Design: pThreads any ETA ?

Some time in the next couple of quarters is probably as precise as anybody can say. But please reserve the issue tracker for filing issues.

This project doesn't have a mailing list or other forum, so the issue tracker is the only place where people can ask questions of the design process. It's valuable to have a place where people can ask questions, so I therefore I encourage people to continue to use the issue tracker to ask questions.

@sunfishcode, fair enough, but then I suggest we create such a list. I don't think that an issue tracker is a proper substitute for that.

There is the w3 public list. It doesn't experience very heavy traffic though. There's also an IRC I believe? I think it was on the website at some point but it'd be nice if those were both directly linked to in the community section of the website. I'd open an issue but I'm not sure which avenues you'd all prefer be ~exposed~ linked to the public

Instructions here:

• https://github.com/WebAssembly/design/blob/master/Contributing.md
• http://webassembly.org/community/feedback/

We've been guiding folks to use github. Not everyone likes it, but it's better to have one place than many.

Hi @rossberg-chromium and thanks for your timely answer.

But please reserve the issue tracker for filing issues.

As pointed out by @jfbastien I asked here because it seems the best place to do it in a standard way.
Besides, thread support is a design feature that is a make or break for webAssembly.
More than the logo context and "Why is it not implemented in java" I see raised as issues.

If I'm wrong please tell me where to follow the shared memory / thread implementation.
It should be an open process, so I'd like to have my say.

--R

@RobertoMalatesta we agree threads are very important, but getting them right was more important than cramming them into WebAssembly's minimally-viable product. Our reasoning is simple: asm.js has enjoyed success without this feature.

Threads are definitely a make or break feature for some important applications, but it's not for all applications. The same is true of GC, zero-cost exception handling, guaranteed tail calls, etc.

Choosing to do an MVP means we don't make everyone happy at first launch. It also means some people get to use WebAssembly at all before we get everything designed, spec'd, and implemented.

That being said, we expect to more-or-less adopt the JavaScript SharedArrayBuffer approach into WebAssembly. Many of us have been involved with that spec from its beginning. It would add low-level capabilities akin to what C++ has, plus a futex-like API. It has room to grow with more memory orderings than sequential consistency. There's ongoing work to create a formal memory model for it.

SAB is at "stage 4" in TC39, which means it's done, ready to ship, and ready to be used. I expect adopting it into WebAssembly won't be too difficult.

@jfbastien thank you for the insightful info.

That being said, we expect to more-or-less adopt the JavaScript SharedArrayBuffer approach into WebAssembly.(...)
SAB is at "stage 4" in TC39, which means it's done, ready to ship, and ready to be used. I expect adopting it into WebAssembly won't be too difficult.

Great. I'm eager to kick-tire it with a lot of C++ code that simply starves for threading.
I regularly use WebWorkers into my TS/JS applications and let me say I'm not a fan of them, but they could go as a first start.

threads are very important, but getting them right was more important than cramming them into WebAssembly's minimally-viable product. Our reasoning is simple: asm.js has enjoyed success without this feature.

Fully understand the needs for a MVP to get out quick (I think that was the reason to drop AST to go stack-based) and the care it needs to plan further steps.

Threads are definitely a make or break feature for some important applications, but it's not for all applications. The same is true of GC, zero-cost exception handling, guaranteed tail calls, etc.

As a long-time user of emscripten, I'd like to see WebAssembly more decoupled from Javascript application stack limits:

Wasm should only 1) aim highest performance and 2) provide sockets, memory/threads, GDI, fs with some Posix/Unix/BSD standard. That way, it would be a thin OS-like layer both embeddable into mobile devices as a unified target platform and into servers to compete for cloud-deployable applications.

Is this the direction wasm is headed on a long term future or I'm completely missing the boat ?

Thanks again for your work,

Roberto

Wasm is definitely decoupled from some stack limits because of how we separate the locals from what a compiler such as LLVM needs to put in the user-accessible Memory.

With threads, especially if they're not Worker based, you'll get more power there.

That being said, it would be good to get more details on your precise ideas. What's the use case, how do you work around it in C++, and how would wasm implementations do it?

I don't expect you to come up with all the answers! Just what comes to mind now. Fwiw C++ has some proposal for stack limits. Check out the SG14 subgroup.

Sorry for low details, I'm under a 👶🌯. Will answer better later!

Sorry for low details, I'm under a :baby::burrito:. Will answer better later!

You're under a baby burrito? LoL

@qwertie

You're under a baby burrito? LoL

Quite literally, a wrapped baby.

Hi @jfbastien ,

it would be good to get more details on your precise ideas. What's the use case, how do you work around it in C++, and how would wasm implementations do it?

Generally my use case is on building simulation software, on the computational part the limit being only the machine, thanks to C++, C/OpenCL and GPUs technologies.

On the front-end the stack HTML5+CSS+Javascript+Typescript has proven to be a valid,proven,portable way of developing mid-large complex applications.

_The JS Web Stack (with the addition of a structured language like Typescript) is so good these days that I don't feel the need to resort to emscripten or try wasm for most applications._

I almost completely agree on the High Level Goals and the Webassembly Use Cases and even more on the Non-Web Embeddings Document.

_WebAssembly will be a relevant technology (at least to me) if it will allow to be faster (to the bone of a native C app plus, say 5% overhead) use more memory and more freely, have ports/sockets fully implemented and allow to run threads like any normal OS does._

Wasm machine should be very slick in size, fast and predictive, so please don't stick into it a GC: GCs are the Ponzi Scheme of IT: you make debts and do not care about allocation up until it's too late and you loose cycles. Besides, if we have a superperforming wasm core machine then a GC can be built on top of it.

For the sake of slickness, I would progressively detach it from Javascript. If I have native compilers in any language then I don't need scripting and, anyway I could embed a Duktape/Lua/whatever simply by compiling in the C sources.

Somewhere on the Mozilla site I saw documents on inplementing wasm native DOM manipulation: I don't think it will be very useful since DOM manipulation is inherently sequential and it's one of the most time consuming aspects of the HTML5/CSS/JS stack: if I write superfast multithreaded wasm only to have my DOM changes serialized waiting in queue then all the performance is gone and it's better transpile some Typescript than using C/C++. In this case I'd favor direct OpenGL access (at first with some WebGL implementation then OpenGL native). Accessing the Graphic Card will prove useful to offer GPU/CUDA capabilities with small or no overhead.
(Just went to check on this and got issue #273 tackling on this)

Having a tiny wasm machine shared by every vendor and independants as well could solve the nightmare of mobile development, where you have at least 4 toolchains for different OSes and versions and architectures.
Graphic and input facilities could vary but we would be able to develop to one wasm and then query the facilities APIs , or even deploy the same application on different app stores.
On the long term: the same with server applications, reaching some kind of consensus on facilities (DB Connection Protocols and Pools, Yellow Pages Systems, Messaging Protocols and Systems etc) and packing them into some standard specification like JEE platform did. (A lot of the conceptual work has already been laid out in the last 20 yrs).

I don't expect you to come up with all the answers! Just what comes to mind now. Fwiw C++ has some proposal for stack limits. Check out the SG14 subgroup.

Sure I'll do.

Thanks for your patience and time.

Rob

As a small addition, when threads are implemented, it would be great if the stack size could be made arbitrarily large, or in lieu of that, if it could be set to at least 64MB. This is because my use case is an application that uses 2 threads, and the non-main thread needs a pretty large stack.

@jjpe that would be a request for a particular toolchain (such as emscripten) that targets wasm, rather than for this repo, because it's an issue of a toolchain's choice of implementation or ABI rather than something about the wasm platform per se. (so e.g. you could open an issue on https://github.com/kripken/emscripten/)

Some of the things I proposed in my former message just seem to be included in the latest WASI proposal.
See:
https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/

Crossing fingers.

--R

FWIW The current status is that Chrome is shipping wasm threads support in version 74, Firefox has an implementation which hasn't shipped yet, and Emscripten has pthreads support (https://emscripten.org/docs/porting/pthreads.html) with wasm available now.

(Also WASI is a system call API, which is mostly orthogonal to whether or not a given wasm embedding or toolchain supports threads/atomics).

Are there any docs on how to use Chrome and Firefox thread APIs?

It's sort of a combination of things. The browser's wasm engine implements the threads/atomics extension to the wasm spec (https://github.com/WebAssembly/threads/tree/master/proposals/threads) which allows shared wasm memories to be instantiated. On the web you can create a web worker, and send the memory (and any associated wasm modules) to the worker via postMessage. Then you instantiate the module with the shared memory on the worker (and on the main thread, or another worker) and the instances share their memory. The spec proposal overview (https://github.com/WebAssembly/threads/blob/master/proposals/threads/Overview.md) has a small example, but I'm not aware of others.

Hi @dschuff and thanks for the answer above.
In my former message I was referring to a previous one, involving not only pthreads but a desirable evolution of wasm into something more kernel-like that some of us following and using emscripten since 2010 saw then as a real breakthrough technology.
See: https://github.com/WebAssembly/design/issues/992#issuecomment-285055578 .

Nine years after emscripten came out, and almost five after wasm announcements, little progress has been done in transforming wasm into a viable server technology.

Sadly, because we need a standard for portable apps, and the best way to do it IMHO would have been to implement a tiny kernel (BSD/Linux like) with POSIX Threads and Berkeley Sockets + a self-contained secure virtual filesystem (with users and groups would suffice). I hold firm on H.Spencer old saying : "Those who don't understand UNIX are condemned to reinvent it".

I still want to keep faith, so WASI is a step on the right direction.

--R

PS: 2010 emscripten still rocks, since it works in the immortal browser, the unmentionable unkillable one used in most enterprises and that will see the bodies of Firefox and Safari float by (their engines at least).