Design: Should WebAssembly consider compatibility with the Open Web Platform critically important?

I broadly agree with the statement as written, but I don't think any kind of formal commitment to it would have avoided many of the disagreements which you have linked to. In particular, your statement is phrased primarily as an additional barrier to the adoption of proposals, but in many of the linked issues you are advocating for additions to the core language.

It might be a useful thought experiment to consider the hypothetical extreme position "if a feature aids compatibility with the Web, we must add it to Wasm", which I wouldn't agree with because it would end up bloating the language with a bunch of JS-specific hooks. I'm not implying that you advocate for such a position, but I do think that your statement as written in the OP doesn't capture the conceptual disagreements which seem to come up in the issues you linked.

There's also a distinction between "WebAssembly" the core language (which I feel absolutely must stay compatible with the "Open Web Platform"), and the ecosystem of tools, host interfaces, and import namespaces which one might choose to standardise/engineer around it. It's less important that WASI stays "Web-friendly", because it's an import namespace which hosts can simply choose not to provide. At the very worst, it would cause duplicated effort/a toolchain split if someone later standardised a more Web-friendly API.

Yeah, some of these issues are a bit on the older side, and I'd certainly approach them differently today (even more so if such a commitment would have existed at the time). I'll think a bit about this one and your other points :)

I was a little surprised by the impression that I am advocating for additions to the core language in many issues, so perhaps it may be good to give some context. Occasionally, I indeed worked out a back pocket alternative (I guess) where it seemed helpful in context of a discussion, which can be seen as advocating for additions I think, most notably Universal Strings where I tried to help resolve a perceived deadlock between Interface Types and GC that were both pointing at each other to be responsible for my concerns. In hindsight, I regret bringing this up again in the IT issue, as GC seems to be the more appropriate place. It may still have blown up there, then, not sure. At the end of the day I never proposed these, also given the existing tensions, but mostly created them out of the desire to explain what I mean concretely. In hindsight, it's a bit of a dilemma I am finding myself in, since proposing anything seems to be doomed anyway due to tension, and working out my arguments in more detail is as well, so if we can make everyone's lives a little easier by making WebAssembly's goals more clear, and work on resolving the general disagreement, I'd very much appreciate it.

The hypothetical extreme position is quite interesting, as I would also argue that there are legacy JS features Wasm might not have to worry about proactively, or perhaps not at all, but then there is still a line to be drawn somewhere. My intuition tells me that one feature too much is better than one too few, for instance, because compatibility is so very important. With the commitment in place, perhaps it would at least be more likely that a feature of interest to a minority can be discussed with less friction, and has a better chance of success to achieve the higher goal?

The distinction between the core language and the broader ecosystem is similarly interesting. I'd argue that if we can, we should try to avoid going down a path of future ecosystem problems and incompatibilities, and should talk more about avoiding fragmentation between Wasm on the Web and Wasm off the Web for example, which seems to be a topic largely non-existent as of today. For example, in the Node.js ecosystem it turned out that all the custom APIs invented for Non-Web were cool initially, but are also a burden today, since sticking to (reuse of) Web APIs is generally preferable over mandatory polyfilling. Deno, the second attempt at Node.js so to say, realized that, but the broader ecosystem is now stuck with it for a long time.

I don't know how much it would be worth to avoid duplicate effort. It's useful to have it as an option, as long as it doesn't lead down a fragmentation path that would otherwise be avoidable perhaps? For example, WASI is a quasi-standard under the WebAssembly umbrella already that is now sneaking into Wasm on the Web, and the more it matures the harder it will become to fix ecosystem problems we discover in the process, so a mechanism to prevent such an outcome before it is too late seems useful.

We cannot have absolutes in our design goals that Wasm must be optimal for a particular eco-system or use case, because fundamentally Wasm serves many eco systems, use cases and languages, which will always require some level of compromise.

Besides web & non-web, the design of Wasm so far has put great emphasis on bringing a broader set of languages to the web, which are often languages that are almost on purpose very different from JS, since they complement JS's existing abilities the most, and thus makes the combination of JS+Wasm the most powerful. It also allows the most amount of existing software to be brought to the web, and allow the web to reach new levels of performance. If instead Wasm had been designed to favor languages very similar to JS it would have been quite pointless.

This is a good thing, but naturally means these languages may be rather different when it comes to implementation details of things like memory management, strings etc, and their needs should be taken into account, especially since the whole point of using these languages is to allow them to be efficient in this new context. When it comes to interop at the boundary, not just interop with JS needs to be considered, but also interop between multiple Wasm languages, and, in the future, interop between these languages and the native code of browsers and other Wasm hosts, which will often be the same or similar languages. Proposals like interface types try to balance all these requirements, not just one of them.

I believe there is another misunderstanding in play that may be good to talk about. I have indeed been advocating to consider my project's needs, more so initially as I hoped its somewhat unique perspective (I believe that some of AssemblyScript's problems of today are a canary for WebAssembly's problems of tomorrow) would be valuable to the CG, but more recently and especially today I am not advocating for a particular language but for the higher goal of compatibility on the Web, while hopefully making good use of the opportunity to resolve disagreement among the group. Just like you, I strongly believe that Wasm should not favor a particular language, and I fear that we are losing track of way more important aspects like compatibility between Web standards over arguing for or against particular languages, leading to discussions that are highly likely to become too heated to be constructive, in turn stalling WebAssembly's evolution. To give an example, I take slight offense by the way you phrased your comment, as I feel that it implies quite a few things that may be on my agenda but are not, while I actually agree with your higher level points of not favoring a particular language, complementing JS's existing abilities, balancing all requirements, making compromises where necessary, and working towards reaching new levels of performance, as long as it doesn't lead to a Web standard that is no longer a Web standard anymore, which would be very unfortunate.

First, I want to say that I see your experience of having adversaries in the CG as a sign of failure of process. It is not an unreasonable goal for us to work non-adversarially, even when we disagree strongly with one another on technical details or priorities.

Second, I agree that interoperability with JS is important for the success of Core WebAssembly. That includes proposals like GC and EH, which have not fully fleshed out JS APIs yet, but will before they are standardized. There is also the stack subgroup, which is considering JS interop as its first matter of business. I would also include in that statement other proposals like IT that are layered, possibly via separate specifications, on top of Core WebAssembly, but still meant to be integrated into the Web platform.

I do think the calculus is different for non-Web consumers of the Core WebAssembly spec in general, and WASI in particular. I don't think the CG should (or can) attempt to impose any control over how Core WebAssembly is used or embedded beyond the Web platform, and since the WASI subgroup is not currently proposing any changes to the Web platform, I don't think it makes sense to encumber it with Web compatibility obligations. Consider that the WASI subgroup could always carry on the same work with the same goals in some other standards organization, but we're all much better off in terms of collaboration, cross-pollination of ideas, and organizational overhead (not to mention compatibility!) because it is a subgroup of the CG.

Speaking of calculus, a hypothetical extreme scenario "what if a subgroup is acting with malicious intent to influence core WebAssembly / work around its higher goals" may be interesting here as well, which I believe many will agree with me is not the case when the subgroup is WASI, but I would still like to raise awareness for, since the inclusion or exclusion of Interface Types features for example, which is a core proposal effectively driven by the same group, can so easily prioritize the needs of WASI, effectively a non-Web certain-language subgroup, over the needs of others. From my perspective as of the OP, I may already have witnessed the latter, so I'd argue that just being wary of the risk and taking precautions isn't necessarily a bad thing, since especially when a subgroup's efforts are otherwise aligned with the interests of a majority, the side effects may be hard to realize before the damage has been done to core WebAssembly, the broader ecosystem, or even worse a Web standard. Or in other words: I think being a subgroup under the WebAssembly umbrella should imply responsibility for core WebAssembly and its higher goals.

I think it would be good to be a little more conservative about ascribing malice to various groups, even hypothetically ;)

A lot of these contentious issues, especially in the IT proposal, which not many people are closely following, only involve and are seen by a handful of folks. For high-visibility design issues, it is hard to reach a decision that a majority of the CG would be against because a more-or-less representative subset of the CG is engaged in the design process, or at least following along without objecting. For niche issues, though, it is possible to reach a decision among a few people that the full CG would be uncomfortable with. In these cases, it can be helpful to raise the specific issue at a CG meeting to take the full CG's pulse on the proposed direction.

One recent example of this was when @lukewagner raised the issue of disallowing duplicate imports. As you experienced, the CG was not entirely comfortable with the direction the module-linking folks had come up with, and as a result there has been wider participation in that discussion and consensus has shifted in a different direction.

Ultimately proposals that the CG would not be comfortable with will not be advanced by the CG. Of course, the earlier issues are surfaced and discussed, the better. I hope everyone feels empowered to raise issues in CG meetings as they identify them.

Stating this first and foremost - I think that interoperability with the web platform is critical for WebAssembly adoption, and I agree in broad strokes to your points about compatibility. That said I'm not sure that adding barriers of entry to proposals would lead to a positive outcome. The WebAssembly CG is made up of the people in the community, and they can all have different goals while contributing to positively advance the standard. Having different goals can sometimes lead to friction, and I'm hoping this is a place where we can respectfully disagree when working towards different goals.

It has been suggested to me to take the following concerns regarding the general direction of WebAssembly before a more CG-level forum, an advice I am following now in the hope that we as a community can eventually resolve them for good.

As some may know, my perspective on WebAssembly's goals has been clashing with the ideas of a certain group responsible for various proposals for quite a while, often leading to more or less heated discussions. So I think that there is an opportunity here to identify the root of the disagreement and work it out, so we can eventually align towards common goals again. I can only speak from my perspective, one that is more common among people who are interested in running WebAssembly on the Web, and _also_ off the Web, which is coincidentally exactly what I am working on, and I am very interested in others thoughts.

From my perspective, the disagreement stems from the expectation induced by initially and still advertising WebAssembly as being part of the Open Web Platform (I'll shorten to "the Web" below). For instance, it has been expressed many times that Wasm isn't meant to replace (or harm) JavaScript, but is rather a companion technology, so I always assumed that it is a priority to achieve excellent compatibility (for me also means: interoperability) with the existing Web, which is a generally desirable goal on the Web. Or, as webassembly.org states:

Unfortunately, over the course of all these heated discussions, I got the impression that my understanding of WebAssembly's goals does not align with what my more influential adversaries within the CG are working towards, which I cannot help but see as attempts to create a new status quo that largely factors out compatibility with the Web. The first issue this disagreement showed in is STRING i32 pairs for UTF-16LE over at Interface Types (in 2017, at the time of creation named Host Bindings), where after 1 1/2 years of silence many arguments have been exchanged on whether UTF-8, a string encoding largely incompatible with Web APIs and JavaScript, should actually be the only supported encoding, or whether it would be useful to also account for compatibility from the start. In my book this should be a given, so I was very surprised by the sheer chain of contrary arguments, in turn fueling the disagreement. The issue eventually (mostly) resolved with the acknowledgement that there are just so many languages using a JavaScript-like string encoding that we should support it, in turn leading to the thorough concept of adapter fusion, yet it was hardly ever acknowledged that compatibility with Web APIs in particular is important. There are certainly more non-technical points to talk about in context of this issue, but I suggest we avoid these for now and focus on WebAssembly's direction.

Another similar issue is GC story for strings? over at the GC proposal, which so far did not reach resolution, and it probably also does not necessarily have to if the goal is an MVP we all can agree on. Yet, discussion so far also indicated that there is disagreement we'll have to resolve eventually, that is as soon as GC may be used in a way that requires interoperability between WebAssembly and Web APIs involving strings, perhaps also involving an arbitrary module graph, so ideally with minimal conversions and copying. The general disagreement also shows in a follow-up issue regarding a better GC story for arrays?, that quickly side-tracked to question my ability and the value of the project I am working on instead of discussing the concern at hand, which I today see more as an unfortunate consequence of a long-standing misunderstanding. If anything, it underlines that there is an opportunity here to align on common ground again.

I was also surprised by the recent efforts to Disallow duplicate imports?, a mechanism useful to integrate with dynamic languages like JavaScript where overloading with multiple signatures is common. The hurry to vote on removal in the same meeting after the presentation had me raise my eyebrows at least, but this may well be just another misunderstanding, or a disagreement on how the process should work.

Not speaking for anyone, but voting on issues has been a historically useful part of the process to gauge opinions of the community, and I wouldn't read into it as a hurry to vote, more of a poll for direction. If you look at the notes, the CG decided that we need more eyes on the issue and the design issue continues to be a place for that discussion.

Similarly, I recently came to question whether WASI may turn out to harm Wasm use cases on the Web?, as to me it appears to establish a new status quo as a side-effect before the more Web-focused part of the community had a chance to find a solution that promotes compatibility with the Web over fragmentation, due to related proposals not being far enough. Personally I consider the priorities at play there problematic for the reasons brought up in the issue. And coincidentally, the efforts at WASI are driven forward by CG members earlier arguing against my points at Interface Types, and while correlation does not necessarily imply causation, it ultimately led me to question the general direction of WebAssembly, leading us back to this issue.

The WASI design principles which I'm sure you've looked at well by now, explicitly state that it isn't limited to APIs that are easy to polyfill on the Web, while also explicitly stating that WASI should work with Web standards where possible. As @conrad-watt points out earlier, there is a distinction between the core language and the ecosystem, and it is a namespace that hosts can simply choose not to provide. WASI as an ecosystem has different design goals, and forcing web compat to be one of their goals seems to be not productive to the problems WASI is trying to solve.

As should be obvious by now, I have grown very worried that we, as a community, are not doing what is necessary to ensure excellent compatibility with the Web, while at the same time tolerating new proposals and related technology to break compatibility with the Web too much, or otherwise affect it negatively, directly, or indirectly. This has led me to wonder, for instance, whether Wasm, or at least its future self, should perhaps not be a W3C recommendation as it compromises backwards compatibility one essential piece at a time, which is certainly an extreme conclusion, but would be the ultimate question we'd have to ask when the level of disagreement increases even more, I think.

The community does consist of several people that have worked on Web standards or are heavily invested in the Web. While I hear your concerns, I'm not sure that the future is quite as dire. Can you add concrete examples where backwards compatibility has already been compromised in a meaningful way?

On that background, I am today turning to the CG with the request to discuss the following addition to WebAssembly's overall goals in order to hopefully reduce unnecessary friction in the future:

Compatibility with the Open Web Platform in general, and Web APIs and JavaScript in particular, is critical for WebAssembly's long-term success, and must not be compromised if it can be avoided. The bar is especially high for proposals promoting functionality not primarily targeting the Web, as there may be little incentive to not negatively affect this goal.

This is the higher level aspect I have been advocating for, in a nutshell, and if we can agree on it, and just write it down, most of the friction in discussions above would become (mostly) obsolete, at least as far as I am concerned, while restoring my faith into the process.

What this could imply:

• Interface Types is fine, but JS string encoding is important due to compatibility, and promoting UTF-8 over it to provide gentle pressure or similar, or justifying it with purported trends, may be problematic
• Threads, SIMD, i31ref etc. are all fine, as these are unrelated and only add functionality, unless they are/do not anymore
• GC is fine, but would be motivated to find a suitable solution for interoperability eventually
• WASI is fine, but would be motivated to consider alternatives to what is essentially a libc currently before going further down the fragmentation route
• WASI-nn is fine, but would be motivated to cooperate for example with the Machine learning for the Web CG

If it turns out that the CG cannot find consensus on making such a paragraph official, or if major modifications to it are necessary, I'd be very worried about the future of Wasm especially in context of being a Web standard and would suggest to re-evaluate our goals, and what we communicate publicly, in more detail. In this case, we may also want to hold a vote to legitimate such a course, I think.

Let me know of your thoughts, thank you! :)

Going back to adding explicit barriers of entry in the form of process doesn't work very well in practice, and given that your concerns are about proposals already in flight, even if we knew exactly where to include this text from a strictly process perspective this would not retroactively apply. Historically we've tried to keep the process for proposals light weight to encourage forward progress, and entrusting to the CG to make the decisions that make sense. Adding process barriers only goes so far if the assumption is that the CG is not operating in good faith. Not saying at all that this is what you are implying, just that barriers of entry are not a particularly good way to address the concerns raised.

@tlively It seems I stroke a nerve in my previous comment. Please bear with me for a moment, though, since I do believe that thinking this through fully can be valuable when avoiding bad outcomes or reducing future tension is the goal. For instance, another more subtle hypothetical extreme scenario could be "what if an actor also responsible for core proposals delays advancing these to give the results of a subgroup with different goals time to flourish in the broader ecosystem - while we cannot prevent it, would we want to discourage it?". Like, I did think about this for a while, but I want to emphasize that this is all hypothetical and mentioning it is serving a higher cause. I just do not want to weaken my suggestion in the OP by holding back on an aspect that may perhaps be relevant to understand my thought process leading to it. This may be another dilemma, I think, and I am trying my best to be as respectful as I can. For instance, I deleted my earlier reply that read "thanks for pointing this out and being so supportive, my apologies", because I wasn't quite confident with holding back on what I had in mind after considering what we may be missing to discuss. I hope this is OK, given what I believe is at stake?

@dtig Thank you so much for your thorough reply. Reading it made me more confident about WebAssembly's future, and also more confident that my experience so far is not as representative for the CG as a whole as I initially thought. The examples I can provide are mostly based on discussions within individual issues I linked in the OP, and not so much on CG activity as a whole as I now realize. That "WASI should work with Web standards where possible" is also an aspect I strongly agree with, and perhaps misunderstood because to me it appears to be relativized in the next sentence "where it isn't essential", since JavaScript, which is the context, is such a Web standard, and does not match my experience so far in related issues over at WASI (see https://github.com/WebAssembly/WASI/issues/402 where the outcome is a syslog-like interface). I will have to rethink the implications here for a bit, if that's OK, but I am interested in discussing this point, and would then likely argue for the higher level point that WASI's design principles shouldn't be nonnegotiable, as was my impression so far, since WASI being under the WebAssembly umbrella doesn't only grant it freedom to create new APIs, but also implies a responsibility due to potential indirect effects (that I would like to raise awareness for) on core WebAssembly.

I think WASI is not a good design. First, web assembly designed for the web virtual machine, and designing some APIs for posix compatible is not in line with the development direction. Second, a better way to introduce third-party APIs is to define interfaces (interaction between two domains) so that the number of APIs can be controlled and security can also be controlled.

another more subtle hypothetical extreme scenario could be "what if an actor also responsible for core proposals delays advancing these to give the results of a subgroup with different goals time to flourish in the broader ecosystem - while we cannot prevent it, would we want to discourage it?"

I believe if there is a disagreement within the proposal, dissenting party can bring their concerns to CG, which has power over the proposals. Adoption requires shipping in an implementation, but it does not require shipping in all implementations (I think two is the number, but I haven't looked at it in a while). I am not sure how this would enable a somebody to delay a proposal against everybody else's wishes, as long as there are implementations.

@penzn Unfortunately, we wouldn't be here if the process was perfect. As an example, we may look at my disagreement with WASI's design principles. I do not know how to approach these properly, since WASI is largely independent by design, which the contemporary majority it serves sees as a good thing, so bigger picture concerns can only be raised at WASI itself, but then my impression was that raising concerns with WASI at WASI is very unlikely to lead to resolution, because most at WASI naturally want WASI to stay the way it is, which is understandable. I have been sent back to Interface Types for instance, even though it is the same group and they know of the troubles I went through there, and more often than not what I point out is "incompatible with WASI's design principles" and that's it. So I tried to identify the higher level problem, that in my opinion boils down to WASI being legitimized to be hardly aligned with core WebAssembly's goals, while at the same time not having responsibility for core WebAssembly's goals. So I suggest to adapt the process, because I either cannot use the process, or do not feel empowered to do so.

Please see Safety of C API discussion at CG meeting on 2021-04-28, it was a similar situation where presenter's concerns were not resolved within the proposal. Of course, the difference in case of WASI is that it is not a proposal, but CG should still have procedural authority. In case of C API safety, there were concrete issues raised that the champion of the proposal had to accept after CG sided with the dissenter, however I believe you can present without a vote at the end as well.

I am sorry, but I do not see how "Safety of the C API" is comparable to more bigger picture concerns. Can you elaborate? For instance, I feel that me giving a pro-Web presentation at WASI would only fuel the disagreement even more. As @dtig said, doing so "seems to be not productive to the problems WASI is trying to solve".

In the C API safety debate there also was a high-level disagreement about goals and needs (whether or not checks are needed for API for an unsafe language), which got resolved in a CG discussion. This is an open standard - making proposals and starting discussions are the only tools we really have to change it.

Some thoughts on the ecosystem issue: I agree that fragmentation between Web and server APIs is a risk. With the addition of WASI we now have two sets of standardized APIs for Wasm, and with no obvious path to unification, since I am personally skeptical browsers will adopt WASI and given WASI's existence the reverse seems unlikely too.

It would have been better for the Web had WASI focused on porting of Web APIs to the server. But that doesn't mean it would have been better for Wasm as a whole. I appreciate the comparison to Node.js and the API history there, but I think it goes both ways: Yes it would have been good for Node.js to use more Web APIs, but maybe Node.js's massive success is due in part to adding the APIs they needed regardless of the Web. Maybe WASI is necessary for Wasm to succeed on the server - given how different the Web and server ecosystems are maybe API fragmentation is unavoidable.

That's one reason why I personally didn't criticize WASI when it started. While from the Web's point of view it isn't optimal, it may still be necessary for Wasm's success on the server, which may be good for Wasm's overall success. (The second reason is that WASI's security model is interesting and may also help Wasm thrive.)

More generally, Wasm is being extended in lots of ways besides the Web and the server, such as plugins, sandboxing solutions, blockchain, and others. As a person working mostly in one area I wouldn't want to criticize people working in others. We shouldn't slow each other down.

I think the only exception to that should be a really serious danger, of fragmentation or otherwise. I don't see such a danger currently. While having two standard APIs makes some things awkward and less optimal, we can polyfill server -> Web already, and eventually we'll be able to do Web -> server as well which will help. For issues like strings I see pre-imports as allowing eventual optimal string usage on the Web - which will also make API fragmentation in Wasm more obvious, but in this case it's pre-existing because of unbridgeable language differences. Overall the risks are real and annoying but they seem manageable and unavoidable.

Good thoughts, @kripken, these are about in line with my understanding of the alternative.

Regarding the Node.js comparison, what I had in mind in particular are things like Buffer vs Uint8Array, which is still responsible for lots of (slow) polyfilling on the Web. Or the avoidable global vs window check in every other portable module. Or TextDecoder not being a global for a while to reduce startup time. Or crypto.getRandomValues still not available globally. Or all the pitfalls with ESM support up until today. I'd argue that Node.js could have done better at some places, while support for fs etc. was probably unavoidable.

In WASI, the situation seems to be similar, in that we'll very likely want a WASI filesystem, but then there are other APIs that provide functionality common across ecosystems, APIs that are essential to create a portable base module in the first place, where I think we can do better than where we are currently heading. Imagine for instance a module that writes to console when something goes wrong, obtains the current time etc., where depending on a (full) WASI polyfill appears to be overkill. I'd imagine that some modules would still need an accompanying polyfill for (just) WASI filesystem on the Web, and I think that's indeed manageable. I just don't think that we are in an all or nothing situation here, but that there is lots of room for cooperation that will eventually pay off over "but the design principles". My argument is: If we can, we probably should, and if we need to slightly adapt the process for it, then we should as well.

Regarding pre-imports, I am not a huge fan of relying on these for very basic interop personally, because I think that strings are just so very essential that fragmenting on this level already is not what we ultimately want. For instance, you mentioned elsewhere that ecosystems with a need for portability might eventually converge to use JS strings, not because it's great to do, but out of sheer necessity. I guess especially those disagreeing with me previously would not be very pleased by such an outcome, including myself.

Perhaps somewhat relevant in context, an excerpt from Deno's latest blog post with honorable mentions of server-side fragmentation and the value of adhering to browser APIs:

Extending web programming beyond the browser is not a novel idea. Indeed, we have done that with moderate success in our “Node.js” project. But over a decade later, we find server-side JavaScript hopelessly fragmented, deeply tied to bad infrastructure, and irrevocably ruled by committees without the incentive to innovate. As the browser platform moves forward at a rapid pace, server-side JavaScript has stagnated.

Deno is our attempt to breathe new life into this ecosystem. To provide a modern, productive programming system that adheres to browser APIs. Deno is not a monolithic system, but rather a set of technologies that we believe can be repurposed to a variety of needs. Not every use-case of server-side JavaScript needs to access the file system; our infrastructure makes it possible to compile out unnecessary bindings. This allows us to create custom runtimes for different applications: Electron-style GUIs, Cloudflare Worker-style Serverless Functions, embedded scripting for databases, etc.

It's of course not exactly the same, but there may still be something to learn from the lessons they learned.

Definitely a relevant article, yes. Another key quote from it:

Many developers, we think, prefer web-first abstraction layers.

Another reason why Web -> server polyfilling (as mentioned earlier) will be important.

Side note: I've sketched out a building block to resolve part of the problems I see with WASI and fragmentation on an essential level meanwhile, named System Essentials for WebAssembly, but would like to have your thoughts on whether something like this sounds good before proposing anything (feel free to open an issue over there). If someone sees value in it as I do (or even wants to co-champion / provide guidance), let me know. If not, I'd be interested in alternative ideas that'd achieve a comparable effect :)

From a quick glance at your doc, the list of "essentials" consists entirely of system _capabilities_. Providing them by default would imply the presence of ambient capabilities, and would thereby undermine the sandboxing model that Wasm was carefully designed to establish.

Moreover, none of the capabilities in that list can be expected to be universally available, even across the environments hosting Wasm today. For example, neither current time (let alone local time), nor randomness, nor logging are available on decentralised systems such as blockchains.

Providing them by default would imply the presence of ambient capabilities

Like the presence of a memory, perhaps. Doesn't seem so different.

and would thereby undermine the sandboxing model was carefully designed to establish.

Only if memory, or non-determinism on different hardware (FP, relaxed SIMD) does as well, I think. I did not intend to, and don't think the doc does, "undermine" Wasm's "careful design".

none of the capabilities in that list can be expected to be universally available

A Wasm binary runs on some machine with some VM on some OS, so I would not agree here. I think it is reasonable to expect these to be universally available, even though special use cases may want to restrict their use.

For example, neither current time (let alone local time), nor randomness, nor logging are available on decentralised systems such as blockchains

Say blockchains for example. These typically disallow floating point math as of today, and may similarly disallow obtaining time or decide to throw log messages away, but nodes still run in some VM on some OS. I'd also say it depends on the exact use case of a ledger, so there are very likely use cases where restrictions are not necessary. I don't think Wasm can solve this in a pure sense anyhow. Another extreme is IOT devices with less than one page of memory, with different implications.

In general I think that battling fragmentation on such an essential level is valuable, and I am obviously interested in thinking this through, even if the outcome is something entirely different than outlined in my doc (I am fine with throwing it away), yet ultimately achieves something comparable. Like a constructive yet well-informed compromise. As such I'd love to have your thoughts on how we could achieve this, together, also personally because that would make me feel like my ideas are considered valuable by people I admire for their technical expertise.

Anyway, it may be better to move this discussion to my doc's repo, or Discord, or E-Mail, or voice chat. So please do (or contact me where it works best), if that's your impression as well :)

I do think there's value in not having the WebAssembly Core spec assume anything about its execution environment, including that there is an OS or a ambient concepts of time or logging. For anything that depends on or mutates state outside Wasm's formal semantics, the Core spec carves out imported functions as the catch-all escape hatch, but of course does not specify that any particular set of imports will be provided.

In general I think that battling fragmentation on such an essential level is valuable

I think this is the crux of the disagreement on this thread. Personally, I feel strongly that it should be a non-goal for the Core Wasm spec to try to provide a unified ecosystem, but that it should try to provide as portable (and performant) an abstraction for pure computation as possible. As an analogy, neither X86 nor ARM attempt to provide unified ecosystems (or even unified ABIs); that's left to the OS layer of abstraction sitting above them, and I think that model is appropriate for WebAssembly as well.

That being said, avoiding ecosystem fragmentation where possible is clearly valuable, but I think it should be done at a layer above Core Wasm. The clear candidate for this ecosystem unification layer is WASI. Would it work if the essential capabilities you identified were provided by a WASI module specifically designed to work with minimal glue both on and off the Web? Where do the current WASI facilities for time and randomness fall short?

Maybe that's what's needed: a standardized set of _optional_ imports; available in Web, most WASI systems (because they are optional, and WASI may run in a restricted embedded environment for example), and any places that may _optionally_ provide the standard imports.

In Web, it would be _as if_ JS glue code passed them in, but no, the system passed them in and the JS dev either didn't have to do anything, or simply had to list them by name so as to choose which ones are available.

Web devs could take it for granted that certain imports exist (or are chosen on the JS side), while this does not force Wasm itself to have it in the language.

A blockchain env may not provide the (optional) imports for example.

The user should be responsible for knowing in which env they are running in and what imports are documented to be available.

This would mean no need for compiler switches to output differing API calls.

WASI would be two things: a provider of standard imports, and a provider of specialized server-only imports.

I think the problem of providing standard imports on the Web without any JS is a slightly different problem (but @dcodeIO, let me know if you disagree). Even given a standard set of imports for use on and off the Web, the current JS API would still require writing JS to compile and instantiate the module, so the marginal benefit to Web developers of letting some imports be implicit would be small. Given a standard set of imports, it might make sense for them to be implicitly provided by something like the ESM integration proposal, but the issue here is that we don't have such a standard set of imports yet.

@dcodeIO

I think the idea of System Essentials is interesting, but I disagree with a new standard for them. Optimally they could go in WASI (as @tlively said), or be Web APIs.

About Web APIs: Fair point in the link about getting the timezone requiring a new Date object. That is cumbersome in wasm. But it is possible with the JS Reflect API (Reflect.construct specifically). We could add more such APIs on either the JS or the Wasm JS API side as needed. The imports would still need to be declared, but the code size cost should be minimal.

About WASI APIs, I strongly believe we need to do more to bridge the Web/WASI divide. Adding more Web-friendly APIs to WASI makes sense to me personally - perhaps similar to System Essentials. I described another possible direction last month and offered to sketch out specific details if there is interest, but have not received a response from WASI people yet. Feedback there would be very welcome!

The crucial aspect to me is that there should not be glue code necessary for essential functionality on the Web respectively common functionality on/off the Web, as I believe that an ecosystem that relies on mandatory polyfills for very basic functionality is not what we should strive for. I also hope that we can find something that works both ways, by neither requiring a polyfill on the Web nor in WASI, for a win-win (even though it may be that polyfilling Web APIs off the Web could be less problematic - but I'd prefer doing something for both).

That's where my somewhat lacking "System Essentials" instructions idea is coming from, mostly based on my observation so far that adding an import namespace probably won't happen in browsers for reasons brought up earlier, so it was my intuition that perhaps instructions may actually be less problematic, even though there are good arguments against. I do agree with your points about keeping core Wasm "core", and hope there is something better we can do on top of core Wasm that works for both Web+WASI.

As such, I am not particularly attached to my particular idea, and would be happy with anything that ticks the boxes of [x] eliminates glue code, be it via instructions, a common import namespace on+off the Web (WASI or not), or indirectly via ESM integration and supporting part of it off the Web, so that we ultimately achieve [x] less fragmentation (same module runs both on and off the Web without polyfilling when only using common functionality; drawing the line at Filesystem and DOM, which are rather environment specific). Does that make sense? :)

Another thought: Perhaps our conversation may be useful to inform the sub-languages idea - say a "profile" for "web+wasi" or something like it? It may then boil down to standardizing the intersection between sub-language API surface.

Also, with this issue being open for nearly a month now, and sadly hardly any of the IT/WASI folks chiming in to improve the situation, I decided to attempt myself in conflict analysis fwiw.

I want to emphasize that the following is very subjective and really only represents my emerging mental model of the Wasm landscape. Nothing of it does imply that anyone is doing anything bad, just that there are varying focuses, which is only natural. Please let me know if you feel treated unfairly, which is not my intention, and I will correct or delete.

My goal was to identify alignment, or the lack thereof, between Wasm players, both in regards to making Wasm richer, i.e. to include more features and use cases, respectively keeping it pure, i.e. pressing the point of a minimal ISA, while also incorporating focus on certain sides of the coin, here primarily interested in running Wasm on the Web Platform respectively on the Edge or within a Blockchain, often with very different requirements.

• Browser: Serving the Web Platform, which is rather rich by nature. "The new desktop OS" so to say.
• Server: Relatively barebones by nature as in "bring your own stuff". Neutral.
• Research: Wants to make Wasm good for everyone, unless attached to another actor.
• Google: Focused on porting stuff to the Web and making it fast; OK with considering others needs as long as it doesn't harm theirs?
• Dfinity: All-in on Blockchain; Needs some richness features apparently; but the more barebones the better?
• Fastly: Edge-focused; Lean/minimal VM is critical for their product; Effectively responsible for IT/WASI?
• Apple: OK with what works for the Web; Not necessarily eager in competing with the App store?
• AssemblyScript: Wants real-world practicability; Tends to the Web but somewhat fuzzy due to stakeholders on Edge/BC?
• Intel?: Not many signs so far; Mostly interested in efficient compute?
• Redhat?: Not many signs so far; Mostly interessted in infrastructure?
• Microsoft?: Not many signs so far; .NET on the Web and eventually everywhere?
• Mozilla?: Non-existent?

While this is likely wrong on many levels, I found that it illustrates my mental model relatively well. I've also drawn a line where I would consider "The Web in WebAssembly" to come into play from my expectations, but this again is arguable. If anything, I hope it helps to understand where I am coming from and why I have gotten the impression that something is off in Wasm-land, while also shining some light on the situation for those who are interested in my condensed view of the issue but may not be aware of every detail that has happened. Take, but with a grain of salt :)

Two pennies... I'm genuinely surprised that this conversation ever had to happen. It was always a bad idea for WebAssembly to allow its scope to be defined by whatever usecases early adopters happened to come up with.

The Web needs WebAssembly in a way that no other platform does, because the Web must restrict what its developers can do, especially at the level of WebAssembly.

If WebAssembly goes in a direction that crypto-miners dislike, they can fork it or whatever. It's not something we need to concern ourselves with when designing Web standards. Everyone else has complete control over the underlying platform. On the Web, we only have the platform we're given, and once it's a standard, we're kind of stuck with it forever.

WASI should be forked off to do its own thing, and the Web should have its own VM that is fully optimized for the browser, exclusively and specifically.

It was always a bad idea for WebAssembly to allow its scope to be defined by whatever usecases early adopters happened to come up with.

Actually, its scope is defined by the CG, which can be joined by anyone, including you. There, people vote on direction, which can be influenced if you want to spend time on being active in it, and present well thought out technical arguments.

Anyone can already fork Wasm if they want to, but there's a lot of advantages to a shared eco-system, so that is what most interested parties go for.

Everyone being able to join the CG means you can't stop people with a non-Web interest from joining, I'm afraid.

I'm genuinely surprised that this conversation ever had to happen. It was always a bad idea for WebAssembly to allow its scope to be defined by whatever usecases early adopters happened to come up with.
...
WASI should be forked off to do its own thing

WASI _is_ doing its own thing; it's an import namespace. It's not being standardised at the same level as WebAssembly language features. Is the suggestion that we should declare non-Web features out-of-scope and not permit WASI specification to happen under the nominal umbrella of the Community Group? What would that accomplish in terms of preventing fragmentation?

There's merit to investigating whether certain parts of WASI could be tweaked to be more neatly shimmable (both Web->WASI, or WASI->Web as @kripken laid out here), but even in the very worst case, we merely end up with an API that can't be imported on the Web, exactly as though WASI was completed as a third-party project for a non-Web runtime.

Again, reading through the issues @dcodeIO linked, I don't see anyone who thinks the Web isn't important for WebAssembly. All I see are people disagreeing (sometimes undecorously) on features/roadmaps for solving pretty specific interop frictions like string encoding. I strongly believe that it is a mistake to characterise these technical disagreements as evidence of a Web vs non-Web power struggle, and that doing so will not result in any positive outcomes.

I don't see it as a power struggle, so much as an issue with the scope being too broad and effectively open-ended. The World Wide Web is important enough to have its own VM, with its own specs and standards, on its own terms, and using its own nomenclature. The attempt to make everything general and abstract has led to specifications, documentation and conversations that are unfamiliar and often alien to web devs.

To be clear, the issue with WASI being an official project is purely symbolic. It makes no technical difference, but its status makes it clear that the Web uses WebAssembly, but that it is not a technology created by and for the Web.

I don't see it as a power struggle...

@7ombie apologies, my final paragraph was intended to be primarily commenting on the above diagram of "conflict".

The attempt to make everything general and abstract has led to specifications, documentation and conversations that are unfamiliar and often alien to web devs.

I agree that some of the technical expectations around interface types specifically have become quite esoteric. There's certainly room in the CG for conversations such as "why interface types rather than first-class string types", and we rely on Web-centric devs like @dcodeIO sharing their opinions and concerns in order to keep the process working. I was dismayed by the way in which the last conversation on this ended, and I hope we can do better in the future.

Understood, @conrad-watt. I haven't seen that conversation, but for my part, I've no desire to play into a Web Vs. non-Web mindset, or anything like that. I'm simply sharing personal gripes with a technology I'm generally very happy with, excited about and grateful for.

My earlier comments were pretty critical and blatant, but I was just being casual. I'm not upset.

@conrad-watt et al. The Interface Types technical description is likely to get more esoteric before it gets less so in the near-ish future. By way of explanation, a lot of this is driven by a desire to combine re-usability with rigor.

I would like to emphasize that I also see value in cooperating (with WASI and others) as much as we reasonably can, hence I was suggesting to investigate solutions that lead to portability between ecosystems in a way that the intersections are covered, including where I'd currently draw the line (DOM, Filesystem). I am aware that what I am suggesting may be hard, or that there may be better ways to achieve something comparable (let me know! :)). It's just that (to quote myself in the OP)

I have grown very worried that we, as a community, are not doing what is necessary to ensure excellent compatibility with the Web, while at the same time tolerating new proposals and related technology to break compatibility with the Web too much, or otherwise affect it negatively, directly, or indirectly.

led me to suggest to re-evaluate our goals somehow by making it clear that

Compatibility with the Open Web Platform in general, and Web APIs and JavaScript in particular, is critical for WebAssembly's long-term success, and must not be compromised if it can be avoided. The bar is especially high for proposals promoting functionality not primarily targeting the Web, as there may be little incentive to not negatively affect this goal.

I am not as attached to making this a requirement for proposals (as was hinted this may become) anymore as I was in the OP, as I agree with many of the points brought up, but would still appreciate a concentrated effort that effectively shifts us into that direction.

@dcodeIO, what is the possible discussion item in your conflict analysis, do all companies you listed need to reveal and compare their plans? Do you think that is possible and what is it going to achieve?

While can't really comment on any of the companies in your chart, I think comparing them like this is misleading, as no single entity is able to change the standard for themselves without others agreeing. I also find the chart axis to be subjective and confusing. For example, running compute (let's say rendering or NN) in wasm on the web should be "edge", since it moves compute down from "the cloud", but then it should also be "web", since that is a crucial part of interactive web now (and definitely not "server", since it runs on the client machine). The other dimension is even more subjective IMO, as pureness and richness in your definition are relative terms. Example - there is a really long running discussion about control flow restrictions motivated by Go's calling conventions. Depending on whom you ask those restrictions would be seen as a useful feature or unnecessary, which I think would affect where that person would place the discussion on "rich vs pure" axis.

To ask a direct question, is the longterm direction of WebAssembly defined by whoever adopts it? For example, if longterm, adoption by web developers was low, but there were big, industrial players with billions invested in using WebAssembly for something unrelated to the Web, would their needs eventually take priority?

To ask a direct question, is the longterm direction of WebAssembly defined by whoever adopts it? For example, if longterm, adoption by web developers was low, but there were big, industrial players with billions invested in using WebAssembly for something unrelated to the Web, would their needs eventually take priority?

We're a community/consensus-driven standard. There's not much use in considering these hypothetical situations, because fundamentally they can't be guarded against by any kind of formal procedure. Welcome to the existential anxiety of community/consensus-building.

Yes, if the only people who cared about Wasm were hardcore edge computing engineers, it's likely that the language would be led in that direction. In this scenario, even if we had somehow managed to engrave in stone a principle that only Web-compatible features can be standardised in the CG, this would just result in a fork, or no one working on the language.

To be clear, I absolutely do not believe that this is the situation we're in. One either has to trust that the Web's interests are being adequately represented by CG members, or get involved oneself.

See, this is something I do not understand. The goal is clearly to produce a Web standard in the end, under the W3C umbrella, which has very clear principles and values and a lot of experience from a long history of similar challenges. I do not see how the Web leading and everyone with a fork aligning with what the Web people come up with, if that's what they want, would be any worse. So yes, I am suggesting to create WebAssembly for the Web first, and for everything else second, as the inverse is an unbearable situation if my experience so far is worth something. Whatever that will require, we should consider and talk about it, as that's the root cause of the problem I think.

As I've said above, while I agree that the Web should be a priority for WebAssembly, I don't believe that the technical disagreements you've been having with other CG members are primarily due to a Web vs non-Web factional split. There are reasons why even purely Web-focused people would want to pursue alternatives to several of your proposals, and you shouldn't interpret this as evidence that the people disagreeing with you don't care about the Web.

WASI is not going to be a W3C spec, and it does not interfere with our production of the W3C WebAssembly spec. Declaring that WASI should not be developed under the CG because it's "non-Web" will not result in the people involved turning their attention to the Web. It would just close down valuable lines of communication and collaboration, that may (for example) allow us to find overlaps with some hypothetical Web-friendly API.

Fair points, but we fundamentally disagree there then, as I cannot reconcile your standpoint with all I have been through. I think there are very clear indications already that the problem is there and must be addressed instead of constantly telling me that there is nothing we can do about it. To me this seems absurd, up to a point where people should be ashamed of what they are doing to WebAssembly, once the brightest star among Web standards that is clearly not living up to what it once went out to become.

I think there are very clear indications already that the problem is there and must be addressed instead of constantly telling me that there is nothing we can do about it.

I think we are failing to grasp the problem, rather than deciding that nothing can be done about it. There is obvious interest in improving WASI or interface types, but it does not appear that there is consensus about those being fundamentally broken or wrong.

Thank you for unlocking the issue :) I want to apologize for my meltdown here, which was not OK and I expect much better from myself. I want to continue engaging with the group, including discussing this (as I think) important issue, and I hope to show that I can do this in a constructive way.

As a start, I have proposed a presentation about my concerns regarding string encodings, and where I think it is relevant for the next steps in Interface Types respectively the more recent development towards a Component Model. And in the process, we ultimately decided to try out a new format of a pre-recorded presentation, with discussion time scheduled for June 22nd's CG video meeting. The recording can be found in the accompanying issue linked right above this comment, and I hope it helps to provide the necessary background in a more condensed, constructive way.

If I may offer a light-hearted suggestion: If someone were to recompile the non-WASM portions of a web-browser's source code in Binaryen or Rust, targetting WASI on the desktop, would it suffice to say that most of the browser is just a big polyfill or would it just turn the argument inward upon itself, cause a catastrophic collapse and turn the Earth into a black hole?

On a serious note: IIRC, the web started out as a hyperlinked documentation of Sir Tim Berners-Lee's phone directory at CERN. The web itself has exceeded all expectations including those of complexity and memory consumption. The web browser has evolved from being a document viewer into an entire publishing medium. That is feature-creep.

WASI, as a programming environment, has the potential to return web technologies to their roots as a branch of computer science where publishing and interactivity merge. Since when does a chatroom have to run in a browser to be a web technology? IRC, the original chat protocol, has been around longer than the web browser has anyway. Likewise, SMTP predates WWW and so does FTP. The Internet is bigger than just the web and that's the way it was supposed to be.