So none of them are using Svelte parser?

svelte2tsx strips out style/script tags and then uses the svelte compiler to parse the htmlx part.
svelte-ts uses the svelte compiler as a second step.
The current language server does not use the svelte compiler for parsing.
I'll update the initial post.

I wasn't sure if I should share this, but some part of it might be useful. Last year (July/August 2019) I started playing around with typechecking Svelte, knowing that I was doing throwaway work just to learn. It supports some unknown amount of a full solution, maybe 30%, and it has some limitations (more on those later). There aren't any tests or external documentation because I've always considered it to be a dead end.

Svelte compilation goes as it does today with a non-typechecking TS preprocessor. If you wanted types in your HTMLx markup, not just your script tag, you'd need a preprocessor to handle that, or Svelte would need to handle TS internally.

The typechecker is a separate build step that uses the un-preprocessed TypeScript code from script blocks and the markup AST from svelte.compile to create a virtual Svelte TypeScript file solely for typechecking purposes. (here's where the Rollup plugin creates the virtual file and then typechecks it) It's similar to @halfnelson's strategy, but different in that it produces normal TS, not TSX, so in the markup it doesn't try to typecheck HTML tags and attributes, only the stuff inside mustache tags and Svelte components. It transforms Svelte components into constructors that get newed with their props. I got partway through supporting a lot of Svelte's constructs before realizing I got carried away more than I intended, and further progress should be Serious Engineering.

The TypeScript compiler API is used to construct a program in memory and typecheck the virtual Svelte TS. Unlike Svelte its view of the world is whole programs, not individual files. At the time it seemed like it would be straightforward to add incremental compilation and worker threads as future optimizations.

The virtual Svelte TS file is generated with a sourcemap which is then used to map TypeScript diagnostics back to the original Svelte source. It was pretty cool to see type errors point back to the Svelte source but I had no confidence in my approach overall.

One problem I remember that I couldn't figure out was with diagnostics that weren't specific enough in some cases. IIRC an example was with typechecking Svelte component constructor props - VSCode somehow would point to individual prop errors, but the TS diagnostics I was getting were pointing only to the constructor, not the problem property.

If you look through the code be aware that parts are a mess and some comments may be outdated. I originally used magic-string and at some point changed over to source-map. And remember it was a doomed endeavor from the start! Note that all of this is on the ts branch of that repo, not master.

Here are the 4 files referenced above:

• virtualSvelteTs.ts
• typecheck.ts
• tsDiagnostics.ts
• rollup-plugin-svelte-typescript-checker.ts

I did some digging in the source code of the current svelte language server. Here are my thoughts on the typescript part of it:

Overview

The language server broadly works likes this:

• DocumentManager handles all documents that get opened.
• TypescriptPlugin registers at the DocumentManager to get invoked on language server events ("doHover" etc). It does not register directly: It gets wrapped with wrapFragmentPlugin which makes sure TypescriptPlugin only gets the content inside the script tag. Mapping of positions (of mouse/text cursor) is done in wrapFragmentPlugin to ensure the infos are shown at the correct position.
• TypescriptPlugin uses service, a typescript language service wrapper. It basically delegates all events to this language server, with some mapping to adhere to the method return types.
• service takes the document to do work on aswell as a createDocument method from TypescriptPlugin. service keeps his own map of documents which are at the moment basically only a delegate to the document of the DocumentsManager. Everytime service is invoked from TypescriptPlugin, service updates its documents map with the given document (after wrapping it). createDocument would be invoked if the language service opens a document that is not yet part of the documents map - but this will never happen since typescript language service does not find other svelte modules (see next section).

Current shortcomings and ideas for improvement

• Since the script tag is given as is to the typescript language service, it cannot deal with special svelte syntax ($).
• The typescript language service tries to lookup other files that are referenced in the currently open svelte file. For .ts/.js files this works, for .svelte files it does not. Reason: The typescript language service does not know about the .svelte ending, so it just assumes it's a normal typescript file and searches for files like ../Component.svelte.ts, which is wrong. In order to fix this, we need to implement the method resolveModuleNames on LanguageServiceHost and reroute all .svelte.ts file lookups to .svelte. Then the language service will walk all svelte files. NOTE: In order to implement this, we also need to fix the following bug: Since the typescript language service now would walk all files, it would invoke createDocument from TypescriptPlugin. But this currently returns the whole document, not only the content inside the script tag. This is a bug inside the wrapFragmentPlugin not wrapping openDocument.
• In order to implement more advanced features like "go to svelte file" by clicking on the svelte component import name, or make typescript grog the $ signs, we need to add a preprocessor. This preprocessor would somehow add the missing parts for typescript. Ideally, all additional stuff can be put on top of the existing code, so that mapping the positions is as easy as mapping the positions with offsets. This poses another challenge though: We now would have two position mappings: One from the whole svelte file to the script tag, and one from the script tag to the preprocessed typescript code. I'm not sure at this point if this is the way to go or if we should rework how TypescriptPlugin (and others) get its document - maybe both extraction of the script tag and preprocessing should be done in one step. This would mean changing or fully replacing the wrapFragmentPlugin. Another argument for reworking that is that, in order to correctly determine things like "unused function", we have to take the html part into account, so the script tag is not enough.

Thoughts?

On resolveModuleNames, here's how a TypeScript team member did it for Vetur, and here's a similar version I did for Svelte, which includes a comment that mentions an alternate strategy that worked, but copying Vetur seemed better.

@dummdidumm do you mind acting as tech lead and breaking these things out to specific issues? i would be happy to volunteer dev time, but i find alot of this overwhelming and would work better in small, defined issues. Unless there is some big architectural change that needs to be done (looks like no?)

Basically just open up a bunch of issues and let people claim them and manage them as our informal tech lead on this. I'd offer to do it but honestly just havent felt the pain yet haha

also linking to orta's original LSP post for others who are browsing https://github.com/sveltejs/svelte/issues/4518

Sure I would love to help on this, but right now I'm just a random dude who is very excited about the language server, digging into the code and making some suggestions 😄 I also need some more time to get familiar with the code, its architecture and lsp in general. But I'm definitely willing to push stuff forward. @orta since you are the maintainer of this repo, what do you think about this?

👋 Yeah, we both are random people who just care - I don't even have any svelte codebases to trigger my "this should be better" itch. So, I'm very happy to see any movement at all - @dummdidumm - push away and I'll back you up and try make sure everything runs smoothly

Ok great! I will go ahead and make some PRs restructuring some of the code and preparing it so we can then parallelize working on individual items.

Maybe we could combine @ryanatkn, svelete-ts's approach and eslint-plugin-svelte3's variable approach:

1. Use svelte preprocess to preprocess script into js.
   Here we could just tranpile to latest or next version of ES to do as little transformation as possible.
2. Let svelte compile the preprocessed source.
3. We can then inject result vars where injected = true into instance ts source or ts AST and then use it with source map to provide type check and auto-completion.

The script can have three fragments: module <script context="module">, instance and mustache (template). The mustache part could just be all dump into a template.js for now. But for the long term we could reflect template into tsx or plain ts like ryanatkn's approach.

Module vars, where module = true, would be injected into template.js and instance but not the other way around.

In this way we don't have to re-implement on how to find reactive variable $: a = 1 or $-prefixed store in typescript AST. These variable injected by svelte compiler derive from top-level statement and we're transpiling it into latest ES. So it should mostly not be rename by typescript.

About reactive variable $: a it could be written like this:

$: a = 1

```ts
let a: number
$: a = 1

```ts
$: a = someFunction(b) as Type

All it's valid ts that doesn't need transform before transpile

And $-prefixed store we can create a generic function like svelte/store get to extract store type

/** injected */
let $store = getType(store)

1. Use svelte preprocess to preprocess script into js.
   Here we could just tranpile to latest or next version of ES to do as little transformation as possible.
2. Let svelte compile the preprocessed source.
3. We can then inject result vars where injected = true into instance ts source or ts AST and then use it with source map to provide type check and auto-completion.

I think svelte-ts does something like that. Seems like a good idea. On injected = true: What code exactly has this property?

The script can have three fragments: module <script context="module">, instance and mustache (template). The mustache part could just be all dump into a template.js for now. But for the long term we could reflect template into tsx or plain ts like ryanatkn's approach.

Module vars, where module = true, would be injected into template.js and instance but not the other way around.

In this way we don't have to re-implement on how to find reactive variable $: a = 1 or $-prefixed store in typescript AST. These variable injected by svelte compiler derive from top-level statement and we're transpiling it into latest ES. So it should mostly not be rename by typescript.

So you want to split up the file into a virtual .ts-file and a .template-file? Why not have the the mustache-parts on the bottom of the .ts-file? This way we also would get rid of "unused method"-warnings. On walking the AST: Yes, I think we can be smart here and skip looking into the AST too much because everything that is referenced in the template must be top level in script.

About reactive variable $: a it could be written like this:

$: a = 1

let a: number
$: a = 1

or

$: a = someFunction(b) as Type

All it's valid ts that doesn't need transform before transpile

Is it possible to infer the type? Maybe this is just something we leave as is and if the user wants to use typescript, he has to define let a: number himself. As an alternative, we could insert let a;, but then it would be any.

And $-prefixed store we can create a generic function like svelte/store get to extract store type

/** injected */
let $store = getType(store)

Yeah this seems nice. Another idea I had was to construct getters/setters.

I think svelte-ts does something like that. Seems like a good idea. On injected = true: What code exactly has this property?

The compile result has a property vars, which is an array of object that has the following property:
name, export_name, injected, module, mutated, reassigned, referenced_from_script and writable
you can see svelte compile api for more info

So you want to split up the file into a virtual .ts-file and a .template-file? Why not have the the mustache-parts on the bottom of the .ts-file? This way we also would get rid of "unused method"-warnings.

This is the approach of eslint-plugin-svelte3. Also I just thought if we want to support ts in template because there is no attribute to specify language.

Is it possible to infer the type?

Since this virtual source is only use for type check and autocomplete, I presume it can achieve by just copy the first assign statement to initialize it but it seems to be need more work.

Is it possible to infer the type? Maybe this is just something we leave as is and if the user wants to use typescript, he has to define let a: number himself. As an alternative, we could insert let a;, but then it would be any.

Looks like normal inference doesn't get us all the way there. (did you mean compile-time inference? not sure if that could work)

let a; // type is "any" initially
$: a = 5;
a; // type is now "number", good
const cb = () => a; // type is implicitly "any", noo

The reason is that in the linear code flow, TypeScript can see a wasn't reassigned, but nested function scopes don't have that guarantee, and TypeScript errs on the side of type safety.

playground example

Exactly, these were my worries with that. But as @jasonlyu123 pointed out, we could just copy the definition (everything after the =).

Before

$: a = true;

After

let a = true;
$: a = true;

But I don't know if that's feasible for all circumstances. Especially with large objects where someone implements an interface, this will not infer everything as the user might expect. Also, the copy-pasta could get pretty big. So I'm still more on the side of "let the user type it".

Thanks for clarifying, I misunderstood what @jasonlyu123 said. It fixes the nested function scope issue I mentioned.

Is there an example of when that's not good default behavior? It seems like it does the ergonomic right thing most of the time, and other cases can be fixed manually. Here's some examples.

Mhm I guess you're right. It is the most ergonomic thing. What I worried about is if the inferred type is wrong, for example property is string instead of a defined union of strings ('a' | 'b'). But as you said, these cases are not that common and then the user still can type it manually.

There's a part of me that's like: "great, the explicit let syntax works awesome" but that would end up not working with JS support out of the box

What about just replace first label to let:

$: a = 1
$: a = 2

get transform to

/** injected  */
let a = 1
$: a = 2

I rethink a bit and can't find the advantage of "copy definition" over "replacing $: with let". Can you guys think of any disadvantage of replacing $: with let?

About inferred type difference to what user expected, can we provide a refactoring actions to convert it to manual define? This action will prompt user to manual type the variable like this, Or refactor to its inferred type if possible.

before

$: a = 'const1'

after

let a : AskUserForType
$: a = 'const1'

Also thinks for the things you point out, Learned something 👍

Ha, smart idea, just entirely replace the $: through a let. I also don't know any situations where that would end up bad.
We also could think about always doing the transformation, not only for the first label. Then there would be a "cannot redeclare"-error which hints the user at "woops, I defined this variable twice" - or are there situations where this would be viable and not a mistake?

Nice, I can't think of any reason replacing the label would fail either. (since break $ doesn't ever work inside assignments, right?)

For redeclaring the same reactive variable, it looks like Svelte allows this and it works in order like you'd expect. (see this repl example) I don't personally see myself using this pattern intentionally and would want the error, but it's in the territory of interpreting Svelte's semantics so don't take my word for it!

OK so redeclaring it is a pattern that actually works. I would say we better allow it then - or make a "redeclare not allowed"-feature flag later which is off by default.

@halfnelson has created a language server based off his svelte2tsx approach in https://github.com/halfnelson/svelte-type-checker-vscode . I really love how good this all works already, but I still would prefer to not use tsx as an intermediate output because I fear it could be a little too slow and have some limitations - but that's just my gut feeling. Still, we could take a lot from svelte2tsx: How the flow of parsing-compiling-transformation is done, and also the extensive test suite. How do others see the approach of transforming the code to tsx?

I though about how other transformations could look like. Here are some examples. Feedback welcome.

How a component definition looks like

Original:

<script>
    import { createEventDispatcher } from 'svelte';
    const dispatch = createEventDispatcher();

    export let todo: string;

    function doneChange() {
        dispatch('doneChange', true);
    }
</script>

Converted:

export default class Component {
   constructor(props: {todo: string; 'on:doneChange': (evt: CustomEvent<boolean>) => void}) {}
}

import { createEventDispatcher } from 'svelte';
const dispatch = createEventDispatcher();

export let todo: string;

function doneChange() {
    dispatch('doneChange', !todo.done);
}

Explanation:
Somehow get all all props (easy) and dispatched events (hard) and add them to one big props object. Adding events with on:-prefix should minimize collisions and make autocomplete suggestions easier to implement.

(in the following examples, component-definition is omitted for brevity)

Use variable in template

Original:

<script>const bla = {bla: 'bla'};</script>
<p>{bla.bla}</p>

Converted:

const bla = 'bla';
const _bla = bla.bla;

Explanation:
Add arbitrary const elements, maybe prefixed with some obscure unicode character, to check correct usages of variables and to also make the "unused"-warning go away. Maybe we then need to think about how to get rid of the unused _bla-warning ...

Use native event listener

Original:

<script>function bla() {return true;}</script>
<button on:click={bla}>bla</button>

Converted:

function bla() {return true;}
const btn = document.createElement('button');
btn.addEventListener('click', bla);

Explanation:
When using native event listeners, create the corresponding element and then add an event listener. This way we take advantage of the addEventListener-typings which have overloads for pretty much every listener out there: .addEventListener('click', ...); -> event is of type MouseEvent. This pattern would be used after having checked if the element is another svelte component which has the event defined (also see next example). Shortcomings: What if Svelte is used in a non-web-environment (Nativescript)?

Use other component

Original:

<script>
import Bla from './bla.svelte';
function blub() {}
</script>
<Bla bla={1} on:blubb={blub} />

Converted:

import Bla from './bla.svelte';
function blub() {}
const bla = new Bla({bla: 1, 'on:blubb': blub});

Explanation:
Every component gets a class definition, so we just instantiate that class.

Each loop

Original:

{#each todos as todo,i (todo.id)}
    <p>{todo.text}</p>
{/each}

Converted:

todos.forEach((todo, i) => {
    const _todoText = todo.text;
});

Explanation:
forEach seems like the easiest to convert to. Inside, we recursively can apply all other features.

If

Original:

{#if x === 1}
    <p>if</p>
{else if x === 2}
    <p>elseif</p>
{else}
    <p>else</p>
{/if}

Converted:

if (x === 1) {

} else if (x === 2) {

} else {

}

Explanation:
Pretty self-explanatory.

Missing pieces

• slots
• special svelte:x components
• other element/component directives such as use:action, transition, bind
• $-syntax

What do you guys think? Are these transformations viable and doable? Did I miss something?

About component class I would prefer extends svelte's own SvelteComponent class, but i don't know if this is easy to implement or not

import { SvelteComponent } from "svelte";

export default class Component extends SvelteComponent {
    constructor(options: ComponentOption<{ propA: string }>) {
        super(options)
    }

    $on(
        event: 'input',
        callback: (event: CustomEvent<string>) => void
    ): () => void
    $on(
        event: 'click',
        callback: (event: CustomEvent<number>) => void
    ): () => void 
    $on(
        event: string,
        callback: (event: CustomEvent<any>) => void
    ): () => void {
        return () => { }
    }
}

and import these interface

interface ComponentOption<TProps, TSlot = undefined> {
    target: Element,
    props: TProps | SlotProp<TSlot>
}

type SlotOption<T> = [unknown, unknown, unknown]

interface SlotProp<TSlot> {
    $$slots: Record<string, SlotOption<TSlot>>
}

Noted $$slots is probably svelte's internal api, I just dig it from compiled code

The reason for this is that it can then be use to emit .d.ts that could be use to type in vanilla ts/js.
Also because, although weird enough,this is valid syntax:

<script>
onMount(() => {
    new Component({
        target: document.getElementById('foo')
    })
})
</script>

Element directives

Transition, use:action and other element directives could be transform like

fade(null as Element, {  } /* option in attribute*/)

Await

{#await promise}
    <p>...waiting</p>
{:then number}
    <p>The number is {number}</p>
{:catch error}
    <p style="color: red">{error.message}</p>
{/await}

to

    promise.then(number => {  number })
    promise.catch(error => { error.message })

bind:this

<script>
let a;
</script>
<div bind:this={a}><div>

to

let a;
onMount(() => {
 a = document.createElement('div')
})

needed to wrap it in callback because it's not immediately available

context="module"

This is a bit tricky

<script context="module">
 let count = 1;
</script>

<script>
import _ from "lodash"
let b;
</script>

should be compile to

import _ from "lodash"

let count = 1;

// any block is ok
{
    let b;
{

And also what if the user somehow use different language in two script, like js in module and ts in instance?

Nice, looking good. For "Use variable in template" const _bla = bla.bla; could be just bla.bla; letting TypeScript do its check without needing to filter out the "unused var" diagnostic. I can't think of any corner cases where this doesn't work.

Some of the Vue team are exploring a typescript language server plugin - https://github.com/znck/vue-developer-experience

It's an interesting angle as today these experiences go from outside TypeScript and in (like vetur does, as it runs it's own tsserver-ish, as well as html/css/etc LSPs) or do you try go from inside TypeScript and work outwards (e.g. this proposal) where you manipulate the TSServer to effectively believe that the .vue files are legitimate TypeScript by masking non-TS code.

Though today it can only work by having patches to typescript

That's an interesting approach. But how would features like autocompletions for html or css work then? Would one have to implement this all "by hand" and not rely on the out-of-the-box-language-servers anymore, or am I missing something?

Do you see any advantages of doing one over the over?

Anything but the JS/TS support would be handled by a separate vscode LSP. It would be like if this extension removes all support for JS/TS and only handled the rest, then the typescript plugin would handle the rest.

Advantage wise you get all of the TS tooling "for free" in this case, jump to symbol, file mappings etc

I'm probably not recommend it, today it requires forks or patches of TypeScript to work which is a pretty high barrier to entry - and the TypeScript team is still not sure if/when compiler plugins could have that kind of access.

Thanks for the insights. I would say we with the current approach then (outside-in).

If we go with the "make a virtual ts file"-approach discussed above, how would we implement this and keep source mappings?

Today, it's easy because it's just "find start of script, find end of script", "extract script part" and for mapping "add offset of svelte file to positions".
If we change the script part now, we need a more sophisticated mapping:

Original

<script lang="typescript">
  let a = 1;
  $: b = a + 1;
</script>
<p>{b}</p>

Mapped:

export default class Bla extends SvelteComponent { ... } // <- prepended, no mapping needed as far as I know
let a = 1; // <- untouched
let b = a + 1; // <- modified in place. How to map? Do we need to adjust all following code positions now to have a new offset/position?
b; // <- appended. Needs mapping from {b} inside svelte-mustache-tag to here. We can get the original position from the html ast which is part of the output of svelte.parse

Any ideas? I never did source mapping stuff before..

Some of the Vue team are exploring a typescript language server plugin - https://github.com/znck/vue-developer-experience

It's an interesting angle as today these experiences go from outside TypeScript and in (like vetur does, as it runs it's own tsserver-ish, as well as html/css/etc LSPs) or do you try go from inside TypeScript and work outwards (e.g. this proposal) where you manipulate the TSServer to effectively believe that the .vue files are legitimate TypeScript by masking non-TS code.

Though today it can only work by having patches to typescript

The remove TypeScript patch PR is merged.
https://github.com/znck/vue-developer-experience/pull/7

So ... @orta this means the disadvantages regarding "unofficial" are no longer the case? Or is it still kind of unofficial, just not through a patch but through some package.json config you have to know of and could change at any time?

Another topic: At the moment I feel we still are in discussion on how to approach this best, and if we decide, then it's going to take some time to implement it. What do you guys think of using svelte2tsx in the meantime? It could serve as a very good starting point, we could get some tests around it and then we either see "ok this actually works great, let's not change it" or gradually migrate away from it. Thoughts?

Aye, there were two ways it was editing TS's default flow, It's still patching TypeScript - but at runtime by replacing TS' functions. I think it's reasonable, given plugin development isn't on the roadmap yet.

I'd be a bad core team member if I recommended it though ;)

Trying out svelte2ts sounds like a good avenue for exploration!

You can see what it would be like with svelte2tsx using this vscode plugin: https://marketplace.visualstudio.com/items?itemName=halfnelson.svelte-type-checker-vscode together with Svelte Beta (works best if you disable svelte beta's typescript options so you don't get double the hints :) )

Even if not using svelte2tsx, I see discussion on which transforms need to be made to svelte's JS code to keep typescript happy. The test suite for svelte2tsx covers not only changes needed to be made to the template, but also the script tag, which if you are going for a script tag only solution might be a good starting point: https://github.com/halfnelson/svelte2tsx/tree/master/test/svelte2tsx/samples

the htmlx2jsx is the folder the template change tests (htmlx), while the svelte2tsx samples are the svelte specific changes needed for svelte's special abuse of js syntax :)

Just discovered this repo https://github.com/pivaszbs/svelte-autoimport . Maybe worth looking into if we want import intellisense independent of svelte2tsx.

I think we're at a point where most of the general direction for this project is sorted and seems to be working out neatly. I'll give this a week for feedback, but I'm pretty happy that we don't need to re-think it architecturally now.

Seems noone has objections, so I'm gonna close this one. Thanks everyone for discussing this!