Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I’m porting the TypeScript type checker tsc to Go

kdy1.dev

Post reply on HN
Print view View original

141–150 of 252 posts

Re: I’m porting the TypeScript type checker tsc to Go

#141

Post by slaymaker1907 »

jupp0r wrote: post #19

> tsc depends on shared mutability and has a cyclical mutable reference. Rust is designed to prevent this behavior. Having two references to the same data (shared mutability) is undefined behavior in Rust. This problem has numerous solutions in Rust - (A)Rc, RefCell etc. I don't understand how this caused the author of the article to stop considering Rust.

While I hate Go and love Rust, there is also the issue that RefCell/Gc/Rc are very different in terms of how expensive a memory allocation is compared to node. Historically there were very similar problems in trying to write C++ like one would write Java. Memory allocators today are much better and closer to GC languages, but I suspect it will still be more expensive for very allocation heavy code.

One way you could get this to work at the cost of some safety (depending on how you do it) would be to use arena style allocation. Obviously there are ways to do this safely in Rust, but it is probably easier to do it unsafely for a port.

However, if you just use a high performance language with a GC like Go/C#/Java, the port will be easier and you won't have to worry about doing too many allocations.

Re: I’m porting the TypeScript type checker tsc to Go

#142

Post by zem »

davepeck wrote: post #68

(In a different direction, Microsoft's pyright type checker for Python -- IMO the fastest and most correct type checker in the Python ecosystem -- is implemented in TypeScript!)

as someone who works on a competing typechecker, i'm definitely very impressed by pyright, especially since it has a single author! we have three people working on pytype and it definitely feels like too little; the way the pyright author manages such a great development velocity is mindblowing, and means they got the initial architecture very right.

Re: I’m porting the TypeScript type checker tsc to Go

#143

Post by reuben364 »

dmitriid wrote: post #87
Waterluvian wrote: post #75

Earlier quoted context omitted.

I am absolutely blown away at how good TypeScript's type checker is, given how much of an unholy mess JavaScript is, and that the TypeScript language cannot just deprecate all the awful parts. More than once I've yelled at my screen, "HOW DO YOU KNOW THIS?!" about some inferred or narrowed type.

However, more than once I've also yelled "HOW DO YOU NOT KNOW THIS?!" at something or other :)

I've managed to get the compiler to typecheck

  const x: Thing = exprA;
  const y: Thing = exprB;
  const z = [x, y];
but somehow not:

  const z: Thing[] = [exprA, exprB];
although exprA and exprB were pretty nasty typing wise.

Re: I’m porting the TypeScript type checker tsc to Go

#144

Post by spion »

Androider wrote: post #139
spion wrote: post #72

Earlier quoted context omitted.

62x improvement is not achievable. 6.2x might be. edit: I see they are running it with 8 threads, in which case yes 50x is achievable. In any sizable codebase though, you should probably split into modules and run with some parallelism via e.g. `wsrun` or `turborepo`

esbuild is > 125x faster than Webpack, doing the exact same job. It's not some theoretical microbenchmark, you'll see it when converting a project from one to the other. If a software hasn't been designed with speed in mind, and I don't think tsc has, there can be massive performance improvements possible if you create a new implementation with performance as a top goal.

Webpack does way more than esbuild, including running a typechecking compiler instead of just transpiling, running compilers able to downlevel emit to ES5 and providing a deep plugin architecture allowing you to hook into any bit you like. But yes, it hasn't been designed with speed in mind - it has been designed for maximum extensibility instead. Its the same reason why Babel is slow compared to sucrase (written in JS, currently faster than SWC and esbuild but doing somewhat less - https://github.com/alangpierce/sucrase)

tsc has in fact been designed with speed in mind (I've been following the project since before it moved to GitHub, which is when they redesigned it for speed). Going beyond 1 order of magnitude performance improvement while implementing the full feature set is highly unlikely.

Re: I’m porting the TypeScript type checker tsc to Go

#145

Post by pcj-github »

hardwaregeek wrote: post #32

This reminds me of when people try to make a new Ruby implementation. They get the basics down and think "this isn't so bad! I can do this!". Fast forward a few months and they're stuck on some nasty little edge case around eval and method_missing that they need to run Rails. Or some weird aspect of C extensions. TypeScript's compiler is an unholy, unsound mess, because JavaScript is an unholy, unsound mess. Sure the…

No spec, but the golden files that check that the compiler is working as intended are very comprehensive. If you can generate the same ~38,000 files in this directory from the test cases, you're pretty much on par with tsc: https://github.com/microsoft/TypeScript/tree/main/tests/base...

Re: I’m porting the TypeScript type checker tsc to Go

#146

Post by marcus_holmes »

moltonel3x wrote: post #138
marcus_holmes wrote: post #51

Given that ESBuild [0] is a way better replacement for Webpack already (and similarly written in Go) I can see how we could pretty soon have an entire toolchain written in Go for dealing with compiling JS. Hopefully the whole toolchain will be embeddable, too. Being able to do configuration-in-code and produce an executable that does the whole build step in milliseconds would be cool. [0] https://esbuild.github.io/

SWC (by the same author as this tentative tsc port) is a part of more and more people's typescript toolchain, and is written in Rust. Having all your ecosystem in a single language feels nice, but pragmatically different tools might be better in different languages.

The point of having it all embeddable in Go is that we could write a single executable that handles the entire process, with the toolchain embedded and the configuration written as code.

Re: I’m porting the TypeScript type checker tsc to Go

#147

Post by m-hilgendorf »

speedgoose wrote: post #44
jupp0r wrote: post #19

> tsc depends on shared mutability and has a cyclical mutable reference. Rust is designed to prevent this behavior. Having two references to the same data (shared mutability) is undefined behavior in Rust. This problem has numerous solutions in Rust - (A)Rc, RefCell etc. I don't understand how this caused the author of the article to stop considering Rust.

I think the point is valid. As a rust beginner I tried to make a graph data structure and it was difficult. Using RefCell is not very nice because it moves the problem from compile time to runtime and you still can’t mutate the same data structure more than once at the same time.

Rust encourages you to use an adjacent list/matrix for your graph representation, which is preferable almost all of the time for graph algorithms. It's a little unwieldy if the graph needs to be mutable during the core algorithms, but almost all algorithms in practice assume it will be stable.

Re: I’m porting the TypeScript type checker tsc to Go

#148

Post by constexpr »

sam0x17 wrote: post #110
ngrilly wrote: post #101

Earlier quoted context omitted.

The only string handling a compiler needs is usually during tokenization. After that the compiler only manipulates tokens, AST, SSA, IR, etc.

And then you face Go's usual awkward issues with having a lack of generics which becomes important when, say, traversing a tree.

I'm the author of esbuild. I hadn't written Go before I started esbuild and I thought I would miss generics more, but I didn't. There is literally only one situation in the whole esbuild code base where I've found myself wishing for generics: sorting an array. The workaround is easy and involves implementing an interface with three methods, each of which is typically a single line: https://pkg.go.dev/sort#Interface. And I only needed a few of these interfaces for a few different types. That wasn't nearly enough of a reason to not use Go, especially considering the other advantages.

Re: I’m porting the TypeScript type checker tsc to Go

#149

Post by jupp0r »

speedgoose wrote: post #109
tomcam wrote: post #57

Earlier quoted context omitted.

This is interesting but I don't know Rust. Presumably your graph data structure grows dynamically, meaning you have to add nodes at runtime. When. you say you can't mutate the same data structure more than once at runtime, what does it mean? That you can't set both a parent and a child pointer in the same function, or what?

I’m very far from being a rust expert but it was difficult to build a bi-directional graph because it’s difficult to connect the nodes together. A simple recursive algorithm will not work. I ended up using a crate. I found the following article interesting on the topic : https://aminb.gitbooks.io/rust-for-c/content/graphs/

It shouldn't be difficult:

1. You have a graph struct that has ownership of all nodes (in Rc containers).

2. Nodes have Weak references to connected nodes

Re: I’m porting the TypeScript type checker tsc to Go

#150

Post by jacobr1 »

pcj-github wrote: post #145
hardwaregeek wrote: post #32

This reminds me of when people try to make a new Ruby implementation. They get the basics down and think "this isn't so bad! I can do this!". Fast forward a few months and they're stuck on some nasty little edge case around eval and method_missing that they need to run Rails. Or some weird aspect of C extensions. TypeScript's compiler is an unholy, unsound mess, because JavaScript is an unholy, unsound mess. Sure the…

No spec, but the golden files that check that the compiler is working as intended are very comprehensive. If you can generate the same ~38,000 files in this directory from the test cases, you're pretty much on par with tsc: https://github.com/microsoft/TypeScript/tree/main/tests/base...

But that only works for a bug-for-bug compiler, no? If you want to build something like a linter, you couldn't use them to validate you are covering the fully syntactic range. Though I guess it would help look at coverage if your linter threw any errors, but it wouldn't validate correctness.
Post reply on HN