> a successful parse consumes at least one rune This avoids infinite loops?
Incremental Parsing in Go
21–30 of 78 posts
Re: Incremental Parsing in Go
#22For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.
Re: Incremental Parsing in Go
#23Earlier quoted context omitted.
This is a premature optimization, and keeping everything in the same language has benefits like greatly simplified tooling and building
It’s not a premature optimization - it’s deciding the maximum that the parser can be optimized in the future. Choosing Go sets a lower ceiling. > Keeping everything in the same language has benefits like greatly simplified tooling and building Surely there are other Go libraries that incorporate C, C++, or Rust? Also if both parsers existed and were equally easy to set up, and you were planning on doing a ton of pars…
https://github.com/aretext/aretext/blob/main/Makefile
Build is literally a `go build ...` and install is `go install`. Adding any other language to the mix would make this a polyglot project and not be "equally easy to set up". The other question is, do both parsers exist? In this write-up they point to tree-sitter as a possibility which is a JS program that produces C code. This would be viable, but here's the author's take:
> I considered integrating tree-sitter, an incremental parsing library with parsers for many existing languages. However, running JavaScript to generate parsers and linking to a C library would have greatly complicated the build process. Today, aretext can be built on almost any platform using a single go install command. I’ve had users install aretext on ARM laptops, FreeBSD servers, Chromebooks, and Android phones. To maintain portability, I wanted a pure Go implementation.
So this wasn't some casual decision, but something they at least considered long enough to describe here.
And the parsing library itself is only around 1200 lines total (comments, blanks, and code). The parsers for each language add a lot more, of course, but should be roughly equivalent given the same library and interface. I imagine that if this project really takes off and performance becomes a real problem they can do the rewrite at that point. Right now, the code works, seems to work fast enough for its author and primary users, and it's trivial to install on any platform supported by Go. So yes, it would have been a premature optimization to complicate the build process, probably reduce the number of supported platforms (or greatly increase the effort to support the same number of platforms), just to have a slightly faster parser.
Re: Incremental Parsing in Go
#24Earlier quoted context omitted.
One of Go's primary goals has always been compilation speed. Go started out in C, and was later (post-1.0) incrementally rewritten to be self-hosting. One of the authors (Ken Thompson) is also one of the co-creators of C. I would argue these guys know what they are doing.
I don’t know, not implementing generics when it was pretty obviously needed was a huge oversight, so I’m not sure. Also, the reason for compiler bootstrapping is more of a “beauty thing”, then practicality. It would definitely be faster in a low-level language, but I doubt it would matter as an end user.
Re: Incremental Parsing in Go
#25For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.
Go is a fast language even with a GC.
Re: Incremental Parsing in Go
#26Earlier quoted context omitted.
One of Go's primary goals has always been compilation speed. Go started out in C, and was later (post-1.0) incrementally rewritten to be self-hosting. One of the authors (Ken Thompson) is also one of the co-creators of C. I would argue these guys know what they are doing.
I don’t know, not implementing generics when it was pretty obviously needed was a huge oversight, so I’m not sure. Also, the reason for compiler bootstrapping is more of a “beauty thing”, then practicality. It would definitely be faster in a low-level language, but I doubt it would matter as an end user.
I get the desire for generics. I do a lot of C# and have used generics for a very many years. Yet I've been writing Go for around 6 or 7 years and other than in the beginning (when I was new to it) I haven't found myself missing them at all.
In other words, for many people the lack of generics comes across as an oversight. For others, including myself (again, a heavy generics user in C#) that really isn't the case. I write Go in the style of Go and it just hasn't been an issue.
Blanket statements are rarely true. YMMV.
Re: Incremental Parsing in Go
#27Earlier quoted context omitted.
If you're making something requiring CPU optimization as a core feature, you might as well go with one of the fastest languages instead of handicapping your project from Day 1. Go is not considered one of the fastest. It's better for network or filesystem logic that is I/O limited.
This is kind of a test of how nuanced your understanding of programming languages can be. Rust with a bit of effort put into optimization will be faster than Go with a bit of effort put into optimization, it is true. However, you need to double-check your intuition for how big and how consequential the delta is, because I'd guesstimate it as roughly a factor of two, possibly a touch less. It is true that Rust does a…
Re: Incremental Parsing in Go
#28Earlier quoted context omitted.
C library for parsing? isn't it dangerous from security perspective?
It's just an example. The options are really Rust (what I'd prefer), C++, C, or perhaps something like Nim that compiles to C. If you’re trying to make an unoptimized parser, then use whatever you want.
Re: Incremental Parsing in Go
#29Earlier quoted context omitted.
I don’t know, not implementing generics when it was pretty obviously needed was a huge oversight, so I’m not sure. Also, the reason for compiler bootstrapping is more of a “beauty thing”, then practicality. It would definitely be faster in a low-level language, but I doubt it would matter as an end user.
> not implementing generics when it was pretty obviously needed was a huge oversight I get the desire for generics. I do a lot of C# and have used generics for a very many years. Yet I've been writing Go for around 6 or 7 years and other than in the beginning (when I was new to it) I haven't found myself missing them at all. In other words, for many people the lack of generics comes across as an oversight. For others…
Sure, it is seldom missed as an end user, but as a library user it is essential. That’s why map and the like had to be hard coded into the language, and why concurrent versions couldn’t be implemented for a long time in the language, the same way it was done for Java forever.