Earlier quoted context omitted.
Sounds cool! If you have something public I'd love to see it. Email is in my profile.
It's not quite ready to share but hopefully soon :)
Best of luck & happy hacking
Incremental Parsing in Go
61–70 of 78 posts
Re: Incremental Parsing in Go
#62Earlier quoted context omitted.
Surely, that’s the incentive part for why the team spent many many work hours improving their GC - just because the JVM typically depends more on a good GC doesn’t make it any less useful - long running processes do make significant use of automatic memory management. Also, Java’s GCs are moving/compacting GCs, so while the immediate memory representation is indeed inefficient, again, for long running processes Java…
> just because the JVM typically depends more on a good GC doesn’t make it any less useful - I mean it feels like personal choice. Do I praise the spouse when they bring whole kitchen down while making a dish and cleaning up quickly afterwards? Or do I take it as "Well, you made mess so it was basic expectation from you to clean up fast for later use"
I would wager that most applications have plenty of object lifetimes that are not regular at all — a web server with some stateful sessions for example. So your analog doesn’t really make sense — go can’t avoid these situations at all and will significantly perform worse in these cases.
Re: Incremental Parsing in Go
#63This article uses the word 'rune' extensively. From the context I assume it means 'lexem' or 'token' (i.e. the unit the lexer/scanner produces and feeds to parser). But then the article uses the word 'token' to mean the output of a parser ('keyword token'), while the usual terminology is that parser output is called a 'parse tree'.
So, in this terminology, the parser consumes 'runes' and outputs 'tokens', while the usual terminology is that parser consumes 'tokens'/'lexems' and outputs 'parse tree'.
Re: Incremental Parsing in Go
#64Earlier quoted context omitted.
Well, cross-language benchmarking is always hard, but for purely testing the GC this is not too bad: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... See how ahead Java is of any other managed language (and it doesn’t really make sense to do this benchmark with non-GCd languages)? Though this is done with the G1 GC, not the low-latency one - this default GC is more throughput oriented with a max pause…
> this is also the reason why java is quite ahead on “energy efficiency” reports as well. Very soon businesses would be asking for "dollar efficiency" also. I think going by effort on Java and their frameworks vendors to pack more instances of Java process/pods on a VM, it is already been asked by tech savvy customers. So that old fact that on sever side programing customers only care for raw throughput and not on ma…
To be honest, I really don’t get this microservice/cloud hype, stackoverflow (which let’s be honest will be bigger than that 34th startup) runs on a single (very beefy though) dedicated server machine.
I pay like 5 dollars a month for a VM with very low params, but even that will happily run anything. Especially that the DB likely can’t be shared the bus factor will remain 1.
Re: Incremental Parsing in Go
#65> The parsers produce a sequence to tokens, not a full syntax tree. Writing a tokenizer is much easier than parsing full syntax trees...Most other editors don’t construct the full syntax tree. Syntax highlighting is nice, but fast semantic analysis is the real holy grail. I have come to think that the best way to develop a new language would be to implement a text editor in that language before releasing the language…
I think it would be sufficient & more valuable to implement a language server for your new language, which keeps you focused on the parts related to your language rather than the struggles of implementing an editor, and then you'll be able to drop into VSCode, neovim, etc.
I agree with the parent. The big advantage of what he's suggesting is code objects exist as first class objects. They don't have to creased by parsing a text file. Which means the editor can directly manipulate them.
If you've ever used CAD programs especially Schematic Capture and PCB Layout programs you'll get the idea. Everything displayed on the screen is an object in a database.
Instead of a parser blindly parsing a text file and coming across a struct definition, a struct is object created by the programmer. Which means you can have hard links between the objects that make up the program. And those can be directly manipulated with manually or programmatically.
The big advantage comes from maintenance and refactoring. Change the name of a field? Happens in exactly one place in the program. So instead of a diff with 1537 files changes you have 'renamed struct fobar to struct foobar'. Change a comment? Well it's just a changed comment.
Re: Incremental Parsing in Go
#66This article uses the word 'rune' extensively. From the context I assume it means 'lexem' or 'token' (i.e. the unit the lexer/scanner produces and feeds to parser). But then the article uses the word 'token' to mean the output of a parser ('keyword token'), while the usual terminology is that parser output is called a 'parse tree'. So, in this terminology, the parser consumes 'runes' and outputs 'tokens', while the u…
Rune is a type alias in go, which more or less maps to the more common words “character” or “code point”.
Re: Incremental Parsing in Go
#67This article uses the word 'rune' extensively. From the context I assume it means 'lexem' or 'token' (i.e. the unit the lexer/scanner produces and feeds to parser). But then the article uses the word 'token' to mean the output of a parser ('keyword token'), while the usual terminology is that parser output is called a 'parse tree'. So, in this terminology, the parser consumes 'runes' and outputs 'tokens', while the u…
Rune is a type alias in go, which more or less maps to the more common words “character” or “code point”. https://go.dev/blog/strings
Thanks! I don't know Go, so this terminology was surprising to me.
Re: Incremental Parsing in Go
#68Earlier quoted context omitted.
I think it would be sufficient & more valuable to implement a language server for your new language, which keeps you focused on the parts related to your language rather than the struggles of implementing an editor, and then you'll be able to drop into VSCode, neovim, etc.
I agree with the parent. The big advantage of what he's suggesting is code objects exist as first class objects. They don't have to creased by parsing a text file. Which means the editor can directly manipulate them. If you've ever used CAD programs especially Schematic Capture and PCB Layout programs you'll get the idea. Everything displayed on the screen is an object in a database. Instead of a parser blindly parsi…
I'd like to see languages like that, but it's a massive lift. Our systems of source control and CI/CD are built around the assumption that languages are text files that are somewhat line oriented. It's valuable for people to be able to use different editors and tools, and text files are the integration point that makes this work.
Fallible parsers are a bit of a hack but they're a hack that works and is widely deployed already, and I'm not convinced the value of moving to a symbolic/binary representation creates enough value to justify the risk involved in the migration. That being said I'd love to see it happen. We'd end the tabs and spaces debate once and for all!
Re: Incremental Parsing in Go
#69This article uses the word 'rune' extensively. From the context I assume it means 'lexem' or 'token' (i.e. the unit the lexer/scanner produces and feeds to parser). But then the article uses the word 'token' to mean the output of a parser ('keyword token'), while the usual terminology is that parser output is called a 'parse tree'. So, in this terminology, the parser consumes 'runes' and outputs 'tokens', while the u…
In Go, `rune` is an alias for `int32` and is used to indicate the value is a Unicode "code point".
For characters in the ASCII range, that means it's just a character encoded using more bits. If you need to worry about the full Unicode range then it's important to understand Unicode Normalization Forms.
Re: Incremental Parsing in Go
#70Earlier quoted context omitted.
You aren't sure if Ken Thompson knows what he's doing?
As a software architect? Absolutely. Programming language designer? Not sure, neither C or Go are good languages in my personal opinion. EDIT: I meant to write that I think very highly of him as an architect/developer.
Experience has shown that often “worse is better”. Go does an amazing job of balancing complexity and power. I haven’t seen a ”better” language that isn’t either slower, harder to become productive, or both.