Incremental Parsing in Go
dev-nonsense.com
Incremental Parsing in Go
1–10 of 78 posts
Re: Incremental Parsing in Go
#2For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.
Re: Incremental Parsing in Go
#3Interesting read. Thank you for sharing. I always found parsers fascinating and mystical. It seems like these parser functions (which i think are analogous to what Rob Pike calls state functions) are a common way to do parsing, though i know very little about it. I especially found the combinators intriguing, though I don't care much for the functional programming syntax in a language like Go.
Anyway, thanks for sharing.
Tangentially, I wrote a little mini parser [0] of my own for my side project. It is inspired by Rob Pike's talk on parsers [1]. It doesn't use state functions, but instead just uses the call stack to keep track of where we are.
[0] https://github.com/binwiederhier/ntfy/blob/main/server/actio...
[1] https://www.youtube.com/watch?v=HxaD_trXwRE and https://go.dev/src/text/template/parse/lex.go
Re: Incremental Parsing in Go
#4For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.
C library for parsing?
isn't it dangerous from security perspective?
Re: Incremental Parsing in Go
#5For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.
For some reason we insist that language parsers are implemented in the language itself, even when the language isn’t great for parsers.
Re: Incremental Parsing in Go
#6For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.
You're in for a big surprise. Try using the language.
Spend some time using Go, and you will be impressed by its performance.
You'll wonder, "Were all those haters on Hacker News misinformed?"
Re: Incremental Parsing in Go
#7For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.
C library for parsing? isn't it dangerous from security perspective?
It's just an example. The options are really Rust (what I'd prefer), C++, C, or perhaps something like Nim that compiles to C.
If you’re trying to make an unoptimized parser, then use whatever you want.
Re: Incremental Parsing in Go
#8For max optimization, wouldn’t it be better to create a Rust or C library for parsing that Go links into? I personally don’t see the usefulness of trying to optimize Go itself too much as it’s handicapped by the runtime and garbage collection.
You're in for a big surprise. Try using the language. Spend some time using Go, and you will be impressed by its performance. You'll wonder, "Were all those haters on Hacker News misinformed?"
If you're making something requiring CPU optimization as a core feature, you might as well go with one of the fastest languages instead of handicapping your project from Day 1. Go is not considered one of the fastest. It's better for network or filesystem logic that is I/O limited.
Re: Incremental Parsing in Go
#9Earlier quoted context omitted.
You're in for a big surprise. Try using the language. Spend some time using Go, and you will be impressed by its performance. You'll wonder, "Were all those haters on Hacker News misinformed?"
If you're making something requiring CPU optimization as a core feature, you might as well go with one of the fastest languages instead of handicapping your project from Day 1. Go is not considered one of the fastest. It's better for network or filesystem logic that is I/O limited.
This is a premature optimization, and keeping everything in the same language has benefits like greatly simplified tooling and building
Re: Incremental Parsing in Go
#10Earlier quoted context omitted.
You're in for a big surprise. Try using the language. Spend some time using Go, and you will be impressed by its performance. You'll wonder, "Were all those haters on Hacker News misinformed?"
If you're making something requiring CPU optimization as a core feature, you might as well go with one of the fastest languages instead of handicapping your project from Day 1. Go is not considered one of the fastest. It's better for network or filesystem logic that is I/O limited.
The optimization here is using incremental parsing, so that changing parse state goes from O(n) to may-as-well-be-O(1). It's probably linear with tree depth.
Any language is fast enough to do this, certainly Go is. Naive parser combinators written in slow languages can tokenize six-figure LOC files fast enough that the user won't notice.