Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a SQL database from scratch in Go

notes.eatonphil.com

Post reply on HN
Print view View original

11–20 of 55 posts

Re: Writing a SQL database from scratch in Go

#11

Post by dilap »

arcticbull wrote: post #10
throwlaplace wrote: post #4

Earlier quoted context omitted.

Lexical analysis using these bespoke methods (writing the finite state machine) is so tedious and error prone. I don't have that much experience but I just went through crafting interpreters and replaced this same module with https://github.com/J-F-Liu/pom , which is a parser combinator library, and it was way easier.

Parser combinators are neat. nom is my go-to now in Rust, it's nice not have to separate your lexer and parser in simpler grammars.

(it’s actually also straightforward to write a recursive descent parser directly on the char stream w/o a lexing step.)

Re: Writing a SQL database from scratch in Go

#12

Post by throwlaplace »

dilap wrote: post #11
arcticbull wrote: post #10

Earlier quoted context omitted.

Parser combinators are neat. nom is my go-to now in Rust, it's nice not have to separate your lexer and parser in simpler grammars.

(it’s actually also straightforward to write a recursive descent parser directly on the char stream w/o a lexing step.)

how? it's not like you can magically skip the actual tokenization. you're basically saying you can do lexical analysis and semantic analysis in the same function. sure but that makes the code that much hairier - there's a reason why they're typically factored into a lexer and a parser.

Re: Writing a SQL database from scratch in Go

#13

Post by userbinator »

ycnewsreader wrote: post #8
userbinator wrote: post #5

selectKeyword keyword = "select" I don't work with Go, so this may be a requirement of the language that I don't know, but whenever I see lines like this, it automatically brings up the question why? --- do you really expect to need to rename the SELECT keyword? Especially when it's named "selectKeyword". Why not just use the string constant? Ditto for the others like "leftparenSymbol" --- I see there's explicit char…

Another reason is that by providing an identifier for this string literal, misspellings of it can be detected by the compiler whereas there is nothing tying separate string literals together.

I can hardly imagine a case where you would misspell "select" and not notice it at some point, nor use it in more than one place (the keyword detector) in the parser.

Re: Writing a SQL database from scratch in Go

#14

Post by arcticbull »

dilap wrote: post #11
arcticbull wrote: post #10

Earlier quoted context omitted.

Parser combinators are neat. nom is my go-to now in Rust, it's nice not have to separate your lexer and parser in simpler grammars.

(it’s actually also straightforward to write a recursive descent parser directly on the char stream w/o a lexing step.)

Possible sure, but without a combinator framework, you have to do a ton of the rote work yourself.

Re: Writing a SQL database from scratch in Go

#15

Post by triyambakam »

userbinator wrote: post #13
ycnewsreader wrote: post #8

Earlier quoted context omitted.

Another reason is that by providing an identifier for this string literal, misspellings of it can be detected by the compiler whereas there is nothing tying separate string literals together.

I can hardly imagine a case where you would misspell "select" and not notice it at some point, nor use it in more than one place (the keyword detector) in the parser.

You'd be surprised...

Re: Writing a SQL database from scratch in Go

#16

Post by Townley »

userbinator wrote: post #13
ycnewsreader wrote: post #8

Earlier quoted context omitted.

Another reason is that by providing an identifier for this string literal, misspellings of it can be detected by the compiler whereas there is nothing tying separate string literals together.

I can hardly imagine a case where you would misspell "select" and not notice it at some point, nor use it in more than one place (the keyword detector) in the parser.

The pattern (which I employ only sometimes) is to have almost all literals defined in this way. Perhaps SELECT isn’t likely to be the word you misspell, but I’m a careless typist and make 10 typos a minute. Taking this added step helps your editor save you from yourself.

Re: Writing a SQL database from scratch in Go

#17

Post by bogomipz »

eatonphil wrote: post #2

Unexpected, but welcome, to see the first part in the series on here right now. I just published the second part [0] today, featuring binary expressions and WHERE filtering. It also updates the REPL to use a prettier table-printing library and a readline implementation. The repo [1] has some additional bare notes on architecture and links to similar, more mature projects (primarily go-mysql-server and ramsql). [0] ht…

This is great. Do you plan on continuing this blog series? If so I hope you post it here. Cheers.

Re: Writing a SQL database from scratch in Go

#18

Post by viraptor »

Townley wrote: post #16
userbinator wrote: post #13

Earlier quoted context omitted.

I can hardly imagine a case where you would misspell "select" and not notice it at some point, nor use it in more than one place (the keyword detector) in the parser.

The pattern (which I employ only sometimes) is to have almost all literals defined in this way. Perhaps SELECT isn’t likely to be the word you misspell, but I’m a careless typist and make 10 typos a minute. Taking this added step helps your editor save you from yourself.

Having standards like that and keeping them helps a lot. Next time you have a different keyword, you don't have to think "does it deserve a constant?" - all of them do.

Similar to how linters stop you from overthinking indentation in specific cases, or some naming standards, and later rename/reformat-wars.

Re: Writing a SQL database from scratch in Go

#20

Post by cube2222 »

Great blog post!

I'd just like to add for the curious, that usually you'd use goyacc for parsing SQL.

And most serious SQL projects in Go have started with the SQL parser from vitess and adapted it to their use case (which is just funny trivia, but for anything big, I recommend it, did the same for OctoSQL [0]).

[0]: https://github.com/cube2222/octosql

Post reply on HN