How long did this take?
I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
11–20 of 79 posts
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#12[flagged]
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#13The thing I would have liked to know is why they don't use an existing fast SQL parser. Was being slightly incompatible with all existing SQL dialects a product requirement?
Our SQL is very similar to ClickHouse SQL, in that we used ClickHouse SQL as a starting point as that's what our underlying DB is. We needed to have our own parser so that we could add additional language features on top.
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#14Well despite my current anti AI sentiment, I have to admit that after reading the article, It was a good use of AI, done by someone with good technical skills. Still I have the feeling that this only works because of the vast accumulated knowledge pre-AI, and if everybody keeps going in this path, it will end up making everyone not advancing their knowledge at the pace they did before. I feel that this AI immersion i…
I think AI is powertool. Period. If you give it to people who are skill, it will create a mess.
I think democratization of intelligence is going to be interesting. You could say the same with same about internet. I think it is part of evolution. May be intelligence or expertise is what does not make us special. May be it is that we are ingenious amd creative with tools and thats how we evolve.
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#15How long did this take?
It took about 2 days to get a proof of concept, and about a week to get something I could ship to production.
I skipped a few features for the PoC (like XML tag support, token positions), so most of the delta was adding those back in!
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#16Well despite my current anti AI sentiment, I have to admit that after reading the article, It was a good use of AI, done by someone with good technical skills. Still I have the feeling that this only works because of the vast accumulated knowledge pre-AI, and if everybody keeps going in this path, it will end up making everyone not advancing their knowledge at the pace they did before. I feel that this AI immersion i…
I think AI is powertool. Period. If you give it to people who are skill, it will create a mess. I think democratization of intelligence is going to be interesting. You could say the same with same about internet. I think it is part of evolution. May be intelligence or expertise is what does not make us special. May be it is that we are ingenious amd creative with tools and thats how we evolve.
> May be it is that we are ingenious amd creative with tools and thats how we evolve.
And every time you use the AI to be ingenious or creative, that will be added to the training data. Then someday the AI can be ingenious and creative without you! (It might take a few more breakthroughs. But investors will literally spend trillions chasing those breakthroughs.)
The endgame here is to replace all human intelligence and labor with machines that are smarter and work cheaper. But who controls the machines?
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#17I cannot believe they're sticking to their guns on this website design. It's awful.
It's awesome!
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#18The key parts of this is how not vibecoded it is. Feels like a model of how you should do software with AI. Now that we can easily set up property testing, fuzzing, etc. there's almost no reason not to.
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#19The key parts of this is how not vibecoded it is. Feels like a model of how you should do software with AI. Now that we can easily set up property testing, fuzzing, etc. there's almost no reason not to.
that is vibecoding these days
Re: I rewrote PostHog's SQL parser, 70x faster, while barely looking at the code
#20Well despite my current anti AI sentiment, I have to admit that after reading the article, It was a good use of AI, done by someone with good technical skills. Still I have the feeling that this only works because of the vast accumulated knowledge pre-AI, and if everybody keeps going in this path, it will end up making everyone not advancing their knowledge at the pace they did before. I feel that this AI immersion i…
I think AI is powertool. Period. If you give it to people who are skill, it will create a mess. I think democratization of intelligence is going to be interesting. You could say the same with same about internet. I think it is part of evolution. May be intelligence or expertise is what does not make us special. May be it is that we are ingenious amd creative with tools and thats how we evolve.
There are some studies that suggest human brain sizes have been shrinking over the last 20,000 years. The theory is that as civilization developed the demand for individual humans to be independently intelligent has weakened because we developed a "collective brain" and also self-domesticated to be more cooperative.