When are they going through with a rename? This sub-optimal name is holding them back.
I'm sure even you'll admit that they have a rather cute looking cockroach as a logo.
How We Built a Vectorized SQL Engine
31–40 of 56 posts
Re: How We Built a Vectorized SQL Engine
#32When are they going through with a rename? This sub-optimal name is holding them back.
Bad publicity is better than no publicity.
Re: How We Built a Vectorized SQL Engine
#33Earlier quoted context omitted.
A database is a server. Your malloc isn't deterministic, either. According to the Go Wikipedia article, as of 2017: > Garbage collection pauses should be significantly shorter than they were in Go 1.7, usually under 100 microseconds and often as low as 10 microseconds. According to [1], "A trivial SELECT can take in the order of 0.1ms to execute server-side", i.e., 100 microseconds. Any I/O will of course significant…
For me, the issue with GC never has been about how long it runs for. It's been about the unpredictability of when it can kick in. Granted a malloc can be unpredictable too (albeit very very rarely), but at least you know a malloc is the only time your process could stall waiting for memory. Databases make extensive use of scratch space for undo logs and redo logs and especially while processing JOINs. It's tremendous…
I'll repeat what was mentioned elsewhere. The GC pauses in Go are in the order of microseconds. We've done a ton of work keeping our memory usage under tight bounds and keeping a close eye on exactly how the go GC processes interplay with query execution threads (what processors are consumed, GC pressure buildup, where/when the stalls are introduced, etc.) Consistently we've found that given you're invariably dealing with network latencies (it's a distributed db after all), the microsecond GC pauses are almost entirely dwarfed to matter (I'm not excluding p99s here).
Re: How We Built a Vectorized SQL Engine
#34Earlier quoted context omitted.
If you pick a happy name, you get complaints that they sound too generic, sound like a medication name, etc. People also complain if your name ends in .com or .net, as apparently that sounds like something from the mid 90s. People just like complaining about names.
On the other hand no one brings this point up about Oracle, PostgreSQL, SQL Server, MySQL, MariaDB etc. People like complaining about names that they wish things didn't have.
I haven't used most of those, but SQL Server is a prime source of griping about names, as Microsoft keeps popping out strangely named variations. See: https://en.wikipedia.org/wiki/Microsoft_SQL_Server#Editions
Re: How We Built a Vectorized SQL Engine
#35Earlier quoted context omitted.
I'm shocked that they chose a garbage-collected language to implement a database with. Golang is great for building servers, but this is domain with potentially much greater resource constraints. How do they deal with GC pause? I'm sure that they have an answer, but they'd have so much more latitude if they had the facility to reason about this from the ground up. They've sort of painted themselves into a corner now.…
We wrote a (brief) blog post about this decision way back in 2015 that you might be interested in: https://www.cockroachlabs.com/blog/why-go-was-the-right-choi... We're generally pretty happy with our choice to use Go for the database, even when it causes us pain like the kind you see in this post. Rust wasn't really ready when we started working on CockroachDB, and we think that we wouldn't have been able to make pr…
How hard would it be to port the code to Rust? (Not trying to make it sound like a "Rewrite in Rust" thing)
Re: How We Built a Vectorized SQL Engine
#36Earlier quoted context omitted.
I'm shocked that they chose a garbage-collected language to implement a database with. Golang is great for building servers, but this is domain with potentially much greater resource constraints. How do they deal with GC pause? I'm sure that they have an answer, but they'd have so much more latitude if they had the facility to reason about this from the ground up. They've sort of painted themselves into a corner now.…
A database is a server. Your malloc isn't deterministic, either. According to the Go Wikipedia article, as of 2017: > Garbage collection pauses should be significantly shorter than they were in Go 1.7, usually under 100 microseconds and often as low as 10 microseconds. According to [1], "A trivial SELECT can take in the order of 0.1ms to execute server-side", i.e., 100 microseconds. Any I/O will of course significant…
C may not be an attractive choice in 2019, but C++ and Rust certainly are.
Especially if you're working at the level where you want/need vectorization.
Re: How We Built a Vectorized SQL Engine
#37Earlier quoted context omitted.
does Go's compiler emit SIMD instructions? it'd be cool to see the disassembly for the final, column order version since that loop would vectorize well. the same question goes for the speed-of-light benchmark -- there may be room to push even further there if it's doing a single multiply at a time.
It does not. You can use something like Avo to lighten the load a bit of writing those assembly routines. It takes care of register allocation, struct fields, and the necessary function stubs.
I was recently profiling some code in utf8 package and it caught my attention that some easily SIMDable code was lighting up in the flame chart. Kind of lame it can’t do that
Re: How We Built a Vectorized SQL Engine
#38Earlier quoted context omitted.
A database is a server. Your malloc isn't deterministic, either. According to the Go Wikipedia article, as of 2017: > Garbage collection pauses should be significantly shorter than they were in Go 1.7, usually under 100 microseconds and often as low as 10 microseconds. According to [1], "A trivial SELECT can take in the order of 0.1ms to execute server-side", i.e., 100 microseconds. Any I/O will of course significant…
Throughput optimization in database engines often relies on operation latency being visible and predictable to the execution scheduler. Disk I/O has this property, a GC does not, so they are not comparable in terms of their impact on throughput. An important class of schedule-based architectural optimizations are rendered ineffective if you are running a GC in the background. This only matters to databases that actua…
So, before we theorize that Cockroach can't possibly be fast enough to meet anyone's needs, perhaps we should look at some performance figures?
I haven't ever touched it myself, but, if the benchmarks that Cockroach Labs posts on their blog are to be believed, it's pretty respectable for what it is.
Re: How We Built a Vectorized SQL Engine
#39Earlier quoted context omitted.
On the other hand no one brings this point up about Oracle, PostgreSQL, SQL Server, MySQL, MariaDB etc. People like complaining about names that they wish things didn't have.
I haven't used most of those, but SQL Server is a prime source of griping about names, as Microsoft keeps popping out strangely named variations. See: https://en.wikipedia.org/wiki/Microsoft_SQL_Server#Editions
My problem with SQL Server is that the name is so generic it sounds more like a product category than a product. Its like naming a word processor "Text Rendering Engine". Microsoft seems to be generally terrible at naming things: I don't know how they expected people to keep .NET Core, Standard, Framework, and Mono separate, not even counting how bad a name .NET is in the first place.
Re: How We Built a Vectorized SQL Engine
#40Earlier quoted context omitted.
We wrote a (brief) blog post about this decision way back in 2015 that you might be interested in: https://www.cockroachlabs.com/blog/why-go-was-the-right-choi... We're generally pretty happy with our choice to use Go for the database, even when it causes us pain like the kind you see in this post. Rust wasn't really ready when we started working on CockroachDB, and we think that we wouldn't have been able to make pr…
How hard would it be to port the code to Rust? (Not trying to make it sound like a "Rewrite in Rust" thing)
Super mega ultra hard. It would take so much time for us to learn rust, port everything including all tooling, and fix new bugs we introduce that we wouldn't add any new features (but lots of new bugs!) for like 2-4 years and the company would die.