Viewing profile — mccanne

Hey really cool! Tcpdump co-author here. If you're interested in the origin story of the tcpdump language, bpf, and pcap, I had fun putting together a talk a number of years ago...…

For sure... the use of JSON here seems orthogonal to Jamie's point of constructing complex nested values in SQL with a single backend query. Modern SQLs support all this as native …

Relevant discussion from a few years back https://news.ycombinator.com/item?id=28221654

And somewhat ironically here, SuperDB not only implements sum-type decoding of JSON in package unpack, but it also implements native sum types in a superset of JSON that we call Su…

Nice article! Decoding sum types into Go interface values is obviously tricky stuff, but it gets even harder when you have recursive data structures as in an abstract syntax tree (…

Really cool though typing ">" after "|" is a pain https://github.com/brimdata/super/blob/main/docs/language/pi...

Necessity is the mother of invention. MapReduce-based systems were developed because the state-of-the-art RDBMS systems of that age could not scale to the needs of the Googles/Yaho…

Yes, the name conflict is unfortunate. We named our project Zed in 2021 before the Zed text editor was a thing.

Apologies if our examples in the article weren't rich enough or clear. The beauty of the error type is that you can wrap any value in an error and make the error as rich as you'd l…

Yes, great point! The idea is that you can fix up a problem with data in place, while you're updating your ingest pipeline to handle whatever is causing the problem. You can do a t…

Hey, thanks for bringing up storage efficiency. This is an often cited objection and a big motivation of the Zed project actually. In Zed, data is organized around a self-describin…

A quick look at the forthcoming release of zui-1.0. Explore gnarly JSON data with ease!

Simon, brilliant observations here and kudos on sqlite-utils. I'm all for layers, a fundamental approach in our field to tame complexity. And the SQL model and SQLite have stood th…

Hear, hear!

Author here. All good points. Yes, you can build a super-structured type system on top of tables. EdgeDB does this well. And you can put JSON into relational columns. Then you migh…

Author here. Agreed! Validation is important. While I didn't make this point in the article, our thinking is schema validation does not require that the serialization format utiliz…

Author here. This totally makes sense. The challenge here is you need to store the type definitions somewhere (e.g., in the .proto files) and any system that processes protocol buf…

zq good/ -

Wow, thanks. Coincidentally, after hearing of a friend's woes dealing with massive amounts of CSV coming from a BPF-instrumental kernel, I played around a bit with integrating Zed …

That's a cool idea. We've had many collaborators using Zed lakes for search at smallish scale and we are still building the breadth of features needed for a serious search platform…

Hi, all. Author here. Thanks for all the great feedback. I've learned a lot from your comments and pointers. The Zed project is broader than "a jq alternative" and my bad for tryin…