Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DuckDB 0.8

duckdb.org

Post reply on HN
Print view View original

21–30 of 101 posts

Re: DuckDB 0.8

#21

Post by mkhattab »

Minor annoyance:

Yesterday I was reading the docs and one of the first things I tried is the following, copy and pasted, which didn't work, using version 0.71:

  COPY (SELECT 42 AS a, 'hello' AS b) TO 'query.json' (FORMAT JSON, ARRAY TRUE);

  >>> Error: Binder Error: Unrecognized option CSV writer "array"
Just tried the above again with this release and it's fixed now.

Ideally, all commands listed in the documentation should be tested or verified to work.

Aside from that, nice work on the release! And I'm learning more and find it pretty cool, especially WASM support.

Re: DuckDB 0.8

#22

Post by 1egg0myegg0 »

mkhattab wrote: post #21

Minor annoyance: Yesterday I was reading the docs and one of the first things I tried is the following, copy and pasted, which didn't work, using version 0.71: COPY (SELECT 42 AS a, 'hello' AS b) TO 'query.json' (FORMAT JSON, ARRAY TRUE); >>> Error: Binder Error: Unrecognized option CSV writer "array" Just tried the above again with this release and it's fixed now. Ideally, all commands listed in the documentation sh…

Howdy! The docs reflect the latest dev branch. For a specific release, you can pick it from the version dropdown in the top right!

Re: DuckDB 0.8

#23

Post by fbdab103 »

1egg0myegg0 wrote: post #19
fbdab103 wrote: post #16

Earlier quoted context omitted.

>Now DuckDB can actually read and write from/to SQLite This feels like it opens up some possibilities, but I am struggling to figure out where. Are you aware of any interesting use cases this has enabled? Does all of the Duck syntax then work against the SQLite database? For example, could I now run a pivot without a bunch of hoops?

You could definitely run a pivot with the SQLite attach! All of the DuckDB syntax works!

Well that is just amazingly interesting. The documentation is failing to do this feature justice! Definitely going to be on my todo list to explore this bridge in the near future.

Re: DuckDB 0.8

#24

Post by mkhattab »

1egg0myegg0 wrote: post #22
mkhattab wrote: post #21

Minor annoyance: Yesterday I was reading the docs and one of the first things I tried is the following, copy and pasted, which didn't work, using version 0.71: COPY (SELECT 42 AS a, 'hello' AS b) TO 'query.json' (FORMAT JSON, ARRAY TRUE); >>> Error: Binder Error: Unrecognized option CSV writer "array" Just tried the above again with this release and it's fixed now. Ideally, all commands listed in the documentation sh…

Howdy! The docs reflect the latest dev branch. For a specific release, you can pick it from the version dropdown in the top right!

Sorry! Just realized that when I checked the docs again. My bad.

Re: DuckDB 0.8

#25

Post by 1egg0myegg0 »

mkhattab wrote: post #24
1egg0myegg0 wrote: post #22

Earlier quoted context omitted.

Howdy! The docs reflect the latest dev branch. For a specific release, you can pick it from the version dropdown in the top right!

Sorry! Just realized that when I checked the docs again. My bad.

No worries! Thanks for reading the docs!

Re: DuckDB 0.8

#26

Post by [deleted] »

de6u99er wrote: post #13
TACIXAT wrote: post #6

I recently pulled DuckDB out of a project after hitting a memory corruption issue in regular usage. Upon investigating, they had an extremely long list of fuzzer-found issues. I just don't understand why someone would start something in a memory unsafe language these days. I cannot in good conscience put that on a customer's machine. We ended up rewriting a component to drop support for Parquet and to just use SQLite…

- DuckDB is written in C++ - SQLite is written in C I wouldn't consider any of those written in a memory safe language. Although SQLite has been battle hardened over many years, while DuckDB is a relatively new project. That being said, has been efforts of reimplementing SQLite in a more memory safe language like Rust. e.g. https://github.com/epilys/rsqlite3

[deleted]

Re: DuckDB 0.8

#27

Post by kristjansson »

de6u99er wrote: post #13
TACIXAT wrote: post #6

I recently pulled DuckDB out of a project after hitting a memory corruption issue in regular usage. Upon investigating, they had an extremely long list of fuzzer-found issues. I just don't understand why someone would start something in a memory unsafe language these days. I cannot in good conscience put that on a customer's machine. We ended up rewriting a component to drop support for Parquet and to just use SQLite…

- DuckDB is written in C++ - SQLite is written in C I wouldn't consider any of those written in a memory safe language. Although SQLite has been battle hardened over many years, while DuckDB is a relatively new project. That being said, has been efforts of reimplementing SQLite in a more memory safe language like Rust. e.g. https://github.com/epilys/rsqlite3

At the level of engineering of SQLite, the choice of language is almost immaterial. Suggesting a low effort transpilation is a competitive peer seems unserious and vaguely disrespectful.

Re: DuckDB 0.8

#28

Post by fiddlerwoaroof »

Icathian wrote: post #15
emmanueloga_ wrote: post #11

Setting up SQL, creating a well-designed schema, and optimizing queries can be a complex, multi-step process. Databases often feel like black boxes, requiring significant training and engineering to get right. One needs to tweak both queries, schema and database configuration to get best results. So why do we need SQL? I know NoSQL came and went (was it nosql or no-relations? not sure...) but honestly I can't think o…

I think you are drastically overestimating the ability of most devs to write performant code. Do you really think you're going to get better results asking devs who, by your measure cannot write good declarative SQL schemas and queries, to roll their own data storage and retrieval raw? RDBMS software is complicated in large part due to it needing to serve a wide range of use cases, so a custom-built version might end…

I generally agree, but I think there’s a mistake here comparing something like Postgres to hand-rolled code: Postgres has to implement generally safe optimizations and so cannot take advantage of invariants that cannot be expressed in a way it understands. For a specific task, with well-understood data, I’d expect a typical programmer should be able to outperform Postgres with a bit of effort.

The trade-off here is maintenance cost and a lack of flexibility due to specialization. So, IMO, one ought to reach for an SQL database first and then something else to optimize hot paths as determined by a profiler.

Re: DuckDB 0.8

#29

Post by bobbylarrybobby »

beoberha wrote: post #14
emmanueloga_ wrote: post #11

Setting up SQL, creating a well-designed schema, and optimizing queries can be a complex, multi-step process. Databases often feel like black boxes, requiring significant training and engineering to get right. One needs to tweak both queries, schema and database configuration to get best results. So why do we need SQL? I know NoSQL came and went (was it nosql or no-relations? not sure...) but honestly I can't think o…

The simple answer to this is that not only engineers write database queries. Personally, I’d be interested to see a system like the one you describe. Part of me thinks that SQL would still be simpler to reason about (though probably not to maintain).

Polars is an interesting library in this space. A rust DataFrame with bindings in other languages, most notably python

Re: DuckDB 0.8

#30

Post by porker »

1egg0myegg0 wrote: post #22
mkhattab wrote: post #21

Minor annoyance: Yesterday I was reading the docs and one of the first things I tried is the following, copy and pasted, which didn't work, using version 0.71: COPY (SELECT 42 AS a, 'hello' AS b) TO 'query.json' (FORMAT JSON, ARRAY TRUE); >>> Error: Binder Error: Unrecognized option CSV writer "array" Just tried the above again with this release and it's fixed now. Ideally, all commands listed in the documentation sh…

Howdy! The docs reflect the latest dev branch. For a specific release, you can pick it from the version dropdown in the top right!

I think it would be a better choice to default to showing the current stable release's docs rather than the dev branch?

That way users who download it see the docs they expect, and those who build from the dev branch or download preview builds have to go through an extra step.

Post reply on HN