Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Harlequin: DuckDB IDE for the terminal

harlequin.sh

Post reply on HN
Print view View original

21–30 of 61 posts

Re: Harlequin: DuckDB IDE for the terminal

#21

Post by zlurker »

quadrature wrote: post #12
zlurker wrote: post #10

Earlier quoted context omitted.

We orchestrate our ETL pipelines with dagster. We only use duckdb in a few of them but are slowly replacing pandas etls with it. For some of our bigger jobs we use spark instead. Essentially it's: 1. Data sources from places such as s3, sftp, rds 2. Use duckdb to load most of these with only extensions (I dont believe there's one for sftp, so we just have some python code to pull the files out.) 3. transform the data…

This is very cool!. Last time I checked duckdb didn't have the concept of a metastore so do you have an internal convention for table locations and folder structure ?. What do you use for reports/visualizations? notebooks ?.

Yeah, dagster has a concept of metadata and assets so we have some code that'll map dagster's own logical representation to physical s3 locations.

Reports and viz varies a lot, the finance department uses tableau where as for more 'data sciencey' stuff we normally just use notebooks.

Re: Harlequin: DuckDB IDE for the terminal

#22

Post by tconbeer »

ayhanfuat wrote: post #19
tconbeer wrote: post #13

Hi everyone! I made this. Tried posting it to Show yesterday, glad this thread is getting more momentum! For the past four months I've been working (part-time, this is OSS after all) on Harlequin, a SQL IDE for DuckDB that runs in your terminal. I built this because I work in Data, and I found myself often reaching for the DuckDB CLI to quickly query CSV or Parquet data, but then hitting a wall when using the DuckDB…

Lovely tool. I'll certainly try it out. The code fragments in the documentation seem to be not selectable, though (or maybe it is not highlighting correctly).

it's an issue with the highlight color. I'll get that fixed shortly.

Re: Harlequin: DuckDB IDE for the terminal

#23

Post by tconbeer »

zokier wrote: post #20

I'm not in love with this style of UI design in terminals: https://harlequin.sh/_app/immutable/assets/export.a0e81d27.p... Every item in the form takes 4 lines (I think?) whereas in more traditional curses UI they would be packed to one line per item, the scrollbar could have easily been avoided here. Smaller nitpick but that style of toggle switches is also form over function; I'd find traditional [X] far more clear…

fair feedback!

Re: Harlequin: DuckDB IDE for the terminal

#25

Post by whobre »

DesiLurker wrote: post #24

I need this but for C++, I kinda miss the old borland turbo C++ ide. mostly what I care about is terminal access and code navigation but other than some vim + plugins monstrosity I cant find any.

There’s Motor, but I don’t know if the project is still being maintained.

https://github.com/rofl0r/motor

Re: Harlequin: DuckDB IDE for the terminal

#26

Post by inshadows »

tconbeer wrote: post #13

Hi everyone! I made this. Tried posting it to Show yesterday, glad this thread is getting more momentum! For the past four months I've been working (part-time, this is OSS after all) on Harlequin, a SQL IDE for DuckDB that runs in your terminal. I built this because I work in Data, and I found myself often reaching for the DuckDB CLI to quickly query CSV or Parquet data, but then hitting a wall when using the DuckDB…

[dead]

Re: Harlequin: DuckDB IDE for the terminal

#27

Post by chrisjc »

tschellenbach wrote: post #3

what are the benefits of DuckDB?

Databases can usually be split into one of two types; OLTP (row-based) or OLAP (columnar). OLTPs are used mostly for transactional workloads whereas OLAP is mostly used for analytics.

Here goes...

Take 1: sqlite is to Postgres, what duckdb is to Snowflake/BigQuery.

Take 2: In a similar way that sqlite is an in-process/memory OLTP, duckdb is an in-process/memory OLAP.

I should mention some caveats/exceptions/notes about my statements above:

- there are OLAP projects out there that use Postgres as their basis.

- HTAPs are DBs that allow you to define tables as either row-based or columnar.

- duckdb works with sqlite tables, and its SQL is heavily based on postgres SQL

- duckdb 0.9.0 is being released next week :)

- It seems duckdb is poised to become an almost ubiquitous component of the analytical stack. Apache Iceberg, dbt, ...

Re: Harlequin: DuckDB IDE for the terminal

#28

Post by rochak »

zokier wrote: post #20

I'm not in love with this style of UI design in terminals: https://harlequin.sh/_app/immutable/assets/export.a0e81d27.p... Every item in the form takes 4 lines (I think?) whereas in more traditional curses UI they would be packed to one line per item, the scrollbar could have easily been avoided here. Smaller nitpick but that style of toggle switches is also form over function; I'd find traditional [X] far more clear…

+1. Lazygit is one of the few tools that does the TUI just right. Super functional UI + Easy discoverability = Chef's kiss.

Re: Harlequin: DuckDB IDE for the terminal

#30

Post by quadrature »

zlurker wrote: post #21
quadrature wrote: post #12

Earlier quoted context omitted.

This is very cool!. Last time I checked duckdb didn't have the concept of a metastore so do you have an internal convention for table locations and folder structure ?. What do you use for reports/visualizations? notebooks ?.

Yeah, dagster has a concept of metadata and assets so we have some code that'll map dagster's own logical representation to physical s3 locations. Reports and viz varies a lot, the finance department uses tableau where as for more 'data sciencey' stuff we normally just use notebooks.

Hadn't heard of that dagster feature, thats really cool.
Post reply on HN