Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Datasette is my data hammer

jeremiak.com

Post reply on HN
Print view View original

21–30 of 71 posts

Re: Datasette is my data hammer

#21

Post by ocdtrekkie »

sally_glance wrote: post #19
ocdtrekkie wrote: post #10

Earlier quoted context omitted.

Question: Did you add plugins? Datasette is basically entirely plugins, such that you can create radically different "sites" and tools out of different combinations of plugins. Datasette without any plugins is useful, but where it's functionality shines is entirely in plugins.

I did not, just had a quick glance at what was available. What is your use case and which plugins would you recommend? Providing some kind of bare bones report/dashboard with the visualization plugin?

In my case, I log the GPS data from my car, but wanted a private way to explore that (obviously sensitive) data.

https://datasette.io/plugins/datasette-cluster-map is a really neat example of a "just works" plugin: If you have latitude and longitude columns, it will plot them!

But there are numerous plugins for certain types of visualizations, there's a bunch for handling editing, inserts, uploads, etc.

Some of the examples are pretty neat examples of what you can do with a bit of tuning: https://global-power-plants.datasettes.com/global-power-plan... is pretty neat.

Re: Datasette is my data hammer

#22

Post by Fuzzwah »

craftsquick wrote: post #17
brunooliv wrote: post #5

Can Datasette be used with a Postgres RDS database with hundred GBs? Feels like it's mostly designed for smaller things? Also if it can, how would I go about it and if it can't, any similar tools I can use? This looks really cool!!

This is a self-plug but shapedata.io does similar things on top of Postgres -- it is much younger than Datasette, so has far fewer features, but should give you a similar exploration experience!

I'm keen to check this out, I tried to sign up but got a 403 during the google auth flow.

Along with the message: The developer hasn’t given you access to this app. It’s currently being tested and it hasn’t been verified by Google. If you think you should have access, contact the developer

Re: Datasette is my data hammer

#23

Post by craftsquick »

Fuzzwah wrote: post #22
craftsquick wrote: post #17

Earlier quoted context omitted.

This is a self-plug but shapedata.io does similar things on top of Postgres -- it is much younger than Datasette, so has far fewer features, but should give you a similar exploration experience!

I'm keen to check this out, I tried to sign up but got a 403 during the google auth flow. Along with the message: The developer hasn’t given you access to this app. It’s currently being tested and it hasn’t been verified by Google. If you think you should have access, contact the developer

Really sorry about that -- I should've turned off test mode now for Google OAuth. Login work ok now?

Re: Datasette is my data hammer

#24

Post by masklinn »

ocdtrekkie wrote: post #21
sally_glance wrote: post #19

Earlier quoted context omitted.

I did not, just had a quick glance at what was available. What is your use case and which plugins would you recommend? Providing some kind of bare bones report/dashboard with the visualization plugin?

In my case, I log the GPS data from my car, but wanted a private way to explore that (obviously sensitive) data. https://datasette.io/plugins/datasette-cluster-map is a really neat example of a "just works" plugin: If you have latitude and longitude columns, it will plot them! But there are numerous plugins for certain types of visualizations, there's a bunch for handling editing, inserts, uploads, etc. Some of the e…

Will datasette automatically provide useful graph views of time series? Or is there a good plug-in of that?

One thing I’m trying to get better at is resource tracking (mostly energy) but it’s pretty clunky using a spreadsheet (likely because I’m not super knowledgeable and really don’t care).

Re: Datasette is my data hammer

#25

Post by Fuzzwah »

craftsquick wrote: post #23
Fuzzwah wrote: post #22

Earlier quoted context omitted.

I'm keen to check this out, I tried to sign up but got a 403 during the google auth flow. Along with the message: The developer hasn’t given you access to this app. It’s currently being tested and it hasn’t been verified by Google. If you think you should have access, contact the developer

Really sorry about that -- I should've turned off test mode now for Google OAuth. Login work ok now?

Yep, in now. Will play with it tomorrow. Thanks, looks like a great project.

Re: Datasette is my data hammer

#27

Post by sgraaf »

Datasette is such a neat project & tool. Simon, its developer, has built an entire ecosystem of tools around sqlite [1], Datasette [2], and automation through GH Actions, like git-scraping [3]. Also, he has been actively blogging for 20+ years [4]. I particularly enjoy his short TIL posts [5].

[1] https://github.com/simonw?tab=repositories&q=sqlite&sort=sta...

[2] https://github.com/simonw?tab=repositories&q=datasette&sort=...

[3] https://simonwillison.net/2020/Oct/9/git-scraping/

[4] https://simonwillison.net/

[5] https://til.simonwillison.net/

Re: Datasette is my data hammer

#28

Post by klondike_klive »

jdblair wrote: post #26

I'm showing my age, but the first thing I thought of when reading the headline was the Commodore Datasette. https://en.m.wikipedia.org/wiki/Commodore_Datasette

I immediately thought of the musician Datassette - proponent of the micro-genre Business Funk.

https://www.youtube.com/watch?v=zgWa7NoJGWc

Re: Datasette is my data hammer

#29

Post by pletnes »

darkteflon wrote: post #2

Interesting project. I would've liked to have heard a brief comparison with Excel - what makes Datasette something that one should reach for over that? Is it just the ability to use SQL in-place?

It’s got plugins for some stuff Excel don’t. I’ve used one for map/gis data to look at lat/long data on a map. Never seen that in Excel. The dataset was too big for Excel, too.

Re: Datasette is my data hammer

#30

Post by matteason »

sgraaf wrote: post #27

Datasette is such a neat project & tool. Simon, its developer, has built an entire ecosystem of tools around sqlite [1], Datasette [2], and automation through GH Actions, like git-scraping [3]. Also, he has been actively blogging for 20+ years [4]. I particularly enjoy his short TIL posts [5]. [1] https://github.com/simonw?tab=repositories&q=sqlite&sort=sta... [2] https://github.com/simonw?tab=repositories&q=datasett…

Simon also did some brilliant stuff with Datasette based on a 2-hour long mp3 of train announcements that ScotRail released as part of a Freedom of Information reponse. A journalist called Jon Brady found it initially on the ScotRail website and tweeted about it, I chopped it up into individual announcements and crowdsourced the transcription, and Simon built a dashboard to explore them all in Datasette.

Simon's blog: https://simonwillison.net/2022/Aug/21/scotrail/

The Datasette dashboard: https://scotrail.datasette.io/

Post reply on HN