Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Datasette is my data hammer

jeremiak.com

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Datasette is my data hammer

#61

Post by simonw »

morgango wrote: post #57
darkteflon wrote: post #2

Interesting project. I would've liked to have heard a brief comparison with Excel - what makes Datasette something that one should reach for over that? Is it just the ability to use SQL in-place?

I think the best comparison would be with Access, seems to be the all-in-one data tool that tried to be.

Yeah, I'm constantly surprised at Microsoft's lack of investment in Access. The world clearly needs a really great, easy to use personal database.

Datasette isn't there yet but it's a useful ambition for the future.

Re: Datasette is my data hammer

#62

Post by klondike_klive »

misterS wrote: post #48
klondike_klive wrote: post #28

Earlier quoted context omitted.

I immediately thought of the musician Datassette - proponent of the micro-genre Business Funk. https://www.youtube.com/watch?v=zgWa7NoJGWc

I did, too :) For those that don't know: Datasette is the author/curator of https://musicforprogramming.net

...and he contributed music and SFX to Space Rubbish, an awesome bullet-hell Asteroids clone. https://vimeo.com/18472447

Re: Datasette is my data hammer

#63

Post by darkteflon »

simonw wrote: post #39
darkteflon wrote: post #2

Interesting project. I would've liked to have heard a brief comparison with Excel - what makes Datasette something that one should reach for over that? Is it just the ability to use SQL in-place?

When I've talked to startups in the past I've always warned them to be careful about accidentally building a product which competes with a potential customer's existing good Excel spreadsheet... so it's ironic that I've ended up myself focusing on something that falls into that bucket! Right now I'd say the big things are: 1. Size of data. Datasette is a great tool for when your data is too big for Excel but not so b…

Hi Simon, thanks so much for taking the time. Been following your excellent work with Django for many years.

We took a look at DS several months ago (I believe it popped up here on HN) but I confess that until I read the linked article I hadn’t fully understood that it was a “hammer”, and a good one at that.

At that time we set it aside in favour of AirTable which for our purposes - to relate this to one of your other comments here - is sort of Access with a web interface plus authentication and (frankly not very granular) authorisation.

The main reason was because it seemed to us that Datasette didn’t have first class support for auth - that it was sort of positioned as a community or internal tool, rather than something we could build out a client interface on. I just had a look at the plugins and noted that there is something for auth, although it too looks rather to be for internal tooling. Is that a fair characterisation, or do we have it wrong? Because we love everything else about DS - open source, built on SQL and SQLite, etc. - and would vastly prefer to use it over AirTable.

Re: Datasette is my data hammer

#64

Post by simonw »

simonw wrote: post #46
sally_glance wrote: post #45

Earlier quoted context omitted.

That publish command looks sweet! For confidential data I'd have to setup some integration with our infra to get the required access control though. Maybe rather go with the package option using docker and a pipeline that deploys to our k8s where I could handle auth using an existing proxy... My current workflow doesn't really involve creating databases, I mostly just create foreign tables, normal tables and views in…

For confidential data you could use one of the plugins that add authentication - datasette-auth0 or datasette-auth-github or datasette-auth-passwords are options there - but yeah, you're probably better off deploying to internet hosting if you already have an auth proxy setup somewhere.

Meant to say internal hosting, not internet hosting.

Re: Datasette is my data hammer

#65

Post by simonw »

darkteflon wrote: post #63
simonw wrote: post #39

Earlier quoted context omitted.

When I've talked to startups in the past I've always warned them to be careful about accidentally building a product which competes with a potential customer's existing good Excel spreadsheet... so it's ironic that I've ended up myself focusing on something that falls into that bucket! Right now I'd say the big things are: 1. Size of data. Datasette is a great tool for when your data is too big for Excel but not so b…

Hi Simon, thanks so much for taking the time. Been following your excellent work with Django for many years. We took a look at DS several months ago (I believe it popped up here on HN) but I confess that until I read the linked article I hadn’t fully understood that it was a “hammer”, and a good one at that. At that time we set it aside in favour of AirTable which for our purposes - to relate this to one of your othe…

Airtable is a fantastic piece of software. I'd love Datasette to work as an alternative for it some day but there is a whole lot of stuff to build to get there.

The most relevant plugin for what you're describing at the moment is probably the Auth0 one: https://datasette.io/plugins/datasette-auth0

Quite a lot of assembly is required at the moment if you want to do interesting custom things with authentication and permissions - the documentation here is the best starting point: https://docs.datasette.io/en/stable/authentication.html

Re: Datasette is my data hammer

#66

Post by darkteflon »

simonw wrote: post #65
darkteflon wrote: post #63

Earlier quoted context omitted.

Hi Simon, thanks so much for taking the time. Been following your excellent work with Django for many years. We took a look at DS several months ago (I believe it popped up here on HN) but I confess that until I read the linked article I hadn’t fully understood that it was a “hammer”, and a good one at that. At that time we set it aside in favour of AirTable which for our purposes - to relate this to one of your othe…

Airtable is a fantastic piece of software. I'd love Datasette to work as an alternative for it some day but there is a whole lot of stuff to build to get there. The most relevant plugin for what you're describing at the moment is probably the Auth0 one: https://datasette.io/plugins/datasette-auth0 Quite a lot of assembly is required at the moment if you want to do interesting custom things with authentication and per…

Fantastic - thanks Simon, much appreciated.

Yes we generally like the AirTable UX, too, and it’s a good fit for client-facing stuff in addition to internal tooling. It is slow, though, and there are no native desktop or mobile clients. Mainly, though: building a business on it comes with all the usual pitfalls of VC-backed SaaS.

It’s very helpful to hear your thoughts on the project’s trajectory. We’ll take another look at DS and the auth0 module, thanks. It looks very promising and we don’t mind a bit of assembly.

Re: Datasette is my data hammer

#67

Post by PurpleRamen »

everybodyknows wrote: post #56
PurpleRamen wrote: post #51

Looking through some of the examples[1], it seems the WebUI is pretty underwhelming and demands much optimization and modernization. Are there alternative UIs available? [1] https://datasette.io/examples

What specifically would you want to be different? It seems to me to serve the purpose well: https://global-power-plants.datasettes.com/global-power-plan...

> What specifically would you want to be different?

It's sold for data exploration, so i except some useful data-app with an up to date interface, dedicated for interactive navigation and presentation of data.

> It seems to me to serve the purpose well:

Not really. This looks like something that could even run on Internet Explorer 2. It has significant flaws in terms of UI and UX, and lacking nearly everything I except here.

It's incredibly wasteful on space. I must scroll down two screens to even see some data. For some reason the map is high jacking the scroll down, and the map is so broken that I end with multiple earths displayed, with only one having actual data. Changes to the query are not in-place, the whole side is reloads every time. Adding a new filter only seems to work after a reload. And all information are really sparse and scattered randomly around. And finally, there seems to be no way to shape the actual output, which would be the main job of a data explorer.

Re: Datasette is my data hammer

#68

Post by PurpleRamen »

simonw wrote: post #60
PurpleRamen wrote: post #51

Looking through some of the examples[1], it seems the WebUI is pretty underwhelming and demands much optimization and modernization. Are there alternative UIs available? [1] https://datasette.io/examples

I'm definitely keen on suggestions for improvements I can make to the default UI. Datasette provides both a JSON API (easily enabled for CORS access) and supports custom templates, so it's possible to customize the UI any way you like. So far I've not seen many examples of extensive customization. I use the custom templates a lot myself - these four sites are all just regular Datasette with some custom templates: - h…

> I'm definitely keen on suggestions for improvements I can make to the default UI.

Have you looked at other modern data-apps? AirTable? Notion? Jupyter Notebooks at least?

> Datasette provides both a JSON API (easily enabled for CORS access) and supports custom templates, so it's possible to customize the UI any way you like.

Seems promising. But how much does the JSON API support interactive work with data? The UI I've seen just reloaded the whole dataset it seems? There seems to be no way to specify queries and views from the UI. Would the API have means to manage such side-data?

Re: Datasette is my data hammer

#69

Post by darkteflon »

darkteflon wrote: post #66
simonw wrote: post #65

Earlier quoted context omitted.

Airtable is a fantastic piece of software. I'd love Datasette to work as an alternative for it some day but there is a whole lot of stuff to build to get there. The most relevant plugin for what you're describing at the moment is probably the Auth0 one: https://datasette.io/plugins/datasette-auth0 Quite a lot of assembly is required at the moment if you want to do interesting custom things with authentication and per…

Fantastic - thanks Simon, much appreciated. Yes we generally like the AirTable UX, too, and it’s a good fit for client-facing stuff in addition to internal tooling. It is slow, though, and there are no native desktop or mobile clients. Mainly, though: building a business on it comes with all the usual pitfalls of VC-backed SaaS. It’s very helpful to hear your thoughts on the project’s trajectory. We’ll take another l…

Replying to myself to say that - with hilarious timing - one of our business-critical AirTable bases is suddenly throwing an error and has become completely inaccessible right when we need it. AirTable support does its best to try to funnel you into the "helpful community".

Cancelling our subscription today and migrating to Datasette, hell or high water. Glad this happened now and not 2 years down the track.

Re: Datasette is my data hammer

#70

Post by djdeutschebahn »

sally_glance wrote: post #8

I tried datasette for the first time not long ago. To me it wasn't obvious what it gave me compared to my local Postgres setup. I have a single docker-compose file with Postgres + pgAdmin and a host mount which I use for data exploration and ad-hoc analysis. I get a nice Web UI and can ingest quickly using the awesome Postgres FDWs. Compared to datasette, it allows me to also transform the data by creating derived vi…

Do you have a solution (with your go-to-tools) to put data with coordinates directly into an interactive map, for example leaflet? That is something I really like (and use extensivly) about datasette respectivly the plugins.
Post reply on HN