This is super cool! I’ve been looking for ways to integrate Observable with my blog posts and make them more interactive and engaging. This might just be it.
Thank you and congrats on the release!
Observable 2.0, a static site generator for data apps
81–90 of 157 posts
Re: Observable 2.0, a static site generator for data apps
#82There's an almost bewildering amount of interesting ideas buried in this. Things like data loaders which are ANY script that can output data (as JSON or something else) to standard output. Markdown files with ```js blocks in that get executed. The reinvention of the core Observable notebook to avoid custom syntax. This is really big.
Yeah data loaders are like a UNIX pipe to a reactive notebook cell (?). There needn’t be any question of “do the data loaders support this or that”; it doesn’t even have a concept of “supporting” beyond supporting stdout… Still thinking through how to understand it myself!!
Data loaders seem like an interesting way to define a multi-language build system without having to write a makefile. Lots of build systems do this, but the boundaries between build steps often isn't as clean and uniform as having a single output per build step and relying on a file naming convention.
It's not truly reactive if you have to do a build to make anything happen. But maybe that doesn't matter, as long as it's reactive during development?
Re: Observable 2.0, a static site generator for data apps
#83A couple of questions:
1. Let's say I got a Sqlite/Duckdb database file on my server. It's got multiple tables and some of them 100M to 150M records. I want to create a plot/table that would have a slider/filter to only bring and show a slice of data at a time. Since it's statically generated data, how is this interactivity achieved? All the possible facets of data filtered by which ever way will be generated? Won't it be huge and how long will it take to generates this static data or is there an actual call back to the server to the duckdb file (I assume it works with .duckdb file too?)
2. If Observable Framework provides the front-end, does it mean I can use any auth library if I want to create a web site with a log in and subscription options?
3. If it's a static web page, does it mean that at any time a user views a chart, they will also be able to go to the Dev Tools and download the file with data that's behind the viz?
4. When (if you can share of course) is the planned release of Plot's interactions: zoom, pan, interactive legend, brush?
5. Deployment - with big parquet, sqlite, csv files, it's impossible to do CI/CO through github or vercel and such. Will your hosting services offer an option to host those files and runtimes to generate them?
Thanks
Re: Observable 2.0, a static site generator for data apps
#84Earlier quoted context omitted.
Hi! Some background first: I'm putting together a blog right now using Hugo and D3. I'm a huge fan of D3's infinite flexibility, as seen in some famous scrollytellers [0-1], and I've spent some time experimenting with that format myself [2]. My question is: what does Observable Framework offer for data storytellers who want to blog? Is this meant to go up against Hugo/Jekyll in terms of full-fledged max-efficiency si…
We’re not expressly targeting the blogging use case — we primarily want to support data apps, dashboards, and reports. But Observable Framework is quite flexible and you can use it for a lot of things; the Framework documentation is itself written using Framework, for example. So I would say that if you are working with data and you want an automated process to keep your data up-to-date, or to work with multiple lang…
How do dashboards work if data is computed at build time? Does that mean every time you want to update the data you need another build? I'm interested in live dashboards, is Obversable framework the wrong tool for the job?
Re: Observable 2.0, a static site generator for data apps
#85This is very cool to see! We've been building something very similar (in some regards, very different in others) in https://rysana.com/bundown
Somehow ideas like this emerge in waves seemingly with no coordination, kind of like what happened when Calculus was first invented
(Of course there's a lot of prior art here but it's interesting to see a specific jump towards polyglot single-file Markdown 'apps' and so on happen around the same time)
Re: Observable 2.0, a static site generator for data apps
#86It is so beautiful
Re: Observable 2.0, a static site generator for data apps
#87Earlier quoted context omitted.
We’re not expressly targeting the blogging use case — we primarily want to support data apps, dashboards, and reports. But Observable Framework is quite flexible and you can use it for a lot of things; the Framework documentation is itself written using Framework, for example. So I would say that if you are working with data and you want an automated process to keep your data up-to-date, or to work with multiple lang…
How do dashboards work if data is computed at build time? Does that mean every time you want to update the data you need another build? I'm interested in live dashboards, is Obversable framework the wrong tool for the job?
Yes, we use continuous deployment (cron) to rebuild as needed. You can also get realtime data on the client if you need to (via fetch or WebSocket to your own servers — it’s “just” JavaScript), but generally we find building static data snapshots a useful constraint because it forces you to think about exactly what data is needed, and as a result the dashboard loads instantly.
Re: Observable 2.0, a static site generator for data apps
#88This looks amazing! I really like the clear seperation of loading/prepping data, and presenting it.
Some requests:
Add simple examples and more clarity to the publish docs.
I assume most ppl would prefer to deploy via github actions[1], which the docs currently just link to a complex deploy file - can you please add some more documentation on this, or add an example of the simplest possible deploy file?
Suggestion: is it possible to use a interface (like in vercel) to connect to a github repo and build/publish on changes?
[1]: https://observablehq.com/framework/getting-started#deploying...
Re: Observable 2.0, a static site generator for data apps
#89Earlier quoted context omitted.
How do dashboards work if data is computed at build time? Does that mean every time you want to update the data you need another build? I'm interested in live dashboards, is Obversable framework the wrong tool for the job?
Yes, we use continuous deployment (cron) to rebuild as needed. You can also get realtime data on the client if you need to (via fetch or WebSocket to your own servers — it’s “just” JavaScript), but generally we find building static data snapshots a useful constraint because it forces you to think about exactly what data is needed, and as a result the dashboard loads instantly.
Row64 dashboards are pretty instant. And interactive.
Edit: link: https://row64.com/
Re: Observable 2.0, a static site generator for data apps
#90It would be really cool if you guys supported Malloy. Maybe you already do? https://www.malloydata.dev/
Observable engineer here. I haven't looked into Malloy much, but Framework's data loaders are very flexible. If you can write a script or binary that uses Malloy and writes to stdout, it can be a data loader. For example, although we use SQL a lot in our internal usage of Framework, Framework doesn't actually have any specific SQL support. We just use our database's normal bindings.
Can you link to the "data loader" API or perhaps even a "data loader" example implementation for something similar to malloy, duckdb, or any other DB/SQL data source/provider?
Would love to see it, thanks in advance!
edit: found it https://observablehq.com/framework/lib/duckdb