Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Observable 2.0, a static site generator for data apps

observablehq.com

Post reply on HN
Print view View original

111–120 of 157 posts

Re: Observable 2.0, a static site generator for data apps

#111

Post by j-pb »

nextworddev wrote: post #106
j-pb wrote: post #73

If you play the history of Observable backwards you start with a company creating a static site generator for dashboards, which then struggles to find a market fit as it tries to bring datascience to middle management, to finally reach a focused, simple and elegant tool for exploratory programming, data visualisation, and interactive documentation in javascript.

Ok but is anyone paying for it?

We used to pay for 4 licenses, until they switched to the weird pricing schemes and the new editor targeted towards people with no programming experience.

I really wish they would open source Observable 0.5, Pluto is currently the only other notebook left that has the flexible data-flow model at reasonable simplicity.

Re: Observable 2.0, a static site generator for data apps

#112

Post by meowtimemania »

tetris11 wrote: post #110

I miss source loading D3 into a simple HTML page, and having D3 tutorials that were up to spec with the latest D3 release. Yeah Observable data pages look cool, but it really feels like excessive JS bloat for the features. I think I miss throwing a D3 viz together without having to load an entire framework library.

@observablehq/plot is what you're looking for if you want to have the same charts loaded via a script tag.

https://observablehq.com/plot/getting-started#plot-in-vanill...

Re: Observable 2.0, a static site generator for data apps

#113

Post by recifs »

tetris11 wrote: post #110

I miss source loading D3 into a simple HTML page, and having D3 tutorials that were up to spec with the latest D3 release. Yeah Observable data pages look cool, but it really feels like excessive JS bloat for the features. I think I miss throwing a D3 viz together without having to load an entire framework library.

You can still use D3 the good old-fashioned way, as a standalone script. See https://d3js.org/getting-started#d3-in-vanilla-html for examples.

Re: Observable 2.0, a static site generator for data apps

#114

Post by time4tea »

mbostock wrote: post #43
jwilber wrote: post #35

Earlier quoted context omitted.

Observable is much more than its library, plot. You mean to compare plot to plotly. There are a number of reasons to choose Observable’s plot over plotly, but to address your point, there is no lock-in here with using plot for the view - you can seemingly use any JS library, including plotly, vega, D3, etc., so I don’t think that’s a huge issue. I agree with your point regarding convincing other scientists to use Jav…

I don’t think the `npm install` will scare people away (Evidence uses that, too), and we’ve definitely tried to make the onboarding process as guided as possible (shout-out to clack.cc for a great CLI prompt library): https://observablehq.com/framework/getting-started And plus you can import libraries directly from a CDN rather than needing to use npm or yarn to manage dependencies. (Though we plan on supporting the…

Importing libs from a cdn is a big no-no for almost any system I work on - they are just another way of surveilling users on the Internet, and building information about the insides of organisations.

Re: Observable 2.0, a static site generator for data apps

#116

Post by recifs »

farhanhubble wrote: post #107

I love how much is possible with the Observable framework and support for libraries like d3.js. However many data apps cannot precompute their outputs. For example, a pipeline that extracts text from documents based on what a user queries, cannot precompute the results and any visualizations must be updated every time. The best hack to accomplish this seems to be rebuilding the app on each update. Or is there another…

The code in a Framework can do whatever you want it to do—it can load data on demand, call an external API, etc. Precomputing data is only an option, not an obligation.

But even when you want things to be very interactive it is a good idea to minimize the data. Expose only the "rows and columns" that you need, and compress it as much as possible. This can be done in a data loader. For example, see the data app we deployed yesterday on hugging-face: its data loaders ingest a large source database (320 files totaling 200GB), and digests it into a single 8MB parquet file that we can then use on the page to "live query" 3 million newspaper titles and dates. https://huggingface.co/spaces/observablehq/fpdn

Re: Observable 2.0, a static site generator for data apps

#117

Post by recifs »

chrisjc wrote: post #91
kuatroka wrote: post #83

A couple of questions: 1. Let's say I got a Sqlite/Duckdb database file on my server. It's got multiple tables and some of them 100M to 150M records. I want to create a plot/table that would have a slider/filter to only bring and show a slice of data at a time. Since it's statically generated data, how is this interactivity achieved? All the possible facets of data filtered by which ever way will be generated? Won't…

Came here with similar questions and Cmd-F "DuckDB". See the comment about "data loaders". Seems like a "data loader" would provide most of what you're asking about. I'm also thinking that a "data loader" combined with duckdb-wasm and arrow would be a pretty nice combination. I imagine that it might not be too difficult to switch two between two implementations of the "data loader" as needed. Switch between reading f…

See the example at https://huggingface.co/spaces/observablehq/fpdn where DuckDB is used both as a data loader (to download and digest 200GB worth of source data into a small 8MB parquet file) and on the client-side to allow the user to do live search queries on the minimized data. Server-side, we're using duckdb-the-binary, and client-side we're using duckdb-wasm.

Re: Observable 2.0, a static site generator for data apps

#118

Post by ipsum2 »

mbostock wrote: post #87
ipsum2 wrote: post #84

Earlier quoted context omitted.

How do dashboards work if data is computed at build time? Does that mean every time you want to update the data you need another build? I'm interested in live dashboards, is Obversable framework the wrong tool for the job?

Yes, we use continuous deployment (cron) to rebuild as needed. You can also get realtime data on the client if you need to (via fetch or WebSocket to your own servers — it’s “just” JavaScript), but generally we find building static data snapshots a useful constraint because it forces you to think about exactly what data is needed, and as a result the dashboard loads instantly.

My use case is monitoring machine learning models as they train, static snapshots doesn't seem like the right approach for me.

Re: Observable 2.0, a static site generator for data apps

#119

Post by asimpletune »

mbostock wrote: post #78
asimpletune wrote: post #63

Earlier quoted context omitted.

One question I have is if there's a way to integrate an observable framework project into an existing static site? I see how I could easily add a project as a subdomain, but what if I wanted to interleave a project I make with observable framework into my existing domain and that static site generator I already use for that domain? By the way, thank you making this. I've been reading and enjoying very much the docume…

Thank you. At a minimum, you could iframe pages built with Framework, or have them live alongside your other pages and link to them. Maybe it would be possible to use Framework’s internal API to generate HTML that could be embedded within another static site generator page but we haven’t explored that idea yet.

Thank you for answering my question. I'm sure after more people use framework an elegant design will make itself more clear. The decision to make data loaders agnostic to specific technology was a welcomed approach, and so I have no doubt a similar result will be achieved with integrating observable framework into existing static sites, however that may look. Thank you!
Post reply on HN