Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Observable 2.0, a static site generator for data apps

observablehq.com

Post reply on HN
Print view View original

51–60 of 157 posts

Re: Observable 2.0, a static site generator for data apps

#51

Post by pzmarzly »

rmnclmnt wrote: post #12
0cf8612b2e1e wrote: post #5

Sounds a bit like the “baked data” pattern. Which I think is a really good idea. I have long been toying with how to use Datasette to make a deliverable dashboard, so this is interesting.

Me too, and that lead to developing the « datasette-dashboards » plugin[0]. I use this for my company where all the data is gathered by connectors scheduled in CI, storing data in Git, and triggering a SQLite db build and Datasette deployment. « BI as Code » if you will [0] https://github.com/rclement/datasette-dashboards

How often do you refresh the data (how often your CI runs)?

Re: Observable 2.0, a static site generator for data apps

#52

Post by zurfer »

mbostock wrote: post #30
ZeroCool2u wrote: post #17

Earlier quoted context omitted.

This seems nice and the plots look great, but I have a hard time imagining switching to Observable from Plotly since there doesn't seem to be a way to make any plots interactive. By which I mean Zoom and Pan. The nearest point high light feature is nice, but what if I want to zoom in? None of the examples here seem to be able to do that and quick google search doesn't make it seem like that's straight forward. That's…

We’re working on zooming and panning for Observable Plot ( https://github.com/observablehq/plot/pull/1738 ) and other interactions such as brushing ( https://github.com/observablehq/plot/pull/721 ) — all of this is already possible, we just haven’t packaged it up in a convenient way yet ( https://github.com/observablehq/plot/pull/1871 ). And as skybrian pointed out, you can also get interactivity “for free” with Obse…

I also love how Observable Plot looks, but agree with top poster, the things that keep me from switching are: - Python wrapper - Out of the box interactivity

Re: Observable 2.0, a static site generator for data apps

#53

Post by kepano »

I appreciate the nod to "File over app"[1] in the announcement. It's so cool that a Markdown file with code blocks can be the source for complex data visualizations and dashboards. Interoperability of this kind makes me giddy. I played around with editing an Observable site from Obsidian and it works great[2].

[1]: https://stephango.com/file-over-app

[2]: https://twitter.com/kepano/status/1758202572446581025

Re: Observable 2.0, a static site generator for data apps

#54

Post by polskibus »

mbostock wrote: post #9

Hey, HN. We’re thrilled to release Observable Framework today — a new open-source tool for developing data apps. I highly recommend viewing this example report adapted from our internal dashboard analyzing web logs: https://observablehq.com/framework/examples/api/ This technique of “just plot everything” (7.6M requests as a scatterplot) has revealed surprising insights we’ve used to optimize our servers and better co…

Thank you Mike for pushing the visualisation envelope for so many years.

Is the new Framework going to support virtualized data access for data sets too large to be sent over network (think of a pivot table that allows to browse huge data warehouse) - it is impossible to prepare entire file upfront, so data queries must happen incrementally with users actions? Or is it completely the other direction from where your vision for Framework is?

Re: Observable 2.0, a static site generator for data apps

#55

Post by d--b »

mbostock wrote: post #9

Hey, HN. We’re thrilled to release Observable Framework today — a new open-source tool for developing data apps. I highly recommend viewing this example report adapted from our internal dashboard analyzing web logs: https://observablehq.com/framework/examples/api/ This technique of “just plot everything” (7.6M requests as a scatterplot) has revealed surprising insights we’ve used to optimize our servers and better co…

At last!

Time to call it quits for https://www.jigdev.com :-D

Godspeed Observable, hope you guys make it big

Re: Observable 2.0, a static site generator for data apps

#56

Post by mbostock »

polskibus wrote: post #54
mbostock wrote: post #9

Hey, HN. We’re thrilled to release Observable Framework today — a new open-source tool for developing data apps. I highly recommend viewing this example report adapted from our internal dashboard analyzing web logs: https://observablehq.com/framework/examples/api/ This technique of “just plot everything” (7.6M requests as a scatterplot) has revealed surprising insights we’ve used to optimize our servers and better co…

Thank you Mike for pushing the visualisation envelope for so many years. Is the new Framework going to support virtualized data access for data sets too large to be sent over network (think of a pivot table that allows to browse huge data warehouse) - it is impossible to prepare entire file upfront, so data queries must happen incrementally with users actions? Or is it completely the other direction from where your v…

If you generate Apache Parquet files you can use DuckDB to make range requests and not download everything to the client. This is pretty magical and allows you to have surprisingly large datasets still queryable at interactive speeds.

But the general idea is to not send everything the client — to be more deliberate and restrictive in what you send, and also what you show. So you probably shouldn’t use this for a general-purpose pivot table that’s trying to show “everything” in your data warehouse and enable ad hoc exploration. You’d instead design more specific, opinionated views, and then craft corresponding data loaders that generate specific pre-aggregated datasets.

Re: Observable 2.0, a static site generator for data apps

#57

Post by tophtucker »

simonw wrote: post #47

There's an almost bewildering amount of interesting ideas buried in this. Things like data loaders which are ANY script that can output data (as JSON or something else) to standard output. Markdown files with ```js blocks in that get executed. The reinvention of the core Observable notebook to avoid custom syntax. This is really big.

Yeah data loaders are like a UNIX pipe to a reactive notebook cell (?). There needn’t be any question of “do the data loaders support this or that”; it doesn’t even have a concept of “supporting” beyond supporting stdout… Still thinking through how to understand it myself!!

Re: Observable 2.0, a static site generator for data apps

#58

Post by mbostock »

zurfer wrote: post #52
mbostock wrote: post #30

Earlier quoted context omitted.

We’re working on zooming and panning for Observable Plot ( https://github.com/observablehq/plot/pull/1738 ) and other interactions such as brushing ( https://github.com/observablehq/plot/pull/721 ) — all of this is already possible, we just haven’t packaged it up in a convenient way yet ( https://github.com/observablehq/plot/pull/1871 ). And as skybrian pointed out, you can also get interactivity “for free” with Obse…

I also love how Observable Plot looks, but agree with top poster, the things that keep me from switching are: - Python wrapper - Out of the box interactivity

We’re working on the interactivity, but we’re not going to do a Python wrapper as the goal of Observable Framework (and Plot) is to leverage web technologies with JavaScript in the front-end for interactive graphics — while doing whatever language you like, including Python, on the back-end for data analysis. There is a R wrapper for Observable Plot (https://github.com/juba/obsplot) and so I imagine someone could write one for Python, but ultimately we don’t think you’ll need it with Framework’s polyglot data loaders since you can seamlessly move between languages (front-end and back-end).

Re: Observable 2.0, a static site generator for data apps

#60

Post by tophtucker »

kepano wrote: post #53

I appreciate the nod to "File over app"[1] in the announcement. It's so cool that a Markdown file with code blocks can be the source for complex data visualizations and dashboards. Interoperability of this kind makes me giddy. I played around with editing an Observable site from Obsidian and it works great[2]. [1]: https://stephango.com/file-over-app [2]: https://twitter.com/kepano/status/1758202572446581025

We’ve talked about and shared your “File over app” manifesto so many times internally over the last few months. It’s one of those tweets that gets immortalized as the perfect crystallization of an ethos that we might otherwise have only been able to gesture at vaguely. It gives the ethos weight and clarity and credibility, and it’s such a relief to be able to point to it! I’m very grateful. —an Observable employee
Post reply on HN