Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Observable 2.0, a static site generator for data apps

observablehq.com

Post reply on HN
Print view View original

151–157 of 157 posts

Re: Observable 2.0, a static site generator for data apps

#151

Post by laurels-marts »

amcaskill wrote: post #143
anentropic wrote: post #125

Earlier quoted context omitted.

This looks very interesting to me, I'm building a BI reporting tool in my company at the moment, but browsing the docs I felt what I was missing was a clear overview of the architecture. e.g. you say above that Evidence takes templated SQL and runs it in DuckDB WASM and then in the docs there's various https://docs.evidence.dev/core-concepts/data-sources/#suppor... like Snowflake, MySQL etc I guess I am wondering whe…

Thanks for the kind words. That’s good feedback on the docs. The tool has evolved pretty dramatically from where it started and we should revisit those diagrams. Evidence is a static site generator. Queries against your sources happen at build time and save to parquet. Queries against the built in DuckDB web assembly instance happen at runtime. Sources (snowflake, Postgres, csv files etc.) run at build time. Pages in…

How fast exactly is DuckDB-Wasm for filtering for interactive coordinated views? Could the inputs be a brush selector range (x0, x1) from a time-series chart and then when you brush it the other components would re-render within… milliseconds? There used to be a cool JS library for this called crossfilter. Not sure if this could be a replacement for it?

Re: Observable 2.0, a static site generator for data apps

#152

Post by cxr »

simonw wrote: post #47

There's an almost bewildering amount of interesting ideas buried in this. Things like data loaders which are ANY script that can output data (as JSON or something else) to standard output. Markdown files with ```js blocks in that get executed. The reinvention of the core Observable notebook to avoid custom syntax. This is really big.

The thing about Markdown code blocks is that they're meant to be read (and therefore rendered—as text). "Markdown files with ```js blocks in that get executed" is just .

Re: Observable 2.0, a static site generator for data apps

#153

Post by amadio »

mbostock wrote: post #14
0cf8612b2e1e wrote: post #5

Sounds a bit like the “baked data” pattern. Which I think is a really good idea. I have long been toying with how to use Datasette to make a deliverable dashboard, so this is interesting.

Yes! It first felt counterintuitive and constraining to prepare data ahead of time, rather than just loading whatever you want on the fly. But we’ve found this to be a great discipline in practice because it forces you to think about what data you actually need to show, and to shift as much of the compute to build time to reduce file sizes (through aggregation and filtering) and accelerate page load. And once you bec…

I think the idea of Framework is really good, but static data limits the applications, excluding monitoring and other cases in which the data is constantly changing, but the dashboard can stay as it is. For example, I'd love to see a revamped Framework version of the LHC beam monitor and related pages (see https://op-webtools.web.cern.ch/vistar/, but check again in 2 months or so, when the accelerator will be running).

In high-energy physics, ROOT is /the/ toolkit for data analysis, and I guess jsROOT (https://root.cern.ch/js/) could also be used to load data to be shown in Framework dashboards. I thought the idea of Framework as a blogging engine with powerful data visualization built-in could be very interesting. Think, for example, about physicists pulling open data (https://opendata.cern.ch) and writing about their analysis or someone pulling data from https://ourworldindata.org/ in their own visualizations to support their case while writing about a particular subject, etc.

Re: Observable 2.0, a static site generator for data apps

#154

Post by rmnclmnt »

pzmarzly wrote: post #51
rmnclmnt wrote: post #12

Earlier quoted context omitted.

Me too, and that lead to developing the « datasette-dashboards » plugin[0]. I use this for my company where all the data is gathered by connectors scheduled in CI, storing data in Git, and triggering a SQLite db build and Datasette deployment. « BI as Code » if you will [0] https://github.com/rclement/datasette-dashboards

How often do you refresh the data (how often your CI runs)?

Depends on the nature of the data and the requirements, can be every hour or once a day.

Most people and organizations have small/mid data and most of all slow data: you do not need fresh data every hour. If you do, you need a way heavier real time analytics data

Re: Observable 2.0, a static site generator for data apps

#155

Post by jaan »

asimpletune wrote: post #63
mbostock wrote: post #9

Hey, HN. We’re thrilled to release Observable Framework today — a new open-source tool for developing data apps. I highly recommend viewing this example report adapted from our internal dashboard analyzing web logs: https://observablehq.com/framework/examples/api/ This technique of “just plot everything” (7.6M requests as a scatterplot) has revealed surprising insights we’ve used to optimize our servers and better co…

One question I have is if there's a way to integrate an observable framework project into an existing static site? I see how I could easily add a project as a subdomain, but what if I wanted to interleave a project I make with observable framework into my existing domain and that static site generator I already use for that domain? By the way, thank you making this. I've been reading and enjoying very much the docume…

Also trying to figure this out for using it with vite! Added a question here: https://github.com/observablehq/framework/discussions/855

Re: Observable 2.0, a static site generator for data apps

#156

Post by tophtucker »

ddanieltan wrote: post #19

I'm super excited to try this out! Couple of questions since I see @mbostock active in the comments. 1. Is the flexibility of languages used in data loaders/backend going to eventually come to the front end/ui? Or will the paradigm always be bring-your-own-language for the data loading but build your dashboard with observablejs/observable plot? 2. Considering ObservableJS is supported by Quarto, can we look forward t…

From the Observable Framework point of view, you’re very welcome to use Apache ECharts or any other library instead of Observable Plot, since you can import whatever you like and it’s all just JavaScript.

Since there was a lot of interest in this thread, Mike added a page to the docs with an ECharts example: https://observablehq.com/framework/lib/echarts

There are two pieces of that example code specific to Framework: the html`` tagged template literal creates a DOM element (see https://github.com/observablehq/htl, also usable outside Framework), and the display function inserts it into the document above the code block (see https://observablehq.com/framework/javascript/display). Note that, whereas Observable Plot takes an options object and returns a DOM element, ECharts instead takes a DOM element and mutates it — but in general they should be equally easy to use in Framework.

Like Plot (and Vega-Lite, another great option), ECharts is also now one of Framework’s built-in “recommended libraries” (see https://observablehq.com/framework/javascript/imports#implic...), meaning that if you reference `echarts` Framework will lazy-load it for you. Adding that was a two-line diff: https://github.com/observablehq/framework/pull/811/files#dif.... But I wanna emphasize that Framework doesn’t have to explicitly “support” a given library for you to use it. “Supporting” in this case just means the convenience of saving you a one-line import statement. But don’t wait for our blessing!! Use whatever.

Re: Observable 2.0, a static site generator for data apps

#157

Post by tophtucker »

kuatroka wrote: post #136
mbostock wrote: post #132

Earlier quoted context omitted.

There are lots of ways to do heatmaps with Observable Plot. See the raster, contour, and cell marks. https://observablehq.com/plot/marks/raster https://observablehq.com/plot/marks/contour https://observablehq.com/plot/marks/cell We generally recommend stacked bar charts over pie and donut charts, so we haven’t prioritized those. But you can already implement them using custom marks, and there’s even a hacky way of do…

I'll check the raster, contour, and cell marks. Thanks. "I don’t understand your comment about the “D3 example.”..." 1. When I visit the Plot Gallery https://observablehq.com/@observablehq/plot-gallery 2. Go down the page to "More from Observable creators" 3. Select an example I like, for example - https://observablehq.com/d/3ea4b4458fed9242?page=2&collectio... It turns out it's D3, not Plot. I think you just have al…

Hah, I just pared down that “More from Observable creators” section the other day to make it less obtrusive, and inadvertently made it more confusingly blend into the notebook contents. This is good feedback; we’ll refine it!
Post reply on HN