Show HN: Microsoft releases Flint, a visualization language for AI agents

>Instead of requiring verbose low-level parameters such as scales, axes, spacing, and layout.

Ok, Microsoft is conflating two different things here: LLMs don't really care about code being low level and verbose, they can read things like Assembly and SPIR-V just fine: visualization is the real issue in that LLMs have no natural understanding of spatial composition through visual comparison because they literally "see" things differently than humans, so the way to get around that is provide them with "visualization" in code form that they can easily reason about and understand, so basically anything that's not deeply nested and has hidden states that they have to reason about.

Also, Flint being stringly typed in JSON is a decision that I don't think I agree with. Looking at the actual spec, this could have just been a normal, human usable TypeScript library, and it would have been 100x better. Using their own example (excuse the formatting):

type SemanticType = "Category" | "YearMonth" | "Profit";

type ChartType = "Heatmap" | "BarChart" | "LineChart" | "ScatterPlot"; // extend as needed

interface ChartEncodings { x: string; y: string; color?: string; size?: string; tooltip?: string; }

interface ChartProperties { colorScheme: string; [key: string]: unknown; // allow other optional properties }

interface ChartSpec { chartType: ChartType; encodings: ChartEncodings; chartProperties: ChartProperties; }

type SemanticTypes = Record;

interface ChartConfig> { data: TData; semantic_types: SemanticTypes; chart_spec: ChartSpec; }

// The actual typed object literal: const chartConfig: ChartConfig = { data: {}, // replace with your actual data shape/type semantic_types: { game: "Category", period: "YearMonth", newUsers: "Profit", }, chart_spec: { chartType: "Heatmap", encodings: { x: "period", y: "game", color: "newUsers", }, chartProperties: { colorScheme: "redblue", }, }, };

EDIT:

Went and actually looked at the source instead of just eyeballing it from the docs, and it was a lot more complete and sophisticated than my assumed mockup already.

Core complaint (string-keyed JSON vs. a real generic authoring surface) still stands, but the specific types I posted aren't what Flint has. My bad.

This comes with a bit of last mile issue, if just chatting with GPT in the chat panel and keep steering it, it's generally fine (as a power user).

But when building it in a tool that serve end users, we are starting to see that a 80% success rate in generating good looking charts can become a big issue. We experienced this when building some data analysis system. So the reliability, expressiveness, and costs (in terms of time and tokens) are hard to achieve all together with directly generating matplotlib, vega-lite etc.

So we essentially designed the langauge as a trade-off across the three, by moving some decisions to the compiler to reduce generation cost while maintain good expressivenss.

This happens if a company has a CEO who presumably can no longer successfully go to the toilet without AI assistance.

Agents, npm, typescript, MCP. All buzzwords are there. Will anyone look at the slop charts? Of course not, the tokens are the goal.

MSFT stock is at 2024 levels. Maybe someone should produce a flint chart and present the agentic work to Nadella. No one buys this AI slop any more.

That's fair, I generally make charts for publication, so I spend much more time and effort on the details. But I can understand this being useful for quick exploration for some people.

Generally speaking, I suggest anyone interested in learning to make charts get familiar with grammar of graphics [0] libraries like Vega-Lite, Observable Plot, ggplot2, Altair. There is a bit of a learning curve if you're used to selecting chart types like in Excel, but once it clicks, it gives you virtually unlimited choices in the kinds of charts you can make. And with ggplot or Observable Plot [1], it's about the same number of lines as something like Flint.

0: https://data.europa.eu/apps/data-visualisation-guide/why-you...

1: https://observablehq.com/@observablehq/plot-gallery

Grammar of graphics has been the foundation of a lot of stuff and definitely worth learning for everyone!

A challenge with GoG is that it assumes configurations as second-class stuff, which makes it quite difficult for users to deal with things like changing formatter, scale, annotations. Flint kinda want to hide this aways (so Flint sets them on behalf of the agent or the user). But yeah, GoG is still the foundation for expressivenss.