Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Microsoft releases Flint, a visualization language for AI agents

microsoft.github.io

Post reply on HN
Print view View original

1–10 of 149 posts

Show HN: Microsoft releases Flint, a visualization language for AI agents

#1

Post by chenglong-hn »

Data visualizations are the bridge between user and data.

But building AI agents that can generate visualizations reliably can be very tricky:

- simple chart specs can be reliable, but generated charts are often of low quality due to reliance on system defaults; - complex chart specs with explicit details can produce good-looking charts, but they are verbose and agents can struggle with reliability

We figured out it is a limitation on the language issue (not just AI capability thing) -- current visualization languages are a bit too low-level for AI agents, requiring them to explicitly make visual decisions that are supposed to be handled by a good compiler. Flint is a visualization intermediate language to address this issue, allow AI agents to solve this last-mile human-agent interaction problem. It provides a simple semantic-type based specification, and contains a layout optimization engine that can produce good-looking charts (filled with derived low-level details) from simple high-level specs. The result is also very human understandable and adaptable. Flint powers data formulator for generating visualizations (another open source project from microsoft https://data-formulator.ai/).

Flint is available open source, and we built a MCP server that you can directly plug flint in your favorite agent app to play with data.

Show HN: Microsoft releases Flint, a visualization language for AI agents
microsoft.github.io

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#4

Post by theK »

> requiring them to explicitly make visual decisions that are supposed to be handled by a good compiler

Isnt graphviz there for the same reason?

Edit: I see it is using JSON as the declaration language, I am OK with llms being "good at json" but a syntax also consumable by humans it is not!

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#6

Post by FailMore »

The charts are very nice, and I think the visualisation layer for LLMs is a very interesting problem.

I’ve been building https://smalldocs.org for this exact reason. It’s an office suite for AI agents - but my main use case is giving a cli based LLM the canvas to express itself - charts, mermaid diagrams, etc. I’ve extended it a bit further to be a format for all types of work so the agent can embed slides and spreadsheets in a document.

Sample document: https://smalldocs.org/blogs/what-is-a-smalldoc

Source: https://github.com/espressoplease/smalldocs

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#7

Post by giancarlostoro »

FailMore wrote: post #6

The charts are very nice, and I think the visualisation layer for LLMs is a very interesting problem. I’ve been building https://smalldocs.org for this exact reason. It’s an office suite for AI agents - but my main use case is giving a cli based LLM the canvas to express itself - charts, mermaid diagrams, etc. I’ve extended it a bit further to be a format for all types of work so the agent can embed slides and spread…

> mermaid diagrams

I'm terrible at diagrams, so I gave GPT very generic descriptions of one of our project, to convert in to that mermaid style, then for Lucid I pasted it in there, and had a visualization of what I needed. Worked out nicely.

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#8

Post by chenglong-hn »

animal_spirits wrote: post #5

It compiles into Echarts, but echarts already has a JSON co figuration spec

It's more like a simple high-level spec to make it easier. The idea is that you don't have to fill position / axes details just to make the chart work. The compiler has a bit of magic of using semantic types to optimize what parameters will be set in ECharts.

In some composite chart examples, the good-looking echart spec is like 5x longer than the simple Flint one!

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#9

Post by chenglong-hn »

theK wrote: post #4

> requiring them to explicitly make visual decisions that are supposed to be handled by a good compiler Isnt graphviz there for the same reason? Edit: I see it is using JSON as the declaration language, I am OK with llms being "good at json" but a syntax also consumable by humans it is not!

In fact, Json as a common language for human in visualization has been around for a while! The benefit of declarative grammar is that users can effective manipulate specs through UI (drag and drop, clicks).

Btw, Flint is intentionally designed to allow agent skip low-level params like scale, axe, zero, step size etc (which are extremely crucial for "GOOD-looking") and they are dynamically optimized by the compiler. So AI agents can have a easier time.

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#10

Post by neomantra »

This is cool to see from a research team. A few weeks ago I was exploring a similar idea with ntcharts, where a user or LLM can specify a chart in a Golang or JSON object...

and then that spec would be rendered either to a Bubble TUI via NTCharts or to HTML/SVG via ECharts. That Echarts HTML could be naturally served by a Golang http service.

But Flint goes much deeper with semantic layers and settings optimizations. Perhaps a NTChart, or whatever terminal chart, could be a rendering target? I'll add it to the list to explore...

https://github.com/NimbleMarkets/ntcharts/blob/spec/spec/REA...

Post reply on HN