Live data from Hacker News

Show HN: Microsoft releases Flint, a visualization language for AI agents

microsoft.github.io

51–60 of 149 posts

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#51

>Instead of requiring verbose low-level parameters such as scales, axes, spacing, and layout. Ok, Microsoft is conflating two different things here: LLMs don't really care about code being low level and verbose, they can read things like Assembly and SPIR-V just fine: visualization is the real issue in that LLMs have no natural understanding of spatial composition through visual comparison because they literally "see…

[deleted]

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#52

"For AI agents". I understand why everything needs to be marketed in this way, but it's just ... an easy-to-generate language for expressing charts. That's impressive! That's useful.

Isn't this literally made for AI agents to be accessed through an MCP server? Seems to me the AI agents part of the marketing is important.

But why be exclusive? Why not "Chart language for computer programs to generate"?

I don't want to use an agent at all, but i wouldn't mind generating some charts with an easy-to-generate markup language...

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#53

Earlier quoted context omitted.

That's fair, I generally make charts for publication, so I spend much more time and effort on the details. But I can understand this being useful for quick exploration for some people. Generally speaking, I suggest anyone interested in learning to make charts get familiar with grammar of graphics [0] libraries like Vega-Lite, Observable Plot, ggplot2, Altair. There is a bit of a learning curve if you're used to selec…

Grammar of graphics has been the foundation of a lot of stuff and definitely worth learning for everyone! A challenge with GoG is that it assumes configurations as second-class stuff, which makes it quite difficult for users to deal with things like changing formatter, scale, annotations. Flint kinda want to hide this aways (so Flint sets them on behalf of the agent or the user). But yeah, GoG is still the foundation…

Can you elaborate on what you mean? Why is it more difficult to deal with formatters, scales, annotations compared with other solutions? Unless I'm misunderstanding something, the defaults are similar to what you would get in Flint, and if you want to add or customise anything, it's usually just one extra line. That's kind of the entire point of the grammar of graphics.

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#54

>Instead of requiring verbose low-level parameters such as scales, axes, spacing, and layout. Ok, Microsoft is conflating two different things here: LLMs don't really care about code being low level and verbose, they can read things like Assembly and SPIR-V just fine: visualization is the real issue in that LLMs have no natural understanding of spatial composition through visual comparison because they literally "see…

I do find the chartType part is not quite elegant, since templates should be more extensible. We will need to fix that. For other parts, it's quite common in visualization and diagram etc libraries to have json, since they are easily portable in different rendering contexts.

I'm sure you know that JSON are object literals in TypeScript, there isn't really even a serialization process that's needed there. The AST/IR can still be in JSON, but the authoring surface can be a restricted subset that does not allow logic to execute, that way you can still get the type safety and functionality of TS when you need conditional/loops/logic without throwing away what TS already gives you.

I mean, it would be great if you guys would have like a "TSON" that is basically "JSON with type declaration and comments" from TS, which I think would just solve a lot of problem straight up. JSON itself is just too restrictive and comes with its own bracket verbosity tax.

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#55

Earlier quoted context omitted.

Isn't this literally made for AI agents to be accessed through an MCP server? Seems to me the AI agents part of the marketing is important.

But why be exclusive? Why not "Chart language for computer programs to generate"? I don't want to use an agent at all, but i wouldn't mind generating some charts with an easy-to-generate markup language...

But for that we already have mermaid.js (and its precursor Graphviz/dot).

The only reason to use this instead of existing, mature ones designed for humans is if you are an AI agent.

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#56

Earlier quoted context omitted.

Grammar of graphics has been the foundation of a lot of stuff and definitely worth learning for everyone! A challenge with GoG is that it assumes configurations as second-class stuff, which makes it quite difficult for users to deal with things like changing formatter, scale, annotations. Flint kinda want to hide this aways (so Flint sets them on behalf of the agent or the user). But yeah, GoG is still the foundation…

Can you elaborate on what you mean? Why is it more difficult to deal with formatters, scales, annotations compared with other solutions? Unless I'm misunderstanding something, the defaults are similar to what you would get in Flint, and if you want to add or customise anything, it's usually just one extra line. That's kind of the entire point of the grammar of graphics.

https://microsoft.github.io/flint-chart/#/

The "how it works" section explains a little bit of this. For example, for the heatmap example showing temporal data, for a "good-looking" chart, we need to (1) reconcile the conflict between banded discrete steps and continuous temporal axis, and it requires understanding and setting stepsize and time parser, (2) for the correlation color, we need to set domain etc under the color axis.

These are supposed to be handled automatically as system defaults, but the tricky part is that these decisions are "semantical", thus requires us to understand the data and design principles, thus existing languages won't stretch that far. And the actual good looking spec is actually over 40 lines of json spec with many low-level paramters, way beyond the simple 5 line encoding promised by GoG. Flint uses semantic type and a layout optimization algorithm to handle this, so 5 lines of encoding + data semantic types can derive rest parameters automatically.

Some examples in the gallery are more extreme: like the waterfall chart example is way over 100 lines of code, and sunbusrt, rose chart are even more since compositions are quite difficult in GoG.

Glad to have a discussion on this level! In fact, we wrote a paper about this, will be putting it online in a week or so!

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#57
Nice to stumble over this thread.

I'm not sure if Flint is the right tool for me. I'd like to have a tool that expresses code in visual form for me. For example, right now I need to reverse engineer some code for debugging purposes.

I already found out there are three tasks:

    * Task one fills task two's queue and waits for an event to get notified
    * Task two reads from its queue, forwards elements to task three's queue.
    * Task three reads from its queue and sends a success/fail message back to task two's queue
    * Task two then notifies the waiting task one.
Visually it's easily expressed: 3 bubbles lined up with 2 connections between the neighboring ones.

Which ML tools suited best for that?

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#58

>Instead of requiring verbose low-level parameters such as scales, axes, spacing, and layout. Ok, Microsoft is conflating two different things here: LLMs don't really care about code being low level and verbose, they can read things like Assembly and SPIR-V just fine: visualization is the real issue in that LLMs have no natural understanding of spatial composition through visual comparison because they literally "see…

sadly, I think we are stuck with JSON as the most reliable way to get data / code in and out of an LLM (could be worse, could be YAML) … I’m interested in custom DSLs that improve LLM predictability and it is quite nice to see that even the Microsoft dinosaur “gets it” … see the Contacts example at https://slangify.org/examples which does VCARD to JCARD round tripping as a way to easily roll your own DSL

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#59

>Instead of requiring verbose low-level parameters such as scales, axes, spacing, and layout. Ok, Microsoft is conflating two different things here: LLMs don't really care about code being low level and verbose, they can read things like Assembly and SPIR-V just fine: visualization is the real issue in that LLMs have no natural understanding of spatial composition through visual comparison because they literally "see…

sadly, I think we are stuck with JSON as the most reliable way to get data / code in and out of an LLM (could be worse, could be YAML) … I’m interested in custom DSLs that improve LLM predictability and it is quite nice to see that even the Microsoft dinosaur “gets it” … see the Contacts example at https://slangify.org/examples which does VCARD to JCARD round tripping as a way to easily roll your own DSL

I felt conflicted as well, json is portable and easy to parse / validate and edit. But many models do still struggle. There are some stuff from functional programming might be worth bringing back here.

Re: Show HN: Microsoft releases Flint, a visualization language for AI agents

#60

Nice to stumble over this thread. I'm not sure if Flint is the right tool for me. I'd like to have a tool that expresses code in visual form for me. For example, right now I need to reverse engineer some code for debugging purposes. I already found out there are three tasks: * Task one fills task two's queue and waits for an event to get notified * Task two reads from its queue, forwards elements to task three's queu…

Seems a bit more like https://mermaid.js.org use cases!
Post reply on HN