Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Automatic Generation of Visualizations and Infographics with LLMs

microsoft.github.io

Post reply on HN
Print view View original

31–40 of 56 posts

Re: Automatic Generation of Visualizations and Infographics with LLMs

#31

Post by computerex »

vimesy wrote: post #30
javajosh wrote: post #24

No, absolutely not. How can you trust the output from such a black box system? Who is to say that the LLM won't add or remove data points to make the chart "look good"? Heaven help us if decision makers start taking this output seriously. But of course they will, because the charts will look professional and plausible, because that's what the prompt requires.

"You are a helpful assistant highly skilled in writing PERFECT code for visualizations. Given some code template, you complete the template to generate a visualization given the dataset and the goal described. The code you write MUST FOLLOW VISUALIZATION BEST PRACTICES ie. meet the specified goal, apply the right transformation, use the right visualization type, use the right data encoding, and use the right aestheti…

Telling the LLM that it must do something is not a guarantee that it'll follow through.

Re: Automatic Generation of Visualizations and Infographics with LLMs

#32

Post by minimaxir »

w-m wrote: post #19
ilovefood wrote: post #15

I took this a step further, turning charts into infographics using stable diffusion (SDXL) https://karimjedda.com/beautiful-data-visualizations-powered...

That's a really nice idea. Have you thought about making it a product?

The technology for this type of generation (ControlNet) is already open source and relatively straightforward to reproduce the charts demoed in that post without shenanigans.

There's no moat.

Re: Automatic Generation of Visualizations and Infographics with LLMs

#33

Post by lmeyerov »

javajosh wrote: post #24

No, absolutely not. How can you trust the output from such a black box system? Who is to say that the LLM won't add or remove data points to make the chart "look good"? Heaven help us if decision makers start taking this output seriously. But of course they will, because the charts will look professional and plausible, because that's what the prompt requires.

Agreed. Our customers on the regulated side cannot use an unexplainable UI like that by law.

We take a middle ground with louie.ai of showing the database queries, data transforms, chart config, and any other decision or generation. It's nice being able to watch & check each step and then write in natural language what you want changed, so ends up feeling more like the easier side of pair programming than a blackbox.

Re: Automatic Generation of Visualizations and Infographics with LLMs

#34

Post by dontupvoteme »

pplonski86 wrote: post #16
dontupvoteme wrote: post #2

Ironically I've found GPT to be pretty terrible with plotting libraries like plotly/dash or even matplotlib compared to just about anything else in python.

I wrote a simple wrapper around Matplotlib and ChatGPT-3.5-turbo. The LLM response is a Python code that is executed to get charts. It is working very nice. Here is a repo https://github.com/mljar/plotai - you will find two videos in the readme. Maybe you should work on your prompts?

Huh, neat. It never really bothered me enough/was important enough to spend specifically time on it since i was able to just hit a button to send them back for fixing but that's good to see the extra passes aren't explicitly needed.

Re: Automatic Generation of Visualizations and Infographics with LLMs

#35

Post by pizza »

javajosh wrote: post #24

No, absolutely not. How can you trust the output from such a black box system? Who is to say that the LLM won't add or remove data points to make the chart "look good"? Heaven help us if decision makers start taking this output seriously. But of course they will, because the charts will look professional and plausible, because that's what the prompt requires.

How do you trust matplotlib? Same way: if you need to audit plots, audit the generated source code.

Re: Automatic Generation of Visualizations and Infographics with LLMs

#36

Post by cooperaustinj »

nologic01 wrote: post #21
Incolotopo wrote: post #10

Earlier quoted context omitted.

And what in particular is now novel or unique to the general 'issue' you mention? Most companies use OpenSource in one way or the other. Nonetheless, a company like MS has probably already build visualizers purely commercially (see excel) or/and is absolutly able to write it themselfs.

I am not sure what you are talking about. If I release a novel visualization library on github under some open source license I want it to be attributed to me. I don't want some specialized LLM to be lifting and offering the same visualization concepts to unnamed corporates for a hefty fee without me ever even knowing about it and these corporates pretending they don't know where that concept is coming from. It is yo…

Everything you described been possible since the dawn of intellectual property. Just replace LLM with "person".

Furthermore, it isn't theft to learn from others' work and reproduce similar qualities.

Re: Automatic Generation of Visualizations and Infographics with LLMs

#37

Post by nologic01 »

cooperaustinj wrote: post #36
nologic01 wrote: post #21

Earlier quoted context omitted.

I am not sure what you are talking about. If I release a novel visualization library on github under some open source license I want it to be attributed to me. I don't want some specialized LLM to be lifting and offering the same visualization concepts to unnamed corporates for a hefty fee without me ever even knowing about it and these corporates pretending they don't know where that concept is coming from. It is yo…

Everything you described been possible since the dawn of intellectual property. Just replace LLM with "person". Furthermore, it isn't theft to learn from others' work and reproduce similar qualities.

Possible is not the same as admissible.

Good to know that the prevailing commercial tech culture now sees plagiarism and stealing ideas without attribution as the modern way of doing business and hopes that dressing things up under some algorithmic veil will hide the act.

I guess the pit of moral decline has no bottom. The consolation is that theft has never been the road to wealth. Once the plundering is over the only thing that is left is a wasteland.

It seems that Microsoft has finally found a way to kill the open source "cancer".

Re: Automatic Generation of Visualizations and Infographics with LLMs

#38

Post by ilovefood »

aliyeysides wrote: post #22
ilovefood wrote: post #20

Earlier quoted context omitted.

Thank you for the kind words. I wouldn't even know where to start. There's a lot I can code and do, but making something into a product is something I have no experience with. What's your recommended approach? Even if open-source, curious to learn.

Continue as an open source project by building features and refining the product. Once you have enough users you can start offering premium features and support to organizations. Maybe even apply to an accelerator? Good Luck!

Thanks for the positivity, I'll give it a shot.

Re: Automatic Generation of Visualizations and Infographics with LLMs

#39

Post by andybak »

nologic01 wrote: post #37
cooperaustinj wrote: post #36

Earlier quoted context omitted.

Everything you described been possible since the dawn of intellectual property. Just replace LLM with "person". Furthermore, it isn't theft to learn from others' work and reproduce similar qualities.

Possible is not the same as admissible. Good to know that the prevailing commercial tech culture now sees plagiarism and stealing ideas without attribution as the modern way of doing business and hopes that dressing things up under some algorithmic veil will hide the act. I guess the pit of moral decline has no bottom. The consolation is that theft has never been the road to wealth. Once the plundering is over the on…

I'm afraid I'm just unclear on exactly what part of this you argue is crossing a moral line.

I.e. what is being stolen without attribution? I'm genuinely not getting what you mean in this specific case.

Re: Automatic Generation of Visualizations and Infographics with LLMs

#40

Post by esafak »

ilovefood wrote: post #15

I took this a step further, turning charts into infographics using stable diffusion (SDXL) https://karimjedda.com/beautiful-data-visualizations-powered...

It's chartjunk and should be used sparingly (e.g., on covers) rather than the actual content. I think it would be used frivolously in the hands of an undisciplined person.

https://en.wikipedia.org/wiki/Chartjunk

Post reply on HN