51–56 of 56 posts
Earlier quoted context omitted.
How do you trust matplotlib? Same way: if you need to audit plots, audit the generated source code.
So instead of auding MPL once (or never because MPL doesn't have a habit of broken output) I should audit the output of this LLM for every query because it does have a habit of hallucinating?
Earlier quoted context omitted.
As they say, people are unwilling to understand something if their monetary gain depends on not understanding it. Let me break it down for you. If I ask for a visualization that squares the circle and there is one repo that has an example of squaring the circle, the LLM will "arrive" at a way of squaring the circle.
That's not really answering my question. If (1) an LLM is able to arrive at solutions in the same class of difficulty as the solution for the target problem and (2) it's not possible to establish the provenance of the solution actually offered by the LLM, then what's the argument for assuming that the solution is based on IP rather than constructive reasoning?
Retrain the LLM without access to the repo data. Ask for the same solution. Enjoy the hallucination. Provenance established.
Was playing with the library this morning, the interesting part to me was the 'goal explorer' which generates the questions to asks of the data. Keen to see more research into this part specially making the questions more specific to the dataset in question and overlaying real-world situations.
But knowing _what_ to look for in the data given a problem statement - that's valuable, and hard to teach. LLMs have such a broad base of "knowledge", they can be reasonably good at this in just about any domain.
Was playing with the library this morning, the interesting part to me was the 'goal explorer' which generates the questions to asks of the data. Keen to see more research into this part specially making the questions more specific to the dataset in question and overlaying real-world situations.
Honestly, that's the more interesting and more difficult part. Anyone with basic training can be coerced to slice and dice schemas and configs until pretty graphs are produced. LLMs might not even be the best for that. But knowing _what_ to look for in the data given a problem statement - that's valuable, and hard to teach. LLMs have such a broad base of "knowledge", they can be reasonably good at this in just about…
Earlier quoted context omitted.
That's not really answering my question. If (1) an LLM is able to arrive at solutions in the same class of difficulty as the solution for the target problem and (2) it's not possible to establish the provenance of the solution actually offered by the LLM, then what's the argument for assuming that the solution is based on IP rather than constructive reasoning?
thats too many ifs. Retrain the LLM without access to the repo data. Ask for the same solution. Enjoy the hallucination. Provenance established.
Earlier quoted context omitted.
Honestly, that's the more interesting and more difficult part. Anyone with basic training can be coerced to slice and dice schemas and configs until pretty graphs are produced. LLMs might not even be the best for that. But knowing _what_ to look for in the data given a problem statement - that's valuable, and hard to teach. LLMs have such a broad base of "knowledge", they can be reasonably good at this in just about…
Right, isn't knowing what to look for a must have on the path to AGI?