LLM tech comes to Wolfram Language • Hacker News Board

Re: LLM tech comes to Wolfram Language

#31

Post by ilaksh » Fri, Jun 30, 2023, 12:49 PM UTC

Relies on GPT-4. Maybe there could be a multistage way to get similar automated API usage with smaller open source models?

I wonder if it would be possible to train a 7B or 13B model to generate code in just one specific programming language. Train it with example problem input/ program output pairs. Then train another small model to translate natural language in a specific domain into an input for the coder model. And maybe a third to translate that into a different real programming language.

The point of this being that you can use smaller GPU instances and dedicate all of the limited power of each model to narrower domain that may be more tractable for it.

Re: LLM tech comes to Wolfram Language

#32

Post by Xen9 » Fri, Jun 30, 2023, 2:07 PM UTC

golol wrote: post #16

Symbolic AI could never become great because it was missing the connection to the intricacies of the real world that you can only get from data. I think a symbioses of symbolic techniques with LLMs or generally multimodal autoregressive foundation models will lead to our first legit "AGIish" agents. The LLM takes the role of a little gremlin inside the machine that provides the magic sauce, that tiny bit of general i…

By far, symbiosis with other LLMs, not traditional programs, will lead to more power AGIs.

Re: LLM tech comes to Wolfram Language

#33

Post by DesiLurker » Fri, Jun 30, 2023, 4:04 PM UTC

golol wrote: post #16

Symbolic AI could never become great because it was missing the connection to the intricacies of the real world that you can only get from data. I think a symbioses of symbolic techniques with LLMs or generally multimodal autoregressive foundation models will lead to our first legit "AGIish" agents. The LLM takes the role of a little gremlin inside the machine that provides the magic sauce, that tiny bit of general i…

I suspect Symbolic AI might become a part of the larger LLM based AI. IMO symbolic AI is really a representation of 'hard truths' where there is no room for fuzziness like physical laws. But real world is fuzzy so the AI needs to develop a discretion on where to apply symbolic AI and how much.

one example may be an AI can look at the car video feed and use it to determine the speed of a car but to recognize the mile/road markers it may have to use the traditional CNN type modules, kinda like what we do.

Overall I feel this has a potential of ballooning into something really interesting like theoretical physics research could be largely automated using this type of combo (in somewhat not-too-distant future).

Re: LLM tech comes to Wolfram Language

#34

Post by mark_l_watson » Fri, Jun 30, 2023, 5:34 PM UTC

ilaksh wrote: post #31

Relies on GPT-4. Maybe there could be a multistage way to get similar automated API usage with smaller open source models? I wonder if it would be possible to train a 7B or 13B model to generate code in just one specific programming language. Train it with example problem input/ program output pairs. Then train another small model to translate natural language in a specific domain into an input for the coder model. A…

I tried it: you get to choose between GPT-3.5 and GPT-4.

Re: LLM tech comes to Wolfram Language

#35

Post by mark_l_watson » Fri, Jun 30, 2023, 6:02 PM UTC

I used to subscribe to Wolfram Cloud/Desktop. I liked it, but never really loved the language itself.

I tried a Chat Notebook on Wolfram Cloud this morning, and asked it to write a script to fetch data from DBPedia and present it. It generated Wolfram Language code, so that was very cool.

Re: LLM tech comes to Wolfram Language

#36

Post by villgax » Fri, Jun 30, 2023, 6:24 PM UTC

All I actually want is an LLM which can adhere to a set of rules, like a language spec, library/framework code & then help me make stuff rather than hallucination of arbitrary version numbers & staying in an infinite loop of trying every iteration of code by myself in an IDE

Re: LLM tech comes to Wolfram Language

#37

Post by rcarmo » Fri, Jun 30, 2023, 8:51 PM UTC

Ycdr4thfdd wrote: post #11

dsign wrote: post #10

Earlier quoted context omitted.

You do. I used it like that for many years, until I could afford the PC version. The raspberry pi version is version-capped though, to whatever they have released up to that point for that platform.

Are there restrictions against running it on a more powerful arm64 server?

At one time, it would test for the existence of Pi-specific devices, but it’s been a long while since I ran it there.

Re: LLM tech comes to Wolfram Language

#38

Post by golol » Fri, Jun 30, 2023, 9:29 PM UTC

DesiLurker wrote: post #33

golol wrote: post #16

Symbolic AI could never become great because it was missing the connection to the intricacies of the real world that you can only get from data. I think a symbioses of symbolic techniques with LLMs or generally multimodal autoregressive foundation models will lead to our first legit "AGIish" agents. The LLM takes the role of a little gremlin inside the machine that provides the magic sauce, that tiny bit of general i…

I suspect Symbolic AI might become a part of the larger LLM based AI. IMO symbolic AI is really a representation of 'hard truths' where there is no room for fuzziness like physical laws. But real world is fuzzy so the AI needs to develop a discretion on where to apply symbolic AI and how much. one example may be an AI can look at the car video feed and use it to determine the speed of a car but to recognize the mile/…

Yes, I agree. Symbolic AI is for hardness, which we can use for the skeleton of the agent. LLMs are the fuzziness that surrounds and interfaces the skeleton. Everyone who writes some simple code that assembles a promot is already doing that.

Re: LLM tech comes to Wolfram Language

#39

Post by golol » Fri, Jun 30, 2023, 9:32 PM UTC

Xen9 wrote: post #32

golol wrote: post #16

Symbolic AI could never become great because it was missing the connection to the intricacies of the real world that you can only get from data. I think a symbioses of symbolic techniques with LLMs or generally multimodal autoregressive foundation models will lead to our first legit "AGIish" agents. The LLM takes the role of a little gremlin inside the machine that provides the magic sauce, that tiny bit of general i…

By far, symbiosis with other LLMs, not traditional programs, will lead to more power AGIs.

If you connect a bunch of LLMs with a hard coded structure (Like sensory module -> planning module -> action agent or something like that) then for me that is like assembling a symbolic AI system out of non-symbolic components.

Re: LLM tech comes to Wolfram Language

#40

Post by Xen9 » Fri, Jun 30, 2023, 11:03 PM UTC

golol wrote: post #39

Xen9 wrote: post #32

Earlier quoted context omitted.

By far, symbiosis with other LLMs, not traditional programs, will lead to more power AGIs.

If you connect a bunch of LLMs with a hard coded structure (Like sensory module -> planning module -> action agent or something like that) then for me that is like assembling a symbolic AI system out of non-symbolic components.

I think the magic of LLMs, if any, is the fact that they can make these kind of "symbolic AI" systems to work.

AutoGPT / AgentGPT / TeenageGPT will inevitably start borrowing ideas from Marvin Minsky & other symbolic AI / cognitive science researchers.

The space of different configurations is much larger than simple connections. It's in fact possibly larger than our imagination. This is because it's hard to comprehend all inputs to a single GPT-4 instance, and the space of LLM-multiagents is essentially the space of *graph theoretical graphs* of GPT-4 instances, practically in four dimensions (if one needs to start thinking about the signals strictly).