Live data from Hacker News

A new way to build with Large Language Models

blog.fixie.ai

31–40 of 46 posts

Re: A new way to build with Large Language Models

#31

I've wanted something like this for years! And just started building it myself last month (mostly for home automation). I like the multi-agent model, and the "thought" steps are pretty interesting. But how do you deal with context length with all the back and forth?

Current context lengths are usually more than adequate for these interactions; the details of each individual step within an execution only need to be retained until the final response is emitted.

Re: A new way to build with Large Language Models

#34
post #9

What's your guidance concerning the risk of prompt injection with applications built on this kind of platform? If developers on your platform are building code that accepts prompt input from untrusted users do you have any measures in place to avoid them tricking the system into executing functions in an inappropriate or unintended way? One thing I'd find useful would be a limit on the number of functions that can be…

I don't know about unintended but why is "inappropriate" a problem? It is user input. You went out of your way to trick it and got what you asked for - how is it different from say user submitted content on regular websites?

I'll never understand this cultural phenomenon. Anybody can open the browser inspector on a random social site and tweak the page to say whatever they want and send screenshots around "implicating" the poor bastards - but none-hypothetically who actually cares?

Sort of akin to running a pub and having a drunk run his mouth. That's not on the establishment that is on the individual.

Re: A new way to build with Large Language Models

#35
post #16

Looks interesting, can you show us more use cases on the landing page? The stock price example is okay, but I want to see what I can use this for that I can't currently do easily. What problems does this solve?

Yeah, great feedback. We'll have a bunch more examples that vary in complexity coming out in the next couple of weeks.

Cool I signed up for access, good luck!

Re: A new way to build with Large Language Models

#36
post #33

Earlier quoted context omitted.

dang comes in with the hammer!!

A user emailed to point this out and I think they made a fair point.

Did the user have a competing project? Never seen this before with much less interesting tech demos.

Re: A new way to build with Large Language Models

#37
post #33

Earlier quoted context omitted.

A user emailed to point this out and I think they made a fair point.

Did the user have a competing project? Never seen this before with much less interesting tech demos.

No, just someone who cares about the quality of HN's front page. The idea that waiting lists don't come with enough information to support a substantive discussion is pretty well established on HN.

What's the interesting demo here? Did I miss something?

Re: A new way to build with Large Language Models

#38
I read a NY Times article by Ted Chiang today in which he made a kind of "stochastic parrot" argument for chatGPT - https://www.newyorker.com/tech/annals-of-technology/chatgpt-.... I believe, on the other hand that chatGPT may eventually be able to generate AGI, but that this will occur emergently and spontaneously. In other words, it will be difficult to predict.

One of the conditions for this, is for chatGPT models to start being able to write their own code in order to produce models of themselves that are more accurate and more efficient. Given this ability, and some fitness criteria, genetic algorithms may be used to create new LLMs. This sounds like science fiction, but once the compute requirements come down for these models (by a couple orders of magnitude), I believe this may be possible.

To what extent does your model allow for semantic models to create semantic models that are themselves more efficient in relation to some fitness criteria? Can I tell a model "You (model) I want you to reproduce using interaction with these other models (some collection of other models) and have the child model offspring be more efficient according to this criteria [for example the resultant models will create short stories that are more likely to receive high ratings on a subreddit devoted to short stories]".

You would need to get around the "model pollution" problem in which LLM models pollute the space for which the models generate data because other models are producing web artifacts (Ted Chiang's Xerox of a Xerox problem). I call this the problem of alpha (direct experience). One of the ways I've thought of to fix this is to have models trained on direct user input (such as cell phone video and pictures from a single user) - I have to admit that I got this idea from Neal Stephenson's Snow Crash (see Gargoyle). If your platform can integrate with visual processing this may have a high information density - object detection in daily videos demonstrating how objects are related to each other in the real world of the user and correlating these into a semantic network.

I'd also suggest that Obsidian integration might be useful.

This is exciting, thanks for making the Fixie SDK public.

Re: A new way to build with Large Language Models

#39
post #37

Earlier quoted context omitted.

Did the user have a competing project? Never seen this before with much less interesting tech demos.

No, just someone who cares about the quality of HN's front page. The idea that waiting lists don't come with enough information to support a substantive discussion is pretty well established on HN. What's the interesting demo here? Did I miss something?

The homepage video with multiple examples was a good demo, but maybe most people don't click that far. That would have probably been a better post.

Re: A new way to build with Large Language Models

#40
post #34
post #9

What's your guidance concerning the risk of prompt injection with applications built on this kind of platform? If developers on your platform are building code that accepts prompt input from untrusted users do you have any measures in place to avoid them tricking the system into executing functions in an inappropriate or unintended way? One thing I'd find useful would be a limit on the number of functions that can be…

I don't know about unintended but why is "inappropriate" a problem? It is user input. You went out of your way to trick it and got what you asked for - how is it different from say user submitted content on regular websites? I'll never understand this cultural phenomenon. Anybody can open the browser inspector on a random social site and tweak the page to say whatever they want and send screenshots around "implicatin…

Whether or not prompt injection is genuinely harmful entirely depends on what developers build with this stuff.

If you're going to give a LLM-driven program the ability to execute functions that can change state in the world you need to understand prompt injection, so you don't accidentally build something that you really shouldn't have built.

The first public prompt injection vulnerability was a Twitter bot which started spam-mentioning people and threatening the president. https://arstechnica.com/information-technology/2022/09/twitt...

Post reply on HN