Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an internal agent: Code-driven vs. LLM-driven workflows

lethain.com

Post reply on HN
Print view View original

31–36 of 36 posts

Re: Building an internal agent: Code-driven vs. LLM-driven workflows

#31

Post by Havoc »

galaxyLogic wrote: post #10

What I'm struggling with is, when you ask AI to do something, its answer is always undeterministically different, more or less. If I start out with a "spec" that tells AI what I want, it can create working software for me. Seems great. But let's say some weeks, or months or even years later I realize I need to change my spec a bit. I would like to give the new spec to the AI and have it produce an improved version of…

You can actually force things like respond with true or false reliably via gbnf. But yeah within those two choices it is still nondeterministic

> exact same output for the exact same input?

If you set temp to zero it gets close but as I understand it not perfect

Re: Building an internal agent: Code-driven vs. LLM-driven workflows

#32

Post by xtiansimon »

heliumtera wrote: post #25
WhiteNoiz3 wrote: post #22

I'm struggling to understand why an LLM even needs to be involved in this at all. Can't you write a script that takes the last 10 slack messages and checks the github status for any URLs and adds an emoji? It could be a script or slack bot and it would work far more reliably and cost nothing in LLM calls. IMO it seems far more efficient to have an LLM write a repeatable workflow once than calling an LLM every time.

Maybe the audience is not developers at all? Someone that does not know anything about computers and computation might not comprehend how easy or complex a given task is. For a whole class of people, checking a key in a json object might be as complex and difficult as creating a compiler. Some of those are in charge of evaluating progress and development of software. Here's the magic, by now everyone can understand t…

> “For a whole class of people, checking a key in a json object might be as complex and difficult as creating a compiler.”

Ugg. I think this is me. I’m self taught (never once made a compiler in a course or class) and making scripts for ETL at work mostly from CSV input. And JSON/APIs are aggravating to me.

I’ve yet to see the Matrix in JSON data structures (Is it storage? Is it wire protocol?). I can follow _examples_ in documentation, but struggle to put parts together from Swagger or some documentation to get the data view I need. For a while I thought some kind of UML diagramming projects would do it for me—to see the Forest and the trees—but the answer was not there.

So, yes, if I can “vibe” code with ChatIA to get over the mental structural hump to make the right joins and calls, I’m all in.

https://docs.clover.com/dev/docs/making-rest-api-calls

https://api.mobilebytes.com/

Re: Building an internal agent: Code-driven vs. LLM-driven workflows

#33

Post by small_scombrus »

xtiansimon wrote: post #32
heliumtera wrote: post #25

Earlier quoted context omitted.

Maybe the audience is not developers at all? Someone that does not know anything about computers and computation might not comprehend how easy or complex a given task is. For a whole class of people, checking a key in a json object might be as complex and difficult as creating a compiler. Some of those are in charge of evaluating progress and development of software. Here's the magic, by now everyone can understand t…

> “For a whole class of people, checking a key in a json object might be as complex and difficult as creating a compiler.” Ugg. I think this is me. I’m self taught (never once made a compiler in a course or class) and making scripts for ETL at work mostly from CSV input. And JSON/APIs are aggravating to me. I’ve yet to see the Matrix in JSON data structures (Is it storage? Is it wire protocol?). I can follow _example…

> Is it storage? Is it wire protocol?

Yes.

It's just a standardised way to represent data structures in text. You can then save that text to a file for storage, or send the text over the wire for data transfer. As long as everyone involved knows they're saving/loading or talking JSON then everyone knows exactly how to read/write the data.

It is a very literal representation of (specifically JavaScript, but generally any) data-structures in text.

Re: Building an internal agent: Code-driven vs. LLM-driven workflows

#34

Post by xtiansimon »

small_scombrus wrote: post #33
xtiansimon wrote: post #32

Earlier quoted context omitted.

> “For a whole class of people, checking a key in a json object might be as complex and difficult as creating a compiler.” Ugg. I think this is me. I’m self taught (never once made a compiler in a course or class) and making scripts for ETL at work mostly from CSV input. And JSON/APIs are aggravating to me. I’ve yet to see the Matrix in JSON data structures (Is it storage? Is it wire protocol?). I can follow _example…

> Is it storage? Is it wire protocol? Yes. It's just a standardised way to represent data structures in text. You can then save that text to a file for storage, or send the text over the wire for data transfer. As long as everyone involved knows they're saving/loading or talking JSON then everyone knows exactly how to read/write the data. It is a very literal representation of (specifically JavaScript, but generally…

Right. Now the problem for me is these structures don’t come with maps. They’re also like relational databases. If you have to add the mixin calls, how do you got them all? Or know you’ve reconstructed the data model correctly? Where’s the blueprint?

Re: Building an internal agent: Code-driven vs. LLM-driven workflows

#35

Post by small_scombrus »

xtiansimon wrote: post #34
small_scombrus wrote: post #33

Earlier quoted context omitted.

> Is it storage? Is it wire protocol? Yes. It's just a standardised way to represent data structures in text. You can then save that text to a file for storage, or send the text over the wire for data transfer. As long as everyone involved knows they're saving/loading or talking JSON then everyone knows exactly how to read/write the data. It is a very literal representation of (specifically JavaScript, but generally…

Right. Now the problem for me is these structures don’t come with maps. They’re also like relational databases. If you have to add the mixin calls, how do you got them all? Or know you’ve reconstructed the data model correctly? Where’s the blueprint?

> Where’s the blueprint?

A JSON Schema file that can be directly linked in your .JSON file!

But otherwise it's the same way you know anything. Documentation and trial and error

Re: Building an internal agent: Code-driven vs. LLM-driven workflows

#36

Post by xtiansimon »

small_scombrus wrote: post #35
xtiansimon wrote: post #34

Earlier quoted context omitted.

Right. Now the problem for me is these structures don’t come with maps. They’re also like relational databases. If you have to add the mixin calls, how do you got them all? Or know you’ve reconstructed the data model correctly? Where’s the blueprint?

> Where’s the blueprint? A JSON Schema file that can be directly linked in your .JSON file! But otherwise it's the same way you know anything. Documentation and trial and error

> "...otherwise it's the same way you know anything."

Not anymore. Now I can harangue ChatAI to explain it to me, and fill-in gaps in my JS knowledge at the same time.

Post reply on HN