It pains me to think of the energy expenditure being used just to see if two addresses are the same.
I replaced 50 lines of code with a single LLM prompt
41–45 of 45 posts
Re: I replaced 50 lines of code with a single LLM prompt
#42On the surface this seems incredibly stupid. But after thinking on it for a minute - maybe use cases with very low tokens in, very low tokens out, makes sense. Still feels awful, but maybe. Probably not. But maybe.
I'm wondering if there's a prototyping use case in there somewhere. Like... throw in a bunch of LLM calls that return vaguely sane data, in order to get the thing running, then replace them with something reliable before you get to production. Would that speed up building a demo enough to be worth doing?
Re: I replaced 50 lines of code with a single LLM prompt
#43They want it to return a single token yes/no, which may not work so well since it doesn't have "space to think". Chain of thought is much more reliable. But that costs more.. but they ended up anyway doing: >The other key will be 'reason' and include a free text explanation of why you chose Yes or No. But they did yes/no FIRST, then reason. So he ended up asking for the answer, and then asked it to _justify_ why that…
This exactly.
When prompted to complete "The moon is made of ", GPT3.5 returns "cheese" or "green cheese" > 52% of the time.[1]
This article suggests a method that will be statistically right most of the time, and confidently wrong the rest of it.
Re: I replaced 50 lines of code with a single LLM prompt
#44This might not be the best solution to the problem but for the developer it worked. I think we are going to see implementations like this more and more. I worry that using LLMs like this will work in 99% of cases but what if you are in that 1% where an LLM can't matchup your address and you can't use a service or can't verify your address because the computer says no?
Re: I replaced 50 lines of code with a single LLM prompt
#45On the surface this seems incredibly stupid. But after thinking on it for a minute - maybe use cases with very low tokens in, very low tokens out, makes sense. Still feels awful, but maybe. Probably not. But maybe.
I'm wondering if there's a prototyping use case in there somewhere. Like... throw in a bunch of LLM calls that return vaguely sane data, in order to get the thing running, then replace them with something reliable before you get to production. Would that speed up building a demo enough to be worth doing?
Yeah.... that sounds like a very good idea. LLMs for prototyping APIs. Basically a stub of sorts.