With the current state of "AI", this strikes me as a "I had a problem and used AI, now I have two problems" kind of situation in most cases.
[deleted]
Throw more AI at your problems
41–50 of 59 posts
Re: Throw more AI at your problems
#42[deleted]
Re: Throw more AI at your problems
#43With the current state of "AI", this strikes me as a "I had a problem and used AI, now I have two problems" kind of situation in most cases.
You have 4 problems because the AI made two more of them up!
Re: Throw more AI at your problems
#44We are truly in the stupidest phase of software engineering yet.
Things will get much more stupid in a few years when we have to maintain all this LLM-generated garbage code.
Re: Throw more AI at your problems
#45The title is deliberately provocative but if I'm reading this article right it seems to push an argument that I've made for ages in the context of my own business, and which I think - as the article itself suggests - actually represents the best practice from before the age of ChatGPT, to wit: have lots of ML models, each of which does some very specific thing well, and wire them up, wherever it makes sense to do so, piping the results from one into the input of another. The article is a fan of doing this with language models specifically - and, of course, in natural language-heavy contexts these ML models will most or all be language models - but the same basic premise applies to ML more generally and, as far as I am aware, this is how it used to be done in commercial applications before everyone started blindly trying to make ChatGPT do everything.
I recently discovered BERTopic, a Python library that bundles a five-step pipeline of now pretty old (relatively) NLP approaches in a way that is very similar to how we were already doing it, now wrapped in a nice handy one-liner. I think it's a great exemplar of the approach that will probably emerge from the hype storm on top.
(Disclaimer: I am not an AI expert and will defer to real data/stats nerds on this.)
Re: Throw more AI at your problems
#46YES (although i'm hesitant to even say anything because on some level this is tightly-guarded personal proprietary knowledge from the trenches that i hold quite dear). why aren't you spinning off like 100 prompts from one input? it works great in a LOT of situations. better than you think it does/would, no matter your estimation of its efficacy.
100 prompts doing what? Something like more selective, focused extraction of structured fields?
a combination of lots of things, with the general theme being focused prompts good at individual, specific subtasks circuits… off the top of my head:
- that try to extract the factual / knowledge content and try to update the rest of the system (e.g. if the user chats you to not send notifications after 9pm, ideally you’d like the whole system to reflect that. if they say they like the color gold, you’d like the recommendation system to know that.)
- detect the emotional valence of the user’s chat and raise an alert if they seem, say, angry
- speculative “given this new information, go over old outputs and see if any of our assumptions were wrong or apt and adjust accordingly”
- learning/feedback systems that run evals after every k runs to update and optimize the prompt
- systems where there is a large state space but any particular user has a very sparse representation (pick the top k adjectives for this user out of a list of 100, where each adjective is evaluated in its own prompt)
- llm circuits with detailed QA rules (and/or many rounds of generation and self-reflection to ensure generation/result quality)
- speculative execution in order to trade increased cost / computation for lower latency. (cf. graph of thoughts prompting)
- out of band knowledge generation, prompt generation, etc.
- alerting / backstopping / monitoring systems
- running multiple independent systems in parallel and then picking the best one or even merge the best results across all of them.
the more, smaller prompts, the easier to do eval and testing, as well as making the system more parallelizable. also, you get stronger, deeper signals that communicate a deeper domain understanding to the user s.t. they think you know what you’re doing.
but the point is every bit as much that you are embodying your own human cognition within the llm— the reason to do that in the first place is because it is virtually infinitely scalable when it makes it out of your brain and onto/into silicon. even if each marginal prompt you add only has a .1% chance of “hitting”, you can just trigger 1000 prompts and voila: one more eureka moment for your system.
sure, there’s diminishing returns in the same way that the CIA wants 100 Iraq analysts but not 10000. but unlike the CIA, you dont need to pag salaries, healthcare, managers, etc. it all scales basically linearly. and, besides, is extremely cheap as long as your prompting is even halfway decent.
Re: Throw more AI at your problems
#47Use Moore's law to achieve unreal battery life and better experiences for users... or use Moore's law to throw more piles of abstractions on abstractions where we end up with solutions like Electron or I Duck Taped An AI on it. Reading through this, I could not tell if this was a parody or real. That robot image slopped in the middle certainly didn't help.
except the computer is the one both writing AND using the abstractions, so the human cost is essentially zero. and thus is absolutely not even similar.
as a general rule, virtually any analogy that involves anthropomorphizing LLMs is at best right for the wrong reasons— a stopped clock— leads to conclusions ranging from misleading to actively harmful.
Re: Throw more AI at your problems
#48We are truly in the stupidest phase of software engineering yet.
Things will get much more stupid in a few years when we have to maintain all this LLM-generated garbage code.
if by “stupid” you mean “worse is better”. except this time it’s actually better. just because it’s apostasy according to the Church of Engineering does not mean it can be dismissed out of hand, no matter how much it hurts your sensibilities. (it used to mine as well, but then i learned to stop worrying and learned to love our new llm overlords)
Re: Throw more AI at your problems
#49We aren’t good at creating software systems from reliable and knowable components. A bit skeptical that the future of software is making a Rube Goldberg machine of black box inter-LLM communication.
“we arent good at it, so we shouldnt admit the possibility that a fundamentally different entity and architecture might be better suited”
Re: Throw more AI at your problems
#50Earlier quoted context omitted.
All the snark contained within aside, I'm reminded of that ranting blog post from the person sick of AI that made the rounds a little ways back, which had one huge, cogent point within: that the same companies that can barely manage to ship and maintain their current software are not magically going to overcome that organizational problem set by virtue of using LLMs. Once they add that in, then they're just going to…
I believe you mean this one: https://ludic.mataroa.blog/blog/i-will-fucking-piledrive-you...
Correct. Also love the dramatic reading someone commissioned.