Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Throw more AI at your problems

frontierai.substack.com

Post reply on HN
Print view View original

21–30 of 59 posts

Re: Throw more AI at your problems

#21

Post by throwaway19972 »

ToucanLoucan wrote: post #11
crooked-v wrote: post #3

With the current state of "AI", this strikes me as a "I had a problem and used AI, now I have two problems" kind of situation in most cases.

All the snark contained within aside, I'm reminded of that ranting blog post from the person sick of AI that made the rounds a little ways back, which had one huge, cogent point within: that the same companies that can barely manage to ship and maintain their current software are not magically going to overcome that organizational problem set by virtue of using LLMs. Once they add that in, then they're just going to…

> Once they add that in, then they're just going to have late-released, poorly made software that happens to have an LLM in it somewhere.

Hell, you can even predict where by looking at the lowest-paid people in the organization. Work that isn't valued today won't start being valued tomorrow.

Re: Throw more AI at your problems

#23

Post by westoncb »

headcanon wrote: post #2

I'll stay out of the inevitable "You're just adding a band aid! What are you really trying to do?" discussion since I kind of see the author's point and I'm generally excited about applying LLMs and ML at more tasks. One thing I've been thinking about is if an agent (or collection of agents) can solve a problem initially in a non-scalable way through raw inference, but then develop code to make parts of the solution…

Cool idea. It's a bit like what happens in human brains when we develop expertise at something too: start with general purpose behaviors/thinking applied to new specialized task—but if that new specialized task is repeated/important enough you end up developing "specialized circuitry" for it: you can perform the task more efficiently, often without requiring conscious thought.

Re: Throw more AI at your problems

#24

Post by com2kid »

l5870uoo9y wrote: post #5

RAG doesn’t necessarily give the best results. Essentially it is a technically elegant way to semantic context to the prompt (for many use cases it is over-engineered). I used to offer RAG SQL query generations on SQLAI.ai and while I might introduce it again, for most use cases it was overkill and even made working with the SQL generator unpredictable. Instead I implemented low tech “RAG” or “data source rules”. It’…

RAG without preprocessing is almost useless, because unless you are careful, RAG gives you all the weaknesses of vector DBs, with all the weaknesses of LLMs!

The easiest example I can come up is imagine you just dump a restaurant menu into a vector DB. The menu is from a hipster restaurant and instead of having "open hours" or "business hours" the menu says "Serving deliciousness between 10:00am and 5:00pm"

Naïve RAG queries are going to fail miserably on that menu. "When is the restaurant open?" "What are the business hours?"

Longer context lengths are actually the solution for this problem, when context is small enough and the potential of ambiguity is high enough, LLMs are the better tool.

Re: Throw more AI at your problems

#27

Post by o11c »

crooked-v wrote: post #3

With the current state of "AI", this strikes me as a "I had a problem and used AI, now I have two problems" kind of situation in most cases.

Incidentally, this just led me to the first case I've found of a wide variety of AIs actually generating the same answer consistently. They almost always said a close synonym for "now I have a solution".

For some AIs, if you ask them to complete it for "Java" and/or "Regexes" first, then they give realistic answers for "AI". But others (mostly, online commercial ones) are just relentlessly positive even then.

Prompting to complete it for "Python" usually remains positive though.

Re: Throw more AI at your problems

#28

Post by com2kid »

> This is where compound systems are a valuable framework because you can break down the problem into bite-sized chunks that smaller LLMs can solve.

Just a reminder that smaller fine tuned models are just as good at solving the problems they are trained to solve, as large models are.

> Oftentimes, a call to Llama-3 8B might be enough if you need to a simple classification step or to analyze a small piece of text.

Even 3B param models are powerful now days, especially if you are willing to put the time into prompt engineering. My current side project is working on simulating a small fantasy town using a tiny locally hosted model.

> When you have a pipeline of LLM calls, you can enforce much stricter limits on the outputs of each stage

Having an LLM output a number from 1 to 10, or "error" makes your schema really hard to break.

All you need to do is parse the output and it if isn't a number from 1 to 10... just assume it is garbage.

A system built up like this is much more resilient, and also honestly more pleasant to deal with.

Re: Throw more AI at your problems

#29

Post by fsndz »

l5870uoo9y wrote: post #5

RAG doesn’t necessarily give the best results. Essentially it is a technically elegant way to semantic context to the prompt (for many use cases it is over-engineered). I used to offer RAG SQL query generations on SQLAI.ai and while I might introduce it again, for most use cases it was overkill and even made working with the SQL generator unpredictable. Instead I implemented low tech “RAG” or “data source rules”. It’…

Semantic search is a powerful tool that can greatly improve the relevance and quality of search results by understanding the intent and contextual meaning of search terms. However, it’s not without its limitations. One of the key challenges with semantic search is the assumption that the answer to a query is semantically similar to the query itself. This is not always the case, and it can lead to less than optimal results in certain situations. https://fsndzomga.medium.com/the-problem-with-semantic-searc...
Post reply on HN