Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What are some actual use cases of AI Agents right now?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 154 posts

Re: Ask HN: What are some actual use cases of AI Agents right now?

#121

Post by usgroup »

Some of the comments reminded me of LeCun's claim regarding the error distribution of an LLM output conditional on content length. Namely, if "e" is the probability of an error, the probability of a sequence of length "n" being error free is p = (1-e)^n. That is to say there is exponentially less chance that an LLM sequence is "within the distribution of correct answers" as token length increases.

This is a consequence of the "auto-regressive" model and its lack of in-built self-correction, and it is a limiting factor in actual applications.

LeCun's tweet:

https://twitter.com/ylecun/status/1640122342570336267

Re: Ask HN: What are some actual use cases of AI Agents right now?

#122

Post by dongecko »

TheLegace wrote: post #109
dongecko wrote: post #74

Earlier quoted context omitted.

That is an interesting observation. I have not gotten to the point of too long cycles and I can think of two reasons for that. Maybe my use case is narrow enough, so that in combination with a rather constraining and strict system message an answer is easy to find. Second, I have lately played a lot with locally running LLMs. Their answers often break the formatting required for the agent to automatically proceed. So…

How do you get the output to be formatted correctly or without any branches. Say for example I want a step-by-step instruction for an action. But the response will have 1. 2. 3. and sometimes if there are multiple pathways there will long answer with 2.a,b,c,d. This is not ideal I would rather have the most simple case(2.a.) and a short summary for other options. I have described it in the prompt but still cannot get…

I have not encountered this problem yet. When I was talking about the format of the answer I meant the following: No matter if you're using Langchain, Llamaindex, something self made, or Instructor (just to get a json back); under the hood there is somewhere the request to the LLM to reply in a structured way, like "answer in the following json format", or "just say 'a', 'b' or 'c'". ChatGPT tends to obey this rather well, most locally running LLMs don't. They answer like:

> Sure my friend, here is your requested json:

> ```

> {

> name: "Daniel",

> age: 47

> }

> ```

Unfortunately, the introductory sentence breaks directly parsing the answer, which means extra coding steps, or tweaking your prompt.

Re: Ask HN: What are some actual use cases of AI Agents right now?

#123

Post by usgroup »

PheonixPharts wrote: post #37

> I'd love to know what are some of the hidden challenges to making a useful product with agents? One thing that is still confusing to me, is that we've been building products with machine learning pretty heavily for a decade now and somehow abandoned all that we have learned about the process now that we're building "AI". The biggest thing any ML practitioner realizes when they step out of a research setting is that…

I'm not sure this argument is in any way specific to LLMs, and the space for their application is still enormous. Search results, ad targeting, recommendation systems, anomaly detection, content flagging, and so on, are all systems using machine learning with a high false positive rate.

Up until fairly recently many systems used non-LLM models for making decisions based on natural language. Their performance would have been far worse but they still did useful work. Examples would include content policy enforcement, semantic search and so on.

There are very many cases where a system will make an automated decision on a heuristic or random basis for lack of better options. ML improved those decision points and spawned new ones. LLMs improve a subset of those decision points and spawn new ones.

Re: Ask HN: What are some actual use cases of AI Agents right now?

#124

Post by PeterisP »

da4id wrote: post #111
alexawarrior3 wrote: post #2

None of these I've seen actually works in practice. Having used LLMs for software development the past year or so, even the latest GPT-4/Gemini doesn't produce anything I can drop in and have it work. I've got to go back and forth with the LLM to get anything useful and even then have to substantially modify it. I really hope there are some big advancements soon and this doesn't just collapse into another AI winter,…

Why do you want it to generate web pages from images? I'm having trouble understanding the workflow here. You see a component you like on another website and want to obtain the code from it? Or if you have a design already, why not just use a Figma to Code tool?

It's not that uncommon to have a workflow where the webpage design gets built and negotiated with stakeholders/customers as a series of photoshop images, and when they're approved, it's forwarded to developers to make a pixel-perfect implementation of that design in HTML/CSS.

Re: Ask HN: What are some actual use cases of AI Agents right now?

#125

Post by krallistic »

PheonixPharts wrote: post #37

> I'd love to know what are some of the hidden challenges to making a useful product with agents? One thing that is still confusing to me, is that we've been building products with machine learning pretty heavily for a decade now and somehow abandoned all that we have learned about the process now that we're building "AI". The biggest thing any ML practitioner realizes when they step out of a research setting is that…

> The biggest thing any ML practitioner realizes when they step out of a research setting is that for most tasks accuracy has to be very high for it be productizable. You can do handwritten digit recognition with 90% accuracy?

It's way more nuanced than this. Of course, you need a decent "accuracy" (not necessarily the metric), but in many business cases, you don't need high accuracy. But you need a solid process: you can catch errors later, you can cross references etc, you need to failsafe, you need to have post-mortem error handling, etc...

I shipped stuff (classical ML) that was nothing more than "a biased coin flip," but that still generates value ($) due to the process around it.

Re: Ask HN: What are some actual use cases of AI Agents right now?

#126

Post by PeterisP »

dongecko wrote: post #122
TheLegace wrote: post #109

Earlier quoted context omitted.

How do you get the output to be formatted correctly or without any branches. Say for example I want a step-by-step instruction for an action. But the response will have 1. 2. 3. and sometimes if there are multiple pathways there will long answer with 2.a,b,c,d. This is not ideal I would rather have the most simple case(2.a.) and a short summary for other options. I have described it in the prompt but still cannot get…

I have not encountered this problem yet. When I was talking about the format of the answer I meant the following: No matter if you're using Langchain, Llamaindex, something self made, or Instructor (just to get a json back); under the hood there is somewhere the request to the LLM to reply in a structured way, like "answer in the following json format", or "just say 'a', 'b' or 'c'". ChatGPT tends to obey this rather…

If that's what you need, it would make all sense to redo the instruction fine-tuning of the model, instead of fiddling with prompt or processing to work around the model settings that go counter to what you want.

Re: Ask HN: What are some actual use cases of AI Agents right now?

#127

Post by treprinum »

choeger wrote: post #7

I am not aware of anything that works today, but I think that there's room for shopping agents. Say you need a new USB Stick or a pair of shoes. Something between $10 and $1000 that you simply have to buy ASAP but doesn't warrant spending one or more evenings on research. A language model could sift through the descriptions and comments and try to eliminate trash and even outright fraud. But then again, it's just ano…

Half of the Internet employs some form of anti-scrapper technology so 95% time spent on building a shopping agent will be on trying to defeat those anti-scrappers.

Re: Ask HN: What are some actual use cases of AI Agents right now?

#128

Post by Sai_ »

thoughtlede wrote: post #53

Answering to your second part of the question about hidden challenges: If you are using AI agents to automate a workflow [1] execution, then the question to ask is where is non-determinism in the workflow. As in, where do humans scratch their head as opposed to rely on deterministic computations. It turns out, a lot of times, as humans, we scratch our head just once for a given kind of objectives to come with a plan.…

I can sense the truth in your reply. Can you suggest some blue oceans where AI can come handy?

Re: Ask HN: What are some actual use cases of AI Agents right now?

#129

Post by dongecko »

PeterisP wrote: post #126
dongecko wrote: post #122

Earlier quoted context omitted.

I have not encountered this problem yet. When I was talking about the format of the answer I meant the following: No matter if you're using Langchain, Llamaindex, something self made, or Instructor (just to get a json back); under the hood there is somewhere the request to the LLM to reply in a structured way, like "answer in the following json format", or "just say 'a', 'b' or 'c'". ChatGPT tends to obey this rather…

If that's what you need, it would make all sense to redo the instruction fine-tuning of the model, instead of fiddling with prompt or processing to work around the model settings that go counter to what you want.

At the very beginning of my journey I did some fine tuning with Lora on a (I believe) Falcon model, but I haven't looked at it since. My impression was that injecting knowledge via fine tuning doesn't work, but tweaking behavior does. So your answer makes much sense to me. Thanks for bringing that up! I will definitively try that out.

Re: Ask HN: What are some actual use cases of AI Agents right now?

#130

Post by Qwero »

chenxi9649 wrote: post #44
Qwero wrote: post #38

Earlier quoted context omitted.

I use already a few ai tools even without perfect accuracy. And a LLM who only needs to call to a few API calls isn't hard. Very little need perfect accuracy and for that we still have classical software.

curious to know, which tools do you use and how do you use em?

I use copilot for coding.

CharGPT for writing emails to bigger audiences, Gramma correction.

Image generator for fun.

LLM for Feature Extraction from random text like a website or PDF (llama).

Post reply on HN