Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Honda: 2 years of ml vs 1 month of prompting - heres what we learned

levs.fyi

Post reply on HN
Print view View original

51–60 of 115 posts

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#51

Post by ticulatedspline »

Aniket-N wrote: post #36

Once you start to recognize AI written, rewritten or even edited articles, it’s hard to stop. It’s not X it’s Y. We didn’t just do A we did B. There’s definitely a lot of hard work that has gone in here. It’s gotten hard to read because of these sentence patterns popping up everywhere.

Dunnow, reads fine to me, also seems we now have a #nothingisreal problem now where everything is AI. Given that LLMS were trained on pre-existing writing it follows that people commonly write like that.

overall I think things have gotten better. I noticed maybe 3 years before chatGPT hit the scene that I would frequent on a page that definitely didn't seem written by a native English speaker. The writing was just weird. I see less of that former style now.

Probably the biggest new trend I notice is this very prominent "Conclusion" block that seems to show up now.

Honestly I'd love to see some data on it. I suspect a lot of "that's LLM slop" isn't and others isn't noticed and lots of LLM tropes were rife within online content long before LLMs but we're now hypersensitive to certain things since they're overused by LLMs.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#53

Post by Der_Einzige »

notanastronaut wrote: post #46
prasoonds wrote: post #24

Earlier quoted context omitted.

Right? This one is also very clear ChatGPTese > That’s not a marginal improvement; it’s a different way of building classifiers. They've replaced an em-dash with a semi-colon.

One of the benefits of being immersed in model usage is being able to spot it in the wild from a mile away. People really hate when you catch them doing it and call them out for it.

And people like you will hate it even more when the normies immunize themselves from being obviously caught by such tells:

https://arxiv.org/abs/2510.15061

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#54

Post by Der_Einzige »

Aniket-N wrote: post #36

Once you start to recognize AI written, rewritten or even edited articles, it’s hard to stop. It’s not X it’s Y. We didn’t just do A we did B. There’s definitely a lot of hard work that has gone in here. It’s gotten hard to read because of these sentence patterns popping up everywhere.

You can remove this easily. We wrote a paper on how to remove this slop from LLMs.

https://arxiv.org/abs/2510.15061

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#55

Post by magicalist »

ieie3366 wrote: post #35
jwong_ wrote: post #23

Wish there was a bit more technical details in how the prompt iterations looked like. > We didn’t just replace a model. We replaced a process. That line sticks out so much now, and I can't unsee it.

[flagged]

> Also HN readers: upvote the most obvious chatgpt slop to the frontpage

Eh, this one was interesting as documentation of real work that people were doing over years. You don't get that many blog posts about this sort of effort without, usually, a bunch of self hype (because the company blogging also sells data analysis AI or whatever) that clouds any interesting part of the story. The slop in it is annoying but it's also noise thats relatively easy to filter out in this case

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#56

Post by Moto7451 »

pjc50 wrote: post #7

Crucially, this is: - text classification, not text generation - operating on existing unstructured input - existing solution was extremely limited (string matching) - comparing LLM to similar but older methods of using neural networks to match - seemingly no negative consequences to warranty customers themselves of mis-classification (the data is used to improve process, not to make decisions)

Which is good because a lot of such matching and ML use cases for products I’ve worked on at several companies fit into this. The problem I’ve seen is when decision making capabilities are inferred from/conflated with text classification and sentiment analysis.

In my current role this seems like a very interesting approach to keep up with pop culture references and internet speak that can change as quickly as it takes the small ML team I work with to train or re-train a model. The limit is not a tech limitation, it’s a person-hours and data labeling problem like this one.

Given I have some people on my team that like to explore this area I’m going to see if I can run a similar case study to this one to see if it’s actually a fit.

Edit: At the risk of being self deprecating and reductive: I’d say a lot of products I’ve worked on are profitable/meaningful versions of Silicon Valley’s Hot Dog/Not Hot Dog.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#57

Post by magicalist »

jwong_ wrote: post #23

Wish there was a bit more technical details in how the prompt iterations looked like. > We didn’t just replace a model. We replaced a process. That line sticks out so much now, and I can't unsee it.

> That line sticks out so much now, and I can't unsee it.

I thought maybe they did it on purpose at first, like a cheeky but too subtle joke about LLM usage, but when it happened twice near the end of the post I just acknowledged, yeah, they did the thing. At least it was at the end or I might have stopped reading way earlier.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#59

Post by Aurornis »

PaulHoule wrote: post #29

I'll note that they had a large annotated data set already that they were using to train and evaluate their own models. Once they decided to start testing LLMs it was straightforward for them to say "LLM 1 outperforms LLM 2" or "Prompt 3 outperforms Prompt 4". I'm afraid that people will draw the wrong conclusion from "We didn’t just replace a model. We replaced a process." and see it as an endorsement of the zero-sh…

> Thing is that to know it is good enough you still have to collect and annotate more data than most people and organizations want to do.

This has been the bottleneck in every ML (not just text/LLM) project I’ve been part of.

Not finding the right AI engineers. Not getting the MLops textbook perfect using the latest trends.

It’s the collecting enough high quality data and getting it properly annotated and verified. Then doing proper evals with humans in the loop to get it right.

People who only know these projects through headlines and podcasts really don’t like to accept this idea. Everyone wants synthetic data with LLMs doing the annotations and evals because they’ve been sold this idea that the AI will do everything for you, you just need to use it right. Then layer on top of that the idea that the LLMs can also write the code for you and it’s a mess when you have to deal with people who only gain their AI knowledge through headlines, LinkedIn posts, and podcasts.

Post reply on HN