Live data from Hacker News

Honda: 2 years of ml vs 1 month of prompting - heres what we learned

levs.fyi

31–40 of 115 posts

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#31
I get that SQL text searches are miserable to write, but it would have flagged it properly in the example.

The text says, "...no leaks..." The case statement says, "...AND LOWER(claim_text) NOT LIKE '%no leak%...'"

It would've properly been marked as a "0".

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#32
post #25

Earlier quoted context omitted.

People generally sleep when you start talking about fine-tuned BERT and CLIP, although they do a fairly decent job as long as you have good data and know what you're doing. But no, they want to pay $0.1 per request to recognize if a photo has a person in it by asking a multimodal LLM deployed across 8x GPUs, for some reason, instead of just spending some hours with CLIP and run it effectively even on CPU.

Are you suggesting use the clip embedding for the text as a feature to train a standard Ml model on?

I think they're suggesting doing that with BERT for text and CLIP for images. Which in my experience is indeed quite effective (and easy/fast).

There have been some developments in the image-of-text/other-than-photograph area though recently. From Meta (although they seem unsure of what exactly their AI division is called): https://arxiv.org/abs/2510.05014 and Qihoo360: https://arxiv.org/abs/2510.27350 for instance.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#33
post #6

Earlier quoted context omitted.

^ This. I'm waiting for an LLM where I can just point it to a repo, slurp it up, and let me ask questions about it.

This is exactly what Devin ( https://devin.ai ) is designed to do. Their deepwiki feature is free. I’ve personally had decent success with it, but YMMV.

Apparently it's also shit. There was a discussion about it a few days ago that contains multiple project maintainers pointing out deepwiki didn't get their repos at all https://news.ycombinator.com/item?id=45884169

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#34
post #2

I wonder if text embeddings and semantic similarity would be effective here?

> We tried multiple vectorization and classification approaches. Our data was heavily imbalanced and skewed towards negative cases. We found that TF-IDF with 1-gram features paired with XGBoost consistently emerged as the winner.

Anthropic found a similar result for retrieval: embeddings + BM25 keyword search (variant of TF-IDF) produced significantly better results.

https://www.anthropic.com/engineering/contextual-retrieval

They also found improvements from augmenting the chunks with Haiku by having it add a summary based on extra context.

That seems to benefit both the keyword search and the embeddings by acting as keyword expansion. (Though it's unclear to me if they tried actual keyword expansion and how that would fare.)

---

Anyway what stands out to me most here is what a Rube Goldberg machine it is. Embeddings, keywords, fusion, contextual augmentation, reranking... each adding marginal gains.

But then the whole thing somehow works really well together (~1% fail rate on most benchmarks. Worse for code retrieval.)

I have to wonder how this would look if it wasn't a bunch of existing solutions taped together, but actually a full integrated system.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#36
Once you start to recognize AI written, rewritten or even edited articles, it’s hard to stop.

It’s not X it’s Y. We didn’t just do A we did B.

There’s definitely a lot of hard work that has gone in here. It’s gotten hard to read because of these sentence patterns popping up everywhere.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#37
post #23

Wish there was a bit more technical details in how the prompt iterations looked like. > We didn’t just replace a model. We replaced a process. That line sticks out so much now, and I can't unsee it.

[flagged]

The two groups can be different but exist in the same community.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#38

Once you start to recognize AI written, rewritten or even edited articles, it’s hard to stop. It’s not X it’s Y. We didn’t just do A we did B. There’s definitely a lot of hard work that has gone in here. It’s gotten hard to read because of these sentence patterns popping up everywhere.

[deleted]

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#39

Once you start to recognize AI written, rewritten or even edited articles, it’s hard to stop. It’s not X it’s Y. We didn’t just do A we did B. There’s definitely a lot of hard work that has gone in here. It’s gotten hard to read because of these sentence patterns popping up everywhere.

[deleted]

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#40
post #23

Wish there was a bit more technical details in how the prompt iterations looked like. > We didn’t just replace a model. We replaced a process. That line sticks out so much now, and I can't unsee it.

Seems like a very natural fit for fine tuning - would have loved to see more on the LLM side.
Post reply on HN