[dead]
Honda: 2 years of ml vs 1 month of prompting - heres what we learned
91–100 of 115 posts
Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#92dayumnnn, this is interesting to say the least
Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#93I get that SQL text searches are miserable to write, but it would have flagged it properly in the example. The text says, "...no leaks..." The case statement says, "...AND LOWER(claim_text) NOT LIKE '%no leak%...'" It would've properly been marked as a "0".
I thought the same. Having said that, the parenthesis in the example are really wrong for what they were trying to convey. I suspect that they built this sql sample for the document and made some mistakes in its generation.
Perhaps I could say, it isn't just generated--it is also hallucinated!
Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#94Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#95Once you start to recognize AI written, rewritten or even edited articles, it’s hard to stop. It’s not X it’s Y. We didn’t just do A we did B. There’s definitely a lot of hard work that has gone in here. It’s gotten hard to read because of these sentence patterns popping up everywhere.
It is some kind of new Law (that ought to be named after someone) that people who write about AI are likely using it to do that writing.
(Even ironically sometimes observed in cases when the writing is disparaging of AI and the use of AI).
If the subject matter is AI, you should instantly pay attention and look for the signs it was AI assisted or generated outright.
Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#96I'll note that they had a large annotated data set already that they were using to train and evaluate their own models. Once they decided to start testing LLMs it was straightforward for them to say "LLM 1 outperforms LLM 2" or "Prompt 3 outperforms Prompt 4". I'm afraid that people will draw the wrong conclusion from "We didn’t just replace a model. We replaced a process." and see it as an endorsement of the zero-sh…
I would offer a stronger more pointed observation, ofen the problem in building a good classifier is having good negative examples. More generally how a classifier identify good negatives is a function of:
1. Data collection technique.
2. Data annotation(labelling).
3. Classfier can learn on your "good" negatives — quantitaively depending on the machine residuals/margin/contrastive/triplet losses — i.e. learn the difference between a negative and positive for a classifier at train time and the optimization minima is higher than at test time.
4. Calibration/Reranking and other Post Processing.
My guess is that they hit a sweet spot with the first 3 techniques.
Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#97Earlier quoted context omitted.
One of the benefits of being immersed in model usage is being able to spot it in the wild from a mile away. People really hate when you catch them doing it and call them out for it.
And people like you will hate it even more when the normies immunize themselves from being obviously caught by such tells: https://arxiv.org/abs/2510.15061
Ah ha! But now the complete lack of emdash and bullet pointed lists from antislop will be the tell! Riposte!
Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#98It's worth highlighting the conditions under which this can help:
> in domains where the taxonomy drifts, the data is scarce, or the requirements shift faster than you can annotate
It's not actually clear if warranty claims really meet these criteria.
For warranty claims, the difficulty is in detecting false negatives, when companies have a strong incentive and opportunity to hide the negatives.
Companies have been trusted to do this kind of market surveillance (auto warranties, drug post-market reporting) largely based on faith that the people involved would do so in earnest. That faith is misplaced when the process is automated (not because the implementors are less diligent, but because they are too removed to tell).
Then the backlash to a few significant injuries might be a much worse regime of bureaucratic oversight, right when companies have replaced knowledge with automation (and replacement labor costs are high).
Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#99This was fun to read “ Fun fact: Translating French and Spanish claims into German first improved technical accuracy—an unexpected perk of Germany’s automotive dominance.”
It really puzzles me how this is helping and how it was done?
Does it make text more clear? How exactly? Does the German language is more descriptive? Does it somehow expands context?
So many questions in this fun fact.
Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned
#100Earlier quoted context omitted.
Amen brother. Working on a computer vision project right now, it's a wild success. This isn't my first CV project, but it's the most successful one. And that chiefly because my client pulled out their wallets and let an army of annotators create all the train data I asked for, and more.
This has been the huge problem in AI research since at least 1998 (and that was just when I was first exposed to it). With data, everything is so much easier, and much simpler machine learning methods. Supervised learning. Took a while to make that work well. And then every few years someone comes up with a way to distill data out of unsupervised examples. GPT is these days the big example of that, but there was "Ima…
I’ve got no problem w/ synthetic data, but it is still more work that most people want to do.