Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Honda: 2 years of ml vs 1 month of prompting - heres what we learned

levs.fyi

Post reply on HN
Print view View original

111–115 of 115 posts

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#111

Post by nostrebored »

juancn wrote: post #77

It could have been done via topic analysis without an LLM. In fact there are companies such as Medallia which specialize in CX and have really strong classification solutions for specifically these use cases (plus all the generative AI stuff for closing the loop).

The topic modeling of every major vendor, mostly awful LDA implementations, is horrendous. On the order of +/-20% absolute percentage points per topic. It would make my life easier if it weren’t so shit. As is, in every customer we have to go in and do legitimate topic modeling and taxonimization.

Their AI implementations are also awful. Just sample 100 contacts with someone who actually understands the business and see their reaction.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#112

Post by prmoustache »

ticulatedspline wrote: post #51
Aniket-N wrote: post #36

Once you start to recognize AI written, rewritten or even edited articles, it’s hard to stop. It’s not X it’s Y. We didn’t just do A we did B. There’s definitely a lot of hard work that has gone in here. It’s gotten hard to read because of these sentence patterns popping up everywhere.

Dunnow, reads fine to me, also seems we now have a #nothingisreal problem now where everything is AI. Given that LLMS were trained on pre-existing writing it follows that people commonly write like that. overall I think things have gotten better. I noticed maybe 3 years before chatGPT hit the scene that I would frequent on a page that definitely didn't seem written by a native English speaker. The writing was just we…

> Dunnow, reads fine to me, also seems we now have a #nothingisreal problem now where everything is AI. Given that LLMS were trained on pre-existing writing it follows that people commonly write like that.

Also we may have already reached a point where people are exposed so much to it they start talking naturally like AI.

We've seen it before, with the advent of internet and short text messages on mobile phone and the evolution of the music genres the writing and speaking capacity of the general population has gown downhill over the last 3 decades. I was watching video archives from the 70's and 80's a few days ago. It was striking to see that bar a few illiterate ones most random people from any social class interviewed in the streets 40-50 years ago would talk in a much more intelligible, eloquent and pleasant way than the best public orators of the 2020's.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#113

Post by twoodfin »

nerdponx wrote: post #85
ieie3366 wrote: post #35

Earlier quoted context omitted.

[flagged]

If you have genuinely interesting and valuable results to report, but you ask AI to do the final writeup for you and it comes across in that generic AI slop style, is it slop? Kind of a gray area for me. It certainly feels lazy and disrespectful to me as a reader, but on the other hand if they don't spend an afternoon proofreading and revising, maybe they can spend that afternoon instead building stuff. I don't know,…

Appreciate this measured response. It’s more thoughtful than my reaction, which is to close the article and strongly consider flagging.

That reaction is visceral, but I’ll try to rationalize it:

Words & language are what separate us from the animals. They’re fundamental to what it means to be human. It’s fantastic that machines can now also use these tools, but when you rely on those machines to communicate your ideas you are effectively substituting the LLM’s weighted agglomeration of others’ ideas for a good chunk of your own.

It’s not just disrespectful to the reader, it’s disrespectful to your own mind.

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#114

Post by Ancapistani »

StefanBatory wrote: post #89
Ancapistani wrote: post #47

Earlier quoted context omitted.

I believe this is likely a consequence of how RLHF is done. I’ve not verified it, but I suspect the frontier model labs are outsourcing it to companies employing primarily non-native English speakers.

Back in the days I've heard it's why delve is so popular; as it's common in Nigerian English. I learned it from MtG and I do believe it's a very cool word and I hate that I can't use it without people raising their eyebrows.

I agree - “delve” is often a perfectly cromulent word :)

Re: Honda: 2 years of ml vs 1 month of prompting - heres what we learned

#115

Post by ghm2180 »

jacquesm wrote: post #108
ghm2180 wrote: post #96

Earlier quoted context omitted.

I would offer a stronger more pointed observation, ofen the problem in building a good classifier is having good negative examples. More generally how a classifier identify good negatives is a function of: 1. Data collection technique. 2. Data annotation(labelling). 3. Classfier can learn on your "good" negatives — quantitaively depending on the machine residuals/margin/contrastive/triplet losses — i.e. learn the dif…

I think the biggest problem with such classifiers is to actually know what is good data and what is bad data. To take a sample of the data and to recognize whether or not this dataset is a general enough representation of both true and false examples (for a binary classifier) to be able to use it to train a model. Because it isn't rare at all to have data sets that are biased 100 to 1 or more for one of the classes,…

So the bias is an issue can be handled in a variety of ways, one which I know to work is to use weights on your rarer class when training. You could also use larger margins to make sure you definitely don't mis-classify the rare class at the cost of mislableling your dominant class — presuming you are ok with it. An example is when doctors order breast biopsies, it happens a lot more than the cancer itself and based on a noisy technique of physical exam.
Post reply on HN