Earlier quoted context omitted.
I was going to say the same. It's probably so much healthier to make note of questions for later research than to stop right then and there and either a) fall down a Wikipedia rabbit hole or b) have an AI strapped to your face perform an info-dump.
Not everyone wants an imagination. This is good for those who don't.
95% of generative AI pilots at companies are failing – MIT report
131–140 of 174 posts
Re: 95% of generative AI pilots at companies are failing – MIT report
#132Warning for gratuitous self promotion: https://humansignal.com/blog/9-criteria-for-successful-ai-pr...
Re: 95% of generative AI pilots at companies are failing – MIT report
#133Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…
I think those kind of glasses may be really useful for blind people. I have seen similar glasses targeted at blind people, that at least in theory, seemed to me like a good idea. I recall the glasses also can write on the screen inside the lens, which makes me think they may be good for deaf people as well. It's just that these use-cases seem uncool, and big companies seem to have to be cool in order to keep either t…
Re: 95% of generative AI pilots at companies are failing – MIT report
#134I'm arriving at the conclusion that deployments of LLMs is most suitable in areas where the cost of false positives and, crucially, false negatives are low. If you cannot tolerate false negatives I don't see how you get around the inaccuracy of LLMs. As long as you can spot false positives and their rate is sufficiently low they are merely an annoyance. I think this is a good consideration before starting a project l…
Management types seem baffled by the idea we would want this, even if they come around the next hour and say "hey user did something can you tell me what happened".
Like guies ... it's not 100%...
Re: 95% of generative AI pilots at companies are failing – MIT report
#135Earlier quoted context omitted.
> I, as a human, rarely have questions to ask This is an eye-opening sentence. It's quite hard to imagine how to live one's daily life with "few questions to ask." Perhaps this is a neurodivergent thing?
I always ponder how many people have a refrigerator in their home their entire life, and what percentage of them don't know how it works. I've asked several gfs, and they don't have even a hint of how it works. Guy friends do a bit better but not as well as you'd think. So yes, people live their entire lives not asking obvious questions.
Re: 95% of generative AI pilots at companies are failing – MIT report
#136Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…
> I, as a human, rarely have questions to ask This is an eye-opening sentence. It's quite hard to imagine how to live one's daily life with "few questions to ask." Perhaps this is a neurodivergent thing?
I think I probably could have worded it more as "I rarely have questions worth knowing the answer to", where the cost of knowing answers is tied to the following rabbit holes and delays/forgotten tasks.
Re: 95% of generative AI pilots at companies are failing – MIT report
#137Earlier quoted context omitted.
> Did several domain experts tell you this or are you making it up? It's an assertion among eight other engineers on the project with ~15 years of experience in the domain. They are domain experts. This part isn't up for debate.
I'm not questioning the credentials of your coworkers - I didn't know they existed! Just so I'm clear then, because this adds a lot of context: Nobody else worked on the 2 softwares you mentioned, but you are on a team? Are the softwares part of a business, one that's making money?
Correct, they didn't directly work on these features. These integrate into a larger pieces of software, both of which directly generate revenue from these features.
Re: 95% of generative AI pilots at companies are failing – MIT report
#138> Despite the rush to integrate powerful new models, about 5% of AI pilot programs achieve rapid revenue acceleration; the vast majority stall, delivering little to no measurable impact on P&L. This summer, I built two very sophisticated pieces of software. A financial ledger to power accrual accounting operations and a code generation framework that scaffolds a database from a defined data model to the frontend comp…
“AI pilots” in the article refers to developing AI-based tools, not to using AI for software development. These projects have a 95% failure rate of successfully deploying the AI tool being developed into production. Regarding use of AI in software development (which is not what the article is about), the proof of the pudding isn’t in greenfield projects, it’s in longer-term software evolution and legacy code. Few dis…
> Generic tools like ChatGPT excel for individuals because of their flexibility, but they stall in enterprise use since they don’t learn from or adapt to workflows, Challapally explained.
I didn't read the actual report (and probably wont) so I was figured "AI Pilots" _could_ (and honestly, should!) include the deployment of models to assist in any and all work (not necessarily even just coding - I just used it as an example).
Re: 95% of generative AI pilots at companies are failing – MIT report
#139Earlier quoted context omitted.
But do you need AI for those answers? I sometimes do the same thing, but Google/DDG/whatever works fine for most, and a niche app works for others (IDing a bird = Merlin app, for example).
Last year one of my berry bushes had browning leaves with some spots. Google search said infection, treatment plan, etc. This year I snapped a pic and sent to chat gpt. Normal end of year die off, cut the brown branches away, here is a fertilizer schedule for end of year to support new growth for the next year. ChatGPT makes gardening so much easier, and that is just one of many areas. Recipes are another, don't trus…
I used to be able to go to the local gardening center and ask the owner who could right away give you the right answer because that was his expertise that came from years of genuine experience. Then Home Depot put him put of business. Same with the local plumbing shop I could walk into with a leaky valve stem from a sink, have a guy glance at it and reply "that's an American Standard" spin around, open a drawer and hand me the part along with new washers.
Now I have to talk to a computer that may or may not be correct. I would rather talk to a real person.
Re: 95% of generative AI pilots at companies are failing – MIT report
#140Earlier quoted context omitted.
What was the last thing you asked about? What was the answer?
The origin of the word calf. 1. Calf (young cow, young of certain other mammals) Old English: cealf (plural calfru or later calves) Proto-Germanic: kalbaz or *kalbaz/kalbazō Cognates: Old Norse kálfr, Old High German kalb, German Kalb, Dutch kalf. Proto-Indo-European root: often linked to gel- (“to swell, be rounded”), possibly referring to the rounded shape of a young animal. Some etymologists, however, leave it as…
This feels similar to a recent conversation with my friend when I was trying to recall the SoC used in the Nintendo Switch and he insisted on using his chatgpt app when I just went to the Wikipedia page for the Switch faster then he could open his app.
I don't want to sound negative, but - to me people who over rely on LLMs are lazy and low effort. I would not hire or work with them.