Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

95% of generative AI pilots at companies are failing – MIT report

fortune.com

Post reply on HN
Print view View original

121–130 of 174 posts

Re: 95% of generative AI pilots at companies are failing – MIT report

#121

Post by lumost »

WD-42 wrote: post #115
agloe_dreams wrote: post #22

Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…

I just finished implementing a chatbot in a box for a clients sass. What problem does it solve? None that I can tell, other than now the sass “has ai”. I still have access to the OpenAI dashboard. I can confirm nobody is actually using it.

Embedded AIs are pretty dumb as a product in my opinion. Why would the customer pay you instead of their existing model vendor of choice? Why do they have to learn your chatbox - when it's probably using a crappier model and lacks the context of their preferred vendor.

I really don't want to pay for 5 different AI subscriptions, I want one subscription that works with all my other services (which I already pay for).

Re: 95% of generative AI pilots at companies are failing – MIT report

#122

Post by richardblythman »

This resonates. Upskilling to AI tools is perhaps the biggest problem of our day. One idea we have to tackle this problem is to bring onboarding/learning directly into the user's work environment, track struggles and offer targeted support, and create continuous feedback loops. If anyone has faced challenges with increasing activation and retention of users on pilots (or external-facing products), would love to chat and see how we can help .

Re: 95% of generative AI pilots at companies are failing – MIT report

#123

Post by noddingham »

longtimelistnr wrote: post #3
amirkabbara wrote: post #2

Why so bad?

Because for the typical office - documents are strewn about on random network drives and are not formatted similarly. This combined with the inability to nail down 100% accuracy on even just internal doc search is just too much to overcome for non-tech industry offices. My office is mind blown if i use Gemini to extract data from a PDF and convert it to an .xlsx or .csv As a technically minded person but not a comp s…

Because you mentioned the use case specifically, I wanted to point you to the fact that Excel has been able to convert images to tables for a while now. Literally screenshot a table from your PDF and it will convert to table. Not trying to diminish any additional capabilities you're getting from Gemini, but this screenshot to table feature has been huge for my finance team.

https://support.microsoft.com/en-us/office/insert-data-from-...

Re: 95% of generative AI pilots at companies are failing – MIT report

#124

Post by soiltype »

brettgriffin wrote: post #93
soiltype wrote: post #29

Earlier quoted context omitted.

> From the outside, it would appear to any domain expert that at least three other people worked on these giving the pace at which they got completed. Did several domain experts tell you this or are you making it up? > I can't imagine how anyone, in any field of information systems, is not multiples more effective than they were five years ago. Perhaps "they are massively more complex than anything I've done"

> Did several domain experts tell you this or are you making it up? It's an assertion among eight other engineers on the project with ~15 years of experience in the domain. They are domain experts. This part isn't up for debate.

I'm not questioning the credentials of your coworkers - I didn't know they existed!

Just so I'm clear then, because this adds a lot of context: Nobody else worked on the 2 softwares you mentioned, but you are on a team? Are the softwares part of a business, one that's making money?

Re: 95% of generative AI pilots at companies are failing – MIT report

#125

Post by poszlem »

throwanem wrote: post #110
poszlem wrote: post #109

Earlier quoted context omitted.

Yes, but I’m not going to. You seem to think I owe you a performance or an explanation. Stop circling around trying to trip me up and just make your point, if you have one.

You were the one who raised the subject, but sure, if that's the way you want it. You are making a mistake which I believe you will regret, outsourcing future time binding to a machine in this way. You seem to believe you are learning something and I do not think that is true, except for a habit of intellectual laziness that I expect will prove as corrosive for you as lucrative to others. You're bragging about your c…

You’re not here to discuss, you’re here to lecture about “intellectual laziness”, which is exactly why I figured you were just trying to trip me up. I use AI the same way people used dictionaries or encyclopedias: to feed curiosity. I knock out little questions as they pop up, and if even a quarter of it sticks, that’s a win. If you want to twist that into “bragging about calf strength,” that’s just your insecurity talking.

Re: 95% of generative AI pilots at companies are failing – MIT report

#126

Post by thisisit »

michaelfm1211 wrote: post #20

> The data also reveals a misalignment in resource allocation. More than half of generative AI budgets are devoted to sales and marketing tools, yet MIT found the biggest ROI in back-office automation—eliminating business process outsourcing, cutting external agency costs, and streamlining operations. Makes sense. The people in charge of setting AI initiatives and policies are office people and managers who could be…

There is a reason why sales and marketing is first. It has to do with hallucination.

People have figured out that even if you mess up sales/support/marketing, worse case you apologize and give a gift coupon. And then there is also the verbose nature of LLMs which makes it better suited to write marketing copies etc.

On business process outsourcing like customer support lot of companies are using LLMs, so that part is unclear to me.

Other BPO processes are accounting & finance, IT, human resources etc. And while companies can take that hallucination risk for customers, they see it as a serious risk. If for example, the accounting and finance operations get messed up due to AI hallucination companies will be in real hot water. Same goes for other back office functions like HR, compliance etc. So, most likely this statement is just hogwash.

Re: 95% of generative AI pilots at companies are failing – MIT report

#127

Post by throwanem »

poszlem wrote: post #125
throwanem wrote: post #110

Earlier quoted context omitted.

You were the one who raised the subject, but sure, if that's the way you want it. You are making a mistake which I believe you will regret, outsourcing future time binding to a machine in this way. You seem to believe you are learning something and I do not think that is true, except for a habit of intellectual laziness that I expect will prove as corrosive for you as lucrative to others. You're bragging about your c…

You’re not here to discuss, you’re here to lecture about “intellectual laziness”, which is exactly why I figured you were just trying to trip me up. I use AI the same way people used dictionaries or encyclopedias: to feed curiosity. I knock out little questions as they pop up, and if even a quarter of it sticks, that’s a win. If you want to twist that into “bragging about calf strength,” that’s just your insecurity t…

"[If] even a quarter of it sticks, that's a win." Sure. Enjoy your day.

Re: 95% of generative AI pilots at companies are failing – MIT report

#128

Post by dvfjsdhgfv »

addaon wrote: post #32
agloe_dreams wrote: post #22

Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…

> But I, as a human, rarely have questions to ask. Wow. This just does not match my personal experience. I do an hour or so walk around the reservoir near my house 4-5 times a week, letting my mind wander freely -- and I find that I stop on average at least five or ten times to take notes about questions to learn the answers to later, and occasionally decide that it's worth it to break pace to start learning the answ…

When I walk around, I have many questions in my head. But I never stop to do something about it. If the question is important enough, it will stick and I'll do something about once I get back.

This is the modern curse: I know I can get an answer to nearly every question, and I can get it quickly, just taking my phone out of my pocket and dictating it, it takes zero effort. I feel it's worth to restrain oneself and just enjoy the walk. It just feels better.

Re: 95% of generative AI pilots at companies are failing – MIT report

#129

Post by rogerkirkness »

com2kid wrote: post #98
alistairSH wrote: post #67

Earlier quoted context omitted.

But do you need AI for those answers? I sometimes do the same thing, but Google/DDG/whatever works fine for most, and a niche app works for others (IDing a bird = Merlin app, for example).

Last year one of my berry bushes had browning leaves with some spots. Google search said infection, treatment plan, etc. This year I snapped a pic and sent to chat gpt. Normal end of year die off, cut the brown branches away, here is a fertilizer schedule for end of year to support new growth for the next year. ChatGPT makes gardening so much easier, and that is just one of many areas. Recipes are another, don't trus…

Is the purpose of gardening to be arms and legs for ChatGPT to grow a garden?

Re: 95% of generative AI pilots at companies are failing – MIT report

#130

Post by duxup »

agloe_dreams wrote: post #22

Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…

Yeah I'm in charge of trying some AI experiments with my company and I look around the landscape for a little inspiration ... is everything just a wrapper on chatgpt or whatever?

I can do that too but it's also not very useful and I'm just shipping data off to some AI company too. Don't know if I want to feed client data elsewhere like that.

Post reply on HN