95% of generative AI pilots at companies are failing – MIT report
1–10 of 174 posts
Re: 95% of generative AI pilots at companies are failing – MIT report
#2Why so bad?
Re: 95% of generative AI pilots at companies are failing – MIT report
#3Why so bad?
Because for the typical office - documents are strewn about on random network drives and are not formatted similarly. This combined with the inability to nail down 100% accuracy on even just internal doc search is just too much to overcome for non-tech industry offices. My office is mind blown if i use Gemini to extract data from a PDF and convert it to an .xlsx or .csv
As a technically minded person but not a comp sci guy, refining document search is like staring into a void and every option uses different (confusing) terminology. This makes it extra difficult for me to both do my regular job AND learn the multiple names/ways to do the exact same thing between platforms.
The only solution that has any reliability for me so far are Gemini instances where i upload only the files i wish to search and just keep it locked to a few questions per instance before it starts to hallucinate.
My attempt at RAG search implementation was a disaster that left me more confused than anything.
Re: 95% of generative AI pilots at companies are failing – MIT report
#4Why so bad?
Turns out that garbage text has very little intrinsic value
Re: 95% of generative AI pilots at companies are failing – MIT report
#5[dead]
Re: 95% of generative AI pilots at companies are failing – MIT report
#6What's the failure rates if technology pilots in general for comparison?
For example, I heard that SAP has an 80-90% deployment failure rate back in the day, but don't have a citable source for it.
Re: 95% of generative AI pilots at companies are failing – MIT report
#7Why so bad?
I think one reason for this is that LLMs are sort of maximally if accidentally designed to fuck up our brains. Despite all the advancements in the last five years I see them as still, fundamentally, text transformation machines which have only very limited sort of intelligence. Yet because nothing in history has been able to generate language except humans, most of us are not prepared to make rational judgements about their capabilities and those of us that may be also often fail to do so.
The fact that we live in an era where tech people have been so investor pilled that overstating the capabilities of technology is basically second nature does not help.
Re: 95% of generative AI pilots at companies are failing – MIT report
#8Re: 95% of generative AI pilots at companies are failing – MIT report
#9Why so bad?
It's in the name: generative AIs.
There are very few use cases at companies where you need to generate something. You want to work with the company's often very private disparate data (with access controls etc.) You wouldn't even have enough data to train a custom LLM, much less use a generic one.
Re: 95% of generative AI pilots at companies are failing – MIT report
#10Am I the only one who looked at this shortened headline and wondered why anyone is allowing AIs to fly airplanes?