Live data from Hacker News

95% of generative AI pilots at companies are failing – MIT report

fortune.com

91–100 of 174 posts

Re: 95% of generative AI pilots at companies are failing – MIT report

#91
post #36

Earlier quoted context omitted.

I think not having those instant answers available is a big part of why your mind wanders in that setting.

I was going to say the same. It's probably so much healthier to make note of questions for later research than to stop right then and there and either a) fall down a Wikipedia rabbit hole or b) have an AI strapped to your face perform an info-dump.

Not everyone wants an imagination. This is good for those who don't.

Re: 95% of generative AI pilots at companies are failing – MIT report

#92
post #46

Earlier quoted context omitted.

> Because the customer wasn't the user - it was their boss and shareholders. It's kinda funny that some online shops are now bragging how great their customer support is because they DON'T use LLM bots xD

Dealing with real humans in the future will be the ultimate VIP treatment.

It already is.

Re: 95% of generative AI pilots at companies are failing – MIT report

#93

> Despite the rush to integrate powerful new models, about 5% of AI pilot programs achieve rapid revenue acceleration; the vast majority stall, delivering little to no measurable impact on P&L. This summer, I built two very sophisticated pieces of software. A financial ledger to power accrual accounting operations and a code generation framework that scaffolds a database from a defined data model to the frontend comp…

> From the outside, it would appear to any domain expert that at least three other people worked on these giving the pace at which they got completed. Did several domain experts tell you this or are you making it up? > I can't imagine how anyone, in any field of information systems, is not multiples more effective than they were five years ago. Perhaps "they are massively more complex than anything I've done"

> Did several domain experts tell you this or are you making it up?

It's an assertion among eight other engineers on the project with ~15 years of experience in the domain. They are domain experts. This part isn't up for debate.

Re: 95% of generative AI pilots at companies are failing – MIT report

#94

Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…

I use my Meta glasses heavily on vacation, and then occasionally else where. The latest Llama isn't as smart as OpenAI, so after a few wrong answers I gave up on day to day queries.

That said, the scenarios they are good at they are really good at. I was traveling in Europe and the glasses where translating engravings on castle walls, translating and summarizing historical plaques, and just generally letting me know what was going on around me.

Re: 95% of generative AI pilots at companies are failing – MIT report

#95

Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…

> I, as a human, rarely have questions to ask This is an eye-opening sentence. It's quite hard to imagine how to live one's daily life with "few questions to ask." Perhaps this is a neurodivergent thing?

Oh what a blissful environment the mind that is not full of constant questions begging to be answered and explored must be.

I'll just be over here, floating (often treading water) in a raging river of "what ifs ...", "I wonder ifs..." And, "Hmmms?"

Re: 95% of generative AI pilots at companies are failing – MIT report

#96

Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…

>Because the customer wasn't the user - it was their boss and shareholders. It was all done to make someone else think 'woah, they are following the trend!'.

I'm seeing this again and again. Customers as users seems like the last concern, if it is a concern at all. Adherence to the narrative du jour, fundraising from investors and hyping the useless product up to dump on retail are the primary concerns.

Vaporware or a useless, unlaunched product are advantageous here. Actual users might report how underwhelming or useless it is. Sky high development costs are touted as wins.

Re: 95% of generative AI pilots at companies are failing – MIT report

#97
post #32

Earlier quoted context omitted.

> But I, as a human, rarely have questions to ask. Wow. This just does not match my personal experience. I do an hour or so walk around the reservoir near my house 4-5 times a week, letting my mind wander freely -- and I find that I stop on average at least five or ten times to take notes about questions to learn the answers to later, and occasionally decide that it's worth it to break pace to start learning the answ…

But do you need AI for those answers? I sometimes do the same thing, but Google/DDG/whatever works fine for most, and a niche app works for others (IDing a bird = Merlin app, for example).

I read that as I-Ding a bird. It was a second of wondering what I-Ding a bird was until I got to "Merlin" and realized it was ID-ing a bird (face-palm emoji here).

Re: 95% of generative AI pilots at companies are failing – MIT report

#98
post #32

Earlier quoted context omitted.

> But I, as a human, rarely have questions to ask. Wow. This just does not match my personal experience. I do an hour or so walk around the reservoir near my house 4-5 times a week, letting my mind wander freely -- and I find that I stop on average at least five or ten times to take notes about questions to learn the answers to later, and occasionally decide that it's worth it to break pace to start learning the answ…

But do you need AI for those answers? I sometimes do the same thing, but Google/DDG/whatever works fine for most, and a niche app works for others (IDing a bird = Merlin app, for example).

Last year one of my berry bushes had browning leaves with some spots. Google search said infection, treatment plan, etc.

This year I snapped a pic and sent to chat gpt. Normal end of year die off, cut the brown branches away, here is a fertilizer schedule for end of year to support new growth for the next year.

ChatGPT makes gardening so much easier, and that is just one of many areas. Recipes are another, don't trust the math, but chat gpt can remix and elevate recipes so much better than Google recipe blog spam posts can.

Re: 95% of generative AI pilots at companies are failing – MIT report

#99

Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…

> There is like one or two really clever uses I've seen - disappointingly, one of them was Jira. The internal jargon dictionary tool was legitimately impressive. Will it make any more money? Probably not. Sounds like Microsoft 365 Copilot at my org. Sucks at nearly everything, but it actually makes a fantastic search engine for emails, teams convos, sharepoint docs, etc. Much better that Microsoft's own global search…

> Though I'm not sure we should be killing the earth for better search

Are we, though? What I have read so far suggests the carbon footprint of training models like gpt4 was "a couple weeks of flights from SFO to NYC" https://andymasley.substack.com/p/individual-ai-use-is-not-b...

They also seem to be coming down in power usage substantially, at least for inference. There's pretty good models that can run on laptops now, and I still very much think we're in the model T phase of this technology so I expect further efficiency refinements. It also seems like they have recently hit a "cap" on the increase in intelligence models are getting for more raw power.

The trendline right now makes me wonder if we'll be talking about "dark datacenters" in the future the same way we talked about dark fiber after the dot com bubble.

Re: 95% of generative AI pilots at companies are failing – MIT report

#100

Nobody actually wants half the useless tools companies are coming up with because most of the solutions are not really novel. They are just wrapping an LLM. It's kinda like what I realized with the meta Ray-Bans: I can have these things on my face, they can tell me the answer to virtually any question in 10 seconds or less. But I, as a human, rarely have questions to ask. When you walk in to your local grocery store…

> I, as a human, rarely have questions to ask This is an eye-opening sentence. It's quite hard to imagine how to live one's daily life with "few questions to ask." Perhaps this is a neurodivergent thing?

Don’t try and diagnose people like this please. Even if you’re qualified, and I doubt you are, it’s very insensitive.
Post reply on HN