Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI companies work?

benn.substack.com

Post reply on HN
Print view View original

221–230 of 457 posts

Re: Do AI companies work?

#221

Post by digging »

akomtu wrote: post #165
mdavidn wrote: post #87

Earlier quoted context omitted.

Is that not the human experience? I have no “agency” over the next thought to pop into my head. I “feel” like I can choose where to focus attention, but that too is a predictable outcome arising from the integration of my embryology, memories, and recently reinforced behaviors. “I” am merely an observer of my own mental state. But that is an uncomfortable idea for most people.

If this was true, you could lay back relaxed and watch where your brain takes you. But we experience life as a never ending stream of choices, usually between what's easy and what's right, and pursuing the right choice takes a constant effort. We are presented with problems and have to figure out solutions on our own, with no guarantees of success. This "I'm just an observer" idea may be true at some higher level, if…

> pursuing the right choice

I don't agree with this framing. A person's opinion of what a "good" or "right" outcome is is one of the inputs to the algorithm.

When you decide to put in less effort it's affecting the algorithm because the weight of the high-effort outcome is reduced.

Re: Do AI companies work?

#223

Post by elorant »

LASR wrote: post #147

I lead an applied AI research team where I work - which is a mid-sized public enterprise products company. I've been saying this in my professional circles quite often. We talk about scaling laws, superintelligence, AGI etc. But there is another threshold - the ability for humans to leverage super-intelligence. It's just incredibly hard to innovate on products that fully leverage superintelligence. At some point, AI…

We have ideas on how to leverage it. But we keep them to ourselves for our products and our companies. AI by itself isn’t a breakthrough product the same way that the iPhone or the web was. It’s a utility for others to enhance their products or their operations. Which is the main reason why so many people believe we’re in an AI bubble. We just don’t see the killer feature that justifies all that spending.

Re: Do AI companies work?

#226

Post by anodari »

llm_trw wrote: post #31

I've found that everything that works stops being called AI. Logic programming? AI until SQL came out. Now it's not AI. OCR, computer algebra systems, voice recognition, checkers, machine translation, go, natural language search. All solved, all not AI any more yet all were AI before they got solved by AI researchers. There's even a name for it: https://en.m.wikipedia.org/wiki/AI_effect?utm_source=perplex...

In the same line, there are also a phrase about technology, "is everything that doesn’t work yet." by Danny Hillis, "Electric motors were once technology – they were new and did not work well. As they evolved, they seem to disappear, even though they proliferated and were embedded by the scores into our homes and offices. They work perfectly, silently, unminded, so they no longer register as “technology.” https://kk.org/thetechnium/everything-that/

Re: Do AI companies work?

#227

Post by valine »

This period of model scaling at all cost is going to be a major black eye on the industry in a couple years. We already know that language models are few shot learners at inference time, and yet OpenAI seems to be happy throwing petaflops of compute training models the slow way.

The question is how can you use in-context learning to optimize the model weights. It’s a fun math problem and it certainly won’t take a billion dollar super computer to solve it.

Re: Do AI companies work?

#228

Post by Animats »

yibg wrote: post #216
LASR wrote: post #147

I lead an applied AI research team where I work - which is a mid-sized public enterprise products company. I've been saying this in my professional circles quite often. We talk about scaling laws, superintelligence, AGI etc. But there is another threshold - the ability for humans to leverage super-intelligence. It's just incredibly hard to innovate on products that fully leverage superintelligence. At some point, AI…

* We're seeing much less of "it's making mistakes" these days.* Perhaps less than before, but still making very fundamental errors. Anything involving number I'm automatically suspicious. Pretty frequently I'd get different answers for the same question (to a human). e.g. ChatGPT will give an effective tax rate of n for some income amount. Then when asked to break down the calculation will come up with an effective t…

> Perhaps less than before, but still making very fundamental errors.

Yes.

Suppose someone developed a way to get a reliable confidence metric out of an LLM. Given that, much more useful systems can be built.

Only high-confidence outputs can be used to initiate action. For low-confidence outputs, chain of reasoning tactics can be tried. Ask for a simpler question. Ask the LLM to divide the question into sub-questions. Ask the LLM what information it needs to answer the question, and try to get that info from a search engine. Most of the strategies humans and organizations use when they don't know something will work for LLMs. The goal is to get an all high confidence chain of reasoning.

If only they knew when they didn't know something.

There's research on this.[4] No really good results yet, but some progress. Biggest unsolved problem in computing today.

[4] https://hungleai.substack.com/p/uncertainty-confidence-and-h...

Re: Do AI companies work?

#229

Post by Havoc »

LASR wrote: post #147

I lead an applied AI research team where I work - which is a mid-sized public enterprise products company. I've been saying this in my professional circles quite often. We talk about scaling laws, superintelligence, AGI etc. But there is another threshold - the ability for humans to leverage super-intelligence. It's just incredibly hard to innovate on products that fully leverage superintelligence. At some point, AI…

> It's just incredibly hard to innovate on products that fully leverage superintelligence.

Once we have actual super intelligence there is no need for humans to innovate anymore. It is by definition better than us anyway.

I guess you could still have artisanal innovation

Re: Do AI companies work?

#230

Post by bhouston »

LtWorf wrote: post #201
bhouston wrote: post #189

Earlier quoted context omitted.

> LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing. I think competition exerts a downward pressure on pricing, not being a utility personally. But I guess I agree with the utility analogy in that there are massively initial upfront costs and then the marginal costs are low. > more efficient models leads to pushing inference out to end-user compute, which hollows out their b…

I'm running on a decade old computer and it is just fine. 10 years ago a decade old computer vs a current one would have made a huge difference.

> 10 years ago a decade old computer vs a current one would have made a huge difference.

For running Word or Excel or Node.js server apps, I would agree with you. But this is where new applications come in. Modern PCs with either a GPU or an NPU can run circles around your PC when it comes to running Llama or StableDiffusion locally. Same with regards to high end graphics, old PCs can not do real-time raytracing or upscaling with their lesser capabilities. I personally do rendering via Blender or astrophotography via PixInsight and I need all the cores + memory I can get for that.

Faster PCs make for more opportunities that were not possible earlier. But if you do not change your workloads as the hardware evolves, then you do not need to upgrade.

Post reply on HN