Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many in the AI field think the bigger-is-better approach is running out of road

economist.com

Post reply on HN
Print view View original

171–180 of 354 posts

Re: Many in the AI field think the bigger-is-better approach is running out of road

#171

Post by TX81Z »

simion314 wrote: post #45
TX81Z wrote: post #32

Earlier quoted context omitted.

GPT4 is really good at code and you can generally verify hallucination easily. The other good use cases are using LLM to turn natural language prompts into API calls to real data.

>GPT4 is really good at code for popular languages, though for JS it most of the time outputs obsolete syntax and code. Today i tried to do a bit of scripting with my son in Garrys Mod, it uses Expression 2 for a Wiremod module. GPT hallucinated a lot of functions and the worst part it switched almost each time from e2 to lua. It is good at solving homeworks for students, or solving popular problems in popular langua…

I’ve been writing JavaScript for 25 years and have always felt it has utterly trash syntax. I’m not sure that’s 100% on GPT.

(And I don’t mean to be rude - I just really hate that syntax!)

Re: Many in the AI field think the bigger-is-better approach is running out of road

#172

Post by chaosjevil »

bemmu wrote: post #111
michaelteter wrote: post #63

Isn't the fundamental problem that LLM's don't actually understand anything (as greater concepts), but rather operate as complex probability machines? My 2 month active experience with ChatGPT-4 gave me the following takeaways: - when it's right, it's amazing; and when you, the operator, can recognize the niche use case where it performs really well, it can be a game-changer (although you could have programmed a tool…

I’m not convinced the language part of my brain isn’t just a complex probability machine, just with different trade-offs.

Nah. Human utterances convey purpose on a discursive level; including your comment or mine. We say stuff because we want to do something, like showing [dis]agreement or inform another speaker or change the actions of the other speaker. This is not just probabilistic - it's a way to handle the world.

In the meantime those large language models simply predict the next word based on the preceding words.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#173

Post by SanderNL »

GPT4 is better than its predecessor and quite impressively so. The (probable) param count reflects that.

It’s not just param count, because not all large models are good, but it clearly is part of the equation.

I always wondered why Altman said going bigger is a dead end. If it were true, saying it would needlessly inform your competition. What’s the use in that? If it were false it might dissuade them from going down that path.. I think I got my answer.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#174

Post by throwaway2037 »

tempodox wrote: post #159
michaelteter wrote: post #63

Isn't the fundamental problem that LLM's don't actually understand anything (as greater concepts), but rather operate as complex probability machines? My 2 month active experience with ChatGPT-4 gave me the following takeaways: - when it's right, it's amazing; and when you, the operator, can recognize the niche use case where it performs really well, it can be a game-changer (although you could have programmed a tool…

Only you, a human developer can be truly creative. An LLM can only ever reproduce what it has seen before.

    An LLM can only ever reproduce what it has seen before.
I don't believe it. How do you explain Midjourney? The art that it produces is incredible by any measurement.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#175

Post by tchaffee »

zappchance wrote: post #163
kedean wrote: post #133

Earlier quoted context omitted.

If it's not actually possible to simulate a system within the confines of physics, does it being deterministic actually matter outside of thought experiments? I feel like a random system and a deterministic system which cannot be simulated are effectively the same thing.

It only matters in matters of free will and ethics. One actual scenario where it's relavant would be the discussion around the criminal justice system. If the universe is deterministic, how can punitive justice be justified?

Why would you need to justify a punishment that was already predetermined? If you are going to excuse crime with the no free will argument you can excuse the punishers with that argument too.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#176

Post by tchaffee »

brutusborn wrote: post #166
zappchance wrote: post #163

Earlier quoted context omitted.

It only matters in matters of free will and ethics. One actual scenario where it's relavant would be the discussion around the criminal justice system. If the universe is deterministic, how can punitive justice be justified?

It can be justified in a deterministic universe because it makes criminals less likely to commit crime in future. As a recipient of punitive justice myself, being punished had a tangible effect on how I thought about crime and thus how I behaved post-punishment. Whether you believe that was deterministic or due to my own free will doesn’t change the outcome.

Punishment is not effective for most people in reducing crime. One thing that is effective is increasing the perceived risk of getting caught. People rarely commit crimes when they are sure they will be caught.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#177

Post by CamperBob2 »

tempodox wrote: post #159
michaelteter wrote: post #63

Isn't the fundamental problem that LLM's don't actually understand anything (as greater concepts), but rather operate as complex probability machines? My 2 month active experience with ChatGPT-4 gave me the following takeaways: - when it's right, it's amazing; and when you, the operator, can recognize the niche use case where it performs really well, it can be a game-changer (although you could have programmed a tool…

Only you, a human developer can be truly creative. An LLM can only ever reproduce what it has seen before.

Where, exactly, did the LLM see this epilogue to The Great Gatsby before?

https://twitter.com/tsimonite/status/1653065940463157248

Re: Many in the AI field think the bigger-is-better approach is running out of road

#178

Post by trashtester »

Legend2440 wrote: post #162
goatlover wrote: post #140

Earlier quoted context omitted.

Or the Many Worlds Interpretation is the correct understanding of quantum mechanics. The MWI people will say that indeterminism comes from the Copenhagen idea of there being a random collapse. But since measuring devices and human brains are also quantum systems, there's no reason to propose a collapse. Decoherence would be the reason we only see one result.

Maybe, but there's no data to support one interpretation over another. I believe that all quantum interpretations are incomplete and therefore wrong. Quantum mechanics is an abstraction over a deeper level of physics we can't measure yet.

Actually, there is a reason to support one interpretation over another. Specifically that it's really hard to define what an "observer" is in the Copenhagen interpretation.

The MW/Everett interpretation is what we have left if we remove the things we cannot define.

There are also reasons to believe other interpretations. For instance, if you're religious, that could pull you towards the Bohr interpretation, since it may make it easier to assume that observation could be linked to an immortal soul.

In any case, it's not natural to assume that below QM there exists a reality that is more similar to our instinctual world model than QM is. If anything, anything below it is likely to be even more abstract and hard to comprehend.

Or it could be that the principles of QM applies all the way down, just as we've seen for the pieces of the SM that we solved after QM was first introduced (strong force, electroweak force).

Re: Many in the AI field think the bigger-is-better approach is running out of road

#179

Post by tchaffee »

Legend2440 wrote: post #170
zappchance wrote: post #163

Earlier quoted context omitted.

It only matters in matters of free will and ethics. One actual scenario where it's relavant would be the discussion around the criminal justice system. If the universe is deterministic, how can punitive justice be justified?

The point of punishment is to discourage you (and other people) from doing that action in the future. You don't need free will for that.

If you don't have free will then how would something discourage you from doing something that is already determined you will do? That doesn't make any sense.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#180

Post by bemmu »

teaearlgraycold wrote: post #154
bemmu wrote: post #111

Earlier quoted context omitted.

I’m not convinced the language part of my brain isn’t just a complex probability machine, just with different trade-offs.

Yes. But modeling language doesn’t mean you’ve made a good model of anything else. Now, I think LLMs have incidentally modeled a ton of small simple systems very well. But those systems, and the ones LLMs haven’t mastered, would be better off modeled with models built for those systems specifically.

Yup, while an LLM might know a cat in language contexts, I know what petting a cat feels like, both physically and emotionally. Can run little physics or visual simulations in my head etc., and have needs and drives which are obviously all still missing.
Post reply on HN