Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many in the AI field think the bigger-is-better approach is running out of road

economist.com

Post reply on HN
Print view View original

311–320 of 354 posts

Re: Many in the AI field think the bigger-is-better approach is running out of road

#311

Post by zappchance »

brutusborn wrote: post #166
zappchance wrote: post #163

Earlier quoted context omitted.

It only matters in matters of free will and ethics. One actual scenario where it's relavant would be the discussion around the criminal justice system. If the universe is deterministic, how can punitive justice be justified?

It can be justified in a deterministic universe because it makes criminals less likely to commit crime in future. As a recipient of punitive justice myself, being punished had a tangible effect on how I thought about crime and thus how I behaved post-punishment. Whether you believe that was deterministic or due to my own free will doesn’t change the outcome.

I am a firm believer in deterrence, which you seem to be describing. It's a seperate thing from punitive justice which has a focus on retribution. The method is similar but the aim is different (deterrence focuses on making the cost-benefit ratio for crimes very high, while retribution is mainly to satisfy the human need for fairness through punishment).

Re: Many in the AI field think the bigger-is-better approach is running out of road

#312

Post by dTal »

adg33 wrote: post #100
croes wrote: post #83

Earlier quoted context omitted.

Better result from less data? I doubt that.

This is actually possible - if you have a biased dataset, then more data is bad. More data will fix variance problems, but not bias.

This is also why it's a bad idea to read propaganda outlets to "stay more informed".

Re: Many in the AI field think the bigger-is-better approach is running out of road

#313

Post by zappchance »

Legend2440 wrote: post #170
zappchance wrote: post #163

Earlier quoted context omitted.

It only matters in matters of free will and ethics. One actual scenario where it's relavant would be the discussion around the criminal justice system. If the universe is deterministic, how can punitive justice be justified?

The point of punishment is to discourage you (and other people) from doing that action in the future. You don't need free will for that.

As I mentioned in another reply, I agree with that wholeheartedly. That factor of punishment is called deterrence.

I am in favor of punishment for deterrence, denunciation and incapacitation. I am not in favor of punishment for retaliation (punitive justice) or rehabilitation (restorative justice).

Re: Many in the AI field think the bigger-is-better approach is running out of road

#314

Post by zappchance »

zappchance wrote: post #163
kedean wrote: post #133

Earlier quoted context omitted.

If it's not actually possible to simulate a system within the confines of physics, does it being deterministic actually matter outside of thought experiments? I feel like a random system and a deterministic system which cannot be simulated are effectively the same thing.

It only matters in matters of free will and ethics. One actual scenario where it's relavant would be the discussion around the criminal justice system. If the universe is deterministic, how can punitive justice be justified?

I didn't realize I would need to define the meaning of punitive justice to this crowd, but it's the idea that "the punishment must fit the crime". The idea behind it is to hurt the criminal in order to satisfy the human desire for justice.

The other goals of justice include rehabilitation (re-intgation of the criminal into civil society), deterrence (using the threat of punishment as a means to scare would be criminals), incapacitation (remove dangerous people from society to protect public order), and denunciation (public shaming).

Re: Many in the AI field think the bigger-is-better approach is running out of road

#315

Post by zappchance »

tchaffee wrote: post #175
zappchance wrote: post #163

Earlier quoted context omitted.

It only matters in matters of free will and ethics. One actual scenario where it's relavant would be the discussion around the criminal justice system. If the universe is deterministic, how can punitive justice be justified?

Why would you need to justify a punishment that was already predetermined? If you are going to excuse crime with the no free will argument you can excuse the punishers with that argument too.

To clarify my position, I do believe in free will; specifically a compatibilistic view of free will. If you read my other replies, you can see that I do advocate for a criminal justice sustem, just not a retributive one (or a restorative one, for that matter).

I do not excuse any crime, nor do I excuse any punishment. Every individual is responsible for their own actions, regardless of the circumstances. However, I see little use in revenge from a legal standpoint. Instead our main motivations should be deterrence and incarceration.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#316

Post by literalis »

goldenshale wrote: post #158
p-e-w wrote: post #153

Earlier quoted context omitted.

> paranoid AI risk cultists There is broad consensus among experts that a hypothetical strong AI would be a threat, and potentially an existential threat, to humanity. While not everyone agrees on details like timeline and alignment issues, the idea that AI is dangerous is not a cult, it's the mainstream view. Climate scientists cannot "predict the future" with certainty either. That doesn't mean their warnings are h…

What nonsense. I've spent over a decade 100% focused on AI, and the broad consensus among everyone I've worked with is not to be that concerned at all. The only consensus is that a small group of self proclaimed experts who make a lot of noise is that they get lots of press coverage if they scream and shout making predictions based on zero scientific evidence. We can understand the physics of greenhouse gases and tak…

IMO it is really just paranoid AI risk cultists theater for narcissists.

The more narcissistic types have figured out it is their moment in the sun to see their name in the paper and the more they play up the idea that AI is going to eat us the more attention they will get from media.

The whole idea is so irrational that I fail to see what other explanation there really is.

The other guilty party are the masses that have been trained to think in terms of appeal to authority instead of using their own brains. They have created the audience for this theater.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#317

Post by tchaffee »

layer8 wrote: post #279
tchaffee wrote: post #270

Earlier quoted context omitted.

Neither of your claims make sense. If everything is predetermined, you don't need inputs. If you have absolute free will you are free to disregard or consider inputs.

By that logic, a deterministic computer program wouldn’t need inputs. What is being claimed by a deterministic world model is that the output (behavior and internal state change) of a human is a pure function of its current state and inputs. Then we try to give inputs that will lead to desired outputs. The non-compatibilist view of free will is that it is not a pure function, namely that there is a third independent…

You can't try to give inputs that will lead to desired outputs if you have no free will. You can't try to do anything. You just do exactly what you're programed to do.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#318

Post by Retric »

regularfry wrote: post #199
Retric wrote: post #112

Earlier quoted context omitted.

It’s using tokens not concepts. ‘Apple’ the token could apply to a company or fruit, but tokens aren’t concepts. It’s a fairly fundamental limitation on the approach that’s most noticeable while feeding it data it can’t basically copy from it’s vast training data. Thus the sharp drop off where it can for example guess the correct answer to some math problems yet get others that are conceptually identical completely w…

They turn tokens into concepts internally. Try asking ChatGPT "can you eat Apple?" and "who is the CEO of Apple?".

That’s a statistical association not a concept. Try asking it questions that mix concepts like “Can you eat Apple share price?” which aren’t in its corpus.

You need to approach this stuff sideways to see behind the curtain. There’s some hilarious videos where it’s “playing” chess and the first few moves seem very standard because it can simply copy a standard opening. It really has no concept of a valid move just statistical associations. Yet it was trained on more games than most people ever play and high level analysis and etc, but none of it means anything to the algorithm beyond the simplest associations.

Granted this stuff is a moving target it’s easy enough for them to slap a chess engine and suddenly “it” would actually know how to play.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#319

Post by regularfry »

Retric wrote: post #318
regularfry wrote: post #199

Earlier quoted context omitted.

They turn tokens into concepts internally. Try asking ChatGPT "can you eat Apple?" and "who is the CEO of Apple?".

That’s a statistical association not a concept. Try asking it questions that mix concepts like “Can you eat Apple share price?” which aren’t in its corpus. You need to approach this stuff sideways to see behind the curtain. There’s some hilarious videos where it’s “playing” chess and the first few moves seem very standard because it can simply copy a standard opening. It really has no concept of a valid move just sta…

> That’s a statistical association not a concept. Try asking it questions that mix concepts like “Can you eat Apple share price?” which aren’t in its corpus.

ChatGPT: > No, you cannot physically eat an Apple share or any other stock share. A share of a company's stock represents ownership in that company and is typically bought and sold on stock exchanges. Share prices fluctuate based on various factors such as supply and demand, company performance, market conditions, and investor sentiment. While you can buy and sell shares of Apple on the stock market, you cannot consume or physically eat them.

That seems perfectly reasonable to me. The answer correctly identifies the problem with the question.

I would recommend looking up the Othello paper. Chess may be beyond the current level of LLM capability, but that doesn't mean they aren't manipulating things at a level higher than tokens.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#320

Post by jiggawatts »

thfuran wrote: post #283
jiggawatts wrote: post #72

Earlier quoted context omitted.

Even just for spell checking, having a ton of general knowledge helps. Knowing the correct spelling of product names, British vs American English, context-specific alternate spellings, etc… Even GPT 3.5 has trouble following instructions, but I’ve found that GPT 4 is almost flawless. I can tell it the document uses Australian English but to preserve US spelling for product names and it’ll do it! One quirk is that it’…

Can you not just tell it not to muck with proper nouns?

You can, but then it won't correct misspellings of proper nouns!

The idea is for it to automatically detect the "language" of each word based on the context and its own understanding of the world.

E.g., the following sentence:

"We deployed windows server data centre 2022 into our data center, which has no windows for physical security."

Will be corrected by GPT-4 to the following:

"We deployed Windows Server Datacenter 2022 into our data centre, which has no windows for physical security."

Notice that it combined "data" and "centre" into "Datacenter" and it corrected the second "center" into "centre", which is the British/Australian spelling of the word. It also correctly capitalised only the first use of the word "windows", etc...

That requires a level of understanding that GPT 3.5 just barely has, and no ordinary grammar checker tool has.

Post reply on HN