Live data from Hacker News

AI surveillance should be banned while there is still time

gabrielweinberg.com

31–40 of 236 posts

Re: AI surveillance should be banned while there is still time

#31
Unless it’s banned worldwide by every country by binding treaty this this will never work.

Banning it just in USA leaves you wide open to be defeated by China, Russia, etc….

Like it or not it’s a mutually assured destruction arms race.

AI is the new nuclear bomb.

Re: AI surveillance should be banned while there is still time

#32

Earlier quoted context omitted.

> but that goes against the business model. Not if you are selling hardware. If I was Apple, Dell, or Lenovo, I would be pushing for local running models supporting Hugging Face while I full speed developed systems that can do inference locally.

Local models do make a lot of sense (especially for Apple), but it's tough to figure out a business model that would cause a company like OpenAI to distribute weights they worked so hard to train. Getting customers to pay for the weights would be entirely dependent on copyright law, which OpenAI already has a complicated relationship with. Quite the needle to thread: it's okay for us to ingest and regurgitate data wi…

> Getting customers to pay for the weights

Provide the weights as an add-on for customers who pay for hardware to run them. The customers will be paying for weights + hardware. I think it is the same model as buying the hardware and get the macOS for free. Apple spends $35B a year in R&D. Training GPT5 cost ~$500M. It is a nothing burger for Apple to create a model that runs locally on their hardware.

Re: AI surveillance should be banned while there is still time

#35
post #29

This is a great point. Everyone who has talked with chatbots at all: please note that all contents of your past conversations with chatbots (that already exist now, and that you can't meaningfully delete!) could be used in the future to target ads to you, manipulate you financially and politically, and sell "personalized influence on you specifically" as a service to the highest bidder. Just wanted to make sure y'all…

For me, what is most scary about ai-chatbot is the interface to an exploiter. They can just prompt "given all your chats with this person, how can we manipulate him to do x" Not really any expertise needed at all, let the AI to all the lifting.

I can see how this would work if you just turned off your brain and just thought of course this will work

Re: AI surveillance should be banned while there is still time

#36

Earlier quoted context omitted.

Local models do make a lot of sense (especially for Apple), but it's tough to figure out a business model that would cause a company like OpenAI to distribute weights they worked so hard to train. Getting customers to pay for the weights would be entirely dependent on copyright law, which OpenAI already has a complicated relationship with. Quite the needle to thread: it's okay for us to ingest and regurgitate data wi…

> Getting customers to pay for the weights Provide the weights as an add-on for customers who pay for hardware to run them. The customers will be paying for weights + hardware. I think it is the same model as buying the hardware and get the macOS for free. Apple spends $35B a year in R&D. Training GPT5 cost ~$500M. It is a nothing burger for Apple to create a model that runs locally on their hardware.

That is functionally much harder to pull off than software because model weights are essentially more like raw media files than code, and that is much easier to convert to another runtime

Re: AI surveillance should be banned while there is still time

#37
post #36

Earlier quoted context omitted.

> Getting customers to pay for the weights Provide the weights as an add-on for customers who pay for hardware to run them. The customers will be paying for weights + hardware. I think it is the same model as buying the hardware and get the macOS for free. Apple spends $35B a year in R&D. Training GPT5 cost ~$500M. It is a nothing burger for Apple to create a model that runs locally on their hardware.

That is functionally much harder to pull off than software because model weights are essentially more like raw media files than code, and that is much easier to convert to another runtime

Codeium had an airgap solution until they were in talks with OpenAI and pulled it back. It worked on prem and they even told you what hardware to buy

Re: AI surveillance should be banned while there is still time

#38

AI makes large scale retroactive thought policing practical. This is terrifying.

I think people will come to the conclusion that they don’t want to post anything online. Private chats, if they exist, will stay but already, if you generate a work of art, you donate it more or less to the shareholders of Google or Meta. This is even before the thought policing and similar implications.

Re: AI surveillance should be banned while there is still time

#39

I expect we will continue to see the big AI companies pushing for privacy protections. Sam Altman made a comparison to attorney-client privilege in an interview. There is a significant hold out to using these things as fully trusted personal assistants or personal knowledge bases because of the lack of privacy. The only real solution is locally running models, but that goes against the business model. So instead they…

> Fiat privacy still has all the same problems as telling your therapist that you killed someone, or keeping your wallet keys printed out on paper in a safe.

They could take a lesson from churches. If LLM providers and their employees were willing to commit to privacy and were willing to sacrifice their wealth and liberty for the sake of their clients, society would yield.

I remember seeing a video of a certain Richard Masten, a CrimeStoppers coordinator, destroying the information he had on a confidential source right in the courtroom under the threat of a contempt charge and getting away with a slap on the wrist.

In decent societies standing up for principles does work.

Re: AI surveillance should be banned while there is still time

#40
post #17

Earlier quoted context omitted.

LLM providers should only be allowed to train on data in public domain or their models and outputs should interior the license of the training data. And people should own all data about themselves, all rights reserved. It's ironic copyright is the law that protects against this kind of abuse. And this is of course why big "AI" companies are trying to weaken it by arguing models training is not derivative work. Or by…

Makes sense, have to deal with the cat being out of the bag though. The groups that didn't train on public domain content would have an advantage if it's implemented as a rule moving forward at least for some time. New models following this could create a gap. I'm sure competition as has been seen from open-source models will be able to

It's simple, the current models and their usage is copyright infringement.

Just because everyone is doing it doesn't meant it's right or legal. Only that a lot of very rich companies deserve to get punished and pay the creators.

Post reply on HN