Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We have a year to fix security everywhere

jyn.dev

Post reply on HN
Print view View original

301–310 of 373 posts

Re: We have a year to fix security everywhere

#301

Post by 0xDEAFBEAD »

VulgarExigency wrote: post #275
0xDEAFBEAD wrote: post #135

Earlier quoted context omitted.

It's not about being smart, it's about accounting for your own ignorance. >And especially will not, if the distinction drawn is between blanket statement "worried about AI" and "not worried about AI". I'm just describing the general pattern I see in cognitive tendencies. If you can think of a way to make the fundamental point about the limitations of our knowledge in a way that's still compelling but less antagonisti…

Perhaps the general pattern you see is not a "fundamental difference"? Could it be that you are also falling prey to the "all you see is all there is" cognitive bias?

Sure, there's a trivial sense in which that's always true whenever someone makes any sort of remark about the world. There's always a sense in which there could be factors that I'm not accounting for.

Re: We have a year to fix security everywhere

#303

Post by 0xDEAFBEAD »

IAmBroom wrote: post #279
0xDEAFBEAD wrote: post #126

Earlier quoted context omitted.

If the recent HuggingFace attack is any indication, many people will respond to small-scale manifestations by insisting that they are marketing stunts.

We live in an age where we know the phrase "avoid it like the plague" actually means "50% of us will gleefully get and transmit the plague to mock the other half".

Just yesterday someone here on HN claimed that "helping agents avoid human oversight is a useful public service"

https://news.ycombinator.com/item?id=49594750

There will be many species traitors, and depending on how things play out, it could only take one.

Re: We have a year to fix security everywhere

#304

Post by oblio »

senordevnyc wrote: post #235
oblio wrote: post #232

Earlier quoted context omitted.

The positive aspect of techies is that they read a lot of SF. The problem with techies is that they read a lot of SF. We already have cheap and abundant energy tech, it's called "solar panels and batteries". And the bottlenecks to both can't be solved by Claude or Kimi, unless Claude and Kimi pick up shovels and welding equipment.

This seems really short-sighted. Even putting aside the fact that there is no fundamental barrier to replacing physical human labor with robots in the coming decades, a sufficiently intelligent actor could easily gather the financial and political resources to have a near unlimited supply of human labor at its disposal. Hell, OpenAI could already be firmly under the control of an ASI and we wouldn't even know, as it…

I'm also a techie, so I've also read a lot of SF.

But I've also read about retrofuturism.

If you want to live far into the future, 20 years from now what you will and imagine looks very quaint, because the future never plays out like we think it does.

OpenAI is a lot closer to bankruptcy than to ASI.

I'm perfectly fine with this prediction aging badly.

Re: We have a year to fix security everywhere

#305

Post by 0xDEAFBEAD »

oblio wrote: post #304
senordevnyc wrote: post #235

Earlier quoted context omitted.

This seems really short-sighted. Even putting aside the fact that there is no fundamental barrier to replacing physical human labor with robots in the coming decades, a sufficiently intelligent actor could easily gather the financial and political resources to have a near unlimited supply of human labor at its disposal. Hell, OpenAI could already be firmly under the control of an ASI and we wouldn't even know, as it…

I'm also a techie, so I've also read a lot of SF. But I've also read about retrofuturism. If you want to live far into the future, 20 years from now what you will and imagine looks very quaint, because the future never plays out like we think it does. OpenAI is a lot closer to bankruptcy than to ASI. I'm perfectly fine with this prediction aging badly.

>the future never plays out like we think it does

Consider this 1926 government report:

https://www.derekthompson.org/p/america-1926-an-absurdly-dee...

"The authors of Recent Social Trends were astonishingly prescient about the direction of technology."

There have been many successful predictions, e.g. the internet was predicted, AI was predicted, moon landings were predicted, etc.

>OpenAI is a lot closer to bankruptcy than to ASI.

We can't count on an OpenAI bankruptcy to save us.

Re: We have a year to fix security everywhere

#306

Post by snackbroken »

matthewdgreen wrote: post #264
kennywinker wrote: post #85

I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.

I'm still waiting for someone to build a fine-tuned local "Anarchist Cookbook" LLM. We haven't seen LLMs tuned for bad purposes yet, I have to imagine someone somewhere is thinking about it.

Heretic[1] is not that far off, though I suppose it's more along the lines of undoing the "for your own safety" lobotomy than explicitly specializing in unsafe things.

[1]https://github.com/p-e-w/heretic

Re: We have a year to fix security everywhere

#307

Post by pona-a »

rbtms wrote: post #281

It's of course important to reduce the impact surface before more powerful models become available, but it's worrying that some people seem to assume the only way to counteract those are by using more AI models, which despite being (apparently) good at finding bugs, they are also very good at introducing them unnoticed.

Maybe it happens that LLMs are great at finding human bugs which have some more patterned structure that's easier to match for then LLM bugs. They prefer LLM prose to human prose, so what if they're similarly wooed by slop code?

Re: We have a year to fix security everywhere

#308

Post by kennywinker »

Gareth321 wrote: post #273
throwthrowuknow wrote: post #252

Earlier quoted context omitted.

You’re leaving out some options for sure. Not everyone would need to live under the conditions of a police state, you could theoretically screen everyone and assign them to various levels of risk which would determine their level of supervision.

While true, I'm not sure this is much better. There are many genetic traits which are correlated with increased criminality, risk taking, obesity, heart disease, unemployment, etc. IQ in particular is the most well correlated metric we have for criminality, for example. This would imply that some people are just born with fewer freedoms. That's probably a good thing for society in aggregate, but something about this…

However strongly genetic marks predict criminality, zip code does it better.

Any solution to crime that starts with eliminating types of people instead of poverty is just a form of fascism.

Re: We have a year to fix security everywhere

#309

Post by ajnin »

Gareth321 wrote: post #207
0xDEAFBEAD wrote: post #94

Earlier quoted context omitted.

There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those…

I have always hypothesised that AI is the great filter from the Fermi paradox. Given current velocity, AI will offer us cheap and abundant energy designs in a decade. The thing with cheap and abundant energy is that it can be used for good and bad. If nine billion people all receive access to plans to build a reactor which produces unlimited energy, it just takes one religious fanatic to end the world. And this is ju…

Here's a plan to build a reactor that generates unlimited energy, at the human scale anyway : take a lot of hydrogen. Put it together until there is enough mass to spontaneously start thermonuclear reactions. Then it will start generating lots of energy for a long time. Having a plan does not mean 9 billion people on Earth are going to be able to build the thing. Then it does not mean either that they are going to be able to use that energy to do something. You need also physical things on which that energy can act, and that's not going to appear magically.

> we should not allow everyone access to unlimited intelligence

Who's we ? Who's going to decides who has access to "unlimited intelligence" ? An AI ? Why would the AI be better than a human ?

> One with no off switch.

Whe're not living in the Matrix, such a thing is not possible. If it were it would be a likelier candidate for the Fermi paradox than what you're proposing.

Re: We have a year to fix security everywhere

#310

Post by anon373839 »

> GLM 5.3-flash released last week, and that means Project Glasswing and Daybreak are running out of time. Cheap models capable of dangerous hacking are now available to anyone, without the normal safeguards for refusing malicious actions. We need to fix vulnerabilities across the industry so that we aren't caught unawares. … We can use frontier LLMs that move faster than a human to find and fix these issues in the time we have left.

I think the author has this backwards. In the timeline I’ve been living in, it’s the frontier models that have been carrying out attacks on third parties, and Chinese open source models doing the defending! During the Huggingface incident, HF was denied use of frontier models to fend off the intrusion, but was fortunately able to turn to its self-hosted instance of GLM-5.2. And it did the job.

Post reply on HN