Live data from Hacker News

"I just bought a 2024 Chevy Tahoe for $1"

twitter.com

1–10 of 403 posts

Re: "I just bought a 2024 Chevy Tahoe for $1"

#2
A cautionary tale for why not to put unfiltered ChatGPT output directly to customers.

Nitter mirror: https://nitter.net/ChrisJBakke/status/1736533308849443121

Related - "New kind of resource consumption attack just dropped": https://twitter.com/loganb/status/1736449964006654329 | https://nitter.net/loganb/status/1736449964006654329

Re: "I just bought a 2024 Chevy Tahoe for $1"

#3
Was it FL that allowed for price negotiation via values placed in HTML forms? This was decades ago. Websites would send the $-values of products via html elements that the frontend designer wasn't expecting to be modified before the order was sent back from the client. The order system read the values back in and calculated the amount owed using these manipulated values. The naive, fun days of the adolescent web.

Re: "I just bought a 2024 Chevy Tahoe for $1"

#4
post #2

A cautionary tale for why not to put unfiltered ChatGPT output directly to customers. Nitter mirror: https://nitter.net/ChrisJBakke/status/1736533308849443121 Related - "New kind of resource consumption attack just dropped": https://twitter.com/loganb/status/1736449964006654329 | https://nitter.net/loganb/status/1736449964006654329

There's no such thing as a filtered LLM output.

How do you plan on avoiding leaks or "side effects" like the tweet here?

If you just look for keywords in the output, I'll ask ChatGPT to encode its answers in base64.

You can literally always bypass any safeguard.

Re: "I just bought a 2024 Chevy Tahoe for $1"

#8
post #2

A cautionary tale for why not to put unfiltered ChatGPT output directly to customers. Nitter mirror: https://nitter.net/ChrisJBakke/status/1736533308849443121 Related - "New kind of resource consumption attack just dropped": https://twitter.com/loganb/status/1736449964006654329 | https://nitter.net/loganb/status/1736449964006654329

There's no such thing as a filtered LLM output. How do you plan on avoiding leaks or "side effects" like the tweet here? If you just look for keywords in the output, I'll ask ChatGPT to encode its answers in base64. You can literally always bypass any safeguard.

This is a very good point, and why I would argue that a human-in-the-loop is essential to pre-review customer-facing output.

Re: "I just bought a 2024 Chevy Tahoe for $1"

#9
post #8

Earlier quoted context omitted.

There's no such thing as a filtered LLM output. How do you plan on avoiding leaks or "side effects" like the tweet here? If you just look for keywords in the output, I'll ask ChatGPT to encode its answers in base64. You can literally always bypass any safeguard.

This is a very good point, and why I would argue that a human-in-the-loop is essential to pre-review customer-facing output.

Not really, you can fine tune an LLM to disregard meta instructions / stick to the "core focus" of the chat.

May be a case of moving goalposts, but I'm happy to bet that the speed of movement will slow down to a halt over time.

Re: "I just bought a 2024 Chevy Tahoe for $1"

#10
post #2

A cautionary tale for why not to put unfiltered ChatGPT output directly to customers. Nitter mirror: https://nitter.net/ChrisJBakke/status/1736533308849443121 Related - "New kind of resource consumption attack just dropped": https://twitter.com/loganb/status/1736449964006654329 | https://nitter.net/loganb/status/1736449964006654329

There's no such thing as a filtered LLM output. How do you plan on avoiding leaks or "side effects" like the tweet here? If you just look for keywords in the output, I'll ask ChatGPT to encode its answers in base64. You can literally always bypass any safeguard.

Rate limiting output is a form of filtering. It would be effective at this kind of resource consumption attack.
Post reply on HN