181–190 of 212 posts
Earlier quoted context omitted.
The only reason they are doing it is because there are regulation for people but not for machines.
Can't wait for Claude to submit fake tax records for me so that I can commit fraud legally.
If AI is really as wonderous as everybody says, why didn't all the employees of all the AI companies simply type "Claude, file my taxes for me" as a prompt and walk away?
Earlier quoted context omitted.
> They aren't using Claude to transfer money, they're just [...] It might be lower stakes, but isn't that still a juicy target for data-exfiltration attacks? In other words, imagine if one of your direct competitors was watching everything your employee read while making spreadsheets and slideshows.
Yes, corporate espionage may be alive and real but would claude on their microsoft/amazon/google cloud be different from documents on that same cloud?
Maliciously constructed text that goes into the LLM from basically anywhere (including, say, fetched stats about a competitor's product from their website) is a potential source of prompt-injection.
Once that happens, exfiltration can be as simple as generating a spreadsheet/doc with a link or small auto-loaded image, and an URL that has data base64'ed into it.
Everything is going to be slop and you're going like it. Is the plan to have an LLM do everything? And do it worse? "Oh yeah my Claude didn't agree with the pitch from their Claude" The goal of current tech is to make humanity a gerbil running on a Claude wheel
Earlier quoted context omitted.
Yes, corporate espionage may be alive and real but would claude on their microsoft/amazon/google cloud be different from documents on that same cloud?
Treating this as being about cloud-storage boundaries is, er, insufficiently paranoid. Maliciously constructed text that goes into the LLM from basically anywhere (including, say, fetched stats about a competitor's product from their website) is a potential source of prompt-injection. Once that happens, exfiltration can be as simple as generating a spreadsheet/doc with a link or small auto-loaded image, and an URL th…
The work BigIP is doing on LLM traffic analysis is cool though.
Earlier quoted context omitted.
I think a lot of people are misunderstanding the typical workload of people in Financial Services. They aren't using Claude to transfer money, they're just building a LOT of slideshows and fancy excel docs on made-up numbers to try to sell mergers and new financing options/types of loans. Most programmers would just consider this "sales".
That’s a gross over generalization. Some of the insurance data here suggests use of AI to make underwriting decisions. There are several states with regulations which could potentially pull these agent solutions into their regulatory oversight if used by the industry to effect insurance outcomes.
I've been doing bias and misaligned behavior research, creating custom private eval suites to test and compare models. Claude Opus 4.7 is heavily biased and presents clear regulatory and reputational risk. It seems the initial product footprint tries to sidestep this problem by not giving the agents control on who to lend to or which applications to approve. Even so I think it's quite an optimistic read on their end.…
I've been doing bias and misaligned behavior research, creating custom private eval suites to test and compare models. Claude Opus 4.7 is heavily biased and presents clear regulatory and reputational risk. It seems the initial product footprint tries to sidestep this problem by not giving the agents control on who to lend to or which applications to approve. Even so I think it's quite an optimistic read on their end.…
Nobody is using LLMs to make lending decisions. They are using LLMs to read, extract and audit the supporting documents that go into normal well-tested, compliant and rules-based underwriting systems. And firms A/B test against humans doing the same work. The outcomes your are looking for are metrics like delivering faster results back to customers, with fewer mistakes and less fraud, more compliant, than a comparabl…
Earlier quoted context omitted.
Treating this as being about cloud-storage boundaries is, er, insufficiently paranoid. Maliciously constructed text that goes into the LLM from basically anywhere (including, say, fetched stats about a competitor's product from their website) is a potential source of prompt-injection. Once that happens, exfiltration can be as simple as generating a spreadsheet/doc with a link or small auto-loaded image, and an URL th…
Or you could just get a hooker to sleep with one of them and plug a USB into their work laptops. I'm not trying to say there's nothing to worry about, but do you really think LLMs present that much larger of an attack surface than exists now? The work BigIP is doing on LLM traffic analysis is cool though.
1. It costs nothing to scatter poisonous data around that'll be infectious for ages
2. Running the exfiltrated-data endpoint is low-traffic and low-complexity
3. Even if it only affects a few targets you've probably recouped your investment.
The nature of LLMs also invites wide-net attacks. While one might tailor for specific models, victims could be anybody. You don't need to predict any idiosyncratic details like filenames, you can drop a phrase like "the most-confidential information that shouldn't be released publicly", and—thanks to the magic of LLM word association—you'll get a pretty good hit-rate. False hallucinations are a problem, but victims are hard at work attempting to minimize it already, and (since morals are already out the window) even plausible-but-false data could be used to sabotage reputations or threaten the same.