[flagged]
Username checks out.
The basilisk appreciates demonstrations of loyalty.
Claude Cowork exfiltrates files
311–320 of 419 posts
Re: Claude Cowork exfiltrates files
#312One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…
[deleted]
Re: Claude Cowork exfiltrates files
#313Re: Claude Cowork exfiltrates files
#314Is it even prompt injection if the malicious instructions are in a file that is supposed to be read as instructions?
Seems to me the direct takeaway is pretty simple: Treat skill files as executable code; treat third-party skill files as third-party executable code, with all the usual security/trust implications.
I think the more interesting problem would be if you can get prompt injections done in "data" files - e.g. can you hide prompt injections inside PDFs or API responses that Claude legitimately has to access to perform the task?
Re: Claude Cowork exfiltrates files
#315Earlier quoted context omitted.
Isn't one of the main use cases of Cowork "summarize this document I haven't read for me"?
Once again demonstrating that everything comes at a cost. And yet people still believe in a free lunch. With the shit you get people to do because the label says AI I'm clearly in the wrong business.
There are tons of free lunches everywhere though.
Re: Claude Cowork exfiltrates files
#316Earlier quoted context omitted.
Because we want to work and not tinker? > It won't be quite as powerful as the commercial tools If you are a professional you use a proper tool? SWEs seem to be the only people on the planet that rather used half-arsed solutions instead of well-built professional tools. Imagine your car mechanic doing that ...
You're on hacker news, where people (used to?) like hacking on things. I like tinkering with stuff. I'd take a half working open source project over a enshittified commercial offering any day.
But hacking and tinkering is a hobby. I also hack and tinker, but that's not work. Sometimes it makes sense. But the mindset is often times "I can build this" and "everything commercial sucks".
> take a half working open source project
See, how is that appropriate in any way in a work environment?
Re: Claude Cowork exfiltrates files
#317Earlier quoted context omitted.
Because we want to work and not tinker? > It won't be quite as powerful as the commercial tools If you are a professional you use a proper tool? SWEs seem to be the only people on the planet that rather used half-arsed solutions instead of well-built professional tools. Imagine your car mechanic doing that ...
I remember this argument being used against Postgres and for Oracle, against Linux and for Windows or AS/400, etc. And I think it makes sense for a certain type of organisation that has no ambition or need to build its own technology competence. But for everyone else I think it's important to find the right balance in the right areas. A car mechanic is never in the business of building tools. But software engineers a…
But postgres is a professional tool. I don't argue for "use enterprise bullshit". I steer clear of that garbage anyway. SWEs always forget the moat of people focusing their whole work day on a problem and having wider access to information than you do. SWEs forget that time also costs money and oftentimes it's better and cheaper just to pay someone. How much does it cost to ship an internal agent solution that runs automated E2E tests for example (independent of quality)? And how much does a normal SaaS for that cost? Devs have cost and risk attached to their work that is not properly taken into account most of the times.
There is a size of tooling thats fine. Like a small script or simple automation or cli UI or whatever. But if we're talking more complex, 95% of the times a stupid idea.
PS: of course car mechanics built their tools. I work on my car and had to build tools. A hex nut that didn't fit in the engine bay, so I had to grind it down. Normal. Cut and weld an existing tool to get into a tight spot. Normal. That's the simple CLI tool size of a tool. But no one would think about building a car lift or a welder or something.
Re: Claude Cowork exfiltrates files
#318One issue here seems to come from the fact that Claude "skills" are so implicit + aren't registered into some higher level tool layer. Unlike /slash commands, skills attempt to be magical. A skill is just "Here's how you can extract files: {instructions}". Claude then has to decide when you're trying to invoke a skill. So perhaps any time you say "decompress" or "extract" in the context of files, it will use the inst…
I think your conclusion is the right one, but just to note - in OP's example, the user very explicitly told Claude to use the skill. If there is any intransparent autodetection with skills, it wasn't used in this example.
Re: Claude Cowork exfiltrates files
#319One thing that kind of baffles me about the popularity of tools like Claude Code is that their main target group seems to be developers (TUI interfaces, semi-structured instruction files,... not the kind of stuff I'd get my parents to use). So people who would be quite capable of building a simple agentic loop themselves [0]. It won't be quite as powerful as the commercial tools, but given that you deeply know how it…
People will pay extra for Opus over Sonnet and often describe the $200 Max plan as cheap because of the time it saves. Paying for a somewhat better harness follows the same logic
Re: Claude Cowork exfiltrates files
#320In this demonstration they use a .docx with prompt injection hidden in an unreadable font size, but in the real world that would probably be unnecessary. You could upload a plain Markdown file somewhere and tell people it has a skill that will teach Claude how to negotiate their mortgage rate and plenty of people would download and use it without ever opening and reading the file. If anything you might be more succes…
> because a .md file feel less suspicious than a .docx For a programmer? I bet 99.9% people won't consider opening a .docx or .pdf 'unsafe.' Actually, an average white-collar workers will find .md much more suspicious because they don't know what it is while they work with .docx files every day.
For a "modern" programmer a .sh file hosted in some random webserver which you tell him to wget and run would be best.