> but they don't realize how hard it is to build these systems correctly.
In this case, it sounds like the SSNs were included in their entirety in the HTML. My first response is that its a stupid and obvious mistake, but I think it might be too suspiciously easy to only blame the developers here.
I think we have a larger problem - which is that there's a hidden cost to adding extra layers of magic to software. And on the web, we seem to just not be able to help ourselves. The cost is that developers often skip actually understanding how the new layers work. And the abstractions are leaky with respect to performance and security, and sometimes functionality.
Its easy to imagine how this bug slipped through. They had a database query which fetched the data for rendering. Then they used some "magic" framework which does server side rendering & hydration. So the server sent the JSON it used to render to the client to dehydrate the page, and that JSON happened to include the raw database rows (with SSNs). The system is magic enough so you don't have to understand how that process works; but not magic enough to protect you from the consequences.
Junior devs use the magic anyway and get stuck, or make mistakes like this. Senior devs feel like we have to learn everything and get overwhelmed.
Other examples of this:
- Recently I wanted to use some rust code (compiled to wasm via wasm-pack) in a svelte project with snowpack or rollup. I know how to include wasm in a webpage, but the bundlers needed special plugins to handle this. And the plugins for wasm are halfbaked, poorly maintained and janky.
- I worked with a team a few years ago who was using some graphql wrapper around contentful. (Before contentful had an official graphql endpoint). The wrapper was very good, but we needed to run some queries that weren't supported by the wrapper. This was close to impossible. Nobody on the team was strong enough to read the graphql code to figure out how to solve our problem. I did it eventually - via some custom endpoints. But I shouldn't have. After I left the team had no idea how to maintain or modify the code I wrote, and they were entirely stuck.
- The "web obesity crisis" comes from projects pulling giant amounts of javascript into their webpages. Our tooling makes this easy (npm install) and safe (incompatible versions of the same package are included separately). So its easy to end up with libraries like web3, which include about a dozen different versions of bn.js resulting in 2.3mb of uncompressed JS which takes nearly a second to parse on a modern computer. - [1] https://github.com/ChainSafe/web3.js/issues/1178
I don't know what the answer here is, but I know when I was writing qbasic as a kid it wasn't like this. Maybe we need to stop going "up the stack", and instead go sideways - throwing things out as we add more. I worry this whole problem will get much worse before it gets any better.