The Stupid Programmer Manifesto • Hacker News Board

Re: The Stupid Programmer Manifesto

#231

Post by bitblender » Fri, Jun 16, 2023, 4:09 PM UTC

hsn915 wrote: post #225

bitblender wrote: post #195

Earlier quoted context omitted.

In this case SignupRequest is your contract representation, Account is your storage representation, and the "backend code path" is the transformer/napping layer. >I don't have two representations of the same object. Rather I have two different object Exactly! Your api contract and your storage are ALWAYS two different objects, because they serve two different concerns. Sometimes by coincidence they can share the same…

> Exactly! Your api contract and your storage are ALWAYS two different objects Not really. I have request objects for everything. "Search" is a request object. List pagination is a request object. Every function exposed through the RPC API takes a request object and returns a response object. The response object is often just a collection of objects straight from "the database". A response to a paginated list request…

Yes really. The fact that you can "often just return an object straight from the database" and have it fulfill the functional requirements of your client is just a coincidence, or more likely it's an invarient that you have decided to enforce. What people have discovered (usually very painfully) is that unavoidable breaking changes to either your client representation or storage representation are bound to happen, and when they do if you haven't separated these concerns this will have a ripple effect through the entire application. This may be fine. If your applications are tiny or downtime is okay, then you likely won't care about this. But to casually dismiss this advice as simply always being overcomplicated and overengineered is a grave oversimplification that you may regret someday. Many of the topics in this post fall into this category - people do it for a good reason, and you might not need it, but everything is a tradeoff and "I'm just going to do the stupid simple thing" is not the silver bullet.

Re: The Stupid Programmer Manifesto

#232

Post by hbrn » Fri, Jun 16, 2023, 4:39 PM UTC

karmakaze wrote: post #220

hbrn wrote: post #201

Earlier quoted context omitted.

Unfortunately it's not a popular belief anymore, but [asynchronous] central event bus is a terrible idea on it's own, and it goes against what [micro]services are about. If you want microservices to work you shouldn't have anything central in them, and you should avoid async as much as possible.

That's not at all what I learned from working with microservices. The worst thing you can do is make sync calls between services, then you end up with high aggregate failure rates, and/or large request latency from retries as well as low uptime due to dependency couplings. One-way async dataflows work much better. That way each microservice is up with all the (potentially delayed) data it needs to respond to requests…

If you have services with lots of dependencies on other services, then you probably ended up the worst of two worlds - distributed monolith.

Typically one-way or two-way is not a choice you can make as an engineer, most processes in your business require two-way, because they are initiated by a user and user wants a definitive response.

The "potentially delayed" approach is very tempting for engineers, but it should be exception rather than the rule.

It

- dilutes responsibilities between services (who is responsible for delays? is service A producing too many messages or is service B too slow to process them?)

- makes your SLA vague (message was processed 3 days later, do we treat it as downtime or not?)

- requires more infrastructure & processes (every service has a queue, dead-letter queue, and a process to deal with dead letters)

- requires a ton monitoring overhead (what delay is acceptable? how do we even measure delay? what if you have different SLA for different messages? we'll have a monitor per message type?)

- introduces a lot of unnecessary complexity and rules (how do you deal with TOCTOU, e.g. admin deactivates a user, but by the time the message gets processed he's no longer an admin)

- ruins user experience (we received your payment information, but we won't immediately tell you that it's wrong).

Despite it's downsides, potentially delayed approach can be a fine tradeoff when it saves you 7-8 digits per year. Most companies never reach this phase.

Re: The Stupid Programmer Manifesto

#233

Post by chrisldgk » Fri, Jun 16, 2023, 4:40 PM UTC

bluefirebrand wrote: post #123

chrisldgk wrote: post #7

It’s important to keep in mind that pretty much all the techniques you’re too „stupid“ for all aren’t necessities for small projects, but rather being able to manage complexity (or workload) at scale. For an MVP or a small product, keeping with the simpler techniques is absolutely fine, and adhering to those standards in many cases can even be considered overengineering. Though there’s one thing I take issue with. „I…

> It’s important to keep in mind that pretty much all the techniques you’re too „stupid“ for all aren’t necessities for small projects, but rather being able to manage complexity (or workload) at scale Most of the techniques people use to manage complexity at scale are too stupid to actually do that. Their only benefit is that following them prevents people from doing something even worse. It also prevents people fro…

Boilerplate and abstraction for the sake of it may be bad, sure, but there’s something to be said about having Standards anyone can adhere to. If I use a certain structure of code that requires more boilerplate but also means that anyone who has worked in that framework before knows exactly where to find and how to edit things, that’s a net gain, even more so in bigger teams. Think about tradeoffs, the most lean and elegant solution may not always be the best.

Re: The Stupid Programmer Manifesto

#234

Post by quickthrower2 » Fri, Jun 16, 2023, 9:18 PM UTC

bbkane wrote: post #203

quickthrower2 wrote: post #89

Earlier quoted context omitted.

For a side project I used some PHP to get a job done. Why? because you can edit the file on the server and keep trying it out till it works. On 90s style hosting that is cheap and honest. The iteration speed is amazing. CI/CD took <100ms.

When I wrote PHP on the server I had real problems with the "CI" part- I would refresh my page an manually test each change. Do you have Continuous Integration tests (i.e. automated) for this code or are you also doing the manual refresh cycle? No shame either way, I think the juice isn't worth the squeeze for automatic testing short-lived code myself

Sorry that was my silly subtle joke. The CI I am referring to here is refreshing the page! And I agree the juice isn't worth the squeeze.

(I love CI systems at work and wouldn't live without them, but that wait time :-(, so everyone tries to get the OODA loop on their local machine )

One idea I had would to build an Elm-like backend language BUT with PHP's "edit the file on the server and it runs" nature. Combine that with some source control, forking and more of a VSCode editor on the server, and you would have a nice DX for small projects.

All those features exist but they live in different languages/stacks, you can't have them all at once (I hope this is where someone replies "you say that... but have you tried X"!)

Re: The Stupid Programmer Manifesto

#235

Post by jagged-chisel » Fri, Jun 16, 2023, 9:26 PM UTC

ResearchCode wrote: post #206

jagged-chisel wrote: post #48

Earlier quoted context omitted.

OT ramble: I met a -0.5x dev once. .NET dude, lots of experience. Standup, every day, was something gone awry with dependency injection. Every. Single. Day.

> standup The one of you who came up with that idea was the true -0.5x engineer.

“Came up with?” None of us invented Agile/Scrum/wtfe malfunction requires daily standup meetings. Management insists, management observes, management signs the paychecks, so … here’s your DSM.

Re: The Stupid Programmer Manifesto

#236

Post by karmakaze » Sat, Jun 17, 2023, 5:03 PM UTC

hbrn wrote: post #232

karmakaze wrote: post #220

Earlier quoted context omitted.

That's not at all what I learned from working with microservices. The worst thing you can do is make sync calls between services, then you end up with high aggregate failure rates, and/or large request latency from retries as well as low uptime due to dependency couplings. One-way async dataflows work much better. That way each microservice is up with all the (potentially delayed) data it needs to respond to requests…

If you have services with lots of dependencies on other services, then you probably ended up the worst of two worlds - distributed monolith. Typically one-way or two-way is not a choice you can make as an engineer, most processes in your business require two-way, because they are initiated by a user and user wants a definitive response. The "potentially delayed" approach is very tempting for engineers, but it should…

A lot of this depends on implementation and infrastructure which is of course an additional detail. In an example I'm recalling it was for a communications app that had services for content, users, groups, sending, and receiving. Sending a message would save content with an id, include user/group recipient ids and write to the send service with them. Each service if it accepts the request completes it unless the service is actually down. The user/group service seems like it could be a sync service, but actually the client caches a list of users/contacts or can search for them.

By abstracting content the only thing that needed to change for new types of content was the content service and clients. Abstracting recipients which can be users/groups, meant that the only service that needed to care about this detail was the one that replicates sent to inboxes in the receiving service. Because of the use of content ids and user/group ids, this is all small idempotent/immutable metadata. The system was complex (yet became manageable over time) and onboarding onto each service was immediate.

I think few have seen well-bounded microservices' contexts leading to the idea that it's a bad distributed monolith. Also worth remembering that advantages of microservices 'done right' is large scaling of developers and isolate failures.

Re: The Stupid Programmer Manifesto

#237

Post by ang_cire » Sat, Jun 17, 2023, 6:31 PM UTC

nerdchum wrote: post #191

ang_cire wrote: post #149

Earlier quoted context omitted.

Well first off, that's not correct anyways; type-conformancy is a very valid and important part of data sanitization. E.g. does the SSN consist of 3 valid integers split on dashes? If not, it ain't a proper SSN. Catching that typeError is much safer than trying to roll regex or character allow/blocklists. But also, the manifesto never mentions data structures. It says > I’m not smart enough to figure out how to trans…

You shouldnt have to "transform" the data though. You should validate the data and sanitize the data...but if youre transforming data youre headed into a state management nightmare. State management is the number one enemy and other than sanitizing and validating both the data and structure should be considered mostly immutable.

You absolutely may need and want to transform data.

You don't want to store code unmodified in a database; that's how you end up with SSTI or stored XSS. You encode special characters in a simple, reversible way (as one example).

Similarly, you don't store passwords untransformed in a db, you hash them. That's a transform.

There are tons of examples like this.

Re: The Stupid Programmer Manifesto

#238

Post by ang_cire » Sat, Jun 17, 2023, 6:37 PM UTC

nerdchum wrote: post #192

ang_cire wrote: post #155

Earlier quoted context omitted.

And in the manifesto the author says > I’m not smart enough to figure out how to transform data between different layers of the system, so I just don’t. I use the same represetnation in the UI layer and the storage layer. That has nothing to do with the data structure, is about how the data is being represented in the backend vs the frontend. It's explicitly about changing the data itself. Stuff like url-encoding and…

Yeah, other than sanitizing and validating, you shouldn't be transforming data or you're headed into a state management bug nightmare.

So you think that if e.g. Stack exchange is storing code examples in their database, they shouldn't transform the special characters into e.g. url-encoding or some other escaped format?

How are you going to validate it doesn't do something harmful?

And also, you keep mentioning sanitization as though it's not inherently a transform. Stripping out whitespace? That's a transform. Just because it's a one-directional transform doesn't make it not one.

Re: The Stupid Programmer Manifesto

#239

Post by Supermancho » Tue, Jun 20, 2023, 3:12 PM UTC

collyw wrote: post #230

yurishimo wrote: post #188

Earlier quoted context omitted.

What do you mean? You can inspect POST request payloads in browser devtools just as easily as GET requests.

It's far faster when you can look at the URL and see the parameters. I have worked on systems with both and I didn't like everything as a POST. If you want to get a page with list of objects with a filter it's a lot easier to copy paste a URL than it is to fart about in developer tools and try top recreate the POST request.

It's also easier to reproduce, for sure. This goes for QA as well as customers, that may not be casually accessing the request (or have access to the logs) for reproducing problematic requests.