Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distributed systems programming has stalled

shadaj.me

Post reply on HN
Print view View original

221–230 of 232 posts

Re: Distributed systems programming has stalled

#221

Post by EtCepeyd »

linkregister wrote: post #215
hinkley wrote: post #42

Earlier quoted context omitted.

I think SaaS and multicore hardware are evolving together because a queue of unrelated, partially ordered tasks running in parallel is a hell of a lot easier to think about than trying to leverage 6-128 cores to keep from ending up with a single user process that’s wasting 84-99% of available resources. Most people are not equipped to contend with Amdahl’s Law. Carving 5% out of the sequential part of a calculation i…

> 8+6+4+4+3+2+1.5+1.5 What is this referring to? It sounds like a fascinating problem.

>> When people find out I got a 30% improvement by doing 8+6+4+4+3+2+1.5+1.5

> What is this referring to?

30 = 8+6+4+4+3+2+1.5+1.5

Re: Distributed systems programming has stalled

#222

Post by jiggawatts »

the_sleaze_ wrote: post #213
jiggawatts wrote: post #124

Earlier quoted context omitted.

In my less humble opinion: the only honest and objective review you’ll get about a system is from a new hire for about a month. Measure the “what the fucks per hour” as a barometer of how bad your org is and how deep a hole it has dug itself into. After that honeymoon period, all but the most autistic people will learn the organisational politics, keep their head down, and “play the game” to be assigned trivial menia…

This is quite a defensive posture. In my current role I've been able to see an incredible raft of insanity, not be obtuse or arrogant enough to dismiss solutions or the intelligence of those who made them, but literally make a communal list of refactor candidates. Then slowly but surely wrangle people and political capital to my side to eventually change them. Years later we still have cruft leftover but there are ma…

Your attitude is commendable! It’s what a true leader should do, and you deserve to be promoted for it.

My comment was a statistical observation of what typically happens in ordinary organisations without a strong-willed, technically capable leader at the helm.

Disclaimer: Also, I have a biased view, because as a consultant I will generally only turn up if there is something already wrong with an organisation that insiders are unable to fix.

Re: Distributed systems programming has stalled

#223

Post by LegionMammal978 »

roadbuster wrote: post #198
gklitt wrote: post #29

This is outside my area of expertise, but the post sounds like it’s asking for “choreographic programming”, where you can write an algorithm in a single function while reasoning explicitly about how it gets distributed: https://en.m.wikipedia.org/wiki/Choreographic_programming I’m curious to what extent the work in that area meets the need.

How does "choreographic programming" differ from the actor model?

From what I can tell, the important distinction is that all actors (and their messages) are described alongside each other, instead of being described separately. There are many implementations of the actor model, but most of them are the 'static-location architectures' that TFA talks about.

Re: Distributed systems programming has stalled

#225

Post by th0ma5 »

spratzt wrote: post #209
th0ma5 wrote: post #133

Earlier quoted context omitted.

I had a longer comment elsewhere but to me this says that the distribution is happening somewhere and what you're also saying is that companies have to decide how much they want or care to control it.

No. The issue is whether you NEED to not whether you want to. 10 to 15 years ago, you could argue, however implausibly, that hardware constraints meant vertical scaling was impossible, and you were forced to adopt a distributed architecture. Subsequent improvement in hardware performance, means that in 2025, vertical scaling is perfect acceptable in nearly all areas, relegating distributed architecture to the most ni…

There is essentially no tooling for this and vendors all default to distributed patterns. Either you directly control the scaling or you're relinquishing it.

Re: Distributed systems programming has stalled

#226

Post by th0ma5 »

CogitoCogito wrote: post #211
th0ma5 wrote: post #117

Earlier quoted context omitted.

I wish I could upvote you again. The complexity balloons when you try to adapt something that wasn't distributed, and often things can be way simpler and more robust if you start with a distributed concept.

I couldn't disagree more. My principle is to write systems extremely simply and then distribute portions of it as it becomes necessary. Almost always it never becomes necessary and the rare cases it does, it is entirely straight forward to do so unless you have an over-complicated design. I don't think I've ever seen it done well when done in the opposite direction. It's always cost more in time and effort and result…

Tons of vendors offer cloud first, distributed deployments. Erlang is distributed by default. Spark is distributed by default. Most databases are distributed by default.

Re: Distributed systems programming has stalled

#227

Post by motorest »

lucyjojo wrote: post #210
motorest wrote: post #200

Earlier quoted context omitted.

> I've found the rush to distributed computing when it's not strictly necessary kinda baffling. I'm not entirely sure you understand the problem domain, or even the high-level problem. The is or ever was a "rush" to distributed computing. What you actually have is this global epifany that having multiple computers communicating over a network to do something actually has a name, and it's called distributed computing.…

you and the guy you are answering too are not talking the same language (technically yes but you are putting different meanings to the same words). this would lead to a pointless conversation, if it were to ever happen.

> you and the guy you are answering too are not talking the same language (technically yes but you are putting different meanings to the same words).

That's the point, isn't it? It's simply wrong to assert that there's a rush to distributed systems when they are already ubiquitous in the real world, even if this comes as a surprise to people like OP. Get acquainted with the definition of distributed computing, and look at reality.

The only epiphany taking place is people looking at distributed systems and thinking that, yes, perhaps they should be treated as distributed systems. Perhaps the interfaces between multiple microservices are points of failure, but replacing them with a monolith does not make it less of a distributed system. Worse, taking down your monolith is also a failure mode, one with higher severity. How do you mitigate that failure mode? Well, educate yourself about distributed computing.

If you look at a distributed system and call it something other than distributed system, are you really speaking a different language, or are you simply misguided?

Re: Distributed systems programming has stalled

#228

Post by worthless-trash »

ignoramous wrote: post #89
hinkley wrote: post #55

I don’t think there’s anyone in the Elixir community who wouldn’t love it if companies would figure out that everyone is writing software that contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Erlang, and start hiring Elixir or Gleam devs. The future is here, but it is not evenly distributed.

> writing software that contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Erlang Since you were at AWS (?), you'd know that Erlang did get its shot at distributed systems there. I'm unsure what went wrong, but if not c/c++, it was all JVM based languages soon after that.

I have noticed that in corporate, langauges dont fail on technical merit, but its fashion merits.

Re: Distributed systems programming has stalled

#229

Post by icedchai »

alabastervlog wrote: post #50
bsnnkv wrote: post #2

Last month I switched from a role working on a distributed system (FAANG) to a role working on embedded software which runs on cards in data center racks. I was in my last role for a year, and 90%+ of my time was spent investigating things that went "missing" at one of many failure points between one of the many distributed components. I wrote less than 200 lines of code that year and I experienced the highest level…

I've found the rush to distributed computing when it's not strictly necessary kinda baffling. The costs in complexity are extreme. I can't imagine the median company doing this stuff is actually getting either better uptime or performance out of it—sure, it maybe recovers better if something breaks, maybe if you did everything right and regularly test that stuff (approximately nobody does though), but there's also so…

I've seen this as well. A relatively simple application becomes a mess of terraform configuration for CloudFront, Lambda, API Gateway, S3, RDS and a half dozen other lesser services because someone had an obsession with "serverless." And performance is worse. And there's as much Terraform as there is actually application code.

Re: Distributed systems programming has stalled

#230

Post by klysm »

herval wrote: post #137
klysm wrote: post #34

Earlier quoted context omitted.

That's okay though! We can just make LLMs grasp it!

ironically or not, the best way to have LLMs be effective at writing valid code is when they work on microservices. Since the scope is smaller and the boundary is clear, tools like Cursor/Windsurf seem to make very few mistakes (compared to pointing them at your monorepo, where they usually end up completely wrong)

Is it then up to the human to specify the services and how they interact?
Post reply on HN