Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distributed systems programming has stalled

shadaj.me

Post reply on HN
Print view View original

141–150 of 232 posts

Re: Distributed systems programming has stalled

#141

Post by Nevermark »

cmrdporcupine wrote: post #106
porridgeraisin wrote: post #43

Earlier quoted context omitted.

I think the reason that distributed systems still are the go-to choice for many software teams is to do with people/career expectations/careers orienting themselves around distributed systems over the time period you mentioned. It will take a while for it to re-orient, and then distributed systems might become a fad again ;) An example of this is typical promotion incentives being easier to get in microservice teams,…

Honestly, I am more cynical and just think people are always looking for ways to make their jobs more interesting than they actually are.

> Honestly, I am more cynical and just think people are always looking for ways to make their jobs more interesting than they actually are.

When you frame the problem that way, unnecessary complexity seems like part of a healthy solution path. /h

Companies get reliability benefits from slack, but creative people abhor wasted slack. Some basic business strategy/wisdom for maintaining/managing creative slack is needed.

Re: Distributed systems programming has stalled

#143

Post by FpUser »

alabastervlog wrote: post #50
bsnnkv wrote: post #2

Last month I switched from a role working on a distributed system (FAANG) to a role working on embedded software which runs on cards in data center racks. I was in my last role for a year, and 90%+ of my time was spent investigating things that went "missing" at one of many failure points between one of the many distributed components. I wrote less than 200 lines of code that year and I experienced the highest level…

I've found the rush to distributed computing when it's not strictly necessary kinda baffling. The costs in complexity are extreme. I can't imagine the median company doing this stuff is actually getting either better uptime or performance out of it—sure, it maybe recovers better if something breaks, maybe if you did everything right and regularly test that stuff (approximately nobody does though), but there's also so…

This is part of what I do for living. C++ backend software running on real hardware which is currently insanely powerful. There is of course spare standby in case things go South. Works like a charm and I have yet to have a client that scratched it anywhere close to overloading server.

I understand that it can not deal with FAANG scale problems, but those are relevant only to a small subset of businesses.

Re: Distributed systems programming has stalled

#144

Post by gmadsen »

hinkley wrote: post #47
gmadsen wrote: post #19

Earlier quoted context omitted.

I know c++ has a lack luster implementation, but do coroutines and channels solve some of these complaints? although not inherently multithreaded, many things shouldn't be multithreaded , just paused. and channels insteaded of shared memory can control order

Coroutines basically make the same observation as transmit windows in TCP/IP: you don’t send data as fast as you can if the other end can’t process it, but also if you send one at a time then you’re going to be twiddling your fingers an awful lot. So you send ten, or twenty, and you wait for signs of progress before you send more. On coroutines it’s not the network but the L1 cache. You’re better off running a functi…

fair enough, that was the design choice c++ went with to not break ABI and have moveable coroutine handles

rust accepted the tradeoff and can do pure stack async,

there are things you can do in c++ to not get the dynamic allocation to heap, but it requires a custom allocator + predefining size of coroutines.

https://pigweed.dev/docs/blog/05-coroutines.html

Re: Distributed systems programming has stalled

#145

Post by yolovoe »

bsnnkv wrote: post #2

Last month I switched from a role working on a distributed system (FAANG) to a role working on embedded software which runs on cards in data center racks. I was in my last role for a year, and 90%+ of my time was spent investigating things that went "missing" at one of many failure points between one of the many distributed components. I wrote less than 200 lines of code that year and I experienced the highest level…

Is the “card” work EC2 Nitro by any chance? Sounds similar to what I used to do

Re: Distributed systems programming has stalled

#146

Post by hintymad »

jbarham wrote: post #135
hintymad wrote: post #122

This reminds me of Rob Pike's article "Systems Software Research is Irrelevant," written about 15 years ago. Perhaps many systems have matured to a point where any improvement appears incremental to engineers, so the conviction to develop a new programming model isn't strong enough. Or perhaps we're in a temporary plateau, and a groundbreaking tool will emerge in a few years. Regarding Laddad's point, building tools…

> This reminds me of Rob Pike's article "Systems Software Research is Irrelevant," written about 15 years ago. 25 years ago: http://herpolhode.com/rob/utah2000.pdf (Time flies.)

Wow, time flies. Thanks!

Re: Distributed systems programming has stalled

#147

Post by englishspot »

bsnnkv wrote: post #2

Last month I switched from a role working on a distributed system (FAANG) to a role working on embedded software which runs on cards in data center racks. I was in my last role for a year, and 90%+ of my time was spent investigating things that went "missing" at one of many failure points between one of the many distributed components. I wrote less than 200 lines of code that year and I experienced the highest level…

curious as to how you made that transition. seems like that'd be tough in today's job market.

Re: Distributed systems programming has stalled

#148

Post by PaulDavisThe1st »

EtCepeyd wrote: post #12
bsnnkv wrote: post #2

Last month I switched from a role working on a distributed system (FAANG) to a role working on embedded software which runs on cards in data center racks. I was in my last role for a year, and 90%+ of my time was spent investigating things that went "missing" at one of many failure points between one of the many distributed components. I wrote less than 200 lines of code that year and I experienced the highest level…

This resonates a lot with me. Distributed systems require insanely hard math at the bottom (paxos, raft, gossip, vector clocks, ...) It's not how the human brain works natively -- we can learn abstract thinking, but it's very hard. Embedded systems sometimes require the parallelization of some hot spots, but those are more like the exception AIUI, and you have a lot more control over things; everything is more local…

> Even data race free multi-threaded programming in modern C and C++ is incredibly annoying; I dislike dealing with both an explicit mesh of peers, and with a leaky abstraction that lies that threads are "symmetric" (as in SMP) while in reality there's a complicated messaging network underneath.

If you're using traditional (p)threads-derived APIs to get work done on a message passing system, I'd say you're using the wrong API.

More likely, I don't understand what you might mean here.

Re: Distributed systems programming has stalled

#149

Post by EtCepeyd »

bigmutant wrote: post #92
hinkley wrote: post #61

Earlier quoted context omitted.

I am 100% convinced we could delete the compiler class from college curricula and replace it with distributed computing and the world would be a better place.

Def agree. Most people will never touch an Abstract Syntax Tree or even Expression Trees. Almost everyone working in back-end will use Cloud Services, will make mistakes based on assumptions of what they provide

I was studying for my MSc in CS some 25 years ago. Our curriculum included both automata/formal languages (multiple courses over multiple semesters) and parallel programming.

The latter course (a) was built on a mathematical formalism that had been developed at the university proper and not used anywhere else, (b) used PVM: https://www.netlib.org/pvm3/>, https://en.wikipedia.org/wiki/Parallel_Virtual_Machine>, for labs.

Since then, I've repeatedly felt that I've seriously benefited from my formal languages courses, while the same couldn't be said about my parallel programming studies. PVM is dead technology (I think it must have counted as "nearly dead" right when we were using it). And the only aspect I recall about the formal parallel stuff is that it resembles nothing that I've read or seen about distributed and/or concurrent programming ever since.

A funny old memory regarding PVM. (This was a time when we used landlines with 56 kbit/s modems and pppd to dial in to university servers.) I bought a cheap second computer just so I could actually "distribute" PVM over a "cluster". For connecting both machines, I used linux's PLIP implementation. I didn't have money for two ethernet cards. IIRC, PLIP allowed for 40 kbyte/s transfers! https://en.wikipedia.org/wiki/Parallel_Line_Internet_Protoco...>

Re: Distributed systems programming has stalled

#150

Post by anitil »

AlotOfReading wrote: post #52
EtCepeyd wrote: post #12

Earlier quoted context omitted.

This resonates a lot with me. Distributed systems require insanely hard math at the bottom (paxos, raft, gossip, vector clocks, ...) It's not how the human brain works natively -- we can learn abstract thinking, but it's very hard. Embedded systems sometimes require the parallelization of some hot spots, but those are more like the exception AIUI, and you have a lot more control over things; everything is more local…

Most embedded systems are distributed systems these days, there's simply a cultural barrier that prevents most practitioners from fully grappling with that fact. A lot of systems I've worked on have benefited from copying ideas invented by distributed systems folks working on networking stuff 20 years ago.

Yes even 'simple' devices these days will have devices (ADC/SPI etc) running in parallel often using DMA, multiple semi-independent clocks, possibly nested interrupts etc. Oh and the UART for some reason always, always has bugs, so hopefully you're using multiple levels of error checking.
Post reply on HN