Live data from Hacker News

Ask HN: Have we screwed ourselves as software engineers?

news.ycombinator.com

241–250 of 430 posts

Re: Ask HN: Have we screwed ourselves as software engineers?

#241

It's another way to favor incumbents with huge resources: it's an anti-competitive psyop to raise barriers to entry and stymie startups in endless shiny thing chasing, because the incumbents are scared of the disruption, so those big players that move a bit slower, seek ways to slow down the scrappy players that can move fast, while those "fast movers" get drunk on the koolaid of buzzwords manufactured on blogs from…

Did you learn the word 'incumbent' yesterday?

Aaw, so cute. My smart piece threatened you and you want to prove you're smarter than me. Nice try but you failed.... I learn all the time, you should try it. XD

Re: Ask HN: Have we screwed ourselves as software engineers?

#242
I think every single field has complexified throughout history. Agriculture now has dozens of chemicals and large machinery. Woodworking now has advanced machinery and a handful of ways to do the same cuts. Music evolved from Gregorian Chant to Bach to Jazz. Software is no different, humans invent more tools and techniques in each field over time.

I don't think engineers are willingly screwing themselves. Does anyone here choose to adopt something they know will screw them over? We may be forced into decisions by higher-ups or by colleagues or associates, but those people generally have some reason behind their actions, they don't willfully screw engineers for fun.

The field as a whole, none of us individually can control where it goes. If your org sticks with proven older tech, it will do zero to prevent new frameworks from cropping up everywhere else. If you adopt any newer technology, you're now becoming a user, increasing its relevance, helping to test it and prove it, finding bugs and errors.

So no, "we" have not "screwed ourselves". It's simply human nature to complexify and add more tools over time.

Re: Ask HN: Have we screwed ourselves as software engineers?

#243
Blame it on the "move fast and break things" mentality. The level of deprecation in this industry is really bad. And it is to a large extent trend-driven.

There is also an increased amount of saturation it seems in the development community. Many people who learn to leetcode before they learn the basic why's.

Re: Ask HN: Have we screwed ourselves as software engineers?

#244
Because software is so malleable, and its inner complexity largely invisible when it is actually running, the complexity tends to grow as much as it can (because it’s easier to add than to remove), that is, until it reaches the point where it is barely (not) manageable anymore. Hence we tend to live on that boundary of maximum still-barely-manageable complexity. Any time a new tool or technology makes some things simpler, the complexity just expands in some other direction (for example in the number of tools).

The only way to counteract this natural course is to explicitly and continuously take the time to simplify and consolidate things and to bear the extra cost of that continuous effort. But the incentives are stacked against that. As long as it (barely) works, the one taking shortcuts and increasing complexity, or just adding something new, will have an edge. It’s also much easier to create yet another leaky-abstraction layer on top of an existing system than improving the underlying system, because the latter is already in use by too many parties and the necessary changes cannot be done without breaking compatibility.

Another factor is that the field is still learning (e.g. type systems, how to best handle concurrency, distributivity, etc., not to speak of changes in the hardware having an effect on what works best, e.g. cache locality, parallelism, GPUs, etc.) and to some degree is still in its infancy. Maybe at some point in the future we will have it all figured out and reach a point of stability where we can concentrate on just making everything as simple and coherent as possible in the ways we by then know work best. But maybe not, and certainly not within our lifetimes.

So, yes, for the time being I’d say there is no real escape. But you can probably find a niche where things are calmer and slower, and stay away from the areas that are the most crazy and quick-moving.

Re: Ask HN: Have we screwed ourselves as software engineers?

#245

Sounds to me like you've worked at trendy tech companies and want to keep working at trendy tech companies. That means you're going to have to work with trendy technology. A massive amount of companies, maybe even the majority, don't do this. They use what works and upgrade when needed, not when its cool to use the new thing. They just don't tend to pay like trendy companies and don't look as good on a resume as tren…

As someone who purposely -doesn't- work at a trendy company, it seeps in here too. People abuse us to get jobs at those trendy tech companies, basically.

Understand we have a relatively small web presence with not a lot of traffic. High dollar, low volume type stuff. About 6 years ago someone decided to move all our stuff to self hosted kubernetes and microservices, because reasons. As you may imagine, he took a job as an infra guy for a much larger company a few months later, leaving others to figure out and clean up the mess.

Not long ago, another did the same thing, but with graphql. Why? Because it's a graph! How cool. Again, left for a sexy tech company not long after, and now our tiny api service is stuck under gazillion lines of autogenerated code.

All of this points to the real problem I guess, which are middle managers. They eat buzzwords up like candy and trust that new and flashy means that person is smarter.

Re: Ask HN: Have we screwed ourselves as software engineers?

#246

The way I look at it is: there are more tools in the toolbox than ever before. Which makes our judgement (the thing they really pay us for) even more important. Kubernetes, for example, is a specific solution to a specific problem. The solution is complex but so is the problem. If k8s give you the right trade-offs for your situation, then it’s not busy work. Of course, there are plenty of project where judgement Is t…

Promotion based architecture is self fulfilling prophecy at least in BI/ data world. I see everybody around me moving to cloud, without really good explanation why. Only reasonable thing I can see as an pattern is that cloud experience on top of data things gets paid 30% more. It made me consider cloud a lot. I was considering switching to cloud, just so I can put in my CV "experience with migration to cloud". For ne…

Hell, I'm working with ~200TB and it still doesn't make sense if you work out the costs.

As far as I can tell it only makes sense if you have a ton of data, and only ever use a fraction of it at a time, for one-off jobs, infrequently.

Re: Ask HN: Have we screwed ourselves as software engineers?

#247

A question for the people here who have a favorable view of kubernetes: what is your level of experience with it, and what problems does it solve that aren't already solved by cloud managed services?

I use DigitalOcean's managed k8s to host quite a few services over the last few years.

There's definitely some complexity and learning curve involved, but it comes with some nice advantages for my particular use case:

- Low vendor lock-in: I actually migrated to DO from a different hosted k8s, and I was able to reuse the majority of my configuration. - Reproducibility: your projects are derived from the resources you upload, so it's hard to end up with a setup that you can't easily reproduce (or migrate!) elsewhere. - High flexibility: I can do some relatively strange things with e.g. routing without k8s batting an eye. There isn't really ever a point of "oh no, I've found something I can't really, do". - I've found it to be cheaper than cloud managed services once you're hosting as much as I am.

More generally, I like the...standardized(?) style. It feels like a sort of "build your own cloud", but the blocks you use look like everyone else's, despite the total product looking a bit different for everyone. I can use k8s managed, in a business setting, or fully self-hosted, and the essentials still work the same way.

A lot of bad experiences wrt complexity I hear come from running the cluster yourself, but nowadays, distributions like k3s make this, dare I say, pretty easy. If you want to use. VPS, DO managed k8s is very nice.

Re: Ask HN: Have we screwed ourselves as software engineers?

#248

Earlier quoted context omitted.

Partially depends but company with 1 IT person that is not data focused, there is low probability that they will have that amount of data. For anything above what we have I think it weirdly depends on country/ salaries. In US, 100k per year is no brainer for cloud as even 1 FTE would cost much more. In non-western Europe, 100k is deal killer as you can hire 2 senior DBAs and you still have enough money for quite a re…

With extensive experience in this vertical, (marketing agency and analytics), 200GB is a _partial_ day for one datasource, let alone table or total DB size.

but does team have 1 IT/ data person?

Edit: sorry I see that you are the same person I was replying to. Then to understand it better, who is consumer of so vast amount of data?

Re: Ask HN: Have we screwed ourselves as software engineers?

#249

Earlier quoted context omitted.

Promotion based architecture is self fulfilling prophecy at least in BI/ data world. I see everybody around me moving to cloud, without really good explanation why. Only reasonable thing I can see as an pattern is that cloud experience on top of data things gets paid 30% more. It made me consider cloud a lot. I was considering switching to cloud, just so I can put in my CV "experience with migration to cloud". For ne…

There's a finance angle behind cloud stuff too that's irrestible for the bean counters: cloud stuff is operations expense, on-prem is a capital expense. Unfortunately these folks are heavily incentivized to favor OpEx I'm not a bean counter, all I know is those guys at my last job would rattle off about it like zombies. IIRC its a tax thing

It's not a tax thing, long-term leases or colocation is a liability for acquisitions. (You could also build a revenue-positive company, but that's the loser way out, apparently.)

Re: Ask HN: Have we screwed ourselves as software engineers?

#250
It's the last stage of the second Dotcom bubble. The industry has been taken over by parasites and the well of innovation left over from the ARPA/PARC years is starting to run dry despite some of the better ideas having yet to have seen their heydey. It will be interesting to see what happens as the money-spigot dries up as the Federal Reserve starts to implement Quantitative Tightening over the next few years. I'm optimistic as all of this might, "reset the board" enough to slow trends toward digital enclosure. Worth looking at:

https://web.law.duke.edu/pd/papers/boyle.pdf

Post reply on HN