Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go + Services = One Goliath Project

engineering.khanacademy.org

Post reply on HN
Print view View original

381–390 of 449 posts

Re: Go + Services = One Goliath Project

#381

Post by [deleted] »

pjmlp wrote: post #314
remote_phone wrote: post #313

What a mistake. Not only going from a monolith to microservices but also changing the language? This is a mistake that rookies make. This will be one of those post-Mortems where they will sheepishly admit they bit more than they could chew, and it wasted years of productivity. There’s no reason to move to Go. Stick with Python for now. Migrate safely to python 3. Once everything is stable, start breaking things up in…

This kind of decisions is usually always taken when the monetary cost is not measured. Apparently not mapping developer hours to real money keeps not being a thing outside consulting shops.

[deleted]

Re: Go + Services = One Goliath Project

#382

Post by StavrosK »

andy_ppp wrote: post #330
scottrogowski wrote: post #296

Earlier quoted context omitted.

A company I am affiliated with made a decision to rewrite their code in microservices-oriented architecture thinking it would only take one year. Now we're 7 years into the transition and starting to come up against some hard deadlines that threaten revenue streams. It seems obvious to everyone except the leadership and the architects that this has been an unmitigated disaster. Other comments on this thread seem to i…

Yes, the micro services + Golang vanity project because you think you’re Google. I really don’t think people understand error states in distributed systems very well, putting possible network partitions everywhere is not a great idea. I would strongly suggest trying a Golang monolith first and seeing if there are one or two heavily used services that need splitting off. Also monorepo. Always.

Microservices have always seemed dangerous to me. "Bugs thrive in the seams between libraries, so let's put more and deeper seams in!"

Microservices have an immense cost, and you have to make sure they're worth it. Many teams years ago found it a nice pattern and implemented it because why not, and now we're at the "oops this isn't actually amazing" part of the cycle.

Re: Go + Services = One Goliath Project

#383

Post by andy_ppp »

StavrosK wrote: post #382
andy_ppp wrote: post #330

Earlier quoted context omitted.

Yes, the micro services + Golang vanity project because you think you’re Google. I really don’t think people understand error states in distributed systems very well, putting possible network partitions everywhere is not a great idea. I would strongly suggest trying a Golang monolith first and seeing if there are one or two heavily used services that need splitting off. Also monorepo. Always.

Microservices have always seemed dangerous to me. "Bugs thrive in the seams between libraries, so let's put more and deeper seams in!" Microservices have an immense cost, and you have to make sure they're worth it. Many teams years ago found it a nice pattern and implemented it because why not, and now we're at the "oops this isn't actually amazing" part of the cycle.

If you work at Google where the standard is higher fine, but 99% of places people end up making very buggy stuff that interrelates in weird ways!

Re: Go + Services = One Goliath Project

#384

Post by snypox »

jordanbeiber wrote: post #359
DelightOne wrote: post #353

Earlier quoted context omitted.

Can you explain what you mean with service lifecycle management in regards to micro services please, or do you have a book on it? I'm currently studying SE and it's the first time it has come up. Thank you :)

Well - I’ve got a “service management” background, so it’s completely natural to talk in these terms. :) A service have in a way two interfaces - one business (the work it’s doing), and one technical (how it does it; a port publishing an endpoint or whatever). No business process to support, no service to manage. The lifecycle of the technical service will consist of a bunch of actions that will be taken, perpetually…

Never heard about this concept, I really like it.

Re: Go + Services = One Goliath Project

#385

Post by AnonymousPlanet »

StavrosK wrote: post #379
AnonymousPlanet wrote: post #374

Earlier quoted context omitted.

Hahaha, sorry but this is a very cute thing to say, in my view. At our company we just barely finished migrating our software with nearly a million lines of legacy Python 2 code to Python 3. This took over a year of nearly exclusive migration effort, just making our code work with both. The entire migration project started way before I joined the company several years ago. So, no, things are not as simple if you're n…

It very much depends on how good the codebase is. I also spent a year on and off porting a large codebase from 2 to 3, and it would have gone an order of magnitude faster if the codebase were in better shape.

I agree that code quality is a big factor. But what is good code quality in Python? In our case, the oldest code is the most "pythonic" and is at the same time the worst to maintain. The better code mitigates the drawbacks of dynamic typing and by that moves away from the pythonic standard you see in many libraries.

But even if you nail the types to the board (e.g. assert isinstance(...)), use (the somewhat weak) Mypy wherever you can, and have good test coverage, you still have to grep your code base for usage of, e.g., .keys(), eyeball hundreds of modules for subtle Unicode madness or hunt for the odd division, replace every sort() that doesn't use key= yet, etc. The todos add up and someone has to go into the code and change those lines.

Re: Go + Services = One Goliath Project

#386

Post by StavrosK »

AnonymousPlanet wrote: post #385
StavrosK wrote: post #379

Earlier quoted context omitted.

It very much depends on how good the codebase is. I also spent a year on and off porting a large codebase from 2 to 3, and it would have gone an order of magnitude faster if the codebase were in better shape.

I agree that code quality is a big factor. But what is good code quality in Python? In our case, the oldest code is the most "pythonic" and is at the same time the worst to maintain. The better code mitigates the drawbacks of dynamic typing and by that moves away from the pythonic standard you see in many libraries. But even if you nail the types to the board (e.g. assert isinstance(...)), use (the somewhat weak) Myp…

Of the things you mentioned, only Unicode has been a problem, and that's exactly what 3 is fixing, so that's to be expected. The rest was automatically handled by 2to3 with only a cursory review.

Re: Go + Services = One Goliath Project

#387

Post by jordanbeiber »

snypox wrote: post #384
jordanbeiber wrote: post #359

Earlier quoted context omitted.

Well - I’ve got a “service management” background, so it’s completely natural to talk in these terms. :) A service have in a way two interfaces - one business (the work it’s doing), and one technical (how it does it; a port publishing an endpoint or whatever). No business process to support, no service to manage. The lifecycle of the technical service will consist of a bunch of actions that will be taken, perpetually…

Never heard about this concept, I really like it.

To me, it’s kind of what ”DevOps” should be about, from a technical perspective:

Take the best/reasonable parts from ITIL (concepts/principles), mix it with principles from the agile manifesto and the 12-factor app. Automate the lot of it.

Doing this in practice gives you dev & ops.

It’s quite a journey that is more difficult the bigger you are. It scales though, so start small, prove the concepts, and grow organically.

Re: Go + Services = One Goliath Project

#388

Post by AnonymousPlanet »

StavrosK wrote: post #386
AnonymousPlanet wrote: post #385

Earlier quoted context omitted.

I agree that code quality is a big factor. But what is good code quality in Python? In our case, the oldest code is the most "pythonic" and is at the same time the worst to maintain. The better code mitigates the drawbacks of dynamic typing and by that moves away from the pythonic standard you see in many libraries. But even if you nail the types to the board (e.g. assert isinstance(...)), use (the somewhat weak) Myp…

Of the things you mentioned, only Unicode has been a problem, and that's exactly what 3 is fixing, so that's to be expected. The rest was automatically handled by 2to3 with only a cursory review.

We tried 2to3 and it gave us poor results. But probably because we deviated too far from being pythonic.

The str/unicode misery is one of the biggest gripes I have with Python. I'm glad this unpleasant knot has been mostly untied in Python 3. I came to the conclusion that the transition would have been much easier if Python 3 just concentrated on the separation of bytes and (unicode) strings. The other features could have been in Python 4.

It's a bit like IPv6. If it would just solve the address space problem, most would have moved to it already. Instead it comes with a lot more baggage. And each additional feature has it's own uphill battle for acceptance. So nearly everyone is dragging their feet, citing their pet peeve with the technology.

Re: Go + Services = One Goliath Project

#389

Post by stult »

AnonymousPlanet wrote: post #385
StavrosK wrote: post #379

Earlier quoted context omitted.

It very much depends on how good the codebase is. I also spent a year on and off porting a large codebase from 2 to 3, and it would have gone an order of magnitude faster if the codebase were in better shape.

I agree that code quality is a big factor. But what is good code quality in Python? In our case, the oldest code is the most "pythonic" and is at the same time the worst to maintain. The better code mitigates the drawbacks of dynamic typing and by that moves away from the pythonic standard you see in many libraries. But even if you nail the types to the board (e.g. assert isinstance(...)), use (the somewhat weak) Myp…

What specific “pythonic” habits have you found are more difficult to maintain? Asking out of genuine curiosity, not to challenge the premise. I work with a lot of data science people that really emphasize being pythonic, but coming from the software/static typing side of the house I always find their code style and architecture a little concerning, and I’m not sure if I’m just not getting it or if they really are writing spaghetti.

Re: Go + Services = One Goliath Project

#390

Post by StavrosK »

AnonymousPlanet wrote: post #388
StavrosK wrote: post #386

Earlier quoted context omitted.

Of the things you mentioned, only Unicode has been a problem, and that's exactly what 3 is fixing, so that's to be expected. The rest was automatically handled by 2to3 with only a cursory review.

We tried 2to3 and it gave us poor results. But probably because we deviated too far from being pythonic. The str/unicode misery is one of the biggest gripes I have with Python. I'm glad this unpleasant knot has been mostly untied in Python 3. I came to the conclusion that the transition would have been much easier if Python 3 just concentrated on the separation of bytes and (unicode) strings. The other features could…

I'm not sure that's true because, as I said, most of it is automatic and worked well with 2to3, leaving us to deal pretty much only with Unicode. I'd certainly prefer to only have to do this upgrade once.
Post reply on HN