Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We decided to go for the big rewrite

tech.channable.com

Post reply on HN
Print view View original

51–60 of 75 posts

Re: We decided to go for the big rewrite

#51

Post by steve_adams_86 »

alexpotato wrote: post #31

Rewriting an application can mean different things: 1. "We are going to start over from scratch and rewrite the whole thing!" Joel Spolksy famously said to "Never do this!" 2. "We are going to slowly refactor the whole codebase." This can, eventually, lead you to a place where none of the original code is there so it's like a rewrite but much simpler. 3. "We are going to slowly add new places to replace the old syste…

I'm not sure about the resume part. My experience refactoring legacy stuff into more modern stuff is actually a big talking point for my resume in my experience. People are always interested in why I did it, how I did it, why didn't I just build anew, etc.

Maybe that's because most jobs I apply for tend to have at least one 'legacy' system laying around that needs some help. But this is somewhat common outside of the startup scene.

Legacy in this case means an app that's maybe 7-10 years old, often written like everything happened on the back of a napkin. Sprawling, inconsistent, monolithic stuff. But not true legacy where it's written in Fortran or something in the 1500s.

At any rate, I think it's worth having refactors on your resume. I know I'd be interested in people who have done it - it's always such an educational experience.

Re: We decided to go for the big rewrite

#52

Post by nemothekid »

kod wrote: post #40
royjacobs wrote: post #11

Reading this article it seems like yet another example of "you don't have big data". Most of the features that are unique to Spark (or Spark-like setups) were not needed, so in the end it's mostly...just an app talking to Postgres? I'm not sure, but reading other articles[0] on the blog seems like they've been jumping on bandwagons before, so it's probably good to come back on those decisions every now and again. Edi…

> "Sure, Scala will work great! It's future-proof and everyone will love it!" Scala is still the least-bad option for a JVM language. Anyone who can't be productive in Scala (not "better java", not "worse haskell", Scala) isn't someone you want on your team anyway.

>Anyone who can't be productive in Scala (not "better java", not "worse haskell", Scala) isn't someone you want on your team anyway.

It’s funny that you gloss over one the big productivity issues with Scala. Somehow everyone who joins your team should be acutely aware of your flavor of Scala.

If the community decided if wanted to be a better java or a worse Haskell then I bet more people would be productive in Scala

Re: We decided to go for the big rewrite

#53

Post by ryanelfman »

goto11 wrote: post #50

Is the current system is so badly architected that it cannot be refactored gradually or rewritten piecemeal? Then the forces which caused these problems will also be in effect during the rewrite, so it will end in the same place when it reaches feature parity. I can only think of a few places where a full rewrite is justified: * You lost the source code * The application is almost purely integration with some 3'rd pa…

That's not necessarily true. Those forces will still be there but the learning of what not to do has also been had. People can improve and learn over time.

Re: We decided to go for the big rewrite

#54

Post by samprotas »

macspoofing wrote: post #47
royjacobs wrote: post #11

Reading this article it seems like yet another example of "you don't have big data". Most of the features that are unique to Spark (or Spark-like setups) were not needed, so in the end it's mostly...just an app talking to Postgres? I'm not sure, but reading other articles[0] on the blog seems like they've been jumping on bandwagons before, so it's probably good to come back on those decisions every now and again. Edi…

>Not trying to come off as too snarky I had a similar thought. Postgres is a great choice, but then they also went with Haskell ... I look forward to another blog post in 2-3 years detailed all the ways that Haskell failed them and that at the day they should have just gone with an industry standard language.

https://tech.channable.com/posts/2017-02-24-how-we-secretly-...

FWIW they have this post from almost 3 years ago about adding Haskell to their stack. I'm guessing the time for your prediction has come and gone.

On the other hand, I find it quite encouraging that Haskell was barely mentioned. It seems they viewed the risk as "changing the projects language" rather than "using a non industry standard language".

Re: We decided to go for the big rewrite

#55

Post by collyw »

alexpotato wrote: post #31

Rewriting an application can mean different things: 1. "We are going to start over from scratch and rewrite the whole thing!" Joel Spolksy famously said to "Never do this!" 2. "We are going to slowly refactor the whole codebase." This can, eventually, lead you to a place where none of the original code is there so it's like a rewrite but much simpler. 3. "We are going to slowly add new places to replace the old syste…

I guess if you are replacing a system part by part as opposed to big bang rewrite, you will be more likely do make sure interfaces are well defined and components are more modular.

Re: We decided to go for the big rewrite

#56

Post by pbourke »

mumblemumble wrote: post #33
royjacobs wrote: post #11

Reading this article it seems like yet another example of "you don't have big data". Most of the features that are unique to Spark (or Spark-like setups) were not needed, so in the end it's mostly...just an app talking to Postgres? I'm not sure, but reading other articles[0] on the blog seems like they've been jumping on bandwagons before, so it's probably good to come back on those decisions every now and again. Edi…

There's that, but Spark is also a little bit tricky because it has such a big feature set that it's not just attractive as a big data tool. On paper, it can be attractive as a tool for easy single-node data parallelism, for easy streaming data processing, for easy machine learning on the Java platform, stuff like that. I'm looking at migrating off of Spark, too, and finding that Spark is still the only way to get a d…

Also, if you collaborate with data scientists and don't have the resources to use fully distinct systems for the production and research/reporting functions then Spark is very nearly the only game in town.

Re: We decided to go for the big rewrite

#57

Post by devonkim »

Darkstryder wrote: post #9
devonkim wrote: post #6

Earlier quoted context omitted.

High availability Postgres setups are a minimum for a production system and a staging system to understand how your system behaves during a failure event. These failure scenarios should be tested not necessarily on every commit but often enough that there’s confidence during a failover you’re not going to drop queries on the floor and pretend it’s all good as well as your monitoring systems report on the event for th…

Yeah. I guess after having been bitten myself a few times with failed MySQL failovers and especially after having read the GitHub October 2018 incident postmortem [1], I stopped considering failover solutions as a reliable availability solution altogether. However this is just a personal opinion that I might revisit at some point. [1] https://github.blog/2018-10-30-oct21-post-incident-analysis/

High availability setups are absolutely required to upgrade / patch running databases as well without significant downtime. The engineering and business costs in time to try to work around these issues are from the 90s and have no place in a modern business environment. Heck, they figured out HA decades before then in commercial, proprietary DBs. Things are much more reliable now with OSS tools than even 4 years ago to the extent few talk about it anymore. There are definitely mistakes and bugs possible but the number of _successful_ failover and failback events must be considered in the calculus.

Upgrades aren’t to be taken lightly of course but again, it’s now a cost of doing business and a reality that we need to engineer properly for.

Re: We decided to go for the big rewrite

#60

Post by commandlinefan »

mason55 wrote: post #39
mumblemumble wrote: post #33

Earlier quoted context omitted.

There's that, but Spark is also a little bit tricky because it has such a big feature set that it's not just attractive as a big data tool. On paper, it can be attractive as a tool for easy single-node data parallelism, for easy streaming data processing, for easy machine learning on the Java platform, stuff like that. I'm looking at migrating off of Spark, too, and finding that Spark is still the only way to get a d…

Curious what you recommend instead of Spark.

Almost 100% of the time when somebody says “curious what you recommend instead of _____” the best answer is “write a program”, but for some reason programmers aren’t supposed to do that any more.
Post reply on HN