Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
61–70 of 78 posts
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#62Take a few steps back and relax. If you were looking at this from the Space Station, who would you say has a problem with the code base? That's right, your client. Not you. Your client. If he/she has 150 installs and 150 angry clients he/she knows that this thing is rotten somewhere. You client may or may not have some technical understanding but rest assured that they understand business. Life often boils down to bi…
Excellent, thoughtful and presumably experience-based reply. This gets my vote.
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#63Lots of good advice in here. I lean toward the "run screaming" side myself.
The fact that they've burned through several contracting companies and still think it's possible to get large numbers of bug fixes in the first month suggests that they're pretty clueless. It's going to take you two or three months (if not longer) just to get your head into the code enough that you can fix anything nontrivial.
If I were advising them, I would say they need two teams: one team of just two or three people (or maybe just one) solely trying to fix bugs in the existing code, and the other team of three or four people doing a complete rewrite from scratch. As others have commented, complete rewrites are normally a bad idea, but this code base is so far gone I don't think it can be incrementally refactored into sanity. Oh, and they should expect the rewrite to take two years.
But despite their experience to this point, it sounds like they're still not ready to hear that. Which leaves you little choice but to run screaming.
EDITED to add: what these people need to understand is that their demand for results in a hurry is what got them into this mess in the first place. Until they get that I don't think there's any hope.
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#64Make sure the code (including any stored procedures or code that is otherwise stored in the database) is in a version control repository. Because any change in this code might have unexpected, subtle consequences (i.e. introduce more bugs). In which case you'd do better rolling back that particular change.
The next step would be to get a grip on deployment. Automate it, so you can roll out updates and roll back updates to all clients without breaking a sweat.
Then set up a proper backlog and bug tracking system, where you can prioritize bugs and work items. (And maybe open it up for bug reports by clients?)
Just like with a real debt, with a technical debt, seeing progress can help to keep you going. At this point, you should have a grip on it, it's just still going to be a lot of hard work. There's good advice on how to approach the refactoring.
Finally, and this is not related to the code, educate stakeholders in your organization about the concept of technical debt. (Back it up by time tracking various work items from the bug tracker.) Somehow your organization got into this situation, so there may be a problem where new features or custom features for clients get priority before bugfixes, and are written without much guidance. Joel Spolsky has written on this subject, you may find his writings help explain the concept, as well as find a way out of this mess (like the '12 steps to better software').
Good luck!
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#65Chances are infinitesimally small you can fix it. If you start to maintain it, you'll be the one who gets the blame if it doesn't work. In an absolute best-case scenario you get to rewrite it whole in your spare time, while trying to keep the legacy code together with ducttape and chickenwire on your employers time.
Get out, get out now. They don't need a maintenance programmer, they need a ninja programmer, the liquidator kind.
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#66I have actually worked on a code base like you're describing, on a contract basis, for a client. I loathe maintenance programming, so the relationship didn't last very long -- just a few months. So: 1. Make sure you have a rock-solid contract in place with the client that will ensure that you get paid, get paid well, and get paid often. Receiving a check in the mail makes it easier to look at the code. If your paymen…
" Everybody that says "rewrite" is dreaming. It is impossible to rewrite something that large without breaking something and spending too much money. Re-factoring a function is doable. Re-factoring a thousand-line file is doable. Re-factoring part of a database is doable. Re-factoring all of it all at once is starry-eyed fiction. Not gonna happen."
I don't think so. Yes, the opportunities to do something like this are rare. It is up to the project lead and the client to decide whether or not this makes sense.
Also, keep in mind that a "complete re-write" doesn't necessarily literally mean that every line of code must be re-written. There's often tons that can be salvaged.
If the code base is an absolute disaster I would not touch it without the understanding that the project might entail massive re-writing of portions of the codes base as well as significant structural modifications. Maybe I'm lucky in that I've never really had to go look for work. I would flat-out reject a project like this without massive client buy-in.
If it is mission critical for the client and they can afford it there is no reason not to step back, truly evaluate the situation and consider a significant redo of the app.
For any non-trivial enterprise having a solid and maintainable code base is nearly priceless. Is it worth investing a year and the corresponding financial commitment to fix the problem once and for all? For the right business, yes! The alternative is to live with a patch-work of code for the next ten years of more.
Because I move across disciplines I have seen this sort of thing in many areas outside of just software code-bases.
I have, as an example, seen data processing facilities with millions of dollars in equipment designed in patch-work fashion that bleed money on a daily basis. In one such cases I proposed a complete redoing of the facility (in staged fashion in order to not affect business). It was very costly, but the owners where under such pain due to the constant bleed that they saw the intelligence in investing a lot of money to lay down an infrastructure that would withstand the test of time, not to mention stopping the bleeding.
Similarly, I have seen this in faulty processes. Process optimization or redesign can be critical to a business. The most well known example of this is the automobile industry.
Car manufactures like Mercedes were devoting fully 20% of their factory floor space to repairs. Cars would come off the assembly line with defects that would have to be repaired after the fact. This consumed a tremendous amount of time, money and resources.
In sharp contrast to this, companies like Toyota where using an approach that aimed to have cars come off the line with zero defects. They'd stop the assembly line when an defect was detected. At first they nearly couldn't make cars. The philosophy was to ensure that detected defects never re-occurred. With time cars started to come off the line with few, if any, defects. Most car manufacturer have now adopted these ideas.
The point is that sometimes a "complete rewrite" is warranted and even necessary. On cannot categorically state that the idea of a re-write is "fiction" any more than stating that it is an absolute necessity while being completely detached from the players and their circumstances. I suggest that it is for the client and consultant to evaluate and decide.
On a personal note. I don't enjoy working with crap. I enjoy my craft. Whether it is writing code, designing electronics or mechanical. I enjoy doing good work and working in quality projects. Life is too short to work on shit projects. You learn nothing and nobody is happy.
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#67I have actually worked on a code base like you're describing, on a contract basis, for a client. I loathe maintenance programming, so the relationship didn't last very long -- just a few months. So: 1. Make sure you have a rock-solid contract in place with the client that will ensure that you get paid, get paid well, and get paid often. Receiving a check in the mail makes it easier to look at the code. If your paymen…
" Everybody that says "rewrite" is dreaming. It is impossible to rewrite something that large without breaking something and spending too much money. Re-factoring a function is doable. Re-factoring a thousand-line file is doable. Re-factoring part of a database is doable. Re-factoring all of it all at once is starry-eyed fiction. Not gonna happen." I don't think so. Yes, the opportunities to do something like this ar…
Yes, I have often said that you should never, ever throw (significant amounts of) code away and start from scratch. I have worked on two bodies of legacy code before and more or less used most of the techniques discussed in the comments, but this is unprecented - for me - in both size and badness. Client and I have agreed that I will work for a month and then see where we stand... I'm hoping that after a month I will have a better feel for whats going on and can outline either a staged or complete rewrite.
Lots of great advice in here, has lifted my spirits a bit. Especially getting complete client buy in (which I have internalised but I guess haven't expressed, either to myself or the client).
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#68Earlier quoted context omitted.
" Everybody that says "rewrite" is dreaming. It is impossible to rewrite something that large without breaking something and spending too much money. Re-factoring a function is doable. Re-factoring a thousand-line file is doable. Re-factoring part of a database is doable. Re-factoring all of it all at once is starry-eyed fiction. Not gonna happen." I don't think so. Yes, the opportunities to do something like this ar…
Yes, I have often said that you should never, ever throw (significant amounts of) code away and start from scratch. I have worked on two bodies of legacy code before and more or less used most of the techniques discussed in the comments, but this is unprecented - for me - in both size and badness. Client and I have agreed that I will work for a month and then see where we stand... I'm hoping that after a month I will…
That's a good approach. Remember, it isn't your problem. It is your client's problem. You are there to help solve his problem. If he doesn't care enough you certainly shouldn't.
In a month you'll know a lot more about what you might be walking into. It is critical that you client also learn what he has to contend with. In other words: Communicate profusely throughout the process.
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#69I have actually worked on a code base like you're describing, on a contract basis, for a client. I loathe maintenance programming, so the relationship didn't last very long -- just a few months. So: 1. Make sure you have a rock-solid contract in place with the client that will ensure that you get paid, get paid well, and get paid often. Receiving a check in the mail makes it easier to look at the code. If your paymen…
" Everybody that says "rewrite" is dreaming. It is impossible to rewrite something that large without breaking something and spending too much money. Re-factoring a function is doable. Re-factoring a thousand-line file is doable. Re-factoring part of a database is doable. Re-factoring all of it all at once is starry-eyed fiction. Not gonna happen." I don't think so. Yes, the opportunities to do something like this ar…
Funny, I had been considering replying to your other comment in this thread advocating for the rewrite. I don't really think you and I would find that we have very different opinions if we could sit down and talk this out over tea.
For example, I agree that sometimes a rewrite is warranted, even necessary. I just don't think this is one of those times.
First, you have to think about this from the client's point of view. To borrow some of patio11's recent wisdom, this is, to them, a business problem, not a technical problem. You, I, OP, others are inclined to look at the code and tell the client, "You have a problem with your code." But, the client doesn't care if the code is beautiful or ugly. They care about just a few things: whether or not they can hire people to work on it, whether or not bugs can be fixed, and whether or not it does what they want it to do. Does it matter to them if the code produces HTML tables as output or stores called code in the database? Nope.
So when it comes to a rewrite, you have to sell that to them as a solution to a business problem that they have, and unlike your examples, that's much harder than pointing to defects on an assembly line or hemorrhaging money. Put yourself in the client's shoes: why should you pay for the software that you already have to be completely rewritten -- and in the end, if you're lucky, end up with the thing that from your point of view you already have -- instead of just hiring a different programmer who's willing to just keep his head down and fix the bugs?
Of course it's possible to make a bunch of arguments to the client that they are losing money on it and that it is a problem they need to address, and if you're really good at that kind of thing, maybe even convince them that a rewrite is worth their money. But it's hard to do, in practice.
Then we have to consider the costs involved. First, there's downtime. To get it done as expediently as possible, they'd have to give up any hope of having any bugs fixed for, what, the next six months at least? They'd be dealing with the very real business problem of angry customers in the meantime, complaining that the software is broken and it's not getting fixed in a timely manner.
Second, there's monetary cost. Another difference between big, ugly software and your examples is that software can be far, far worse in terms of the amount of effort required to understand it before changing it. That software didn't get to be a million lines of code overnight; it probably started out, in its infancy, as maybe a few tens of thousands piled on top of a bad framework. The rest is probably largely business logic edge-cases and lazy programming. There is almost certainly some really important business logic deeply hidden somewhere in that code; one way or another that business logic needs to be in whatever version of the software they're using. If a full rewrite is done, that means either the programmers read and fully understand the existing code before rewriting it, or they build the new version without that business logic and force the business to deal with a problem that they've already had and solved once before.
What would be the cost of reading and understanding every line of code, and then writing a new version? Conservatively, $1/line? Heck, assuming you could somehow side-step that whole problem, what would the cost be for an entirely new version? Still $1/line? And how much smaller could the new version be? 100,000 lines, a 90% reduction from the previous version? That's still awfully expensive.
So I think that a full rewrite for this project isn't realistic. It's what many of us would like to dream about -- taking something crappy, throwing it out, erasing it from the world, building something beautiful in its place -- that's psychological sugar. But it's not realistic.
But what can be done is to rewrite it piecemeal. It's like home improvement in that sense -- you don't move in to a home and then demolish it and raze the entire lot and build something from scratch and landscape it and everything. But, you can move in, and replace a water main one month, put in a sprinkler system another month, put sod in the following month, and so on. You can take a really ugly, overwhelming, huge project and break it down into manageable pieces and fix it one piece at a time.
I agree wholeheartedly that life is too short to work on shit projects, assuming of course that you have the money to be in a position to never work on a shit project. That's why I was pretty up-front about this maybe not being a good project to work on. (And also why I didn't stick with my client's project when I was presented with something really similar to OP's.)
Re: Ask HN: I just inherited 700K+ lines of bad PHP. Advice?
#70OK, so everyone else has pretty much covered the arguments for "don't do it, run" and for "re-write it". But assuming that you either
a) have to maintain it anyway (can't afford to lose job etc.)
or
b) are going to re-write it but don't have a definte spec to know what it has to do
then you are going to need to try and understand the code base. Here are some PHP specific tools to help you.
- Use XHGUI[1] (which is a fork of Facebooks XHProf) to profile the code as it runs. It can draw call-graphs for you (if you have Graphviz installed) which will help you to visuallise the code flow.
- Use PHPdoc[2] to generate API docs. This will help you get a simplified overview of the code to use as a reference.
- Use Xdebug[3] as you make changes and execute code to get more insight into how it is running and to trace variables etc. through the execution. You can use KCacheGrind [4] to visualise the output of Xdebug.
- Use a staging/development environment for everything you do with this code, and don't push any changes into to production until you really, really have to. When you do, use version control (e.g. Git, SVN etc.) and use an automated build system (Phing[5] is a great PHP specific one) to try and keep everything consistent.
Good luck! Quick plug : I'm currently writing a book [6] about PHP development (called PHP Everywhere : Programming beyond the web with PHP) which covers the tools above (albeit not for the kind of job you are taking on!). The one small mercy you may have when tackling a project like this is that it is written in PHP. PHP is usually quite a verbose language, which while it doesn't always produce sexy code, does mean that its straight forward to read and understand (at the local level!). An extra space here and there doesn't usually alter the meaning of the code as it does in some languages!
[1] https://github.com/preinheimer/xhprof [2] http://www.phpdoc.org/ [3] http://www.xdebug.org/ [4] http://kcachegrind.sourceforge.net [5] http://www.phing.info/ [6] http://leanpub.com/php