> Data structures, not algorithms, are central to programming I'm a big time leetcode interview hater and reading that felt validating. Why the f*ck am I always asked about algorithms.
[dead]
Rob Pike’s Rules of Programming (1989)
461–470 of 483 posts
Re: Rob Pike’s Rules of Programming (1989)
#462Earlier quoted context omitted.
I usually defer this until a PM does the research to highlight that speed is a burning issue. I find 98% of the time that users are clamoring to get something implemented or fixed which isnt speed related so I work on that instead. When I do drill down what I tend to find in the flame graphs is that your scope for making performance improvements a user will actually notice is bottlenecked primarily by I/O not by code…
The problem with ignoring performance is that you'll always end up with slow software that is awful to use but ticks all the feature boxes. As soon as someone comes along that is fast and nice people will switch to that. People don't ask for software to be fast and usable because it obviously should be. Why would they ask? They might complain when it's unusably slow. But that doesn't mean they don't want it to be fas…
The problem with saying "one must validate that speed is a problem before doing something about it" is that some people hear "one must ignore speed".
Users not being up front about their desires and needs is an argument for better research, not presuming on their behalf. It is true that they are not necessarily adept communicators.
Re: Rob Pike’s Rules of Programming (1989)
#463Earlier quoted context omitted.
It’s about coupling and being able to maintain that in the long term. A narrow focus helps to test each individual unit in isolation from each other. It is true that a database appears to be a single datastructure with hundreds of methods from the users perspective and that is fine, because someone else engineered and tested it for you. However if you were to look into how a database is implemented you would get to s…
It’s about coupling and being able to maintain that in the long term. What does that mean? This is all the kind of abstract programming advice that sounds nice until someone needs an example. A narrow focus helps to test each individual unit in isolation from each other. A function operating on a data structure is already a narrow focus. It is true that a database appears to be a single datastructure with hundreds of…
To put it simply if you have a function f(ctx) = ctx.a + ctx.b it is hard to see what are the arguments producing the output. Which are the elements in the datastructure you need to vary in order to have exhaustive tests. Whereas if one refactors it as f(ctx) = g(ctx.a, ctx.b) you only need to test function g with respect to (a, b) whereas forwarding of methods can be simply covered in integration tests without any care whether function g is implemented correctly.
To make such testing strategy to work data structures need to be small. It is better to have multiple small data structures rather than one big universal one where methods are defined at the ctx level making exhaustive tests difficult.
Perhaps, I haven’t been clear. I agree with Pike’s advice strongly. What I am trying to say here is that the Perl’s rule 9 is diametrically opposite of what Pike says.
Re: Rob Pike’s Rules of Programming (1989)
#464Earlier quoted context omitted.
Ignoring optimization opportunities until you see the profile only works when you actually profile! Profiling never achieved its place in most developers’ core loop the way that compiling, linting, or unit testing did. How many real CI/CD pipelines spit out flame graphs alongside test results?
I usually defer this until a PM does the research to highlight that speed is a burning issue. I find 98% of the time that users are clamoring to get something implemented or fixed which isnt speed related so I work on that instead. When I do drill down what I tend to find in the flame graphs is that your scope for making performance improvements a user will actually notice is bottlenecked primarily by I/O not by code…
> I usually defer this until a PM does the research to highlight that speed is a burning issue.
Does a carpenter wait until you get a splinter before pulling out the sandpaper?
We should be actively taking pride in our work, not churning out crap until someone notices.
Re: Rob Pike’s Rules of Programming (1989)
#465"Epigrams in Programming" by Alan J. Perlis has a lot more, if you like short snippets of wisdom :) https://www.cs.yale.edu/homes/perlis-alan/quotes.html > Rule 5. Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming. Always preferred Perlis' version, that might be slightly o…
With 100 functions and one datastructure it is almost as programming with a global variables where new instance is equivalent to a new process. Doesn’t seem like a good rule to follow.
In languages without generics, like C, people tend to build a custom list type that only holds their particular type. This leads to a proliferation of list types each with sparse feature support.
It is better to have one collection type and use it everywhere directly.
Some Java developers had a tendency to encapsulate collections in Manager classes that grant limited operations/access to the list type. In practice this doesn't really buy you anything and you could have had a regular List instead.
Re: Rob Pike’s Rules of Programming (1989)
#466Earlier quoted context omitted.
Was 3 years a long time for an indie platformer in the early 2000s? Looking at some similar examples: * Braid was 3 years * Cave Story was 5 years * World of Goo was 2 years * Limbo was about 3 years (but with 8-16 people) So Braid seems pretty average.
Wow! I'm shocked Limbo took 3 years and had a larger team. I've played a bit of the game (but didn't complete it). Looking at the Steam reviews and a quick search shows me it takes 3-10 hours to beat the game. :O
IIRC the Limbo team grew over time, it shows in gameplay, the second half feels less personal and more like a generic platformer. They hired more people to get done with it. It don't mean it is rushed, but to me, the second half lacks personality compared to the first half.
Also, I don't like the idea of using gameplay time as a value statement. Maybe I say that because I don't have money problems, but I find that the tendency certain gamers have of judging games in terms of dollars per hour of gameplay is pretty damaging, as it incentivizes developers to focus on gameplay time more than polish. 3-10 hours is already plenty for that style of game. Note that there are many AAA games in the 10 hour range.
Re: Rob Pike’s Rules of Programming (1989)
#467Earlier quoted context omitted.
Huh, I've always understood that quote very differently, with emphasis on "premature" ... not as in, "don't optimize" but more as in "don't optimize before you've understood the problem" ... or, as a CS professor of mine said "Make it work first, THEN make it work fast" ...
And if you know in advance that a function will be in the critical path, and it needs to perform some operation on N items, and N will be large, it’s not premature to consider the speed of that loop.
Charles Eames said that "design depends largely on constraints".
If you know in advance, then one of the constraints on the design of that function is that it's on the critical path and that it has to meet the specification.
You're stating the obvious that you need to consider its implementation.
But that's not the same as "I have a function that is not on the critical path and the performance constraint is not the most important, but I'll still spend time on optimizing it instead of making it clear and easy to understand."
Re: Rob Pike’s Rules of Programming (1989)
#468[flagged]
> In practice what I see fail most often is not premature optimization but premature abstraction This matches my experience as well. Someone here commented once that abstractions should be emergent, not speculative, and I loved that line so much I use it with my team all the time now when I see the craziness starting.
> abstractions should be emergent, not speculative
Dang, I love that line, too. It is 100% correct, and the opposite of what OOP teaches.
Re: Rob Pike’s Rules of Programming (1989)
#469Earlier quoted context omitted.
It’s about coupling and being able to maintain that in the long term. What does that mean? This is all the kind of abstract programming advice that sounds nice until someone needs an example. A narrow focus helps to test each individual unit in isolation from each other. A function operating on a data structure is already a narrow focus. It is true that a database appears to be a single datastructure with hundreds of…
To put it simply if you have a function f(ctx) = ctx.a + ctx.b it is hard to see what are the arguments producing the output. Which are the elements in the datastructure you need to vary in order to have exhaustive tests. Whereas if one refactors it as f(ctx) = g(ctx.a, ctx.b) you only need to test function g with respect to (a, b) whereas forwarding of methods can be simply covered in integration tests without any c…
To make such testing strategy to work data structures need to be small.
Why would that be true?
It is better to have multiple small data structures rather than one big universal one where methods are defined at the ctx level making exhaustive tests difficult.
I don't think you are backing this up at all, you just keep saying it over and over. It's also not even about big data structures it's about having fewer data structures and using them over and over. You can see where this is effective even in javascript and lua with their tables that have hash maps and arrays.
Re: Rob Pike’s Rules of Programming (1989)
#470Earlier quoted context omitted.
I only agree if you have a bounded dataset size that you know will never grow. If it can grow in future (and if you're not sure, you should assume it can), not only will many data structures and algorithms scale poorly along the way, but they will grow to dominate the bottleneck as well. By the time it no longer meets requirements and you get a trouble ticket, you're now under time pressure to develop, qualify, and d…
> You're much more likely to encounter regressions when doing this under time pressure. There is nothing to suggest you should wait to optimize under pressure, only that you should optimize only after you have measured. Benchmark tests are still best written during the development cycle, not while running hot in production. Starting with the naive solution helps quickly ensure that your API is sensible and that your…
Also there's no reason why a horizontally scalable service can't be stress tested with 2x or 10x of prod load in non-prod environments.