Meta-habit: learn to adopt different habits for different situations. With that in mind, some techniques I've found useful for various situations: "Researchey" green-field development for data-science-like problems: 1. If it can be done manually first, do it manually. You'll gain an intuition for how you might approach it. 2. Collect examples. Start with a spreadsheet of data that highlights the data you have availab…
Great advice. One nit to pick: > Don't ignore constant factors. Sometimes an algorithm with asymptotically worse performance will perform better in practice because it has much better cache locality. Forget the cache, sometimes they're just plain faster (edit in response to comment: I mean faster for your use case ). I've e.g. found that convolutions can be much faster with the naive algorithm than with an FFT in a p…
> sometimes they're just plain faster
Not faster for sufficiently large N (by definition).
But your general point is correct.
I've best seen this expressed in Rob Pike's 5 Rules of Programming [0], Rule 3:
Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy algorithms have big constants. Until you know that n is frequently going to be big, don't get fancy.