Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing is better than data structures and algorithms

nedbatchelder.com

Post reply on HN
Print view View original

61–70 of 178 posts

Re: Testing is better than data structures and algorithms

#61

Post by hinkley »

JackSlateur wrote: post #53

I think the author is mislead Let's grab a simple use case: some basic CRUD http API. Easy, you say, no need to know fancy stuff ! Just test it and that's all. You do your test, all good, you can roll in production ! But sadly, in production, you have multiple users (what an idea ..). Suddenly, your CRUD api has become a concurrent system. Suddenly, you have data corruption, because you never thought about anything a…

Mediocre testing can also lead to a situation where there is friction for improvement because the tests are brittle and coupled (with each other and the misfeature you’re interested in fixing).

I like a more uniform distribution in my testing efforts. Start earlier, end later than most, and it’s experiences like this that inform that preference. And also production bugs in code with supposed 100% test coverage.

Re: Testing is better than data structures and algorithms

#62

Post by wubrr »

JackSlateur wrote: post #59
wubrr wrote: post #57

Earlier quoted context omitted.

I mean, in your example you just have an incomplete test suite. (Though writing a complete one is often unrealistic) While understanding algorithms and data structures is important, the only way you really know how well it works, and how well it's implemented is by thoroughly testing it. There are an infinite amount of clever algorithms out there with terrible implementations. You need both.

Testing concurrency is extremely hard For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Or is it because one query ran 10ms earlier and, thus, you avoided the issue ? I truly wonder if there is real world tests around this; I bet there is only algorithm and fuzzing;

> Testing concurrency is extremely hard

Writing a non-trivial concurrent system based on your understanding of the 'algorithm' , without relying on testing is much harder.

> I truly wonder if there is real world tests around this

Of course there are. There are many tools, methods, and test suites out there for concurrency testing, for almost any major language out there. Of course, understanding your algorithm, and the systems involved is required to write a proper test suite.

> For instance, get sql queries; You ran them, and you have no issue; Is your code sane ?

Take those queries and run them 1000x+ times concurrently in a loop. That will catch most common issues. If you want to go a step further you can build a setup to execute your queries in any desired order.

Re: Testing is better than data structures and algorithms

#63

Post by sarchertech »

hinkley wrote: post #61
JackSlateur wrote: post #53

I think the author is mislead Let's grab a simple use case: some basic CRUD http API. Easy, you say, no need to know fancy stuff ! Just test it and that's all. You do your test, all good, you can roll in production ! But sadly, in production, you have multiple users (what an idea ..). Suddenly, your CRUD api has become a concurrent system. Suddenly, you have data corruption, because you never thought about anything a…

Mediocre testing can also lead to a situation where there is friction for improvement because the tests are brittle and coupled (with each other and the misfeature you’re interested in fixing). I like a more uniform distribution in my testing efforts. Start earlier, end later than most, and it’s experiences like this that inform that preference. And also production bugs in code with supposed 100% test coverage.

> Mediocre testing can also lead to a situation where there is friction for improvement because the tests are brittle and coupled

This is very very common among inexperienced devs and in immature organizations that think that more tests necessarily means better.

Re: Testing is better than data structures and algorithms

#64

Post by jkhdigital »

jerf wrote: post #23

This is one of the things I'd tune in the current curriculum. When I went to college in the late 1990s, we were right on the verge of a major transition to DSAs being something every programmer would implement themselves to something that you just pick up out of your libraries. So it makes sense that we would have some pretty heavy-duty labs on implementing very basic data structures. That said, I escaped into the dy…

About 20 years ago I failed out of the undergrad CS program at UIUC because I thought I was smart enough to skip most lectures. I did manage to get an A in the C++ Data Structures course because the lectures were recorded and I just binged them all the night before each test.

Anyways, now I’m a full-time lecturer teaching undergraduate CS courses (long story) and I’m actually shaping curriculum. As soon as I read this article I thought “I need to tell my data structures students to read this” because it echoes a lot of what I’ve been saying in class.

Case in point: right after two lectures covering the ArrayList versus LinkedList implementations of the Java List interface, I spent an entire lecture on JUnit and live-coded a performance test suite that produced actual data to back up our discussions of big-O complexity. The best part of all? They learned about JIT compilation in the JVM firsthand because it completely blew apart the expected test results.

Re: Testing is better than data structures and algorithms

#65

Post by themafia »

JackSlateur wrote: post #53

I think the author is mislead Let's grab a simple use case: some basic CRUD http API. Easy, you say, no need to know fancy stuff ! Just test it and that's all. You do your test, all good, you can roll in production ! But sadly, in production, you have multiple users (what an idea ..). Suddenly, your CRUD api has become a concurrent system. Suddenly, you have data corruption, because you never thought about anything a…

Macro Pierre White says "perfection is lots of little things done well."

Which is something I've always agreed with, so, I never understand articles that seek to eschew an important part of releasing software because they believe their approach elsewhere is enough to overcome these intentionally suboptimal choices.

Re: Testing is better than data structures and algorithms

#66

Post by IvyMike »

Always gonna have to side with Peter Norvig on this one: https://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-...

> They said, “Look at the contrast—here’s Norvig’s Sudoku thing and then there’s this other guy, whose name I’ve forgotten, one of these test-driven design gurus. He starts off and he says, “Well, I’m going to do Sudoku and I’m going to have this class and first thing I’m going to do is write a bunch of tests.” But then he never got anywhere. He had five different blog posts and in each one he wrote a little bit more and wrote lots of tests but he never got anything working because he didn’t know how to solve the problem. I actually knew—from AI—that, well, there’s this field of constraint propagation—I know how that works. There’s this field of recursive search—I know how that works. And I could see, right from the start, you put these two together, and you could solve this Sudoku thing. He didn’t know that so he was sort of blundering in the dark even though all his code “worked” because he had all these test cases.

Re: Testing is better than data structures and algorithms

#67

Post by IvyMike »

IvyMike wrote: post #66

Always gonna have to side with Peter Norvig on this one: https://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-... > They said, “Look at the contrast—here’s Norvig’s Sudoku thing and then there’s this other guy, whose name I’ve forgotten, one of these test-driven design gurus. He starts off and he says, “Well, I’m going to do Sudoku and I’m going to have this class and first thing I’m going to do is write a bun…

More context, from an earlier HN comment: https://news.ycombinator.com/item?id=3033446

Re: Testing is better than data structures and algorithms

#68

Post by sarchertech »

wubrr wrote: post #62
JackSlateur wrote: post #59

Earlier quoted context omitted.

Testing concurrency is extremely hard For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Or is it because one query ran 10ms earlier and, thus, you avoided the issue ? I truly wonder if there is real world tests around this; I bet there is only algorithm and fuzzing;

> Testing concurrency is extremely hard Writing a non-trivial concurrent system based on your understanding of the 'algorithm' , without relying on testing is much harder. > I truly wonder if there is real world tests around this Of course there are. There are many tools, methods, and test suites out there for concurrency testing, for almost any major language out there. Of course, understanding your algorithm, and t…

I’ve never worked somewhere (in 20 years from big tech companies to small startups) that was generally and reliably testing for concurrency bugs.

And I’ve seen dozens of bugs caused by people assuming that transactions (with the default isolation level) protect against race conditions.

Re: Testing is better than data structures and algorithms

#69

Post by matheusmoreira »

hvb2 wrote: post #6

This feels backwards. When you have a good understanding of data structures you have the luxury of testing. If you focus on testing over data structures, you might end up testing something that you didn't need to test because you used the wrong data structures. IMHO too often people dont consider big O because it works fine with their 10 row test case.... And then it grinds to a halt when given a real problem

> too often people dont consider big O because it works fine with their 10 row test case.... And then it grinds to a halt when given a real problem

The reverse also happens frustratingly often. One could spend a lot of time obsessing over theoretical complexity only for it to amount to nothing. One might carefully choose a data structure and algorithm based on these theoretical properties and discover that in practice they get smoked by dumb contiguous arrays just because they fit in caches.

The sad fact is the sheer brute force of modern processors is often enough in the vast majority of cases so long as people avoid accidentally making things quadratic.

Sometimes people don't even do that and we get things such as the GTA5 dumpster fire.

https://news.ycombinator.com/item?id=26296339

Re: Testing is better than data structures and algorithms

#70

Post by JackSlateur »

wubrr wrote: post #62
JackSlateur wrote: post #59

Earlier quoted context omitted.

Testing concurrency is extremely hard For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Or is it because one query ran 10ms earlier and, thus, you avoided the issue ? I truly wonder if there is real world tests around this; I bet there is only algorithm and fuzzing;

> Testing concurrency is extremely hard Writing a non-trivial concurrent system based on your understanding of the 'algorithm' , without relying on testing is much harder. > I truly wonder if there is real world tests around this Of course there are. There are many tools, methods, and test suites out there for concurrency testing, for almost any major language out there. Of course, understanding your algorithm, and t…

Running 1000x queries in a loop is called luck.
Post reply on HN