Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a tool to get instant test results

github.com

Post reply on HN
Print view View original

51–60 of 68 posts

Re: Show HN: I built a tool to get instant test results

#51

Post by yardstick »

> hypertest is an open-source CLI tool designed to help you maintain focus on the current coding subtask, built specifically for the easily distracted.

I don’t quite get this proposition. If easily distracted, and wanting to stay focused, wouldn’t you rather want to run tests only on demand and not be interrupted by failing tests after changing just a line or two and still not finished with the rest of your changes.

I can see the benefits of a fast feedback loop. But not sure the pitch of keeping you focused is how i would see it work for me. YMMV.

Re: Show HN: I built a tool to get instant test results

#52

Post by TeMPOraL »

marginalia_nu wrote: post #49
jasamer wrote: post #17

Earlier quoted context omitted.

This tool doesn't run all the tests, only the ones affected by the code changes since the last run. It figures out which tests to run by initially running all tests, and storing the code coverage of each test. It doesn't need to rely on "test change likelihood" - if a code change is outside of the code coverage of the test, it doesn't affect the test.

> If a code change is outside of the code coverage of the test, it doesn't affect the test. This assertion questionable, the extent it is true it is language specific. Here is a counterexample in Java: // foo.java public class Foo { public static final int LENGTH = 10; } // bar.java public class Bar { public int getFooLength() { return Foo.LENGTH; } } Code coverage tools will not highlight Foo as being touched when B…

You'd want to augment the code analysis with the dependency graph of object/class files (or use it instead of what the OP is using). I'm not sure if it's bullet-proof, but except for runtime dynamic linking, you should get a superset of all affected changes if you just use... whatever your IDE uses to determine which files need to be rebuild when you make a change. Following that graph should give you a superset of all unit tests that need to be run.

Like, e.g. if I change a macro in a header file and press "rebuild" on the project, MSVC (or MSBuild driven by CMake) will figure out that the header was changed, chase down which translation units include it directly or transitively, and rebuild those, then link the output and... chase down everyone else who links to the output and relink them, etc.

I bet you could produce a counterexample that breaks this mechanism (C++ being what it is), but I don't expect to see it in an actual codebase.

Re: Show HN: I built a tool to get instant test results

#53

Post by grncdr »

twic wrote: post #38

This sounds a lot like a couple of old JUnit test runners: InfiniTest - https://infinitest.github.io/ an d https://infinitest.github.io/doc/intellij#how-it-works JUnit Max - https://web.archive.org/web/20090206151635/http://www.threer... partially surviving as https://junit.org/junit4/javadoc/4.12/org/junit/experimental... Ruby has something similar: autotest - https://github.com/grosser/autotest I think it's a good…

Ruby also has crystalball (https://toptal.github.io/crystalball/) which is similar, but not quite the same.

Re: Show HN: I built a tool to get instant test results

#54

Post by bagels »

yardstick wrote: post #51

> hypertest is an open-source CLI tool designed to help you maintain focus on the current coding subtask, built specifically for the easily distracted. I don’t quite get this proposition. If easily distracted, and wanting to stay focused, wouldn’t you rather want to run tests only on demand and not be interrupted by failing tests after changing just a line or two and still not finished with the rest of your changes.…

If you are already running tests after every save, and watching for the outcome, but the test takes too long, you might get distracted waiting for the test

Re: Show HN: I built a tool to get instant test results

#55

Post by _cfl0 »

TeMPOraL wrote: post #52
marginalia_nu wrote: post #49

Earlier quoted context omitted.

> If a code change is outside of the code coverage of the test, it doesn't affect the test. This assertion questionable, the extent it is true it is language specific. Here is a counterexample in Java: // foo.java public class Foo { public static final int LENGTH = 10; } // bar.java public class Bar { public int getFooLength() { return Foo.LENGTH; } } Code coverage tools will not highlight Foo as being touched when B…

You'd want to augment the code analysis with the dependency graph of object/class files (or use it instead of what the OP is using). I'm not sure if it's bullet-proof, but except for runtime dynamic linking, you should get a superset of all affected changes if you just use... whatever your IDE uses to determine which files need to be rebuild when you make a change. Following that graph should give you a superset of a…

Not bulletproof, but a-ok if your running a full run just before the commit.

Re: Show HN: I built a tool to get instant test results

#56

Post by _cfl0 »

gjadi wrote: post #34
_cfl0 wrote: post #15

Earlier quoted context omitted.

Love that mantra! Dont really agree, it’s not intended for that. I use it as I would run tests through a terminal normally. Built this because I hate running tests.

Got it. Very much like the idea to check the code coverage to find out which tests you should run. First time I saw it. Is it common in other tools?

I also have a CI solution I’m launching soon.

Other than that, most of this tech was forgotten in the late 90s early 2000s

Re: Show HN: I built a tool to get instant test results

#57

Post by [deleted] »

marginalia_nu wrote: post #49
jasamer wrote: post #17

Earlier quoted context omitted.

This tool doesn't run all the tests, only the ones affected by the code changes since the last run. It figures out which tests to run by initially running all tests, and storing the code coverage of each test. It doesn't need to rely on "test change likelihood" - if a code change is outside of the code coverage of the test, it doesn't affect the test.

> If a code change is outside of the code coverage of the test, it doesn't affect the test. This assertion questionable, the extent it is true it is language specific. Here is a counterexample in Java: // foo.java public class Foo { public static final int LENGTH = 10; } // bar.java public class Bar { public int getFooLength() { return Foo.LENGTH; } } Code coverage tools will not highlight Foo as being touched when B…

[deleted]

Re: Show HN: I built a tool to get instant test results

#58

Post by asdev »

jwhiles wrote: post #6

I'm a bit confused how this differs from the existing watch modes that are built into test runners I'm familiar with, such as Jest. Is this something that I'm used to in TypeScript, but which just doesn't exist in most language's ecosystems?

I'm confused too. You can add regex patterns for both filenames and test names in Jest to run only the tests you want in watch mode. This tool just makes it so you don't have to input that configuration? Doesn't seem like a huge value add.

Re: Show HN: I built a tool to get instant test results

#59

Post by _cfl0 »

Cthulhu_ wrote: post #41
twic wrote: post #38

This sounds a lot like a couple of old JUnit test runners: InfiniTest - https://infinitest.github.io/ an d https://infinitest.github.io/doc/intellij#how-it-works JUnit Max - https://web.archive.org/web/20090206151635/http://www.threer... partially surviving as https://junit.org/junit4/javadoc/4.12/org/junit/experimental... Ruby has something similar: autotest - https://github.com/grosser/autotest I think it's a good…

Jest, probably the most popular JS test runner of today, also has had it since forever.

Had what since forever? Does jest analyze the changes you make and compare them to code coverage? (hint: no)

The key innovation piece here is the speed that the extreme selectively enables you.

Re: Show HN: I built a tool to get instant test results

#60

Post by _cfl0 »

solardev wrote: post #48
Cthulhu_ wrote: post #41

Earlier quoted context omitted.

Jest, probably the most popular JS test runner of today, also has had it since forever.

My Jest can run on only changed files, but it still takes forever. Much longer than 250ms

Right.. Because out of a 100 tests it will run close to that number. when you change code most of the time only a handful of tests or less need to run.
Post reply on HN