Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a tool to get instant test results

github.com

Post reply on HN
Print view View original

41–50 of 68 posts

Re: Show HN: I built a tool to get instant test results

#41

Post by Cthulhu_ »

twic wrote: post #38

This sounds a lot like a couple of old JUnit test runners: InfiniTest - https://infinitest.github.io/ an d https://infinitest.github.io/doc/intellij#how-it-works JUnit Max - https://web.archive.org/web/20090206151635/http://www.threer... partially surviving as https://junit.org/junit4/javadoc/4.12/org/junit/experimental... Ruby has something similar: autotest - https://github.com/grosser/autotest I think it's a good…

Jest, probably the most popular JS test runner of today, also has had it since forever.

Re: Show HN: I built a tool to get instant test results

#42

Post by Cthulhu_ »

_cfl0 wrote: post #9
jwhiles wrote: post #6

I'm a bit confused how this differs from the existing watch modes that are built into test runners I'm familiar with, such as Jest. Is this something that I'm used to in TypeScript, but which just doesn't exist in most language's ecosystems?

Well, I would say the biggest difference is while the watch mode you described runs all the tests for every change. hypertest only runs the tests the were impacted by the changed code. Resulting is blazing fast performance. I’d say it’s even fast than apples alt+tab animation most of the time.

> Well, I would say the biggest difference is while the watch mode you described runs all the tests for every change.

This is not true though. Might depend on the test tooling used, but most of the ones I've used only run changed tests (either directly or through change detection via git, e.g. https://nx.dev/concepts/affected)

Re: Show HN: I built a tool to get instant test results

#44

Post by funcDropShadow »

_cfl0 wrote: post #31
volkk wrote: post #24

Earlier quoted context omitted.

> hypertest only runs the tests the were impacted by the changed code with all due respect, this is also a solved problem with test watchers. unless you mean this is smart enough to not just run unit tests that are directly related to the file changed, but also understands code changes from the perspective of downstream dependencies?

The second part is the aim here. This is more of a code watcher than a test watcher specifically.

sbt ~test uses the dependency analysis of the scala compiler to determine which tests to run.

Re: Show HN: I built a tool to get instant test results

#46

Post by lupire »

Cthulhu_ wrote: post #42
_cfl0 wrote: post #9

Earlier quoted context omitted.

Well, I would say the biggest difference is while the watch mode you described runs all the tests for every change. hypertest only runs the tests the were impacted by the changed code. Resulting is blazing fast performance. I’d say it’s even fast than apples alt+tab animation most of the time.

> Well, I would say the biggest difference is while the watch mode you described runs all the tests for every change. This is not true though. Might depend on the test tooling used, but most of the ones I've used only run changed tests (either directly or through change detection via git, e.g. https://nx.dev/concepts/affected )

Let's say you have 100 tests that depend on the file being changed, but only 10 that depend on the method or line changed since last year run.

How many tests will your system run? That's the important question.

Re: Show HN: I built a tool to get instant test results

#47

Post by lupire »

funcDropShadow wrote: post #44
_cfl0 wrote: post #31

Earlier quoted context omitted.

The second part is the aim here. This is more of a code watcher than a test watcher specifically.

sbt ~test uses the dependency analysis of the scala compiler to determine which tests to run.

Are you sure that's actually true, all the way down to the method or line level?

https://www.scala-sbt.org/1.x/docs/Testing.html

Re: Show HN: I built a tool to get instant test results

#48

Post by solardev »

Cthulhu_ wrote: post #41
twic wrote: post #38

This sounds a lot like a couple of old JUnit test runners: InfiniTest - https://infinitest.github.io/ an d https://infinitest.github.io/doc/intellij#how-it-works JUnit Max - https://web.archive.org/web/20090206151635/http://www.threer... partially surviving as https://junit.org/junit4/javadoc/4.12/org/junit/experimental... Ruby has something similar: autotest - https://github.com/grosser/autotest I think it's a good…

Jest, probably the most popular JS test runner of today, also has had it since forever.

My Jest can run on only changed files, but it still takes forever. Much longer than 250ms

Re: Show HN: I built a tool to get instant test results

#49

Post by marginalia_nu »

jasamer wrote: post #17
daveguy wrote: post #10

I like the concept and it seems that "run them all and see what changed" is a great model to avoid missing any side effects from a code change. However, if you're running all the tests on every change, doesn't that mean the responsiveness is necessarily greater than 250ms if your test suite is longer than 250ms? Regardless, running tests in the background to eliminate human delay is good. It would be nice if the (~te…

This tool doesn't run all the tests, only the ones affected by the code changes since the last run. It figures out which tests to run by initially running all tests, and storing the code coverage of each test. It doesn't need to rely on "test change likelihood" - if a code change is outside of the code coverage of the test, it doesn't affect the test.

> If a code change is outside of the code coverage of the test, it doesn't affect the test.

This assertion questionable, the extent it is true it is language specific.

Here is a counterexample in Java:

  // foo.java 
  public class Foo {
    public static final int LENGTH = 10;
  }

  // bar.java 
  public class Bar {
    public int getFooLength() {
      return Foo.LENGTH;
    }
  }
Code coverage tools will not highlight Foo as being touched when Bar.getFooLength is invoked, even though a test asserting that getFooLength() is 10 will fail or succeed depending on Foo.LENGTH. The reason for this is that Foo.LENGTH is inlined in Bar. If you for example wanted to patch a new version of the value, you need to supply Bar.class to affect the change; patching Foo alone would do nothing.

C and C++ likewise do not satisfy this requirement because macros.

Re: Show HN: I built a tool to get instant test results

#50

Post by theptip »

“Test impact analysis” is one name for this general approach. There are a few projects doing this in the Python ecosystem that don’t fully work (Smother, pytest-tia, and some others). Great to see another entry.

Can you cache the coverage DB so this can be run in a CI pipeline?

Post reply on HN