Test Anything Protocol
testanything.org
Test Anything Protocol
1–10 of 31 posts
Re: Test Anything Protocol
#2The sharness[1] project originated from the Git project and is a really great and easy way to write tests using shell. Its output is TAP-compatible, so you can use `prove` to verify sharness tests.
Re: Test Anything Protocol
#3[deleted]
Re: Test Anything Protocol
#4I looked at TAP when I had to implement a testing tool for my own language. In the end, I didn't really see the point of using TAP. Is anyone using it outside Perl? What are its specific advantages compared to using common testing libraries?
Re: Test Anything Protocol
#5I prefer Cram [1] or mdx[2]
Re: Test Anything Protocol
#6TAP is fantastic for testing your Postgres database (indexes, views, functions, triggers, etc.) without depending on any specific client language: https://pgtap.org/
Re: Test Anything Protocol
#7I looked at TAP when I had to implement a testing tool for my own language. In the end, I didn't really see the point of using TAP. Is anyone using it outside Perl? What are its specific advantages compared to using common testing libraries?
Same motivation as JUnit, I guess: it's a common protocol that different testing tools can all agree to speak/understand.
Re: Test Anything Protocol
#8I looked at TAP when I had to implement a testing tool for my own language. In the end, I didn't really see the point of using TAP. Is anyone using it outside Perl? What are its specific advantages compared to using common testing libraries?
> Is anyone using it outside Perl?
I first saw it used with JavaScript, so I assume yes; but the site itself answers this, too. (specifically the producers and consumers pages.)
I think it is of limited use if you can use any of the more sophisticated defacto standards like JUnit compatible output. Still, you can imagine scenarios where a stupidly simple test protocol would be nice. One fairly esoteric use case might be test ROMs for CPU emulators; today many of them output results to serial in some format, but AFAIK there is no unification of the format at all, across different test ROMs.
Re: Test Anything Protocol
#9I like it a lot, and am surprised I didn't know about it considering I've been looking for a dead-simple git + Makefile + shell script + (chroot or Docker-like) container + simplistic topsort-prio'd shell runner pool + minimal Web UI solution for CI for some time now, of which this could totally be an adequate part to comprehend and render test reports. As a protocol, it might benefit from some enhancements, such as capturing and linking to detailed test result output files for failed tests.
Re: Test Anything Protocol
#10I looked into TAP a while ago, but had a couple of problems with it:
1) The whole point is to allow language/harness agnostic tooling, but the only tool of note seems to be smoulder. Everything else seems to be trivial things like showing pass/fail with emoji tick/cross symbols, etc.
2) The TAP standard allows tests to be numbered, and a total count to be given up-front. In practice, all tools seem to require this, and barf when given unnumbered tests. This makes TAP far less useful, since it requires some global coordinating process to know how many tests we're going to run.
I was interested in TAP since I could write test scripts in any language which spit out a simple format to stdout, and they could be nested; e.g. we can have an overall "test everything" script which runs the test script for each of our projects/repos; projects might choose to have different scripts for unit tests, functional tests, integration tests, etc. and those tests might be spread across many files. Ideally we shouldn't have to care about any of that structure. With TAP we need some way to count the tests before running them (or else we have to wait for all of the tests to finish first, which is naff); this requires some sort of declarative structure, at which point our scripts can no longer be opaque black-boxes, so we might as well use a "proper" test harness like xUnit.