Love using tester army to validate PRs against my preview environment. Skips the manual check much of the time and helps me ship more confidently.
Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
41–50 of 87 posts
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#42Great presentation On a slight tangent, since we are all here... Does anyone still believe there is a long-term future in traditional UI/UX? It feels like a lot of attention is still going into landing pages, dashboards, and CRUD apps, while overlooking a bigger shift where fewer people will actually need to interact with those interfaces directly when the same tools can perform the underlying tasks automatically, wi…
How can it perform tasks automatically? It's not magic, there has to be an UI/UX for interacting with it. Will that UI/UX be more optimized and easier to use is the question. Like would you prefer saying "close window computer" or press alt+f4 or just click on the little cross thing or equivalent. Why are we assuming all AI automagic UI/UX will be better for all tasks?
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#43Love using tester army to validate PRs against my preview environment. Skips the manual check much of the time and helps me ship more confidently.
Happy to hear that!
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#44Does it support testing on all Apple platforms (macOS, iOS, iPadOS, watchOS, tvOS, and visionOS)?
Currently, only iOS, but we can add iPadOS too!
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#45Great presentation On a slight tangent, since we are all here... Does anyone still believe there is a long-term future in traditional UI/UX? It feels like a lot of attention is still going into landing pages, dashboards, and CRUD apps, while overlooking a bigger shift where fewer people will actually need to interact with those interfaces directly when the same tools can perform the underlying tasks automatically, wi…
How can it perform tasks automatically? It's not magic, there has to be an UI/UX for interacting with it. Will that UI/UX be more optimized and easier to use is the question. Like would you prefer saying "close window computer" or press alt+f4 or just click on the little cross thing or equivalent. Why are we assuming all AI automagic UI/UX will be better for all tasks?
AI agents can perfectly do a lot of the data entry tasks and build dashboards. You practically need to build none of that when you can ask an AI agent to pull the data and build a chart or provide a file or a paste to insert into a database.
Basically that.
If the app requires a mouse then it should have UI, if not, unless critical, it can be driven by an agent.
That's my point.
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#46"Traditional E2E tests are slow to set up and expensive to maintain." I don't really understand this. If I'm already using Opus to write the code, surely it would know best what E2E tests to write to be able to verify its own output? This seems like an unnecessary external step.
Unfortunately from our experience tests don’t scale as well as code. First of all static tests are very brittle, you rely on selectors, need wait times and can’t really test a lot of dynamic content (think AI chats/interactions). Then it’s all the infrastructure around it: solving captchas, handling auth, handling email OTP (each of our agents has access to its own inbox) and handling video recording and screenshots.…
I would respectfully disagree on this. How i write tests right now I ask claude/codex to create an eval and it just spins up a bg LLM agent worker which verifies the tests in the sandbox/internally.
So i would say that atm in house testing is easier than external testing for us
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#47Hey, I just gave it a try and ran a quick test on booking.com. It took ~3 mins for a basic test. Do you cache the test steps so that future runs are faster and they don't call LLMs for the subsequent runs?
Also your current pricing is $300 for 1K tests which means $0.3 for each test. We tried out playwright mcp and it easily consumes 1M+ tokens for a test with ~20 steps (including image input). So with this pricing are you guys default alive?
Also is there a benchmark which you ran to prove the efficacy of your testing agent? because in the current stage it is a trust me bro kinda thing.
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#48Some digging
FAST_MODEL = "google/gemini-3-flash" (fast mode primary)
DEEP_MODEL = "openai/gpt-5.4" (deep mode primary)
VISION_CLICK_MODEL= "openai/gpt-5.4" (the visual grounder)
fast: gemini-3-flash, falls back to gpt-5.4, 15-min run timeout, max 2 visual calls/step. deep: gpt-5.4, 15-min timeout, max 3 visual calls/step.
Why such a hard timeout, and why not latest models?
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#49Hey, I just gave it a try and ran a quick test on booking.com. It took ~3 mins for a basic test. Do you cache the test steps so that future runs are faster and they don't call LLMs for the subsequent runs? Also your current pricing is $300 for 1K tests which means $0.3 for each test. We tried out playwright mcp and it easily consumes 1M+ tokens for a test with ~20 steps (including image input). So with this pricing a…
We've been doing quite a lot of context engineering and optimizations to make sure it's not as expensive. The subsequent runs are faster because we cache the trajectory of the agent (not the whole test run yet, as we want to keep the agent in the loop, more like a manual QA engineer, not a test script).
We currently do not have any benchmarks; much of the experience depends on the test plan. We've been mostly focusing on the customer experience not benchmarking.
Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps
#50Some digging FAST_MODEL = "google/gemini-3-flash" (fast mode primary) DEEP_MODEL = "openai/gpt-5.4" (deep mode primary) VISION_CLICK_MODEL= "openai/gpt-5.4" (the visual grounder) fast: gemini-3-flash, falls back to gpt-5.4, 15-min run timeout, max 2 visual calls/step. deep: gpt-5.4, 15-min timeout, max 3 visual calls/step. Why such a hard timeout, and why not latest models?
We found gemini 3 flash to be the best model as of now, when it comes to bang for the buck, GPT 5.5 is also a bit more expensive than 5.4, if we run tests at scale it has to be affordable. Once a newer model that fits into the criteria is released, we will update it!