Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps

tester.army

Post reply on HN
Print view View original

31–40 of 87 posts

Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps

#31

Post by skinfaxi »

okwasniewski wrote: post #30
zuzululu wrote: post #25

not sure the pain point you mentioned resonate. with LLMs its very easy to do E2E testing. also I feel uneasy about outsourcing this part with all the security issues these days.

Unfortunately from our experience tests don’t scale as well as code. First of all, static tests are very brittle: you rely on selectors, need wait times, and can’t really test a lot of dynamic content (think AI chats/interactions). Then it’s all the infrastructure around it: solving captchas, handling auth, handling email OTP (each of our agents has access to its own inbox), spinning up simulators and handling video…

Goodness I really didn't expect such lazy copy-pasting of responses for a YC company.

Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps

#32

Post by okwasniewski »

tootubular wrote: post #28
okwasniewski wrote: post #10

Earlier quoted context omitted.

We are thinking whether to change this.. We also have testerarmy.com/.ai

You have the .com? That's a no-brainer imo. I have a domain for a saas where the .com is squatted so I settled for .ai (and other surrounding TLDs / host permutations) and right out of the gate ran into some issues with firewall vendors in corpo environments.

Yeah, we have all of them. I saw it too where in bigger companies our emails were going straight to spam. Will migrate to it soon

Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps

#33

Post by ai_slop_hater »

okwasniewski wrote: post #21
poisonborz wrote: post #18

E2E tests are now quick to write due to LLMs, and are then deterministic AND cheap to run. How would this compare to the token costs of running an agent the whole time for each test? How do you make sure results stay stable regardless of the nondeterministic nature? Do customers still need to create test cases - any way to import from test case management system - based on which they could have already generate e2e t…

Unfortunately from our experience tests don’t scale as well as code. First of all, static tests are very brittle: you rely on selectors, need wait times, and can’t really test a lot of dynamic content (think AI chats/interactions). Then it’s all the infrastructure around it: solving captchas, handling auth, handling email OTP (each of our agents has access to its own inbox) and handling video recording and screenshot…

Why can't you test AI chats?

Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps

#35

Post by _pdp_ »

Great presentation

On a slight tangent, since we are all here...

Does anyone still believe there is a long-term future in traditional UI/UX?

It feels like a lot of attention is still going into landing pages, dashboards, and CRUD apps, while overlooking a bigger shift where fewer people will actually need to interact with those interfaces directly when the same tools can perform the underlying tasks automatically, without much UI at all.

So the bigger question is does UI/UX evolve into something else, or does a large part of it simply disappear?

I might be a bit too early. Recently I started a project and decided to skip all of that and focus to make it more friendly to AI agents and frankly so far it has been great purely from user experience but also what it delivers.

Re: Launch HN: TesterArmy (YC P26) – Agents that test web and mobile apps

#39

Post by [deleted] »

okwasniewski wrote: post #21
poisonborz wrote: post #18

E2E tests are now quick to write due to LLMs, and are then deterministic AND cheap to run. How would this compare to the token costs of running an agent the whole time for each test? How do you make sure results stay stable regardless of the nondeterministic nature? Do customers still need to create test cases - any way to import from test case management system - based on which they could have already generate e2e t…

Unfortunately from our experience tests don’t scale as well as code. First of all, static tests are very brittle: you rely on selectors, need wait times, and can’t really test a lot of dynamic content (think AI chats/interactions). Then it’s all the infrastructure around it: solving captchas, handling auth, handling email OTP (each of our agents has access to its own inbox) and handling video recording and screenshot…

[deleted]
Post reply on HN