Launch HN: CamelQA (YC W24) – AI that tests mobile apps • Hacker News Board

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#31

Post by jondwillis » Thu, Mar 21, 2024, 1:36 AM UTC

bellareed wrote: post #20

jondwillis wrote: post #18

Any support for non-mobile native apps? e.g. macOS?

Not yet but it's on the road map! We have a monthly newsletter if you'd like to stay up to date

Will do, and good luck with getting some traction! I am bullish on automation like this, and loved the demo.

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#32

Post by garyiskidding » Thu, Mar 21, 2024, 5:06 AM UTC

This is a great idea. I assume that inference costs will be higher for the time being, but it does aim to solve a real problem. Kudos..!

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#33

Post by ksvarma » Thu, Mar 21, 2024, 6:14 AM UTC

hugs wrote: post #21

Appium project starter here. Congrats on the launch! If you ever want to talk shop, let me know! I'm glad to see more vision-first, AI-powered testing tools in the world.

Props for everything you do Jason and share openly, also your robots!

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#34

Post by ngokevin » Thu, Mar 21, 2024, 6:35 AM UTC

Yeah, the two big issues with UI tests: flaky and slow.

Curious how using GPT and vision combats flakiness? I'd feel the entropy of GPT and anything less than 100% accuracy in the computer vision pieces would lead to more flakiness.

I also wonder about the speed and costs of running the tests. When E2E tests are traditionally slow and expensive already. The computer vision and GPT elements seem costlier and less fast.

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#35

Post by tetek » Thu, Mar 21, 2024, 9:01 AM UTC

would be nice to just invite the AI tester to TestFlight instead of uploading the build :)

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#36

Post by onion2k » Thu, Mar 21, 2024, 9:04 AM UTC

This is very cool. Is there a version for browsers?

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#37

Post by sumedh » Thu, Mar 21, 2024, 9:44 AM UTC

bellareed wrote: post #26

chw9e wrote: post #25

This looks awesome, automated UI testing is so hard to get right but also very important. Great work so far!

Thank you! It is hard to get right. And what we've noticed from the user's we've been talking to, no one company has won in this space just yet.

> no one company has won in this space just yet

Detox from Wix for React Native Apps is pretty good.

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#38

Post by bartekpacia » Thu, Mar 21, 2024, 12:47 PM UTC

Seems similar to App Quality Copilot - https://www.mobile.dev/app-quality-copilot

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#39

Post by ahoog42 » Thu, Mar 21, 2024, 1:11 PM UTC

Congrats, very exciting. Do you support configuring things like usernames and passwords? How do you handle MFA?

Re: Launch HN: CamelQA (YC W24) – AI that tests mobile apps

#40

Post by bellareed » Thu, Mar 21, 2024, 2:05 PM UTC

ahoog42 wrote: post #39

Congrats, very exciting. Do you support configuring things like usernames and passwords? How do you handle MFA?

Yes and yes. Check out the sole GIF we've provided. It should show you an example of camelQA navigating user sign up MFA by navigating to the email app through a deep link, verifying the email, navigating back to the app and confirming sign up.