Launch HN: GPT Driver (YC S21) – End-to-end app testing in natural language • Hacker News Board

Re: Launch HN: GPT Driver (YC S21) – End-to-end app testing in natural language

#81

Post by drothlis » Fri, Oct 25, 2024, 8:14 AM UTC

I noticed in your demo it generated the prompt "tap on the 'Log in' button located directly below the 'Facebook Password' field".

Does your model consistently get the positions right? (above, below, etc). Every time I play with ChatGPT, even GPT-4o, it can't do basic spatial reasoning. For example, here's a typical output (emphasis mine):

> If YouTube is to the upper *left* of ESPN, press "Up" once, then *"Right"* to move the focus.

(I test TV apps where the input is a remote control, rather than tapping directly on the UI elements.)

Re: Launch HN: GPT Driver (YC S21) – End-to-end app testing in natural language

#82

Post by archerx » Fri, Oct 25, 2024, 8:45 AM UTC

chrtng wrote: post #30

archerx wrote: post #29

Curious question, what ever happened with the OpenAI drama with trademarking “GPT”. I’m guessing they were not successful?

From what we understand the term GPT was deemed too general for OpenAI to claim as its own. https://www.theverge.com/2024/2/16/24075304/trademark-pto-op...

Thank you.

Re: Launch HN: GPT Driver (YC S21) – End-to-end app testing in natural language

#83

Post by ec109685 » Fri, Oct 25, 2024, 2:30 PM UTC

ec109685 wrote: post #46

> In terms of trying the product out: since the service is resource-intensive (we provide hosted virtual/real phone instances), we don't currently have a playground available. However, you can see some examples here https://mobileboost.io/showcases and book a demo of GPT Driver testing your app through our website. Have you considered an approach like what Anthropic is doing for their computer control where an agent…

Or even the actual device on the latest Mac OS.

Re: Launch HN: GPT Driver (YC S21) – End-to-end app testing in natural language

#84

Post by egeek » Fri, Oct 25, 2024, 10:37 PM UTC

edelans wrote: post #76

fullstackchris wrote: post #18

Earlier quoted context omitted.

To be fair, this is NOT the case with native mobile apps. There are some projects like detox that are trying to make e2e tests easier, but the tests themselves can be painful, run fairly slow on emulators, etc. Maybe someday the tooling for mobile will be as good as headless chrome is for web :) Agreed though that the followup debugging of a failed test could be hard to automate in some cases.

I think we can claim that at Waldo. Check for yourself: I've just recorded this [1] scripted test on the wikipedia mobile app, and it yields this [2] Replay. In less than a minute we spin up a fresh virtual device, install your app on it, execute the 8 steps of the script. As a result, you get the Replay of the session : video synchronized with interaction timeline, device & network logs, so you can debug in full con…

Do you have any pricing info available? All I can see is get started for free, but no info on what it might cost later