Live data from Hacker News

Using GPT-4 Vision with Vimium to browse the web

github.com

131–133 of 133 posts

Re: Using GPT-4 Vision with Vimium to browse the web

#131

I'm curious to see what it does when it sees a captcha.

From OpenAI docs[1]: "For safety reasons, we have implemented a system to block the submission of CAPTCHAs." [1] https://platform.openai.com/docs/guides/vision

There was an exploit that let you access the GPT-4 vision model months before release (and this restriction) and it could do this: https://media.discordapp.net/attachments/1020661972322230272...

Re: Using GPT-4 Vision with Vimium to browse the web

#132
post #40

Earlier quoted context omitted.

Yeah, I've been feeding screenshots from selenium to the vision API, and when I trigger bot detection on a website, chatgpt refuses to process the image.

It does solve, or at least try to solve, captchas for me. It gets like half the characters correct, it's very bad at it.

Maybe try asking it to list what's in each square before giving the final answer

Re: Using GPT-4 Vision with Vimium to browse the web

#133

Love the idea. It also shows that GPT-4V created a new angle in web scraping. I guess, this or similar code would be leveraged in many projects like: 1. Scrape XXX websites, say LinkedIn or Twitter use all types of methods in the DOM to prevent it, but fighting working well GPT-4V + OCR would be ultra hard. 2. Give me an analysis of what these XXX companies are doing. And this could be done for competitors, to unders…

It took me a while to get what you meant, because... I'm not sure "XXX websites" usually means what you intended to convey here :)

I feel very innocent now, as it did not even cross my mind ;)
Post reply on HN