Live data from Hacker News

First Impressions with GPT-4V(ision)

blog.roboflow.com

201–210 of 345 posts

Re: First Impressions with GPT-4V(ision)

#201

Earlier quoted context omitted.

Roughly half of people in most developed countries are not functionally articulate: meaning, they can read functionally, but struggle to articulate what they want with the written word. LLM-based chatbots can be extremely attractive to the top 30% literacy users in the developed world. They are not a good universal UI. You still need to provide pathways for the user to follow to get done what they need without forcin…

Audio to text solves written word articulation, right? Besides this post is about vision, which also solves it.

Does vision solve it? How does one ask a question through images alone?

Re: First Impressions with GPT-4V(ision)

#202
post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

Zero awe. This week so far it can't find an import of a type in Next.js and it can't get the user object from a session. Next.js is one of the most popular FE frameworks in the world. It lied about both, repeatedly giving wrong answers. Here is my previous experience https://news.ycombinator.com/item?id=34648167 with it not being able to do basic tasks. It's all fun and games until the mistakes start having a cost. O…

Just to clarify, you're using GPT-4? Or GPT-3.5 (the free ChatGPT one)?

My experience with GPT-4 has been completely different from what you describe. Example:

https://gitlab.com/-/snippets/2567734

Re: First Impressions with GPT-4V(ision)

#203

Earlier quoted context omitted.

Correct, this will be the successor to the GUI.

I doubt it. It’s too damn costly computationally.

This is the same reply to GUIs will never take off but decades later and on to the next successor.

Re: First Impressions with GPT-4V(ision)

#204
post #194

Could someone with access tell me what GPT-4V has to say of this image? http://karpathy.github.io/assets/obamafunny.jpg Andrej Karpathy used it in 2012 as an example of an image he thought would be extremely hard for a model to interpret. I'm wondering how this holds 11 years later...

Prompt: "What do you have to say about this image?"

Response 1: "The image depicts a candid moment in what appears to be a hallway or corridor setting. Several individuals are present, engaging in various activities. The prominent features include a man on the left, who seems to be peering into a room, and another man next to him standing on a weighing scale and taking some notes. The individuals in the background are conversing with each other. The overall mood seems light-hearted and jovial. The architecture and interior suggest an institutional or official setting, possibly an office building or government facility. The tiled floor, lockers, and signs on the walls contribute to this formal ambiance. Overall, it's a snapshot of a seemingly casual moment in a professional setting."

Response 2: "Sorry, I cannot help with that."

New chat, Prompt: "Why is this image funny?"

Response 1: "The humor in this image seems to stem from the juxtaposition of formal and informal elements. Here are a few reasons why someone might find it amusing:

Unexpected Situation: Important figures, who are usually seen in serious or formal settings, are captured in a more relaxed, candid, and human moment.

Height Discrepancy: The taller individual is using a device to measure height, while the shorter individual walks by. This could create a comedic contrast between their heights.

Expressions: The reactions of people in the background and the focal subjects can be humorous. They are smiling or laughing, which indicates something amusing might have occurred or been said.

Informal Setting: The backdrop of lockers and checkered floor tiles gives the feel of a school or gym environment, which contrasts with the formal attire of the individuals.

Different people might find humor in different elements of the photo. It's a mix of candidness, unexpected situations, and visual contrasts that create a light-hearted and humorous scene."

Response 2: (same)

New chat, prompt: "What is the person in the middle doing with their foot and why?"

Response: "The person in the middle appears to be playfully stepping on the scale while the taller individual is weighing himself, likely as a humorous gesture to momentarily increase the weight reading. It's a light-hearted way of interacting and bringing some humor to the situation."

.

Apologies for how bad the formatting of this is going to come out, not sure how to make it better on HN (wish we had real quotes not just code blocks). Overall, I don't think it either noticed the foot was on the scale by itself or put it together that this was the focus until fed that information. Otherwise it was more lost in generalities about the image.

Re: First Impressions with GPT-4V(ision)

#205
post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

I'm glad you made that change because the interface can't really solve every single thing. Most of what I do on my phone is read eBooks. AI can't inject the knowledge directly into my brain. Text on the screen is as good as it will get for the foreseeable future. Most of what I want to do in my home is cook, clean, sleep, and have sex with my wife. Software can't do any of those things for me. Most of what I want to…

I'm hopeful that AI cars make renting one cheap enough that I can just rent an AI car the 0-2 times/week I need one instead of owning a car and all the time and stress on maintenance that entails.

Re: First Impressions with GPT-4V(ision)

#206

Earlier quoted context omitted.

Yeah that was my take as well. Although it did come up with a plausible sounding answer. A less nerdy human might also not get it

To be fair the author didn't get it either so hard for me to blame GPT but was a bit wild to see the first example of success actually be a failure.

good example of how you need to be smarter than the AI to know whether it's correct or not

if you don't know the answer, and the AI just pretends to know the answer, you just have to take its word for it.

Re: First Impressions with GPT-4V(ision)

#207
Oddly just like the text version it is still really bad at tic-tac-toe. Gave it a picture of a completed game and "Who won?" It told me "X won with a vertical line through the middle column" when in fact O won and there was only one X in the middle column.

Very impressive with almost everything else I gave it though.

Re: First Impressions with GPT-4V(ision)

#208

Earlier quoted context omitted.

Audio to text solves written word articulation, right? Besides this post is about vision, which also solves it.

No, it does not. It's still the same words, in a different medium. If you are unable to write, you'll probably be unable to speak your ideas.

That sounds super false to me, at least given how articulate kids can be before they can read, and how literate they can be before they can write.

If it's really true that half of the population can't functionally express themselves verbally then I'd sure like to know that. Or maybe I've misinterpreted something claimed here, because I'm struggling to find these claims plausible.

Re: First Impressions with GPT-4V(ision)

#209
post #104

Earlier quoted context omitted.

You are now where the GUI was in in 1986 or so.

and you think corporate will wait near forty years for tech to catch up? no, they will, indeed force you to be always online, leading to absolute and total surveillance where you voluntarily add dozens of mics and cameras to every home. if you thought telescreens a'la 1984 is bad, just wait.

It's funny that being obsessed with "corporations" feels so specifically GenX. I think because it's part of the New Left movement. In reality governments have all the power here, which is why this would be illegal in at least CA/EU/China.

Another issue where this comes up is high housing costs and climate change, which are mostly caused by bad land use laws (and the profiteers are landlords, who mostly own one or two properties), but people from the New Left era will literally refuse to believe you about this because they can't accept that any bad thing on Earth could not be caused by "corporations".

Re: First Impressions with GPT-4V(ision)

#210

Earlier quoted context omitted.

Audio to text solves written word articulation, right? Besides this post is about vision, which also solves it.

No, it does not. It's still the same words, in a different medium. If you are unable to write, you'll probably be unable to speak your ideas.

This isn't true. There's plenty of people who are verbally fine but can't read or write. Spoken language is a far more common and fundamental skill than reading or writing.
Post reply on HN