Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

First Impressions with GPT-4V(ision)

blog.roboflow.com

Post reply on HN
Print view View original

281–290 of 345 posts

Re: First Impressions with GPT-4V(ision)

#281

Post by nmfisher »

raincole wrote: post #230
yieldcrv wrote: post #40

Earlier quoted context omitted.

The more people say that, the less convincing it is There is no way I would have a UI developer onboarded when I can generate many iterations of layouts in midjourney, copy them into chatgpt4 and get code in NextJS with Typescript instantly non devs will have trouble doing this or thinking of the prompts to ask, but the dev team asking for headcount simply wont ask for headcount, and the engineering manager is going…

> when I can generate many iterations of layouts in midjourney, copy them into chatgpt4 and get code in NextJS with Typescript instantly Have you actually tried this? I did the first step and even that didn't work well. The "iterations of layout in MidJourney" step. If people can make it work, well bless them, but we're not getting rid of our graphic designer now.

The best counterargument to “GPT4 is going to replace us all” is actually using it for a couple of weeks.

It has a few neat tricks but it’s not reliable and at least half of what it generates is totally unusable, the other half requires heavy intervention and supervision.

Re: First Impressions with GPT-4V(ision)

#282

Post by Veedrac »

crazygringo wrote: post #109
cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

> they will become a superior user interface to almost every thing you want to do No they won't. They're actually a pretty terrible user interface from a design perspective. Primarily because they provide zero affordances, but also because of speed. UX is about providing an intuitive understanding of available capabilities at a glance, and allowing you to do things with a single tap that then reflect the new state ba…

As we all know, the one human skill computers can never replace is doing repetitive tasks quickly.

Re: First Impressions with GPT-4V(ision)

#283

Post by chx »

SoftTalker wrote: post #274
chx wrote: post #244

Earlier quoted context omitted.

> few edge-case failures and mistakes Like identifying traffic lights in 4th and 5th squares in the second and third row both when there are only four squares?

I often fail at those too. If a square contains a small corner of a thing, but is mostly not the thing, do I check it or not?

read the problem again please. it's not that it misidentified what's in the squares, it "miscounted" the number of squares -- because it doesn't count, it's just a plausible sentence generator.

Re: First Impressions with GPT-4V(ision)

#284

Post by fy20 »

> For example, GPT-4V avoids identifying a specific person in an image and does not respond to prompts pertaining to hate symbols.

How does it handle pictures of the swastika?

For those that don't know, before the Nazis used it, it was a symbol of hope and prosperity in the West and even appeared on Coca Cola marketing. Today it still is in Eastern cultures.

https://www.bbc.co.uk/news/magazine-29644591

Re: First Impressions with GPT-4V(ision)

#285

Post by StevenNunez »

jihadjihad wrote: post #85
mbb70 wrote: post #37

The "Why is this image funny?" test reminds me of https://karpathy.github.io/2012/10/22/state-of-computer-visi... In 10 years we went from "SoTA is so far from achieving this I don't even know where to start" to "That'll be $0.0004 per token and have a nice day"

Has anyone tried GPT-4V on that image?

"The humor in this picture stems from the juxtaposition of the formal setting and attire with the playful and unexpected situation. It appears that one of the men is being measured for his height, which is not a typical activity during formal events or gatherings, especially with other distinguished individuals observing. Additionally, the expressions on the faces of the participants add to the humor, as they seem to be enjoying the lighthearted moment. The surprise element of this unexpected situation amidst an otherwise formal setting creates a humorous contrast." Womp. My prompt was "Why is this picture funny?"

Re: First Impressions with GPT-4V(ision)

#286

Post by StevenNunez »

varenc wrote: post #262
jihadjihad wrote: post #85

Earlier quoted context omitted.

Has anyone tried GPT-4V on that image?

Another response for this image a friend sent: " The humor in this picture comes from the juxtaposition of a formal setting with an informal, unexpected action. It shows a group of well-dressed individuals, possibly political figures or professionals, in what appears to be a formal event or gathering. However, amidst this serious environment, one tall man is casually standing on a scale to check his weight, seemingly…

I got this. "The humor in this picture stems from the juxtaposition of the formal setting and attire with the playful and unexpected situation. It appears that one of the men is being measured for his height, which is not a typical activity during formal events or gatherings, especially with other distinguished individuals observing. Additionally, the expressions on the faces of the participants add to the humor, as they seem to be enjoying the lighthearted moment. The surprise element of this unexpected situation amidst an otherwise formal setting creates a humorous contrast." Pretty similar!

Re: First Impressions with GPT-4V(ision)

#287

Post by starfallg »

cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

Would you rather do something yourself with an accessible user interface or API, or open a ticket to ask somebody to do it for you?

Sometimes communicating with an intelligent agent is harder than doing things yourself with a good structured user interface where you can communicate your intent clearly.

Until we have mind reading AI that is.

Re: First Impressions with GPT-4V(ision)

#288

Post by fomine3 »

dmazzoni wrote: post #178
hypertele-Xii wrote: post #89

Earlier quoted context omitted.

GPT-3 requires 700 gigabytes of GPU RAM. I'm looking at my cheapest computer components retailer listing a 48 gigabyte GPU at $5k. So to run the previous generation of GPT would cost me about $70k right now. When do you think I can expect to run GPT-4 on my consumer $device? :)

First of all, you're off by an order of magnitude. Second, I don't think it will be that long. There are already LLMs as good as GPT-3 running on average laptops and even phones. In the next couple of years, you'll see: - Ordinary PCs, tablets, and phones with dedicated AI chips, like TPUs - they'll be more tuned specifically for LLMs - Mathematical and algorithmic optimizations will make existing LLMs faster on the…

I don't know. RAM is $$ and currently usable LLMs needs huge RAM with higher bandwidth. I don't see any story that it will be solved with future AI chips. Do you know anything?

Re: First Impressions with GPT-4V(ision)

#289

Post by Jakob »

not2b wrote: post #277
jiggawatts wrote: post #216

Earlier quoted context omitted.

> I hate the idea of having to use a mouse to click on a visual GUI to navigate a file system in order to make use of its functionality. Ironically, studies have shown that mouse-based interfaces are more efficient for practically all filesystem use-cases compared to CLI interfaces. Despite objectively faster-time-to-solution, people self-report that they "feel" that the mouse GUIs are slower. That's because there's…

Pratically all? I regularly need to select a subset of a very large number of files to do something with. Example: I have a directory, or possibly a tree, with thousands of photos. The photos have the date in ISO 9000 format embedded in their name. Your job is to find all the files from February 2021 and copy them to specified location, and there are 50 of them. Good luck using a visual GUI for that, unless it has th…

Personally, I would use the terminal, too for a throwaway use case like this. But GUIs are very good at that and faster when you do more than the naïve case, they also allow to see intermediate steps.

For your use case, on macOS open Automator.app and add three actions

  1. "ask for finder items" (the source folder)
  2. "filter finder items" (by name)
  3. "copy finder items" (to target folder)
This takes roughly 5 clicks, 10 seconds at most.

Repeatability and configurability is where the GUI action shines. With only one click more you can

- add filtering by size, opening date, modification date, etc. in addition or a combination thereof

- do the same action for multiple source folders and the same target folder

- choose whether you want to replace existing files

- add it as a folder action that runs automatically on modification of the source folder

Arguably much slower on the terminal.

Alternative on macOS, that works on all other major OS with similar shortcuts and a similar feature set, just not repeatable:

  1. Go to source folder (shift-cmd-G)
  2. Filter (cmd-shift-F)
  3. Copy (cmd-A, cmd-C)
  4. Go to target folder (shift-cmd-G)
  5. Paste (cmd-V)
This repeatability, configurability and automation is where GPT falls short, for now.

Re: First Impressions with GPT-4V(ision)

#290

Post by dumpsterdiver »

wilg wrote: post #264
crazygringo wrote: post #109

Earlier quoted context omitted.

> they will become a superior user interface to almost every thing you want to do No they won't. They're actually a pretty terrible user interface from a design perspective. Primarily because they provide zero affordances, but also because of speed. UX is about providing an intuitive understanding of available capabilities at a glance, and allowing you to do things with a single tap that then reflect the new state ba…

There’s no reason chatbots have to be the interface to an LLM. Imagine dynamically generated interfaces redesigning themselves to your needs as you work through a task.

So to your point, I do think it would be nice to have an AI assistant suggest to make changes to the view mode of an application. For instance, if I navigate to a certain area the AI might suggest that I make that option visible in my menu. I might say no, because I prefer to use the hotkey. Aside from that, I'm going to have to absolutely side with the parent commenter on this one - When I'm doing serious work I want a familiar interface that I know how to move around efficiently in. I don't want any surprises.

Suggestions to improve workflow sound great. But nullifying hard earned knowledge of an interface... I can't see that helping me.

The shining example in my mind is audio/video/graphics applications, where there are good reasons to routinely switch between different views. Knowing your way around those views (which might be custom, but still static), and being able to navigate through them quickly is very valuable.

Post reply on HN