Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

First Impressions with GPT-4V(ision)

blog.roboflow.com

Post reply on HN
Print view View original

301–310 of 345 posts

Re: First Impressions with GPT-4V(ision)

#301

Post by jes5199 »

p1esk wrote: post #221
cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

If the rate of improvement continues at the current pace - which is GPT 1 to 2 to 3 to 4 in the last five years - we are just one or two improvements away from a full blown AGI/superintelligence/singularity/etc. At that point, a superior user interface is probably the least interesting (or scary) thing that would happen. I personally doubt GPT-5 will be as much of an improvement over GPT-4 as GPT-4 was over GPT-3, bu…

I think you’re assuming a lot about the nature of intelligence. GPT is growing fast, but we have no real idea how big/deep the gap is between it and the human mind

Re: First Impressions with GPT-4V(ision)

#302

Post by yieldcrv »

raincole wrote: post #230
yieldcrv wrote: post #40

Earlier quoted context omitted.

The more people say that, the less convincing it is There is no way I would have a UI developer onboarded when I can generate many iterations of layouts in midjourney, copy them into chatgpt4 and get code in NextJS with Typescript instantly non devs will have trouble doing this or thinking of the prompts to ask, but the dev team asking for headcount simply wont ask for headcount, and the engineering manager is going…

> when I can generate many iterations of layouts in midjourney, copy them into chatgpt4 and get code in NextJS with Typescript instantly Have you actually tried this? I did the first step and even that didn't work well. The "iterations of layout in MidJourney" step. If people can make it work, well bless them, but we're not getting rid of our graphic designer now.

I don’t have multimodal access yet

Re: First Impressions with GPT-4V(ision)

#303

Post by monkpit »

Curious that it sets up the math problem right, got the value wrong, but it was close enough that it got the answer right.

I wonder why it gets it wrong when it spits out the value? I figure 25/cos(10°) is around 25.38. GPT says it’s 25.44.

I can’t wait for the next iteration of these tools that have agency to reach out to a service for an answer, like Wolfram or a Python interpreter or any expert/oracle.

I think it would be cool to see which circumstances even prompted the AI to delegate to the expert for an answer - what criteria would be used to signal that it doesn’t quite know the answer, or that it shouldn’t guess?

I know there’s something along these lines with autogpt and/or agentgpt but I wasn’t super impressed with it when I looked at them both. Granted this was a few months ago.

Re: First Impressions with GPT-4V(ision)

#304

Post by fragmede »

monkpit wrote: post #303

Curious that it sets up the math problem right, got the value wrong, but it was close enough that it got the answer right. I wonder why it gets it wrong when it spits out the value? I figure 25/cos(10°) is around 25.38. GPT says it’s 25.44. I can’t wait for the next iteration of these tools that have agency to reach out to a service for an answer, like Wolfram or a Python interpreter or any expert/oracle. I think it…

> I can’t wait for the next iteration of these tools that have agency to reach out to a service for an answer, like Wolfram or a Python interpreter or any expert/oracle.

ChatGPT-4 has a plugin system, and there is already a Wolfram plugin.

Using that plugin, ChatGPT-4 is happy to tell me that the exact answer: 25 sec(π/18), as well as the decimal approximation of 25.3857.

https://chat.openai.com/share/468db5e9-4983-4bf6-9efb-f42783...

That link doesn't properly show that off, so here's a screenshot: https://i.imgur.com/foE5hgR.png

Re: First Impressions with GPT-4V(ision)

#305

Post by mycall »

sarchertech wrote: post #250
nuancebydefault wrote: post #200

Earlier quoted context omitted.

Most user interfaces already have a much finer granularity and number of options than your examples. When taking a shower, I would like fine control over the water temperature, preferably with a feedback loop regulating the temperature. (Preferably also the regulation changes over the duration of the showering.) Choosing to read the NY times indeed is only a few taps away, but navigating through and within its list o…

> When taking a shower, I would like fine control over the water temperature, preferably with a feedback loop regulating the temperature. (Preferably also the regulation changes over the duration of the showering.) That exists, but it’s expensive because of the electronics and mechanics involved. There are so many interfaces with this exact problem. You also almost certainly don’t want non-deterministic hallucination…

Prompt engineering and using multiple AI models in parallel might find ways to cancel out most hallucinations similar to how consensus-based replication works.

Re: First Impressions with GPT-4V(ision)

#306

Post by fragmede »

theK wrote: post #272
artursapek wrote: post #268

Earlier quoted context omitted.

Exactly. Chat is just the first, most general purpose UI. Like a terminal was for computers.

Agreed but lets keep in mind that graphical UIs involve significantly more effort than CLIs. In the case where you shoerhorn an ai to be your primary UI I would bet this is as hard if not harder since you typically have to code around significant limitations of current llms

http://gitwit.dev shows us that LLMs can generate various GUIs just fine, when prompted.

Re: First Impressions with GPT-4V(ision)

#307

Post by thebruce87m »

crazygringo wrote: post #109
cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

> they will become a superior user interface to almost every thing you want to do No they won't. They're actually a pretty terrible user interface from a design perspective. Primarily because they provide zero affordances, but also because of speed. UX is about providing an intuitive understanding of available capabilities at a glance, and allowing you to do things with a single tap that then reflect the new state ba…

This is the Dropbox comment of our time.

Re: First Impressions with GPT-4V(ision)

#308

Post by bmicraft »

3abiton wrote: post #189
hypertele-Xii wrote: post #39

Earlier quoted context omitted.

So you won't be able to do anything without Internet connection to the AI mainframe? No thanks.

At some point Open-source models' performance will follow. Llava will improve.

The vram and processing power in my phone will never follow what is available when not limited by the constraints of a phone

Re: First Impressions with GPT-4V(ision)

#309

Post by dumpsterdiver »

wilg wrote: post #300
dumpsterdiver wrote: post #290

Earlier quoted context omitted.

So to your point, I do think it would be nice to have an AI assistant suggest to make changes to the view mode of an application. For instance, if I navigate to a certain area the AI might suggest that I make that option visible in my menu. I might say no, because I prefer to use the hotkey. Aside from that, I'm going to have to absolutely side with the parent commenter on this one - When I'm doing serious work I wan…

I think there’s room for an AI-native interface that knows the full context of the project you’re working on and can make extremely intelligent suggestions or tailor the interface for a specific task. Or you could just literally describe the interface you want for something and it redesigns it for you immediately.

There is room, absolutely. Perhaps I took too hard of a line. I do recognize that a compromise can be made between a user retaining their knowledge of a system, and that system evolving to accommodate user requirements. It's easy to be fearful of the rug being pulled right from under you, but I suppose the reality might be closer to earnestly asking to place things upon the rug. In other words, the things that are familiar will probably remain familiar and accessible, but AI will help us arrange them in a way that optimizes our workflow.

Re: First Impressions with GPT-4V(ision)

#310

Post by phs318u »

cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

I am currently working in the higher education space and previously in banking. I agree with you. We are already workshopping where it makes sense to provide this type of interface on top of headless core systems. In the examples we've considered, this will drive a reduction in head count in some areas.

I'd also suggest that one of the early "killer apps" for this may be as an "IVR co-pilot" for actual humans on the phone with customers for their tricky issues.

Post reply on HN