First Impressions with GPT-4V(ision) • Hacker News Board

Re: First Impressions with GPT-4V(ision)

#141

Post by kossTKR » Wed, Sep 27, 2023, 5:15 PM UTC

generic92034 wrote: post #125

kossTKR wrote: post #62

Earlier quoted context omitted.

But if this development continues AI will surely be able to just parse the entire frontend, then look at the repo, then look at the specifications, then when you ask for a specific feature it will instantly implement it gracefully. I don't see why it wouldn't understand piles of hotfixes on top of each other, or even refactor technical debt in tight coupling with existing or historical specification. Or is there a re…

It depends. Did you ever work in Development Support? Understanding requests or bug reports from customers is quite challenging, for trained and experienced developers. In my eyes that would require AGI, which we do not know of, if that can be achieved with the current approach.

I actually did , and yes it's extremely challenging and can be rather rage inducing; "it doesn't work" --> what doesn't work --> "the thing" --> what thing --> ∞ ...

But the thing is conversations like the above ie. both external support and internal feature requests could theoretically be handled by a GPT-like system also ending up in a ai created custom specification that could both be implemented and documented by the ai system instead of humans?

I know we're a few versions out, but still.

Re: First Impressions with GPT-4V(ision)

#142

Post by crazygringo » Wed, Sep 27, 2023, 5:17 PM UTC

bratbag wrote: post #136

crazygringo wrote: post #109

Earlier quoted context omitted.

> they will become a superior user interface to almost every thing you want to do No they won't. They're actually a pretty terrible user interface from a design perspective. Primarily because they provide zero affordances, but also because of speed. UX is about providing an intuitive understanding of available capabilities at a glance, and allowing you to do things with a single tap that then reflect the new state ba…

If something is a repetitive habit that you can do almost without thinking, there is a good chance an AI could infer that entire chain. I think what's more likely is that an AI based interface will end up being superior after it has had a chance to observe your personal preferences and approach on a conventional UI. So both will still be needed, with an AI helping at the low end and high end of experience and the mid…

I think I wasn't clear enough -- these habits I'm talking about are things like "press cold water button, press start" or "press warm water button, press start" or "tap 'News' app grouping, tap 'NY Times' icon".

There's nothing to infer. The sequence is already short. There are no benefits from AI here.

But you raise a good point, which is that there are occasionally things like 15-step processes that people repeat a bunch of times, that the AI can observe and then take over. So basically useful in programming macros/shortcuts as well. But that still requires the original UI -- it doesn't replace it.

Re: First Impressions with GPT-4V(ision)

#143

Post by mbil » Wed, Sep 27, 2023, 5:26 PM UTC

RandomLensman wrote: post #84

mbil wrote: post #71

Earlier quoted context omitted.

I agree. I think apps that would initially benefit from LLM-powered conversational interfaces are those that have the following traits: - constrained context - part of a hands-free workflow A couple use-cases I have been pondering are driving assistant and cooking assistant. People are already used to using their phone or car's nav system to give them directions to an unfamiliar place. But even with such a system it'…

In the cooking example, you either need the AI to have full awareness of the step you are at or you need to describe the step you are at, which could be cumbersome ("I did ..., how much sugar do I need now"). I venture, having the recipe projected in front of you would be much faster.

I imagined the AI would be reading the steps aloud to you, and so would be aware of your progress. I don’t think an AI assistant precludes the recipe being projected tho, just as in the driving example it wouldn’t replace an on screen map.

Re: First Impressions with GPT-4V(ision)

#144

Post by purplecats » Wed, Sep 27, 2023, 5:28 PM UTC

these first impressions don't mean anything besides what they are capable of (which does not mean you will have access to). they will do the same thing that anything does in a capitalist environment, which is to give you a taste of something amazing at first to hook you in (like with GPT4) then render it to the point of uselessness in value right above of the cusp of what you will tolerate to continue paying.

if anything, this shows the power disparity between the haves (they have this technology which gets better with time) and have nots (certainly me, but possibly also you) who get the super diluted version of this

Re: First Impressions with GPT-4V(ision)

#145

Post by abrichr » Wed, Sep 27, 2023, 5:28 PM UTC

stackedinserter wrote: post #74

Am I the only one who never saw these extra capabilities? OpenAI keep spreading news about "multimodal chatgpt", video, voice etc but to me it still looks like the initial version, but for $20/mo. It's still a text bot that gives unreliable answers.

I have the same experience. In the "Beta features" settings dialog, I've enabled "Advanced data analysis":

> Try a version of ChatGPT that knows how to write and execute python code, and can work with file uploads. Try asking for help with data analysis, image conversions, or editing a code file. Note: files will not persist beyond a single session.

Transcript:

```

In your settings it says:

> Try a version of ChatGPT that knows how to write and execute python code, and can work with file uploads. Try asking for help with data analysis, image conversions, or editing a code file. Note: files will not persist beyond a single session.

How do I upload an image?

ChatGPT I apologize for the confusion. The version of ChatGPT you're currently interacting with doesn't have the capability to accept file uploads. My apologies for any inconvenience this may have caused. If you have any other questions or need guidance on how to convert an image using software or online tools, feel free to ask.

```

Hopefully it's just a matter of time, but either way it's jarring for their product to contradict itself.

Re: First Impressions with GPT-4V(ision)

#146

Post by OnlyMortal » Wed, Sep 27, 2023, 5:28 PM UTC

have_faith wrote: post #111

cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

I personally can't wait for brands to start specifically advertising "doesn't use AI" as a selling point because everyone got burned by their AI controlled toaster, literally and figuratively. For anything designated an "appliance" I want it to be fixable by someone locally, or it's just throwing money away.

Here’s where a toaster will be in the near future.

https://youtu.be/7folKbch3U8

Do we want this?

Re: First Impressions with GPT-4V(ision)

#147

Post by sir_brickalot » Wed, Sep 27, 2023, 6:07 PM UTC

cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

If AI or LLMs or similar would be everywhere... how much more renewable energy do we have to produce, for very little to no gain in so many cases?

Madness!

Re: First Impressions with GPT-4V(ision)

#148

Post by dns_snek » Wed, Sep 27, 2023, 6:22 PM UTC

dr_dshiv wrote: post #51

nonameiguess wrote: post #30

Earlier quoted context omitted.

I'm glad you made that change because the interface can't really solve every single thing. Most of what I do on my phone is read eBooks. AI can't inject the knowledge directly into my brain. Text on the screen is as good as it will get for the foreseeable future. Most of what I want to do in my home is cook, clean, sleep, and have sex with my wife. Software can't do any of those things for me. Most of what I want to…

Also, keep in mind that Robots may be scifi level in 2 years. Like, able to cook and clean with hands! Crazy, but I think we will see it happen so fast: https://www.tri.global/news/toyota-research-institute-unveil...

I'm not sure if we watched the same video, but I saw a robot fumble though a few mechanical motions with dexterity and speed of a toddler to achieve a few isolated, best-case tasks where all the hard parts were taken care of by a human.

Cool demo, I suppose, but nobody is going to buy this as anything other than a toy.

Re: First Impressions with GPT-4V(ision)

#149

Post by renewiltord » Wed, Sep 27, 2023, 6:51 PM UTC

have_faith wrote: post #111

cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

I personally can't wait for brands to start specifically advertising "doesn't use AI" as a selling point because everyone got burned by their AI controlled toaster, literally and figuratively. For anything designated an "appliance" I want it to be fixable by someone locally, or it's just throwing money away.

I'm sure that in 2045, the Hacker News of its time will be complaining about lack of AI-less devices the way it complains about lack of smart-less TVs. It'll be available, it'll be more expensive, and the people who claim they want it don't want it enough to pay the premium.

Re: First Impressions with GPT-4V(ision)

#150

Post by HDThoreaun » Wed, Sep 27, 2023, 6:53 PM UTC

bamboozled wrote: post #132

yieldcrv wrote: post #40

Earlier quoted context omitted.

The more people say that, the less convincing it is There is no way I would have a UI developer onboarded when I can generate many iterations of layouts in midjourney, copy them into chatgpt4 and get code in NextJS with Typescript instantly non devs will have trouble doing this or thinking of the prompts to ask, but the dev team asking for headcount simply wont ask for headcount, and the engineering manager is going…

Will you then use the AI to scale your platform ? Optimise your database ? Improve your test coverage, implement new features, write new backend services, integrate with old difficult but critical systems? At some stage you must realise that you’re still working…

what does any of this have to do with frontend?