Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

First Impressions with GPT-4V(ision)

blog.roboflow.com

Post reply on HN
Print view View original

331–340 of 345 posts

Re: First Impressions with GPT-4V(ision)

#331

Post by RealityVoid »

a2dam wrote: post #210
RealityVoid wrote: post #199

Earlier quoted context omitted.

No, it does not. It's still the same words, in a different medium. If you are unable to write, you'll probably be unable to speak your ideas.

This isn't true. There's plenty of people who are verbally fine but can't read or write. Spoken language is a far more common and fundamental skill than reading or writing.

I am surprised about the amount of resistance this is getting. Yes, some people talk more than they write. Yes, some people can't write but they can talk. But, that's just the environment the words travel through. The words themselves, they will be the same. I conceptualize expressing in writing as more lenient than communicating verbally. Verbal communication leaves much less time to ponder your ideas and pick your words carefully, thus, it's harder. I maintain my stance, if you can put words to paper (and the vast majority of people in the first world can do that) but are unable to communicate your ideas this way, you probably will fail doing so verbally as well.

If my supposition (Speculation? Stronger than it should assertion?) Is true then just interpreting requests verbally will not help

Re: First Impressions with GPT-4V(ision)

#332

Post by RealityVoid »

furyofantares wrote: post #208
RealityVoid wrote: post #199

Earlier quoted context omitted.

No, it does not. It's still the same words, in a different medium. If you are unable to write, you'll probably be unable to speak your ideas.

That sounds super false to me, at least given how articulate kids can be before they can read, and how literate they can be before they can write. If it's really true that half of the population can't functionally express themselves verbally then I'd sure like to know that. Or maybe I've misinterpreted something claimed here, because I'm struggling to find these claims plausible.

I mean to say that if you can write and are unable to express your ideas, you probably won't be able to do so verbally either.

In the case of kids, of course, that's true, but just because they can't write.

But if you can (and most people can) just having the option of voice input won't help.

I refrain to take a stance about how much of the population is unable to articulate thoughts in writing, (it's probably not great though) but it's probably going to be comparable with how many can't express themselves verbally as well.

I'm talking here about more complex ideas of course. I'm sure average communication is functional.

Re: First Impressions with GPT-4V(ision)

#333

Post by wilg »

JHonaker wrote: post #327
wilg wrote: post #264

Earlier quoted context omitted.

There’s no reason chatbots have to be the interface to an LLM. Imagine dynamically generated interfaces redesigning themselves to your needs as you work through a task.

I would hate that. An interface is useful when you can anticipate and remember what things are available and where they are. It doesn't matter if it's a visual UI or keyboard shortcuts. If you start moving stuff around on us, we revolt. (See every UI update on an MS Office product ever.)

Well you have to make it useful obviously. Keep an open mind!

Re: First Impressions with GPT-4V(ision)

#334

Post by KETpXDDzR »

I think that solves any web scraping issues. Most issues I have with scraping websites are random/seldom changes in the pages. E.g., a bot detection pop-up that requires a captcha to solve. With GPT4V, I could just ask the model what to do and how.

Re: First Impressions with GPT-4V(ision)

#335

Post by kristopolous »

gdubs wrote: post #291
kristopolous wrote: post #151

This actually doesn't seem like it's a giant lift using modern image classifiers. The basic idea is to use diffusion classifiers to caption the image to generate descriptive text and append the prompt. The work part is getting the ensemble right since you'll need to use a general classifier, like BLIP, to identify say a bunch of text from a plant and then, in this example, use structured OCR and pl@ntnet to get more…

The main difference here is that GPT4 is truly multimodal, and so the model itself understands images and doesn't require a preprocessor to tell it what's in the image. As you show, that's a viable approach – but it has limitations. Mainly, things get lost in translation. It's like the difference between you telling me what's outside the window and then asking me questions about it – versus me being able to look out…

well sure, you'd in practice connect things at a lower level. The point was to show I'm not just purporting a vague idea and then retreating behind jargon or expressed expertise and hand-waving. This is something you could do with a couple browser tabs open and like a 3 page manual.

How OpenAI processes things so quickly, that's the thing to marvel. I've got 4090s, I know what the fast-as-it-can-go speed is for a single machine. Doing this for tens of thousands of people simultaneously faster than I can do with high end software on my $10,000 workstation? Alright, that's impressive.

I know there's some incredibly expensive NVIDIA cards but they must be doing some more magic on top of that.

Re: First Impressions with GPT-4V(ision)

#336

Post by _l219 »

I had access to this a few months back due to an insecure API endpoint by the name of “rainbow”. It was extremely useful until they noticed and shut it down. This was before GPT-4 itself was released to the public.

Re: First Impressions with GPT-4V(ision)

#337

Post by sarchertech »

nuancebydefault wrote: post #329
sarchertech wrote: post #250

Earlier quoted context omitted.

> When taking a shower, I would like fine control over the water temperature, preferably with a feedback loop regulating the temperature. (Preferably also the regulation changes over the duration of the showering.) That exists, but it’s expensive because of the electronics and mechanics involved. There are so many interfaces with this exact problem. You also almost certainly don’t want non-deterministic hallucination…

It would become less expensive, using less sensors and actuators, when using the predictive and learning abilities of an ai. You can, for safety reasons, keep a mechanical temperature limiter in the loop.

My background is in embedded systems. There is absolutely no way that an AI could make a temperature controlled shower cheaper.

There’s no way to use fewer actuators, and the control system is already dead simple, and uses one temperature sensor per outlet.

Re: First Impressions with GPT-4V(ision)

#338

Post by sarchertech »

nuancebydefault wrote: post #330
bodhi_mind wrote: post #276

Earlier quoted context omitted.

Maybe I misunderstood the op, but isn’t the mechanical faucet and our body’s temperature receptors this “fine control with feedback loop”?

Indeed, and to take the UI a step further, humans often prefer automation, if it works reliable. A complicated UI would become simple, just step into the shower.

There’s no complicated UI. You just turn a knob that sets a digital temperature readout.

If you want the shower to save your temperature preferences and start automatically, there’s no reason to build in a computer capable of running an AI.

But in reality you almost certainly don’t want a system like this because you don’t want an AI accidentally turning on your shower when you’re not home, when you do ok to clean it, or grab a razor, or when your toddler wanders in.

Granted an AI could try to determine intent, but it’s never going to get it 100% right. Which is why for physical systems like this you almost always want a physical button to signal intent.

Re: First Impressions with GPT-4V(ision)

#339

Post by theK »

fragmede wrote: post #306
theK wrote: post #272

Earlier quoted context omitted.

Agreed but lets keep in mind that graphical UIs involve significantly more effort than CLIs. In the case where you shoerhorn an ai to be your primary UI I would bet this is as hard if not harder since you typically have to code around significant limitations of current llms

http://gitwit.dev shows us that LLMs can generate various GUIs just fine, when prompted.

This is not what I said. Sure you can use such tools or even just classical boilerplate scripts (like we used for a decade now) to get started with react fast. But building out a system that fails well when the underlying llm starts behaving erratically or not at all is a completely different league of engineering as executing a boilerplate script.

Re: First Impressions with GPT-4V(ision)

#340

Post by PeterStuer »

have_faith wrote: post #111
cs702 wrote: post #10

Sure, there are a few edge-case failures and mistakes here and there, but I can't help but be in awe . AWE. Let me state the obvious, in case anyone here isn't clear about the implications: If the rate of improvement of these AI models continues at the current pace, they will become a superior user interface to almost every thing you want to do on your mobile phone, your tablet, your desktop computer, your car, your…

I personally can't wait for brands to start specifically advertising "doesn't use AI" as a selling point because everyone got burned by their AI controlled toaster, literally and figuratively. For anything designated an "appliance" I want it to be fixable by someone locally, or it's just throwing money away.

I'd rather pay for 'does not collect and sell your data. Does not play advertisements, is not remotely operable or accesible by anyone but the owner'

Sadly the market goes 100% full steam in the opposite direction.

Post reply on HN