Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are beginning to roll out new voice and image capabilities in ChatGPT

openai.com

Post reply on HN
Print view View original

871–880 of 914 posts

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#871

Post by swsieber »

TheEzEzz wrote: post #633
modeless wrote: post #459

Voice has the potential to be awesome. This demo is really underwhelming to me because of the multi-second latency between the query and response, just like every other lame voice assistant. It doesn't have to be this way! I have a local demo using Llama 2 that responds in about half a second and it feels like talking to an actual person instead of like Siri or something. I really should package it up so people can t…

Completely agree, latency is key for unlocking great voice experiences. Here's a quick demo I'm working on for voice ordering https://youtu.be/WfvLIEHwiyo Total end-to-end latency is a few hundred milliseconds: starting from speech to text, to the LLM, then to a POS to validate the SKU (no hallucinations are possible!), and finally back to generated speech. The latency is starting to feel really natural. Building out…

That's way slick.

Can I ask what your background is, and what things you're used to working with? I don't have the chops to build what you built, but I'd love to get there.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#872

Post by chankstein38 »

kkielhofner wrote: post #799
chankstein38 wrote: post #228

Earlier quoted context omitted.

I've wanted a ChatGPT Pod equivalent to a Google Home pod for a while! I have been intending to build it at some point. I am with you, talking to Google sucks. "Hey Google, why do ____ happen?" "I'm sorry, I don't know anything about that" But you're GOOGLE! Google it! What the heck lol So yeah, ChatGPT being able to hear what I say and give me info about it would be great! My holdup has been wakewords.

We have hardware and wake words: https://heywillow.io/ Our REST endpoint can talk to whatever you want and we’ll have native ChatGPT soon.

Ooooo looking, thanks for the recommendation!

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#873

Post by hugs »

chintler wrote: post #841
hugs wrote: post #791

As someone deep in the software test automation space, the thing I'm waiting for is robust AI-powered image recognition of app user interfaces. Combined with an AI ability to write test automation code, I'm looking forward to the ability to generate executable Selenium or Appium test code from a single screenshot (or sequence of screenshots). Feels like we're almost there.

I'll recommend the Spotlight paper by Google[1]. There are very interesting datasets they created for this purpose. They mention they have a screen-action-screen dataset that is in-house and it doesn't look like they'll open it. Maybe owning Android has its advantages. There's a recent paper by Huggingface called IDEFICS[2] that claims to be an open source implementation of Flamingo(an older paper about few-shot mult…

Thanks!

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#874

Post by DrScientist »

whimsicalism wrote: post #597
londons_explore wrote: post #14

Earlier quoted context omitted.

Statistical diagnoses models have offered similar possibilities in medicine for 50 years. Pretty much, the idea is that you can get a far more accurate diagnosis if you take into account the medical history of everyone else in your family, town, workplace, residence and put all of it into a big statistical model, on top of your symptoms and history. However, medical secrecy, processes and laws prevent such things, ev…

Nonsense. The medical possibilities that will be unlocked by large generative deep multimodal models are on an entirely different scale from "statistical diagnoses." Imagine feeding in an MRI image, asking if this person has cancer, and then asking the model to point out why it thinks the person has cancer. That will be possible within a few years at most . The regulatory challenges will be surmounted eventually once…

But in your scenario - which part is adding the value?

Your deep multimodal models or the MRI imaging?

What you are essentially saying is the signal is so subtle that only a large NN can reliably extract it.

While that may well be the case, it would be better to have a scan/diagnostic that doesn't need that level of signal processing to interpret.

For example - you don't need a large generative deep multimodal model to read a Covid antigen or PCR test.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#875

Post by cyrialize »

Legend2440 wrote: post #811
cyrialize wrote: post #389

Earlier quoted context omitted.

I do think it is much different from the past twenty years. Twenty years ago we didn't have ChatGPT. There are things we could compare it to, but there also isn't anything like it. My biggest fear is just a lack of jobs. When people need experience to work, and the work you give to people to give them experience is replaced by ChatGPT - then what do we do? Of course there will still be companies hiring people, but wh…

The only way there will be no jobs is if every conceivable human need is met by robots. In which case there will also be no need to work.

I understand that. The fear I'm describing isn't no jobs, but less jobs than there are people.

There's multiple ways to address this, but it's difficult for me to imagine a future with our current economic system (in the US) that allows that to happen (like UBI).

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#877

Post by grantcas »

adroniser wrote: post #69
dmd wrote: post #49

Earlier quoted context omitted.

I also don't believe LLMs are "conscious", but I also don't know what that means, and I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word.

I keep feeling that consciousness is a bit of a red herring when it comes to AI. People have intuitions that things other than humans cannot develop consciousness which they then extrapolate to thinking AI can't get past a certain intelligence level. In fact my view is that consciousness is just a mysterious side effect of the human brain, and is completely irrelevant to the behaviour of a human. You can be intellige…

[dead]

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#878

Post by eshack94 »

spencersolberg wrote: post #809
eshack94 wrote: post #62

I like how they silently removed the web browsing (Bing browsing) chat feature after first having it disabled for several months. A proper notice about them removing the feature would've been nice. Maybe I missed it (someone please correct me if wrong), but the last I heard officially it was temporarily disabled while they fix something. Next thing I know, it's completely gone from the platform without another peep.

Just made an account to say that I currently have this feature. It was gone for a few months but it came back to me I think this past week. Not as a plugin, either, it is its own “model” to select.

Hey, thanks for the info! I did not know about this, but this is actually good to hear. I'll keep an eye open for it. Are you using ChatGPT or the API? Did you have to take any action to get it to reappear, or is it just a slow rollout as they re-enable?

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#879

Post by eshack94 »

ashot wrote: post #845
eshack94 wrote: post #526

Earlier quoted context omitted.

Perplexity has gone downhill a lot since its initial rollout. Anecdotally, from my experience as a non-paying user of the service.

give vello.ai a try

Thanks I'll check it out. Are there other similar sites you like?

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#880

Post by laurels-marts »

famouswaffles wrote: post #786
laurels-marts wrote: post #559

I'm very curious about this feature: > analyze a complex graph for work-related data Does this mean that I can take a screenshot of e.g. Apple stock chart and it will be able to reason about it and provide insights and analysis? GPT-4 currently can display images but cannot reason or understand them at all. I think it's one thing to have some image recognition and be able to detect that the picture "contains a time-s…

Look at this link of GPT-4 Vision analyzing charts(last image). https://imgur.com/a/iOYTmt0

This is brilliant. Thank you very much for this link. The analysis on the last image was impressive and quite thorough (given the simple prompt).

Every chart has an equivalent tabular representation. One way to get "charts" analysed like this before GPT Vision was to just pass tabular representations of charts to GPT-4. This makes implementing chart analysis a lot simpler. I do wonder though if for absolute best result it still wouldn't be better to pass both - image of the chart and the tabular representation of the chart.

Imagine having a dashboard with 5 different visualisations. You could capture the state of the entire dashboard in one screenshot and then pass tabular representations of the each individual chart all in one prompt to GPT-4 for a very comprehensive analysis and summary.

Post reply on HN