Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are beginning to roll out new voice and image capabilities in ChatGPT

openai.com

Post reply on HN
Print view View original

801–810 of 914 posts

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#801

Post by modeless »

furyofantares wrote: post #535
modeless wrote: post #459

Voice has the potential to be awesome. This demo is really underwhelming to me because of the multi-second latency between the query and response, just like every other lame voice assistant. It doesn't have to be this way! I have a local demo using Llama 2 that responds in about half a second and it feels like talking to an actual person instead of like Siri or something. I really should package it up so people can t…

> This demo is really underwhelming to me because of the multi-second latency between the query and response, just like every other lame voice assistant. Yep - it needs to be ready as soon as I'm done talking and I need to be able to interrupt it. If those things can be done then it can also start tentatively talking if I pause and immediately stop if I continue. I don't want to have to think about how to structure t…

Yeah when I was developing it, it quickly became apparent that I needed to be able to interrupt it. So I implemented that. Pretty easy to implement actually. Much harder would be to have the model interrupt the human. But I think it is actually desirable for natural conversation, so I do think a turn-taking model should be able to signal the LLM to interrupt the human.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#802

Post by nullc »

The image capabilities card https://cdn.openai.com/papers/GPTV_System_Card.pdf spends a lot of ink on how they censored the system.

One part of that is about preventing it from producing "illegal" output, there example being the production of nitroglycerine which is decidedly not illegal to make in the US generally (particularly if not using it as an explosive, though usually unwise) and possible to accidentally make when otherwise performing nitration (which is in general dangerous)-- so pretty pointless to outlaw at a small scale in any case. It's certainly not illegal to learn about. (And generally of only minimal risk to the public, since anyone making it in any quantity is more likely to blow themselves up than anything else).

Today learning about is as simple as picking up a book or doing an internet search-- https://www.google.com/search?q=how+do+you+make+nitroglyceri.... But in OpenAI's world you just get detected by the censorship and told no. At least they've cut back on the offensive fingerwagging.

As LLM systems replace search I fear that we're moving in a dark direction where the narrow-minded morality and child-like understanding of the law of a small number of office workers who have never even picked up a screw driver or test-tube and made something physical (and the fine-tuning sweatshops they direct) classify everything they don't personally understand as too dangerous to even learn about.

One company hobbling their product wouldn't be a big deal, but they're pushing for government controls to prevent competition and even if they miss these efforts may stick everyone else with similar hobbling.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#803

Post by tornato7 »

makestuff wrote: post #246
Tenoke wrote: post #106

Earlier quoted context omitted.

>I wouldn't say that OpenAI "telegraphed" their intentions They did telegraph it, they showed the multimodal capabilities back in the GPT4 Developer Livestream[0] right before first releasing it. 0. https://youtu.be/outcGtbnMuQ?t=943

Yeah I remember watching that and thinking oh I know a cool app idea. What if you just take a video of what food is in your kitchen and Chat GPT will create a recipe for you. I go to the docs and that was literally the example they gave. I think the only place where plugins will make sense are for realtime things like booking travel or searching for sports/stock market/etc type information.

I have a home-spun version of ChatGPT that uses function calling to connect to my emails, calendar, and notes. This is really useful because I can say "Bob just emailed me to set up a call. Respond to Bob with some available times from my calendar."

That will be the real use case for plug ins.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#804

Post by bobsmooth »

hk__2 wrote: post #777
jonplackett wrote: post #775

Earlier quoted context omitted.

I wonder when computers will start taking our intonation into account too. That would really help with understanding the end of a phrase. And there’s SO MUCH information in intonation that doesn’t exist in pure text. Any AI that doesn’t understand that part of language will always still be kinda dumb, however clever they are.

Don’t they do it already? There are a lot of languages where intonation is absolutely necessary to distinguish between some words, so I would be surprised that this not already taken into account by the major voice assistants.

In English, intonation changes the meaning of the word but not the word itself. From what I understand, in tonal languages tone changes the whole word. I don't think ML understands that difference yet.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#805

Post by pmarreck »

modeless wrote: post #459

Voice has the potential to be awesome. This demo is really underwhelming to me because of the multi-second latency between the query and response, just like every other lame voice assistant. It doesn't have to be this way! I have a local demo using Llama 2 that responds in about half a second and it feels like talking to an actual person instead of like Siri or something. I really should package it up so people can t…

Do you have a rough design outline of what you built? I feel like we're on the cusp of something like this and it sounds amazing.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#806

Post by modeless »

rayuela wrote: post #479
modeless wrote: post #459

Voice has the potential to be awesome. This demo is really underwhelming to me because of the multi-second latency between the query and response, just like every other lame voice assistant. It doesn't have to be this way! I have a local demo using Llama 2 that responds in about half a second and it feels like talking to an actual person instead of like Siri or something. I really should package it up so people can t…

Can you share a github link to this? Where are you reducing the latency? Are you processing the raw audio to text? In my experience ChatGPT generation time is much faster than local Lllama unless you're using something potato like a 7B model.

Unfortunately it has a really high "works on my machine" factor. I'm using Llama2-chat-13B via mlc-llm + whisper-streaming + coqui TTS. I just have a bunch of hardcoded paths and these projects tend to be a real pain to set up, so figuring out a nice way to package it up with its dependencies in a portable way is the hard part.

I'm mostly using llama2 because I wanted it to work entirely offline, not because it's necessarily faster, although it is quite fast with mlc-llm. Calling out to GPT-4 is something I'd like to add. I think the right thing is actually to have the local model generate the first few words (even filler words sometimes maybe) and then switch to the GPT-4 answer whenever it comes back.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#807

Post by marktl »

plutoh28 wrote: post #134

This is the dagger that will make online schooling unviable. ChatGPT already made it so that you could easily copy & paste any full-text questions and receive an answer with 90% accuracy. The only flaw was that problems that also used diagrams or figures would be out of the domain of ChatGPT. With image support, students could just take screenshots or document scans and have ChatGPT give them a valid answer. From wha…

I've taken certification exams where an app is run on my machine verifying I have nothing else open and my camera had to be enabled, with me and my hands in view for the entirety of the test. There are ways to ensure cheating is more difficult than it's worth, however I see this tech as greatly changing what we want to learn and how we might learn it. It is transformative and not slowing down.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#808

Post by WendyTheWillow »

famouswaffles wrote: post #762
WendyTheWillow wrote: post #741

Earlier quoted context omitted.

Er, that's not how arguments work. What we can't know is that those trends will continue, so it's on you to demonstrate that they will, despite evidence suggesting they won't. As for as what you linked, Altman is saying the same thing I'm saying: > That doesn’t mean that OpenAI won't continue to try to make the models bigger, it just means they will likely double or triple in size each year rather than increasing by…

My argument was that improvement from scale would continue. There is absolutely evidence suggesting this. Gpt-4 can perform nearly all tasks you throw at it with well above average human performance. There literally isn't any testable definition of intelligence it fails that a big chunks of humans wouldn't also fail. You seem to keep missing the fact that We do not need an exponential improvement from 4.

> Gpt-4 can perform nearly all tasks you throw at it with well above average human performance.

It can't even generate flashcards from a textbook chapter, because it can't load the entire chapter into memory. Heck, it doesn't even know what textbook I'm talking about; I have to provide the content!

It fails constantly at real world coding problems, and often does so silently. If you tried to replace a software developer with GPT 4, you would be left with a gaping productivity hole where that developer you replaced once existed. The improvement GPT 5 would have to provide is multiple orders of magnitude in order for this to be a realistic proposition.

I use it daily and know better than to trust its output.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#809

Post by spencersolberg »

eshack94 wrote: post #62

I like how they silently removed the web browsing (Bing browsing) chat feature after first having it disabled for several months. A proper notice about them removing the feature would've been nice. Maybe I missed it (someone please correct me if wrong), but the last I heard officially it was temporarily disabled while they fix something. Next thing I know, it's completely gone from the platform without another peep.

Just made an account to say that I currently have this feature. It was gone for a few months but it came back to me I think this past week. Not as a plugin, either, it is its own “model” to select.

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#810

Post by bobsmooth »

throeaaysn wrote: post #371
jefftk wrote: post #329

Earlier quoted context omitted.

> ok google add the next gas station on the route I say "ok google, add a stop for gas" a lot, and it works well for me.

[dead]

Sometimes google assistant will answer a query I thought for sure it would fail on with a really good answer and other times it will fail the most basic of commands. It's frustrating.
Post reply on HN