Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New models and developer products

openai.com

Post reply on HN
Print view View original

471–480 of 561 posts

Re: New models and developer products

#471

Post by karmasimida »

estreeper wrote: post #292
karmasimida wrote: post #224

Earlier quoted context omitted.

Embedding is poor man's context length increase. It essentially increases your context length but with loss. There is a cost argument to make still, embedding-based approach will be cheaper and faster, but worse result than full text. That being said, I don't see how those embedding startups compete with OpenAI, no one will be able to offer better embedding than OpenAI itself. It is hardly a convincing business. The…

For embeddings specifically, there are multiple open source models that outperform OpenAI’s best model (text-embedding-ada-002) that you can see on the MTEB Leaderboard [1] > embedding-based approach will be cheaper and faster, but worse result than full text I’m not sure results would be worse, I think it depends on the extent to which the models are able to ignore irrelevant context, which is a problem [2]. Using r…

> on the MTEB Leaderboard

The point isn't about leaderboard. With increasing context length, the question is on whether we need embeddings or not. With longer context length, embeddings is no longer a necessity, and it lowers its value.

Re: New models and developer products

#472

Post by StanAngeloff »

throw03172019 wrote: post #244
StanAngeloff wrote: post #153

Earlier quoted context omitted.

I personally use Whisper to transcribe painfully long meetings (2+ hours). The transcripts are then segmented and, you guessed it, entered right into GPT-4 for clean up, summarisation, minutes, etc. So in a sense it's a great way to get more people to use their other products?

This sounds amazing. Would you be willing to share your code? Thanks!

I run this[0] on Google Colab. The way I have it set up is to encode the meeting minutes to .ogg, push them to Google Drive, then adjust the script to tell it how many speakers there were and the topic of conversation. The `initial_prompt` really helps the model especially if you are talking about brand names, etc. that it may not know how to correctly transcribe. I've added a comment at the bottom of the Gist with some of the prompts I've used in the past. I've successfully managed to produce reports on week-long meetings (~18 hours) that were essential to get the team up to speed.

As a company we are currently shifting to Otter.ai[1] which gives good enough results for everyday meetings.

[0]: https://gist.github.com/StanAngeloff/91480fac18a74d8aff3e4cf... [1]: https://otter.ai/

Re: New models and developer products

#473

Post by icelancer »

nmfisher wrote: post #384
unshavedyak wrote: post #381

Earlier quoted context omitted.

Can you go into depth? I’ve used ChatGPT Pro and Phind extensively, didn’t know about Claude and code. Curious to give it a try

I generally use it for boilerplate tasks like “here’s some code, write unit tests” or “here’s a JSON object, write a model class and parser function”. Claude is significantly faster, so even if it requires a couple more prompt iterations than GPT4, I still get the result I need earlier than with GPT4. GPT4 also recently developed this annoying tendency to only give you one or two examples of what you asked for, then…

> GPT4 also recently developed this annoying tendency to only give you one or two examples of what you asked for, then say “you can write the rest on your own based on this template”. I can’t overstate how annoying this was.

The last model "update" has really ruined GPT-4 in this regard.

Re: New models and developer products

#474

Post by asaddhamani »

ChatGTP wrote: post #427
asaddhamani wrote: post #425

Earlier quoted context omitted.

Their products are incredible though. I’ve tried the alternatives and even Claude is not nearly as good as even ChatGPT. Claude gives an ethics lecture with every second reply, which costs me money each time and makes their product very difficult to (want to) embed.

What are you using it for? I want to know what people actually use these things for damn it !

I'd built a bot to use ChatGPT from Telegram (this was before the ChatGPT API), and currently building a tool to help make writing easier (https://www.penpersona.com). This is the API.

Apart from that, it's pretty much replaced 80% of my search engine usage, I can ask it to collate reviews for a product from reddit and other sites, get the critical reception of a book, etc. You don't have to go and read long posts and articles, have GPT do it for you. There's many other use cases like this. For the second part, I'm using a UI called Typing Mind (which also works with the API).

Re: New models and developer products

#475

Post by joegibbs »

leobg wrote: post #435
gwern wrote: post #338

Earlier quoted context omitted.

Considering how well simply reusing GPT-3.5/4 outputs has worked to juice rival model performance, at least in relatively narrow benchmarking, I dunno how many GBs it'd take, but probably not that many, and it's a straightforward easy way to turn money into performance at a much lower cost than buying a few thousand more H100s.

OpenAI does not strike me as a company that would be naive about this. Didn’t they just recently manipulate the outputs of an endpoint when they realized people were misusing it? (“CatGPT”) The most sinister interpretation is that the logits are a red herring. People who are tied up in stealing them aren’t free to do actual rival work.

What happened with CatGPT?

Re: New models and developer products

#476

Post by stavros »

9dev wrote: post #448
lucubratory wrote: post #278

Earlier quoted context omitted.

They are my content, actually, from the last ~15 years of being on the internet. I don't care about it personally, and even if I did it is really obviously fair use so even if I find it objectionable I don't get to actually legally compel someone to stop.

What on earth is fair use about a public company deriving its whole valuation from the processing of content taken from the internet without any regard for licensing, or robots.txt rules?? The technology is cool, I get it. But saying ”I don’t mind, they can use my content“ is on par with ”I don’t need privacy, I have nothing to hide“ in terms of statement quality.

I agree with you on OpenAI, I disagree on LLMs in general. I wish someone would use all the content available ever to train a great open-source LLM.

Re: New models and developer products

#477

Post by jeppebemad »

Alifatisk wrote: post #458
topicseed wrote: post #6

128,000 token context, Assistants API, JSON mode, April 2023 knowledge cutoff, GPT 4 Turbo, lower pricing, custom GPTs, a good bunch of announcements all-round! https://openai.com/pricing

I thought GPT-4 had access to internet now?

The “browse with bing” feature allows it to fetch a single webpage into the context, but the new cutoff allows _everything crawled_ to be context (up to the new date, that is)

Re: New models and developer products

#478

Post by stavros »

DaiPlusPlus wrote: post #271
tornato7 wrote: post #58

A few notes on pricing: - GPT-4 Turbo vision is much cheaper than I expected. A 768*768 px image costs $0.00765 to input. That's practical to replace more specialized computer vision models for many use-cases. - ElevenLabs is $0.24 per 1K characters while OpenAI TTS HD is $0.03 per 1K characters. Elevenlabs still has voice copying but for many use-cases it's no longer competitive. - It appears that there's no additio…

> GPT-4 Turbo vision is much cheaper than I expected. A 768*768 px image costs $0.00765 to input. That's practical to replace more specialized computer vision models for many use-cases That's still on-the-orders-of $0.01/image - whereas a simple binary-classifier I wrote using OpenCV and simple histograms (no NNs here) would be like $0.0000001/image (if I had to put a price on it - on the basis that I wrote it 8 year…

Your binary classifier can't tell me that my image contains a photo of a cat on a painting of a surfboard.

Re: New models and developer products

#479

Post by stavros »

weird-eye-issue wrote: post #416
OddMerlin wrote: post #405

Is langchain still relevant with the release of AssistantAI? It seems managing the context window, state, etc is now all taken care of by Assistant AI. I guess langchain is still relevant for non-OpenAI options?

The average langchain dev will struggle to even call the API without langchain so I think it's safe

I find the opposite to be true, I use the OpenAI APIs as they are, but really struggle to figure out how the hell LangChain works.

Re: New models and developer products

#480

Post by cubefox »

freedomben wrote: post #426
qingcharles wrote: post #380

Earlier quoted context omitted.

I was in prison when ChatGPT came out. All I knew of it was a headline that flashed past really fast on CNN and I called my buddy and said "What the hell is Chat OPT?" I'd just finished reading The Singularity is Near for the second time too...

The singularity is near is a great book. Hilarious that you read that for the second time and then got out and saw chat gpt! I love kurzweil but his estimates of timeline are often pretty over optimistic, so I'd be really wondering.

On Metaculus the arrival for weakly general AI was predicted for 2045 two years ago. Now it's at 2026.

https://www.metaculus.com/questions/3479/date-weakly-general...

Post reply on HN