New models and developer products • Hacker News Board

Re: New models and developer products

#241

Post by simonw » Mon, Nov 06, 2023, 9:35 PM UTC

I just released a new version of my LLM CLI tool with support for the new GPT-4 Turbo model: https://llm.datasette.io/en/stable/changelog.html#v0-12

You can install it like this:

    pipx install llm

Then set an API key:

    llm keys set openai

Then run a prompt through GPT-4 Turbo like this:

    llm -m gpt-4-turbo "Ten great names for a pet walrus"
    # Or a shortcut:
    llm -m 4t "Ten great names for a pet walrus"

Here's a one-liner that summarizes all of the comments in this Hacker News conversation (taking advantage of the new long context length):

    curl -s "https://hn.algolia.com/api/v1/items/38166420" | \
      jq -r 'recurse(.children[]) | .author + ": " + .text' | \
      llm -m gpt-4-turbo 'Summarize the themes of the opinions expressed here,
      including direct quotes in quote markers (with author attribution) for each theme.
      Fix HTML entities. Output markdown. Go long.'

Example output here: https://gist.github.com/simonw/d50c8634320d339bd88f0ef17dea0...

Re: New models and developer products

#242

Post by oezi » Mon, Nov 06, 2023, 9:36 PM UTC

dragonwriter wrote: post #150

m3kw9 wrote: post #141

Earlier quoted context omitted.

How does this absolve vectordbs

If you are using OpenAI, the new Assistants API looks like itnwill handle internally what you used to handle externally with a vector DB for RAG (and for some things, GPT-4-Turbo’s 128k context window will make it unnecessary entirely.) There are some other uses for Vector DBs than RAG for LLMs, and there are reasons people might use non-OpenAI LLMs with RAG, so there is still a role for VectorDBs, but it shrunk a lo…

OpenAI is still way too expensive to run a corporate knowledge base on top

Re: New models and developer products

#243

Post by 0x0000000 » Mon, Nov 06, 2023, 9:37 PM UTC

PH95VuimJjqBqy wrote: post #230

airstrike wrote: post #194

Earlier quoted context omitted.

> Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data. https://news.ycombinator.com/newsguidelines.html

likewise, stop posting these references to the guidelines, if you feel the guidelines are being broken report it to the moderators and move on. This type of post is actually more disruptive than the post you're replying to.

> This type of post is actually more disruptive than the post you're replying to.

As is yours. The author of that comment has contact information in their information, why not unicast them?

Re: New models and developer products

#244

Post by throw03172019 » Mon, Nov 06, 2023, 9:37 PM UTC

StanAngeloff wrote: post #153

WanderPanda wrote: post #28

Does anyone have an idea why they are so open about Whisper? Is it the poster child project for OAI people scratching their open source itch? Is there just no commercial value in speech to text?

I personally use Whisper to transcribe painfully long meetings (2+ hours). The transcripts are then segmented and, you guessed it, entered right into GPT-4 for clean up, summarisation, minutes, etc. So in a sense it's a great way to get more people to use their other products?

This sounds amazing. Would you be willing to share your code? Thanks!

Re: New models and developer products

#245

Post by teaearlgraycold » Mon, Nov 06, 2023, 9:38 PM UTC

whytai wrote: post #71

Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…

I’ve been keeping my eye on a YC startup for the last few months that I interviewed with this summer. They’ve been set back so many times. It looks like they’re just “ball chasing”. They started as a chatbot app before chatgpt launched. Then they were a RAG file processing app, then enterprise-hosted chat. I lost track of where they are now but they were certainly affected by this announcement.

You know you’re doing the wrong thing if you dread the OpenAI keynotes. Pick a niche, stop riding on OpenAI’s coat tails.

Re: New models and developer products

#246

Post by tomjen3 » Mon, Nov 06, 2023, 9:43 PM UTC

thethimble wrote: post #214

shmatt wrote: post #208

Earlier quoted context omitted.

The investors will only get their 1000x if OpenAI can convince people its risk free to use. So they'll happily cover the legal battle to prove it or spent every last company penny trying

Or, alternatively, copyright risk is a major concern for real customers, and this is a major step forward in addressing that. Not everything needs to be so cynical. What’s good for investors can be good for users as well.

Or if it turns out that copyright is a major issue OpenAI now have standing to fight any claims.

Re: New models and developer products

#247

Post by muttled » Mon, Nov 06, 2023, 9:43 PM UTC

faeriechangling wrote: post #51

minimaxir wrote: post #4

Most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform, which is not surprising given increased competition in the space. The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'm mixed on the presentation and will need to read the fine print on…

It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lot more money spent. I've looked into it and maybe the most practical end-to-end system for owning my own LLM is to run a couple of 3090s on a consumer motherboard at substantial running cost to keep them up 24/7 and that's not powerful enough to cut it and rath…

For me personally, being able to fine-tune the local LLM's at a much higher rank and training more layers is very useful for (somewhat unreliably) embedding information. AFAIK the OpenAI fine-tuning is more geared towards formatting the output.

Re: New models and developer products

#248

Post by singularity2001 » Mon, Nov 06, 2023, 9:45 PM UTC

petercooper wrote: post #236

singularity2001 wrote: post #131

did they break the api? from openai import OpenAI Traceback (most recent call last): File " ", line 1, in ImportError: cannot import name 'OpenAI' from 'openai' If so where is the current documentation?

v1.0/1.1 of the `openai` Python package differ significantly from the 0.x versions. You'll want to upgrade the package before following the instructions you were following. More info here: https://github.com/openai/openai-python/discussions/631

thanks, migration guide included!

I wished this was linked or integrated visibly in public documentation

Re: New models and developer products

#249

Post by jddj » Mon, Nov 06, 2023, 9:46 PM UTC

128k context is getting up to the point where it will fit all of a moderately large codebase, right?

Re: New models and developer products

#250

Post by PH95VuimJjqBqy » Mon, Nov 06, 2023, 9:46 PM UTC

0x0000000 wrote: post #243

PH95VuimJjqBqy wrote: post #230

Earlier quoted context omitted.

likewise, stop posting these references to the guidelines, if you feel the guidelines are being broken report it to the moderators and move on. This type of post is actually more disruptive than the post you're replying to.

> This type of post is actually more disruptive than the post you're replying to. As is yours. The author of that comment has contact information in their information, why not unicast them?

https://en.wikipedia.org/wiki/Paradox_of_tolerance

> The paradox of tolerance states that if a society's practice of tolerance is inclusive of the intolerant, intolerance will ultimately dominate, eliminating the tolerant and the practice of tolerance with them. Karl Popper described it as the seemingly self-contradictory idea that, in order to maintain a tolerant society, the society must retain the right to be intolerant of intolerance.

To answer your question more succinctly, because the poster isn't the only person who will read these comments.