Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: BBC “In Our Time”, categorised by Dewey Decimal, heavy lifting by GPT

genmon.github.io

Post reply on HN
Print view View original

1–10 of 177 posts

Show HN: BBC “In Our Time”, categorised by Dewey Decimal, heavy lifting by GPT

#1

Post by genmon »

I'm a big fan of the BBC podcast In Our Time -- and (like most people) I've been playing with the OpenAI APIs.

In Our Time has almost 1,000 episodes on everything from Cleopatra to the evolution of teeth to plasma physics, all still available, so it's my starting point to learn about most topics. But it's not well organised.

So here are the episodes sorted by library code. It's fun to explore.

Web scraping is usually pretty tedious, but I found that I could send the minimised HTML to GPT-3 and get (almost) perfect JSON back: the prompt includes the Typescript definition.

At the same time I asked for a Dewey classification... and it worked. So I replaced a few days of fiddly work with 3 cents per inference and an overnight data run.

My takeaway is that I'll be using LLMs as function call way more in the future. This isn't "generative" AI, more "programmatic" AI perhaps?

So I'm interested in what temperature=0 LLM usage looks like (you want it to be pretty deterministic), at scale, and what a language that treats that as a first-class concept might look like.

Show HN: BBC “In Our Time”, categorised by Dewey Decimal, heavy lifting by GPT
genmon.github.io

Re: Show HN: BBC “In Our Time”, categorised by Dewey Decimal, heavy lifting by GPT

#6

Post by valgaze »

LeoPanthera wrote: post #5
haolez wrote: post #3

Wait. Is temperature=0 "pretty deterministic", or is it deterministic?

It’s my understanding that 0 is completely deterministic, except for when the model is updated, which does happen occasionally.

@haolez I just tested w/ davinci-003 w/ temperature to 0

Prompt: https://imgur.com/YtQ4fbf --

Reveal the question marks in an interesting way:

The dog goes ????????????? --

With temperature == 0, it consistently ("pretty deterministically"?) generated "woof!"

ex. The dog goes woof!

Re: Show HN: BBC “In Our Time”, categorised by Dewey Decimal, heavy lifting by GPT

#7

Post by genmon »

haolez wrote: post #3

Wait. Is temperature=0 "pretty deterministic", or is it deterministic?

In the strict definition, it's deterministic: you get the same response for the same prompt, each time (given the exact same model).

But the prompt is parameterised. The bulk of the prompt is requesting a list of guests and speakers to be extracted, and the episode synopsis is appended as the "parameter". And I've noticed that the variation of the parameter changes what the overall prompt returns... so it might start being less reliable at responding with valid JSON, for example.

So it's instance-deterministic but, across a range of parameters, class-fuzzy, if that makes sense?

Re: Show HN: BBC “In Our Time”, categorised by Dewey Decimal, heavy lifting by GPT

#10

Post by genmon »

PaulHoule wrote: post #4

What's the prompt?

One of the prompts is

  Extract the description and a list of guests from the supplied episode notes from a podcast.

  Also provide a Dewey Decimal Classification code and label for the description

  Return valid JSON conforming to the following Typescript type definition:

  {
    "description": string,
    "guests": {"name": string, "affiliation": string | null}[]
    "dewey_decimal": {"code": string, "label": string},
  }

  Episode synopsis (Markdown):

  {notes}

  Valid JSON:
(And the completion tends to be JSON, but not always.)
Post reply on HN