ChatGPT is a blurry JPEG of the web

Post by bbor » Thu, Feb 09, 2023, 8:16 PM UTC

avereveard wrote: post #72

chapium wrote: post #55

Earlier quoted context omitted.

I’m just searching the comments for novel use cases where its effective. Most articles I’ve read seem like either moral panics or snake oil. I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. It really does seem curated at times, leading me to think this will eventually become a fad or replaced by a more advanced algorithm.

Want niche use case? I feed chatgpt the whole warstuff one page rule miniature combat system, and I can ask it to generate thematic units. I generated a wide variety of content from LOTR to ships. it even created new trait to complement the existing one when needed by special units, along with explanation of the mechanics of the trait. It doesn't quite understand positioning, but it will simulate round of combat betw…

Post by Der_Einzige » Thu, Feb 09, 2023, 8:20 PM UTC

hnfong wrote: post #46

Ari_Rahikkala wrote: post #31

> Models like ChatGPT aren’t eligible for the Hutter Prize for a variety of reasons, one of which is that they don’t reconstruct the original text precisely—i.e., they don’t perform lossless compression. Small nit: The lossiness is not a problem at all. Entropy coding turns an imperfect, lossy predictor into a lossless data compressor, and the better the predictor, the better the compression ratio. All Hutter Prize c…

Fabrice Bellard has a project that does precisly this. And does it extremely well, apparently. Previously on HN: https://news.ycombinator.com/item?id=27244004 Apparently it leads the compression of enwik9 ( http://www.mattmahoney.net/dc/text.html ) . Not sure why it isn't eligible for the Hutter Prize, there's some speculations in the previous discussion but I don't know whether they're true.

Post by nuggets_ » Thu, Feb 09, 2023, 8:36 PM UTC

abecedarius wrote: post #208

An essay making reasonable points, but overall it strikes me like a dismissal circa 1980 of personal computers as toys. My first day with ChatGPT I tried teaching it my hobby dialect of Lisp (unlikely to be in its training set) and then asking it to implement symbolic differentiation. Its attempt was very scatterbrained, but not completely hopeless. If you don't think that required any thinking from it, I don't want…

Post by dgb23 » Thu, Feb 09, 2023, 8:44 PM UTC

mrtksn wrote: post #7

They can always use AI based solutions to unblur the JPEG, like this: https://twitter.com/maxhkw/status/1373063086282739715

Post by williamcotton » Thu, Feb 09, 2023, 8:45 PM UTC

PaulHoule wrote: post #203

williamcotton wrote: post #179

Earlier quoted context omitted.

There is definitely a misconception about how to use a tool like ChatGPT. If you give it an analytic prompt like "turn this baseball box score into an entertaining outline" it will reliably act as a translator because all of the facts about the game are contained in the prompt. If you give it a synthetic prompt like "give me quotes from the broadcasters" it will reliably acts as a synthesizer because none of the fact…

That's a touch beyond state of the art but we might get there. If there was one big problem w/ today's LLMs it is that the attention window is too short to hold a "complete" document. I can put the headline of an HN submission through BERT and expect BERT to capture it but there is (as of yet) no way to cut up a document up into 512 (BERT) or 4096 (ChatGPT) token slices and then mash those embeddings together to make…

  It's built with Supabase/Postgres, and consists of several key parts:
  
  Parsing the Supabase docs into sections.
  Creating embeddings for each section using OpenAI's embeddings API.
  Storing the embeddings in Postgres using the pgvector extension.
  Getting a user's question.
  Query the Postgres database for the most relevant documents related to the question.
  Inject these documents as context for GPT-3 to reference in its answer.
  Streaming the results back to the user in realtime.