Deploy your own GPT-3 API with Docker and fly

#1

Post by mayyue » Sat, Feb 04, 2023, 12:29 PM UTC

Deploy your own GPT-3 API with Docker and fly
github.com

Re: Deploy your own GPT-3 API with Docker and fly

#2

Post by amrb » Sat, Feb 04, 2023, 1:13 PM UTC

There's also the option of caching and sessions with https://github.com/HazyResearch/manifest

Re: Deploy your own GPT-3 API with Docker and fly

#3

Post by kolinko » Sat, Feb 04, 2023, 1:25 PM UTC

Why like this, and not just using the API directly through Python or sth?

Re: Deploy your own GPT-3 API with Docker and fly

#4

Post by supermatt » Sat, Feb 04, 2023, 1:27 PM UTC

kolinko wrote: post #3

Why like this, and not just using the API directly through Python or sth?

Because they are trying to get inference for free by scraping chatGPT. Of course, "free" here also means "mostly unavailable" given the reliability of chatGPT recently.

Re: Deploy your own GPT-3 API with Docker and fly

#5

Post by mshake2 » Sat, Feb 04, 2023, 1:38 PM UTC

So this builds a 100mb docker image that proxies an http request to openai? Am I missing something? I don't want to discourage new engineers from tinkering, but can someone please explain why this is reaching the front page of HN?

Re: Deploy your own GPT-3 API with Docker and fly

#6

Post by [deleted] » Sat, Feb 04, 2023, 1:46 PM UTC

mshake2 wrote: post #5

So this builds a 100mb docker image that proxies an http request to openai? Am I missing something? I don't want to discourage new engineers from tinkering, but can someone please explain why this is reaching the front page of HN?

[deleted]

Re: Deploy your own GPT-3 API with Docker and fly

#7

Post by [deleted] » Sat, Feb 04, 2023, 1:56 PM UTC

mshake2 wrote: post #5

So this builds a 100mb docker image that proxies an http request to openai? Am I missing something? I don't want to discourage new engineers from tinkering, but can someone please explain why this is reaching the front page of HN?

[deleted]

Re: Deploy your own GPT-3 API with Docker and fly

#8

Post by krzepah » Sat, Feb 04, 2023, 1:59 PM UTC

mshake2 wrote: post #5

So this builds a 100mb docker image that proxies an http request to openai? Am I missing something? I don't want to discourage new engineers from tinkering, but can someone please explain why this is reaching the front page of HN?

I agree with you, the caching mechanism and ways to communicate with the API from your scripts would have been way more interesting to read about

Re: Deploy your own GPT-3 API with Docker and fly

#9

Post by eternalban » Sat, Feb 04, 2023, 2:00 PM UTC

mshake2 wrote: post #5

So this builds a 100mb docker image that proxies an http request to openai? Am I missing something? I don't want to discourage new engineers from tinkering, but can someone please explain why this is reaching the front page of HN?

~somewhat +agree re. these gpt novelty hacks/projects, however note the kitty is doing more than just proxying a request:

https://github.com/queercat/node-chatgpt-api#features

(I feel we're back in the 90s and someone has released something called Mosaic .. same vibes. A frenzy is coming and right now we're in potential low hanging fruit riches just like those html (yes, just html) millionaires -- I know at least 2)

Re: Deploy your own GPT-3 API with Docker and fly

#10

Post by mayyue » Sat, Feb 04, 2023, 2:04 PM UTC

mshake2 wrote: post #5

So this builds a 100mb docker image that proxies an http request to openai? Am I missing something? I don't want to discourage new engineers from tinkering, but can someone please explain why this is reaching the front page of HN?

i thought it was interesting and few (3-4) people would like it. i ABSOLUTELY did not expect for anyone to actually look at this with any amount of thought nor care.

it is a hack that this works and is guaranteed something that does not make OpenAI happy. rewriting to reflect this.