Viewing profile — jne2356

The resources for the GPT-3 replication will be provided by the cloud company CoreWeave.

They get this suggestion a lot. There's a section in their FAQ that explains why it's infeasible. https://github.com/EleutherAI/info

OpenAI only trained the full sized GPT-3 once. Hyperparameter sweep was conducted on significantly smaller models (see: https://arxiv.org/abs/2001.08361 )

This is not true. The OpenAI team only trained one full-sized GPT-3, and conducted their hyperparameter sweep on significantly smaller models (see: https://arxiv.org/abs/2001.08361…

EleutherAI has already secured the resources necessary. They get the seti@home suggestion a lot. There's a section in their FAQ that explains why it's infeasible. https://github.co…

The cloud company CoreWeave has agreed to provide the GPU resources necessary.

GPT-3 does not fit in any one GPU that exists at present. It's already spread out across multiple GPUs.

They get this suggestion a lot. There's a section in their FAQ that explains why it's infeasible. https://github.com/EleutherAI/info