191–194 of 194 posts
Does anyone else find it ironic that all these ChatGPT "clones" are popping up when OpenAi is supposed to be the ones open sourcing and sharing their work? I guess: "You Either Die A Hero, Or You Live Long Enough To See Yourself Become The Villain"?
Sam Altman has turned into a megalomaniac.
Earlier quoted context omitted.
The 3 hours is the instruction fine-tuning. The base foundational model is GPT-J which was already provided by Eleuther-AI and has been around for a couple of years. Note: I work at Databricks and am familiar with this project but didn't work on it.
Do you know why GPT-J is being used instead of NeoX or any of the other larger open source models?
Anyone care to comment on why the output of these models changes so dramatically given so little Q&A training? It's a 6 billion parameter model with only 50 thousand Q&A samples. It's clear the model already "knows" the format of a Tweet (short length, attention-grabbing, contains hashtags). The model also knows stuff about language models (word2vec, tokenization), and can include entities from the question in its re…