161–170 of 217 posts
Are we heading to the (distant) future where to make progress in any field you have to spend big $$$ to train a model?
This is akin to teaching an adult human about a specific domain. Better to just do that than make a whole new human from scratch!
I never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
I never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
I never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
The concern, nominally, is that it’s too good. They’re worried that it'll lead to a huge influx of things like spam comments and fake news articles.
The future is not as dark as it seems because of the rat race of megacorps. You can use reduced versions of language models with extremely good results. I was involved in training the first-ever GPT2 for Bengali language, but with 117 million parameters. It took a month's effort (training + writing code + setup) and about $6k in TPU cost, but Google Cloud covered it. Anyway, it is surprisingly good. We fine-tuned the…
I'm glad this all worked out for you. This is unrelated, but I just want to say that I hate how many people Google managed to convert to TPU with their research program and that their managed TPU/GPU offerings are absolutely horrible and infuriating to work with unless you somehow get on their radar.
People were blaming cryptocurrencies miners for the prices of GPUs, when in fact it was the AI researchers who bought all the GPUs. :D I wonder what if somebody designs an electronic currency rewarded as payment for general GPU computations instead of just computing hashes? You pay some $, to train your model and the miner gets some coins. Every one is happy, electricity is not wasted and the GPUs gets used for a rea…
Earlier quoted context omitted.
1. There's a world of problems (such as "perception-related" e.g. vision and NLP) which we tried to solve for decades with symbolic AI and got worse results than what nowadays first-year students can do as a homework with ML; 2. For your example of chess, for some time now ML engines are pretty much untouchable by engines based on pre-ML methods.
>> 1. There's a world of problems (such as "perception-related" e.g. vision and NLP) which we tried to solve for decades with symbolic AI and got worse results than what nowadays first-year students can do as a homework with ML; Perception tasks were traditionally attempted with statistical machine learning approaches rather than symbolic AI, for example the Perceptron was a very early neural network that was used in…
I never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
I never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
OpenAI has strict requirements for the usage of GPT-3. For instance, you cannot automate posting to social media without a human in the middle.