11–20 of 29 posts
Finally the Metaverse is taking shape. Those human friends on Facebook can now be replaced with Metafriends
Can someone ELI5 me why suddenly LLM and transformers became all the rage in AI scene?
* 2019 - GPT2 is presented to the public and demonstrates transformer-based LLMs and their emergent capabilities
* 2020 - GPT3 is released and shows that throwing more compute at LLMs yields significantly better LLMs
* 2022 - ChatGPT is openly released to the public, showcasing the versatility of an LLM-based chatbot
In my experience transformers have been all the rage in the researcher/enthusiast scene since 2019. The technology has just gradually matured enough to become viable for consumer use, which is why you see the industry rushing to adopt it. ChatGPT was the watershed moment for the tech because suddenly anyone in the world could sign up for free, open a chat dialogue and start getting legible LLM output without needing to understand the tech or prompt engineering.
Finally the Metaverse is taking shape. Those human friends on Facebook can now be replaced with Metafriends
The funny thing is, something like the Metaverse might actually be helpful to construct multimodal models - it could provide higher fidelity training data.
Can someone ELI5 me why suddenly LLM and transformers became all the rage in AI scene?
Transformers were used on text to train models without needing labeled data. People realized that simply scaling the data and models meant better performance. when they scaled even further, emergent intelligence started appearing and the models were dominating every known task. Now everyone wants an LLM
I don't think this is very accurate. How well does LLM perform on image segmentation, for example?
The chain-of-thought prompting in section 4.5 is extremely interesting to me, but it looks like they're missing a test group - what is the performance if the image is simply described and then the task is evaluated using only the text of the description, not only when combined with the image.
Slightly interestingly, the last commit changes a heading from "AI" to "AGI" https://github.com/microsoft/unilm/commit/bbbb5b4b06c2dd501d...
The model is relatively small, 1.6B. I am guessing the goal is to be able to run on a user's home PC. But it would be interesting to see how much better it gets if you scale it up by a factor of 10 or 100.
Earlier quoted context omitted.
Transformers were used on text to train models without needing labeled data. People realized that simply scaling the data and models meant better performance. when they scaled even further, emergent intelligence started appearing and the models were dominating every known task. Now everyone wants an LLM
> and the models were dominating every known task I don't think this is very accurate. How well does LLM perform on image segmentation, for example?
Can someone ELI5 me why suddenly LLM and transformers became all the rage in AI scene?
* 2017 - the Attention Is All You Need paper proposes transformers, suggests language translation as their primary application * 2019 - GPT2 is presented to the public and demonstrates transformer-based LLMs and their emergent capabilities * 2020 - GPT3 is released and shows that throwing more compute at LLMs yields significantly better LLMs * 2022 - ChatGPT is openly released to the public, showcasing the versatilit…