521–530 of 1000 posts
I was able to see LaMDA in action about 1.5 years ago. It was just as awe-inducing as ChatGPT. Google is not behind in this race, they're simply slower to roll out new functionality. And maybe rightly so? Some of the things I saw ChatGPT say would have been multi-day international bad headlines for Google, if a Google AI had said them instead of an upstart startup.
Earlier quoted context omitted.
Google has far more resources for training models and inference. Likely more than all their nearest competitors combined.
Source?
For few weeks now I had a thought experiment of creating a LLM search engine trained on books. Such LLM search engine would be most reliable if you seek knowledge but as others mentioned if you want up to date information, search engine is probably your fastest and easiest way to go. But actually I wonder and somewhat doubt that data, information and knowledge in books is lagging significantly behind Web's data, information and knowledge. One big advantage of books is that they are more reliable and more in depth source of information and knowledge than some random site, blog or Wikipedia article. Scientific research papers also come to mind as a highly reliable source of information and knowledge.
Earlier quoted context omitted.
You are missing the Microsoft partnership. MS has a 48% stake in OpenAI and provides all the infrastructure through Azure, including purpose built machines for model training. Microsoft has also launched GitHub Copilot, summarization features in Teams, and is widely reported to be adding GPT features to Bing.
MS cannot deliver successful stuff on the web at scale. They can deliver unsuccessful stuff on the web at scale, they can deliver successful stuff that turns out to be inconsequential for their bottom line on the web in the long-term (AJAX came from MS), but it's just not in their DNA to take over the web. They had lots of chances to do it during the last 20 years or so, they had all the silver bullets at their dispo…
Not sure what decade you're stuck in here with comments about ajax
Question for AI experts out there: ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021. Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog O…
At least the Bing ChatGPT integration can get data from websites and all of that, and cite the sources. Probably using REALM. Not sure if is ChatGPT-4 that would use REALM together or if is MSFT integrating both.
https://medium.com/@owenyin/scoop-oh-the-things-youll-do-wit...
Similar to how http://perplexity.ai/ works to get info on current data.
Question for AI experts out there: ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021. Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog O…
> So my question is, how hard of a problem is that with LLMs? I get the sense that LLMs are trained on a very large data set all at once, but that it is difficult to incrementally update them with new data. It's the opposite. Getting an LLM to learn the basics of grammar and paragraph-level language structure is the "hard" part. Once you have that, further fine-tuning, specialisation, or other incremental changes are…
That sounds like InstructGPT
Question for AI experts out there: ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021. Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog O…
There is already LLM search engines out there https://perplexity.ai
Earlier quoted context omitted.
Those new language models are "Google killers" because they reset all the assumptions that people have made about search for several decades. Imagine that people start using those chat bots massively as a replacement for Google search. Then the notion of keyword disappears. Google AdSense becomes mostly irrelevant. Of course, Google is a giant today with a history of machine learning innovation. So they have a good c…
The point about the economics of running these models is an important one that slides under the radar a lot of times. The training costs for large language models like GPT are enormous, and the inference costs are substantial too. Right now things like ChatGPT are very cool parlor tricks, but there's absolutely no way to justify them in terms of the economics of running the service today. Obviously this is all going…
2. Don’t Google have specialised hardware for training neural networks? If the costs of training/inference are very significant won’t Google (with their ASIC and hardware design team) have a significant advantage? It seems to me that their AI hardware was developed because they saw this problem coming a long way off.
Question for AI experts out there: ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021. Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog O…
> So my question is, how hard of a problem is that with LLMs? I get the sense that LLMs are trained on a very large data set all at once, but that it is difficult to incrementally update them with new data. It's the opposite. Getting an LLM to learn the basics of grammar and paragraph-level language structure is the "hard" part. Once you have that, further fine-tuning, specialisation, or other incremental changes are…