641–650 of 1000 posts
Earlier quoted context omitted.
Google has far more resources for training models and inference. Likely more than all their nearest competitors combined.
If you’re talking about physical hardware, Google pales in comparison to Amazon, and Gcloud is still smaller than Azure. It’s possible google’s private compute makes up for the Azure difference but it’s not like they’re in different leagues in terms of access to hardware.
Question for AI experts out there: ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021. Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog O…
I actually think Microsoft is betting on the wrong dog. But let's see :)
Question for AI experts out there: ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021. Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog O…
That's not quite correct. It's trivial to update the model with new information. What's not trivial is to encode priorities or time into the model. Fundamentally, the model doesn't understand about the concept of time, so you can't easily condition it on "give me only information from last year". You can get there and nudge it in the right direction by generating training datasets for such queries and fine-tuning the model on them, by oversampling more recent data in your training procedure, or by adjusting your gradients for more recent data, but all of these are still "fuzzy" - the model may or may not do exactly what you want during prediction time and could still give you old results.
However, I would assume that Google is doing something more sophisticated and that's it's not just a plain LLM like GPT3. It's probably a more complex architecture with external systems and data sources around it that tries to solve the above using heuristics.
That being said, the current Google search is extremely bad at time relevance too...
Earlier quoted context omitted.
I think that counts as teaching.
Close enough, I'm just peevish about that word. Teaching is miles away from anything in ML or deep learning practice today.
Anybody remember how we used to learn new words and build our vocabulary? Perhaps it went something like: "What does this word mean"? "Look it up in the dictionary" "Why can't you just tell me??" I remember the hassle of looking things up in a paper dictionary. Over the long term, it massively helped improve my ability to not just recall a simple definition, but also learn new words, as they were adjacent to what I w…
FWIW, I'm not sure I've ever done that and I've learned two languages (outside of a classroom environment, which I expect makes a difference). I usually just encounter words in books and infer their meaning from context. As I encounter the word more and more and tried to use it myself I gained an understanding of what it meant.
It is an interesting bootstrapping problem.
Earlier quoted context omitted.
That may all be the case but then why has Google been failing to innovate and also maintain anything the past few years? The past fews years we’ve seen Google products die and deteriorate. Search has gotten worse, they’ve killed dozens of products and services (even ones that they promised would be around for the long haul). GPhotos is no longer free (after putting most competitors out of business). Google lack any i…
I'm still salty about Google Reader, but they've innovated (Bard is the subject in the Op and a pretty big AI innovation) and maintained (Gmail, Docs, Chrome, Golang, Android) a bunch of stuff.
Bard has yet to be released or evaluated and they are already using marketing speak to let people know it won't be as good as ChatGPT.
>and maintained (Gmail, Docs, Chrome, Golang, Android)
I wouldn't call any of those maintained... stagnated at best with degradation around the edges all around. For each one of those there are 10 projects Google has killed or let wither on the vine over the past 5 years.
And we then fired anyone internally who attempted to actually apply them...
Earlier quoted context omitted.
That may all be the case but then why has Google been failing to innovate and also maintain anything the past few years? The past fews years we’ve seen Google products die and deteriorate. Search has gotten worse, they’ve killed dozens of products and services (even ones that they promised would be around for the long haul). GPhotos is no longer free (after putting most competitors out of business). Google lack any i…
>GPhotos is no longer free (after putting most competitors out of business). Maybe the putting most competitors out of business has something to do with GPhotos not being free? I think it does.
Fundamentally, google would have never created a chatGPT, and this response feels like youtube shorts in the face of tiktok.
Question for AI experts out there: ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021. Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog O…
(source: I helped build Sparrow, DeepMind's RLHF model, that does this by learning to use google under the hood)