In the download script, it skips parts of the model (02 and 83); any ML people have ideas why you'd do that?
YaLM-100B: Pretrained language model with 100B parameters
291–300 of 666 posts
Re: YaLM-100B: Pretrained language model with 100B parameters
#292Earlier quoted context omitted.
Quoted post unavailable.
It's not that Russia Bad, it's that if you know a search engine will serve you censored, biased results that makes it an unreliable search engine.
People using duckduckgo bangs often use different search engines for different topics.
I usually try ddg first, if it's tech I use Bing, if it's local I use Google.
Re: YaLM-100B: Pretrained language model with 100B parameters
#293Re: YaLM-100B: Pretrained language model with 100B parameters
#294Earlier quoted context omitted.
That's definitely the future, personalized entertainment and social interactions will be big. I could watch a movie made for me, and discuss it with a bunch of chat bots. The future will be bubbly as hell, people will be decaying in their safe places as the hellscape rages on outside.
> I could watch a movie made for me We're a long, long way from this. Stringing words/images together into a coherent sequence is arguably the easy bit of creating novels/films, and computers still lag a long way behind humans in this regard. Structuring a narrative is a harder, subtler step. Our most advanced ML solutions are improving rapidly, but often struggle with coherence over a single paragraph; they're not g…
> We're a long, long way from this.
We’re probably 18 months away from this. We’re probably less than 5 years away from being able to do this on local hardware. AI/ML is advancing faster than most people realise.
Re: YaLM-100B: Pretrained language model with 100B parameters
#295Earlier quoted context omitted.
> Google overlords neutered their own product out of fear over lawyers/regulation What kind of lawyers/regulation do you have in mind? If anything, I'd find the opposite: lawyers and copyright holders should be grateful for such a tool that - when it was still working - allowed you to trace websites using your images illegally. Now they all use Yandex for this purpose, with relatively good results.
You misunderstood parent post. It's about Google not being sued for discrimination. https://www.washingtonpost.com/news/the-intersect/wp/2016/08... https://www.theguardian.com/technology/2016/apr/08/does-goog... https://www.bloomberg.com/news/articles/2021-10-19/google-qu... https://theconversation.com/googles-algorithms-discriminate-...
Oh I see. What I'm looking for is the reason why they broke the reverse image search. It was working well many years ago but some time after that they switched it to some strange image classifier (I upload an image of an apple to find exactly the same image to track its license of origin, and it says "possibly an image of an apple" - oh thank you Google I didn't know that.)
Re: YaLM-100B: Pretrained language model with 100B parameters
#296is this the first GPT-like models which is fully opensource ? none of the others are right ?
Aren't eleutherai's model so?
Re: YaLM-100B: Pretrained language model with 100B parameters
#297Earlier quoted context omitted.
Do you have the same level of concern about the leverage Five-Eyes intelligence agencies have over Facebook and Google?
There's a world of difference between Five-Eyes and being harrassed, mobbed, jailed, having a "Z" and "traitor" spray painted on your apartment door or being murdered. By conflating those two clearly means you don't understand what's going on Russia and its Putin-controlled satellites like Belarus.
There's a world of difference between living in Russia and using Yandex to search for how to kill Putin and living in the west and using Yandex to search for how to spin up a FastAPI server.
By conflating those two clearly means you don't understand that everyone isn't in the same situation as yourself.
Re: YaLM-100B: Pretrained language model with 100B parameters
#298Earlier quoted context omitted.
Just take it to America from us, thank you. Along with VK. Great search engine and a social network. Full of backdoors for thugs and corrupt police, censorship and other lovely stuff... but you'll probably say that Google is full of it too, because you had no experience of living in Russia.
and you haven't had experience of living in the states.
This comment needs expansion. Tell us your experience of police brutality and corruption in the US.
Re: YaLM-100B: Pretrained language model with 100B parameters
#299Earlier quoted context omitted.
I believe you're confusing the amount of A100 graphics cards used to train the model (the cluster was actually made up of 800 A100s), and the amount you need to run the model : > The model [...] is supposed to run on multiple GPUs with tensor parallelism. > It was tested on 4 (A100 80g) and 8 (V100 32g) GPUs, [but should work] with ≈200GB of GPU memory. I don't know what the price of a V100 is, but given $10k a piece…
The $10k price is for an A100 with 40GB ram, so you need 8 of those. If you can get your hands on the 80GB variant, 4 are enough. Also, if you want to have a machine with eight of these cards, it will need to be a pretty high-spec rack-mounted or large tower. To feed these GPUs, you will want to have a decent amount of PCIe-4 lanes, meaning EPYC are the logical choice. So that's $20k for an AMD EPYC server with at le…
There is also $5k A6000 with 48GB
Re: YaLM-100B: Pretrained language model with 100B parameters
#300Earlier quoted context omitted.
You misunderstood parent post. It's about Google not being sued for discrimination. https://www.washingtonpost.com/news/the-intersect/wp/2016/08... https://www.theguardian.com/technology/2016/apr/08/does-goog... https://www.bloomberg.com/news/articles/2021-10-19/google-qu... https://theconversation.com/googles-algorithms-discriminate-...
Oh I see. What I'm looking for is the reason why they broke the reverse image search. It was working well many years ago but some time after that they switched it to some strange image classifier (I upload an image of an apple to find exactly the same image to track its license of origin, and it says "possibly an image of an apple" - oh thank you Google I didn't know that.)
Do they at least tell you the type of Apple it is?