21–25 of 25 posts
we've like barely trained these things? the entirety of common crawl is 424 terabytes. that's merely 6 days of 8K raw video.
> No matter how fast Searle is, he won't be able to come up with a beautiful and original Chinese poem that has the creative spark special to humans Why not? > Of course, at some level of complexity, it will be stuck in a local maximum of work quality simply because the book has no guide on how to solve the problem at hand. I find this a pretty un-optimistic view, especially from someone building a coding autopilot.…
It seems we've reached the point that understanding of LLMs would be a great candidate for the beginner/intermediate/expert meme. "It's just autocomplete" -> "It's got a world model, it's thinking for itself" -> "It's just autocomplete".
we've like barely trained these things? the entirety of common crawl is 424 terabytes. that's merely 6 days of 8K raw video.
This is for LLM's which deal mainly with text. An entire book can be stored as .42 MB according to https://www.quora.com/How-many-megabytes-are-in-a-book . 424 terrabytes text is over a billion books worth of data. On the common crawl website it even says "Over 250 billion pages spanning 17 years." That's an impressive amount of information.
Earlier quoted context omitted.
This is for LLM's which deal mainly with text. An entire book can be stored as .42 MB according to https://www.quora.com/How-many-megabytes-are-in-a-book . 424 terrabytes text is over a billion books worth of data. On the common crawl website it even says "Over 250 billion pages spanning 17 years." That's an impressive amount of information.
LLMs can deal with more than text. Impressive today is nothing tomorrow
Earlier quoted context omitted.
LLMs can deal with more than text. Impressive today is nothing tomorrow
The technology that allows an LLM to "see" images and video is completely different though. It's not what is being trained on common crawl.