21–30 of 62 posts
Other than this sentence:
> We curated a large dataset of videos and languages from public book and video datasets, consisting of videos of diverse activities and long-form books.
I didn’t see any other mention of datasets used, is this on intentional?
What does "Million-Length" mean?
This looks really promising! Other than this sentence: > We curated a large dataset of videos and languages from public book and video datasets, consisting of videos of diverse activities and long-form books. I didn’t see any other mention of datasets used, is this on intentional?
the claim is that a dataset was created. of words and of videos, and that it was created from public datasets of books and videos, those datasets containing books, and videos.
it takes too many words to say almost nothing.
nothing to see here.
if that isn’t the intent, then the authors need to do better.
Earlier quoted context omitted.
Look at how RingAttention is implemented: it's blockwise attention distributed among many GPUs, in other words bruteforce parallelization. For inference they use TPUs v4-128, not running this at home any time soon.
What is this month's best choice to run at home?
This looks really promising! Other than this sentence: > We curated a large dataset of videos and languages from public book and video datasets, consisting of videos of diverse activities and long-form books. I didn’t see any other mention of datasets used, is this on intentional?
https://huggingface.co/LargeWorldModel
And the model page specifically mention Books3
This looks really promising! Other than this sentence: > We curated a large dataset of videos and languages from public book and video datasets, consisting of videos of diverse activities and long-form books. I didn’t see any other mention of datasets used, is this on intentional?
the sentence itself should be alarming. the claim is that a dataset was created. of words and of videos, and that it was created from public datasets of books and videos, those datasets containing books, and videos. it takes too many words to say almost nothing. nothing to see here. if that isn’t the intent, then the authors need to do better.
Books3 dataset 700B text-image pairs from Laion-2B-en, filtered to only keep images with at least 256 resolution 400M text-image pairs from COYO-700M, filtered to only keep images with at least 256 resolution 10M text-video pairs from WebVid10M 3M text-video pairs from a subset of InternVid10M 73K text-video chat pairs from Valley-Instruct-73K 100K text-video chat pairs from Video-ChatGPT