11–20 of 123 posts
Earlier quoted context omitted.
"just" All the complexity wrapped up into one word.
You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it can't be cheap
Earlier quoted context omitted.
You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it can't be cheap
Web scrape... what, exactly? How will "web scraping" lead to high quality results?
Earlier quoted context omitted.
You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it can't be cheap
Training an AI requires high quality, clean/normalized data, which is very difficult to do at millions/billions of data points and is very frequently done incorrectly with silent failures.
hint: they are all about one thing and there are a lot of eager volunteers to help on those websites. It would be easy to "normalize/clean/classify" because the pictures would have a consistent theme, thus reducing the amount of parameters.
We are not trying to generate elephants getting railed on a SpaceX rocket flying in oil painting style (although I'm sure theres people into that and its not my place to judge), we are just trying to remove the human cost out of this necessary evil.
I can't believe nobody is investing in "DALL-E-2-4-PORN". This sounds like an X amount of money thrown at a hugely sticky product that can be iterated (with the current trend in hardware) to the point where it literally generates billions of dollars in revenues for ages to come (no pun intended).
How much would it cost to train something like this? Is there even a good dataset for it?
There is a dataset of 5 billion image-text pairs (laion-5b) scraped by various parties. This can then be filtered and used to train these models. Cost is a bit of an issue but there are orgs that have provided compute for open model training. And Imagen is nice because the text encoder part is already available and doesn't need more training, so it would just be the diffusion model components being trained. I'd guess…
I did the pip installs and installed Cuda. I changed prompt and ran the sample code. It ran to completion. How do I save the image from trainer.sample?
Why don't they just keep their findings to themselfes and build products on top of them?
Public companies can't do stuff just for the fun of it, right? So there must be some commercial reasoning behind it?
Earlier quoted context omitted.
"just" All the complexity wrapped up into one word.
You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it can't be cheap
https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454...
And this is just for a text-only scrape.
Earlier quoted context omitted.
"just" All the complexity wrapped up into one word.
You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it can't be cheap
For example, in a generative model I'm working on, I have a dataset consisting of ~5M images just blindly scraped from a website. After filtering, this drops down to ~500k images, yet a model trained on that performs worse than one trained on a set of 30k curated images (picked based on a manually evaluated list of "known good" artists), where filtering brings it down to ~18k images. The larger dataset, while containing more information, also contains more errors, many of them pretty hard to filter out.
Earlier quoted context omitted.
You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it can't be cheap
Web scrape... what, exactly? How will "web scraping" lead to high quality results?
What is the reason Google published their research details about Imagen? Why don't they just keep their findings to themselfes and build products on top of them? Public companies can't do stuff just for the fun of it, right? So there must be some commercial reasoning behind it?