61–70 of 112 posts
Images still look AI generated
Yeah, there are still imperfections. But, it’s surprising to us how much the quality can be improved without the need of a whole pre-training (re-) run.
Tried a simple prompt, and got some pretty interesting results: "Octopus DJ spinning the turntables at a rave." The human like hands the DJ sprouts are interesting, and no amount of prompting seems to stop them. Opinionated, as the paper says.
Describing it as "Octopus DJ with no fingers" got rid of the hands for me, but interestingly, also removed every anthropomorphized element of the octopus, so that it was literally just an octopus spinning turntables.
Maybe you got a lucky roll :)
hey hn! I'm one of the founders at Krea. we prepared a blogpost about how we trained FLUX Krea if you're interested in learning more: https://www.krea.ai/blog/flux-krea-open-source-release
.scrollbar-hide {
-ms-overflow-style: none;
scrollbar-width: none;
}yoo i'm also a researcher on the krea 1 project and happy to answer any questions :)
How large was the dataset used for post-training?
We used two types of datasets for post-training. Supervised finetuning data and preference data used for RLHF stage. You can actually use less than < 1M samples to significantly boost the aesthetics. Quality matters A LOT. Quantity helps with generalisation and stability of the checkpoints though.
Earlier quoted context omitted.
Yeah, there are still imperfections. But, it’s surprising to us how much the quality can be improved without the need of a whole pre-training (re-) run.
Is it possible (or do people already do this), to train a classifier to identify the AI look and use it as an adversary to try and maximise both 'quality' and 'not that sort of quality'?
Another note about preference optimisation and RL is that it has really high quality ceiling but needs to be very carefully tuned. It's easy to get perfect anatomy and structure if you decide to completely "collapse" the model. For instance, ChatGPT images are collapsed to have slight yellow color palette. FLUX images always have this glossy, plastic texture with overly blurry background. It's similar to reward hacking behavior you see in LLMs where they sound overly nice and chatty.
I had to make a few compromises to balance between "stable, collapsed, boring model" and "unstable, diverse, explorative" model.
How did you train while ensuring only images consensually acquired were used?
How much data is the model trained on?
Copying and pasting Sangwu’s answer: We used two types of datasets for post-training. Supervised finetuning data and preference data used for RLHF stage. You can actually use less than < 1M samples to significantly boost the aesthetics. Quality matters A LOT. Quantity helps with generalisation and stability of the checkpoints though.
Earlier quoted context omitted.
We used two types of datasets for post-training. Supervised finetuning data and preference data used for RLHF stage. You can actually use less than < 1M samples to significantly boost the aesthetics. Quality matters A LOT. Quantity helps with generalisation and stability of the checkpoints though.
How is the data collected?