Helpful blog post for understanding what kind of data is needed for these models!
Does this have any application for generating realistic scenes for robotics training?
Releasing weights for FLUX.1 Krea
11–20 of 112 posts
Re: Releasing weights for FLUX.1 Krea
#12Helpful blog post for understanding what kind of data is needed for these models! Does this have any application for generating realistic scenes for robotics training?
Hey thanks! I’ll ask Sangwu to hop here to answer this and give a more research-oriented answer
Re: Releasing weights for FLUX.1 Krea
#13Helpful blog post for understanding what kind of data is needed for these models! Does this have any application for generating realistic scenes for robotics training?
Thank you! Glad you find it helpful.
The model is focused on photorealism so it should be able to generate most realistic scenes. Although, I think using 3D engines would be more suitable for typical cases for robotics training since it gives you ground truth data on objects, location, etc.
One interesting use case would be if you are focusing on a robotics task that would require perception of realistic scenes.
Re: Releasing weights for FLUX.1 Krea
#14Amazing. I can practically smell that owl it looks so darned owl-like.
From the article it doesn’t seem as though photorealism per se was a goal in training; was that just emergent from human preferences, or did it take some specific dataset construction mojo?
Re: Releasing weights for FLUX.1 Krea
#15Amazing. I can practically smell that owl it looks so darned owl-like. From the article it doesn’t seem as though photorealism per se was a goal in training; was that just emergent from human preferences, or did it take some specific dataset construction mojo?
I love owls. Photorealism was one of the focus areas for training because "AI look" (e.g. plastic skin) was biggest complaint for FLUX.1 model series. Photorealism was achieved with both careful curation of finetuning and preference dataset.
Re: Releasing weights for FLUX.1 Krea
#16Relevant links:
- GitHub repository: https://github.com/krea-ai/flux-krea
- Model Technical Report: https://www.krea.ai/blog/flux-krea-open-source-release
- Huggingface model card: https://huggingface.co/black-forest-labs/FLUX.1-Krea-dev
Re: Releasing weights for FLUX.1 Krea
#17Can someone ELI5 why the safetensor file is 23.8 GB, given the 12B parameter model? Does the model use closer to 24 GB of VRAM or 12 GB of VRAM. I've always associated a 1 billion parameter = 1 GB of VRAM. Is this estimate inaccurate?
Re: Releasing weights for FLUX.1 Krea
#18Can someone ELI5 why the safetensor file is 23.8 GB, given the 12B parameter model? Does the model use closer to 24 GB of VRAM or 12 GB of VRAM. I've always associated a 1 billion parameter = 1 GB of VRAM. Is this estimate inaccurate?
That's a good ballpark for something quantized to 8 bits per parameter. But you can 2x/4x that for 16 and 32 bit.
Re: Releasing weights for FLUX.1 Krea
#19Can someone ELI5 why the safetensor file is 23.8 GB, given the 12B parameter model? Does the model use closer to 24 GB of VRAM or 12 GB of VRAM. I've always associated a 1 billion parameter = 1 GB of VRAM. Is this estimate inaccurate?
A parameter can be any size float. Lots of downloadable models are FP8 (8 bits per parameter), but it appears this model is FP16 (16 bits per parameter)
Often, the training is done in FP16 then quantized down to FP8 or FP4 for distribution.
Re: Releasing weights for FLUX.1 Krea
#20Can someone ELI5 why the safetensor file is 23.8 GB, given the 12B parameter model? Does the model use closer to 24 GB of VRAM or 12 GB of VRAM. I've always associated a 1 billion parameter = 1 GB of VRAM. Is this estimate inaccurate?
Quick napkin math assuming bfloat16 format : 1B * 16 bits = 16B bits = 2GB.
Since it's a 12B parameter model, you get around ~24GB. Downcasting to bfloat16 from float32 comes with pretty minimal performance degradation, so we uploaded the weights in bfloat16 format.