Is the architecture outlined anywhere? Any publications or word on if they will publish something in the future? To be fair to them, they seemed to have launched this company today so I doubt they have a lot of time right now. Or maybe I just missed it?
Flux: Open-source text-to-image model with 12B parameters
61–70 of 239 posts
Re: Flux: Open-source text-to-image model with 12B parameters
#62whenever I see a new model I always see if it can do engineering diagrams (e.g. "two square boxes at a distance of 3.5mm"), still no dice on this one. https://x.com/seveibar/status/1819081632575611279 Would love to see an AI company attack engineering diagrams head on, my current hunch is that they just aren't in the training dataset (I'm very tempted to make a synthetic dataset/benchmark)
I hope you find manual tagging of diagrams interesting, as that is what you'll be doing a lot of!
Re: Flux: Open-source text-to-image model with 12B parameters
#63Earlier quoted context omitted.
Its already available for comfyui
It’ll need time for the goodies beyond the base model though I would guess
Works great as is right now, I can see some workflows being affected or having to wait for an update, but even those can do with some temporary workarounds (like having to load another model for later inpainting steps).
So if you're wanting to experiment and have a 24GB card, have at it!
Re: Flux: Open-source text-to-image model with 12B parameters
#64Tested it using prompts from ideogram (login walled) which has great prompt adherence. Flux generated very very good images. I have been playing with ideogram but i don't want their filters and want to have a similar powerful system running locally. If this runs locally, this is very very close to that in terms of both image quality and prompt adherence. I did fail at writing text clearly when text was a bit complica…
Ideogram handles text really well but I don’t want to be on some weird social network.
If this thing can mint memes with captions in it on a single node I guess that’s the weekend gone.
Thanks for the useful review.
Re: Flux: Open-source text-to-image model with 12B parameters
#65Tested it using prompts from ideogram (login walled) which has great prompt adherence. Flux generated very very good images. I have been playing with ideogram but i don't want their filters and want to have a similar powerful system running locally. If this runs locally, this is very very close to that in terms of both image quality and prompt adherence. I did fail at writing text clearly when text was a bit complica…
Ideogram handles text really well but I don’t want to be on some weird social network. If this thing can mint memes with captions in it on a single node I guess that’s the weekend gone. Thanks for the useful review.
Flux is amazing actually. See my other comment where I verified a prompt on their fastest model. Check the linked reddit thread too.
Re: Flux: Open-source text-to-image model with 12B parameters
#66Is the architecture outlined anywhere? Any publications or word on if they will publish something in the future? To be fair to them, they seemed to have launched this company today so I doubt they have a lot of time right now. Or maybe I just missed it?
You can look at the model config params for diffusers, e.g.: https://huggingface.co/black-forest-labs/FLUX.1-schnell/comm...
Re: Flux: Open-source text-to-image model with 12B parameters
#67hi friends! burkay from fal.ai here. would like to clarify that the model is NOT built by fal. all credit should go to Black Forest Labs ( https://blackforestlabs.ai/ ) which is a new co by the OG stable diffusion team. what we did at fal is take the model and run it on our inference engine optimized to run these kinds of models really really fast. feel free to give it a shot on the playgrounds. https://fal.ai/models…
The name is a bit unfortunate given that Julia's most popular ML library is called Flux. See: https://fluxml.ai . This library is quite well known, 3rd most starred project in Julia: https://juliapackages.com/packages?sort=stars . It has been around since, at least, 2016: https://github.com/FluxML/Flux.jl/graphs/code-frequency .
There was a looong distracting thread a month ago about something similar, niche language, might have been Julia, had a package with the same name as $NEW_THING.
I hope this one doesn't stir as much discussion. It has 4000 stars, there isnt a large mass of people who view the world through the lens of "Flux is ML library". No one will end up in a "who is on first?" discussion because of it. If this line of argument is held sacrosanct, it ends up in an infinite loop until everyone gives up and starts using UUIDs.
Re: Flux: Open-source text-to-image model with 12B parameters
#68The [schnell] model variant is Apache-licensed and is open sourced on Hugging Face: https://huggingface.co/black-forest-labs/FLUX.1-schnell It is very fast and very good at rendering text, and appears to have a text encoder such that the model can handle both text and positioning much better: https://x.com/minimaxir/status/1819041076872908894 A fun consequence of better text rendering is that it means text watermarks…
That’s not really fair to conclude that the training data contains vanity fair images since the prompt includes “by Vanity Fair”. I could write “with text that says Shutterstock” in the prompt but that doesn’t necessairly mean the dataset contains that
Are you suggesting that the model independently came up with Vanity Fair's logo, including font and kerning?
https://www.vanityfair.com/verso/static/vanity-fair/assets/l...
Re: Flux: Open-source text-to-image model with 12B parameters
#69whenever I see a new model I always see if it can do engineering diagrams (e.g. "two square boxes at a distance of 3.5mm"), still no dice on this one. https://x.com/seveibar/status/1819081632575611279 Would love to see an AI company attack engineering diagrams head on, my current hunch is that they just aren't in the training dataset (I'm very tempted to make a synthetic dataset/benchmark)
Can’t you get this done via an LLM and have it generate code for mermaid or D2 or something? I’ve been fiddling around with that a bit in order to create flowcharts and datamodels, and I’m pretty sure I’ve seen at least one of those languages handle absolute positioning of object.
Re: Flux: Open-source text-to-image model with 12B parameters
#70Is the architecture outlined anywhere? Any publications or word on if they will publish something in the future? To be fair to them, they seemed to have launched this company today so I doubt they have a lot of time right now. Or maybe I just missed it?
You can look at the model config params for diffusers, e.g.: https://huggingface.co/black-forest-labs/FLUX.1-schnell/comm...
I don't have anything to compare it to as I'm not that familiar with other diffusion models in the first place. I was kind of hoping to read the key changes they made to the diffusion architecture and how they collected and curated their dataset. I'd assume their are also using LAION but I wonder if they are doing anything to filter out low quality images (separate from what LAION atheistic already does). Or maybe if they have their own dataset.