Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral releases Pixtral 12B, its first multimodal model

techcrunch.com

Post reply on HN
Print view View original

21–30 of 44 posts

Re: Mistral releases Pixtral 12B, its first multimodal model

#21

Post by esafak »

GaggiX wrote: post #18
dartos wrote: post #11

Earlier quoted context omitted.

The issue with image models is that their style becomes identifiable and stale quite quickly, so you’ll need a fresh intake of different, newer, styles every so often and that’s going to be harder and harder to get.

The style becoming identifiable and stale has mostly to do with CFG and almost nothing with the dataset, the heavy use of CFG by most models trades diversity with coherency. You don't need a costant intake of new images and styles, it's like saying that an image created two years ago is stale because it doesn't follow a new style or something. Also Pixtral is not a text-to-image model.

CFG for Classifier-Free Guidance?

Re: Mistral releases Pixtral 12B, its first multimodal model

#22

Post by whimsicalism »

Flockster wrote: post #10

Could this be used for a selfhosted handwritten text recognition instance? Like writing on an ePaper tablet, exporting the PDF and feed this into this model to extract todos from notes for example. Or what would be the SotA for this application?

if you have a 3090, you could self host

Re: Mistral releases Pixtral 12B, its first multimodal model

#23

Post by GaggiX »

esafak wrote: post #21
GaggiX wrote: post #18

Earlier quoted context omitted.

The style becoming identifiable and stale has mostly to do with CFG and almost nothing with the dataset, the heavy use of CFG by most models trades diversity with coherency. You don't need a costant intake of new images and styles, it's like saying that an image created two years ago is stale because it doesn't follow a new style or something. Also Pixtral is not a text-to-image model.

CFG for Classifier-Free Guidance?

Exactly, https://arxiv.org/abs/2207.12598

Jonathan Ho, one of the authors of the CFG paper, now works for Ideogram, and Ideogram 2 is one of the very few models (or perhaps the only one) where I don't see the artifacts caused by the CFG, maybe he has achieved a breakthrough.

Re: Mistral releases Pixtral 12B, its first multimodal model

#24

Post by jhgg »

Flockster wrote: post #10

Could this be used for a selfhosted handwritten text recognition instance? Like writing on an ePaper tablet, exporting the PDF and feed this into this model to extract todos from notes for example. Or what would be the SotA for this application?

Try out https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct

Re: Mistral releases Pixtral 12B, its first multimodal model

#26

Post by [deleted] »

bronco21016 wrote: post #13
buran77 wrote: post #7

The "Mistral Pixtral multimodal model" really rolls off the tongue. > It’s unclear which image data Mistral might have used to develop Pixtral 12B. The days of free web scraping especially for the richer sources of material are almost gone, with anything between technical (API restrictions) and legal (copyright) measures building deep moats. I also wonder what they trained it on. They're not Meta or Google with endle…

At what point does an agent sitting at a browser collecting information differ from a human? I have multiple ad-blockers running, how am I different from a bot scouring the “free” web? I get the idea of copyright and creators wanting to be paid for their content. However, I think there are plenty of human users out there not “paying” for “free” content either. Which one is a greater loss of revenue? A collection of o…

[deleted]

Re: Mistral releases Pixtral 12B, its first multimodal model

#27

Post by htrp »

buran77 wrote: post #7

The "Mistral Pixtral multimodal model" really rolls off the tongue. > It’s unclear which image data Mistral might have used to develop Pixtral 12B. The days of free web scraping especially for the richer sources of material are almost gone, with anything between technical (API restrictions) and legal (copyright) measures building deep moats. I also wonder what they trained it on. They're not Meta or Google with endle…

there are torrents all over the internet of AI training data for images and video....

img2dataset also exists

Re: Mistral releases Pixtral 12B, its first multimodal model

#28

Post by a2128 »

bronco21016 wrote: post #13
buran77 wrote: post #7

The "Mistral Pixtral multimodal model" really rolls off the tongue. > It’s unclear which image data Mistral might have used to develop Pixtral 12B. The days of free web scraping especially for the richer sources of material are almost gone, with anything between technical (API restrictions) and legal (copyright) measures building deep moats. I also wonder what they trained it on. They're not Meta or Google with endle…

At what point does an agent sitting at a browser collecting information differ from a human? I have multiple ad-blockers running, how am I different from a bot scouring the “free” web? I get the idea of copyright and creators wanting to be paid for their content. However, I think there are plenty of human users out there not “paying” for “free” content either. Which one is a greater loss of revenue? A collection of o…

Humans use Google Chrome from their home IP address that isn't on any blacklists, and they're always happy to make an account and download an app instead of accessing a website. Or at least that's what companies think humans are

Re: Mistral releases Pixtral 12B, its first multimodal model

#29

Post by jazzyjackson »

buran77 wrote: post #7

The "Mistral Pixtral multimodal model" really rolls off the tongue. > It’s unclear which image data Mistral might have used to develop Pixtral 12B. The days of free web scraping especially for the richer sources of material are almost gone, with anything between technical (API restrictions) and legal (copyright) measures building deep moats. I also wonder what they trained it on. They're not Meta or Google with endle…

You don't need a contract with reddit to scrape it, you can just add `.json` to any url and you'll get the entire thread as one object.

Re: Mistral releases Pixtral 12B, its first multimodal model

#30

Post by jazzyjackson »

edude03 wrote: post #3

12B is pretty small, so I’m doubting it’ll be anywhere close to internvl2 however mistral does great work and likely this model is still useful for on device tasks

I've found llama 3.1 8B to be effective at transforming unstructured text into structured data, now that LM Studio accepts a json schema parameter.

For a general knowledge chatbot it doesn't know much of course, but its a good worker bee.

Post reply on HN