Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral releases Pixtral 12B, its first multimodal model

techcrunch.com

Post reply on HN
Print view View original

11–20 of 44 posts

Re: Mistral releases Pixtral 12B, its first multimodal model

#11

Post by dartos »

simonw wrote: post #9
buran77 wrote: post #7

The "Mistral Pixtral multimodal model" really rolls off the tongue. > It’s unclear which image data Mistral might have used to develop Pixtral 12B. The days of free web scraping especially for the richer sources of material are almost gone, with anything between technical (API restrictions) and legal (copyright) measures building deep moats. I also wonder what they trained it on. They're not Meta or Google with endle…

What do you mean by copyright measures? Has anything changed on that front in the last two years? My hunch is that most AI labs are already sitting on a pretty sizable collection of scraped image data - and that data from two years ago will be almost as effective as data scraped today, at least as far as image training goes.

The issue with image models is that their style becomes identifiable and stale quite quickly, so you’ll need a fresh intake of different, newer, styles every so often and that’s going to be harder and harder to get.

Re: Mistral releases Pixtral 12B, its first multimodal model

#12

Post by Eisenstein »

dartos wrote: post #11
simonw wrote: post #9

Earlier quoted context omitted.

What do you mean by copyright measures? Has anything changed on that front in the last two years? My hunch is that most AI labs are already sitting on a pretty sizable collection of scraped image data - and that data from two years ago will be almost as effective as data scraped today, at least as far as image training goes.

The issue with image models is that their style becomes identifiable and stale quite quickly, so you’ll need a fresh intake of different, newer, styles every so often and that’s going to be harder and harder to get.

> Built on one of Mistral’s text models, Nemo 12B, the new model can answer questions about an arbitrary number of images of an arbitrary size given either URLs or images encoded using base64, the binary-to-text encoding scheme. Similar to other multimodal models such as Anthropic’s Claude family and OpenAI’s GPT-4o, Pixtral 12B should — at least in theory — be able to perform tasks like captioning images and counting the number of objects in a photo.

This is a not a diffusion model -- it doesn't create images, it answers questions.

Re: Mistral releases Pixtral 12B, its first multimodal model

#13

Post by bronco21016 »

buran77 wrote: post #7

The "Mistral Pixtral multimodal model" really rolls off the tongue. > It’s unclear which image data Mistral might have used to develop Pixtral 12B. The days of free web scraping especially for the richer sources of material are almost gone, with anything between technical (API restrictions) and legal (copyright) measures building deep moats. I also wonder what they trained it on. They're not Meta or Google with endle…

At what point does an agent sitting at a browser collecting information differ from a human?

I have multiple ad-blockers running, how am I different from a bot scouring the “free” web? I get the idea of copyright and creators wanting to be paid for their content. However, I think there are plenty of human users out there not “paying” for “free” content either. Which one is a greater loss of revenue? A collection of over a million humans? Or 100 or so corporate bots?

Re: Mistral releases Pixtral 12B, its first multimodal model

#14

Post by tonygiorgio »

Flockster wrote: post #10

Could this be used for a selfhosted handwritten text recognition instance? Like writing on an ePaper tablet, exporting the PDF and feed this into this model to extract todos from notes for example. Or what would be the SotA for this application?

> the 12-billion-parameter model is about 24GB in size

Probably not on the device itself but I would love that use case as well. At least going to my own server. I’d want to protect notes in particular, which is why I don’t do any cloud backup on my RM2. But some self hosted, AI assisted OCR workflows could be really nice.

Re: Mistral releases Pixtral 12B, its first multimodal model

#16

Post by kaoD »

Jackson__ wrote: post #8
edude03 wrote: post #3

12B is pretty small, so I’m doubting it’ll be anywhere close to internvl2 however mistral does great work and likely this model is still useful for on device tasks

It appears to be slightly worse than Qwen2VL 7B, a model almost half it's size, if you look at the Qwen's official benchmarks instead of Mistral's. https://xcancel.com/_philschmid/status/1833954941624615151

But Qwen is not multimodal, or is it?

Re: Mistral releases Pixtral 12B, its first multimodal model

#17

Post by whimsicalism »

dartos wrote: post #11
simonw wrote: post #9

Earlier quoted context omitted.

What do you mean by copyright measures? Has anything changed on that front in the last two years? My hunch is that most AI labs are already sitting on a pretty sizable collection of scraped image data - and that data from two years ago will be almost as effective as data scraped today, at least as far as image training goes.

The issue with image models is that their style becomes identifiable and stale quite quickly, so you’ll need a fresh intake of different, newer, styles every so often and that’s going to be harder and harder to get.

solvable without additional images

Re: Mistral releases Pixtral 12B, its first multimodal model

#18

Post by GaggiX »

dartos wrote: post #11
simonw wrote: post #9

Earlier quoted context omitted.

What do you mean by copyright measures? Has anything changed on that front in the last two years? My hunch is that most AI labs are already sitting on a pretty sizable collection of scraped image data - and that data from two years ago will be almost as effective as data scraped today, at least as far as image training goes.

The issue with image models is that their style becomes identifiable and stale quite quickly, so you’ll need a fresh intake of different, newer, styles every so often and that’s going to be harder and harder to get.

The style becoming identifiable and stale has mostly to do with CFG and almost nothing with the dataset, the heavy use of CFG by most models trades diversity with coherency. You don't need a costant intake of new images and styles, it's like saying that an image created two years ago is stale because it doesn't follow a new style or something.

Also Pixtral is not a text-to-image model.

Re: Mistral releases Pixtral 12B, its first multimodal model

#19

Post by Jackson__ »

kaoD wrote: post #16
Jackson__ wrote: post #8

Earlier quoted context omitted.

It appears to be slightly worse than Qwen2VL 7B, a model almost half it's size, if you look at the Qwen's official benchmarks instead of Mistral's. https://xcancel.com/_philschmid/status/1833954941624615151

But Qwen is not multimodal, or is it?

https://qwen2.org/vl/

>Qwen2-VL is the latest addition to the vision-language models in the Qwen series, building upon the capabilities of Qwen-VL. Compared to its predecessor, Qwen2-VL offers:

>State-of-the-Art Image Understanding

>Extended Video Comprehension

Besides, it'd have been pretty silly for them to mention it on their slides if it wasn't.

Re: Mistral releases Pixtral 12B, its first multimodal model

#20

Post by GaggiX »

buran77 wrote: post #7

The "Mistral Pixtral multimodal model" really rolls off the tongue. > It’s unclear which image data Mistral might have used to develop Pixtral 12B. The days of free web scraping especially for the richer sources of material are almost gone, with anything between technical (API restrictions) and legal (copyright) measures building deep moats. I also wonder what they trained it on. They're not Meta or Google with endle…

>The days of free web scraping especially for the richer sources of material are almost gone

I would say the opposite, it has never been easier to collect a huge amount of data, in particular if you have a target, also you don't even need to write a line of code if you are good at explaining Claude 3.5 Sonnet what you want to achieve and the details.

Post reply on HN