Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

261–270 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#261

Post by Deathmax »

faangguyindia wrote: post #203
simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

The free tier API isn't US-only, Google has removed the free tier restriction for UK/EEA countries for a while now, with the added bonus of not training on your data if making a request from the UK/CH/EEA.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#262

Post by phillipcarter »

a2128 wrote: post #234
faangguyindia wrote: post #203

Earlier quoted context omitted.

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same am…

I think there's a few things to consider:

They make a ton of money on large enterprise package deals through Google Cloud. That includes API access but also support and professional services. Most orgs that pay for this stuff don't really need it, but they buy it anyways, as is consistent with most enterprise sales. That can give Google a significant margin to make up the cost elsewhere.

Gemini Flash is probably super cheap to run compared to other models. The cost of inference for many tasks has gone down tremendously over the past 1.5 years, and it's still going down. Every economic incentive aligns with running these models more efficiently.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#263

Post by euroderf »

kristopolous wrote: post #187
nextworddev wrote: post #142

Earlier quoted context omitted.

They are literally training on all the free personal data you provided, so they owe you this much

Given what I see in Facebook comments I'm surprised the AI doesn't just respond with "Amen. Happy Birthday" to every query. They're clearly majorly scrubbing things somehow

In a few years (or months?) Faceborg will offer a new service "EverYou" trained on your entire Faceborg corpus. It will speak like you to others (whomever you permit) and it will like what you like (acting as a web gopher for you) and it will be able stay up late talking to tipsy you about life, the universe, and everything, and it will be... "long-term affordable".

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#264

Post by Jackson__ »

chriskanan wrote: post #221

The assessments of visual capability really need to be more robust. They are still using datasets like VQAv2, which while providing some insight, have many issues. There are many newer datasets that serve as much more robust tests and that are less prone to being affected by linguistic bias. I'd like to see more head-to-head comparisons with community created multi-modal LLMs as done in these papers: https://arxiv.or…

Looking at their benchmark results and my own experience with their 11B vision model, I think while not perfect they represent the model well.

Meaning it's doing impressively bad compared to other models I've tried in similar sizes(for vision).

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#266

Post by svieira »

1986 wrote: post #229
mike_hearn wrote: post #195

Earlier quoted context omitted.

You're objectively correct but judging from your downvotes there seems to be some denial here about that! The atheism alone means it's different from a big chunk of the world's population, possibly the majority. Supposedly around 80% of the world's population identify with a religion though I guess you can debate how many people are truly devout. The good news is that the big AI labs seem to be slowly getting a grip…

You want the computer to believe in God?

If God was real, wouldn't you? If God is real and you're wrong about that (or if you don't yet know the real God) would you want the computer to agree with your misconception or would you want it to know the truth?

Cut out "computer" here - would you want any person to hold a falsehood as the truth?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#267

Post by grahamj »

knicholes wrote: post #235
sumedh wrote: post #205

Earlier quoted context omitted.

I tried some OCR use cases, Claude Sonnet just blows Molmo.

When you say "blows," do you mean in a subservient sense or more like, "it blows it out of the water?"

yeah does it suck or does it suck?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#268

Post by kenmacd »

nickpsecurity wrote: post #122
warkdarrior wrote: post #65

Earlier quoted context omitted.

Do you have some concrete example of biases in their models? Or are you just fishing for something to complain about?

Google’s and OpenAI often answered far-left, Progressive, and atheist. Google’s was censoring white people at one point. Facebook seems to espouse similar values. They’ve funded work to increase those values. Many mention topics relevant to these things in their papers in the bias or alignment sections. These political systems don’t represent the majority of the world. They might not even represent half the U.S.. Peo…

> Google’s and OpenAI often answered far-left, Progressive, and atheist.

Can you share some conversations where the AI answers fall in to these categories. I'm especially interested in seeing an honest conversation that results in a response you'd consider 'far-left'.

> These political systems don’t represent the majority of the world.

Okay… but just because the majority of people believe something doesn't necessarily make it true. You should also be willing to accept the possibly that it's not 'targeted suppression' but that the model has 'learned' and to show both sides would be a form of suppression.

For example while it's not the majority, there's a scarily large number of people that believe the Earth is flat. If you tell an LLM that the Earth is flat it'll likely disagree. Someone that actually believes the Earth is flat could see this as the Round-Earther creators promoting their own views when the 'alignment' could simply be to focus on ideas with some amount of scientific backing.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#269

Post by [deleted] »

faangguyindia wrote: post #203
simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

[deleted]

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#270

Post by rcpt »

a2128 wrote: post #234
faangguyindia wrote: post #203

Earlier quoted context omitted.

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same am…

Aren't API calls essentially swappable now between vendors now?

If you wanted to switch from Gemini to Chatgpt you could copy/paste your code into Chatgpt and ask it to switch to their API.

Disclaimer I work at Google but not on Gemini

Post reply on HN