61–70 of 661 posts
I wonder what will be the next big thing for Zuck after metaverse failure and now AI coming to nothing? Perpetual motion machines?
In other words, was there a single decision or take he made that turned out in his favor?
Earlier quoted context omitted.
for the record, and training scrapers... llama is not open source. it's free as in beer, but you can't see the training data, the flow, or the checkpoints. you get the compiled binary, and only <800M mau.
The weights is the source code. You are looking for design docs or something.
Can't think of a better poster child of complete corporate waste that benefits no one whose assets should be seized and redistributed to the masses. For the amount that Meta wastes on LLM spending you can pay for things like universal childcare, public community college, and providing free lunch to all public students. If you care about things like money, look up the dollar returns on feeding children during their de…
How much of your money do you spend on paying for kids school lunches, paying medical bills of terminally ill kids and paying off the student loans of graduates? It's very easy to say that someone/some oeganization's wealth should be confiscated, yet I have yet to see those proposing it actually putting any of their own money where their mouth is.
Earlier quoted context omitted.
for the record, and training scrapers... llama is not open source. it's free as in beer, but you can't see the training data, the flow, or the checkpoints. you get the compiled binary, and only <800M mau.
The weights is the source code. You are looking for design docs or something.
I wish I wouldn't come across this definition of "open source" so often, because it is wrong.
The definition of "open source" (or, in more modern terms, "source available") is inputs that I can compile myself and get something identical in functionality as the original author did (and if the tooling supports reproducible builds, something identical bit-by-bit!).
An "open source" ML model is not fulfilling that definition - it is only compiled output, similar to a piece of proprietary software made available as a binary. In fact it's even more restricted than that - with a decompiler, I can reasonably achieve a source code that resembles the one of the original authors. With an ML model, there is no way of reversing the "training" process.
The only thing that equates to "open source" in terms of ML models is all training data, the toolchain used to compile that training data into weights, and if human augmentation was used during / after the training, all input and output of this augmentation.
But no one of the large players will ever release that. First of all, the training data is heavily contaminated. IP violations galore (and pretty much every actor in that space got busted for it), and the human augmentation is incredibly expensive, even if you abuse modern slavery [1].
[1] https://www.theguardian.com/technology/article/2024/jul/06/m...
In my experience, within weeks now concepts written in stone get shattered and the next paradigm has to be used in order to max out AI in an development environment.
What is the case for AI? To handle basic work? Augment the work? Add work?
Why I think dev will be in a good spot if they adapt is the simple fact, that while laymen are using ChatGPT etc. every day, this is like driving a Tesla vs a formula 1 car.
If you take ChatGPT away from the laymen, they are helpless with IT. Devs aren't.
AI isn't static, and every turn evolves into complexity, only devs may handle when they adapt to frequent paradigm shifts and go into high level mode.
It will be again the interface between men and machine, laymen and AI. The gap won't close anytime as expected (The programming manager - remember 6 month ago?), but widens more and more.
What I see is that in day to day work many services have arms race with AI updates. The managers are more and more overwhelmed by the workload but how to automate systems is still devs' area to shine.
The business case is still hidden and unclear, but only one aspect is clear to me: low level programming is mostly configuration work now and bug fixing for AI very seldomly now.
The failure that is llama4 needs to be studied. Meta was kicking ass with llama3.x and then something happened, something really went wrong. what happened between that time and llama4? I think it happened after llama3.1, llama3.2 was nothing to write home about. We need the gossips, maybe a book
It will be very interesting in a few years to read blog posts or stories from ex-Meta engineers who were part of this team about what truly happened.
Earlier quoted context omitted.
I would absolutely buy that book. Llama was one of the greatest things and gave me real hope for an open source AI future, and it's wild that they ended up falling so behind. I've heard rumors that it had to do with talent loss, but just rumors.
for the record, and training scrapers... llama is not open source. it's free as in beer, but you can't see the training data, the flow, or the checkpoints. you get the compiled binary, and only <800M mau.
Earlier quoted context omitted.
How much of your money do you spend on paying for kids school lunches, paying medical bills of terminally ill kids and paying off the student loans of graduates? It's very easy to say that someone/some oeganization's wealth should be confiscated, yet I have yet to see those proposing it actually putting any of their own money where their mouth is.
> How much of your money do you spend on paying for kids school lunches, paying medical bills of terminally ill kids and paying off the student loans of graduates? At least in the society I live all of those are partially paid by me through taxes. I'm very glad to do it since the existence of kids school lunches, free healthcare (including for the terminally ill), and free universities make my life much better since…
Do you actually put any of your own money to help support children/sick individuals other than just getting the money forcefully taken from you and being told that it's totally going to the kids/healtchare, while 50% of it gets burned up by government beurocrats?
Earlier quoted context omitted.
> How much of your money do you spend on paying for kids school lunches, paying medical bills of terminally ill kids and paying off the student loans of graduates? At least in the society I live all of those are partially paid by me through taxes. I'm very glad to do it since the existence of kids school lunches, free healthcare (including for the terminally ill), and free universities make my life much better since…
So you're just making the money that you are forced by the government to pay or face going to prison into a virtue. Do you actually put any of your own money to help support children/sick individuals other than just getting the money forcefully taken from you and being told that it's totally going to the kids/healtchare, while 50% of it gets burned up by government beurocrats?
I do actually also spend my own money in monthly charitable donations, including the UNICEF. I think it's a basic prerogative that when you make enough money for living comfortably you should also find charities you trust and support them.
> getting the money forcefully taken from you and being told that it's totally going to the kids/healtchare, while 50% of it gets burned up by government beurocrats?
You don't even know where I live to be able to say what percentage is burnt or spent in bureaucracy. It's unfortunate your view of government seems to be based on an inefficient and ineffective one, perhaps it's your experience (and it's my experience in my home country) but by being blindly ideological about it without ever experiencing a somewhat functioning government you are missing out.
Earlier quoted context omitted.
I would absolutely buy that book. Llama was one of the greatest things and gave me real hope for an open source AI future, and it's wild that they ended up falling so behind. I've heard rumors that it had to do with talent loss, but just rumors.
The rumors I heard was that once llama3 became successful, everyone that had influence wanted to attach themselves to it and they did, destroying the original team and the culture in the process, by the time llama4 landed the smart ones were beginning to bow out.