Earlier quoted context omitted.
It'll be available in about 60 minutes.
are you an aggressive hn-lurker or do you have some keyword alerts set up for this, just curious.
I have automation that emails me when certain keywords are mention on HN.
Meta Llama 3
751–760 of 965 posts
Re: Meta Llama 3
#752I downloaded llama3:8b-instruct-q4_0 in ollama and said "hi" and it answered with 10 screen long rant. This is an exerpt. > You're welcome! It was a pleasure chatting with you. Bye for now!assistant > Bye for now!assistant > Bye!assistant
Sorry about this. It should be fixed now. There was an issue with the vocabulary we had to fix and re-push! ollama pull llama3:8b-instruct-q4_0 should update it.
Works well now. That was an amazingly quick reaction from you, I am impressed.
Re: Meta Llama 3
#753Earlier quoted context omitted.
> China is a suppressive state that strives to control its citizens China's central government also believes it is protecting its citizens. > while the EU privacy protection laws are put in place to protect citizens The fact that they CAN exert so much power on information access in the name of "protection" is a bad precedent, and opens the door to future, less-benevolent authoritarian leadership being formed. (Even…
>China's central government also believes it is protecting its citizens. Anyone who's taking a course in epistemology can tell you that there's more to assessing veracity of a belief than noting its equivalence to other beliefs. There can be symmetry in psychology without symmetry in underlying facts. So noting an equivalence of belief is not enough to establish an equivalence in fact. I'm not even saying I'm for or…
The previous poster said that the EU is not restricting the freedom of its citizens, but protecting them (from themselves?). I fail to see how one can say that with a straight face. If you had a basic understanding of history of dictorships you would know that every dictatorship starts off by "protecting" its citizens.
Re: Meta Llama 3
#754Earlier quoted context omitted.
I haven't tried Llama 3 yet, but Llama 2 is indeed extremely "safe." (I'm old enough to remember when AI safety was about not having AI take over the world and kill all humans, not when it might offend a Puritan's sexual sensibilities or hurt somebody's feelings, so I hate using the word "safe" for it, but I can't think of a better word that others would understand). It's not quite as bad as Gemini, but in the same c…
So whereabouts are you that a "Puritan's sexual sensibilities" holds any sway?
I think the point is Silicon Valley is such a place.
"Visible nipples? The defining characteristic of all mammals, which infants necessarily have to put in their mouths to feed? On this website? Your account has been banned!"
Meanwhile in Berlin, topless calendars in shopping malls and spinning-cube billboards for Dildo King all over the place.
Re: Meta Llama 3
#755Earlier quoted context omitted.
You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.
For sure. I just started watching the new Dwarkesh interview with Zuck that was just released ( https://t.co/f4h7ko0M7q ) and you can just tell from the first few minutes that he simply has a different level of enthusiasm and passion and level of engagement than 99% of big tech CEOs.
I've never heard of this person, but many of the questions he asks Zuck show a total lack of any insight in this field. How did this interview even happen?
Re: Meta Llama 3
#756Earlier quoted context omitted.
I'm seeing the same behaviour. It's as if they have a post-processor that evaluates the quality of the response after a certain number of tokens have been generated, and reverts the response if it's below a threshold.
I've noticed Gemini exhibiting similar behaviour. It will start to answer, for example, a programming question - only to delete the answer and replace it with something along the lines of "I'm only a language model, I don't know how to do that"
They have both pre and post-LLM filters.
Re: Meta Llama 3
#757Earlier quoted context omitted.
Also, being open source adds phenomenal value for Meta: 1. It attracts the world's best academic talent, who deeply want their work shared. AI experts can join any company, so ones which commit to open AI have a huge advantage. 2. Having armies of SWEs contributing millions of free labor hours to test/fix/improve/expand your stuff is incredible. 3. The industry standardizes around their tech, driving down costs and d…
Yes, I completely agree with every point you made. It’s going to be so satisfying when all the AI safety people realize that their attempts to cram this protectionist/alarmist control down our throats are all for nothing, because there is an even stronger model that is totally open weights, and you can never put the genie back in the bottle!
> you can never put the genie back in the bottle
That's specifically why OpenAI don't release weights, and why everyone who cares about safety talks about laws, and why Yud says the laws only matter if you're willing to enforce them internationally via air strikes.
> It’s going to be so satisfying
I won't be feeling Schadenfreude if a low budget group or individual takes an open weights model, does a white-box analysis to determine what it knows and to overcome any RLFH, in order to force it to work as an assistant helping walk them though the steps to make VX nerve agent.
Given how old VX is, it's fairly likely all the info is on the public internet already, but even just LLMs-as-a-better-search / knowledge synthesis from disparate sources, that makes a difference, especially for domain specific "common sense": You don't need to know what to ask for, you can ask a model to ask itself a better question first.
Re: Meta Llama 3
#758Earlier quoted context omitted.
> Driving up market pay for workers via competition for their labour is exactly how we get progress for workers. There's a difference between "paying higher salaries in fair competition for talents" and "buying people to let them rot to make sure they don't work for competition". It's the same as "lowering prices to the benefit of consumer" vs "price dumping to become a monopoly". Facebook never did it at scale thoug…
> It's the same as "lowering prices to the benefit of consumer" vs "price dumping to become a monopoly". Where has that ever worked? Predatory pricing is highly unlikely. See eg https://www.econlib.org/library/Columns/y2017/Hendersonpreda... and https://www.econlib.org/archives/2014/03/public_schoolin.htm... > Facebook never did it at scale though. Google did. Please provide some examples. > There's a difference betw…
> Where has that ever worked? Predatory pricing is highly unlikely.
> See eg
Neither of the articles understand how predatory pricing works, assuming it's a single-market process. In the most usual case you fuel price dumping in one market by profits from the other. This way you can run it potentially indefinitely and you're doing it not in a hope of making profits on this market some day but to make sure no one else does. Funnily enough the second author got a good example but still failed to see it under his nose: public schools do have 90% of the market, and in many countries almost 100%. Obviously it works. Netscape died despite having a superior product because it was competing with a public school so to speak. Browser market is dead up to this date.
> And I'm not sure why as a worker you would decide to rot? If someone pays me a lot to put in a token effort, just so I don't work for the competition, I might happily take that over and practice my trumpet playing while 'working from home'.
That's exactly what happens and people proceed to degrade professionally.
> Perhaps someone else has actual interesting work, and comparable pay.
Not unless that someone sits on the ads money pipe.
> Please provide some examples
What kind of example do you expect? If it helps, half the people I personally know in Google "practice the trumpet" in your words. Situation is slowly improving though in the past two years.
I'm not saying it should be made illegal. I'm saying it's definitely happening and it's sad for me to see. I want the tech industry to move forward, not the amateur trumpet one.
Re: Meta Llama 3
#759Earlier quoted context omitted.
They also stated that they are still training larger variants that will be more competitive: > Our largest models are over 400B parameters and, while these models are still training, our team is excited about how they’re trending. Over the coming months, we’ll release multiple models with new capabilities including multimodality, the ability to converse in multiple languages, a much longer context window, and stronge…
Anyone have any informed guesstimations as to where we might expect a 400b parameter model for llama 3 to land benchmark wise and performance wise, relative to this current llama 3 and relative to GPT-4? I understand that parameters mean different things for different models, and llama two had 70 b parameters, so I'm wondering if anyone can contribute some guesstimation as to what might be expected with the larger mo…
The benchmark for the latest checkpoint is pretty good: https://x.com/teknium1/status/1780991928726905050?s=46
Re: Meta Llama 3
#760I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…
Why is Meta doing it though? This is an astronomical investment. What do they gain from it?
I would assume it's related to fair use and how OpenAI and Google have closed models that are built on copyrighted material. Easier to make the case that it's for the public good if it's open and free than not...