181–190 of 860 posts
> I understand that you have concerns about Facebook and their impact on society. However, it's important to note that Facebook is a complex organization with many different teams and initiatives, and it's not accurate to make sweeping generalizations about the company as a whole. While some decisions and products may be controversial or have unintended consequences, it's important to recognize that Facebook has also brought many positive innovations and opportunities to the world.
Earlier quoted context omitted.
> Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. Yes I would agree with you if Google wasn't set on to full on panic mode by their investors about releasing something vs Open AI due to Chat GPT's buzz. Bard was just a "hey we can do this too" thing, it was released half assed, had next to no marketing or hype. Vertex AI…
I can already tell you that PaLM is not anywhere near as good and PaLM-2 is at least not as good before RLHF. Not going to keep replying, believe what you want about Google's capabilities
You seem to be under the mistaken belief that: 1. Google has competent high-level organization that effectively sets and pursues long term goals. 2. There is some advantage to developing a highly capable LLM but not releasing it.
(2) could be the case if Google had built an extremely large model which was too expensive to deploy. Having been privy to what they had been working on up until mid-2022 and knowing how much work, compute and planning goes into extremely large models, this would very much surprise me.
Note: I did not have much visibility into what deepmind was up to. Maybe they had something.
Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…
That's an oddly high number for blocking competition. OpenAI's ChatGPT hit 100 million MAUs in January, and has gone down since. It's essentially a "Amazon and Google don't use this k thx."
In the things you can't do (at https://ai.meta.com/llama/use-policy/ ): "Military, warfare, *nuclear industries or applications*" Odd given the climate situation to say the least...
Earlier quoted context omitted.
Start searching SuperHOT and RoPE together. 8k-32k context length on regular old Llama models that were originally intended to only have 2k context lengths.
Any trick which is not doing full quadratic attention cripples a models ability to reason "in the middle" more than they already are crippled. Good long context length models are currently a mirage. This is why no one is seriously using GPT-4-32k or Claude-100k in production right now. Edit: even if it's doing full attention like the commentator says, turns out that's not good enough! https://arxiv.org/abs/2307.03172
A single email to their public complaint system from anyone could have your license revoked.