211–220 of 648 posts
Earlier quoted context omitted.
Their translation service is based on llms and is commercially a successful product.
Transformer models rather than LLMs surely. ChatGPT behaves nothing like Google Translate.
Earlier quoted context omitted.
>There are no license issues like with the facebook llama. OpenLLaMa uses a dataset which does not seem to have gotten propper commercial licensing for the training data. There is potential licensing issues because the copyright situation is not well defended.
So far as anyone knows, this is not a derivative work, its transformative, and therefore not subject to any licensing requirement. You're right though, that's arguably still up for debate, but I think the precedent of transformative work is pretty well attested.
Earlier quoted context omitted.
Counterpoint: there's way too much "art makers", copyright is keeping the quality ceiling down, good content loses monetization game with spam - all while that "predictive text generator" is, for better or worse, pretty much the most magnificent piece of technology invented this century, and - for better or worse - it's likely to become next major shift to economy and life.
- HN on social media : The powers are too centralized, future is decentralization, question is how - HN on free software: is good - HN on copyright : WAY TOO MUCH PEASANTS CLAIMING INDIVIDUAL RIGHTS, RIGHTS THAT ARENT EVEN REAL, ART BE CENTRALIZED FOR MAXIMUM MONOPOLY
"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…
I mean the guy who created GPT-4 literally demanded a ban of any system more powerful than GPT-4.
Earlier quoted context omitted.
Interesting on a meta point that the more clickbaity title "GPT-4 details leaked" won out over the more dispassionate but drier "GPT-4 Architecture, Infrastructure, Training Dataset, Costs".
When choosing titles for my own submissions, yeah, the accurate title that HN says they desire gets no votes whatsoever. Any clickbait on here, people bring upon themselves (and this isn't even a clickbait-level title)
The whole point of accurate titles is that you'll get less votes on uninteresting content.
Earlier quoted context omitted.
Yep they (would) basically have 8-16 "experts" that are each about the size of GPT-3. Since they each see different batches of the dataset, they learn to model those distributions independently rather than the distribution of the whole dataset. Some of the attention is shared between them however. Then another "routing model" decides which model is most suitable for the given user prompt. Given they use relatively fe…
This is not how mixture of experts works at all. The experts are chosen on each layer, not for the whole network, and attention is shared between all of them.
So you’re saying the experts chosen are a more literal mixture of layers from each model? Rather than a simple “pick which model to run”?
Earlier quoted context omitted.
Counterpoint: there's way too much "art makers", copyright is keeping the quality ceiling down, good content loses monetization game with spam - all while that "predictive text generator" is, for better or worse, pretty much the most magnificent piece of technology invented this century, and - for better or worse - it's likely to become next major shift to economy and life.
How do you believe copyright is keeping the quality down in particular? As I see it, the reason is marketability and trying to conform to standards. Making a new Marvel Movie is sure to have a payoff. Making Parasite MIGHT have a higher payoff, but the risk is way higher. Removing copyright might help to get various more spin-offs and reiterations on previous work, which does increase the scope, but taking the abilit…
Earlier quoted context omitted.
LLaMA 30B or 60B can be very impressive when correctly prompted. Deploying the 60B version is a challenge though and you might need to apply 4-bit quantization with something like https://github.com/PanQiWei/AutoGPTQ or https://github.com/qwopqwop200/GPTQ-for-LLaMa . Then you can improve the inference speed by using https://github.com/turboderp/exllama . If you prefer to use an "instruct" model à la ChatGPT (i.e. tha…
4-bit quantization removes a lot of the model's sophistication, and 60B parameters is still smaller than what GPT4 is using.
Earlier quoted context omitted.
Yeah, they went from v1 to regulatory capture in the span of months
> regulatory capture If this leak is correct, regulatory capture is likely the only moat OpenAI could have hoped for. It would explain why Sam was so absolutely adamant that this tech needed to receive oversight. If correct, every big tech company now has a recipe to build their own GPT-4. I'd expect for the open source efforts to try to duplicate the results as well. LLMs will increase in quality across the board an…
Earlier quoted context omitted.
From the post: [Rumors that start to become lawsuits] Some speculations are: - LibGen (4M+ books) - Sci-Hub (80M+ papers) - All of GitHub This is the most funny, but in the end sad aspect. If ChatGPT was indeed trained on pirated content and is able to be(come) such a powerful tool, then the copyright laws should have been abolished yesterday. If ChatGPT was not trained on all these resources out there, then think ho…
That is a very dangerous way to approach copyright laws. They are definitely abused by corporations like Disney, infamously so, but abolishing them is absolutely not the answer. Art makers are already struggling en masse, taking away their ability to earn money off their work isn't an answer, especially if it's just to train a predictive text generator.