[flagged]
Hello Dolly: Democratizing the magic of ChatGPT with open models
171–180 of 194 posts
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#172[flagged]
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#173[flagged]
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#174Earlier quoted context omitted.
That would be anticompetitive practice that is actually against the law in many countries[1]. In the unlikely event of OpenAI ever engaging in such things they will be sued into oblivion. [1] https://en.wikipedia.org/wiki/Refusal_to_deal
No it wouldn't. Wikipedia has a crap definition that inexplicably focuses on cartels where multiple companies coordinate the refusal, which this definitely isn't. The FTC has a better definition for US law [1]. Companies routinely ban users for ToS violations. Just look at any thread about Google on here to see people complaining about it. [1]: https://www.ftc.gov/advice-guidance/competition-guidance/gui...
The FTC link has an example of the only newspaper in town refusing to deal with customers who are also running ads on a radio station. Do you think if the newspaper dressed such refusal as a ToS violation it would fly with FTC?
Google might be banning people for enforceable violations of their ToS but imagine the uproar if they banned a Bing engineer for using Google search to find solutions for some Bing problem (which is similar to the problem here). The upside for Google or OpenAI would be somewhat limited but the downside is almost boundless.
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#175Earlier quoted context omitted.
That’s the repo with the code to train the model to get the weights, not the trained weights.
Did you try emailing?
"Hi, could I have the weights? I'd like to upload them as a torrent so anyone can download them freely without having to ask so as to broaden access."
Can you guess what their reply would be?
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#176[flagged]
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#177Earlier quoted context omitted.
The README also says this: > This fine-tunes the [GPT-J 6B]( https://huggingface.co/EleutherAI/gpt-j-6B ) model on the [Alpaca]( https://huggingface.co/datasets/tatsu-lab/alpaca ) dataset using a Databricks notebook. > Please note that while GPT-J 6B is Apache 2.0 licensed, the Alpaca dataset is licensed under Creative Commons NonCommercial (CC BY-NC 4.0). ...so, this cannot be used for commercial purposes
> ...so, this cannot be used for commercial purposes The implication being that you're only "democratizing" something if people can make money off of it?
Kinda? I (personally) read "democratizing" as intending to be for the benefit of many over the few. Bit duplicitous to preclude access to the means of production in that definition, "for many rather than the few (BUT, wait, the actual economic benefit and utility is still locked away for the few)".
But maybe "democratize" is starting to mean something similar to "open". All the good words.
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#178Earlier quoted context omitted.
Better, yes, and for that we have evidence. But is the improvement stemming simply from even more data? That's what I'm questioning.
This paper is pretty approachable and goes over the "scaling laws" in detail: https://arxiv.org/abs/2206.07682 In short, yes. More data, higher quality data, more epochs on the data. That is the name of the game.
That paper doesn't discuss GPT-4 at all. It does however contain this interesting excerpt (emphasis mine):
> Although we may observe an emergent ability to occur at a certain scale, it is possible that the ability could be later achieved at a smaller scale—in other words, model scale is not the singular factor for unlocking an emergent ability. As the science of training large language models progresses, certain abilities may be unlocked for smaller models with new architectures, higher-quality data, or improved training procedures. For example, there are 14 BIG-Bench tasks5 for which LaMDA 137B and GPT-3 175B models perform at near-random, but PaLM 62B in fact achieves above-random performance, despite having fewer model parameters and training FLOPs.
So it's not obvious that it should be so straightforward.
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#179Does anyone else find it ironic that all these ChatGPT "clones" are popping up when OpenAi is supposed to be the ones open sourcing and sharing their work? I guess: "You Either Die A Hero, Or You Live Long Enough To See Yourself Become The Villain"?
AI and high-performance semiconductors are the only technological fields where the US and allies haven't been surpassed by Russia and China. There is probably a lot of political pressure on OpenAI to be as closed as possible. Remember the US government has banned Nvidia from exporting A100/H100 to China/Russia. Those are the same chips OpenAI uses for both training and inference.
In which fields have Russia surpassed the US? I get China, but Russia?
Re: Hello Dolly: Democratizing the magic of ChatGPT with open models
#180Earlier quoted context omitted.
I think all of the "AI alignment" talk is mostly fearmongering. It's a cunningly smart way to get ignorant people scared enough of AI so they have no choice but to trust the OpenAI overlords when they say they need AI to be closed. Then OpenAI gets a free pass to be the gatekeeper of the model, and people stop questioning the fact that they went from Open to Closed. AI being tuned to be "safe" by an exceedingly small…
I don't know what press releases you've been reading, but the model is closed so they can make money off it, that's pretty obvious.
Seems like it would make sense for that to be the real reason, and the safety concerns to be a convenient scapegoat, although from talking with several people who work at OpenAI, they really do seem to believe the safety/alignment issue deep in their bones. I could almost be led to believe that the massive business advantage of keeping it closed is a happy side effect for them and not the actual reason.