Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hello Dolly: Democratizing the magic of ChatGPT with open models

databricks.com

Post reply on HN
Print view View original

61–70 of 194 posts

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#61

Post by mejutoco »

sillysaurusx wrote: post #22
chatmasta wrote: post #19

Earlier quoted context omitted.

I figured it was a reference to the Dalai Lama (which doesn't invalidate your comment, since that's also pronounced like Dalí). LLM -> Llama -> Dalai Lama

Hmm. Is Salvador Dalí pronounced differently than Dolly or Dalai? The wikipedia page has "dah-lee" as the phonetic, and https://www.google.com/search?q=pronounce+salvador+dali sounds the same as https://www.google.com/search?q=pronounce+dalai+lama . So it seems like all three are identical.

Dalí has the stress on the last syllable, hence the accent (but Dall-e probably not). In my native language Dalai is pronounced "Da-lie", like another comment says above. TIL Dolly is pronounced so similarly. I thought the Do sounded like Doberman, but apparently not.

https://www.merriam-webster.com/dictionary/dolly

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#62

Post by bob1029 »

> Surprisingly, instruction-following does not seem to require the latest or largest models: our model is only 6 billion parameters, compared to 175 billion for GPT-3.

We started seeing this in our testing. OpenAI's Curie model is responding very well to our fine-tuning experiments for chatbot-style interface. I am trying to keep us focused on quality of training data rather than obsessing over raw network size. Davinci (and derivatives) might turn out to be overkill for our use cases.

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#63

Post by amrb »

thewataccount wrote: post #27

I might be having a moment - but I can't find any links to a git repo, huggingface, or anything about the models/weights/checkpoints directly from the article. I just see a zip download that AFAIK also doesn't contain the weights/checkpoints. I find this a bit odd, the contents of the zip (from the gdrive preview) look like they should be in a git repo, and I assume they download the model from somewhere? GDrive usua…

Are they pulling a Facebook, on model access?

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#64

Post by neilv »

leobg wrote: post #7
Mizza wrote: post #3

It's immediately become difficult to untangle the licensing here. Is this safe for production use - I have no idea if I can expect a DMCA from Mark if I step out of bounds with this or other post-Alpaca models, unless I'm missing something important. Meta really botched the Llama release.

Why? Dolly had nothing to do with Llama or its weights. Besides: How would anyone ever know which model generated the output you are serving? AFAIK there is no fingerprint in any model’s output. And even if there was, it would probably be destroyed by fine tuning “over it”.

> Besides: How would anyone ever know which model generated the output you are serving?

There's precedent for "whatever you can get away with" in tech companies, but establishing a culture of that at the start of this new big change could end up undesirable for most people.

For example, it could relieve demand for more legal and sustainable ways, until it's too late. (Look at the history of digital entertainment media piracy and DRM and legislation, for example. Or look at the history of software piracy, where some big companies seem to actually want their product to be pirated, partly because it builds a bigger moat against competitors, and they can legally strongarm some of those pirates later.)

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#65

Post by ricardobeat »

rockzom wrote: post #58
renewiltord wrote: post #29

Earlier quoted context omitted.

Wow, just discovered that the American pronunciation for Dalai Lama is Da-lee. Well, that's a discovery. This is like when Khan Academy came out and there was a guy online saying it's a terrible brand because it sounds like Con Academy which it doesn't in my dialect. Took a while to get it.

How do you say Khan?

Kãn / k-ä-n, A like in “father”

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#66

Post by joseda-hg »

sillysaurusx wrote: post #9

Interesting. DALL-E, Dalai ( https://cocktailpeanut.github.io/dalai/ ), and now Dolly are all pronounced the same way. It feels like there should be an xkcd for this.

Are they? (Not sarcastic, I'm not native and I wouldn't pronounce them all that similar at first sight)

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#67

Post by sp332 »

rnosov wrote: post #17
Mizza wrote: post #12

Earlier quoted context omitted.

So, this is a "dirty" model, in that is was created by data which violated OpenAI ToS. Obviously, this kind of violation is basically fine if you're a massive corporation who the rules don't apply to, but it's a huge risk if you're a small fish.

ToS are not the law. It would be similar to your power company claiming copyright over the code written using "their" electricity. Not going to happen. I wouldn't be too concerned.

No, but you could be banned from using OpenAI products in the future, which seems like quite a liability for a researcher or company.

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#68

Post by rburhum »

chatmasta wrote: post #19
mejutoco wrote: post #18

Earlier quoted context omitted.

AFAIK DALL-E is pronounced as Dalí, as in Salvador Dalí. https://en.wikipedia.org/wiki/Salvador_Dal%C3%AD

I figured it was a reference to the Dalai Lama (which doesn't invalidate your comment, since that's also pronounced like Dalí). LLM -> Llama -> Dalai Lama

Dalí has an accent at the end, which has the emphasis in the last letter. Dalai does not. They sound very different. “dah-lee” vs https://m.youtube.com/watch?v=JhFbvuKn45w

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#69

Post by slimsag »

thewataccount wrote: post #27

I might be having a moment - but I can't find any links to a git repo, huggingface, or anything about the models/weights/checkpoints directly from the article. I just see a zip download that AFAIK also doesn't contain the weights/checkpoints. I find this a bit odd, the contents of the zip (from the gdrive preview) look like they should be in a git repo, and I assume they download the model from somewhere? GDrive usua…

The README also says this:

> This fine-tunes the [GPT-J 6B](https://huggingface.co/EleutherAI/gpt-j-6B) model on the [Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) dataset using a Databricks notebook.

> Please note that while GPT-J 6B is Apache 2.0 licensed, the Alpaca dataset is licensed under Creative Commons NonCommercial (CC BY-NC 4.0).

...so, this cannot be used for commercial purposes

Re: Hello Dolly: Democratizing the magic of ChatGPT with open models

#70

Post by gremlinsinc »

Mizza wrote: post #12
rnosov wrote: post #11

Earlier quoted context omitted.

They are instruction tuning it using the dataset released by stanford-alpaca team. The dataset itself is synthetic (created using GPT-3) and somewhat noisy and in my view can be easily recreated if OpenAI ever tries to go after it (which is very unlikely). Anyway, facebook has nothing to do with anything used by this project.

So, this is a "dirty" model, in that is was created by data which violated OpenAI ToS. Obviously, this kind of violation is basically fine if you're a massive corporation who the rules don't apply to, but it's a huge risk if you're a small fish.

If you use output, from a non-profit who open sourced the output gained by following the TOS, as in they aren't using it 'for profit', it's not illegal, because:

A. it's an output gained via following the letter of the law (TOS).

B. TOS only applies directly to people who've accepted the TOS, unless alpaca's license/TOS ALSO forwards the same criterion as it's source at openai, then derivatives wouldn't apply.

It's like if an app developer on IOS violated a TOS, and apple tried to go after everybody who ever used the app, they didn't agree directly to the TOS, only the developer did.

Post reply on HN