Live data from Hacker News

The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

thesequence.substack.com

311–320 of 527 posts

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#311
post #299

Earlier quoted context omitted.

Shouldn't that be the default position? The training methods are certainly patentable, but the actual input to the algorithm is usually public domain, and outputs of algorithms are not generally copyrightable as new works (think of to_lowercase(Harry Potter), which is not a copyrightable work), so the model weights would be a derivative work of public domain materials, and hence also forced into the public domain fro…

> the model weights would be a derivative work of public domain materials, and hence also forced into the public domain from a copyright perspective. I don’t think “Public domain” means what you think it means.

Yes, the person to whom you are responding appears to be mixing up "publicly available" (made available to general public) with "public domain" (not protected by copyright).

IANAL but, I think, as far as US law goes, they have the right conclusion for the wrong reasons. Unsupervised training is an automated process, and the US Copyright Office has said [0] that the product of automated processes can't be copyrighted. While that statement was focused on the output of running an AI model, not the output of its training process (the parameters), I can't see how – for a model produced by unsupervised training – the conclusion would be any different.

This is probably not the case in many non-US jurisdictions, such as the EU, UK, Australia, etc – all of which have far weaker standards for copyrightability than the US does. It may not apply for supervised training – the supervision may be sufficient human input for copyrightability even in the US. It may not apply for AI models trained from copyrighted datasets, where the copyright owner of the dataset is claiming ownership of the model – that is not the case for OpenAI/Google/Meta/etc, who are all using training datasets predominantly copyright by third parties, but maybe Getty Images will build their own Stable Diffusion-style AI based on their image library, and that might give them a way of copyrighting their model which OpenAI/Google/Meta/etc lack.

It is always possible that US Congress will amend the law to make AI parameters copyrightable, or introduce some sui generis non-copyright legal protection for them, like the semiconductor mask work rights which were legislated in response to court rulings that semiconductor masks could not be copyrighted. I think the odds are reasonably high they will in fact do that sooner or later, but nobody knows for certain how things will pan out.

[0] https://www.federalregister.gov/documents/2023/03/16/2023-05...

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#312
post #293

Earlier quoted context omitted.

I feel like there would be a good chunk of real humans who would be incapable of answering a question like this.

This is true of all questions.

I remember a possibly apocryphal quote from a park ranger saying that there was a significant overlap between the smartest bears and the dumbest tourists.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#313
post #204

Earlier quoted context omitted.

I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly

Here's one piece of evidence suggesting it's more like rote pattern matching than reasoning. > All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it >> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear rev…

You're reading the promo materials wrong.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#314

I dont think people know many cool things you can do out of the box with text-generation-webui interface for chat models. With extensions for voice in/out, stable diffusion images in/out, long term memory and custom npc backstories its pretty much virtual reality in a box. Just some examples of things you can do. How about create a D&D (or any RPG) game, the NPC can be the dungeon master, creating monsters/loot and r…

An SMS, email, or even physical mail only agent would be pretty interesting. With the socially accepted inherent limitations (text only, async, some level of misinterpretation expected) of those interfaces I'd wager it'd be possible to convincingly jump the uncanny valley today. Forcing to a comms path that's not just real-time chat affords a little more suspension of disbelief.

Particularly with growing token counts, you can have a pen pal, virtual colleague, or friend to bounce ideas off, return to previous thoughts, and chat to in cases where a real one may not exist or be available. A little ELIZA-esque, but adaptable to different needs.

My only concern is this would be be primed for misuse by people already experiencing isolation to further retreat miss opportunity to grow real social connections. Also, any semblance of privacy over those mediums would be a nightmare.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#315

"The training and serving code, along with an online demo, are publicly available for non-commercial use." (from Vicuna's home page.) In what universe is that "open source"?!

https://www.gnu.org/philosophy/open-source-misses-the-point....

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#316

Earlier quoted context omitted.

Haha good luck with that now… it’s in the digital ether available to all on IPFS… at worst you might have to ask around for someone to help you, but its “distributed” widely enough now I don’t think even a billionaire can put this back into the bottle.

And in 6 months it will be outdated. So long LLaMA, and thanks for all the fish. You will be remembered as the slightly-sexier version of GPT-J that was most well-renowned for... checks clipboard ...Macbook acceleration.

LLaMA is part of LLM history in a way that Bard will probably never be

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#317

I dont think people know many cool things you can do out of the box with text-generation-webui interface for chat models. With extensions for voice in/out, stable diffusion images in/out, long term memory and custom npc backstories its pretty much virtual reality in a box. Just some examples of things you can do. How about create a D&D (or any RPG) game, the NPC can be the dungeon master, creating monsters/loot and r…

An SMS, email, or even physical mail only agent would be pretty interesting. With the socially accepted inherent limitations (text only, async, some level of misinterpretation expected) of those interfaces I'd wager it'd be possible to convincingly jump the uncanny valley today. Forcing to a comms path that's not just real-time chat affords a little more suspension of disbelief. Particularly with growing token counts…

People are already using it for dating sites and catfishing on twitter.

I'm interested trying to make an RPG in the style of bards tale, generate the scenes for the game. Each game would be different. Cant get the client side voice gen working yet, but the online voice api works, but thats pay.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#318

Earlier quoted context omitted.

The difference for me with GPT-4 is its ‘understanding’ of the scene and its explanation of WHY you should push or pull. It talks an out a door with people approaching from different directions. It has some idea of what those people would be thinking. That seems different to just ‘mirror writing means do the opposite’.

I asked GPT4 to draw a dog or a skull in openscad and even though the end result was buggy, commenting things in the code here and there and making some volumes transparent I figured out he got it okay. For instance the dog had two eyes two ears one long nose (potatoids). It understood the symmetry of both pairs but was unable to place them at the right place. It's not like it was just misaligned, things were in the…

I think things like this (or simpler things like asking ChatGPT for ascii art of a circle) really show the difference between LLMs and humans. The issue is that it’s a language model rather then an image one, so it doesn’t understand the concept of ‘looks like a dog’.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#319
post #297

Earlier quoted context omitted.

Googlers I've talked to feel that OpenAI was irresponsible by not instituting enough safeguards, and testing it enough before releasing it.

I agree with them. It does feel like Google could match OpenAI if they didn't have a gigantic brand with tons of reputation on the line.

Nah, Google just doesn’t wanna lose all the ad money by building a search killer. As soon as they figure out how to put ads in your chats, they’re gonna release the full models.

Re: The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

#320

Earlier quoted context omitted.

Or they are capable of some level of reasoning.

At this point I'm weakly convinced that, with high-dimensional enough latent space, adjacency search is reasoning.

Yeh- my feel is, language is the framework by which we developed reasoning and we used an organic NN to do it. And at scale an complexity approaching the human brain we get similar results
Post reply on HN