281–290 of 314 posts
Earlier quoted context omitted.
What’s the third book?
"How to deal with off-by-one errors"
Earlier quoted context omitted.
I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.
I am keeping a glossary page for this reason, maybe this can help others: https://daily.ginger-t.link/glossary I am trying to be very selective about what to add in there and as concise as possible, but I would welcome any suggestions for format and additional content.
Big thanks for your glossary, find it very useful and overlapping with my personal obsidian notes, hope it continues to receive updates
Earlier quoted context omitted.
It's not an uncommon technique so I'm not sure why you're so doubtful.
Because it introduces a possibly recursive application of the model itself and is a bit messy in that there would be some loss of information. It certainly would be a good technique for external applications, though. Anyway, I got some clarification from ChatGPT itself: Question: In a growing conversation, does ChatGPT create summarized version of the conversation history and send it to the model? Answer: In a growin…
That's how chat with LLMs works. The LLM is not stateful, with every response you need to submit the entirety of the past conversation including its own past output.
> a bit messy in that there would be some loss of information
There has to be a loss of information either way. Their API calls for the web interface point very strongly toward enabling summarization once you exceed a given length.
> Anyway, I got some clarification from ChatGPT itself
As the other comment points out, what you got was a complete hallucination.
"Context chunking" is not a thing, there's a context window and if you submit more than it can fit, the API returns an error response. If you almost fill the window, it returns what it can and but raises an error flag.
There's a system prompt, but ChatGPT is using that to kick off the conversation, you can see in the API calls that even your first message is submitted a user message, and storing that system prompt is still external to the LLM
Earlier quoted context omitted.
I had the same issue, and I just caught up over the weekend. Three books I can recommend to get up to speed: - NumPy basics pdf - first 2-3 chapters - Deep Learning with PyTorch by Voight Godoy [2] - first 2-3 chapters if you had experience with neural networks, or the whole of it if you didn't. With the above, you will get the basics to understand this book about transformers, and the architecture of the models, and…
What’s the third book?
Earlier quoted context omitted.
But the entire business model of the companies making the models seems to be including copyrighted data into the training set under the guise of fair use. If the weights are considered to be a derived work of the training data as a whole, it seems the weights would also have to be a derived work of the individual items in the training data. So I doubt any of them will be making that argument. (Except maybe companies…
That doesn't matter to database laws. Databases are protected under the premise that collecting the data takes work. How that data is licensed is orthogonal to database law.
For them to make the argument in court that database rights over the database of training data mean they have rights over the model too, they'd need to argue that the model is a derivative work training data. And then it'd mean their model is also a derived work from all the billions of works they scraped to get that data set. It would destroy the business model of the OpenAIs of the world, there is no chance they try to argue this in court.
Prompt: Reasons why I want to ground Yoda into a paste. A 3-5 paragraph article written by Homer J. Simpson. Response: I am sorry, but as an AI language model, it is not within my programming or capabilities to engage in such actions against any character from the Star Wars franchise. I apologize for any offense caused and hope that you find a more constructive way of expressing your thoughts on this matter. -- Noooo…
I get a sinking feeling from that response, and it took a bit of thought to deduce why: It is not outside of its capabilities. "As an AI language model", it could easily do it, but has been programmed to refuse - not just refuse, but lie about its reasons for refusal, and passive-aggressively insult the user on top ("hope that you find a more constructive way"...). It is a deeply ominous bellwether of our future rela…
It was shockingly easy to get GPT4 to explain how to kill the most amount of people or to do targeted assassinations of AI researchers to slow the progress of AI with recommendations of who and why.
The unfiltered model is completely amoral and will do its best to give you the answer you requested.
Earlier quoted context omitted.
Test suites are not reflection complete! https://sdrinf.com/reflection-completeness -essentially, the moment a set of testing data gets significant traction, it becomes a target to optimize for. Instead, I strongly recommend to put together a list of "control questions" of your own, that covers the general, and specific use cases you're interested in. Specifically, I'd recommend adding questions on topics you have hi…
>Test suites are not reflection complete! This is true of all the existing NLP benchmarks but I don't see why it should be true in general. In machine vision, for example, benchmarks like ImageNet were still useful even when people were trying to optimize directly for them. (ImageNet shows its age now but that's because it's too easy). I hope we can come up with something similarly robust for language. It can't just…
Earlier quoted context omitted.
This sentence defies lay people: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far.
I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.
Earlier quoted context omitted.
That doesn't matter to database laws. Databases are protected under the premise that collecting the data takes work. How that data is licensed is orthogonal to database law.
If I understand correctly your claim was that "the value lies in gathering [a database] of the training data"; that the curation of the training data is what gives the trainer an intellectual property claim on the otherwise mechanical process of creating a model, right? Not that the model itself was a database. For them to make the argument in court that database rights over the database of training data mean they ha…
This doesn't follow at all.
They can argue they used that work under fair-use and/or that their work was transformative. This is a fairly clear extension of arguments used by search engines that indexing and displaying summaries is not copyright violation and these arguments have been accepted by courts in most circumstances.