221–230 of 314 posts
Earlier quoted context omitted.
What is a lora or llama? Google gives me nothing.
LoRA: https://arxiv.org/pdf/2106.09685.pdf LLaMA: https://ai.facebook.com/blog/large-language-model-llama-meta... Both have been the subjects of numerous HN posts in the last month.
Earlier quoted context omitted.
I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.
Ah, my fellow citizen of the interwebs, fear not! Your intellectual frustrations are but a natural reaction to the tsunami of technological jargon. You see, the AI wave is the epitome of obfuscation, a testament to the labyrinthine lexicon of the digital age. It's as if a group of caffeinated, sleep-deprived tech enthusiasts assembled in the dark of night and decided to create an impenetrable fortress of vernacular,…
Earlier quoted context omitted.
This sentence defies lay people: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far.
I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.
- NumPy basics pdf - first 2-3 chapters - Deep Learning with PyTorch by Voight Godoy [2] - first 2-3 chapters if you had experience with neural networks, or the whole of it if you didn't.
With the above, you will get the basics to understand this book about transformers, and the architecture of the models, and everything else, from this book:
- Natural Language Processing with Transformers: Building Language Applications with Hugging Face ( https://www.amazon.com/Natural-Language-Processing-Transform... ).
I took a weekend to go through the books in this order, and now I finally understand what people mean with all that jargon :)
1 - https://numpy.org/doc/1.18/numpy-user.pdf - 2 - https://www.amazon.com/Deep-Learning-PyTorch-Step-Step/dp/B0...
Earlier quoted context omitted.
There's a neat trick when you encounter jargon. 1. Identify the jargon terms you don't understand 2. Lookup papers that introduce the jargon terms 3. Skim-read the paper to get the gist of the jargon If you don't want to do this, then you don't have to feel uneducated. You can simply choose to feel like your time is more important than skimming a dozen AI papers a week. But for example, here's what I did to understan…
> 3. Skim-read the paper to get the gist of the jargon Or, you know, you could ask ChatGPT to explain it to you... Granted the term was coined 2021>=. Even if it wasn't but the paper is less than 32k tokens... 0.6c for the answer doesn't seem all that steep. edit: grammar
It works astoundingly well with poorly written technical manuals. Looking at you, CMake reference manual O_O. It also helps translate unix man pages from Neckbeardese into clean and modern speech.
With science papers it's a bit more work. You must copy section by section into GPT4, despite the increased token limit.
But sure. Here's how it can work:
1. Copy relevant sections of the paper
2. As questions about the jargon:
"Explain ____ like I'm 5. What is ____ useful for? Why do we even need it?"
"Ah, now I understand _____. But I'm still confused about _____. Why do you mean when you say _____?"
"I'm starting to get it. One final question. What does it mean when ______?"
"I am now enlightened. Please lay down a sick beat and perform the Understanding Dance with me. Dances"
This actually works surprisingly well.
Earlier quoted context omitted.
I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.
I feel you! Did you miss the web 3 wave though? I still can't imagine what that was about.
Also, building websites that don't store user data at all. Everything is kept in browser storage. You could say that the chat-gpt interfaces people are building now are web3, because they don't store your api keys, nor your converstation history.
Second part was decentralising as much as possible. Decentralised domain-name systems (ENS), storage, hosting, and money of course. So that you own your data, and your identity.
The last time I checked, the decentralised storage and hosting were the most difficult to solve. That is - we have torrents of course, but if you wanted to pay decentralised web to host and run your scripts indefinitely, it was not feasible.
The important takeaway for both GPT4All and Alpaca is that once an expensive proprietary model is released, people can easily train cheaper OSS models on input/output pairs. ⇒ LLMs are not defensible ⇒ LLMs will become commoditized ⇒ Prices will drop ⇒ Great for open source
This isn't the stable diffusion vs DALLE-2 moment yet. The performance lacks so much they ain't the same thing.
Earlier quoted context omitted.
It’s not a licensing issue because OpenAI does not own the copyright to the output of GPT-4. The people who might have a copyright claim are: the authors of the training data, if there is a clear resemblance between the training data input and the output, and rarely the author of the prompt. OpenAI could argue that because some of the training data was written by them, they have a copyright claim to the output. Howev…
This can thus be circumvented by an intermediary that happens to release the output somewhere. Since you don't need permission from the copyright owner to train models, you just.. take it from their website or something. As long as you don't accept their terms of service, there is nothing they can do.
Attempting to control what users do in their own time with public domain information from your website may be a step too far for a click or browse wrap ToS.
Earlier quoted context omitted.
We're a long long long way off that. So check back in two months. Jokes aside, the limiting factor will be either a technique to pack all of the code into smaller tokens like semantic search (someone else will be able to comment on this as that's the limit of my understanding) or GPU memory for input tokens. Buying a "high end GPU" isn't buying a 4090 or even two, it's 250k on a DGX unit and putting it in a datacentr…
"Buying a "high end GPU" isn't buying a 4090 or even two, it's 250k on a DGX unit and putting it in a datacentre. You will probably be able to find a service that would sign a confidentiality agreement and provide you with this service for less than 250k." Does anyone have a projection based on historical GPU cost reductions as to how long we have to wait until a DGX unit costs as little as a 4090 does today?