161–170 of 214 posts
Earlier quoted context omitted.
spam and especially phishing will become much, much better it will be hard to trust anything at all
Which will force drive signed communications. So... win.
Earlier quoted context omitted.
Nothing about a zero or few-shot classifier precludes you from validating on a data set.
You can certainly hold out a validation set while you are writing your prompt, but you can't know whether the model is over fitting for your data set since you don't know what data was in the training set.
But I posit that most text classification tasks don't have such strict accuracy requirements. For one, no text classifier is 100% accurate. For instance, I have genuine mail in my spam folder frequently. I see spam on social networks, etc. I struggle to think of cases that aren't at least somewhat tolerant to some amount of incorrect classification.
Earlier quoted context omitted.
I already have a custom browser plugin that calls out to GPT (gpt-3.5-turbo is cheap and good enough for this) to classify and filter out low-effort, overly negative, or intellectually dishonest HN comments. It significantly improves the experience on this site. Bonus points: I had never written a browser plugin, but GPT4 helped me do it in under half an hour.
I don’t know how anyone can trust to run code on their machine that they don’t understand and hasn’t been reviewed by a third party.
Earlier quoted context omitted.
I already have a custom browser plugin that calls out to GPT (gpt-3.5-turbo is cheap and good enough for this) to classify and filter out low-effort, overly negative, or intellectually dishonest HN comments. It significantly improves the experience on this site. Bonus points: I had never written a browser plugin, but GPT4 helped me do it in under half an hour.
I don’t know how anyone can trust to run code on their machine that they don’t understand and hasn’t been reviewed by a third party.
How sure are you that all that code has been reviewed by a 3rd party? How many CVEs a year impact your laptop/desktop?
Do you have any reason to think that increased productivity with LLM assistance will result in lower quality code? Personally I find LLM assistance increases productivity, decreases the penalty of using a more difficult language like rust, and makes it more palatable to spend more (LLM assisted) time writing tests.
Earlier quoted context omitted.
I already have a custom browser plugin that calls out to GPT (gpt-3.5-turbo is cheap and good enough for this) to classify and filter out low-effort, overly negative, or intellectually dishonest HN comments. It significantly improves the experience on this site. Bonus points: I had never written a browser plugin, but GPT4 helped me do it in under half an hour.
This sounds super cool, I’m very curious how you implemented it. I’m only vaguely familiar with the API.. if I had to guess I would say you send - a system instruction that its job is to filter unwanted content - examples of unwanted content - an instruction like “filter the following html:” For every web request you want to filter, you would re-send all of those messages followed by the page HTML as the final messag…
- Examples of unwanted content
- Then I give a large numbered list of comments and ask which numbers should be filtered
- The plugin then just deletes those comment nodes from the DOM. If HN ever updates their HTML I will have to tweak this code.
The reason to send a large list of comments is just to save on costs. It's cheaper to do it this way than one comment at a time.
So the main difference from what you've proposed is GPT never sees the HTML. My code enumerates the comments in the HTML and splices them in to the prompt in a nice numbered list, then does the reverse translation from list number to DOM element in the other direction.
Earlier quoted context omitted.
I already have a custom browser plugin that calls out to GPT (gpt-3.5-turbo is cheap and good enough for this) to classify and filter out low-effort, overly negative, or intellectually dishonest HN comments. It significantly improves the experience on this site. Bonus points: I had never written a browser plugin, but GPT4 helped me do it in under half an hour.
I don’t know how anyone can trust to run code on their machine that they don’t understand and hasn’t been reviewed by a third party.
Earlier quoted context omitted.
is there a difference in the quality of llm one would be able to train or run on a gpu with 8, 12, 16, all the way up to 24gb? I'm trying to decide whether it's worth while to splurge on a more expensive 4090 vs a 4070 or whatever.
It makes 0 economical sense to buy a GPU to train a model. If you want to train a model, train it on the cloud.
So finetuning with LoRAs and a few other methods is fine on higher end consumer hardware like a 4090 and finishes in a reasonable amount of time - IMO definitely worth it if you're experimenting with this especially for the inference.
The base training though yeah I totally agree with you - train in the cloud, don't buy hardware when you need a month of 8x A100's or whatnot.
Earlier quoted context omitted.
It makes 0 economical sense to buy a GPU to train a model. If you want to train a model, train it on the cloud.
It's more that I'm building a gaming pc this summer, and I can either target 1440p (4070) for 2k or 4k for 5k (4090). If I can do a lot more with a 4090 over a 4070 it might make sense, but I know a lot of cs students use google colab these days, so I may just rely on that.
The 24GB of VRAM is 100% worth it alone. If you want to do local ML stuff you _need_ that 24gb of VRAM.
64GB of ram + 24GB of vram lets you run a lot of the medium size models at decent speeds. I don't use Colab personally but AFAIK it should work fine for you if you don't want to do it locally.
Also worth noting is the newer ray tracing rendering that cyberpunk is doing. You should checkout the demos IMO it looks sick. It only runs at 18fps on a 4090 so it's only playable on a 4090 + dlss, and I'm not sure if the newer rending tech will be super achievable on any of the other cards - if that's of interest to you.
Earlier quoted context omitted.
It makes 0 economical sense to buy a GPU to train a model. If you want to train a model, train it on the cloud.
This isn't necessarily true with LoRAs - a 4090 can train/compute the alpaca dataset with LoRA in under 6 hours (it might be 3, I forget what it was). So finetuning with LoRAs and a few other methods is fine on higher end consumer hardware like a 4090 and finishes in a reasonable amount of time - IMO definitely worth it if you're experimenting with this especially for the inference. The base training though yeah I to…