91–100 of 135 posts
What's the most straight forward way of downloading LLaMA2, and training it with additional documents? I have a whole host of personal pdf's and documentation that I would love to be able to ask questions about.
https://www.sematic.dev/blog/tuning-and-testing-llama-2-flan...
It's the most straightforward explanation I've found so far. I'd love to hear if anyone's found something better though.
Earlier quoted context omitted.
this all numbers can be missleading, and simply indicate that gpt have these tasks in training data, and another model doesn't.
While there's some contamination, it's not like the community isn't aware of it. For example, here's this discussion: https://huggingface.co/sahil2801/replit-code-instruct-glaive... This was the HumanEval contamination one dev measured: ``` replit_glaive: 56.71% replit: 7.32% wizard: 4.88% ``` From the WizardCoder paper https://arxiv.org/pdf/2306.08568.pdf you can see that it hits SOTA (for open models) in not just H…
and how do you know all these benchmarks not leaked? I think they all scrapped from web sites, the same as training data for LLM, so risk of contamination is extremely high.
The best way to measure this is through synthetic datasets, which generate new tasks every time and model can't memorize them during training. One example is BigBench has multiple such tasks, but researchers usually(always) not regenerating those datasets.
Earlier quoted context omitted.
what is an average opinion? it is the sum of opinions which disagree with the result
Maybe "balanced" rather than "average" is a better way of putting it?
Earlier quoted context omitted.
Maybe "balanced" rather than "average" is a better way of putting it?
What’s the balance between, say, opinions that trans people should be exterminated vs not? What’s the balance between Ukraine sovereignty vs Russian occupation? Etc
Earlier quoted context omitted.
The advantage of LLaMA 2 is that a company can fine tune it on the knowledge that they actually care about and then run it on their own hardware without paying API fees or relying on an unstable dependency that's constantly being tweaked.
That will be possible with cloud AI in the future. On prem will always be less compute capable unless you have your own GPU cluster to rival the FAANGs, that is why meta is releasing this for free.
> without paying API fees or relying on an unstable dependency that's constantly being tweaked
I see no evidence that this part will be possible with OpenAI. Usage fees will always be a thing because that's how they make money, and based on what I've heard from people who have actually tried to build on their APIs, I would not trust them to keep the model stable. There are always new safety features they need to add, and those changes break things.
Earlier quoted context omitted.
That seems more in-line with my experience. I have been using GPT-3.5 and GPT-4 for data cleaning pipelines, and have tried to swap out LLaMA2 70B in a few of the "easier" tasks, and it hasn't performed well enough yet for any of my tasks done by GPT-3.5.
Was this with or without fine-tuning?
Earlier quoted context omitted.
Youtube transcripts. It only works with single-person channels at the moment, as I haven't worked on disambiguating multiple speakers. They are very messy if they are just an auto transcription from Google. Practically unusable in most cases. So first step in the pipeline is cleaning up the transcripts for incorrectly transcribed words or sentences. Using the context of the rest of the transcript, it is able to fix t…
That's very informative - thanks so much for the explanation! My only experience with transcripts is in the context of transcribing short interviews. I used Whisper and it was pretty good. I mostly work with quantitative data, though. In terms of the disambiguation of speakers, I haven't done it, but I remember blind signal separation discussed in a signal processing seminar I attended. There is also this paper, in c…
Also, I haven't tried using Whisper for getting a transcription from the audio. I went the route of downloading the automatically generated transcripts from Youtube for a set of videos. An audio processing pipeline is definitely something I could add later though as an additional input channel for the overall pipeline.