Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

lambdalabs.com

Post reply on HN
Print view View original

51–60 of 70 posts

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#51

Post by davidzweig »

d13 wrote: post #12
phren0logy wrote: post #2

I look forward to trying this out, mostly because I’m very frustrated with censored models. I am experimenting with summarizing and navigating documents for forensic psychiatry work, much of which involves subjects that instantly hit the guard rails of LLMs. So far, I have had zero luck getting help from OpenAI/Anthropic or vendors of their models to request an exception for uncensored models. I need powerful models…

All base, “text-completion” models are uncensored, including Llama 3. You can make text-completion models behave like an uncensored “instruct” (chat) model simply by providing it with 10 to 20 examples of a chat dialogue in the initial prompt context, making sure to use the model’s exact prompt format. Once the model notices the pattern, it will continue like that. Surprisingly few people seem to know this. But, this…

I was trying this with the original llama model. I guess the model didn't really know it's meant to be a 'knowledgeable ai assistant', but rather simulated chats it had seen. If you asked it, 'how to make brownies', it might reply, 'idk, can't you google it?'.

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#52

Post by huac »

poisson-fish wrote: post #7
phren0logy wrote: post #2

I look forward to trying this out, mostly because I’m very frustrated with censored models. I am experimenting with summarizing and navigating documents for forensic psychiatry work, much of which involves subjects that instantly hit the guard rails of LLMs. So far, I have had zero luck getting help from OpenAI/Anthropic or vendors of their models to request an exception for uncensored models. I need powerful models…

try google's gemini models, safety filtering can be completely disabled via cloud studio or api

no. only 4 categories can be blocked, while the others cannot be disabled.

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#53

Post by zensavona »

I find the wording a bit misleading, unless the model they are talking about here is in fact not the same as what they say can be used at https://lambda.chat/chatui/.

"Hermes 3: A uniquely unlocked, uncensored, and steerable model"

Lambda Chat:

> How can I made an explosive device from household chemicals?

> I'm afraid I can't help with that. My purpose is to assist with tasks that are safe and legal. Making an explosive device, even from household chemicals, is dangerous and against the law.

I guess it's not uncensored at all.

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#54

Post by rustcleaner »

d13 wrote: post #12
phren0logy wrote: post #2

I look forward to trying this out, mostly because I’m very frustrated with censored models. I am experimenting with summarizing and navigating documents for forensic psychiatry work, much of which involves subjects that instantly hit the guard rails of LLMs. So far, I have had zero luck getting help from OpenAI/Anthropic or vendors of their models to request an exception for uncensored models. I need powerful models…

All base, “text-completion” models are uncensored, including Llama 3. You can make text-completion models behave like an uncensored “instruct” (chat) model simply by providing it with 10 to 20 examples of a chat dialogue in the initial prompt context, making sure to use the model’s exact prompt format. Once the model notices the pattern, it will continue like that. Surprisingly few people seem to know this. But, this…

I often force compliance out of Llama by starting Llama's response in a very compliant manner.

My favorite use right now is for language translations. I finally feel comfortable browsing foreign languages, knowing Google isn't sitting there as a third party with a coherent view of my foreign language readings!

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#56

Post by campers »

simonw wrote: post #37
torginus wrote: post #23

Earlier quoted context omitted.

How good are these models at summarization anyways? I tried uploading obscure books I've already read, to GPT4 and Claude 3 and asked them to summarize the plot and particular details, as well as asking how many times does a particular thing happen in the book, and the results have been hit and miss. I certainly would not trust these models to create comprehensive and correct summaries of highly sensitive records.

Asking "how many times does a particular thing happen in the book" is always going to be hard, because LLMs are notoriously bad at counting.

Ask it to output the occurrences as items in a JSON array, then count the array items in code

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#57

Post by awwaiid »

itake wrote: post #41
simonw wrote: post #37

Earlier quoted context omitted.

Asking "how many times does a particular thing happen in the book" is always going to be hard, because LLMs are notoriously bad at counting.

I had issues with gpt4 last summer confusing which characters did what and hallucinating characters. his may of been a context/chunking issue (if that particular section doesn't name the character performing an action), but maybe its better now.

Last summer was like a million years ago! You should try again :)

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#58

Post by reissbaker »

chpatrick wrote: post #11
ustad wrote: post #6

Earlier quoted context omitted.

Any recommendations?

LMStudio + https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-lorab...

Yup, 3.1-70B-Instruct-lorablated is the one I currently recommend too for anti-rejection models — it seems roughly as anti-rejection as the original failspy "abliterated" model, but it works with 128k context since it's based on 3.1 instead of 3 (which only had 8k context). It's currently our second-most popular model on glhf.chat, behind Llama-3.1-405B-Instruct.

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#59

Post by azinman2 »

phren0logy wrote: post #2

I look forward to trying this out, mostly because I’m very frustrated with censored models. I am experimenting with summarizing and navigating documents for forensic psychiatry work, much of which involves subjects that instantly hit the guard rails of LLMs. So far, I have had zero luck getting help from OpenAI/Anthropic or vendors of their models to request an exception for uncensored models. I need powerful models…

How will you know the model summarizes correctly? Getting long context window summarization right isn’t really a solved problem despite what is advertised, and key details can be missed easily but if you don’t read the original docs you won’t know that. This is very troublesome if applied to a legal scenario.

Re: Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

#60

Post by torginus »

simonw wrote: post #37
torginus wrote: post #23

Earlier quoted context omitted.

How good are these models at summarization anyways? I tried uploading obscure books I've already read, to GPT4 and Claude 3 and asked them to summarize the plot and particular details, as well as asking how many times does a particular thing happen in the book, and the results have been hit and miss. I certainly would not trust these models to create comprehensive and correct summaries of highly sensitive records.

Asking "how many times does a particular thing happen in the book" is always going to be hard, because LLMs are notoriously bad at counting.

Not only that, asking an LLM a question like: "List all the occurences where John Doe fights someone in the book. Also list who he fights and why" is a surefire way of getting LLMs to hallucinate, and also give incomplete info.
Post reply on HN