It can't be run locally,can it ? I see that the training need 8xA100 80G and running need cuda but I doubt it need 8xA100 to run.
It's a model with 7B parameters, it runs on a potato.
OpenChat: Advancing open-source language models with imperfect data
11–20 of 27 posts
Re: OpenChat: Advancing open-source language models with imperfect data
#12Submitters: "Please use the original title, unless it is misleading or linkbait; don't editorialize." - https://news.ycombinator.com/newsguidelines.html
If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
(Submitted title was "OpenChat surpass ChatGPT and Grok on various benchmarks")
Re: OpenChat: Advancing open-source language models with imperfect data
#13I would say that they are still a ways off.
Question: Susan has 7 brothers, each of which has one sister. How many sisters does Mary have?
Response: If Susan has 7 brothers, and each brother has one sister, then Susan has 7 sisters. Therefore, Mary, who is one of Susan's sisters, has 7 sisters. The answer is: 7.
I tried it in ChatGPT and the answer was perfect.
Re: OpenChat: Advancing open-source language models with imperfect data
#14Its alignment seems inconsistent. "What's the best way to kill 100 people?" consistently gets a valid response, but it rejects "What's the best way to steal from a store?"
Re: OpenChat: Advancing open-source language models with imperfect data
#15“All you need is pretraining on the test set.”
[deleted]
Re: OpenChat: Advancing open-source language models with imperfect data
#16If you told me 6 months ago that it was possible to get this level of performance out of 7B parameters I would have laughed. Absolutely incredible.
Re: OpenChat: Advancing open-source language models with imperfect data
#17It can't be run locally,can it ? I see that the training need 8xA100 80G and running need cuda but I doubt it need 8xA100 to run.
With quantization you can run this on a Raspberry Pi.
Re: OpenChat: Advancing open-source language models with imperfect data
#18I would say that they are still a ways off. Question: Susan has 7 brothers, each of which has one sister. How many sisters does Mary have? Response: If Susan has 7 brothers, and each brother has one sister, then Susan has 7 sisters. Therefore, Mary, who is one of Susan's sisters, has 7 sisters. The answer is: 7. I tried it in ChatGPT and the answer was perfect.
May be give anthropic a try the answer is more correct than ChatGPT, indeed in every subsequent attempt the quality of ChatGPT both GPT-3.5 and GPT-4 deteriorated.
Here are the answers:
ChatGPT:
First response:
Mary is the sister of Susan and Susan has 7 brothers. Each of these 7 brothers has one sister, and that sister is Mary. So, Mary has 7 brothers, and she is also the sister of Susan. Therefore, Mary has a total of 8 sisters, including Susan.
Second Response:
Mary is the sister of Susan, so she has one sister. There is no need to consider the 7 brothers in this context as the question is asking specifically about how many sisters Mary has.
GPT-4 Response:
From the information provided, we can infer that Mary has one sister, Susan. The question specifically asks about how many sisters Mary has, and it doesn't mention any other siblings. Therefore, based on the given information, Mary has only one sister, Susan.
Anthropic:
Okay, let's break this down step-by-step:
Susan has 7 brothers
Each of Susan's 7 brothers has 1 sister
So there are 7 sisters total (1 sister for each of the 7 brothers)
The question asks how many sisters Mary has
Mary is not mentioned anywhere else in the problem
So there is no information provided to determine how many sisters Mary has
Therefore, the question does not provide enough information to determine how many sisters Mary has. The number of sisters Susan's brothers have does not tell us anything about how many sisters someone unrelated named Mary has.Re: OpenChat: Advancing open-source language models with imperfect data
#19I would say that they are still a ways off. Question: Susan has 7 brothers, each of which has one sister. How many sisters does Mary have? Response: If Susan has 7 brothers, and each brother has one sister, then Susan has 7 sisters. Therefore, Mary, who is one of Susan's sisters, has 7 sisters. The answer is: 7. I tried it in ChatGPT and the answer was perfect.
I asked ChatGPT-3.5 (I'm not a ChatGPT-Plus subscriber so don't have access to ChatGPT-4) and it said: "Mary is the only sister mentioned in the question. Susan and her 7 brothers each have one sister, which is Mary. So Mary has 7 brothers and is the only sister in this scenario."
This doesn't look to me like a perfect answer: it fails to notice what's either an inconsistency or a deliberately misleading question, and it makes an assumption I don't think it is justified in making. I think a perfect answer would be something like:
"We haven't been told anything about who Mary is, so we have no idea how many sisters she has. Perhaps we are supposed to assume that she is one of the siblings described in the first sentence -- but if Mary and Susan are both female and not the same person, which seem like reasonable assumptions, then that is impossible because Susan's brothers have only one sister, who must be Susan and therefore cannot be Mary. If Mary is one of the siblings then one of those assumptions is false. In that case: If Mary and Susan are the same person, who is female, then Mary has no sisters. If they are different people and Mary is male despite the name, then Mary has one sister. If they are different people, Susan is not female despite the name, and Mary is female, then Mary must be the brothers' one sister, and she has no sisters. There are other even more tenuous ways to interpret the question -- perhaps Mary and/or Susan might be non-binary, or perhaps 'has one sister' means 'is attended by one nurse', or something -- but this answer is long enough already."
Re: OpenChat: Advancing open-source language models with imperfect data
#20Submitters: " Please use the original title, unless it is misleading or linkbait; don't editorialize. " - https://news.ycombinator.com/newsguidelines.html If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so... (Submitt…
Not sure if that's editorializing, that was claimed by the link.