361–370 of 380 posts
Why would they use LMSYS rather than A/B testing with the regular ChatGPT service? Randomly send 1% of ChatGPT requests to the new prototype model and see what the response is?
Earlier quoted context omitted.
You can ask it's knowledge cutoff and it will respond November 2023. It have no idea of the big events of the beginning of 2024, like the earthquake in Japan.
> You can ask it's knowledge cutoff and it will respond November 2023 It probably just repeated something based on what common AI cutoffs there are, LLMs doesn't have a sense or self or thought process, they don't know more about themselves than the text given to them about themselves, and even then it is likely to default to some common text from the internet.
I certainly hope it's not GPT-5. This model struggles with reasoning tasks Opus does wonderfully with. A cheaper GPT-4 that's this good? Neat, I guess. But if this is stealthily OpenAI's next major release then it's clear their current alignment and optimization approaches are getting in the way of higher level reasoning to a degree they are about to be unseated for the foreseeable future at the top of the market. (T…
Earlier quoted context omitted.
Me: 478700000000+99000000+580000+7000? GPT4: 478799650000 Me: Well? GPT4: Apologies for the confusion. The sum of 478700000000, 99000000, 580000 and 7000 is 478799058000. I will be patient. The answer is 478799587000 by the way. You just put the digits side by side.
I recently tried a Fermi estimation problem on a bunch of LLMs and they all failed spectacularly. It was crossing too many orders of magnitude, all the zeroes muddled them up. E.g.: the right way to work with numbers like a “trillion trillion” is to concentrate on the powers of ten, not to write the number out in full.
For anyone wondering how to access Go to - https://chat.lmsys.org/ Select Direct Chat Select gpt2-chatbot Other twitter mentions - https://twitter.com/kosenjuu/status/1784952955294421215
Very impressive Prompt: > there are 3 black blocks on top of an block that we don't know the color of and beneath them there is a blue block. We remove all blocks and shuffle the blocks with one additional green block, then put them back on top of each other. the yellow block is on top of blue block. What color is the block we don't know the color of? only answer in one word. the color of block we didn't know the col…
Earlier quoted context omitted.
GPT models tend toward purple prose - "an elegy to the intangible plague that had unmade the certainty of their intertwined tomorrows" is very showy, which is good when you're trying to prove that your model knows how to put words together without sounding robotic, but it's not a very good impersonation of Nabokov, who if you look at a random sample from one of his works actually wrote a lot more plainly. The same wi…
Apparently much of ChatGPT's purple prose and occasional rare word usage is because it's speaking African-accented English because they used Kenyan/Nigerian workers for training.
Do you have a source for the 2nd part of your claim that they use Nigerians for RLHF?
I’m know OpenAI use Kenyans via Sama / Samasource [0] and it has been discussed here [1] before.
1: OpenAI used Kenyan workers on less than $2 per hour to make ChatGPT less toxic https://news.ycombinator.com/item?id=34426421
I'm impressed. I gave the same prompt to opus, gpt-4, and this model. I'm very impressed with the quality. I feel like it addresses my ask better than the other 2 models. GPT2-Chatbot: https://pastebin.com/vpYvTf3T Claude: https://pastebin.com/SzNbAaKP GPT-4: https://pastebin.com/D60fjEVR Prompt: I am a senate aid, my political affliation does not matter. My goal is to once and for all fix the American healthcare sys…
That reason — an independent analysis might conclude — is increasing centralization.
See the biggest shift in the US healthcare system over the last 50 years for example:
There was a 3,200 percent increase in the number of healthcare administrators between 1975 and 2010, compared to a 150% increase in physicians, due to an increasing number of regulations:
https://www.athenahealth.com/knowledge-hub/practice-manageme...
>Supporters say the growing number of administrators is needed to keep pace with the drastic changes in healthcare delivery during that timeframe, particularly change driven by technology and by ever-more-complex regulations. (To cite just a few industry-disrupting regulations, consider the Prospective Payment System of 1983 [1]; the Health Insurance Portability & Accountability Act of 1996 [2]; and the Health Information Technology for Economic and Clinical Act of 2009.) [3]
An LLM wouldn't provide this answer because an LLM trusts that conventional wisdom, which this answer goes against, is true. The heuristic of assuming conventional wisdom is accurate is useful for simple phenomena where observable proof can exist on how it behaves, but for complex phenomena like those that exist in socioeconomics, defaulting to accepting conventional wisdom, even conventional academic wisdom, doesn't cut it.
[1] https://www.cms.gov/medicare/payment/prospective-payment-sys...
[2] https://www.hhs.gov/hipaa/for-professionals/privacy/laws-reg...
[3] https://www.hhs.gov/hipaa/for-professionals/special-topics/h...
Earlier quoted context omitted.
I like your interpretation, but how would they refer back to a plan if it isn’t stored in the input/output? Wouldn’t this be lost/recalculated with each token?
The internal state at layer M of token N is available at every following token > N and layer > M via attention heads. Transformed by a matrix but a very direct lookup mechanism. The state after the final attention layer is not addressable in this way, but it immediately becomes the output token which is of course accessible. Note also that sequential computations such as loops translate nicely to parallel ones, e.g.…