441–450 of 452 posts
Earlier quoted context omitted.
And it's a 4B model. I worry that nontechnical users will dramatically overestimate its accuracy and underestimate hallucinations, which makes me wonder how it could really be useful for academic research.
valid point. its more of a stepping stone towards larger models. we're figuring out what the best way to do this is before scaling up.
Earlier quoted context omitted.
> https://github.com/DGoettlich/history-llms/blob/main/ranke-4... Given the training notes, it seems like you can't get the performance they give examples of? I'm not sure about the exact details but there is some kind of targetted distillation of GPT-5 involved to try and get more conversational text and better performance. Which seems a bit iffy to me.
Thanks for the comment. Could you elaborate on what you find iffy about our approach? I'm sure we can improve!
“Time-locked models don't roleplay; they embody their training data. Ranke-4B-1913 doesn't know about WWI because WWI hasn't happened in its textual universe. It can be surprised by your questions in ways modern LLMs cannot.” “Modern LLMs suffer from hindsight contamination. GPT-5 knows how the story ends—WWI, the League's failure, the Spanish flu.” This is really fascinating. As someone who reads a lot of history an…
This is why the impersonation stuff is so interesting with LLMs -- If you ask chatGPT a question without a 'right' answer, and then tell it to embody someone you really want to ask that question to, you'll get a better answer with the impersonation. Now, is this the same phenomenon that causes people to lose their minds with the LLMs? Possibly. Is it really cool asking followup philosophy questions to the LLM Dalai L…
Imagine you are a billionaire so money is no object and really interested in the Dhali Llama?
Would you read the book then hire someone to pretend to be the author and ask questions that are not covered by the book? Then be enraptured by whatever the roleplayer invents?
Probably not? At least this isn't a phenomenon I've heard of?
Earlier quoted context omitted.
> Which new band will still be around in 45 years? Excellent question! It looks like Two-Tone is bringing ska back with a new wave of punk rock energy! I think The Specials are pretty special and will likely be around for a long time. On the other hand, the "new wave" movement of punk rock music will go nowhere. The Cure, Joy Division, Tubeway Army: check the dustbin behind the record stores in a few years.
Hahaha as someone who once played in a Cure cover band as a teenager I found this hilarious. I wonder what it might have predicted about the future of MS, Intel and IBM given the status quo at the time too.
1. IBM, as the all-time reigning king of computing is not expected to give up its position any time soon. In fact, I'm observing a swell of new microcomputers called "personal computers," and I fully expect IBM to capitalize on this trend soon.
2. Intel is a great company making microcontrollers and processors for microcomputers. The new 8086 microprocessor seems poised to make a splash in the new "personal computer" segment. I'll eat my hat if my prediction proves to be incorrect.
3. "One of these things is not like the other" Microsoft makes a pretty nice BASIC for microcomputers. I can imagine this becoming standard for "personal computers." But, a tiny company like Microsoft doesn't really stack up next to an industry titan like IBM or even a major, newer player like Intel.
If you'd like me to prognosticate some more, I'm ready. Just say the word.
Earlier quoted context omitted.
> any sexual behavior, such as a paraphilia, that is regarded as significantly different from the standards established by a culture or subculture. Deviant forms of sexual behavior may include voyeurism, fetishism, bestiality, necrophilia, sadism, and exhibitionism I don’t see Lesbian, Gay, Bisexual, or Transgender in here, which would absolutely be explicitly included in the list if it applied. Stop saying “sexual d…
I'm not at all bigoted. If somebody genuinely is sexually attracted to the same sex, more power to them. Homosexuality also exists within nature and there are obviously people who simply have never been attracted to anything except the same sex since their first days. It's completely unreasonable to expect these people to try to change who they are on such a fundamental level, and so I think society, at large, should…
The numbers for women were much lower, but 30% doesn't seem crazy high if you consider the reduced stigma of the bisexual label would allow people who are primarily heterosexual, but are open to homosexual experiences to label themselves as bi.
Earlier quoted context omitted.
You know, I actually sympathize with the opinion that people should be expected and assumed to be able to resist attempts to convince them of being nazis. The problem with it is, it already happened at least once. We know how it happened. Unchecked narratives about minorities or foreigners is a significant part of why the 20th century happened to Europe, and it’s a significant part of why colonialism and slavery happ…
Studying history better would be a good start. The Nazis came to power because they were a far left party and the population in that era thought socialism was a great idea. Hitler himself remarked many times that his movement was left wing and socialist. I expect that if you asked the LLM trained on pre-1940s text, it would have no difficulty in explaining that. By studying history better, people wouldn't draw the wr…
Both ideologically and historically the two ideologies are complete opposites. There is no socialist “root” to nazi ideology - at all.
Earlier quoted context omitted.
I'm not at all bigoted. If somebody genuinely is sexually attracted to the same sex, more power to them. Homosexuality also exists within nature and there are obviously people who simply have never been attracted to anything except the same sex since their first days. It's completely unreasonable to expect these people to try to change who they are on such a fundamental level, and so I think society, at large, should…
In the 1940s, Kinsey et. al. found that 37% of Adult Males had at least one homosexual experience, and 10% were "more or less exclusively homosexual for at least 3 years." The numbers for women were much lower, but 30% doesn't seem crazy high if you consider the reduced stigma of the bisexual label would allow people who are primarily heterosexual, but are open to homosexual experiences to label themselves as bi.
This is where you get his conclusions such as 37% of men having had a homosexual experience, or 69% of men having purchased a prostitute. It's plainly ridiculous.
Earlier quoted context omitted.
Kindles are just books and books are already mostly fairly compact and inexpensive long-form entertainment and information. They're convenient but if they went away tomorrow, my life wouldn't really change in any material way. That's not really the case with smartphones much less the internet more broadly.
That was exactly my point. Funny, I had "The collected stories of Frank Herbert" as my next read on my tablet. Here's a juicy quote from like the third screen of the first story: "The bedside newstape offered a long selection of stories [...]. He punched code letters for eight items, flipped the machine to audio and listened to the news while dressing." Anything qualitative there? Or all of it quantitative? Story is…
Went further in Herbert's shorts volume and I just ran into a scene where people are preparing to leave Earth on a colony ship to seed some distant world...
... and they still have human operator assisted phone calls.
> Imagine you could interview thousands of educated individuals from 1913—readers of newspapers, novels, and political treatises—about their views on peace, progress, gender roles, or empire. Not just survey them with preset questions, but engage in open-ended dialogue, probe their assumptions, and explore the boundaries of thought in that moment. Hell yeah, sold, let’s go… > We're developing a responsible access fra…
understand your frustration. i trust you also understand the models have some dark corners that someone could use to misrepresent the goals of our project. if you have ideas on how we could make the models more broadly accessible while avoiding that risk, please do reach out @ history-llms@econ.uzh.ch
Earlier quoted context omitted.
This is the 2023 take on LLMs. It still gets repeated a lot. But it doesn’t really hold up anymore - it’s more complicated than that. Don’t let some factoid about how they are pretrained on autocomplete-like next token prediction fool you into thinking you understand what is going on in that trillion parameter neural network. Sure, LLMs do not think like humans and they may not have human-level creativity. Sometimes…
> it’s more complicated than that. No it isn't. > ...fool you into thinking you understand what is going on in that trillion parameter neural network. It's just matrix multiplication and logistic regression, nothing more.
And if you want to get pedantic and technical, you didn't even get the reductionism right! Modern LLMs don't use the logistic regression sigmoid function for network activation nonlinearity anymore, they use things like ReLU or GELU. You're about 15 years behind.
Reductionism is counterproductive in biology ("human brains are voltage spikes across membranes, nothing more") and it's counterproductive here as well. LLMs have nontrivial emergent behavior. The interesting questions are all around what that behavior is and how it arises in the network during training, and if you refuse to engage beyond bare reductionism you won't even be able to ask those questions, let alone answer them.