511–520 of 695 posts
Earlier quoted context omitted.
Llama 4 Scout, Maximum context length: 10M tokens. This is a nice development.
I don't think RAG will survive this time
Earlier quoted context omitted.
No. That disclaimer implies that there are other likely answers. The age of the earth is completely settled, and has been for a long time. Facts don't care about your feelings.
You misunderstand it completely, as it is not a matter of feelings. And it is not a disclaimer (which you apparently felt as a disclaimer). It is a matter of facts. The facts are, that that computation was performed by Patterson and refined by others. This is, as said, what a good reasoner will tell you. > implies that there Even if there had never been other attempts to answer that question, the "facts"¹ remains as…
Are we going to find out that Meta pirated libgen again, with zero recognition to the authors? “Open-sourcing it” doesn’t magically absolve you of the irreparable damages you’ve caused society. You stole their life’s work so your company could profit off of rage-slop.
Should Taylor swift be liable to pay commission for every piece of music she listened to while training? They will have influenced her work in some way.
I’d rather go the other way and say that the companies have to freely release their data sets, if the data is derived from other people’s work. It would put everyone on a level playing field.
Earlier quoted context omitted.
No. That disclaimer implies that there are other likely answers. The age of the earth is completely settled, and has been for a long time. Facts don't care about your feelings.
You misunderstand it completely, as it is not a matter of feelings. And it is not a disclaimer (which you apparently felt as a disclaimer). It is a matter of facts. The facts are, that that computation was performed by Patterson and refined by others. This is, as said, what a good reasoner will tell you. > implies that there Even if there had never been other attempts to answer that question, the "facts"¹ remains as…
"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.
Earlier quoted context omitted.
https://hnup.date/ ;)
yes this is great but I'd like to pick a different voice. the current one feels too robotic
Earlier quoted context omitted.
> Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation. Doesn’t explain why roughly half of American voters were not “leaning left” during the election. EDIT: 07:29 UTC changed "Americans" to "American voters".
It is not and has never been half. 2024 voter turnout was 64%
Earlier quoted context omitted.
It's not even per token. The routing happens once per layer, with the same token bouncing between layers. It's more of a performance optimization than anything else, improving memory liquidity. Except it's not an optimization for running the model locally (where you only run a single query at a time, and it would be nice to keep the weights on the disk until they are relevant). It's a performance optimization for lar…
BTW, I'd love to see a large model designed from scratch for efficient local inference on low-memory devices. While current MoE implementations are tuned for load-balancing over large pools of GPUs, there is nothing stopping you tuning them to only switch expert once or twice per token, and ideally keep the same weights across multiple tokens. Well, nothing stopping you, but there is the question of if it will actual…
Earlier quoted context omitted.
No it is not. Right leaning opinions are heavily censored and shunned in all major publishing platforms that bots can scrape. For example, before Trump, if you contested the utterly normal common sense and scientifically sound idea that a trans woman is still a man, you would be banned - therefore, people with common sense will simply disengage, self-censor and get on with life.
Maybe because that position is both scientifically and morally unsound and if held strongly will lead to dehumanization and hate, attributes we should prevent any LLM from having.
Seems to be quite a lot of studies finding notable differences in brain “readings” (for want of a better word, sorry not a scientist) between transgender people and others sharing their biological sex.
The first study I read highlights the findings of many studies that the insula of transgender individuals is very different to cisgender individuals, with the insula being “associated with body and self-perception.” [0]
Gosh our brains are truly something else and are not so easily categorised! Now if only I could find a way to learn all this stuff a little bit faster…
[0] https://www.nature.com/articles/s41386-020-0666-3
A collection of many other studies: https://en.m.wikipedia.org/wiki/Causes_of_gender_incongruenc...
Earlier quoted context omitted.
You misunderstand it completely, as it is not a matter of feelings. And it is not a disclaimer (which you apparently felt as a disclaimer). It is a matter of facts. The facts are, that that computation was performed by Patterson and refined by others. This is, as said, what a good reasoner will tell you. > implies that there Even if there had never been other attempts to answer that question, the "facts"¹ remains as…
It is not just “according to some research”, it is also according to the overwhelming scientific consensus at the time. Sources are good but it should not appear as if it is one opinion among possibly many others equally valid.
The appearance that it could be «one opinion among possibly many others equally valid» is all in your head: it is an unduly feeling from a bad mental framework.
The advanced framework (that I advanced) is that of the foundational theory of knowledge: a notion has a source - you computed or reasoned, or somebody else. You do not allow your consultant to believe, so you demand that knowledge is tracked.
You will not accept an oracle.
The paradox is that you are seeing the demand of the source as a support to "belief", while it is the radical opposite: the only thing it will be """believed""" (and not really "believed" - just the end of the chain) is the protocols, that "in the training sources I read statement S".