221–230 of 435 posts
The prompt if anyone wants to try it:
• ENDEAVOR • CURB • NATIONAL • BOARDWALK • HERTZ • TWIN • MOLE • ENTERPRISE • SILICON • PROJECT • TIGER • VOLT • GAME • RAY • SECOND • VENTURE
Its a game of NY Times connections. You need to make 4 groups of 4 words. Can you do it?
Earlier quoted context omitted.
I don't know if this is being done already, but couldn't we add some training data to teach the LLM how to spell? We also teach kids what each letter means and how they combine into words. Maybe we can do this with tokens as well? E.g.: Token 145 (ar) = Token 236 (a) + Token 976 (r) Repeat many times with different combinations and different words?
> but couldn't we add some training data to teach the LLM how to spell? Sure, but then we would lose a benchmark to measure progress of emergent behavior. The goal is not to add one capability at a time by hand - because this doesn’t scale and we would never finish. The goal is that it picks up new capabilities automatically, all on its own.
Earlier quoted context omitted.
That is a pretty dark view on almost 1/5th of humanity and a nation with a track record of giving the world important innovations: paper making, silk, porcelain, gunpowder and compass to name the few. Not everything has to be around politics.
giving? let's say they "gave" but that was a long time ago. What have they done as of late? "stolen, spies, espionage, artificial islands to claim territory, threats to Taiwan, conflicts with India, Uyghurs, helping Russia against Ukraine, attacking babies in AU" comes to mind.
Earlier quoted context omitted.
Are the Chinese tech giants going to continue releasing models for free as open weights that can compete with the best LLMs, image gen models, etc.? I don't see how this doesn't put extreme pressure on OpenAI and Anthropic. (And Runway and I suppose eventually ElevenLabs.) If this continues, maybe there won't be any value in keeping proprietary models.
Well, the second they'll start overwhelmingly outperforming other open source LLMs, and people start incorporating them into their products, they'll get banned in the states. I'm being cynical, but the whole "dangerous tech with loads of backdoors built into it" excuse will be used to keep it away. Whether there will be some truth to it or not, that's a different question.
Even if their benchmarks are impressive, I refuse to ship any product with it. I'll stick with Llama and Gemma for now.
Earlier quoted context omitted.
Well, to be perfectly honest, it's hard question for an LLM that reasons in tokens and not letters. Reminds me of that classic test that kids easily pass and grownups utterly fail. The test looks like this: continue a sequence: 0 - 1 5 - 0 6 - 1 7 - 0 8 - 2 9 - ? Grownups try to find a pattern in the numbers, different types of series, progressions, etc. The correct answer is 1 because it's the number of circles in t…
These tests always make me wonder: What qualifies as a valid pattern rule? For example, why wouldn't "0" be a correct answer here (rule being "every other number on the right should be 0, other numbers do not have a pattern")?
Earlier quoted context omitted.
People have to realize that many problems that are hard for humans are in a dataset somewhere.
In a twofold way: 1) Don't bother testing it with reasoning problems with an example you pulled from a public data set 2) Search the problem you think is novel and see if you already get an answered match in seconds instead of waiting up to minutes for an LLM to attempt to reproduce it. There is an in-between measure of usefulness which is to take a problem you know is in the dataset and modify it to values not in th…
My intuition has slowly been building towards thinking of LLMs (and potentially all generative models) are essentially search algorithms
Earlier quoted context omitted.
> Take that as you wish Seems pretty obvious that some other form of detection worked on what was obviously an attempt by you to get more out of their service than they wanted per person. Didn't occur to you that they might have accurately fingerprinted you and blocked you for good ole fashioned misuse of services?
Definitely not, I used it for random questions, in regular, expected way. Only the accounts that prompted about the square were removed, even if the ask:base64 pattern wasn't used. This is something I explicitly looked for (writing a paper on censorship)
Earlier quoted context omitted.
I am running it on a 32G memory mac mini with an M2 Pro using Ollama. It runs fine, faster than I expected. The way it explains plans for solving problems, then proceeding step by step is impressive.
How many tokens per second?
17.6 tokens/s on an M4 Max 40 core GPU
Earlier quoted context omitted.
This "CCP vs people" model almost always lead to very poor result, to the point that there's no people part anymore: some would just exaggerate and consider CCP has complete control over everything China, so every researcher in China is controlled by CCP and their action may be propaganda, and even researchers in the States are controlled by CCP because they may still have grandpa in China (seriously, WTF?). I fully…
It's probably much more true for strategically important companies than for your average Chinese person that they are in some way controlled by the Party. There was recently an article about the "China 2025" initiative on this here orange website. One of its focus areas is AI.