11–20 of 38 posts
> The first few tokens often carried minimal semantic information—sometimes just a start-of-sequence marker or common words like "the" or "a." I wonder if it makes sense to use the first word as a title of sorts rather than going straight in grammatically correct sentence when prompting
Another approach I've seen is the "Diff transformer" from MS Research (https://github.com/microsoft/unilm/tree/master/Diff-Transfor...).
> The first few tokens often carried minimal semantic information—sometimes just a start-of-sequence marker or common words like "the" or "a." I wonder if it makes sense to use the first word as a title of sorts rather than going straight in grammatically correct sentence when prompting
> The first few tokens often carried minimal semantic information—sometimes just a start-of-sequence marker or common words like "the" or "a." I wonder if it makes sense to use the first word as a title of sorts rather than going straight in grammatically correct sentence when prompting
"Magnets. How do they work?"
Seems like this was a better solution to the same problem https://www.evanmiller.org/attention-is-off-by-one.html
> The first few tokens often carried minimal semantic information—sometimes just a start-of-sequence marker or common words like "the" or "a." I wonder if it makes sense to use the first word as a title of sorts rather than going straight in grammatically correct sentence when prompting
Some people start their prompts with "Hello" or "Please" or something similar, out of some habitual sense of politeness, I think. It would be hilarious if those prompts really work better because the model can use those words as attention sinks.
However, that seems to be contradicted by what was shown recently with the successful International Math Olympiad effort. Their prompts, such as https://github.com/aw31/openai-imo-2025-proofs/blob/main/pro... , were very terse. It's hard to tell where the prompt stops and the CoT response starts, in fact.
So there is probably some interplay between the need for attention sinks and the use of step-by-step reasoning. It might not be too surprising if the latter works because it's an indirect way to optimize the former.
Earlier quoted context omitted.
Some people start their prompts with "Hello" or "Please" or something similar, out of some habitual sense of politeness, I think. It would be hilarious if those prompts really work better because the model can use those words as attention sinks.
One point that Karpathy has made in some of his videos is that using additional tokens in the prompt can facilitate computation. If you ask a transformer to do some basic math, it will be more likely to get the right answer (or at least a better approximation) with a more verbose prompt. To me, this backs up the use of more conversational language ("Please," etc.) when prompting. However, that seems to be contradicte…