11–20 of 62 posts
[flagged]
Maybe like… focus on the actual content instead of perceived writing patterns? Crazy I know.
Earlier quoted context omitted.
Maybe like… focus on the actual content instead of perceived writing patterns? Crazy I know.
This. I don't particularly like the LLM writing style, but we've read a ton of very poorly written texts over the year with no complaints. LLMs are average writers with an annoying style, but not bad writers. If the content is good, I don't care if a LLM wrote it.
[flagged]
Maybe like… focus on the actual content instead of perceived writing patterns? Crazy I know.
Like what does the second sentence even mean? Is it even a sentence? "The roofline math, the prompt-processing catch, the NPU red herring, and the owner-measured speeds."
Let's also ensure the SSD doesn't age prematurely.
I was under the impression that when you're streaming the weights from disk because the full model won't fit in memory, that it is solely reading from the SSD, not writing, so it wouldn't be causing wear on your SSD.
Can't really run it as well, though. My "mini PC" is an M4 Max with 128GB of unified memory and the memory bandwidth is still sorely lacking for inference (although it's far better than any non-unified consumer architecture!).
To be fair it's "only" half the throughput of a 4090 and a third of an RTX 6000. Significant but not an order of magnitude.
Can't really run it as well, though. My "mini PC" is an M4 Max with 128GB of unified memory and the memory bandwidth is still sorely lacking for inference (although it's far better than any non-unified consumer architecture!).
To be fair it's "only" half the throughput of a 4090 and a third of an RTX 6000. Significant but not an order of magnitude.
[flagged]
Earlier quoted context omitted.
This. I don't particularly like the LLM writing style, but we've read a ton of very poorly written texts over the year with no complaints. LLMs are average writers with an annoying style, but not bad writers. If the content is good, I don't care if a LLM wrote it.
It’s a flag that no care went into it. Web surfing involves lots of little decisions following cues of “is this worth my time”
Putting an LLM on it means you care to make it look nice, but not enough to actually do it. Why bother?
Earlier quoted context omitted.
Maybe like… focus on the actual content instead of perceived writing patterns? Crazy I know.
The writing style is this staccato LLM-like style that is difficult to read as it has zero flow and meaningless sentences. Like what does the second sentence even mean? Is it even a sentence? "The roofline math, the prompt-processing catch, the NPU red herring, and the owner-measured speeds."
` The mini PC's slowness is not a driver problem or a weak chip. It is arithmetic on the bandwidth number. `