71–80 of 87 posts
Cerebras really has impressed me with their technicality and their approach in the modern LLM era. I hope they do well, as I've heard they are en-route to IPO. It will be interesting to see if they can make a dent vs NVIDIA and other players in this space.
Apparently so. You can also buy in via various PE outfits before IPO, if you so desire. I did.
Earlier quoted context omitted.
You don't need quantization aware training on larger models. 4 bit 70b and 405b models exhibit close to zero degradation in output with post training quantization[1][2]. [1]: https://arxiv.org/pdf/2409.11055v1 [2]: https://lmarena.ai/
I wonder why that is? because they are trained with dropout?
Same reason why you can get a pretty good reconstruction when you add random noise to an image and then apply a binary threshold function to it. The more pixels there are, the more recognizable will be the B&W reconstruction.
Earlier quoted context omitted.
Perhaps. But CCTV cameras and smartphones are huge sources of raw content of the real world. Unless you want to take the argument of Morpheus in The Marix and ask "what is real?"
So let’s crank up total surveillance for better auto descriptions of a picture. We aren’t exchanging freedom for security anymore, what could be reasonable under certain conditions, we just get convenience. Bad deal.
Total surveillance may be necessary for other reasons, like making sure organised crime can't blackmail anyone because the state already knows it all, but it's overkill for AI.
Earlier quoted context omitted.
I refer you to Wachowski & Wachowski (1999)*, building on previous work including Descartes and A. J. Ayer. To whit: humans can't either, so that's an unreasonable question. More formally, the tripartite definition of knowledge is flawed, and everything you think you know has a Munchausen trilemma. * Genuinely part of my A-level in philosophy
I wasn’t expecting your response to be “the truth is unknowable”, but was hoping for something of more substance to discuss.
1. AI can do what we can do, in much the same way we can do it, because it's biologically inspired. Not a perfect copy, but close enough for the general case of this argument.
2. AI can't ever be perfect because of the same reasons we can't ever be perfect: it's impossible to become certain of anything in finite time and with finite examples.
3. AI can still reach higher performance in specific things than us — not everything, not yet — because the information processing speedup going from synapses to transistors is of the same order of magnitude as walking is to continental drift, so when there exists sufficient training data to overcome the inefficiency of the model, we can make models absorb approximately all of that information.
Earlier quoted context omitted.
I refer you to Wachowski & Wachowski (1999)*, building on previous work including Descartes and A. J. Ayer. To whit: humans can't either, so that's an unreasonable question. More formally, the tripartite definition of knowledge is flawed, and everything you think you know has a Munchausen trilemma. * Genuinely part of my A-level in philosophy
So we get the same flaws as before with a higher power consumption. And because it’s fast and easy we now get more fakes, scams and disinformation. That makes AI a lose-lose not to mention further negative consequences.
Are you treating "the internet" as "reality" with this line of questions?
The internet is the map, don't mistake the map for the territory — it's fine as a bootstrap but not the final result, just like it's OK for a human to research a topic by reading on Wikipedia but not to use it as the only source.
Earlier quoted context omitted.
To be honest most LLM's are reasonable at coding, they're not great. Sure they can code small stuff. But the can't refactor large software projects, or upgrade them.
Honestly, most software tasks aren’t refactoring large projects, so it’s probably OK. As the world gets more internet connected and more online, we’ll have an ever expanding list of “small stuff” - glue code that mixes and ever growing list of data sources/sinks and visualizations together. Many of which are “write once” and leave running. Big companies (eg google) have built complex build systems (eg bazel ) to isol…
Earlier quoted context omitted.
I refer you to Wachowski & Wachowski (1999)*, building on previous work including Descartes and A. J. Ayer. To whit: humans can't either, so that's an unreasonable question. More formally, the tripartite definition of knowledge is flawed, and everything you think you know has a Munchausen trilemma. * Genuinely part of my A-level in philosophy
So we get the same flaws as before with a higher power consumption. And because it’s fast and easy we now get more fakes, scams and disinformation. That makes AI a lose-lose not to mention further negative consequences.
I wonder at what point does increasing LLM throughput only start to serve negative uses of AI. This is already 2 orders of magnitude faster than humans can read. Are there any significant legitimate uses beyond just spamming AI-generated SEO articles and fake Amazon books more quickly and cheaply?
It sort of is starting to look like you can linearly boost utility by exponentially scaling token usage per query. If so we might see companies slowing on scaling parameters and instead focusing on scaling token usage.
python3 examples/basic/chat.py -m cerebras/llama3.1-70b
import langroid.language_models as lm
import langroid as lr
llm_config = lm.OpenAIGPTConfig(chat_model= "cerebras/llama3.1-70b")
agent = lr.ChatAgent(
lr.ChatAgentConfig(llm=llm_config, system_message="Be helpful but concise"))
)
task = lr.Task(agent)
task.run()
Earlier quoted context omitted.
I don't understand your question. This isn't turtles all the way down, it's grounded in real world data, and increasingly large varieties of it.
How does the AI know it’s reality and not a fake image or text fed to the system?