The Myth of AI Omniscience: AI's Epistemological Limits

Online portfolios are the worst lie we tell young developers. It’s the software industry’s version of “exposure” gigs: they get a few more bits they can shove into their models, you get jack shit.

Every, and I mean, every job I got I got because I either knew a guy or I knew a guy that knew a guy. You are far better off going to conferences and networking.

The same as it takes to teach a human to recognise objects and output something cogent, except for machine learning models the effort does not need to be replicated from scratch each time.

But you're entirely missing the point of the comment you replied to, which is that giving them feedback loops and ability to carry out actions, and open ended problems to do things like web searches allows for discovery in the same way that a lot of human discovery happens, by posing questions, engaging in discovery and feed their findings back into further training. The only thing stopping us from doing that at scale right now is that training costs are still high enough that people are being conservative and taking the well known paths of feeding in cleaned and well labelled data. But we've already seen a lot of efforts in using LLMs to generate new training data for LLMs, and letting them drive that process without human intervention is an obvious next step (which will run into a huge number of failures, but the successes will keep accelerating future advances).

> The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated.

For humans to be anything more than a "mechanical transformer from its input to its output" (assuming you're using "transformer" in the general sense here rather than in the sense of the specific LLM architecture) presumes not just a non-deterministic, non-materialistic world, but one that fundamentally breaks our understanding of both logic and physics. There's no evidence to suggest we are anything more than well trained, well-structured automatons. And on that basis, while there's every reason to acknowledge that there are additional big leaps to be made, this constant stream of assumptions that there's some inherent barrier between us and LLMs rather than just a set of refinements is pretty much a religious belief and nothing more.

But that was not my point. My assumption to start with is that most of us do not put much on Github that more than a few other people will care about. But most individual accounts are also of minimal value to Github or other people training LLMs. So however small recognition you get from having code accessible publicly (for me it's the occasional recognition when doing job interviews and a few e-mails from people now and again), you lose that to take away far less value to people training LLMs.

For the vast majority of us, it takes a vastly inflated ego to think withholding our code from a public repository will even be noticed by more than a handful of people. There are exceptions, to be sure, but they are vanishingly small proportion of us.

I believe this can be served as example of combination of existing concepts in novel ways, applied to math, which is very formal and, on the other hand, is the basis of many other areas of human knowledge.

If this can serve as a such example, then you are wrong.

Also, what you have shown as your examples are not very much interesting outside of entertainment.