The diminishing half-life of knowledge

> If it were a battery break through it would take 10 years to turn it into a mass producible product

I think it really did take approximately that much time for LLMs as well. first transformers paper came in 2017, almost 6 years back.

text to code came almost 2-3 years before ChatGPT: https://www.microsoft.com/en-us/research/video/intellicode-c...

so even for software with the same underlying tech i.e transformers, it took almost 5 years to get to a breakthrough that can be scaled.

I really like paulg's observation that "knowledge grows fractally". If you put the scope of chatgpt as all human jobs, it would still seem that its we have only scratched the surface, same in terms of throwing money at it, we are only throwing a very little fraction of the money

> We need to value readable code over DRY, or Design patterns or what ever the next FOTM is. We need to laud people who make systems simpler, who reduces costs, who reshape and extend existing things rather than build new ones and pile more on top because it's "easy" or looks good on the resume.

not just in tech, its always been hard to quantify and reward people based on non-functional attributes of a system's output.

> I am part of the problem too, and I need to stop. I need to stop reaching for that next shiny bit of tech, next framework, next library. Because acting like a schizophrenic ADHD child finding candy hasn't really served anyone.

referencing paulg again, I think this reach for next shiny bit of tech should still happen, but reaching fractally i.e in context of everything that you do, reaching for new tech should be a small part of it but still an essential component to grow

while I agree that there is a half-life to certain type of knowledge but I think it would be overstating to reasonably apply to all knowledge.

its very easy to retain a significant portion of the knowledge by building mental models. sure you will forget about the API surface of a technology, but you would surely remember the underlying knowledge models and you would surely apply it in many other contexts.

like remembering the REST API inputs and output data types is also a knowledge whose half life is much shorter. but building mental models of those would stay for a long time. the point never is to remember and include REST API input and output data as part of your skill set, the point is to include their underlying knowledge. and you can't treat the API surface of a library as knowledge, its essentially a volatile memory and supposed to be cleared away

like who would care if you can bring certain type of knowledge's half life to 200 years. since you can't live anymore for that much time, you don't even need to think about solving that problem. just need to think about how to convert knowledge whose half life is less to the knowledge that has higher half life. (like converting API surfaces to mental models)

> Cutting costs by adopting a better architecture was a big thing at one of my previous jobs. People were praised and promoted for cutting thousands of dollars off monthly AWS bills.

I think this is a slippery slope. praising is fine, but aws bills is essentially a non-functional quality attribute of the software. the job is to never even let it become a problem in the first place

what about teams that have built their software in time and with quality, are they essentially losing one of the opportunities to get promoted because they built a better software in the first place?

>I do. We need to do better. We need to stop piling on more and strip back to clear and well understood. We need to value readable code over DRY, or Design patterns or what ever the next FOTM

This sentiment is common in people that lack understanding why each of the stack elements currently in place has been put in there for. I'm not picking on parent specifically, but having been working for "big tech" for quite some time I'm meeting young-er people that are starting their first gig in a "big tech stack" company(at a senior position due to their entire 5 years of experience) and their first instinct is as the above. "Why are you using all this crap? Just rip it all out and start from scratch!

No

I was like that a couple of times in my career and being more convincing I was allowed to" rip it all out" on more than one occasion. A year later my system was better than the original, but by the time I finished it was already out of date with "modern practices" and during that year I rediscovered every single seemingly stupid decision I saw made in the original system.

Now, when I see something that doesn't make sense that looks like a mind boggling tech stack doing almost nothing(and yet it works well) I ask myself, what is it I don't know about this. What documentation is lacking/out of date? (all of it usually). I then dive into the source code and learn why things were done the way they are. Also knowing how the entire stack works from the bare metal to k8 and "server less" helps.

If I was an educator I'd make up an It curriculum teaching the basics of how computers work with something like basic on 8 bit, later assembly.

Then I'd go through features of modern hardware, multi-cores, caches, etc.

Then networking basics with focus on low level protocols "tcp/Ip Ethernet, vlans, vpns". With some layer 7 stuff added like HTTP(S),SMTP etc at the same time as OS level knowledge based on Linux on the console and Windows Server is introduced(as well as bash/powershell/python scripting) . I'd have students code simple servers/clients, stand up their own SMTP gateway and Web server on bare metal. Data science should run from this time on too.

Only after they've been using and learning on bate metal for at least a year I'd start with hypervisors. Teaching things like distributed switching, more advanced features like FT and HA using virtual machines. Also NAS and Sans at this stage.

Then and only then comes containers and k8 at the same time as serverless and cloud. Then topic like resilience, meeting SLAs, SLOs (risk calculation) , business continuity, DR in depth, etc

As mentioned before few things should be thought alongside this throughout, probably java programming, data science with python. Maybe ML basics.

I don't know what is thought on It courses these days, but I strongly suspect not what I listed above. /rant over