Ask HN: What is the most important publicly unknown fact in your industry?

Having an educational system is a sign that the society is wealthy and can afford it, but doesn't mean it's the best use of resources involved (thus it might have a bad ROI).

You said education was overlooked where you come from, was there an educational system there ?

Li-ion battery energy density grows in spurts. Realistically, there have been a few major improvements, with very small incremental improvements in between. For example, in 2001 we were at 180 Wh/kg. In just a handful of months we jumped to 260 Wh/kg, then to 280 Wh/kg as manufacturing processes improved. In the last 11 years, however, improvements have been maybe a few Wh/kg per year, and there have been literally zero improvements whatsoever in the past four years, by anyone.

Eleven years is a very long time, especially for something that everybody just assumes is constantly silently improving. And it's not like the battery industry stopped investing in growth: there are absolutely insane amounts of money invested in battery R&D, since whoever figures out how to make a cheap, energy-dense battery will become absolutely infathomably rich and make Madoff-level growth look like US savings bonds.

Bonus fact: nobody really understands lithium-ion batteries. One production run might have substantially higher or lower capacity. Some batteries might explode. Some batteries are high discharge and some are low discharge. The "explanation" for all of these things is 'heat'.

Imagine that happening to any other industry (for example the auto industry.)

- "Why did my car just explode!?" "Heat"

- "Why does this car last a tenth the lifetime of this car?" "Heat"

- "Why does this car go a thousand times faster than any others?" "Heat"

- "Why is this entire production run of cars not working?" "Heat"

People say "heat" for two reasons: 1) because nobody understands li-ion batteries and 2) because yes, the real answer does have something to do with heat.

Batteries are one of the most important industries and critical to every company on HN. But the state of the art of batteries is putting together random metals with a bunch of random chemicals and watching which ones don't explode.

Imagine talking to the CEO of Boeing: "We just did our millionth test, engines made of cheese and wings made of coconut shells. It crashed during takeoff, of course. We haven't had any improvements in over a decade, but we're sure we'll get there eventually - we've spent billions on research so far."

Everything from 1% to 66% illiteracy has been attempted to sell to people.

It does seem that illiteracy clusters with various citizenship status, racial and other demographics, simple age distribution of a geographic location, and socioeconomic level to an extent that if you successfully filter those effects out you end up with a boring flat line at 100% literacy when ends up meaning nothing. The largest factor is literacy graphs and maps are basically a restatement of immigration data.

I'd like to see salaries raised if only for the "a rising tide raises all boats".

Vast majority of calculations are performed in excel. Most engineers know at least a bit of programming from one mandatory class in first year, but for some it's fortran, others it's matlab, vba, etc. Excel spreadsheets are the common denominator.

Thankfully, most sheets are rather simple, with one calculation per line, with inputs coming from previous intermediate calculations on the above lines.

Unfortunately, sometimes, the logic needs to be more complex, and that's where things go south. Despite all the best intentions, spreadsheets are very hard to test, audit and version-track, and errors will inevitably sneak in.

Other things get harder too, when all information is in spreadsheets. Recently I was tasked with updating and reporting about 200 calculations that were contained in 200 almost-identical sheets. A single factor needed to be changed on those 200 sheets, and then I needed to compile the new results in a summary table: relatively simple VBA macro, right? Except multiple of those sheets were just slighlty different, due to multiple engineers modifying them for slightly different cases. The solution ended being similar to scraping really screwed up html with something like BS. A variety of regexes, string searches, and even heuristics based on cell formatting needed to be used. A lot of information was conveyed visually in those sheets (ie, a new calculation block started with a title that was typically bold, blue, and 20pt, but sometimes a bit different).

The same thing could've been implemented in python/julia/fortran/heck even C, version tracked and unit tested. Verificating and understanding the calculations is way easier, as all the equations are there in front of you when you open the source file. The program could take input written in simple text file (think ini or even yaml if something more complex is needed), then spit out a report in another text file, or even fancy formatted html. Then, future modifications to handle "special" need not a different program (spreadsheet) for each new case, but in msot cases the master program could be updated while still ensuring backwards compatibility, and all calculations could be rerun in batch from the input txts.

Anyways, that's what I dream of sometimes while I'm either doing rote manual excel-scraping or trying to think of a new convoluted heuristic to program it. Unfortunately, the single most important thing for documentation is that colleagues can understand it, and most MEs still see programming as something too complicated and time consuming. I also work in a big corporation, and IT utterly useless. Getting a custom tool made by IT would be a hell of budget approvals, resource plannings and a bunch of three-letter acronyms for at least a year (seriously) before a programmer would strike a single key. Comparatively, a spreadsheet can be bashed out in an afternoon by basically anyone.