We’re approaching the limits of computer power – we need new programmers • Hacker News Board

Re: We’re approaching the limits of computer power – we need new programmers

#141

Post by throwaway17_17 » Sun, Jan 12, 2020, 6:55 AM UTC

Shorel wrote: post #124

cmdshiftf4 wrote: post #49

Earlier quoted context omitted.

>The renewed interest in C++ and other compiled languages is an indication of the need to get more efficient. I kind of disagree with this based on intuition alone. Most developers, professional developers, are using web tech (JS stack in particular - Node and hyped front-end frameworks). Yet we"re seeing "interest" in compiled languages such as Rust, despite almost nobody using it professionally and almost nobody do…

At first we optimized languages for memory and CPU use, because there was so little of it. When we could, we optimized for ease of writing code, and it lead to bloated and slow systems. This is the current status of things. We are optimizing again for performance, but we want to have our cake and eat it, we want both the performance and the ease of writing code. And the reduction of bugs in the end result. This chang…

The following is not a particularly deep thought, and that is why it is stated simply:

The way to maintain the easy abstractions, or even increase the level of abstraction while still increasing performance is to ditch the proliferation of massive general purpose programming languages and adopt specific DSLs that fit the problem domain. I feel like the language-oriented programming paradigm that Felliesen of Racket champions is certainly in this spirit, but I would like to see a language core that is specifically tailored to performance concerns.

Not that this is important to the general concept, but this is the current focus of my language design efforts, hoping for first public showing in April 2020,

Re: We’re approaching the limits of computer power – we need new programmers

#142

Post by falcolas » Sun, Jan 12, 2020, 7:01 AM UTC

dnautics wrote: post #58

falcolas wrote: post #56

Earlier quoted context omitted.

> paying through the nose for hardware in general You also have to consider that there are limits to how parallel an application can be - Amdahl's Law - at some point even throwing hardware at a scaling issues has its limits. Of course, there's also a truism that the team who implemented the first pass won't have to support (financially or as a developer) the software when it no longer scales.

No, amdahl's law is (roughly speaking) a limit to how parallel an algorithm can be. Applications (in the sense of web apps) generally have the potential to scale via Gustafson's law, but we are (IMO) largely held back by framework and old ways of programming. https://en.wikipedia.org/wiki/Gustafson's_law

So long as an application needs to share state between worker processes, (database, redis cluster, etc) then Amdahl’s law still applies. There’s very few modern applications that can truly scale linearly.

Re: We’re approaching the limits of computer power – we need new programmers

#143

Post by leetcrew » Sun, Jan 12, 2020, 7:05 AM UTC

billti wrote: post #60

leetcrew wrote: post #46

Earlier quoted context omitted.

do you ever compile code? at work I have a machine with an i7-7700 (4C/8T), 32GB of RAM, and an SSD. it still takes about 45 minutes to do a full build of the project I work on, which can easily be triggered by modifying any of the important header files. if I had to do my job on your laptop from 2009, I'd never get anything done.

Came here to say exactly this. I’m waiting for Amazon to deliver the new CPU cooler I ordered today so my i9-9900k can run flat out for longer when building, after I spent so much of yesterday waiting 20 mins for another build when my attempted fixed didn’t pan out. (Though probably more of a reflection on how painful C++ as a language is if you have to touch a common header file).

obviously the whole "textually including header files" model introduces a lot of overhead in compilation, but as I understand it, optimizations are inherently expensive. if you want to do stuff like inlining across module boundaries, full rebuilds are inevitable when you touch commonly used code. I think it's reasonable to expect that compiling optimized builds of complex code will always hunger for more compute power.

Re: We’re approaching the limits of computer power – we need new programmers

#144

Post by ahartmetz » Sun, Jan 12, 2020, 7:23 AM UTC

uncle_j wrote: post #92

gameswithgo wrote: post #87

Earlier quoted context omitted.

except now it has actually been true for years. clock rates aren't increasing. advances in performance have been only from things that are tricky for developers to efficiently leverage (cache, simd, more cores). We need developers who understand these new low level details as much now as we needed that kind of developer in the past.

> except now it has actually been true for years Sure it is true. It isn't a tech journo writing a quick piece to get some clicks. I am quite cynical these days. There hasn't been any competition in the Desktop CPU space for years until 2019. Also clock rates haven't increased since the mid-2000s (there were 5ghz P4 chips). Clock rates being an indication of speed stopped being a thing back then when I could buy a "s…

Minor correction about desktop CPUs: Ryzen 1 came out in Q1/Q2 2017. Initial problems were mostly solved by 2018.

Re: We’re approaching the limits of computer power – we need new programmers

#145

Post by thewebcount » Sun, Jan 12, 2020, 7:26 AM UTC

I don't think this article is seeing the whole picture. The author talks about how programmers used to have to cram a program into 16KB of RAM (or ROM) and it had to be efficient. But that came at a huge cost. Reading 6502 Assembly with variables that could only have up to 6 characters for their names, and were all global was a huge pain in the ass!

We have great optimization tools freely available these days, and when necessary they are used. We also have great standard libraries with most languages that make it fairly easy to choose the right types of containers and other data structures. (You can still screw it up if you want, though.)

As soon as it becomes economically necessary to write more efficient code, we will be tasked with that. I work on professional software and we do a hell of a lot of optimization. Some of it is hard, but a lot of it could be done by regular programmers if they were taught how to use the tools.

Re: We’re approaching the limits of computer power – we need new programmers

#146

Post by atq2119 » Sun, Jan 12, 2020, 7:33 AM UTC

yyhhsj0521 wrote: post #131

burfog wrote: post #121

Earlier quoted context omitted.

What about battery life? If you prefer, call it carbon footprint. Python has a huge carbon footprint. We should get rid of slow languages for environmental reasons.

But then we'd have to pour in more manpower, which involves more commute, more upkeeping (AC, lunch), etc.

So... are you saying people are going to have more kids because we stop using Python?

For equal numbers of humans, all those energy/environmental costs you mention are going to be there regardless of which programming language is used...

Re: We’re approaching the limits of computer power – we need new programmers

#147

Post by fuzzfactor » Sun, Jan 12, 2020, 7:36 AM UTC

nickpinkston wrote: post #59

We're mostly fighting Murphy's Law, not Moore's law. As said below, most problems are so far from being compute/$ or otherwise technically limited and far more about organizational / political issues putting vast inefficiencies into these systems and priorities that fund their creation.

From the article:

>In a lecture in 1997, Nathan Myhrvold, who was once Bill Gates’s chief technology officer, set out his Four Laws of Software. 1: software is like a gas – it expands to fill its container. 2: software grows until it is limited by Moore’s law. 3: software growth makes Moore’s law possible – people buy new hardware because the software requires it. And, finally, 4: software is only limited by human ambition and expectation.

Codified anti-recycling.

Re: We’re approaching the limits of computer power – we need new programmers

#148

Post by rs23296008n1 » Sun, Jan 12, 2020, 7:37 AM UTC

btrettel wrote: post #34

jryb wrote: post #31

Earlier quoted context omitted.

It’s critical for a lot of scientific and industrial applications. Too many things just can’t be done without a supercomputer.

I disagree. I did my masters' on computational fluid dynamics (CFD) and I'd say that a large fraction of supercomputer use (in fluid dynamics at least) is wasted. Mostly because people take naive approaches and end up computing the wrong thing, set up their simulation poorly, reinvent the wheel, use HPC on something that can be computed by hand, etc. If they read more of the literature they'd have a more solid grasp…

Many moons ago when Beowulf clusters were still new, I remember a project where I was given a months worth of then new cluster time to spend. Due to a delay we had to wait a few months before we could exercise it. One weekend I was playing around with some junk computers I'd assembled for a LAN party. Long story short, I tried out a portion of the project on that LAN, got a useful result then we ran the rest of the project on that overall system and joined more machines. We completed the project using no special machines, often just idle machines added and removed upon their availability. Along the way we rewrote the entire project code as a set of modules then used python to orchestrate them.

None of this is extraordinary now but the result was we reduced multiple times the budget requirements and processing times just using code improvements. A lot of times, the head-on solution just needed optimisation. Sideways improvements also helped such as small optimisations also helped. The more exotic equipment is still useful but its an accelerator.

One last thing: we received a LOT of grief and criticism for our approach. There was peer pressure to use particular solution types even though they were wildly inappropriate. We had funding pulled or threatened to be pulled by some and other backers. One lessons we learnt includes: Don't underestimate how vested certain interests are for the use of various toolkits. "Use this or else!" is the not-so-subtle threat.

I'm so glad to not rely on only academic work now.

Re: We’re approaching the limits of computer power – we need new programmers

#149

Post by rs23296008n1 » Sun, Jan 12, 2020, 7:56 AM UTC

leetcrew wrote: post #46

qwerty456127 wrote: post #27

Why do we need more computer power? I haven't upgraded my laptop since 2009 (well, I've replaced its HDD with an SSD 2 years ago and it made a huge difference) and I'm okay. Some people insist on photorealistic 3D graphics in the games they play, I agree that's cool but wouldn't say that's anything close to important.

do you ever compile code? at work I have a machine with an i7-7700 (4C/8T), 32GB of RAM, and an SSD. it still takes about 45 minutes to do a full build of the project I work on, which can easily be triggered by modifying any of the important header files. if I had to do my job on your laptop from 2009, I'd never get anything done.

That is the choice of software tool. You are literally grinding through gigabytes of data. If your tool didn't require so much data processing it would be faster. This may or not be possible to improve by you. Often there are workarounds.

Case in point: a matrix library I used to use needed to a full row/column pass each time. We put a layer in between it and our code. Reduced lookups required by 30%. We were processing the same amount of data and getting the same results but requiring far less time. That layer also reduced memory requirements. Now we could process larger datasets faster with the same hardware. Thats just one example.

Your choice of CPU and other hardware isn't always the limiting factor. Even the language choice has an impact. Some languages/solutions require more data processing overhead than others to get the same final result.

Even the way your program's Makefile or module composition can have an effect on compiling performance. I remember the use of a code generator we included that meant it had to regenerate a massive amount of code each run due to its input files being changed. We improved it by a ridiculous amount simply by hashing its inputs and comparing the hashes prior to running the code generator. Simply not running that code generator each time meant we sped up the build significantly. 30 minute build times reduced by 5-10 minutes. Same hardware. And that was easily triggered by a trivial file change.

Re: We’re approaching the limits of computer power – we need new programmers

#150

Post by mjpuser » Sun, Jan 12, 2020, 8:04 AM UTC

We also have a mantra against optimization until you know that you need to. It seems too cost and time prohibitive to put these things on the programmer to maintain, and that we need to develop tools to help optimize our code. Maybe the next generation of optimization techniques will be runtime instead of compile time. We already have dbs with optimizers, so maybe there will be programming languages with optimizers?