Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We’re approaching the limits of computer power – we need new programmers

theguardian.com

Post reply on HN
Print view View original

81–90 of 265 posts

Re: We’re approaching the limits of computer power – we need new programmers

#81

Post by champtar »

One of the best 2h practical course that I had was just write the fastest square matrix multiplication. You could use any language, any algorithm, just no libraries. The target was a 32 core CPU server (this was ~10 years ago). At 5000x5000 all the Java and Python attempts were running out of memory. In C, We tried some openmp, some optimized algorithm, but in the end the best trick was to flip one of the matrix so that memory could be always prefetched. Out of curiosity another student tried GNU Scientific Library, it turned out to be ~100 times faster. My take away was find the right tool for the job!

A fun read on cloud scale vs optimized code is this recent article comparing ClickHouse and ScyllaDB (https://www.altinity.com/blog/2020/1/1/clickhouse-cost-effic...)

Re: We’re approaching the limits of computer power – we need new programmers

#82

Post by pchiusano »

tluyben2 wrote: post #21
falcolas wrote: post #13

Earlier quoted context omitted.

Even if the software only needs to respond at a certain speed, scale will quickly make you either pay through the nose for better hardware or optimize the software so that it can respond in a small fraction of the original speed. The trick, as always, is finding balance between paying for hardware and paying developers.

But that is the case now too and in my experience it swung to paying through the nose for hardware in general; as more or less a sidetrack I take on projects where I optimise (mostly online) systems. Example; a few weeks ago a startup asked me to check out their setup as they were spending almost 30k$/mo on AWS. I spent a few days optimising and now they are down to less than 10k$. With some more work it will be a fe…

I don’t know if you feel this way, but my complaint about the more hyped cloud services is that not only can they be expensive (fine) but the promised time-savings and simplicity of operating the system often doesn’t really materialize either, except in restricted circumstances that you don’t appreciate in advance and only find out later after you’ve already committed.

If it really did save time and were simpler, some companies would (quite reasonably) be willing to pay a premium for that - time is money and all that. In reality it seems like people often end up with the worst of both worlds - it’s expensive, complicated, still needs a huge staff to maintain, and doesn’t even work that well.

Re: We’re approaching the limits of computer power – we need new programmers

#83

Post by tluyben2 »

johnny22 wrote: post #71
tluyben2 wrote: post #33

Earlier quoted context omitted.

Sure, but my point was that they cut that bill with 20k PER month by giving me 5k one off... They gave me 10k runway to poke around but 5k was enough to fix it; it was simply that bad to start with. The low hanging fruit in most systems I see is really trivial to fix; they just have no one to do it... I bet other people here have seen that before when thrown into an existing project (and I read Spolsky at an impressi…

seems like you deserve more of a cut than that.

Well, the premise going in after a quick (very quick) review of the system was: 'I will check what I can do in 5 days at $10k; I believe I can help, but if I cannot, you lose $10k. If I can help you in less time, you only pay that time.'. I do not think I can move that to some other deal with that premise. Maybe if I say; 'I will do this for 50% of the money you save in 12 months after I am done' that would work, but this is is a side thing which I do because I like optimizing things; if I sell it in another way, it's not bound to time which will make it a timesink and risk. It is a choice.

Re: We’re approaching the limits of computer power – we need new programmers

#84

Post by sp332 »

username90 wrote: post #65
errantspark wrote: post #57

Earlier quoted context omitted.

Well it can't, but it doesn't matter in the way your colleagues thought it would. Excel is a much better product than Google Sheets, but having the better product doesn't mean having the winning product.

> Excel is a much better product than Google Sheets, but having the better product doesn't mean having the winning product. Much better product? Sheets takes literally seconds to download and install and runs on all your devices. Also it automatically syncs your data between devices and sharing data with other people is as easy as sharing a website. These are very important features in my view and makes Sheets into a…

It's been possible for multiple people to edit an Excel file simultaneously for a long time. Since 2017 (at least) you can use OneDrive as the file location, so you get all those syncing and sharing benefits you mentioned. The newer Click-to-Run installer takes about 2 minutes to get the app to a usable state, and if that's just too long, there's always https://office.live.com/start/Excel.aspx

Re: We’re approaching the limits of computer power – we need new programmers

#85

Post by dehrmann »

narag wrote: post #54
pjc50 wrote: post #23

The major problem areas are those where it's economically "best" to do the computationally inefficient thing. The obvious example is the quick-to-build MVP, but many of the bigger problems come from platform conflicts. Because we have at least five different actively uncooperating operating system platforms, it's hard to build portable native apps - so people build electron apps instead. We also use the web browser a…

I just realized that this situation hasn't improved much in twenty years. But the problems are really commercial (or maybe "political") not technical. Then there's the ongoing War On Native Apps. I'm no prophet, but I did predict then that the browser would have eaten all the business applications space by now. It was just obvious. Colleagues objected that the web could not match rich native programs.

> this situation hasn't improved much in twenty years

It's gotten much worse. Now you have iOS, Android, Windows, Mac, wed, and Linux(?). In 2000, you had Windows. You couldn't do anything interesting on mobile, Web 2.0 (cringe) wasn't a thing, and Mac's market share was about 3%.

Re: We’re approaching the limits of computer power – we need new programmers

#86

Post by dehrmann »

ledauphin wrote: post #10

I just don't buy this. I cut my teeth as a HPC programmer working with C and writing no-lock algorithms. There will always be a need for that, but realistically the vast majority of software being developed is simply not performance-critical. It's designed to work at human speed. Advances in language, compiler, and runtime implementations will continue to keep up with any growth in the need for performant application…

I have a few PCs running Windows 10 that are older than 5 years. As long as they have SSDs and you're not gaming, they're still plenty fast, even for modern websites.

Re: We’re approaching the limits of computer power – we need new programmers

#87

Post by gameswithgo »

uncle_j wrote: post #9

Every few years something like this gets written. I remember similar things being written in 2004-2005 before the Core 2 line of processors came out. There is still improvements being made to the current tech or new takes on the current tech that aren't incorporated yet in the current bunch of consumer processors. Also I happen to think that what makes a computer fast is the removal of bottlenecks in the hardware. Yo…

except now it has actually been true for years. clock rates aren't increasing. advances in performance have been only from things that are tricky for developers to efficiently leverage (cache, simd, more cores). We need developers who understand these new low level details as much now as we needed that kind of developer in the past.

Re: We’re approaching the limits of computer power – we need new programmers

#88

Post by tluyben2 »

pchiusano wrote: post #82
tluyben2 wrote: post #21

Earlier quoted context omitted.

But that is the case now too and in my experience it swung to paying through the nose for hardware in general; as more or less a sidetrack I take on projects where I optimise (mostly online) systems. Example; a few weeks ago a startup asked me to check out their setup as they were spending almost 30k$/mo on AWS. I spent a few days optimising and now they are down to less than 10k$. With some more work it will be a fe…

I don’t know if you feel this way, but my complaint about the more hyped cloud services is that not only can they be expensive (fine) but the promised time-savings and simplicity of operating the system often doesn’t really materialize either, except in restricted circumstances that you don’t appreciate in advance and only find out later after you’ve already committed. If it really did save time and were simpler, som…

Well it is better than making even trivial architectures with actual hardware (I have some pictures of me hauling servers over xmas a long time ago; that was cheap (in monthlies and hardware, not in hours!) but I would and do pay a premium for that). Otherwise I do agree somewhat; most overbearing systems can be done much simpler but we are all preparing (and thus paying) for eventualities that most likely will never occur or will not influence the bottom line.

Tech like AWS Lambda (of which I like the theoretical idea) are meant to remedy the issues with complexity for a premium. But that premium makes, personally, my eyes water. I cannot see any high volume operation justifying going live with it. Are there big examples of those? And how is it justified vs the alternatives (which are, besides some programmer+admin time and scalability) far more efficient?

Re: We’re approaching the limits of computer power – we need new programmers

#89

Post by kick »

username90 wrote: post #65
errantspark wrote: post #57

Earlier quoted context omitted.

Well it can't, but it doesn't matter in the way your colleagues thought it would. Excel is a much better product than Google Sheets, but having the better product doesn't mean having the winning product.

> Excel is a much better product than Google Sheets, but having the better product doesn't mean having the winning product. Much better product? Sheets takes literally seconds to download and install and runs on all your devices. Also it automatically syncs your data between devices and sharing data with other people is as easy as sharing a website. These are very important features in my view and makes Sheets into a…

Excel also has a web client that can be downloaded in seconds and has more features than Google Sheets.

Re: We’re approaching the limits of computer power – we need new programmers

#90

Post by kraig911 »

How many years was it where bridges were made only out of wood? I feel engineers before us had the foresight to see the possibilities but lacked the tools and understanding.

I fear only when people realized the economy needed to support large mammals crossing a bridge at one time did they really engineer bridges to support that weight. I think the same metaphor could be said for computing.

Post reply on HN