Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We’re approaching the limits of computer power – we need new programmers

theguardian.com

Post reply on HN
Print view View original

241–250 of 265 posts

Re: We’re approaching the limits of computer power – we need new programmers

#241

Post by pdimitar »

rs23296008n1 wrote: post #233
pdimitar wrote: post #230

Earlier quoted context omitted.

We have a good number of them indeed but nobody wants to pay them to fix most of the IT area. Ironic, right.

Or we can't get them past the HR hiring policies that eliminate all candidates. Been through that myself. I even got multiple tech leads to sit the testing and watched them fail. They were already on the team yet would not be able to get on the team . Absurd but true. Contracting is such a strange world. I've drifted so far into it I've lost the ability to see how salary based people even get work. All I can do is ke…

Can you tell me more? Sounds quite humorous. And quite usual...

Re: We’re approaching the limits of computer power – we need new programmers

#242

Post by pdimitar »

rs23296008n1 wrote: post #149
leetcrew wrote: post #46

Earlier quoted context omitted.

do you ever compile code? at work I have a machine with an i7-7700 (4C/8T), 32GB of RAM, and an SSD. it still takes about 45 minutes to do a full build of the project I work on, which can easily be triggered by modifying any of the important header files. if I had to do my job on your laptop from 2009, I'd never get anything done.

That is the choice of software tool. You are literally grinding through gigabytes of data. If your tool didn't require so much data processing it would be faster. This may or not be possible to improve by you. Often there are workarounds. Case in point: a matrix library I used to use needed to a full row/column pass each time. We put a layer in between it and our code. Reduced lookups required by 30%. We were process…

That's just picking the lowest hanging fruit which is quite common sense. Not to diminish your achievement but if tomorrow, say, I figure Rust's compiler is slow, there's nothing I can do [in the foreseeable future].

Having much better hardware in these cases helps you be actually productive and not twiddle your thumbs waiting for the compiler.

Re: We’re approaching the limits of computer power – we need new programmers

#243

Post by tluyben2 »

pdimitar wrote: post #229
tluyben2 wrote: post #83

Earlier quoted context omitted.

Well, the premise going in after a quick (very quick) review of the system was: 'I will check what I can do in 5 days at $10k; I believe I can help, but if I cannot, you lose $10k. If I can help you in less time, you only pay that time.'. I do not think I can move that to some other deal with that premise. Maybe if I say; 'I will do this for 50% of the money you save in 12 months after I am done' that would work, but…

I am curious how do you even find a side job like that. I am definitely a spiritual brother with you because I love optimising things. But I am very unsure how do I even start a side career with that premise. Any advice?

> Any advice?

Spend a lot of time with funded startups. Meetups, conferences etc. They will be happy to talk about this. But also online; you need to 'dox' nicks some times, but when you see quite broad questions in slack/reddit about performance of systems and you find out this is some (tech) (co-)founder you can ask them to help. I do no-cure-no-pay if the system is an MVP and crud; I do no-cure-still-pay if the system is larger and already live. That is not because I want to blackmail the company (and if I like the idea you can give me a % as well instead, all fun and games), but usually because 'wanting to help' is punished when it's 'free' as in no good deed will go unpunished. I Did no-cure-no-pay with optimising (and other services) live systems in the past, but as soon as I touch it, people blame me for all kinds of dataloss (while i'm very careful and absolutely make (offsite) backups always) and other misery. So when basically what I do is connect with (co)founders who are in a jam and when they don't have production data yet, I will go no-cure-no-pay; when they have production data they need to keep, I will explore but if I cannot do anything (for that price, mind you; there is always something to do), I still get paid.

There are probably literally 1m projects and growing at any time in this world that have serious issues and that are burning money and that will crash (all the time or sooner or later) that need help. For instance, I know of a large state own postal/courier tracking system that crashes under load every 48 hours. We tried to help them but they are fine just rebooting (manually!). Fine, that happens too.

Re: We’re approaching the limits of computer power – we need new programmers

#244

Post by pdimitar »

luord wrote: post #193

There were several good points both for and against the article in this comment section. I was pleasantly surprised, usually the threads caused by posts like this turn into "static typing vs dynamic typing" or "functional vs object oriented" flamewars. As for my own opinion: yes, optimization is key, but we gotta remember not to make it premature. Take advantage of the fast hardware to actually create something; once…

Literally every experienced programmer would like to do this. But when you get to that last stage the shot-callers are like "nah, it's fine" and you never get to the optimisation.

I've seen many products die simply because customers get frustrated with laggy or buggy experience and leave.

By the time the businessmen wake up, it's usually too late.

Re: We’re approaching the limits of computer power – we need new programmers

#245

Post by pdimitar »

missblit wrote: post #41
tomrod wrote: post #12

This is a topic that really interests me, but I couldn't read the article -- either a paywall, ad-wall, or some other reader-hostile blocker incongruent with the foundation of the Internet prevents usability. Ah well. I'll join the conversation regardless. For all the programmers out there -- _how do we do this?_. I came into programming through Matlab and Python in Economics and Data Science. I don't have formal tra…

The days of everything being hand-optimized assembly are behind us. It still has it's niche, but for anything outside hot inner loops or extremely frequently called functions (like malloc), straightforward C++ will be just as fast. Meaning there's no point in optimizing an expensive function if 99% of your program's memory and run time is spent in a different function. This means the absolute most important skill to…

> Meaning there's no point in optimizing an expensive function if 99% of your program's memory and run time is spent in a different function.

Maybe there's no business point in optimizing those. But I feel this line of thinking got us into the current mess to begin with. Everybody is either like "we can't afford to optimize" (blatant lie at least 80% of the time btw) or "nah, not my effing job".

Plus that philosophy only really works when your business is fighting for survival in its initial stages. After you stabilize a little and have some runway you absolutely definitely should invest in technical excellence because it also lends itself pretty well to preventing laggy and/or buggy user experience (and those can bleed your subscriber numbers).

Re: We’re approaching the limits of computer power – we need new programmers

#246

Post by pdimitar »

at_a_remove wrote: post #30
tomrod wrote: post #12

This is a topic that really interests me, but I couldn't read the article -- either a paywall, ad-wall, or some other reader-hostile blocker incongruent with the foundation of the Internet prevents usability. Ah well. I'll join the conversation regardless. For all the programmers out there -- _how do we do this?_. I came into programming through Matlab and Python in Economics and Data Science. I don't have formal tra…

Honestly? I used to jabber on about this with regards to the still distant future of actual nanotechnology ... we need to find the guys who wrote videogames for arcades in the 1980s and press them for their secrets before so many brilliant tricks will be lost to time. They did so much with so little! My guess is that we will slowly approach this wall and spend a lot of time trying for incremental gains, trying to avo…

I have this theory that a lot of corporations know this but they don't want to be the pioneers who volunteer their money and man-hours, only for their competitors to then reap the fruits of their labour for free.

I can't prove it but I intuitively feel there's a lot of spite out there. Many people are unhappy with the status quo but are also unhappy with the idea to sacrifice their resources for everybody else -- and they will likely not only be non-grateful; they might try and pull an Oracle or Amazon and sue the creators over the rights of their own labour.

Things really do seem stuck in this giant tug of war game lately.

Re: We’re approaching the limits of computer power – we need new programmers

#247

Post by pdimitar »

bluestreak wrote: post #206
tomrod wrote: post #12

This is a topic that really interests me, but I couldn't read the article -- either a paywall, ad-wall, or some other reader-hostile blocker incongruent with the foundation of the Internet prevents usability. Ah well. I'll join the conversation regardless. For all the programmers out there -- _how do we do this?_. I came into programming through Matlab and Python in Economics and Data Science. I don't have formal tra…

The path forward is to be economical with hardware resources. I always try to imagine a physical character performing a task that i'm trying to code. How far does imaginary character needs to travel, how many trips do they need to make. Is everything they do is absolutely necessary. If they delegate work, is their sub-contractor efficient? There isn't a single place to learn how to be efficient, it is better to start…

Not sure what you are getting at here.

> I always try to imagine a physical character performing a task that i'm trying to code. How far does imaginary character needs to travel, how many trips do they need to make.

Dude, that's why we have optimising compilers. Functional programming is demonstrably less efficient on our imperative/mutable CPU architectures but a lot of compilers are extremely smart and turn those higher-level FP languages into very decently efficient machine code that's not much worse than what GCC for C++ produces. Especially compilers like those of OCaml and Haskell are famous for this. They shrunk the gap between FP and the languages that are closer to the metal. They shrunk that gap by a lot and even if they are not 100% there, I'm seeing results that make me think they are 75% - 85% there.

We need languages that rid us of endlessly thinking about minutiae and we must start assembling bigger LEGO constructs in our heads if we want anything in IT to actually get unstuck and start progressing again. (Of course, this paragraph doesn't apply to kernel and driver authors. They have to micro-optimise everything they can on the lowest level they can. That's a given.)

> Scary number of people I've met do not even attempt to learn how a library functions they use actually work.

I couldn't care less. How a library function works is an implementation detail. I only need to know what does it do. That's why it's a 3rd party library after all. The creator might notice a hot path during stress tests and optimise that implementation detail into some entirely another algorithm and/or data structure. And boom, your code that optimises for an implementation quirk you weren't supposed to look at in the first place, is now slow or even buggy.

Re: We’re approaching the limits of computer power – we need new programmers

#248

Post by pdimitar »

bordercases wrote: post #19
tomrod wrote: post #12

This is a topic that really interests me, but I couldn't read the article -- either a paywall, ad-wall, or some other reader-hostile blocker incongruent with the foundation of the Internet prevents usability. Ah well. I'll join the conversation regardless. For all the programmers out there -- _how do we do this?_. I came into programming through Matlab and Python in Economics and Data Science. I don't have formal tra…

The fundamental tradeoff is between control and abstraction. Better control typically means going closer to machine/operational semantics, better abstraction typically means going to denotational semantics. Compilers are what mediate between these two domains, but tend to become more bloated as they have to accommodate both more diverse hardware and more numerous languages. This helps the working programmer ignore th…

> Under these discretions languages like OCaml and Rust seem to make the cut.

I absolutely agree! I am gradually learning both and I am just getting so angry that I didn't know about OCaml like 10-15 years ago. :( I was just so damn busy surviving and being depressed for a heckton of [dumb] reasons for 15 years. And then I woke up.

Now I am just a regular web CRUD idiot dev who, even though he was very clever and inventive and creative in the past, nowadays seems to get pissed at small details like configuring web frameworks (even though I am still much better than a lot of others, I dare say -- proven with practice... or so I like to think). And now I have to work against the negative inertia of my last 15 years and learn the truly valuable concepts and how they are implemented in those two extremely efficient, if a bit quirky in syntax, languages.

But it seems every time somebody says "let's just keep these N languages and kill everything else", no discussion is possible... And I feel we really must only keep a few languages/runtimes around and scrap everything else.

Re: We’re approaching the limits of computer power – we need new programmers

#249

Post by pdimitar »

kraig911 wrote: post #90

How many years was it where bridges were made only out of wood? I feel engineers before us had the foresight to see the possibilities but lacked the tools and understanding. I fear only when people realized the economy needed to support large mammals crossing a bridge at one time did they really engineer bridges to support that weight. I think the same metaphor could be said for computing.

Yeah, most people only get creative only when they absolutely must, and not one minute earlier.

Re: We’re approaching the limits of computer power – we need new programmers

#250

Post by lern_too_spel »

saagarjha wrote: post #240
lern_too_spel wrote: post #187

Earlier quoted context omitted.

A double array takes O(1) more space in Java vs. C, so running out of memory wouldn't be a problem.

Unfortunately big-O algorithmic analysis doesn't translate cleanly to the real world due to constants and other inefficiencies.

It does in this case. Champtar says it's something about the size of the matrix that makes it run out of space. The overhead of the double array is already paid for.
Post reply on HN