The computers are fast, but you don't know it

Post by dataflow » Thu, Jun 16, 2022, 8:49 PM UTC

eterm wrote: post #180

dataflow wrote: post #175

Earlier quoted context omitted.

Nah that part I'm not worried about. The "cheating" is omitting the rest of the line. What you really needed was: var y = Enumerable.Range(1, 50).Where((x, i) => i % 4 == 0).Where(e => e % 3 == 0).Skip(1).Select(e => e + 4).ToArray(); Compare that against: y = [t[1] for t in enumerate(range(1, 50, 4)) if t[0] % 3 == 0][2:] It's almost twice as long, and doesn't exactly make up for it with readability either.

It's not "twice as long" in any syntactic sense, and readability is easily fixed: Enumerable.Range(1,50) .Where(e => e % 4 == 0 && e % 3 == 0) .Skip(1) .Select(e => e + 1) That's very understandable, it's clear what it does, and if your complaint is that dotnet prefers to name expressions like Skip rather than magic syntax, we can disagree on what make things readable and easy to maintain.

  var y = Enumerable.Range(1,50)
      .Where(e => e % 4 == 0 && e % 3 == 0)
      .Skip(1)
      .Select(e => e + 1)
      .ToArray();

  y = [t[1] for t in enumerate(range(1, 50, 4))
       if t[0] % 3 == 0][2:]

Post by geysersam » Thu, Jun 16, 2022, 8:50 PM UTC

willis936 wrote: post #115

lynguist wrote: post #98

Earlier quoted context omitted.

I agree except for the Python bit, which is factually wrong. Python allows you to program as if you’re a jazz pianist. You can improvise, iterate and have fun. And when you found a solution you just refactor it and use numba. Boom, it runs the same speed as a compiled language. I once wrote one little program that ran in 24 min without numba and ca. 8 seconds with numba.

Dozens of instances of a C GUI can launch in the time it takes to launch a hello world python program.

Post by geph2021 » Thu, Jun 16, 2022, 8:52 PM UTC

eterm wrote: post #145

geph2021 wrote: post #20

Earlier quoted context omitted.

The author says: "The function looks something like this:" And then shows some grouping and sorting functions using pandas. Then he says: "I replaced Pandas with simple python lists and implemented the algorithm manually to do the group-by and sort." I think the point of the first optimization is you can do the relatively expenseive group/sort operations without pandas, and improve performance. For the rest of the ar…

We never get a good sense of how much time was actually saved with that change not least because the original function calls "initialise weights" inside every loop, the new function does not. It would have been interesting to see what difference that alone made. The takeaway of the article, that computers are blindingly fast and we make them do unecessary work (and often sit around waiting on I/O) with most their tim…

   We never get a good sense of how much time was actually saved with that change not least because the original function calls "initialise weights" inside every loop, the new function does not. 

   [1] Please don't get me wrong. Pandas is pretty fast for a typical dataset but it's not the processing that slows down pandas in my case. It's the creation of Pandas objects itself which can be slow. If your service needs to respond in less than 500ms, then you will feel the effect of each line of Pandas code.

Post by kzrdude » Thu, Jun 16, 2022, 8:52 PM UTC

spatley wrote: post #63

porcoda wrote: post #11

Yup. We have gotten into the habit of leaving a lot of potential performance on the floor in the interest of productivity/accessibility. What always amazes me is when I have to work with a person who only speaks Python or only speaks JS and is completely unaware of the actual performance potential of a system. I think a lot of people just accept the performance they get as normal even if they are doing things that ta…

Agreed that switching to lower level languages give the potential of many orders of magnitude. But the thing that was most enlightening was that removing pandas made a 9900% increase in speed without even a change to language. 20 minutes down to 12 seconds is a very big deal, and I still don't have to remember how to manage pointers.

Post by theandrewbailey » Thu, Jun 16, 2022, 8:54 PM UTC

41b696ef1113 wrote: post #168

kossTKR wrote: post #132

Earlier quoted context omitted.

I hope one day latency in general will be "back to normal". I still remember how fast console based computing, an old gameboy or a 90's macintosh would be - click a button and stuff would show up instantly. There was a tactility present with computers that's gone today. Today everything feels sluggish - just writing this comment on my $3000 Macbook Pro and i can feel the latency, sometimes there's even small pauses.…

>Hopefully the focus on 100hz+ screens in tech Come again? I think anything beyond 60hz still qualifies as niche. Vendors are still selling 720p laptops.