Live data from Hacker News

For algorithms, a little memory outweighs a lot of time

quantamagazine.org

21–30 of 144 posts

Re: For algorithms, a little memory outweighs a lot of time

#21

Lookup tables with precalculated things for the win! In fact I don’t think we would need processors anymore if we were centrally storing all of the operations ever done in our processors. Now fast retrieval is another problem for another thread.

Oh, that's not a problem. Just cache the retrieval lookups too.

Re: For algorithms, a little memory outweighs a lot of time

#23
post #4

I think it is very intuitive that more space beats the pants off of more time. In time O(n) you can use O(n) cells on a tape, but there are O(2^n) possible configurations of symbols on a tape of length n (for an alphabet with 2 symbols), so you can do so much more with n space than with n time.

Also, the O(1) random memory access assumption makes it easy to take memory for granted. Really it's something like O(n^(1/3)) when you're scaling the computer to the size of the problem, and you can see this in practice in datacenters.

I forget the name of the O(1) access model. Not UMA, something else.

Re: For algorithms, a little memory outweighs a lot of time

#24
I am confused. If a single-tape turing machine receives a digit N in binary, and is supposed to write N ones on the tape, on the right side of the digit N, it performs N steps.

If you expect N ones at the output, how can this machine be simulated in the space smaller than N?

This machine must decrement the digit N at the beginning of the tape, and move to the end of the tape to write "1", so it runs in time O(N^2), not O(N)? (as it takes N "trips" to the end of the tape, and each "trip" takes 1, 2, 3 .. N steps)

Since turing machines can not jump to any place on a tape in constant time (like computers can), does it have any impact on real computers?

Re: For algorithms, a little memory outweighs a lot of time

#25
post #4

I think it is very intuitive that more space beats the pants off of more time. In time O(n) you can use O(n) cells on a tape, but there are O(2^n) possible configurations of symbols on a tape of length n (for an alphabet with 2 symbols), so you can do so much more with n space than with n time.

Also, the O(1) random memory access assumption makes it easy to take memory for granted. Really it's something like O(n^(1/3)) when you're scaling the computer to the size of the problem, and you can see this in practice in datacenters. I forget the name of the O(1) access model. Not UMA, something else.

O(n^(1/2)) really, since data centers are 2 dimensional, not 3 dimensional.

(Quite aside from the practical "we build on the surface of the earth" consideration, heat dissipation considerations limit you to a 2 dimensional circuit in 3-space.)

Re: For algorithms, a little memory outweighs a lot of time

#26

I am confused. If a single-tape turing machine receives a digit N in binary, and is supposed to write N ones on the tape, on the right side of the digit N, it performs N steps. If you expect N ones at the output, how can this machine be simulated in the space smaller than N? This machine must decrement the digit N at the beginning of the tape, and move to the end of the tape to write "1", so it runs in time O(N^2), n…

Multitape Turing machines are far more powerful (in terms of how fast they can run, not computability) than single-tape machines.

But to answer your question: "space" here refers to working space, excluding the input and output.

Re: For algorithms, a little memory outweighs a lot of time

#27

Earlier quoted context omitted.

Intuitive yes, but since P != PSPACE is still unproven it's clearly hard to demonstrate.

There's not even a proof that P != EXPTIME haha EDIT: I am a dumbass and misremembered.

I thought there was some simple proof of this, but all I can think of is time hierarchy theorem.

Re: For algorithms, a little memory outweighs a lot of time

#28
post #4

I think it is very intuitive that more space beats the pants off of more time. In time O(n) you can use O(n) cells on a tape, but there are O(2^n) possible configurations of symbols on a tape of length n (for an alphabet with 2 symbols), so you can do so much more with n space than with n time.

Also, the O(1) random memory access assumption makes it easy to take memory for granted. Really it's something like O(n^(1/3)) when you're scaling the computer to the size of the problem, and you can see this in practice in datacenters. I forget the name of the O(1) access model. Not UMA, something else.

On the other hand, actual computers can work in parallel when you scale the hardware, something that the TM formulation doesn't cover. It can be interesting which algorithms work well with lots of computing power subject to data locality. (Brains being the classic example of this.)

Re: For algorithms, a little memory outweighs a lot of time

#29

Lookup tables with precalculated things for the win! In fact I don’t think we would need processors anymore if we were centrally storing all of the operations ever done in our processors. Now fast retrieval is another problem for another thread.

Oh, that's not a problem. Just cache the retrieval lookups too.

it's pointers all the way down

Re: For algorithms, a little memory outweighs a lot of time

#30

Earlier quoted context omitted.

Also, the O(1) random memory access assumption makes it easy to take memory for granted. Really it's something like O(n^(1/3)) when you're scaling the computer to the size of the problem, and you can see this in practice in datacenters. I forget the name of the O(1) access model. Not UMA, something else.

O(n^(1/2)) really, since data centers are 2 dimensional, not 3 dimensional. (Quite aside from the practical "we build on the surface of the earth" consideration, heat dissipation considerations limit you to a 2 dimensional circuit in 3-space.)

If you have rows of racks of machines, isn't that 3 dimensions? A machine can be on top of, behind, or next to another that it's directly connected to. And the components inside have their own non-uniform memory access.

Or if you're saying heat dissipation scales with surface area and is 2D, I don't know. Would think that water cooling makes it more about volume, but I'm not an expert on that.

Post reply on HN