Live data from Hacker News

For algorithms, a little memory outweighs a lot of time

quantamagazine.org

131–140 of 144 posts

Re: For algorithms, a little memory outweighs a lot of time

#131

Earlier quoted context omitted.

If your program runs in O(n) time, it cannot use more than O(n) memory (upper bound on memory usage. If your program uses O(n) memory, it must run at least in O(n) time (lower bound on time).

This is pretty easy to refute: > If your program runs in O(n) time, it cannot use more than O(n) memory (upper bound on memory usage.[sic] This is clearly refuted by all software running today. Programs (especially games) clearly use more memory than there are instructions in the program. > If your program uses O(n) memory, it must run at least in O(n) time (lower bound on time). Memory bombs use an incredible amount…

>Programs (especially games) clearly use more memory than there are instructions in the program.

How can you access a piece of memory without issuing an instruction to the CPU? Also, "clearly" is not an argument.

>Memory bombs use an incredible amount of memory and do it incredibly quickly.

How can you access a piece of memory without issuing an instruction to the CPU? Also "incredibly quickly" is not an argument. Also also, O(n) is incredibly quick.

Re: For algorithms, a little memory outweighs a lot of time

#132
post #82

It's kind of insulting to the reader that they explain P complexity class without using the word polynomial ("all problems that can be solved in a reasonable amount of time")

I think this is actually a pretty reasonable description but I also have read Quantum Computing Since Democritus.

Re: For algorithms, a little memory outweighs a lot of time

#133

Earlier quoted context omitted.

https://conwaylife.com/wiki/HashLife is an algorithm for doing basically this in Conway’s Game of Life, which is Turing complete. I remember my first impression being complete confusion: here’s a tick-by-tick simulation too varied and complex to encapsulate in a formula, and you’re telling me I can just skip way into its future?

If I read that page correctly, it does this for areas with empty space between them? Makes sense. Say you have a pattern (surrounded by empty space) that 'flickers': A-B-A-B-A... etc. Then as long as nothing intrudes , nth generation is the same pattern as in n+1000,000th generation. Similar for patterns that do a 3-cycle, 4-cycle etc. All you'd need is a) a way to detect repeating patterns, and b) do some kind of co…

I don’t fully understand the algorithm, but no, to my understanding it’s much more general than that. In each tick a cell’s state is solely determined by its immediate neighbors, which means the simulation has a “speed of light” of 1 cell/second: to look N ticks into the future, you need only consider cells within N cells of the area you’re computing, no matter what’s outside that. So, for example, if you want to skip a 10x10 area 100 ticks into the future, you consider a 210x210 area centered on your 10x10, compute it once, then in the future use that 210x210 area as a lookup key for the 10x10 100 ticks into the future. I think HashLife is also somehow doing this on multiple scales at once, and some other tricks.

Re: For algorithms, a little memory outweighs a lot of time

#134

Earlier quoted context omitted.

This is pretty easy to refute: > If your program runs in O(n) time, it cannot use more than O(n) memory (upper bound on memory usage.[sic] This is clearly refuted by all software running today. Programs (especially games) clearly use more memory than there are instructions in the program. > If your program uses O(n) memory, it must run at least in O(n) time (lower bound on time). Memory bombs use an incredible amount…

>Programs (especially games) clearly use more memory than there are instructions in the program. How can you access a piece of memory without issuing an instruction to the CPU? Also, "clearly" is not an argument. >Memory bombs use an incredible amount of memory and do it incredibly quickly. How can you access a piece of memory without issuing an instruction to the CPU? Also "incredibly quickly" is not an argument. Al…

> Also, "clearly" is not an argument.

As in your assertion is literally self-evidently false. It is on you to provide a burden of proof here; especially since there are instructions that can load more than a single bit of memory.

> How can you access a piece of memory without issuing an instruction to the CPU?

Let me rather ask you this: where do the instructions exist that are running? That is right: in memory. However, just because instructions exist in memory doesn’t mean they’re accessed. There is not a relationship between the number of instructions and the amount of memory accessed/used.

Re: For algorithms, a little memory outweighs a lot of time

#135
post #73

It's unfortunate that Quanta links are so popular, when they include so much pseudo-poetic fluff around the mathematics. Below there's an entire thread to dismiss a misconception introduced by the quanta article. "I think it is very intuitive that more space beats the pants off of more time." (poster is absolutely right) The The article say "Until now, the only known algorithms for accomplishing certain tasks require…

I'm the author of this article. If you ask a complexity theorist, they will tell you that they did in fact have a general intuition that certain problems require space close to to linear in time to solve (see e.g., Ryan's comment #22 on Scott Aaronson's blog post about the result: https://scottaaronson.blog/?p=8680, and the comments after that). The most intuitive way to see this is in a circuit/DAG picture, where the goal is to get from the input nodes of the graph to the output nodes. Some graphs are very "wide": cut the graph at some intermediate point, and there will be a lot of edges crossing the cut, each of which represents some information from an earlier stage in the computation that you'll need to remember to get to the output. Ryan's first result is a general-purpose method for doing any computation, even ones whose graph structure looks like this, in asymptotically far less space. That is precisely what makes the result so surprising!

My article was quite explicit in multiple places that the universal/comprehensive character of the result was that counterintuitive part:

- In the first paragraph: "memory was more powerful than computer scientists believed: A small amount would be as helpful as a lot of time in all conceivable computations."

- Further down in the introduction, in the passage you quoted: "Until now, the only known algorithms for accomplishing certain tasks required an amount of space roughly proportional to their runtime, and researchers had long assumed there’s no way to do better. Williams’ proof established a mathematical procedure for transforming any algorithm — no matter what it does — into a form that uses much less space.

- In the third section, I explicitly state that researchers do believe space is more powerful than time in the specific sense that you're criticizing my article for misrepresenting: "But complexity theorists suspect that PSPACE is a much larger class, containing many problems that aren’t in P. In other words, they believe that space is a far more powerful computational resource than time. This belief stems from the fact that algorithms can use the same small chunk of memory over and over, while time isn’t as forgiving — once it passes, you can’t get it back."

- In the fourth section, I explain why researchers didn't think the HPV75 result could be improved further, despite their intuition that space is more powerful than time in the above sense: "While many problems can be solved with much less space than time, some intuitively seemed like they’d need nearly as much space as time."

TCS (and complexity theory specifically) are complicated subjects. I spend a lot of time interviewing researchers and thinking about how to distill the results of my reporting into a form that is accessible to readers with widely varying levels of familiarity with the subject matter. You are of course well within your rights to critique my stylistic choices, the narrative aspects of the story, and the order in which I presented information, but I will push back against the claim that my article is spreading misinformation about complexity theory. You're referring to a misconception that arises, by your own admission, when you don't read carefully. If it's the headline you object to, you could lodge a similar complaint against the complexity theorist Lance Fortnow: https://blog.computationalcomplexity.org/2025/02/you-need-mu....

Re: For algorithms, a little memory outweighs a lot of time

#136

Earlier quoted context omitted.

>Programs (especially games) clearly use more memory than there are instructions in the program. How can you access a piece of memory without issuing an instruction to the CPU? Also, "clearly" is not an argument. >Memory bombs use an incredible amount of memory and do it incredibly quickly. How can you access a piece of memory without issuing an instruction to the CPU? Also "incredibly quickly" is not an argument. Al…

> Also, "clearly" is not an argument. As in your assertion is literally self-evidently false. It is on you to provide a burden of proof here; especially since there are instructions that can load more than a single bit of memory. > How can you access a piece of memory without issuing an instruction to the CPU? Let me rather ask you this: where do the instructions exist that are running? That is right: in memory. Howe…

This is about time and memory complexity, which is a formal field of computer science. Your replies are about your own vague understanding of computing, which is not the topic here.

Re: For algorithms, a little memory outweighs a lot of time

#137
post #75

Earlier quoted context omitted.

This can work both ways. If the program needs more memory than the computer has, it can't run until you buy more. But if it takes twice as long, at least it runs at all.

Modern computers have so much memory it feels like it doesn't matter. Spending that memory on arrays for algorithms or things like a Garbage Collector just make sense. And, extra memory is worthless. You WANT the summation of all your programs to use all your memory. The processor, on the other hand, can context switch and do everything in it's power to make sure it stays busy. The CPU is like an engine and memory is…

Only if running one such memory-hungry program at a time, which usually cannot be afforded. Multi-program workloads are much more common and the strategy of using as much ram as possible can't work in that environment.

Re: For algorithms, a little memory outweighs a lot of time

#138
post #58

Earlier quoted context omitted.

This might be completely naive but can a reversible time component be incorporated into distinguishing two hash calculations? Meaning when unpacked/extrapolated it is a unique signifier but when decomposed it folds back into the standard calculation - is this feasible?

hashes by definition are not reversible. you could store a timestamp together with a hash, and/or you could include a timestamp in the digested content, but the timestamp can’t be part of the hash.

Oh, of course, the timestamp could instead be metadata!

Re: For algorithms, a little memory outweighs a lot of time

#139

Earlier quoted context omitted.

> Also, "clearly" is not an argument. As in your assertion is literally self-evidently false. It is on you to provide a burden of proof here; especially since there are instructions that can load more than a single bit of memory. > How can you access a piece of memory without issuing an instruction to the CPU? Let me rather ask you this: where do the instructions exist that are running? That is right: in memory. Howe…

This is about time and memory complexity, which is a formal field of computer science. Your replies are about your own vague understanding of computing, which is not the topic here.

Yes, but you are asserting the relationship is directly connected -- which is clearly not true. You said that it is O(n) memory and O(n) time, both using n. That means a program containing x bytes can only run for x seconds. This is clearly not true.

Re: For algorithms, a little memory outweighs a lot of time

#140

Earlier quoted context omitted.

This is about time and memory complexity, which is a formal field of computer science. Your replies are about your own vague understanding of computing, which is not the topic here.

Yes, but you are asserting the relationship is directly connected -- which is clearly not true. You said that it is O(n) memory and O(n) time, both using n. That means a program containing x bytes can only run for x seconds. This is clearly not true.

>That means a program containing x bytes can only run for x seconds.

That is not what it means. Again, if you are not familiar with the notation then all you are doing is slapping your personal ideas about computing to some symbols

Post reply on HN