Live data from Hacker News

Everything old is new again: memory optimization

nibblestew.blogspot.com

21–30 of 168 posts

Re: Everything old is new again: memory optimization

#21
I'm always confused as hell how little insight we have in memory consumption.

I look at memory profiles of rnomal apps and often think "what is burning that memory".

Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this.

For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it doing?

Alone for chrome to implement tab suspend took YEARS despite everyone being aware of the issue. And addons existed which were able to do this.

I bought more ram just for chrome...

Re: Everything old is new again: memory optimization

#23
post #13

Earlier quoted context omitted.

Tells you right away where this is coming from.

The critisism is valid. The problem is how you value this critism. I agree they are stealing it but I also see the benefit of it for society and for myself. Suckerberg downloaded terabytes of books for training, while people around me got sued to hell 20 years ago for downloading one mp3 file.

Anna's Archive. Aaron Swartz.

FB and so are CIA fronts and they can do anything they please. Until they hit against Disney and lobbying giants and if a CIA idiot tries to sue/bribe/blackmail them they can order Hollywood to rot their images into pieces with all the wars they promoted in Middle East and Latin America just to fill the wallets of CEO's. That among some social critique movie on FB about getting illegal user data all over the world to deny insurances and whatnot. And OFC with a clear mention of the Epstein case with related people, just in case the Americans forgot about it.

Then the US industry and military complex would collapse in months with brainwashed kids running away from the army. Not to mention to the Call of Duty franchise and the like. It would be the end of Boeing and several more, of course. To hell to profit driven wars for nothing.

Ah, yes, AIPAC lobbies and the like. Good luck taming right wing wackos hating the MAGA cult more than the 'woke' people themselves. These will be the first ones against you after sinking the US image for decades, even more than the illegal Iraq war with no WMD's and the Bush/Cheney mafia.

The outcome of this? proper and serious engineering a la Airbus. Instant profit-driven MBA and war sickos being kicked out from the spot. OFC the AI snakeoil sellers except for the classical AI/NN against concrete cases (image detection and the like), these will survive fine, even better because these kind of jobs are highly specific and they are not statistical text parrots. They can provide granted results unlike LLM's prone to degrade because the human based content feeding needs to be continuous, while for tumour detection a big enough sample can cover a 99% of the cases.

R&D on electric vehicles/energy and nuclear power like nowhere else. And, for sure, the EV equivalent of a Ford T for Americans. A cheap and reliable one, good enough for the common Joe/Mary without being a luxury item. A new Golden Age would rise, for sure. But the oil mafia will try to fight them like crazy.

Re: Everything old is new again: memory optimization

#24
post #13

Earlier quoted context omitted.

Tells you right away where this is coming from.

The critisism is valid. The problem is how you value this critism. I agree they are stealing it but I also see the benefit of it for society and for myself. Suckerberg downloaded terabytes of books for training, while people around me got sued to hell 20 years ago for downloading one mp3 file.

they got sued for uploading actually

and Zuck isn’t sued for downloading either, he is sued for reproduction by the AI not being derivative enough, but so far all branches of government support that

Re: Everything old is new again: memory optimization

#26
post #4

"copyright infringement factories"

I don't know how anyone can call the most amazing invention in computer science of the last 20 years "copyright infringement factories". We went from the ST:NG ship computer being futuristic tech to "we kinda have this now". Its like calling cars "air pollution factories", as if that was their only purpose and use.

A fundamentally anti-civilisational mindset.

Re: Everything old is new again: memory optimization

#28
post #4

"copyright infringement factories"

I don't know how anyone can call the most amazing invention in computer science of the last 20 years "copyright infringement factories". We went from the ST:NG ship computer being futuristic tech to "we kinda have this now". Its like calling cars "air pollution factories", as if that was their only purpose and use. A fundamentally anti-civilisational mindset.

I'm sorry, but you're acting obtuse if you pretend you don't know why they're being called that.

Re: Everything old is new again: memory optimization

#29
post #6

Well, we can use memoryview for the dict generation avoiding creation of string objects until the time for the output: import re, operator def count_words(filename): with open(filename, 'rb') as fp: data= memoryview(fp.read()) word_counts= {} for match in re.finditer(br'\S+', data): word= data[match.start(): match.end()] try: word_counts[word]+= 1 except KeyError: word_counts[word]= 1 word_counts= sorted(word_counts.…

This doesn't do the same thing though, since it's not Unicode aware.

    >>> 'x\u2009   a'.split()
    ['x', 'a']
    # incorrect; in bytes mode, `\S` doesn't know about unicode whitespace
    >>> list(re.finditer(br'\S+', 'x\u2009   a'.encode()))
    [, ]
    # correct, in unicode mode
    >>> list(re.finditer(r'\S+', 'x\u2009   a'))
    [, ]

Re: Everything old is new again: memory optimization

#30

I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it…

200Mb for Sublime does not seem so bad when compared to Postman using 4Gb on my machine...
Post reply on HN