Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adding 16 kb page size to Android

android-developers.googleblog.com

Post reply on HN
Print view View original

111–120 of 173 posts

Re: Adding 16 kb page size to Android

#111

Post by afr0ck »

ignoramous wrote: post #105
lanigone wrote: post #37

Earlier quoted context omitted.

you can also do 2M and 1G huge pages on x86, it gets kind of silly fast.

What? Any pointers on how 1G speeds things up? I'd have taken a bigger page size to wreak havoc on process scheduling and filesystem.

Because of virtual address translation [1] speed up. When a memory access is made by a program, the CPU must first translate the virtual address to a physical address, by walking a hierarchical data structure called a page table [2]. Walking the page tables is slow, thus CPUs implement a small on-CPU cache of virtual-to-physical translations called a TLB [1]. The TLB has a limited number of entries for each page size. With 4 KiB pages, the contention on this cache is very high, especially if the workload has a very large workingset size, therefore causing frequent cache evictions and slow walk of the page tables. With 2 MiB or 1 GiB pages, there is less contention and more workingset size is covered by the TLB. For example, a TLB with 1024 entries can cover a maximum of 4 MiB of workingset memory. With 2 MiB pages, it can cover up to 2 GiB of workingset memory. Often, the CPU has different number of entries for each page size.

However, it is known that larger page sizes have higher internal fragmentation and thus lead to memory wastage. It's a trade off. But generally speaking, for modern systems, the overhead of managing memory in 4 KiB is very high and we are at a point where switching to 16/64 KiB is almost always a win. 2 MiB is still a bit of a stretch, though, but transparent 2 MiB pages for heap memory is enabled by default on most major Linux distributions, aka THP [2]

Source: my PhD is on memory management and address translation on large memory systems, having worked both on hardware architecture of address translation and TLBs as well as the Linux kernel. I'm happy to talk about this all day!

[1] https://blogs.vmware.com/vsphere/2020/03/how-is-virtual-memo... [2] https://docs.kernel.org/admin-guide/mm/transhuge.html

Re: Adding 16 kb page size to Android

#112

Post by ignoramous »

afr0ck wrote: post #111
ignoramous wrote: post #105

Earlier quoted context omitted.

What? Any pointers on how 1G speeds things up? I'd have taken a bigger page size to wreak havoc on process scheduling and filesystem.

Because of virtual address translation [1] speed up. When a memory access is made by a program, the CPU must first translate the virtual address to a physical address, by walking a hierarchical data structure called a page table [2]. Walking the page tables is slow, thus CPUs implement a small on-CPU cache of virtual-to-physical translations called a TLB [1]. The TLB has a limited number of entries for each page size…

Thanks!

> I'm happy to talk about this all day!

With noobs, too? ;)

> Often, the CPU has different number of entries for each page size.

- Does it mean userspace is free to allocate up to a maximum of 1G? I took pages to have a fixed size.

- Or, you mean CPUs reserve TLB sizes depending on the requested page size?

> With 2 MiB or 1 GiB pages, there is less contention and more workingset size is covered by the TLB

- Would memory allocators / GCs need to be changed to deal with blocks of 1G? Would you say, the current ones found in popular runtimes/implementations are adept at doing so?

- Does it not adversely affect databases accustomed to smaller page sizes now finding themselves paging in 1G at once?

> my PhD is on memory management and address translation on large memory systems

If the dissertation is public, please do link it, if you're comfortable doing so.

Re: Adding 16 kb page size to Android

#113

Post by fensgrim »

ignoramous wrote: post #107
saagarjha wrote: post #34

Earlier quoted context omitted.

Android apps can call into native code via JNI, which the platform supports.

Wonder if Android apps can also be fully native (C++)?

It is possible to have a project set up with a manifest which contains only a single activity with android.app.NativeActivity pointing to a .so, and zero lines of java/kotlin/flutter/whatever else - though your app initialization will go through usual hoops of spawning a java-based instance.

Minimal example would be https://github.com/android/ndk-samples/blob/master/native-ac..., though there are well established Qt based apps as well

Re: Adding 16 kb page size to Android

#114

Post by Veserv »

sweetjuly wrote: post #102
Veserv wrote: post #59

Earlier quoted context omitted.

Having both 4KB and 16KB simultaneously is either easy or hard depending on which hardware feature they are using for 16KB pages. If they are using the configurable granule size, then that is a system-wide hardware configuration option. You literally can not map at smaller granularity while that bit is set. You might be able to design a CPU that allows your idea of partial pages, but there be dragons. If they are not…

Hmm, I'm not sure that's quite right. ARMv8 supports per TTBR translation granules [1] and so you can have 4K and 16K user processes coexisting under an arbitrary page size kernel by just context switching TCR.TG0 at the same time as TTBR0. There is no such thing as a global granule size. [1]: https://arm.jonpalmisc.com/2023_09_sysreg/AArch64-tcr_el2#fi...

Well, if you want to run headfirst into the magical land of hardware errata, I guess you could go around creating heterogeneous, switched mappings.

I doubt the TCRs were ever intended to support rapid runtime switching or that the TLBs were ever intended to support heterogeneous entrys even with ASID tagging.

Re: Adding 16 kb page size to Android

#115

Post by evilduck »

Alifatisk wrote: post #84
evilduck wrote: post #13

Earlier quoted context omitted.

Question I wrote: > I encountered the typo "anablibg" in the sentence "I wonder how much help they had by asahi doing a lot of the kernel and ecosystem work anablibg 16k pages." What did they actually mean? GPT-4o and Sonnet 3.5 understood it perfectly. This isn't really a problem for the large models. For local small models: * Gemma2 9b did not get it and thought it meant "analyzing". * Codestral (22b) did not it ge…

Is there any task Gemma is better at compared to others?

Local LLM topics are a treadmill of “what’s best and what is preferred” changing basically weekly to monthly, it’s a rapidly evolving field, but right now I actually tend to gravitate to Gemma2 9b for coding assistance for Typescript work or general question and answer stuff. Its embedded knowledge and speed on the computers that I have (32GB M2 Max, 16GB M1 Air, 4080 gaming desktop) make for a good balance while also using the computer for other stuff, bigger models limit what else I can run simultaneously and are slower than my reading speed, smaller models have less utility and the speed increase is pointless if they’re dumb.

Re: Adding 16 kb page size to Android

#116

Post by monocasa »

ignoramous wrote: post #105
lanigone wrote: post #37

Earlier quoted context omitted.

you can also do 2M and 1G huge pages on x86, it gets kind of silly fast.

What? Any pointers on how 1G speeds things up? I'd have taken a bigger page size to wreak havoc on process scheduling and filesystem.

It's nice for type 1 hypervisors when carving up memory for guests. When page walks for guest virtual to host physical end up taking sixteen levels, a 1G page short circuits that in half to eight.

Re: Adding 16 kb page size to Android

#117

Post by versteegen »

afr0ck wrote: post #111
ignoramous wrote: post #105

Earlier quoted context omitted.

What? Any pointers on how 1G speeds things up? I'd have taken a bigger page size to wreak havoc on process scheduling and filesystem.

Because of virtual address translation [1] speed up. When a memory access is made by a program, the CPU must first translate the virtual address to a physical address, by walking a hierarchical data structure called a page table [2]. Walking the page tables is slow, thus CPUs implement a small on-CPU cache of virtual-to-physical translations called a TLB [1]. The TLB has a limited number of entries for each page size…

> I'm happy to talk about this all day!

Oh really :)

I'd like to ask how applications should change their memory allocation or usage patterns to maximise the benefit of THP. Do memory allocators (glibc mainly) need config tweaking to coalesce tiny mallocs into 2MB+ mmaps, will they just always do that automatically, do you need to use a custom pool allocator so you're doing large allocations, or are you never going to get the full benefit of huge tables without madvise/libhugetlbfs? And does this apply to Mac/Windows/*BSD at all?

[Edit: ouch, I see /sys/kernel/mm/transparent_hugepage/enabled is default set to 'madvise' on my system (Slackware) and as a result doing nearly nothing. But I saw it enabled in the past. Well that answers a lot of my questions: got to use madvise/libhugetlbfs.]

I read you also need to ensure ELF segments are properly aligned to get transparent huge pages for code/data.

Another question. From your link [2]:

> An application may mmap a large region but only touch 1 byte of it, in that case a 2M page might be allocated instead of a 4k page for no good.

Do the heuristics used by Linux THP (khugepaged) really allow completely ignoring whether pages have actually been page-faulted in or even initialised? Is a possibility unlikely to happen in practice?

Re: Adding 16 kb page size to Android

#118

Post by CalChris »

afr0ck wrote: post #111
ignoramous wrote: post #105

Earlier quoted context omitted.

What? Any pointers on how 1G speeds things up? I'd have taken a bigger page size to wreak havoc on process scheduling and filesystem.

Because of virtual address translation [1] speed up. When a memory access is made by a program, the CPU must first translate the virtual address to a physical address, by walking a hierarchical data structure called a page table [2]. Walking the page tables is slow, thus CPUs implement a small on-CPU cache of virtual-to-physical translations called a TLB [1]. The TLB has a limited number of entries for each page size…

Are huge pages expected to share code (X) and data (RW)?

Re: Adding 16 kb page size to Android

#119

Post by dataflow »

a1o wrote: post #7

> The very first 16 KB enabled Android system will be made available on select devices as a developer option. This is so you can use the developer option to test and fix > once an application is fixed to be page size agnostic, the same application binary can run on both 4 KB and 16 KB devices I am curious about this. When could an app NOT be agnostic to this? Like what an app must be doing to cause this to be noticea…

> When could an app NOT be agnostic to this

When the app has a custom memory allocator, the allocator might have hardcoded the page size for performance. Otherwise you have to load a static variable (knocks out a cache line you could've used for something else) and then do a multiplication (or bit shift, if you assume power of 2) by a runtime value instead of a shift by a constant, which can be slower.

No idea if Android apps are ever this performance sensitive, though.

Re: Adding 16 kb page size to Android

#120

Post by wren6991 »

monocasa wrote: post #24
IshKebab wrote: post #5

Earlier quoted context omitted.

Probably wouldn't be too hard to add a 16 kB page size extension. But I think the Svnapot extension is their solution to this problem. If you're not familiar it lets you mark a set of pages as being part of a contiguously mapped 64 kB region. No idea how the performance characteristics vary. It relieves TLB pressure, but you still have to create 16 4kB page table entries.

Svnapot is a poor solution to the problem. On one hand it means that that each page table entry takes up half a cache line for the 16KB case, and two whole cache lines in the 64KB case. This really cuts down on the page walker hardware's ability to effectively prefetch TLB entries, leading to basically the same issues as this classic discussion about why tree based page tables are generally more effective than hash b…

> This means that practically only the untranslated bits of the address can be used by the set selection portion of the cache lookup

It's true that this makes things difficult, but Arm have been shipping D caches with way size > page size for decades. The problem you get is that virtual synonyms of the same physical cache block can become incoherent with one another. You solve this by extending your coherence protocol to cover the potential synonyms of each index in the set (so for example with 16 kB/way and 4 kB pages, there are four potential indices for each physical cache block, and you need to maintain their coherence). It has some cost and the cost scales with the ratio of way size : page size, so it's still desirable to stay under the limit, e.g. by just increasing the number of cache ways.

Post reply on HN