Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adding 16 kb page size to Android

android-developers.googleblog.com

Post reply on HN
Print view View original

91–100 of 173 posts

Re: Adding 16 kb page size to Android

#91

Post by nox101 »

a1o wrote: post #7

> The very first 16 KB enabled Android system will be made available on select devices as a developer option. This is so you can use the developer option to test and fix > once an application is fixed to be page size agnostic, the same application binary can run on both 4 KB and 16 KB devices I am curious about this. When could an app NOT be agnostic to this? Like what an app must be doing to cause this to be noticea…

I don't know if this fits but I've seen code that allocated say 32 bytes from a function that allocated 1meg under the hood. Not knowing that's what was happening the app quickly ran out of memory. It arguably was not the app's fault. The API it was calling into was poorly designed and poorly named, such that the fact that you might need to know the block size to use the function was in no way indicated by the name of the function nor the names of any of its parameters.

Re: Adding 16 kb page size to Android

#92

Post by quotemstr »

IshKebab wrote: post #82
stingraycharles wrote: post #76

Earlier quoted context omitted.

I’m a total idiot, how exactly is page size a CPU issue rather than a kernel issue? Is it about memory channel protocols / communication? Disks have been slowly migrating away from the 4kb sector size, is this a same thing going on? That you need to actual drive to support it, because of internal structuring (i.e. how exactly the CPU aligns things in RAM), and on some super low level 4kb / 16kb being the smallest uni…

The CPU has hardware that does a page table walk automatically when you access an address for which the translation is not cached in the TLB. Otherwise virtual memory would be really slow. Since the CPU hardware itself is doing the page table walk it needs to understand page tables and page table entries etc. including how big pages are. Also you need to know how big pages are for the TLB itself. The value of 4kB its…

As an aside, it's shame that hardware page table walking won out over software filled TLBs, as some older computers had. I wonder what clever and wonderful hacks we might have been able to invent had we not needed to give the CPU a raw pointer to a data structure the layout of which is fixed forever.

Re: Adding 16 kb page size to Android

#93

Post by bewaretheirs »

arghwhat wrote: post #21
jsheard wrote: post #19

Earlier quoted context omitted.

Makes me wonder how much performance Windows is leaving on the table with its primitive support for large pages. It does support them, but it doesn't coalesce pages transparently like Linux does, and explicitly allocating them requires special permissions and is very likely to fail due to fragmentation if the system has been running for a while. In practice it's scarcely used outside of server software which immediat…

Quite a bit, but 2M is an annoying size and the transparent handling is suboptimal. Without userspace cooperating, the kernel might end up having to split the pages at random due to an unfortunate unaligned munmap/madvise from an application not realizing it was being served 2M pages. Having Intel/AMD add 16-128K page support, or making it common for userspace to explicitly ask for 2M pages for their heap arenas is l…

Intel's menu of page sizes is an artifact of its page table structure.

On x86 in 64-bit mode, page table entries are 64 bits each; the lowest level in the hierarchy (L1) is a 4K page containing 512 64-bit of PTEs which in total map 2M of memory, which is not coincidentally the large page size.

The L1 page table pages are themselves found via a PTE in a L2 page table; one L2 page table page maps 512*2M = 1G of virtual address space, which is again, not coincidentally, the huge page size.

Large pages are mapped by a L2 PTE (sometimes called a PDE, "page directory entry") with a particular bit set indicating that the PTE points at the large page rather than a PTE page. The hardware page table walker just stops at that point.

And huge pages are similarly mapped by an L3 PTE with a bit set indicating that the L3 PTE is a huge page.

Shoehorning an intermediate size would complicate page table updates or walks or probably both.

Note that an OS can, of its own accord independent of hardware maintain allocations as a coarser granularity and sometimes get some savings out of this. For one historic example, the VAX had a tiny 512-byte page size; IIRC, BSD unix pretended it had a 1K page size and always updated PTEs in pairs.

Re: Adding 16 kb page size to Android

#94

Post by pwg »

stingraycharles wrote: post #76
Veserv wrote: post #59

Earlier quoted context omitted.

Having both 4KB and 16KB simultaneously is either easy or hard depending on which hardware feature they are using for 16KB pages. If they are using the configurable granule size, then that is a system-wide hardware configuration option. You literally can not map at smaller granularity while that bit is set. You might be able to design a CPU that allows your idea of partial pages, but there be dragons. If they are not…

I’m a total idiot, how exactly is page size a CPU issue rather than a kernel issue? Is it about memory channel protocols / communication? Disks have been slowly migrating away from the 4kb sector size, is this a same thing going on? That you need to actual drive to support it, because of internal structuring (i.e. how exactly the CPU aligns things in RAM), and on some super low level 4kb / 16kb being the smallest uni…

> I’m a total idiot, how exactly is page size a CPU issue rather than a kernel issue?

Because the size of a page is a hardware defined size for Intel and ARM CPU's (well, more modern Intel and ARM CPU's give the OS a choice of sizes from a small set of options).

It (page size) is baked into the CPU hardware.

> And does that then mean that there’s less overhead in all kinds of memory (pre)fetchers in the CPU, because more can be achieved in less clock cycles?

For the same size TLB (Translation Look-aside Buffer -- the CPU hardware that stores the "referencing info" for the currently active set of pages being used by the code running on the CPU) a larger page size allows more total memory to be accessible before taking a page fault and having to replace one or more of the entries in the TLB. So yes, it means less overhead, because CPU cycles are not used up in replacing as many TLB entries as often.

Re: Adding 16 kb page size to Android

#95

Post by IshKebab »

quotemstr wrote: post #92
IshKebab wrote: post #82

Earlier quoted context omitted.

The CPU has hardware that does a page table walk automatically when you access an address for which the translation is not cached in the TLB. Otherwise virtual memory would be really slow. Since the CPU hardware itself is doing the page table walk it needs to understand page tables and page table entries etc. including how big pages are. Also you need to know how big pages are for the TLB itself. The value of 4kB its…

As an aside, it's shame that hardware page table walking won out over software filled TLBs, as some older computers had. I wonder what clever and wonderful hacks we might have been able to invent had we not needed to give the CPU a raw pointer to a data structure the layout of which is fixed forever.

Yeah maybe, though in practice I think it would be just too slow.

Re: Adding 16 kb page size to Android

#96

Post by reissbaker »

phh wrote: post #54
devit wrote: post #41

Seems pretty dubious to do this without adding support for having both 4KB and 16KB processes at once to the Linux kernel, since it means all old binaries break and emulators which emulate normal systems with 4KB pages (Wine, console emulators, etc.) might dramatically lose performance if they need to emulate the MMU. Hopefully they don't actually ship a 16KB default before supporting 4KB pages as well in the same ke…

Google/Android doesn't care much about backward compatibility and broke programs released on Pixel 3 in Pixel 7. (the interdiction of 32bit-only apps is 2019 on Play Store, Pixel 7 is first 64bits only device, while Google still released 32bits only device in 2023...). They quite regularly break apps in new Android versions (despite their infrastructure to handle backward compatibility), and app developers are used t…

Generally I've found Google to care much more about not breaking old apps compared to Apple, which often expects developers to rebuild apps for OS updates or else the apps stop working entirely (or buy entirely new machines to get OS updates at all, e.g. the Intel/Apple Silicon transition). Google isn't on the level of Windows "we will watch for specific binaries and re-introduce bugs in the kernel specifically for those binaries that they depend on" in terms of backwards compatibility, but I wouldn't go so far as to say they don't care. I'm not sure whether that's better or worse: there's definitely merit to Apple's approach, since it keeps them able to iterate quickly on UX and performance by dropping support for the old stuff.

Re: Adding 16 kb page size to Android

#97

Post by hinkley »

andai wrote: post #22
jsheard wrote: post #19

Earlier quoted context omitted.

Makes me wonder how much performance Windows is leaving on the table with its primitive support for large pages. It does support them, but it doesn't coalesce pages transparently like Linux does, and explicitly allocating them requires special permissions and is very likely to fail due to fragmentation if the system has been running for a while. In practice it's scarcely used outside of server software which immediat…

A lot of low level stuff is a lot slower on Windows, let alone the GUI. There's also entire blogs cataloging an abundance of pathological performance issues. The one I notice the most is the filesystem. Running Linux in VirtualBox, I got 7x the host speed for many small file operations. (On top of that Explorer itself has its own random lag.) I think a better question is how much performance are they leaving on the t…

> The one I notice the most is the filesystem. Running Linux in VirtualBox, I got 7x the host speed for many small file operations. (On top of that Explorer itself has its own random lag.)

That’s a very old problem. In early days of subversion, the metadata for every directory existed in the directory. The rationale was that you could check out just a directory in svn. It was disastrously slow on Windows and the subversion maintainers had no answer for it, except insulting ones like “turn off virus scanning”. Telling a windows user to turn off virus scanning is equivalent to telling someone to play freeze tag in traffic. You might as well just tell them, “go fuck yourself with a rusty chainsaw”

Someone reorganized the data so it all happened at the root directory and the CLI just searched upward until it found the single metadata file. If memory serves that made large checkouts and updates about 2-3 times faster on Linux and 20x faster on windows.

Re: Adding 16 kb page size to Android

#98

Post by lxgr »

Now I wonder: Does increased page size have any negative impacts on I/O performance or flash lifetime, e.g. for writebacks of dirty pages of memory-mapped files where only a small part was changed?

Or is the write granularity of modern managed flash devices (such as eMMCs as used in Android smartphones) much larger than either 4 or 16 kB anyway?

Re: Adding 16 kb page size to Android

#99

Post by ShroudedNight »

lanigone wrote: post #37
o11c wrote: post #23

Earlier quoted context omitted.

The fundamental problem is that system headers don't provide enough information. In particular, many programs need both "min runtime page size" and "max runtime page size" (and by this I mean non-huge pages). If you call `mmap` without constraint, you need to assume the result will be aligned to at least "min runtime page size". In practice it is probably safe to assume 4K for this for "normal" systems, but I've seen…

you can also do 2M and 1G huge pages on x86, it gets kind of silly fast.

1G huge pages had (have?) performance benefits on managed runtimes for certain scenarios (Both the JIT code cache and the GC space saw uplift on the SpecJ benchmarks if I recall correctly)

If using relatively large quantities of memory 2M should enable much higher TLB hit rates assuming the CPU doesn't do something silly like only having 4 slots for pages larger than 4k ¬.¬

Re: Adding 16 kb page size to Android

#100

Post by 95014_refugee »

scottlamb wrote: post #63
daghamm wrote: post #57

Can someone explain those numbers to me? 5-10% performance boost sounds huge. Wouldn't we have much larger TLBd if page walk was really this expensive? On the other hand 9% increase in memory usage also sounds huge. How did this affect memory usage that much?

> 5-10% performance boost sounds huge. Wouldn't we have much larger TLBd if page walk was really this expensive? It's pretty typical for large programs to spend 15+% of their "CPU time" waiting for the TLB. [1] So larger pages really help, including changing the base 4 KiB -> 16 KiB (4x reduction in TLB pressure) and using 2 MiB huge pages (512x reduction where it works out). I've also wondered why the TLB isn't larg…

> I've also wondered why the TLB isn't larger.

Fast CAMs are (relatively) expensive, is the excuse I always hear.

Post reply on HN