Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adding 16 kb page size to Android

android-developers.googleblog.com

Post reply on HN
Print view View original

151–160 of 173 posts

Re: Adding 16 kb page size to Android

#151

Post by gmokki »

afr0ck wrote: post #133
ignoramous wrote: post #112

Earlier quoted context omitted.

Thanks! > I'm happy to talk about this all day! With noobs, too? ;) > Often, the CPU has different number of entries for each page size. - Does it mean userspace is free to allocate up to a maximum of 1G? I took pages to have a fixed size. - Or, you mean CPUs reserve TLB sizes depending on the requested page size? > With 2 MiB or 1 GiB pages, there is less contention and more workingset size is covered by the TLB - W…

> - Does it mean userspace is free to allocate up to a maximum of 1G? I took pages to have a fixed size. > - Or, you mean CPUs reserve TLB sizes depending on the requested page size? The TLB is a hardware cache with a limited number of entries that cannot dynamically change. Your CPU is shipped with a fixed number of entries dedicated for each page size. Translations of base 4 KiB pages could, for example, have 1024…

I think modern Intel/AMD have same amount of dTLB entries for all page sizes. For example a modern CPU with 3k TLB entries one can access at max: - 12MB with 4k page size - 6GB with 2M page size - 3TB with 1G page size

If the working set per core is bigger than above numbers you get 10-20% slower memory accesses due to TLB miss penalty.

Re: Adding 16 kb page size to Android

#152

Post by alephr »

mananaysiempre wrote: post #66
tredre3 wrote: post #55

Earlier quoted context omitted.

Windows uses 4KB pages.

Right (on x86-32 and -64, because you can’t have 64KB pages there, though larger page sizes do exist and get used). You still cannot (e.g.) MapViewOfFile() on an address not divisible by 64KB, because Alpha[1]. As far as I understand, Windows is mostly why the docs for the Blink emulator[2] (a companion project of Cosmopolitan libc) tell you any programs under it need to use sysconf(_SC_PAGESIZE) [aka getpagesize() a…

this is no longer true with MapViewOfFile3: Third time's a charm, now you can map to page boundaries

Re: Adding 16 kb page size to Android

#153

Post by mananaysiempre »

alephr wrote: post #152
mananaysiempre wrote: post #66

Earlier quoted context omitted.

Right (on x86-32 and -64, because you can’t have 64KB pages there, though larger page sizes do exist and get used). You still cannot (e.g.) MapViewOfFile() on an address not divisible by 64KB, because Alpha[1]. As far as I understand, Windows is mostly why the docs for the Blink emulator[2] (a companion project of Cosmopolitan libc) tell you any programs under it need to use sysconf(_SC_PAGESIZE) [aka getpagesize() a…

this is no longer true with MapViewOfFile3: Third time's a charm, now you can map to page boundaries

TIL about MapViewOfFile3 and NtMapViewOfSectionEx, thanks! Still, the Microsoft docs say[1]:

> [in, optional] BaseAddress

> The desired base address of the view (the address is rounded down to the nearest 64k boundary).

> [...]

> [in] Offset

> The offset from the beginning of the section.

> The offset must be 64k aligned.

The peculiar part is where base address and offset must be divisible by 64K (also referred to as the “allocation granularity”) but the size only needs to be divisible by the page size. Maybe you’re right and the docs are wrong?..

[1] https://learn.microsoft.com/en-us/windows/win32/api/memoryap...

Re: Adding 16 kb page size to Android

#154

Post by quotemstr »

Denvercoder9 wrote: post #101
quotemstr wrote: post #92

Earlier quoted context omitted.

As an aside, it's shame that hardware page table walking won out over software filled TLBs, as some older computers had. I wonder what clever and wonderful hacks we might have been able to invent had we not needed to give the CPU a raw pointer to a data structure the layout of which is fixed forever.

Page table layout isn't really fixed forever, x86 has changed its multiple times.

Not without revving the hardware though

Re: Adding 16 kb page size to Android

#155

Post by alephr »

mananaysiempre wrote: post #153
alephr wrote: post #152

Earlier quoted context omitted.

this is no longer true with MapViewOfFile3: Third time's a charm, now you can map to page boundaries

TIL about MapViewOfFile3 and NtMapViewOfSectionEx, thanks! Still, the Microsoft docs say[1]: > [in, optional] BaseAddress > The desired base address of the view (the address is rounded down to the nearest 64k boundary). > [...] > [in] Offset > The offset from the beginning of the section. > The offset must be 64k aligned. The peculiar part is where base address and offset must be divisible by 64K (also referred to as…

the new behavior works under the MEM_REPLACE_PLACEHOLDER flag, you can create those regions with VirtualAlloc2

Re: Adding 16 kb page size to Android

#156

Post by SloopJon »

ignoramous wrote: post #105
lanigone wrote: post #37

Earlier quoted context omitted.

you can also do 2M and 1G huge pages on x86, it gets kind of silly fast.

What? Any pointers on how 1G speeds things up? I'd have taken a bigger page size to wreak havoc on process scheduling and filesystem.

Search for huge pages in the documentation of a DBMS that implements its own caching in shared memory: Oracle [1], PostgreSQL [2], MySQL [3], etc. When you're caching hundreds of gigabytes, it makes a difference. Here's a benchmark comparing PostgreSQL performance with regular, large, and huge pages [4].

There was a really bad performance regression in Linux a couple of years ago that killed performance with large memory regions like this (can't find a useful link at the moment), and the short-term mitigation was to increase the huge page size from 2MB to 1GB.

[1] https://blogs.oracle.com/exadata/post/huge-pages-in-the-cont...

[2] https://www.postgresql.org/docs/current/kernel-resources.htm...

[3] https://dev.mysql.com/doc/refman/8.4/en/large-page-support.h...

[4] https://www.percona.com/blog/benchmark-postgresql-with-linux...

Re: Adding 16 kb page size to Android

#157

Post by dotancohen »

david_allison wrote: post #79
dotancohen wrote: post #32

Earlier quoted context omitted.

Yes, but the context here is Java or Kotlin running on Android, not embedded C. Or do some Android applications run embedded C with only a Java UI? I'm not an Android dev.

The Android Native Development Kit (NDK) allows building native code libraries for Android (typically C/C++, but this can include Rust). These can then be loaded and accessed by JNI on the Java/Kotlin side * Brief overview of the NDK: https://developer.android.com/ndk/guides * Guide to supporting 16KB page sizes with the NDK https://developer.android.com/guide/practices/page-sizes

Good to know, thank you!

Re: Adding 16 kb page size to Android

#158

Post by dotancohen »

fpoling wrote: post #81
dotancohen wrote: post #32

Earlier quoted context omitted.

Yes, but the context here is Java or Kotlin running on Android, not embedded C. Or do some Android applications run embedded C with only a Java UI? I'm not an Android dev.

Chrome browser on Android uses the same code base as Chrome on desktop including multi-process architecture. But it’s UI is in Java communicating with C++ using JNI.

I had no idea, thank you!

Re: Adding 16 kb page size to Android

#159

Post by CalChris »

arghwhat wrote: post #11
twoodfin wrote: post #4

A little additional background: iOS has used 16KB pages since the 64-bit transition, and ARM Macs have inherited that design.

A more relevant bit of background is that 4KB pages lead to quite a lot of overhead due to the sheer number of mappings needing to be configured and cached. Using larger pages reduce overhead, in particular TLB misses as fewer entries are needed to describe the same memory range. While x86 chips mainly supports 4K, 2M and 1G pages, ARM chips tend to support more practical 16K page sizes - a nice balance between perfo…

Armv8-A also supports 4K pages: FEAT_TGran4K. So Apple did indeed make a choice to instead use 16K, FEAT_TGran16K. Microsoft uses 4K for AArch64 Windows.

Re: Adding 16 kb page size to Android

#160

Post by extraduder_ire »

ignoramous wrote: post #107
saagarjha wrote: post #34

Earlier quoted context omitted.

Android apps can call into native code via JNI, which the platform supports.

Wonder if Android apps can also be fully native (C++)?

I saw a project posted on here a while back about writing android apps with no java, only c.

There is no good reason to do it, but it is apparently possible.

https://github.com/cnlohr/rawdrawandroid

Post reply on HN