Add support for symbolicating APK/ZIP-embedded libraries on Android #662
Conversation
|
Code size changes for a hello-world Rust program linked with libstd with backtrace: On platform
|
sudoBash418
force-pushed
the
pr/android-apk-symbolication
branch
from
2b61d3c
to
3534f66
Compare
|
Code size changes for a hello-world Rust program linked with libstd with backtrace: On platform
|
sudoBash418
force-pushed
the
pr/android-apk-symbolication
branch
from
3534f66
to
3b8533e
Compare
|
Code size changes for a hello-world Rust program linked with libstd with backtrace: On platform
|
sudoBash418
force-pushed
the
pr/android-apk-symbolication
branch
from
3b8533e
to
80f5df4
Compare
|
Code size changes for a hello-world Rust program linked with libstd with backtrace: On platform
|
sudoBash418
force-pushed
the
pr/android-apk-symbolication
branch
from
80f5df4
to
1020bf4
Compare
|
Code size changes for a hello-world Rust program linked with libstd with backtrace: On platform
|
|
The remaining macOS CI failure seems unrelated to this PR. Not sure why the binary size check reports an increase on Ubuntu; AFAIK my changes should have minimal impact on non-Android targets. |
|
The macos failure is caused by a new lint in the nightly compiler. #664 fixes it. |
sudoBash418
force-pushed
the
pr/android-apk-symbolication
branch
from
1020bf4
to
d881495
Compare
|
Rebased and added requested changes. Also updated the |
|
Looks like the binary size check is broken by the dependency bump I just rebased on, whoops. |
|
That's fine. It really ought to be fixed but whatever. |
There was a problem hiding this comment.
One last(?) fussing about types, and most of the rest is just code and logic organization, and documenting the invariants correctly so as to make auditing and refactoring easy. This is an... oddly-organized library that has grown over quite a long time, before I even started programming, nevermind learned Rust. A significant amount of effort necessarily goes into just keeping this overgrown garden from finally bursting out of its walls, and the current organization doesn't really help you in that. You have my sympathies.
|
@sudoBash418 The reason that Windows demands you align the mapping to 64KiB is because that is the value of dwAllocationGranularity... a sort of virtual page size? Except not really. dwAllocationGranularity means that you lock down that much address space, but the actual pages are still faulted one by one within that region. |
| // find MapsEntry matching library's base address and get its file offset | ||
| maps.iter() | ||
| .find(|m| m.ip_matches(dlpi_addr as usize)) | ||
| .map(|m| m.offset()) |
There was a problem hiding this comment.
Note to self: reread the MapsEntry struct and its FromStr if you somehow develop another question about this.
|
Hm, this looks good but... ...ah, right. Can you construct a test for specifically symbolicating a .so in a zip for Android? It likely has to be its own crate, like the other tests. Honestly, though, a bash script recorded from what you've been doing a zillion times I'm sure, and just plonked into the workflow yaml with a conditional to run it only on android, would also be fine. With that, I can accept this! |
|
...wait. ...nevermind, we're only testing linkage on Android apparently. |
|
I believe I can accept this with a rebase. |
By default, modern Android build tools will store native libraries uncompressed, and the loader will map them directly from the APK (instead of the package manager extracting them on installation). This commit adds support for symbolicating these embedded libraries. To avoid parsing ZIP structures, the offset of the library within the archive is determined via /proc/self/maps. ref: https://android.googlesource.com/platform/bionic/+/main/android-changes-for-ndk-developers.md#opening-shared-libraries-directly-from-an-apk
sudoBash418
force-pushed
the
pr/android-apk-symbolication
branch
from
6103b0a
to
6319c02
Compare
Fixes #661
Add support for symbolicating APK/ZIP-embedded libraries by doing the following:
In
native_libraries:!/substring/proc/self/mapsand use theoffsetfield to determine where the library starts in the archiveIn
Mapping::new_android(non-embedded libraries use the same logic as before):native_librariesand ending at the end of the APK.(the cost of mmap-ing the unused remainder of the APK should be minimal)
Aside: my previous attempt used the
zipcrate to determine the exact size of the library and avoid mmap-ing the rest of the archive. But I figured it wasn't worth bringing in a new dependency and bloating the example binary by ~25% 😅As for potential confusion between a file literally named
/path/to/apk!/lib.soand a file namedlib.sowithin/path/to/apk, I tried to follow the dynamic linker's behaviour: treat it as an embedded file first, and fallback to a literal path if mapping fails for any reason.Tests have been skipped because there is no CI that runs the emulated binaries anyways.