# Symbolization and deobfuscation This document describes how to turn raw instruction addresses and obfuscated Java/Kotlin names in a collected trace into human-readable function names, source locations, and class/method names. The right approach depends on **what kind of trace you have**, so this page is organised around that question. Two definitions used throughout: - **Symbolization**: mapping native instruction addresses back to function names, source files, and line numbers, using the unstripped ELF binaries (or equivalent Breakpad symbol files) that were loaded in the profiled process. - **Deobfuscation**: mapping the obfuscated Java/Kotlin names emitted by R8/ProGuard (e.g. `fsd.a`) back to the original identifiers, using the `mapping.txt` produced at build time. ## Which workflow do you need? {#which-workflow} Match your trace to one of the categories below and follow the link. Picking the wrong workflow is the most common reason symbols "don't work". The key rule of thumb: **userspace** symbols are resolved offline on the host (`trace_processor bundle`), while **kernel** symbols are always resolved at record time on the device (Perfetto never stores absolute kernel addresses, to avoid disclosing [KASLR](https://en.wikipedia.org/wiki/Address_space_layout_randomization)). | Your trace contains… | Examples | What you need | | --- | --- | --- | | **Callstacks** | Native heap profiler, `traced_perf` / Linux perf CPU sampling, ART heap dumps | [Symbolization & deobfuscation](#callstacks). Userspace frames are resolved offline (`trace_processor bundle`); kernel frames are symbolized on-device automatically. | | **Kernel ftrace events** | `function_graph` tracing, `sched_blocked_reason`, kprobes | [Record-time `symbolize_ksyms`](#ftrace). These addresses **cannot** be symbolized after the fact. | | **Userspace event names** | atrace slice names, ART method tracing | [Not currently supported](#userspace-event-names) for offline deobfuscation; emit readable names at instrumentation time. | ## Callstacks: symbolization and deobfuscation {#callstacks} This applies to any data source that captures callstacks: the native heap profiler, the perf-based CPU profiler (`traced_perf` and imported Linux `perf` data), and the ART allocation profiler. These data sources record raw **userspace** instruction addresses (and, on Android, obfuscated Java/Kotlin frames), which you resolve on the host **after recording** with the steps below. You do **not** need to re-record to get userspace symbols or deobfuscated names, as long as you still have the matching binaries and mapping files. Callstacks can also contain **kernel** frames, which are handled differently; see [Kernel frames in callstacks](#callstack-kernel-frames) at the end of this section. ### {#option-1-traceconv-bundle} Option 1: `trace_processor bundle` (recommended) `trace_processor bundle` is a one-shot command that takes a trace and produces an **enriched trace**: the original trace plus all the symbol and deobfuscation data needed to analyse it, packaged together in a single file. ```bash trace_processor bundle input.perfetto-trace enriched-trace ``` The enriched trace can be opened in the [Perfetto UI](https://ui.perfetto.dev) or in `trace_processor_shell` like any other trace, with symbols and deobfuscated names already applied. NOTE: As an implementation detail, the enriched trace is currently packaged as a TAR archive containing the original trace, native symbol packets, and Java/Kotlin deobfuscation packets. The UI and `trace_processor_shell` read this format transparently, so you normally don't need to unpack it yourself. **Requirements:** - `llvm-symbolizer` on `$PATH` for native symbolization to produce function names and line numbers (`sudo apt install llvm` on Debian/Ubuntu). - Input and output must be file paths; stdin/stdout are not supported. - Matching unstripped binaries / Breakpad symbols on disk (Build IDs must match what was recorded on device). - For Java/Kotlin: the `mapping.txt` produced by the build that ran on the device. #### Automatic path discovery The main advantage over [Option 2](#option-2-legacy-traceconv-symbolize-deobfuscate) is that `bundle` looks for symbols and mapping files in all the obvious places without configuration. It searches: - The AOSP build output (`$ANDROID_PRODUCT_OUT/symbols`) when running inside a `lunch`-ed AOSP checkout. - Standard system debug directories (`$HOME/.debug`, `/usr/lib/debug`). - Absolute library paths recorded in the trace's `stack_profile_mapping` (useful when profiling on the same machine you are analysing on). - The standard Android Gradle project layout for ProGuard/R8 mapping files (`./app/build/outputs/mapping//mapping.txt`). #### Supplementing discovery with flags When auto-discovery isn't enough: ```bash trace_processor bundle \ --symbol-paths /path/to/symbols1,/path/to/symbols2 \ --proguard-map com.example.app=/path/to/mapping.txt \ --verbose \ input.perfetto-trace enriched-trace ``` The properties of the `bundle` flags are: - `--symbol-paths PATH1,PATH2,...`: additional directories to search for native symbols (in addition to the auto-discovered ones). - `--no-auto-symbol-paths`: disable auto-discovery of native symbol paths. Only paths given via `--symbol-paths` are searched. - `--proguard-map [pkg=]PATH`: additional ProGuard/R8 `mapping.txt` to apply for Java/Kotlin deobfuscation. Repeat the flag for multiple maps. The optional `pkg=` prefix scopes a map to a specific Java package. - `--no-auto-proguard-maps`: disable auto-discovery of ProGuard/R8 mapping files (e.g. the standard Android Gradle layout). Only maps given via `--proguard-map` are applied. - `--verbose`: print every path tried and every library looked up — useful when debugging "could not find" errors. ### {#option-2-legacy-traceconv-symbolize-deobfuscate} Option 2: Legacy `trace_processor util symbolize` / `util deobfuscate` NOTE: This flow is kept for backwards compatibility with existing scripts and CI pipelines that already depend on it. For new usage, always prefer [Option 1](#option-1-traceconv-bundle) — it is simpler, has auto-discovery, and works on non-Perfetto trace formats. The older `trace_processor util symbolize` and `trace_processor util deobfuscate` subcommands produce standalone symbol and deobfuscation files driven entirely by environment variables, which must then be concatenated onto the trace by hand. #### Native symbolization All tools (`trace_processor`, the `heap_profile` script) honour the `PERFETTO_BINARY_PATH` environment variable: ```bash PERFETTO_BINARY_PATH=somedir tools/heap_profile android --name ${NAME} ``` To produce a standalone symbol file for a trace you already collected: ```bash PERFETTO_BINARY_PATH=somedir trace_processor util symbolize raw-trace > symbols ``` Alternatively, set `PERFETTO_SYMBOLIZER_MODE=index` and the symbolizer will recursively index the directory for ELF files by Build ID, so filenames do not need to match. #### Java/Kotlin deobfuscation Provide ProGuard/R8 maps via `PERFETTO_PROGUARD_MAP`, using the format `packagename=map_filename[:packagename=map_filename...]`: ```bash PERFETTO_PROGUARD_MAP=com.example.pkg1=foo.txt:com.example.pkg2=bar.txt \ ./tools/heap_profile android -n com.example.app ``` To produce a standalone deobfuscation file for an existing trace: ```bash PERFETTO_PROGUARD_MAP=com.example.pkg=proguard_map.txt \ trace_processor util deobfuscate ${TRACE} > deobfuscation_map ``` #### Attaching the output to a trace Both `symbols` and `deobfuscation_map` above are serialized `TracePacket` protos, so for a **Perfetto protobuf trace** you can simply concatenate them: ```bash cat ${TRACE} symbols > symbolized-trace cat ${TRACE} deobfuscation_map > deobfuscated-trace # or both: cat ${TRACE} symbols deobfuscation_map > enriched-trace ``` The `tools/heap_profile` script does this automatically in its output directory when `PERFETTO_BINARY_PATH` is set. **Limitations:** - The concatenation trick **only works for Perfetto protobuf traces**. Other trace formats (Chrome JSON, systrace, Firefox profile, etc.) cannot have `TracePacket` bytes appended this way. For those formats, use [Option 1](#option-1-traceconv-bundle) and load the symbols via `trace_processor_shell`. - You must manage `PERFETTO_BINARY_PATH` / `PERFETTO_PROGUARD_MAP` by hand; none of the auto-discovery from Option 1 applies. ### Symbol lookup order For each native mapping in the trace, the symbolizer looks for a file with matching Build ID. For each search path `P`, it tries (in order): 1. Absolute path of the library file relative to `P`. 2. Same, with `base.apk!` stripped from the filename. 3. Basename of the library file relative to `P`. 4. Basename, with `base.apk!` stripped. 5. `P/.build-id//.debug` (the standard [Fedora Build ID layout](https://fedoraproject.org/wiki/RolandMcGrath/BuildID#Find_files_by_build_ID)). For example, `/system/lib/base.apk!foo.so` with build id `abcd1234...` is looked up under a symbol path `P` at: 1. `P/system/lib/base.apk!foo.so` 2. `P/system/lib/foo.so` 3. `P/base.apk!foo.so` 4. `P/foo.so` 5. `P/.build-id/ab/cd1234...debug` The first file with a matching Build ID wins. If the Build ID on disk differs from the one recorded in the trace, the file is skipped. ### Using symbolization/deobfuscation from a C++ library There is currently **no stable public C++ API** for performing symbolization or deobfuscation in-process. The underlying implementation exists (`TraceToBundle` in `src/traceconv/trace_to_bundle.h`, backed by `EnrichTrace` in `src/trace_processor/util/trace_enrichment/trace_enrichment.h`), but it lives under `src/` rather than `include/` and is not part of the public API surface. If you need this, please +1 on [GitHub issue #5534](https://github.com/google/perfetto/issues/5534) so we can gauge demand and prioritise. ### Troubleshooting `trace_processor bundle` always produces a bundle containing at least the original trace. When it cannot add all the enrichment it wants, it prints a summary of what is missing and how to fix it, then still exits successfully — so check the output of the command even when it succeeds. It exits non-zero only for genuine failures (unreadable input, unwritable output, or an explicitly-provided `--proguard-map` that cannot be read). Common messages and what they mean: - **`N frames could not be symbolized and will appear as "unknown"`** with a `hint: use --symbol-paths ...` line: the tool searched the auto-discovered paths (plus any `--symbol-paths` you gave) but found no binary with a matching Build ID. Follow the hint, or re-run with `--verbose` to see every path that was tried. - **`N frames ... no build IDs in trace, symbol lookup requires build IDs`**: the trace's mappings have no Build ID, so symbols cannot be matched even with the right binaries. Rebuild the binaries with Build IDs (linker flag `-Wl,--build-id`) and re-record. - **`Kernel function names: this trace contains function_graph events ...`**: the trace contains kernel addresses from `function_graph` (or similar ftrace events) recorded **without** `symbolize_ksyms`. These cannot be symbolized offline; re-record with `symbolize_ksyms: true`. See [Kernel ftrace events](#ftrace). - **`no symbol paths were searched`**: automatic discovery was disabled (`--no-auto-symbol-paths`) and no explicit paths were given. Pass `--symbol-paths` with the directories to search. - **`failed to open output file ...`**: the output path could not be created (e.g. the parent directory does not exist or is not writable). Check the path. #### Could not find library When symbolizing a profile you may see messages like: ```text Could not find /data/app/invalid.app-wFgo3GRaod02wSvPZQ==/lib/arm64/somelib.so (Build ID: 44b7138abd5957b8d0a56ce86216d478). ``` Check that `somelib.so` exists somewhere under one of the search paths (`--symbol-paths` or an auto-discovered location). Then compare the Build ID on disk to the one reported in the message using `readelf -n /path/to/somelib.so`. If they do not match, the copy on disk is a different build than the one on device and cannot be used. Re-running `trace_processor bundle` with `--verbose` prints every path tried, which usually makes it clear whether the file was missing entirely or found with the wrong Build ID. ### Kernel frames in callstacks {#callstack-kernel-frames} A sampled callstack can include **kernel** frames (e.g. perf sampling with `callstack_sampling { kernel_frames: true }`). Unlike the userspace frames above, these are symbolized **automatically on the device at record time** from `/proc/kallsyms` — the offline tools in this section do not touch them. For kernel frames to be named, the recording must be able to read `/proc/kallsyms`, which requires running as root or lowering `kptr_restrict`: ```bash echo 0 | sudo tee /proc/sys/kernel/kptr_restrict ``` If kernel frames show as hex addresses, this is a record-time permissions issue and you have to re-record. This is the same KASLR constraint as for [kernel ftrace events](#ftrace) below, but note the two use different mechanisms: callstack kernel frames do **not** use the `symbolize_ksyms` ftrace option — that flag only affects ftrace events. ## Kernel ftrace events: `symbolize_ksyms` {#ftrace} If you are doing **system tracing** and seeing raw hexadecimal addresses where you expected kernel function names — for example in [function graph tracing](/docs/data-sources/funcgraph.md), in the `blocked_function` field of an uninterruptible-sleep [scheduling blockage](/docs/case-studies/scheduling-blockages.md), or in kprobe events — the fix is **not** offline symbolization. These kernel addresses are resolved **at record time** by enabling `symbolize_ksyms` in the ftrace config: ```protobuf data_sources: { config { name: "linux.ftrace" ftrace_config { symbolize_ksyms: true # ... your ftrace_events / function_graph config ... } } } ``` This reads `/proc/kallsyms` on the device and embeds the (mangled) symbol map in the trace. It requires that either `traced_probes` runs as root or `kptr_restrict` has been lowered manually. WARNING: `trace_processor bundle` and the offline symbolizers above **cannot** recover kernel symbols. Perfetto deliberately does not store absolute kernel addresses in the trace, because doing so would defeat [KASLR](https://en.wikipedia.org/wiki/Address_space_layout_randomization) and disclose the kernel memory layout. The symbol names are mangled on device so this works without leaking absolute addresses. If you forgot to set `symbolize_ksyms`, you have to re-record. This flag applies only to ftrace **events**. Kernel frames captured inside sampled callstacks are handled separately; see [Kernel frames in callstacks](#callstack-kernel-frames). ## Userspace event names: atrace and ART method tracing {#userspace-event-names} Some data sources record human-readable **name strings** rather than addresses or stack frames. When those strings are obfuscated (e.g. an R8-obfuscated class name), there is **no offline mechanism to deobfuscate them** — the name must be emitted in a readable form at instrumentation time. This is distinct from the Java/Kotlin **stack-frame** deobfuscation in [the callstacks section](#callstacks), which applies only to heap dumps and sampled callstacks. This affects two cases today: - **atrace / userspace slice names**: [atrace](/docs/data-sources/atrace.md) slice names (and other strings that ended up in a `TRACE_EVENT` literal) are recorded verbatim. There is no post-hoc mapping step. - **ART method tracing**: the method names captured by ART method tracing are not run through the ProGuard/R8 deobfuscation path, so obfuscated builds will show obfuscated method names. A `mapping.txt`-based deobfuscation path for these is in principle possible but not currently implemented. Support is under discussion; see [GitHub issue #6391](https://github.com/google/perfetto/issues/6391) for context and to register interest.