Tags: hgn/linux
Tags
Disable building BPF based features by default for v6.4. We need to better polish building with BPF skels, so revert back to making it an experimental feature that has to be explicitely enabled using BUILD_BPF_SKEL=1. Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
Additional fixes on top of the previous v6.4 perf tools pull request: - Remove the use of bpftool to generate a vmlinux.h file to be used when building BPF bytecode that enables perf tools features. Instead add a vmlinux.h file with just the set of structs and fields used in the tools/perf/util/bpf_skel/*.bpf.c files and rely on libbpf's CO-RE functionality to compare the BTF info generated by clang for the BPF target against the BPF info for the kernel where the resulting tool runs to adjust it before loading into the kernel. - Fix 'perf stat -b <prog id>' to match counter values obtained via 'bpftool prog profile <prog id>', as fentry/fexit progs gets loaded, but the corresponding perf event wasn't being enabled and added into the events BPF map. - Fix copy of 'perf stat' metric threshold to avoid SEGV on using uninitialized memory. Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
perf tools changes and fixes for v6.4:
Build:
- Require libtraceevent to build, one can disable it using NO_LIBTRACEEVENT=1.
It is required for tools like 'perf sched', 'perf kvm', 'perf trace', etc.
libtraceevent is available in most distros so installing 'libtraceevent-devel' should
be a one-time event to continue building perf as usual.
Using NO_LIBTRACEEVENT=1 produces tooling that is functional and sufficient
for lots of users not interested in those libtraceevent dependent features.
- Allow Python support in 'perf script' when libtraceevent isn't linked, as not
all features requires it, for instance Intel PT does not use tracepoints.
- Error if the python interpreter needed for jevents to work isn't available
and NO_JEVENTS=1 isn't set, preventing a build without support for JSON
vendor events, which is a rare but possible condition. The two check error
messages:
$(error ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.)
$(error ERROR: Python interpreter needed for jevents generation too old (older than 3.6). Install a newer python or build with NO_JEVENTS=1.)
- Make libbpf 1.0 the minimum required when building with out of tree, distro
provided libbpf.
- Use libsdtc++'s and LLVM's libcxx's __cxa_demangle, a portable C++ demangler, add
'perf test' entry for it.
- Make binutils libraries opt in, as distros disable building with it due to licensing,
they were used for C++ demangling, for instance.
- Switch libpfm4 to opt-out rather than opt-in, if libpfm-devel (or equivalent) isn't installed, we'll
just have a build warning:
Makefile.config:1144: libpfm4 not found, disables libpfm4 support. Please install libpfm4-dev
- Add a feature test for scandirat(), that is not implemented so far in musl and uclibc,
disabling features that need it, such as scanning for tracepoints in /sys/kernel/tracing/events.
- Build by default with BPF skels, to disable it use NO_BPF_SKEL=1 in the make
command line.
This make the following features to be available by default, which were
introduced in previous kernel versions built required building with BUILD_BPF_SKEL=1:
- perf lock contention
- perf kwork
- off-cpu profiling
- Filtering events using BPF
- 'perf stat' for counting events on BPF programs
- Features are being implemented using BPF skels, where BPF is built and linked
with the native perf tool to then be loaded and attached to places in the
kernel where information can be obtained to implement features that previously
required kernel changes.
These load and attach operations are only performed when the feature that
uses it is requested by the user in the perf command line.
One can look at the git history for the files in the
tools/perf/util/bpf_skel/ directory to find out about them, in addition to the
perf tools man pages, as of this writing the contents of this directory:
$ cd tools/perf/util/bpf_skel/
$ ls
bperf_follower.bpf.c bperf_leader.bpf.c
bpf_prog_profiler.bpf.c func_latency.bpf.c
kwork_trace.bpf.c lock_contention.bpf.c
off_cpu.bpf.c sample_filter.bpf.c
perf BPF filters:
- New feature where BPF can be used to filter samples, for instance:
$ sudo ./perf record -e cycles --filter 'period > 1000' true
$ sudo ./perf script
perf-exec 2273949 546850.708501: 5029 cycles: ffffffff826f9e25 finish_wait+0x5 ([kernel.kallsyms])
perf-exec 2273949 546850.708508: 32409 cycles: ffffffff826f9e25 finish_wait+0x5 ([kernel.kallsyms])
perf-exec 2273949 546850.708526: 143369 cycles: ffffffff82b4cdbf xas_start+0x5f ([kernel.kallsyms])
perf-exec 2273949 546850.708600: 372650 cycles: ffffffff8286b8f7 __pagevec_lru_add+0x117 ([kernel.kallsyms])
perf-exec 2273949 546850.708791: 482953 cycles: ffffffff829190de __mod_memcg_lruvec_state+0x4e ([kernel.kallsyms])
true 2273949 546850.709036: 501985 cycles: ffffffff828add7c tlb_gather_mmu+0x4c ([kernel.kallsyms])
true 2273949 546850.709292: 503065 cycles: 7f2446d97c03 _dl_map_object_deps+0x973 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
- In addition to 'period' (PERF_SAMPLE_PERIOD), the other PERF_SAMPLE_ can be
used for filtering, and also some other sample accessible values, from
tools/perf/Documentation/perf-record.txt:
Essentially the BPF filter expression is:
<term> <operator> <value> (("," | "||") <term> <operator> <value>)*
The <term> can be one of:
ip, id, tid, pid, cpu, time, addr, period, txn, weight, phys_addr,
code_pgsz, data_pgsz, weight1, weight2, weight3, ins_lat, retire_lat,
p_stage_cyc, mem_op, mem_lvl, mem_snoop, mem_remote, mem_lock,
mem_dtlb, mem_blk, mem_hops
The <operator> can be one of:
==, !=, >, >=, <, <=, &
The <value> can be one of:
<number> (for any term)
na, load, store, pfetch, exec (for mem_op)
l1, l2, l3, l4, cxl, io, any_cache, lfb, ram, pmem (for mem_lvl)
na, none, hit, miss, hitm, fwd, peer (for mem_snoop)
remote (for mem_remote)
na, locked (for mem_locked)
na, l1_hit, l1_miss, l2_hit, l2_miss, any_hit, any_miss, walk, fault (for mem_dtlb)
na, by_data, by_addr (for mem_blk)
hops0, hops1, hops2, hops3 (for mem_hops)
perf lock contention:
- Show lock type with address.
- Track and show mmap_lock, siglock and per-cpu rq_lock with address. This is
done for mmap_lock by following the current->mm pointer:
$ sudo ./perf lock con -abl -- sleep 10
contended total wait max wait avg wait address symbol
...
16344 312.30 ms 2.22 ms 19.11 us ffff8cc702595640
17686 310.08 ms 1.49 ms 17.53 us ffff8cc7025952c0
3 84.14 ms 45.79 ms 28.05 ms ffff8cc78114c478 mmap_lock
3557 76.80 ms 68.75 us 21.59 us ffff8cc77ca3af58
1 68.27 ms 68.27 ms 68.27 ms ffff8cda745dfd70
9 54.53 ms 7.96 ms 6.06 ms ffff8cc7642a48b8 mmap_lock
14629 44.01 ms 60.00 us 3.01 us ffff8cc7625f9ca0
3481 42.63 ms 140.71 us 12.24 us ffffffff937906ac vmap_area_lock
16194 38.73 ms 42.15 us 2.39 us ffff8cd397cbc560
11 38.44 ms 10.39 ms 3.49 ms ffff8ccd6d12fbb8 mmap_lock
1 5.43 ms 5.43 ms 5.43 ms ffff8cd70018f0d8
1674 5.38 ms 422.93 us 3.21 us ffffffff92e06080 tasklist_lock
581 4.51 ms 130.68 us 7.75 us ffff8cc9b1259058
5 3.52 ms 1.27 ms 703.23 us ffff8cc754510070
112 3.47 ms 56.47 us 31.02 us ffff8ccee38b3120
381 3.31 ms 73.44 us 8.69 us ffffffff93790690 purge_vmap_area_lock
255 3.19 ms 36.35 us 12.49 us ffff8d053ce30c80
- Update default map size to 16384.
- Allocate single letter option -M for --map-nr-entries, as it is proving being
frequently used.
- Fix struct rq lock access for older kernels with BPF's CO-RE (Compile once,
run everywhere).
- Fix problems found with MSAn.
perf report/top:
- Add inline information when using --call-graph=fp or lbr, as was already done
to the --call-graph=dwarf callchain mode.
- Improve the 'srcfile' sort key performance by really using an optimization
introduced in 6.2 for the 'srcline' sort key that avoids calling addr2line for
comparision with each sample.
perf sched:
- Make 'perf sched latency/map/replay' to use "sched:sched_waking" instead of
"sched:sched_waking", consistent with 'perf record' since d566a9c ("perf
sched: Prefer sched_waking event when it exists").
perf ftrace:
- Make system wide the default target for latency subcommand, run the following
command then generate some network traffic and press control+C:
# perf ftrace latency -T __kfree_skb
^C# DURATION | COUNT | GRAPH |
0 - 1 us | 27 | ############# |
1 - 2 us | 22 | ########### |
2 - 4 us | 8 | #### |
4 - 8 us | 5 | ## |
8 - 16 us | 24 | ############ |
16 - 32 us | 2 | # |
32 - 64 us | 1 | |
64 - 128 us | 0 | |
128 - 256 us | 0 | |
256 - 512 us | 0 | |
512 - 1024 us | 0 | |
1 - 2 ms | 0 | |
2 - 4 ms | 0 | |
4 - 8 ms | 0 | |
8 - 16 ms | 0 | |
16 - 32 ms | 0 | |
32 - 64 ms | 0 | |
64 - 128 ms | 0 | |
128 - 256 ms | 0 | |
256 - 512 ms | 0 | |
512 - 1024 ms | 0 | |
1 - ... s | 0 | |
#
perf top:
- Add --branch-history (LBR: Last Branch Record) option, just like already
available for 'perf record'.
- Fix segfault in thread__comm_len() where thread->comm was being used outside
thread->comm_lock.
perf annotate:
- Allow configuring objdump and addr2line in ~/.perfconfig., so that you can
use alternative binaries, such as llvm's.
perf kvm:
- Add TUI mode for 'perf kvm stat report'.
Reference counting:
- Add reference count checking infrastructure to check for use after free, done
to the 'cpumap', 'namespaces', 'maps' and 'map' structs, more to come. To
build with it use -DREFCNT_CHECKING=1 in the make command line to build
tools/perf.
Documented at: https://perf.wiki.kernel.org/index.php/Reference_Count_Checking
- This caught, for instance, fix, present in this series:
- Fix maps use after put in 'perf test "Share thread maps"':
'maps' is copied from leader, but the leader is put on line 79 and then
'maps' is used to read the reference count below - so a use after put, with the
put of maps happening within thread__put. Fix by reversing the order of puts so
that the leader is put last.
- Also several fixes were made to places where reference counts were not being held.
- Make this one of the tests in 'make -C tools/perf build-test' to regularly
build test it and to make sure no direct access to the reference counted
structs are made, doing that via accessors to check the validity of the struct
pointer.
ARM64:
- Fix 'perf report' segfault when filtering coresight traces by sparse lists of CPUs.
- Add support for 'simd' as a sort field for 'perf report', to show ARM's NEON SIMD's
predicate flags: "partial" and "empty".
Vendor events (JSON):
arm64:
- Add N1 metrics.
Intel:
- Add graniterapids, grandridge and sierraforrest events.
- Refresh events for: alderlake, aldernaken, broadwell, broadwellde,
broadwellx, cascadelakx, haswell, haswellx, icelake, icelakex, jaketown,
meteorlake, knightslanding, sandybridge, sapphirerapids, silvermont, skylake,
tigerlake and westmereep-dp
- Refresh metrics for alderlake-n, broadwell, broadwellde, broadwellx, haswell,
haswellx, icelakex, ivybridge, ivytown and skylakex.
perf stat:
- Implement --topdown using JSON metrics.
- Add TopdownL1 JSON metric as a default if present, but disable it for
now for some Intel hybrid architectures, a series of patches
addressing this is being reviewed and will be submitted for v6.5.
- Use metrics for --smi-cost.
- Update topdown documentation.
Vendor events (JSON) infrastructure:
- Add support for computing and printing metric threshold values. For instance,
here is one found in the tools/perf/pmu-events/arch/x86/sapphirerapids/spr-metrics.json
file:
{
"BriefDescription": "Percentage of cycles spent in System Management Interrupts.",
"MetricExpr": "((msr@aperf@ - cycles) / msr@aperf@ if msr@smi@ > 0 else 0)",
"MetricGroup": "smi",
"MetricName": "smi_cycles",
"MetricThreshold": "smi_cycles > 0.1",
"ScaleUnit": "100%"
},
- Test parsing metric thresholds with the fake PMU in 'perf test pmu-events'.
- Support for printing metric thresholds in 'perf list'.
- Add --metric-no-threshold option to 'perf stat'.
- Add rand (reverse and) and has_pmem (optane memory) support to metrics.
- Sort list of input files to avoid depending on the order from readdir()
helping in obtaining reproducible builds.
S/390:
- Add common metrics: - CPI (cycles per instruction), prbstate (ratio of
instructions executed in problem state compared to total number of
instructions), l1mp (Level one instruction and data cache misses per 100
instructions).
- Add cache metrics for z13, z14, z15 and z16.
- Add metric for TLB and cache.
ARM:
- Add raw decoding for SPE (Statistical Profiling Extension) v1.3 MTE (Memory
Tagging Extension) and MOPS (Memory Operations) load/store.
Hardware tracing:
Intel PT:
- Add event type names UINTR (User interrupt delivered) and UIRET (Exiting from
user interrupt routine), documented in table 32-50 "CFE Packet Type and Vector
Fields Details" in the Intel Processor Trace chapter of The Intel SDM
Volume 3 version 078.
- Add support for new branch instructions ERETS and ERETU.
- Fix CYC timestamps after standalone CBR
ARM CoreSight:
- Allow user to override timestamp and contextid settings.
- Fix segfault in dso lookup.
- Fix timeless decode mode detection.
- Add separate decode paths for timeless and per-thread modes.
auxtrace:
- Fix address filter entire kernel size.
Miscellaneous:
- Fix use-after-free and unaligned bugs in the PLT handling routines.
- Use zfree() to reduce chances of use after free.
- Add missing 0x prefix for addresses printed in hexadecimal in 'perf probe'.
- Suppress massive unsupported target platform errors in the unwind code.
- Fix return incorrect build_id size in elf_read_build_id().
- Fix 'perf scripts intel-pt-events.py' IPC output for Python 2 .
- Add missing new parameter in kfree_skb tracepoint to the python scripts using it.
- Add 'perf bench syscall fork' benchmark.
- Add support for printing PERF_MEM_LVLNUM_UNC (Uncached access) in 'perf mem'.
- Fix wrong size expectation for perf test 'Setup struct perf_event_attr' caused by
the patch adding perf_event_attr::config3.
- Fix some spelling mistakes.
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
PreviousNext