[ptrace] 201766a20e: kernel_selftests.seccomp.make_fail
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 201766a20e30f982ccfe36bebfad9602c3ff574a ("ptrace: add PTRACE_GET_SYSCALL_INFO request")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master
in testcase: kernel_selftests
with following parameters:
group: kselftests-02
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
2019-07-26 16:52:03 make run_tests -C seccomp
make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp'
gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf
In file included from seccomp_bpf.c:51:0:
seccomp_bpf.c: In function ‘tracer_ptrace’:
seccomp_bpf.c:1787:20: error: ‘PTRACE_EVENTMSG_SYSCALL_ENTRY’ undeclared (first use in this function)
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
seccomp_bpf.c:1787:20: note: each undeclared identifier is reported only once for each function it appears in
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
seccomp_bpf.c:1788:6: error: ‘PTRACE_EVENTMSG_SYSCALL_EXIT’ undeclared (first use in this function)
: PTRACE_EVENTMSG_SYSCALL_EXIT, msg);
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
Makefile:12: recipe for target 'seccomp_bpf' failed
make: *** [seccomp_bpf] Error 1
make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp'
To reproduce:
# build kernel
cd linux
cp config-5.2.0-10889-g201766a20e30f9 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
2 months, 2 weeks
- 3
- 3
[ext4] b1b4705d54: filebench.sum_bytes_mb/s -20.2% regression
by kernel test robot
Greeting,
FYI, we noticed a -20.2% regression of filebench.sum_bytes_mb/s due to commit:
commit: b1b4705d54abedfd69dcdf42779c521aa1e0fbd3 ("ext4: introduce direct I/O read using iomap infrastructure")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: filebench
on test machine: 8 threads Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 8G memory
with following parameters:
disk: 1HDD
fs: ext4
test: fivestreamreaddirect.f
cpufreq_governor: performance
ucode: 0x27
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/1HDD/ext4/x86_64-rhel-7.6/debian-x86_64-2019-11-14.cgz/lkp-hsw-d01/fivestreamreaddirect.f/filebench/0x27
commit:
09edf4d381 ("ext4: introduce new callback for IOMAP_REPORT")
b1b4705d54 ("ext4: introduce direct I/O read using iomap infrastructure")
09edf4d381957b14 b1b4705d54abedfd69dcdf42779
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
:4 25% 1:4 dmesg.WARNING:stack_recursion
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:da
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:db
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:dc
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:dd
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:de
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:df
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ea
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:eb
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ec
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ed
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ee
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ef
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fa
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fb
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fc
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fd
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fe
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ff
:4 25% 1:4 kmsg.file:libnss_files-#so_fault:filemap_fault_mmap:generic_file_mmap_readpage:simple_readpage
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a07fffffa0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a17fffffa1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a27fffffa2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a37fffffa3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a47fffffa4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a57fffffa5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a67fffffa6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a77fffffa7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a7fffff8aff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a7fffff9aff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a87fffffa8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a97fffffa9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#aa7fffffaaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ab7fffffabff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ac7fffffacff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ad7fffffadff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ae7fffffaeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#af7fffffafff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b07fffffb0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b17fffffb1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b27fffffb2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b37fffffb3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b47fffffb4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b57fffffb5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b67fffffb6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b77fffffb7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b7fffff8bff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b7fffff9bff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b87fffffb8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b97fffffb9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ba7fffffbaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bb7fffffbbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bc7fffffbcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bd7fffffbdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#be7fffffbeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bf7fffffbfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c07fffffc0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c17fffffc1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c27fffffc2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c37fffffc3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c47fffffc4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c57fffffc5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c67fffffc6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c77fffffc7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c7fffff8cff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c7fffff9cff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c87fffffc8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c97fffffc9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ca7fffffcaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cb7fffffcbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cc7fffffccff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cd7fffffcdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ce7fffffceff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cf7fffffcfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d07fffffd0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d17fffffd1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d27fffffd2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d37fffffd3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d47fffffd4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d57fffffd5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d67fffffd6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d77fffffd7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d7fffff8dff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d7fffff9dff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d87fffffd8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d97fffffd9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#da7fffffdaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#db7fffffdbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#dc7fffffdcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#dd7fffffddff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#de7fffffdeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#df7fffffdfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e07fffffe0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e17fffffe1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e27fffffe2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e37fffffe3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e47fffffe4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e57fffffe5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e67fffffe6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e77fffffe7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e7fffff8eff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e7fffff9eff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e87fffffe8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e97fffffe9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ea7fffffeaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#eb7fffffebff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ec7fffffecff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ed7fffffedff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ee7fffffeeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ef7fffffefff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f07ffffff0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f17ffffff1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f27ffffff2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f37ffffff3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f47ffffff4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f57ffffff5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f67ffffff6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f77ffffff7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f7fffff8fff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f7fffff9fff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f87ffffff8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f97ffffff9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fa7ffffffaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fb7ffffffbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fc7ffffffcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fd7ffffffdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fe7ffffffeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ff7fffffffff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff80ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff81ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff82ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff83ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff84ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff85ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff86ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff87ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff88ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff89ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff90ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff91ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff92ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff93ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff94ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff95ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff96ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff97ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff98ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff99ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffffffffff
%stddev %change %stddev
\ | \
58.30 ± 2% -20.2% 46.53 filebench.sum_bytes_mb/s
3503 ± 2% -20.2% 2796 filebench.sum_operations
58.39 ± 2% -20.2% 46.61 filebench.sum_operations/s
58.33 ± 2% -19.9% 46.75 filebench.sum_reads/s
84.34 ± 4% +27.1% 107.17 filebench.sum_time_ms/op
7174922 ± 2% -20.2% 5727328 filebench.time.file_system_inputs
10963 ± 2% -8.5% 10035 filebench.time.voluntary_context_switches
58.37 ± 3% -9.0 49.38 ± 3% mpstat.cpu.all.idle%
41.08 ± 5% +8.2 49.24 ± 2% mpstat.cpu.all.iowait%
3061 ± 6% +6.1% 3246 ± 4% turbostat.Bzy_MHz
5.48 ± 97% +260.7% 19.78 ± 30% turbostat.Pkg%pc2
27.33 ± 32% -52.0% 13.12 ± 19% sched_debug.cpu.nr_uninterruptible.max
-67.67 -60.8% -26.50 sched_debug.cpu.nr_uninterruptible.min
28.48 ± 30% -57.9% 11.98 ± 23% sched_debug.cpu.nr_uninterruptible.stddev
8210 ± 44% +76.1% 14454 ± 6% softirqs.CPU0.RCU
8200 ± 40% +70.2% 13954 ± 5% softirqs.CPU2.RCU
9772 ± 36% +59.6% 15597 ± 15% softirqs.CPU3.RCU
108.33 ± 27% +113.5% 231.25 ± 26% interrupts.CPU1.RES:Rescheduling_interrupts
1032 -23.9% 786.00 ± 30% interrupts.CPU5.CAL:Function_call_interrupts
229.33 ± 46% +50.1% 344.25 ± 31% interrupts.CPU5.NMI:Non-maskable_interrupts
229.33 ± 46% +50.1% 344.25 ± 31% interrupts.CPU5.PMI:Performance_monitoring_interrupts
457.00 ± 61% -74.2% 117.75 ± 41% interrupts.CPU5.RES:Rescheduling_interrupts
53.33 ± 60% +206.6% 163.50 ± 42% interrupts.CPU7.RES:Rescheduling_interrupts
58.99 ± 3% -14.9% 50.18 ± 3% iostat.cpu.idle
40.36 ± 5% +19.8% 48.35 ± 2% iostat.cpu.iowait
0.18 +386.3% 0.87 ±137% iostat.cpu.user
33012 ± 2% -19.7% 26520 iostat.sda.rkB/s
40.60 ± 6% +16.2% 47.18 ± 4% iostat.sda.w/s
43038 ± 4% +17.1% 50399 ± 2% iostat.sda.wkB/s
58.67 ± 3% -15.6% 49.50 ± 3% vmstat.cpu.id
40.00 ± 5% +19.4% 47.75 ± 2% vmstat.cpu.wa
33010 ± 2% -19.7% 26518 vmstat.io.bi
42837 ± 4% +17.2% 50200 ± 2% vmstat.io.bo
5519500 -15.5% 4664152 vmstat.memory.cache
2183603 +39.2% 3039439 ± 2% vmstat.memory.free
1065500 -19.2% 861375 slabinfo.buffer_head.active_objs
27320 -19.1% 22093 slabinfo.buffer_head.active_slabs
1065521 -19.1% 861671 slabinfo.buffer_head.num_objs
27320 -19.1% 22093 slabinfo.buffer_head.num_slabs
1306 +18.4% 1546 ± 7% slabinfo.kmalloc-512.active_objs
1306 +18.4% 1546 ± 7% slabinfo.kmalloc-512.num_objs
28277 -9.3% 25651 slabinfo.radix_tree_node.active_objs
5388173 -15.3% 4561843 meminfo.Cached
989375 ± 2% -22.6% 765315 ± 3% meminfo.Dirty
4310025 -19.2% 3483318 meminfo.Inactive
4261041 -19.4% 3434223 meminfo.Inactive(file)
162676 -13.7% 140448 meminfo.KReclaimable
2151883 ± 2% +39.5% 3000927 ± 2% meminfo.MemFree
5902816 -14.4% 5053772 meminfo.Memused
162676 -13.7% 140448 meminfo.SReclaimable
197794 -11.3% 175489 meminfo.Slab
80314276 ± 5% +6.5% 85566891 ± 3% perf-stat.i.branch-instructions
4810221 ± 5% +7.5% 5172535 perf-stat.i.cache-references
7.022e+08 ± 3% +6.9% 7.509e+08 ± 2% perf-stat.i.cpu-cycles
3.891e+08 ± 5% +6.3% 4.135e+08 ± 3% perf-stat.i.instructions
195094 ± 2% +10.8% 216142 perf-stat.i.node-loads
79563934 ± 5% +6.5% 84757473 ± 3% perf-stat.ps.branch-instructions
4765201 ± 5% +7.5% 5123552 perf-stat.ps.cache-references
6.956e+08 ± 3% +6.9% 7.438e+08 ± 2% perf-stat.ps.cpu-cycles
3.855e+08 ± 5% +6.3% 4.096e+08 ± 3% perf-stat.ps.instructions
193276 ± 2% +10.8% 214098 perf-stat.ps.node-loads
4.114e+10 ± 5% +5.3% 4.333e+10 ± 3% perf-stat.total.instructions
247900 ± 3% -22.9% 191187 ± 3% proc-vmstat.nr_dirty
1349353 -15.5% 1140545 proc-vmstat.nr_file_pages
535810 ± 2% +40.1% 750444 ± 2% proc-vmstat.nr_free_pages
1067252 -19.6% 858356 proc-vmstat.nr_inactive_file
40723 -13.8% 35112 proc-vmstat.nr_slab_reclaimable
1110681 ± 2% +17.1% 1300126 proc-vmstat.nr_written
1067251 -19.6% 858384 proc-vmstat.nr_zone_inactive_file
249604 ± 3% -22.7% 193058 ± 3% proc-vmstat.nr_zone_write_pending
660225 -3.8% 635276 proc-vmstat.pgalloc_dma32
871298 +2.7% 894930 proc-vmstat.pgalloc_normal
3587465 ± 2% -20.2% 2863670 proc-vmstat.pgpgin
4645372 ± 4% +16.6% 5414779 proc-vmstat.pgpgout
13.16 ± 14% -6.6 6.58 ± 52% perf-profile.calltrace.cycles-pp.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.06 ± 15% -6.6 6.50 ± 52% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.50 ± 52% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.02 ± 14% -6.5 6.48 ± 52% perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat.do_syscall_64
10.39 ± 36% -4.6 5.78 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel
11.21 ± 33% -4.6 6.64 ± 7% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64
5.71 ± 17% -2.8 2.95 ± 54% perf-profile.calltrace.cycles-pp.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
3.68 ± 14% -2.0 1.69 ± 50% perf-profile.calltrace.cycles-pp.release_pages.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode.evict
3.70 ± 14% -2.0 1.74 ± 48% perf-profile.calltrace.cycles-pp.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
2.90 ± 23% -1.6 1.32 ± 56% perf-profile.calltrace.cycles-pp.try_to_free_buffers.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode.evict
2.42 ± 20% -1.4 1.01 ± 75% perf-profile.calltrace.cycles-pp.delete_from_page_cache_batch.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
2.02 ± 17% -1.3 0.68 ±104% perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode
1.33 ± 28% -0.9 0.45 ±100% perf-profile.calltrace.cycles-pp.free_buffer_head.try_to_free_buffers.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode
1.21 ± 22% -0.8 0.40 ±106% perf-profile.calltrace.cycles-pp.xas_store.delete_from_page_cache_batch.truncate_inode_pages_range.ext4_evict_inode.evict
1.17 ± 3% +0.4 1.57 ± 13% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
0.35 ±141% +0.7 1.07 ± 37% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
3.71 ± 34% +1.6 5.34 ± 14% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
64.92 ± 5% +9.2 74.17 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
63.80 ± 4% +9.4 73.19 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
69.95 ± 5% +10.4 80.37 ± 5% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
13.16 ± 14% -6.6 6.58 ± 52% perf-profile.children.cycles-pp.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.children.cycles-pp.do_unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.children.cycles-pp.ext4_evict_inode
13.03 ± 14% -6.5 6.49 ± 53% perf-profile.children.cycles-pp.truncate_inode_pages_range
13.07 ± 15% -6.4 6.64 ± 53% perf-profile.children.cycles-pp.evict
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.children.cycles-pp.start_kernel
5.71 ± 17% -2.7 2.99 ± 54% perf-profile.children.cycles-pp.truncate_cleanup_page
3.74 ± 13% -1.9 1.85 ± 49% perf-profile.children.cycles-pp.__pagevec_release
3.75 ± 12% -1.9 1.88 ± 49% perf-profile.children.cycles-pp.release_pages
2.91 ± 22% -1.5 1.40 ± 53% perf-profile.children.cycles-pp.try_to_free_buffers
2.43 ± 19% -1.3 1.12 ± 57% perf-profile.children.cycles-pp.delete_from_page_cache_batch
2.03 ± 17% -1.1 0.95 ± 58% perf-profile.children.cycles-pp.free_unref_page_list
1.34 ± 27% -0.7 0.65 ± 45% perf-profile.children.cycles-pp.free_buffer_head
1.23 ± 20% -0.6 0.64 ± 50% perf-profile.children.cycles-pp.xas_store
0.78 ± 15% -0.4 0.36 ± 52% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers
0.73 ± 16% -0.4 0.33 ± 52% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head
0.50 ± 39% -0.4 0.15 ± 53% perf-profile.children.cycles-pp.unaccount_page_cache_page
0.42 ± 24% -0.3 0.14 ± 93% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.76 ± 19% -0.2 0.53 ± 6% perf-profile.children.cycles-pp.lapic_next_deadline
0.32 ± 22% -0.2 0.14 ± 69% perf-profile.children.cycles-pp.free_unref_page_commit
0.25 ± 25% -0.2 0.08 ±102% perf-profile.children.cycles-pp.uncharge_page
0.19 ± 47% -0.1 0.07 ± 68% perf-profile.children.cycles-pp.__might_sleep
0.16 ± 28% -0.1 0.05 ±112% perf-profile.children.cycles-pp.__libc_fork
0.15 ± 22% -0.1 0.04 ±102% perf-profile.children.cycles-pp.xas_find
0.13 ± 12% -0.1 0.08 ± 34% perf-profile.children.cycles-pp.leave_mm
0.04 ± 71% +0.0 0.07 ± 14% perf-profile.children.cycles-pp.irq_work_needs_cpu
0.02 ±141% +0.1 0.09 ± 29% perf-profile.children.cycles-pp.__msecs_to_jiffies
0.04 ± 71% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.cpuidle_not_available
0.09 ± 75% +0.1 0.18 ± 23% perf-profile.children.cycles-pp.__schedule
0.00 +0.1 0.10 ± 40% perf-profile.children.cycles-pp.link_path_walk
0.09 ± 41% +0.1 0.20 ± 19% perf-profile.children.cycles-pp.pm_qos_read_value
0.04 ± 71% +0.1 0.14 ± 52% perf-profile.children.cycles-pp.raise_softirq
0.04 ±141% +0.1 0.15 ± 33% perf-profile.children.cycles-pp.__test_set_page_writeback
0.00 +0.1 0.12 ± 17% perf-profile.children.cycles-pp.__delete_from_page_cache
0.05 ±141% +0.1 0.17 ± 60% perf-profile.children.cycles-pp.exit_mmap
0.19 ± 54% +0.1 0.32 ± 21% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.05 ±141% +0.1 0.18 ± 64% perf-profile.children.cycles-pp.mmput
0.08 ± 30% +0.2 0.24 ± 33% perf-profile.children.cycles-pp.run_local_timers
0.05 ±141% +0.2 0.21 ± 41% perf-profile.children.cycles-pp.ext4_bio_write_page
0.14 ± 32% +0.2 0.30 ± 46% perf-profile.children.cycles-pp.update_rq_clock
0.15 ± 71% +0.2 0.33 ± 17% perf-profile.children.cycles-pp.rcu_idle_exit
0.38 ± 29% +0.2 0.56 ± 16% perf-profile.children.cycles-pp.sched_clock_cpu
0.14 ± 78% +0.2 0.33 ± 41% perf-profile.children.cycles-pp.mpage_map_and_submit_buffers
0.00 +0.2 0.20 ± 52% perf-profile.children.cycles-pp.invalidate_inode_pages2_range
0.07 ± 97% +0.2 0.30 ± 58% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.07 ±141% +0.2 0.31 ± 37% perf-profile.children.cycles-pp.mpage_submit_page
0.38 ± 21% +0.3 0.64 ± 14% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.08 ±101% +0.3 0.34 ± 51% perf-profile.children.cycles-pp.do_group_exit
0.08 ±101% +0.3 0.34 ± 51% perf-profile.children.cycles-pp.do_exit
0.23 ± 29% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.do_writepages
0.23 ± 29% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.ext4_writepages
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.wb_workfn
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.wb_writeback
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.__writeback_inodes_wb
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.writeback_sb_inodes
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.__writeback_single_inode
0.00 +0.3 0.32 ± 49% perf-profile.children.cycles-pp.ext4_file_read_iter
0.00 +0.3 0.32 ± 49% perf-profile.children.cycles-pp.iomap_dio_rw
1.30 ± 4% +0.4 1.72 ± 14% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.74 ± 41% +0.5 1.25 ± 33% perf-profile.children.cycles-pp.scheduler_tick
75.40 +4.7 80.05 ± 4% perf-profile.children.cycles-pp.cpuidle_enter
75.35 +4.7 80.02 ± 4% perf-profile.children.cycles-pp.cpuidle_enter_state
81.28 +5.9 87.18 ± 4% perf-profile.children.cycles-pp.do_idle
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.children.cycles-pp.start_secondary
1.16 ± 22% -0.6 0.56 ± 63% perf-profile.self.cycles-pp.free_pcppages_bulk
0.73 ± 16% -0.4 0.33 ± 52% perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head
0.46 ± 21% -0.3 0.16 ± 36% perf-profile.self.cycles-pp.xas_store
0.70 ± 5% -0.3 0.45 ± 38% perf-profile.self.cycles-pp.release_pages
0.29 ± 17% -0.2 0.07 ± 66% perf-profile.self.cycles-pp.free_unref_page_commit
0.32 ± 26% -0.2 0.13 ± 86% perf-profile.self.cycles-pp.workingset_update_node
0.18 ± 14% -0.1 0.06 ±100% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.11 -0.1 0.05 ± 60% perf-profile.self.cycles-pp.leave_mm
0.04 ± 71% +0.0 0.07 ± 14% perf-profile.self.cycles-pp.irq_work_needs_cpu
0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.__msecs_to_jiffies
0.04 ± 71% +0.1 0.11 ± 26% perf-profile.self.cycles-pp.cpuidle_not_available
0.08 ± 81% +0.1 0.18 ± 23% perf-profile.self.cycles-pp.rcu_idle_exit
0.09 ± 41% +0.1 0.20 ± 19% perf-profile.self.cycles-pp.pm_qos_read_value
0.04 ± 71% +0.1 0.14 ± 52% perf-profile.self.cycles-pp.raise_softirq
0.04 ±141% +0.1 0.17 ± 41% perf-profile.self.cycles-pp.calc_global_load_tick
0.16 ± 12% +0.2 0.32 ± 27% perf-profile.self.cycles-pp.run_timer_softirq
0.08 ± 81% +0.2 0.26 ± 46% perf-profile.self.cycles-pp.update_rq_clock
0.17 ± 49% +0.2 0.36 ± 22% perf-profile.self.cycles-pp.load_balance
0.26 ± 8% +0.2 0.47 ± 40% perf-profile.self.cycles-pp.hrtimer_interrupt
filebench.time.file_system_inputs
7.6e+06 +-+---------------------------------------------------------------+
7.4e+06 +-++ .+ +.+ |
|: ++ + +.+.+.+.++. .+ : : +.+.+. +.|
7.2e+06 +-+ : : + + .++ + : : .+ + + + |
7e+06 +-+ : : + + + +: +.+ + :+ + |
6.8e+06 +-+ + + + + +.+ : |
6.6e+06 +-+ +: |
| + |
6.4e+06 +-+ |
6.2e+06 +-+ |
6e+06 +-+ |
5.8e+06 O-+ O O O O |
| O OO O O O O O OO O O O OO O O OO O O O O |
5.6e+06 +-+ O O |
5.4e+06 +-+---------------------------------------------------------------+
filebench.sum_operations
3700 +-+------------------------------------------------------------------+
3600 +-++ .+. +.+ |
|: +. + +.+.+.+.++. .+ : : +.+.+. +.|
3500 +-+ + : + + .+.+ + : : : + + : |
3400 +-+ :: + + :+ : :.+.+ + + + : |
| + + + + + +.+ + |
3300 +-+ :+ |
3200 +-+ + |
3100 +-+ |
| |
3000 +-+ |
2900 +-+ |
O O O O O O O O |
2800 +-O O OO O O O O O O O O O O O OO O O O O |
2700 +-+---------------------------------------------O--------------------+
filebench.sum_operations_s
62 +-+--------------------------------------------------------------------+
| + .+ .+ |
60 +-++ + +.+.+.+.+. .+ + : +. .+. +.|
58 +-+ +.+ : +.+ + .+.+ + : : + + + + : |
|: + : + + :+ : :.+.+ + :+ : |
56 +-+ + + + + + +.+ + |
54 +-+ + + |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ O |
46 O-O O O O O O O O O O O O O O O OO O O O O O O O O O |
| O O |
44 +-+--------------------------------------------------------------------+
filebench.sum_reads_s
62 +-+--------------------------------------------------------------------+
| + |
60 +-++ .++. .+. +.+ +. .+. .|
58 +-+ +.+ + +.+ +.+.+.+. .+.+ + : : + + + + + |
|: + + + + :+ : : .+ + :+ + |
56 +-+ + + + +.+ + +.+ + |
| : : |
54 +-+ : : |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ |
O O O O O O O O O O |
46 +-O-O-O-O---OO-O-----O---O-O-O-O-O-OO-O-O-O-O-----O--------------------+
filebench.sum_bytes_mb_s
62 +-+--------------------------------------------------------------------+
| + |
60 +-++ +.++.+.+.+.+. +.+ +. .+ +.|
58 +-+ +.+ : +.+.+. .+. : : + + + : |
|: + : + +. +. : : .+.+ +. +. : |
56 +-+ + + + + + +.+ + |
54 +-+ + + |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ O |
46 O-O O O O O O O O O O O O O O O O O O O O O O O O |
| O O O O |
44 +-+--------------------------------------------------------------------+
filebench.sum_time_ms_op
110 +-+-O------O----------------------O--------------O--------------------+
| O O OO O O O O O OO O O O O O O O OO O O O O |
105 O-+ O |
100 +-+ |
| |
95 +-+ |
| .+. |
90 +-+ + .+. .+ + .+. .++ + |
|: : : + .+.+.+ + + : +.+. + + + |
85 +-+ +. : +. +.+ :+ +. : +.+. + +.|
80 +-++ + +. .+. + : : + + |
| + + + : : |
75 +-+ : : |
| : |
70 +-+-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
7 months, 1 week
- 5
- 10
[f2fs] fe1897eaa6: aim7.jobs-per-min -60.9% regression
by kernel test robot
Greeting,
FYI, we noticed a -60.9% regression of aim7.jobs-per-min due to commit:
commit: fe1897eaa6646f5a64a4cee0e6473ed9887d324b ("f2fs: fix to update time in lazytime mode")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: aim7
on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory
with following parameters:
disk: 4BRD_12G
md: RAID1
fs: f2fs
test: sync_disk_rw
load: 200
cpufreq_governor: performance
ucode: 0x42e
test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system.
test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/4BRD_12G/f2fs/x86_64-rhel-7.6/200/RAID1/debian-x86_64-2019-11-14.cgz/lkp-ivb-2ep1/sync_disk_rw/aim7/0x42e
commit:
b145b0eb20 (" ARM and x86 bugfixes of all kinds. The most visible one is that migrating")
fe1897eaa6 ("f2fs: fix to update time in lazytime mode")
b145b0eb2031a620 fe1897eaa6646f5a64a4cee0e64
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
1208 -60.9% 473.12 aim7.jobs-per-min
992.70 +155.6% 2537 aim7.time.elapsed_time
992.70 +155.6% 2537 aim7.time.elapsed_time.max
1.078e+08 +60.0% 1.724e+08 aim7.time.file_system_outputs
9646301 -4.2% 9238717 aim7.time.involuntary_context_switches
24107 ± 3% +58.2% 38130 ± 17% aim7.time.minor_page_faults
3295 +202.3% 9963 aim7.time.system_time
26.75 +51.4% 40.49 aim7.time.user_time
5.02e+08 +210.5% 1.559e+09 aim7.time.voluntary_context_switches
9294452 ± 4% -15.7% 7835497 ± 16% numa-numastat.node0.local_node
9298417 ± 4% -15.7% 7840991 ± 16% numa-numastat.node0.numa_hit
3.66 ± 3% +4.2 7.82 ± 18% mpstat.cpu.all.idle%
0.01 ± 32% -0.0 0.00 ± 31% mpstat.cpu.all.soft%
0.06 -0.0 0.04 mpstat.cpu.all.usr%
3.85 ± 3% +105.0% 7.89 ± 17% iostat.cpu.idle
89.80 -4.9% 85.36 iostat.cpu.iowait
6.29 +6.6% 6.70 iostat.cpu.system
23968 -48.4% 12356 iostat.md0.w/s
53780 -37.1% 33824 iostat.md0.wkB/s
89.00 -4.8% 84.75 vmstat.cpu.wa
53779 -37.1% 33833 vmstat.io.bo
29311046 -28.6% 20935254 vmstat.memory.free
1026388 +19.4% 1225080 vmstat.system.cs
103289 +8.5% 112038 vmstat.system.in
24013 ± 29% -58.5% 9976 ±120% numa-meminfo.node0.Dirty
119019 ± 10% +44.6% 172139 ± 13% numa-meminfo.node1.Active
3503 ± 56% +464.8% 19786 ± 38% numa-meminfo.node1.Active(file)
40477 ± 16% -38.5% 24896 ± 47% numa-meminfo.node1.Dirty
19966008 ± 2% -29.5% 14084594 ± 44% numa-meminfo.node1.MemFree
13014643 ± 4% +45.2% 18896052 ± 32% numa-meminfo.node1.MemUsed
12018 ± 41% +184.8% 34228 ± 35% numa-meminfo.node1.Shmem
2.09e+10 +127.4% 4.752e+10 ± 4% cpuidle.C1.time
2.558e+08 +140.1% 6.143e+08 ± 4% cpuidle.C1.usage
3.017e+09 ± 10% +301.2% 1.211e+10 ± 9% cpuidle.C1E.time
18225961 ± 9% +430.1% 96624314 ± 6% cpuidle.C1E.usage
4.627e+08 ± 14% +147.5% 1.145e+09 ± 20% cpuidle.C6.time
802166 ± 17% +186.9% 2301023 ± 26% cpuidle.C6.usage
1.799e+10 +148.9% 4.476e+10 cpuidle.POLL.time
2.616e+08 ± 2% +246.4% 9.063e+08 ± 2% cpuidle.POLL.usage
6017 ± 29% -58.5% 2496 ±120% numa-vmstat.node0.nr_dirty
5742 ± 29% -65.0% 2011 ±120% numa-vmstat.node0.nr_zone_write_pending
874.50 ± 56% +465.6% 4946 ± 38% numa-vmstat.node1.nr_active_file
10139 ± 16% -38.6% 6224 ± 47% numa-vmstat.node1.nr_dirty
4992144 ± 2% -29.5% 3521200 ± 44% numa-vmstat.node1.nr_free_pages
3003 ± 41% +184.9% 8556 ± 35% numa-vmstat.node1.nr_shmem
874.50 ± 56% +465.6% 4946 ± 38% numa-vmstat.node1.nr_zone_active_file
9775 ± 16% -47.9% 5089 ± 47% numa-vmstat.node1.nr_zone_write_pending
2.558e+08 +140.1% 6.143e+08 ± 4% turbostat.C1
43.77 -4.8 38.98 ± 3% turbostat.C1%
18225642 ± 9% +430.2% 96624037 ± 6% turbostat.C1E
6.31 ± 10% +3.6 9.94 ± 9% turbostat.C1E%
795117 ± 17% +188.4% 2292823 ± 26% turbostat.C6
0.38 ± 42% -63.6% 0.14 ± 13% turbostat.CPU%c6
1.028e+08 +176.7% 2.844e+08 turbostat.IRQ
0.23 ± 33% -69.1% 0.07 ± 22% turbostat.Pkg%pc2
275064 +12.3% 308846 ± 4% meminfo.Active
8036 ± 21% +261.7% 29072 ± 52% meminfo.Active(file)
64400 -45.9% 34850 meminfo.Dirty
16481 +10.8% 18264 meminfo.Inactive(anon)
147067 +12.8% 165858 ± 2% meminfo.KReclaimable
29120417 -28.8% 20740075 meminfo.MemAvailable
29335165 -28.6% 20936804 meminfo.MemFree
36511133 +23.0% 44909491 meminfo.Memused
147067 +12.8% 165858 ± 2% meminfo.SReclaimable
36567 ± 6% +37.2% 50173 meminfo.Shmem
51580 -60.8% 20220 meminfo.max_used_kB
4.27 ± 7% -13.9% 3.67 ± 4% perf-stat.i.MPKI
0.71 ± 3% -0.0 0.67 perf-stat.i.branch-miss-rate%
17.93 +3.5 21.41 ± 2% perf-stat.i.cache-miss-rate%
1029337 +19.1% 1226025 perf-stat.i.context-switches
11421 ± 4% +41.1% 16120 ± 6% perf-stat.i.cpu-migrations
0.42 ± 2% +0.0 0.45 ± 6% perf-stat.i.dTLB-load-miss-rate%
0.11 ± 27% -0.0 0.08 ± 3% perf-stat.i.dTLB-store-miss-rate%
84.04 ± 2% -2.3 81.75 ± 2% perf-stat.i.iTLB-load-miss-rate%
7375571 +7.3% 7917282 perf-stat.i.iTLB-load-misses
3.91 ± 7% -9.2% 3.56 ± 4% perf-stat.overall.MPKI
17.90 +4.1 21.96 ± 3% perf-stat.overall.cache-miss-rate%
0.41 ± 2% +0.0 0.45 ± 6% perf-stat.overall.dTLB-load-miss-rate%
0.10 ± 27% -0.0 0.08 ± 3% perf-stat.overall.dTLB-store-miss-rate%
83.99 ± 2% -2.3 81.72 ± 2% perf-stat.overall.iTLB-load-miss-rate%
1028292 +19.2% 1225532 perf-stat.ps.context-switches
11410 ± 4% +41.2% 16114 ± 6% perf-stat.ps.cpu-migrations
7368087 +7.4% 7914119 perf-stat.ps.iTLB-load-misses
2.558e+13 ± 2% +157.0% 6.575e+13 ± 3% perf-stat.total.instructions
9128 ± 2% -25.3% 6821 ± 4% slabinfo.dmaengine-unmap-16.active_objs
9175 ± 2% -24.4% 6938 ± 4% slabinfo.dmaengine-unmap-16.num_objs
28309 +12.5% 31840 slabinfo.f2fs_extent_node.active_objs
28704 +13.0% 32429 slabinfo.f2fs_extent_node.num_objs
964.00 ± 5% -13.3% 836.00 ± 6% slabinfo.kmem_cache_node.active_objs
1008 ± 5% -12.7% 880.00 ± 6% slabinfo.kmem_cache_node.num_objs
13017 -33.1% 8707 slabinfo.nat_entry.active_objs
13036 -33.2% 8707 slabinfo.nat_entry.num_objs
433.75 ± 9% +32.4% 574.50 ± 2% slabinfo.nfs_read_data.active_objs
433.75 ± 9% +32.4% 574.50 ± 2% slabinfo.nfs_read_data.num_objs
697.50 ± 3% +48.6% 1036 ± 33% slabinfo.numa_policy.active_objs
697.50 ± 3% +48.6% 1036 ± 33% slabinfo.numa_policy.num_objs
153098 +22.3% 187296 slabinfo.radix_tree_node.active_objs
2733 +22.3% 3344 slabinfo.radix_tree_node.active_slabs
153098 +22.3% 187296 slabinfo.radix_tree_node.num_objs
2733 +22.3% 3344 slabinfo.radix_tree_node.num_slabs
542.00 ± 10% +29.5% 702.00 ± 6% slabinfo.xfrm_state.active_objs
542.00 ± 10% +29.5% 702.00 ± 6% slabinfo.xfrm_state.num_objs
66757 +4.8% 69942 proc-vmstat.nr_active_anon
2009 ± 21% +261.7% 7267 ± 52% proc-vmstat.nr_active_file
96.00 +3.9% 99.75 proc-vmstat.nr_anon_transparent_hugepages
13466177 +60.0% 21542226 proc-vmstat.nr_dirtied
16154 -46.0% 8719 proc-vmstat.nr_dirty
726643 -28.8% 517541 proc-vmstat.nr_dirty_background_threshold
1455064 -28.8% 1036348 proc-vmstat.nr_dirty_threshold
311402 +2.4% 318957 proc-vmstat.nr_file_pages
7332853 -28.6% 5234441 proc-vmstat.nr_free_pages
4120 +10.8% 4565 proc-vmstat.nr_inactive_anon
13939 -1.2% 13772 proc-vmstat.nr_kernel_stack
2845 -1.5% 2803 proc-vmstat.nr_page_table_pages
9142 ± 6% +37.2% 12542 proc-vmstat.nr_shmem
36767 +12.8% 41463 ± 2% proc-vmstat.nr_slab_reclaimable
13387539 +60.3% 21463781 proc-vmstat.nr_written
66757 +4.8% 69942 proc-vmstat.nr_zone_active_anon
2009 ± 21% +261.7% 7267 ± 52% proc-vmstat.nr_zone_active_file
4120 +10.8% 4565 proc-vmstat.nr_zone_inactive_anon
15494 -54.2% 7089 proc-vmstat.nr_zone_write_pending
582146 ±102% +628.1% 4238850 ± 6% proc-vmstat.numa_foreign
13506 ± 3% +114.5% 28974 ± 31% proc-vmstat.numa_hint_faults
8913 ± 6% +148.6% 22154 ± 35% proc-vmstat.numa_hint_faults_local
582146 ±102% +628.1% 4238850 ± 6% proc-vmstat.numa_miss
597885 ± 99% +611.6% 4254585 ± 6% proc-vmstat.numa_other
1014174 ± 20% -46.1% 547135 ± 57% proc-vmstat.pgalloc_dma32
16157005 +22.4% 19769138 proc-vmstat.pgalloc_normal
2516271 +152.8% 6360808 proc-vmstat.pgfault
4866589 +64.4% 8002519 proc-vmstat.pgfree
53553588 +60.3% 85856803 proc-vmstat.pgpgout
1.55 ± 4% +0.1 1.65 perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback
1.58 ± 4% +0.1 1.69 perf-profile.calltrace.cycles-pp.schedule.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback
1.60 ± 4% +0.1 1.71 perf-profile.calltrace.cycles-pp.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback.f2fs_do_sync_file
0.75 ± 2% +0.2 0.91 ± 2% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.70 ± 4% -0.3 0.36 ± 7% perf-profile.children.cycles-pp.__generic_file_write_iter
0.69 ± 4% -0.3 0.35 ± 7% perf-profile.children.cycles-pp.generic_perform_write
0.57 ± 5% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
0.58 ± 5% -0.3 0.26 ± 7% perf-profile.children.cycles-pp.file_write_and_wait_range
0.57 ± 6% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.do_writepages
0.57 ± 5% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.f2fs_write_data_pages
0.64 ± 5% -0.3 0.34 ± 6% perf-profile.children.cycles-pp.f2fs_write_begin
0.57 ± 5% -0.2 0.32 ± 7% perf-profile.children.cycles-pp.f2fs_put_page
0.31 ± 10% -0.2 0.11 ± 7% perf-profile.children.cycles-pp.blk_finish_plug
0.31 ± 10% -0.2 0.11 ± 7% perf-profile.children.cycles-pp.blk_flush_plug_list
0.30 ± 11% -0.2 0.11 ± 6% perf-profile.children.cycles-pp.raid1_unplug
0.26 -0.1 0.14 ± 12% perf-profile.children.cycles-pp.f2fs_write_cache_pages
0.22 ± 20% -0.1 0.11 ± 41% perf-profile.children.cycles-pp.brd_do_bvec
0.14 ± 12% -0.1 0.03 ±100% perf-profile.children.cycles-pp.submit_bio_wait
0.21 ± 10% -0.1 0.11 ± 10% perf-profile.children.cycles-pp.submit_bio
0.23 ± 17% -0.1 0.13 ± 31% perf-profile.children.cycles-pp.worker_thread
0.20 ± 3% -0.1 0.11 ± 14% perf-profile.children.cycles-pp.__write_data_page
0.14 ± 15% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.__submit_flush_wait
0.17 ± 12% -0.1 0.09 ± 12% perf-profile.children.cycles-pp.md_make_request
0.16 ± 11% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.md_handle_request
0.15 ± 9% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.raid1_make_request
0.14 ± 17% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.f2fs_issue_flush
0.14 ± 6% -0.1 0.08 ± 19% perf-profile.children.cycles-pp.f2fs_do_write_data_page
0.10 ± 33% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.reschedule_interrupt
0.09 ± 37% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.scheduler_ipi
0.06 ± 6% +0.0 0.08 ± 10% perf-profile.children.cycles-pp.up_read
0.10 ± 4% +0.0 0.12 ± 13% perf-profile.children.cycles-pp.pick_next_task_idle
0.20 ± 2% +0.0 0.23 ± 5% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.16 ± 6% +0.0 0.19 ± 4% perf-profile.children.cycles-pp.irq_exit
0.01 ±173% +0.1 0.07 ± 7% perf-profile.children.cycles-pp.put_prev_task_fair
0.27 ± 3% +0.1 0.33 ± 4% perf-profile.children.cycles-pp.tick_nohz_next_event
0.35 ± 3% +0.1 0.42 ± 3% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.60 ± 4% +0.1 1.72 perf-profile.children.cycles-pp.io_schedule
0.78 ± 2% +0.1 0.92 ± 2% perf-profile.children.cycles-pp.menu_select
0.06 ± 6% +0.0 0.08 ± 10% perf-profile.self.cycles-pp.up_read
0.08 ± 5% +0.0 0.11 ± 11% perf-profile.self.cycles-pp.cpuidle_enter_state
0.35 ± 5% +0.0 0.39 ± 5% perf-profile.self.cycles-pp.f2fs_wait_on_node_pages_writeback
0.35 ± 4% +0.1 0.41 ± 3% perf-profile.self.cycles-pp.menu_select
8096 ± 28% +278.9% 30679 ± 20% sched_debug.cfs_rq:/.MIN_vruntime.avg
158062 ± 17% +299.5% 631442 ± 8% sched_debug.cfs_rq:/.MIN_vruntime.max
32629 ± 22% +289.7% 127148 ± 13% sched_debug.cfs_rq:/.MIN_vruntime.stddev
44053 +192.3% 128774 sched_debug.cfs_rq:/.exec_clock.avg
140530 ± 9% +377.8% 671471 ± 29% sched_debug.cfs_rq:/.exec_clock.max
33108 ± 2% +109.9% 69484 ± 2% sched_debug.cfs_rq:/.exec_clock.min
20291 ± 7% +386.1% 98642 ± 21% sched_debug.cfs_rq:/.exec_clock.stddev
0.25 ± 82% -88.4% 0.03 ±131% sched_debug.cfs_rq:/.load_avg.min
8096 ± 28% +278.9% 30679 ± 20% sched_debug.cfs_rq:/.max_vruntime.avg
158062 ± 17% +299.5% 631442 ± 8% sched_debug.cfs_rq:/.max_vruntime.max
32629 ± 22% +289.7% 127148 ± 13% sched_debug.cfs_rq:/.max_vruntime.stddev
184170 +218.6% 586859 ± 2% sched_debug.cfs_rq:/.min_vruntime.avg
251516 ± 4% +254.5% 891702 ± 11% sched_debug.cfs_rq:/.min_vruntime.max
165614 +131.3% 383025 ± 2% sched_debug.cfs_rq:/.min_vruntime.min
15286 ± 9% +1057.3% 176903 ± 10% sched_debug.cfs_rq:/.min_vruntime.stddev
0.67 ± 6% +45.5% 0.98 ± 9% sched_debug.cfs_rq:/.nr_spread_over.avg
0.74 ± 20% +32.4% 0.98 ± 13% sched_debug.cfs_rq:/.nr_spread_over.stddev
3.43 ± 29% -60.7% 1.35 ±100% sched_debug.cfs_rq:/.removed.load_avg.avg
75.10 ± 34% -84.1% 11.91 ±100% sched_debug.cfs_rq:/.removed.load_avg.max
15.24 ± 23% -75.5% 3.74 ±100% sched_debug.cfs_rq:/.removed.load_avg.stddev
157.58 ± 29% -60.5% 62.19 ±100% sched_debug.cfs_rq:/.removed.runnable_sum.avg
3459 ± 34% -84.0% 552.75 ± 99% sched_debug.cfs_rq:/.removed.runnable_sum.max
700.79 ± 23% -75.4% 172.66 ±100% sched_debug.cfs_rq:/.removed.runnable_sum.stddev
34.81 ± 42% -87.0% 4.53 ±100% sched_debug.cfs_rq:/.removed.util_avg.max
5.89 ± 31% -79.0% 1.23 ±100% sched_debug.cfs_rq:/.removed.util_avg.stddev
59642 ± 58% +506.4% 361657 ± 61% sched_debug.cfs_rq:/.spread0.max
15286 ± 9% +1057.2% 176904 ± 10% sched_debug.cfs_rq:/.spread0.stddev
15.81 ± 15% +28.7% 20.34 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.avg
553.04 ± 15% +42.8% 789.60 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.max
83.37 ± 15% +37.7% 114.78 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev
130721 -17.9% 107295 sched_debug.cpu.avg_idle.avg
32650 ± 7% -16.9% 27145 ± 6% sched_debug.cpu.avg_idle.min
509212 +151.7% 1281639 sched_debug.cpu.clock.avg
509215 +151.7% 1281642 sched_debug.cpu.clock.max
509209 +151.7% 1281636 sched_debug.cpu.clock.min
509212 +151.7% 1281639 sched_debug.cpu.clock_task.avg
509215 +151.7% 1281642 sched_debug.cpu.clock_task.max
509209 +151.7% 1281636 sched_debug.cpu.clock_task.min
480.69 ± 2% +89.2% 909.42 ± 3% sched_debug.cpu.curr->pid.avg
14037 +135.4% 33040 sched_debug.cpu.curr->pid.max
2146 ± 2% +125.6% 4843 sched_debug.cpu.curr->pid.stddev
0.12 ± 5% +10.2% 0.13 ± 3% sched_debug.cpu.nr_running.avg
10174129 +208.2% 31360192 sched_debug.cpu.nr_switches.avg
10849045 +290.1% 42321541 ± 3% sched_debug.cpu.nr_switches.max
564008 ± 15% +1719.4% 10261545 ± 8% sched_debug.cpu.nr_switches.stddev
-87.16 +262.4% -315.83 sched_debug.cpu.nr_uninterruptible.min
66.84 ± 14% +143.1% 162.48 ± 21% sched_debug.cpu.nr_uninterruptible.stddev
10172460 +208.3% 31358619 sched_debug.cpu.sched_count.avg
10847113 +290.1% 42319816 ± 3% sched_debug.cpu.sched_count.max
563945 ± 15% +1719.6% 10261603 ± 8% sched_debug.cpu.sched_count.stddev
4957584 +211.1% 15425344 sched_debug.cpu.sched_goidle.avg
5286410 +291.7% 20709361 ± 3% sched_debug.cpu.sched_goidle.max
272603 ± 14% +1717.5% 4954528 ± 7% sched_debug.cpu.sched_goidle.stddev
5120024 +209.4% 15841794 sched_debug.cpu.ttwu_count.avg
58965381 ± 14% +567.2% 3.934e+08 ± 33% sched_debug.cpu.ttwu_count.max
132694 ± 2% -41.0% 78274 ± 5% sched_debug.cpu.ttwu_count.min
11266172 ± 8% +482.6% 65635267 ± 21% sched_debug.cpu.ttwu_count.stddev
114955 +26.1% 144930 ± 3% sched_debug.cpu.ttwu_local.max
83595 ± 4% -53.0% 39284 ± 30% sched_debug.cpu.ttwu_local.min
5967 ± 16% +460.9% 33468 ± 9% sched_debug.cpu.ttwu_local.stddev
509209 +151.7% 1281636 sched_debug.cpu_clk
506629 +152.5% 1279056 sched_debug.ktime
509627 +151.6% 1282053 sched_debug.sched_clk
18876 +138.7% 45051 ± 4% softirqs.CPU0.RCU
131251 ± 5% +153.2% 332324 ± 8% softirqs.CPU0.SCHED
327888 ± 3% +181.5% 923135 ± 10% softirqs.CPU0.TIMER
19003 ± 4% +121.7% 42132 ± 12% softirqs.CPU1.RCU
125587 ± 4% +130.3% 289267 ± 22% softirqs.CPU1.SCHED
331057 ± 2% +205.9% 1012636 ± 18% softirqs.CPU1.TIMER
19567 ± 5% +137.8% 46538 ± 2% softirqs.CPU10.RCU
131681 +161.3% 344047 softirqs.CPU10.SCHED
318774 +177.5% 884714 ± 8% softirqs.CPU10.TIMER
17876 ± 5% +158.8% 46258 softirqs.CPU11.RCU
128299 ± 4% +167.5% 343196 softirqs.CPU11.SCHED
320847 ± 3% +166.4% 854607 ± 8% softirqs.CPU11.TIMER
19129 ± 4% +121.1% 42292 ± 4% softirqs.CPU12.RCU
135251 +172.0% 367942 softirqs.CPU12.SCHED
310663 +201.8% 937514 ± 24% softirqs.CPU12.TIMER
17993 +143.9% 43882 ± 3% softirqs.CPU13.RCU
137747 ± 4% +157.5% 354715 ± 4% softirqs.CPU13.SCHED
359507 ± 23% +126.2% 813048 ± 13% softirqs.CPU13.TIMER
18173 +141.9% 43961 ± 3% softirqs.CPU14.RCU
134564 +163.8% 355019 ± 4% softirqs.CPU14.SCHED
310599 ± 2% +160.3% 808494 ± 13% softirqs.CPU14.TIMER
18180 +139.4% 43524 ± 3% softirqs.CPU15.RCU
134993 +160.8% 352061 ± 6% softirqs.CPU15.SCHED
308373 +164.7% 816152 ± 12% softirqs.CPU15.TIMER
19918 +144.6% 48716 softirqs.CPU16.RCU
134609 +162.6% 353418 ± 4% softirqs.CPU16.SCHED
307916 +163.0% 809772 ± 13% softirqs.CPU16.TIMER
19672 ± 2% +148.8% 48938 ± 3% softirqs.CPU17.RCU
132926 ± 2% +165.9% 353452 ± 4% softirqs.CPU17.SCHED
314316 +157.0% 807880 ± 13% softirqs.CPU17.TIMER
20021 ± 3% +141.2% 48298 ± 2% softirqs.CPU18.RCU
133347 ± 2% +164.0% 352038 ± 5% softirqs.CPU18.SCHED
312605 +158.6% 808358 ± 13% softirqs.CPU18.TIMER
19458 +139.3% 46570 ± 6% softirqs.CPU19.RCU
134001 +150.1% 335170 ± 14% softirqs.CPU19.SCHED
310521 +171.0% 841639 ± 14% softirqs.CPU19.TIMER
19131 ± 3% +138.3% 45586 ± 3% softirqs.CPU2.RCU
129111 ± 4% +154.4% 328419 ± 9% softirqs.CPU2.SCHED
329064 ± 3% +172.7% 897239 ± 10% softirqs.CPU2.TIMER
19249 ± 2% +145.5% 47257 ± 3% softirqs.CPU20.RCU
132532 +167.8% 354983 ± 3% softirqs.CPU20.SCHED
310659 +163.4% 818268 ± 13% softirqs.CPU20.TIMER
19473 +142.2% 47163 ± 4% softirqs.CPU21.RCU
133856 +166.0% 356005 ± 4% softirqs.CPU21.SCHED
308960 +164.3% 816637 ± 13% softirqs.CPU21.TIMER
19216 ± 2% +146.1% 47292 ± 3% softirqs.CPU22.RCU
133039 +165.4% 353140 ± 4% softirqs.CPU22.SCHED
307621 +166.0% 818260 ± 13% softirqs.CPU22.TIMER
19244 ± 2% +145.9% 47314 ± 2% softirqs.CPU23.RCU
133897 +163.1% 352326 ± 5% softirqs.CPU23.SCHED
310036 ± 2% +161.6% 811061 ± 13% softirqs.CPU23.TIMER
18954 ± 4% +143.9% 46235 ± 4% softirqs.CPU24.RCU
128754 ± 3% +157.8% 331927 ± 7% softirqs.CPU24.SCHED
323118 +169.7% 871609 ± 8% softirqs.CPU24.TIMER
18942 ± 6% +136.0% 44695 ± 3% softirqs.CPU25.RCU
129580 ± 3% +162.2% 339786 ± 3% softirqs.CPU25.SCHED
328907 +168.0% 881566 ± 7% softirqs.CPU25.TIMER
18913 ± 2% +147.2% 46757 ± 2% softirqs.CPU26.RCU
124714 ± 2% +176.4% 344692 softirqs.CPU26.SCHED
328463 ± 4% +170.2% 887463 ± 8% softirqs.CPU26.TIMER
19015 +137.2% 45107 ± 5% softirqs.CPU27.RCU
128551 ± 6% +152.0% 323916 ± 13% softirqs.CPU27.SCHED
325384 ± 2% +193.3% 954266 ± 13% softirqs.CPU27.TIMER
19094 +149.9% 47725 ± 2% softirqs.CPU28.RCU
131563 +161.9% 344579 ± 2% softirqs.CPU28.SCHED
319151 +176.7% 883105 ± 9% softirqs.CPU28.TIMER
19050 ± 4% +145.4% 46756 ± 2% softirqs.CPU29.RCU
124211 ± 6% +176.0% 342763 ± 2% softirqs.CPU29.SCHED
329526 ± 4% +158.9% 853085 ± 8% softirqs.CPU29.TIMER
22721 ± 8% +121.7% 50370 ± 4% softirqs.CPU3.RCU
132516 ± 2% +156.9% 340453 ± 4% softirqs.CPU3.SCHED
323552 ± 2% +174.3% 887612 ± 9% softirqs.CPU3.TIMER
19911 ± 3% +137.4% 47276 softirqs.CPU30.RCU
130778 +161.4% 341879 softirqs.CPU30.SCHED
317264 +181.0% 891407 ± 8% softirqs.CPU30.TIMER
19232 ± 4% +144.4% 47012 softirqs.CPU31.RCU
126109 ± 10% +174.9% 346628 ± 2% softirqs.CPU31.SCHED
329439 ± 4% +169.4% 887674 ± 9% softirqs.CPU31.TIMER
18139 ± 4% +141.4% 43795 ± 4% softirqs.CPU32.RCU
125566 ± 4% +170.9% 340112 ± 4% softirqs.CPU32.SCHED
328833 +184.0% 934027 ± 9% softirqs.CPU32.TIMER
19309 ± 10% +128.3% 44077 ± 3% softirqs.CPU33.RCU
128438 ± 3% +167.7% 343879 ± 2% softirqs.CPU33.SCHED
320949 +172.2% 873631 ± 9% softirqs.CPU33.TIMER
18041 ± 4% +146.1% 44399 ± 2% softirqs.CPU34.RCU
129791 +165.0% 343914 ± 2% softirqs.CPU34.SCHED
319403 ± 2% +177.3% 885711 ± 8% softirqs.CPU34.TIMER
17991 ± 5% +146.2% 44290 ± 2% softirqs.CPU35.RCU
137112 ± 7% +148.6% 340824 ± 3% softirqs.CPU35.SCHED
416552 ± 21% +119.5% 914435 ± 8% softirqs.CPU35.TIMER
20036 ± 4% +138.7% 47823 ± 4% softirqs.CPU36.RCU
138048 ± 3% +165.7% 366854 ± 7% softirqs.CPU36.SCHED
359442 ± 22% +196.1% 1064343 ± 19% softirqs.CPU36.TIMER
19421 ± 5% +153.4% 49203 ± 2% softirqs.CPU37.RCU
135462 +158.7% 350500 ± 6% softirqs.CPU37.SCHED
305531 +166.8% 815287 ± 12% softirqs.CPU37.TIMER
19963 +150.5% 50014 softirqs.CPU38.RCU
132257 +167.5% 353848 ± 5% softirqs.CPU38.SCHED
311979 ± 2% +160.3% 812118 ± 12% softirqs.CPU38.TIMER
20119 ± 2% +143.6% 49011 ± 2% softirqs.CPU39.RCU
135450 +161.9% 354713 ± 4% softirqs.CPU39.SCHED
307797 +164.3% 813377 ± 12% softirqs.CPU39.TIMER
20593 ± 9% +132.1% 47796 ± 5% softirqs.CPU4.RCU
129881 +159.2% 336643 ± 3% softirqs.CPU4.SCHED
321940 +175.0% 885297 ± 8% softirqs.CPU4.TIMER
19874 ± 2% +149.6% 49611 softirqs.CPU40.RCU
134660 +161.9% 352636 ± 5% softirqs.CPU40.SCHED
309870 +160.6% 807374 ± 13% softirqs.CPU40.TIMER
19897 ± 2% +147.2% 49186 ± 2% softirqs.CPU41.RCU
131978 ± 2% +167.4% 352864 ± 5% softirqs.CPU41.SCHED
315198 +157.6% 812013 ± 12% softirqs.CPU41.TIMER
19988 ± 2% +143.9% 48749 ± 2% softirqs.CPU42.RCU
133326 ± 2% +165.6% 354135 ± 4% softirqs.CPU42.SCHED
313042 +159.5% 812298 ± 12% softirqs.CPU42.TIMER
19723 ± 2% +141.9% 47704 ± 4% softirqs.CPU43.RCU
131944 +165.7% 350566 ± 5% softirqs.CPU43.SCHED
314428 +162.9% 826595 ± 13% softirqs.CPU43.TIMER
19824 ± 2% +143.0% 48177 ± 4% softirqs.CPU44.RCU
132047 ± 2% +157.8% 340362 ± 12% softirqs.CPU44.SCHED
312615 ± 2% +165.2% 829094 ± 12% softirqs.CPU44.TIMER
20140 ± 2% +138.6% 48049 ± 7% softirqs.CPU45.RCU
133846 +151.7% 336901 ± 14% softirqs.CPU45.SCHED
309086 ± 2% +170.8% 837094 ± 12% softirqs.CPU45.TIMER
20103 ± 3% +138.8% 48009 ± 3% softirqs.CPU46.RCU
133860 +154.1% 340108 ± 11% softirqs.CPU46.SCHED
311323 +166.1% 828498 ± 13% softirqs.CPU46.TIMER
19628 ± 2% +149.0% 48881 ± 2% softirqs.CPU47.RCU
132835 ± 2% +164.6% 351487 ± 5% softirqs.CPU47.SCHED
309264 +162.8% 812666 ± 13% softirqs.CPU47.TIMER
19773 ± 2% +139.0% 47263 ± 2% softirqs.CPU5.RCU
130891 ± 2% +163.7% 345185 softirqs.CPU5.SCHED
323979 ± 2% +182.2% 914392 ± 8% softirqs.CPU5.TIMER
19904 ± 3% +141.4% 48045 ± 4% softirqs.CPU6.RCU
131836 +160.9% 344009 softirqs.CPU6.SCHED
316150 +181.7% 890614 ± 8% softirqs.CPU6.TIMER
19077 ± 3% +145.1% 46751 ± 3% softirqs.CPU7.RCU
127906 ± 2% +157.6% 329548 ± 5% softirqs.CPU7.SCHED
325087 +181.6% 915494 ± 9% softirqs.CPU7.TIMER
20183 ± 2% +114.9% 43380 ± 10% softirqs.CPU8.RCU
131974 +118.3% 288124 ± 30% softirqs.CPU8.SCHED
319626 +190.1% 927161 ± 13% softirqs.CPU8.TIMER
19292 ± 3% +137.6% 45839 ± 3% softirqs.CPU9.RCU
129715 +158.9% 335788 ± 4% softirqs.CPU9.SCHED
321209 +178.0% 893035 ± 8% softirqs.CPU9.TIMER
931061 +140.9% 2242787 softirqs.RCU
6322319 +160.8% 16487173 softirqs.SCHED
15417545 +169.7% 41581783 ± 8% softirqs.TIMER
853.75 ± 12% +139.9% 2048 ± 7% interrupts.35:PCI-MSI.2621441-edge.eth0-TxRx-0
586.00 ± 14% +130.1% 1348 ± 5% interrupts.36:PCI-MSI.2621442-edge.eth0-TxRx-1
651.00 ± 9% +136.3% 1538 ± 9% interrupts.37:PCI-MSI.2621443-edge.eth0-TxRx-2
565.25 ± 14% +129.7% 1298 interrupts.38:PCI-MSI.2621444-edge.eth0-TxRx-3
591.00 ± 14% +123.8% 1322 ± 2% interrupts.39:PCI-MSI.2621445-edge.eth0-TxRx-4
660.25 ± 13% +111.4% 1396 ± 7% interrupts.40:PCI-MSI.2621446-edge.eth0-TxRx-5
529.50 ± 4% +227.1% 1732 ± 14% interrupts.41:PCI-MSI.2621447-edge.eth0-TxRx-6
585.75 ± 16% +163.2% 1541 ± 10% interrupts.42:PCI-MSI.2621448-edge.eth0-TxRx-7
404968 +145.6% 994665 ± 3% interrupts.CAL:Function_call_interrupts
8396 ± 6% +133.7% 19627 ± 6% interrupts.CPU0.CAL:Function_call_interrupts
1988743 +155.3% 5077644 interrupts.CPU0.LOC:Local_timer_interrupts
8698 ± 5% +121.4% 19255 ± 2% interrupts.CPU1.CAL:Function_call_interrupts
1986989 +155.5% 5077206 interrupts.CPU1.LOC:Local_timer_interrupts
8487 ± 5% +132.3% 19713 ± 8% interrupts.CPU10.CAL:Function_call_interrupts
1988375 +155.4% 5077322 interrupts.CPU10.LOC:Local_timer_interrupts
8770 ± 5% +149.6% 21889 ± 3% interrupts.CPU11.CAL:Function_call_interrupts
1987964 +155.4% 5077484 interrupts.CPU11.LOC:Local_timer_interrupts
78678 ± 8% +541.4% 504605 ±122% interrupts.CPU11.RES:Rescheduling_interrupts
8731 ± 9% +119.5% 19167 ± 6% interrupts.CPU12.CAL:Function_call_interrupts
1988222 +155.4% 5077524 interrupts.CPU12.LOC:Local_timer_interrupts
7960 ± 3% +153.5% 20176 ± 9% interrupts.CPU13.CAL:Function_call_interrupts
1988667 +155.3% 5076359 interrupts.CPU13.LOC:Local_timer_interrupts
8518 ± 6% +155.0% 21723 ± 7% interrupts.CPU14.CAL:Function_call_interrupts
1987350 +155.5% 5077482 interrupts.CPU14.LOC:Local_timer_interrupts
37.00 ± 31% +124.3% 83.00 ± 24% interrupts.CPU14.TLB:TLB_shootdowns
8831 ± 4% +137.4% 20964 ± 10% interrupts.CPU15.CAL:Function_call_interrupts
1987501 +155.5% 5077129 interrupts.CPU15.LOC:Local_timer_interrupts
3708 ± 27% -63.0% 1370 ±103% interrupts.CPU15.NMI:Non-maskable_interrupts
3708 ± 27% -63.0% 1370 ±103% interrupts.CPU15.PMI:Performance_monitoring_interrupts
23.25 ± 24% +253.8% 82.25 ± 28% interrupts.CPU15.TLB:TLB_shootdowns
8708 ± 8% +132.7% 20268 ± 7% interrupts.CPU16.CAL:Function_call_interrupts
1988418 +155.3% 5076923 interrupts.CPU16.LOC:Local_timer_interrupts
8548 ± 7% +148.3% 21226 ± 7% interrupts.CPU17.CAL:Function_call_interrupts
1988416 +155.3% 5077317 interrupts.CPU17.LOC:Local_timer_interrupts
33.00 ± 18% +147.0% 81.50 ± 21% interrupts.CPU17.TLB:TLB_shootdowns
8456 ± 9% +157.0% 21731 ± 6% interrupts.CPU18.CAL:Function_call_interrupts
1988574 +155.3% 5077430 interrupts.CPU18.LOC:Local_timer_interrupts
30.25 ± 15% +181.8% 85.25 ± 25% interrupts.CPU18.TLB:TLB_shootdowns
8884 ± 6% +144.5% 21720 ± 3% interrupts.CPU19.CAL:Function_call_interrupts
1987747 +155.4% 5077260 interrupts.CPU19.LOC:Local_timer_interrupts
37.00 ± 19% +127.0% 84.00 ± 26% interrupts.CPU19.TLB:TLB_shootdowns
8298 ± 11% +145.3% 20352 ± 9% interrupts.CPU2.CAL:Function_call_interrupts
1988446 +155.4% 5077740 interrupts.CPU2.LOC:Local_timer_interrupts
8594 ± 5% +146.1% 21149 ± 5% interrupts.CPU20.CAL:Function_call_interrupts
1987855 +155.4% 5077226 interrupts.CPU20.LOC:Local_timer_interrupts
8376 ± 8% +151.7% 21084 ± 9% interrupts.CPU21.CAL:Function_call_interrupts
1988171 +155.4% 5077212 interrupts.CPU21.LOC:Local_timer_interrupts
41.25 ± 20% +106.1% 85.00 ± 27% interrupts.CPU21.TLB:TLB_shootdowns
9024 ± 2% +141.1% 21757 ± 3% interrupts.CPU22.CAL:Function_call_interrupts
1987613 +155.4% 5077125 interrupts.CPU22.LOC:Local_timer_interrupts
8095 ± 7% +154.9% 20633 ± 6% interrupts.CPU23.CAL:Function_call_interrupts
1989315 +155.2% 5077451 interrupts.CPU23.LOC:Local_timer_interrupts
853.75 ± 12% +139.9% 2048 ± 7% interrupts.CPU24.35:PCI-MSI.2621441-edge.eth0-TxRx-0
8463 ± 5% +147.0% 20901 ± 7% interrupts.CPU24.CAL:Function_call_interrupts
1988251 +155.4% 5077879 interrupts.CPU24.LOC:Local_timer_interrupts
74848 ± 8% +525.2% 467985 ±128% interrupts.CPU24.RES:Rescheduling_interrupts
586.00 ± 14% +130.1% 1348 ± 5% interrupts.CPU25.36:PCI-MSI.2621442-edge.eth0-TxRx-1
8156 ± 7% +170.0% 22019 ± 2% interrupts.CPU25.CAL:Function_call_interrupts
1988295 +155.3% 5077010 interrupts.CPU25.LOC:Local_timer_interrupts
930.75 ±123% +260.9% 3359 ± 15% interrupts.CPU25.NMI:Non-maskable_interrupts
930.75 ±123% +260.9% 3359 ± 15% interrupts.CPU25.PMI:Performance_monitoring_interrupts
72678 ± 7% +575.9% 491267 ±120% interrupts.CPU25.RES:Rescheduling_interrupts
651.00 ± 9% +136.3% 1538 ± 9% interrupts.CPU26.37:PCI-MSI.2621443-edge.eth0-TxRx-2
8538 ± 6% +141.5% 20625 ± 6% interrupts.CPU26.CAL:Function_call_interrupts
1988053 +155.4% 5077342 interrupts.CPU26.LOC:Local_timer_interrupts
75367 ± 8% +522.0% 468794 ±133% interrupts.CPU26.RES:Rescheduling_interrupts
565.25 ± 14% +129.7% 1298 interrupts.CPU27.38:PCI-MSI.2621444-edge.eth0-TxRx-3
8157 ± 5% +132.5% 18969 ± 3% interrupts.CPU27.CAL:Function_call_interrupts
1988437 +155.3% 5077428 interrupts.CPU27.LOC:Local_timer_interrupts
591.00 ± 14% +123.8% 1322 ± 2% interrupts.CPU28.39:PCI-MSI.2621445-edge.eth0-TxRx-4
8181 ± 6% +148.1% 20300 ± 3% interrupts.CPU28.CAL:Function_call_interrupts
1987807 +155.4% 5077738 interrupts.CPU28.LOC:Local_timer_interrupts
71071 ± 7% +581.0% 483987 ±128% interrupts.CPU28.RES:Rescheduling_interrupts
660.25 ± 13% +111.4% 1396 ± 7% interrupts.CPU29.40:PCI-MSI.2621446-edge.eth0-TxRx-5
8178 ± 5% +150.4% 20475 ± 7% interrupts.CPU29.CAL:Function_call_interrupts
1987852 +155.4% 5077382 interrupts.CPU29.LOC:Local_timer_interrupts
70205 ± 8% +585.5% 481286 ±127% interrupts.CPU29.RES:Rescheduling_interrupts
17.75 ± 36% +163.4% 46.75 ± 80% interrupts.CPU29.TLB:TLB_shootdowns
8684 ± 5% +151.7% 21860 ± 4% interrupts.CPU3.CAL:Function_call_interrupts
1988786 +155.3% 5077436 interrupts.CPU3.LOC:Local_timer_interrupts
80407 ± 4% +518.6% 497419 ±122% interrupts.CPU3.RES:Rescheduling_interrupts
529.50 ± 4% +227.1% 1732 ± 14% interrupts.CPU30.41:PCI-MSI.2621447-edge.eth0-TxRx-6
8415 ± 4% +138.0% 20031 ± 3% interrupts.CPU30.CAL:Function_call_interrupts
1988609 +155.3% 5077494 interrupts.CPU30.LOC:Local_timer_interrupts
74875 ± 8% +518.8% 463337 ±134% interrupts.CPU30.RES:Rescheduling_interrupts
585.75 ± 16% +163.2% 1541 ± 10% interrupts.CPU31.42:PCI-MSI.2621448-edge.eth0-TxRx-7
7917 +151.5% 19913 ± 3% interrupts.CPU31.CAL:Function_call_interrupts
1988677 +155.3% 5077502 interrupts.CPU31.LOC:Local_timer_interrupts
67596 ± 3% +596.5% 470844 ±134% interrupts.CPU31.RES:Rescheduling_interrupts
8230 ± 4% +161.9% 21551 ± 8% interrupts.CPU32.CAL:Function_call_interrupts
1988317 +155.4% 5077410 interrupts.CPU32.LOC:Local_timer_interrupts
70449 ± 5% +597.2% 491174 ±127% interrupts.CPU32.RES:Rescheduling_interrupts
8684 ± 6% +137.3% 20610 ± 6% interrupts.CPU33.CAL:Function_call_interrupts
1988440 +155.3% 5077281 interrupts.CPU33.LOC:Local_timer_interrupts
77401 ± 5% +495.8% 461151 ±134% interrupts.CPU33.RES:Rescheduling_interrupts
8412 ± 8% +150.3% 21061 ± 4% interrupts.CPU34.CAL:Function_call_interrupts
1988447 +155.4% 5077546 interrupts.CPU34.LOC:Local_timer_interrupts
75351 ± 8% +537.5% 480376 ±128% interrupts.CPU34.RES:Rescheduling_interrupts
7890 +147.7% 19543 ± 5% interrupts.CPU35.CAL:Function_call_interrupts
1988875 +155.3% 5077717 interrupts.CPU35.LOC:Local_timer_interrupts
70158 ± 2% +577.1% 475011 ±135% interrupts.CPU35.RES:Rescheduling_interrupts
7908 ± 2% +184.9% 22530 ± 3% interrupts.CPU36.CAL:Function_call_interrupts
1988403 +155.4% 5077694 interrupts.CPU36.LOC:Local_timer_interrupts
31.75 ± 20% +178.0% 88.25 ± 28% interrupts.CPU36.TLB:TLB_shootdowns
9078 ± 2% +133.8% 21229 ± 6% interrupts.CPU37.CAL:Function_call_interrupts
1986971 +155.5% 5076871 interrupts.CPU37.LOC:Local_timer_interrupts
34.75 ± 36% +154.7% 88.50 ± 25% interrupts.CPU37.TLB:TLB_shootdowns
8574 ± 7% +131.7% 19864 ± 3% interrupts.CPU38.CAL:Function_call_interrupts
1986780 +155.5% 5076769 interrupts.CPU38.LOC:Local_timer_interrupts
29.25 ± 35% +195.7% 86.50 ± 18% interrupts.CPU38.TLB:TLB_shootdowns
8184 ± 7% +158.0% 21112 ± 4% interrupts.CPU39.CAL:Function_call_interrupts
1986972 +155.5% 5077337 interrupts.CPU39.LOC:Local_timer_interrupts
8368 ± 8% +144.7% 20481 ± 12% interrupts.CPU4.CAL:Function_call_interrupts
1987774 +155.4% 5077506 interrupts.CPU4.LOC:Local_timer_interrupts
8165 ± 3% +163.7% 21528 ± 6% interrupts.CPU40.CAL:Function_call_interrupts
1987659 +155.4% 5077421 interrupts.CPU40.LOC:Local_timer_interrupts
32.00 ± 12% +154.7% 81.50 ± 29% interrupts.CPU40.TLB:TLB_shootdowns
8454 ± 6% +158.4% 21847 ± 8% interrupts.CPU41.CAL:Function_call_interrupts
1987504 +155.5% 5077467 interrupts.CPU41.LOC:Local_timer_interrupts
37.75 ± 45% +122.5% 84.00 ± 17% interrupts.CPU41.TLB:TLB_shootdowns
8529 ± 7% +136.4% 20164 ± 8% interrupts.CPU42.CAL:Function_call_interrupts
1988401 +155.4% 5077407 interrupts.CPU42.LOC:Local_timer_interrupts
32.75 ± 39% +171.8% 89.00 ± 37% interrupts.CPU42.TLB:TLB_shootdowns
8102 ± 5% +150.4% 20283 ± 9% interrupts.CPU43.CAL:Function_call_interrupts
1987414 +155.5% 5077903 interrupts.CPU43.LOC:Local_timer_interrupts
32.50 ± 18% +150.8% 81.50 ± 34% interrupts.CPU43.TLB:TLB_shootdowns
8206 ± 7% +157.3% 21112 ± 8% interrupts.CPU44.CAL:Function_call_interrupts
1987278 +155.5% 5077309 interrupts.CPU44.LOC:Local_timer_interrupts
37.00 ± 25% +119.6% 81.25 ± 24% interrupts.CPU44.TLB:TLB_shootdowns
8467 ± 7% +139.3% 20261 ± 5% interrupts.CPU45.CAL:Function_call_interrupts
1987858 +155.4% 5077234 interrupts.CPU45.LOC:Local_timer_interrupts
38.25 ± 22% +113.1% 81.50 ± 27% interrupts.CPU45.TLB:TLB_shootdowns
7773 ± 4% +162.0% 20367 ± 9% interrupts.CPU46.CAL:Function_call_interrupts
1988584 +155.3% 5077417 interrupts.CPU46.LOC:Local_timer_interrupts
37.50 ± 14% +142.7% 91.00 ± 34% interrupts.CPU46.TLB:TLB_shootdowns
8746 ± 4% +138.7% 20880 ± 8% interrupts.CPU47.CAL:Function_call_interrupts
1988495 +155.3% 5077204 interrupts.CPU47.LOC:Local_timer_interrupts
8722 ± 7% +134.1% 20414 ± 10% interrupts.CPU5.CAL:Function_call_interrupts
1987875 +155.4% 5077418 interrupts.CPU5.LOC:Local_timer_interrupts
8435 ± 8% +143.9% 20570 ± 11% interrupts.CPU6.CAL:Function_call_interrupts
1988389 +155.4% 5077630 interrupts.CPU6.LOC:Local_timer_interrupts
8837 ± 5% +139.1% 21133 ± 9% interrupts.CPU7.CAL:Function_call_interrupts
1987853 +155.4% 5077370 interrupts.CPU7.LOC:Local_timer_interrupts
8852 ± 5% +123.8% 19810 ± 3% interrupts.CPU8.CAL:Function_call_interrupts
1988702 +155.3% 5077569 interrupts.CPU8.LOC:Local_timer_interrupts
8273 ± 6% +151.1% 20771 ± 9% interrupts.CPU9.CAL:Function_call_interrupts
1988033 +155.4% 5077588 interrupts.CPU9.LOC:Local_timer_interrupts
70739 ± 6% +578.9% 480243 ±129% interrupts.CPU9.RES:Rescheduling_interrupts
95428175 +155.4% 2.437e+08 interrupts.LOC:Local_timer_interrupts
144.00 +133.3% 336.00 interrupts.MCP:Machine_check_polls
6773889 ± 8% +482.9% 39483682 ± 7% interrupts.RES:Rescheduling_interrupts
1544 ± 5% +91.5% 2957 ± 11% interrupts.TLB:TLB_shootdowns
aim7.jobs-per-min
1300 +-+------------------------------------------------------------------+
1200 +-+.++.+.+.+.+. +.+.+.+. +.+.+ +.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| + + + |
1100 +-+ |
1000 +-+ |
| |
900 +-+ |
800 +-+ |
700 +-+ |
| |
600 +-+ |
500 +-+ O |
| O O O O O O O |
400 O-O OO O O O O OO O O O OO O O O O OO O O O O |
300 +-+------------------------------------------------------------------+
aim7.time.user_time
55 +-+--------------------------------------------------------------------+
| |
50 +-+ O |
| O |
| |
45 +-+ |
| O O O O O O O O O O |
40 O-O O OO O O O O O O O O O OO O O O O O |
| |
35 +-+ |
| |
| |
30 +-+ |
|.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.++.+. .+.+.+.+.++.+.+.|
25 +-+--------------------------------------------------------------------+
aim7.time.system_time
11000 +-+-----------------------------------------------------------------+
O O OO O O O OO O O OO O O O OO O O O OO O O |
10000 +-+ O OO O O OO O |
9000 +-+ |
| |
8000 +-+ |
| |
7000 +-+ |
| |
6000 +-+ |
5000 +-+ |
| |
4000 +-+ |
| .++. .+. .+.+. .+.+. .++. .+ .+.+.+ .+. .++.+.|
3000 +-+-----------------------------------------------------------------+
aim7.time.elapsed_time
3500 +-+------------------------------------------------------------------+
O O OO O O O O OO O O OO O O O O OO O O O O |
3000 +-+ O |
| |
| O O O O O O |
2500 +-+ OO |
| |
2000 +-+ |
| |
1500 +-+ |
| |
| |
1000 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| |
500 +-+------------------------------------------------------------------+
aim7.time.elapsed_time.max
3500 +-+------------------------------------------------------------------+
O O OO O O O O OO O O OO O O O O OO O O O O |
3000 +-+ O |
| |
| O O O O O O |
2500 +-+ OO |
| |
2000 +-+ |
| |
1500 +-+ |
| |
| |
1000 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| |
500 +-+------------------------------------------------------------------+
aim7.time.voluntary_context_switches
1.8e+09 +-+---------------------------------------------------------------+
O O O O OO O OO O O OO O O OO O OO O O |
1.6e+09 +-O O O O O O OO O O O |
| |
1.4e+09 +-+ |
| |
1.2e+09 +-+ |
| |
1e+09 +-+ |
| |
8e+08 +-+ |
| |
6e+08 +-+ |
|.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|
4e+08 +-+---------------------------------------------------------------+
aim7.time.file_system_outputs
1.8e+08 +-+---------------------------------------------------------------+
O OO O O OO O O OO O O OO O O OO O OO O O OO O O OO O O O |
1.7e+08 +-+ |
1.6e+08 +-+ |
| |
1.5e+08 +-+ |
| |
1.4e+08 +-+ |
| |
1.3e+08 +-+ |
1.2e+08 +-+ |
| |
1.1e+08 +-+ |
|.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|
1e+08 +-+---------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
7 months, 2 weeks
- 3
- 3
[btrfs] 8d47a0d8f7: fio.write_bw_MBps -28.6% regression
by kernel test robot
Greeting,
FYI, we noticed a -28.6% regression of fio.write_bw_MBps due to commit:
commit: 8d47a0d8f7947422dd359ac8e462687f81a7a137 ("btrfs: Do mandatory tree block check before submitting bio")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: fio-basic
on test machine: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory
with following parameters:
disk: 2pmem
fs: btrfs
runtime: 200s
nr_task: 50%
time_based: tb
rw: randwrite
bs: 4k
ioengine: libaio
test_size: 100G
cpufreq_governor: performance
ucode: 0x3d
test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
4k/gcc-7/performance/2pmem/btrfs/libaio/x86_64-rhel-7.6/50%/debian-x86_64-2018-04-03.cgz/200s/randwrite/lkp-hsw-ep2/100G/fio-basic/tb/0x3d
commit:
ff2ac107fa ("btrfs: tree-checker: Remove comprehensive root owner check")
8d47a0d8f7 ("btrfs: Do mandatory tree block check before submitting bio")
ff2ac107fae2440b 8d47a0d8f7947422dd359ac8e46
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.BUG:unable_to_handle_kernel
:4 25% 1:4 dmesg.Kernel_panic-not_syncing:Fatal_exception_in_interrupt
:4 25% 1:4 dmesg.Oops:#[##]
:4 25% 1:4 dmesg.RIP:cpuidle_enter_state
:4 25% 1:4 dmesg.RIP:native_write_msr
:4 25% 1:4 dmesg.RIP:perf_prepare_sample
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
0.01 ± 21% +0.3 0.28 ± 20% fio.latency_1000ms%
4.14 ± 13% +1.8 5.95 ± 9% fio.latency_10ms%
8.73 ± 15% +3.0 11.68 ± 8% fio.latency_20ms%
1.91 ± 5% -0.4 1.52 ± 3% fio.latency_250ms%
66.81 -5.9 60.90 ± 2% fio.latency_4ms%
0.18 ± 15% +0.1 0.29 ± 16% fio.latency_750us%
1.017e+08 -28.5% 72701525 fio.time.file_system_outputs
109.75 -26.8% 80.33 ± 2% fio.time.percent_of_cpu_this_job_got
197.97 -26.4% 145.79 ± 2% fio.time.system_time
38.66 ± 4% -24.0% 29.39 ± 3% fio.time.user_time
12388015 -28.5% 8858878 fio.time.voluntary_context_switches
12711059 -28.5% 9083954 fio.workload
247.91 -28.6% 176.96 fio.write_bw_MBps
309248 ± 2% +71.7% 531114 ± 4% fio.write_clat_99%_us
17574 +40.0% 24609 fio.write_clat_mean_us
57989 ± 4% +82.2% 105661 fio.write_clat_stddev
63463 -28.6% 45302 fio.write_iops
564.61 +40.2% 791.54 fio.write_slat_mean_us
9375 ± 7% +96.3% 18406 fio.write_slat_stddev
21.25 +8.5% 23.06 ± 9% boot-time.dhcp
93.32 +1.4% 94.58 iostat.cpu.idle
6.43 -18.6% 5.23 ± 2% iostat.cpu.system
6.48 -1.2 5.27 ± 2% mpstat.cpu.all.sys%
0.24 ± 5% -0.1 0.18 ± 2% mpstat.cpu.all.usr%
93.00 +1.1% 94.00 vmstat.cpu.id
591200 -30.0% 413921 vmstat.io.bo
31653776 ± 2% -10.9% 28196588 ± 2% vmstat.memory.cache
11927341 ± 7% +29.8% 15486805 ± 4% vmstat.memory.free
5.00 -26.7% 3.67 ± 12% vmstat.procs.r
635910 ± 2% -27.6% 460384 vmstat.system.cs
7822384 ± 2% -25.0% 5870498 numa-numastat.node0.local_node
7839861 ± 2% -24.8% 5893785 numa-numastat.node0.numa_hit
224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node0.numa_miss
242465 ± 44% -84.3% 37964 ± 18% numa-numastat.node0.other_node
7762623 ± 12% -26.8% 5681358 ± 6% numa-numastat.node1.local_node
224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node1.numa_foreign
7768618 ± 12% -26.9% 5681573 ± 6% numa-numastat.node1.numa_hit
9.429e+08 ± 10% -35.4% 6.094e+08 cpuidle.C1.time
53504154 ± 6% -31.8% 36504731 cpuidle.C1.usage
1.093e+09 ± 22% -50.3% 5.435e+08 ± 3% cpuidle.C1E.time
16541925 ± 13% -45.0% 9103775 cpuidle.C1E.usage
5.366e+09 ± 36% +71.7% 9.214e+09 ± 18% cpuidle.C3.time
16163664 ± 33% +57.0% 25379738 ± 5% cpuidle.C3.usage
9697055 ± 16% -35.8% 6225168 ± 19% cpuidle.C6.usage
19313595 ± 17% -16.3% 16166123 cpuidle.POLL.time
10291838 ± 9% -26.8% 7537045 ± 7% meminfo.Active
10034782 ± 9% -27.4% 7280710 ± 7% meminfo.Active(file)
31524905 ± 2% -10.9% 28082249 ± 2% meminfo.Cached
115157 ± 4% +61.1% 185560 ± 3% meminfo.CmaFree
6430720 ± 8% +19.1% 7660885 ± 6% meminfo.DirectMap2M
534230 ± 5% -12.8% 465932 ± 6% meminfo.Dirty
11737322 ± 7% +30.2% 15281189 ± 4% meminfo.MemFree
37604344 ± 2% -9.4% 34060483 meminfo.Memused
806.50 ± 37% +137.4% 1914 ± 8% meminfo.Writeback
370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.active_objs
370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.num_objs
15147 ± 5% -8.3% 13897 ± 2% slabinfo.blkdev_ioc.active_objs
17839 ± 6% -11.7% 15745 ± 2% slabinfo.blkdev_ioc.num_objs
12178 ± 4% -7.4% 11275 ± 2% slabinfo.kmalloc-512.num_objs
7794 ± 3% +25.4% 9774 slabinfo.proc_inode_cache.active_objs
8017 ± 2% +23.4% 9895 slabinfo.proc_inode_cache.num_objs
2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.active_objs
2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.num_objs
263.75 ± 2% -15.1% 224.00 ± 2% turbostat.Avg_MHz
9.46 ± 2% -1.4 8.07 ± 2% turbostat.Busy%
53502248 ± 6% -31.8% 36503898 turbostat.C1
5.86 ± 8% -2.0 3.82 turbostat.C1%
16540396 ± 13% -45.0% 9102733 turbostat.C1E
6.80 ± 22% -3.4 3.40 ± 4% turbostat.C1E%
16163332 ± 33% +57.0% 25379351 ± 5% turbostat.C3
33.55 ± 36% +24.1 57.61 ± 18% turbostat.C3%
9689445 ± 16% -35.8% 6221033 ± 19% turbostat.C6
18.04 ± 26% +49.1% 26.90 ± 9% turbostat.CPU%c3
53.25 ± 3% +10.8% 59.00 ± 4% turbostat.CoreTmp
4.67 ± 14% +62.7% 7.59 ± 26% turbostat.Pkg%pc2
56.50 +9.7% 62.00 ± 2% turbostat.PkgTmp
11.84 -4.8% 11.28 turbostat.RAMWatt
7232478 ± 10% -37.2% 4543037 ± 22% numa-meminfo.node0.Active
7072564 ± 10% -37.6% 4410135 ± 23% numa-meminfo.node0.Active(file)
308167 ± 6% -24.7% 232152 ± 10% numa-meminfo.node0.Dirty
8784028 ± 13% +21.7% 10686327 ± 2% numa-meminfo.node0.Inactive
8669548 ± 12% +20.7% 10465444 ± 2% numa-meminfo.node0.Inactive(file)
572.50 ± 22% +115.9% 1236 ± 25% numa-meminfo.node0.Writeback
15098140 ± 3% -17.7% 12425670 ± 3% numa-meminfo.node1.FilePages
11599549 ± 7% -22.3% 9018051 ± 4% numa-meminfo.node1.Inactive
105067 ± 92% -99.3% 725.33 ± 51% numa-meminfo.node1.Inactive(anon)
11494482 ± 7% -21.6% 9017325 ± 4% numa-meminfo.node1.Inactive(file)
5504 ± 9% -12.5% 4819 ± 2% numa-meminfo.node1.KernelStack
6709938 ± 11% +45.8% 9781880 ± 5% numa-meminfo.node1.MemFree
18029136 ± 4% -17.0% 14957200 ± 3% numa-meminfo.node1.MemUsed
4427 ± 69% -77.5% 996.00 ± 8% numa-meminfo.node1.PageTables
857660 ± 32% -37.2% 538646 ± 8% numa-meminfo.node1.SUnreclaim
109232 ± 88% -96.1% 4274 ± 10% numa-meminfo.node1.Shmem
986740 ± 35% -40.0% 592452 ± 10% numa-meminfo.node1.Slab
575.21 ± 15% +30.1% 748.58 ± 2% sched_debug.cfs_rq:/.runnable_load_avg.max
100.45 ± 15% +28.6% 129.19 ± 5% sched_debug.cfs_rq:/.runnable_load_avg.stddev
830.19 ± 9% +15.7% 960.25 ± 6% sched_debug.cfs_rq:/.util_avg.max
29.97 ± 16% -25.2% 22.43 ± 31% sched_debug.cfs_rq:/.util_est_enqueued.avg
591.17 ± 13% -22.2% 459.83 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.max
107.52 ± 13% -23.0% 82.84 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.stddev
2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock.stddev
2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock_task.stddev
24.51 ± 7% +21.5% 29.77 ± 8% sched_debug.cpu.cpu_load[0].avg
541.29 ± 15% +38.3% 748.58 ± 2% sched_debug.cpu.cpu_load[0].max
93.97 ± 7% +27.8% 120.09 ± 2% sched_debug.cpu.cpu_load[0].stddev
519.08 ± 6% +40.5% 729.42 ± 4% sched_debug.cpu.cpu_load[1].max
87.45 ± 5% +28.5% 112.39 ± 4% sched_debug.cpu.cpu_load[1].stddev
480.83 ± 7% +39.5% 670.92 ± 13% sched_debug.cpu.cpu_load[2].max
80.92 ± 5% +25.7% 101.75 ± 8% sched_debug.cpu.cpu_load[2].stddev
408.83 ± 4% +54.0% 629.58 ± 19% sched_debug.cpu.cpu_load[3].max
71.62 ± 2% +30.7% 93.61 ± 11% sched_debug.cpu.cpu_load[3].stddev
381.50 ± 8% +66.7% 636.00 ± 18% sched_debug.cpu.cpu_load[4].max
64.40 ± 6% +39.7% 89.96 ± 12% sched_debug.cpu.cpu_load[4].stddev
106.75 ± 20% -88.4% 12.33 ± 10% proc-vmstat.kswapd_high_wmark_hit_quickly
2508660 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_active_file
33315360 -30.4% 23192301 proc-vmstat.nr_dirtied
133447 ± 5% -12.6% 116580 ± 6% proc-vmstat.nr_dirty
7881043 ± 2% -10.9% 7019533 ± 2% proc-vmstat.nr_file_pages
28790 ± 4% +61.1% 46392 ± 3% proc-vmstat.nr_free_cma
2934503 ± 7% +30.2% 3821369 ± 4% proc-vmstat.nr_free_pages
5044168 -3.4% 4872965 proc-vmstat.nr_inactive_file
430282 -6.0% 404589 proc-vmstat.nr_slab_unreclaimable
197.75 ± 38% +136.5% 467.67 ± 10% proc-vmstat.nr_writeback
33243480 -30.4% 23143756 proc-vmstat.nr_written
2508665 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_zone_active_file
5044399 -3.4% 4873026 proc-vmstat.nr_zone_inactive_file
134972 ± 5% -12.4% 118262 ± 5% proc-vmstat.nr_zone_write_pending
15632301 ± 5% -25.8% 11598535 ± 3% proc-vmstat.numa_hit
15608826 ± 5% -25.8% 11575029 ± 3% proc-vmstat.numa_local
2300530 ± 10% -31.1% 1585363 ± 10% proc-vmstat.pgactivate
16955745 -26.3% 12499611 ± 2% proc-vmstat.pgalloc_normal
17488000 -25.9% 12959812 ± 2% proc-vmstat.pgfree
1.33e+08 -30.4% 92575440 proc-vmstat.pgpgout
5728177 ± 6% -78.5% 1232433 ± 25% proc-vmstat.pgscan_kswapd
5727967 ± 6% -78.5% 1232244 ± 25% proc-vmstat.pgsteal_kswapd
32760 -7.6% 30285 proc-vmstat.slabs_scanned
1768211 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_active_file
10410188 ± 7% -32.5% 7023939 ± 7% numa-vmstat.node0.nr_dirtied
76995 ± 6% -24.6% 58035 ± 10% numa-vmstat.node0.nr_dirty
2167457 ± 12% +20.7% 2616147 ± 2% numa-vmstat.node0.nr_inactive_file
142.00 ± 24% +120.4% 313.00 ± 24% numa-vmstat.node0.nr_writeback
10308758 ± 7% -32.6% 6949497 ± 7% numa-vmstat.node0.nr_written
1768216 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_zone_active_file
2167563 ± 12% +20.7% 2616189 ± 2% numa-vmstat.node0.nr_zone_inactive_file
77786 ± 7% -24.3% 58899 ± 10% numa-vmstat.node0.nr_zone_write_pending
5889240 ± 3% -14.3% 5046874 ± 3% numa-vmstat.node0.numa_hit
5837851 ± 4% -14.0% 5023120 ± 3% numa-vmstat.node0.numa_local
68376 ± 48% -96.9% 2149 ± 46% numa-vmstat.node0.numa_miss
119766 ± 42% -78.4% 25904 ± 3% numa-vmstat.node0.numa_other
30.50 ± 38% -100.0% 0.00 numa-vmstat.node0.workingset_nodes
6829126 ± 6% -22.8% 5269898 ± 10% numa-vmstat.node1.nr_dirtied
3774645 ± 3% -17.7% 3106081 ± 3% numa-vmstat.node1.nr_file_pages
28796 ± 4% +61.1% 46405 ± 3% numa-vmstat.node1.nr_free_cma
1677332 ± 11% +45.8% 2445778 ± 5% numa-vmstat.node1.nr_free_pages
26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_inactive_anon
2873702 ± 7% -21.6% 2254116 ± 4% numa-vmstat.node1.nr_inactive_file
5506 ± 9% -12.5% 4820 ± 2% numa-vmstat.node1.nr_kernel_stack
29031 ± 85% -88.2% 3436 numa-vmstat.node1.nr_mapped
1106 ± 69% -77.5% 249.00 ± 8% numa-vmstat.node1.nr_page_table_pages
27308 ± 88% -96.1% 1068 ± 10% numa-vmstat.node1.nr_shmem
214418 ± 32% -37.2% 134663 ± 8% numa-vmstat.node1.nr_slab_unreclaimable
75.00 ± 35% +145.3% 184.00 ± 18% numa-vmstat.node1.nr_writeback
6761224 ± 6% -23.1% 5202586 ± 10% numa-vmstat.node1.nr_written
26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_zone_inactive_anon
2873810 ± 7% -21.6% 2254135 ± 4% numa-vmstat.node1.nr_zone_inactive_file
68381 ± 48% -96.9% 2152 ± 46% numa-vmstat.node1.numa_foreign
5035283 ± 15% -26.9% 3679318 ± 7% numa-vmstat.node1.numa_hit
4905249 ± 15% -28.2% 3522229 ± 8% numa-vmstat.node1.numa_local
18.43 ± 6% -28.3% 13.21 ± 3% perf-stat.i.MPKI
2.205e+09 ± 3% +10.5% 2.437e+09 perf-stat.i.branch-instructions
1.83 ± 5% -0.6 1.24 ± 7% perf-stat.i.branch-miss-rate%
37019551 ± 3% -29.4% 26140844 ± 5% perf-stat.i.branch-misses
28932020 ± 6% -31.1% 19933819 ± 3% perf-stat.i.cache-misses
2.03e+08 ± 4% -24.8% 1.526e+08 perf-stat.i.cache-references
645248 ± 2% -27.4% 468301 perf-stat.i.context-switches
1.97 ± 7% -22.1% 1.53 ± 2% perf-stat.i.cpi
1.983e+10 ± 3% -15.1% 1.684e+10 perf-stat.i.cpu-cycles
1.03 ± 5% -0.4 0.64 ± 10% perf-stat.i.dTLB-load-miss-rate%
31879502 ± 6% -37.9% 19807598 ± 9% perf-stat.i.dTLB-load-misses
3.201e+09 ± 3% +18.8% 3.804e+09 perf-stat.i.dTLB-loads
0.13 ± 7% -0.0 0.09 ± 10% perf-stat.i.dTLB-store-miss-rate%
2516181 ± 10% -28.3% 1805272 ± 10% perf-stat.i.dTLB-store-misses
2.143e+09 ± 3% +19.8% 2.568e+09 ± 2% perf-stat.i.dTLB-stores
13781489 ± 3% -28.4% 9866239 ± 3% perf-stat.i.iTLB-loads
1.164e+10 ± 3% +21.6% 1.415e+10 perf-stat.i.instructions
8435 ± 17% +42.5% 12023 ± 14% perf-stat.i.instructions-per-iTLB-miss
0.57 ± 3% +43.6% 0.82 perf-stat.i.ipc
16050361 ± 9% -30.7% 11127721 ± 3% perf-stat.i.node-load-misses
8120610 ± 9% -31.3% 5580996 perf-stat.i.node-loads
2591649 ± 13% -31.1% 1785932 ± 6% perf-stat.i.node-store-misses
1809775 ± 12% -31.2% 1244360 ± 2% perf-stat.i.node-stores
17.45 ± 4% -38.2% 10.79 perf-stat.overall.MPKI
1.68 ± 3% -0.6 1.07 ± 5% perf-stat.overall.branch-miss-rate%
1.70 ± 3% -30.2% 1.19 perf-stat.overall.cpi
689.69 ± 9% +22.6% 845.48 ± 2% perf-stat.overall.cycles-between-cache-misses
0.99 ± 5% -0.5 0.52 ± 10% perf-stat.overall.dTLB-load-miss-rate%
0.12 ± 8% -0.0 0.07 ± 10% perf-stat.overall.dTLB-store-miss-rate%
10.81 ± 14% +2.3 13.09 ± 10% perf-stat.overall.iTLB-load-miss-rate%
7119 ± 14% +35.9% 9679 ± 13% perf-stat.overall.instructions-per-iTLB-miss
0.59 ± 3% +43.1% 0.84 perf-stat.overall.ipc
202912 ± 2% +68.8% 342615 ± 3% perf-stat.overall.path-length
2.195e+09 ± 3% +10.5% 2.425e+09 perf-stat.ps.branch-instructions
36847010 ± 3% -29.4% 26015686 ± 5% perf-stat.ps.branch-misses
28799092 ± 6% -31.1% 19837306 ± 3% perf-stat.ps.cache-misses
2.021e+08 ± 4% -24.8% 1.519e+08 perf-stat.ps.cache-references
642266 ± 2% -27.4% 465968 perf-stat.ps.context-switches
1.974e+10 ± 3% -15.1% 1.676e+10 perf-stat.ps.cpu-cycles
31729472 ± 6% -37.9% 19715594 ± 9% perf-stat.ps.dTLB-load-misses
3.186e+09 ± 3% +18.8% 3.785e+09 perf-stat.ps.dTLB-loads
2504443 ± 10% -28.2% 1797154 ± 10% perf-stat.ps.dTLB-store-misses
2.133e+09 ± 3% +19.8% 2.556e+09 ± 2% perf-stat.ps.dTLB-stores
13717699 ± 3% -28.4% 9816979 ± 3% perf-stat.ps.iTLB-loads
1.159e+10 ± 3% +21.5% 1.408e+10 perf-stat.ps.instructions
15979371 ± 9% -30.7% 11073468 ± 3% perf-stat.ps.node-load-misses
8081205 ± 9% -31.3% 5554114 perf-stat.ps.node-loads
2580198 ± 13% -31.1% 1777169 ± 6% perf-stat.ps.node-store-misses
1801590 ± 12% -31.3% 1238321 ± 2% perf-stat.ps.node-stores
2.579e+12 +20.6% 3.111e+12 ± 2% perf-stat.total.instructions
36149 ± 4% -26.2% 26686 ± 5% softirqs.CPU0.RCU
37255 ± 3% -26.8% 27256 ± 2% softirqs.CPU1.RCU
37834 ± 5% -26.5% 27801 ± 9% softirqs.CPU10.RCU
39075 ± 3% -24.5% 29494 ± 2% softirqs.CPU11.RCU
38137 ± 6% -23.1% 29340 ± 4% softirqs.CPU12.RCU
37480 ± 3% -20.5% 29797 ± 2% softirqs.CPU13.RCU
37690 ± 5% -28.7% 26861 ± 4% softirqs.CPU14.RCU
33863 ± 9% -29.5% 23885 ± 5% softirqs.CPU15.RCU
34306 ± 9% -31.1% 23641 ± 12% softirqs.CPU16.RCU
34374 ± 9% -29.0% 24402 ± 7% softirqs.CPU17.RCU
44013 ± 9% -31.7% 30059 softirqs.CPU18.RCU
43114 ± 8% -29.9% 30219 softirqs.CPU19.RCU
36276 ± 8% -23.3% 27825 ± 5% softirqs.CPU2.RCU
67049 ± 7% +39.1% 93267 ± 18% softirqs.CPU2.TIMER
39721 ± 13% -31.4% 27233 ± 3% softirqs.CPU20.RCU
42340 ± 5% -28.6% 30215 ± 6% softirqs.CPU21.RCU
40896 ± 7% -30.3% 28506 ± 2% softirqs.CPU22.RCU
42422 ± 7% -33.2% 28357 ± 2% softirqs.CPU23.RCU
42330 ± 9% -33.5% 28150 ± 5% softirqs.CPU24.RCU
41406 ± 9% -31.4% 28398 softirqs.CPU25.RCU
39754 ± 5% -29.4% 28063 softirqs.CPU26.RCU
40207 ± 6% -27.9% 28995 ± 12% softirqs.CPU27.RCU
40351 ± 9% -29.4% 28487 ± 3% softirqs.CPU28.RCU
41880 ± 5% -29.6% 29472 softirqs.CPU29.RCU
36293 ± 4% -17.7% 29864 ± 7% softirqs.CPU3.RCU
39341 ± 9% -34.3% 25847 softirqs.CPU30.RCU
39687 ± 9% -33.5% 26400 ± 2% softirqs.CPU31.RCU
38062 ± 14% -39.9% 22878 ± 10% softirqs.CPU32.RCU
39769 ± 13% -33.8% 26314 ± 5% softirqs.CPU33.RCU
39504 ± 10% -37.2% 24824 ± 6% softirqs.CPU34.RCU
39591 ± 11% -38.6% 24300 ± 5% softirqs.CPU35.RCU
40398 ± 8% -27.0% 29488 ± 12% softirqs.CPU36.RCU
39464 ± 12% -25.3% 29478 ± 10% softirqs.CPU37.RCU
41177 ± 9% -34.0% 27190 ± 19% softirqs.CPU38.RCU
38734 ± 13% -24.8% 29145 ± 5% softirqs.CPU39.RCU
36804 ± 5% -23.0% 28354 softirqs.CPU4.RCU
40421 ± 12% -28.8% 28776 ± 6% softirqs.CPU40.RCU
40371 ± 11% -31.1% 27834 ± 8% softirqs.CPU41.RCU
40633 ± 10% -29.9% 28469 ± 10% softirqs.CPU42.RCU
41018 ± 9% -33.6% 27252 ± 6% softirqs.CPU43.RCU
38181 ± 12% -27.0% 27856 ± 10% softirqs.CPU44.RCU
39605 ± 12% -32.3% 26803 ± 12% softirqs.CPU45.RCU
42686 ± 11% -29.7% 30016 ± 9% softirqs.CPU47.RCU
40433 ± 9% -31.8% 27586 ± 11% softirqs.CPU48.RCU
38371 ± 12% -23.0% 29528 ± 7% softirqs.CPU49.RCU
37172 ± 2% -25.9% 27562 ± 3% softirqs.CPU5.RCU
39660 ± 12% -27.5% 28752 ± 11% softirqs.CPU50.RCU
40375 ± 10% -26.7% 29582 ± 3% softirqs.CPU51.RCU
40568 ± 9% -28.8% 28886 ± 6% softirqs.CPU52.RCU
39967 ± 10% -27.3% 29073 ± 11% softirqs.CPU53.RCU
40874 ± 11% -37.1% 25725 ± 6% softirqs.CPU54.RCU
41227 ± 14% -37.9% 25606 ± 6% softirqs.CPU55.RCU
38856 ± 13% -34.7% 25380 ± 8% softirqs.CPU56.RCU
39618 ± 13% -39.1% 24119 ± 4% softirqs.CPU57.RCU
39412 ± 12% -36.6% 24970 ± 3% softirqs.CPU58.RCU
41572 ± 14% -41.3% 24399 ± 10% softirqs.CPU59.RCU
36927 ± 3% -24.4% 27923 ± 22% softirqs.CPU6.RCU
37057 ± 10% -31.8% 25261 ± 2% softirqs.CPU60.RCU
36584 ± 8% -27.8% 26419 ± 2% softirqs.CPU61.RCU
35859 ± 4% -28.2% 25729 ± 5% softirqs.CPU62.RCU
35849 ± 7% -33.2% 23933 ± 5% softirqs.CPU63.RCU
35379 ± 6% -26.4% 26041 ± 4% softirqs.CPU64.RCU
36908 ± 6% -26.5% 27110 ± 2% softirqs.CPU65.RCU
37595 ± 3% -26.1% 27783 ± 4% softirqs.CPU66.RCU
36327 ± 2% -27.0% 26533 ± 2% softirqs.CPU67.RCU
35136 ± 7% -31.7% 23998 ± 9% softirqs.CPU68.RCU
35638 ± 4% -28.6% 25435 ± 4% softirqs.CPU69.RCU
37268 ± 6% -25.4% 27786 ± 4% softirqs.CPU7.RCU
36693 ± 4% -29.2% 25986 ± 5% softirqs.CPU70.RCU
36859 ± 3% -28.8% 26260 ± 4% softirqs.CPU71.RCU
37539 ± 7% -27.7% 27144 ± 4% softirqs.CPU8.RCU
37584 ± 4% -26.0% 27802 ± 15% softirqs.CPU9.RCU
2795313 ± 7% -29.7% 1964363 ± 4% softirqs.RCU
249.50 ± 76% -50.4% 123.67 ± 2% interrupts.41:PCI-MSI.1572870-edge.eth0-TxRx-6
154.50 ± 13% -10.9% 137.67 ± 10% interrupts.42:PCI-MSI.1572871-edge.eth0-TxRx-7
150.50 ± 43% -28.0% 108.33 interrupts.84:PCI-MSI.1572913-edge.eth0-TxRx-49
10393 ± 10% -26.4% 7653 ± 10% interrupts.CPU1.RES:Rescheduling_interrupts
12518 ± 15% -31.0% 8642 ± 21% interrupts.CPU10.RES:Rescheduling_interrupts
945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.NMI:Non-maskable_interrupts
945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.PMI:Performance_monitoring_interrupts
13068 ± 18% -30.6% 9075 ± 23% interrupts.CPU11.RES:Rescheduling_interrupts
11949 ± 11% -49.8% 6001 ± 29% interrupts.CPU12.RES:Rescheduling_interrupts
13711 ± 8% -19.4% 11052 ± 24% interrupts.CPU13.RES:Rescheduling_interrupts
13361 ± 6% -38.8% 8170 ± 25% interrupts.CPU14.RES:Rescheduling_interrupts
7628 ± 12% -29.2% 5401 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts
7909 ± 25% -30.4% 5501 ± 37% interrupts.CPU19.RES:Rescheduling_interrupts
1482 ± 39% -100.0% 0.00 interrupts.CPU2.NMI:Non-maskable_interrupts
1482 ± 39% -100.0% 0.00 interrupts.CPU2.PMI:Performance_monitoring_interrupts
2116 +23.2% 2606 ± 10% interrupts.CPU20.CAL:Function_call_interrupts
194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.NMI:Non-maskable_interrupts
194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.PMI:Performance_monitoring_interrupts
2096 ± 4% +18.0% 2474 ± 9% interrupts.CPU21.CAL:Function_call_interrupts
11313 ± 20% -56.4% 4935 ± 23% interrupts.CPU21.RES:Rescheduling_interrupts
2111 ± 3% +20.1% 2534 ± 11% interrupts.CPU22.CAL:Function_call_interrupts
2109 ± 4% +22.6% 2585 ± 10% interrupts.CPU23.CAL:Function_call_interrupts
10305 ± 16% -40.7% 6115 ± 17% interrupts.CPU23.RES:Rescheduling_interrupts
2128 ± 2% +17.8% 2507 ± 10% interrupts.CPU24.CAL:Function_call_interrupts
12271 ± 20% -39.0% 7488 ± 29% interrupts.CPU24.RES:Rescheduling_interrupts
2123 +19.7% 2541 ± 7% interrupts.CPU25.CAL:Function_call_interrupts
9780 ± 21% -32.9% 6563 ± 25% interrupts.CPU25.RES:Rescheduling_interrupts
980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.NMI:Non-maskable_interrupts
980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.PMI:Performance_monitoring_interrupts
11163 ± 28% -49.8% 5605 ± 13% interrupts.CPU27.RES:Rescheduling_interrupts
11071 ± 12% -56.3% 4835 ± 26% interrupts.CPU29.RES:Rescheduling_interrupts
728.50 ± 66% -100.0% 0.00 interrupts.CPU3.NMI:Non-maskable_interrupts
728.50 ± 66% -100.0% 0.00 interrupts.CPU3.PMI:Performance_monitoring_interrupts
10599 ± 10% -38.7% 6501 ± 16% interrupts.CPU31.RES:Rescheduling_interrupts
2213 ± 4% +14.2% 2527 ± 7% interrupts.CPU32.CAL:Function_call_interrupts
8862 ± 16% -39.6% 5354 ± 28% interrupts.CPU32.RES:Rescheduling_interrupts
2156 ± 6% +20.5% 2599 ± 9% interrupts.CPU33.CAL:Function_call_interrupts
2169 ± 5% +19.9% 2601 ± 9% interrupts.CPU34.CAL:Function_call_interrupts
10483 ± 18% -48.8% 5371 ± 33% interrupts.CPU34.RES:Rescheduling_interrupts
2049 +24.4% 2549 ± 8% interrupts.CPU35.CAL:Function_call_interrupts
10929 ± 15% -49.0% 5578 ± 14% interrupts.CPU35.RES:Rescheduling_interrupts
1993 ± 4% +28.5% 2561 ± 12% interrupts.CPU36.CAL:Function_call_interrupts
161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.NMI:Non-maskable_interrupts
161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.PMI:Performance_monitoring_interrupts
1987 ± 4% +30.5% 2594 ± 13% interrupts.CPU37.CAL:Function_call_interrupts
1863 ± 10% +35.7% 2529 ± 14% interrupts.CPU38.CAL:Function_call_interrupts
1984 ± 6% +26.5% 2509 ± 12% interrupts.CPU41.CAL:Function_call_interrupts
12830 ± 30% -26.4% 9445 ± 9% interrupts.CPU43.RES:Rescheduling_interrupts
2067 ± 2% +27.4% 2632 ± 9% interrupts.CPU44.CAL:Function_call_interrupts
2066 ± 3% +27.2% 2629 ± 9% interrupts.CPU45.CAL:Function_call_interrupts
2158 ± 3% +19.6% 2582 ± 9% interrupts.CPU46.CAL:Function_call_interrupts
2036 ± 6% +30.3% 2652 ± 10% interrupts.CPU47.CAL:Function_call_interrupts
150.50 ± 43% -28.0% 108.33 interrupts.CPU49.84:PCI-MSI.1572913-edge.eth0-TxRx-49
11823 ± 12% -36.9% 7460 ± 18% interrupts.CPU49.RES:Rescheduling_interrupts
1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.NMI:Non-maskable_interrupts
1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.PMI:Performance_monitoring_interrupts
11748 ± 22% -40.0% 7048 ± 27% interrupts.CPU5.RES:Rescheduling_interrupts
2102 ± 5% +19.6% 2515 ± 7% interrupts.CPU52.CAL:Function_call_interrupts
2115 ± 6% +19.5% 2527 ± 7% interrupts.CPU53.CAL:Function_call_interrupts
11318 ± 17% -20.4% 9004 ± 26% interrupts.CPU53.RES:Rescheduling_interrupts
2177 ± 5% +17.4% 2556 ± 9% interrupts.CPU54.CAL:Function_call_interrupts
10968 ± 23% -62.3% 4134 ± 21% interrupts.CPU54.RES:Rescheduling_interrupts
2161 ± 5% +20.2% 2598 ± 9% interrupts.CPU55.CAL:Function_call_interrupts
9334 ± 19% -55.4% 4158 ± 34% interrupts.CPU55.RES:Rescheduling_interrupts
1443 ± 22% -100.0% 0.00 interrupts.CPU56.NMI:Non-maskable_interrupts
1443 ± 22% -100.0% 0.00 interrupts.CPU56.PMI:Performance_monitoring_interrupts
11377 ± 31% -56.9% 4907 ± 43% interrupts.CPU56.RES:Rescheduling_interrupts
601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.NMI:Non-maskable_interrupts
601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.PMI:Performance_monitoring_interrupts
8295 ± 31% -47.7% 4340 ± 24% interrupts.CPU57.RES:Rescheduling_interrupts
9182 ± 37% -59.7% 3701 ± 46% interrupts.CPU58.RES:Rescheduling_interrupts
249.50 ± 76% -50.4% 123.67 ± 2% interrupts.CPU6.41:PCI-MSI.1572870-edge.eth0-TxRx-6
12473 ± 27% -44.4% 6940 ± 13% interrupts.CPU6.RES:Rescheduling_interrupts
7886 ± 12% -42.2% 4555 ± 31% interrupts.CPU60.RES:Rescheduling_interrupts
971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.NMI:Non-maskable_interrupts
971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.PMI:Performance_monitoring_interrupts
9253 ± 26% -56.4% 4033 ± 16% interrupts.CPU61.RES:Rescheduling_interrupts
2103 ± 5% +19.1% 2505 ± 11% interrupts.CPU63.CAL:Function_call_interrupts
2059 ± 2% +20.6% 2483 ± 10% interrupts.CPU64.CAL:Function_call_interrupts
8545 ± 17% -41.2% 5025 ± 8% interrupts.CPU65.RES:Rescheduling_interrupts
7378 ± 23% -51.0% 3614 ± 30% interrupts.CPU68.RES:Rescheduling_interrupts
8573 ± 28% -50.1% 4277 ± 6% interrupts.CPU69.RES:Rescheduling_interrupts
154.50 ± 13% -10.9% 137.67 ± 10% interrupts.CPU7.42:PCI-MSI.1572871-edge.eth0-TxRx-7
9910 ± 28% -51.2% 4839 ± 40% interrupts.CPU70.RES:Rescheduling_interrupts
8836 ± 26% -52.7% 4180 ± 6% interrupts.CPU71.RES:Rescheduling_interrupts
911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.NMI:Non-maskable_interrupts
911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.PMI:Performance_monitoring_interrupts
12326 ± 15% -32.3% 8350 ± 10% interrupts.CPU8.RES:Rescheduling_interrupts
1392 ± 58% -100.0% 0.00 interrupts.CPU9.NMI:Non-maskable_interrupts
1392 ± 58% -100.0% 0.00 interrupts.CPU9.PMI:Performance_monitoring_interrupts
60185 ± 19% -30.5% 41822 ± 11% interrupts.NMI:Non-maskable_interrupts
60185 ± 19% -30.5% 41822 ± 11% interrupts.PMI:Performance_monitoring_interrupts
749419 -31.9% 510225 ± 3% interrupts.RES:Rescheduling_interrupts
15.05 ± 6% -4.1 10.97 ± 20% perf-profile.calltrace.cycles-pp.normal_work_helper.process_one_work.worker_thread.kthread.ret_from_fork
12.66 ± 5% -4.0 8.68 ± 20% perf-profile.calltrace.cycles-pp.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread.kthread
5.99 ± 8% -1.6 4.37 ± 6% perf-profile.calltrace.cycles-pp.io_submit
5.83 ± 8% -1.6 4.25 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.io_submit
5.82 ± 8% -1.6 4.24 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.81 ± 8% -1.6 4.23 ± 6% perf-profile.calltrace.cycles-pp.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.76 ± 8% -1.6 4.19 ± 6% perf-profile.calltrace.cycles-pp.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.65 ± 6% -1.6 4.08 ± 20% perf-profile.calltrace.cycles-pp.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread
4.52 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread
6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback
6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes
5.58 ± 8% -1.5 4.07 ± 6% perf-profile.calltrace.cycles-pp.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.50 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work
5.44 ± 8% -1.5 3.96 ± 7% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64
6.63 ± 8% -1.5 5.17 ± 10% perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode
5.34 ± 9% -1.4 3.90 ± 6% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit
5.55 ± 8% -1.2 4.32 ± 10% perf-profile.calltrace.cycles-pp.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
4.98 ± 8% -1.1 3.86 ± 10% perf-profile.calltrace.cycles-pp.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages
5.00 ± 8% -1.1 3.88 ± 10% perf-profile.calltrace.cycles-pp.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages
2.80 ± 4% -1.0 1.78 ± 22% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work
1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io
1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper
2.32 ± 3% -0.6 1.72 ± 18% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper
0.91 ± 8% -0.5 0.42 ± 73% perf-profile.calltrace.cycles-pp.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper
0.88 ± 8% -0.5 0.41 ± 74% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io
0.83 ± 5% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums
1.90 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread
1.89 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work
0.82 ± 4% -0.4 0.42 ± 71% perf-profile.calltrace.cycles-pp.run_one_async_done.normal_work_helper.process_one_work.worker_thread.kthread
0.81 ± 4% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.btrfs_map_bio.run_one_async_done.normal_work_helper.process_one_work.worker_thread
0.74 ± 6% -0.4 0.36 ± 70% perf-profile.calltrace.cycles-pp.push_leaf_right.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks
1.19 ± 3% -0.4 0.83 ± 16% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io
1.00 ± 6% -0.4 0.65 ± 7% perf-profile.calltrace.cycles-pp.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.27 ± 5% -0.4 0.92 ± 7% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.25 ± 6% -0.3 0.90 ± 7% perf-profile.calltrace.cycles-pp.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
0.93 ± 8% -0.3 0.59 ± 7% perf-profile.calltrace.cycles-pp.__set_extent_bit.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter
0.94 ± 7% -0.3 0.60 ± 6% perf-profile.calltrace.cycles-pp.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write
0.70 ± 8% -0.3 0.37 ± 70% perf-profile.calltrace.cycles-pp.__lookup_extent_mapping.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter
0.79 ± 6% -0.3 0.47 ± 71% perf-profile.calltrace.cycles-pp.setup_items_for_insert.btrfs_duplicate_item.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper
0.71 ± 21% -0.3 0.43 ± 74% perf-profile.calltrace.cycles-pp.__btrfs_free_extent.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space
1.15 ± 11% -0.3 0.88 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.15 ± 11% -0.3 0.88 ± 8% perf-profile.calltrace.cycles-pp.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
1.43 ± 13% -0.3 1.17 ± 5% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.06 ± 5% -0.3 0.81 ± 7% perf-profile.calltrace.cycles-pp.btrfs_drop_extent_cache.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.30 ± 15% -0.2 1.06 ± 4% perf-profile.calltrace.cycles-pp.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write
1.03 ± 13% -0.2 0.79 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range
1.20 ± 4% -0.2 0.96 ± 15% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
1.24 ± 3% -0.2 1.01 ± 15% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.05 ± 8% -0.2 0.82 ± 12% perf-profile.calltrace.cycles-pp.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
0.88 ± 13% -0.2 0.71 ± 2% perf-profile.calltrace.cycles-pp.wait_reserve_ticket.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter
1.01 ± 9% -0.2 0.84 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.01 ± 10% -0.2 0.85 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
0.72 ± 7% -0.2 0.56 ± 6% perf-profile.calltrace.cycles-pp.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write
0.75 ± 8% -0.1 0.60 ± 10% perf-profile.calltrace.cycles-pp.submit_extent_page.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages
0.92 ± 15% +0.2 1.14 ± 3% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
1.01 ± 15% +0.2 1.25 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
1.15 ± 15% +0.3 1.44 ± 6% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
0.27 ±100% +0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
1.74 ± 9% +0.4 2.11 ± 2% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
2.63 ± 10% +0.6 3.26 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle
0.00 +0.8 0.75 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook
2.37 ± 19% +0.8 3.19 ± 5% perf-profile.calltrace.cycles-pp.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread.ret_from_fork
2.33 ± 19% +0.8 3.17 ± 5% perf-profile.calltrace.cycles-pp.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread
0.00 +0.9 0.90 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook
4.71 ± 11% +1.0 5.74 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry
5.04 ± 10% +1.2 6.19 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
0.00 +1.3 1.30 ± 19% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range
0.00 +1.4 1.36 ± 19% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents
0.29 ±101% +1.4 1.67 ± 18% perf-profile.calltrace.cycles-pp.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space
0.00 +1.5 1.52 ± 18% perf-profile.calltrace.cycles-pp.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio
0.00 +2.2 2.20 ± 14% perf-profile.calltrace.cycles-pp.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_writeback.wb_workfn.process_one_work.worker_thread.kthread
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_workfn.process_one_work.worker_thread.kthread.ret_from_fork
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work.worker_thread
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work
49.45 ± 2% +4.3 53.79 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.calltrace.cycles-pp.secondary_startup_64
1.07 ± 18% +5.0 6.06 ± 20% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback
0.93 ± 18% +5.0 5.95 ± 20% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes
61.26 ± 2% +5.1 66.36 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
61.27 ± 2% +5.1 66.37 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
61.21 ± 2% +5.1 66.32 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.33 ±100% +5.3 5.68 ± 20% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode
55.45 ± 2% +5.4 60.88 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +6.4 6.39 ± 13% perf-profile.calltrace.cycles-pp.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page
0.00 +6.5 6.50 ± 13% perf-profile.calltrace.cycles-pp.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb
0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages
0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages
15.05 ± 6% -4.1 10.97 ± 20% perf-profile.children.cycles-pp.normal_work_helper
12.66 ± 5% -4.0 8.68 ± 20% perf-profile.children.cycles-pp.btrfs_finish_ordered_io
7.84 ± 5% -2.4 5.39 ± 10% perf-profile.children.cycles-pp.btrfs_search_slot
7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_writepages
7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_write_cache_pages
7.03 ± 7% -1.6 5.38 ± 12% perf-profile.children.cycles-pp.__extent_writepage
6.19 ± 8% -1.6 4.56 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
6.17 ± 8% -1.6 4.55 ± 7% perf-profile.children.cycles-pp.do_syscall_64
6.01 ± 8% -1.6 4.39 ± 7% perf-profile.children.cycles-pp.io_submit
5.81 ± 8% -1.6 4.23 ± 6% perf-profile.children.cycles-pp.__x64_sys_io_submit
5.76 ± 8% -1.6 4.19 ± 6% perf-profile.children.cycles-pp.io_submit_one
5.66 ± 5% -1.6 4.09 ± 20% perf-profile.children.cycles-pp.btrfs_mark_extent_written
4.52 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.add_pending_csums
5.59 ± 8% -1.5 4.07 ± 6% perf-profile.children.cycles-pp.aio_write
4.51 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.btrfs_csum_file_blocks
5.44 ± 8% -1.5 3.96 ± 7% perf-profile.children.cycles-pp.btrfs_file_write_iter
5.34 ± 9% -1.4 3.90 ± 6% perf-profile.children.cycles-pp.btrfs_buffered_write
5.88 ± 7% -1.4 4.47 ± 11% perf-profile.children.cycles-pp.writepage_delalloc
5.30 ± 8% -1.3 4.01 ± 11% perf-profile.children.cycles-pp.btrfs_run_delalloc_range
5.28 ± 8% -1.3 3.99 ± 11% perf-profile.children.cycles-pp.run_delalloc_nocow
2.55 ± 2% -0.9 1.67 ± 16% perf-profile.children.cycles-pp.btrfs_cow_block
2.54 ± 2% -0.9 1.66 ± 16% perf-profile.children.cycles-pp.__btrfs_cow_block
2.15 ± 13% -0.8 1.31 ± 8% perf-profile.children.cycles-pp.__lookup_extent_mapping
1.87 ± 12% -0.8 1.04 ± 12% perf-profile.children.cycles-pp.__etree_search
0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.printk
0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.vprintk_emit
2.60 ± 2% -0.7 1.92 ± 14% perf-profile.children.cycles-pp.btrfs_insert_empty_items
3.00 ± 5% -0.7 2.33 ± 6% perf-profile.children.cycles-pp._raw_spin_lock
1.89 ± 2% -0.7 1.23 ± 7% perf-profile.children.cycles-pp.__set_extent_bit
2.78 ± 4% -0.6 2.14 ± 14% perf-profile.children.cycles-pp.__schedule
1.57 ± 2% -0.5 1.02 ± 9% perf-profile.children.cycles-pp.lock_extent_bits
1.19 ± 14% -0.5 0.70 ± 11% perf-profile.children.cycles-pp.__clear_extent_bit
1.51 ± 7% -0.5 1.04 ± 9% perf-profile.children.cycles-pp.read_block_for_search
2.08 ± 5% -0.5 1.62 ± 18% perf-profile.children.cycles-pp.setup_items_for_insert
2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.btrfs_run_delayed_refs
2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.__btrfs_run_delayed_refs
1.60 ± 6% -0.4 1.20 ± 12% perf-profile.children.cycles-pp.schedule
1.95 ± 11% -0.4 1.58 ± 12% perf-profile.children.cycles-pp.try_to_wake_up
1.31 ± 6% -0.4 0.94 ± 9% perf-profile.children.cycles-pp.create_io_em
0.99 ± 4% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.alloc_tree_block_no_bg_flush
1.00 ± 6% -0.4 0.65 ± 7% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need
1.27 ± 9% -0.4 0.92 ± 11% perf-profile.children.cycles-pp.__wake_up_common_lock
0.98 ± 3% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.btrfs_alloc_tree_block
1.27 ± 5% -0.4 0.92 ± 7% perf-profile.children.cycles-pp.btrfs_dirty_pages
1.34 ± 7% -0.3 0.99 ± 10% perf-profile.children.cycles-pp.btrfs_map_bio
1.11 ± 9% -0.3 0.77 ± 11% perf-profile.children.cycles-pp.generic_bin_search
1.16 ± 6% -0.3 0.82 ± 9% perf-profile.children.cycles-pp.find_extent_buffer
0.91 ± 7% -0.3 0.57 ± 21% perf-profile.children.cycles-pp.btrfs_lookup_csum
1.21 ± 11% -0.3 0.90 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_csums_range
1.21 ± 11% -0.3 0.91 ± 9% perf-profile.children.cycles-pp.csum_exist_in_range
1.10 ± 10% -0.3 0.80 ± 9% perf-profile.children.cycles-pp.__wake_up_common
0.37 ± 70% -0.3 0.07 ± 18% perf-profile.children.cycles-pp.test_range_bit
1.61 ± 4% -0.3 1.32 ± 15% perf-profile.children.cycles-pp.push_leaf_right
0.60 ± 4% -0.3 0.31 ± 25% perf-profile.children.cycles-pp.btrfs_extend_item
1.02 ± 9% -0.3 0.73 ± 10% perf-profile.children.cycles-pp.autoremove_wake_function
1.12 ± 4% -0.3 0.84 ± 9% perf-profile.children.cycles-pp.btrfs_drop_extent_cache
0.82 ± 6% -0.3 0.55 ± 4% perf-profile.children.cycles-pp.btrfs_release_path
0.85 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_extent_buffer_full
0.99 ± 4% -0.3 0.73 ± 13% perf-profile.children.cycles-pp.btrfs_set_token_32
1.44 ± 13% -0.3 1.18 ± 5% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
0.84 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_page
0.87 ± 11% -0.3 0.62 ± 11% perf-profile.children.cycles-pp.pagecache_get_page
0.84 ± 7% -0.2 0.60 ± 11% perf-profile.children.cycles-pp.kmem_cache_alloc
1.12 ± 7% -0.2 0.88 ± 17% perf-profile.children.cycles-pp.__extent_writepage_io
1.30 ± 15% -0.2 1.06 ± 5% perf-profile.children.cycles-pp.reserve_metadata_bytes
0.82 ± 4% -0.2 0.58 ± 13% perf-profile.children.cycles-pp.run_one_async_done
1.25 ± 3% -0.2 1.01 ± 15% perf-profile.children.cycles-pp.schedule_idle
0.34 ± 53% -0.2 0.10 ± 28% perf-profile.children.cycles-pp.delay_tsc
0.70 ± 13% -0.2 0.47 ± 11% perf-profile.children.cycles-pp.prepare_pages
0.50 ± 22% -0.2 0.28 ± 13% perf-profile.children.cycles-pp.clear_state_bit
0.84 ± 5% -0.2 0.63 ± 6% perf-profile.children.cycles-pp.btrfs_get_extent
0.54 ± 11% -0.2 0.34 ± 4% perf-profile.children.cycles-pp.btrfs_free_path
1.16 ± 11% -0.2 0.96 ± 19% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.53 ± 9% -0.2 0.33 ± 18% perf-profile.children.cycles-pp.unpin_extent_cache
1.07 ± 10% -0.2 0.88 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_file_extent
0.50 ± 4% -0.2 0.31 ± 15% perf-profile.children.cycles-pp.clear_extent_bit
0.45 ± 13% -0.2 0.26 ± 11% perf-profile.children.cycles-pp.kmem_cache_free
0.40 ± 14% -0.2 0.21 ± 50% perf-profile.children.cycles-pp.__writeback_inodes_wb
0.64 ± 7% -0.2 0.45 ± 10% perf-profile.children.cycles-pp.dequeue_task_fair
0.50 ± 8% -0.2 0.33 ± 23% perf-profile.children.cycles-pp.btrfs_add_delayed_tree_ref
0.88 ± 13% -0.2 0.71 ± 2% perf-profile.children.cycles-pp.wait_reserve_ticket
0.52 ± 9% -0.2 0.35 ± 7% perf-profile.children.cycles-pp.__radix_tree_lookup
0.52 ± 8% -0.2 0.36 ± 8% perf-profile.children.cycles-pp.xas_load
0.78 ± 19% -0.2 0.62 ± 23% perf-profile.children.cycles-pp.__btrfs_free_extent
0.56 ± 17% -0.1 0.42 ± 15% perf-profile.children.cycles-pp.btrfs_del_items
0.51 ± 5% -0.1 0.36 ± 9% perf-profile.children.cycles-pp.dequeue_entity
0.39 ± 7% -0.1 0.24 ± 6% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers
0.82 ± 14% -0.1 0.68 ± 18% perf-profile.children.cycles-pp.ttwu_do_activate
0.55 ± 8% -0.1 0.41 ± 8% perf-profile.children.cycles-pp.btrfs_submit_bio_hook
0.53 ± 5% -0.1 0.40 ± 13% perf-profile.children.cycles-pp.update_load_avg
0.40 ± 8% -0.1 0.27 ± 27% perf-profile.children.cycles-pp.add_delayed_ref_head
0.35 ± 13% -0.1 0.22 ± 3% perf-profile.children.cycles-pp.set_extent_bit
0.52 ± 9% -0.1 0.39 ± 7% perf-profile.children.cycles-pp.btrfs_wq_submit_bio
0.35 ± 3% -0.1 0.23 ± 32% perf-profile.children.cycles-pp.queued_write_lock_slowpath
0.33 ± 6% -0.1 0.21 ± 15% perf-profile.children.cycles-pp.__slab_alloc
0.63 ± 19% -0.1 0.51 ± 13% perf-profile.children.cycles-pp.sched_ttwu_pending
0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode
0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode_fallback
0.57 ± 5% -0.1 0.46 ± 10% perf-profile.children.cycles-pp.find_lock_delalloc_range
0.45 ± 5% -0.1 0.34 ± 9% perf-profile.children.cycles-pp.btrfs_cross_ref_exist
0.29 ± 3% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.free_extent_buffer
0.38 ± 3% -0.1 0.26 ± 8% perf-profile.children.cycles-pp.btrfs_mark_buffer_dirty
0.46 ± 5% -0.1 0.35 ± 13% perf-profile.children.cycles-pp.find_delalloc_range
0.41 ± 9% -0.1 0.30 ± 2% perf-profile.children.cycles-pp.btrfs_copy_from_user
0.40 ± 9% -0.1 0.29 ± 2% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
0.38 ± 10% -0.1 0.27 perf-profile.children.cycles-pp.find_get_entry
0.31 ± 12% -0.1 0.20 ± 31% perf-profile.children.cycles-pp.btrfs_remove_ordered_extent
0.30 ± 6% -0.1 0.20 ± 14% perf-profile.children.cycles-pp.___slab_alloc
0.26 ± 8% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.merge_state
0.38 ± 7% -0.1 0.28 ± 3% perf-profile.children.cycles-pp.copyin
0.26 ± 17% -0.1 0.15 ± 26% perf-profile.children.cycles-pp.btrfs_delayed_update_inode
0.25 ± 4% -0.1 0.15 ± 5% perf-profile.children.cycles-pp.insert_state
0.40 ± 13% -0.1 0.30 ± 13% perf-profile.children.cycles-pp.extent_clear_unlock_delalloc
0.36 ± 9% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.alloc_extent_state
0.38 ± 7% -0.1 0.29 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.39 ± 3% -0.1 0.29 ± 14% perf-profile.children.cycles-pp.alloc_extent_buffer
0.31 ± 4% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.set_extent_buffer_dirty
0.26 ± 11% -0.1 0.17 ± 23% perf-profile.children.cycles-pp.unlock_up
0.32 ± 3% -0.1 0.24 ± 13% perf-profile.children.cycles-pp.__btrfs_map_block
0.23 ± 13% -0.1 0.15 ± 28% perf-profile.children.cycles-pp.btrfs_add_delayed_data_ref
0.25 ± 12% -0.1 0.17 ± 2% perf-profile.children.cycles-pp.__slab_free
0.47 ± 6% -0.1 0.39 ± 15% perf-profile.children.cycles-pp.__push_leaf_right
0.26 ± 9% -0.1 0.18 ± 19% perf-profile.children.cycles-pp.__test_set_page_writeback
0.32 ± 11% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.btrfs_add_ordered_extent
0.15 ± 8% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.__xa_set_mark
0.23 ± 7% -0.1 0.15 ± 3% perf-profile.children.cycles-pp.release_extent_buffer
0.18 ± 8% -0.1 0.11 ± 26% perf-profile.children.cycles-pp.alloc_extent_map
0.32 ± 10% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.__btrfs_add_ordered_extent
0.32 ± 17% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.btrfs_tree_read_lock_atomic
0.20 ± 13% -0.1 0.13 ± 27% perf-profile.children.cycles-pp.btrfs_inc_extent_ref
0.14 ± 21% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.find_ref_head
0.13 ± 32% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.rb_erase_cached
0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_submit_bio_start
0.53 ± 12% -0.1 0.46 ± 16% perf-profile.children.cycles-pp.btrfs_tree_read_lock
0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_csum_one_bio
0.17 ± 16% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.rb_erase
0.21 ± 7% -0.1 0.14 ± 16% perf-profile.children.cycles-pp.run_one_async_start
0.22 ± 3% -0.1 0.15 ± 22% perf-profile.children.cycles-pp.btrfs_try_tree_write_lock
0.21 ± 9% -0.1 0.15 ± 21% perf-profile.children.cycles-pp.btrfs_leaf_free_space
0.15 ± 23% -0.1 0.09 ± 35% perf-profile.children.cycles-pp.free_extent_map
0.20 ± 9% -0.1 0.14 ± 20% perf-profile.children.cycles-pp.leaf_space_used
0.19 ± 8% -0.1 0.13 ± 13% perf-profile.children.cycles-pp.check_delayed_ref
0.17 ± 13% -0.1 0.12 ± 31% perf-profile.children.cycles-pp.get_page_from_freelist
0.13 ± 8% -0.1 0.08 ± 26% perf-profile.children.cycles-pp.__update_load_avg_se
0.17 ± 2% -0.1 0.12 ± 18% perf-profile.children.cycles-pp.btrfs_set_range_writeback
0.15 ± 8% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.xas_set_mark
0.18 ± 6% -0.1 0.13 ± 16% perf-profile.children.cycles-pp.___might_sleep
0.20 ± 11% -0.1 0.15 ± 35% perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.10 ± 9% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_unlock
0.14 ± 11% -0.1 0.09 ± 28% perf-profile.children.cycles-pp.btrfs_reserve_extent
0.10 ± 33% -0.1 0.05 ± 72% perf-profile.children.cycles-pp.__btrfs_release_delayed_node
0.22 ± 9% -0.0 0.17 ± 25% perf-profile.children.cycles-pp.mark_extent_buffer_accessed
0.09 ± 13% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_read_unlock_blocking
0.18 ± 8% -0.0 0.14 ± 24% perf-profile.children.cycles-pp.__list_del_entry_valid
0.09 ± 13% -0.0 0.04 ± 73% perf-profile.children.cycles-pp.verify_parent_transid
0.11 ± 19% -0.0 0.07 ± 11% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.22 ± 5% -0.0 0.18 ± 12% perf-profile.children.cycles-pp.check_committed_ref
0.11 ± 19% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.select_idle_sibling
0.14 ± 11% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.do_io_getevents
0.15 ± 7% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__x64_sys_io_getevents
0.08 ± 10% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_release_extent_buffer_pages
0.17 ± 18% -0.0 0.13 ± 9% perf-profile.children.cycles-pp.clear_page_dirty_for_io
0.14 ± 5% -0.0 0.10 ± 25% perf-profile.children.cycles-pp.block_group_cache_tree_search
0.18 ± 4% -0.0 0.14 ± 11% perf-profile.children.cycles-pp.btrfs_get_chunk_map
0.11 ± 13% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.btrfs_buffer_uptodate
0.15 ± 15% -0.0 0.11 ± 19% perf-profile.children.cycles-pp.xas_find_marked
0.12 ± 15% -0.0 0.08 ± 20% perf-profile.children.cycles-pp.btrfs_unlock_up_safe
0.12 ± 14% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.btrfs_set_path_blocking
0.11 ± 11% -0.0 0.07 ± 28% perf-profile.children.cycles-pp.find_free_extent
0.24 ± 5% -0.0 0.20 ± 6% perf-profile.children.cycles-pp.btrfs_root_node
0.07 ± 6% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.extent_mergeable
0.13 ± 12% -0.0 0.10 ± 9% perf-profile.children.cycles-pp.__switch_to
0.10 ± 18% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.rb_insert_color_cached
0.07 ± 14% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.rb_prev
0.11 ± 11% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.get_io_u
0.10 ± 14% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.available_idle_cpu
0.17 ± 10% -0.0 0.13 ± 3% perf-profile.children.cycles-pp._raw_write_lock
0.10 ± 11% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.__might_sleep
0.17 ± 7% -0.0 0.14 ± 12% perf-profile.children.cycles-pp._raw_read_lock
0.17 ± 10% -0.0 0.14 ± 12% perf-profile.children.cycles-pp.can_overcommit
0.10 ± 11% -0.0 0.07 perf-profile.children.cycles-pp.__list_add_valid
0.09 ± 7% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.aio_read_events
0.11 ± 10% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.___perf_sw_event
0.11 ± 9% -0.0 0.09 ± 18% perf-profile.children.cycles-pp.get_alloc_profile
0.11 ± 12% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.pick_next_task_idle
0.10 ± 13% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.read_events
0.08 ± 6% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.replace_extent_mapping
0.10 ± 5% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.menu_reflect
0.07 ± 17% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.run_posix_cpu_timers
0.05 ± 58% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.rcu_irq_enter
0.19 ± 11% +0.1 0.26 ± 17% perf-profile.children.cycles-pp.native_irq_return_iret
0.20 ± 14% +0.1 0.27 ± 8% perf-profile.children.cycles-pp.btrfs_comp_cpu_keys
0.38 ± 9% +0.1 0.46 perf-profile.children.cycles-pp.load_balance
0.30 ± 8% +0.1 0.38 ± 15% perf-profile.children.cycles-pp.irq_enter
0.21 ± 24% +0.1 0.30 ± 9% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.22 ± 3% +0.1 0.32 ± 9% perf-profile.children.cycles-pp.lapic_next_deadline
0.52 ± 12% +0.1 0.63 ± 3% perf-profile.children.cycles-pp.rebalance_domains
1.90 ± 10% +0.3 2.23 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.23 ± 12% +0.4 0.67 ± 15% perf-profile.children.cycles-pp.read_extent_buffer
2.88 ± 11% +0.5 3.41 perf-profile.children.cycles-pp.hrtimer_interrupt
0.09 ± 12% +0.7 0.83 ± 15% perf-profile.children.cycles-pp.btrfs_get_token_8
2.37 ± 19% +0.8 3.19 ± 5% perf-profile.children.cycles-pp.btrfs_async_reclaim_metadata_space
2.33 ± 19% +0.8 3.17 ± 5% perf-profile.children.cycles-pp.flush_space
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_and_wait_transaction
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_marked_extents
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
0.42 ± 43% +1.3 1.67 ± 18% perf-profile.children.cycles-pp.btrfs_commit_transaction
0.25 ± 5% +1.9 2.17 ± 10% perf-profile.children.cycles-pp.btrfs_get_token_64
1.71 ± 5% +2.1 3.79 ± 5% perf-profile.children.cycles-pp.btrfs_get_token_32
0.43 ± 8% +2.4 2.84 ± 10% perf-profile.children.cycles-pp.map_private_extent_buffer
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_writeback
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.writeback_sb_inodes
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.__writeback_single_inode
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_workfn
50.06 ± 2% +3.8 53.82 ± 4% perf-profile.children.cycles-pp.intel_idle
61.99 ± 2% +4.4 66.39 ± 3% perf-profile.children.cycles-pp.do_idle
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.secondary_startup_64
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.cpu_startup_entry
8.51 ± 7% +4.5 13.04 ± 12% perf-profile.children.cycles-pp.do_writepages
56.43 ± 3% +4.9 61.30 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state
61.27 ± 2% +5.1 66.37 ± 3% perf-profile.children.cycles-pp.start_secondary
1.43 ± 7% +6.2 7.64 ± 13% perf-profile.children.cycles-pp.submit_extent_page
1.20 ± 17% +6.2 7.45 ± 13% perf-profile.children.cycles-pp.btree_write_cache_pages
1.05 ± 17% +6.3 7.31 ± 13% perf-profile.children.cycles-pp.write_one_eb
1.11 ± 7% +6.3 7.37 ± 13% perf-profile.children.cycles-pp.submit_one_bio
0.55 ± 16% +6.4 6.95 ± 13% perf-profile.children.cycles-pp.btree_submit_bio_hook
0.00 +6.5 6.48 ± 13% perf-profile.children.cycles-pp.check_leaf
0.00 +6.5 6.53 ± 13% perf-profile.children.cycles-pp.btree_csum_one_bio
2.12 ± 11% -0.8 1.29 ± 7% perf-profile.self.cycles-pp.__lookup_extent_mapping
1.83 ± 12% -0.8 1.03 ± 13% perf-profile.self.cycles-pp.__etree_search
2.06 ± 5% -0.5 1.59 ± 5% perf-profile.self.cycles-pp._raw_spin_lock
0.81 ± 9% -0.3 0.52 ± 10% perf-profile.self.cycles-pp.generic_bin_search
0.84 ± 7% -0.3 0.58 ± 17% perf-profile.self.cycles-pp.copy_page
0.33 ± 54% -0.2 0.10 ± 28% perf-profile.self.cycles-pp.delay_tsc
0.89 ± 9% -0.2 0.69 ± 14% perf-profile.self.cycles-pp.btrfs_set_token_32
0.81 ± 4% -0.2 0.63 ± 14% perf-profile.self.cycles-pp.__schedule
0.51 ± 9% -0.2 0.35 ± 6% perf-profile.self.cycles-pp.__radix_tree_lookup
0.45 ± 8% -0.1 0.32 ± 10% perf-profile.self.cycles-pp.xas_load
0.43 ± 4% -0.1 0.31 ± 6% perf-profile.self.cycles-pp.find_extent_buffer
0.77 ± 5% -0.1 0.66 ± 11% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.31 ± 10% -0.1 0.20 ± 28% perf-profile.self.cycles-pp.add_delayed_ref_head
0.39 ± 16% -0.1 0.29 ± 16% perf-profile.self.cycles-pp.try_to_wake_up
0.31 ± 4% -0.1 0.20 ± 26% perf-profile.self.cycles-pp.queued_write_lock_slowpath
0.38 ± 7% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.20 ± 17% -0.1 0.11 ± 14% perf-profile.self.cycles-pp.kmem_cache_free
0.25 ± 11% -0.1 0.17 ± 2% perf-profile.self.cycles-pp.__slab_free
0.16 ± 9% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.free_extent_buffer
0.28 ± 17% -0.1 0.20 ± 11% perf-profile.self.cycles-pp.btrfs_search_slot
0.31 ± 4% -0.1 0.24 ± 20% perf-profile.self.cycles-pp.kmem_cache_alloc
0.14 ± 21% -0.1 0.06 ± 19% perf-profile.self.cycles-pp.find_ref_head
0.16 ± 10% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.insert_state
0.12 ± 8% -0.1 0.06 ± 72% perf-profile.self.cycles-pp.__update_load_avg_se
0.16 ± 15% -0.1 0.10 ± 8% perf-profile.self.cycles-pp.rb_erase
0.10 ± 9% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_unlock
0.15 ± 13% -0.1 0.10 ± 12% perf-profile.self.cycles-pp.___slab_alloc
0.28 ± 13% -0.1 0.22 ± 5% perf-profile.self.cycles-pp.queued_read_lock_slowpath
0.15 ± 21% -0.1 0.09 ± 35% perf-profile.self.cycles-pp.free_extent_map
0.18 ± 6% -0.1 0.12 ± 11% perf-profile.self.cycles-pp.update_load_avg
0.10 ± 11% -0.1 0.04 ± 70% perf-profile.self.cycles-pp.btrfs_extend_item
0.15 ± 23% -0.1 0.09 ± 9% perf-profile.self.cycles-pp.add_extent_mapping
0.15 ± 8% -0.1 0.09 ± 20% perf-profile.self.cycles-pp.xas_set_mark
0.18 ± 6% -0.1 0.13 ± 16% perf-profile.self.cycles-pp.___might_sleep
0.18 ± 6% -0.0 0.13 ± 22% perf-profile.self.cycles-pp.__list_del_entry_valid
0.18 ± 4% -0.0 0.14 ± 22% perf-profile.self.cycles-pp.pick_next_task_fair
0.09 ± 13% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.verify_parent_transid
0.09 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_read_unlock_blocking
0.16 ± 13% -0.0 0.12 ± 24% perf-profile.self.cycles-pp.mark_page_accessed
0.12 ± 14% -0.0 0.08 ± 20% perf-profile.self.cycles-pp.dequeue_task_fair
0.10 ± 19% -0.0 0.07 ± 18% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.08 ± 14% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.unlock_up
0.15 ± 15% -0.0 0.11 ± 19% perf-profile.self.cycles-pp.xas_find_marked
0.24 ± 6% -0.0 0.20 ± 8% perf-profile.self.cycles-pp.btrfs_root_node
0.17 ± 6% -0.0 0.13 ± 10% perf-profile.self.cycles-pp._raw_read_lock
0.10 ± 13% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.btrfs_csum_one_bio
0.16 ± 7% -0.0 0.13 ± 9% perf-profile.self.cycles-pp.dequeue_entity
0.07 -0.0 0.04 ± 71% perf-profile.self.cycles-pp.io_u_queued_complete
0.08 -0.0 0.05 ± 70% perf-profile.self.cycles-pp.mutex_lock
0.11 ± 11% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.get_io_u
0.16 ± 9% -0.0 0.13 perf-profile.self.cycles-pp._raw_write_lock
0.07 ± 17% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.btrfs_tree_read_lock
0.12 ± 8% -0.0 0.09 ± 13% perf-profile.self.cycles-pp.__switch_to
0.10 ± 11% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.__clear_extent_bit
0.07 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.rb_prev
0.10 ± 15% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.find_get_pages_range_tag
0.10 ± 12% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.available_idle_cpu
0.10 ± 11% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.__list_add_valid
0.08 ± 13% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.__might_sleep
0.12 ± 17% -0.0 0.09 ± 15% perf-profile.self.cycles-pp.release_extent_buffer
0.09 ± 17% -0.0 0.07 ± 14% perf-profile.self.cycles-pp.___perf_sw_event
0.08 ± 5% -0.0 0.06 ± 13% perf-profile.self.cycles-pp.__push_leaf_right
0.11 ± 7% -0.0 0.09 ± 9% perf-profile.self.cycles-pp.set_extent_buffer_dirty
0.11 ± 4% -0.0 0.08 ± 11% perf-profile.self.cycles-pp.__set_extent_bit
0.09 ± 7% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.hrtimer_interrupt
0.07 ± 17% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.run_posix_cpu_timers
0.07 ± 7% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.load_balance
0.01 ±173% +0.1 0.07 ± 17% perf-profile.self.cycles-pp.lapic_next_deadline
0.00 +0.1 0.06 ± 23% perf-profile.self.cycles-pp.find_next_and_bit
0.19 ± 11% +0.1 0.26 ± 17% perf-profile.self.cycles-pp.native_irq_return_iret
0.19 ± 24% +0.1 0.28 ± 7% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.50 ± 8% +0.2 0.69 ± 17% perf-profile.self.cycles-pp.cpuidle_enter_state
0.07 ± 11% +0.4 0.46 ± 17% perf-profile.self.cycles-pp.btrfs_get_token_8
0.22 ± 13% +0.4 0.66 ± 16% perf-profile.self.cycles-pp.read_extent_buffer
0.00 +0.6 0.65 ± 14% perf-profile.self.cycles-pp.check_leaf
0.15 ± 11% +1.0 1.13 ± 11% perf-profile.self.cycles-pp.btrfs_get_token_64
1.48 ± 7% +1.0 2.51 ± 5% perf-profile.self.cycles-pp.btrfs_get_token_32
0.41 ± 8% +2.2 2.61 ± 9% perf-profile.self.cycles-pp.map_private_extent_buffer
49.99 ± 2% +3.7 53.74 ± 4% perf-profile.self.cycles-pp.intel_idle
fio.write_bw_MBps
300 +-+-------------------------------------------------------------------+
| |
250 +-++.+ +. .+.. .+..+..+.+.. .+..+.. .+ |
| : : +. +..+ + +..+ +. + |
| : : : : : : |
200 +-+ : : : : : : O O |
O O O: O :O O O O O O O O O O: O:O O: O: O O O O O
150 +-+ : : : : : : |
| : : : : : : |
100 +-+ : : : : : : |
| : : : : : : |
| : : : : : : |
50 +-+ :: :: :: |
| : : : |
0 +-+---------------------------------------------O-------------O-------+
fio.write_iops
70000 +-+-----------------------------------------------------------------+
|..+.+ +..+..+.+.. .+.+..+.+..+ +.. +..+.+..+..+ |
60000 +-+ : : +. : : + : |
| : : : : : : |
50000 +-+ : : : : : : |
O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O
40000 +-+ : O: : : : : |
| : : : : : : |
30000 +-+ : : : : : : |
| : : : : : : |
20000 +-+ : : : : : : |
| :: :: :: |
10000 +-+ : : : |
| : : : |
0 +-+--------------------------------------------O------------O-------+
fio.write_clat_mean_us
30000 +-+-----------------------------------------------------------------+
| |
25000 O-+O O O O O O O O O O O O O O O O O O O |
| O O O O O
| |
20000 +-+ |
|..+.+ +..+..+.+..+..+.+..+.+..+ +..+ +..+.+..+..+ |
15000 +-+ : : : : : : |
| : : : : : : |
10000 +-+ : : : : : : |
| : : : : : : |
| : : : : : : |
5000 +-+ : : : : :: |
| : : : |
0 +-+--------------------------------------------O------------O-------+
fio.write_clat_stddev
160000 +-+----------------------------------------------------------------+
| |
140000 O-+O O O O O O O O O O O O O O |
120000 +-+ O O |
| |
100000 +-+ O O O O O O O O
| |
80000 +-+ |
| .+.. |
60000 +-++.+ +..+.+. +.+..+.+..+..+ +.+ +..+..+.+..+ |
40000 +-+ : : : : : : |
| : : : : : : |
20000 +-+ : : : : : : |
| :: :: :: |
0 +-+-------------------------------------------O------------O-------+
fio.write_slat_mean_us
900 +-+-------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O |
800 +-+O O O O O O O
700 +-+ |
| |
600 +-++.+ +.+..+..+..+.+..+..+.+..+ +..+ +..+..+..+.+ |
500 +-+ : : : : : : |
| : : : : : : |
400 +-+ : : : : : : |
300 +-+ : : : : : : |
| : : : : : : |
200 +-+ : : : : : : |
100 +-+ : : :: :: |
| : : : |
0 +-+---------------------------------------------O-------------O-------+
fio.write_slat_stddev
25000 +-+-----O---------O-------------------------------------------------+
O O O O O O O O O O O O O O O |
| |
20000 +-+ |
| O O O O O O O O
| |
15000 +-+ |
| |
10000 +-++. .+.. .+.. .+..+ .+ .+..+ |
|. + +..+..+ +..+ + : +. : +..+.+. |
| : : : : : : |
5000 +-+ : : : : : : |
| : : : : : : |
| :: : :: |
0 +-+--------------------------------------------O------------O-------+
fio.latency_1000ms_
0.4 +-+------------------------------------------------------------------+
| |
0.35 +-+ O |
0.3 +-+ |
| O O |
0.25 +-+ O O |
| O O O
0.2 O-+ O O O O O O O O O O |
| O O |
0.15 +-+O O O O |
0.1 +-+ |
| |
0.05 +-+ |
| .+. .+..+..+.+..+.. .+. .+.+..+.. |
0 +-+---------------------------------------------O------------O-------+
fio.workload
1.4e+07 +-+---------------------------------------------------------------+
|..+.+ +..+.+..+.+..+.+..+.+..+ +.. +.+..+.+..+ |
1.2e+07 +-+ : : : : + : |
| : : : : : : |
1e+07 +-+ : : : : : : |
O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O
8e+06 +-+ : O: : : : : |
| : : : : : : |
6e+06 +-+ : : : : : : |
| : : : : : : |
4e+06 +-+ : : : : : : |
| :: :: :: |
2e+06 +-+ : : : |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
fio.time.voluntary_context_switches
1.4e+07 +-+---------------------------------------------------------------+
| |
1.2e+07 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ |
| : : : : : : |
1e+07 +-+ : : : : : : |
| O O: : O O O : : :O : O O O O O O O
8e+06 O-+ : O:O O O O O O O: O:O O: : |
| : : : : : : |
6e+06 +-+ : : : : : : |
| : : : : : : |
4e+06 +-+ : : : : : : |
| : : : : :: |
2e+06 +-+ : : :: |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
fio.time.file_system_outputs
1.2e+08 +-+---------------------------------------------------------------+
| |
1e+08 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ |
| : : : : : : |
| : : : : : : |
8e+07 +-+ : : : : : : O O O O O
O O O: O:O O O O O O O O O O: O:O O:O : O O |
6e+07 +-+ : : : : : : |
| : : : : : : |
4e+07 +-+ : : : : : : |
| : : : : : : |
| : : : : :: |
2e+07 +-+ : : :: |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
7 months, 2 weeks
- 4
- 4
9ff4452912 ("mm/mremap: use pmd_addr_end to calculate next in .."): [ 70.987730] BUG: Bad rss-counter state mm:01f5c40b type:MM_SHMEMPAGES val:15
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
commit 9ff4452912d63fe076ed01324bc651d51d807d5c
Author: Wei Yang <richardw.yang(a)linux.intel.com>
AuthorDate: Fri Jan 24 16:17:30 2020 +1100
Commit: Stephen Rothwell <sfr(a)canb.auug.org.au>
CommitDate: Fri Jan 24 16:17:30 2020 +1100
mm/mremap: use pmd_addr_end to calculate next in move_page_tables()
Use the general helper instead of doing it by hand.
Link: http://lkml.kernel.org/r/20200117232254.2792-4-richardw.yang@linux.intel.com
Signed-off-by: Wei Yang <richardw.yang(a)linux.intel.com>
Cc: Aneesh Kumar K.V <aneesh.kumar(a)linux.ibm.com>
Cc: Dan Williams <dan.j.williams(a)intel.com>
Cc: Kirill A. Shutemov <kirill(a)shutemov.name>
Cc: Thomas Hellstrom <thellstrom(a)vmware.com>
Cc: Yang Shi <yang.shi(a)linux.alibaba.com>
Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org>
Signed-off-by: Stephen Rothwell <sfr(a)canb.auug.org.au>
cc3cd8721a mm/mremap: it is sure to have enough space when extent meets requirement
9ff4452912 mm/mremap: use pmd_addr_end to calculate next in move_page_tables()
+-----------------------------------------------------------------------------------+------------+------------+
| | cc3cd8721a | 9ff4452912 |
+-----------------------------------------------------------------------------------+------------+------------+
| boot_successes | 878 | 70 |
| boot_failures | 50 | 228 |
| WARNING:possible_circular_locking_dependency_detected | 8 | 2 |
| WARNING:at_mm/usercopy.c:#usercopy_warn | 4 | 1 |
| EIP:usercopy_warn | 4 | 1 |
| INFO:rcu_preempt_detected_stalls_on_CPUs/tasks | 2 | |
| EIP:___might_sleep | 1 | |
| EIP:lock_is_held_type | 1 | |
| BUG:kernel_hang_in_test_stage | 7 | 2 |
| BUG:sleeping_function_called_from_invalid_context_at_include/linux/percpu-rwsem.h | 2 | |
| Mem-Info | 1 | 3 |
| BUG:kernel_hang_in_boot_stage | 24 | 6 |
| EIP:_raw_spin_unlock_irq | 1 | |
| EIP:drain_array | 1 | |
| EIP:lock_acquire | 1 | |
| EIP:schedule_timeout | 1 | |
| BUG:kernel_timeout_in_torture_test_stage | 1 | |
| WARNING:at_kernel/rcu/tree_stall.h:#rcu_check_gp_start_stall | 1 | |
| EIP:rcu_check_gp_start_stall | 1 | |
| EIP:default_idle | 1 | |
| invalid_opcode:#[##] | 1 | 38 |
| EIP:serial8250_ports | 1 | |
| Kernel_panic-not_syncing:Fatal_exception | 1 | 149 |
| BUG:kernel_timeout_in_boot_stage | 3 | |
| BUG:Bad_rss-counter_state_mm:#type:MM_FILEPAGES_val | 0 | 10 |
| BUG:Bad_rss-counter_state_mm:#type:MM_ANONPAGES_val | 0 | 17 |
| BUG:non-zero_pgtables_bytes_on_freeing_mm | 0 | 2 |
| BUG:Bad_page_map_in_process | 0 | 60 |
| kernel_BUG_at_include/linux/mm.h | 0 | 26 |
| EIP:release_pages | 0 | 32 |
| BUG:Bad_rss-counter_state_mm:#type:MM_SHMEMPAGES_val | 0 | 33 |
| BUG:kernel_NULL_pointer_dereference,address | 0 | 89 |
| Oops:#[##] | 0 | 118 |
| EIP:___cache_free | 0 | 19 |
| BUG:unable_to_handle_page_fault_for_address | 0 | 38 |
| EIP:native_machine_emergency_restart | 0 | 15 |
| EIP:do_swap_page | 0 | 9 |
| EIP:__copy_user_ll | 0 | 1 |
| EIP:filp_close | 0 | 11 |
| BUG:Bad_rss-counter_state_mm:(ptrval)type:MM_ANONPAGES_val | 0 | 6 |
| BUG:Bad_page_state_in_process | 0 | 6 |
| BUG:Bad_rss-counter_state_mm:#type:MM_SWAPENTS_val | 0 | 8 |
| EIP:__rb_insert_augmented | 0 | 17 |
| WARNING:at_lib/refcount.c:#refcount_warn_saturate | 0 | 3 |
| EIP:refcount_warn_saturate | 0 | 3 |
| EIP:do_exit | 0 | 1 |
| INFO:trying_to_register_non-static_key | 0 | 11 |
| EIP:close_pdeo | 0 | 2 |
| EIP:umount_tree | 0 | 1 |
| EIP:__rb_erase_color | 0 | 8 |
| WARNING:at_kernel/locking/rwsem.c:#down_write | 0 | 4 |
| EIP:down_write | 0 | 5 |
| WARNING:at_kernel/locking/rwsem.c:#up_write | 0 | 3 |
| EIP:up_write | 0 | 3 |
| EIP:locks_remove_posix | 0 | 1 |
| EIP:follow_page_mask | 0 | 2 |
| EIP:unlink_anon_vmas | 0 | 3 |
| EIP:copy_page_to_iter | 0 | 4 |
| EIP:vma_interval_tree_augment_rotate | 0 | 1 |
| EIP:fib_rules_event | 0 | 1 |
| WARNING:at_kernel/locking/lockdep.c:#mark_lock | 0 | 1 |
| EIP:mark_lock | 0 | 1 |
| EIP:neigh_parms_release | 0 | 1 |
| Kernel_panic-not_syncing:Fatal_exception_in_interrupt | 0 | 2 |
| EIP:list_lru_del | 0 | 1 |
| EIP:handle_mm_fault | 0 | 1 |
| EIP:clear_user | 0 | 1 |
| WARNING:at_fs/proc/generic.c:#remove_proc_entry | 0 | 1 |
| EIP:remove_proc_entry | 0 | 1 |
| EIP:rb_erase | 0 | 1 |
| EIP:__lock_acquire | 0 | 4 |
| EIP:dnotify_flush | 0 | 1 |
| EIP:unmap_page_range | 0 | 9 |
| EIP:shmem_mapping | 0 | 5 |
| EIP:visit_groups_merge | 0 | 1 |
| sysctl_link_missing_during_unregister | 0 | 1 |
| EIP:sysctl_print_dir | 0 | 1 |
| Kernel_panic-not_syncing:corrupted_stack_end_detected_inside_scheduler | 0 | 2 |
| WARNING:at_fs/proc/proc_sysctl.c:#retire_sysctl_set | 0 | 4 |
| EIP:retire_sysctl_set | 0 | 4 |
| EIP:count_subheaders | 0 | 1 |
| EIP:do_raw_spin_trylock | 0 | 1 |
| EIP:event_sched_in | 0 | 1 |
| EIP:anon_vma_interval_tree_insert | 0 | 1 |
| EIP:__wake_up_common | 0 | 2 |
| EIP:snmp6_unregister_dev | 0 | 1 |
| kernel_BUG_at_mm/usercopy.c | 0 | 1 |
| EIP:usercopy_abort | 0 | 1 |
| EIP:perf_iterate_ctx | 0 | 4 |
| BUG:Bad_rss-counter_state_mm:(ptrval)type:MM_SHMEMPAGES_val | 0 | 2 |
| kernel_BUG_at_mm/filemap.c | 0 | 6 |
| EIP:unaccount_page_cache_page | 0 | 6 |
| kernel_BUG_at_mm/internal.h | 0 | 1 |
| EIP:rmap_walk_file | 0 | 1 |
| WARNING:at_fs/sysfs/group.c:#sysfs_remove_group | 0 | 1 |
| EIP:sysfs_remove_group | 0 | 1 |
| WARNING:at_fs/kernfs/dir.c:#__kernfs_remove | 0 | 2 |
| EIP:__kernfs_remove | 0 | 2 |
| EIP:idr_remove | 0 | 2 |
| EIP:munlock_vma_pages_range | 0 | 1 |
| EIP:__put_anon_vma | 0 | 1 |
| WARNING:at_net/xfrm/xfrm_state.c:#xfrm_state_fini | 0 | 1 |
| EIP:xfrm_state_fini | 0 | 1 |
| EIP:anon_vma_clone | 0 | 2 |
| EIP:iov_iter_fault_in_readable | 0 | 2 |
| EIP:strlen | 0 | 1 |
| WARNING:at_mm/rmap.c:#anon_vma_clone | 0 | 1 |
| EIP:apparmor_file_free_security | 0 | 1 |
| EIP:uevent_net_exit | 0 | 1 |
| kernel_BUG_at_mm/slab.c | 0 | 1 |
| EIP:__check_heap_object | 0 | 1 |
| EIP:mremap_userfaultfd_prep | 0 | 1 |
+-----------------------------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 58.385214] LoadPin: kernel-module old-api-pinning-ignored obj=<unknown> pid=1148 cmdline="/bin/trinity -q -d -x get_robust_list -N 999999"
[main] 10264 iterations. [F:7282 S:2943 HI:1665]
[main] 11562 iterations. [F:8107 S:3402 HI:2254]
[ 66.557674] cfg80211: failed to load regulatory.db
[ 66.558831] Writes: Total: 47169752 Max/Min: 0/0 Fail: 0
[ 70.987730] BUG: Bad rss-counter state mm:01f5c40b type:MM_SHMEMPAGES val:15
[ 70.991024] LoadPin: kernel-module old-api-pinning-ignored obj=<unknown> pid=1305 cmdline="/bin/trinity -q -d -N 999999"
[ 89.787760] ------------[ cut here ]------------
[ 89.789359] WARNING: CPU: 0 PID: 7 at fs/proc/proc_sysctl.c:1714 retire_sysctl_set+0xd/0x20
[ 89.790335] Modules linked in:
[ 89.790643] CPU: 0 PID: 7 Comm: kworker/u4:0 Not tainted 5.5.0-rc7-00100-g9ff4452912d63 #1
[ 89.791429] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 89.792266] Workqueue: netns cleanup_net
[ 89.792651] EIP: retire_sysctl_set+0xd/0x20
[ 89.793063] Code: 00 00 c7 43 28 00 00 00 00 c7 43 2c 00 00 00 00 8b 5d f4 89 ec 5d c3 8d b6 00 00 00 00 55 89 e5 53 89 c3 8b 43 30 85 c0 74 02 <0f> 0b 5b 5d c3 cc cc cc cc cc cc cc cc cc cc cc cc cc cc 55 89 e5
[ 89.794811] EAX: f2fd9e70 EBX: f2f7ae9c ECX: c21b0210 EDX: c2e2f1b4
[ 89.795432] ESI: c2e2f1b0 EDI: f2f7adc0 EBP: f48e7eb4 ESP: f48e7eb0
[ 89.796032] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010282
[ 89.796684] CR0: 80050033 CR2: 00000020 CR3: 30984000 CR4: 000406d0
[ 89.797284] Call Trace:
[ 89.797532] sysctl_net_exit+0x11/0x20
[ 89.797897] ops_exit_list+0x24/0x50
[ 89.798308] cleanup_net+0x1ae/0x2a0
[ 89.798658] process_one_work+0x335/0x5e0
[ 89.799047] ? process_one_work+0x1e0/0x5e0
[ 89.799447] worker_thread+0x28c/0x4b0
[ 89.799805] kthread+0xfa/0x100
[ 89.800115] ? process_one_work+0x5e0/0x5e0
[ 89.800533] ? kthread_create_worker_on_cpu+0x20/0x20
[ 89.801017] ret_from_fork+0x1e/0x28
[ 89.801365] irq event stamp: 302164
[ 89.801703] hardirqs last enabled at (302163): [<c222189d>] _raw_spin_unlock_irqrestore+0x3d/0x60
[ 89.802544] hardirqs last disabled at (302164): [<c1001b6e>] trace_hardirqs_off_thunk+0xc/0x10
[ 89.803357] softirqs last enabled at (302148): [<c1efddf5>] netlink_release+0x805/0x890
[ 89.804113] softirqs last disabled at (302146): [<c1efddf5>] netlink_release+0x805/0x890
[ 89.804873] ---[ end trace 1be77e98e8984beb ]---
[main] Error opening /boot/trace.txt : No such file or directory
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 702ccea170f07783bd002055a353a0866c062267 v5.4 --
git bisect good 48a8dd171993569cb77dee0882a47abad47d2837 # 03:04 G 300 0 10 10 Merge tag 'drm-fixes-2019-12-28' of git://anongit.freedesktop.org/drm/drm
git bisect good 88eec99fc15ed5abc3d56b444d60718a23f1a87c # 04:32 G 300 0 6 6 Merge remote-tracking branch 'mediatek/for-next'
git bisect good d09cfb473db33e1b8f9bbac1a2d236907dbc2734 # 06:30 G 301 0 7 7 Merge remote-tracking branch 'sound/for-next'
git bisect good 0a45358c872e756d9b0f9ae31cbc7e15809df908 # 07:22 G 308 0 7 7 Merge remote-tracking branch 'chrome-platform/for-next'
git bisect good 94e26012eb8955f973af6e4f069bc4240c6bd846 # 08:14 G 300 0 7 7 Merge remote-tracking branch 'pinctrl-intel/for-next'
git bisect good f029b5bb02f9507115275ca8f56a4951bdadacd2 # 09:19 G 302 0 12 13 Merge remote-tracking branch 'at24/at24/for-next'
git bisect good ae3d536cd9783e30790af9a6fe78e340cc7475a6 # 10:21 G 301 0 16 16 Merge remote-tracking branch 'pidfd/for-next'
git bisect good ebaedb3eb2435ef1f30289a3c32da1de0d26f965 # 12:34 G 304 0 12 12 Merge remote-tracking branch 'kunit-next/kunit'
git bisect bad 9dafa06d235e112a653fc5c1430c3d9be75ec0fd # 12:50 B 21 1 21 21 Merge branch 'akpm-current/current'
git bisect bad b3437b1c1440a341b273ab182c288d8ca71a11b5 # 13:20 B 62 1 46 46 mm/huge_memory.c: reduce critical section protected by split_queue_lock
git bisect good 44d82a4867542ef86e279996863ade52de2eae51 # 16:29 G 310 0 11 11 mm/gup: introduce pin_user_pages*() and FOLL_PIN
git bisect bad 0347aa789d573fce9c7153ddf97e338501e54c0e # 17:50 B 148 1 111 111 mm/mremap: start addresses are properly aligned
git bisect good 66cf11e1741eded78bd9d981bbd3660c195756f1 # 18:57 G 300 0 10 10 mm/gup_benchmark: use proper FOLL_WRITE flags instead of hard-coding "1"
git bisect good 4bd4e0f691e7efbe00558717e2b7d680c3aa9bb2 # 19:57 G 304 0 7 7 mm/mmap.c: fix the adjusted length error
git bisect good cc3cd8721a75db76fbd76532f1df6234fbc38a0a # 21:31 G 300 0 18 18 mm/mremap: it is sure to have enough space when extent meets requirement
git bisect bad 65fbe0ffd2a3ff3bb8b5be3a7565a6dbe896bbe3 # 00:44 B 47 1 38 39 mm/mremap: calculate extent in one place
git bisect bad 9ff4452912d63fe076ed01324bc651d51d807d5c # 01:48 B 40 1 34 34 mm/mremap: use pmd_addr_end to calculate next in move_page_tables()
# first bad commit: [9ff4452912d63fe076ed01324bc651d51d807d5c] mm/mremap: use pmd_addr_end to calculate next in move_page_tables()
git bisect good cc3cd8721a75db76fbd76532f1df6234fbc38a0a # 05:20 G 901 0 35 57 mm/mremap: it is sure to have enough space when extent meets requirement
# extra tests with debug options
git bisect good 9ff4452912d63fe076ed01324bc651d51d807d5c # 13:22 G 300 0 246 246 mm/mremap: use pmd_addr_end to calculate next in move_page_tables()
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation
9 months, 4 weeks
- 2
- 1
[locking/qspinlock] ae9a1c0ae4: will-it-scale.per_thread_ops 132.6% improvement
by kernel test robot
Greeting,
FYI, we noticed a 132.6% improvement of will-it-scale.per_thread_ops due to commit:
commit: ae9a1c0ae45c35d2861c24ac27a5285123f8e30d ("locking/qspinlock: Introduce CNA into the slow path of qspinlock")
https://github.com/0day-ci/linux/commits/Alex-Kogan/Add-NUMA-awareness-to...
in testcase: will-it-scale
on test machine: 104 threads Skylake with 192G memory
with following parameters:
nr_task: 100%
mode: thread
test: unlink2
cpufreq_governor: performance
ucode: 0x2000065
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
In addition to that, the commit also has significant impact on the following tests:
+------------------+---------------------------------------------------------------------------+
| testcase: change | aim7: aim7.jobs-per-min 41.0% improvement |
| test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters | cpufreq_governor=performance |
| | disk=1BRD_48G |
| | fs=f2fs |
| | load=1500 |
| | test=creat-clo |
| | ucode=0x500002c |
+------------------+---------------------------------------------------------------------------+
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-skl-fpga01/unlink2/will-it-scale/0x2000065
commit:
7169a01e0c ("locking/qspinlock: Refactor the qspinlock slow path")
ae9a1c0ae4 ("locking/qspinlock: Introduce CNA into the slow path of qspinlock")
7169a01e0c99633b ae9a1c0ae45c35d2861c24ac27a
---------------- ---------------------------
%stddev %change %stddev
\ | \
8344 ± 2% +132.6% 19410 ± 8% will-it-scale.per_thread_ops
301.64 +49.7% 451.44 ± 8% will-it-scale.time.elapsed_time
301.64 +49.7% 451.44 ± 8% will-it-scale.time.elapsed_time.max
10356 -7.1% 9618 will-it-scale.time.percent_of_cpu_this_job_got
30942 +38.0% 42709 ± 8% will-it-scale.time.system_time
299.88 ± 2% +139.8% 719.26 ± 8% will-it-scale.time.user_time
867845 ± 2% +132.6% 2018782 ± 8% will-it-scale.workload
3427 +2.2% 3502 boot-time.idle
5723156 ± 4% +402.0% 28733048 ± 27% numa-numastat.node0.local_node
5728194 ± 4% +401.7% 28739751 ± 27% numa-numastat.node0.numa_hit
5680305 ± 5% +341.3% 25066930 ± 13% numa-numastat.node1.local_node
5695391 ± 5% +340.4% 25080394 ± 13% numa-numastat.node1.numa_hit
0.98 +0.6 1.53 mpstat.cpu.all.gnice%
0.59 ± 15% +1.4 1.99 ± 38% mpstat.cpu.all.idle%
0.46 ± 15% +7.3 7.80 ± 5% mpstat.cpu.all.soft%
97.00 -9.8 87.16 mpstat.cpu.all.sys%
0.98 +0.6 1.53 mpstat.cpu.all.usr%
97.00 -1.8% 95.25 vmstat.cpu.sy
1274221 +128.8% 2914840 ± 3% vmstat.memory.cache
103.00 +59.0% 163.75 vmstat.procs.r
1697 ± 4% +10.1% 1869 ± 2% vmstat.system.cs
213524 +43.9% 307285 vmstat.system.in
157814 ± 37% +157.5% 406318 ± 39% cpuidle.C1.time
5721 ± 30% +189.7% 16576 ± 33% cpuidle.C1.usage
28196190 ± 30% +860.8% 2.709e+08 ± 37% cpuidle.C1E.time
123827 ± 21% +510.4% 755832 ± 31% cpuidle.C1E.usage
37450 ± 36% +352.7% 169537 ± 23% cpuidle.POLL.time
1877 ± 14% +284.6% 7219 ± 38% cpuidle.POLL.usage
110948 +1510.3% 1786635 ± 2% meminfo.KReclaimable
2471747 +425.2% 12980847 ± 2% meminfo.Memused
110948 +1510.3% 1786635 ± 2% meminfo.SReclaimable
355350 +2491.9% 9210381 ± 2% meminfo.SUnreclaim
466299 +2258.4% 10997018 ± 2% meminfo.Slab
10275 ± 4% +1425.3% 156721 ± 12% meminfo.max_used_kB
4460 ± 69% +203.3% 13529 ± 30% turbostat.C1
121615 ± 22% +519.9% 753893 ± 31% turbostat.C1E
0.09 ± 28% +0.5 0.58 ± 38% turbostat.C1E%
0.38 ± 33% +223.7% 1.23 ± 44% turbostat.CPU%c1
65242150 +47.3% 96089129 ± 8% turbostat.IRQ
132.58 -15.0% 112.71 turbostat.RAMWatt
4827550 -5.2% 4574251 proc-vmstat.nr_dirty_background_threshold
9666905 -5.2% 9159687 proc-vmstat.nr_dirty_threshold
48564619 -5.2% 46023595 proc-vmstat.nr_free_pages
27758 +1459.8% 432976 ± 4% proc-vmstat.nr_slab_reclaimable
89019 +2404.7% 2229685 ± 4% proc-vmstat.nr_slab_unreclaimable
11437217 ± 4% +370.4% 53800046 ± 8% proc-vmstat.numa_hit
11417078 ± 4% +371.0% 53779868 ± 8% proc-vmstat.numa_local
41390823 ± 4% +325.9% 1.763e+08 ± 8% proc-vmstat.pgalloc_normal
41333857 ± 4% +320.8% 1.74e+08 ± 8% proc-vmstat.pgfree
64817 ± 6% -10.6% 57927 ± 5% numa-vmstat.node0.nr_active_anon
14704 ± 8% +1506.2% 236186 ± 25% numa-vmstat.node0.nr_slab_reclaimable
46552 ± 4% +2516.0% 1217825 ± 26% numa-vmstat.node0.nr_slab_unreclaimable
64817 ± 6% -10.6% 57927 ± 5% numa-vmstat.node0.nr_zone_active_anon
3455933 ± 4% +314.1% 14310891 ± 26% numa-vmstat.node0.numa_hit
3415231 ± 4% +318.8% 14303349 ± 26% numa-vmstat.node0.numa_local
13060 ± 11% +1419.7% 198487 ± 21% numa-vmstat.node1.nr_slab_reclaimable
42619 ± 5% +2295.1% 1020779 ± 22% numa-vmstat.node1.nr_slab_unreclaimable
3431424 ± 5% +283.3% 13151338 ± 13% numa-vmstat.node1.numa_hit
3282191 ± 6% +295.1% 12969052 ± 13% numa-vmstat.node1.numa_local
259176 ± 6% -10.3% 232594 ± 5% numa-meminfo.node0.Active
259046 ± 6% -10.2% 232503 ± 5% numa-meminfo.node0.Active(anon)
58795 ± 8% +1473.5% 925121 ± 24% numa-meminfo.node0.KReclaimable
1302386 +413.3% 6684751 ± 21% numa-meminfo.node0.MemUsed
58795 ± 8% +1473.5% 925121 ± 24% numa-meminfo.node0.SReclaimable
185796 ± 4% +2465.9% 4767407 ± 25% numa-meminfo.node0.SUnreclaim
244591 ± 5% +2227.4% 5692529 ± 25% numa-meminfo.node0.Slab
52252 ± 11% +1477.6% 824327 ± 21% numa-meminfo.node1.KReclaimable
1170598 ± 2% +418.0% 6063147 ± 17% numa-meminfo.node1.MemUsed
52252 ± 11% +1477.6% 824327 ± 21% numa-meminfo.node1.SReclaimable
170522 ± 5% +2389.5% 4245068 ± 22% numa-meminfo.node1.SUnreclaim
222775 ± 6% +2175.6% 5069395 ± 22% numa-meminfo.node1.Slab
347350 +2486.4% 8983942 ± 4% slabinfo.Acpi-Parse.active_objs
4759 +2486.0% 123079 ± 4% slabinfo.Acpi-Parse.active_slabs
347475 +2485.7% 8984832 ± 4% slabinfo.Acpi-Parse.num_objs
4759 +2486.0% 123079 ± 4% slabinfo.Acpi-Parse.num_slabs
1573 ± 3% +17.7% 1851 ± 7% slabinfo.UNIX.active_objs
1573 ± 3% +17.7% 1851 ± 7% slabinfo.UNIX.num_objs
6010 +34.2% 8063 ± 4% slabinfo.cred_jar.active_objs
6010 +34.2% 8063 ± 4% slabinfo.cred_jar.num_objs
277551 +3135.5% 8980098 ± 4% slabinfo.dentry.active_objs
6654 +3113.7% 213846 ± 4% slabinfo.dentry.active_slabs
279498 +3113.5% 8981562 ± 4% slabinfo.dentry.num_objs
6654 +3113.7% 213846 ± 4% slabinfo.dentry.num_slabs
4184 ± 6% +11.7% 4672 ± 5% slabinfo.files_cache.active_objs
4184 ± 6% +11.7% 4672 ± 5% slabinfo.files_cache.num_objs
187757 ± 2% +4641.3% 8902186 ± 4% slabinfo.filp.active_objs
5933 +4589.3% 278216 ± 4% slabinfo.filp.active_slabs
189878 +4588.8% 8902929 ± 4% slabinfo.filp.num_objs
5933 +4589.3% 278216 ± 4% slabinfo.filp.num_slabs
94.50 +106.9% 195.50 ± 21% slabinfo.nfs_read_data.active_objs
94.50 +106.9% 195.50 ± 21% slabinfo.nfs_read_data.num_objs
4511 +18.0% 5324 slabinfo.pid.active_objs
4511 +18.0% 5324 slabinfo.pid.num_objs
191949 ± 2% +4538.2% 8902980 ± 4% slabinfo.shmem_inode_cache.active_objs
4236 +4469.4% 193561 ± 4% slabinfo.shmem_inode_cache.active_slabs
194891 +4468.6% 8903831 ± 4% slabinfo.shmem_inode_cache.num_objs
4236 +4469.4% 193561 ± 4% slabinfo.shmem_inode_cache.num_slabs
3829 ± 4% +24.8% 4780 ± 3% slabinfo.signal_cache.active_objs
3829 ± 4% +24.9% 4783 ± 3% slabinfo.signal_cache.num_objs
2688 +14.1% 3068 ± 4% slabinfo.sock_inode_cache.active_objs
2688 +14.1% 3068 ± 4% slabinfo.sock_inode_cache.num_objs
5894 +12.5% 6633 ± 2% slabinfo.task_delay_info.active_objs
5894 +12.5% 6633 ± 2% slabinfo.task_delay_info.num_objs
105.00 ± 17% +107.4% 217.75 ± 31% slabinfo.xfrm_state.active_objs
105.00 ± 17% +107.4% 217.75 ± 31% slabinfo.xfrm_state.num_objs
13.70 ± 6% +22.6% 16.80 ± 6% perf-stat.i.MPKI
4.808e+09 +32.6% 6.373e+09 perf-stat.i.branch-instructions
1.00 +0.1 1.11 perf-stat.i.branch-miss-rate%
48169497 ± 2% +41.7% 68233089 perf-stat.i.branch-misses
43.03 -33.0 10.03 ± 5% perf-stat.i.cache-miss-rate%
1.345e+08 ± 5% -62.3% 50717542 perf-stat.i.cache-misses
3.125e+08 ± 5% +65.0% 5.157e+08 ± 6% perf-stat.i.cache-references
1662 ± 5% -16.6% 1385 ± 2% perf-stat.i.context-switches
12.65 -26.4% 9.31 perf-stat.i.cpi
6.37 ± 10% +183.5% 18.06 ± 7% perf-stat.i.cpu-migrations
2152 ± 5% +307.8% 8778 perf-stat.i.cycles-between-cache-misses
0.07 ± 4% +0.0 0.11 ± 19% perf-stat.i.dTLB-load-miss-rate%
4356223 ± 5% +120.6% 9611496 ± 18% perf-stat.i.dTLB-load-misses
6.427e+09 +38.0% 8.868e+09 ± 2% perf-stat.i.dTLB-loads
274203 ± 7% +129.3% 628638 ± 29% perf-stat.i.dTLB-store-misses
3.114e+09 ± 2% +54.5% 4.811e+09 perf-stat.i.dTLB-stores
98.37 -2.7 95.64 perf-stat.i.iTLB-load-miss-rate%
2207761 ± 2% +83.7% 4054811 perf-stat.i.iTLB-load-misses
44223 ± 40% +325.7% 188244 ± 16% perf-stat.i.iTLB-loads
2.282e+10 +35.9% 3.1e+10 perf-stat.i.instructions
10372 -22.3% 8060 perf-stat.i.instructions-per-iTLB-miss
0.08 +38.4% 0.11 perf-stat.i.ipc
2514 -23.0% 1935 perf-stat.i.minor-faults
88.91 -49.8 39.12 ± 2% perf-stat.i.node-load-miss-rate%
25465414 ± 8% -98.8% 316344 ± 8% perf-stat.i.node-load-misses
68.94 -65.4 3.51 ± 29% perf-stat.i.node-store-miss-rate%
12879640 ± 7% -96.8% 407503 ± 17% perf-stat.i.node-store-misses
5787040 ± 4% +76.0% 10184031 ± 3% perf-stat.i.node-stores
2514 -22.8% 1941 perf-stat.i.page-faults
13.70 ± 6% +21.2% 16.61 ± 7% perf-stat.overall.MPKI
1.00 +0.1 1.06 perf-stat.overall.branch-miss-rate%
43.04 -33.6 9.39 ± 7% perf-stat.overall.cache-miss-rate%
12.65 -27.2% 9.22 perf-stat.overall.cpi
2152 ± 5% +175.9% 5938 perf-stat.overall.cycles-between-cache-misses
0.07 ± 4% +0.0 0.11 ± 17% perf-stat.overall.dTLB-load-miss-rate%
0.01 ± 6% +0.0 0.01 ± 26% perf-stat.overall.dTLB-store-miss-rate%
98.04 -2.0 96.09 perf-stat.overall.iTLB-load-miss-rate%
10338 -26.2% 7629 perf-stat.overall.instructions-per-iTLB-miss
0.08 +37.3% 0.11 perf-stat.overall.ipc
88.91 -79.1 9.76 ± 8% perf-stat.overall.node-load-miss-rate%
68.94 -65.4 3.58 ± 18% perf-stat.overall.node-store-miss-rate%
7896490 -12.9% 6879794 perf-stat.overall.path-length
4.792e+09 +32.1% 6.332e+09 perf-stat.ps.branch-instructions
48010074 ± 2% +39.3% 66892751 ± 2% perf-stat.ps.branch-misses
1.34e+08 ± 5% -64.4% 47760401 perf-stat.ps.cache-misses
3.114e+08 ± 5% +63.9% 5.105e+08 ± 5% perf-stat.ps.cache-references
1656 ± 5% -25.0% 1242 perf-stat.ps.context-switches
2.876e+11 -1.4% 2.836e+11 perf-stat.ps.cpu-cycles
6.35 ± 10% +156.7% 16.31 ± 6% perf-stat.ps.cpu-migrations
4341603 ± 5% +121.6% 9620245 ± 15% perf-stat.ps.dTLB-load-misses
6.406e+09 +37.8% 8.825e+09 ± 3% perf-stat.ps.dTLB-loads
273284 ± 7% +128.2% 623657 ± 25% perf-stat.ps.dTLB-store-misses
3.103e+09 ± 2% +53.7% 4.769e+09 perf-stat.ps.dTLB-stores
2200347 ± 2% +83.3% 4034144 ± 2% perf-stat.ps.iTLB-load-misses
44087 ± 40% +272.6% 164273 ± 17% perf-stat.ps.iTLB-loads
2.274e+10 +35.3% 3.078e+10 perf-stat.ps.instructions
2505 -36.6% 1588 perf-stat.ps.minor-faults
25379917 ± 8% -98.9% 291377 ± 10% perf-stat.ps.node-load-misses
3161287 ± 7% -15.0% 2688539 perf-stat.ps.node-loads
12836399 ± 7% -97.1% 373482 ± 16% perf-stat.ps.node-store-misses
5767639 ± 4% +75.1% 10101195 ± 3% perf-stat.ps.node-stores
2506 -36.6% 1588 perf-stat.ps.page-faults
6.852e+12 +102.6% 1.388e+13 ± 8% perf-stat.total.instructions
10839 ±172% +54121.1% 5877405 ± 63% sched_debug.cfs_rq:/.MIN_vruntime.avg
1127316 ±172% +1774.8% 21135287 ± 20% sched_debug.cfs_rq:/.MIN_vruntime.max
110010 ±172% +5871.6% 6569376 ± 46% sched_debug.cfs_rq:/.MIN_vruntime.stddev
148797 +39.9% 208188 ± 10% sched_debug.cfs_rq:/.exec_clock.avg
150120 +40.8% 211431 ± 10% sched_debug.cfs_rq:/.exec_clock.max
147571 +35.7% 200227 ± 12% sched_debug.cfs_rq:/.exec_clock.min
281.51 ± 14% +814.2% 2573 ± 22% sched_debug.cfs_rq:/.exec_clock.stddev
10953 ± 13% +2236.0% 255873 ± 50% sched_debug.cfs_rq:/.load.avg
193044 ± 62% +492.2% 1143238 ± 19% sched_debug.cfs_rq:/.load.max
20205 ± 59% +1280.5% 278935 ± 36% sched_debug.cfs_rq:/.load.stddev
16.14 ± 12% +1767.4% 301.32 ± 47% sched_debug.cfs_rq:/.load_avg.avg
199.67 ± 2% +497.4% 1192 ± 13% sched_debug.cfs_rq:/.load_avg.max
7.50 +330.1% 32.26 ± 86% sched_debug.cfs_rq:/.load_avg.min
31.78 ± 17% +763.5% 274.40 ± 27% sched_debug.cfs_rq:/.load_avg.stddev
10839 ±172% +54121.1% 5877405 ± 63% sched_debug.cfs_rq:/.max_vruntime.avg
1127316 ±172% +1774.8% 21135287 ± 20% sched_debug.cfs_rq:/.max_vruntime.max
110010 ±172% +5871.6% 6569376 ± 46% sched_debug.cfs_rq:/.max_vruntime.stddev
16820406 +31.5% 22110572 ± 10% sched_debug.cfs_rq:/.min_vruntime.avg
16938980 +35.2% 22893221 ± 10% sched_debug.cfs_rq:/.min_vruntime.max
16235991 +23.1% 19989003 ± 9% sched_debug.cfs_rq:/.min_vruntime.min
135956 ± 11% +311.7% 559778 ± 15% sched_debug.cfs_rq:/.min_vruntime.stddev
0.85 +30.9% 1.12 ± 11% sched_debug.cfs_rq:/.nr_running.avg
1.08 ± 7% +80.7% 1.96 ± 9% sched_debug.cfs_rq:/.nr_running.max
0.06 ± 17% +407.0% 0.30 ± 31% sched_debug.cfs_rq:/.nr_running.stddev
764.54 ± 28% -35.6% 492.31 ± 31% sched_debug.cfs_rq:/.nr_spread_over.max
57.08 ± 41% +214.6% 179.61 ± 14% sched_debug.cfs_rq:/.nr_spread_over.min
167.02 ± 25% -76.6% 39.07 ± 22% sched_debug.cfs_rq:/.nr_spread_over.stddev
7.97 +2791.3% 230.53 ± 49% sched_debug.cfs_rq:/.runnable_load_avg.avg
22.96 ± 19% +3924.6% 923.98 ± 13% sched_debug.cfs_rq:/.runnable_load_avg.max
2.18 ± 14% +11226.4% 246.85 ± 36% sched_debug.cfs_rq:/.runnable_load_avg.stddev
10826 ± 13% +2261.8% 255687 ± 50% sched_debug.cfs_rq:/.runnable_weight.avg
192124 ± 63% +494.1% 1141432 ± 19% sched_debug.cfs_rq:/.runnable_weight.max
20076 ± 60% +1288.2% 278706 ± 36% sched_debug.cfs_rq:/.runnable_weight.stddev
-31244 -2590.9% 778258 ± 39% sched_debug.cfs_rq:/.spread0.avg
86667 ± 32% +1700.5% 1560442 ± 24% sched_debug.cfs_rq:/.spread0.max
-615171 +118.4% -1343411 sched_debug.cfs_rq:/.spread0.min
135884 ± 11% +311.5% 559142 ± 15% sched_debug.cfs_rq:/.spread0.stddev
1006 ± 7% +44.3% 1452 ± 22% sched_debug.cfs_rq:/.util_est_enqueued.max
1045889 ± 4% +41.9% 1484052 ± 12% sched_debug.cpu.avg_idle.max
89274 ± 22% +58.5% 141492 ± 10% sched_debug.cpu.avg_idle.stddev
192292 +33.2% 256212 ± 7% sched_debug.cpu.clock.avg
192306 +33.3% 256262 ± 7% sched_debug.cpu.clock.max
192279 +33.1% 256002 ± 7% sched_debug.cpu.clock.min
8.36 +851.9% 79.59 ±140% sched_debug.cpu.clock.stddev
192292 +33.2% 256212 ± 7% sched_debug.cpu.clock_task.avg
192306 +33.3% 256262 ± 7% sched_debug.cpu.clock_task.max
192279 +33.1% 256002 ± 7% sched_debug.cpu.clock_task.min
8.36 +851.9% 79.59 ±140% sched_debug.cpu.clock_task.stddev
1838 ± 2% -20.5% 1461 ± 17% sched_debug.cpu.curr->pid.min
514860 ± 3% +19.3% 614079 ± 7% sched_debug.cpu.max_idle_balance_cost.max
2432 ±129% +517.4% 15019 ± 49% sched_debug.cpu.max_idle_balance_cost.stddev
0.87 +43.8% 1.25 ± 15% sched_debug.cpu.nr_running.avg
1.83 ± 6% +84.9% 3.39 ± 24% sched_debug.cpu.nr_running.max
0.16 ± 12% +213.1% 0.51 ± 32% sched_debug.cpu.nr_running.stddev
1446 ± 7% +16.8% 1690 ± 7% sched_debug.cpu.nr_switches.min
2513 ± 5% +128.2% 5734 ± 10% sched_debug.cpu.sched_count.avg
483.42 ± 11% +619.4% 3477 ± 9% sched_debug.cpu.sched_count.min
53.52 ± 12% +195.5% 158.16 ± 28% sched_debug.cpu.sched_goidle.avg
1835 +27.3% 2337 ± 14% sched_debug.cpu.sched_goidle.max
2.50 ± 42% +231.3% 8.28 ± 27% sched_debug.cpu.sched_goidle.min
211.34 ± 10% +78.9% 378.04 ± 22% sched_debug.cpu.sched_goidle.stddev
192279 +33.1% 255999 ± 7% sched_debug.cpu_clk
189412 +33.6% 253129 ± 7% sched_debug.ktime
192681 +33.1% 256393 ± 7% sched_debug.sched_clk
100412 ± 2% +84.5% 185216 ± 10% softirqs.CPU0.RCU
9764 ± 6% +42.6% 13922 ± 15% softirqs.CPU0.SCHED
94451 ± 21% +81.4% 171304 ± 14% softirqs.CPU1.RCU
80870 ± 4% +106.3% 166819 ± 14% softirqs.CPU10.RCU
80014 ± 23% +98.2% 158562 ± 5% softirqs.CPU100.RCU
105851 ± 2% +47.0% 155567 ± 8% softirqs.CPU100.TIMER
68652 ± 3% +130.7% 158403 ± 2% softirqs.CPU101.RCU
104028 ± 2% +48.4% 154370 ± 8% softirqs.CPU101.TIMER
91114 ± 23% +75.3% 159723 ± 3% softirqs.CPU102.RCU
103977 ± 2% +48.1% 153964 ± 7% softirqs.CPU102.TIMER
69089 +132.7% 160776 ± 3% softirqs.CPU103.RCU
104589 +55.8% 162958 ± 11% softirqs.CPU103.TIMER
91256 ± 19% +83.0% 166975 ± 16% softirqs.CPU11.RCU
102284 ± 21% +63.0% 166684 ± 15% softirqs.CPU12.RCU
91374 ± 19% +82.1% 166382 ± 15% softirqs.CPU13.RCU
81052 ± 5% +106.4% 167310 ± 13% softirqs.CPU14.RCU
84049 ± 3% +100.9% 168821 ± 12% softirqs.CPU15.RCU
92354 ± 18% +79.7% 165972 ± 16% softirqs.CPU16.RCU
78917 +110.2% 165869 ± 12% softirqs.CPU17.RCU
90660 ± 20% +84.3% 167088 ± 13% softirqs.CPU18.RCU
90523 ± 20% +86.0% 168331 ± 13% softirqs.CPU19.RCU
82191 ± 3% +107.8% 170827 ± 13% softirqs.CPU2.RCU
79685 ± 5% +113.0% 169689 ± 12% softirqs.CPU20.RCU
78964 ± 3% +113.4% 168548 ± 12% softirqs.CPU21.RCU
88494 ± 20% +89.4% 167616 ± 12% softirqs.CPU22.RCU
78594 ± 5% +117.4% 170884 ± 12% softirqs.CPU23.RCU
103142 ± 17% +61.3% 166355 ± 15% softirqs.CPU24.RCU
80771 +103.9% 164673 ± 14% softirqs.CPU25.RCU
76673 ± 6% +122.1% 170303 ± 5% softirqs.CPU26.RCU
109075 +43.3% 156320 ± 7% softirqs.CPU26.TIMER
72057 ± 2% +120.9% 159152 ± 4% softirqs.CPU27.RCU
110068 +42.3% 156592 ± 7% softirqs.CPU27.TIMER
72911 ± 6% +122.5% 162250 ± 3% softirqs.CPU28.RCU
108856 ± 2% +45.2% 158064 ± 7% softirqs.CPU28.TIMER
72889 ± 4% +107.8% 151457 ± 2% softirqs.CPU29.RCU
106372 ± 2% +26.8% 134847 ± 4% softirqs.CPU29.TIMER
77832 ± 2% +115.7% 167869 ± 11% softirqs.CPU3.RCU
72522 ± 3% +123.0% 161702 ± 6% softirqs.CPU30.RCU
106786 +55.3% 165848 ± 12% softirqs.CPU30.TIMER
72345 ± 5% +117.2% 157134 ± 3% softirqs.CPU31.RCU
106581 +47.1% 156760 ± 7% softirqs.CPU31.TIMER
71719 ± 6% +124.8% 161212 ± 5% softirqs.CPU32.RCU
105525 +44.4% 152342 ± 7% softirqs.CPU32.TIMER
72502 ± 2% +127.6% 165042 ± 4% softirqs.CPU33.RCU
108774 +43.9% 156488 ± 8% softirqs.CPU33.TIMER
71062 ± 4% +123.3% 158681 ± 4% softirqs.CPU34.RCU
108673 ± 2% +43.1% 155521 ± 7% softirqs.CPU34.TIMER
80505 ± 21% +98.8% 160061 ± 4% softirqs.CPU35.RCU
106188 ± 2% +46.6% 155631 ± 7% softirqs.CPU35.TIMER
71161 ± 2% +123.6% 159113 ± 3% softirqs.CPU36.RCU
107046 ± 4% +45.3% 155494 ± 7% softirqs.CPU36.TIMER
69979 ± 3% +128.7% 160051 ± 2% softirqs.CPU37.RCU
105652 ± 2% +46.7% 155000 ± 7% softirqs.CPU37.TIMER
72075 ± 4% +128.0% 164335 ± 3% softirqs.CPU38.RCU
105249 ± 2% +47.0% 154765 ± 7% softirqs.CPU38.TIMER
73488 ± 6% +112.1% 155843 ± 3% softirqs.CPU39.RCU
108405 ± 2% +43.8% 155892 ± 7% softirqs.CPU39.TIMER
79885 ± 4% +109.8% 167568 ± 14% softirqs.CPU4.RCU
73250 ± 4% +112.7% 155789 ± 5% softirqs.CPU40.RCU
107897 +44.9% 156384 ± 7% softirqs.CPU40.TIMER
72089 ± 3% +124.8% 162054 ± 5% softirqs.CPU41.RCU
106170 ± 2% +47.3% 156357 ± 8% softirqs.CPU41.TIMER
70291 ± 2% +126.1% 158949 ± 3% softirqs.CPU42.RCU
103427 +65.5% 171135 ± 12% softirqs.CPU42.TIMER
72542 ± 2% +127.9% 165299 ± 4% softirqs.CPU43.RCU
103524 +50.3% 155647 ± 7% softirqs.CPU43.TIMER
71337 ± 3% +126.6% 161640 ± 4% softirqs.CPU44.RCU
103240 +50.7% 155534 ± 8% softirqs.CPU44.TIMER
77853 ± 13% +106.5% 160787 ± 2% softirqs.CPU45.RCU
103446 ± 2% +49.0% 154177 ± 7% softirqs.CPU45.TIMER
76067 ± 5% +103.3% 154644 ± 4% softirqs.CPU46.RCU
110212 +41.8% 156321 ± 7% softirqs.CPU46.TIMER
72476 ± 2% +123.6% 162053 ± 5% softirqs.CPU47.RCU
107617 +45.3% 156313 ± 7% softirqs.CPU47.TIMER
73117 ± 5% +111.6% 154686 ± 5% softirqs.CPU48.RCU
105754 +46.5% 154897 ± 8% softirqs.CPU48.TIMER
84349 ± 22% +83.4% 154700 ± 4% softirqs.CPU49.RCU
104750 +46.2% 153186 ± 7% softirqs.CPU49.TIMER
80769 ± 5% +112.6% 171704 ± 13% softirqs.CPU5.RCU
72530 +116.7% 157179 ± 4% softirqs.CPU50.RCU
104531 ± 2% +47.4% 154051 ± 8% softirqs.CPU50.TIMER
76998 ± 2% +111.2% 162628 ± 4% softirqs.CPU51.RCU
105267 ± 2% +65.9% 174689 ± 23% softirqs.CPU51.TIMER
79714 ± 3% +116.9% 172889 ± 12% softirqs.CPU52.RCU
92142 ± 20% +83.6% 169140 ± 14% softirqs.CPU53.RCU
91482 ± 22% +85.8% 169988 ± 13% softirqs.CPU54.RCU
82341 ± 3% +105.5% 169199 ± 13% softirqs.CPU55.RCU
101745 ± 21% +65.1% 168015 ± 12% softirqs.CPU56.RCU
90221 ± 23% +85.5% 167381 ± 14% softirqs.CPU57.RCU
104878 ± 14% +62.0% 169918 ± 15% softirqs.CPU58.RCU
101109 ± 23% +71.5% 173450 ± 10% softirqs.CPU59.RCU
81891 ± 3% +104.0% 167085 ± 14% softirqs.CPU6.RCU
89877 ± 18% +91.3% 171947 ± 14% softirqs.CPU60.RCU
77777 ± 2% +121.4% 172188 ± 14% softirqs.CPU61.RCU
80015 +109.7% 167786 ± 11% softirqs.CPU62.RCU
88572 ± 21% +89.5% 167825 ± 13% softirqs.CPU63.RCU
90477 ± 18% +88.4% 170427 ± 12% softirqs.CPU64.RCU
113980 ± 35% +49.7% 170640 ± 13% softirqs.CPU65.RCU
78250 ± 4% +113.4% 167009 ± 14% softirqs.CPU66.RCU
90308 ± 18% +90.4% 171925 ± 12% softirqs.CPU67.RCU
78807 ± 3% +117.5% 171403 ± 11% softirqs.CPU68.RCU
79295 ± 3% +112.9% 168847 ± 13% softirqs.CPU69.RCU
80563 ± 2% +108.7% 168101 ± 11% softirqs.CPU7.RCU
90877 ± 18% +87.2% 170085 ± 10% softirqs.CPU70.RCU
90744 ± 18% +91.8% 174011 ± 11% softirqs.CPU71.RCU
81756 ± 9% +106.6% 168888 ± 16% softirqs.CPU72.RCU
100001 ± 20% +70.5% 170545 ± 10% softirqs.CPU73.RCU
78085 ± 3% +115.9% 168562 ± 13% softirqs.CPU74.RCU
88915 ± 20% +92.4% 171110 ± 13% softirqs.CPU75.RCU
89823 ± 21% +91.0% 171562 ± 12% softirqs.CPU76.RCU
91041 ± 18% +90.0% 172960 ± 14% softirqs.CPU77.RCU
84844 ± 18% +88.5% 159898 softirqs.CPU78.RCU
106434 +47.4% 156892 ± 7% softirqs.CPU78.TIMER
84742 ± 22% +88.5% 159700 ± 3% softirqs.CPU79.RCU
107496 +44.7% 155578 ± 7% softirqs.CPU79.TIMER
92221 ± 18% +81.9% 167739 ± 14% softirqs.CPU8.RCU
76168 ± 6% +116.5% 164885 ± 5% softirqs.CPU80.RCU
107679 ± 3% +45.7% 156892 ± 7% softirqs.CPU80.TIMER
73865 ± 6% +121.6% 163693 ± 4% softirqs.CPU81.RCU
105024 ± 2% +48.8% 156224 ± 7% softirqs.CPU81.TIMER
73094 ± 4% +115.0% 157154 ± 5% softirqs.CPU82.RCU
105187 +66.5% 175185 ± 20% softirqs.CPU82.TIMER
72609 ± 4% +123.2% 162068 ± 2% softirqs.CPU83.RCU
105428 +48.8% 156835 ± 8% softirqs.CPU83.TIMER
69012 ± 3% +136.1% 162961 ± 3% softirqs.CPU84.RCU
103569 +49.1% 154450 ± 7% softirqs.CPU84.TIMER
71273 ± 4% +127.1% 161866 ± 4% softirqs.CPU85.RCU
107166 +46.9% 157387 ± 7% softirqs.CPU85.TIMER
73920 ± 4% +114.0% 158157 ± 4% softirqs.CPU86.RCU
106782 +45.6% 155519 ± 7% softirqs.CPU86.TIMER
70889 ± 2% +125.9% 160114 ± 3% softirqs.CPU87.RCU
104785 +49.3% 156428 ± 7% softirqs.CPU87.TIMER
71683 ± 5% +130.0% 164840 ± 5% softirqs.CPU88.RCU
105153 +50.3% 158068 ± 8% softirqs.CPU88.TIMER
73717 ± 5% +130.6% 169985 ± 3% softirqs.CPU89.RCU
3524 ± 5% +80.5% 6361 ± 36% softirqs.CPU89.SCHED
104448 +52.0% 158748 ± 8% softirqs.CPU89.TIMER
79472 ± 4% +112.9% 169172 ± 13% softirqs.CPU9.RCU
71726 ± 5% +123.7% 160453 ± 4% softirqs.CPU90.RCU
104820 ± 2% +47.8% 154920 ± 7% softirqs.CPU90.TIMER
72303 ± 4% +118.9% 158291 ± 5% softirqs.CPU91.RCU
106382 +48.5% 158024 ± 9% softirqs.CPU91.TIMER
81381 ± 21% +103.2% 165386 ± 3% softirqs.CPU92.RCU
105797 ± 2% +48.2% 156824 ± 8% softirqs.CPU92.TIMER
91365 ± 24% +80.6% 165015 ± 2% softirqs.CPU93.RCU
105028 +49.9% 157413 ± 8% softirqs.CPU93.TIMER
70463 ± 2% +134.1% 164988 ± 6% softirqs.CPU94.RCU
104130 +55.0% 161385 ± 5% softirqs.CPU94.TIMER
82135 ± 21% +96.3% 161243 ± 2% softirqs.CPU95.RCU
103162 +50.4% 155143 ± 7% softirqs.CPU95.TIMER
82367 ± 23% +96.2% 161563 ± 5% softirqs.CPU96.RCU
103303 +51.0% 155945 ± 8% softirqs.CPU96.TIMER
71859 ± 3% +130.0% 165302 ± 3% softirqs.CPU97.RCU
103733 ± 2% +49.1% 154615 ± 8% softirqs.CPU97.TIMER
71721 ± 3% +124.2% 160821 ± 6% softirqs.CPU98.RCU
109690 +44.9% 158963 ± 7% softirqs.CPU98.TIMER
70619 ± 2% +132.1% 163916 ± 2% softirqs.CPU99.RCU
106893 +46.7% 156786 ± 7% softirqs.CPU99.TIMER
8450379 ± 3% +103.3% 17178867 ± 7% softirqs.RCU
380275 +42.6% 542185 ± 7% softirqs.SCHED
12568587 +30.3% 16373516 ± 9% softirqs.TIMER
28.20 ± 36% -28.2 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.evict.do_unlinkat.do_syscall_64
28.13 ± 36% -28.1 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.inode_sb_list_add.new_inode.shmem_get_inode
12.50 ± 81% -12.5 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.inode_doinit_with_dentry.security_d_instantiate.d_instantiate
12.46 ± 81% -12.5 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.selinux_inode_free_security.security_inode_free.__destroy_inode
13.24 ± 76% -11.2 2.05 ± 21% perf-profile.calltrace.cycles-pp.__destroy_inode.destroy_inode.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.05 ± 78% -11.1 2.00 ± 21% perf-profile.calltrace.cycles-pp.d_instantiate.shmem_mknod.path_openat.do_filp_open.do_sys_open
12.76 ± 79% -11.0 1.72 ± 25% perf-profile.calltrace.cycles-pp.security_inode_free.__destroy_inode.destroy_inode.do_unlinkat.do_syscall_64
12.76 ± 79% -11.0 1.72 ± 25% perf-profile.calltrace.cycles-pp.selinux_inode_free_security.security_inode_free.__destroy_inode.destroy_inode.do_unlinkat
12.87 ± 79% -11.0 1.85 ± 22% perf-profile.calltrace.cycles-pp.security_d_instantiate.d_instantiate.shmem_mknod.path_openat.do_filp_open
12.85 ± 79% -11.0 1.84 ± 23% perf-profile.calltrace.cycles-pp.inode_doinit_with_dentry.security_d_instantiate.d_instantiate.shmem_mknod.path_openat
12.60 ± 80% -11.0 1.63 ± 27% perf-profile.calltrace.cycles-pp._raw_spin_lock.selinux_inode_free_security.security_inode_free.__destroy_inode.destroy_inode
12.65 ± 80% -10.9 1.73 ± 24% perf-profile.calltrace.cycles-pp._raw_spin_lock.inode_doinit_with_dentry.security_d_instantiate.d_instantiate.shmem_mknod
2.38 ± 27% -1.1 1.24 ± 21% perf-profile.calltrace.cycles-pp._raw_spin_lock.__alloc_fd.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.32 ± 28% -1.1 1.23 ± 22% perf-profile.calltrace.cycles-pp._raw_spin_lock.__close_fd.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.65 ± 5% +0.1 0.71 ± 2% perf-profile.calltrace.cycles-pp.new_inode_pseudo.new_inode.shmem_get_inode.shmem_mknod.path_openat
0.53 ± 2% +0.3 0.80 ± 14% perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_open.do_syscall_64
0.28 ±100% +0.3 0.57 ± 3% perf-profile.calltrace.cycles-pp.alloc_inode.new_inode_pseudo.new_inode.shmem_get_inode.shmem_mknod
0.54 ± 3% +0.4 0.91 ± 13% perf-profile.calltrace.cycles-pp.path_parentat.filename_parentat.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.27 ±100% +0.5 0.72 ± 13% perf-profile.calltrace.cycles-pp.alloc_empty_file.path_openat.do_filp_open.do_sys_open.do_syscall_64
0.00 +0.7 0.65 ± 13% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.__cna_queued_spin_lock_slowpath._raw_spin_lock.evict
0.00 +0.7 0.67 ± 14% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.__cna_queued_spin_lock_slowpath._raw_spin_lock.evict.do_unlinkat
0.00 +0.7 0.73 ± 14% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.shmem_reserve_inode.shmem_get_inode.shmem_mknod
0.00 +0.7 0.74 ± 13% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.__cna_queued_spin_lock_slowpath._raw_spin_lock.inode_sb_list_add
0.00 +0.8 0.75 ± 12% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.__cna_queued_spin_lock_slowpath._raw_spin_lock.inode_sb_list_add.new_inode
0.00 +0.8 0.78 ± 15% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.shmem_free_inode.shmem_evict_inode.evict
0.00 +0.8 0.80 ± 16% perf-profile.calltrace.cycles-pp.link_path_walk.path_parentat.filename_parentat.do_unlinkat.do_syscall_64
0.00 +1.1 1.13 ± 23% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.__close_fd.__x64_sys_close.do_syscall_64
0.00 +1.1 1.15 ± 22% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.__alloc_fd.do_sys_open.do_syscall_64
0.00 +1.6 1.56 ± 28% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.selinux_inode_free_security.security_inode_free.__destroy_inode
0.00 +1.6 1.62 ± 26% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.inode_doinit_with_dentry.security_d_instantiate.d_instantiate
0.00 +2.2 2.24 ± 62% perf-profile.calltrace.cycles-pp.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd
0.00 +3.3 3.27 ± 60% perf-profile.calltrace.cycles-pp.file_free_rcu.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd
0.14 ±173% +6.0 6.14 ± 60% perf-profile.calltrace.cycles-pp.ret_from_fork
0.14 ±173% +6.0 6.14 ± 60% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
0.00 +6.0 6.00 ± 61% perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn
0.00 +6.0 6.00 ± 61% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread
0.00 +6.0 6.01 ± 61% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork
0.00 +6.0 6.01 ± 61% perf-profile.calltrace.cycles-pp.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork
0.00 +6.0 6.02 ± 61% perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork
30.67 ± 33% +11.0 41.68 ± 5% perf-profile.calltrace.cycles-pp.shmem_get_inode.shmem_mknod.path_openat.do_filp_open.do_sys_open
28.42 ± 36% +11.2 39.64 ± 5% perf-profile.calltrace.cycles-pp.inode_sb_list_add.new_inode.shmem_get_inode.shmem_mknod.path_openat
28.25 ± 36% +11.3 39.52 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.inode_sb_list_add.new_inode.shmem_get_inode.shmem_mknod
29.08 ± 35% +11.3 40.36 ± 5% perf-profile.calltrace.cycles-pp.new_inode.shmem_get_inode.shmem_mknod.path_openat.do_filp_open
0.00 +34.3 34.25 ± 5% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.evict.do_unlinkat.do_syscall_64
0.00 +39.4 39.36 ± 5% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.inode_sb_list_add.new_inode.shmem_get_inode
87.78 -87.8 0.00 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
13.29 ± 76% -11.2 2.10 ± 21% perf-profile.children.cycles-pp.destroy_inode
13.24 ± 76% -11.2 2.06 ± 21% perf-profile.children.cycles-pp.__destroy_inode
13.06 ± 78% -11.1 2.00 ± 22% perf-profile.children.cycles-pp.d_instantiate
12.76 ± 79% -11.0 1.72 ± 25% perf-profile.children.cycles-pp.security_inode_free
12.76 ± 79% -11.0 1.72 ± 25% perf-profile.children.cycles-pp.selinux_inode_free_security
12.87 ± 79% -11.0 1.85 ± 22% perf-profile.children.cycles-pp.security_d_instantiate
12.85 ± 79% -11.0 1.84 ± 23% perf-profile.children.cycles-pp.inode_doinit_with_dentry
89.34 -6.5 82.80 ± 4% perf-profile.children.cycles-pp._raw_spin_lock
98.94 -6.0 92.90 ± 3% perf-profile.children.cycles-pp.do_syscall_64
98.96 -6.0 92.93 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
45.30 -4.8 40.55 ± 4% perf-profile.children.cycles-pp.do_unlinkat
2.61 ± 25% -1.2 1.40 ± 18% perf-profile.children.cycles-pp.__alloc_fd
2.42 ± 27% -1.1 1.31 ± 21% perf-profile.children.cycles-pp.__close_fd
2.59 ± 25% -1.1 1.48 ± 19% perf-profile.children.cycles-pp.__x64_sys_close
0.36 ± 15% -0.2 0.21 ± 18% perf-profile.children.cycles-pp.__mnt_want_write
0.39 ± 16% -0.1 0.27 ± 13% perf-profile.children.cycles-pp.mnt_want_write
0.20 ± 2% -0.1 0.11 ± 7% perf-profile.children.cycles-pp.__list_del_entry_valid
0.26 ± 7% -0.1 0.18 ± 8% perf-profile.children.cycles-pp.__list_add_valid
0.27 ± 7% -0.1 0.20 ± 4% perf-profile.children.cycles-pp.rcu_segcblist_enqueue
0.13 ± 6% -0.1 0.07 perf-profile.children.cycles-pp.drop_nlink
0.23 ± 5% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.shmem_unlink
0.09 ± 4% -0.0 0.04 ± 57% perf-profile.children.cycles-pp.find_next_zero_bit
0.26 +0.0 0.28 ± 2% perf-profile.children.cycles-pp.d_alloc_parallel
0.06 ± 11% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.fsnotify_grab_connector
0.05 +0.0 0.07 ± 10% perf-profile.children.cycles-pp.path_init
0.05 +0.0 0.07 ± 10% perf-profile.children.cycles-pp.__follow_mount_rcu
0.06 ± 7% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.dentry_unlink_inode
0.08 ± 6% +0.0 0.10 ± 8% perf-profile.children.cycles-pp.fsnotify_destroy_marks
0.08 ± 6% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.avc_has_perm
0.07 ± 10% +0.0 0.09 ± 11% perf-profile.children.cycles-pp.__d_lookup_rcu
0.17 ± 4% +0.0 0.20 ± 2% perf-profile.children.cycles-pp.simple_lookup
0.14 ± 5% +0.0 0.17 ± 2% perf-profile.children.cycles-pp.__d_alloc
0.08 ± 5% +0.0 0.11 ± 10% perf-profile.children.cycles-pp.__might_sleep
0.04 ± 58% +0.0 0.07 perf-profile.children.cycles-pp.lockref_get
0.06 ± 6% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.lockref_put_or_lock
0.04 ± 57% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.down_write
0.04 ± 57% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.inode_security_rcu
0.05 ± 8% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.legitimize_path
0.11 ± 4% +0.0 0.14 ± 5% perf-profile.children.cycles-pp.security_inode_alloc
0.08 ± 6% +0.0 0.11 perf-profile.children.cycles-pp.__dentry_kill
0.16 ± 2% +0.0 0.20 ± 3% perf-profile.children.cycles-pp.d_alloc
0.07 ± 7% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.06 +0.0 0.10 ± 4% perf-profile.children.cycles-pp.unlazy_walk
0.10 ± 12% +0.0 0.14 ± 15% perf-profile.children.cycles-pp.memcpy_erms
0.08 +0.0 0.12 ± 5% perf-profile.children.cycles-pp.avc_has_perm_noaudit
0.08 ± 5% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.d_add
0.07 ± 6% +0.0 0.11 ± 4% perf-profile.children.cycles-pp.complete_walk
0.15 +0.0 0.19 ± 3% perf-profile.children.cycles-pp.__fput
0.07 ± 11% +0.0 0.12 ± 5% perf-profile.children.cycles-pp.__check_object_size
0.07 ± 5% +0.0 0.12 ± 5% perf-profile.children.cycles-pp.inode_init_once
0.08 ± 5% +0.0 0.12 ± 8% perf-profile.children.cycles-pp.shmem_undo_range
0.14 ± 5% +0.0 0.19 ± 7% perf-profile.children.cycles-pp.lookup_fast
0.12 ± 7% +0.0 0.17 ± 6% perf-profile.children.cycles-pp.shmem_truncate_range
0.10 ± 8% +0.0 0.15 ± 4% perf-profile.children.cycles-pp.__inode_security_revalidate
0.52 ± 6% +0.1 0.57 ± 3% perf-profile.children.cycles-pp.alloc_inode
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.__virt_addr_valid
0.01 ±173% +0.1 0.07 ± 6% perf-profile.children.cycles-pp.__d_lookup_done
0.66 ± 5% +0.1 0.71 ± 2% perf-profile.children.cycles-pp.new_inode_pseudo
0.17 ± 5% +0.1 0.22 ± 6% perf-profile.children.cycles-pp.walk_component
0.17 ± 9% +0.1 0.23 ± 9% perf-profile.children.cycles-pp.___might_sleep
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.find_get_entries
0.21 ± 10% +0.1 0.28 ± 7% perf-profile.children.cycles-pp.shmem_alloc_inode
0.12 +0.1 0.18 ± 7% perf-profile.children.cycles-pp.__x64_sys_unlink
0.14 ± 3% +0.1 0.20 ± 6% perf-profile.children.cycles-pp.strncpy_from_user
0.23 +0.1 0.30 ± 3% perf-profile.children.cycles-pp.task_work_run
0.08 ± 6% +0.1 0.15 ± 5% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.25 +0.1 0.33 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop
0.12 ± 5% +0.1 0.20 ± 19% perf-profile.children.cycles-pp.may_open
0.14 ± 3% +0.1 0.23 ± 6% perf-profile.children.cycles-pp.__slab_alloc
0.11 ± 4% +0.1 0.20 ± 7% perf-profile.children.cycles-pp.new_slab
0.13 ± 6% +0.1 0.23 ± 6% perf-profile.children.cycles-pp.___slab_alloc
0.16 ± 10% +0.1 0.26 ± 20% perf-profile.children.cycles-pp.may_delete
0.21 ± 2% +0.1 0.32 ± 5% perf-profile.children.cycles-pp.getname_flags
0.24 ± 2% +0.1 0.35 ± 2% perf-profile.children.cycles-pp.dput
0.70 ± 3% +0.1 0.81 ± 6% perf-profile.children.cycles-pp.vfs_unlink
0.29 ± 5% +0.1 0.40 ± 5% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.28 ± 6% +0.1 0.40 ± 10% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.39 ± 5% +0.2 0.55 ± 5% perf-profile.children.cycles-pp.selinux_inode_permission
0.41 ± 5% +0.2 0.58 ± 5% perf-profile.children.cycles-pp.security_inode_permission
0.00 +0.2 0.17 ± 44% perf-profile.children.cycles-pp.shmem_free_in_core_inode
0.44 ± 6% +0.2 0.61 ± 5% perf-profile.children.cycles-pp.kmem_cache_alloc
0.52 ± 3% +0.2 0.72 ± 13% perf-profile.children.cycles-pp.alloc_empty_file
0.45 ± 3% +0.2 0.66 ± 15% perf-profile.children.cycles-pp.__alloc_file
0.00 +0.2 0.25 ± 4% perf-profile.children.cycles-pp.cna_scan_main_queue
0.08 ± 10% +0.3 0.34 ± 45% perf-profile.children.cycles-pp.rcu_cblist_dequeue
1.35 ± 2% +0.3 1.69 ± 9% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
0.99 ± 2% +0.3 1.34 ± 12% perf-profile.children.cycles-pp.irq_exit
1.42 ± 2% +0.4 1.79 ± 9% perf-profile.children.cycles-pp.apic_timer_interrupt
0.54 ± 3% +0.4 0.92 ± 14% perf-profile.children.cycles-pp.path_parentat
0.56 ± 3% +0.4 0.94 ± 13% perf-profile.children.cycles-pp.filename_parentat
1.00 ± 2% +0.6 1.61 ± 15% perf-profile.children.cycles-pp.link_path_walk
0.01 ±173% +0.8 0.78 ± 67% perf-profile.children.cycles-pp.free_one_page
0.01 ±173% +0.8 0.81 ± 67% perf-profile.children.cycles-pp.__free_pages_ok
0.03 ±102% +0.9 0.97 ± 66% perf-profile.children.cycles-pp.unfreeze_partials
0.03 ±105% +1.0 0.98 ± 65% perf-profile.children.cycles-pp.put_cpu_partial
0.24 ± 10% +1.2 1.44 ± 48% perf-profile.children.cycles-pp.__slab_free
0.36 ± 11% +2.2 2.60 ± 53% perf-profile.children.cycles-pp.kmem_cache_free
0.70 ± 10% +3.3 3.99 ± 47% perf-profile.children.cycles-pp.file_free_rcu
0.29 ± 40% +5.7 6.01 ± 61% perf-profile.children.cycles-pp.run_ksoftirqd
0.29 ± 40% +5.7 6.02 ± 61% perf-profile.children.cycles-pp.smpboot_thread_fn
0.38 ± 32% +5.8 6.14 ± 60% perf-profile.children.cycles-pp.ret_from_fork
0.38 ± 32% +5.8 6.14 ± 60% perf-profile.children.cycles-pp.kthread
1.17 ± 10% +6.0 7.17 ± 49% perf-profile.children.cycles-pp.rcu_do_batch
1.25 ± 9% +6.1 7.32 ± 48% perf-profile.children.cycles-pp.rcu_core
1.27 ± 9% +6.1 7.34 ± 48% perf-profile.children.cycles-pp.__softirqentry_text_start
30.68 ± 33% +11.0 41.68 ± 5% perf-profile.children.cycles-pp.shmem_get_inode
28.42 ± 36% +11.2 39.65 ± 5% perf-profile.children.cycles-pp.inode_sb_list_add
29.08 ± 35% +11.3 40.36 ± 5% perf-profile.children.cycles-pp.new_inode
0.00 +81.4 81.42 ± 4% perf-profile.children.cycles-pp.__cna_queued_spin_lock_slowpath
86.52 -86.5 0.00 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
1.56 ± 3% -0.2 1.39 perf-profile.self.cycles-pp._raw_spin_lock
0.36 ± 15% -0.2 0.21 ± 18% perf-profile.self.cycles-pp.__mnt_want_write
0.20 ± 6% -0.1 0.12 ± 3% perf-profile.self.cycles-pp.__destroy_inode
0.19 ± 5% -0.1 0.11 ± 10% perf-profile.self.cycles-pp.__list_del_entry_valid
0.26 ± 7% -0.1 0.18 ± 7% perf-profile.self.cycles-pp.__list_add_valid
0.27 ± 7% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.rcu_segcblist_enqueue
0.13 ± 6% -0.1 0.07 perf-profile.self.cycles-pp.drop_nlink
0.09 -0.0 0.04 ± 58% perf-profile.self.cycles-pp.find_next_zero_bit
0.17 ± 8% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.shmem_get_inode
0.12 ± 8% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.shmem_reserve_inode
0.13 ± 7% -0.0 0.10 ± 8% perf-profile.self.cycles-pp.__alloc_fd
0.10 ± 7% -0.0 0.07 ± 17% perf-profile.self.cycles-pp.__close_fd
0.10 ± 4% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.d_alloc_parallel
0.10 ± 7% -0.0 0.08 ± 6% perf-profile.self.cycles-pp.shmem_free_inode
0.07 ± 6% +0.0 0.09 ± 13% perf-profile.self.cycles-pp.__d_lookup_rcu
0.06 +0.0 0.08 ± 10% perf-profile.self.cycles-pp.strncpy_from_user
0.07 ± 6% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.__might_sleep
0.05 +0.0 0.08 ± 6% perf-profile.self.cycles-pp.lockref_put_or_lock
0.08 ± 6% +0.0 0.10 ± 4% perf-profile.self.cycles-pp.__call_rcu
0.07 ± 6% +0.0 0.10 ± 5% perf-profile.self.cycles-pp.avc_has_perm
0.04 ± 58% +0.0 0.07 perf-profile.self.cycles-pp.lockref_get
0.06 +0.0 0.09 perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.08 +0.0 0.11 ± 7% perf-profile.self.cycles-pp.link_path_walk
0.08 +0.0 0.11 ± 4% perf-profile.self.cycles-pp.avc_has_perm_noaudit
0.06 ± 14% +0.0 0.10 ± 4% perf-profile.self.cycles-pp.inode_init_once
0.10 ± 15% +0.0 0.14 ± 15% perf-profile.self.cycles-pp.memcpy_erms
0.15 ± 5% +0.0 0.19 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc
0.00 +0.1 0.05 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.do_dentry_open
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.__virt_addr_valid
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.__inode_security_revalidate
0.16 ± 9% +0.1 0.22 ± 9% perf-profile.self.cycles-pp.___might_sleep
0.26 ± 10% +0.1 0.35 ± 8% perf-profile.self.cycles-pp.selinux_inode_permission
0.27 ± 4% +0.1 0.37 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.07 ± 6% +0.1 0.17 ± 23% perf-profile.self.cycles-pp.kmem_cache_free
0.29 ± 5% +0.1 0.40 ± 5% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.00 +0.2 0.17 ± 44% perf-profile.self.cycles-pp.shmem_free_in_core_inode
0.00 +0.2 0.25 ± 5% perf-profile.self.cycles-pp.cna_scan_main_queue
0.08 ± 11% +0.3 0.33 ± 46% perf-profile.self.cycles-pp.rcu_cblist_dequeue
0.58 ± 2% +0.3 0.86 ± 6% perf-profile.self.cycles-pp.do_syscall_64
0.24 ± 10% +1.2 1.43 ± 48% perf-profile.self.cycles-pp.__slab_free
0.70 ± 10% +3.3 3.97 ± 47% perf-profile.self.cycles-pp.file_free_rcu
0.00 +79.7 79.66 ± 4% perf-profile.self.cycles-pp.__cna_queued_spin_lock_slowpath
385.75 ± 13% +363.4% 1787 ± 55% interrupts.38:PCI-MSI.67633153-edge.eth0-TxRx-0
697757 ± 7% -21.8% 545585 ± 12% interrupts.CAL:Function_call_interrupts
6024 ± 7% -20.5% 4786 ± 12% interrupts.CPU0.CAL:Function_call_interrupts
605078 +49.4% 904260 ± 8% interrupts.CPU0.LOC:Local_timer_interrupts
2219 ± 6% -37.8% 1380 ± 9% interrupts.CPU0.TLB:TLB_shootdowns
5992 ± 5% -16.2% 5019 ± 12% interrupts.CPU1.CAL:Function_call_interrupts
605216 +49.4% 904048 ± 8% interrupts.CPU1.LOC:Local_timer_interrupts
2206 ± 6% -30.5% 1533 ± 4% interrupts.CPU1.TLB:TLB_shootdowns
6619 ± 9% -23.7% 5050 ± 12% interrupts.CPU10.CAL:Function_call_interrupts
605367 +49.3% 903560 ± 8% interrupts.CPU10.LOC:Local_timer_interrupts
2214 ± 6% -31.5% 1516 ± 6% interrupts.CPU10.TLB:TLB_shootdowns
6929 ± 8% -21.3% 5455 ± 11% interrupts.CPU100.CAL:Function_call_interrupts
604784 +49.4% 903719 ± 8% interrupts.CPU100.LOC:Local_timer_interrupts
2219 ± 6% -16.2% 1860 ± 11% interrupts.CPU100.TLB:TLB_shootdowns
6945 ± 7% -20.7% 5509 ± 12% interrupts.CPU101.CAL:Function_call_interrupts
605011 +49.4% 903743 ± 8% interrupts.CPU101.LOC:Local_timer_interrupts
2211 ± 5% -15.3% 1873 ± 12% interrupts.CPU101.TLB:TLB_shootdowns
6927 ± 7% -21.1% 5466 ± 13% interrupts.CPU102.CAL:Function_call_interrupts
605053 +49.4% 903662 ± 8% interrupts.CPU102.LOC:Local_timer_interrupts
2212 ± 5% -17.0% 1835 ± 13% interrupts.CPU102.TLB:TLB_shootdowns
6747 ± 8% -19.6% 5423 ± 13% interrupts.CPU103.CAL:Function_call_interrupts
605061 +49.3% 903314 ± 8% interrupts.CPU103.LOC:Local_timer_interrupts
6616 ± 9% -23.3% 5077 ± 11% interrupts.CPU11.CAL:Function_call_interrupts
605325 +49.3% 904015 ± 8% interrupts.CPU11.LOC:Local_timer_interrupts
2213 ± 5% -32.2% 1499 ± 3% interrupts.CPU11.TLB:TLB_shootdowns
6569 ± 8% -22.7% 5076 ± 9% interrupts.CPU12.CAL:Function_call_interrupts
604975 +49.4% 903644 ± 8% interrupts.CPU12.LOC:Local_timer_interrupts
2203 ± 5% -28.9% 1565 ± 2% interrupts.CPU12.TLB:TLB_shootdowns
605093 +49.4% 904084 ± 8% interrupts.CPU13.LOC:Local_timer_interrupts
2221 ± 6% -31.2% 1527 ± 3% interrupts.CPU13.TLB:TLB_shootdowns
6464 ± 12% -21.2% 5095 ± 12% interrupts.CPU14.CAL:Function_call_interrupts
605279 +49.4% 904124 ± 8% interrupts.CPU14.LOC:Local_timer_interrupts
470.00 ±119% -90.6% 44.00 ± 26% interrupts.CPU14.RES:Rescheduling_interrupts
2215 ± 6% -29.8% 1555 ± 5% interrupts.CPU14.TLB:TLB_shootdowns
6608 ± 9% -23.2% 5075 ± 9% interrupts.CPU15.CAL:Function_call_interrupts
605265 +49.3% 903448 ± 8% interrupts.CPU15.LOC:Local_timer_interrupts
2217 ± 6% -25.9% 1643 ± 3% interrupts.CPU15.TLB:TLB_shootdowns
6600 ± 9% -21.9% 5154 ± 12% interrupts.CPU16.CAL:Function_call_interrupts
604749 +49.4% 903692 ± 8% interrupts.CPU16.LOC:Local_timer_interrupts
2223 ± 6% -30.0% 1555 ± 5% interrupts.CPU16.TLB:TLB_shootdowns
6600 ± 9% -23.0% 5080 ± 11% interrupts.CPU17.CAL:Function_call_interrupts
604741 +49.4% 903577 ± 8% interrupts.CPU17.LOC:Local_timer_interrupts
2217 ± 6% -29.5% 1564 ± 7% interrupts.CPU17.TLB:TLB_shootdowns
6580 ± 9% -23.6% 5025 ± 11% interrupts.CPU18.CAL:Function_call_interrupts
604772 +49.5% 903856 ± 8% interrupts.CPU18.LOC:Local_timer_interrupts
2216 ± 6% -29.0% 1573 ± 2% interrupts.CPU18.TLB:TLB_shootdowns
604709 +49.5% 903818 ± 8% interrupts.CPU19.LOC:Local_timer_interrupts
2196 ± 6% -30.0% 1536 ± 4% interrupts.CPU19.TLB:TLB_shootdowns
6247 ± 10% -20.7% 4956 ± 11% interrupts.CPU2.CAL:Function_call_interrupts
605082 +49.4% 904044 ± 8% interrupts.CPU2.LOC:Local_timer_interrupts
2206 ± 5% -27.8% 1593 ± 6% interrupts.CPU2.TLB:TLB_shootdowns
7021 ± 10% -28.2% 5039 ± 12% interrupts.CPU20.CAL:Function_call_interrupts
604458 +49.5% 903696 ± 8% interrupts.CPU20.LOC:Local_timer_interrupts
2217 ± 6% -32.1% 1505 ± 4% interrupts.CPU20.TLB:TLB_shootdowns
7026 ± 11% -28.3% 5036 ± 12% interrupts.CPU21.CAL:Function_call_interrupts
604498 +49.6% 904028 ± 8% interrupts.CPU21.LOC:Local_timer_interrupts
2227 ± 6% -31.8% 1518 ± 4% interrupts.CPU21.TLB:TLB_shootdowns
7013 ± 11% -26.4% 5158 ± 10% interrupts.CPU22.CAL:Function_call_interrupts
604469 +49.6% 904118 ± 8% interrupts.CPU22.LOC:Local_timer_interrupts
2215 ± 6% -28.8% 1578 ± 7% interrupts.CPU22.TLB:TLB_shootdowns
7017 ± 10% -26.5% 5155 ± 9% interrupts.CPU23.CAL:Function_call_interrupts
604486 +49.5% 903885 ± 8% interrupts.CPU23.LOC:Local_timer_interrupts
2220 ± 6% -26.9% 1623 ± 6% interrupts.CPU23.TLB:TLB_shootdowns
604622 +49.4% 903393 ± 8% interrupts.CPU24.LOC:Local_timer_interrupts
2196 ± 5% -28.0% 1581 ± 8% interrupts.CPU24.TLB:TLB_shootdowns
6994 ± 11% -27.3% 5084 ± 11% interrupts.CPU25.CAL:Function_call_interrupts
604791 +49.4% 903564 ± 8% interrupts.CPU25.LOC:Local_timer_interrupts
2223 ± 6% -30.0% 1556 ± 7% interrupts.CPU25.TLB:TLB_shootdowns
6950 ± 11% -24.8% 5229 ± 12% interrupts.CPU26.CAL:Function_call_interrupts
604805 +49.5% 904034 ± 8% interrupts.CPU26.LOC:Local_timer_interrupts
2221 ± 6% -19.4% 1790 ± 15% interrupts.CPU26.TLB:TLB_shootdowns
605095 +49.4% 903979 ± 8% interrupts.CPU27.LOC:Local_timer_interrupts
2212 ± 6% -15.0% 1880 ± 12% interrupts.CPU27.TLB:TLB_shootdowns
604657 +49.4% 903264 ± 8% interrupts.CPU28.LOC:Local_timer_interrupts
7032 ± 10% -23.9% 5349 ± 12% interrupts.CPU29.CAL:Function_call_interrupts
604852 +49.5% 903982 ± 8% interrupts.CPU29.LOC:Local_timer_interrupts
2226 ± 6% -18.8% 1808 ± 13% interrupts.CPU29.TLB:TLB_shootdowns
5858 ± 5% -15.3% 4963 ± 10% interrupts.CPU3.CAL:Function_call_interrupts
605204 +49.4% 903949 ± 8% interrupts.CPU3.LOC:Local_timer_interrupts
20.75 ± 78% +530.1% 130.75 ± 84% interrupts.CPU3.RES:Rescheduling_interrupts
2217 ± 6% -31.6% 1516 ± 5% interrupts.CPU3.TLB:TLB_shootdowns
385.75 ± 13% +363.4% 1787 ± 55% interrupts.CPU30.38:PCI-MSI.67633153-edge.eth0-TxRx-0
604784 +49.5% 903854 ± 8% interrupts.CPU30.LOC:Local_timer_interrupts
6997 ± 10% -23.3% 5367 ± 14% interrupts.CPU31.CAL:Function_call_interrupts
604908 +49.3% 903233 ± 8% interrupts.CPU31.LOC:Local_timer_interrupts
2217 ± 6% -15.7% 1868 ± 12% interrupts.CPU31.TLB:TLB_shootdowns
7028 ± 9% -23.3% 5392 ± 14% interrupts.CPU32.CAL:Function_call_interrupts
604943 +49.4% 903732 ± 8% interrupts.CPU32.LOC:Local_timer_interrupts
21.00 ± 86% +315.5% 87.25 ± 92% interrupts.CPU32.RES:Rescheduling_interrupts
2221 ± 6% -15.9% 1868 ± 13% interrupts.CPU32.TLB:TLB_shootdowns
604947 +49.4% 903620 ± 8% interrupts.CPU33.LOC:Local_timer_interrupts
2208 ± 6% -15.9% 1857 ± 13% interrupts.CPU33.TLB:TLB_shootdowns
6996 ± 9% -25.2% 5233 ± 12% interrupts.CPU34.CAL:Function_call_interrupts
605056 +49.4% 903727 ± 8% interrupts.CPU34.LOC:Local_timer_interrupts
6967 ± 9% -22.5% 5398 ± 14% interrupts.CPU35.CAL:Function_call_interrupts
604764 +49.4% 903512 ± 8% interrupts.CPU35.LOC:Local_timer_interrupts
2214 ± 6% -15.2% 1877 ± 12% interrupts.CPU35.TLB:TLB_shootdowns
6972 ± 9% -22.1% 5433 ± 12% interrupts.CPU36.CAL:Function_call_interrupts
604790 +49.4% 903509 ± 8% interrupts.CPU36.LOC:Local_timer_interrupts
2220 ± 6% -16.3% 1859 ± 12% interrupts.CPU36.TLB:TLB_shootdowns
6989 ± 9% -22.2% 5434 ± 12% interrupts.CPU37.CAL:Function_call_interrupts
604776 +49.4% 903536 ± 8% interrupts.CPU37.LOC:Local_timer_interrupts
2220 ± 6% -16.3% 1859 ± 12% interrupts.CPU37.TLB:TLB_shootdowns
6962 ± 10% -23.2% 5344 ± 12% interrupts.CPU38.CAL:Function_call_interrupts
605034 +49.4% 903873 ± 8% interrupts.CPU38.LOC:Local_timer_interrupts
11.00 ±126% +2281.8% 262.00 ±155% interrupts.CPU38.RES:Rescheduling_interrupts
2215 ± 6% -17.1% 1835 ± 12% interrupts.CPU38.TLB:TLB_shootdowns
6995 ± 9% -21.2% 5511 ± 13% interrupts.CPU39.CAL:Function_call_interrupts
605001 +49.4% 903831 ± 8% interrupts.CPU39.LOC:Local_timer_interrupts
6159 ± 9% -18.7% 5010 ± 9% interrupts.CPU4.CAL:Function_call_interrupts
605400 +49.3% 904029 ± 8% interrupts.CPU4.LOC:Local_timer_interrupts
2219 ± 6% -31.5% 1519 ± 4% interrupts.CPU4.TLB:TLB_shootdowns
6989 ± 9% -22.1% 5447 ± 13% interrupts.CPU40.CAL:Function_call_interrupts
605052 +49.4% 903846 ± 8% interrupts.CPU40.LOC:Local_timer_interrupts
2217 ± 6% -16.6% 1849 ± 12% interrupts.CPU40.TLB:TLB_shootdowns
6945 ± 10% -22.9% 5357 ± 12% interrupts.CPU41.CAL:Function_call_interrupts
605062 +49.3% 903620 ± 8% interrupts.CPU41.LOC:Local_timer_interrupts
2224 ± 6% -17.3% 1839 ± 13% interrupts.CPU41.TLB:TLB_shootdowns
6977 ± 9% -24.0% 5300 ± 15% interrupts.CPU42.CAL:Function_call_interrupts
605055 +49.3% 903090 ± 8% interrupts.CPU42.LOC:Local_timer_interrupts
2220 ± 6% -18.6% 1807 ± 16% interrupts.CPU42.TLB:TLB_shootdowns
6976 ± 9% -21.9% 5447 ± 10% interrupts.CPU43.CAL:Function_call_interrupts
605063 +49.4% 903693 ± 8% interrupts.CPU43.LOC:Local_timer_interrupts
2224 ± 6% -16.7% 1853 ± 13% interrupts.CPU43.TLB:TLB_shootdowns
6963 ± 9% -23.7% 5313 ± 8% interrupts.CPU44.CAL:Function_call_interrupts
604214 +49.6% 903660 ± 8% interrupts.CPU44.LOC:Local_timer_interrupts
2214 ± 5% -16.4% 1850 ± 12% interrupts.CPU44.TLB:TLB_shootdowns
604751 +49.4% 903354 ± 8% interrupts.CPU45.LOC:Local_timer_interrupts
604757 +49.4% 903692 ± 8% interrupts.CPU46.LOC:Local_timer_interrupts
2191 ± 8% -17.1% 1816 ± 14% interrupts.CPU46.TLB:TLB_shootdowns
6983 ± 8% -22.5% 5409 ± 11% interrupts.CPU47.CAL:Function_call_interrupts
604639 +49.4% 903355 ± 8% interrupts.CPU47.LOC:Local_timer_interrupts
2220 ± 6% -16.8% 1848 ± 13% interrupts.CPU47.TLB:TLB_shootdowns
6972 ± 8% -23.6% 5328 ± 11% interrupts.CPU48.CAL:Function_call_interrupts
605090 +49.3% 903375 ± 8% interrupts.CPU48.LOC:Local_timer_interrupts
2221 ± 6% -17.9% 1822 ± 14% interrupts.CPU48.TLB:TLB_shootdowns
6960 ± 9% -19.9% 5572 ± 12% interrupts.CPU49.CAL:Function_call_interrupts
605028 +49.3% 903376 ± 8% interrupts.CPU49.LOC:Local_timer_interrupts
2262 ± 9% -17.2% 1873 ± 11% interrupts.CPU49.TLB:TLB_shootdowns
6300 ± 10% -20.5% 5010 ± 11% interrupts.CPU5.CAL:Function_call_interrupts
605265 +49.4% 904108 ± 8% interrupts.CPU5.LOC:Local_timer_interrupts
7393 ± 6% -30.0% 5172 ± 24% interrupts.CPU5.NMI:Non-maskable_interrupts
7393 ± 6% -30.0% 5172 ± 24% interrupts.CPU5.PMI:Performance_monitoring_interrupts
2218 ± 6% -31.5% 1520 ± 4% interrupts.CPU5.TLB:TLB_shootdowns
6942 ± 8% -21.3% 5466 ± 12% interrupts.CPU50.CAL:Function_call_interrupts
604743 +49.4% 903416 ± 8% interrupts.CPU50.LOC:Local_timer_interrupts
2217 ± 6% -15.9% 1865 ± 13% interrupts.CPU50.TLB:TLB_shootdowns
605464 +49.2% 903502 ± 8% interrupts.CPU51.LOC:Local_timer_interrupts
2221 ± 6% -21.4% 1744 ± 15% interrupts.CPU51.TLB:TLB_shootdowns
6697 ± 10% -24.8% 5034 ± 11% interrupts.CPU52.CAL:Function_call_interrupts
604902 +49.4% 903660 ± 8% interrupts.CPU52.LOC:Local_timer_interrupts
2218 ± 6% -32.4% 1499 ± 4% interrupts.CPU52.TLB:TLB_shootdowns
6598 ± 14% -22.1% 5141 ± 12% interrupts.CPU53.CAL:Function_call_interrupts
605189 +49.3% 903556 ± 8% interrupts.CPU53.LOC:Local_timer_interrupts
2207 ± 5% -29.2% 1563 ± 5% interrupts.CPU53.TLB:TLB_shootdowns
605229 +49.3% 903617 ± 8% interrupts.CPU54.LOC:Local_timer_interrupts
2184 ± 6% -28.1% 1570 ± 7% interrupts.CPU54.TLB:TLB_shootdowns
6984 ± 8% -27.2% 5085 ± 11% interrupts.CPU55.CAL:Function_call_interrupts
605334 +49.3% 903851 ± 8% interrupts.CPU55.LOC:Local_timer_interrupts
2205 ± 5% -32.5% 1489 ± 6% interrupts.CPU55.TLB:TLB_shootdowns
6964 ± 8% -26.1% 5143 ± 12% interrupts.CPU56.CAL:Function_call_interrupts
605158 +49.4% 903830 ± 8% interrupts.CPU56.LOC:Local_timer_interrupts
2216 ± 6% -29.6% 1560 ± 8% interrupts.CPU56.TLB:TLB_shootdowns
6994 ± 8% -27.4% 5077 ± 10% interrupts.CPU57.CAL:Function_call_interrupts
604926 +49.4% 903638 ± 8% interrupts.CPU57.LOC:Local_timer_interrupts
2218 ± 6% -31.9% 1511 ± 6% interrupts.CPU57.TLB:TLB_shootdowns
6961 ± 8% -27.7% 5034 ± 13% interrupts.CPU58.CAL:Function_call_interrupts
605119 +49.4% 904050 ± 8% interrupts.CPU58.LOC:Local_timer_interrupts
2214 ± 6% -31.5% 1516 ± 8% interrupts.CPU58.TLB:TLB_shootdowns
6964 ± 8% -27.3% 5060 ± 11% interrupts.CPU59.CAL:Function_call_interrupts
605144 +49.4% 903922 ± 8% interrupts.CPU59.LOC:Local_timer_interrupts
2214 ± 6% -34.0% 1462 ± 4% interrupts.CPU59.TLB:TLB_shootdowns
6188 ± 11% -19.3% 4993 ± 12% interrupts.CPU6.CAL:Function_call_interrupts
605296 +49.3% 903962 ± 8% interrupts.CPU6.LOC:Local_timer_interrupts
2222 ± 6% -32.1% 1509 ± 6% interrupts.CPU6.TLB:TLB_shootdowns
6979 ± 8% -26.1% 5156 ± 11% interrupts.CPU60.CAL:Function_call_interrupts
605168 +49.4% 904155 ± 8% interrupts.CPU60.LOC:Local_timer_interrupts
2222 ± 6% -33.5% 1478 ± 3% interrupts.CPU60.TLB:TLB_shootdowns
6964 ± 8% -26.9% 5088 ± 12% interrupts.CPU61.CAL:Function_call_interrupts
605138 +49.4% 904017 ± 8% interrupts.CPU61.LOC:Local_timer_interrupts
2212 ± 5% -32.4% 1496 ± 6% interrupts.CPU61.TLB:TLB_shootdowns
605099 +49.4% 904082 ± 8% interrupts.CPU62.LOC:Local_timer_interrupts
2208 ± 5% -31.6% 1510 ± 7% interrupts.CPU62.TLB:TLB_shootdowns
6972 ± 8% -26.1% 5150 ± 13% interrupts.CPU63.CAL:Function_call_interrupts
604984 +49.4% 903589 ± 8% interrupts.CPU63.LOC:Local_timer_interrupts
2210 ± 5% -29.9% 1549 ± 3% interrupts.CPU63.TLB:TLB_shootdowns
6975 ± 8% -26.4% 5134 ± 12% interrupts.CPU64.CAL:Function_call_interrupts
605205 +49.4% 903939 ± 8% interrupts.CPU64.LOC:Local_timer_interrupts
2218 ± 6% -30.0% 1551 ± 4% interrupts.CPU64.TLB:TLB_shootdowns
605118 +49.4% 904115 ± 8% interrupts.CPU65.LOC:Local_timer_interrupts
2220 ± 6% -33.1% 1485 ± 6% interrupts.CPU65.TLB:TLB_shootdowns
6978 ± 8% -27.1% 5085 ± 13% interrupts.CPU66.CAL:Function_call_interrupts
605220 +49.3% 903866 ± 8% interrupts.CPU66.LOC:Local_timer_interrupts
2216 ± 6% -31.6% 1516 ± 5% interrupts.CPU66.TLB:TLB_shootdowns
6669 ± 12% -22.2% 5192 ± 14% interrupts.CPU67.CAL:Function_call_interrupts
605109 +49.4% 904123 ± 8% interrupts.CPU67.LOC:Local_timer_interrupts
2218 ± 6% -29.7% 1559 ± 6% interrupts.CPU67.TLB:TLB_shootdowns
6917 ± 8% -25.1% 5181 ± 13% interrupts.CPU68.CAL:Function_call_interrupts
604787 +49.5% 904120 ± 8% interrupts.CPU68.LOC:Local_timer_interrupts
2219 ± 6% -32.3% 1502 ± 5% interrupts.CPU68.TLB:TLB_shootdowns
6956 ± 8% -25.5% 5185 ± 12% interrupts.CPU69.CAL:Function_call_interrupts
604874 +49.5% 904010 ± 8% interrupts.CPU69.LOC:Local_timer_interrupts
2219 ± 6% -33.3% 1479 ± 5% interrupts.CPU69.TLB:TLB_shootdowns
605105 +49.4% 903797 ± 8% interrupts.CPU7.LOC:Local_timer_interrupts
7385 ± 6% -29.9% 5175 ± 24% interrupts.CPU7.NMI:Non-maskable_interrupts
7385 ± 6% -29.9% 5175 ± 24% interrupts.CPU7.PMI:Performance_monitoring_interrupts
2218 ± 6% -30.2% 1548 ± 7% interrupts.CPU7.TLB:TLB_shootdowns
6973 ± 8% -26.1% 5154 ± 12% interrupts.CPU70.CAL:Function_call_interrupts
604989 +49.4% 904038 ± 8% interrupts.CPU70.LOC:Local_timer_interrupts
2222 ± 6% -31.2% 1529 ± 2% interrupts.CPU70.TLB:TLB_shootdowns
6897 ± 7% -26.7% 5059 ± 9% interrupts.CPU71.CAL:Function_call_interrupts
604989 +49.4% 903916 ± 8% interrupts.CPU71.LOC:Local_timer_interrupts
2195 ± 5% -30.8% 1519 ± 6% interrupts.CPU71.TLB:TLB_shootdowns
605300 +49.3% 903967 ± 8% interrupts.CPU72.LOC:Local_timer_interrupts
2212 ± 5% -33.8% 1465 ± 4% interrupts.CPU72.TLB:TLB_shootdowns
6960 ± 8% -25.5% 5184 ± 12% interrupts.CPU73.CAL:Function_call_interrupts
604879 +49.4% 903619 ± 8% interrupts.CPU73.LOC:Local_timer_interrupts
2217 ± 6% -32.1% 1504 ± 5% interrupts.CPU73.TLB:TLB_shootdowns
6939 ± 8% -26.0% 5137 ± 13% interrupts.CPU74.CAL:Function_call_interrupts
604856 +49.4% 903444 ± 8% interrupts.CPU74.LOC:Local_timer_interrupts
7.75 ± 43% +561.3% 51.25 ± 57% interrupts.CPU74.RES:Rescheduling_interrupts
2219 ± 6% -32.0% 1509 ± 3% interrupts.CPU74.TLB:TLB_shootdowns
6943 ± 8% -26.7% 5089 ± 12% interrupts.CPU75.CAL:Function_call_interrupts
604991 +49.4% 903802 ± 8% interrupts.CPU75.LOC:Local_timer_interrupts
2213 ± 6% -32.1% 1502 ± 5% interrupts.CPU75.TLB:TLB_shootdowns
6896 ± 8% -25.6% 5127 ± 14% interrupts.CPU76.CAL:Function_call_interrupts
604998 +49.4% 903922 ± 8% interrupts.CPU76.LOC:Local_timer_interrupts
2193 ± 5% -30.2% 1530 ± 9% interrupts.CPU76.TLB:TLB_shootdowns
6936 ± 8% -27.5% 5030 ± 11% interrupts.CPU77.CAL:Function_call_interrupts
604995 +49.4% 903602 ± 8% interrupts.CPU77.LOC:Local_timer_interrupts
2218 ± 6% -34.0% 1463 ± 4% interrupts.CPU77.TLB:TLB_shootdowns
605036 +49.4% 903728 ± 8% interrupts.CPU78.LOC:Local_timer_interrupts
2215 ± 6% -15.4% 1875 ± 12% interrupts.CPU78.TLB:TLB_shootdowns
6936 ± 8% -22.1% 5403 ± 12% interrupts.CPU79.CAL:Function_call_interrupts
604797 +49.4% 903303 ± 8% interrupts.CPU79.LOC:Local_timer_interrupts
339.50 ± 83% -91.5% 29.00 ± 49% interrupts.CPU79.RES:Rescheduling_interrupts
2217 ± 6% -15.9% 1865 ± 12% interrupts.CPU79.TLB:TLB_shootdowns
6178 ± 11% -19.5% 4971 ± 12% interrupts.CPU8.CAL:Function_call_interrupts
605074 +49.3% 903560 ± 8% interrupts.CPU8.LOC:Local_timer_interrupts
2215 ± 6% -31.8% 1510 ± 3% interrupts.CPU8.TLB:TLB_shootdowns
7006 ± 8% -23.2% 5383 ± 14% interrupts.CPU80.CAL:Function_call_interrupts
604775 +49.5% 903930 ± 8% interrupts.CPU80.LOC:Local_timer_interrupts
2213 ± 6% -16.2% 1855 ± 14% interrupts.CPU80.TLB:TLB_shootdowns
6996 ± 7% -22.3% 5432 ± 12% interrupts.CPU81.CAL:Function_call_interrupts
604370 +49.6% 903867 ± 8% interrupts.CPU81.LOC:Local_timer_interrupts
2225 ± 6% -16.2% 1866 ± 13% interrupts.CPU81.TLB:TLB_shootdowns
605046 +49.4% 903912 ± 8% interrupts.CPU82.LOC:Local_timer_interrupts
2297 -19.7% 1845 ± 11% interrupts.CPU82.TLB:TLB_shootdowns
6998 ± 8% -22.8% 5404 ± 12% interrupts.CPU83.CAL:Function_call_interrupts
604950 +49.4% 903618 ± 8% interrupts.CPU83.LOC:Local_timer_interrupts
2207 ± 6% -15.9% 1856 ± 13% interrupts.CPU83.TLB:TLB_shootdowns
7002 ± 8% -20.9% 5540 ± 11% interrupts.CPU84.CAL:Function_call_interrupts
605061 +49.4% 904027 ± 8% interrupts.CPU84.LOC:Local_timer_interrupts
2220 ± 6% -15.4% 1877 ± 12% interrupts.CPU84.TLB:TLB_shootdowns
605088 +49.4% 904045 ± 8% interrupts.CPU85.LOC:Local_timer_interrupts
6998 ± 8% -22.8% 5406 ± 12% interrupts.CPU86.CAL:Function_call_interrupts
605132 +49.3% 903673 ± 8% interrupts.CPU86.LOC:Local_timer_interrupts
2223 ± 5% -17.0% 1845 ± 14% interrupts.CPU86.TLB:TLB_shootdowns
7007 ± 8% -22.5% 5432 ± 14% interrupts.CPU87.CAL:Function_call_interrupts
604950 +49.4% 903948 ± 8% interrupts.CPU87.LOC:Local_timer_interrupts
2214 ± 6% -16.1% 1857 ± 13% interrupts.CPU87.TLB:TLB_shootdowns
6989 ± 8% -22.7% 5400 ± 11% interrupts.CPU88.CAL:Function_call_interrupts
605140 +49.4% 903906 ± 8% interrupts.CPU88.LOC:Local_timer_interrupts
2220 ± 6% -16.4% 1855 ± 13% interrupts.CPU88.TLB:TLB_shootdowns
6991 ± 8% -22.0% 5451 ± 12% interrupts.CPU89.CAL:Function_call_interrupts
605154 +49.4% 903867 ± 8% interrupts.CPU89.LOC:Local_timer_interrupts
2213 ± 5% -16.7% 1843 ± 14% interrupts.CPU89.TLB:TLB_shootdowns
6198 ± 11% -19.0% 5018 ± 11% interrupts.CPU9.CAL:Function_call_interrupts
605213 +49.3% 903851 ± 8% interrupts.CPU9.LOC:Local_timer_interrupts
2225 ± 6% -31.2% 1532 ± 4% interrupts.CPU9.TLB:TLB_shootdowns
6995 ± 8% -22.7% 5406 ± 11% interrupts.CPU90.CAL:Function_call_interrupts
604939 +49.4% 903792 ± 8% interrupts.CPU90.LOC:Local_timer_interrupts
2220 ± 6% -16.1% 1862 ± 13% interrupts.CPU90.TLB:TLB_shootdowns
604799 +49.5% 903918 ± 8% interrupts.CPU91.LOC:Local_timer_interrupts
2212 ± 6% -16.1% 1856 ± 13% interrupts.CPU91.TLB:TLB_shootdowns
6864 ± 8% -20.6% 5449 ± 12% interrupts.CPU92.CAL:Function_call_interrupts
605039 +49.4% 903911 ± 8% interrupts.CPU92.LOC:Local_timer_interrupts
2217 ± 6% -15.9% 1865 ± 12% interrupts.CPU92.TLB:TLB_shootdowns
6952 ± 7% -23.6% 5310 ± 14% interrupts.CPU93.CAL:Function_call_interrupts
604763 +49.5% 903835 ± 8% interrupts.CPU93.LOC:Local_timer_interrupts
10.75 ±101% +1490.7% 171.00 ±103% interrupts.CPU93.RES:Rescheduling_interrupts
2212 ± 6% -19.0% 1792 ± 16% interrupts.CPU93.TLB:TLB_shootdowns
6972 ± 7% -23.0% 5366 ± 13% interrupts.CPU94.CAL:Function_call_interrupts
605014 +49.4% 903749 ± 8% interrupts.CPU94.LOC:Local_timer_interrupts
2222 ± 6% -20.1% 1776 ± 15% interrupts.CPU94.TLB:TLB_shootdowns
6970 ± 7% -21.5% 5473 ± 13% interrupts.CPU95.CAL:Function_call_interrupts
604375 +49.5% 903623 ± 8% interrupts.CPU95.LOC:Local_timer_interrupts
2225 ± 6% -17.0% 1846 ± 14% interrupts.CPU95.TLB:TLB_shootdowns
6965 ± 8% -22.7% 5386 ± 11% interrupts.CPU96.CAL:Function_call_interrupts
605087 +49.3% 903388 ± 8% interrupts.CPU96.LOC:Local_timer_interrupts
2216 ± 6% -16.5% 1850 ± 13% interrupts.CPU96.TLB:TLB_shootdowns
605055 +49.4% 903704 ± 8% interrupts.CPU97.LOC:Local_timer_interrupts
2195 ± 6% -15.1% 1864 ± 12% interrupts.CPU97.TLB:TLB_shootdowns
6884 ± 9% -21.4% 5408 ± 11% interrupts.CPU98.CAL:Function_call_interrupts
605119 +49.3% 903214 ± 8% interrupts.CPU98.LOC:Local_timer_interrupts
2224 ± 6% -16.8% 1850 ± 12% interrupts.CPU98.TLB:TLB_shootdowns
6941 ± 7% -21.4% 5457 ± 14% interrupts.CPU99.CAL:Function_call_interrupts
605081 +49.3% 903350 ± 8% interrupts.CPU99.LOC:Local_timer_interrupts
100.25 ± 45% -68.8% 31.25 ± 14% interrupts.CPU99.RES:Rescheduling_interrupts
2214 ± 6% -18.3% 1810 ± 14% interrupts.CPU99.TLB:TLB_shootdowns
62917283 +49.4% 93991011 ± 8% interrupts.LOC:Local_timer_interrupts
230455 ± 6% -23.8% 175528 ± 8% interrupts.TLB:TLB_shootdowns
will-it-scale.per_thread_ops
22000 +-+-----------------------------------------------------------------+
| O O |
20000 +-+ |
18000 +-+ O |
| |
16000 +-+ O |
| |
14000 +-+ |
O O O O O O OO O O O O O O O O O O O |
12000 +-O |
10000 +-+ O |
| |
8000 +-+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.|
| |
6000 +-+-----------------------------------------------------------------+
will-it-scale.workload
2.2e+06 +-+----------------------------------------O----------------------+
| O |
2e+06 +-+ O |
| |
1.8e+06 +-+ |
| O |
1.6e+06 +-+ |
| |
1.4e+06 +-+ O O O |
O O OO O O O O O OO O O O O O O |
1.2e+06 +-+ |
| O |
1e+06 +-+ |
| .+. .++. .+.+.+.+.+.+.++.+.+. .+.+.++.+. .+.+. .+.+.++.+.+. .|
800000 +-+---------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-skl-fpga01: 104 threads Skylake with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-skl-fpga01/mmap2/will-it-scale/0x2000065
commit:
7169a01e0c ("locking/qspinlock: Refactor the qspinlock slow path")
ae9a1c0ae4 ("locking/qspinlock: Introduce CNA into the slow path of qspinlock")
7169a01e0c99633b ae9a1c0ae45c35d2861c24ac27a
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:2 50% 1:2 dmesg.INFO:rcu_sched_self-detected_stall_on_CPU
:2 50% 1:2 dmesg.Kernel_panic-not_syncing:Hard_LOCKUP
:2 100% 2:2 dmesg.RIP:__cna_queued_spin_lock_slowpath
:2 50% 1:2 dmesg.RIP:_raw_spin_unlock_irqrestore
:2 50% 1:2 dmesg.RIP:native_smp_send_reschedule
:2 50% 1:2 dmesg.RIP:smp_call_function_many
:2 50% 1:2 dmesg.RIP:smp_call_function_single
1:2 -50% :2 dmesg.WARNING:at#for_ip_interrupt_entry/0x
1:2 -50% :2 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
:2 50% 1:2 dmesg.WARNING:at_arch/x86/kernel/apic/ipi.c:#native_smp_send_reschedule
2:2 -100% :2 dmesg.WARNING:stack_recursion
***************************************************************************************************
lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/1BRD_48G/f2fs/x86_64-rhel-7.6/1500/debian-x86_64-20191114.cgz/lkp-csl-2ap2/creat-clo/aim7/0x500002c
commit:
7169a01e0c ("locking/qspinlock: Refactor the qspinlock slow path")
ae9a1c0ae4 ("locking/qspinlock: Introduce CNA into the slow path of qspinlock")
7169a01e0c99633b ae9a1c0ae45c35d2861c24ac27a
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
%stddev %change %stddev
\ | \
39853 ± 3% +41.0% 56213 aim7.jobs-per-min
226.21 ± 3% -29.2% 160.23 aim7.time.elapsed_time
226.21 ± 3% -29.2% 160.23 aim7.time.elapsed_time.max
843318 ± 12% -35.7% 542630 ± 2% aim7.time.involuntary_context_switches
246472 ± 5% -21.6% 193305 ± 2% aim7.time.minor_page_faults
40440 ± 3% -32.8% 27158 aim7.time.system_time
19.24 ± 15% -20.9% 15.22 aim7.time.user_time
370991 ± 5% -52.2% 177411 ± 3% aim7.time.voluntary_context_switches
7.89 ± 6% +65.6% 13.06 ± 3% iostat.cpu.idle
92.05 -5.6% 86.86 iostat.cpu.system
7.10 ± 6% +4.9 12.02 ± 3% mpstat.cpu.all.idle%
0.00 ±131% +0.0 0.01 ± 9% mpstat.cpu.all.soft%
3270 ± 25% +32.5% 4332 ± 25% numa-meminfo.node1.Active(file)
3866 ±128% +324.7% 16419 ± 29% numa-meminfo.node2.Inactive
3740 ±127% +320.6% 15732 ± 30% numa-meminfo.node2.Inactive(anon)
8064 ± 33% +71.2% 13808 ± 6% numa-meminfo.node2.Mapped
202862 +31.8% 267418 slabinfo.filp.active_objs
3279 +35.6% 4448 slabinfo.filp.active_slabs
209931 +35.6% 284714 slabinfo.filp.num_objs
3279 +35.6% 4448 slabinfo.filp.num_slabs
7.75 ± 5% +58.1% 12.25 ± 3% vmstat.cpu.id
91.25 -4.9% 86.75 vmstat.cpu.sy
9640 ± 2% -11.7% 8515 vmstat.system.cs
395512 +1.1% 400004 vmstat.system.in
817.50 ± 25% +32.5% 1083 ± 25% numa-vmstat.node1.nr_active_file
817.50 ± 25% +32.5% 1083 ± 25% numa-vmstat.node1.nr_zone_active_file
946.75 ±128% +306.9% 3852 ± 31% numa-vmstat.node2.nr_inactive_anon
2099 ± 34% +61.3% 3386 ± 4% numa-vmstat.node2.nr_mapped
946.75 ±128% +306.9% 3852 ± 31% numa-vmstat.node2.nr_zone_inactive_anon
13174 +15.8% 15261 meminfo.Active(file)
167500 ± 4% -10.8% 149490 ± 6% meminfo.AnonHugePages
33478 ± 4% +12.3% 37600 ± 2% meminfo.Inactive
31945 ± 4% +12.9% 36051 ± 2% meminfo.Inactive(anon)
35790 ± 5% +18.1% 42252 ± 2% meminfo.Mapped
16089 ± 3% +41.1% 22709 ± 2% meminfo.max_used_kB
1473 -4.9% 1400 turbostat.Avg_MHz
6.44 ± 7% +60.8% 10.36 ± 6% turbostat.CPU%c1
0.29 ± 47% +213.6% 0.92 ± 47% turbostat.CPU%c6
46231878 ± 3% -27.9% 33312672 turbostat.IRQ
0.43 ± 62% +202.9% 1.30 ± 38% turbostat.Pkg%pc2
275.83 -4.4% 263.78 turbostat.PkgWatt
3294 +15.9% 3817 proc-vmstat.nr_active_file
1381 -3.9% 1328 proc-vmstat.nr_dirty
7970 ± 4% +13.1% 9017 ± 2% proc-vmstat.nr_inactive_anon
9049 ± 5% +18.3% 10708 proc-vmstat.nr_mapped
93037 +4.3% 97080 proc-vmstat.nr_slab_unreclaimable
3294 +15.9% 3817 proc-vmstat.nr_zone_active_file
7970 ± 4% +13.1% 9017 ± 2% proc-vmstat.nr_zone_inactive_anon
1381 -3.9% 1328 proc-vmstat.nr_zone_write_pending
170518 ± 7% -31.9% 116060 ± 4% proc-vmstat.numa_hint_faults
128536 ± 8% -34.8% 83815 ± 5% proc-vmstat.numa_hint_faults_local
1280417 +2.3% 1309336 proc-vmstat.numa_hit
1224699 +2.4% 1253630 proc-vmstat.numa_local
179353 ± 6% -31.1% 123642 ± 4% proc-vmstat.numa_pte_updates
2204040 ± 3% +26.8% 2794380 proc-vmstat.pgalloc_normal
12038 ± 6% +254.8% 42707 ± 3% proc-vmstat.pgdeactivate
1059852 ± 3% -24.9% 795459 proc-vmstat.pgfault
2246071 ± 3% +25.8% 2825427 proc-vmstat.pgfree
12041 ± 6% +254.7% 42711 ± 3% proc-vmstat.pgrotated
80033 -45.6% 43539 ± 26% sched_debug.cfs_rq:/.exec_clock.avg
81224 -44.9% 44745 ± 26% sched_debug.cfs_rq:/.exec_clock.max
75636 -47.7% 39532 ± 28% sched_debug.cfs_rq:/.exec_clock.min
8.62 ± 10% +82.6% 15.74 ± 12% sched_debug.cfs_rq:/.load_avg.avg
28.82 ± 32% +83.3% 52.83 ± 20% sched_debug.cfs_rq:/.load_avg.stddev
16740170 -45.5% 9122599 ± 26% sched_debug.cfs_rq:/.min_vruntime.avg
17423077 ± 2% -45.5% 9499479 ± 26% sched_debug.cfs_rq:/.min_vruntime.max
15401097 ± 2% -47.0% 8162981 ± 27% sched_debug.cfs_rq:/.min_vruntime.min
261607 ± 18% -31.5% 179296 ± 24% sched_debug.cfs_rq:/.min_vruntime.stddev
0.75 -17.6% 0.61 ± 9% sched_debug.cfs_rq:/.nr_running.avg
0.14 ± 16% +40.4% 0.20 ± 15% sched_debug.cfs_rq:/.nr_running.stddev
0.22 ± 27% +71604.7% 157.55 ± 27% sched_debug.cfs_rq:/.nr_spread_over.avg
7.94 ± 34% +3205.0% 262.33 ± 26% sched_debug.cfs_rq:/.nr_spread_over.max
0.87 ± 18% +1812.1% 16.69 ± 20% sched_debug.cfs_rq:/.nr_spread_over.stddev
261635 ± 18% -31.4% 179385 ± 24% sched_debug.cfs_rq:/.spread0.stddev
786.20 -15.3% 665.85 ± 9% sched_debug.cfs_rq:/.util_avg.avg
272.00 ± 17% -41.2% 160.00 ± 53% sched_debug.cfs_rq:/.util_avg.min
923366 ± 2% +10.2% 1017241 sched_debug.cpu.avg_idle.avg
141282 ± 9% +40.7% 198778 ± 4% sched_debug.cpu.avg_idle.stddev
138608 -26.0% 102504 ± 14% sched_debug.cpu.clock.avg
138663 -26.0% 102561 ± 14% sched_debug.cpu.clock.max
138546 -26.1% 102437 ± 14% sched_debug.cpu.clock.min
138608 -26.0% 102504 ± 14% sched_debug.cpu.clock_task.avg
138663 -26.0% 102561 ± 14% sched_debug.cpu.clock_task.max
138546 -26.1% 102437 ± 14% sched_debug.cpu.clock_task.min
2885 -17.8% 2371 ± 10% sched_debug.cpu.curr->pid.avg
6908 -19.4% 5570 ± 9% sched_debug.cpu.curr->pid.max
502666 +10.9% 557444 sched_debug.cpu.max_idle_balance_cost.avg
0.84 ± 4% -15.4% 0.71 ± 10% sched_debug.cpu.nr_running.avg
6428 ± 2% -34.3% 4226 ± 13% sched_debug.cpu.nr_switches.avg
4806 ± 6% -38.1% 2974 ± 18% sched_debug.cpu.nr_switches.min
5.04 -16.5% 4.21 ± 12% sched_debug.cpu.nr_uninterruptible.avg
-62.38 -27.5% -45.21 sched_debug.cpu.nr_uninterruptible.min
35.72 ± 16% -25.2% 26.70 ± 10% sched_debug.cpu.nr_uninterruptible.stddev
4666 ± 5% -48.4% 2410 ± 24% sched_debug.cpu.sched_count.avg
11190 ± 12% -28.4% 8014 ± 22% sched_debug.cpu.sched_count.max
3733 ± 7% -52.7% 1765 ± 28% sched_debug.cpu.sched_count.min
1019 ± 7% -19.5% 820.55 ± 17% sched_debug.cpu.sched_count.stddev
497.85 ± 2% -38.7% 305.40 ± 21% sched_debug.cpu.sched_goidle.avg
242.00 ± 9% -53.5% 112.58 ± 27% sched_debug.cpu.sched_goidle.min
2857 -54.8% 1290 ± 25% sched_debug.cpu.ttwu_count.avg
6766 ± 6% -35.6% 4357 ± 27% sched_debug.cpu.ttwu_count.max
2347 -57.6% 996.00 ± 28% sched_debug.cpu.ttwu_count.min
584.18 ± 4% -29.7% 410.47 ± 20% sched_debug.cpu.ttwu_count.stddev
1250 -48.9% 638.75 ± 26% sched_debug.cpu.ttwu_local.avg
3515 ± 11% -42.5% 2021 ± 22% sched_debug.cpu.ttwu_local.max
995.06 ± 2% -49.7% 500.12 ± 28% sched_debug.cpu.ttwu_local.min
289.08 ± 9% -40.6% 171.74 ± 18% sched_debug.cpu.ttwu_local.stddev
138545 -26.1% 102437 ± 14% sched_debug.cpu_clk
134497 -26.8% 98387 ± 14% sched_debug.ktime
2.84 -16.7% 2.36 ± 11% sched_debug.rt_rq:/.rt_runtime.stddev
139160 -26.0% 103042 ± 13% sched_debug.sched_clk
2.28 ± 10% +110.8% 4.81 ± 24% perf-stat.i.MPKI
1.726e+10 ± 3% -8.6% 1.577e+10 perf-stat.i.branch-instructions
0.20 +0.1 0.33 ± 29% perf-stat.i.branch-miss-rate%
25431008 +18.6% 30149004 ± 2% perf-stat.i.branch-misses
56.52 -46.6 9.92 ± 4% perf-stat.i.cache-miss-rate%
85078601 ± 12% -69.1% 26321756 ± 12% perf-stat.i.cache-misses
1.472e+08 ± 11% +91.7% 2.821e+08 ± 11% perf-stat.i.cache-references
9686 ± 2% -11.6% 8558 perf-stat.i.context-switches
5.553e+11 -4.6% 5.295e+11 perf-stat.i.cpu-cycles
1080 ± 2% -18.1% 885.31 perf-stat.i.cpu-migrations
6661 ± 11% +212.6% 20828 ± 14% perf-stat.i.cycles-between-cache-misses
0.01 ± 3% +0.0 0.03 ± 33% perf-stat.i.dTLB-load-miss-rate%
997340 ± 7% +35.5% 1350940 ± 13% perf-stat.i.dTLB-load-misses
2.366e+10 ± 4% -11.9% 2.085e+10 perf-stat.i.dTLB-loads
56117 ± 3% +48.5% 83312 ± 16% perf-stat.i.dTLB-store-misses
1.553e+09 ± 3% +38.7% 2.153e+09 perf-stat.i.dTLB-stores
94.75 -1.5 93.20 perf-stat.i.iTLB-load-miss-rate%
11529887 ± 5% +12.0% 12917345 ± 2% perf-stat.i.iTLB-load-misses
523515 ± 11% +41.2% 739299 ± 2% perf-stat.i.iTLB-loads
8.205e+10 ± 3% -10.4% 7.351e+10 perf-stat.i.instructions
6945 ± 6% -21.0% 5484 ± 2% perf-stat.i.instructions-per-iTLB-miss
4416 +4.5% 4613 perf-stat.i.minor-faults
94.33 -7.5 86.80 perf-stat.i.node-load-miss-rate%
18389129 ± 14% -71.0% 5334408 ± 11% perf-stat.i.node-load-misses
1094365 ± 17% -26.0% 810217 ± 5% perf-stat.i.node-loads
99.22 -3.4 95.78 perf-stat.i.node-store-miss-rate%
10516492 -73.8% 2752315 ± 2% perf-stat.i.node-store-misses
61297 ± 8% +74.0% 106651 ± 2% perf-stat.i.node-stores
4416 +4.5% 4613 perf-stat.i.page-faults
1.81 ± 15% +112.7% 3.84 ± 11% perf-stat.overall.MPKI
0.15 ± 3% +0.0 0.19 ± 2% perf-stat.overall.branch-miss-rate%
57.71 -48.4 9.31 perf-stat.overall.cache-miss-rate%
6620 ± 10% +209.8% 20510 ± 14% perf-stat.overall.cycles-between-cache-misses
0.00 ± 5% +0.0 0.01 ± 13% perf-stat.overall.dTLB-load-miss-rate%
95.66 -1.1 94.57 perf-stat.overall.iTLB-load-miss-rate%
7135 ± 6% -20.2% 5692 ± 2% perf-stat.overall.instructions-per-iTLB-miss
94.40 -7.7 86.74 perf-stat.overall.node-load-miss-rate%
99.42 -3.2 96.27 perf-stat.overall.node-store-miss-rate%
1.717e+10 ± 3% -8.8% 1.566e+10 perf-stat.ps.branch-instructions
25310366 +18.3% 29946644 ± 2% perf-stat.ps.branch-misses
84667928 ± 12% -69.1% 26141865 ± 12% perf-stat.ps.cache-misses
1.465e+08 ± 11% +91.3% 2.802e+08 ± 11% perf-stat.ps.cache-references
9643 ± 2% -11.8% 8505 perf-stat.ps.context-switches
5.526e+11 -4.9% 5.258e+11 perf-stat.ps.cpu-cycles
1075 ± 2% -18.3% 879.16 perf-stat.ps.cpu-migrations
993108 ± 8% +35.3% 1343864 ± 13% perf-stat.ps.dTLB-load-misses
2.354e+10 ± 4% -12.1% 2.07e+10 perf-stat.ps.dTLB-loads
55888 ± 3% +48.5% 83004 ± 16% perf-stat.ps.dTLB-store-misses
1.545e+09 ± 2% +38.4% 2.138e+09 perf-stat.ps.dTLB-stores
11474914 ± 5% +11.8% 12827782 ± 2% perf-stat.ps.iTLB-load-misses
522199 ± 11% +41.0% 736255 ± 2% perf-stat.ps.iTLB-loads
8.165e+10 ± 3% -10.6% 7.299e+10 perf-stat.ps.instructions
4396 +4.3% 4587 perf-stat.ps.minor-faults
18300279 ± 14% -71.1% 5297385 ± 11% perf-stat.ps.node-load-misses
1089093 ± 17% -26.1% 804594 ± 5% perf-stat.ps.node-loads
10465460 -73.9% 2733026 ± 2% perf-stat.ps.node-store-misses
61024 ± 8% +73.7% 106007 ± 2% perf-stat.ps.node-stores
4396 +4.3% 4587 perf-stat.ps.page-faults
1.84e+13 ± 2% -36.7% 1.165e+13 perf-stat.total.instructions
20.58 ± 7% -20.6 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_truncate
20.45 ± 7% -20.5 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_setattr
20.26 ± 7% -20.3 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_truncate_blocks
20.52 ± 7% -15.7 4.79 ± 6% perf-profile.calltrace.cycles-pp.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_setattr.notify_change.do_truncate
20.52 ± 7% -15.7 4.79 ± 6% perf-profile.calltrace.cycles-pp.f2fs_mark_inode_dirty_sync.f2fs_setattr.notify_change.do_truncate.path_openat
20.51 ± 7% -15.7 4.79 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_setattr.notify_change
20.65 ± 7% -15.1 5.52 ± 6% perf-profile.calltrace.cycles-pp.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_truncate.f2fs_setattr.notify_change
20.65 ± 7% -15.1 5.52 ± 6% perf-profile.calltrace.cycles-pp.f2fs_mark_inode_dirty_sync.f2fs_truncate.f2fs_setattr.notify_change.do_truncate
20.64 ± 7% -15.1 5.52 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_truncate.f2fs_setattr
32.71 ± 12% -14.0 18.68 ± 5% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_open.do_syscall_64
32.64 ± 12% -14.0 18.63 ± 5% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_open
32.42 ± 12% -14.0 18.43 ± 5% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.path_openat.do_filp_open
62.22 ± 7% -8.5 53.68 ± 6% perf-profile.calltrace.cycles-pp.f2fs_setattr.notify_change.do_truncate.path_openat.do_filp_open
62.31 ± 7% -8.5 53.78 ± 6% perf-profile.calltrace.cycles-pp.notify_change.do_truncate.path_openat.do_filp_open.do_sys_open
62.36 ± 7% -8.5 53.85 ± 6% perf-profile.calltrace.cycles-pp.do_truncate.path_openat.do_filp_open.do_sys_open.do_syscall_64
98.33 -0.7 97.64 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.39 ±102% +0.9 1.33 ± 18% perf-profile.calltrace.cycles-pp._raw_spin_lock.dput.fscrypt_file_open.f2fs_file_open.do_dentry_open
0.39 ±102% +1.1 1.50 ± 17% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_get_not_zero.dget_parent.fscrypt_file_open.f2fs_file_open
0.40 ±102% +1.2 1.57 ± 15% perf-profile.calltrace.cycles-pp.dput.fscrypt_file_open.f2fs_file_open.do_dentry_open.path_openat
0.00 +1.3 1.33 ± 18% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.dput.fscrypt_file_open.f2fs_file_open
0.42 ±101% +1.4 1.78 ± 13% perf-profile.calltrace.cycles-pp.lockref_get_not_zero.dget_parent.fscrypt_file_open.f2fs_file_open.do_dentry_open
0.42 ±101% +1.4 1.78 ± 13% perf-profile.calltrace.cycles-pp.dget_parent.fscrypt_file_open.f2fs_file_open.do_dentry_open.path_openat
0.00 +1.5 1.49 ± 17% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.lockref_get_not_zero.dget_parent.fscrypt_file_open
1.01 ± 70% +2.4 3.38 ± 14% perf-profile.calltrace.cycles-pp.fscrypt_file_open.f2fs_file_open.do_dentry_open.path_openat.do_filp_open
1.01 ± 70% +2.4 3.38 ± 14% perf-profile.calltrace.cycles-pp.f2fs_file_open.do_dentry_open.path_openat.do_filp_open.do_sys_open
1.07 ± 69% +2.4 3.51 ± 13% perf-profile.calltrace.cycles-pp.do_dentry_open.path_openat.do_filp_open.do_sys_open.do_syscall_64
0.00 +4.7 4.75 ± 6% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_setattr
0.00 +5.5 5.47 ± 6% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_truncate
41.43 ± 7% +7.2 48.61 ± 6% perf-profile.calltrace.cycles-pp.f2fs_truncate.f2fs_setattr.notify_change.do_truncate.path_openat
0.38 ±102% +8.4 8.82 ± 19% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_get_not_dead.legitimize_path.unlazy_walk.complete_walk
0.43 ±102% +8.7 9.08 ± 18% perf-profile.calltrace.cycles-pp.lockref_get_not_dead.legitimize_path.unlazy_walk.complete_walk.path_openat
0.43 ±102% +8.7 9.12 ± 18% perf-profile.calltrace.cycles-pp.legitimize_path.unlazy_walk.complete_walk.path_openat.do_filp_open
0.43 ±102% +8.7 9.12 ± 18% perf-profile.calltrace.cycles-pp.complete_walk.path_openat.do_filp_open.do_sys_open.do_syscall_64
0.43 ±102% +8.7 9.12 ± 18% perf-profile.calltrace.cycles-pp.unlazy_walk.complete_walk.path_openat.do_filp_open.do_sys_open
0.00 +8.8 8.79 ± 19% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.lockref_get_not_dead.legitimize_path.unlazy_walk
0.39 ±102% +10.8 11.14 ± 19% perf-profile.calltrace.cycles-pp._raw_spin_lock.dput.path_openat.do_filp_open.do_sys_open
0.43 ±101% +11.0 11.42 ± 18% perf-profile.calltrace.cycles-pp.dput.path_openat.do_filp_open.do_sys_open.do_syscall_64
0.00 +11.1 11.12 ± 19% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.dput.path_openat.do_filp_open
20.60 ± 7% +22.3 42.94 ± 6% perf-profile.calltrace.cycles-pp.f2fs_truncate_blocks.f2fs_truncate.f2fs_setattr.notify_change.do_truncate
20.37 ± 7% +22.3 42.71 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_truncate_blocks.f2fs_truncate
20.38 ± 7% +22.4 42.73 ± 6% perf-profile.calltrace.cycles-pp.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_truncate_blocks.f2fs_truncate.f2fs_setattr
20.38 ± 7% +22.4 42.73 ± 6% perf-profile.calltrace.cycles-pp.f2fs_mark_inode_dirty_sync.f2fs_truncate_blocks.f2fs_truncate.f2fs_setattr.notify_change
0.00 +42.6 42.61 ± 6% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.f2fs_inode_dirtied.f2fs_mark_inode_dirty_sync.f2fs_truncate_blocks
63.49 ± 6% -63.5 0.00 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
32.74 ± 12% -14.0 18.70 ± 5% perf-profile.children.cycles-pp.rwsem_down_write_slowpath
32.68 ± 12% -14.0 18.65 ± 5% perf-profile.children.cycles-pp.rwsem_optimistic_spin
32.46 ± 12% -14.0 18.46 ± 5% perf-profile.children.cycles-pp.osq_lock
62.22 ± 7% -8.5 53.68 ± 6% perf-profile.children.cycles-pp.f2fs_setattr
62.31 ± 7% -8.5 53.78 ± 6% perf-profile.children.cycles-pp.notify_change
61.58 ± 7% -8.5 53.05 ± 6% perf-profile.children.cycles-pp.f2fs_mark_inode_dirty_sync
62.36 ± 7% -8.5 53.85 ± 6% perf-profile.children.cycles-pp.do_truncate
61.58 ± 7% -8.5 53.07 ± 6% perf-profile.children.cycles-pp.f2fs_inode_dirtied
98.35 -0.7 97.66 perf-profile.children.cycles-pp.do_filp_open
98.33 -0.7 97.64 perf-profile.children.cycles-pp.path_openat
98.45 -0.7 97.79 perf-profile.children.cycles-pp.do_sys_open
0.40 ± 7% -0.1 0.31 ± 4% perf-profile.children.cycles-pp.update_process_times
0.40 ± 7% -0.1 0.31 ± 4% perf-profile.children.cycles-pp.tick_sched_handle
0.36 ± 8% -0.1 0.28 ± 6% perf-profile.children.cycles-pp.scheduler_tick
0.55 ± 6% -0.1 0.47 ± 4% perf-profile.children.cycles-pp.tick_sched_timer
0.34 ± 7% -0.1 0.26 ± 5% perf-profile.children.cycles-pp.task_tick_fair
0.31 ± 5% -0.1 0.24 perf-profile.children.cycles-pp.rwsem_wake
0.21 ± 8% -0.1 0.15 ± 5% perf-profile.children.cycles-pp.update_cfs_group
0.11 ± 7% -0.0 0.09 perf-profile.children.cycles-pp.wake_up_q
0.10 ± 8% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.try_to_wake_up
0.07 -0.0 0.05 perf-profile.children.cycles-pp.rwsem_mark_wake
0.08 ± 6% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.update_load_avg
0.05 +0.0 0.07 ± 7% perf-profile.children.cycles-pp.___might_sleep
0.07 ± 7% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.__fput
0.05 ± 8% +0.0 0.07 ± 5% perf-profile.children.cycles-pp.getname_flags
0.07 ± 7% +0.0 0.09 perf-profile.children.cycles-pp.selinux_inode_permission
0.14 ± 3% +0.0 0.16 ± 2% perf-profile.children.cycles-pp.down_write
0.07 ± 6% +0.0 0.10 ± 5% perf-profile.children.cycles-pp.security_inode_permission
0.07 ± 7% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.alloc_empty_file
0.06 ± 6% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.__alloc_file
0.12 ± 3% +0.0 0.15 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop
0.11 ± 7% +0.0 0.15 ± 7% perf-profile.children.cycles-pp.rcu_core
0.11 ± 4% +0.0 0.15 ± 3% perf-profile.children.cycles-pp.task_work_run
0.14 ± 8% +0.0 0.18 ± 4% perf-profile.children.cycles-pp.__softirqentry_text_start
0.01 ±173% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.kmem_cache_alloc
0.01 ±173% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.__follow_mount_rcu
0.00 +0.1 0.05 perf-profile.children.cycles-pp.strncpy_from_user
0.00 +0.1 0.05 perf-profile.children.cycles-pp.run_ksoftirqd
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.__x64_sys_close
0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.pagecache_get_page
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.smpboot_thread_fn
0.02 ±173% +0.1 0.08 ± 5% perf-profile.children.cycles-pp.lookup_fast
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.02 ±173% +0.1 0.08 ± 5% perf-profile.children.cycles-pp.walk_component
0.08 ± 5% +0.1 0.16 ± 6% perf-profile.children.cycles-pp.do_unlinkat
0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.f2fs_evict_inode
0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.f2fs_remove_inode_page
0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.truncate_node
0.00 +0.1 0.09 ± 12% perf-profile.children.cycles-pp.evict
0.14 ± 15% +0.1 0.23 ± 3% perf-profile.children.cycles-pp.link_path_walk
0.00 +0.1 0.10 ± 7% perf-profile.children.cycles-pp.f2fs_inode_synced
0.00 +0.1 0.12 ± 4% perf-profile.children.cycles-pp.cna_scan_main_queue
0.12 ± 13% +0.3 0.46 ± 6% perf-profile.children.cycles-pp.lockref_put_return
0.56 ± 52% +1.2 1.78 ± 13% perf-profile.children.cycles-pp.lockref_get_not_zero
0.56 ± 52% +1.2 1.78 ± 13% perf-profile.children.cycles-pp.dget_parent
1.10 ± 54% +2.3 3.38 ± 14% perf-profile.children.cycles-pp.fscrypt_file_open
1.10 ± 54% +2.3 3.38 ± 14% perf-profile.children.cycles-pp.f2fs_file_open
1.19 ± 48% +2.3 3.51 ± 13% perf-profile.children.cycles-pp.do_dentry_open
41.43 ± 7% +7.2 48.61 ± 6% perf-profile.children.cycles-pp.f2fs_truncate
0.58 ± 52% +8.5 9.09 ± 18% perf-profile.children.cycles-pp.lockref_get_not_dead
0.59 ± 51% +8.5 9.12 ± 18% perf-profile.children.cycles-pp.legitimize_path
0.59 ± 50% +8.5 9.13 ± 18% perf-profile.children.cycles-pp.complete_walk
0.59 ± 50% +8.5 9.13 ± 18% perf-profile.children.cycles-pp.unlazy_walk
1.14 ± 51% +11.9 13.03 ± 18% perf-profile.children.cycles-pp.dput
63.59 ± 6% +12.4 76.00 perf-profile.children.cycles-pp._raw_spin_lock
20.64 ± 7% +22.3 42.95 ± 6% perf-profile.children.cycles-pp.f2fs_truncate_blocks
0.00 +75.9 75.91 perf-profile.children.cycles-pp.__cna_queued_spin_lock_slowpath
62.89 ± 6% -62.9 0.00 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
32.14 ± 12% -13.9 18.27 ± 5% perf-profile.self.cycles-pp.osq_lock
0.21 ± 8% -0.1 0.15 ± 5% perf-profile.self.cycles-pp.update_cfs_group
0.11 ± 7% -0.0 0.09 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.10 ± 4% +0.0 0.12 ± 7% perf-profile.self.cycles-pp.down_write
0.04 ± 57% +0.0 0.06 ± 6% perf-profile.self.cycles-pp.___might_sleep
0.00 +0.1 0.05 perf-profile.self.cycles-pp.dput
0.00 +0.1 0.05 perf-profile.self.cycles-pp.link_path_walk
0.01 ±173% +0.1 0.10 ± 7% perf-profile.self.cycles-pp.path_openat
0.00 +0.1 0.12 ± 4% perf-profile.self.cycles-pp.cna_scan_main_queue
0.09 ± 11% +0.2 0.26 ± 9% perf-profile.self.cycles-pp.lockref_get_not_dead
0.07 ± 22% +0.2 0.27 ± 7% perf-profile.self.cycles-pp.lockref_get_not_zero
0.12 ± 13% +0.3 0.46 ± 6% perf-profile.self.cycles-pp.lockref_put_return
0.00 +75.1 75.09 perf-profile.self.cycles-pp.__cna_queued_spin_lock_slowpath
147317 ± 2% -22.7% 113881 softirqs.CPU0.RCU
89302 ± 4% -27.6% 64619 ± 2% softirqs.CPU0.TIMER
148746 ± 2% -23.1% 114328 ± 2% softirqs.CPU1.RCU
82526 ± 4% -29.8% 57937 ± 4% softirqs.CPU1.TIMER
145190 ± 2% -22.6% 112320 ± 2% softirqs.CPU10.RCU
83341 ± 4% -26.8% 61027 ± 3% softirqs.CPU10.TIMER
143250 ± 3% -22.2% 111445 softirqs.CPU100.RCU
81773 ± 4% -25.5% 60915 ± 2% softirqs.CPU100.TIMER
144710 ± 2% -23.2% 111208 softirqs.CPU101.RCU
82353 ± 4% -26.2% 60790 ± 4% softirqs.CPU101.TIMER
143111 -22.9% 110336 softirqs.CPU102.RCU
81699 ± 3% -27.7% 59090 ± 2% softirqs.CPU102.TIMER
144336 -23.2% 110806 softirqs.CPU103.RCU
82687 ± 2% -28.5% 59129 ± 2% softirqs.CPU103.TIMER
142816 ± 2% -22.5% 110684 softirqs.CPU104.RCU
80722 ± 4% -25.8% 59886 ± 3% softirqs.CPU104.TIMER
144132 ± 2% -23.5% 110313 ± 2% softirqs.CPU105.RCU
83668 ± 4% -29.3% 59123 ± 2% softirqs.CPU105.TIMER
144943 ± 2% -24.0% 110097 softirqs.CPU106.RCU
82974 ± 4% -28.8% 59081 ± 2% softirqs.CPU106.TIMER
145187 ± 4% -24.2% 110051 softirqs.CPU107.RCU
83685 ± 8% -29.3% 59134 softirqs.CPU107.TIMER
144525 ± 3% -23.6% 110476 softirqs.CPU108.RCU
82368 ± 5% -28.8% 58642 softirqs.CPU108.TIMER
145103 ± 2% -24.1% 110205 softirqs.CPU109.RCU
82881 ± 4% -29.4% 58536 ± 2% softirqs.CPU109.TIMER
145917 ± 3% -24.0% 110909 softirqs.CPU11.RCU
84518 ± 6% -28.1% 60780 ± 3% softirqs.CPU11.TIMER
144712 ± 2% -23.7% 110350 softirqs.CPU110.RCU
82653 ± 4% -28.1% 59391 ± 2% softirqs.CPU110.TIMER
142581 ± 4% -21.7% 111683 softirqs.CPU111.RCU
81902 ± 5% -26.5% 60224 ± 3% softirqs.CPU111.TIMER
145740 ± 2% -23.4% 111699 ± 2% softirqs.CPU112.RCU
83678 ± 4% -28.0% 60220 ± 3% softirqs.CPU112.TIMER
144697 ± 2% -22.8% 111667 softirqs.CPU113.RCU
82563 ± 3% -27.1% 60207 ± 2% softirqs.CPU113.TIMER
143253 ± 2% -23.6% 109467 ± 2% softirqs.CPU114.RCU
81295 ± 3% -27.6% 58891 ± 3% softirqs.CPU114.TIMER
144182 ± 2% -23.0% 111034 ± 2% softirqs.CPU115.RCU
82003 ± 3% -27.3% 59615 softirqs.CPU115.TIMER
145377 ± 4% -22.9% 112084 softirqs.CPU116.RCU
83723 ± 7% -28.6% 59800 softirqs.CPU116.TIMER
146321 ± 2% -24.2% 110984 softirqs.CPU117.RCU
84161 ± 5% -29.6% 59235 softirqs.CPU117.TIMER
143383 -23.1% 110284 softirqs.CPU118.RCU
81282 ± 3% -27.7% 58789 ± 2% softirqs.CPU118.TIMER
144588 -22.5% 112016 ± 2% softirqs.CPU119.RCU
81160 ± 3% -26.3% 59795 ± 5% softirqs.CPU119.TIMER
145254 ± 2% -23.7% 110795 ± 2% softirqs.CPU12.RCU
83492 ± 5% -27.4% 60599 softirqs.CPU12.TIMER
145222 ± 2% -22.3% 112794 ± 2% softirqs.CPU120.RCU
81790 ± 4% -26.8% 59852 ± 3% softirqs.CPU120.TIMER
146363 ± 2% -23.4% 112128 softirqs.CPU121.RCU
83774 ± 5% -29.1% 59422 softirqs.CPU121.TIMER
145543 ± 3% -22.7% 112477 ± 2% softirqs.CPU122.RCU
82480 ± 4% -27.1% 60101 softirqs.CPU122.TIMER
144952 ± 3% -22.0% 113085 softirqs.CPU123.RCU
82026 ± 4% -25.9% 60810 ± 2% softirqs.CPU123.TIMER
145097 ± 2% -22.9% 111905 ± 2% softirqs.CPU124.RCU
82318 ± 4% -27.5% 59675 ± 4% softirqs.CPU124.TIMER
144339 -22.9% 111253 softirqs.CPU125.RCU
82110 ± 3% -27.7% 59359 ± 2% softirqs.CPU125.TIMER
144791 ± 2% -23.1% 111315 softirqs.CPU126.RCU
81783 ± 3% -28.1% 58777 ± 2% softirqs.CPU126.TIMER
144003 ± 2% -22.0% 112352 ± 2% softirqs.CPU127.RCU
81767 ± 4% -26.2% 60338 ± 4% softirqs.CPU127.TIMER
143977 ± 3% -21.4% 113125 ± 3% softirqs.CPU128.RCU
81315 ± 4% -24.3% 61571 ± 7% softirqs.CPU128.TIMER
144784 ± 3% -22.5% 112280 softirqs.CPU129.RCU
82025 ± 4% -26.3% 60436 ± 2% softirqs.CPU129.TIMER
145985 ± 2% -23.8% 111216 softirqs.CPU13.RCU
83767 ± 4% -28.8% 59631 softirqs.CPU13.TIMER
144477 ± 3% -22.8% 111593 softirqs.CPU130.RCU
82052 ± 3% -27.9% 59169 softirqs.CPU130.TIMER
143558 ± 3% -22.3% 111597 ± 2% softirqs.CPU131.RCU
81615 ± 4% -26.2% 60240 ± 2% softirqs.CPU131.TIMER
143118 ± 3% -21.5% 112342 ± 3% softirqs.CPU132.RCU
81107 ± 4% -25.6% 60321 ± 4% softirqs.CPU132.TIMER
144625 ± 3% -23.0% 111308 softirqs.CPU133.RCU
81672 ± 4% -27.7% 59048 ± 2% softirqs.CPU133.TIMER
145056 ± 3% -23.5% 110929 softirqs.CPU134.RCU
82852 ± 5% -29.0% 58836 ± 2% softirqs.CPU134.TIMER
145438 ± 3% -23.5% 111229 softirqs.CPU135.RCU
82718 ± 5% -28.0% 59598 ± 2% softirqs.CPU135.TIMER
144527 ± 3% -23.3% 110838 softirqs.CPU136.RCU
82411 ± 5% -28.3% 59062 softirqs.CPU136.TIMER
144957 ± 3% -23.5% 110898 softirqs.CPU137.RCU
82707 ± 5% -29.2% 58540 ± 2% softirqs.CPU137.TIMER
144151 ± 3% -23.2% 110703 softirqs.CPU138.RCU
82764 ± 3% -29.0% 58732 softirqs.CPU138.TIMER
143826 ± 2% -22.2% 111881 ± 3% softirqs.CPU139.RCU
82039 ± 4% -26.2% 60509 ± 5% softirqs.CPU139.TIMER
146354 -24.2% 110929 ± 2% softirqs.CPU14.RCU
85193 ± 2% -29.7% 59920 ± 2% softirqs.CPU14.TIMER
143552 -23.1% 110392 softirqs.CPU140.RCU
81365 ± 3% -27.8% 58746 ± 3% softirqs.CPU140.TIMER
144373 ± 3% -22.5% 111848 ± 2% softirqs.CPU141.RCU
81349 ± 4% -27.1% 59326 ± 3% softirqs.CPU141.TIMER
143979 ± 3% -22.1% 112202 softirqs.CPU142.RCU
81827 ± 4% -27.2% 59575 ± 2% softirqs.CPU142.TIMER
147062 ± 3% -23.9% 111956 softirqs.CPU143.RCU
83990 ± 4% -28.6% 59954 ± 3% softirqs.CPU143.TIMER
145158 ± 3% -24.5% 109601 softirqs.CPU144.RCU
82800 ± 3% -29.8% 58144 ± 2% softirqs.CPU144.TIMER
143290 -23.9% 109052 ± 2% softirqs.CPU145.RCU
81618 ± 2% -29.0% 57922 ± 2% softirqs.CPU145.TIMER
144603 ± 2% -24.2% 109678 softirqs.CPU146.RCU
82056 ± 3% -28.3% 58871 ± 2% softirqs.CPU146.TIMER
144756 ± 2% -23.8% 110238 ± 2% softirqs.CPU147.RCU
82996 ± 2% -28.6% 59275 ± 3% softirqs.CPU147.TIMER
144538 ± 2% -24.4% 109207 ± 2% softirqs.CPU148.RCU
82372 ± 3% -29.5% 58080 ± 2% softirqs.CPU148.TIMER
144549 ± 2% -24.4% 109258 softirqs.CPU149.RCU
82007 ± 3% -28.5% 58641 softirqs.CPU149.TIMER
144303 ± 2% -23.0% 111150 ± 2% softirqs.CPU15.RCU
83123 ± 4% -27.4% 60380 ± 2% softirqs.CPU15.TIMER
143430 ± 3% -23.8% 109253 softirqs.CPU150.RCU
81186 ± 3% -28.3% 58197 ± 2% softirqs.CPU150.TIMER
143809 ± 3% -24.1% 109088 ± 2% softirqs.CPU151.RCU
81603 ± 3% -28.7% 58178 ± 2% softirqs.CPU151.TIMER
144093 ± 2% -24.0% 109489 softirqs.CPU152.RCU
82010 ± 2% -28.9% 58322 ± 2% softirqs.CPU152.TIMER
143481 ± 3% -23.3% 110044 softirqs.CPU153.RCU
82143 ± 3% -28.9% 58434 softirqs.CPU153.TIMER
143106 ± 3% -22.8% 110446 ± 3% softirqs.CPU154.RCU
92065 ± 23% -35.0% 59825 ± 6% softirqs.CPU154.TIMER
143115 ± 3% -23.3% 109772 softirqs.CPU155.RCU
81569 ± 2% -28.7% 58139 softirqs.CPU155.TIMER
143529 ± 2% -23.4% 109952 softirqs.CPU156.RCU
81453 ± 3% -28.4% 58280 ± 2% softirqs.CPU156.TIMER
143375 ± 2% -24.0% 108990 softirqs.CPU157.RCU
81909 ± 2% -29.6% 57659 ± 2% softirqs.CPU157.TIMER
142963 ± 3% -22.7% 110529 ± 2% softirqs.CPU158.RCU
81567 ± 3% -27.1% 59468 ± 3% softirqs.CPU158.TIMER
143212 ± 2% -22.1% 111604 ± 2% softirqs.CPU159.RCU
82160 ± 3% -27.2% 59783 ± 2% softirqs.CPU159.TIMER
146915 ± 2% -24.7% 110686 softirqs.CPU16.RCU
83941 ± 3% -28.8% 59761 ± 2% softirqs.CPU16.TIMER
143790 ± 3% -23.2% 110414 softirqs.CPU160.RCU
81562 ± 4% -27.5% 59094 softirqs.CPU160.TIMER
142218 ± 3% -21.5% 111579 softirqs.CPU161.RCU
80937 ± 4% -25.9% 59973 softirqs.CPU161.TIMER
144186 ± 2% -23.8% 109912 ± 2% softirqs.CPU162.RCU
81766 ± 3% -28.6% 58412 ± 2% softirqs.CPU162.TIMER
144565 ± 2% -23.7% 110276 ± 2% softirqs.CPU163.RCU
82888 ± 3% -29.1% 58738 ± 3% softirqs.CPU163.TIMER
143788 ± 3% -23.7% 109743 softirqs.CPU164.RCU
81315 ± 3% -28.2% 58355 softirqs.CPU164.TIMER
143220 ± 2% -23.0% 110248 softirqs.CPU165.RCU
81198 ± 3% -27.9% 58540 ± 2% softirqs.CPU165.TIMER
143603 ± 3% -22.9% 110727 softirqs.CPU166.RCU
81311 ± 3% -27.6% 58838 softirqs.CPU166.TIMER
143034 ± 2% -21.3% 112579 ± 2% softirqs.CPU167.RCU
80538 ± 3% -26.3% 59323 ± 2% softirqs.CPU167.TIMER
144997 ± 2% -21.8% 113410 ± 4% softirqs.CPU168.RCU
87670 ± 5% -28.0% 63130 ± 7% softirqs.CPU168.TIMER
144210 ± 2% -23.3% 110564 softirqs.CPU169.RCU
86254 ± 5% -29.5% 60810 softirqs.CPU169.TIMER
144351 ± 2% -23.1% 111043 softirqs.CPU17.RCU
82818 ± 4% -27.8% 59772 ± 2% softirqs.CPU17.TIMER
144800 ± 2% -23.5% 110763 softirqs.CPU170.RCU
87153 ± 4% -30.8% 60352 softirqs.CPU170.TIMER
145295 ± 3% -23.2% 111627 ± 2% softirqs.CPU171.RCU
144851 ± 2% -23.3% 111092 ± 2% softirqs.CPU172.RCU
86828 ± 5% -29.6% 61113 ± 3% softirqs.CPU172.TIMER
144793 ± 2% -23.5% 110783 softirqs.CPU173.RCU
86374 ± 4% -29.9% 60515 softirqs.CPU173.TIMER
144306 ± 3% -23.2% 110791 softirqs.CPU174.RCU
85489 ± 5% -29.7% 60066 ± 2% softirqs.CPU174.TIMER
145551 ± 3% -24.0% 110564 ± 2% softirqs.CPU175.RCU
97852 ± 17% -38.7% 60017 ± 2% softirqs.CPU175.TIMER
145888 ± 3% -24.2% 110644 softirqs.CPU176.RCU
88036 ± 6% -31.3% 60506 softirqs.CPU176.TIMER
145055 ± 2% -24.5% 109477 ± 2% softirqs.CPU177.RCU
86924 ± 4% -31.0% 59940 softirqs.CPU177.TIMER
143529 -23.1% 110346 ± 2% softirqs.CPU178.RCU
86631 ± 5% -30.9% 59849 ± 2% softirqs.CPU178.TIMER
144621 ± 2% -22.8% 111703 softirqs.CPU179.RCU
86538 ± 4% -28.9% 61499 softirqs.CPU179.TIMER
143931 ± 2% -22.7% 111259 softirqs.CPU18.RCU
82193 ± 4% -27.1% 59925 ± 2% softirqs.CPU18.TIMER
144797 ± 3% -24.3% 109684 softirqs.CPU180.RCU
86723 ± 6% -31.4% 59529 ± 2% softirqs.CPU180.TIMER
143244 ± 2% -23.0% 110287 softirqs.CPU181.RCU
85815 ± 5% -29.5% 60462 softirqs.CPU181.TIMER
143727 ± 2% -23.5% 109933 softirqs.CPU182.RCU
85808 ± 4% -29.9% 60138 ± 2% softirqs.CPU182.TIMER
144005 ± 2% -23.8% 109777 softirqs.CPU183.RCU
87085 ± 4% -30.7% 60333 ± 2% softirqs.CPU183.TIMER
144302 -23.8% 109897 softirqs.CPU184.RCU
86187 ± 4% -30.5% 59906 ± 2% softirqs.CPU184.TIMER
143642 ± 2% -23.9% 109340 softirqs.CPU185.RCU
86001 ± 4% -30.6% 59688 softirqs.CPU185.TIMER
142942 ± 3% -23.8% 108859 softirqs.CPU186.RCU
85489 ± 5% -29.8% 59977 ± 2% softirqs.CPU186.TIMER
143781 ± 3% -23.2% 110376 softirqs.CPU187.RCU
85590 ± 6% -30.2% 59732 softirqs.CPU187.TIMER
143295 ± 2% -23.0% 110310 softirqs.CPU188.RCU
85515 ± 5% -29.7% 60079 ± 2% softirqs.CPU188.TIMER
144280 ± 3% -23.1% 110926 softirqs.CPU189.RCU
146576 ± 2% -23.8% 111667 ± 2% softirqs.CPU19.RCU
85293 ± 5% -29.2% 60390 ± 2% softirqs.CPU19.TIMER
142286 ± 2% -22.7% 110006 softirqs.CPU190.RCU
85556 ± 5% -29.2% 60538 softirqs.CPU190.TIMER
143148 ± 2% -22.3% 111277 softirqs.CPU191.RCU
86075 ± 5% -30.8% 59593 softirqs.CPU191.TIMER
145640 ± 3% -23.4% 111569 ± 2% softirqs.CPU2.RCU
83534 ± 5% -27.4% 60677 ± 4% softirqs.CPU2.TIMER
145621 -23.9% 110855 ± 2% softirqs.CPU20.RCU
83960 ± 3% -28.4% 60138 ± 2% softirqs.CPU20.TIMER
145117 ± 3% -22.5% 112440 ± 3% softirqs.CPU21.RCU
83116 ± 6% -25.8% 61652 ± 4% softirqs.CPU21.TIMER
144182 -22.6% 111535 softirqs.CPU22.RCU
82570 ± 4% -26.8% 60472 softirqs.CPU22.TIMER
145142 ± 3% -23.6% 110900 ± 2% softirqs.CPU23.RCU
82631 ± 6% -27.2% 60148 ± 2% softirqs.CPU23.TIMER
146904 ± 3% -22.3% 114126 ± 2% softirqs.CPU24.RCU
84490 ± 5% -27.1% 61604 ± 2% softirqs.CPU24.TIMER
147378 ± 3% -23.3% 113028 softirqs.CPU25.RCU
84420 ± 5% -28.7% 60201 ± 2% softirqs.CPU25.TIMER
147313 ± 2% -23.2% 113161 softirqs.CPU26.RCU
83470 ± 4% -27.4% 60569 ± 2% softirqs.CPU26.TIMER
145228 ± 3% -22.6% 112377 softirqs.CPU27.RCU
83293 ± 3% -27.4% 60459 ± 3% softirqs.CPU27.TIMER
146418 ± 2% -23.9% 111405 softirqs.CPU28.RCU
83727 ± 4% -29.1% 59331 softirqs.CPU28.TIMER
145975 ± 2% -22.9% 112557 softirqs.CPU29.RCU
83482 ± 3% -27.7% 60332 ± 2% softirqs.CPU29.TIMER
144798 ± 2% -22.6% 112079 softirqs.CPU3.RCU
83081 ± 4% -27.3% 60375 ± 2% softirqs.CPU3.TIMER
146422 ± 2% -22.9% 112865 ± 2% softirqs.CPU30.RCU
83152 ± 4% -27.5% 60269 ± 3% softirqs.CPU30.TIMER
146737 ± 2% -23.9% 111717 softirqs.CPU31.RCU
84103 ± 3% -29.5% 59257 softirqs.CPU31.TIMER
145106 ± 2% -22.8% 111953 softirqs.CPU32.RCU
82755 ± 4% -27.2% 60209 ± 2% softirqs.CPU32.TIMER
144002 ± 3% -21.8% 112643 softirqs.CPU33.RCU
82032 ± 4% -25.8% 60831 ± 3% softirqs.CPU33.TIMER
144603 ± 3% -22.0% 112859 softirqs.CPU34.RCU
82542 ± 4% -25.9% 61186 ± 3% softirqs.CPU34.TIMER
146647 ± 4% -22.7% 113392 softirqs.CPU35.RCU
84499 ± 4% -26.9% 61802 ± 2% softirqs.CPU35.TIMER
144046 ± 3% -22.8% 111137 ± 2% softirqs.CPU36.RCU
82020 ± 4% -28.0% 59043 ± 3% softirqs.CPU36.TIMER
145081 ± 2% -23.0% 111733 softirqs.CPU37.RCU
82599 ± 4% -27.4% 59964 ± 2% softirqs.CPU37.TIMER
144811 ± 3% -22.8% 111843 ± 2% softirqs.CPU38.RCU
82325 ± 4% -27.6% 59629 ± 2% softirqs.CPU38.TIMER
144105 ± 3% -22.4% 111770 softirqs.CPU39.RCU
82002 ± 5% -26.8% 60050 softirqs.CPU39.TIMER
146607 ± 2% -24.8% 110189 softirqs.CPU4.RCU
84721 ± 4% -29.3% 59901 ± 2% softirqs.CPU4.TIMER
144997 ± 2% -22.4% 112453 softirqs.CPU40.RCU
82780 ± 4% -26.3% 61047 ± 2% softirqs.CPU40.TIMER
145672 ± 3% -23.2% 111874 softirqs.CPU41.RCU
83428 ± 5% -27.6% 60441 ± 2% softirqs.CPU41.TIMER
145349 ± 3% -23.4% 111406 ± 2% softirqs.CPU42.RCU
93374 ± 19% -35.6% 60105 ± 2% softirqs.CPU42.TIMER
144820 ± 2% -22.8% 111785 softirqs.CPU43.RCU
83174 ± 3% -27.4% 60362 ± 2% softirqs.CPU43.TIMER
143863 ± 3% -22.4% 111601 softirqs.CPU44.RCU
82193 ± 4% -26.9% 60074 ± 2% softirqs.CPU44.TIMER
144368 ± 3% -22.7% 111575 softirqs.CPU45.RCU
82383 ± 4% -27.3% 59927 ± 2% softirqs.CPU45.TIMER
145430 ± 3% -23.3% 111558 softirqs.CPU46.RCU
82457 ± 4% -27.5% 59806 softirqs.CPU46.TIMER
144521 ± 2% -23.3% 110820 softirqs.CPU47.RCU
82389 ± 4% -27.8% 59515 ± 2% softirqs.CPU47.TIMER
145731 ± 2% -22.7% 112628 softirqs.CPU48.RCU
82261 ± 3% -26.4% 60580 ± 2% softirqs.CPU48.TIMER
145102 ± 3% -23.2% 111417 softirqs.CPU49.RCU
82446 ± 4% -28.0% 59323 softirqs.CPU49.TIMER
146455 ± 3% -24.5% 110547 softirqs.CPU5.RCU
84618 ± 4% -29.1% 59977 ± 2% softirqs.CPU5.TIMER
145635 ± 3% -23.1% 112034 ± 2% softirqs.CPU50.RCU
82651 ± 4% -27.8% 59690 ± 2% softirqs.CPU50.TIMER
144500 ± 2% -22.1% 112603 softirqs.CPU51.RCU
82521 ± 3% -27.5% 59834 ± 2% softirqs.CPU51.TIMER
144050 ± 3% -22.8% 111226 ± 2% softirqs.CPU52.RCU
82349 ± 3% -27.4% 59780 ± 2% softirqs.CPU52.TIMER
143173 ± 3% -22.0% 111742 softirqs.CPU53.RCU
145161 ± 2% -23.8% 110577 softirqs.CPU54.RCU
81997 ± 3% -28.3% 58780 softirqs.CPU54.TIMER
144566 ± 2% -23.5% 110626 softirqs.CPU55.RCU
82062 ± 3% -28.2% 58938 ± 2% softirqs.CPU55.TIMER
143566 ± 3% -22.7% 110976 ± 2% softirqs.CPU56.RCU
81643 ± 3% -27.0% 59582 softirqs.CPU56.TIMER
143899 ± 3% -23.0% 110789 softirqs.CPU57.RCU
82255 ± 3% -27.7% 59503 ± 2% softirqs.CPU57.TIMER
143695 ± 3% -22.6% 111255 ± 3% softirqs.CPU58.RCU
83656 ± 5% -28.4% 59903 ± 3% softirqs.CPU58.TIMER
142480 ± 3% -22.2% 110817 ± 2% softirqs.CPU59.RCU
81482 ± 3% -26.8% 59676 ± 3% softirqs.CPU59.TIMER
145453 ± 2% -23.3% 111508 softirqs.CPU6.RCU
83213 ± 4% -27.4% 60375 ± 2% softirqs.CPU6.TIMER
144535 ± 2% -22.4% 112175 softirqs.CPU60.RCU
82566 ± 2% -27.5% 59899 softirqs.CPU60.TIMER
147031 ± 2% -23.7% 112180 ± 4% softirqs.CPU61.RCU
83473 ± 3% -27.1% 60876 ± 5% softirqs.CPU61.TIMER
144650 ± 3% -23.6% 110450 ± 2% softirqs.CPU62.RCU
82545 ± 3% -27.7% 59699 softirqs.CPU62.TIMER
144775 ± 2% -22.7% 111870 ± 2% softirqs.CPU63.RCU
83272 ± 3% -27.9% 60004 softirqs.CPU63.TIMER
145135 ± 3% -24.2% 110014 ± 3% softirqs.CPU64.RCU
82884 ± 3% -28.5% 59223 ± 2% softirqs.CPU64.TIMER
145969 ± 3% -23.9% 111065 ± 2% softirqs.CPU65.RCU
83738 ± 3% -28.5% 59893 ± 2% softirqs.CPU65.TIMER
144824 ± 3% -22.5% 112209 ± 2% softirqs.CPU66.RCU
82567 ± 3% -27.3% 60046 ± 2% softirqs.CPU66.TIMER
144546 ± 3% -23.5% 110526 softirqs.CPU67.RCU
82519 ± 3% -28.7% 58828 softirqs.CPU67.TIMER
145508 ± 2% -24.5% 109801 ± 3% softirqs.CPU68.RCU
83087 ± 3% -28.9% 59066 ± 2% softirqs.CPU68.TIMER
145652 ± 3% -23.2% 111805 ± 2% softirqs.CPU69.RCU
82903 ± 3% -28.0% 59674 ± 3% softirqs.CPU69.TIMER
144831 ± 2% -23.7% 110447 softirqs.CPU7.RCU
83217 ± 4% -27.6% 60235 ± 2% softirqs.CPU7.TIMER
144167 ± 3% -22.6% 111554 softirqs.CPU70.RCU
81998 ± 3% -27.6% 59343 ± 2% softirqs.CPU70.TIMER
144459 ± 2% -23.9% 109873 ± 2% softirqs.CPU71.RCU
82025 ± 4% -28.6% 58550 ± 2% softirqs.CPU71.TIMER
145860 ± 2% -21.2% 114905 softirqs.CPU72.RCU
87304 ± 4% -27.4% 63382 ± 4% softirqs.CPU72.TIMER
145902 ± 2% -23.0% 112392 softirqs.CPU73.RCU
87669 ± 4% -30.5% 60957 softirqs.CPU73.TIMER
146060 -23.5% 111803 ± 2% softirqs.CPU74.RCU
87696 ± 4% -30.1% 61263 ± 2% softirqs.CPU74.TIMER
147053 -24.5% 111018 ± 2% softirqs.CPU75.RCU
88710 ± 2% -31.5% 60797 ± 2% softirqs.CPU75.TIMER
145305 ± 2% -23.3% 111435 ± 2% softirqs.CPU76.RCU
87102 ± 4% -29.4% 61523 softirqs.CPU76.TIMER
144707 ± 2% -23.6% 110504 softirqs.CPU77.RCU
86521 ± 5% -30.2% 60379 ± 2% softirqs.CPU77.TIMER
145248 ± 2% -22.8% 112117 softirqs.CPU78.RCU
86177 ± 4% -29.1% 61099 softirqs.CPU78.TIMER
146152 ± 2% -23.6% 111663 softirqs.CPU79.RCU
87096 ± 4% -29.5% 61370 softirqs.CPU79.TIMER
145117 ± 2% -23.0% 111775 softirqs.CPU8.RCU
83540 ± 4% -25.9% 61869 softirqs.CPU8.TIMER
144631 ± 2% -23.1% 111158 softirqs.CPU80.RCU
86567 ± 4% -29.5% 61061 softirqs.CPU80.TIMER
144946 ± 2% -22.8% 111834 softirqs.CPU81.RCU
87373 ± 4% -28.9% 62107 softirqs.CPU81.TIMER
145145 -22.7% 112134 ± 2% softirqs.CPU82.RCU
144227 ± 2% -22.7% 111449 softirqs.CPU83.RCU
86902 ± 4% -29.0% 61721 softirqs.CPU83.TIMER
144536 ± 2% -23.0% 111261 softirqs.CPU84.RCU
86380 ± 4% -29.4% 61000 softirqs.CPU84.TIMER
145033 ± 3% -23.4% 111100 softirqs.CPU85.RCU
87385 ± 6% -29.9% 61250 ± 2% softirqs.CPU85.TIMER
145333 ± 2% -23.7% 110839 softirqs.CPU86.RCU
87039 ± 4% -29.7% 61221 softirqs.CPU86.TIMER
145178 ± 2% -23.4% 111137 softirqs.CPU87.RCU
87346 ± 5% -30.0% 61159 softirqs.CPU87.TIMER
144445 ± 3% -22.9% 111363 softirqs.CPU88.RCU
86748 ± 4% -29.0% 61556 softirqs.CPU88.TIMER
144967 ± 3% -23.2% 111301 softirqs.CPU89.RCU
87135 ± 5% -29.2% 61693 softirqs.CPU89.TIMER
146168 ± 3% -21.4% 114910 ± 3% softirqs.CPU9.RCU
94328 ± 17% -32.1% 64050 ± 6% softirqs.CPU9.TIMER
144608 ± 2% -24.0% 109965 softirqs.CPU90.RCU
86850 ± 4% -29.9% 60845 softirqs.CPU90.TIMER
144181 ± 2% -23.6% 110125 softirqs.CPU91.RCU
86494 ± 4% -30.1% 60443 ± 2% softirqs.CPU91.TIMER
144968 ± 3% -23.8% 110502 softirqs.CPU92.RCU
86849 ± 6% -29.8% 60943 softirqs.CPU92.TIMER
144602 ± 2% -23.4% 110746 softirqs.CPU93.RCU
86758 ± 5% -30.0% 60736 softirqs.CPU93.TIMER
145455 ± 3% -23.5% 111300 softirqs.CPU94.RCU
87903 ± 6% -29.5% 61994 ± 3% softirqs.CPU94.TIMER
146051 ± 3% -24.1% 110816 softirqs.CPU95.RCU
88500 ± 5% -30.5% 61529 ± 3% softirqs.CPU95.TIMER
143359 ± 2% -23.6% 109510 softirqs.CPU96.RCU
81080 ± 6% -27.3% 58927 ± 2% softirqs.CPU96.TIMER
145624 ± 3% -24.0% 110725 ± 2% softirqs.CPU97.RCU
81898 ± 6% -27.5% 59400 ± 2% softirqs.CPU97.TIMER
145465 ± 4% -20.9% 115101 ± 2% softirqs.CPU98.RCU
83965 ± 6% -24.4% 63442 ± 7% softirqs.CPU98.TIMER
145576 ± 2% -23.5% 111417 ± 2% softirqs.CPU99.RCU
83350 ± 4% -27.1% 60735 ± 5% softirqs.CPU99.TIMER
27792781 ± 2% -23.2% 21353960 softirqs.RCU
16107152 ± 4% -28.3% 11555611 softirqs.TIMER
456.00 ± 3% -28.9% 324.00 interrupts.9:IO-APIC.9-fasteoi.acpi
766234 ± 8% -15.7% 646279 ± 14% interrupts.CAL:Function_call_interrupts
452803 ± 3% -29.3% 320233 interrupts.CPU0.LOC:Local_timer_interrupts
456.00 ± 3% -28.9% 324.00 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
453219 ± 3% -29.5% 319566 interrupts.CPU1.LOC:Local_timer_interrupts
3986 ± 7% -16.0% 3349 ± 13% interrupts.CPU10.CAL:Function_call_interrupts
453056 ± 3% -29.2% 320584 interrupts.CPU10.LOC:Local_timer_interrupts
1208 ± 3% -56.4% 527.50 interrupts.CPU10.RES:Rescheduling_interrupts
451576 ± 3% -29.0% 320461 interrupts.CPU100.LOC:Local_timer_interrupts
1227 -60.0% 490.50 ± 6% interrupts.CPU100.RES:Rescheduling_interrupts
452705 ± 3% -29.4% 319529 interrupts.CPU101.LOC:Local_timer_interrupts
1181 ± 2% -56.8% 510.00 ± 4% interrupts.CPU101.RES:Rescheduling_interrupts
451731 ± 3% -29.2% 319984 interrupts.CPU102.LOC:Local_timer_interrupts
1165 ± 5% -55.7% 516.75 ± 5% interrupts.CPU102.RES:Rescheduling_interrupts
451647 ± 3% -29.4% 318778 interrupts.CPU103.LOC:Local_timer_interrupts
1160 ± 3% -54.1% 532.25 ± 10% interrupts.CPU103.RES:Rescheduling_interrupts
449658 ± 3% -28.8% 320256 interrupts.CPU104.LOC:Local_timer_interrupts
1139 ± 3% -55.6% 506.00 ± 11% interrupts.CPU104.RES:Rescheduling_interrupts
450773 ± 3% -29.0% 319851 interrupts.CPU105.LOC:Local_timer_interrupts
1150 ± 4% -55.2% 515.75 ± 2% interrupts.CPU105.RES:Rescheduling_interrupts
452849 ± 3% -29.4% 319657 interrupts.CPU106.LOC:Local_timer_interrupts
1160 ± 4% -56.6% 503.25 ± 9% interrupts.CPU106.RES:Rescheduling_interrupts
450792 ± 3% -29.1% 319817 interrupts.CPU107.LOC:Local_timer_interrupts
1178 ± 2% -57.2% 504.00 ± 7% interrupts.CPU107.RES:Rescheduling_interrupts
451844 ± 3% -29.2% 319824 interrupts.CPU108.LOC:Local_timer_interrupts
1158 ± 3% -57.3% 494.75 ± 7% interrupts.CPU108.RES:Rescheduling_interrupts
453101 ± 3% -29.3% 320289 interrupts.CPU109.LOC:Local_timer_interrupts
1143 -54.8% 516.50 ± 9% interrupts.CPU109.RES:Rescheduling_interrupts
3993 ± 7% -16.9% 3318 ± 13% interrupts.CPU11.CAL:Function_call_interrupts
451333 ± 3% -29.1% 320076 interrupts.CPU11.LOC:Local_timer_interrupts
1239 ± 3% -56.3% 541.50 ± 8% interrupts.CPU11.RES:Rescheduling_interrupts
451872 ± 3% -29.1% 320242 interrupts.CPU110.LOC:Local_timer_interrupts
1199 ± 3% -56.5% 521.50 ± 9% interrupts.CPU110.RES:Rescheduling_interrupts
450972 ± 3% -29.1% 319690 interrupts.CPU111.LOC:Local_timer_interrupts
1164 ± 3% -57.7% 493.00 ± 8% interrupts.CPU111.RES:Rescheduling_interrupts
451615 ± 3% -29.5% 318582 interrupts.CPU112.LOC:Local_timer_interrupts
1212 ± 3% -57.3% 517.50 ± 3% interrupts.CPU112.RES:Rescheduling_interrupts
450605 ± 3% -29.1% 319563 interrupts.CPU113.LOC:Local_timer_interrupts
1232 -56.9% 531.75 ± 2% interrupts.CPU113.RES:Rescheduling_interrupts
450763 ± 3% -29.1% 319557 interrupts.CPU114.LOC:Local_timer_interrupts
1186 -56.1% 521.25 ± 7% interrupts.CPU114.RES:Rescheduling_interrupts
452258 ± 3% -29.2% 320416 interrupts.CPU115.LOC:Local_timer_interrupts
1228 ± 2% -55.2% 549.75 ± 10% interrupts.CPU115.RES:Rescheduling_interrupts
452821 ± 3% -29.2% 320511 interrupts.CPU116.LOC:Local_timer_interrupts
1190 ± 2% -55.7% 527.00 ± 7% interrupts.CPU116.RES:Rescheduling_interrupts
451871 ± 3% -29.1% 320276 interrupts.CPU117.LOC:Local_timer_interrupts
1193 ± 4% -54.8% 538.75 ± 8% interrupts.CPU117.RES:Rescheduling_interrupts
450690 ± 3% -29.0% 319863 interrupts.CPU118.LOC:Local_timer_interrupts
1230 -60.2% 490.00 ± 9% interrupts.CPU118.RES:Rescheduling_interrupts
451936 ± 3% -29.3% 319727 interrupts.CPU119.LOC:Local_timer_interrupts
1218 ± 4% -58.7% 503.25 ± 9% interrupts.CPU119.RES:Rescheduling_interrupts
452182 ± 3% -29.2% 320074 interrupts.CPU12.LOC:Local_timer_interrupts
1156 ± 4% -55.1% 518.75 ± 8% interrupts.CPU12.RES:Rescheduling_interrupts
451922 ± 3% -28.9% 321093 interrupts.CPU120.LOC:Local_timer_interrupts
1317 ± 7% -58.3% 549.75 ± 6% interrupts.CPU120.RES:Rescheduling_interrupts
452283 ± 3% -29.2% 320344 interrupts.CPU121.LOC:Local_timer_interrupts
1276 ± 5% -58.1% 534.50 ± 6% interrupts.CPU121.RES:Rescheduling_interrupts
453239 ± 3% -29.3% 320319 interrupts.CPU122.LOC:Local_timer_interrupts
1263 ± 10% -56.1% 555.00 ± 4% interrupts.CPU122.RES:Rescheduling_interrupts
452529 ± 3% -29.1% 321000 interrupts.CPU123.LOC:Local_timer_interrupts
1241 ± 11% -57.0% 533.75 ± 3% interrupts.CPU123.RES:Rescheduling_interrupts
452455 ± 3% -29.1% 320893 interrupts.CPU124.LOC:Local_timer_interrupts
1244 ± 6% -59.7% 501.50 ± 3% interrupts.CPU124.RES:Rescheduling_interrupts
452320 ± 3% -29.2% 320283 interrupts.CPU125.LOC:Local_timer_interrupts
1280 ± 6% -58.1% 536.50 ± 6% interrupts.CPU125.RES:Rescheduling_interrupts
453492 ± 3% -29.2% 321030 interrupts.CPU126.LOC:Local_timer_interrupts
1237 ± 12% -55.0% 557.50 ± 3% interrupts.CPU126.RES:Rescheduling_interrupts
452537 ± 3% -29.0% 321276 interrupts.CPU127.LOC:Local_timer_interrupts
1255 ± 9% -57.0% 539.75 ± 8% interrupts.CPU127.RES:Rescheduling_interrupts
450802 ± 3% -28.7% 321259 interrupts.CPU128.LOC:Local_timer_interrupts
1263 ± 8% -58.0% 530.25 ± 3% interrupts.CPU128.RES:Rescheduling_interrupts
452541 ± 3% -29.1% 320958 interrupts.CPU129.LOC:Local_timer_interrupts
1251 ± 9% -58.6% 518.50 ± 6% interrupts.CPU129.RES:Rescheduling_interrupts
3990 ± 7% -16.9% 3317 ± 14% interrupts.CPU13.CAL:Function_call_interrupts
451812 ± 3% -29.1% 320117 interrupts.CPU13.LOC:Local_timer_interrupts
1269 ± 10% -56.4% 553.00 ± 10% interrupts.CPU13.RES:Rescheduling_interrupts
452510 ± 3% -29.1% 320964 interrupts.CPU130.LOC:Local_timer_interrupts
1219 ± 8% -55.5% 543.25 ± 4% interrupts.CPU130.RES:Rescheduling_interrupts
451568 ± 3% -29.0% 320754 interrupts.CPU131.LOC:Local_timer_interrupts
1276 ± 10% -56.8% 551.75 ± 3% interrupts.CPU131.RES:Rescheduling_interrupts
451568 ± 3% -28.8% 321353 interrupts.CPU132.LOC:Local_timer_interrupts
1260 ± 10% -56.1% 552.75 ± 3% interrupts.CPU132.RES:Rescheduling_interrupts
450614 ± 3% -28.8% 320860 interrupts.CPU133.LOC:Local_timer_interrupts
1208 ± 8% -56.6% 524.50 ± 5% interrupts.CPU133.RES:Rescheduling_interrupts
452540 ± 3% -29.3% 319820 interrupts.CPU134.LOC:Local_timer_interrupts
1198 ± 9% -54.7% 543.50 ± 5% interrupts.CPU134.RES:Rescheduling_interrupts
451774 ± 3% -29.0% 320779 interrupts.CPU135.LOC:Local_timer_interrupts
1243 ± 9% -57.8% 524.25 ± 9% interrupts.CPU135.RES:Rescheduling_interrupts
451764 ± 3% -29.0% 320862 interrupts.CPU136.LOC:Local_timer_interrupts
1242 ± 11% -57.8% 524.00 ± 6% interrupts.CPU136.RES:Rescheduling_interrupts
452544 ± 3% -29.2% 320355 interrupts.CPU137.LOC:Local_timer_interrupts
1217 ± 8% -53.8% 561.75 ± 2% interrupts.CPU137.RES:Rescheduling_interrupts
452420 ± 3% -29.3% 319898 interrupts.CPU138.LOC:Local_timer_interrupts
1229 ± 10% -56.5% 535.25 ± 7% interrupts.CPU138.RES:Rescheduling_interrupts
3998 ± 8% -15.3% 3387 ± 14% interrupts.CPU139.CAL:Function_call_interrupts
451228 ± 3% -29.0% 320225 interrupts.CPU139.LOC:Local_timer_interrupts
1259 ± 8% -57.9% 530.00 ± 2% interrupts.CPU139.RES:Rescheduling_interrupts
3993 ± 7% -16.6% 3330 ± 14% interrupts.CPU14.CAL:Function_call_interrupts
452711 ± 3% -29.3% 320074 interrupts.CPU14.LOC:Local_timer_interrupts
1210 ± 3% -58.3% 505.25 ± 4% interrupts.CPU14.RES:Rescheduling_interrupts
450711 ± 3% -29.0% 320022 interrupts.CPU140.LOC:Local_timer_interrupts
1203 ± 10% -57.3% 513.25 ± 2% interrupts.CPU140.RES:Rescheduling_interrupts
451669 ± 3% -29.0% 320865 interrupts.CPU141.LOC:Local_timer_interrupts
1248 ± 10% -57.4% 531.25 ± 3% interrupts.CPU141.RES:Rescheduling_interrupts
452660 ± 3% -29.2% 320643 interrupts.CPU142.LOC:Local_timer_interrupts
1230 ± 9% -56.5% 535.75 ± 5% interrupts.CPU142.RES:Rescheduling_interrupts
451450 ± 3% -29.1% 319878 interrupts.CPU143.LOC:Local_timer_interrupts
1278 ± 11% -60.3% 508.25 ± 3% interrupts.CPU143.RES:Rescheduling_interrupts
452073 ± 3% -29.3% 319424 interrupts.CPU144.LOC:Local_timer_interrupts
1233 ± 3% -57.8% 520.50 ± 11% interrupts.CPU144.RES:Rescheduling_interrupts
452499 ± 3% -29.6% 318666 interrupts.CPU145.LOC:Local_timer_interrupts
1239 ± 5% -59.9% 496.25 ± 9% interrupts.CPU145.RES:Rescheduling_interrupts
452757 ± 3% -29.4% 319517 interrupts.CPU146.LOC:Local_timer_interrupts
1177 ± 6% -59.5% 477.50 ± 11% interrupts.CPU146.RES:Rescheduling_interrupts
4019 ± 8% -16.2% 3368 ± 15% interrupts.CPU147.CAL:Function_call_interrupts
453095 ± 3% -29.3% 320412 interrupts.CPU147.LOC:Local_timer_interrupts
1253 ± 3% -58.2% 523.50 ± 19% interrupts.CPU147.RES:Rescheduling_interrupts
452905 ± 3% -29.5% 319425 interrupts.CPU148.LOC:Local_timer_interrupts
1264 ± 10% -62.5% 474.00 ± 10% interrupts.CPU148.RES:Rescheduling_interrupts
451603 ± 3% -28.9% 320926 interrupts.CPU149.LOC:Local_timer_interrupts
1223 ± 5% -57.6% 519.00 ± 12% interrupts.CPU149.RES:Rescheduling_interrupts
3964 ± 7% -16.0% 3331 ± 14% interrupts.CPU15.CAL:Function_call_interrupts
450909 ± 3% -29.0% 320079 interrupts.CPU15.LOC:Local_timer_interrupts
1204 ± 4% -54.8% 544.75 ± 5% interrupts.CPU15.RES:Rescheduling_interrupts
451623 ± 3% -29.2% 319931 interrupts.CPU150.LOC:Local_timer_interrupts
1243 ± 4% -61.0% 485.00 ± 14% interrupts.CPU150.RES:Rescheduling_interrupts
4002 ± 8% -15.7% 3372 ± 14% interrupts.CPU151.CAL:Function_call_interrupts
451616 ± 3% -29.1% 320037 interrupts.CPU151.LOC:Local_timer_interrupts
1255 ± 10% -58.5% 520.25 ± 6% interrupts.CPU151.RES:Rescheduling_interrupts
452725 ± 3% -29.3% 319944 interrupts.CPU152.LOC:Local_timer_interrupts
1288 ± 6% -64.5% 457.00 ± 18% interrupts.CPU152.RES:Rescheduling_interrupts
451866 ± 3% -29.1% 320479 interrupts.CPU153.LOC:Local_timer_interrupts
1245 ± 5% -60.2% 496.00 ± 10% interrupts.CPU153.RES:Rescheduling_interrupts
451042 ± 3% -29.1% 320014 interrupts.CPU154.LOC:Local_timer_interrupts
1223 ± 10% -59.8% 491.25 ± 7% interrupts.CPU154.RES:Rescheduling_interrupts
4001 ± 7% -16.4% 3344 ± 14% interrupts.CPU155.CAL:Function_call_interrupts
452491 ± 3% -29.2% 320494 interrupts.CPU155.LOC:Local_timer_interrupts
1211 ± 4% -58.4% 503.75 ± 9% interrupts.CPU155.RES:Rescheduling_interrupts
451790 ± 3% -29.1% 320406 interrupts.CPU156.LOC:Local_timer_interrupts
1223 ± 7% -59.2% 498.75 ± 10% interrupts.CPU156.RES:Rescheduling_interrupts
451158 ± 3% -29.2% 319508 interrupts.CPU157.LOC:Local_timer_interrupts
1256 ± 4% -61.5% 484.00 ± 8% interrupts.CPU157.RES:Rescheduling_interrupts
451754 ± 3% -29.1% 320491 interrupts.CPU158.LOC:Local_timer_interrupts
1231 ± 8% -60.1% 491.25 ± 7% interrupts.CPU158.RES:Rescheduling_interrupts
451517 ± 3% -29.0% 320669 interrupts.CPU159.LOC:Local_timer_interrupts
1249 ± 4% -58.7% 515.50 ± 5% interrupts.CPU159.RES:Rescheduling_interrupts
452638 ± 3% -29.3% 320070 interrupts.CPU16.LOC:Local_timer_interrupts
1220 ± 2% -56.4% 532.00 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts
450384 ± 3% -28.9% 320033 interrupts.CPU160.LOC:Local_timer_interrupts
1227 ± 7% -59.5% 496.75 ± 6% interrupts.CPU160.RES:Rescheduling_interrupts
449410 ± 3% -28.9% 319687 interrupts.CPU161.LOC:Local_timer_interrupts
1299 ± 6% -59.9% 520.25 ± 13% interrupts.CPU161.RES:Rescheduling_interrupts
451627 ± 3% -29.4% 318895 interrupts.CPU162.LOC:Local_timer_interrupts
1228 ± 5% -60.2% 488.75 ± 9% interrupts.CPU162.RES:Rescheduling_interrupts
452345 ± 3% -29.3% 319815 interrupts.CPU163.LOC:Local_timer_interrupts
1229 ± 3% -59.8% 493.75 ± 10% interrupts.CPU163.RES:Rescheduling_interrupts
451073 ± 3% -29.0% 320277 interrupts.CPU164.LOC:Local_timer_interrupts
1269 ± 7% -59.3% 516.50 ± 10% interrupts.CPU164.RES:Rescheduling_interrupts
451353 ± 3% -29.3% 318934 interrupts.CPU165.LOC:Local_timer_interrupts
1194 ± 4% -53.5% 555.75 ± 11% interrupts.CPU165.RES:Rescheduling_interrupts
451335 ± 3% -28.9% 320880 interrupts.CPU166.LOC:Local_timer_interrupts
1259 ± 7% -61.0% 491.25 ± 5% interrupts.CPU166.RES:Rescheduling_interrupts
449763 ± 3% -28.5% 321454 interrupts.CPU167.LOC:Local_timer_interrupts
1280 ± 7% -60.3% 508.50 ± 5% interrupts.CPU167.RES:Rescheduling_interrupts
452805 ± 3% -29.2% 320695 interrupts.CPU168.LOC:Local_timer_interrupts
1270 ± 7% -57.5% 540.50 ± 6% interrupts.CPU168.RES:Rescheduling_interrupts
451812 ± 3% -29.1% 320547 interrupts.CPU169.LOC:Local_timer_interrupts
1252 ± 6% -55.2% 561.50 ± 10% interrupts.CPU169.RES:Rescheduling_interrupts
450954 ± 3% -29.1% 319575 interrupts.CPU17.LOC:Local_timer_interrupts
1233 ± 3% -54.6% 559.75 ± 3% interrupts.CPU17.RES:Rescheduling_interrupts
452765 ± 3% -29.1% 320816 interrupts.CPU170.LOC:Local_timer_interrupts
1211 ± 6% -56.3% 529.50 ± 4% interrupts.CPU170.RES:Rescheduling_interrupts
452036 ± 3% -29.1% 320711 interrupts.CPU171.LOC:Local_timer_interrupts
1224 ± 7% -56.0% 538.25 ± 4% interrupts.CPU171.RES:Rescheduling_interrupts
452830 ± 3% -29.2% 320825 interrupts.CPU172.LOC:Local_timer_interrupts
1237 ± 8% -57.5% 525.75 ± 11% interrupts.CPU172.RES:Rescheduling_interrupts
451604 ± 3% -29.1% 320045 interrupts.CPU173.LOC:Local_timer_interrupts
1250 ± 4% -55.5% 556.25 ± 5% interrupts.CPU173.RES:Rescheduling_interrupts
450654 ± 3% -28.8% 320649 interrupts.CPU174.LOC:Local_timer_interrupts
1269 ± 9% -55.9% 560.25 ± 3% interrupts.CPU174.RES:Rescheduling_interrupts
451650 ± 3% -29.2% 319874 interrupts.CPU175.LOC:Local_timer_interrupts
1231 ± 3% -54.9% 555.25 ± 3% interrupts.CPU175.RES:Rescheduling_interrupts
450952 ± 3% -28.9% 320762 interrupts.CPU176.LOC:Local_timer_interrupts
1269 ± 4% -57.6% 537.75 ± 6% interrupts.CPU176.RES:Rescheduling_interrupts
452398 ± 3% -29.2% 320245 interrupts.CPU177.LOC:Local_timer_interrupts
1247 ± 4% -56.9% 537.50 ± 6% interrupts.CPU177.RES:Rescheduling_interrupts
451527 ± 3% -29.1% 320251 interrupts.CPU178.LOC:Local_timer_interrupts
1214 ± 7% -55.5% 540.25 ± 9% interrupts.CPU178.RES:Rescheduling_interrupts
4032 ± 6% -16.1% 3381 ± 14% interrupts.CPU179.CAL:Function_call_interrupts
452598 ± 3% -29.2% 320544 interrupts.CPU179.LOC:Local_timer_interrupts
1224 ± 8% -53.8% 566.00 ± 4% interrupts.CPU179.RES:Rescheduling_interrupts
449159 ± 3% -28.7% 320143 interrupts.CPU18.LOC:Local_timer_interrupts
1156 ± 4% -55.6% 513.50 ± 5% interrupts.CPU18.RES:Rescheduling_interrupts
453035 ± 3% -29.3% 320145 interrupts.CPU180.LOC:Local_timer_interrupts
1201 ± 5% -55.8% 531.50 ± 6% interrupts.CPU180.RES:Rescheduling_interrupts
4029 ± 8% -15.9% 3389 ± 15% interrupts.CPU181.CAL:Function_call_interrupts
451550 ± 3% -28.9% 320853 interrupts.CPU181.LOC:Local_timer_interrupts
1227 ± 5% -56.2% 537.75 interrupts.CPU181.RES:Rescheduling_interrupts
4046 ± 8% -15.7% 3410 ± 15% interrupts.CPU182.CAL:Function_call_interrupts
451619 ± 3% -29.1% 319989 interrupts.CPU182.LOC:Local_timer_interrupts
1206 ± 6% -53.8% 557.75 ± 4% interrupts.CPU182.RES:Rescheduling_interrupts
452472 ± 3% -29.3% 320059 interrupts.CPU183.LOC:Local_timer_interrupts
1211 ± 8% -57.4% 516.00 ± 6% interrupts.CPU183.RES:Rescheduling_interrupts
4032 ± 8% -15.8% 3393 ± 13% interrupts.CPU184.CAL:Function_call_interrupts
451657 ± 3% -29.1% 320432 interrupts.CPU184.LOC:Local_timer_interrupts
1181 ± 4% -53.1% 553.75 ± 3% interrupts.CPU184.RES:Rescheduling_interrupts
451618 ± 3% -29.1% 320062 interrupts.CPU185.LOC:Local_timer_interrupts
1205 ± 7% -53.9% 555.75 ± 6% interrupts.CPU185.RES:Rescheduling_interrupts
4044 ± 8% -16.7% 3370 ± 14% interrupts.CPU186.CAL:Function_call_interrupts
450125 ± 3% -28.9% 320072 interrupts.CPU186.LOC:Local_timer_interrupts
1200 ± 7% -56.1% 526.75 ± 3% interrupts.CPU186.RES:Rescheduling_interrupts
4042 ± 8% -15.8% 3403 ± 14% interrupts.CPU187.CAL:Function_call_interrupts
450734 ± 3% -28.8% 320793 interrupts.CPU187.LOC:Local_timer_interrupts
1234 ± 5% -55.7% 547.00 ± 6% interrupts.CPU187.RES:Rescheduling_interrupts
4037 ± 8% -16.2% 3384 ± 16% interrupts.CPU188.CAL:Function_call_interrupts
451745 ± 3% -29.0% 320669 interrupts.CPU188.LOC:Local_timer_interrupts
1200 ± 8% -56.1% 527.00 ± 4% interrupts.CPU188.RES:Rescheduling_interrupts
450992 ± 3% -28.9% 320635 interrupts.CPU189.LOC:Local_timer_interrupts
1211 ± 9% -55.3% 541.00 ± 7% interrupts.CPU189.RES:Rescheduling_interrupts
451859 ± 3% -29.2% 320034 interrupts.CPU19.LOC:Local_timer_interrupts
1204 ± 3% -54.8% 544.50 ± 5% interrupts.CPU19.RES:Rescheduling_interrupts
4049 ± 8% -15.0% 3443 ± 14% interrupts.CPU190.CAL:Function_call_interrupts
451733 ± 3% -29.1% 320182 interrupts.CPU190.LOC:Local_timer_interrupts
1198 ± 5% -57.4% 510.75 ± 6% interrupts.CPU190.RES:Rescheduling_interrupts
4019 ± 8% -16.8% 3343 ± 13% interrupts.CPU191.CAL:Function_call_interrupts
450960 ± 3% -28.9% 320576 interrupts.CPU191.LOC:Local_timer_interrupts
1216 ± 8% -53.7% 562.50 ± 8% interrupts.CPU191.RES:Rescheduling_interrupts
452806 ± 3% -29.4% 319745 interrupts.CPU2.LOC:Local_timer_interrupts
1206 ± 3% -54.9% 544.25 ± 6% interrupts.CPU2.RES:Rescheduling_interrupts
452871 ± 3% -29.5% 319377 interrupts.CPU20.LOC:Local_timer_interrupts
1240 ± 3% -56.0% 545.75 ± 8% interrupts.CPU20.RES:Rescheduling_interrupts
451600 ± 3% -29.0% 320698 interrupts.CPU21.LOC:Local_timer_interrupts
1170 ± 5% -43.1% 666.25 ± 39% interrupts.CPU21.RES:Rescheduling_interrupts
450939 ± 3% -29.0% 320327 interrupts.CPU22.LOC:Local_timer_interrupts
1239 ± 3% -55.4% 552.25 ± 10% interrupts.CPU22.RES:Rescheduling_interrupts
451819 ± 3% -29.2% 319818 interrupts.CPU23.LOC:Local_timer_interrupts
1216 ± 2% -56.1% 534.25 ± 7% interrupts.CPU23.RES:Rescheduling_interrupts
452031 ± 3% -29.1% 320472 interrupts.CPU24.LOC:Local_timer_interrupts
1736 ± 30% -55.7% 769.25 ± 10% interrupts.CPU24.RES:Rescheduling_interrupts
451844 ± 3% -28.9% 321154 interrupts.CPU25.LOC:Local_timer_interrupts
1277 ± 8% -54.2% 585.25 ± 7% interrupts.CPU25.RES:Rescheduling_interrupts
453247 ± 3% -29.4% 320029 interrupts.CPU26.LOC:Local_timer_interrupts
1286 ± 8% -56.1% 565.00 interrupts.CPU26.RES:Rescheduling_interrupts
452266 ± 3% -29.1% 320864 interrupts.CPU27.LOC:Local_timer_interrupts
1267 ± 12% -56.5% 551.50 ± 5% interrupts.CPU27.RES:Rescheduling_interrupts
452656 ± 3% -29.3% 319990 interrupts.CPU28.LOC:Local_timer_interrupts
1295 ± 9% -56.8% 560.00 interrupts.CPU28.RES:Rescheduling_interrupts
451745 ± 3% -29.0% 320762 interrupts.CPU29.LOC:Local_timer_interrupts
1288 ± 7% -56.9% 555.75 ± 3% interrupts.CPU29.RES:Rescheduling_interrupts
450582 ± 3% -28.9% 320203 interrupts.CPU3.LOC:Local_timer_interrupts
1454 ± 18% -62.9% 539.00 ± 9% interrupts.CPU3.RES:Rescheduling_interrupts
452851 ± 3% -29.4% 319633 interrupts.CPU30.LOC:Local_timer_interrupts
1309 ± 7% -54.9% 589.75 ± 8% interrupts.CPU30.RES:Rescheduling_interrupts
452719 ± 3% -29.6% 318935 interrupts.CPU31.LOC:Local_timer_interrupts
1254 ± 12% -45.2% 687.75 ± 31% interrupts.CPU31.RES:Rescheduling_interrupts
452588 ± 3% -29.1% 320666 interrupts.CPU32.LOC:Local_timer_interrupts
1267 ± 11% -56.0% 557.75 ± 2% interrupts.CPU32.RES:Rescheduling_interrupts
450718 ± 3% -28.9% 320675 interrupts.CPU33.LOC:Local_timer_interrupts
1265 ± 8% -56.6% 549.75 ± 8% interrupts.CPU33.RES:Rescheduling_interrupts
451592 ± 3% -29.2% 319802 interrupts.CPU34.LOC:Local_timer_interrupts
1210 ± 12% -53.1% 568.25 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts
451616 ± 3% -28.9% 321232 interrupts.CPU35.LOC:Local_timer_interrupts
8638 -50.5% 4272 interrupts.CPU35.NMI:Non-maskable_interrupts
8638 -50.5% 4272 interrupts.CPU35.PMI:Performance_monitoring_interrupts
1247 ± 8% -57.2% 533.50 ± 7% interrupts.CPU35.RES:Rescheduling_interrupts
450805 ± 3% -29.0% 320131 interrupts.CPU36.LOC:Local_timer_interrupts
1242 ± 10% -57.6% 527.00 ± 5% interrupts.CPU36.RES:Rescheduling_interrupts
452558 ± 3% -29.0% 321267 interrupts.CPU37.LOC:Local_timer_interrupts
1202 ± 8% -54.1% 551.50 ± 5% interrupts.CPU37.RES:Rescheduling_interrupts
451740 ± 3% -29.1% 320351 interrupts.CPU38.LOC:Local_timer_interrupts
1275 ± 12% -57.1% 547.00 ± 6% interrupts.CPU38.RES:Rescheduling_interrupts
451568 ± 3% -29.0% 320731 interrupts.CPU39.LOC:Local_timer_interrupts
1251 ± 9% -57.5% 532.50 ± 8% interrupts.CPU39.RES:Rescheduling_interrupts
452624 ± 3% -29.3% 319788 interrupts.CPU4.LOC:Local_timer_interrupts
1310 ± 13% -49.1% 666.50 ± 33% interrupts.CPU4.RES:Rescheduling_interrupts
452569 ± 3% -29.0% 321282 interrupts.CPU40.LOC:Local_timer_interrupts
1267 ± 9% -58.2% 530.25 ± 8% interrupts.CPU40.RES:Rescheduling_interrupts
452692 ± 3% -29.1% 320863 interrupts.CPU41.LOC:Local_timer_interrupts
1243 ± 6% -58.2% 519.50 ± 8% interrupts.CPU41.RES:Rescheduling_interrupts
452080 ± 3% -29.0% 320754 interrupts.CPU42.LOC:Local_timer_interrupts
1260 ± 10% -57.4% 536.25 ± 9% interrupts.CPU42.RES:Rescheduling_interrupts
451798 ± 3% -28.9% 321215 interrupts.CPU43.LOC:Local_timer_interrupts
1234 ± 11% -56.5% 536.50 ± 5% interrupts.CPU43.RES:Rescheduling_interrupts
451698 ± 3% -29.0% 320858 interrupts.CPU44.LOC:Local_timer_interrupts
1242 ± 7% -56.8% 536.25 ± 2% interrupts.CPU44.RES:Rescheduling_interrupts
451499 ± 3% -29.0% 320649 interrupts.CPU45.LOC:Local_timer_interrupts
1268 ± 11% -56.7% 549.50 ± 3% interrupts.CPU45.RES:Rescheduling_interrupts
452051 ± 3% -29.0% 321012 interrupts.CPU46.LOC:Local_timer_interrupts
1284 ± 9% -57.4% 547.50 ± 3% interrupts.CPU46.RES:Rescheduling_interrupts
451902 ± 3% -29.1% 320365 interrupts.CPU47.LOC:Local_timer_interrupts
1290 ± 7% -57.4% 549.25 ± 5% interrupts.CPU47.RES:Rescheduling_interrupts
451631 ± 3% -28.9% 320954 interrupts.CPU48.LOC:Local_timer_interrupts
1557 ± 7% -59.7% 627.75 ± 23% interrupts.CPU48.RES:Rescheduling_interrupts
451549 ± 3% -29.0% 320751 interrupts.CPU49.LOC:Local_timer_interrupts
1281 ± 7% -58.5% 532.25 ± 15% interrupts.CPU49.RES:Rescheduling_interrupts
453101 ± 3% -29.4% 319817 interrupts.CPU5.LOC:Local_timer_interrupts
1336 ± 19% -61.6% 513.00 ± 10% interrupts.CPU5.RES:Rescheduling_interrupts
3950 ± 8% -16.2% 3309 ± 14% interrupts.CPU50.CAL:Function_call_interrupts
452101 ± 3% -29.0% 321131 interrupts.CPU50.LOC:Local_timer_interrupts
1271 ± 5% -57.7% 537.75 ± 13% interrupts.CPU50.RES:Rescheduling_interrupts
451517 ± 3% -28.9% 320886 interrupts.CPU51.LOC:Local_timer_interrupts
1286 ± 7% -61.1% 500.50 ± 11% interrupts.CPU51.RES:Rescheduling_interrupts
451335 ± 3% -29.2% 319430 interrupts.CPU52.LOC:Local_timer_interrupts
1253 ± 7% -60.1% 500.25 ± 11% interrupts.CPU52.RES:Rescheduling_interrupts
3937 ± 9% -17.0% 3269 ± 15% interrupts.CPU53.CAL:Function_call_interrupts
451178 ± 3% -28.9% 320907 interrupts.CPU53.LOC:Local_timer_interrupts
1253 ± 5% -61.1% 487.25 ± 9% interrupts.CPU53.RES:Rescheduling_interrupts
451332 ± 3% -29.0% 320283 interrupts.CPU54.LOC:Local_timer_interrupts
1257 ± 4% -58.4% 523.00 ± 13% interrupts.CPU54.RES:Rescheduling_interrupts
450908 ± 3% -29.0% 320274 interrupts.CPU55.LOC:Local_timer_interrupts
1247 ± 8% -59.0% 511.50 ± 11% interrupts.CPU55.RES:Rescheduling_interrupts
450386 ± 3% -28.9% 320379 interrupts.CPU56.LOC:Local_timer_interrupts
1248 ± 5% -61.0% 486.25 ± 10% interrupts.CPU56.RES:Rescheduling_interrupts
451338 ± 3% -28.9% 320735 interrupts.CPU57.LOC:Local_timer_interrupts
1298 ± 9% -60.5% 512.75 ± 8% interrupts.CPU57.RES:Rescheduling_interrupts
451270 ± 3% -28.9% 320866 interrupts.CPU58.LOC:Local_timer_interrupts
1246 ± 7% -60.0% 498.75 ± 10% interrupts.CPU58.RES:Rescheduling_interrupts
3968 ± 8% -16.7% 3304 ± 14% interrupts.CPU59.CAL:Function_call_interrupts
450285 ± 3% -28.7% 320885 interrupts.CPU59.LOC:Local_timer_interrupts
1226 ± 7% -59.7% 494.50 ± 7% interrupts.CPU59.RES:Rescheduling_interrupts
451970 ± 3% -29.2% 320106 interrupts.CPU6.LOC:Local_timer_interrupts
1178 ± 4% -54.3% 539.00 ± 6% interrupts.CPU6.RES:Rescheduling_interrupts
452638 ± 3% -29.1% 320755 interrupts.CPU60.LOC:Local_timer_interrupts
1246 ± 2% -62.9% 462.00 ± 10% interrupts.CPU60.RES:Rescheduling_interrupts
450454 ± 3% -28.7% 320996 interrupts.CPU61.LOC:Local_timer_interrupts
1256 ± 9% -61.1% 488.50 ± 11% interrupts.CPU61.RES:Rescheduling_interrupts
449233 ± 3% -28.8% 320066 interrupts.CPU62.LOC:Local_timer_interrupts
1260 ± 6% -61.5% 485.25 ± 10% interrupts.CPU62.RES:Rescheduling_interrupts
452202 ± 3% -29.0% 321015 interrupts.CPU63.LOC:Local_timer_interrupts
1237 ± 8% -58.1% 518.25 ± 8% interrupts.CPU63.RES:Rescheduling_interrupts
450644 ± 3% -29.1% 319320 interrupts.CPU64.LOC:Local_timer_interrupts
7559 ± 24% -30.4% 5263 ± 33% interrupts.CPU64.NMI:Non-maskable_interrupts
7559 ± 24% -30.4% 5263 ± 33% interrupts.CPU64.PMI:Performance_monitoring_interrupts
1272 ± 5% -60.0% 508.75 ± 10% interrupts.CPU64.RES:Rescheduling_interrupts
452334 ± 3% -29.2% 320162 interrupts.CPU65.LOC:Local_timer_interrupts
1271 ± 6% -59.8% 511.50 ± 13% interrupts.CPU65.RES:Rescheduling_interrupts
451504 ± 3% -28.9% 321018 interrupts.CPU66.LOC:Local_timer_interrupts
1271 ± 9% -59.8% 511.25 ± 10% interrupts.CPU66.RES:Rescheduling_interrupts
450384 ± 3% -28.9% 320015 interrupts.CPU67.LOC:Local_timer_interrupts
6478 ± 33% -18.3% 5291 ± 34% interrupts.CPU67.NMI:Non-maskable_interrupts
6478 ± 33% -18.3% 5291 ± 34% interrupts.CPU67.PMI:Performance_monitoring_interrupts
1255 ± 5% -58.2% 524.25 ± 10% interrupts.CPU67.RES:Rescheduling_interrupts
451257 ± 3% -29.2% 319513 interrupts.CPU68.LOC:Local_timer_interrupts
1253 ± 8% -59.4% 508.75 ± 17% interrupts.CPU68.RES:Rescheduling_interrupts
450403 ± 3% -28.9% 320425 interrupts.CPU69.LOC:Local_timer_interrupts
1271 ± 6% -58.2% 531.25 ± 17% interrupts.CPU69.RES:Rescheduling_interrupts
452323 ± 3% -29.0% 321210 interrupts.CPU7.LOC:Local_timer_interrupts
1253 ± 2% -57.7% 530.75 ± 6% interrupts.CPU7.RES:Rescheduling_interrupts
451237 ± 3% -29.0% 320186 interrupts.CPU70.LOC:Local_timer_interrupts
1260 ± 8% -47.9% 656.50 ± 29% interrupts.CPU70.RES:Rescheduling_interrupts
450774 ± 3% -29.2% 319134 interrupts.CPU71.LOC:Local_timer_interrupts
1232 ± 6% -57.6% 522.25 ± 9% interrupts.CPU71.RES:Rescheduling_interrupts
451796 ± 3% -29.1% 320386 interrupts.CPU72.LOC:Local_timer_interrupts
1565 ± 14% -46.1% 843.50 ± 13% interrupts.CPU72.RES:Rescheduling_interrupts
450827 ± 3% -29.0% 319884 interrupts.CPU73.LOC:Local_timer_interrupts
1275 ± 3% -50.2% 635.50 ± 9% interrupts.CPU73.RES:Rescheduling_interrupts
453097 ± 3% -29.3% 320261 interrupts.CPU74.LOC:Local_timer_interrupts
1269 ± 3% -56.5% 552.25 ± 4% interrupts.CPU74.RES:Rescheduling_interrupts
452368 ± 3% -29.2% 320270 interrupts.CPU75.LOC:Local_timer_interrupts
1205 ± 6% -53.3% 563.25 ± 7% interrupts.CPU75.RES:Rescheduling_interrupts
452333 ± 3% -29.2% 320261 interrupts.CPU76.LOC:Local_timer_interrupts
1225 ± 8% -54.6% 556.25 interrupts.CPU76.RES:Rescheduling_interrupts
451718 ± 3% -29.2% 319867 interrupts.CPU77.LOC:Local_timer_interrupts
1231 ± 4% -51.9% 592.75 ± 4% interrupts.CPU77.RES:Rescheduling_interrupts
452388 ± 3% -29.2% 320265 interrupts.CPU78.LOC:Local_timer_interrupts
1271 ± 6% -56.7% 550.50 ± 4% interrupts.CPU78.RES:Rescheduling_interrupts
452138 ± 3% -29.0% 321185 interrupts.CPU79.LOC:Local_timer_interrupts
1208 ± 5% -54.3% 551.75 ± 6% interrupts.CPU79.RES:Rescheduling_interrupts
3957 ± 6% -15.7% 3337 ± 14% interrupts.CPU8.CAL:Function_call_interrupts
453208 ± 3% -29.0% 321694 interrupts.CPU8.LOC:Local_timer_interrupts
1201 ± 2% -55.0% 540.75 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts
4057 ± 7% -17.0% 3369 ± 16% interrupts.CPU80.CAL:Function_call_interrupts
452808 ± 3% -29.2% 320807 interrupts.CPU80.LOC:Local_timer_interrupts
1259 ± 6% -55.5% 560.25 ± 9% interrupts.CPU80.RES:Rescheduling_interrupts
4019 ± 7% -15.7% 3388 ± 14% interrupts.CPU81.CAL:Function_call_interrupts
452385 ± 3% -29.0% 321252 interrupts.CPU81.LOC:Local_timer_interrupts
1209 ± 3% -51.2% 590.25 ± 9% interrupts.CPU81.RES:Rescheduling_interrupts
4014 ± 7% -16.0% 3373 ± 14% interrupts.CPU82.CAL:Function_call_interrupts
452385 ± 3% -29.0% 320984 interrupts.CPU82.LOC:Local_timer_interrupts
1218 ± 4% -52.5% 579.25 ± 6% interrupts.CPU82.RES:Rescheduling_interrupts
3991 ± 7% -15.6% 3367 ± 14% interrupts.CPU83.CAL:Function_call_interrupts
451641 ± 3% -28.9% 321045 interrupts.CPU83.LOC:Local_timer_interrupts
1218 ± 6% -54.5% 554.00 ± 4% interrupts.CPU83.RES:Rescheduling_interrupts
3989 ± 7% -16.1% 3348 ± 13% interrupts.CPU84.CAL:Function_call_interrupts
451421 ± 3% -28.9% 320942 interrupts.CPU84.LOC:Local_timer_interrupts
1218 ± 7% -55.7% 539.25 ± 5% interrupts.CPU84.RES:Rescheduling_interrupts
452343 ± 3% -29.1% 320664 interrupts.CPU85.LOC:Local_timer_interrupts
1228 ± 8% -55.1% 552.00 ± 8% interrupts.CPU85.RES:Rescheduling_interrupts
4013 ± 7% -15.4% 3396 ± 13% interrupts.CPU86.CAL:Function_call_interrupts
452435 ± 3% -29.4% 319327 interrupts.CPU86.LOC:Local_timer_interrupts
1219 ± 4% -56.2% 534.25 ± 3% interrupts.CPU86.RES:Rescheduling_interrupts
4004 ± 7% -16.0% 3363 ± 14% interrupts.CPU87.CAL:Function_call_interrupts
452402 ± 3% -29.1% 320973 interrupts.CPU87.LOC:Local_timer_interrupts
1257 ± 6% -56.1% 552.00 ± 6% interrupts.CPU87.RES:Rescheduling_interrupts
4007 ± 7% -15.7% 3380 ± 13% interrupts.CPU88.CAL:Function_call_interrupts
452862 ± 3% -29.2% 320660 interrupts.CPU88.LOC:Local_timer_interrupts
1195 ± 5% -51.6% 578.75 ± 8% interrupts.CPU88.RES:Rescheduling_interrupts
452593 ± 3% -29.1% 320937 interrupts.CPU89.LOC:Local_timer_interrupts
1252 ± 6% -54.8% 566.50 ± 2% interrupts.CPU89.RES:Rescheduling_interrupts
452336 ± 3% -29.0% 321162 interrupts.CPU9.LOC:Local_timer_interrupts
1209 ± 3% -56.1% 530.25 ± 10% interrupts.CPU9.RES:Rescheduling_interrupts
452508 ± 3% -29.2% 320382 interrupts.CPU90.LOC:Local_timer_interrupts
1247 ± 6% -55.6% 553.75 interrupts.CPU90.RES:Rescheduling_interrupts
4020 ± 7% -15.4% 3400 ± 15% interrupts.CPU91.CAL:Function_call_interrupts
451670 ± 3% -29.2% 319988 interrupts.CPU91.LOC:Local_timer_interrupts
7569 ± 24% -29.7% 5320 ± 35% interrupts.CPU91.NMI:Non-maskable_interrupts
7569 ± 24% -29.7% 5320 ± 35% interrupts.CPU91.PMI:Performance_monitoring_interrupts
1245 ± 9% -58.2% 520.25 ± 5% interrupts.CPU91.RES:Rescheduling_interrupts
4018 ± 7% -16.8% 3342 ± 15% interrupts.CPU92.CAL:Function_call_interrupts
452080 ± 3% -29.0% 320782 interrupts.CPU92.LOC:Local_timer_interrupts
1203 ± 8% -55.0% 542.00 ± 5% interrupts.CPU92.RES:Rescheduling_interrupts
452511 ± 3% -29.2% 320184 interrupts.CPU93.LOC:Local_timer_interrupts
1247 ± 6% -55.9% 549.75 ± 3% interrupts.CPU93.RES:Rescheduling_interrupts
3990 ± 7% -15.8% 3361 ± 15% interrupts.CPU94.CAL:Function_call_interrupts
452343 ± 3% -29.1% 320501 interrupts.CPU94.LOC:Local_timer_interrupts
1202 ± 6% -54.2% 551.25 ± 8% interrupts.CPU94.RES:Rescheduling_interrupts
4007 ± 7% -16.7% 3337 ± 14% interrupts.CPU95.CAL:Function_call_interrupts
452589 ± 3% -29.0% 321186 interrupts.CPU95.LOC:Local_timer_interrupts
1624 ± 14% -50.3% 807.25 ± 28% interrupts.CPU95.RES:Rescheduling_interrupts
450410 ± 3% -29.1% 319460 interrupts.CPU96.LOC:Local_timer_interrupts
1168 ± 4% -53.6% 541.50 ± 9% interrupts.CPU96.RES:Rescheduling_interrupts
452859 ± 3% -29.3% 320256 interrupts.CPU97.LOC:Local_timer_interrupts
1180 ± 5% -57.0% 507.75 ± 7% interrupts.CPU97.RES:Rescheduling_interrupts
451879 ± 3% -29.1% 320200 interrupts.CPU98.LOC:Local_timer_interrupts
1221 ± 2% -58.4% 507.75 ± 9% interrupts.CPU98.RES:Rescheduling_interrupts
453217 ± 3% -29.6% 318991 interrupts.CPU99.LOC:Local_timer_interrupts
1193 ± 4% -58.6% 494.75 ± 8% interrupts.CPU99.RES:Rescheduling_interrupts
86750516 ± 3% -29.1% 61503668 interrupts.LOC:Local_timer_interrupts
240586 ± 5% -56.1% 105580 ± 3% interrupts.RES:Rescheduling_interrupts
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
10 months
- 1
- 0
[blk] 78acda3832: kmsg.blk_update_request:critical_target_error,dev_sr#,sector#op#:(READ)flags#phys_seg#prio_class
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 78acda3832d1324cd7fd726d8dc5b8d95727a60e ("blk-map: add kernel address validation in blk_rq_map_kern func")
https://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git for-5.6/block-test
in testcase: blktests
with following parameters:
disk: 1SSD
test: block-group1
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 42.716630] blk_update_request: critical target error, dev sr1, sector 2097136 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[ 42.721177] sr 3:0:0:0: [sr1] tag#30 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=0s
[ 42.721180] sr 3:0:0:0: [sr1] tag#30 Sense Key : Illegal Request [current]
[ 42.721181] sr 3:0:0:0: [sr1] tag#30 Add. Sense: Logical block address out of range
[ 42.721183] sr 3:0:0:0: [sr1] tag#30 CDB: Read(10) 28 00 00 07 ff fc 00 00 01 00
[ 42.721184] blk_update_request: critical target error, dev sr1, sector 2097136 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 42.721194] sr 3:0:0:0: [sr1] tag#31 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=0s
[ 42.721196] sr 3:0:0:0: [sr1] tag#31 Sense Key : Illegal Request [current]
[ 42.721197] sr 3:0:0:0: [sr1] tag#31 Add. Sense: Logical block address out of range
[ 42.721198] sr 3:0:0:0: [sr1] tag#31 CDB: Read(10) 28 00 00 07 ff fd 00 00 01 00
[ 42.721199] blk_update_request: critical target error, dev sr1, sector 2097140 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 42.722276] sr 3:0:0:0: [sr1] tag#1 unaligned transfer
[ 42.722284] blk_update_request: I/O error, dev sr1, sector 2 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[ 42.726003] sr 3:0:0:0: [sr1] tag#4 unaligned transfer
[ 42.726005] blk_update_request: I/O error, dev sr1, sector 1 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 42.726008] sr 3:0:0:0: [sr1] tag#5 unaligned transfer
[ 42.726010] blk_update_request: I/O error, dev sr1, sector 5 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
To reproduce:
# build kernel
cd linux
cp config-5.5.0-rc2-00192-g78acda3832d13 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
10 months
- 1
- 0
[timer] 5acb72bc56: WARNING:at_kernel/time/timer.c:#schedule_timeout
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 5acb72bc5642be30e358f43eaa010d06a6ea29e7 ("timer: Warn about schedule_timeout() called for tasks in TASK_RUNNING state")
https://github.com/0day-ci/linux/commits/Alexander-Popov/timer-Warn-about...
in testcase: blktests
with following parameters:
disk: 1SSD
test: block-group1
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+--------------------------------------------------+------------+------------+
| | a67ca893ae | 5acb72bc56 |
+--------------------------------------------------+------------+------------+
| boot_successes | 9 | 3 |
| boot_failures | 0 | 9 |
| WARNING:at_kernel/time/timer.c:#schedule_timeout | 0 | 9 |
| RIP:schedule_timeout | 0 | 9 |
+--------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 9.394278] WARNING: CPU: 1 PID: 1 at kernel/time/timer.c:1892 schedule_timeout+0x254/0x340
[ 9.395601] Modules linked in: ip_tables
[ 9.396169] CPU: 1 PID: 1 Comm: systemd Not tainted 5.5.0-rc5-00071-g5acb72bc5642b #1
[ 9.397240] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 9.398362] RIP: 0010:schedule_timeout+0x254/0x340
[ 9.399021] Code: 08 00 41 89 c6 e9 a4 fe ff ff 80 3d 41 7a d4 00 00 0f 85 15 fe ff ff 48 c7 c7 28 99 f3 8c c6 05 2d 7a d4 00 01 e8 9c 81 5e ff <0f> 0b e9 fb fd ff ff 4d 89 ec e9 ec fe ff ff e8 38 b9 61 ff 48 c7
[ 9.400914] RSP: 0018:ffffa8fc80013d38 EFLAGS: 00010282
[ 9.401492] RAX: 0000000000000000 RBX: 000000000000752f RCX: 0000000000000000
[ 9.402222] RDX: 0000000000000001 RSI: ffff96be3fd19b38 RDI: ffff96be3fd19b38
[ 9.402956] RBP: ffff96bdd0f58000 R08: 0000000000000258 R09: 0000000000000030
[ 9.403687] R10: 000000000000077b R11: ffffa8fc80013bd8 R12: 0000000000000000
[ 9.404421] R13: ffff96bdd0f58208 R14: ffff96be3fd38a20 R15: 0000000000000001
[ 9.405163] FS: 00007f1abe5ec680(0000) GS:ffff96be3fd00000(0000) knlGS:0000000000000000
[ 9.406057] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9.406672] CR2: 0000563498ad1294 CR3: 00000001d0d52000 CR4: 00000000000406e0
[ 9.407404] Call Trace:
[ 9.407764] ? prepare_to_wait_event+0x99/0x190
[ 9.408287] tty_wait_until_sent+0x157/0x1a0
[ 9.408790] ? finish_wait+0x80/0x80
[ 9.409238] tty_port_close_start+0x11e/0x1b0
[ 9.409798] tty_port_close+0x27/0x60
[ 9.410242] tty_release+0x124/0x600
[ 9.410680] __fput+0xd1/0x270
[ 9.411079] task_work_run+0xa3/0xe0
[ 9.411525] exit_to_usermode_loop+0xeb/0xf0
[ 9.412022] do_syscall_64+0x1c6/0x1f0
[ 9.412475] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 9.413045] RIP: 0033:0x7f1ac02325b1
[ 9.413517] Code: 2d aa 00 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 8b 05 9a ee 00 00 85 c0 75 16 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 3f c3 66 0f 1f 44 00 00 53 89 fb 48 83 ec 10
[ 9.415390] RSP: 002b:00007ffed608ea98 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
[ 9.416243] RAX: 0000000000000000 RBX: 0000000000000029 RCX: 00007f1ac02325b1
[ 9.416975] RDX: 0000000000000005 RSI: 00007ffed608eaf0 RDI: 0000000000000029
[ 9.417718] RBP: 00007f1abe5ec460 R08: 000000000000fce0 R09: 0000563498acc700
[ 9.418452] R10: 0000000000000003 R11: 0000000000000246 R12: 0000000000000000
[ 9.419186] R13: 0000000000000005 R14: 0000000000000000 R15: 0000000000000030
[ 9.419924] ---[ end trace 029e41b457dc5ff5 ]---
To reproduce:
# build kernel
cd linux
cp config-5.5.0-rc5-00071-g5acb72bc5642b .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
10 months
- 1
- 0
[char] 987f028b86: stack_segment:#[##]
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 987f028b8637cfa7658aa456ae73f8f21a7a7f6f ("char: hpet: Use flexible-array member")
https://git.kernel.org/cgit/linux/kernel/git/gregkh/char-misc.git char-misc-next
in testcase: blktests
with following parameters:
disk: 1SSD
test: nvme-005
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------------+------------+------------+
| | eb143f8756 | 987f028b86 |
+------------------------------------------+------------+------------+
| boot_successes | 7 | 0 |
| boot_failures | 0 | 7 |
| stack_segment:#[##] | 0 | 7 |
| RIP:__kmalloc | 0 | 7 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 7 |
+------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 1.025991] NetLabel: unlabeled traffic allowed by default
[ 1.026375] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 1.027093] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[ 1.030128] clocksource: Switched to clocksource kvm-clock
[ 1.084907] VFS: Disk quotas dquot_6.6.0
[ 1.086158] stack segment: 0000 [#1] SMP PTI
[ 1.087100] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.5.0-rc7-00103-g987f028b8637c #1
[ 1.087100] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 1.087100] RIP: 0010:__kmalloc+0x9d/0x270
[ 1.087100] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 4a 37 b7 75 49 8b 28 48 85 ed 0f 84 a1 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b
[ 1.087100] RSP: 0000:ffffa4cb80013d28 EFLAGS: 00010202
[ 1.087100] RAX: 0000000000000000 RBX: 0000000000000dc0 RCX: 0000000000000133
[ 1.087100] RDX: 0000000000000132 RSI: 0000000000000dc0 RDI: ffff981247c02fc0
[ 1.087100] RBP: 009896808086a201 R08: 0000000000031120 R09: ffff981274887e00
[ 1.087100] R10: ffffa4cb80013d78 R11: 0000000000000000 R12: 0000000000000dc0
[ 1.087100] R13: 00000000000000fe R14: ffff981247c02fc0 R15: ffff981247c02fc0
[ 1.087100] FS: 0000000000000000(0000) GS:ffff98137fc00000(0000) knlGS:0000000000000000
[ 1.087100] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1.087100] CR2: 0000000000000000 CR3: 0000000133a0a000 CR4: 00000000000406f0
[ 1.087100] Call Trace:
[ 1.087100] ? __register_sysctl_table+0x44e/0x5e0
[ 1.087100] __register_sysctl_table+0x44e/0x5e0
[ 1.087100] ? kmem_cache_alloc_trace+0x3b/0x230
[ 1.087100] __register_sysctl_paths+0x1c0/0x1f0
[ 1.087100] ? set_debug_rodata+0x11/0x11
[ 1.087100] dquot_init+0x2f/0x116
[ 1.087100] ? iomap_init+0x20/0x20
[ 1.087100] do_one_initcall+0x46/0x220
[ 1.087100] kernel_init_freeable+0x1c2/0x23b
[ 1.087100] ? rest_init+0xd0/0xd0
[ 1.087100] kernel_init+0xa/0x110
[ 1.087100] ret_from_fork+0x35/0x40
[ 1.087100] Modules linked in:
[ 1.115829] ---[ end trace bc28001aba492303 ]---
[ 1.116611] RIP: 0010:__kmalloc+0x9d/0x270
[ 1.117313] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 4a 37 b7 75 49 8b 28 48 85 ed 0f 84 a1 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b
[ 1.120074] RSP: 0000:ffffa4cb80013d28 EFLAGS: 00010202
[ 1.120940] RAX: 0000000000000000 RBX: 0000000000000dc0 RCX: 0000000000000133
[ 1.122042] RDX: 0000000000000132 RSI: 0000000000000dc0 RDI: ffff981247c02fc0
[ 1.123138] RBP: 009896808086a201 R08: 0000000000031120 R09: ffff981274887e00
[ 1.124258] R10: ffffa4cb80013d78 R11: 0000000000000000 R12: 0000000000000dc0
[ 1.125378] R13: 00000000000000fe R14: ffff981247c02fc0 R15: ffff981247c02fc0
[ 1.126453] FS: 0000000000000000(0000) GS:ffff98137fc00000(0000) knlGS:0000000000000000
[ 1.127787] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1.128716] CR2: 0000000000000000 CR3: 0000000133a0a000 CR4: 00000000000406f0
[ 1.129807] Kernel panic - not syncing: Fatal exception
To reproduce:
# build kernel
cd linux
cp config-5.5.0-rc7-00103-g987f028b8637c .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
10 months
- 1
- 0
8868232560 ("char: hpet: Use flexible-array member"): [ 0.552714] BUG: unable to handle page fault for address: 01691dcf
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://github.com/0day-ci/linux/commits/Gustavo-A-R-Silva/char-hpet-Use-...
commit 88682325604256d47565700a3e36ded20aba935f
Author: Gustavo A. R. Silva <gustavo(a)embeddedor.com>
AuthorDate: Mon Jan 20 17:53:26 2020 -0600
Commit: 0day robot <lkp(a)intel.com>
CommitDate: Fri Jan 24 03:13:16 2020 +0800
char: hpet: Use flexible-array member
Old code in the kernel uses 1-byte and 0-byte arrays to indicate the
presence of a "variable length array":
struct something {
int length;
u8 data[1];
};
struct something *instance;
instance = kmalloc(sizeof(*instance) + size, GFP_KERNEL);
instance->length = size;
memcpy(instance->data, source, size);
There is also 0-byte arrays. Both cases pose confusion for things like
sizeof(), CONFIG_FORTIFY_SOURCE, etc.[1] Instead, the preferred mechanism
to declare variable-length types such as the one above is a flexible array
member[2] which need to be the last member of a structure and empty-sized:
struct something {
int stuff;
u8 data[];
};
Also, by making use of the mechanism above, we will get a compiler warning
in case the flexible array does not occur last in the structure, which
will help us prevent some kind of undefined behavior bugs from being
unadvertenly introduced[3] to the codebase from now on.
[1] https://github.com/KSPP/linux/issues/21
[2] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
[3] commit 76497732932f ("cxgb3/l2t: Fix undefined behaviour")
Signed-off-by: Gustavo A. R. Silva <gustavo(a)embeddedor.com>
eb143f8756 binder: fix log spam for existing debugfs file creation.
8868232560 char: hpet: Use flexible-array member
+-----------------------------------------------------------+------------+------------+
| | eb143f8756 | 8868232560 |
+-----------------------------------------------------------+------------+------------+
| boot_successes | 295 | 47 |
| boot_failures | 6 | 69 |
| BUG:kernel_hang_in_test_stage | 2 | 6 |
| Kernel_panic-not_syncing:Attempted_to_kill_init!exitcode= | 4 | 2 |
| Assertion_failed | 2 | |
| BUG:kernel_NULL_pointer_dereference,address | 0 | 24 |
| Oops:#[##] | 0 | 58 |
| EIP:strlen | 0 | 14 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 61 |
| BUG:unable_to_handle_page_fault_for_address | 0 | 31 |
| EIP:__kmalloc | 0 | 18 |
| EIP:insert_header | 0 | 9 |
| EIP:find_entry | 0 | 14 |
| EIP:qdisc_alloc | 0 | 3 |
| kernel_BUG_at_mm/slub.c | 0 | 3 |
| invalid_opcode:#[##] | 0 | 3 |
| EIP:kfree | 0 | 3 |
+-----------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 0.546173] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.547606] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.548869] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[ 0.550220] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
[ 0.551805] NET: Registered protocol family 2
[ 0.552714] BUG: unable to handle page fault for address: 01691dcf
[ 0.553954] #PF: supervisor read access in kernel mode
[ 0.554993] #PF: error_code(0x0000) - not-present page
[ 0.555628] *pde = 00000000
[ 0.555628] Oops: 0000 [#1]
[ 0.555628] CPU: 0 PID: 1 Comm: swapper Not tainted 5.5.0-rc7-00103-g8868232560425 #1
[ 0.555628] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 0.555628] EIP: __kmalloc+0x59/0x230
[ 0.555628] Code: f6 c7 04 75 44 89 da 89 f0 e8 83 08 fd ff 85 c0 75 5f 85 f6 74 5b 8b 0e 8b 69 04 8b 19 85 db 0f 84 54 01 00 00 8b 56 14 01 da <8b> 02 89 44 24 08 8b 46 7c 89 44 24 04 9c 59 fa 8b 06 3b 18 74 61
[ 0.555628] EAX: 00000000 EBX: 01691dcf ECX: f568b080 EDX: 01691dcf
[ 0.555628] ESI: 800989c0 EDI: 00000dc0 EBP: 00000042 ESP: 80141eec
[ 0.555628] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00010206
[ 0.555628] CR0: 80050033 CR2: 01691dcf CR3: 021bb000 CR4: 00140690
[ 0.555628] Call Trace:
[ 0.555628] ? __register_sysctl_table+0x43e/0x5d0
[ 0.555628] ? printk+0xd/0xe
[ 0.555628] ? sock_register+0x3d/0x70
[ 0.555628] ? inet_init_net+0xdb/0xdb
[ 0.555628] ? inet_init+0x74/0x20b
[ 0.555628] ? do_one_initcall+0x31/0x1a0
[ 0.555628] ? parse_args+0x60/0x400
[ 0.555628] ? kernel_init_freeable+0xe5/0x162
[ 0.555628] ? kernel_init_freeable+0x119/0x162
[ 0.555628] ? rest_init+0x80/0x80
[ 0.555628] ? kernel_init+0x5/0xe0
[ 0.555628] ? ret_from_fork+0x19/0x30
[ 0.555628] Modules linked in:
[ 0.555628] CR2: 0000000001691dcf
[ 0.555628] ---[ end trace 8754433795f8cb33 ]---
[ 0.555628] EIP: __kmalloc+0x59/0x230
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 17379081fe45690bf8f591bb389ae184f92cb94a def9d2780727cec3313ed3522d0123158d87224d --
git bisect bad 09ffb4921e5ffecf390dc996e6886510c123d517 # 17:36 B 9 1 5 5 Merge 'linux-review/Colin-King/usb-musb-fix-spelling-mistake-periperal-peripheral/20200118-060848' into devel-hourly-2020012415
git bisect bad 84f764efb0edc16db815df823221fff3778fa995 # 03:14 B 43 5 18 18 Merge 'peterz-queue/x86/pti' into devel-hourly-2020012415
git bisect bad d23c13017a2336059d0551d1fcfd593eebedbd23 # 16:56 B 10 4 2 3 Merge 'linux-review/Yue-Hu/zram-do-not-set-ZRAM_IDLE-bit-for-idlepage-writeback-in-writeback_store/20200124-065342' into devel-hourly-2020012415
git bisect bad 63ff12bb3f17064ffb4f848c79d346dd1b8b3071 # 04:38 B 8 1 4 4 Merge 'pwm/for-next' into devel-hourly-2020012415
git bisect good 891626a66558c45620e5c3a45a3aea5d2e1a6b35 # 06:03 G 100 0 0 0 Merge 'linux-review/Eric-Dumazet/gtp-make-sure-only-SOCK_DGRAM-UDP-sockets-are-accepted/20200124-134655' into devel-hourly-2020012415
git bisect good d56d251c699342238622f9133a2e95fad430d78a # 06:40 G 104 0 0 0 Merge 'linux-review/Alex-Shi/lib-zstd-remove-unused-macros/20200124-044302' into devel-hourly-2020012415
git bisect bad 22478a522756522c3bc023d0f5ca182b9f94c287 # 06:59 B 3 1 1 1 Merge 'linux-review/Gustavo-A-R-Silva/char-hpet-Use-flexible-array-member/20200124-031315' into devel-hourly-2020012415
git bisect good 1660557bd7cfced58b12389d7e99e21cde7e1cbe # 07:24 G 99 0 0 0 Merge tag 'extcon-next-for-5.6' of git://git.kernel.org/pub/scm/linux/kernel/git/chanwoo/extcon into char-misc-next
git bisect good 451ed9682babad0592c882ca303a4634cc1bb045 # 07:50 G 96 0 0 0 Merge branch 'topic/sdw_intel' into next
git bisect good 30c8fa3ec61a46da80698e1f8ab95df4d42bf374 # 08:20 G 98 0 0 0 interconnect: qcom: Add MSM8916 interconnect provider driver
git bisect good 559e575a8946a6561dfe8880de341d4ef78d5994 # 08:43 G 100 0 0 0 mei: me: add comet point (lake) H device ids
git bisect good 28910cee898c2a43fcdd8c26e1378ae8084a93a0 # 09:20 G 96 0 0 3 fpga: xilinx-pr-decoupler: Remove clk_get error message for probe defer
git bisect good 10d3e38c7923853967cea97513213bba923dde64 # 09:49 G 99 0 0 2 Merge tag 'icc-5.6-rc1' of https://git.linaro.org/people/georgi.djakov/linux into char-misc-next
git bisect bad 88682325604256d47565700a3e36ded20aba935f # 10:07 B 12 1 4 4 char: hpet: Use flexible-array member
git bisect good eb143f8756e77c8fcfc4d574922ae9efd3a43ca9 # 10:35 G 102 0 0 2 binder: fix log spam for existing debugfs file creation.
# first bad commit: [88682325604256d47565700a3e36ded20aba935f] char: hpet: Use flexible-array member
git bisect good eb143f8756e77c8fcfc4d574922ae9efd3a43ca9 # 10:49 G 286 0 0 3 binder: fix log spam for existing debugfs file creation.
# extra tests with debug options
git bisect good 88682325604256d47565700a3e36ded20aba935f # 11:40 G 99 0 99 104 char: hpet: Use flexible-array member
# extra tests on head commit of linux-review/Gustavo-A-R-Silva/char-hpet-Use-flexible-array-member/20200124-031315
git bisect bad 88682325604256d47565700a3e36ded20aba935f # 11:55 B 47 18 0 51 char: hpet: Use flexible-array member
# bad: [88682325604256d47565700a3e36ded20aba935f] char: hpet: Use flexible-array member
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation
10 months
- 1
- 0