[ptrace] 201766a20e: kernel_selftests.seccomp.make_fail
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 201766a20e30f982ccfe36bebfad9602c3ff574a ("ptrace: add PTRACE_GET_SYSCALL_INFO request")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master
in testcase: kernel_selftests
with following parameters:
group: kselftests-02
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
2019-07-26 16:52:03 make run_tests -C seccomp
make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp'
gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf
In file included from seccomp_bpf.c:51:0:
seccomp_bpf.c: In function ‘tracer_ptrace’:
seccomp_bpf.c:1787:20: error: ‘PTRACE_EVENTMSG_SYSCALL_ENTRY’ undeclared (first use in this function)
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
seccomp_bpf.c:1787:20: note: each undeclared identifier is reported only once for each function it appears in
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
seccomp_bpf.c:1788:6: error: ‘PTRACE_EVENTMSG_SYSCALL_EXIT’ undeclared (first use in this function)
: PTRACE_EVENTMSG_SYSCALL_EXIT, msg);
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
Makefile:12: recipe for target 'seccomp_bpf' failed
make: *** [seccomp_bpf] Error 1
make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp'
To reproduce:
# build kernel
cd linux
cp config-5.2.0-10889-g201766a20e30f9 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 month, 2 weeks
- 3
- 3
[ext4] b1b4705d54: filebench.sum_bytes_mb/s -20.2% regression
by kernel test robot
Greeting,
FYI, we noticed a -20.2% regression of filebench.sum_bytes_mb/s due to commit:
commit: b1b4705d54abedfd69dcdf42779c521aa1e0fbd3 ("ext4: introduce direct I/O read using iomap infrastructure")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: filebench
on test machine: 8 threads Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 8G memory
with following parameters:
disk: 1HDD
fs: ext4
test: fivestreamreaddirect.f
cpufreq_governor: performance
ucode: 0x27
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/1HDD/ext4/x86_64-rhel-7.6/debian-x86_64-2019-11-14.cgz/lkp-hsw-d01/fivestreamreaddirect.f/filebench/0x27
commit:
09edf4d381 ("ext4: introduce new callback for IOMAP_REPORT")
b1b4705d54 ("ext4: introduce direct I/O read using iomap infrastructure")
09edf4d381957b14 b1b4705d54abedfd69dcdf42779
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
:4 25% 1:4 dmesg.WARNING:stack_recursion
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:da
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:db
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:dc
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:dd
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:de
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:df
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ea
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:eb
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ec
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ed
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ee
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ef
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fa
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fb
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fc
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fd
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fe
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ff
:4 25% 1:4 kmsg.file:libnss_files-#so_fault:filemap_fault_mmap:generic_file_mmap_readpage:simple_readpage
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a07fffffa0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a17fffffa1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a27fffffa2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a37fffffa3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a47fffffa4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a57fffffa5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a67fffffa6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a77fffffa7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a7fffff8aff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a7fffff9aff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a87fffffa8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a97fffffa9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#aa7fffffaaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ab7fffffabff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ac7fffffacff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ad7fffffadff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ae7fffffaeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#af7fffffafff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b07fffffb0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b17fffffb1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b27fffffb2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b37fffffb3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b47fffffb4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b57fffffb5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b67fffffb6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b77fffffb7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b7fffff8bff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b7fffff9bff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b87fffffb8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b97fffffb9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ba7fffffbaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bb7fffffbbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bc7fffffbcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bd7fffffbdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#be7fffffbeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bf7fffffbfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c07fffffc0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c17fffffc1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c27fffffc2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c37fffffc3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c47fffffc4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c57fffffc5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c67fffffc6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c77fffffc7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c7fffff8cff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c7fffff9cff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c87fffffc8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c97fffffc9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ca7fffffcaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cb7fffffcbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cc7fffffccff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cd7fffffcdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ce7fffffceff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cf7fffffcfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d07fffffd0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d17fffffd1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d27fffffd2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d37fffffd3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d47fffffd4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d57fffffd5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d67fffffd6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d77fffffd7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d7fffff8dff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d7fffff9dff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d87fffffd8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d97fffffd9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#da7fffffdaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#db7fffffdbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#dc7fffffdcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#dd7fffffddff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#de7fffffdeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#df7fffffdfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e07fffffe0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e17fffffe1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e27fffffe2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e37fffffe3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e47fffffe4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e57fffffe5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e67fffffe6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e77fffffe7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e7fffff8eff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e7fffff9eff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e87fffffe8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e97fffffe9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ea7fffffeaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#eb7fffffebff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ec7fffffecff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ed7fffffedff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ee7fffffeeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ef7fffffefff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f07ffffff0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f17ffffff1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f27ffffff2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f37ffffff3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f47ffffff4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f57ffffff5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f67ffffff6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f77ffffff7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f7fffff8fff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f7fffff9fff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f87ffffff8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f97ffffff9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fa7ffffffaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fb7ffffffbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fc7ffffffcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fd7ffffffdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fe7ffffffeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ff7fffffffff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff80ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff81ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff82ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff83ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff84ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff85ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff86ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff87ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff88ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff89ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff90ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff91ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff92ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff93ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff94ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff95ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff96ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff97ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff98ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff99ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffffffffff
%stddev %change %stddev
\ | \
58.30 ± 2% -20.2% 46.53 filebench.sum_bytes_mb/s
3503 ± 2% -20.2% 2796 filebench.sum_operations
58.39 ± 2% -20.2% 46.61 filebench.sum_operations/s
58.33 ± 2% -19.9% 46.75 filebench.sum_reads/s
84.34 ± 4% +27.1% 107.17 filebench.sum_time_ms/op
7174922 ± 2% -20.2% 5727328 filebench.time.file_system_inputs
10963 ± 2% -8.5% 10035 filebench.time.voluntary_context_switches
58.37 ± 3% -9.0 49.38 ± 3% mpstat.cpu.all.idle%
41.08 ± 5% +8.2 49.24 ± 2% mpstat.cpu.all.iowait%
3061 ± 6% +6.1% 3246 ± 4% turbostat.Bzy_MHz
5.48 ± 97% +260.7% 19.78 ± 30% turbostat.Pkg%pc2
27.33 ± 32% -52.0% 13.12 ± 19% sched_debug.cpu.nr_uninterruptible.max
-67.67 -60.8% -26.50 sched_debug.cpu.nr_uninterruptible.min
28.48 ± 30% -57.9% 11.98 ± 23% sched_debug.cpu.nr_uninterruptible.stddev
8210 ± 44% +76.1% 14454 ± 6% softirqs.CPU0.RCU
8200 ± 40% +70.2% 13954 ± 5% softirqs.CPU2.RCU
9772 ± 36% +59.6% 15597 ± 15% softirqs.CPU3.RCU
108.33 ± 27% +113.5% 231.25 ± 26% interrupts.CPU1.RES:Rescheduling_interrupts
1032 -23.9% 786.00 ± 30% interrupts.CPU5.CAL:Function_call_interrupts
229.33 ± 46% +50.1% 344.25 ± 31% interrupts.CPU5.NMI:Non-maskable_interrupts
229.33 ± 46% +50.1% 344.25 ± 31% interrupts.CPU5.PMI:Performance_monitoring_interrupts
457.00 ± 61% -74.2% 117.75 ± 41% interrupts.CPU5.RES:Rescheduling_interrupts
53.33 ± 60% +206.6% 163.50 ± 42% interrupts.CPU7.RES:Rescheduling_interrupts
58.99 ± 3% -14.9% 50.18 ± 3% iostat.cpu.idle
40.36 ± 5% +19.8% 48.35 ± 2% iostat.cpu.iowait
0.18 +386.3% 0.87 ±137% iostat.cpu.user
33012 ± 2% -19.7% 26520 iostat.sda.rkB/s
40.60 ± 6% +16.2% 47.18 ± 4% iostat.sda.w/s
43038 ± 4% +17.1% 50399 ± 2% iostat.sda.wkB/s
58.67 ± 3% -15.6% 49.50 ± 3% vmstat.cpu.id
40.00 ± 5% +19.4% 47.75 ± 2% vmstat.cpu.wa
33010 ± 2% -19.7% 26518 vmstat.io.bi
42837 ± 4% +17.2% 50200 ± 2% vmstat.io.bo
5519500 -15.5% 4664152 vmstat.memory.cache
2183603 +39.2% 3039439 ± 2% vmstat.memory.free
1065500 -19.2% 861375 slabinfo.buffer_head.active_objs
27320 -19.1% 22093 slabinfo.buffer_head.active_slabs
1065521 -19.1% 861671 slabinfo.buffer_head.num_objs
27320 -19.1% 22093 slabinfo.buffer_head.num_slabs
1306 +18.4% 1546 ± 7% slabinfo.kmalloc-512.active_objs
1306 +18.4% 1546 ± 7% slabinfo.kmalloc-512.num_objs
28277 -9.3% 25651 slabinfo.radix_tree_node.active_objs
5388173 -15.3% 4561843 meminfo.Cached
989375 ± 2% -22.6% 765315 ± 3% meminfo.Dirty
4310025 -19.2% 3483318 meminfo.Inactive
4261041 -19.4% 3434223 meminfo.Inactive(file)
162676 -13.7% 140448 meminfo.KReclaimable
2151883 ± 2% +39.5% 3000927 ± 2% meminfo.MemFree
5902816 -14.4% 5053772 meminfo.Memused
162676 -13.7% 140448 meminfo.SReclaimable
197794 -11.3% 175489 meminfo.Slab
80314276 ± 5% +6.5% 85566891 ± 3% perf-stat.i.branch-instructions
4810221 ± 5% +7.5% 5172535 perf-stat.i.cache-references
7.022e+08 ± 3% +6.9% 7.509e+08 ± 2% perf-stat.i.cpu-cycles
3.891e+08 ± 5% +6.3% 4.135e+08 ± 3% perf-stat.i.instructions
195094 ± 2% +10.8% 216142 perf-stat.i.node-loads
79563934 ± 5% +6.5% 84757473 ± 3% perf-stat.ps.branch-instructions
4765201 ± 5% +7.5% 5123552 perf-stat.ps.cache-references
6.956e+08 ± 3% +6.9% 7.438e+08 ± 2% perf-stat.ps.cpu-cycles
3.855e+08 ± 5% +6.3% 4.096e+08 ± 3% perf-stat.ps.instructions
193276 ± 2% +10.8% 214098 perf-stat.ps.node-loads
4.114e+10 ± 5% +5.3% 4.333e+10 ± 3% perf-stat.total.instructions
247900 ± 3% -22.9% 191187 ± 3% proc-vmstat.nr_dirty
1349353 -15.5% 1140545 proc-vmstat.nr_file_pages
535810 ± 2% +40.1% 750444 ± 2% proc-vmstat.nr_free_pages
1067252 -19.6% 858356 proc-vmstat.nr_inactive_file
40723 -13.8% 35112 proc-vmstat.nr_slab_reclaimable
1110681 ± 2% +17.1% 1300126 proc-vmstat.nr_written
1067251 -19.6% 858384 proc-vmstat.nr_zone_inactive_file
249604 ± 3% -22.7% 193058 ± 3% proc-vmstat.nr_zone_write_pending
660225 -3.8% 635276 proc-vmstat.pgalloc_dma32
871298 +2.7% 894930 proc-vmstat.pgalloc_normal
3587465 ± 2% -20.2% 2863670 proc-vmstat.pgpgin
4645372 ± 4% +16.6% 5414779 proc-vmstat.pgpgout
13.16 ± 14% -6.6 6.58 ± 52% perf-profile.calltrace.cycles-pp.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.06 ± 15% -6.6 6.50 ± 52% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.50 ± 52% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.02 ± 14% -6.5 6.48 ± 52% perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat.do_syscall_64
10.39 ± 36% -4.6 5.78 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel
11.21 ± 33% -4.6 6.64 ± 7% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64
5.71 ± 17% -2.8 2.95 ± 54% perf-profile.calltrace.cycles-pp.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
3.68 ± 14% -2.0 1.69 ± 50% perf-profile.calltrace.cycles-pp.release_pages.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode.evict
3.70 ± 14% -2.0 1.74 ± 48% perf-profile.calltrace.cycles-pp.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
2.90 ± 23% -1.6 1.32 ± 56% perf-profile.calltrace.cycles-pp.try_to_free_buffers.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode.evict
2.42 ± 20% -1.4 1.01 ± 75% perf-profile.calltrace.cycles-pp.delete_from_page_cache_batch.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
2.02 ± 17% -1.3 0.68 ±104% perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode
1.33 ± 28% -0.9 0.45 ±100% perf-profile.calltrace.cycles-pp.free_buffer_head.try_to_free_buffers.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode
1.21 ± 22% -0.8 0.40 ±106% perf-profile.calltrace.cycles-pp.xas_store.delete_from_page_cache_batch.truncate_inode_pages_range.ext4_evict_inode.evict
1.17 ± 3% +0.4 1.57 ± 13% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
0.35 ±141% +0.7 1.07 ± 37% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
3.71 ± 34% +1.6 5.34 ± 14% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
64.92 ± 5% +9.2 74.17 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
63.80 ± 4% +9.4 73.19 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
69.95 ± 5% +10.4 80.37 ± 5% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
13.16 ± 14% -6.6 6.58 ± 52% perf-profile.children.cycles-pp.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.children.cycles-pp.do_unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.children.cycles-pp.ext4_evict_inode
13.03 ± 14% -6.5 6.49 ± 53% perf-profile.children.cycles-pp.truncate_inode_pages_range
13.07 ± 15% -6.4 6.64 ± 53% perf-profile.children.cycles-pp.evict
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.children.cycles-pp.start_kernel
5.71 ± 17% -2.7 2.99 ± 54% perf-profile.children.cycles-pp.truncate_cleanup_page
3.74 ± 13% -1.9 1.85 ± 49% perf-profile.children.cycles-pp.__pagevec_release
3.75 ± 12% -1.9 1.88 ± 49% perf-profile.children.cycles-pp.release_pages
2.91 ± 22% -1.5 1.40 ± 53% perf-profile.children.cycles-pp.try_to_free_buffers
2.43 ± 19% -1.3 1.12 ± 57% perf-profile.children.cycles-pp.delete_from_page_cache_batch
2.03 ± 17% -1.1 0.95 ± 58% perf-profile.children.cycles-pp.free_unref_page_list
1.34 ± 27% -0.7 0.65 ± 45% perf-profile.children.cycles-pp.free_buffer_head
1.23 ± 20% -0.6 0.64 ± 50% perf-profile.children.cycles-pp.xas_store
0.78 ± 15% -0.4 0.36 ± 52% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers
0.73 ± 16% -0.4 0.33 ± 52% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head
0.50 ± 39% -0.4 0.15 ± 53% perf-profile.children.cycles-pp.unaccount_page_cache_page
0.42 ± 24% -0.3 0.14 ± 93% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.76 ± 19% -0.2 0.53 ± 6% perf-profile.children.cycles-pp.lapic_next_deadline
0.32 ± 22% -0.2 0.14 ± 69% perf-profile.children.cycles-pp.free_unref_page_commit
0.25 ± 25% -0.2 0.08 ±102% perf-profile.children.cycles-pp.uncharge_page
0.19 ± 47% -0.1 0.07 ± 68% perf-profile.children.cycles-pp.__might_sleep
0.16 ± 28% -0.1 0.05 ±112% perf-profile.children.cycles-pp.__libc_fork
0.15 ± 22% -0.1 0.04 ±102% perf-profile.children.cycles-pp.xas_find
0.13 ± 12% -0.1 0.08 ± 34% perf-profile.children.cycles-pp.leave_mm
0.04 ± 71% +0.0 0.07 ± 14% perf-profile.children.cycles-pp.irq_work_needs_cpu
0.02 ±141% +0.1 0.09 ± 29% perf-profile.children.cycles-pp.__msecs_to_jiffies
0.04 ± 71% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.cpuidle_not_available
0.09 ± 75% +0.1 0.18 ± 23% perf-profile.children.cycles-pp.__schedule
0.00 +0.1 0.10 ± 40% perf-profile.children.cycles-pp.link_path_walk
0.09 ± 41% +0.1 0.20 ± 19% perf-profile.children.cycles-pp.pm_qos_read_value
0.04 ± 71% +0.1 0.14 ± 52% perf-profile.children.cycles-pp.raise_softirq
0.04 ±141% +0.1 0.15 ± 33% perf-profile.children.cycles-pp.__test_set_page_writeback
0.00 +0.1 0.12 ± 17% perf-profile.children.cycles-pp.__delete_from_page_cache
0.05 ±141% +0.1 0.17 ± 60% perf-profile.children.cycles-pp.exit_mmap
0.19 ± 54% +0.1 0.32 ± 21% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.05 ±141% +0.1 0.18 ± 64% perf-profile.children.cycles-pp.mmput
0.08 ± 30% +0.2 0.24 ± 33% perf-profile.children.cycles-pp.run_local_timers
0.05 ±141% +0.2 0.21 ± 41% perf-profile.children.cycles-pp.ext4_bio_write_page
0.14 ± 32% +0.2 0.30 ± 46% perf-profile.children.cycles-pp.update_rq_clock
0.15 ± 71% +0.2 0.33 ± 17% perf-profile.children.cycles-pp.rcu_idle_exit
0.38 ± 29% +0.2 0.56 ± 16% perf-profile.children.cycles-pp.sched_clock_cpu
0.14 ± 78% +0.2 0.33 ± 41% perf-profile.children.cycles-pp.mpage_map_and_submit_buffers
0.00 +0.2 0.20 ± 52% perf-profile.children.cycles-pp.invalidate_inode_pages2_range
0.07 ± 97% +0.2 0.30 ± 58% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.07 ±141% +0.2 0.31 ± 37% perf-profile.children.cycles-pp.mpage_submit_page
0.38 ± 21% +0.3 0.64 ± 14% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.08 ±101% +0.3 0.34 ± 51% perf-profile.children.cycles-pp.do_group_exit
0.08 ±101% +0.3 0.34 ± 51% perf-profile.children.cycles-pp.do_exit
0.23 ± 29% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.do_writepages
0.23 ± 29% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.ext4_writepages
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.wb_workfn
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.wb_writeback
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.__writeback_inodes_wb
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.writeback_sb_inodes
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.__writeback_single_inode
0.00 +0.3 0.32 ± 49% perf-profile.children.cycles-pp.ext4_file_read_iter
0.00 +0.3 0.32 ± 49% perf-profile.children.cycles-pp.iomap_dio_rw
1.30 ± 4% +0.4 1.72 ± 14% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.74 ± 41% +0.5 1.25 ± 33% perf-profile.children.cycles-pp.scheduler_tick
75.40 +4.7 80.05 ± 4% perf-profile.children.cycles-pp.cpuidle_enter
75.35 +4.7 80.02 ± 4% perf-profile.children.cycles-pp.cpuidle_enter_state
81.28 +5.9 87.18 ± 4% perf-profile.children.cycles-pp.do_idle
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.children.cycles-pp.start_secondary
1.16 ± 22% -0.6 0.56 ± 63% perf-profile.self.cycles-pp.free_pcppages_bulk
0.73 ± 16% -0.4 0.33 ± 52% perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head
0.46 ± 21% -0.3 0.16 ± 36% perf-profile.self.cycles-pp.xas_store
0.70 ± 5% -0.3 0.45 ± 38% perf-profile.self.cycles-pp.release_pages
0.29 ± 17% -0.2 0.07 ± 66% perf-profile.self.cycles-pp.free_unref_page_commit
0.32 ± 26% -0.2 0.13 ± 86% perf-profile.self.cycles-pp.workingset_update_node
0.18 ± 14% -0.1 0.06 ±100% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.11 -0.1 0.05 ± 60% perf-profile.self.cycles-pp.leave_mm
0.04 ± 71% +0.0 0.07 ± 14% perf-profile.self.cycles-pp.irq_work_needs_cpu
0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.__msecs_to_jiffies
0.04 ± 71% +0.1 0.11 ± 26% perf-profile.self.cycles-pp.cpuidle_not_available
0.08 ± 81% +0.1 0.18 ± 23% perf-profile.self.cycles-pp.rcu_idle_exit
0.09 ± 41% +0.1 0.20 ± 19% perf-profile.self.cycles-pp.pm_qos_read_value
0.04 ± 71% +0.1 0.14 ± 52% perf-profile.self.cycles-pp.raise_softirq
0.04 ±141% +0.1 0.17 ± 41% perf-profile.self.cycles-pp.calc_global_load_tick
0.16 ± 12% +0.2 0.32 ± 27% perf-profile.self.cycles-pp.run_timer_softirq
0.08 ± 81% +0.2 0.26 ± 46% perf-profile.self.cycles-pp.update_rq_clock
0.17 ± 49% +0.2 0.36 ± 22% perf-profile.self.cycles-pp.load_balance
0.26 ± 8% +0.2 0.47 ± 40% perf-profile.self.cycles-pp.hrtimer_interrupt
filebench.time.file_system_inputs
7.6e+06 +-+---------------------------------------------------------------+
7.4e+06 +-++ .+ +.+ |
|: ++ + +.+.+.+.++. .+ : : +.+.+. +.|
7.2e+06 +-+ : : + + .++ + : : .+ + + + |
7e+06 +-+ : : + + + +: +.+ + :+ + |
6.8e+06 +-+ + + + + +.+ : |
6.6e+06 +-+ +: |
| + |
6.4e+06 +-+ |
6.2e+06 +-+ |
6e+06 +-+ |
5.8e+06 O-+ O O O O |
| O OO O O O O O OO O O O OO O O OO O O O O |
5.6e+06 +-+ O O |
5.4e+06 +-+---------------------------------------------------------------+
filebench.sum_operations
3700 +-+------------------------------------------------------------------+
3600 +-++ .+. +.+ |
|: +. + +.+.+.+.++. .+ : : +.+.+. +.|
3500 +-+ + : + + .+.+ + : : : + + : |
3400 +-+ :: + + :+ : :.+.+ + + + : |
| + + + + + +.+ + |
3300 +-+ :+ |
3200 +-+ + |
3100 +-+ |
| |
3000 +-+ |
2900 +-+ |
O O O O O O O O |
2800 +-O O OO O O O O O O O O O O O OO O O O O |
2700 +-+---------------------------------------------O--------------------+
filebench.sum_operations_s
62 +-+--------------------------------------------------------------------+
| + .+ .+ |
60 +-++ + +.+.+.+.+. .+ + : +. .+. +.|
58 +-+ +.+ : +.+ + .+.+ + : : + + + + : |
|: + : + + :+ : :.+.+ + :+ : |
56 +-+ + + + + + +.+ + |
54 +-+ + + |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ O |
46 O-O O O O O O O O O O O O O O O OO O O O O O O O O O |
| O O |
44 +-+--------------------------------------------------------------------+
filebench.sum_reads_s
62 +-+--------------------------------------------------------------------+
| + |
60 +-++ .++. .+. +.+ +. .+. .|
58 +-+ +.+ + +.+ +.+.+.+. .+.+ + : : + + + + + |
|: + + + + :+ : : .+ + :+ + |
56 +-+ + + + +.+ + +.+ + |
| : : |
54 +-+ : : |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ |
O O O O O O O O O O |
46 +-O-O-O-O---OO-O-----O---O-O-O-O-O-OO-O-O-O-O-----O--------------------+
filebench.sum_bytes_mb_s
62 +-+--------------------------------------------------------------------+
| + |
60 +-++ +.++.+.+.+.+. +.+ +. .+ +.|
58 +-+ +.+ : +.+.+. .+. : : + + + : |
|: + : + +. +. : : .+.+ +. +. : |
56 +-+ + + + + + +.+ + |
54 +-+ + + |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ O |
46 O-O O O O O O O O O O O O O O O O O O O O O O O O |
| O O O O |
44 +-+--------------------------------------------------------------------+
filebench.sum_time_ms_op
110 +-+-O------O----------------------O--------------O--------------------+
| O O OO O O O O O OO O O O O O O O OO O O O O |
105 O-+ O |
100 +-+ |
| |
95 +-+ |
| .+. |
90 +-+ + .+. .+ + .+. .++ + |
|: : : + .+.+.+ + + : +.+. + + + |
85 +-+ +. : +. +.+ :+ +. : +.+. + +.|
80 +-++ + +. .+. + : : + + |
| + + + : : |
75 +-+ : : |
| : |
70 +-+-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
6 months, 2 weeks
- 5
- 10
[f2fs] fe1897eaa6: aim7.jobs-per-min -60.9% regression
by kernel test robot
Greeting,
FYI, we noticed a -60.9% regression of aim7.jobs-per-min due to commit:
commit: fe1897eaa6646f5a64a4cee0e6473ed9887d324b ("f2fs: fix to update time in lazytime mode")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: aim7
on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory
with following parameters:
disk: 4BRD_12G
md: RAID1
fs: f2fs
test: sync_disk_rw
load: 200
cpufreq_governor: performance
ucode: 0x42e
test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system.
test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/4BRD_12G/f2fs/x86_64-rhel-7.6/200/RAID1/debian-x86_64-2019-11-14.cgz/lkp-ivb-2ep1/sync_disk_rw/aim7/0x42e
commit:
b145b0eb20 (" ARM and x86 bugfixes of all kinds. The most visible one is that migrating")
fe1897eaa6 ("f2fs: fix to update time in lazytime mode")
b145b0eb2031a620 fe1897eaa6646f5a64a4cee0e64
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
1208 -60.9% 473.12 aim7.jobs-per-min
992.70 +155.6% 2537 aim7.time.elapsed_time
992.70 +155.6% 2537 aim7.time.elapsed_time.max
1.078e+08 +60.0% 1.724e+08 aim7.time.file_system_outputs
9646301 -4.2% 9238717 aim7.time.involuntary_context_switches
24107 ± 3% +58.2% 38130 ± 17% aim7.time.minor_page_faults
3295 +202.3% 9963 aim7.time.system_time
26.75 +51.4% 40.49 aim7.time.user_time
5.02e+08 +210.5% 1.559e+09 aim7.time.voluntary_context_switches
9294452 ± 4% -15.7% 7835497 ± 16% numa-numastat.node0.local_node
9298417 ± 4% -15.7% 7840991 ± 16% numa-numastat.node0.numa_hit
3.66 ± 3% +4.2 7.82 ± 18% mpstat.cpu.all.idle%
0.01 ± 32% -0.0 0.00 ± 31% mpstat.cpu.all.soft%
0.06 -0.0 0.04 mpstat.cpu.all.usr%
3.85 ± 3% +105.0% 7.89 ± 17% iostat.cpu.idle
89.80 -4.9% 85.36 iostat.cpu.iowait
6.29 +6.6% 6.70 iostat.cpu.system
23968 -48.4% 12356 iostat.md0.w/s
53780 -37.1% 33824 iostat.md0.wkB/s
89.00 -4.8% 84.75 vmstat.cpu.wa
53779 -37.1% 33833 vmstat.io.bo
29311046 -28.6% 20935254 vmstat.memory.free
1026388 +19.4% 1225080 vmstat.system.cs
103289 +8.5% 112038 vmstat.system.in
24013 ± 29% -58.5% 9976 ±120% numa-meminfo.node0.Dirty
119019 ± 10% +44.6% 172139 ± 13% numa-meminfo.node1.Active
3503 ± 56% +464.8% 19786 ± 38% numa-meminfo.node1.Active(file)
40477 ± 16% -38.5% 24896 ± 47% numa-meminfo.node1.Dirty
19966008 ± 2% -29.5% 14084594 ± 44% numa-meminfo.node1.MemFree
13014643 ± 4% +45.2% 18896052 ± 32% numa-meminfo.node1.MemUsed
12018 ± 41% +184.8% 34228 ± 35% numa-meminfo.node1.Shmem
2.09e+10 +127.4% 4.752e+10 ± 4% cpuidle.C1.time
2.558e+08 +140.1% 6.143e+08 ± 4% cpuidle.C1.usage
3.017e+09 ± 10% +301.2% 1.211e+10 ± 9% cpuidle.C1E.time
18225961 ± 9% +430.1% 96624314 ± 6% cpuidle.C1E.usage
4.627e+08 ± 14% +147.5% 1.145e+09 ± 20% cpuidle.C6.time
802166 ± 17% +186.9% 2301023 ± 26% cpuidle.C6.usage
1.799e+10 +148.9% 4.476e+10 cpuidle.POLL.time
2.616e+08 ± 2% +246.4% 9.063e+08 ± 2% cpuidle.POLL.usage
6017 ± 29% -58.5% 2496 ±120% numa-vmstat.node0.nr_dirty
5742 ± 29% -65.0% 2011 ±120% numa-vmstat.node0.nr_zone_write_pending
874.50 ± 56% +465.6% 4946 ± 38% numa-vmstat.node1.nr_active_file
10139 ± 16% -38.6% 6224 ± 47% numa-vmstat.node1.nr_dirty
4992144 ± 2% -29.5% 3521200 ± 44% numa-vmstat.node1.nr_free_pages
3003 ± 41% +184.9% 8556 ± 35% numa-vmstat.node1.nr_shmem
874.50 ± 56% +465.6% 4946 ± 38% numa-vmstat.node1.nr_zone_active_file
9775 ± 16% -47.9% 5089 ± 47% numa-vmstat.node1.nr_zone_write_pending
2.558e+08 +140.1% 6.143e+08 ± 4% turbostat.C1
43.77 -4.8 38.98 ± 3% turbostat.C1%
18225642 ± 9% +430.2% 96624037 ± 6% turbostat.C1E
6.31 ± 10% +3.6 9.94 ± 9% turbostat.C1E%
795117 ± 17% +188.4% 2292823 ± 26% turbostat.C6
0.38 ± 42% -63.6% 0.14 ± 13% turbostat.CPU%c6
1.028e+08 +176.7% 2.844e+08 turbostat.IRQ
0.23 ± 33% -69.1% 0.07 ± 22% turbostat.Pkg%pc2
275064 +12.3% 308846 ± 4% meminfo.Active
8036 ± 21% +261.7% 29072 ± 52% meminfo.Active(file)
64400 -45.9% 34850 meminfo.Dirty
16481 +10.8% 18264 meminfo.Inactive(anon)
147067 +12.8% 165858 ± 2% meminfo.KReclaimable
29120417 -28.8% 20740075 meminfo.MemAvailable
29335165 -28.6% 20936804 meminfo.MemFree
36511133 +23.0% 44909491 meminfo.Memused
147067 +12.8% 165858 ± 2% meminfo.SReclaimable
36567 ± 6% +37.2% 50173 meminfo.Shmem
51580 -60.8% 20220 meminfo.max_used_kB
4.27 ± 7% -13.9% 3.67 ± 4% perf-stat.i.MPKI
0.71 ± 3% -0.0 0.67 perf-stat.i.branch-miss-rate%
17.93 +3.5 21.41 ± 2% perf-stat.i.cache-miss-rate%
1029337 +19.1% 1226025 perf-stat.i.context-switches
11421 ± 4% +41.1% 16120 ± 6% perf-stat.i.cpu-migrations
0.42 ± 2% +0.0 0.45 ± 6% perf-stat.i.dTLB-load-miss-rate%
0.11 ± 27% -0.0 0.08 ± 3% perf-stat.i.dTLB-store-miss-rate%
84.04 ± 2% -2.3 81.75 ± 2% perf-stat.i.iTLB-load-miss-rate%
7375571 +7.3% 7917282 perf-stat.i.iTLB-load-misses
3.91 ± 7% -9.2% 3.56 ± 4% perf-stat.overall.MPKI
17.90 +4.1 21.96 ± 3% perf-stat.overall.cache-miss-rate%
0.41 ± 2% +0.0 0.45 ± 6% perf-stat.overall.dTLB-load-miss-rate%
0.10 ± 27% -0.0 0.08 ± 3% perf-stat.overall.dTLB-store-miss-rate%
83.99 ± 2% -2.3 81.72 ± 2% perf-stat.overall.iTLB-load-miss-rate%
1028292 +19.2% 1225532 perf-stat.ps.context-switches
11410 ± 4% +41.2% 16114 ± 6% perf-stat.ps.cpu-migrations
7368087 +7.4% 7914119 perf-stat.ps.iTLB-load-misses
2.558e+13 ± 2% +157.0% 6.575e+13 ± 3% perf-stat.total.instructions
9128 ± 2% -25.3% 6821 ± 4% slabinfo.dmaengine-unmap-16.active_objs
9175 ± 2% -24.4% 6938 ± 4% slabinfo.dmaengine-unmap-16.num_objs
28309 +12.5% 31840 slabinfo.f2fs_extent_node.active_objs
28704 +13.0% 32429 slabinfo.f2fs_extent_node.num_objs
964.00 ± 5% -13.3% 836.00 ± 6% slabinfo.kmem_cache_node.active_objs
1008 ± 5% -12.7% 880.00 ± 6% slabinfo.kmem_cache_node.num_objs
13017 -33.1% 8707 slabinfo.nat_entry.active_objs
13036 -33.2% 8707 slabinfo.nat_entry.num_objs
433.75 ± 9% +32.4% 574.50 ± 2% slabinfo.nfs_read_data.active_objs
433.75 ± 9% +32.4% 574.50 ± 2% slabinfo.nfs_read_data.num_objs
697.50 ± 3% +48.6% 1036 ± 33% slabinfo.numa_policy.active_objs
697.50 ± 3% +48.6% 1036 ± 33% slabinfo.numa_policy.num_objs
153098 +22.3% 187296 slabinfo.radix_tree_node.active_objs
2733 +22.3% 3344 slabinfo.radix_tree_node.active_slabs
153098 +22.3% 187296 slabinfo.radix_tree_node.num_objs
2733 +22.3% 3344 slabinfo.radix_tree_node.num_slabs
542.00 ± 10% +29.5% 702.00 ± 6% slabinfo.xfrm_state.active_objs
542.00 ± 10% +29.5% 702.00 ± 6% slabinfo.xfrm_state.num_objs
66757 +4.8% 69942 proc-vmstat.nr_active_anon
2009 ± 21% +261.7% 7267 ± 52% proc-vmstat.nr_active_file
96.00 +3.9% 99.75 proc-vmstat.nr_anon_transparent_hugepages
13466177 +60.0% 21542226 proc-vmstat.nr_dirtied
16154 -46.0% 8719 proc-vmstat.nr_dirty
726643 -28.8% 517541 proc-vmstat.nr_dirty_background_threshold
1455064 -28.8% 1036348 proc-vmstat.nr_dirty_threshold
311402 +2.4% 318957 proc-vmstat.nr_file_pages
7332853 -28.6% 5234441 proc-vmstat.nr_free_pages
4120 +10.8% 4565 proc-vmstat.nr_inactive_anon
13939 -1.2% 13772 proc-vmstat.nr_kernel_stack
2845 -1.5% 2803 proc-vmstat.nr_page_table_pages
9142 ± 6% +37.2% 12542 proc-vmstat.nr_shmem
36767 +12.8% 41463 ± 2% proc-vmstat.nr_slab_reclaimable
13387539 +60.3% 21463781 proc-vmstat.nr_written
66757 +4.8% 69942 proc-vmstat.nr_zone_active_anon
2009 ± 21% +261.7% 7267 ± 52% proc-vmstat.nr_zone_active_file
4120 +10.8% 4565 proc-vmstat.nr_zone_inactive_anon
15494 -54.2% 7089 proc-vmstat.nr_zone_write_pending
582146 ±102% +628.1% 4238850 ± 6% proc-vmstat.numa_foreign
13506 ± 3% +114.5% 28974 ± 31% proc-vmstat.numa_hint_faults
8913 ± 6% +148.6% 22154 ± 35% proc-vmstat.numa_hint_faults_local
582146 ±102% +628.1% 4238850 ± 6% proc-vmstat.numa_miss
597885 ± 99% +611.6% 4254585 ± 6% proc-vmstat.numa_other
1014174 ± 20% -46.1% 547135 ± 57% proc-vmstat.pgalloc_dma32
16157005 +22.4% 19769138 proc-vmstat.pgalloc_normal
2516271 +152.8% 6360808 proc-vmstat.pgfault
4866589 +64.4% 8002519 proc-vmstat.pgfree
53553588 +60.3% 85856803 proc-vmstat.pgpgout
1.55 ± 4% +0.1 1.65 perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback
1.58 ± 4% +0.1 1.69 perf-profile.calltrace.cycles-pp.schedule.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback
1.60 ± 4% +0.1 1.71 perf-profile.calltrace.cycles-pp.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback.f2fs_do_sync_file
0.75 ± 2% +0.2 0.91 ± 2% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.70 ± 4% -0.3 0.36 ± 7% perf-profile.children.cycles-pp.__generic_file_write_iter
0.69 ± 4% -0.3 0.35 ± 7% perf-profile.children.cycles-pp.generic_perform_write
0.57 ± 5% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
0.58 ± 5% -0.3 0.26 ± 7% perf-profile.children.cycles-pp.file_write_and_wait_range
0.57 ± 6% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.do_writepages
0.57 ± 5% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.f2fs_write_data_pages
0.64 ± 5% -0.3 0.34 ± 6% perf-profile.children.cycles-pp.f2fs_write_begin
0.57 ± 5% -0.2 0.32 ± 7% perf-profile.children.cycles-pp.f2fs_put_page
0.31 ± 10% -0.2 0.11 ± 7% perf-profile.children.cycles-pp.blk_finish_plug
0.31 ± 10% -0.2 0.11 ± 7% perf-profile.children.cycles-pp.blk_flush_plug_list
0.30 ± 11% -0.2 0.11 ± 6% perf-profile.children.cycles-pp.raid1_unplug
0.26 -0.1 0.14 ± 12% perf-profile.children.cycles-pp.f2fs_write_cache_pages
0.22 ± 20% -0.1 0.11 ± 41% perf-profile.children.cycles-pp.brd_do_bvec
0.14 ± 12% -0.1 0.03 ±100% perf-profile.children.cycles-pp.submit_bio_wait
0.21 ± 10% -0.1 0.11 ± 10% perf-profile.children.cycles-pp.submit_bio
0.23 ± 17% -0.1 0.13 ± 31% perf-profile.children.cycles-pp.worker_thread
0.20 ± 3% -0.1 0.11 ± 14% perf-profile.children.cycles-pp.__write_data_page
0.14 ± 15% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.__submit_flush_wait
0.17 ± 12% -0.1 0.09 ± 12% perf-profile.children.cycles-pp.md_make_request
0.16 ± 11% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.md_handle_request
0.15 ± 9% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.raid1_make_request
0.14 ± 17% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.f2fs_issue_flush
0.14 ± 6% -0.1 0.08 ± 19% perf-profile.children.cycles-pp.f2fs_do_write_data_page
0.10 ± 33% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.reschedule_interrupt
0.09 ± 37% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.scheduler_ipi
0.06 ± 6% +0.0 0.08 ± 10% perf-profile.children.cycles-pp.up_read
0.10 ± 4% +0.0 0.12 ± 13% perf-profile.children.cycles-pp.pick_next_task_idle
0.20 ± 2% +0.0 0.23 ± 5% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.16 ± 6% +0.0 0.19 ± 4% perf-profile.children.cycles-pp.irq_exit
0.01 ±173% +0.1 0.07 ± 7% perf-profile.children.cycles-pp.put_prev_task_fair
0.27 ± 3% +0.1 0.33 ± 4% perf-profile.children.cycles-pp.tick_nohz_next_event
0.35 ± 3% +0.1 0.42 ± 3% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.60 ± 4% +0.1 1.72 perf-profile.children.cycles-pp.io_schedule
0.78 ± 2% +0.1 0.92 ± 2% perf-profile.children.cycles-pp.menu_select
0.06 ± 6% +0.0 0.08 ± 10% perf-profile.self.cycles-pp.up_read
0.08 ± 5% +0.0 0.11 ± 11% perf-profile.self.cycles-pp.cpuidle_enter_state
0.35 ± 5% +0.0 0.39 ± 5% perf-profile.self.cycles-pp.f2fs_wait_on_node_pages_writeback
0.35 ± 4% +0.1 0.41 ± 3% perf-profile.self.cycles-pp.menu_select
8096 ± 28% +278.9% 30679 ± 20% sched_debug.cfs_rq:/.MIN_vruntime.avg
158062 ± 17% +299.5% 631442 ± 8% sched_debug.cfs_rq:/.MIN_vruntime.max
32629 ± 22% +289.7% 127148 ± 13% sched_debug.cfs_rq:/.MIN_vruntime.stddev
44053 +192.3% 128774 sched_debug.cfs_rq:/.exec_clock.avg
140530 ± 9% +377.8% 671471 ± 29% sched_debug.cfs_rq:/.exec_clock.max
33108 ± 2% +109.9% 69484 ± 2% sched_debug.cfs_rq:/.exec_clock.min
20291 ± 7% +386.1% 98642 ± 21% sched_debug.cfs_rq:/.exec_clock.stddev
0.25 ± 82% -88.4% 0.03 ±131% sched_debug.cfs_rq:/.load_avg.min
8096 ± 28% +278.9% 30679 ± 20% sched_debug.cfs_rq:/.max_vruntime.avg
158062 ± 17% +299.5% 631442 ± 8% sched_debug.cfs_rq:/.max_vruntime.max
32629 ± 22% +289.7% 127148 ± 13% sched_debug.cfs_rq:/.max_vruntime.stddev
184170 +218.6% 586859 ± 2% sched_debug.cfs_rq:/.min_vruntime.avg
251516 ± 4% +254.5% 891702 ± 11% sched_debug.cfs_rq:/.min_vruntime.max
165614 +131.3% 383025 ± 2% sched_debug.cfs_rq:/.min_vruntime.min
15286 ± 9% +1057.3% 176903 ± 10% sched_debug.cfs_rq:/.min_vruntime.stddev
0.67 ± 6% +45.5% 0.98 ± 9% sched_debug.cfs_rq:/.nr_spread_over.avg
0.74 ± 20% +32.4% 0.98 ± 13% sched_debug.cfs_rq:/.nr_spread_over.stddev
3.43 ± 29% -60.7% 1.35 ±100% sched_debug.cfs_rq:/.removed.load_avg.avg
75.10 ± 34% -84.1% 11.91 ±100% sched_debug.cfs_rq:/.removed.load_avg.max
15.24 ± 23% -75.5% 3.74 ±100% sched_debug.cfs_rq:/.removed.load_avg.stddev
157.58 ± 29% -60.5% 62.19 ±100% sched_debug.cfs_rq:/.removed.runnable_sum.avg
3459 ± 34% -84.0% 552.75 ± 99% sched_debug.cfs_rq:/.removed.runnable_sum.max
700.79 ± 23% -75.4% 172.66 ±100% sched_debug.cfs_rq:/.removed.runnable_sum.stddev
34.81 ± 42% -87.0% 4.53 ±100% sched_debug.cfs_rq:/.removed.util_avg.max
5.89 ± 31% -79.0% 1.23 ±100% sched_debug.cfs_rq:/.removed.util_avg.stddev
59642 ± 58% +506.4% 361657 ± 61% sched_debug.cfs_rq:/.spread0.max
15286 ± 9% +1057.2% 176904 ± 10% sched_debug.cfs_rq:/.spread0.stddev
15.81 ± 15% +28.7% 20.34 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.avg
553.04 ± 15% +42.8% 789.60 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.max
83.37 ± 15% +37.7% 114.78 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev
130721 -17.9% 107295 sched_debug.cpu.avg_idle.avg
32650 ± 7% -16.9% 27145 ± 6% sched_debug.cpu.avg_idle.min
509212 +151.7% 1281639 sched_debug.cpu.clock.avg
509215 +151.7% 1281642 sched_debug.cpu.clock.max
509209 +151.7% 1281636 sched_debug.cpu.clock.min
509212 +151.7% 1281639 sched_debug.cpu.clock_task.avg
509215 +151.7% 1281642 sched_debug.cpu.clock_task.max
509209 +151.7% 1281636 sched_debug.cpu.clock_task.min
480.69 ± 2% +89.2% 909.42 ± 3% sched_debug.cpu.curr->pid.avg
14037 +135.4% 33040 sched_debug.cpu.curr->pid.max
2146 ± 2% +125.6% 4843 sched_debug.cpu.curr->pid.stddev
0.12 ± 5% +10.2% 0.13 ± 3% sched_debug.cpu.nr_running.avg
10174129 +208.2% 31360192 sched_debug.cpu.nr_switches.avg
10849045 +290.1% 42321541 ± 3% sched_debug.cpu.nr_switches.max
564008 ± 15% +1719.4% 10261545 ± 8% sched_debug.cpu.nr_switches.stddev
-87.16 +262.4% -315.83 sched_debug.cpu.nr_uninterruptible.min
66.84 ± 14% +143.1% 162.48 ± 21% sched_debug.cpu.nr_uninterruptible.stddev
10172460 +208.3% 31358619 sched_debug.cpu.sched_count.avg
10847113 +290.1% 42319816 ± 3% sched_debug.cpu.sched_count.max
563945 ± 15% +1719.6% 10261603 ± 8% sched_debug.cpu.sched_count.stddev
4957584 +211.1% 15425344 sched_debug.cpu.sched_goidle.avg
5286410 +291.7% 20709361 ± 3% sched_debug.cpu.sched_goidle.max
272603 ± 14% +1717.5% 4954528 ± 7% sched_debug.cpu.sched_goidle.stddev
5120024 +209.4% 15841794 sched_debug.cpu.ttwu_count.avg
58965381 ± 14% +567.2% 3.934e+08 ± 33% sched_debug.cpu.ttwu_count.max
132694 ± 2% -41.0% 78274 ± 5% sched_debug.cpu.ttwu_count.min
11266172 ± 8% +482.6% 65635267 ± 21% sched_debug.cpu.ttwu_count.stddev
114955 +26.1% 144930 ± 3% sched_debug.cpu.ttwu_local.max
83595 ± 4% -53.0% 39284 ± 30% sched_debug.cpu.ttwu_local.min
5967 ± 16% +460.9% 33468 ± 9% sched_debug.cpu.ttwu_local.stddev
509209 +151.7% 1281636 sched_debug.cpu_clk
506629 +152.5% 1279056 sched_debug.ktime
509627 +151.6% 1282053 sched_debug.sched_clk
18876 +138.7% 45051 ± 4% softirqs.CPU0.RCU
131251 ± 5% +153.2% 332324 ± 8% softirqs.CPU0.SCHED
327888 ± 3% +181.5% 923135 ± 10% softirqs.CPU0.TIMER
19003 ± 4% +121.7% 42132 ± 12% softirqs.CPU1.RCU
125587 ± 4% +130.3% 289267 ± 22% softirqs.CPU1.SCHED
331057 ± 2% +205.9% 1012636 ± 18% softirqs.CPU1.TIMER
19567 ± 5% +137.8% 46538 ± 2% softirqs.CPU10.RCU
131681 +161.3% 344047 softirqs.CPU10.SCHED
318774 +177.5% 884714 ± 8% softirqs.CPU10.TIMER
17876 ± 5% +158.8% 46258 softirqs.CPU11.RCU
128299 ± 4% +167.5% 343196 softirqs.CPU11.SCHED
320847 ± 3% +166.4% 854607 ± 8% softirqs.CPU11.TIMER
19129 ± 4% +121.1% 42292 ± 4% softirqs.CPU12.RCU
135251 +172.0% 367942 softirqs.CPU12.SCHED
310663 +201.8% 937514 ± 24% softirqs.CPU12.TIMER
17993 +143.9% 43882 ± 3% softirqs.CPU13.RCU
137747 ± 4% +157.5% 354715 ± 4% softirqs.CPU13.SCHED
359507 ± 23% +126.2% 813048 ± 13% softirqs.CPU13.TIMER
18173 +141.9% 43961 ± 3% softirqs.CPU14.RCU
134564 +163.8% 355019 ± 4% softirqs.CPU14.SCHED
310599 ± 2% +160.3% 808494 ± 13% softirqs.CPU14.TIMER
18180 +139.4% 43524 ± 3% softirqs.CPU15.RCU
134993 +160.8% 352061 ± 6% softirqs.CPU15.SCHED
308373 +164.7% 816152 ± 12% softirqs.CPU15.TIMER
19918 +144.6% 48716 softirqs.CPU16.RCU
134609 +162.6% 353418 ± 4% softirqs.CPU16.SCHED
307916 +163.0% 809772 ± 13% softirqs.CPU16.TIMER
19672 ± 2% +148.8% 48938 ± 3% softirqs.CPU17.RCU
132926 ± 2% +165.9% 353452 ± 4% softirqs.CPU17.SCHED
314316 +157.0% 807880 ± 13% softirqs.CPU17.TIMER
20021 ± 3% +141.2% 48298 ± 2% softirqs.CPU18.RCU
133347 ± 2% +164.0% 352038 ± 5% softirqs.CPU18.SCHED
312605 +158.6% 808358 ± 13% softirqs.CPU18.TIMER
19458 +139.3% 46570 ± 6% softirqs.CPU19.RCU
134001 +150.1% 335170 ± 14% softirqs.CPU19.SCHED
310521 +171.0% 841639 ± 14% softirqs.CPU19.TIMER
19131 ± 3% +138.3% 45586 ± 3% softirqs.CPU2.RCU
129111 ± 4% +154.4% 328419 ± 9% softirqs.CPU2.SCHED
329064 ± 3% +172.7% 897239 ± 10% softirqs.CPU2.TIMER
19249 ± 2% +145.5% 47257 ± 3% softirqs.CPU20.RCU
132532 +167.8% 354983 ± 3% softirqs.CPU20.SCHED
310659 +163.4% 818268 ± 13% softirqs.CPU20.TIMER
19473 +142.2% 47163 ± 4% softirqs.CPU21.RCU
133856 +166.0% 356005 ± 4% softirqs.CPU21.SCHED
308960 +164.3% 816637 ± 13% softirqs.CPU21.TIMER
19216 ± 2% +146.1% 47292 ± 3% softirqs.CPU22.RCU
133039 +165.4% 353140 ± 4% softirqs.CPU22.SCHED
307621 +166.0% 818260 ± 13% softirqs.CPU22.TIMER
19244 ± 2% +145.9% 47314 ± 2% softirqs.CPU23.RCU
133897 +163.1% 352326 ± 5% softirqs.CPU23.SCHED
310036 ± 2% +161.6% 811061 ± 13% softirqs.CPU23.TIMER
18954 ± 4% +143.9% 46235 ± 4% softirqs.CPU24.RCU
128754 ± 3% +157.8% 331927 ± 7% softirqs.CPU24.SCHED
323118 +169.7% 871609 ± 8% softirqs.CPU24.TIMER
18942 ± 6% +136.0% 44695 ± 3% softirqs.CPU25.RCU
129580 ± 3% +162.2% 339786 ± 3% softirqs.CPU25.SCHED
328907 +168.0% 881566 ± 7% softirqs.CPU25.TIMER
18913 ± 2% +147.2% 46757 ± 2% softirqs.CPU26.RCU
124714 ± 2% +176.4% 344692 softirqs.CPU26.SCHED
328463 ± 4% +170.2% 887463 ± 8% softirqs.CPU26.TIMER
19015 +137.2% 45107 ± 5% softirqs.CPU27.RCU
128551 ± 6% +152.0% 323916 ± 13% softirqs.CPU27.SCHED
325384 ± 2% +193.3% 954266 ± 13% softirqs.CPU27.TIMER
19094 +149.9% 47725 ± 2% softirqs.CPU28.RCU
131563 +161.9% 344579 ± 2% softirqs.CPU28.SCHED
319151 +176.7% 883105 ± 9% softirqs.CPU28.TIMER
19050 ± 4% +145.4% 46756 ± 2% softirqs.CPU29.RCU
124211 ± 6% +176.0% 342763 ± 2% softirqs.CPU29.SCHED
329526 ± 4% +158.9% 853085 ± 8% softirqs.CPU29.TIMER
22721 ± 8% +121.7% 50370 ± 4% softirqs.CPU3.RCU
132516 ± 2% +156.9% 340453 ± 4% softirqs.CPU3.SCHED
323552 ± 2% +174.3% 887612 ± 9% softirqs.CPU3.TIMER
19911 ± 3% +137.4% 47276 softirqs.CPU30.RCU
130778 +161.4% 341879 softirqs.CPU30.SCHED
317264 +181.0% 891407 ± 8% softirqs.CPU30.TIMER
19232 ± 4% +144.4% 47012 softirqs.CPU31.RCU
126109 ± 10% +174.9% 346628 ± 2% softirqs.CPU31.SCHED
329439 ± 4% +169.4% 887674 ± 9% softirqs.CPU31.TIMER
18139 ± 4% +141.4% 43795 ± 4% softirqs.CPU32.RCU
125566 ± 4% +170.9% 340112 ± 4% softirqs.CPU32.SCHED
328833 +184.0% 934027 ± 9% softirqs.CPU32.TIMER
19309 ± 10% +128.3% 44077 ± 3% softirqs.CPU33.RCU
128438 ± 3% +167.7% 343879 ± 2% softirqs.CPU33.SCHED
320949 +172.2% 873631 ± 9% softirqs.CPU33.TIMER
18041 ± 4% +146.1% 44399 ± 2% softirqs.CPU34.RCU
129791 +165.0% 343914 ± 2% softirqs.CPU34.SCHED
319403 ± 2% +177.3% 885711 ± 8% softirqs.CPU34.TIMER
17991 ± 5% +146.2% 44290 ± 2% softirqs.CPU35.RCU
137112 ± 7% +148.6% 340824 ± 3% softirqs.CPU35.SCHED
416552 ± 21% +119.5% 914435 ± 8% softirqs.CPU35.TIMER
20036 ± 4% +138.7% 47823 ± 4% softirqs.CPU36.RCU
138048 ± 3% +165.7% 366854 ± 7% softirqs.CPU36.SCHED
359442 ± 22% +196.1% 1064343 ± 19% softirqs.CPU36.TIMER
19421 ± 5% +153.4% 49203 ± 2% softirqs.CPU37.RCU
135462 +158.7% 350500 ± 6% softirqs.CPU37.SCHED
305531 +166.8% 815287 ± 12% softirqs.CPU37.TIMER
19963 +150.5% 50014 softirqs.CPU38.RCU
132257 +167.5% 353848 ± 5% softirqs.CPU38.SCHED
311979 ± 2% +160.3% 812118 ± 12% softirqs.CPU38.TIMER
20119 ± 2% +143.6% 49011 ± 2% softirqs.CPU39.RCU
135450 +161.9% 354713 ± 4% softirqs.CPU39.SCHED
307797 +164.3% 813377 ± 12% softirqs.CPU39.TIMER
20593 ± 9% +132.1% 47796 ± 5% softirqs.CPU4.RCU
129881 +159.2% 336643 ± 3% softirqs.CPU4.SCHED
321940 +175.0% 885297 ± 8% softirqs.CPU4.TIMER
19874 ± 2% +149.6% 49611 softirqs.CPU40.RCU
134660 +161.9% 352636 ± 5% softirqs.CPU40.SCHED
309870 +160.6% 807374 ± 13% softirqs.CPU40.TIMER
19897 ± 2% +147.2% 49186 ± 2% softirqs.CPU41.RCU
131978 ± 2% +167.4% 352864 ± 5% softirqs.CPU41.SCHED
315198 +157.6% 812013 ± 12% softirqs.CPU41.TIMER
19988 ± 2% +143.9% 48749 ± 2% softirqs.CPU42.RCU
133326 ± 2% +165.6% 354135 ± 4% softirqs.CPU42.SCHED
313042 +159.5% 812298 ± 12% softirqs.CPU42.TIMER
19723 ± 2% +141.9% 47704 ± 4% softirqs.CPU43.RCU
131944 +165.7% 350566 ± 5% softirqs.CPU43.SCHED
314428 +162.9% 826595 ± 13% softirqs.CPU43.TIMER
19824 ± 2% +143.0% 48177 ± 4% softirqs.CPU44.RCU
132047 ± 2% +157.8% 340362 ± 12% softirqs.CPU44.SCHED
312615 ± 2% +165.2% 829094 ± 12% softirqs.CPU44.TIMER
20140 ± 2% +138.6% 48049 ± 7% softirqs.CPU45.RCU
133846 +151.7% 336901 ± 14% softirqs.CPU45.SCHED
309086 ± 2% +170.8% 837094 ± 12% softirqs.CPU45.TIMER
20103 ± 3% +138.8% 48009 ± 3% softirqs.CPU46.RCU
133860 +154.1% 340108 ± 11% softirqs.CPU46.SCHED
311323 +166.1% 828498 ± 13% softirqs.CPU46.TIMER
19628 ± 2% +149.0% 48881 ± 2% softirqs.CPU47.RCU
132835 ± 2% +164.6% 351487 ± 5% softirqs.CPU47.SCHED
309264 +162.8% 812666 ± 13% softirqs.CPU47.TIMER
19773 ± 2% +139.0% 47263 ± 2% softirqs.CPU5.RCU
130891 ± 2% +163.7% 345185 softirqs.CPU5.SCHED
323979 ± 2% +182.2% 914392 ± 8% softirqs.CPU5.TIMER
19904 ± 3% +141.4% 48045 ± 4% softirqs.CPU6.RCU
131836 +160.9% 344009 softirqs.CPU6.SCHED
316150 +181.7% 890614 ± 8% softirqs.CPU6.TIMER
19077 ± 3% +145.1% 46751 ± 3% softirqs.CPU7.RCU
127906 ± 2% +157.6% 329548 ± 5% softirqs.CPU7.SCHED
325087 +181.6% 915494 ± 9% softirqs.CPU7.TIMER
20183 ± 2% +114.9% 43380 ± 10% softirqs.CPU8.RCU
131974 +118.3% 288124 ± 30% softirqs.CPU8.SCHED
319626 +190.1% 927161 ± 13% softirqs.CPU8.TIMER
19292 ± 3% +137.6% 45839 ± 3% softirqs.CPU9.RCU
129715 +158.9% 335788 ± 4% softirqs.CPU9.SCHED
321209 +178.0% 893035 ± 8% softirqs.CPU9.TIMER
931061 +140.9% 2242787 softirqs.RCU
6322319 +160.8% 16487173 softirqs.SCHED
15417545 +169.7% 41581783 ± 8% softirqs.TIMER
853.75 ± 12% +139.9% 2048 ± 7% interrupts.35:PCI-MSI.2621441-edge.eth0-TxRx-0
586.00 ± 14% +130.1% 1348 ± 5% interrupts.36:PCI-MSI.2621442-edge.eth0-TxRx-1
651.00 ± 9% +136.3% 1538 ± 9% interrupts.37:PCI-MSI.2621443-edge.eth0-TxRx-2
565.25 ± 14% +129.7% 1298 interrupts.38:PCI-MSI.2621444-edge.eth0-TxRx-3
591.00 ± 14% +123.8% 1322 ± 2% interrupts.39:PCI-MSI.2621445-edge.eth0-TxRx-4
660.25 ± 13% +111.4% 1396 ± 7% interrupts.40:PCI-MSI.2621446-edge.eth0-TxRx-5
529.50 ± 4% +227.1% 1732 ± 14% interrupts.41:PCI-MSI.2621447-edge.eth0-TxRx-6
585.75 ± 16% +163.2% 1541 ± 10% interrupts.42:PCI-MSI.2621448-edge.eth0-TxRx-7
404968 +145.6% 994665 ± 3% interrupts.CAL:Function_call_interrupts
8396 ± 6% +133.7% 19627 ± 6% interrupts.CPU0.CAL:Function_call_interrupts
1988743 +155.3% 5077644 interrupts.CPU0.LOC:Local_timer_interrupts
8698 ± 5% +121.4% 19255 ± 2% interrupts.CPU1.CAL:Function_call_interrupts
1986989 +155.5% 5077206 interrupts.CPU1.LOC:Local_timer_interrupts
8487 ± 5% +132.3% 19713 ± 8% interrupts.CPU10.CAL:Function_call_interrupts
1988375 +155.4% 5077322 interrupts.CPU10.LOC:Local_timer_interrupts
8770 ± 5% +149.6% 21889 ± 3% interrupts.CPU11.CAL:Function_call_interrupts
1987964 +155.4% 5077484 interrupts.CPU11.LOC:Local_timer_interrupts
78678 ± 8% +541.4% 504605 ±122% interrupts.CPU11.RES:Rescheduling_interrupts
8731 ± 9% +119.5% 19167 ± 6% interrupts.CPU12.CAL:Function_call_interrupts
1988222 +155.4% 5077524 interrupts.CPU12.LOC:Local_timer_interrupts
7960 ± 3% +153.5% 20176 ± 9% interrupts.CPU13.CAL:Function_call_interrupts
1988667 +155.3% 5076359 interrupts.CPU13.LOC:Local_timer_interrupts
8518 ± 6% +155.0% 21723 ± 7% interrupts.CPU14.CAL:Function_call_interrupts
1987350 +155.5% 5077482 interrupts.CPU14.LOC:Local_timer_interrupts
37.00 ± 31% +124.3% 83.00 ± 24% interrupts.CPU14.TLB:TLB_shootdowns
8831 ± 4% +137.4% 20964 ± 10% interrupts.CPU15.CAL:Function_call_interrupts
1987501 +155.5% 5077129 interrupts.CPU15.LOC:Local_timer_interrupts
3708 ± 27% -63.0% 1370 ±103% interrupts.CPU15.NMI:Non-maskable_interrupts
3708 ± 27% -63.0% 1370 ±103% interrupts.CPU15.PMI:Performance_monitoring_interrupts
23.25 ± 24% +253.8% 82.25 ± 28% interrupts.CPU15.TLB:TLB_shootdowns
8708 ± 8% +132.7% 20268 ± 7% interrupts.CPU16.CAL:Function_call_interrupts
1988418 +155.3% 5076923 interrupts.CPU16.LOC:Local_timer_interrupts
8548 ± 7% +148.3% 21226 ± 7% interrupts.CPU17.CAL:Function_call_interrupts
1988416 +155.3% 5077317 interrupts.CPU17.LOC:Local_timer_interrupts
33.00 ± 18% +147.0% 81.50 ± 21% interrupts.CPU17.TLB:TLB_shootdowns
8456 ± 9% +157.0% 21731 ± 6% interrupts.CPU18.CAL:Function_call_interrupts
1988574 +155.3% 5077430 interrupts.CPU18.LOC:Local_timer_interrupts
30.25 ± 15% +181.8% 85.25 ± 25% interrupts.CPU18.TLB:TLB_shootdowns
8884 ± 6% +144.5% 21720 ± 3% interrupts.CPU19.CAL:Function_call_interrupts
1987747 +155.4% 5077260 interrupts.CPU19.LOC:Local_timer_interrupts
37.00 ± 19% +127.0% 84.00 ± 26% interrupts.CPU19.TLB:TLB_shootdowns
8298 ± 11% +145.3% 20352 ± 9% interrupts.CPU2.CAL:Function_call_interrupts
1988446 +155.4% 5077740 interrupts.CPU2.LOC:Local_timer_interrupts
8594 ± 5% +146.1% 21149 ± 5% interrupts.CPU20.CAL:Function_call_interrupts
1987855 +155.4% 5077226 interrupts.CPU20.LOC:Local_timer_interrupts
8376 ± 8% +151.7% 21084 ± 9% interrupts.CPU21.CAL:Function_call_interrupts
1988171 +155.4% 5077212 interrupts.CPU21.LOC:Local_timer_interrupts
41.25 ± 20% +106.1% 85.00 ± 27% interrupts.CPU21.TLB:TLB_shootdowns
9024 ± 2% +141.1% 21757 ± 3% interrupts.CPU22.CAL:Function_call_interrupts
1987613 +155.4% 5077125 interrupts.CPU22.LOC:Local_timer_interrupts
8095 ± 7% +154.9% 20633 ± 6% interrupts.CPU23.CAL:Function_call_interrupts
1989315 +155.2% 5077451 interrupts.CPU23.LOC:Local_timer_interrupts
853.75 ± 12% +139.9% 2048 ± 7% interrupts.CPU24.35:PCI-MSI.2621441-edge.eth0-TxRx-0
8463 ± 5% +147.0% 20901 ± 7% interrupts.CPU24.CAL:Function_call_interrupts
1988251 +155.4% 5077879 interrupts.CPU24.LOC:Local_timer_interrupts
74848 ± 8% +525.2% 467985 ±128% interrupts.CPU24.RES:Rescheduling_interrupts
586.00 ± 14% +130.1% 1348 ± 5% interrupts.CPU25.36:PCI-MSI.2621442-edge.eth0-TxRx-1
8156 ± 7% +170.0% 22019 ± 2% interrupts.CPU25.CAL:Function_call_interrupts
1988295 +155.3% 5077010 interrupts.CPU25.LOC:Local_timer_interrupts
930.75 ±123% +260.9% 3359 ± 15% interrupts.CPU25.NMI:Non-maskable_interrupts
930.75 ±123% +260.9% 3359 ± 15% interrupts.CPU25.PMI:Performance_monitoring_interrupts
72678 ± 7% +575.9% 491267 ±120% interrupts.CPU25.RES:Rescheduling_interrupts
651.00 ± 9% +136.3% 1538 ± 9% interrupts.CPU26.37:PCI-MSI.2621443-edge.eth0-TxRx-2
8538 ± 6% +141.5% 20625 ± 6% interrupts.CPU26.CAL:Function_call_interrupts
1988053 +155.4% 5077342 interrupts.CPU26.LOC:Local_timer_interrupts
75367 ± 8% +522.0% 468794 ±133% interrupts.CPU26.RES:Rescheduling_interrupts
565.25 ± 14% +129.7% 1298 interrupts.CPU27.38:PCI-MSI.2621444-edge.eth0-TxRx-3
8157 ± 5% +132.5% 18969 ± 3% interrupts.CPU27.CAL:Function_call_interrupts
1988437 +155.3% 5077428 interrupts.CPU27.LOC:Local_timer_interrupts
591.00 ± 14% +123.8% 1322 ± 2% interrupts.CPU28.39:PCI-MSI.2621445-edge.eth0-TxRx-4
8181 ± 6% +148.1% 20300 ± 3% interrupts.CPU28.CAL:Function_call_interrupts
1987807 +155.4% 5077738 interrupts.CPU28.LOC:Local_timer_interrupts
71071 ± 7% +581.0% 483987 ±128% interrupts.CPU28.RES:Rescheduling_interrupts
660.25 ± 13% +111.4% 1396 ± 7% interrupts.CPU29.40:PCI-MSI.2621446-edge.eth0-TxRx-5
8178 ± 5% +150.4% 20475 ± 7% interrupts.CPU29.CAL:Function_call_interrupts
1987852 +155.4% 5077382 interrupts.CPU29.LOC:Local_timer_interrupts
70205 ± 8% +585.5% 481286 ±127% interrupts.CPU29.RES:Rescheduling_interrupts
17.75 ± 36% +163.4% 46.75 ± 80% interrupts.CPU29.TLB:TLB_shootdowns
8684 ± 5% +151.7% 21860 ± 4% interrupts.CPU3.CAL:Function_call_interrupts
1988786 +155.3% 5077436 interrupts.CPU3.LOC:Local_timer_interrupts
80407 ± 4% +518.6% 497419 ±122% interrupts.CPU3.RES:Rescheduling_interrupts
529.50 ± 4% +227.1% 1732 ± 14% interrupts.CPU30.41:PCI-MSI.2621447-edge.eth0-TxRx-6
8415 ± 4% +138.0% 20031 ± 3% interrupts.CPU30.CAL:Function_call_interrupts
1988609 +155.3% 5077494 interrupts.CPU30.LOC:Local_timer_interrupts
74875 ± 8% +518.8% 463337 ±134% interrupts.CPU30.RES:Rescheduling_interrupts
585.75 ± 16% +163.2% 1541 ± 10% interrupts.CPU31.42:PCI-MSI.2621448-edge.eth0-TxRx-7
7917 +151.5% 19913 ± 3% interrupts.CPU31.CAL:Function_call_interrupts
1988677 +155.3% 5077502 interrupts.CPU31.LOC:Local_timer_interrupts
67596 ± 3% +596.5% 470844 ±134% interrupts.CPU31.RES:Rescheduling_interrupts
8230 ± 4% +161.9% 21551 ± 8% interrupts.CPU32.CAL:Function_call_interrupts
1988317 +155.4% 5077410 interrupts.CPU32.LOC:Local_timer_interrupts
70449 ± 5% +597.2% 491174 ±127% interrupts.CPU32.RES:Rescheduling_interrupts
8684 ± 6% +137.3% 20610 ± 6% interrupts.CPU33.CAL:Function_call_interrupts
1988440 +155.3% 5077281 interrupts.CPU33.LOC:Local_timer_interrupts
77401 ± 5% +495.8% 461151 ±134% interrupts.CPU33.RES:Rescheduling_interrupts
8412 ± 8% +150.3% 21061 ± 4% interrupts.CPU34.CAL:Function_call_interrupts
1988447 +155.4% 5077546 interrupts.CPU34.LOC:Local_timer_interrupts
75351 ± 8% +537.5% 480376 ±128% interrupts.CPU34.RES:Rescheduling_interrupts
7890 +147.7% 19543 ± 5% interrupts.CPU35.CAL:Function_call_interrupts
1988875 +155.3% 5077717 interrupts.CPU35.LOC:Local_timer_interrupts
70158 ± 2% +577.1% 475011 ±135% interrupts.CPU35.RES:Rescheduling_interrupts
7908 ± 2% +184.9% 22530 ± 3% interrupts.CPU36.CAL:Function_call_interrupts
1988403 +155.4% 5077694 interrupts.CPU36.LOC:Local_timer_interrupts
31.75 ± 20% +178.0% 88.25 ± 28% interrupts.CPU36.TLB:TLB_shootdowns
9078 ± 2% +133.8% 21229 ± 6% interrupts.CPU37.CAL:Function_call_interrupts
1986971 +155.5% 5076871 interrupts.CPU37.LOC:Local_timer_interrupts
34.75 ± 36% +154.7% 88.50 ± 25% interrupts.CPU37.TLB:TLB_shootdowns
8574 ± 7% +131.7% 19864 ± 3% interrupts.CPU38.CAL:Function_call_interrupts
1986780 +155.5% 5076769 interrupts.CPU38.LOC:Local_timer_interrupts
29.25 ± 35% +195.7% 86.50 ± 18% interrupts.CPU38.TLB:TLB_shootdowns
8184 ± 7% +158.0% 21112 ± 4% interrupts.CPU39.CAL:Function_call_interrupts
1986972 +155.5% 5077337 interrupts.CPU39.LOC:Local_timer_interrupts
8368 ± 8% +144.7% 20481 ± 12% interrupts.CPU4.CAL:Function_call_interrupts
1987774 +155.4% 5077506 interrupts.CPU4.LOC:Local_timer_interrupts
8165 ± 3% +163.7% 21528 ± 6% interrupts.CPU40.CAL:Function_call_interrupts
1987659 +155.4% 5077421 interrupts.CPU40.LOC:Local_timer_interrupts
32.00 ± 12% +154.7% 81.50 ± 29% interrupts.CPU40.TLB:TLB_shootdowns
8454 ± 6% +158.4% 21847 ± 8% interrupts.CPU41.CAL:Function_call_interrupts
1987504 +155.5% 5077467 interrupts.CPU41.LOC:Local_timer_interrupts
37.75 ± 45% +122.5% 84.00 ± 17% interrupts.CPU41.TLB:TLB_shootdowns
8529 ± 7% +136.4% 20164 ± 8% interrupts.CPU42.CAL:Function_call_interrupts
1988401 +155.4% 5077407 interrupts.CPU42.LOC:Local_timer_interrupts
32.75 ± 39% +171.8% 89.00 ± 37% interrupts.CPU42.TLB:TLB_shootdowns
8102 ± 5% +150.4% 20283 ± 9% interrupts.CPU43.CAL:Function_call_interrupts
1987414 +155.5% 5077903 interrupts.CPU43.LOC:Local_timer_interrupts
32.50 ± 18% +150.8% 81.50 ± 34% interrupts.CPU43.TLB:TLB_shootdowns
8206 ± 7% +157.3% 21112 ± 8% interrupts.CPU44.CAL:Function_call_interrupts
1987278 +155.5% 5077309 interrupts.CPU44.LOC:Local_timer_interrupts
37.00 ± 25% +119.6% 81.25 ± 24% interrupts.CPU44.TLB:TLB_shootdowns
8467 ± 7% +139.3% 20261 ± 5% interrupts.CPU45.CAL:Function_call_interrupts
1987858 +155.4% 5077234 interrupts.CPU45.LOC:Local_timer_interrupts
38.25 ± 22% +113.1% 81.50 ± 27% interrupts.CPU45.TLB:TLB_shootdowns
7773 ± 4% +162.0% 20367 ± 9% interrupts.CPU46.CAL:Function_call_interrupts
1988584 +155.3% 5077417 interrupts.CPU46.LOC:Local_timer_interrupts
37.50 ± 14% +142.7% 91.00 ± 34% interrupts.CPU46.TLB:TLB_shootdowns
8746 ± 4% +138.7% 20880 ± 8% interrupts.CPU47.CAL:Function_call_interrupts
1988495 +155.3% 5077204 interrupts.CPU47.LOC:Local_timer_interrupts
8722 ± 7% +134.1% 20414 ± 10% interrupts.CPU5.CAL:Function_call_interrupts
1987875 +155.4% 5077418 interrupts.CPU5.LOC:Local_timer_interrupts
8435 ± 8% +143.9% 20570 ± 11% interrupts.CPU6.CAL:Function_call_interrupts
1988389 +155.4% 5077630 interrupts.CPU6.LOC:Local_timer_interrupts
8837 ± 5% +139.1% 21133 ± 9% interrupts.CPU7.CAL:Function_call_interrupts
1987853 +155.4% 5077370 interrupts.CPU7.LOC:Local_timer_interrupts
8852 ± 5% +123.8% 19810 ± 3% interrupts.CPU8.CAL:Function_call_interrupts
1988702 +155.3% 5077569 interrupts.CPU8.LOC:Local_timer_interrupts
8273 ± 6% +151.1% 20771 ± 9% interrupts.CPU9.CAL:Function_call_interrupts
1988033 +155.4% 5077588 interrupts.CPU9.LOC:Local_timer_interrupts
70739 ± 6% +578.9% 480243 ±129% interrupts.CPU9.RES:Rescheduling_interrupts
95428175 +155.4% 2.437e+08 interrupts.LOC:Local_timer_interrupts
144.00 +133.3% 336.00 interrupts.MCP:Machine_check_polls
6773889 ± 8% +482.9% 39483682 ± 7% interrupts.RES:Rescheduling_interrupts
1544 ± 5% +91.5% 2957 ± 11% interrupts.TLB:TLB_shootdowns
aim7.jobs-per-min
1300 +-+------------------------------------------------------------------+
1200 +-+.++.+.+.+.+. +.+.+.+. +.+.+ +.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| + + + |
1100 +-+ |
1000 +-+ |
| |
900 +-+ |
800 +-+ |
700 +-+ |
| |
600 +-+ |
500 +-+ O |
| O O O O O O O |
400 O-O OO O O O O OO O O O OO O O O O OO O O O O |
300 +-+------------------------------------------------------------------+
aim7.time.user_time
55 +-+--------------------------------------------------------------------+
| |
50 +-+ O |
| O |
| |
45 +-+ |
| O O O O O O O O O O |
40 O-O O OO O O O O O O O O O OO O O O O O |
| |
35 +-+ |
| |
| |
30 +-+ |
|.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.++.+. .+.+.+.+.++.+.+.|
25 +-+--------------------------------------------------------------------+
aim7.time.system_time
11000 +-+-----------------------------------------------------------------+
O O OO O O O OO O O OO O O O OO O O O OO O O |
10000 +-+ O OO O O OO O |
9000 +-+ |
| |
8000 +-+ |
| |
7000 +-+ |
| |
6000 +-+ |
5000 +-+ |
| |
4000 +-+ |
| .++. .+. .+.+. .+.+. .++. .+ .+.+.+ .+. .++.+.|
3000 +-+-----------------------------------------------------------------+
aim7.time.elapsed_time
3500 +-+------------------------------------------------------------------+
O O OO O O O O OO O O OO O O O O OO O O O O |
3000 +-+ O |
| |
| O O O O O O |
2500 +-+ OO |
| |
2000 +-+ |
| |
1500 +-+ |
| |
| |
1000 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| |
500 +-+------------------------------------------------------------------+
aim7.time.elapsed_time.max
3500 +-+------------------------------------------------------------------+
O O OO O O O O OO O O OO O O O O OO O O O O |
3000 +-+ O |
| |
| O O O O O O |
2500 +-+ OO |
| |
2000 +-+ |
| |
1500 +-+ |
| |
| |
1000 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| |
500 +-+------------------------------------------------------------------+
aim7.time.voluntary_context_switches
1.8e+09 +-+---------------------------------------------------------------+
O O O O OO O OO O O OO O O OO O OO O O |
1.6e+09 +-O O O O O O OO O O O |
| |
1.4e+09 +-+ |
| |
1.2e+09 +-+ |
| |
1e+09 +-+ |
| |
8e+08 +-+ |
| |
6e+08 +-+ |
|.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|
4e+08 +-+---------------------------------------------------------------+
aim7.time.file_system_outputs
1.8e+08 +-+---------------------------------------------------------------+
O OO O O OO O O OO O O OO O O OO O OO O O OO O O OO O O O |
1.7e+08 +-+ |
1.6e+08 +-+ |
| |
1.5e+08 +-+ |
| |
1.4e+08 +-+ |
| |
1.3e+08 +-+ |
1.2e+08 +-+ |
| |
1.1e+08 +-+ |
|.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|
1e+08 +-+---------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
6 months, 2 weeks
- 3
- 3
[btrfs] 8d47a0d8f7: fio.write_bw_MBps -28.6% regression
by kernel test robot
Greeting,
FYI, we noticed a -28.6% regression of fio.write_bw_MBps due to commit:
commit: 8d47a0d8f7947422dd359ac8e462687f81a7a137 ("btrfs: Do mandatory tree block check before submitting bio")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: fio-basic
on test machine: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory
with following parameters:
disk: 2pmem
fs: btrfs
runtime: 200s
nr_task: 50%
time_based: tb
rw: randwrite
bs: 4k
ioengine: libaio
test_size: 100G
cpufreq_governor: performance
ucode: 0x3d
test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
4k/gcc-7/performance/2pmem/btrfs/libaio/x86_64-rhel-7.6/50%/debian-x86_64-2018-04-03.cgz/200s/randwrite/lkp-hsw-ep2/100G/fio-basic/tb/0x3d
commit:
ff2ac107fa ("btrfs: tree-checker: Remove comprehensive root owner check")
8d47a0d8f7 ("btrfs: Do mandatory tree block check before submitting bio")
ff2ac107fae2440b 8d47a0d8f7947422dd359ac8e46
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.BUG:unable_to_handle_kernel
:4 25% 1:4 dmesg.Kernel_panic-not_syncing:Fatal_exception_in_interrupt
:4 25% 1:4 dmesg.Oops:#[##]
:4 25% 1:4 dmesg.RIP:cpuidle_enter_state
:4 25% 1:4 dmesg.RIP:native_write_msr
:4 25% 1:4 dmesg.RIP:perf_prepare_sample
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
0.01 ± 21% +0.3 0.28 ± 20% fio.latency_1000ms%
4.14 ± 13% +1.8 5.95 ± 9% fio.latency_10ms%
8.73 ± 15% +3.0 11.68 ± 8% fio.latency_20ms%
1.91 ± 5% -0.4 1.52 ± 3% fio.latency_250ms%
66.81 -5.9 60.90 ± 2% fio.latency_4ms%
0.18 ± 15% +0.1 0.29 ± 16% fio.latency_750us%
1.017e+08 -28.5% 72701525 fio.time.file_system_outputs
109.75 -26.8% 80.33 ± 2% fio.time.percent_of_cpu_this_job_got
197.97 -26.4% 145.79 ± 2% fio.time.system_time
38.66 ± 4% -24.0% 29.39 ± 3% fio.time.user_time
12388015 -28.5% 8858878 fio.time.voluntary_context_switches
12711059 -28.5% 9083954 fio.workload
247.91 -28.6% 176.96 fio.write_bw_MBps
309248 ± 2% +71.7% 531114 ± 4% fio.write_clat_99%_us
17574 +40.0% 24609 fio.write_clat_mean_us
57989 ± 4% +82.2% 105661 fio.write_clat_stddev
63463 -28.6% 45302 fio.write_iops
564.61 +40.2% 791.54 fio.write_slat_mean_us
9375 ± 7% +96.3% 18406 fio.write_slat_stddev
21.25 +8.5% 23.06 ± 9% boot-time.dhcp
93.32 +1.4% 94.58 iostat.cpu.idle
6.43 -18.6% 5.23 ± 2% iostat.cpu.system
6.48 -1.2 5.27 ± 2% mpstat.cpu.all.sys%
0.24 ± 5% -0.1 0.18 ± 2% mpstat.cpu.all.usr%
93.00 +1.1% 94.00 vmstat.cpu.id
591200 -30.0% 413921 vmstat.io.bo
31653776 ± 2% -10.9% 28196588 ± 2% vmstat.memory.cache
11927341 ± 7% +29.8% 15486805 ± 4% vmstat.memory.free
5.00 -26.7% 3.67 ± 12% vmstat.procs.r
635910 ± 2% -27.6% 460384 vmstat.system.cs
7822384 ± 2% -25.0% 5870498 numa-numastat.node0.local_node
7839861 ± 2% -24.8% 5893785 numa-numastat.node0.numa_hit
224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node0.numa_miss
242465 ± 44% -84.3% 37964 ± 18% numa-numastat.node0.other_node
7762623 ± 12% -26.8% 5681358 ± 6% numa-numastat.node1.local_node
224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node1.numa_foreign
7768618 ± 12% -26.9% 5681573 ± 6% numa-numastat.node1.numa_hit
9.429e+08 ± 10% -35.4% 6.094e+08 cpuidle.C1.time
53504154 ± 6% -31.8% 36504731 cpuidle.C1.usage
1.093e+09 ± 22% -50.3% 5.435e+08 ± 3% cpuidle.C1E.time
16541925 ± 13% -45.0% 9103775 cpuidle.C1E.usage
5.366e+09 ± 36% +71.7% 9.214e+09 ± 18% cpuidle.C3.time
16163664 ± 33% +57.0% 25379738 ± 5% cpuidle.C3.usage
9697055 ± 16% -35.8% 6225168 ± 19% cpuidle.C6.usage
19313595 ± 17% -16.3% 16166123 cpuidle.POLL.time
10291838 ± 9% -26.8% 7537045 ± 7% meminfo.Active
10034782 ± 9% -27.4% 7280710 ± 7% meminfo.Active(file)
31524905 ± 2% -10.9% 28082249 ± 2% meminfo.Cached
115157 ± 4% +61.1% 185560 ± 3% meminfo.CmaFree
6430720 ± 8% +19.1% 7660885 ± 6% meminfo.DirectMap2M
534230 ± 5% -12.8% 465932 ± 6% meminfo.Dirty
11737322 ± 7% +30.2% 15281189 ± 4% meminfo.MemFree
37604344 ± 2% -9.4% 34060483 meminfo.Memused
806.50 ± 37% +137.4% 1914 ± 8% meminfo.Writeback
370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.active_objs
370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.num_objs
15147 ± 5% -8.3% 13897 ± 2% slabinfo.blkdev_ioc.active_objs
17839 ± 6% -11.7% 15745 ± 2% slabinfo.blkdev_ioc.num_objs
12178 ± 4% -7.4% 11275 ± 2% slabinfo.kmalloc-512.num_objs
7794 ± 3% +25.4% 9774 slabinfo.proc_inode_cache.active_objs
8017 ± 2% +23.4% 9895 slabinfo.proc_inode_cache.num_objs
2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.active_objs
2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.num_objs
263.75 ± 2% -15.1% 224.00 ± 2% turbostat.Avg_MHz
9.46 ± 2% -1.4 8.07 ± 2% turbostat.Busy%
53502248 ± 6% -31.8% 36503898 turbostat.C1
5.86 ± 8% -2.0 3.82 turbostat.C1%
16540396 ± 13% -45.0% 9102733 turbostat.C1E
6.80 ± 22% -3.4 3.40 ± 4% turbostat.C1E%
16163332 ± 33% +57.0% 25379351 ± 5% turbostat.C3
33.55 ± 36% +24.1 57.61 ± 18% turbostat.C3%
9689445 ± 16% -35.8% 6221033 ± 19% turbostat.C6
18.04 ± 26% +49.1% 26.90 ± 9% turbostat.CPU%c3
53.25 ± 3% +10.8% 59.00 ± 4% turbostat.CoreTmp
4.67 ± 14% +62.7% 7.59 ± 26% turbostat.Pkg%pc2
56.50 +9.7% 62.00 ± 2% turbostat.PkgTmp
11.84 -4.8% 11.28 turbostat.RAMWatt
7232478 ± 10% -37.2% 4543037 ± 22% numa-meminfo.node0.Active
7072564 ± 10% -37.6% 4410135 ± 23% numa-meminfo.node0.Active(file)
308167 ± 6% -24.7% 232152 ± 10% numa-meminfo.node0.Dirty
8784028 ± 13% +21.7% 10686327 ± 2% numa-meminfo.node0.Inactive
8669548 ± 12% +20.7% 10465444 ± 2% numa-meminfo.node0.Inactive(file)
572.50 ± 22% +115.9% 1236 ± 25% numa-meminfo.node0.Writeback
15098140 ± 3% -17.7% 12425670 ± 3% numa-meminfo.node1.FilePages
11599549 ± 7% -22.3% 9018051 ± 4% numa-meminfo.node1.Inactive
105067 ± 92% -99.3% 725.33 ± 51% numa-meminfo.node1.Inactive(anon)
11494482 ± 7% -21.6% 9017325 ± 4% numa-meminfo.node1.Inactive(file)
5504 ± 9% -12.5% 4819 ± 2% numa-meminfo.node1.KernelStack
6709938 ± 11% +45.8% 9781880 ± 5% numa-meminfo.node1.MemFree
18029136 ± 4% -17.0% 14957200 ± 3% numa-meminfo.node1.MemUsed
4427 ± 69% -77.5% 996.00 ± 8% numa-meminfo.node1.PageTables
857660 ± 32% -37.2% 538646 ± 8% numa-meminfo.node1.SUnreclaim
109232 ± 88% -96.1% 4274 ± 10% numa-meminfo.node1.Shmem
986740 ± 35% -40.0% 592452 ± 10% numa-meminfo.node1.Slab
575.21 ± 15% +30.1% 748.58 ± 2% sched_debug.cfs_rq:/.runnable_load_avg.max
100.45 ± 15% +28.6% 129.19 ± 5% sched_debug.cfs_rq:/.runnable_load_avg.stddev
830.19 ± 9% +15.7% 960.25 ± 6% sched_debug.cfs_rq:/.util_avg.max
29.97 ± 16% -25.2% 22.43 ± 31% sched_debug.cfs_rq:/.util_est_enqueued.avg
591.17 ± 13% -22.2% 459.83 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.max
107.52 ± 13% -23.0% 82.84 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.stddev
2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock.stddev
2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock_task.stddev
24.51 ± 7% +21.5% 29.77 ± 8% sched_debug.cpu.cpu_load[0].avg
541.29 ± 15% +38.3% 748.58 ± 2% sched_debug.cpu.cpu_load[0].max
93.97 ± 7% +27.8% 120.09 ± 2% sched_debug.cpu.cpu_load[0].stddev
519.08 ± 6% +40.5% 729.42 ± 4% sched_debug.cpu.cpu_load[1].max
87.45 ± 5% +28.5% 112.39 ± 4% sched_debug.cpu.cpu_load[1].stddev
480.83 ± 7% +39.5% 670.92 ± 13% sched_debug.cpu.cpu_load[2].max
80.92 ± 5% +25.7% 101.75 ± 8% sched_debug.cpu.cpu_load[2].stddev
408.83 ± 4% +54.0% 629.58 ± 19% sched_debug.cpu.cpu_load[3].max
71.62 ± 2% +30.7% 93.61 ± 11% sched_debug.cpu.cpu_load[3].stddev
381.50 ± 8% +66.7% 636.00 ± 18% sched_debug.cpu.cpu_load[4].max
64.40 ± 6% +39.7% 89.96 ± 12% sched_debug.cpu.cpu_load[4].stddev
106.75 ± 20% -88.4% 12.33 ± 10% proc-vmstat.kswapd_high_wmark_hit_quickly
2508660 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_active_file
33315360 -30.4% 23192301 proc-vmstat.nr_dirtied
133447 ± 5% -12.6% 116580 ± 6% proc-vmstat.nr_dirty
7881043 ± 2% -10.9% 7019533 ± 2% proc-vmstat.nr_file_pages
28790 ± 4% +61.1% 46392 ± 3% proc-vmstat.nr_free_cma
2934503 ± 7% +30.2% 3821369 ± 4% proc-vmstat.nr_free_pages
5044168 -3.4% 4872965 proc-vmstat.nr_inactive_file
430282 -6.0% 404589 proc-vmstat.nr_slab_unreclaimable
197.75 ± 38% +136.5% 467.67 ± 10% proc-vmstat.nr_writeback
33243480 -30.4% 23143756 proc-vmstat.nr_written
2508665 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_zone_active_file
5044399 -3.4% 4873026 proc-vmstat.nr_zone_inactive_file
134972 ± 5% -12.4% 118262 ± 5% proc-vmstat.nr_zone_write_pending
15632301 ± 5% -25.8% 11598535 ± 3% proc-vmstat.numa_hit
15608826 ± 5% -25.8% 11575029 ± 3% proc-vmstat.numa_local
2300530 ± 10% -31.1% 1585363 ± 10% proc-vmstat.pgactivate
16955745 -26.3% 12499611 ± 2% proc-vmstat.pgalloc_normal
17488000 -25.9% 12959812 ± 2% proc-vmstat.pgfree
1.33e+08 -30.4% 92575440 proc-vmstat.pgpgout
5728177 ± 6% -78.5% 1232433 ± 25% proc-vmstat.pgscan_kswapd
5727967 ± 6% -78.5% 1232244 ± 25% proc-vmstat.pgsteal_kswapd
32760 -7.6% 30285 proc-vmstat.slabs_scanned
1768211 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_active_file
10410188 ± 7% -32.5% 7023939 ± 7% numa-vmstat.node0.nr_dirtied
76995 ± 6% -24.6% 58035 ± 10% numa-vmstat.node0.nr_dirty
2167457 ± 12% +20.7% 2616147 ± 2% numa-vmstat.node0.nr_inactive_file
142.00 ± 24% +120.4% 313.00 ± 24% numa-vmstat.node0.nr_writeback
10308758 ± 7% -32.6% 6949497 ± 7% numa-vmstat.node0.nr_written
1768216 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_zone_active_file
2167563 ± 12% +20.7% 2616189 ± 2% numa-vmstat.node0.nr_zone_inactive_file
77786 ± 7% -24.3% 58899 ± 10% numa-vmstat.node0.nr_zone_write_pending
5889240 ± 3% -14.3% 5046874 ± 3% numa-vmstat.node0.numa_hit
5837851 ± 4% -14.0% 5023120 ± 3% numa-vmstat.node0.numa_local
68376 ± 48% -96.9% 2149 ± 46% numa-vmstat.node0.numa_miss
119766 ± 42% -78.4% 25904 ± 3% numa-vmstat.node0.numa_other
30.50 ± 38% -100.0% 0.00 numa-vmstat.node0.workingset_nodes
6829126 ± 6% -22.8% 5269898 ± 10% numa-vmstat.node1.nr_dirtied
3774645 ± 3% -17.7% 3106081 ± 3% numa-vmstat.node1.nr_file_pages
28796 ± 4% +61.1% 46405 ± 3% numa-vmstat.node1.nr_free_cma
1677332 ± 11% +45.8% 2445778 ± 5% numa-vmstat.node1.nr_free_pages
26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_inactive_anon
2873702 ± 7% -21.6% 2254116 ± 4% numa-vmstat.node1.nr_inactive_file
5506 ± 9% -12.5% 4820 ± 2% numa-vmstat.node1.nr_kernel_stack
29031 ± 85% -88.2% 3436 numa-vmstat.node1.nr_mapped
1106 ± 69% -77.5% 249.00 ± 8% numa-vmstat.node1.nr_page_table_pages
27308 ± 88% -96.1% 1068 ± 10% numa-vmstat.node1.nr_shmem
214418 ± 32% -37.2% 134663 ± 8% numa-vmstat.node1.nr_slab_unreclaimable
75.00 ± 35% +145.3% 184.00 ± 18% numa-vmstat.node1.nr_writeback
6761224 ± 6% -23.1% 5202586 ± 10% numa-vmstat.node1.nr_written
26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_zone_inactive_anon
2873810 ± 7% -21.6% 2254135 ± 4% numa-vmstat.node1.nr_zone_inactive_file
68381 ± 48% -96.9% 2152 ± 46% numa-vmstat.node1.numa_foreign
5035283 ± 15% -26.9% 3679318 ± 7% numa-vmstat.node1.numa_hit
4905249 ± 15% -28.2% 3522229 ± 8% numa-vmstat.node1.numa_local
18.43 ± 6% -28.3% 13.21 ± 3% perf-stat.i.MPKI
2.205e+09 ± 3% +10.5% 2.437e+09 perf-stat.i.branch-instructions
1.83 ± 5% -0.6 1.24 ± 7% perf-stat.i.branch-miss-rate%
37019551 ± 3% -29.4% 26140844 ± 5% perf-stat.i.branch-misses
28932020 ± 6% -31.1% 19933819 ± 3% perf-stat.i.cache-misses
2.03e+08 ± 4% -24.8% 1.526e+08 perf-stat.i.cache-references
645248 ± 2% -27.4% 468301 perf-stat.i.context-switches
1.97 ± 7% -22.1% 1.53 ± 2% perf-stat.i.cpi
1.983e+10 ± 3% -15.1% 1.684e+10 perf-stat.i.cpu-cycles
1.03 ± 5% -0.4 0.64 ± 10% perf-stat.i.dTLB-load-miss-rate%
31879502 ± 6% -37.9% 19807598 ± 9% perf-stat.i.dTLB-load-misses
3.201e+09 ± 3% +18.8% 3.804e+09 perf-stat.i.dTLB-loads
0.13 ± 7% -0.0 0.09 ± 10% perf-stat.i.dTLB-store-miss-rate%
2516181 ± 10% -28.3% 1805272 ± 10% perf-stat.i.dTLB-store-misses
2.143e+09 ± 3% +19.8% 2.568e+09 ± 2% perf-stat.i.dTLB-stores
13781489 ± 3% -28.4% 9866239 ± 3% perf-stat.i.iTLB-loads
1.164e+10 ± 3% +21.6% 1.415e+10 perf-stat.i.instructions
8435 ± 17% +42.5% 12023 ± 14% perf-stat.i.instructions-per-iTLB-miss
0.57 ± 3% +43.6% 0.82 perf-stat.i.ipc
16050361 ± 9% -30.7% 11127721 ± 3% perf-stat.i.node-load-misses
8120610 ± 9% -31.3% 5580996 perf-stat.i.node-loads
2591649 ± 13% -31.1% 1785932 ± 6% perf-stat.i.node-store-misses
1809775 ± 12% -31.2% 1244360 ± 2% perf-stat.i.node-stores
17.45 ± 4% -38.2% 10.79 perf-stat.overall.MPKI
1.68 ± 3% -0.6 1.07 ± 5% perf-stat.overall.branch-miss-rate%
1.70 ± 3% -30.2% 1.19 perf-stat.overall.cpi
689.69 ± 9% +22.6% 845.48 ± 2% perf-stat.overall.cycles-between-cache-misses
0.99 ± 5% -0.5 0.52 ± 10% perf-stat.overall.dTLB-load-miss-rate%
0.12 ± 8% -0.0 0.07 ± 10% perf-stat.overall.dTLB-store-miss-rate%
10.81 ± 14% +2.3 13.09 ± 10% perf-stat.overall.iTLB-load-miss-rate%
7119 ± 14% +35.9% 9679 ± 13% perf-stat.overall.instructions-per-iTLB-miss
0.59 ± 3% +43.1% 0.84 perf-stat.overall.ipc
202912 ± 2% +68.8% 342615 ± 3% perf-stat.overall.path-length
2.195e+09 ± 3% +10.5% 2.425e+09 perf-stat.ps.branch-instructions
36847010 ± 3% -29.4% 26015686 ± 5% perf-stat.ps.branch-misses
28799092 ± 6% -31.1% 19837306 ± 3% perf-stat.ps.cache-misses
2.021e+08 ± 4% -24.8% 1.519e+08 perf-stat.ps.cache-references
642266 ± 2% -27.4% 465968 perf-stat.ps.context-switches
1.974e+10 ± 3% -15.1% 1.676e+10 perf-stat.ps.cpu-cycles
31729472 ± 6% -37.9% 19715594 ± 9% perf-stat.ps.dTLB-load-misses
3.186e+09 ± 3% +18.8% 3.785e+09 perf-stat.ps.dTLB-loads
2504443 ± 10% -28.2% 1797154 ± 10% perf-stat.ps.dTLB-store-misses
2.133e+09 ± 3% +19.8% 2.556e+09 ± 2% perf-stat.ps.dTLB-stores
13717699 ± 3% -28.4% 9816979 ± 3% perf-stat.ps.iTLB-loads
1.159e+10 ± 3% +21.5% 1.408e+10 perf-stat.ps.instructions
15979371 ± 9% -30.7% 11073468 ± 3% perf-stat.ps.node-load-misses
8081205 ± 9% -31.3% 5554114 perf-stat.ps.node-loads
2580198 ± 13% -31.1% 1777169 ± 6% perf-stat.ps.node-store-misses
1801590 ± 12% -31.3% 1238321 ± 2% perf-stat.ps.node-stores
2.579e+12 +20.6% 3.111e+12 ± 2% perf-stat.total.instructions
36149 ± 4% -26.2% 26686 ± 5% softirqs.CPU0.RCU
37255 ± 3% -26.8% 27256 ± 2% softirqs.CPU1.RCU
37834 ± 5% -26.5% 27801 ± 9% softirqs.CPU10.RCU
39075 ± 3% -24.5% 29494 ± 2% softirqs.CPU11.RCU
38137 ± 6% -23.1% 29340 ± 4% softirqs.CPU12.RCU
37480 ± 3% -20.5% 29797 ± 2% softirqs.CPU13.RCU
37690 ± 5% -28.7% 26861 ± 4% softirqs.CPU14.RCU
33863 ± 9% -29.5% 23885 ± 5% softirqs.CPU15.RCU
34306 ± 9% -31.1% 23641 ± 12% softirqs.CPU16.RCU
34374 ± 9% -29.0% 24402 ± 7% softirqs.CPU17.RCU
44013 ± 9% -31.7% 30059 softirqs.CPU18.RCU
43114 ± 8% -29.9% 30219 softirqs.CPU19.RCU
36276 ± 8% -23.3% 27825 ± 5% softirqs.CPU2.RCU
67049 ± 7% +39.1% 93267 ± 18% softirqs.CPU2.TIMER
39721 ± 13% -31.4% 27233 ± 3% softirqs.CPU20.RCU
42340 ± 5% -28.6% 30215 ± 6% softirqs.CPU21.RCU
40896 ± 7% -30.3% 28506 ± 2% softirqs.CPU22.RCU
42422 ± 7% -33.2% 28357 ± 2% softirqs.CPU23.RCU
42330 ± 9% -33.5% 28150 ± 5% softirqs.CPU24.RCU
41406 ± 9% -31.4% 28398 softirqs.CPU25.RCU
39754 ± 5% -29.4% 28063 softirqs.CPU26.RCU
40207 ± 6% -27.9% 28995 ± 12% softirqs.CPU27.RCU
40351 ± 9% -29.4% 28487 ± 3% softirqs.CPU28.RCU
41880 ± 5% -29.6% 29472 softirqs.CPU29.RCU
36293 ± 4% -17.7% 29864 ± 7% softirqs.CPU3.RCU
39341 ± 9% -34.3% 25847 softirqs.CPU30.RCU
39687 ± 9% -33.5% 26400 ± 2% softirqs.CPU31.RCU
38062 ± 14% -39.9% 22878 ± 10% softirqs.CPU32.RCU
39769 ± 13% -33.8% 26314 ± 5% softirqs.CPU33.RCU
39504 ± 10% -37.2% 24824 ± 6% softirqs.CPU34.RCU
39591 ± 11% -38.6% 24300 ± 5% softirqs.CPU35.RCU
40398 ± 8% -27.0% 29488 ± 12% softirqs.CPU36.RCU
39464 ± 12% -25.3% 29478 ± 10% softirqs.CPU37.RCU
41177 ± 9% -34.0% 27190 ± 19% softirqs.CPU38.RCU
38734 ± 13% -24.8% 29145 ± 5% softirqs.CPU39.RCU
36804 ± 5% -23.0% 28354 softirqs.CPU4.RCU
40421 ± 12% -28.8% 28776 ± 6% softirqs.CPU40.RCU
40371 ± 11% -31.1% 27834 ± 8% softirqs.CPU41.RCU
40633 ± 10% -29.9% 28469 ± 10% softirqs.CPU42.RCU
41018 ± 9% -33.6% 27252 ± 6% softirqs.CPU43.RCU
38181 ± 12% -27.0% 27856 ± 10% softirqs.CPU44.RCU
39605 ± 12% -32.3% 26803 ± 12% softirqs.CPU45.RCU
42686 ± 11% -29.7% 30016 ± 9% softirqs.CPU47.RCU
40433 ± 9% -31.8% 27586 ± 11% softirqs.CPU48.RCU
38371 ± 12% -23.0% 29528 ± 7% softirqs.CPU49.RCU
37172 ± 2% -25.9% 27562 ± 3% softirqs.CPU5.RCU
39660 ± 12% -27.5% 28752 ± 11% softirqs.CPU50.RCU
40375 ± 10% -26.7% 29582 ± 3% softirqs.CPU51.RCU
40568 ± 9% -28.8% 28886 ± 6% softirqs.CPU52.RCU
39967 ± 10% -27.3% 29073 ± 11% softirqs.CPU53.RCU
40874 ± 11% -37.1% 25725 ± 6% softirqs.CPU54.RCU
41227 ± 14% -37.9% 25606 ± 6% softirqs.CPU55.RCU
38856 ± 13% -34.7% 25380 ± 8% softirqs.CPU56.RCU
39618 ± 13% -39.1% 24119 ± 4% softirqs.CPU57.RCU
39412 ± 12% -36.6% 24970 ± 3% softirqs.CPU58.RCU
41572 ± 14% -41.3% 24399 ± 10% softirqs.CPU59.RCU
36927 ± 3% -24.4% 27923 ± 22% softirqs.CPU6.RCU
37057 ± 10% -31.8% 25261 ± 2% softirqs.CPU60.RCU
36584 ± 8% -27.8% 26419 ± 2% softirqs.CPU61.RCU
35859 ± 4% -28.2% 25729 ± 5% softirqs.CPU62.RCU
35849 ± 7% -33.2% 23933 ± 5% softirqs.CPU63.RCU
35379 ± 6% -26.4% 26041 ± 4% softirqs.CPU64.RCU
36908 ± 6% -26.5% 27110 ± 2% softirqs.CPU65.RCU
37595 ± 3% -26.1% 27783 ± 4% softirqs.CPU66.RCU
36327 ± 2% -27.0% 26533 ± 2% softirqs.CPU67.RCU
35136 ± 7% -31.7% 23998 ± 9% softirqs.CPU68.RCU
35638 ± 4% -28.6% 25435 ± 4% softirqs.CPU69.RCU
37268 ± 6% -25.4% 27786 ± 4% softirqs.CPU7.RCU
36693 ± 4% -29.2% 25986 ± 5% softirqs.CPU70.RCU
36859 ± 3% -28.8% 26260 ± 4% softirqs.CPU71.RCU
37539 ± 7% -27.7% 27144 ± 4% softirqs.CPU8.RCU
37584 ± 4% -26.0% 27802 ± 15% softirqs.CPU9.RCU
2795313 ± 7% -29.7% 1964363 ± 4% softirqs.RCU
249.50 ± 76% -50.4% 123.67 ± 2% interrupts.41:PCI-MSI.1572870-edge.eth0-TxRx-6
154.50 ± 13% -10.9% 137.67 ± 10% interrupts.42:PCI-MSI.1572871-edge.eth0-TxRx-7
150.50 ± 43% -28.0% 108.33 interrupts.84:PCI-MSI.1572913-edge.eth0-TxRx-49
10393 ± 10% -26.4% 7653 ± 10% interrupts.CPU1.RES:Rescheduling_interrupts
12518 ± 15% -31.0% 8642 ± 21% interrupts.CPU10.RES:Rescheduling_interrupts
945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.NMI:Non-maskable_interrupts
945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.PMI:Performance_monitoring_interrupts
13068 ± 18% -30.6% 9075 ± 23% interrupts.CPU11.RES:Rescheduling_interrupts
11949 ± 11% -49.8% 6001 ± 29% interrupts.CPU12.RES:Rescheduling_interrupts
13711 ± 8% -19.4% 11052 ± 24% interrupts.CPU13.RES:Rescheduling_interrupts
13361 ± 6% -38.8% 8170 ± 25% interrupts.CPU14.RES:Rescheduling_interrupts
7628 ± 12% -29.2% 5401 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts
7909 ± 25% -30.4% 5501 ± 37% interrupts.CPU19.RES:Rescheduling_interrupts
1482 ± 39% -100.0% 0.00 interrupts.CPU2.NMI:Non-maskable_interrupts
1482 ± 39% -100.0% 0.00 interrupts.CPU2.PMI:Performance_monitoring_interrupts
2116 +23.2% 2606 ± 10% interrupts.CPU20.CAL:Function_call_interrupts
194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.NMI:Non-maskable_interrupts
194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.PMI:Performance_monitoring_interrupts
2096 ± 4% +18.0% 2474 ± 9% interrupts.CPU21.CAL:Function_call_interrupts
11313 ± 20% -56.4% 4935 ± 23% interrupts.CPU21.RES:Rescheduling_interrupts
2111 ± 3% +20.1% 2534 ± 11% interrupts.CPU22.CAL:Function_call_interrupts
2109 ± 4% +22.6% 2585 ± 10% interrupts.CPU23.CAL:Function_call_interrupts
10305 ± 16% -40.7% 6115 ± 17% interrupts.CPU23.RES:Rescheduling_interrupts
2128 ± 2% +17.8% 2507 ± 10% interrupts.CPU24.CAL:Function_call_interrupts
12271 ± 20% -39.0% 7488 ± 29% interrupts.CPU24.RES:Rescheduling_interrupts
2123 +19.7% 2541 ± 7% interrupts.CPU25.CAL:Function_call_interrupts
9780 ± 21% -32.9% 6563 ± 25% interrupts.CPU25.RES:Rescheduling_interrupts
980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.NMI:Non-maskable_interrupts
980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.PMI:Performance_monitoring_interrupts
11163 ± 28% -49.8% 5605 ± 13% interrupts.CPU27.RES:Rescheduling_interrupts
11071 ± 12% -56.3% 4835 ± 26% interrupts.CPU29.RES:Rescheduling_interrupts
728.50 ± 66% -100.0% 0.00 interrupts.CPU3.NMI:Non-maskable_interrupts
728.50 ± 66% -100.0% 0.00 interrupts.CPU3.PMI:Performance_monitoring_interrupts
10599 ± 10% -38.7% 6501 ± 16% interrupts.CPU31.RES:Rescheduling_interrupts
2213 ± 4% +14.2% 2527 ± 7% interrupts.CPU32.CAL:Function_call_interrupts
8862 ± 16% -39.6% 5354 ± 28% interrupts.CPU32.RES:Rescheduling_interrupts
2156 ± 6% +20.5% 2599 ± 9% interrupts.CPU33.CAL:Function_call_interrupts
2169 ± 5% +19.9% 2601 ± 9% interrupts.CPU34.CAL:Function_call_interrupts
10483 ± 18% -48.8% 5371 ± 33% interrupts.CPU34.RES:Rescheduling_interrupts
2049 +24.4% 2549 ± 8% interrupts.CPU35.CAL:Function_call_interrupts
10929 ± 15% -49.0% 5578 ± 14% interrupts.CPU35.RES:Rescheduling_interrupts
1993 ± 4% +28.5% 2561 ± 12% interrupts.CPU36.CAL:Function_call_interrupts
161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.NMI:Non-maskable_interrupts
161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.PMI:Performance_monitoring_interrupts
1987 ± 4% +30.5% 2594 ± 13% interrupts.CPU37.CAL:Function_call_interrupts
1863 ± 10% +35.7% 2529 ± 14% interrupts.CPU38.CAL:Function_call_interrupts
1984 ± 6% +26.5% 2509 ± 12% interrupts.CPU41.CAL:Function_call_interrupts
12830 ± 30% -26.4% 9445 ± 9% interrupts.CPU43.RES:Rescheduling_interrupts
2067 ± 2% +27.4% 2632 ± 9% interrupts.CPU44.CAL:Function_call_interrupts
2066 ± 3% +27.2% 2629 ± 9% interrupts.CPU45.CAL:Function_call_interrupts
2158 ± 3% +19.6% 2582 ± 9% interrupts.CPU46.CAL:Function_call_interrupts
2036 ± 6% +30.3% 2652 ± 10% interrupts.CPU47.CAL:Function_call_interrupts
150.50 ± 43% -28.0% 108.33 interrupts.CPU49.84:PCI-MSI.1572913-edge.eth0-TxRx-49
11823 ± 12% -36.9% 7460 ± 18% interrupts.CPU49.RES:Rescheduling_interrupts
1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.NMI:Non-maskable_interrupts
1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.PMI:Performance_monitoring_interrupts
11748 ± 22% -40.0% 7048 ± 27% interrupts.CPU5.RES:Rescheduling_interrupts
2102 ± 5% +19.6% 2515 ± 7% interrupts.CPU52.CAL:Function_call_interrupts
2115 ± 6% +19.5% 2527 ± 7% interrupts.CPU53.CAL:Function_call_interrupts
11318 ± 17% -20.4% 9004 ± 26% interrupts.CPU53.RES:Rescheduling_interrupts
2177 ± 5% +17.4% 2556 ± 9% interrupts.CPU54.CAL:Function_call_interrupts
10968 ± 23% -62.3% 4134 ± 21% interrupts.CPU54.RES:Rescheduling_interrupts
2161 ± 5% +20.2% 2598 ± 9% interrupts.CPU55.CAL:Function_call_interrupts
9334 ± 19% -55.4% 4158 ± 34% interrupts.CPU55.RES:Rescheduling_interrupts
1443 ± 22% -100.0% 0.00 interrupts.CPU56.NMI:Non-maskable_interrupts
1443 ± 22% -100.0% 0.00 interrupts.CPU56.PMI:Performance_monitoring_interrupts
11377 ± 31% -56.9% 4907 ± 43% interrupts.CPU56.RES:Rescheduling_interrupts
601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.NMI:Non-maskable_interrupts
601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.PMI:Performance_monitoring_interrupts
8295 ± 31% -47.7% 4340 ± 24% interrupts.CPU57.RES:Rescheduling_interrupts
9182 ± 37% -59.7% 3701 ± 46% interrupts.CPU58.RES:Rescheduling_interrupts
249.50 ± 76% -50.4% 123.67 ± 2% interrupts.CPU6.41:PCI-MSI.1572870-edge.eth0-TxRx-6
12473 ± 27% -44.4% 6940 ± 13% interrupts.CPU6.RES:Rescheduling_interrupts
7886 ± 12% -42.2% 4555 ± 31% interrupts.CPU60.RES:Rescheduling_interrupts
971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.NMI:Non-maskable_interrupts
971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.PMI:Performance_monitoring_interrupts
9253 ± 26% -56.4% 4033 ± 16% interrupts.CPU61.RES:Rescheduling_interrupts
2103 ± 5% +19.1% 2505 ± 11% interrupts.CPU63.CAL:Function_call_interrupts
2059 ± 2% +20.6% 2483 ± 10% interrupts.CPU64.CAL:Function_call_interrupts
8545 ± 17% -41.2% 5025 ± 8% interrupts.CPU65.RES:Rescheduling_interrupts
7378 ± 23% -51.0% 3614 ± 30% interrupts.CPU68.RES:Rescheduling_interrupts
8573 ± 28% -50.1% 4277 ± 6% interrupts.CPU69.RES:Rescheduling_interrupts
154.50 ± 13% -10.9% 137.67 ± 10% interrupts.CPU7.42:PCI-MSI.1572871-edge.eth0-TxRx-7
9910 ± 28% -51.2% 4839 ± 40% interrupts.CPU70.RES:Rescheduling_interrupts
8836 ± 26% -52.7% 4180 ± 6% interrupts.CPU71.RES:Rescheduling_interrupts
911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.NMI:Non-maskable_interrupts
911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.PMI:Performance_monitoring_interrupts
12326 ± 15% -32.3% 8350 ± 10% interrupts.CPU8.RES:Rescheduling_interrupts
1392 ± 58% -100.0% 0.00 interrupts.CPU9.NMI:Non-maskable_interrupts
1392 ± 58% -100.0% 0.00 interrupts.CPU9.PMI:Performance_monitoring_interrupts
60185 ± 19% -30.5% 41822 ± 11% interrupts.NMI:Non-maskable_interrupts
60185 ± 19% -30.5% 41822 ± 11% interrupts.PMI:Performance_monitoring_interrupts
749419 -31.9% 510225 ± 3% interrupts.RES:Rescheduling_interrupts
15.05 ± 6% -4.1 10.97 ± 20% perf-profile.calltrace.cycles-pp.normal_work_helper.process_one_work.worker_thread.kthread.ret_from_fork
12.66 ± 5% -4.0 8.68 ± 20% perf-profile.calltrace.cycles-pp.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread.kthread
5.99 ± 8% -1.6 4.37 ± 6% perf-profile.calltrace.cycles-pp.io_submit
5.83 ± 8% -1.6 4.25 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.io_submit
5.82 ± 8% -1.6 4.24 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.81 ± 8% -1.6 4.23 ± 6% perf-profile.calltrace.cycles-pp.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.76 ± 8% -1.6 4.19 ± 6% perf-profile.calltrace.cycles-pp.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.65 ± 6% -1.6 4.08 ± 20% perf-profile.calltrace.cycles-pp.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread
4.52 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread
6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback
6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes
5.58 ± 8% -1.5 4.07 ± 6% perf-profile.calltrace.cycles-pp.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.50 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work
5.44 ± 8% -1.5 3.96 ± 7% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64
6.63 ± 8% -1.5 5.17 ± 10% perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode
5.34 ± 9% -1.4 3.90 ± 6% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit
5.55 ± 8% -1.2 4.32 ± 10% perf-profile.calltrace.cycles-pp.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
4.98 ± 8% -1.1 3.86 ± 10% perf-profile.calltrace.cycles-pp.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages
5.00 ± 8% -1.1 3.88 ± 10% perf-profile.calltrace.cycles-pp.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages
2.80 ± 4% -1.0 1.78 ± 22% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work
1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io
1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper
2.32 ± 3% -0.6 1.72 ± 18% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper
0.91 ± 8% -0.5 0.42 ± 73% perf-profile.calltrace.cycles-pp.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper
0.88 ± 8% -0.5 0.41 ± 74% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io
0.83 ± 5% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums
1.90 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread
1.89 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work
0.82 ± 4% -0.4 0.42 ± 71% perf-profile.calltrace.cycles-pp.run_one_async_done.normal_work_helper.process_one_work.worker_thread.kthread
0.81 ± 4% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.btrfs_map_bio.run_one_async_done.normal_work_helper.process_one_work.worker_thread
0.74 ± 6% -0.4 0.36 ± 70% perf-profile.calltrace.cycles-pp.push_leaf_right.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks
1.19 ± 3% -0.4 0.83 ± 16% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io
1.00 ± 6% -0.4 0.65 ± 7% perf-profile.calltrace.cycles-pp.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.27 ± 5% -0.4 0.92 ± 7% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.25 ± 6% -0.3 0.90 ± 7% perf-profile.calltrace.cycles-pp.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
0.93 ± 8% -0.3 0.59 ± 7% perf-profile.calltrace.cycles-pp.__set_extent_bit.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter
0.94 ± 7% -0.3 0.60 ± 6% perf-profile.calltrace.cycles-pp.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write
0.70 ± 8% -0.3 0.37 ± 70% perf-profile.calltrace.cycles-pp.__lookup_extent_mapping.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter
0.79 ± 6% -0.3 0.47 ± 71% perf-profile.calltrace.cycles-pp.setup_items_for_insert.btrfs_duplicate_item.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper
0.71 ± 21% -0.3 0.43 ± 74% perf-profile.calltrace.cycles-pp.__btrfs_free_extent.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space
1.15 ± 11% -0.3 0.88 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.15 ± 11% -0.3 0.88 ± 8% perf-profile.calltrace.cycles-pp.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
1.43 ± 13% -0.3 1.17 ± 5% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.06 ± 5% -0.3 0.81 ± 7% perf-profile.calltrace.cycles-pp.btrfs_drop_extent_cache.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.30 ± 15% -0.2 1.06 ± 4% perf-profile.calltrace.cycles-pp.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write
1.03 ± 13% -0.2 0.79 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range
1.20 ± 4% -0.2 0.96 ± 15% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
1.24 ± 3% -0.2 1.01 ± 15% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.05 ± 8% -0.2 0.82 ± 12% perf-profile.calltrace.cycles-pp.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
0.88 ± 13% -0.2 0.71 ± 2% perf-profile.calltrace.cycles-pp.wait_reserve_ticket.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter
1.01 ± 9% -0.2 0.84 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.01 ± 10% -0.2 0.85 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
0.72 ± 7% -0.2 0.56 ± 6% perf-profile.calltrace.cycles-pp.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write
0.75 ± 8% -0.1 0.60 ± 10% perf-profile.calltrace.cycles-pp.submit_extent_page.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages
0.92 ± 15% +0.2 1.14 ± 3% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
1.01 ± 15% +0.2 1.25 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
1.15 ± 15% +0.3 1.44 ± 6% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
0.27 ±100% +0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
1.74 ± 9% +0.4 2.11 ± 2% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
2.63 ± 10% +0.6 3.26 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle
0.00 +0.8 0.75 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook
2.37 ± 19% +0.8 3.19 ± 5% perf-profile.calltrace.cycles-pp.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread.ret_from_fork
2.33 ± 19% +0.8 3.17 ± 5% perf-profile.calltrace.cycles-pp.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread
0.00 +0.9 0.90 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook
4.71 ± 11% +1.0 5.74 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry
5.04 ± 10% +1.2 6.19 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
0.00 +1.3 1.30 ± 19% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range
0.00 +1.4 1.36 ± 19% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents
0.29 ±101% +1.4 1.67 ± 18% perf-profile.calltrace.cycles-pp.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space
0.00 +1.5 1.52 ± 18% perf-profile.calltrace.cycles-pp.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio
0.00 +2.2 2.20 ± 14% perf-profile.calltrace.cycles-pp.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_writeback.wb_workfn.process_one_work.worker_thread.kthread
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_workfn.process_one_work.worker_thread.kthread.ret_from_fork
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work.worker_thread
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work
49.45 ± 2% +4.3 53.79 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.calltrace.cycles-pp.secondary_startup_64
1.07 ± 18% +5.0 6.06 ± 20% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback
0.93 ± 18% +5.0 5.95 ± 20% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes
61.26 ± 2% +5.1 66.36 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
61.27 ± 2% +5.1 66.37 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
61.21 ± 2% +5.1 66.32 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.33 ±100% +5.3 5.68 ± 20% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode
55.45 ± 2% +5.4 60.88 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +6.4 6.39 ± 13% perf-profile.calltrace.cycles-pp.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page
0.00 +6.5 6.50 ± 13% perf-profile.calltrace.cycles-pp.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb
0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages
0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages
15.05 ± 6% -4.1 10.97 ± 20% perf-profile.children.cycles-pp.normal_work_helper
12.66 ± 5% -4.0 8.68 ± 20% perf-profile.children.cycles-pp.btrfs_finish_ordered_io
7.84 ± 5% -2.4 5.39 ± 10% perf-profile.children.cycles-pp.btrfs_search_slot
7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_writepages
7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_write_cache_pages
7.03 ± 7% -1.6 5.38 ± 12% perf-profile.children.cycles-pp.__extent_writepage
6.19 ± 8% -1.6 4.56 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
6.17 ± 8% -1.6 4.55 ± 7% perf-profile.children.cycles-pp.do_syscall_64
6.01 ± 8% -1.6 4.39 ± 7% perf-profile.children.cycles-pp.io_submit
5.81 ± 8% -1.6 4.23 ± 6% perf-profile.children.cycles-pp.__x64_sys_io_submit
5.76 ± 8% -1.6 4.19 ± 6% perf-profile.children.cycles-pp.io_submit_one
5.66 ± 5% -1.6 4.09 ± 20% perf-profile.children.cycles-pp.btrfs_mark_extent_written
4.52 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.add_pending_csums
5.59 ± 8% -1.5 4.07 ± 6% perf-profile.children.cycles-pp.aio_write
4.51 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.btrfs_csum_file_blocks
5.44 ± 8% -1.5 3.96 ± 7% perf-profile.children.cycles-pp.btrfs_file_write_iter
5.34 ± 9% -1.4 3.90 ± 6% perf-profile.children.cycles-pp.btrfs_buffered_write
5.88 ± 7% -1.4 4.47 ± 11% perf-profile.children.cycles-pp.writepage_delalloc
5.30 ± 8% -1.3 4.01 ± 11% perf-profile.children.cycles-pp.btrfs_run_delalloc_range
5.28 ± 8% -1.3 3.99 ± 11% perf-profile.children.cycles-pp.run_delalloc_nocow
2.55 ± 2% -0.9 1.67 ± 16% perf-profile.children.cycles-pp.btrfs_cow_block
2.54 ± 2% -0.9 1.66 ± 16% perf-profile.children.cycles-pp.__btrfs_cow_block
2.15 ± 13% -0.8 1.31 ± 8% perf-profile.children.cycles-pp.__lookup_extent_mapping
1.87 ± 12% -0.8 1.04 ± 12% perf-profile.children.cycles-pp.__etree_search
0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.printk
0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.vprintk_emit
2.60 ± 2% -0.7 1.92 ± 14% perf-profile.children.cycles-pp.btrfs_insert_empty_items
3.00 ± 5% -0.7 2.33 ± 6% perf-profile.children.cycles-pp._raw_spin_lock
1.89 ± 2% -0.7 1.23 ± 7% perf-profile.children.cycles-pp.__set_extent_bit
2.78 ± 4% -0.6 2.14 ± 14% perf-profile.children.cycles-pp.__schedule
1.57 ± 2% -0.5 1.02 ± 9% perf-profile.children.cycles-pp.lock_extent_bits
1.19 ± 14% -0.5 0.70 ± 11% perf-profile.children.cycles-pp.__clear_extent_bit
1.51 ± 7% -0.5 1.04 ± 9% perf-profile.children.cycles-pp.read_block_for_search
2.08 ± 5% -0.5 1.62 ± 18% perf-profile.children.cycles-pp.setup_items_for_insert
2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.btrfs_run_delayed_refs
2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.__btrfs_run_delayed_refs
1.60 ± 6% -0.4 1.20 ± 12% perf-profile.children.cycles-pp.schedule
1.95 ± 11% -0.4 1.58 ± 12% perf-profile.children.cycles-pp.try_to_wake_up
1.31 ± 6% -0.4 0.94 ± 9% perf-profile.children.cycles-pp.create_io_em
0.99 ± 4% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.alloc_tree_block_no_bg_flush
1.00 ± 6% -0.4 0.65 ± 7% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need
1.27 ± 9% -0.4 0.92 ± 11% perf-profile.children.cycles-pp.__wake_up_common_lock
0.98 ± 3% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.btrfs_alloc_tree_block
1.27 ± 5% -0.4 0.92 ± 7% perf-profile.children.cycles-pp.btrfs_dirty_pages
1.34 ± 7% -0.3 0.99 ± 10% perf-profile.children.cycles-pp.btrfs_map_bio
1.11 ± 9% -0.3 0.77 ± 11% perf-profile.children.cycles-pp.generic_bin_search
1.16 ± 6% -0.3 0.82 ± 9% perf-profile.children.cycles-pp.find_extent_buffer
0.91 ± 7% -0.3 0.57 ± 21% perf-profile.children.cycles-pp.btrfs_lookup_csum
1.21 ± 11% -0.3 0.90 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_csums_range
1.21 ± 11% -0.3 0.91 ± 9% perf-profile.children.cycles-pp.csum_exist_in_range
1.10 ± 10% -0.3 0.80 ± 9% perf-profile.children.cycles-pp.__wake_up_common
0.37 ± 70% -0.3 0.07 ± 18% perf-profile.children.cycles-pp.test_range_bit
1.61 ± 4% -0.3 1.32 ± 15% perf-profile.children.cycles-pp.push_leaf_right
0.60 ± 4% -0.3 0.31 ± 25% perf-profile.children.cycles-pp.btrfs_extend_item
1.02 ± 9% -0.3 0.73 ± 10% perf-profile.children.cycles-pp.autoremove_wake_function
1.12 ± 4% -0.3 0.84 ± 9% perf-profile.children.cycles-pp.btrfs_drop_extent_cache
0.82 ± 6% -0.3 0.55 ± 4% perf-profile.children.cycles-pp.btrfs_release_path
0.85 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_extent_buffer_full
0.99 ± 4% -0.3 0.73 ± 13% perf-profile.children.cycles-pp.btrfs_set_token_32
1.44 ± 13% -0.3 1.18 ± 5% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
0.84 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_page
0.87 ± 11% -0.3 0.62 ± 11% perf-profile.children.cycles-pp.pagecache_get_page
0.84 ± 7% -0.2 0.60 ± 11% perf-profile.children.cycles-pp.kmem_cache_alloc
1.12 ± 7% -0.2 0.88 ± 17% perf-profile.children.cycles-pp.__extent_writepage_io
1.30 ± 15% -0.2 1.06 ± 5% perf-profile.children.cycles-pp.reserve_metadata_bytes
0.82 ± 4% -0.2 0.58 ± 13% perf-profile.children.cycles-pp.run_one_async_done
1.25 ± 3% -0.2 1.01 ± 15% perf-profile.children.cycles-pp.schedule_idle
0.34 ± 53% -0.2 0.10 ± 28% perf-profile.children.cycles-pp.delay_tsc
0.70 ± 13% -0.2 0.47 ± 11% perf-profile.children.cycles-pp.prepare_pages
0.50 ± 22% -0.2 0.28 ± 13% perf-profile.children.cycles-pp.clear_state_bit
0.84 ± 5% -0.2 0.63 ± 6% perf-profile.children.cycles-pp.btrfs_get_extent
0.54 ± 11% -0.2 0.34 ± 4% perf-profile.children.cycles-pp.btrfs_free_path
1.16 ± 11% -0.2 0.96 ± 19% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.53 ± 9% -0.2 0.33 ± 18% perf-profile.children.cycles-pp.unpin_extent_cache
1.07 ± 10% -0.2 0.88 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_file_extent
0.50 ± 4% -0.2 0.31 ± 15% perf-profile.children.cycles-pp.clear_extent_bit
0.45 ± 13% -0.2 0.26 ± 11% perf-profile.children.cycles-pp.kmem_cache_free
0.40 ± 14% -0.2 0.21 ± 50% perf-profile.children.cycles-pp.__writeback_inodes_wb
0.64 ± 7% -0.2 0.45 ± 10% perf-profile.children.cycles-pp.dequeue_task_fair
0.50 ± 8% -0.2 0.33 ± 23% perf-profile.children.cycles-pp.btrfs_add_delayed_tree_ref
0.88 ± 13% -0.2 0.71 ± 2% perf-profile.children.cycles-pp.wait_reserve_ticket
0.52 ± 9% -0.2 0.35 ± 7% perf-profile.children.cycles-pp.__radix_tree_lookup
0.52 ± 8% -0.2 0.36 ± 8% perf-profile.children.cycles-pp.xas_load
0.78 ± 19% -0.2 0.62 ± 23% perf-profile.children.cycles-pp.__btrfs_free_extent
0.56 ± 17% -0.1 0.42 ± 15% perf-profile.children.cycles-pp.btrfs_del_items
0.51 ± 5% -0.1 0.36 ± 9% perf-profile.children.cycles-pp.dequeue_entity
0.39 ± 7% -0.1 0.24 ± 6% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers
0.82 ± 14% -0.1 0.68 ± 18% perf-profile.children.cycles-pp.ttwu_do_activate
0.55 ± 8% -0.1 0.41 ± 8% perf-profile.children.cycles-pp.btrfs_submit_bio_hook
0.53 ± 5% -0.1 0.40 ± 13% perf-profile.children.cycles-pp.update_load_avg
0.40 ± 8% -0.1 0.27 ± 27% perf-profile.children.cycles-pp.add_delayed_ref_head
0.35 ± 13% -0.1 0.22 ± 3% perf-profile.children.cycles-pp.set_extent_bit
0.52 ± 9% -0.1 0.39 ± 7% perf-profile.children.cycles-pp.btrfs_wq_submit_bio
0.35 ± 3% -0.1 0.23 ± 32% perf-profile.children.cycles-pp.queued_write_lock_slowpath
0.33 ± 6% -0.1 0.21 ± 15% perf-profile.children.cycles-pp.__slab_alloc
0.63 ± 19% -0.1 0.51 ± 13% perf-profile.children.cycles-pp.sched_ttwu_pending
0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode
0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode_fallback
0.57 ± 5% -0.1 0.46 ± 10% perf-profile.children.cycles-pp.find_lock_delalloc_range
0.45 ± 5% -0.1 0.34 ± 9% perf-profile.children.cycles-pp.btrfs_cross_ref_exist
0.29 ± 3% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.free_extent_buffer
0.38 ± 3% -0.1 0.26 ± 8% perf-profile.children.cycles-pp.btrfs_mark_buffer_dirty
0.46 ± 5% -0.1 0.35 ± 13% perf-profile.children.cycles-pp.find_delalloc_range
0.41 ± 9% -0.1 0.30 ± 2% perf-profile.children.cycles-pp.btrfs_copy_from_user
0.40 ± 9% -0.1 0.29 ± 2% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
0.38 ± 10% -0.1 0.27 perf-profile.children.cycles-pp.find_get_entry
0.31 ± 12% -0.1 0.20 ± 31% perf-profile.children.cycles-pp.btrfs_remove_ordered_extent
0.30 ± 6% -0.1 0.20 ± 14% perf-profile.children.cycles-pp.___slab_alloc
0.26 ± 8% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.merge_state
0.38 ± 7% -0.1 0.28 ± 3% perf-profile.children.cycles-pp.copyin
0.26 ± 17% -0.1 0.15 ± 26% perf-profile.children.cycles-pp.btrfs_delayed_update_inode
0.25 ± 4% -0.1 0.15 ± 5% perf-profile.children.cycles-pp.insert_state
0.40 ± 13% -0.1 0.30 ± 13% perf-profile.children.cycles-pp.extent_clear_unlock_delalloc
0.36 ± 9% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.alloc_extent_state
0.38 ± 7% -0.1 0.29 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.39 ± 3% -0.1 0.29 ± 14% perf-profile.children.cycles-pp.alloc_extent_buffer
0.31 ± 4% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.set_extent_buffer_dirty
0.26 ± 11% -0.1 0.17 ± 23% perf-profile.children.cycles-pp.unlock_up
0.32 ± 3% -0.1 0.24 ± 13% perf-profile.children.cycles-pp.__btrfs_map_block
0.23 ± 13% -0.1 0.15 ± 28% perf-profile.children.cycles-pp.btrfs_add_delayed_data_ref
0.25 ± 12% -0.1 0.17 ± 2% perf-profile.children.cycles-pp.__slab_free
0.47 ± 6% -0.1 0.39 ± 15% perf-profile.children.cycles-pp.__push_leaf_right
0.26 ± 9% -0.1 0.18 ± 19% perf-profile.children.cycles-pp.__test_set_page_writeback
0.32 ± 11% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.btrfs_add_ordered_extent
0.15 ± 8% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.__xa_set_mark
0.23 ± 7% -0.1 0.15 ± 3% perf-profile.children.cycles-pp.release_extent_buffer
0.18 ± 8% -0.1 0.11 ± 26% perf-profile.children.cycles-pp.alloc_extent_map
0.32 ± 10% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.__btrfs_add_ordered_extent
0.32 ± 17% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.btrfs_tree_read_lock_atomic
0.20 ± 13% -0.1 0.13 ± 27% perf-profile.children.cycles-pp.btrfs_inc_extent_ref
0.14 ± 21% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.find_ref_head
0.13 ± 32% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.rb_erase_cached
0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_submit_bio_start
0.53 ± 12% -0.1 0.46 ± 16% perf-profile.children.cycles-pp.btrfs_tree_read_lock
0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_csum_one_bio
0.17 ± 16% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.rb_erase
0.21 ± 7% -0.1 0.14 ± 16% perf-profile.children.cycles-pp.run_one_async_start
0.22 ± 3% -0.1 0.15 ± 22% perf-profile.children.cycles-pp.btrfs_try_tree_write_lock
0.21 ± 9% -0.1 0.15 ± 21% perf-profile.children.cycles-pp.btrfs_leaf_free_space
0.15 ± 23% -0.1 0.09 ± 35% perf-profile.children.cycles-pp.free_extent_map
0.20 ± 9% -0.1 0.14 ± 20% perf-profile.children.cycles-pp.leaf_space_used
0.19 ± 8% -0.1 0.13 ± 13% perf-profile.children.cycles-pp.check_delayed_ref
0.17 ± 13% -0.1 0.12 ± 31% perf-profile.children.cycles-pp.get_page_from_freelist
0.13 ± 8% -0.1 0.08 ± 26% perf-profile.children.cycles-pp.__update_load_avg_se
0.17 ± 2% -0.1 0.12 ± 18% perf-profile.children.cycles-pp.btrfs_set_range_writeback
0.15 ± 8% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.xas_set_mark
0.18 ± 6% -0.1 0.13 ± 16% perf-profile.children.cycles-pp.___might_sleep
0.20 ± 11% -0.1 0.15 ± 35% perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.10 ± 9% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_unlock
0.14 ± 11% -0.1 0.09 ± 28% perf-profile.children.cycles-pp.btrfs_reserve_extent
0.10 ± 33% -0.1 0.05 ± 72% perf-profile.children.cycles-pp.__btrfs_release_delayed_node
0.22 ± 9% -0.0 0.17 ± 25% perf-profile.children.cycles-pp.mark_extent_buffer_accessed
0.09 ± 13% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_read_unlock_blocking
0.18 ± 8% -0.0 0.14 ± 24% perf-profile.children.cycles-pp.__list_del_entry_valid
0.09 ± 13% -0.0 0.04 ± 73% perf-profile.children.cycles-pp.verify_parent_transid
0.11 ± 19% -0.0 0.07 ± 11% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.22 ± 5% -0.0 0.18 ± 12% perf-profile.children.cycles-pp.check_committed_ref
0.11 ± 19% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.select_idle_sibling
0.14 ± 11% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.do_io_getevents
0.15 ± 7% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__x64_sys_io_getevents
0.08 ± 10% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_release_extent_buffer_pages
0.17 ± 18% -0.0 0.13 ± 9% perf-profile.children.cycles-pp.clear_page_dirty_for_io
0.14 ± 5% -0.0 0.10 ± 25% perf-profile.children.cycles-pp.block_group_cache_tree_search
0.18 ± 4% -0.0 0.14 ± 11% perf-profile.children.cycles-pp.btrfs_get_chunk_map
0.11 ± 13% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.btrfs_buffer_uptodate
0.15 ± 15% -0.0 0.11 ± 19% perf-profile.children.cycles-pp.xas_find_marked
0.12 ± 15% -0.0 0.08 ± 20% perf-profile.children.cycles-pp.btrfs_unlock_up_safe
0.12 ± 14% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.btrfs_set_path_blocking
0.11 ± 11% -0.0 0.07 ± 28% perf-profile.children.cycles-pp.find_free_extent
0.24 ± 5% -0.0 0.20 ± 6% perf-profile.children.cycles-pp.btrfs_root_node
0.07 ± 6% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.extent_mergeable
0.13 ± 12% -0.0 0.10 ± 9% perf-profile.children.cycles-pp.__switch_to
0.10 ± 18% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.rb_insert_color_cached
0.07 ± 14% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.rb_prev
0.11 ± 11% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.get_io_u
0.10 ± 14% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.available_idle_cpu
0.17 ± 10% -0.0 0.13 ± 3% perf-profile.children.cycles-pp._raw_write_lock
0.10 ± 11% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.__might_sleep
0.17 ± 7% -0.0 0.14 ± 12% perf-profile.children.cycles-pp._raw_read_lock
0.17 ± 10% -0.0 0.14 ± 12% perf-profile.children.cycles-pp.can_overcommit
0.10 ± 11% -0.0 0.07 perf-profile.children.cycles-pp.__list_add_valid
0.09 ± 7% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.aio_read_events
0.11 ± 10% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.___perf_sw_event
0.11 ± 9% -0.0 0.09 ± 18% perf-profile.children.cycles-pp.get_alloc_profile
0.11 ± 12% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.pick_next_task_idle
0.10 ± 13% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.read_events
0.08 ± 6% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.replace_extent_mapping
0.10 ± 5% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.menu_reflect
0.07 ± 17% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.run_posix_cpu_timers
0.05 ± 58% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.rcu_irq_enter
0.19 ± 11% +0.1 0.26 ± 17% perf-profile.children.cycles-pp.native_irq_return_iret
0.20 ± 14% +0.1 0.27 ± 8% perf-profile.children.cycles-pp.btrfs_comp_cpu_keys
0.38 ± 9% +0.1 0.46 perf-profile.children.cycles-pp.load_balance
0.30 ± 8% +0.1 0.38 ± 15% perf-profile.children.cycles-pp.irq_enter
0.21 ± 24% +0.1 0.30 ± 9% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.22 ± 3% +0.1 0.32 ± 9% perf-profile.children.cycles-pp.lapic_next_deadline
0.52 ± 12% +0.1 0.63 ± 3% perf-profile.children.cycles-pp.rebalance_domains
1.90 ± 10% +0.3 2.23 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.23 ± 12% +0.4 0.67 ± 15% perf-profile.children.cycles-pp.read_extent_buffer
2.88 ± 11% +0.5 3.41 perf-profile.children.cycles-pp.hrtimer_interrupt
0.09 ± 12% +0.7 0.83 ± 15% perf-profile.children.cycles-pp.btrfs_get_token_8
2.37 ± 19% +0.8 3.19 ± 5% perf-profile.children.cycles-pp.btrfs_async_reclaim_metadata_space
2.33 ± 19% +0.8 3.17 ± 5% perf-profile.children.cycles-pp.flush_space
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_and_wait_transaction
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_marked_extents
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
0.42 ± 43% +1.3 1.67 ± 18% perf-profile.children.cycles-pp.btrfs_commit_transaction
0.25 ± 5% +1.9 2.17 ± 10% perf-profile.children.cycles-pp.btrfs_get_token_64
1.71 ± 5% +2.1 3.79 ± 5% perf-profile.children.cycles-pp.btrfs_get_token_32
0.43 ± 8% +2.4 2.84 ± 10% perf-profile.children.cycles-pp.map_private_extent_buffer
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_writeback
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.writeback_sb_inodes
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.__writeback_single_inode
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_workfn
50.06 ± 2% +3.8 53.82 ± 4% perf-profile.children.cycles-pp.intel_idle
61.99 ± 2% +4.4 66.39 ± 3% perf-profile.children.cycles-pp.do_idle
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.secondary_startup_64
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.cpu_startup_entry
8.51 ± 7% +4.5 13.04 ± 12% perf-profile.children.cycles-pp.do_writepages
56.43 ± 3% +4.9 61.30 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state
61.27 ± 2% +5.1 66.37 ± 3% perf-profile.children.cycles-pp.start_secondary
1.43 ± 7% +6.2 7.64 ± 13% perf-profile.children.cycles-pp.submit_extent_page
1.20 ± 17% +6.2 7.45 ± 13% perf-profile.children.cycles-pp.btree_write_cache_pages
1.05 ± 17% +6.3 7.31 ± 13% perf-profile.children.cycles-pp.write_one_eb
1.11 ± 7% +6.3 7.37 ± 13% perf-profile.children.cycles-pp.submit_one_bio
0.55 ± 16% +6.4 6.95 ± 13% perf-profile.children.cycles-pp.btree_submit_bio_hook
0.00 +6.5 6.48 ± 13% perf-profile.children.cycles-pp.check_leaf
0.00 +6.5 6.53 ± 13% perf-profile.children.cycles-pp.btree_csum_one_bio
2.12 ± 11% -0.8 1.29 ± 7% perf-profile.self.cycles-pp.__lookup_extent_mapping
1.83 ± 12% -0.8 1.03 ± 13% perf-profile.self.cycles-pp.__etree_search
2.06 ± 5% -0.5 1.59 ± 5% perf-profile.self.cycles-pp._raw_spin_lock
0.81 ± 9% -0.3 0.52 ± 10% perf-profile.self.cycles-pp.generic_bin_search
0.84 ± 7% -0.3 0.58 ± 17% perf-profile.self.cycles-pp.copy_page
0.33 ± 54% -0.2 0.10 ± 28% perf-profile.self.cycles-pp.delay_tsc
0.89 ± 9% -0.2 0.69 ± 14% perf-profile.self.cycles-pp.btrfs_set_token_32
0.81 ± 4% -0.2 0.63 ± 14% perf-profile.self.cycles-pp.__schedule
0.51 ± 9% -0.2 0.35 ± 6% perf-profile.self.cycles-pp.__radix_tree_lookup
0.45 ± 8% -0.1 0.32 ± 10% perf-profile.self.cycles-pp.xas_load
0.43 ± 4% -0.1 0.31 ± 6% perf-profile.self.cycles-pp.find_extent_buffer
0.77 ± 5% -0.1 0.66 ± 11% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.31 ± 10% -0.1 0.20 ± 28% perf-profile.self.cycles-pp.add_delayed_ref_head
0.39 ± 16% -0.1 0.29 ± 16% perf-profile.self.cycles-pp.try_to_wake_up
0.31 ± 4% -0.1 0.20 ± 26% perf-profile.self.cycles-pp.queued_write_lock_slowpath
0.38 ± 7% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.20 ± 17% -0.1 0.11 ± 14% perf-profile.self.cycles-pp.kmem_cache_free
0.25 ± 11% -0.1 0.17 ± 2% perf-profile.self.cycles-pp.__slab_free
0.16 ± 9% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.free_extent_buffer
0.28 ± 17% -0.1 0.20 ± 11% perf-profile.self.cycles-pp.btrfs_search_slot
0.31 ± 4% -0.1 0.24 ± 20% perf-profile.self.cycles-pp.kmem_cache_alloc
0.14 ± 21% -0.1 0.06 ± 19% perf-profile.self.cycles-pp.find_ref_head
0.16 ± 10% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.insert_state
0.12 ± 8% -0.1 0.06 ± 72% perf-profile.self.cycles-pp.__update_load_avg_se
0.16 ± 15% -0.1 0.10 ± 8% perf-profile.self.cycles-pp.rb_erase
0.10 ± 9% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_unlock
0.15 ± 13% -0.1 0.10 ± 12% perf-profile.self.cycles-pp.___slab_alloc
0.28 ± 13% -0.1 0.22 ± 5% perf-profile.self.cycles-pp.queued_read_lock_slowpath
0.15 ± 21% -0.1 0.09 ± 35% perf-profile.self.cycles-pp.free_extent_map
0.18 ± 6% -0.1 0.12 ± 11% perf-profile.self.cycles-pp.update_load_avg
0.10 ± 11% -0.1 0.04 ± 70% perf-profile.self.cycles-pp.btrfs_extend_item
0.15 ± 23% -0.1 0.09 ± 9% perf-profile.self.cycles-pp.add_extent_mapping
0.15 ± 8% -0.1 0.09 ± 20% perf-profile.self.cycles-pp.xas_set_mark
0.18 ± 6% -0.1 0.13 ± 16% perf-profile.self.cycles-pp.___might_sleep
0.18 ± 6% -0.0 0.13 ± 22% perf-profile.self.cycles-pp.__list_del_entry_valid
0.18 ± 4% -0.0 0.14 ± 22% perf-profile.self.cycles-pp.pick_next_task_fair
0.09 ± 13% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.verify_parent_transid
0.09 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_read_unlock_blocking
0.16 ± 13% -0.0 0.12 ± 24% perf-profile.self.cycles-pp.mark_page_accessed
0.12 ± 14% -0.0 0.08 ± 20% perf-profile.self.cycles-pp.dequeue_task_fair
0.10 ± 19% -0.0 0.07 ± 18% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.08 ± 14% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.unlock_up
0.15 ± 15% -0.0 0.11 ± 19% perf-profile.self.cycles-pp.xas_find_marked
0.24 ± 6% -0.0 0.20 ± 8% perf-profile.self.cycles-pp.btrfs_root_node
0.17 ± 6% -0.0 0.13 ± 10% perf-profile.self.cycles-pp._raw_read_lock
0.10 ± 13% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.btrfs_csum_one_bio
0.16 ± 7% -0.0 0.13 ± 9% perf-profile.self.cycles-pp.dequeue_entity
0.07 -0.0 0.04 ± 71% perf-profile.self.cycles-pp.io_u_queued_complete
0.08 -0.0 0.05 ± 70% perf-profile.self.cycles-pp.mutex_lock
0.11 ± 11% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.get_io_u
0.16 ± 9% -0.0 0.13 perf-profile.self.cycles-pp._raw_write_lock
0.07 ± 17% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.btrfs_tree_read_lock
0.12 ± 8% -0.0 0.09 ± 13% perf-profile.self.cycles-pp.__switch_to
0.10 ± 11% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.__clear_extent_bit
0.07 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.rb_prev
0.10 ± 15% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.find_get_pages_range_tag
0.10 ± 12% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.available_idle_cpu
0.10 ± 11% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.__list_add_valid
0.08 ± 13% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.__might_sleep
0.12 ± 17% -0.0 0.09 ± 15% perf-profile.self.cycles-pp.release_extent_buffer
0.09 ± 17% -0.0 0.07 ± 14% perf-profile.self.cycles-pp.___perf_sw_event
0.08 ± 5% -0.0 0.06 ± 13% perf-profile.self.cycles-pp.__push_leaf_right
0.11 ± 7% -0.0 0.09 ± 9% perf-profile.self.cycles-pp.set_extent_buffer_dirty
0.11 ± 4% -0.0 0.08 ± 11% perf-profile.self.cycles-pp.__set_extent_bit
0.09 ± 7% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.hrtimer_interrupt
0.07 ± 17% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.run_posix_cpu_timers
0.07 ± 7% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.load_balance
0.01 ±173% +0.1 0.07 ± 17% perf-profile.self.cycles-pp.lapic_next_deadline
0.00 +0.1 0.06 ± 23% perf-profile.self.cycles-pp.find_next_and_bit
0.19 ± 11% +0.1 0.26 ± 17% perf-profile.self.cycles-pp.native_irq_return_iret
0.19 ± 24% +0.1 0.28 ± 7% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.50 ± 8% +0.2 0.69 ± 17% perf-profile.self.cycles-pp.cpuidle_enter_state
0.07 ± 11% +0.4 0.46 ± 17% perf-profile.self.cycles-pp.btrfs_get_token_8
0.22 ± 13% +0.4 0.66 ± 16% perf-profile.self.cycles-pp.read_extent_buffer
0.00 +0.6 0.65 ± 14% perf-profile.self.cycles-pp.check_leaf
0.15 ± 11% +1.0 1.13 ± 11% perf-profile.self.cycles-pp.btrfs_get_token_64
1.48 ± 7% +1.0 2.51 ± 5% perf-profile.self.cycles-pp.btrfs_get_token_32
0.41 ± 8% +2.2 2.61 ± 9% perf-profile.self.cycles-pp.map_private_extent_buffer
49.99 ± 2% +3.7 53.74 ± 4% perf-profile.self.cycles-pp.intel_idle
fio.write_bw_MBps
300 +-+-------------------------------------------------------------------+
| |
250 +-++.+ +. .+.. .+..+..+.+.. .+..+.. .+ |
| : : +. +..+ + +..+ +. + |
| : : : : : : |
200 +-+ : : : : : : O O |
O O O: O :O O O O O O O O O O: O:O O: O: O O O O O
150 +-+ : : : : : : |
| : : : : : : |
100 +-+ : : : : : : |
| : : : : : : |
| : : : : : : |
50 +-+ :: :: :: |
| : : : |
0 +-+---------------------------------------------O-------------O-------+
fio.write_iops
70000 +-+-----------------------------------------------------------------+
|..+.+ +..+..+.+.. .+.+..+.+..+ +.. +..+.+..+..+ |
60000 +-+ : : +. : : + : |
| : : : : : : |
50000 +-+ : : : : : : |
O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O
40000 +-+ : O: : : : : |
| : : : : : : |
30000 +-+ : : : : : : |
| : : : : : : |
20000 +-+ : : : : : : |
| :: :: :: |
10000 +-+ : : : |
| : : : |
0 +-+--------------------------------------------O------------O-------+
fio.write_clat_mean_us
30000 +-+-----------------------------------------------------------------+
| |
25000 O-+O O O O O O O O O O O O O O O O O O O |
| O O O O O
| |
20000 +-+ |
|..+.+ +..+..+.+..+..+.+..+.+..+ +..+ +..+.+..+..+ |
15000 +-+ : : : : : : |
| : : : : : : |
10000 +-+ : : : : : : |
| : : : : : : |
| : : : : : : |
5000 +-+ : : : : :: |
| : : : |
0 +-+--------------------------------------------O------------O-------+
fio.write_clat_stddev
160000 +-+----------------------------------------------------------------+
| |
140000 O-+O O O O O O O O O O O O O O |
120000 +-+ O O |
| |
100000 +-+ O O O O O O O O
| |
80000 +-+ |
| .+.. |
60000 +-++.+ +..+.+. +.+..+.+..+..+ +.+ +..+..+.+..+ |
40000 +-+ : : : : : : |
| : : : : : : |
20000 +-+ : : : : : : |
| :: :: :: |
0 +-+-------------------------------------------O------------O-------+
fio.write_slat_mean_us
900 +-+-------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O |
800 +-+O O O O O O O
700 +-+ |
| |
600 +-++.+ +.+..+..+..+.+..+..+.+..+ +..+ +..+..+..+.+ |
500 +-+ : : : : : : |
| : : : : : : |
400 +-+ : : : : : : |
300 +-+ : : : : : : |
| : : : : : : |
200 +-+ : : : : : : |
100 +-+ : : :: :: |
| : : : |
0 +-+---------------------------------------------O-------------O-------+
fio.write_slat_stddev
25000 +-+-----O---------O-------------------------------------------------+
O O O O O O O O O O O O O O O |
| |
20000 +-+ |
| O O O O O O O O
| |
15000 +-+ |
| |
10000 +-++. .+.. .+.. .+..+ .+ .+..+ |
|. + +..+..+ +..+ + : +. : +..+.+. |
| : : : : : : |
5000 +-+ : : : : : : |
| : : : : : : |
| :: : :: |
0 +-+--------------------------------------------O------------O-------+
fio.latency_1000ms_
0.4 +-+------------------------------------------------------------------+
| |
0.35 +-+ O |
0.3 +-+ |
| O O |
0.25 +-+ O O |
| O O O
0.2 O-+ O O O O O O O O O O |
| O O |
0.15 +-+O O O O |
0.1 +-+ |
| |
0.05 +-+ |
| .+. .+..+..+.+..+.. .+. .+.+..+.. |
0 +-+---------------------------------------------O------------O-------+
fio.workload
1.4e+07 +-+---------------------------------------------------------------+
|..+.+ +..+.+..+.+..+.+..+.+..+ +.. +.+..+.+..+ |
1.2e+07 +-+ : : : : + : |
| : : : : : : |
1e+07 +-+ : : : : : : |
O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O
8e+06 +-+ : O: : : : : |
| : : : : : : |
6e+06 +-+ : : : : : : |
| : : : : : : |
4e+06 +-+ : : : : : : |
| :: :: :: |
2e+06 +-+ : : : |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
fio.time.voluntary_context_switches
1.4e+07 +-+---------------------------------------------------------------+
| |
1.2e+07 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ |
| : : : : : : |
1e+07 +-+ : : : : : : |
| O O: : O O O : : :O : O O O O O O O
8e+06 O-+ : O:O O O O O O O: O:O O: : |
| : : : : : : |
6e+06 +-+ : : : : : : |
| : : : : : : |
4e+06 +-+ : : : : : : |
| : : : : :: |
2e+06 +-+ : : :: |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
fio.time.file_system_outputs
1.2e+08 +-+---------------------------------------------------------------+
| |
1e+08 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ |
| : : : : : : |
| : : : : : : |
8e+07 +-+ : : : : : : O O O O O
O O O: O:O O O O O O O O O O: O:O O:O : O O |
6e+07 +-+ : : : : : : |
| : : : : : : |
4e+07 +-+ : : : : : : |
| : : : : : : |
| : : : : :: |
2e+07 +-+ : : :: |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
6 months, 2 weeks
- 4
- 4
87c4696d57 ("mm/debug: Add tests validating architecture page .."): [ 1.395296] kernel BUG at include/linux/mm.h:2007!
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-t...
commit 87c4696d57b5e380bc2720fdff0772b042462b7d
Author: Anshuman Khandual <anshuman.khandual(a)arm.com>
AuthorDate: Tue Dec 24 10:08:42 2019 +0530
Commit: 0day robot <lkp(a)intel.com>
CommitDate: Wed Dec 25 18:02:30 2019 +0800
mm/debug: Add tests validating architecture page table helpers
This adds tests which will validate architecture page table helpers and
other accessors in their compliance with expected generic MM semantics.
This will help various architectures in validating changes to existing
page table helpers or addition of new ones.
This test covers basic page table entry transformations including but not
limited to old, young, dirty, clean, write, write protect etc at various
level along with populating intermediate entries with next page table page
and validating them.
Test page table pages are allocated from system memory with required size
and alignments. The mapped pfns at page table levels are derived from a
real pfn representing a valid kernel text symbol. This test gets called
right after page_alloc_init_late().
This gets build and run when CONFIG_DEBUG_VM_PGTABLE is selected along with
CONFIG_VM_DEBUG. Architectures willing to subscribe this test also need to
select CONFIG_ARCH_HAS_DEBUG_VM_PGTABLE which for now is limited to x86 and
arm64. Going forward, other architectures too can enable this after fixing
build or runtime problems (if any) with their page table helpers.
Folks interested in making sure that a given platform's page table helpers
conform to expected generic MM semantics should enable the above config
which will just trigger this test during boot. Any non conformity here will
be reported as an warning which would need to be fixed. This test will help
catch any changes to the agreed upon semantics expected from generic MM and
enable platforms to accommodate it thereafter.
Cc: Andrew Morton <akpm(a)linux-foundation.org>
Cc: Vlastimil Babka <vbabka(a)suse.cz>
Cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
Cc: Thomas Gleixner <tglx(a)linutronix.de>
Cc: Mike Rapoport <rppt(a)linux.vnet.ibm.com>
Cc: Jason Gunthorpe <jgg(a)ziepe.ca>
Cc: Dan Williams <dan.j.williams(a)intel.com>
Cc: Peter Zijlstra <peterz(a)infradead.org>
Cc: Michal Hocko <mhocko(a)kernel.org>
Cc: Mark Rutland <mark.rutland(a)arm.com>
Cc: Mark Brown <broonie(a)kernel.org>
Cc: Steven Price <Steven.Price(a)arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel(a)linaro.org>
Cc: Masahiro Yamada <yamada.masahiro(a)socionext.com>
Cc: Kees Cook <keescook(a)chromium.org>
Cc: Tetsuo Handa <penguin-kernel(a)i-love.sakura.ne.jp>
Cc: Matthew Wilcox <willy(a)infradead.org>
Cc: Sri Krishna chowdary <schowdary(a)nvidia.com>
Cc: Dave Hansen <dave.hansen(a)intel.com>
Cc: Russell King - ARM Linux <linux(a)armlinux.org.uk>
Cc: Michael Ellerman <mpe(a)ellerman.id.au>
Cc: Paul Mackerras <paulus(a)samba.org>
Cc: Martin Schwidefsky <schwidefsky(a)de.ibm.com>
Cc: Heiko Carstens <heiko.carstens(a)de.ibm.com>
Cc: "David S. Miller" <davem(a)davemloft.net>
Cc: Vineet Gupta <vgupta(a)synopsys.com>
Cc: James Hogan <jhogan(a)kernel.org>
Cc: Paul Burton <paul.burton(a)mips.com>
Cc: Ralf Baechle <ralf(a)linux-mips.org>
Cc: Kirill A. Shutemov <kirill(a)shutemov.name>
Cc: Gerald Schaefer <gerald.schaefer(a)de.ibm.com>
Cc: Christophe Leroy <christophe.leroy(a)c-s.fr>
Cc: Ingo Molnar <mingo(a)kernel.org>
Cc: linux-snps-arc(a)lists.infradead.org
Cc: linux-mips(a)vger.kernel.org
Cc: linux-arm-kernel(a)lists.infradead.org
Cc: linux-ia64(a)vger.kernel.org
Cc: linuxppc-dev(a)lists.ozlabs.org
Cc: linux-s390(a)vger.kernel.org
Cc: linux-sh(a)vger.kernel.org
Cc: sparclinux(a)vger.kernel.org
Cc: x86(a)kernel.org
Cc: linux-kernel(a)vger.kernel.org
Tested-by: Christophe Leroy <christophe.leroy(a)c-s.fr> #PPC32
Reviewed-by: Ingo Molnar <mingo(a)kernel.org>
Suggested-by: Catalin Marinas <catalin.marinas(a)arm.com>
Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org>
Signed-off-by: Christophe Leroy <christophe.leroy(a)c-s.fr>
Signed-off-by: Anshuman Khandual <anshuman.khandual(a)arm.com>
46cf053efe Linux 5.5-rc3
87c4696d57 mm/debug: Add tests validating architecture page table helpers
+------------------------------------------+----------+------------+
| | v5.5-rc3 | 87c4696d57 |
+------------------------------------------+----------+------------+
| boot_successes | 32 | 0 |
| boot_failures | 0 | 11 |
| kernel_BUG_at_include/linux/mm.h | 0 | 11 |
| invalid_opcode:#[##] | 0 | 11 |
| EIP:pgtable_pmd_page_dtor | 0 | 11 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 11 |
+------------------------------------------+----------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 1.390624] smp: Brought up 1 node, 2 CPUs
[ 1.390624] smpboot: Max logical packages: 2
[ 1.390624] smpboot: Total of 2 processors activated (8783.48 BogoMIPS)
[ 1.391537] debug_vm_pgtable: debug_vm_pgtable: Validating architecture page table helpers
[ 1.392382] page:f29b85c0 refcount:0 mapcount:0 mapping:00000000 index:0x0
[ 1.393415] raw: 02800000 f29b8624 f29b8584 00000000 00000000 edc22280 ffffffff 00000000
[ 1.394178] page dumped because: VM_BUG_ON_PAGE(page->pmd_huge_pte)
[ 1.394820] ------------[ cut here ]------------
[ 1.395296] kernel BUG at include/linux/mm.h:2007!
[ 1.395942] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC PTI
[ 1.396463] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.5.0-rc3-00001-g87c4696d57b5e #1
[ 1.396722] EIP: pgtable_pmd_page_dtor+0x1a/0x23
[ 1.396722] Code: d4 8a 27 c2 e8 16 81 04 00 b2 01 5b 88 d0 5d c3 55 89 e5 52 89 45 fc 8b 45 fc 83 78 08 00 74 0c ba e1 e2 e0 c1 e8 14 99 13 00 <0f> 0b e8 92 eb 13 00 c9 c3 55 89 e5 52 89 45 fc 8b 45 fc 90 8d 74
[ 1.396722] EAX: c1e0e2e1 EBX: 2dc2e000 ECX: 00000000 EDX: c1e0e2e1
[ 1.396722] ESI: edc2b000 EDI: edc4e010 EBP: ee287f14 ESP: ee287f10
[ 1.396722] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010246
[ 1.396722] CR0: 80050033 CR2: ffffffff CR3: 0226a000 CR4: 001406b0
[ 1.396722] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1.396722] DR6: fffe0ff0 DR7: 00000400
[ 1.396722] Call Trace:
[ 1.396722] mop_up_one_pmd+0x48/0x62
[ 1.396722] pgd_free+0x35/0xe0
[ 1.396722] __mmdrop+0x42/0x96
[ 1.396722] debug_vm_pgtable+0x460/0x47c
[ 1.396722] kernel_init_freeable+0x84/0x172
[ 1.396722] ? rest_init+0xe9/0xe9
[ 1.396722] kernel_init+0xd/0xe9
[ 1.396722] ret_from_fork+0x1e/0x28
[ 1.396722] Modules linked in:
[ 1.396742] ---[ end trace 9c6f11143a94c590 ]---
[ 1.397197] EIP: pgtable_pmd_page_dtor+0x1a/0x23
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start 59548166b20af3032c081a8d567551fa5edce034 46cf053efec6a3a5f343fead837777efe8252a46 --
git bisect good be0d040c5c4bf0be561b49030efc5e61f96a2c48 # 00:50 G 11 0 0 0 Merge 'linux-review/Madalin-Bucur/dpaa_eth-fix-DMA-mapping-leak/20191225-042314' into devel-hourly-2019122517
git bisect good 9f24ee23f7dcd753208b8379dc07750a2f9e38be # 01:17 G 11 0 0 0 Merge 'omap/omap-for-v5.6/ti-sysc-dt' into devel-hourly-2019122517
git bisect good 437fd0a2fcebf42ca5a9495c75929f4217d93f8c # 01:35 G 11 0 0 0 Merge 'tty/tty-testing' into devel-hourly-2019122517
git bisect good 74e55c4af3d3fdac084ff3a74b3888a468a33179 # 01:49 G 11 0 0 0 Merge 'pinctrl-samsung/pinctrl-next' into devel-hourly-2019122517
git bisect good fccd6f98a5b3ff2bebdd60669fca895790935c2a # 02:07 G 11 0 0 0 Merge 'shawnguo/for-next' into devel-hourly-2019122517
git bisect good c9315ddfffc58512ff94311e43fee9e19e8c72d0 # 02:32 G 11 0 0 0 Merge 'tip/perf/core' into devel-hourly-2019122517
git bisect good 9e6e6e4c8f466df27a95913e82606c9904ed7e83 # 02:53 G 10 0 0 0 Merge 'linux-review/Viresh-Kumar/sched-fair-Load-balance-aggressively-for-SCHED_IDLE-CPUs/20191225-183419' into devel-hourly-2019122517
git bisect good ab38db6c3c65d49beb4efa8950b96cb0b33dfac0 # 03:19 G 11 0 0 0 Merge 'linux-review/Saravana-Kannan/efi-arm-defer-probe-of-PCIe-backed-efifb-on-DT-systems/20191225-182253' into devel-hourly-2019122517
git bisect bad 3eb52936430c315fa36857e50ce55b40bd7153ef # 04:50 B 0 11 27 0 Merge 'linux-review/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20191225-180158' into devel-hourly-2019122517
git bisect bad 87c4696d57b5e380bc2720fdff0772b042462b7d # 05:19 B 0 11 27 0 mm/debug: Add tests validating architecture page table helpers
# first bad commit: [87c4696d57b5e380bc2720fdff0772b042462b7d] mm/debug: Add tests validating architecture page table helpers
git bisect good 46cf053efec6a3a5f343fead837777efe8252a46 # 06:16 G 31 0 0 0 Linux 5.5-rc3
# extra tests with debug options
git bisect good 87c4696d57b5e380bc2720fdff0772b042462b7d # 07:04 G 11 0 2 2 mm/debug: Add tests validating architecture page table helpers
# extra tests on head commit of linux-review/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20191225-180158
git bisect bad 87c4696d57b5e380bc2720fdff0772b042462b7d # 07:06 B 0 11 27 0 mm/debug: Add tests validating architecture page table helpers
# bad: [87c4696d57b5e380bc2720fdff0772b042462b7d] mm/debug: Add tests validating architecture page table helpers
# extra tests on revert first bad commit
git bisect good dd0462bb7fdbb7c31a88e93334199d4f6ea6a6b6 # 08:50 G 10 0 0 0 Revert "mm/debug: Add tests validating architecture page table helpers"
# good: [dd0462bb7fdbb7c31a88e93334199d4f6ea6a6b6] Revert "mm/debug: Add tests validating architecture page table helpers"
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation
9 months
- 4
- 5
[drm/mgag200] 90f479ae51: vm-scalability.median -18.8% regression
by kernel test robot
Greeting,
FYI, we noticed a -18.8% regression of vm-scalability.median due to commit:
commit: 90f479ae51afa45efab97afdde9b94b9660dd3e4 ("drm/mgag200: Replace struct mga_fbdev with generic framebuffer emulation")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git master
in testcase: vm-scalability
on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory
with following parameters:
runtime: 300s
size: 8T
test: anon-cow-seq-hugetlb
cpufreq_governor: performance
test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase:
gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2019-05-14.cgz/300s/8T/lkp-knm01/anon-cow-seq-hugetlb/vm-scalability
commit:
f1f8555dfb ("drm/bochs: Use shadow buffer for bochs framebuffer console")
90f479ae51 ("drm/mgag200: Replace struct mga_fbdev with generic framebuffer emulation")
f1f8555dfb9a70a2 90f479ae51afa45efab97afdde9
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
2:4 -50% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
:4 25% 1:4 dmesg.WARNING:at_ip___perf_sw_event/0x
:4 25% 1:4 dmesg.WARNING:at_ip__fsnotify_parent/0x
%stddev %change %stddev
\ | \
43955 ± 2% -18.8% 35691 vm-scalability.median
0.06 ± 7% +193.0% 0.16 ± 2% vm-scalability.median_stddev
14906559 ± 2% -17.9% 12237079 vm-scalability.throughput
87651 ± 2% -17.4% 72374 vm-scalability.time.involuntary_context_switches
2086168 -23.6% 1594224 vm-scalability.time.minor_page_faults
15082 ± 2% -10.4% 13517 vm-scalability.time.percent_of_cpu_this_job_got
29987 -8.9% 27327 vm-scalability.time.system_time
15755 -12.4% 13795 vm-scalability.time.user_time
122011 -19.3% 98418 vm-scalability.time.voluntary_context_switches
3.034e+09 -23.6% 2.318e+09 vm-scalability.workload
242478 ± 12% +68.5% 408518 ± 23% cpuidle.POLL.time
2788 ± 21% +117.4% 6062 ± 26% cpuidle.POLL.usage
56653 ± 10% +64.4% 93144 ± 20% meminfo.Mapped
120392 ± 7% +14.0% 137212 ± 4% meminfo.Shmem
47221 ± 11% +77.1% 83634 ± 22% numa-meminfo.node0.Mapped
120465 ± 7% +13.9% 137205 ± 4% numa-meminfo.node0.Shmem
2885513 -16.5% 2409384 numa-numastat.node0.local_node
2885471 -16.5% 2409354 numa-numastat.node0.numa_hit
11813 ± 11% +76.3% 20824 ± 22% numa-vmstat.node0.nr_mapped
30096 ± 7% +13.8% 34238 ± 4% numa-vmstat.node0.nr_shmem
43.72 ± 2% +5.5 49.20 mpstat.cpu.all.idle%
0.03 ± 4% +0.0 0.05 ± 6% mpstat.cpu.all.soft%
19.51 -2.4 17.08 mpstat.cpu.all.usr%
1012 -7.9% 932.75 turbostat.Avg_MHz
32.38 ± 10% +25.8% 40.73 turbostat.CPU%c1
145.51 -3.1% 141.01 turbostat.PkgWatt
15.09 -19.2% 12.19 turbostat.RAMWatt
43.50 ± 2% +13.2% 49.25 vmstat.cpu.id
18.75 ± 2% -13.3% 16.25 ± 2% vmstat.cpu.us
152.00 ± 2% -9.5% 137.50 vmstat.procs.r
4800 -13.1% 4173 vmstat.system.cs
156170 -11.9% 137594 slabinfo.anon_vma.active_objs
3395 -11.9% 2991 slabinfo.anon_vma.active_slabs
156190 -11.9% 137606 slabinfo.anon_vma.num_objs
3395 -11.9% 2991 slabinfo.anon_vma.num_slabs
1716 ± 5% +11.5% 1913 ± 8% slabinfo.dmaengine-unmap-16.active_objs
1716 ± 5% +11.5% 1913 ± 8% slabinfo.dmaengine-unmap-16.num_objs
1767 ± 2% -19.0% 1431 ± 2% slabinfo.hugetlbfs_inode_cache.active_objs
1767 ± 2% -19.0% 1431 ± 2% slabinfo.hugetlbfs_inode_cache.num_objs
3597 ± 5% -16.4% 3006 ± 3% slabinfo.skbuff_ext_cache.active_objs
3597 ± 5% -16.4% 3006 ± 3% slabinfo.skbuff_ext_cache.num_objs
1330122 -23.6% 1016557 proc-vmstat.htlb_buddy_alloc_success
77214 ± 3% +6.4% 82128 ± 2% proc-vmstat.nr_active_anon
67277 +2.9% 69246 proc-vmstat.nr_anon_pages
218.50 ± 3% -10.6% 195.25 proc-vmstat.nr_dirtied
288628 +1.4% 292755 proc-vmstat.nr_file_pages
360.50 -2.7% 350.75 proc-vmstat.nr_inactive_file
14225 ± 9% +63.8% 23304 ± 20% proc-vmstat.nr_mapped
30109 ± 7% +13.8% 34259 ± 4% proc-vmstat.nr_shmem
99870 -1.3% 98597 proc-vmstat.nr_slab_unreclaimable
204.00 ± 4% -12.1% 179.25 proc-vmstat.nr_written
77214 ± 3% +6.4% 82128 ± 2% proc-vmstat.nr_zone_active_anon
360.50 -2.7% 350.75 proc-vmstat.nr_zone_inactive_file
8810 ± 19% -66.1% 2987 ± 42% proc-vmstat.numa_hint_faults
8810 ± 19% -66.1% 2987 ± 42% proc-vmstat.numa_hint_faults_local
2904082 -16.4% 2427026 proc-vmstat.numa_hit
2904081 -16.4% 2427025 proc-vmstat.numa_local
6.828e+08 -23.5% 5.221e+08 proc-vmstat.pgalloc_normal
2900008 -17.2% 2400195 proc-vmstat.pgfault
6.827e+08 -23.5% 5.22e+08 proc-vmstat.pgfree
1.635e+10 -17.0% 1.357e+10 perf-stat.i.branch-instructions
1.53 ± 4% -0.1 1.45 ± 3% perf-stat.i.branch-miss-rate%
2.581e+08 ± 3% -20.5% 2.051e+08 ± 2% perf-stat.i.branch-misses
12.66 +1.1 13.78 perf-stat.i.cache-miss-rate%
72720849 -12.0% 63958986 perf-stat.i.cache-misses
5.766e+08 -18.6% 4.691e+08 perf-stat.i.cache-references
4674 ± 2% -13.0% 4064 perf-stat.i.context-switches
4.29 +12.5% 4.83 perf-stat.i.cpi
2.573e+11 -7.4% 2.383e+11 perf-stat.i.cpu-cycles
231.35 -21.5% 181.56 perf-stat.i.cpu-migrations
3522 +4.4% 3677 perf-stat.i.cycles-between-cache-misses
0.09 ± 13% +0.0 0.12 ± 5% perf-stat.i.iTLB-load-miss-rate%
5.894e+10 -15.8% 4.961e+10 perf-stat.i.iTLB-loads
5.901e+10 -15.8% 4.967e+10 perf-stat.i.instructions
1291 ± 14% -21.8% 1010 perf-stat.i.instructions-per-iTLB-miss
0.24 -11.0% 0.21 perf-stat.i.ipc
9476 -17.5% 7821 perf-stat.i.minor-faults
9478 -17.5% 7821 perf-stat.i.page-faults
9.76 -3.6% 9.41 perf-stat.overall.MPKI
1.59 ± 4% -0.1 1.52 perf-stat.overall.branch-miss-rate%
12.61 +1.1 13.71 perf-stat.overall.cache-miss-rate%
4.38 +10.5% 4.83 perf-stat.overall.cpi
3557 +5.3% 3747 perf-stat.overall.cycles-between-cache-misses
0.08 ± 12% +0.0 0.10 perf-stat.overall.iTLB-load-miss-rate%
1268 ± 15% -23.0% 976.22 perf-stat.overall.instructions-per-iTLB-miss
0.23 -9.5% 0.21 perf-stat.overall.ipc
5815 +9.7% 6378 perf-stat.overall.path-length
1.634e+10 -17.5% 1.348e+10 perf-stat.ps.branch-instructions
2.595e+08 ± 3% -21.2% 2.043e+08 ± 2% perf-stat.ps.branch-misses
72565205 -12.2% 63706339 perf-stat.ps.cache-misses
5.754e+08 -19.2% 4.646e+08 perf-stat.ps.cache-references
4640 ± 2% -12.5% 4060 perf-stat.ps.context-switches
2.581e+11 -7.5% 2.387e+11 perf-stat.ps.cpu-cycles
229.91 -22.0% 179.42 perf-stat.ps.cpu-migrations
5.889e+10 -16.3% 4.927e+10 perf-stat.ps.iTLB-loads
5.899e+10 -16.3% 4.938e+10 perf-stat.ps.instructions
9388 -18.2% 7677 perf-stat.ps.minor-faults
9389 -18.2% 7677 perf-stat.ps.page-faults
1.764e+13 -16.2% 1.479e+13 perf-stat.total.instructions
46803 ± 3% -18.8% 37982 ± 6% sched_debug.cfs_rq:/.exec_clock.min
5320 ± 3% +23.7% 6581 ± 3% sched_debug.cfs_rq:/.exec_clock.stddev
6737 ± 14% +58.1% 10649 ± 10% sched_debug.cfs_rq:/.load.avg
587978 ± 17% +58.2% 930382 ± 9% sched_debug.cfs_rq:/.load.max
46952 ± 16% +64.8% 77388 ± 11% sched_debug.cfs_rq:/.load.stddev
7.12 ± 4% +49.1% 10.62 ± 6% sched_debug.cfs_rq:/.load_avg.avg
474.40 ± 23% +67.5% 794.60 ± 10% sched_debug.cfs_rq:/.load_avg.max
37.70 ± 11% +74.8% 65.90 ± 9% sched_debug.cfs_rq:/.load_avg.stddev
13424269 ± 4% -15.6% 11328098 ± 2% sched_debug.cfs_rq:/.min_vruntime.avg
15411275 ± 3% -12.4% 13505072 ± 2% sched_debug.cfs_rq:/.min_vruntime.max
7939295 ± 6% -17.5% 6551322 ± 7% sched_debug.cfs_rq:/.min_vruntime.min
21.44 ± 7% -56.1% 9.42 ± 4% sched_debug.cfs_rq:/.nr_spread_over.avg
117.45 ± 11% -60.6% 46.30 ± 14% sched_debug.cfs_rq:/.nr_spread_over.max
19.33 ± 8% -66.4% 6.49 ± 9% sched_debug.cfs_rq:/.nr_spread_over.stddev
4.32 ± 15% +84.4% 7.97 ± 3% sched_debug.cfs_rq:/.runnable_load_avg.avg
353.85 ± 29% +118.8% 774.35 ± 11% sched_debug.cfs_rq:/.runnable_load_avg.max
27.30 ± 24% +118.5% 59.64 ± 9% sched_debug.cfs_rq:/.runnable_load_avg.stddev
6729 ± 14% +58.2% 10644 ± 10% sched_debug.cfs_rq:/.runnable_weight.avg
587978 ± 17% +58.2% 930382 ± 9% sched_debug.cfs_rq:/.runnable_weight.max
46950 ± 16% +64.8% 77387 ± 11% sched_debug.cfs_rq:/.runnable_weight.stddev
5305069 ± 4% -17.4% 4380376 ± 7% sched_debug.cfs_rq:/.spread0.avg
7328745 ± 3% -9.9% 6600897 ± 3% sched_debug.cfs_rq:/.spread0.max
2220837 ± 4% +55.8% 3460596 ± 5% sched_debug.cpu.avg_idle.avg
4590666 ± 9% +76.8% 8117037 ± 15% sched_debug.cpu.avg_idle.max
485052 ± 7% +80.3% 874679 ± 10% sched_debug.cpu.avg_idle.stddev
561.50 ± 26% +37.7% 773.30 ± 15% sched_debug.cpu.clock.stddev
561.50 ± 26% +37.7% 773.30 ± 15% sched_debug.cpu.clock_task.stddev
3.20 ± 10% +109.6% 6.70 ± 3% sched_debug.cpu.cpu_load[0].avg
309.10 ± 20% +150.3% 773.75 ± 12% sched_debug.cpu.cpu_load[0].max
21.02 ± 14% +160.8% 54.80 ± 9% sched_debug.cpu.cpu_load[0].stddev
3.19 ± 8% +109.8% 6.70 ± 3% sched_debug.cpu.cpu_load[1].avg
299.75 ± 19% +158.0% 773.30 ± 12% sched_debug.cpu.cpu_load[1].max
20.32 ± 12% +168.7% 54.62 ± 9% sched_debug.cpu.cpu_load[1].stddev
3.20 ± 8% +109.1% 6.69 ± 4% sched_debug.cpu.cpu_load[2].avg
288.90 ± 20% +167.0% 771.40 ± 12% sched_debug.cpu.cpu_load[2].max
19.70 ± 12% +175.4% 54.27 ± 9% sched_debug.cpu.cpu_load[2].stddev
3.16 ± 8% +110.9% 6.66 ± 6% sched_debug.cpu.cpu_load[3].avg
275.50 ± 24% +178.4% 766.95 ± 12% sched_debug.cpu.cpu_load[3].max
18.92 ± 15% +184.2% 53.77 ± 10% sched_debug.cpu.cpu_load[3].stddev
3.08 ± 8% +115.7% 6.65 ± 7% sched_debug.cpu.cpu_load[4].avg
263.55 ± 28% +188.7% 760.85 ± 12% sched_debug.cpu.cpu_load[4].max
18.03 ± 18% +196.6% 53.46 ± 11% sched_debug.cpu.cpu_load[4].stddev
14543 -9.6% 13150 sched_debug.cpu.curr->pid.max
5293 ± 16% +74.7% 9248 ± 11% sched_debug.cpu.load.avg
587978 ± 17% +58.2% 930382 ± 9% sched_debug.cpu.load.max
40887 ± 19% +78.3% 72891 ± 9% sched_debug.cpu.load.stddev
1141679 ± 4% +56.9% 1790907 ± 5% sched_debug.cpu.max_idle_balance_cost.avg
2432100 ± 9% +72.6% 4196779 ± 13% sched_debug.cpu.max_idle_balance_cost.max
745656 +29.3% 964170 ± 5% sched_debug.cpu.max_idle_balance_cost.min
239032 ± 9% +81.9% 434806 ± 10% sched_debug.cpu.max_idle_balance_cost.stddev
0.00 ± 27% +92.1% 0.00 ± 31% sched_debug.cpu.next_balance.stddev
1030 ± 4% -10.4% 924.00 ± 2% sched_debug.cpu.nr_switches.min
0.04 ± 26% +139.0% 0.09 ± 41% sched_debug.cpu.nr_uninterruptible.avg
830.35 ± 6% -12.0% 730.50 ± 2% sched_debug.cpu.sched_count.min
912.00 ± 2% -9.5% 825.38 sched_debug.cpu.ttwu_count.avg
433.05 ± 3% -19.2% 350.05 ± 3% sched_debug.cpu.ttwu_count.min
160.70 ± 3% -12.5% 140.60 ± 4% sched_debug.cpu.ttwu_local.min
9072 ± 11% -36.4% 5767 ± 8% softirqs.CPU1.RCU
12769 ± 5% +15.3% 14718 ± 3% softirqs.CPU101.SCHED
13198 +11.5% 14717 ± 3% softirqs.CPU102.SCHED
12981 ± 4% +13.9% 14788 ± 3% softirqs.CPU105.SCHED
13486 ± 3% +11.8% 15071 ± 4% softirqs.CPU111.SCHED
12794 ± 4% +14.1% 14601 ± 9% softirqs.CPU112.SCHED
12999 ± 4% +10.1% 14314 ± 4% softirqs.CPU115.SCHED
12844 ± 4% +10.6% 14202 ± 2% softirqs.CPU120.SCHED
13336 ± 3% +9.4% 14585 ± 3% softirqs.CPU122.SCHED
12639 ± 4% +20.2% 15195 softirqs.CPU123.SCHED
13040 ± 5% +15.2% 15024 ± 5% softirqs.CPU126.SCHED
13123 +15.1% 15106 ± 5% softirqs.CPU127.SCHED
9188 ± 6% -35.7% 5911 ± 2% softirqs.CPU13.RCU
13054 ± 3% +13.1% 14761 ± 5% softirqs.CPU130.SCHED
13158 ± 2% +13.9% 14985 ± 5% softirqs.CPU131.SCHED
12797 ± 6% +13.5% 14524 ± 3% softirqs.CPU133.SCHED
12452 ± 5% +14.8% 14297 softirqs.CPU134.SCHED
13078 ± 3% +10.4% 14439 ± 3% softirqs.CPU138.SCHED
12617 ± 2% +14.5% 14442 ± 5% softirqs.CPU139.SCHED
12974 ± 3% +13.7% 14752 ± 4% softirqs.CPU142.SCHED
12579 ± 4% +19.1% 14983 ± 3% softirqs.CPU143.SCHED
9122 ± 24% -44.6% 5053 ± 5% softirqs.CPU144.RCU
13366 ± 2% +11.1% 14848 ± 3% softirqs.CPU149.SCHED
13246 ± 2% +22.0% 16162 ± 7% softirqs.CPU150.SCHED
13452 ± 3% +20.5% 16210 ± 7% softirqs.CPU151.SCHED
13507 +10.1% 14869 softirqs.CPU156.SCHED
13808 ± 3% +9.2% 15079 ± 4% softirqs.CPU157.SCHED
13442 ± 2% +13.4% 15248 ± 4% softirqs.CPU160.SCHED
13311 +12.1% 14920 ± 2% softirqs.CPU162.SCHED
13544 ± 3% +8.5% 14695 ± 4% softirqs.CPU163.SCHED
13648 ± 3% +11.2% 15179 ± 2% softirqs.CPU166.SCHED
13404 ± 4% +12.5% 15079 ± 3% softirqs.CPU168.SCHED
13421 ± 6% +16.0% 15568 ± 8% softirqs.CPU169.SCHED
13115 ± 3% +23.1% 16139 ± 10% softirqs.CPU171.SCHED
13424 ± 6% +10.4% 14822 ± 3% softirqs.CPU175.SCHED
13274 ± 3% +13.7% 15087 ± 9% softirqs.CPU185.SCHED
13409 ± 3% +12.3% 15063 ± 3% softirqs.CPU190.SCHED
13181 ± 7% +13.4% 14946 ± 3% softirqs.CPU196.SCHED
13578 ± 3% +10.9% 15061 softirqs.CPU197.SCHED
13323 ± 5% +24.8% 16627 ± 6% softirqs.CPU198.SCHED
14072 ± 2% +12.3% 15798 ± 7% softirqs.CPU199.SCHED
12604 ± 13% +17.9% 14865 softirqs.CPU201.SCHED
13380 ± 4% +14.8% 15356 ± 3% softirqs.CPU203.SCHED
13481 ± 8% +14.2% 15390 ± 3% softirqs.CPU204.SCHED
12921 ± 2% +13.8% 14710 ± 3% softirqs.CPU206.SCHED
13468 +13.0% 15218 ± 2% softirqs.CPU208.SCHED
13253 ± 2% +13.1% 14992 softirqs.CPU209.SCHED
13319 ± 2% +14.3% 15225 ± 7% softirqs.CPU210.SCHED
13673 ± 5% +16.3% 15895 ± 3% softirqs.CPU211.SCHED
13290 +17.0% 15556 ± 5% softirqs.CPU212.SCHED
13455 ± 4% +14.4% 15392 ± 3% softirqs.CPU213.SCHED
13454 ± 4% +14.3% 15377 ± 3% softirqs.CPU215.SCHED
13872 ± 7% +9.7% 15221 ± 5% softirqs.CPU220.SCHED
13555 ± 4% +17.3% 15896 ± 5% softirqs.CPU222.SCHED
13411 ± 4% +20.8% 16197 ± 6% softirqs.CPU223.SCHED
8472 ± 21% -44.8% 4680 ± 3% softirqs.CPU224.RCU
13141 ± 3% +16.2% 15265 ± 7% softirqs.CPU225.SCHED
14084 ± 3% +8.2% 15242 ± 2% softirqs.CPU226.SCHED
13528 ± 4% +11.3% 15063 ± 4% softirqs.CPU228.SCHED
13218 ± 3% +16.3% 15377 ± 4% softirqs.CPU229.SCHED
14031 ± 4% +10.2% 15467 ± 2% softirqs.CPU231.SCHED
13770 ± 3% +14.0% 15700 ± 3% softirqs.CPU232.SCHED
13456 ± 3% +12.3% 15105 ± 3% softirqs.CPU233.SCHED
13137 ± 4% +13.5% 14909 ± 3% softirqs.CPU234.SCHED
13318 ± 2% +14.7% 15280 ± 2% softirqs.CPU235.SCHED
13690 ± 2% +13.7% 15563 ± 7% softirqs.CPU238.SCHED
13771 ± 5% +20.8% 16634 ± 7% softirqs.CPU241.SCHED
13317 ± 7% +19.5% 15919 ± 9% softirqs.CPU243.SCHED
8234 ± 16% -43.9% 4616 ± 5% softirqs.CPU244.RCU
13845 ± 6% +13.0% 15643 ± 3% softirqs.CPU244.SCHED
13179 ± 3% +16.3% 15323 softirqs.CPU246.SCHED
13754 +12.2% 15438 ± 3% softirqs.CPU248.SCHED
13769 ± 4% +10.9% 15276 ± 2% softirqs.CPU252.SCHED
13702 +10.5% 15147 ± 2% softirqs.CPU254.SCHED
13315 ± 2% +12.5% 14980 ± 3% softirqs.CPU255.SCHED
13785 ± 3% +12.9% 15568 ± 5% softirqs.CPU256.SCHED
13307 ± 3% +15.0% 15298 ± 3% softirqs.CPU257.SCHED
13864 ± 3% +10.5% 15313 ± 2% softirqs.CPU259.SCHED
13879 ± 2% +11.4% 15465 softirqs.CPU261.SCHED
13815 +13.6% 15687 ± 5% softirqs.CPU264.SCHED
119574 ± 2% +11.8% 133693 ± 11% softirqs.CPU266.TIMER
13688 +10.9% 15180 ± 6% softirqs.CPU267.SCHED
11716 ± 4% +19.3% 13974 ± 8% softirqs.CPU27.SCHED
13866 ± 3% +13.7% 15765 ± 4% softirqs.CPU271.SCHED
13887 ± 5% +12.5% 15621 softirqs.CPU272.SCHED
13383 ± 3% +19.8% 16031 ± 2% softirqs.CPU274.SCHED
13347 +14.1% 15232 ± 3% softirqs.CPU275.SCHED
12884 ± 2% +21.0% 15593 ± 4% softirqs.CPU276.SCHED
13131 ± 5% +13.4% 14891 ± 5% softirqs.CPU277.SCHED
12891 ± 2% +19.2% 15371 ± 4% softirqs.CPU278.SCHED
13313 ± 4% +13.0% 15049 ± 2% softirqs.CPU279.SCHED
13514 ± 3% +10.2% 14897 ± 2% softirqs.CPU280.SCHED
13501 ± 3% +13.7% 15346 softirqs.CPU281.SCHED
13261 +17.5% 15577 softirqs.CPU282.SCHED
8076 ± 15% -43.7% 4546 ± 5% softirqs.CPU283.RCU
13686 ± 3% +12.6% 15413 ± 2% softirqs.CPU284.SCHED
13439 ± 2% +9.2% 14670 ± 4% softirqs.CPU285.SCHED
8878 ± 9% -35.4% 5735 ± 4% softirqs.CPU35.RCU
11690 ± 2% +13.6% 13274 ± 5% softirqs.CPU40.SCHED
11714 ± 2% +19.3% 13975 ± 13% softirqs.CPU41.SCHED
11763 +12.5% 13239 ± 4% softirqs.CPU45.SCHED
11662 ± 2% +9.4% 12757 ± 3% softirqs.CPU46.SCHED
11805 ± 2% +9.3% 12902 ± 2% softirqs.CPU50.SCHED
12158 ± 3% +12.3% 13655 ± 8% softirqs.CPU55.SCHED
11716 ± 4% +8.8% 12751 ± 3% softirqs.CPU58.SCHED
11922 ± 2% +9.9% 13100 ± 4% softirqs.CPU64.SCHED
9674 ± 17% -41.8% 5625 ± 6% softirqs.CPU66.RCU
11818 +12.0% 13237 softirqs.CPU66.SCHED
124682 ± 7% -6.1% 117088 ± 5% softirqs.CPU66.TIMER
8637 ± 9% -34.0% 5700 ± 7% softirqs.CPU70.RCU
11624 ± 2% +11.0% 12901 ± 2% softirqs.CPU70.SCHED
12372 ± 2% +13.2% 14003 ± 3% softirqs.CPU71.SCHED
9949 ± 25% -33.9% 6574 ± 31% softirqs.CPU72.RCU
10392 ± 26% -35.1% 6745 ± 35% softirqs.CPU73.RCU
12766 ± 3% +11.1% 14188 ± 3% softirqs.CPU76.SCHED
12611 ± 2% +18.8% 14984 ± 5% softirqs.CPU78.SCHED
12786 ± 3% +17.9% 15079 ± 7% softirqs.CPU79.SCHED
11947 ± 4% +9.7% 13103 ± 4% softirqs.CPU8.SCHED
13379 ± 7% +11.8% 14962 ± 4% softirqs.CPU83.SCHED
13438 ± 5% +9.7% 14738 ± 2% softirqs.CPU84.SCHED
12768 +19.4% 15241 ± 6% softirqs.CPU88.SCHED
8604 ± 13% -39.3% 5222 ± 3% softirqs.CPU89.RCU
13077 ± 2% +17.1% 15308 ± 7% softirqs.CPU89.SCHED
11887 ± 3% +20.1% 14272 ± 5% softirqs.CPU9.SCHED
12723 ± 3% +11.3% 14165 ± 4% softirqs.CPU90.SCHED
8439 ± 12% -38.9% 5153 ± 4% softirqs.CPU91.RCU
13429 ± 3% +10.3% 14806 ± 2% softirqs.CPU95.SCHED
12852 ± 4% +10.3% 14174 ± 5% softirqs.CPU96.SCHED
13010 ± 2% +14.4% 14888 ± 5% softirqs.CPU97.SCHED
2315644 ± 4% -36.2% 1477200 ± 4% softirqs.RCU
1572 ± 10% +63.9% 2578 ± 39% interrupts.CPU0.NMI:Non-maskable_interrupts
1572 ± 10% +63.9% 2578 ± 39% interrupts.CPU0.PMI:Performance_monitoring_interrupts
252.00 ± 11% -35.2% 163.25 ± 13% interrupts.CPU104.RES:Rescheduling_interrupts
2738 ± 24% +52.4% 4173 ± 19% interrupts.CPU105.NMI:Non-maskable_interrupts
2738 ± 24% +52.4% 4173 ± 19% interrupts.CPU105.PMI:Performance_monitoring_interrupts
245.75 ± 19% -31.0% 169.50 ± 7% interrupts.CPU105.RES:Rescheduling_interrupts
228.75 ± 13% -24.7% 172.25 ± 19% interrupts.CPU106.RES:Rescheduling_interrupts
2243 ± 15% +66.3% 3730 ± 35% interrupts.CPU113.NMI:Non-maskable_interrupts
2243 ± 15% +66.3% 3730 ± 35% interrupts.CPU113.PMI:Performance_monitoring_interrupts
2703 ± 31% +67.0% 4514 ± 33% interrupts.CPU118.NMI:Non-maskable_interrupts
2703 ± 31% +67.0% 4514 ± 33% interrupts.CPU118.PMI:Performance_monitoring_interrupts
2613 ± 25% +42.2% 3715 ± 24% interrupts.CPU121.NMI:Non-maskable_interrupts
2613 ± 25% +42.2% 3715 ± 24% interrupts.CPU121.PMI:Performance_monitoring_interrupts
311.50 ± 23% -47.7% 163.00 ± 9% interrupts.CPU122.RES:Rescheduling_interrupts
266.75 ± 19% -31.6% 182.50 ± 15% interrupts.CPU124.RES:Rescheduling_interrupts
293.75 ± 33% -32.3% 198.75 ± 19% interrupts.CPU125.RES:Rescheduling_interrupts
2601 ± 36% +43.2% 3724 ± 29% interrupts.CPU127.NMI:Non-maskable_interrupts
2601 ± 36% +43.2% 3724 ± 29% interrupts.CPU127.PMI:Performance_monitoring_interrupts
2258 ± 21% +68.2% 3797 ± 29% interrupts.CPU13.NMI:Non-maskable_interrupts
2258 ± 21% +68.2% 3797 ± 29% interrupts.CPU13.PMI:Performance_monitoring_interrupts
3338 ± 29% +54.6% 5160 ± 9% interrupts.CPU139.NMI:Non-maskable_interrupts
3338 ± 29% +54.6% 5160 ± 9% interrupts.CPU139.PMI:Performance_monitoring_interrupts
219.50 ± 27% -23.0% 169.00 ± 21% interrupts.CPU139.RES:Rescheduling_interrupts
290.25 ± 25% -32.5% 196.00 ± 11% interrupts.CPU14.RES:Rescheduling_interrupts
243.50 ± 4% -16.0% 204.50 ± 12% interrupts.CPU140.RES:Rescheduling_interrupts
1797 ± 15% +135.0% 4223 ± 46% interrupts.CPU147.NMI:Non-maskable_interrupts
1797 ± 15% +135.0% 4223 ± 46% interrupts.CPU147.PMI:Performance_monitoring_interrupts
2537 ± 22% +89.6% 4812 ± 28% interrupts.CPU15.NMI:Non-maskable_interrupts
2537 ± 22% +89.6% 4812 ± 28% interrupts.CPU15.PMI:Performance_monitoring_interrupts
292.25 ± 34% -33.9% 193.25 ± 6% interrupts.CPU15.RES:Rescheduling_interrupts
424.25 ± 37% -58.5% 176.25 ± 14% interrupts.CPU158.RES:Rescheduling_interrupts
312.50 ± 42% -54.2% 143.00 ± 18% interrupts.CPU159.RES:Rescheduling_interrupts
725.00 ±118% -75.7% 176.25 ± 14% interrupts.CPU163.RES:Rescheduling_interrupts
2367 ± 6% +59.9% 3786 ± 24% interrupts.CPU177.NMI:Non-maskable_interrupts
2367 ± 6% +59.9% 3786 ± 24% interrupts.CPU177.PMI:Performance_monitoring_interrupts
239.50 ± 30% -46.6% 128.00 ± 14% interrupts.CPU179.RES:Rescheduling_interrupts
320.75 ± 15% -24.0% 243.75 ± 20% interrupts.CPU20.RES:Rescheduling_interrupts
302.50 ± 17% -47.2% 159.75 ± 8% interrupts.CPU200.RES:Rescheduling_interrupts
2166 ± 5% +92.0% 4157 ± 40% interrupts.CPU207.NMI:Non-maskable_interrupts
2166 ± 5% +92.0% 4157 ± 40% interrupts.CPU207.PMI:Performance_monitoring_interrupts
217.00 ± 11% -34.6% 142.00 ± 12% interrupts.CPU214.RES:Rescheduling_interrupts
2610 ± 36% +47.4% 3848 ± 35% interrupts.CPU215.NMI:Non-maskable_interrupts
2610 ± 36% +47.4% 3848 ± 35% interrupts.CPU215.PMI:Performance_monitoring_interrupts
2046 ± 13% +118.6% 4475 ± 43% interrupts.CPU22.NMI:Non-maskable_interrupts
2046 ± 13% +118.6% 4475 ± 43% interrupts.CPU22.PMI:Performance_monitoring_interrupts
289.50 ± 28% -41.1% 170.50 ± 8% interrupts.CPU22.RES:Rescheduling_interrupts
2232 ± 6% +33.0% 2970 ± 24% interrupts.CPU221.NMI:Non-maskable_interrupts
2232 ± 6% +33.0% 2970 ± 24% interrupts.CPU221.PMI:Performance_monitoring_interrupts
4552 ± 12% -27.6% 3295 ± 15% interrupts.CPU222.NMI:Non-maskable_interrupts
4552 ± 12% -27.6% 3295 ± 15% interrupts.CPU222.PMI:Performance_monitoring_interrupts
2013 ± 15% +80.9% 3641 ± 27% interrupts.CPU226.NMI:Non-maskable_interrupts
2013 ± 15% +80.9% 3641 ± 27% interrupts.CPU226.PMI:Performance_monitoring_interrupts
2575 ± 49% +67.1% 4302 ± 34% interrupts.CPU227.NMI:Non-maskable_interrupts
2575 ± 49% +67.1% 4302 ± 34% interrupts.CPU227.PMI:Performance_monitoring_interrupts
248.00 ± 36% -36.3% 158.00 ± 19% interrupts.CPU228.RES:Rescheduling_interrupts
2441 ± 24% +43.0% 3490 ± 30% interrupts.CPU23.NMI:Non-maskable_interrupts
2441 ± 24% +43.0% 3490 ± 30% interrupts.CPU23.PMI:Performance_monitoring_interrupts
404.25 ± 69% -65.5% 139.50 ± 17% interrupts.CPU236.RES:Rescheduling_interrupts
566.50 ± 40% -73.6% 149.50 ± 31% interrupts.CPU237.RES:Rescheduling_interrupts
243.50 ± 26% -37.1% 153.25 ± 21% interrupts.CPU248.RES:Rescheduling_interrupts
258.25 ± 12% -53.5% 120.00 ± 18% interrupts.CPU249.RES:Rescheduling_interrupts
2888 ± 27% +49.4% 4313 ± 30% interrupts.CPU253.NMI:Non-maskable_interrupts
2888 ± 27% +49.4% 4313 ± 30% interrupts.CPU253.PMI:Performance_monitoring_interrupts
2468 ± 44% +67.3% 4131 ± 37% interrupts.CPU256.NMI:Non-maskable_interrupts
2468 ± 44% +67.3% 4131 ± 37% interrupts.CPU256.PMI:Performance_monitoring_interrupts
425.00 ± 59% -60.3% 168.75 ± 34% interrupts.CPU258.RES:Rescheduling_interrupts
1859 ± 16% +106.3% 3834 ± 44% interrupts.CPU268.NMI:Non-maskable_interrupts
1859 ± 16% +106.3% 3834 ± 44% interrupts.CPU268.PMI:Performance_monitoring_interrupts
2684 ± 28% +61.2% 4326 ± 36% interrupts.CPU269.NMI:Non-maskable_interrupts
2684 ± 28% +61.2% 4326 ± 36% interrupts.CPU269.PMI:Performance_monitoring_interrupts
2171 ± 6% +108.8% 4533 ± 20% interrupts.CPU270.NMI:Non-maskable_interrupts
2171 ± 6% +108.8% 4533 ± 20% interrupts.CPU270.PMI:Performance_monitoring_interrupts
2262 ± 14% +61.8% 3659 ± 37% interrupts.CPU273.NMI:Non-maskable_interrupts
2262 ± 14% +61.8% 3659 ± 37% interrupts.CPU273.PMI:Performance_monitoring_interrupts
2203 ± 11% +50.7% 3320 ± 38% interrupts.CPU279.NMI:Non-maskable_interrupts
2203 ± 11% +50.7% 3320 ± 38% interrupts.CPU279.PMI:Performance_monitoring_interrupts
2433 ± 17% +52.9% 3721 ± 25% interrupts.CPU280.NMI:Non-maskable_interrupts
2433 ± 17% +52.9% 3721 ± 25% interrupts.CPU280.PMI:Performance_monitoring_interrupts
2778 ± 33% +63.1% 4531 ± 36% interrupts.CPU283.NMI:Non-maskable_interrupts
2778 ± 33% +63.1% 4531 ± 36% interrupts.CPU283.PMI:Performance_monitoring_interrupts
331.75 ± 32% -39.8% 199.75 ± 17% interrupts.CPU29.RES:Rescheduling_interrupts
2178 ± 22% +53.9% 3353 ± 31% interrupts.CPU3.NMI:Non-maskable_interrupts
2178 ± 22% +53.9% 3353 ± 31% interrupts.CPU3.PMI:Performance_monitoring_interrupts
298.50 ± 30% -39.7% 180.00 ± 6% interrupts.CPU34.RES:Rescheduling_interrupts
2490 ± 3% +58.7% 3953 ± 28% interrupts.CPU35.NMI:Non-maskable_interrupts
2490 ± 3% +58.7% 3953 ± 28% interrupts.CPU35.PMI:Performance_monitoring_interrupts
270.50 ± 24% -31.1% 186.25 ± 3% interrupts.CPU36.RES:Rescheduling_interrupts
2493 ± 7% +57.0% 3915 ± 27% interrupts.CPU43.NMI:Non-maskable_interrupts
2493 ± 7% +57.0% 3915 ± 27% interrupts.CPU43.PMI:Performance_monitoring_interrupts
286.75 ± 36% -32.4% 193.75 ± 7% interrupts.CPU45.RES:Rescheduling_interrupts
259.00 ± 12% -23.6% 197.75 ± 13% interrupts.CPU46.RES:Rescheduling_interrupts
244.00 ± 21% -35.6% 157.25 ± 11% interrupts.CPU47.RES:Rescheduling_interrupts
230.00 ± 7% -21.3% 181.00 ± 11% interrupts.CPU48.RES:Rescheduling_interrupts
281.00 ± 13% -27.4% 204.00 ± 15% interrupts.CPU53.RES:Rescheduling_interrupts
256.75 ± 5% -18.4% 209.50 ± 12% interrupts.CPU54.RES:Rescheduling_interrupts
2433 ± 9% +68.4% 4098 ± 35% interrupts.CPU58.NMI:Non-maskable_interrupts
2433 ± 9% +68.4% 4098 ± 35% interrupts.CPU58.PMI:Performance_monitoring_interrupts
316.00 ± 25% -41.4% 185.25 ± 13% interrupts.CPU59.RES:Rescheduling_interrupts
2703 ± 38% +56.0% 4217 ± 31% interrupts.CPU60.NMI:Non-maskable_interrupts
2703 ± 38% +56.0% 4217 ± 31% interrupts.CPU60.PMI:Performance_monitoring_interrupts
2425 ± 16% +39.9% 3394 ± 27% interrupts.CPU61.NMI:Non-maskable_interrupts
2425 ± 16% +39.9% 3394 ± 27% interrupts.CPU61.PMI:Performance_monitoring_interrupts
2388 ± 18% +69.5% 4047 ± 29% interrupts.CPU66.NMI:Non-maskable_interrupts
2388 ± 18% +69.5% 4047 ± 29% interrupts.CPU66.PMI:Performance_monitoring_interrupts
2322 ± 11% +93.4% 4491 ± 35% interrupts.CPU67.NMI:Non-maskable_interrupts
2322 ± 11% +93.4% 4491 ± 35% interrupts.CPU67.PMI:Performance_monitoring_interrupts
319.00 ± 40% -44.7% 176.25 ± 9% interrupts.CPU67.RES:Rescheduling_interrupts
2512 ± 8% +28.1% 3219 ± 25% interrupts.CPU70.NMI:Non-maskable_interrupts
2512 ± 8% +28.1% 3219 ± 25% interrupts.CPU70.PMI:Performance_monitoring_interrupts
2290 ± 39% +78.7% 4094 ± 28% interrupts.CPU74.NMI:Non-maskable_interrupts
2290 ± 39% +78.7% 4094 ± 28% interrupts.CPU74.PMI:Performance_monitoring_interrupts
2446 ± 40% +94.8% 4764 ± 23% interrupts.CPU75.NMI:Non-maskable_interrupts
2446 ± 40% +94.8% 4764 ± 23% interrupts.CPU75.PMI:Performance_monitoring_interrupts
426.75 ± 61% -67.7% 138.00 ± 8% interrupts.CPU75.RES:Rescheduling_interrupts
192.50 ± 13% +45.6% 280.25 ± 45% interrupts.CPU76.RES:Rescheduling_interrupts
274.25 ± 34% -42.2% 158.50 ± 34% interrupts.CPU77.RES:Rescheduling_interrupts
2357 ± 9% +73.0% 4078 ± 23% interrupts.CPU78.NMI:Non-maskable_interrupts
2357 ± 9% +73.0% 4078 ± 23% interrupts.CPU78.PMI:Performance_monitoring_interrupts
348.50 ± 53% -47.3% 183.75 ± 29% interrupts.CPU80.RES:Rescheduling_interrupts
2650 ± 43% +46.2% 3874 ± 36% interrupts.CPU84.NMI:Non-maskable_interrupts
2650 ± 43% +46.2% 3874 ± 36% interrupts.CPU84.PMI:Performance_monitoring_interrupts
2235 ± 10% +117.8% 4867 ± 10% interrupts.CPU90.NMI:Non-maskable_interrupts
2235 ± 10% +117.8% 4867 ± 10% interrupts.CPU90.PMI:Performance_monitoring_interrupts
2606 ± 33% +38.1% 3598 ± 21% interrupts.CPU92.NMI:Non-maskable_interrupts
2606 ± 33% +38.1% 3598 ± 21% interrupts.CPU92.PMI:Performance_monitoring_interrupts
408.75 ± 58% -56.8% 176.75 ± 25% interrupts.CPU92.RES:Rescheduling_interrupts
399.00 ± 64% -63.6% 145.25 ± 16% interrupts.CPU93.RES:Rescheduling_interrupts
314.75 ± 36% -44.2% 175.75 ± 13% interrupts.CPU94.RES:Rescheduling_interrupts
191.00 ± 15% -29.1% 135.50 ± 9% interrupts.CPU97.RES:Rescheduling_interrupts
94.00 ± 8% +50.0% 141.00 ± 12% interrupts.IWI:IRQ_work_interrupts
841457 ± 7% +16.6% 980751 ± 3% interrupts.NMI:Non-maskable_interrupts
841457 ± 7% +16.6% 980751 ± 3% interrupts.PMI:Performance_monitoring_interrupts
12.75 ± 11% -4.1 8.67 ± 31% perf-profile.calltrace.cycles-pp.do_rw_once
1.02 ± 16% -0.6 0.47 ± 59% perf-profile.calltrace.cycles-pp.sched_clock.sched_clock_cpu.cpuidle_enter_state.cpuidle_enter.do_idle
1.10 ± 15% -0.4 0.66 ± 14% perf-profile.calltrace.cycles-pp.sched_clock_cpu.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
1.05 ± 16% -0.4 0.61 ± 14% perf-profile.calltrace.cycles-pp.native_sched_clock.sched_clock.sched_clock_cpu.cpuidle_enter_state.cpuidle_enter
1.58 ± 4% +0.3 1.91 ± 7% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.copy_page
0.79 ± 26% +0.5 1.27 ± 18% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.79 ± 26% +0.5 1.27 ± 18% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.79 ± 26% +0.5 1.27 ± 18% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.11 ± 4% +0.5 2.60 ± 7% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.osq_lock.__mutex_lock.hugetlb_fault.handle_mm_fault
0.83 ± 26% +0.5 1.32 ± 18% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
0.83 ± 26% +0.5 1.32 ± 18% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.90 ± 5% +0.6 2.45 ± 7% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.copy_page.copy_subpage
0.65 ± 62% +0.6 1.20 ± 15% perf-profile.calltrace.cycles-pp.alloc_fresh_huge_page.alloc_surplus_huge_page.alloc_huge_page.hugetlb_cow.hugetlb_fault
0.60 ± 62% +0.6 1.16 ± 18% perf-profile.calltrace.cycles-pp.free_huge_page.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap
0.95 ± 17% +0.6 1.52 ± 8% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.mutex_spin_on_owner
0.61 ± 62% +0.6 1.18 ± 18% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput
0.61 ± 62% +0.6 1.19 ± 19% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit
0.61 ± 62% +0.6 1.19 ± 19% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit
0.64 ± 61% +0.6 1.23 ± 18% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.64 ± 61% +0.6 1.23 ± 18% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
1.30 ± 9% +0.6 1.92 ± 8% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.mutex_spin_on_owner.__mutex_lock
0.19 ±173% +0.7 0.89 ± 20% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_huge_page.release_pages.tlb_flush_mmu
0.19 ±173% +0.7 0.90 ± 20% perf-profile.calltrace.cycles-pp._raw_spin_lock.free_huge_page.release_pages.tlb_flush_mmu.tlb_finish_mmu
0.00 +0.8 0.77 ± 30% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prep_new_huge_page.alloc_fresh_huge_page.alloc_surplus_huge_page
0.00 +0.8 0.78 ± 30% perf-profile.calltrace.cycles-pp._raw_spin_lock.prep_new_huge_page.alloc_fresh_huge_page.alloc_surplus_huge_page.alloc_huge_page
0.00 +0.8 0.79 ± 29% perf-profile.calltrace.cycles-pp.prep_new_huge_page.alloc_fresh_huge_page.alloc_surplus_huge_page.alloc_huge_page.hugetlb_cow
0.82 ± 67% +0.9 1.72 ± 22% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.alloc_huge_page.hugetlb_cow.hugetlb_fault
0.84 ± 66% +0.9 1.74 ± 20% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.alloc_surplus_huge_page.alloc_huge_page.hugetlb_cow
2.52 ± 6% +0.9 3.44 ± 9% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.copy_page.copy_subpage.copy_user_huge_page
0.83 ± 67% +0.9 1.75 ± 21% perf-profile.calltrace.cycles-pp._raw_spin_lock.alloc_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault
0.84 ± 66% +0.9 1.77 ± 20% perf-profile.calltrace.cycles-pp._raw_spin_lock.alloc_surplus_huge_page.alloc_huge_page.hugetlb_cow.hugetlb_fault
1.64 ± 12% +1.0 2.67 ± 7% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.mutex_spin_on_owner.__mutex_lock.hugetlb_fault
1.65 ± 45% +1.3 2.99 ± 18% perf-profile.calltrace.cycles-pp.alloc_surplus_huge_page.alloc_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault
1.74 ± 13% +1.4 3.16 ± 6% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.mutex_spin_on_owner.__mutex_lock.hugetlb_fault.handle_mm_fault
2.56 ± 48% +2.2 4.81 ± 19% perf-profile.calltrace.cycles-pp.alloc_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.__do_page_fault
12.64 ± 14% +3.6 16.20 ± 8% perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.hugetlb_fault.handle_mm_fault.__do_page_fault
2.97 ± 7% +3.8 6.74 ± 9% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.copy_page.copy_subpage.copy_user_huge_page.hugetlb_cow
19.99 ± 9% +4.1 24.05 ± 6% perf-profile.calltrace.cycles-pp.hugetlb_cow.hugetlb_fault.handle_mm_fault.__do_page_fault.do_page_fault
1.37 ± 15% -0.5 0.83 ± 13% perf-profile.children.cycles-pp.sched_clock_cpu
1.31 ± 16% -0.5 0.78 ± 13% perf-profile.children.cycles-pp.sched_clock
1.29 ± 16% -0.5 0.77 ± 13% perf-profile.children.cycles-pp.native_sched_clock
1.80 ± 2% -0.3 1.47 ± 10% perf-profile.children.cycles-pp.task_tick_fair
0.73 ± 2% -0.2 0.54 ± 11% perf-profile.children.cycles-pp.update_curr
0.42 ± 17% -0.2 0.27 ± 16% perf-profile.children.cycles-pp.account_process_tick
0.73 ± 10% -0.2 0.58 ± 9% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.27 ± 6% -0.1 0.14 ± 14% perf-profile.children.cycles-pp.__acct_update_integrals
0.27 ± 18% -0.1 0.16 ± 13% perf-profile.children.cycles-pp.rcu_segcblist_ready_cbs
0.40 ± 12% -0.1 0.30 ± 14% perf-profile.children.cycles-pp.__next_timer_interrupt
0.47 ± 7% -0.1 0.39 ± 13% perf-profile.children.cycles-pp.update_rq_clock
0.29 ± 12% -0.1 0.21 ± 15% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.21 ± 7% -0.1 0.14 ± 12% perf-profile.children.cycles-pp.account_system_index_time
0.38 ± 2% -0.1 0.31 ± 12% perf-profile.children.cycles-pp.timerqueue_add
0.26 ± 11% -0.1 0.20 ± 13% perf-profile.children.cycles-pp.find_next_bit
0.23 ± 15% -0.1 0.17 ± 15% perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit
0.14 ± 8% -0.1 0.07 ± 14% perf-profile.children.cycles-pp.account_user_time
0.17 ± 6% -0.0 0.12 ± 10% perf-profile.children.cycles-pp.cpuacct_charge
0.18 ± 20% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.irq_work_tick
0.11 ± 13% -0.0 0.07 ± 25% perf-profile.children.cycles-pp.tick_sched_do_timer
0.12 ± 10% -0.0 0.08 ± 15% perf-profile.children.cycles-pp.get_cpu_device
0.07 ± 11% -0.0 0.04 ± 58% perf-profile.children.cycles-pp.raise_softirq
0.12 ± 3% -0.0 0.09 ± 8% perf-profile.children.cycles-pp.write
0.11 ± 13% +0.0 0.14 ± 8% perf-profile.children.cycles-pp.native_write_msr
0.09 ± 9% +0.0 0.11 ± 7% perf-profile.children.cycles-pp.finish_task_switch
0.10 ± 10% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.schedule_idle
0.07 ± 6% +0.0 0.10 ± 12% perf-profile.children.cycles-pp.__read_nocancel
0.04 ± 58% +0.0 0.07 ± 15% perf-profile.children.cycles-pp.__free_pages_ok
0.06 ± 7% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.perf_read
0.07 +0.0 0.11 ± 14% perf-profile.children.cycles-pp.perf_evsel__read_counter
0.07 +0.0 0.11 ± 13% perf-profile.children.cycles-pp.cmd_stat
0.07 +0.0 0.11 ± 13% perf-profile.children.cycles-pp.__run_perf_stat
0.07 +0.0 0.11 ± 13% perf-profile.children.cycles-pp.process_interval
0.07 +0.0 0.11 ± 13% perf-profile.children.cycles-pp.read_counters
0.07 ± 22% +0.0 0.11 ± 19% perf-profile.children.cycles-pp.__handle_mm_fault
0.07 ± 19% +0.1 0.13 ± 8% perf-profile.children.cycles-pp.rb_erase
0.03 ±100% +0.1 0.09 ± 9% perf-profile.children.cycles-pp.smp_call_function_single
0.01 ±173% +0.1 0.08 ± 11% perf-profile.children.cycles-pp.perf_event_read
0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.__perf_event_read_value
0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.08 ± 17% +0.1 0.15 ± 8% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.04 ±103% +0.1 0.13 ± 58% perf-profile.children.cycles-pp.shmem_getpage_gfp
0.38 ± 14% +0.1 0.51 ± 6% perf-profile.children.cycles-pp.run_timer_softirq
0.11 ± 4% +0.3 0.37 ± 32% perf-profile.children.cycles-pp.worker_thread
0.20 ± 5% +0.3 0.48 ± 25% perf-profile.children.cycles-pp.ret_from_fork
0.20 ± 4% +0.3 0.48 ± 25% perf-profile.children.cycles-pp.kthread
0.00 +0.3 0.29 ± 38% perf-profile.children.cycles-pp.memcpy_erms
0.00 +0.3 0.29 ± 38% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work
0.00 +0.3 0.31 ± 37% perf-profile.children.cycles-pp.process_one_work
0.47 ± 48% +0.4 0.91 ± 19% perf-profile.children.cycles-pp.prep_new_huge_page
0.70 ± 29% +0.5 1.16 ± 18% perf-profile.children.cycles-pp.free_huge_page
0.73 ± 29% +0.5 1.19 ± 18% perf-profile.children.cycles-pp.tlb_flush_mmu
0.72 ± 29% +0.5 1.18 ± 18% perf-profile.children.cycles-pp.release_pages
0.73 ± 29% +0.5 1.19 ± 18% perf-profile.children.cycles-pp.tlb_finish_mmu
0.76 ± 27% +0.5 1.23 ± 18% perf-profile.children.cycles-pp.exit_mmap
0.77 ± 27% +0.5 1.24 ± 18% perf-profile.children.cycles-pp.mmput
0.79 ± 26% +0.5 1.27 ± 18% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.79 ± 26% +0.5 1.27 ± 18% perf-profile.children.cycles-pp.do_group_exit
0.79 ± 26% +0.5 1.27 ± 18% perf-profile.children.cycles-pp.do_exit
1.28 ± 29% +0.5 1.76 ± 9% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.77 ± 28% +0.5 1.26 ± 13% perf-profile.children.cycles-pp.alloc_fresh_huge_page
1.53 ± 15% +0.7 2.26 ± 14% perf-profile.children.cycles-pp.do_syscall_64
1.53 ± 15% +0.7 2.27 ± 14% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
1.13 ± 3% +0.9 2.07 ± 14% perf-profile.children.cycles-pp.interrupt_entry
0.79 ± 9% +1.0 1.76 ± 5% perf-profile.children.cycles-pp.perf_event_task_tick
1.71 ± 39% +1.4 3.08 ± 16% perf-profile.children.cycles-pp.alloc_surplus_huge_page
2.66 ± 42% +2.3 4.94 ± 17% perf-profile.children.cycles-pp.alloc_huge_page
2.89 ± 45% +2.7 5.54 ± 18% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
3.34 ± 35% +2.7 6.02 ± 17% perf-profile.children.cycles-pp._raw_spin_lock
12.77 ± 14% +3.9 16.63 ± 7% perf-profile.children.cycles-pp.mutex_spin_on_owner
20.12 ± 9% +4.0 24.16 ± 6% perf-profile.children.cycles-pp.hugetlb_cow
15.40 ± 10% -3.6 11.84 ± 28% perf-profile.self.cycles-pp.do_rw_once
4.02 ± 9% -1.3 2.73 ± 30% perf-profile.self.cycles-pp.do_access
2.00 ± 14% -0.6 1.41 ± 13% perf-profile.self.cycles-pp.cpuidle_enter_state
1.26 ± 16% -0.5 0.74 ± 13% perf-profile.self.cycles-pp.native_sched_clock
0.42 ± 17% -0.2 0.27 ± 16% perf-profile.self.cycles-pp.account_process_tick
0.27 ± 19% -0.2 0.12 ± 17% perf-profile.self.cycles-pp.timerqueue_del
0.53 ± 3% -0.1 0.38 ± 11% perf-profile.self.cycles-pp.update_curr
0.27 ± 6% -0.1 0.14 ± 14% perf-profile.self.cycles-pp.__acct_update_integrals
0.27 ± 18% -0.1 0.16 ± 13% perf-profile.self.cycles-pp.rcu_segcblist_ready_cbs
0.61 ± 4% -0.1 0.51 ± 8% perf-profile.self.cycles-pp.task_tick_fair
0.20 ± 8% -0.1 0.12 ± 14% perf-profile.self.cycles-pp.account_system_index_time
0.23 ± 15% -0.1 0.16 ± 17% perf-profile.self.cycles-pp.rcu_dynticks_eqs_exit
0.25 ± 11% -0.1 0.18 ± 14% perf-profile.self.cycles-pp.find_next_bit
0.10 ± 11% -0.1 0.03 ±100% perf-profile.self.cycles-pp.tick_sched_do_timer
0.29 -0.1 0.23 ± 11% perf-profile.self.cycles-pp.timerqueue_add
0.12 ± 10% -0.1 0.06 ± 17% perf-profile.self.cycles-pp.account_user_time
0.22 ± 15% -0.1 0.16 ± 6% perf-profile.self.cycles-pp.scheduler_tick
0.17 ± 6% -0.0 0.12 ± 10% perf-profile.self.cycles-pp.cpuacct_charge
0.18 ± 20% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.irq_work_tick
0.07 ± 13% -0.0 0.03 ±100% perf-profile.self.cycles-pp.update_process_times
0.12 ± 7% -0.0 0.08 ± 15% perf-profile.self.cycles-pp.get_cpu_device
0.07 ± 11% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.raise_softirq
0.12 ± 11% -0.0 0.09 ± 7% perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
0.11 ± 11% +0.0 0.14 ± 6% perf-profile.self.cycles-pp.native_write_msr
0.10 ± 5% +0.1 0.15 ± 8% perf-profile.self.cycles-pp.__remove_hrtimer
0.07 ± 23% +0.1 0.13 ± 8% perf-profile.self.cycles-pp.rb_erase
0.08 ± 17% +0.1 0.15 ± 7% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.00 +0.1 0.08 ± 10% perf-profile.self.cycles-pp.smp_call_function_single
0.32 ± 17% +0.1 0.42 ± 7% perf-profile.self.cycles-pp.run_timer_softirq
0.22 ± 5% +0.1 0.34 ± 4% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.45 ± 15% +0.2 0.60 ± 12% perf-profile.self.cycles-pp.rcu_irq_enter
0.31 ± 8% +0.2 0.46 ± 16% perf-profile.self.cycles-pp.irq_enter
0.29 ± 10% +0.2 0.44 ± 16% perf-profile.self.cycles-pp.apic_timer_interrupt
0.71 ± 30% +0.2 0.92 ± 8% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.00 +0.3 0.28 ± 37% perf-profile.self.cycles-pp.memcpy_erms
1.12 ± 3% +0.9 2.02 ± 15% perf-profile.self.cycles-pp.interrupt_entry
0.79 ± 9% +0.9 1.73 ± 5% perf-profile.self.cycles-pp.perf_event_task_tick
2.49 ± 45% +2.1 4.55 ± 20% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
10.95 ± 15% +2.7 13.61 ± 8% perf-profile.self.cycles-pp.mutex_spin_on_owner
vm-scalability.throughput
1.6e+07 +-+---------------------------------------------------------------+
|..+.+ +..+.+..+.+. +. +..+.+..+.+..+.+..+.+..+ + |
1.4e+07 +-+ : : O O O O |
1.2e+07 O-+O O O O O O O O O O O O O O O O O O
| : : O O O O |
1e+07 +-+ : : |
| : : |
8e+06 +-+ : : |
| : : |
6e+06 +-+ : : |
4e+06 +-+ : : |
| :: |
2e+06 +-+ : |
| : |
0 +-+---------------------------------------------------------------+
vm-scalability.time.minor_page_faults
2.5e+06 +-+---------------------------------------------------------------+
| |
|..+.+ +..+.+..+.+..+.+..+.+.. .+. .+.+..+.+..+.+..+.+..+ |
2e+06 +-+ : : +. +. |
O O O: O O O O O O O O O O |
| : : O O O O O O O O O O O O O O
1.5e+06 +-+ : : |
| : : |
1e+06 +-+ : : |
| : : |
| : : |
500000 +-+ : : |
| : |
| : |
0 +-+---------------------------------------------------------------+
vm-scalability.workload
3.5e+09 +-+---------------------------------------------------------------+
| .+. .+.+.. .+.. |
3e+09 +-+ + +..+.+..+.+..+.+. +..+.+..+.+..+.+..+.+..+ + |
| : : O O O |
2.5e+09 O-+O O: O O O O O O O O O |
| : : O O O O O O O O O O O O
2e+09 +-+ : : |
| : : |
1.5e+09 +-+ : : |
| : : |
1e+09 +-+ : : |
| : : |
5e+08 +-+ : |
| : |
0 +-+---------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
9 months, 3 weeks
- 8
- 50
[mm/debug] 87c4696d57: kernel_BUG_at_include/linux/mm.h
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 87c4696d57b5e380bc2720fdff0772b042462b7d ("[PATCH V11 RESEND] mm/debug: Add tests validating architecture page table helpers")
url: https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-t...
in testcase: trinity
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+--------------------------------------------------+----------+------------+
| | v5.5-rc3 | 87c4696d57 |
+--------------------------------------------------+----------+------------+
| boot_successes | 0 | 0 |
| boot_failures | 153 | 12 |
| BUG:workqueue_lockup-pool | 121 | |
| BUG:soft_lockup-CPU##stuck_for#s![dma-fence:#:#] | 25 | |
| EIP:thread_signal_callback | 14 | |
| Kernel_panic-not_syncing:softlockup:hung_tasks | 25 | |
| EIP:kthread_should_stop | 11 | |
| BUG:kernel_hang_in_boot_stage | 7 | |
| kernel_BUG_at_include/linux/mm.h | 0 | 12 |
| invalid_opcode:#[##] | 0 | 12 |
| EIP:__free_pages | 0 | 12 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 12 |
+--------------------------------------------------+----------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 9.781974] kernel BUG at include/linux/mm.h:592!
[ 9.782810] invalid opcode: 0000 [#1] PTI
[ 9.783443] CPU: 0 PID: 1 Comm: swapper Not tainted 5.5.0-rc3-00001-g87c4696d57b5e #1
[ 9.784528] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 9.785756] EIP: __free_pages+0x14/0x40
[ 9.786442] Code: 0c 9c 5e fa 89 d8 e8 5b f3 ff ff 56 9d 5b 5e 5d c3 8d 74 26 00 90 8b 48 1c 55 89 e5 85 c9 75 16 ba b4 b6 84 d6 e8 ac 49 fe ff <0f> 0b 8d b4 26 00 00 00 00 8d 76 00 ff 48 1c 75 10 85 d2 75 07 e8
[ 9.789697] EAX: d68761f7 EBX: ea52f000 ECX: ea4f8520 EDX: d684b6b4
[ 9.790850] ESI: 00000000 EDI: ef45e000 EBP: ea501f08 ESP: ea501f08
[ 9.791879] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00010286
[ 9.792783] CR0: 80050033 CR2: ffffffff CR3: 16d00000 CR4: 000406b0
[ 9.792783] Call Trace:
[ 9.792783] free_pages+0x3c/0x50
[ 9.792783] pgd_free+0x5a/0x170
[ 9.792783] __mmdrop+0x42/0xe0
[ 9.792783] debug_vm_pgtable+0x54f/0x567
[ 9.792783] kernel_init_freeable+0x90/0x1e3
[ 9.792783] ? rest_init+0xf0/0xf0
[ 9.792783] kernel_init+0x8/0xf0
[ 9.792783] ret_from_fork+0x19/0x24
[ 9.792783] Modules linked in:
[ 9.792803] ---[ end trace 91b7335adcf0b656 ]---
To reproduce:
# build kernel
cd linux
cp config-5.5.0-rc3-00001-g87c4696d57b5e .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=i386 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
9 months, 3 weeks
- 3
- 2
[cpuidle] 259231a045: will-it-scale.per_process_ops -12.6% regression
by kernel test robot
Greeting,
FYI, we noticed a -12.6% regression of will-it-scale.per_process_ops due to commit:
commit: 259231a045616c4101d023a8f4dcc8379af265a6 ("cpuidle: add poll_limit_ns to cpuidle_device structure")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git master
in testcase: will-it-scale
on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory
with following parameters:
nr_task: 100%
mode: process
test: mmap1
cpufreq_governor: performance
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-2019-05-14.cgz/lkp-knm01/mmap1/will-it-scale
commit:
fa86ee90eb ("add cpuidle-haltpoll driver")
259231a045 ("cpuidle: add poll_limit_ns to cpuidle_device structure")
fa86ee90eb111126 259231a045616c4101d023a8f4d
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
%stddev %change %stddev
\ | \
1611 -12.6% 1408 will-it-scale.per_process_ops
464144 -12.6% 405580 will-it-scale.workload
1581 ± 2% +3.3% 1633 vmstat.system.cs
35.13 -1.0% 34.78 boot-time.dhcp
11888 -1.0% 11765 boot-time.idle
5207 ± 4% +25.7% 6547 ± 7% slabinfo.kmalloc-rcl-64.active_objs
5207 ± 4% +25.7% 6547 ± 7% slabinfo.kmalloc-rcl-64.num_objs
1.07 ± 53% -38.8% 0.66 ± 6% turbostat.CPU%c6
2.63 -14.9% 2.23 ± 2% turbostat.RAMWatt
874.11 ± 2% +10.7% 967.81 ± 5% sched_debug.cfs_rq:/.exec_clock.stddev
4672082 ± 24% +60.3% 7488441 ± 26% sched_debug.cpu.avg_idle.max
662.84 ± 3% +21.4% 804.72 ± 6% sched_debug.cpu.clock.stddev
662.84 ± 3% +21.4% 804.72 ± 6% sched_debug.cpu.clock_task.stddev
1185029 ± 14% +51.4% 1793617 ± 28% sched_debug.cpu.max_idle_balance_cost.max
75638 ± 25% +62.8% 123124 ± 17% sched_debug.cpu.max_idle_balance_cost.stddev
0.00 ± 2% +21.3% 0.00 ± 6% sched_debug.cpu.next_balance.stddev
1.16 ± 6% -0.1 1.05 perf-stat.i.branch-miss-rate%
1.104e+08 -5.6% 1.042e+08 perf-stat.i.branch-misses
45924834 -3.9% 44147120 perf-stat.i.cache-misses
1455 +2.1% 1485 perf-stat.i.context-switches
9825 +4.4% 10255 perf-stat.i.cycles-between-cache-misses
0.18 ± 8% -0.0 0.16 perf-stat.i.iTLB-load-miss-rate%
67884172 -6.4% 63509655 perf-stat.i.iTLB-load-misses
596.76 +7.3% 640.20 perf-stat.i.instructions-per-iTLB-miss
1.12 -0.1 1.05 perf-stat.overall.branch-miss-rate%
9817 +4.3% 10239 perf-stat.overall.cycles-between-cache-misses
0.17 -0.0 0.16 perf-stat.overall.iTLB-load-miss-rate%
596.10 +7.1% 638.47 perf-stat.overall.instructions-per-iTLB-miss
26461122 +13.8% 30121468 perf-stat.overall.path-length
1.098e+08 -5.7% 1.035e+08 perf-stat.ps.branch-misses
45700304 -3.9% 43918015 perf-stat.ps.cache-misses
67526222 -6.5% 63149942 perf-stat.ps.iTLB-load-misses
481381 ± 18% +17.7% 566437 ± 7% interrupts.CPU0.LOC:Local_timer_interrupts
79.50 ±111% -76.4% 18.75 ± 99% interrupts.CPU100.RES:Rescheduling_interrupts
5783 ± 33% -17.7% 4757 ± 34% interrupts.CPU103.NMI:Non-maskable_interrupts
5783 ± 33% -17.7% 4757 ± 34% interrupts.CPU103.PMI:Performance_monitoring_interrupts
128.00 ±105% -92.8% 9.25 ± 37% interrupts.CPU107.RES:Rescheduling_interrupts
5769 ± 33% -33.5% 3837 interrupts.CPU109.NMI:Non-maskable_interrupts
5769 ± 33% -33.5% 3837 interrupts.CPU109.PMI:Performance_monitoring_interrupts
6774 ± 24% -29.4% 4785 ± 34% interrupts.CPU111.NMI:Non-maskable_interrupts
6774 ± 24% -29.4% 4785 ± 34% interrupts.CPU111.PMI:Performance_monitoring_interrupts
140.50 ± 43% -78.3% 30.50 ±118% interrupts.CPU118.RES:Rescheduling_interrupts
5776 ± 33% -33.7% 3830 interrupts.CPU120.NMI:Non-maskable_interrupts
5776 ± 33% -33.7% 3830 interrupts.CPU120.PMI:Performance_monitoring_interrupts
5801 ± 33% -17.7% 4776 ± 34% interrupts.CPU126.NMI:Non-maskable_interrupts
5801 ± 33% -17.7% 4776 ± 34% interrupts.CPU126.PMI:Performance_monitoring_interrupts
5772 ± 33% -17.7% 4749 ± 34% interrupts.CPU138.NMI:Non-maskable_interrupts
5772 ± 33% -17.7% 4749 ± 34% interrupts.CPU138.PMI:Performance_monitoring_interrupts
5786 ± 33% -18.4% 4722 ± 34% interrupts.CPU14.NMI:Non-maskable_interrupts
5786 ± 33% -18.4% 4722 ± 34% interrupts.CPU14.PMI:Performance_monitoring_interrupts
3844 +74.8% 6718 ± 24% interrupts.CPU167.NMI:Non-maskable_interrupts
3844 +74.8% 6718 ± 24% interrupts.CPU167.PMI:Performance_monitoring_interrupts
96.50 ± 98% -82.9% 16.50 ± 98% interrupts.CPU172.RES:Rescheduling_interrupts
5757 ± 33% -33.8% 3809 interrupts.CPU175.NMI:Non-maskable_interrupts
5757 ± 33% -33.8% 3809 interrupts.CPU175.PMI:Performance_monitoring_interrupts
481026 ± 19% +18.0% 567722 ± 7% interrupts.CPU18.LOC:Local_timer_interrupts
58.50 ± 91% -71.4% 16.75 ± 94% interrupts.CPU180.RES:Rescheduling_interrupts
6737 ± 24% -43.7% 3795 interrupts.CPU184.NMI:Non-maskable_interrupts
6737 ± 24% -43.7% 3795 interrupts.CPU184.PMI:Performance_monitoring_interrupts
5770 ± 32% -18.1% 4728 ± 34% interrupts.CPU188.NMI:Non-maskable_interrupts
5770 ± 32% -18.1% 4728 ± 34% interrupts.CPU188.PMI:Performance_monitoring_interrupts
281.00 ± 87% -80.5% 54.75 ± 49% interrupts.CPU188.RES:Rescheduling_interrupts
529.50 ±124% -95.6% 23.50 ± 81% interrupts.CPU189.RES:Rescheduling_interrupts
7713 -50.7% 3803 interrupts.CPU192.NMI:Non-maskable_interrupts
7713 -50.7% 3803 interrupts.CPU192.PMI:Performance_monitoring_interrupts
5762 ± 33% -33.9% 3809 interrupts.CPU203.NMI:Non-maskable_interrupts
5762 ± 33% -33.9% 3809 interrupts.CPU203.PMI:Performance_monitoring_interrupts
5783 ± 33% -17.9% 4750 ± 35% interrupts.CPU217.NMI:Non-maskable_interrupts
5783 ± 33% -17.9% 4750 ± 35% interrupts.CPU217.PMI:Performance_monitoring_interrupts
16.75 ± 49% +443.3% 91.00 ±129% interrupts.CPU224.RES:Rescheduling_interrupts
6779 ± 24% -43.5% 3830 interrupts.CPU239.NMI:Non-maskable_interrupts
6779 ± 24% -43.5% 3830 interrupts.CPU239.PMI:Performance_monitoring_interrupts
478215 ± 19% +17.7% 562671 ± 8% interrupts.CPU258.LOC:Local_timer_interrupts
350.50 ± 89% -88.3% 41.00 ±121% interrupts.CPU259.RES:Rescheduling_interrupts
493.00 ±139% -95.1% 24.25 ± 69% interrupts.CPU261.RES:Rescheduling_interrupts
6726 ± 24% -43.3% 3813 interrupts.CPU263.NMI:Non-maskable_interrupts
6726 ± 24% -43.3% 3813 interrupts.CPU263.PMI:Performance_monitoring_interrupts
5755 ± 33% -34.2% 3789 interrupts.CPU270.NMI:Non-maskable_interrupts
5755 ± 33% -34.2% 3789 interrupts.CPU270.PMI:Performance_monitoring_interrupts
5776 ± 32% -33.8% 3825 interrupts.CPU275.NMI:Non-maskable_interrupts
5776 ± 32% -33.8% 3825 interrupts.CPU275.PMI:Performance_monitoring_interrupts
5799 ± 33% -17.8% 4769 ± 35% interrupts.CPU276.NMI:Non-maskable_interrupts
5799 ± 33% -17.8% 4769 ± 35% interrupts.CPU276.PMI:Performance_monitoring_interrupts
6708 ± 24% -29.1% 4754 ± 34% interrupts.CPU277.NMI:Non-maskable_interrupts
6708 ± 24% -29.1% 4754 ± 34% interrupts.CPU277.PMI:Performance_monitoring_interrupts
5809 ± 32% -34.2% 3820 interrupts.CPU3.NMI:Non-maskable_interrupts
5809 ± 32% -34.2% 3820 interrupts.CPU3.PMI:Performance_monitoring_interrupts
150.25 ± 27% -61.9% 57.25 ± 21% interrupts.CPU3.RES:Rescheduling_interrupts
7721 -50.0% 3858 interrupts.CPU34.NMI:Non-maskable_interrupts
7721 -50.0% 3858 interrupts.CPU34.PMI:Performance_monitoring_interrupts
135.00 ± 57% -77.6% 30.25 ± 24% interrupts.CPU36.RES:Rescheduling_interrupts
6764 ± 24% -43.4% 3831 interrupts.CPU39.NMI:Non-maskable_interrupts
6764 ± 24% -43.4% 3831 interrupts.CPU39.PMI:Performance_monitoring_interrupts
7661 -38.2% 4733 ± 33% interrupts.CPU4.NMI:Non-maskable_interrupts
7661 -38.2% 4733 ± 33% interrupts.CPU4.PMI:Performance_monitoring_interrupts
117.25 ± 52% -66.1% 39.75 ± 76% interrupts.CPU40.RES:Rescheduling_interrupts
480972 ± 18% +17.6% 565776 ± 7% interrupts.CPU42.LOC:Local_timer_interrupts
1034 ± 65% -85.0% 154.75 ±127% interrupts.CPU45.RES:Rescheduling_interrupts
24.00 ± 68% +2232.3% 559.75 ±103% interrupts.CPU50.RES:Rescheduling_interrupts
4794 ± 34% +20.8% 5790 ± 33% interrupts.CPU51.NMI:Non-maskable_interrupts
4794 ± 34% +20.8% 5790 ± 33% interrupts.CPU51.PMI:Performance_monitoring_interrupts
48.25 ± 96% +590.7% 333.25 ±110% interrupts.CPU51.RES:Rescheduling_interrupts
6767 ± 24% -43.3% 3837 interrupts.CPU53.NMI:Non-maskable_interrupts
6767 ± 24% -43.3% 3837 interrupts.CPU53.PMI:Performance_monitoring_interrupts
16.25 ± 88% +696.9% 129.50 ±137% interrupts.CPU53.RES:Rescheduling_interrupts
479539 ± 19% +17.4% 562923 ± 8% interrupts.CPU58.LOC:Local_timer_interrupts
6744 ± 24% -43.2% 3830 interrupts.CPU58.NMI:Non-maskable_interrupts
6744 ± 24% -43.2% 3830 interrupts.CPU58.PMI:Performance_monitoring_interrupts
6731 ± 24% -29.5% 4747 ± 35% interrupts.CPU6.NMI:Non-maskable_interrupts
6731 ± 24% -29.5% 4747 ± 35% interrupts.CPU6.PMI:Performance_monitoring_interrupts
7693 -50.2% 3834 interrupts.CPU65.NMI:Non-maskable_interrupts
7693 -50.2% 3834 interrupts.CPU65.PMI:Performance_monitoring_interrupts
5821 ± 33% -34.2% 3829 interrupts.CPU66.NMI:Non-maskable_interrupts
5821 ± 33% -34.2% 3829 interrupts.CPU66.PMI:Performance_monitoring_interrupts
18.75 ± 35% +392.0% 92.25 ± 75% interrupts.CPU72.RES:Rescheduling_interrupts
6782 ± 24% -43.4% 3837 interrupts.CPU80.NMI:Non-maskable_interrupts
6782 ± 24% -43.4% 3837 interrupts.CPU80.PMI:Performance_monitoring_interrupts
5776 ± 33% -17.5% 4764 ± 33% interrupts.CPU97.NMI:Non-maskable_interrupts
5776 ± 33% -17.5% 4764 ± 33% interrupts.CPU97.PMI:Performance_monitoring_interrupts
48.96 -0.3 48.70 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mmap64
48.94 -0.3 48.67 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
49.21 -0.3 48.95 perf-profile.calltrace.cycles-pp.mmap64
46.12 -0.2 45.89 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.percpu_counter_add_batch.__vm_enough_memory.mmap_region.do_mmap
46.08 -0.2 45.85 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.percpu_counter_add_batch.__vm_enough_memory.mmap_region
48.68 -0.2 48.47 perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
48.65 -0.2 48.45 perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
48.47 -0.2 48.27 perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.27 -0.1 1.16 perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
1.19 -0.1 1.09 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap
1.61 -0.1 1.54 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64
50.20 +0.2 50.37 perf-profile.calltrace.cycles-pp.munmap
49.95 +0.2 50.13 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
49.96 +0.2 50.15 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap
46.38 +0.2 46.56 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.percpu_counter_add_batch.__do_munmap.__vm_munmap
49.67 +0.2 49.88 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
49.64 +0.2 49.85 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
49.51 +0.3 49.76 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe
47.52 +0.4 47.88 perf-profile.calltrace.cycles-pp.percpu_counter_add_batch.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64
0.00 +1.0 1.03 ± 4% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.percpu_counter_add_batch.__vm_enough_memory
0.00 +1.0 1.03 ± 6% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.percpu_counter_add_batch.__do_munmap
0.00 +1.1 1.10 ± 4% perf-profile.calltrace.cycles-pp.apic_timer_interrupt._raw_spin_unlock_irqrestore.percpu_counter_add_batch.__vm_enough_memory.mmap_region
0.00 +1.1 1.10 ± 6% perf-profile.calltrace.cycles-pp.apic_timer_interrupt._raw_spin_unlock_irqrestore.percpu_counter_add_batch.__do_munmap.__vm_munmap
0.00 +1.1 1.12 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.percpu_counter_add_batch.__vm_enough_memory.mmap_region.do_mmap
0.00 +1.1 1.13 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.percpu_counter_add_batch.__do_munmap.__vm_munmap.__x64_sys_munmap
0.00 +1.4 1.40 ± 4% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore
0.00 +1.7 1.72 ± 5% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.percpu_counter_add_batch
49.22 -0.3 48.96 perf-profile.children.cycles-pp.mmap64
48.68 -0.2 48.47 perf-profile.children.cycles-pp.ksys_mmap_pgoff
48.66 -0.2 48.45 perf-profile.children.cycles-pp.vm_mmap_pgoff
48.47 -0.2 48.27 perf-profile.children.cycles-pp.do_mmap
1.27 -0.1 1.16 perf-profile.children.cycles-pp.unmap_vmas
0.36 ± 3% -0.1 0.25 ± 5% perf-profile.children.cycles-pp.perf_event_mmap
1.23 -0.1 1.13 perf-profile.children.cycles-pp.unmap_page_range
1.62 -0.1 1.54 perf-profile.children.cycles-pp.unmap_region
0.20 ± 6% -0.1 0.14 ± 8% perf-profile.children.cycles-pp.perf_iterate_sb
0.55 ± 2% -0.0 0.51 ± 2% perf-profile.children.cycles-pp.___might_sleep
0.17 ± 6% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.22 -0.0 0.19 ± 2% perf-profile.children.cycles-pp.get_unmapped_area
0.16 ± 6% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.17 -0.0 0.15 ± 3% perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown
0.30 ± 2% -0.0 0.28 perf-profile.children.cycles-pp._cond_resched
0.19 ± 3% -0.0 0.17 ± 2% perf-profile.children.cycles-pp.free_p4d_range
0.11 -0.0 0.10 ± 5% perf-profile.children.cycles-pp.unmapped_area_topdown
0.12 +0.0 0.14 perf-profile.children.cycles-pp.security_vm_enough_memory_mm
0.08 +0.0 0.10 ± 4% perf-profile.children.cycles-pp.selinux_vm_enough_memory
0.07 +0.0 0.10 ± 4% perf-profile.children.cycles-pp.cred_has_capability
0.04 ± 57% +0.0 0.07 ± 7% perf-profile.children.cycles-pp.__might_sleep
0.11 ± 4% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.15 ± 12% +0.0 0.19 ± 6% perf-profile.children.cycles-pp.update_curr
0.20 ± 33% +0.1 0.26 ± 30% perf-profile.children.cycles-pp.vfs_write
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.perf_event_task_tick
0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.avc_has_perm_noaudit
0.53 ± 5% +0.1 0.61 ± 3% perf-profile.children.cycles-pp.task_tick_fair
0.00 +0.1 0.09 perf-profile.children.cycles-pp.lru_add_drain_cpu
0.00 +0.1 0.10 ± 11% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.01 ±173% +0.1 0.11 ± 7% perf-profile.children.cycles-pp.irq_enter
0.00 +0.1 0.10 ± 4% perf-profile.children.cycles-pp.lru_add_drain
0.69 ± 7% +0.1 0.82 ± 4% perf-profile.children.cycles-pp.scheduler_tick
0.87 ± 6% +0.2 1.02 ± 4% perf-profile.children.cycles-pp.update_process_times
0.94 ± 6% +0.2 1.09 ± 4% perf-profile.children.cycles-pp.tick_sched_timer
0.89 ± 6% +0.2 1.04 ± 4% perf-profile.children.cycles-pp.tick_sched_handle
50.23 +0.2 50.40 perf-profile.children.cycles-pp.munmap
1.28 ± 7% +0.2 1.48 ± 4% perf-profile.children.cycles-pp.__hrtimer_run_queues
49.67 +0.2 49.88 perf-profile.children.cycles-pp.__x64_sys_munmap
49.64 +0.2 49.86 perf-profile.children.cycles-pp.__vm_munmap
49.52 +0.3 49.77 perf-profile.children.cycles-pp.__do_munmap
94.81 +0.3 95.07 perf-profile.children.cycles-pp.percpu_counter_add_batch
1.54 ± 9% +0.3 1.83 ± 5% perf-profile.children.cycles-pp.hrtimer_interrupt
1.83 ± 9% +0.4 2.18 ± 5% perf-profile.children.cycles-pp.smp_apic_timer_interrupt
1.95 ± 10% +0.4 2.34 ± 5% perf-profile.children.cycles-pp.apic_timer_interrupt
0.00 +2.4 2.38 ± 5% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.15 ± 5% -0.0 0.10 ± 11% perf-profile.self.cycles-pp.perf_iterate_sb
0.17 ± 6% -0.0 0.13 perf-profile.self.cycles-pp.syscall_return_via_sysret
0.51 -0.0 0.46 ± 2% perf-profile.self.cycles-pp.unmap_page_range
0.52 ± 2% -0.0 0.48 ± 2% perf-profile.self.cycles-pp.___might_sleep
0.10 ± 7% -0.0 0.06 perf-profile.self.cycles-pp.perf_event_mmap
0.15 ± 4% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.15 ± 3% -0.0 0.13 ± 3% perf-profile.self.cycles-pp._cond_resched
0.10 ± 4% -0.0 0.09 perf-profile.self.cycles-pp.unmapped_area_topdown
0.09 -0.0 0.08 perf-profile.self.cycles-pp.vma_compute_subtree_gap
0.08 +0.0 0.10 ± 4% perf-profile.self.cycles-pp.mmap_region
0.05 +0.0 0.08 perf-profile.self.cycles-pp.mmap64
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.task_tick_fair
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.irq_enter
0.04 ± 59% +0.1 0.10 ± 12% perf-profile.self.cycles-pp.hrtimer_interrupt
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.perf_event_task_tick
0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.avc_has_perm_noaudit
0.01 ±173% +0.1 0.08 ± 5% perf-profile.self.cycles-pp.munmap
0.00 +0.1 0.09 ± 5% perf-profile.self.cycles-pp.lru_add_drain_cpu
0.00 +0.1 0.10 ± 11% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.00 +0.1 0.15 ± 5% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
will-it-scale.per_process_ops
1800 +-+------------------------------------------------------------------+
| .+ .++.++.++. .++.++.++.+ .++.++.++.+ .+ .+|
1600 +-++.++ + ++.+ ++.+ ++.++ +.++ + + |
1400 +-+O OO OO OO OO OO :O :O OO OO O OO OO OO OO OO |
O O OO O O : : O |
1200 +-+ : : : : |
1000 +-+ : : : : |
| : : : : |
800 +-+ :: :: |
600 +-+ :: :: |
| :: :: |
400 +-+ :: :: |
200 +-+ : : |
| : : |
0 +-+------------------------------------------------------------------+
will-it-scale.workload
500000 +-+----------------------------------------------------------------+
450000 +-+++.++.++.++.+++.++.+ ++.+ ++.+++.++.++.++.++.+++.++.++.++.++.+|
| O O : : : : O O |
400000 OO+OO O OO O OOO OO OO OO OO OO OO O OO OO OO O |
350000 +-+ : : : : |
| : : : : |
300000 +-+ : : : : |
250000 +-+ : : : : |
200000 +-+ :: :: |
| :: :: |
150000 +-+ :: :: |
100000 +-+ : : |
| : : |
50000 +-+ : : |
0 +-+----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
9 months, 3 weeks
- 3
- 4
[block] f216fdd77b: BUG:kernel_reboot-without-warning_in_boot_stage
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: f216fdd77b5654f8c4f6fac6020d6aabc58878ef ("block: replace seq_zones_bitmap with conv_zones_bitmap")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: kernel_selftests
with following parameters:
group: kselftests-x86
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-------------------------------------------------------------------------+------------+------------+
| | 9b38bb4b1e | f216fdd77b |
+-------------------------------------------------------------------------+------------+------------+
| boot_successes | 0 | 0 |
| boot_failures | 4 | 4 |
| WARNING:at_drivers/base/core.c:#device_links_supplier_sync_state_resume | 4 | |
| RIP:device_links_supplier_sync_state_resume | 4 | |
| BUG:kernel_reboot-without-warning_in_boot_stage | 0 | 4 |
+-------------------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
needed_size: 0x0000000006800000
trampoline_32bit: 0x000000000009d000
Decompressing Linux... Parsing ELF... done.
Booting the kernel.
BUG: kernel reboot-without-warning in boot stage
Linux version 5.4.0-10082-gf216fdd77b565 #1
Command line: ip=::::vm-snb-f9eef226434d::dhcp root=/dev/ram0 user=lkp job=/lkp/jobs/scheduled/vm-snb-f9eef226434d/kernel_selftests-kselftests-x86-debian-x86_64-2019-11-14.cgz-f216fdd77-20191206-11813-1cb6s7n-3.yaml ARCH=x86_64 kconfig=x86_64-randconfig-h002-20191205 branch=block/for-linus commit=f216fdd77b5654f8c4f6fac6020d6aabc58878ef BOOT_IMAGE=/pkg/linux/x86_64-randconfig-h002-20191205/gcc-7/f216fdd77b5654f8c4f6fac6020d6aabc58878ef/vmlinuz-5.4.0-10082-gf216fdd77b565 erst_disable max_uptime=3600 RESULT_ROOT=/result/kernel_selftests/kselftests-x86/vm-snb/debian-x86_64-2019-11-14.cgz/x86_64-randconfig-h002-20191205/gcc-7/f216fdd77b5654f8c4f6fac6020d6aabc58878ef/3 LKP_SERVER=inn debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 net.ifnames=0 printk.devkmsg=on panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 drbd.minor_count=8 systemd.log_level=err ignore_loglevel console=tty0 earlyprintk=ttyS0,115200 console=ttyS0,115200 vga=normal rw rcuperf.shutdown=0 watchdog_thresh=60
Elapsed time: 60
To reproduce:
# build kernel
cd linux
cp config-5.4.0-10082-gf216fdd77b565 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
9 months, 3 weeks
- 3
- 4
[iomap] 3af5053534: WARNING:at_fs/iomap/buffered-io.c:#iomap_readpages
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 3af5053534e4744e3c2374016a939cd0390efa33 ("iomap: add struct iomap_data")
https://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git buffered-uncached
in testcase: ltp
with following parameters:
disk: 1HDD
fs: btrfs
test: syscalls_part1
test-description: The LTP testsuite contains a collection of tools for testing the Linux kernel and related features.
test-url: http://linux-test-project.github.io/
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+----------------------------------------------------+------------+------------+
| | 625c62c892 | 3af5053534 |
+----------------------------------------------------+------------+------------+
| boot_successes | 10 | 2 |
| boot_failures | 0 | 30 |
| WARNING:at_fs/iomap/buffered-io.c:#iomap_readpages | 0 | 30 |
| RIP:iomap_readpages | 0 | 30 |
| BUG:kernel_hang_in_test_stage | 0 | 1 |
+----------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 76.725353] WARNING: CPU: 1 PID: 17101 at fs/iomap/buffered-io.c:481 iomap_readpages+0x1a3/0x200
[ 76.727353] Modules linked in: fuse vfat fat xfs overlay ext4 mbcache jbd2 loop btrfs blake2b_generic xor zstd_decompress zstd_compress raid6_pq libcrc32c dm_mod bochs_drm drm_vram_helper drm_ttm_helper intel_rapl_msr intel_rapl_common ttm drm_kms_helper sr_mod crct10dif_pclmul cdrom crc32_pclmul syscopyarea sysfillrect sysimgblt sg crc32c_intel ghash_clmulni_intel fb_sys_fops ata_generic pata_acpi drm ppdev snd_pcm snd_timer aesni_intel snd crypto_simd cryptd glue_helper ata_piix joydev libata soundcore pcspkr serio_raw parport_pc i2c_piix4 floppy parport ip_tables
[ 76.748388] CPU: 1 PID: 17101 Comm: fallocate04 Not tainted 5.5.0-rc1-00031-g3af5053534e47 #1
[ 76.751738] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 76.754930] RIP: 0010:iomap_readpages+0x1a3/0x200
[ 76.757335] Code: 57 48 8b 47 08 8b 80 d0 00 00 00 83 e8 01 83 f8 01 77 45 e8 af db f6 ff 48 85 db 75 0f 48 8b 44 24 18 48 8b 10 48 39 d0 74 02 <0f> 0b 48 8b 4c 24 48 65 48 33 0c 25 28 00 00 00 89 e8 75 35 48 83
[ 76.764548] RSP: 0018:ffff97b8c0817b38 EFLAGS: 00010293
[ 76.767466] RAX: ffff97b8c0817c80 RBX: 0000000000000000 RCX: 0000000000000000
[ 76.770620] RDX: ffffeaf546997148 RSI: 0000000000000000 RDI: ffffeaf54697ca80
[ 76.773840] RBP: 0000000000000000 R08: ffff97b8c0817970 R09: ffff8964a5f2a000
[ 76.777008] R10: 0000000000000000 R11: 0000000000001000 R12: ffffffffa4358100
[ 76.780263] R13: ffff89648697cd58 R14: ffff97b8c0817c80 R15: ffff89648697cec8
[ 76.782251] FS: 00007f6b905e5500(0000) GS:ffff89653fd00000(0000) knlGS:0000000000000000
[ 76.784334] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 76.786021] CR2: 00007f6b90195120 CR3: 0000000194a72000 CR4: 00000000000406e0
[ 76.787930] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 76.789813] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 76.791726] Call Trace:
[ 76.792997] read_pages+0x6d/0x190
[ 76.794151] ? __do_page_cache_readahead+0x191/0x1d0
[ 76.795584] __do_page_cache_readahead+0x191/0x1d0
[ 76.797062] ondemand_readahead+0x117/0x300
[ 76.798403] generic_file_read_iter+0x988/0xd00
[ 76.799676] ? __switch_to_asm+0x34/0x70
[ 76.800978] ? __switch_to_asm+0x40/0x70
[ 76.802144] ? __switch_to_asm+0x34/0x70
[ 76.803428] ? __switch_to_asm+0x40/0x70
[ 76.804744] ? _cond_resched+0x19/0x30
[ 76.805922] xfs_file_buffered_aio_read+0x53/0xf0 [xfs]
[ 76.807264] xfs_file_read_iter+0x6e/0xd0 [xfs]
[ 76.808694] new_sync_read+0x12a/0x1c0
[ 76.809716] vfs_read+0x9b/0x160
[ 76.810673] ksys_read+0xa1/0xe0
[ 76.811690] do_syscall_64+0x5b/0x1f0
[ 76.812935] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 76.814120] RIP: 0033:0x7f6b90106910
[ 76.815069] Code: b6 fe ff ff 48 8d 3d 0f be 08 00 48 83 ec 08 e8 06 db 01 00 66 0f 1f 44 00 00 83 3d f9 2d 2c 00 00 75 10 b8 00 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 de 9b 01 00 48 89 04 24
[ 76.818680] RSP: 002b:00007ffe885b26a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 76.820295] RAX: ffffffffffffffda RBX: 0000000000003000 RCX: 00007f6b90106910
[ 76.821833] RDX: 0000000000003000 RSI: 00007ffe885b2710 RDI: 0000000000000007
[ 76.823369] RBP: 00007ffe885b2710 R08: 0000000000000007 R09: 00007ffe885b2710
[ 76.824915] R10: 0000000000000073 R11: 0000000000000246 R12: 0000000000000007
[ 76.826343] R13: 000055718f1b9739 R14: 0000000000000045 R15: 0000000000000001
[ 76.827829] ---[ end trace 23e1a55ba58f0ee2 ]---
To reproduce:
# build kernel
cd linux
cp config-5.5.0-rc1-00031-g3af5053534e47 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
Rong Chen
9 months, 4 weeks
- 2
- 1