September 2020 - LKP - Ml01.01.Org

[drm/i915] 04ff178484: phoronix-test-suite.supertuxkart.1024x768.Fullscreen.Ultimate.1.GranParadisoIsland.frames_per_second -30.4% regression

by kernel test robot

Greeting, FYI, we noticed a -30.4% regression of phoronix-test-suite.supertuxkart.1024x768.Fullscreen.Ultimate.1.GranParadisoIsland.frames_per_second due to commit: commit: 04ff1784840f5f954a656c7e8795c76467e29128 ("[Intel-gfx] [PATCH] drm/i915: Enable WaProgramMgsrForCorrectSliceSpecificMmioReads for Gen9") url: https://github.com/0day-ci/linux/commits/Cooper-Chiou/drm-i915-Enable-WaP... base: git://anongit.freedesktop.org/drm-intel for-linux-next in testcase: phoronix-test-suite on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory with following parameters: need_x: true test: supertuxkart-1.5.2 option_a: Fullscreen option_b: Ultimate option_c: 1 option_d: Gran Paradiso Island [Approximately 275k triangles; advanced graphics] cpufreq_governor: performance ucode: 0xd6 test-description: The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. test-url: http://www.phoronix-test-suite.com/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/need_x/option_a/option_b/option_c/option_d/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/true/Fullscreen/Ultimate/1/Gran Paradiso Island [Approximately 275k triangles; advanced graphics]/debian-x86_64-phoronix/lkp-kbl-nuc1/supertuxkart-1.5.2/phoronix-test-suite/0xd6 commit: 400d4953f1 ("drm/i915/pll: Centralize PLL_ENABLE register lookup") 04ff178484 ("drm/i915: Enable WaProgramMgsrForCorrectSliceSpecificMmioReads for Gen9") 400d4953f1f434d5 04ff1784840f5f954a656c7e879 ---------------- --------------------------- %stddev %change %stddev \ | \ 35.32 -30.4% 24.59 phoronix-test-suite.supertuxkart.1024x768.Fullscreen.Ultimate.1.GranParadisoIsland.frames_per_second 3638 ± 15% -39.2% 2211 ± 4% phoronix-test-suite.time.involuntary_context_switches 32.00 -30.5% 22.25 phoronix-test-suite.time.percent_of_cpu_this_job_got 36 +----------------------------------------------------------------------+ |.+.+..+.+.+.+..+.+.+.+.+..+.+.+.+..+.+.+.+.+..+.+.+.+.+..+ +.+..+.+.| 34 |-+ | | | | | 32 |-+ | | | 30 |-+ | | | 28 |-+ | | | | | 26 |-+ | | O O O O O O O O O O O O O O O O O O O O O O | 24 +----------------------------------------------------------------------+ phoronix-test-suite.time.percent_of_cpu_this_job_got 32 +----------------------------------------------------------------------+ | + +. + + +.+.+. | 30 |-+ | | | | | 28 |-+ | | | 26 |-+ | | | 24 |-+ | | O O | | | 22 |-O O O O O O O O O O O O O O O O O O O | | O | 20 +----------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 2 months

3
4
0 / 0

[sched/fair] 3c29e651e1: hackbench.throughput -15.2% regression

by kernel test robot

Greeting, FYI, we noticed a -15.2% regression of hackbench.throughput due to commit: commit: 3c29e651e16dd3b3179cfb2d055ee9538e37515c ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: hackbench on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory with following parameters: nr_threads: 100% mode: threads ipc: pipe cpufreq_governor: performance ucode: 0xca test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler. test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sc... In addition to that, the commit also has significant impact on the following tests: +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -11.3% regression | | test machine | 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | iterations=18 | | | mode=threads | | | nr_threads=1600% | | | ucode=0x43 | +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -10.6% regression | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | mode=process | | | nr_threads=50% | | | ucode=0xb000038 | +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -6.8% regression | | test machine | 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | mode=threads | | | nr_threads=100% | | | ucode=0xb8 | +------------------+-----------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-20191114.cgz/lkp-cfl-e1/hackbench/0xca commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- %stddev %change %stddev \ | \ 69492 ± 3% -15.2% 58899 hackbench.throughput 7.727e+08 ± 7% +38.3% 1.069e+09 ± 2% hackbench.time.involuntary_context_switches 131708 ± 4% -15.4% 111382 hackbench.time.minor_page_faults 684.98 ± 2% -2.5% 667.72 hackbench.time.user_time 1.755e+09 ± 2% +11.2% 1.953e+09 hackbench.time.voluntary_context_switches 4.32e+08 ± 4% -15.6% 3.648e+08 hackbench.workload 186.50 ± 12% -16.6% 155.50 ± 11% interrupts.TLB:TLB_shootdowns 4147892 ± 5% +19.6% 4961643 vmstat.system.cs 0.72 ± 10% -0.1 0.58 ± 17% mpstat.cpu.all.idle% 0.00 ±156% +0.0 0.01 ± 62% mpstat.cpu.all.soft% 1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.active_objs 1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.num_objs 40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_hit 40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_local 40724029 ± 10% -34.5% 26661546 proc-vmstat.pgalloc_normal 763906 ± 2% -7.5% 706781 proc-vmstat.pgfault 40701242 ± 10% -34.6% 26637947 proc-vmstat.pgfree 1636058 ± 11% -45.0% 899629 ± 11% turbostat.C1 0.10 ± 7% -0.0 0.07 ± 7% turbostat.C1% 15201 ± 25% +113.3% 32422 ± 8% turbostat.C1E 0.01 ± 57% +0.0 0.03 ± 13% turbostat.C1E% 9093 ± 11% +217.2% 28841 ± 11% turbostat.C3 0.00 ±173% +0.0 0.03 ± 13% turbostat.C3% 16977 ± 2% -92.8% 1222 ± 98% turbostat.C8 0.17 ± 4% -0.2 0.01 ±110% turbostat.C8% 0.13 -100.0% 0.00 turbostat.CPU%c7 9623106 ± 5% -35.1% 6241651 ± 8% cpuidle.C1.time 1636668 ± 11% -45.0% 899926 ± 11% cpuidle.C1.usage 899494 ± 33% +272.6% 3351412 ± 5% cpuidle.C1E.time 15327 ± 25% +112.6% 32592 ± 8% cpuidle.C1E.usage 435674 ± 20% +643.5% 3239351 ± 14% cpuidle.C3.time 9179 ± 11% +215.2% 28933 ± 11% cpuidle.C3.usage 16850924 ± 2% -94.2% 979516 ± 86% cpuidle.C8.time 17150 ± 2% -92.0% 1378 ± 88% cpuidle.C8.usage 6022328 ± 18% -57.1% 2584773 ± 7% cpuidle.POLL.time 5339703 ± 19% -60.1% 2130089 ± 7% cpuidle.POLL.usage 14.06 ± 25% +64.0% 23.07 ± 13% sched_debug.cfs_rq:/.runnable_load_avg.min 25.10 ± 5% -15.8% 21.12 ± 7% sched_debug.cfs_rq:/.runnable_load_avg.stddev 13339 ± 24% +45.7% 19433 ± 7% sched_debug.cfs_rq:/.runnable_weight.min 166.51 ± 24% +48.9% 247.86 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.min 253.56 ± 2% -12.3% 222.30 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.stddev 159602 ± 23% +55.6% 248411 ± 6% sched_debug.cpu.avg_idle.max 51567 ± 24% +49.1% 76905 ± 11% sched_debug.cpu.avg_idle.stddev 11576 ± 5% -9.5% 10478 sched_debug.cpu.curr->pid.avg 9976 ± 8% -16.8% 8305 ± 9% sched_debug.cpu.curr->pid.min 4.81 ± 15% +43.3% 6.89 ± 8% sched_debug.cpu.nr_running.min 6.68 ± 4% -17.0% 5.54 ± 8% sched_debug.cpu.nr_running.stddev 76003928 ± 2% +21.0% 91975904 sched_debug.cpu.nr_switches.avg 78115404 ± 2% +23.4% 96397551 sched_debug.cpu.nr_switches.max 74275755 ± 2% +19.0% 88359903 sched_debug.cpu.nr_switches.min 1043054 ± 15% +83.9% 1917752 ± 10% sched_debug.cpu.nr_switches.stddev -685.26 +64.8% -1129 sched_debug.cpu.nr_uninterruptible.min 15470 ± 9% -31.0% 10674 ± 9% softirqs.CPU0.SCHED 13756 ± 10% -40.8% 8140 ± 4% softirqs.CPU1.SCHED 13645 ± 11% -39.7% 8227 ± 3% softirqs.CPU10.SCHED 14065 ± 12% -38.8% 8609 ± 2% softirqs.CPU11.SCHED 13948 ± 11% -39.9% 8381 ± 6% softirqs.CPU12.SCHED 13823 ± 12% -39.7% 8332 ± 9% softirqs.CPU13.SCHED 73323 +13.4% 83165 ± 11% softirqs.CPU14.RCU 13872 ± 12% -36.5% 8809 ± 7% softirqs.CPU14.SCHED 13726 ± 12% -40.4% 8175 ± 4% softirqs.CPU15.SCHED 13780 ± 11% -34.8% 8987 ± 12% softirqs.CPU2.SCHED 13895 ± 10% -40.5% 8266 ± 6% softirqs.CPU3.SCHED 221879 +13.4% 251589 ± 14% softirqs.CPU3.TIMER 13873 ± 10% -40.4% 8262 ± 6% softirqs.CPU4.SCHED 72625 +12.0% 81355 ± 10% softirqs.CPU5.RCU 13876 ± 13% -41.4% 8137 ± 3% softirqs.CPU5.SCHED 14118 ± 12% -40.8% 8354 ± 3% softirqs.CPU6.SCHED 13503 ± 12% -40.0% 8099 softirqs.CPU7.SCHED 13549 ± 11% -38.2% 8374 ± 8% softirqs.CPU8.SCHED 14273 ± 10% -40.1% 8548 ± 6% softirqs.CPU9.SCHED 223184 ± 11% -38.9% 136383 ± 5% softirqs.SCHED 55.80 +9.9% 61.34 perf-stat.i.MPKI 1.32 +0.1 1.39 perf-stat.i.branch-miss-rate% 1.066e+08 +3.7% 1.105e+08 perf-stat.i.branch-misses 0.15 ± 8% -0.0 0.11 ± 3% perf-stat.i.cache-miss-rate% 2521584 ± 9% -26.5% 1854353 ± 3% perf-stat.i.cache-misses 2.27e+09 +8.0% 2.451e+09 perf-stat.i.cache-references 4162783 ± 4% +19.7% 4981708 perf-stat.i.context-switches 1.52 +2.0% 1.55 perf-stat.i.cpi 227633 ± 14% +75.1% 398694 ± 3% perf-stat.i.cpu-migrations 33931 ± 12% +42.7% 48428 ± 2% perf-stat.i.cycles-between-cache-misses 0.01 -0.0 0.01 ± 4% perf-stat.i.dTLB-load-miss-rate% 1408442 ± 2% -16.4% 1177727 ± 4% perf-stat.i.dTLB-load-misses 1.229e+10 -2.1% 1.203e+10 perf-stat.i.dTLB-loads 0.00 ± 2% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate% 35757 ± 2% -16.0% 30032 ± 3% perf-stat.i.dTLB-store-misses 7.449e+09 -2.3% 7.279e+09 perf-stat.i.dTLB-stores 39246970 -5.5% 37079604 perf-stat.i.iTLB-load-misses 76813 ± 8% -26.2% 56724 ± 6% perf-stat.i.iTLB-loads 4.096e+10 -1.9% 4.02e+10 perf-stat.i.instructions 1062 +4.1% 1105 perf-stat.i.instructions-per-iTLB-miss 0.66 -1.9% 0.65 perf-stat.i.ipc 1210 -7.1% 1124 perf-stat.i.minor-faults 105889 ± 10% -24.8% 79596 ± 5% perf-stat.i.node-loads 174508 ± 7% -24.4% 131915 perf-stat.i.node-stores 1210 -7.1% 1124 perf-stat.i.page-faults 55.42 ± 2% +10.0% 60.98 perf-stat.overall.MPKI 1.31 +0.1 1.38 perf-stat.overall.branch-miss-rate% 0.11 ± 11% -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate% 1.51 +1.9% 1.54 perf-stat.overall.cpi 24740 ± 9% +34.9% 33377 ± 3% perf-stat.overall.cycles-between-cache-misses 0.01 -0.0 0.01 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 2% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 1043 +3.9% 1084 perf-stat.overall.instructions-per-iTLB-miss 0.66 -1.9% 0.65 perf-stat.overall.ipc 57826 ± 3% +15.6% 66874 perf-stat.overall.path-length 1.064e+08 +3.7% 1.104e+08 perf-stat.ps.branch-misses 2517524 ± 9% -26.5% 1851335 ± 3% perf-stat.ps.cache-misses 2.266e+09 +8.0% 2.447e+09 perf-stat.ps.cache-references 4155898 ± 4% +19.7% 4973458 perf-stat.ps.context-switches 227258 ± 14% +75.1% 398041 ± 3% perf-stat.ps.cpu-migrations 1406119 ± 2% -16.4% 1175777 ± 4% perf-stat.ps.dTLB-load-misses 1.227e+10 -2.1% 1.201e+10 perf-stat.ps.dTLB-loads 35699 ± 2% -16.0% 29983 ± 3% perf-stat.ps.dTLB-store-misses 7.437e+09 -2.3% 7.267e+09 perf-stat.ps.dTLB-stores 39182154 -5.5% 37018206 perf-stat.ps.iTLB-load-misses 76695 ± 8% -26.2% 56638 ± 6% perf-stat.ps.iTLB-loads 4.09e+10 -1.9% 4.013e+10 perf-stat.ps.instructions 1208 -7.1% 1122 perf-stat.ps.minor-faults 105716 ± 10% -24.8% 79465 ± 5% perf-stat.ps.node-loads 174234 ± 7% -24.4% 131703 perf-stat.ps.node-stores 1208 -7.1% 1122 perf-stat.ps.page-faults 29.39 ±104% -29.4 0.00 perf-profile.calltrace.cycles-pp.start_thread 15.86 ±105% -15.9 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread 14.88 ±105% -14.9 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 14.75 ±105% -14.8 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 13.53 ±104% -13.5 0.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 12.51 ±104% -12.5 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread 12.45 ±105% -12.4 0.00 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 11.21 ±105% -11.2 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 11.08 ±105% -11.1 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 10.47 ±105% -10.5 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 9.73 ±105% -9.7 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 3.09 ± 12% -0.6 2.50 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 2.45 ± 14% -0.5 1.95 ± 2% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 1.15 ± 13% -0.2 0.95 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 0.99 ± 13% -0.2 0.83 ± 5% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write 0.92 ± 15% +0.2 1.10 perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.84 ± 23% +0.3 1.10 ± 2% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.99 ± 19% +0.3 1.26 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.97 ± 23% +0.3 1.26 perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.29 ±100% +0.3 0.60 ± 5% perf-profile.calltrace.cycles-pp.reschedule_interrupt.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write 0.55 ± 62% +0.3 0.89 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 0.28 ±100% +0.4 0.63 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop 0.33 ±100% +0.4 0.69 ± 2% perf-profile.calltrace.cycles-pp.prepare_to_wait.pipe_wait.pipe_read.new_sync_read.vfs_read 0.29 ±100% +0.4 0.65 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 1.00 ± 27% +0.4 1.37 ± 2% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.57 ± 63% +0.4 0.95 ± 5% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.46 ± 31% +0.4 1.86 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.72 ± 60% +0.4 1.14 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.29 ±100% +0.4 0.70 ± 6% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.34 ±100% +0.4 0.77 ± 7% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task 1.26 ± 26% +0.5 1.72 perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common 1.13 ± 26% +0.5 1.59 perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.35 ±100% +0.5 0.83 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 1.28 ± 28% +0.5 1.77 ± 3% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.40 ± 26% +0.5 1.89 perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.44 ±100% +0.5 0.95 ± 7% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 2.33 ± 17% +0.5 2.88 ± 2% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read 1.33 ± 28% +0.6 1.89 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.40 ± 24% +0.6 1.97 ± 4% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 0.83 ± 66% +0.6 1.40 ± 5% perf-profile.calltrace.cycles-pp.native_write_msr 5.01 ± 8% +0.8 5.81 ± 2% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.75 ± 39% +0.9 2.61 ± 6% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 1.33 ± 67% +1.0 2.28 ± 6% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.56 ± 12% +1.2 7.78 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 8.96 ± 9% +1.6 10.56 perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 9.23 ± 10% +1.6 10.87 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 9.17 ± 10% +1.6 10.81 perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 3.72 ± 31% +1.7 5.44 ± 6% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.37 ± 8% +1.9 5.32 ± 4% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.37 ± 10% +2.3 6.65 ± 5% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 15.27 ± 10% +2.3 17.59 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 15.59 ± 10% +2.4 17.96 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 3.00 ± 71% +2.6 5.60 ± 6% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.10 ± 71% +2.7 5.77 ± 6% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.05 ± 10% +2.7 20.73 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 28.21 ± 5% +3.7 31.87 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 24.29 ± 8% +4.7 28.98 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 23.38 ± 8% +4.7 28.08 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 23.72 ± 8% +4.7 28.43 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 31.18 ± 41% +12.7 43.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 33.44 ± 40% +13.2 46.61 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 64.81 ± 43% +27.5 92.34 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 65.30 ± 43% +27.7 92.99 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 29.39 ±104% -29.4 0.00 perf-profile.children.cycles-pp.start_thread 16.09 ±105% -16.1 0.00 perf-profile.children.cycles-pp.__GI___libc_write 12.73 ±104% -12.7 0.00 perf-profile.children.cycles-pp.__GI___libc_read 5.80 ± 10% -1.2 4.65 ± 3% perf-profile.children.cycles-pp.security_file_permission 4.10 ± 16% -1.0 3.10 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 3.21 ± 9% -0.7 2.55 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter 1.45 ± 32% -0.6 0.83 perf-profile.children.cycles-pp.entry_SYSCALL_64 3.04 ± 8% -0.6 2.44 ± 3% perf-profile.children.cycles-pp.selinux_file_permission 2.56 ± 11% -0.6 1.99 ± 2% perf-profile.children.cycles-pp.copy_page_from_iter 2.80 ± 9% -0.5 2.25 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 3.38 ± 7% -0.5 2.87 ± 2% perf-profile.children.cycles-pp.mutex_lock 1.62 ± 12% -0.4 1.23 ± 3% perf-profile.children.cycles-pp.___might_sleep 1.82 ± 9% -0.4 1.43 ± 4% perf-profile.children.cycles-pp.syscall_return_via_sysret 2.09 ± 12% -0.4 1.72 ± 6% perf-profile.children.cycles-pp.file_has_perm 0.90 ± 20% -0.3 0.57 ± 9% perf-profile.children.cycles-pp.__mutex_lock 1.14 ± 12% -0.3 0.82 ± 3% perf-profile.children.cycles-pp.__inode_security_revalidate 1.26 ± 10% -0.3 0.97 ± 4% perf-profile.children.cycles-pp.copyin 1.70 ± 8% -0.3 1.42 ± 3% perf-profile.children.cycles-pp.copyout 1.25 ± 11% -0.3 0.98 ± 3% perf-profile.children.cycles-pp.fsnotify 1.04 ± 9% -0.3 0.79 ± 3% perf-profile.children.cycles-pp._cond_resched 1.72 ± 7% -0.2 1.48 ± 5% perf-profile.children.cycles-pp.fput_many 1.16 ± 10% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.__might_sleep 1.09 ± 8% -0.2 0.85 ± 3% perf-profile.children.cycles-pp.__fsnotify_parent 0.83 ± 13% -0.2 0.63 ± 3% perf-profile.children.cycles-pp.__might_fault 0.69 ± 11% -0.1 0.56 perf-profile.children.cycles-pp.current_time 0.99 ± 4% -0.1 0.89 ± 4% perf-profile.children.cycles-pp.touch_atime 0.65 ± 6% -0.1 0.55 ± 6% perf-profile.children.cycles-pp.atime_needs_update 0.39 ± 4% -0.1 0.29 perf-profile.children.cycles-pp.rcu_all_qs 0.13 ± 39% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.34 ± 11% -0.1 0.26 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common 0.49 ± 5% -0.1 0.41 ± 2% perf-profile.children.cycles-pp.wake_up_q 0.24 ± 2% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.30 ± 11% -0.1 0.23 ± 3% perf-profile.children.cycles-pp.inode_has_perm 0.23 ± 8% -0.1 0.18 ± 6% perf-profile.children.cycles-pp.__sb_end_write 0.21 ± 9% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.__x64_sys_read 0.29 ± 8% -0.0 0.24 ± 2% perf-profile.children.cycles-pp.timespec64_trunc 0.21 ± 8% -0.0 0.17 ± 8% perf-profile.children.cycles-pp.__x64_sys_write 0.22 ± 7% -0.0 0.19 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.08 ± 16% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.bpf_fd_pass 0.10 ± 4% +0.0 0.11 ± 3% perf-profile.children.cycles-pp.perf_exclude_event 0.08 ± 24% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.rcu_note_context_switch 0.17 ± 19% +0.1 0.24 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.23 ± 19% +0.1 0.32 ± 3% perf-profile.children.cycles-pp.resched_curr 0.61 ± 8% +0.1 0.72 ± 6% perf-profile.children.cycles-pp.sched_clock 0.67 ± 8% +0.1 0.78 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu 0.50 ± 9% +0.1 0.62 ± 3% perf-profile.children.cycles-pp.account_entity_enqueue 0.61 ± 10% +0.2 0.79 perf-profile.children.cycles-pp.account_entity_dequeue 0.43 ± 19% +0.2 0.62 ± 7% perf-profile.children.cycles-pp.clear_buddies 1.42 ± 14% +0.2 1.66 ± 4% perf-profile.children.cycles-pp.native_write_msr 1.17 ± 13% +0.3 1.42 ± 2% perf-profile.children.cycles-pp.check_preempt_wakeup 0.85 ± 23% +0.3 1.12 ± 7% perf-profile.children.cycles-pp.put_prev_entity 0.94 ± 14% +0.3 1.22 ± 5% perf-profile.children.cycles-pp.___perf_sw_event 1.45 ± 11% +0.3 1.73 perf-profile.children.cycles-pp.check_preempt_curr 1.45 ± 12% +0.3 1.76 perf-profile.children.cycles-pp.__enqueue_entity 1.60 ± 10% +0.3 1.91 perf-profile.children.cycles-pp.ttwu_do_wakeup 1.12 ± 9% +0.3 1.43 ± 3% perf-profile.children.cycles-pp.__update_load_avg_se 1.64 ± 8% +0.4 2.00 ± 4% perf-profile.children.cycles-pp.available_idle_cpu 0.96 ± 19% +0.4 1.34 ± 7% perf-profile.children.cycles-pp.pick_next_entity 1.85 ± 16% +0.4 2.28 ± 6% perf-profile.children.cycles-pp.switch_fpu_return 1.90 ± 9% +0.6 2.50 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 3.22 ± 13% +0.6 3.87 perf-profile.children.cycles-pp.update_load_avg 4.03 ± 10% +0.6 4.68 ± 2% perf-profile.children.cycles-pp.update_curr 3.01 ± 14% +0.7 3.72 ± 2% perf-profile.children.cycles-pp.reweight_entity 5.10 ± 8% +0.8 5.87 ± 2% perf-profile.children.cycles-pp.enqueue_entity 4.71 ± 15% +1.1 5.77 ± 4% perf-profile.children.cycles-pp.pick_next_task_fair 6.73 ± 11% +1.1 7.87 perf-profile.children.cycles-pp.dequeue_task_fair 9.07 ± 9% +1.5 10.61 perf-profile.children.cycles-pp.enqueue_task_fair 9.32 ± 9% +1.6 10.89 perf-profile.children.cycles-pp.ttwu_do_activate 91.46 +1.6 93.03 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 9.27 ± 9% +1.6 10.84 perf-profile.children.cycles-pp.activate_task 4.16 ± 26% +1.6 5.78 ± 6% perf-profile.children.cycles-pp.exit_to_usermode_loop 90.79 +1.7 92.47 perf-profile.children.cycles-pp.do_syscall_64 3.48 ± 8% +1.9 5.36 ± 4% perf-profile.children.cycles-pp.select_idle_sibling 4.44 ± 10% +2.2 6.67 ± 5% perf-profile.children.cycles-pp.select_task_rq_fair 18.46 ± 9% +2.3 20.81 perf-profile.children.cycles-pp.pipe_wait 19.93 ± 11% +3.5 23.46 ± 2% perf-profile.children.cycles-pp.__schedule 28.82 ± 5% +3.6 32.45 perf-profile.children.cycles-pp.__wake_up_common_lock 19.89 ± 12% +3.7 23.62 ± 2% perf-profile.children.cycles-pp.schedule 23.96 ± 8% +4.5 28.45 perf-profile.children.cycles-pp.try_to_wake_up 24.53 ± 8% +4.5 29.06 perf-profile.children.cycles-pp.__wake_up_common 23.91 ± 8% +4.6 28.46 perf-profile.children.cycles-pp.autoremove_wake_function 2.76 ± 9% -0.5 2.23 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.46 ± 18% -0.4 1.06 ± 4% perf-profile.self.cycles-pp.pipe_write 1.81 ± 9% -0.4 1.43 ± 4% perf-profile.self.cycles-pp.syscall_return_via_sysret 1.56 ± 12% -0.4 1.19 ± 3% perf-profile.self.cycles-pp.___might_sleep 2.09 ± 7% -0.3 1.81 ± 2% perf-profile.self.cycles-pp.mutex_lock 1.86 ± 8% -0.3 1.59 ± 3% perf-profile.self.cycles-pp.selinux_file_permission 1.21 ± 11% -0.3 0.96 ± 2% perf-profile.self.cycles-pp.fsnotify 1.70 ± 7% -0.2 1.46 ± 5% perf-profile.self.cycles-pp.fput_many 1.04 ± 11% -0.2 0.82 ± 3% perf-profile.self.cycles-pp.__might_sleep 1.01 ± 8% -0.2 0.80 ± 3% perf-profile.self.cycles-pp.__fsnotify_parent 1.04 ± 10% -0.2 0.83 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.67 ± 10% -0.2 0.51 ± 5% perf-profile.self.cycles-pp.new_sync_write 0.64 ± 12% -0.2 0.48 ± 5% perf-profile.self.cycles-pp.vfs_write 1.67 -0.2 1.51 ± 2% perf-profile.self.cycles-pp.pipe_read 0.58 ± 17% -0.1 0.44 ± 7% perf-profile.self.cycles-pp.file_has_perm 0.44 ± 18% -0.1 0.31 ± 7% perf-profile.self.cycles-pp.__mutex_lock 0.49 ± 13% -0.1 0.35 ± 3% perf-profile.self.cycles-pp.copy_page_from_iter 0.64 ± 8% -0.1 0.52 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter 0.67 ± 7% -0.1 0.55 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.30 ± 5% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.ksys_write 0.40 ± 12% -0.1 0.30 ± 7% perf-profile.self.cycles-pp.security_file_permission 0.13 ± 38% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.36 ± 11% -0.1 0.28 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate 0.76 ± 5% -0.1 0.68 ± 5% perf-profile.self.cycles-pp.do_syscall_64 0.21 ± 2% -0.1 0.14 ± 8% perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.26 ± 12% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.inode_has_perm 0.29 ± 11% -0.1 0.23 ± 6% perf-profile.self.cycles-pp.current_time 0.28 ± 4% -0.1 0.21 ± 2% perf-profile.self.cycles-pp.rcu_all_qs 0.22 ± 7% -0.1 0.17 ± 7% perf-profile.self.cycles-pp.__sb_end_write 0.31 -0.1 0.26 ± 7% perf-profile.self.cycles-pp.ksys_read 0.20 ± 11% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_read 0.26 ± 8% -0.0 0.22 perf-profile.self.cycles-pp.timespec64_trunc 0.20 ± 5% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_write 0.24 ± 9% -0.0 0.21 ± 6% perf-profile.self.cycles-pp.__might_fault 0.09 ± 9% -0.0 0.06 perf-profile.self.cycles-pp.copyin 0.14 ± 5% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.07 ± 10% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.sched_clock 0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.ttwu_do_wakeup 0.20 ± 12% +0.0 0.24 ± 5% perf-profile.self.cycles-pp.__list_add_valid 0.10 ± 26% +0.0 0.14 ± 10% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.15 ± 16% +0.1 0.21 ± 11% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.35 ± 7% +0.1 0.43 ± 4% perf-profile.self.cycles-pp.prepare_to_wait 0.55 ± 2% +0.1 0.63 ± 3% perf-profile.self.cycles-pp.__fget_light 0.22 ± 18% +0.1 0.30 ± 3% perf-profile.self.cycles-pp.resched_curr 0.48 ± 11% +0.1 0.58 ± 3% perf-profile.self.cycles-pp.check_preempt_wakeup 0.39 ± 11% +0.1 0.50 perf-profile.self.cycles-pp.account_entity_enqueue 0.85 ± 7% +0.1 0.97 ± 4% perf-profile.self.cycles-pp.enqueue_entity 0.43 ± 16% +0.1 0.55 ± 7% perf-profile.self.cycles-pp.pick_next_entity 0.86 ± 10% +0.1 0.99 perf-profile.self.cycles-pp.dequeue_task_fair 0.37 ± 21% +0.2 0.55 ± 6% perf-profile.self.cycles-pp.clear_buddies 0.45 ± 14% +0.2 0.65 perf-profile.self.cycles-pp.account_entity_dequeue 1.41 ± 14% +0.2 1.65 ± 4% perf-profile.self.cycles-pp.native_write_msr 0.80 ± 16% +0.3 1.06 ± 4% perf-profile.self.cycles-pp.___perf_sw_event 1.09 ± 9% +0.3 1.39 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se 0.84 ± 16% +0.3 1.14 ± 7% perf-profile.self.cycles-pp.select_task_rq_fair 1.45 ± 12% +0.3 1.75 perf-profile.self.cycles-pp.__enqueue_entity 1.62 ± 8% +0.4 1.99 ± 4% perf-profile.self.cycles-pp.available_idle_cpu 1.83 ± 16% +0.4 2.25 ± 6% perf-profile.self.cycles-pp.switch_fpu_return 2.15 ± 11% +0.5 2.60 ± 5% perf-profile.self.cycles-pp.update_curr 1.88 ± 9% +0.6 2.45 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 1.21 ± 9% +0.7 1.92 ± 3% perf-profile.self.cycles-pp._raw_spin_lock 0.80 ± 11% +1.4 2.17 ± 3% perf-profile.self.cycles-pp.select_idle_sibling hackbench.throughput 74000 +-+-----------------------------------------------------------------+ 72000 +-+.+ .+ | |.+ : .+. .+. + | 70000 +-+ : + + +. : | 68000 +-+ + + : | | :: | 66000 +-+ + | 64000 +-+ | 62000 +-+ | | | 60000 +-+ O O OO O O O O O O O OO | 58000 +-+ O O O O O O O O O O O | O O OO O O | 56000 O-+ O O O O O O | 54000 +-O-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-hsw-ep4: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory ========================================================================================= compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/18/x86_64-rhel-7.6/threads/1600%/debian-x86_64-2019-11-14.cgz/lkp-hsw-ep4/hackbench/0x43 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 176075 ± 2% -11.3% 156152 hackbench.throughput 891.79 +12.3% 1001 hackbench.time.elapsed_time 891.79 +12.3% 1001 hackbench.time.elapsed_time.max 1.889e+09 ± 9% +33.1% 2.514e+09 ± 3% hackbench.time.involuntary_context_switches 42939 ± 3% +17.0% 50220 hackbench.time.system_time 18978 +3.2% 19592 hackbench.time.user_time 2.558e+09 ± 8% +33.5% 3.415e+09 ± 2% hackbench.time.voluntary_context_switches 43133704 ± 3% +39.2% 60035429 ± 33% cpuidle.C1.time 48661 +17.7% 57274 ± 5% meminfo.Shmem 19043 ± 2% -14.0% 16377 meminfo.max_used_kB 2502 +1.3% 2535 turbostat.Avg_MHz 3.368e+08 ± 5% +30.6% 4.398e+08 turbostat.IRQ 66.25 +4.5% 69.25 vmstat.cpu.sy 29.25 ± 2% -8.5% 26.75 vmstat.cpu.us 5000443 ± 7% +18.6% 5931772 vmstat.system.cs 370173 ± 3% +17.1% 433616 vmstat.system.in 98786923 ± 7% +74.5% 1.724e+08 numa-numastat.node0.local_node 98791606 ± 7% +74.5% 1.724e+08 numa-numastat.node0.numa_hit 4686 ± 99% +297.8% 18642 ± 24% numa-numastat.node0.other_node 1.818e+08 -51.4% 88447082 ± 3% numa-numastat.node1.local_node 1.819e+08 -51.4% 88451833 ± 3% numa-numastat.node1.numa_hit 18694 ± 25% -74.6% 4753 ± 97% numa-numastat.node1.other_node 155316 +1.5% 157632 proc-vmstat.nr_active_anon 99.50 +5.3% 104.75 ± 3% proc-vmstat.nr_dirtied 4440 +4.9% 4656 proc-vmstat.nr_inactive_anon 6283 +1.9% 6399 proc-vmstat.nr_mapped 12248 ± 2% +16.6% 14276 ± 5% proc-vmstat.nr_shmem 98.75 +5.3% 104.00 proc-vmstat.nr_written 155316 +1.5% 157632 proc-vmstat.nr_zone_active_anon 4440 +4.9% 4656 proc-vmstat.nr_zone_inactive_anon 2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_hit 2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_local 2.83e+08 -7.0% 2.632e+08 proc-vmstat.pgalloc_normal 3212744 +2.3% 3286078 proc-vmstat.pgfault 2.828e+08 -7.0% 2.631e+08 proc-vmstat.pgfree 39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_active_anon 31864 ± 44% +211.0% 99111 ± 4% numa-vmstat.node0.nr_anon_pages 11183 ± 3% +4212.8% 482321 ± 2% numa-vmstat.node0.nr_kernel_stack 499.50 ± 23% +66.9% 833.75 ± 8% numa-vmstat.node0.nr_page_table_pages 11417 -63.5% 4168 ± 5% numa-vmstat.node0.nr_shmem 15905 ± 4% -12.5% 13919 ± 2% numa-vmstat.node0.nr_slab_reclaimable 17996 ± 3% +587.4% 123708 ± 2% numa-vmstat.node0.nr_slab_unreclaimable 39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_zone_active_anon 52471566 ± 6% +73.3% 90915199 numa-vmstat.node0.numa_hit 52466577 ± 6% +73.2% 90896366 numa-vmstat.node0.numa_local 4990 ± 92% +277.5% 18836 ± 24% numa-vmstat.node0.numa_other 114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_active_anon 114545 ± 12% -58.2% 47874 ± 9% numa-vmstat.node1.nr_anon_pages 462075 -96.6% 15717 ± 67% numa-vmstat.node1.nr_kernel_stack 871.50 ± 13% -37.9% 541.00 ± 13% numa-vmstat.node1.nr_page_table_pages 691.25 ± 22% +1359.2% 10086 ± 8% numa-vmstat.node1.nr_shmem 11357 ± 5% +18.1% 13414 ± 4% numa-vmstat.node1.nr_slab_reclaimable 118594 -85.5% 17243 ± 16% numa-vmstat.node1.nr_slab_unreclaimable 114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_zone_active_anon 93308245 ± 2% -49.7% 46921467 ± 4% numa-vmstat.node1.numa_hit 93128224 ± 2% -49.8% 46755145 ± 4% numa-vmstat.node1.numa_local 157762 ± 35% +154.4% 401333 ± 4% numa-meminfo.node0.Active 157721 ± 35% +154.4% 401249 ± 4% numa-meminfo.node0.Active(anon) 128232 ± 44% +212.2% 400337 ± 4% numa-meminfo.node0.AnonPages 63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.KReclaimable 11691 ± 3% +4087.3% 489553 numa-meminfo.node0.KernelStack 1170596 ± 4% +104.2% 2390602 numa-meminfo.node0.MemUsed 2003 ± 23% +67.3% 3351 ± 8% numa-meminfo.node0.PageTables 63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.SReclaimable 72301 ± 3% +591.9% 500285 numa-meminfo.node0.SUnreclaim 45749 -63.5% 16679 ± 5% numa-meminfo.node0.Shmem 135789 ± 3% +309.6% 556136 numa-meminfo.node0.Slab 460025 ± 11% -50.2% 228882 ± 7% numa-meminfo.node1.Active 459902 ± 11% -50.2% 228802 ± 7% numa-meminfo.node1.Active(anon) 459237 ± 11% -58.3% 191354 ± 9% numa-meminfo.node1.AnonPages 45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.KReclaimable 468955 -96.7% 15461 ± 67% numa-meminfo.node1.KernelStack 2241450 -51.9% 1078199 ± 3% numa-meminfo.node1.MemUsed 3493 ± 13% -38.0% 2167 ± 13% numa-meminfo.node1.PageTables 45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.SReclaimable 479010 -85.6% 68750 ± 16% numa-meminfo.node1.SUnreclaim 2779 ± 23% +1354.1% 40419 ± 8% numa-meminfo.node1.Shmem 524580 -76.7% 122408 ± 8% numa-meminfo.node1.Slab 250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.avg 1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.stddev 20766 ± 9% -25.3% 15519 ± 10% sched_debug.cfs_rq:/.load.avg 43820 ± 19% -40.6% 26044 ± 36% sched_debug.cfs_rq:/.load.stddev 250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.max_vruntime.avg 1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.max_vruntime.stddev 10990567 ± 9% +22.9% 13509735 ± 7% sched_debug.cfs_rq:/.min_vruntime.stddev 102.40 -33.3% 68.27 ± 57% sched_debug.cfs_rq:/.removed.load_avg.max 4711 -33.4% 3136 ± 57% sched_debug.cfs_rq:/.removed.runnable_sum.max 19930 ± 10% -25.1% 14917 ± 10% sched_debug.cfs_rq:/.runnable_weight.avg -5713498 -349.0% 14226386 ± 8% sched_debug.cfs_rq:/.spread0.avg 8106715 ± 13% +283.1% 31057182 ± 6% sched_debug.cfs_rq:/.spread0.max -18025645 -99.7% -45393 sched_debug.cfs_rq:/.spread0.min 11027552 ± 9% +21.5% 13398318 ± 7% sched_debug.cfs_rq:/.spread0.stddev 570.46 ± 5% +109.2% 1193 ± 21% sched_debug.cfs_rq:/.util_est_enqueued.avg 2898 ± 11% +90.0% 5507 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.max 563.27 ± 13% +149.3% 1404 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.stddev 155578 ± 22% +84.1% 286457 ± 28% sched_debug.cpu.avg_idle.min 1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock.stddev 1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock_task.stddev 50207 -12.3% 44017 ± 7% sched_debug.cpu.curr->pid.avg 82673 -19.5% 66514 ± 5% sched_debug.cpu.curr->pid.max 746.05 ± 11% +554.3% 4881 ± 8% sched_debug.cpu.curr->pid.min 30728 ± 2% -40.8% 18187 ± 10% sched_debug.cpu.curr->pid.stddev 0.00 ± 43% +106.0% 0.00 ± 31% sched_debug.cpu.next_balance.stddev 20.49 ± 6% +149.0% 51.03 ± 23% sched_debug.cpu.nr_running.avg 157.83 ± 9% +77.0% 279.37 ± 8% sched_debug.cpu.nr_running.max 29.95 ± 11% +137.1% 70.99 ± 16% sched_debug.cpu.nr_running.stddev 29803727 ± 9% +29.6% 38623870 ± 3% sched_debug.cpu.nr_switches.avg 51549476 ± 10% +22.5% 63171598 ± 3% sched_debug.cpu.nr_switches.max 12792570 ± 3% +43.5% 18362678 ± 4% sched_debug.cpu.nr_switches.min 2.03 ± 80% +430.9% 10.76 ± 38% sched_debug.cpu.nr_uninterruptible.avg 6.99 ± 4% +48.7% 10.39 ± 5% perf-stat.i.MPKI 7.49 ± 5% -1.2 6.29 ± 5% perf-stat.i.cache-miss-rate% 27202880 ± 2% +46.9% 39966416 ± 2% perf-stat.i.cache-misses 4.662e+08 ± 5% +71.5% 7.998e+08 ± 5% perf-stat.i.cache-references 5989366 ± 4% +12.9% 6760958 ± 5% perf-stat.i.context-switches 197319 ± 12% -19.2% 159504 ± 3% perf-stat.i.cpu-migrations 7415 ± 2% -23.3% 5686 ± 3% perf-stat.i.cycles-between-cache-misses 0.61 ± 35% -0.3 0.35 ± 4% perf-stat.i.dTLB-load-miss-rate% 1.506e+08 ± 38% -44.9% 82967539 ± 5% perf-stat.i.dTLB-load-misses 0.22 ± 4% -0.0 0.20 ± 4% perf-stat.i.dTLB-store-miss-rate% 35974976 ± 5% -13.4% 31146077 ± 6% perf-stat.i.dTLB-store-misses 1.627e+10 -3.7% 1.566e+10 perf-stat.i.dTLB-stores 46665620 ± 3% -14.6% 39838962 ± 5% perf-stat.i.iTLB-load-misses 2191 ± 5% +21.4% 2660 ± 7% perf-stat.i.instructions-per-iTLB-miss 0.47 -2.5% 0.46 perf-stat.i.ipc 6660 ± 8% -19.1% 5391 ± 8% perf-stat.i.minor-faults 62.93 ± 2% +11.1 74.06 perf-stat.i.node-load-miss-rate% 12537290 ± 4% +73.5% 21750457 ± 4% perf-stat.i.node-load-misses 5459308 ± 3% -7.4% 5057404 ± 3% perf-stat.i.node-loads 29.39 ± 2% -4.1 25.29 ± 2% perf-stat.i.node-store-miss-rate% 2863317 +13.6% 3252632 ± 2% perf-stat.i.node-store-misses 6381925 ± 2% +53.5% 9796918 ± 4% perf-stat.i.node-stores 6638 ± 8% -19.4% 5348 ± 8% perf-stat.i.page-faults 4.89 ± 4% +78.4% 8.73 perf-stat.overall.MPKI 1.29 +0.0 1.31 perf-stat.overall.branch-miss-rate% 6.13 ± 4% -1.3 4.85 ± 3% perf-stat.overall.cache-miss-rate% 2.15 +2.0% 2.19 perf-stat.overall.cpi 7187 -27.9% 5182 ± 3% perf-stat.overall.cycles-between-cache-misses 0.63 ± 39% -0.3 0.36 ± 5% perf-stat.overall.dTLB-load-miss-rate% 0.23 ± 5% -0.0 0.21 ± 3% perf-stat.overall.dTLB-store-miss-rate% 75.65 -1.1 74.59 perf-stat.overall.iTLB-load-miss-rate% 1604 ± 3% +15.9% 1859 perf-stat.overall.instructions-per-iTLB-miss 0.47 -2.0% 0.46 perf-stat.overall.ipc 63.14 ± 2% +13.6 76.70 perf-stat.overall.node-load-miss-rate% 30.74 ± 2% -5.1 25.62 ± 2% perf-stat.overall.node-store-miss-rate% 47754 ± 2% +11.7% 53321 perf-stat.overall.path-length 24893507 +40.6% 34993733 ± 2% perf-stat.ps.cache-misses 4.072e+08 ± 5% +77.1% 7.212e+08 perf-stat.ps.cache-references 4968881 ± 7% +19.0% 5911450 perf-stat.ps.context-switches 1.789e+11 +1.3% 1.812e+11 perf-stat.ps.cpu-cycles 1.59e+08 ± 39% -44.1% 88801351 ± 6% perf-stat.ps.dTLB-load-misses 2.502e+10 -2.0% 2.453e+10 perf-stat.ps.dTLB-loads 39138578 ± 5% -12.6% 34190129 ± 3% perf-stat.ps.dTLB-store-misses 1.675e+10 -3.7% 1.613e+10 perf-stat.ps.dTLB-stores 51901139 ± 2% -14.3% 44461197 perf-stat.ps.iTLB-load-misses 16711264 ± 2% -9.3% 15153636 ± 2% perf-stat.ps.iTLB-loads 3573 -9.1% 3247 perf-stat.ps.minor-faults 10427704 ± 3% +73.2% 18055799 ± 3% perf-stat.ps.node-load-misses 6085553 ± 4% -10.0% 5479559 ± 2% perf-stat.ps.node-loads 2566511 +13.4% 2911367 ± 2% perf-stat.ps.node-store-misses 5784179 ± 2% +46.2% 8454210 ± 3% perf-stat.ps.node-stores 3573 -9.1% 3247 perf-stat.ps.page-faults 7.427e+13 ± 2% +11.7% 8.293e+13 perf-stat.total.instructions 127394 ± 5% +19.1% 151665 softirqs.CPU0.RCU 321761 ± 3% +12.8% 363107 ± 3% softirqs.CPU0.TIMER 124526 ± 5% +20.2% 149697 ± 3% softirqs.CPU1.RCU 123768 ± 4% +24.7% 154332 ± 2% softirqs.CPU10.RCU 130315 ± 3% +22.1% 159111 softirqs.CPU11.RCU 126684 ± 4% +26.1% 159755 ± 2% softirqs.CPU12.RCU 124649 ± 9% +27.8% 159298 ± 3% softirqs.CPU13.RCU 124528 ± 4% +23.2% 153372 ± 2% softirqs.CPU14.RCU 326150 ± 2% +9.8% 357991 ± 4% softirqs.CPU14.TIMER 121923 ± 6% +17.4% 143119 softirqs.CPU15.RCU 122114 ± 5% +16.7% 142560 softirqs.CPU16.RCU 121425 ± 6% +14.4% 138861 ± 3% softirqs.CPU17.RCU 330948 ± 2% +22.2% 404570 ± 16% softirqs.CPU17.TIMER 312317 ± 4% +18.2% 369091 ± 2% softirqs.CPU18.TIMER 9626 ± 2% +15.0% 11067 ± 3% softirqs.CPU19.SCHED 322308 ± 3% +21.1% 390454 ± 2% softirqs.CPU19.TIMER 124281 ± 5% +22.8% 152670 ± 2% softirqs.CPU2.RCU 9452 ± 2% +15.6% 10927 ± 3% softirqs.CPU20.SCHED 318575 ± 6% +22.3% 389741 ± 3% softirqs.CPU20.TIMER 9327 ± 3% +16.7% 10886 ± 3% softirqs.CPU21.SCHED 306628 ± 4% +23.3% 377962 ± 3% softirqs.CPU21.TIMER 9424 ± 2% +17.6% 11084 ± 3% softirqs.CPU22.SCHED 302796 ± 4% +24.2% 376126 ± 2% softirqs.CPU22.TIMER 9798 ± 2% +11.4% 10918 ± 4% softirqs.CPU23.SCHED 312009 ± 4% +23.6% 385626 ± 2% softirqs.CPU23.TIMER 9559 ± 3% +13.2% 10822 softirqs.CPU24.SCHED 311772 ± 4% +23.0% 383504 ± 2% softirqs.CPU24.TIMER 9514 ± 4% +13.2% 10766 ± 3% softirqs.CPU25.SCHED 307640 ± 4% +22.4% 376479 ± 2% softirqs.CPU25.TIMER 9573 ± 3% +13.8% 10897 ± 3% softirqs.CPU26.SCHED 314607 ± 9% +21.1% 380918 ± 2% softirqs.CPU26.TIMER 9385 +13.7% 10668 ± 2% softirqs.CPU27.SCHED 301949 ± 4% +25.2% 378140 ± 3% softirqs.CPU27.TIMER 9483 ± 2% +14.7% 10874 ± 2% softirqs.CPU28.SCHED 299462 ± 3% +24.9% 373896 ± 2% softirqs.CPU28.TIMER 9505 ± 3% +13.3% 10769 ± 3% softirqs.CPU29.SCHED 307087 ± 4% +23.8% 380066 ± 3% softirqs.CPU29.TIMER 124992 ± 4% +19.4% 149253 ± 3% softirqs.CPU3.RCU 9630 ± 4% +13.6% 10942 ± 3% softirqs.CPU30.SCHED 306382 ± 4% +25.4% 384355 ± 2% softirqs.CPU30.TIMER 9496 ± 6% +12.6% 10689 ± 2% softirqs.CPU31.SCHED 308689 ± 4% +24.5% 384419 ± 2% softirqs.CPU31.TIMER 9714 +12.6% 10937 ± 4% softirqs.CPU32.SCHED 304137 ± 3% +35.9% 413240 ± 13% softirqs.CPU32.TIMER 9598 ± 2% +13.4% 10882 ± 5% softirqs.CPU33.SCHED 9622 +13.8% 10949 ± 3% softirqs.CPU34.SCHED 305882 ± 4% +27.3% 389315 ± 4% softirqs.CPU34.TIMER 9409 ± 4% +15.3% 10853 ± 4% softirqs.CPU35.SCHED 306215 ± 4% +23.8% 379222 ± 2% softirqs.CPU35.TIMER 321617 ± 4% +12.9% 362966 ± 3% softirqs.CPU36.TIMER 134843 ± 6% +19.4% 161053 ± 2% softirqs.CPU37.RCU 130954 ± 6% +20.4% 157716 softirqs.CPU38.RCU 129884 ± 5% +17.8% 152978 softirqs.CPU39.RCU 124744 ± 4% +20.1% 149790 ± 3% softirqs.CPU4.RCU 130593 ± 5% +17.2% 153092 ± 2% softirqs.CPU40.RCU 134185 ± 5% +19.9% 160871 softirqs.CPU41.RCU 133336 ± 5% +18.1% 157409 softirqs.CPU42.RCU 134486 ± 4% +18.7% 159613 softirqs.CPU43.RCU 129857 ± 6% +18.5% 153887 softirqs.CPU44.RCU 131816 ± 5% +20.0% 158144 softirqs.CPU45.RCU 132691 ± 5% +22.4% 162412 softirqs.CPU46.RCU 138736 ± 3% +18.7% 164668 softirqs.CPU47.RCU 132971 ± 6% +24.2% 165142 softirqs.CPU48.RCU 136142 ± 3% +20.8% 164493 softirqs.CPU49.RCU 125011 ± 4% +23.6% 154462 ± 2% softirqs.CPU5.RCU 133660 ± 4% +19.5% 159753 ± 2% softirqs.CPU50.RCU 140405 ± 4% +19.5% 167772 softirqs.CPU51.RCU 140523 ± 4% +18.5% 166576 softirqs.CPU52.RCU 138849 ± 5% +14.6% 159157 ± 4% softirqs.CPU53.RCU 332171 ± 2% +21.4% 403260 ± 15% softirqs.CPU53.TIMER 133037 ± 4% +14.0% 151688 ± 2% softirqs.CPU54.RCU 9584 +14.3% 10959 ± 2% softirqs.CPU54.SCHED 310603 ± 4% +18.0% 366472 ± 3% softirqs.CPU54.TIMER 9565 ± 2% +13.8% 10889 ± 5% softirqs.CPU55.SCHED 321231 ± 3% +21.3% 389753 ± 2% softirqs.CPU55.TIMER 136633 ± 4% +12.6% 153854 softirqs.CPU56.RCU 9430 ± 2% +15.6% 10898 ± 5% softirqs.CPU56.SCHED 319477 ± 6% +22.1% 390107 ± 2% softirqs.CPU56.TIMER 128672 ± 4% +14.1% 146791 ± 3% softirqs.CPU57.RCU 9493 ± 2% +13.1% 10740 ± 3% softirqs.CPU57.SCHED 305670 ± 4% +23.5% 377597 ± 2% softirqs.CPU57.TIMER 127129 ± 4% +15.6% 146993 ± 2% softirqs.CPU58.RCU 9529 ± 3% +13.6% 10824 ± 4% softirqs.CPU58.SCHED 302317 ± 4% +23.8% 374371 ± 3% softirqs.CPU58.TIMER 9427 ± 2% +14.0% 10743 ± 2% softirqs.CPU59.SCHED 310985 ± 4% +23.5% 384119 ± 3% softirqs.CPU59.TIMER 124937 ± 4% +21.6% 151962 ± 3% softirqs.CPU6.RCU 9435 ± 2% +14.1% 10762 ± 3% softirqs.CPU60.SCHED 311646 ± 4% +22.8% 382731 ± 3% softirqs.CPU60.TIMER 9585 +13.5% 10877 ± 2% softirqs.CPU61.SCHED 306857 ± 4% +22.8% 376944 ± 2% softirqs.CPU61.TIMER 9483 ± 2% +12.5% 10671 ± 5% softirqs.CPU62.SCHED 9620 ± 4% +9.7% 10556 ± 4% softirqs.CPU63.SCHED 301085 ± 3% +24.7% 375397 ± 3% softirqs.CPU63.TIMER 300443 ± 4% +23.7% 371620 ± 2% softirqs.CPU64.TIMER 9667 +11.2% 10751 ± 4% softirqs.CPU65.SCHED 307194 ± 4% +23.4% 379020 ± 3% softirqs.CPU65.TIMER 9493 ± 3% +14.9% 10909 ± 3% softirqs.CPU66.SCHED 306226 ± 4% +25.1% 383140 ± 3% softirqs.CPU66.TIMER 9674 ± 3% +13.8% 11010 ± 3% softirqs.CPU67.SCHED 307870 ± 4% +25.0% 384793 ± 2% softirqs.CPU67.TIMER 9451 ± 2% +14.2% 10792 ± 4% softirqs.CPU68.SCHED 301873 ± 3% +26.1% 380748 softirqs.CPU68.TIMER 9542 +13.8% 10863 ± 3% softirqs.CPU69.SCHED 316206 ± 8% +20.5% 380893 ± 2% softirqs.CPU69.TIMER 128158 ± 4% +20.4% 154294 ± 2% softirqs.CPU7.RCU 9456 ± 3% +13.5% 10734 ± 3% softirqs.CPU70.SCHED 305361 ± 4% +36.4% 416546 ± 15% softirqs.CPU70.TIMER 9520 ± 3% +15.4% 10985 ± 4% softirqs.CPU71.SCHED 305605 ± 4% +24.6% 380653 ± 2% softirqs.CPU71.TIMER 122831 ± 4% +21.6% 149424 ± 2% softirqs.CPU8.RCU 123870 ± 4% +23.0% 152396 ± 2% softirqs.CPU9.RCU 9357957 ± 5% +15.4% 10794953 softirqs.RCU 23209193 ± 3% +15.5% 26806870 ± 3% softirqs.TIMER 4.79 ± 4% -3.1 1.71 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.85 ± 4% -2.4 1.41 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.39 ± 4% -2.0 1.35 ± 16% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 2.93 ± 5% -1.9 0.99 ± 18% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 3.25 ± 4% -1.9 1.31 ± 10% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write 2.14 ± 4% -1.2 0.91 ± 14% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64 2.06 ± 4% -1.2 0.85 ± 13% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64 1.52 ± 3% -1.0 0.49 ± 59% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 1.46 ± 3% -1.0 0.46 ± 59% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read 1.53 ± 8% -0.8 0.68 ± 12% perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read 1.65 ± 4% -0.8 0.81 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.45 ± 5% -0.7 0.74 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.35 ± 5% -0.7 0.68 ± 10% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64 1.28 ± 8% -0.7 0.62 ± 12% perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.new_sync_write.vfs_write.ksys_write 1.29 ± 3% -0.6 0.67 ± 9% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 0.64 ± 11% +0.3 0.96 ± 7% perf-profile.calltrace.cycles-pp.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 0.00 +0.5 0.55 ± 3% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.00 +0.6 0.55 ± 5% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.00 +0.6 0.57 ± 6% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.6 0.61 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.7 0.69 ± 8% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.7 0.70 ± 6% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.00 +0.7 0.71 ± 11% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule 0.00 +0.7 0.72 ± 7% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.7 0.72 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +0.8 0.76 ± 10% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.8 0.77 ± 2% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.8 0.77 ± 5% perf-profile.calltrace.cycles-pp.__switch_to 0.00 +0.8 0.77 ± 6% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.8 0.79 ± 5% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.8 0.81 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +0.9 0.90 ± 3% perf-profile.calltrace.cycles-pp.find_next_bit.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up 0.00 +0.9 0.91 ± 4% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.00 +0.9 0.91 ± 6% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.00 +0.9 0.92 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr 0.00 +1.1 1.05 ± 5% perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.00 +1.1 1.08 ± 8% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.00 +1.1 1.13 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.00 +1.2 1.17 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.00 +1.2 1.18 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 0.00 +1.2 1.20 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read 0.00 +1.3 1.30 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +1.3 1.34 ± 6% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +1.6 1.62 ± 4% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.9 1.88 ± 5% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read 0.00 +1.9 1.93 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.13 ±173% +2.0 2.11 ± 5% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 0.00 +2.1 2.06 ± 4% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +2.4 2.38 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.00 +2.7 2.65 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.35 ±103% +4.3 4.67 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.22 ±173% +4.6 4.81 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.23 ±173% +4.7 4.97 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.72 ± 23% +5.0 5.72 ± 5% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 0.69 ± 17% +5.6 6.25 ± 4% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 0.74 ± 17% +5.8 6.50 ± 4% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.74 ± 18% +5.8 6.55 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 15.99 ± 54% +7.5 23.51 ± 6% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.13 +7.7 20.86 ± 3% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.49 +8.7 20.19 ± 3% perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 1.31 ± 27% +8.9 10.21 ± 7% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 1.81 ± 22% +11.2 12.98 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 1.89 ± 22% +11.4 13.28 ± 5% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 2.18 ± 21% +13.1 15.28 ± 4% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 2.01 ± 25% +19.1 21.10 ± 7% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 2.38 ± 23% +20.0 22.40 ± 7% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 19.01 ± 3% +23.4 42.38 ± 4% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 21.13 ± 53% +23.8 44.92 ± 5% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 17.85 ± 4% +23.9 41.76 ± 4% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 19.16 ± 53% +24.6 43.74 ± 5% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.79 ± 22% +30.7 34.45 ± 6% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 3.84 ± 23% +30.9 34.69 ± 6% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 4.13 ± 21% +31.0 35.10 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 5.77 ± 14% +31.0 36.76 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 9.20 ± 6% -6.0 3.16 ± 12% perf-profile.children.cycles-pp.syscall_return_via_sysret 8.88 ± 3% -5.8 3.03 ± 12% perf-profile.children.cycles-pp.entry_SYSCALL_64 8.72 ± 4% -5.3 3.39 ± 11% perf-profile.children.cycles-pp.security_file_permission 4.27 ± 3% -2.4 1.89 ± 9% perf-profile.children.cycles-pp.selinux_file_permission 3.15 ± 5% -2.1 1.07 ± 12% perf-profile.children.cycles-pp.file_has_perm 3.48 ± 3% -2.0 1.46 ± 10% perf-profile.children.cycles-pp.copy_page_to_iter 3.80 ± 3% -2.0 1.79 ± 6% perf-profile.children.cycles-pp.mutex_unlock 3.02 ± 5% -1.9 1.11 ± 11% perf-profile.children.cycles-pp.copy_page_from_iter 3.31 ± 4% -1.6 1.75 ± 6% perf-profile.children.cycles-pp.__fdget_pos 2.55 ± 3% -1.5 1.02 ± 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 3.17 ± 4% -1.5 1.70 ± 6% perf-profile.children.cycles-pp.__fget_light 2.10 ± 7% -1.4 0.71 ± 12% perf-profile.children.cycles-pp.fsnotify 2.52 ± 6% -1.3 1.23 ± 8% perf-profile.children.cycles-pp.__fget 1.77 ± 6% -1.1 0.66 ± 9% perf-profile.children.cycles-pp.avc_has_perm 1.89 ± 5% -1.1 0.78 ± 10% perf-profile.children.cycles-pp.___might_sleep 1.44 ± 4% -1.0 0.48 ± 12% perf-profile.children.cycles-pp.__inode_security_revalidate 1.33 ± 10% -0.9 0.40 ± 16% perf-profile.children.cycles-pp.current_time 1.56 ± 3% -0.9 0.66 ± 9% perf-profile.children.cycles-pp.copyout 1.56 ± 8% -0.8 0.73 ± 7% perf-profile.children.cycles-pp.touch_atime 1.33 ± 4% -0.8 0.53 ± 10% perf-profile.children.cycles-pp.__might_sleep 1.15 ± 9% -0.8 0.36 ± 12% perf-profile.children.cycles-pp.atime_needs_update 1.17 ± 4% -0.7 0.43 ± 13% perf-profile.children.cycles-pp.copyin 1.23 ± 5% -0.7 0.54 ± 9% perf-profile.children.cycles-pp.__fsnotify_parent 1.32 ± 7% -0.6 0.68 ± 6% perf-profile.children.cycles-pp.file_update_time 1.02 ± 3% -0.6 0.38 ± 12% perf-profile.children.cycles-pp.__might_fault 2.54 ± 2% -0.5 2.03 perf-profile.children.cycles-pp.mutex_lock 0.78 ± 4% -0.4 0.36 ± 10% perf-profile.children.cycles-pp.fput_many 0.53 ± 4% -0.4 0.15 ± 21% perf-profile.children.cycles-pp.inode_has_perm 0.44 ± 12% -0.3 0.11 ± 22% perf-profile.children.cycles-pp.iov_iter_init 0.76 ± 4% -0.3 0.44 ± 8% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.53 ± 3% -0.3 0.21 ± 6% perf-profile.children.cycles-pp.rcu_all_qs 0.41 ± 9% -0.3 0.12 ± 15% perf-profile.children.cycles-pp.timespec64_trunc 0.97 -0.3 0.71 ± 3% perf-profile.children.cycles-pp._cond_resched 0.47 ± 7% -0.3 0.21 ± 11% perf-profile.children.cycles-pp.__x64_sys_read 0.45 ± 7% -0.3 0.20 ± 10% perf-profile.children.cycles-pp.__x64_sys_write 0.37 ± 4% -0.2 0.17 ± 16% perf-profile.children.cycles-pp.generic_pipe_buf_confirm 0.44 ± 12% -0.2 0.24 ± 3% perf-profile.children.cycles-pp.__sb_start_write 0.31 ± 6% -0.2 0.12 ± 13% perf-profile.children.cycles-pp.__sb_end_write 0.36 ± 7% -0.2 0.18 ± 6% perf-profile.children.cycles-pp.rw_verify_area 0.26 ± 14% -0.2 0.09 ± 14% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.29 ± 8% -0.2 0.14 ± 16% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.22 ± 39% -0.2 0.06 ± 11% perf-profile.children.cycles-pp.__vfs_read 0.21 ± 19% -0.1 0.11 ± 9% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 0.12 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_write 0.12 ± 17% -0.0 0.08 ± 11% perf-profile.children.cycles-pp.wake_up_q 0.08 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.kill_fasync 0.11 ± 7% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.09 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.01 ±173% +0.1 0.06 ± 6% perf-profile.children.cycles-pp.__mnt_want_write 0.07 ± 25% +0.1 0.12 ± 27% perf-profile.children.cycles-pp.__softirqentry_text_start 0.08 ± 30% +0.1 0.14 ± 26% perf-profile.children.cycles-pp.irq_exit 0.06 ± 6% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.__mark_inode_dirty 0.00 +0.1 0.06 perf-profile.children.cycles-pp.preempt_schedule_common 0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.is_cpu_allowed 0.03 ±100% +0.1 0.10 ± 8% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.__x2apic_send_IPI_dest 0.01 ±173% +0.1 0.09 ± 32% perf-profile.children.cycles-pp.migrate_task_rq_fair 0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.update_cfs_rq_h_load 0.05 ± 61% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.native_irq_return_iret 0.00 +0.1 0.08 ± 8% perf-profile.children.cycles-pp.rcu_note_context_switch 0.00 +0.1 0.08 ± 13% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.native_load_tls 0.03 ±102% +0.1 0.12 ± 30% perf-profile.children.cycles-pp.set_task_cpu 0.00 +0.1 0.09 ± 11% perf-profile.children.cycles-pp.check_cfs_rq_runtime 0.00 +0.1 0.10 ± 5% perf-profile.children.cycles-pp.rb_next 0.10 ± 10% +0.1 0.20 ± 9% perf-profile.children.cycles-pp.generic_update_time 0.00 +0.1 0.10 ± 29% perf-profile.children.cycles-pp.__x64_sys_exit 0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.do_exit 0.00 +0.1 0.11 ± 8% perf-profile.children.cycles-pp.rb_insert_color 0.00 +0.1 0.11 ± 7% perf-profile.children.cycles-pp.__list_add_valid 0.00 +0.1 0.12 ± 7% perf-profile.children.cycles-pp.resched_curr 0.00 +0.1 0.12 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.13 ± 10% +0.1 0.26 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid 0.00 +0.1 0.15 ± 4% perf-profile.children.cycles-pp.set_next_buddy 0.00 +0.2 0.16 ± 5% perf-profile.children.cycles-pp.deactivate_task 0.05 ± 9% +0.2 0.22 ± 7% perf-profile.children.cycles-pp.rb_erase 0.06 ± 13% +0.2 0.23 ± 9% perf-profile.children.cycles-pp.finish_wait 0.17 ± 12% +0.2 0.35 ± 2% perf-profile.children.cycles-pp.anon_pipe_buf_release 0.01 ±173% +0.2 0.20 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.00 +0.2 0.21 ± 4% perf-profile.children.cycles-pp.cpumask_next 0.07 ± 22% +0.3 0.40 ± 2% perf-profile.children.cycles-pp.prepare_to_wait 0.03 ±100% +0.4 0.39 ± 5% perf-profile.children.cycles-pp.update_min_vruntime 0.03 ±100% +0.4 0.39 ± 12% perf-profile.children.cycles-pp.cpus_share_cache 0.06 ± 14% +0.4 0.43 perf-profile.children.cycles-pp.clear_buddies 0.11 ± 15% +0.4 0.50 ± 3% perf-profile.children.cycles-pp.cpuacct_charge 0.10 ± 26% +0.4 0.49 ± 4% perf-profile.children.cycles-pp.native_sched_clock 0.32 ± 31% +0.4 0.75 ± 14% perf-profile.children.cycles-pp.sched_ttwu_pending 0.11 ± 26% +0.4 0.56 ± 3% perf-profile.children.cycles-pp.sched_clock 0.34 ± 32% +0.5 0.80 ± 15% perf-profile.children.cycles-pp.scheduler_ipi 0.08 ± 12% +0.5 0.56 ± 7% perf-profile.children.cycles-pp.account_entity_enqueue 0.13 ± 26% +0.5 0.62 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.14 ± 26% +0.6 0.77 ± 4% perf-profile.children.cycles-pp.__calc_delta 0.08 ± 16% +0.7 0.76 ± 3% perf-profile.children.cycles-pp.account_entity_dequeue 0.84 ± 5% +0.7 1.54 ± 5% perf-profile.children.cycles-pp.__lock_text_start 1.61 ± 16% +0.7 2.31 ± 9% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.42 ± 31% +0.7 1.12 ± 13% perf-profile.children.cycles-pp.reschedule_interrupt 0.24 ± 28% +0.9 1.11 perf-profile.children.cycles-pp.native_write_msr 0.14 ± 22% +0.9 1.01 ± 5% perf-profile.children.cycles-pp.check_preempt_wakeup 0.24 ± 32% +0.9 1.12 ± 7% perf-profile.children.cycles-pp.finish_task_switch 0.22 ± 22% +0.9 1.10 ± 3% perf-profile.children.cycles-pp.find_next_bit 0.22 ± 23% +0.9 1.12 ± 3% perf-profile.children.cycles-pp.set_next_entity 0.21 ± 32% +0.9 1.14 ± 3% perf-profile.children.cycles-pp.update_cfs_group 0.16 ± 23% +1.0 1.15 ± 4% perf-profile.children.cycles-pp.check_preempt_curr 0.21 ± 30% +1.0 1.21 ± 2% perf-profile.children.cycles-pp.__switch_to 0.23 ± 21% +1.0 1.24 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 0.16 ± 38% +1.0 1.20 ± 4% perf-profile.children.cycles-pp.___perf_sw_event 0.15 ± 26% +1.0 1.20 ± 3% perf-profile.children.cycles-pp.pick_next_entity 0.15 ± 20% +1.1 1.21 ± 7% perf-profile.children.cycles-pp.put_prev_entity 0.17 ± 22% +1.1 1.23 ± 4% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.19 ± 10% +1.3 1.52 ± 10% perf-profile.children.cycles-pp.__enqueue_entity 0.24 ± 23% +1.3 1.59 ± 6% perf-profile.children.cycles-pp.__update_load_avg_se 0.28 ± 23% +1.4 1.64 ± 3% perf-profile.children.cycles-pp.switch_fpu_return 0.31 ± 29% +1.6 1.88 ± 5% perf-profile.children.cycles-pp.update_rq_clock 0.44 ± 21% +1.8 2.24 ± 4% perf-profile.children.cycles-pp.cpumask_next_wrap 0.33 ± 20% +1.8 2.15 ± 4% perf-profile.children.cycles-pp.dequeue_entity 23.28 ± 2% +2.0 25.27 ± 2% perf-profile.children.cycles-pp.ksys_read 0.46 ± 25% +2.1 2.57 ± 5% perf-profile.children.cycles-pp.update_load_avg 0.20 ± 21% +2.1 2.34 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.91 ± 29% +2.2 3.12 ± 11% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.38 ± 23% +2.4 2.80 ± 4% perf-profile.children.cycles-pp.reweight_entity 0.52 ± 18% +2.4 2.96 ± 3% perf-profile.children.cycles-pp.enqueue_entity 0.49 ± 23% +2.6 3.10 ± 4% perf-profile.children.cycles-pp.update_curr 20.92 ± 2% +3.2 24.12 ± 2% perf-profile.children.cycles-pp.vfs_read 0.21 ± 23% +3.4 3.65 ± 7% perf-profile.children.cycles-pp._raw_spin_lock 0.76 ± 24% +3.9 4.67 ± 3% perf-profile.children.cycles-pp.pick_next_task_fair 0.66 ± 34% +4.4 5.06 ± 2% perf-profile.children.cycles-pp.exit_to_usermode_loop 0.81 ± 22% +5.0 5.83 ± 4% perf-profile.children.cycles-pp.dequeue_task_fair 1.01 ± 21% +5.8 6.86 ± 4% perf-profile.children.cycles-pp.enqueue_task_fair 1.06 ± 21% +6.0 7.10 ± 4% perf-profile.children.cycles-pp.activate_task 1.07 ± 21% +6.1 7.15 ± 4% perf-profile.children.cycles-pp.ttwu_do_activate 13.19 +7.8 20.94 ± 3% perf-profile.children.cycles-pp.new_sync_read 11.61 +8.7 20.29 ± 3% perf-profile.children.cycles-pp.pipe_read 1.45 ± 25% +8.9 10.35 ± 7% perf-profile.children.cycles-pp.available_idle_cpu 78.85 ± 2% +9.8 88.66 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 77.90 ± 2% +10.1 88.00 perf-profile.children.cycles-pp.do_syscall_64 2.42 ± 21% +12.9 15.36 ± 4% perf-profile.children.cycles-pp.pipe_wait 2.71 ± 26% +15.4 18.12 ± 4% perf-profile.children.cycles-pp.__schedule 2.76 ± 25% +15.6 18.37 ± 4% perf-profile.children.cycles-pp.schedule 27.59 ± 2% +18.6 46.17 ± 3% perf-profile.children.cycles-pp.ksys_write 2.26 ± 23% +19.0 21.30 ± 7% perf-profile.children.cycles-pp.select_idle_sibling 24.98 ± 2% +19.9 44.88 ± 3% perf-profile.children.cycles-pp.vfs_write 2.48 ± 23% +20.0 22.46 ± 7% perf-profile.children.cycles-pp.select_task_rq_fair 19.06 ± 3% +23.3 42.41 ± 4% perf-profile.children.cycles-pp.new_sync_write 17.98 ± 4% +23.9 41.93 ± 4% perf-profile.children.cycles-pp.pipe_write 4.03 ± 21% +30.6 34.60 ± 6% perf-profile.children.cycles-pp.try_to_wake_up 3.97 ± 22% +30.8 34.74 ± 6% perf-profile.children.cycles-pp.autoremove_wake_function 4.30 ± 20% +30.9 35.19 ± 6% perf-profile.children.cycles-pp.__wake_up_common 6.06 ± 14% +31.2 37.21 ± 5% perf-profile.children.cycles-pp.__wake_up_common_lock 24.41 ± 6% -15.5 8.96 ± 10% perf-profile.self.cycles-pp.do_syscall_64 9.18 ± 6% -6.0 3.14 ± 12% perf-profile.self.cycles-pp.syscall_return_via_sysret 8.63 ± 5% -5.6 3.00 ± 12% perf-profile.self.cycles-pp.entry_SYSCALL_64 3.71 ± 3% -2.0 1.73 ± 6% perf-profile.self.cycles-pp.mutex_unlock 2.47 ± 3% -1.5 0.96 ± 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 2.76 ± 3% -1.4 1.35 ± 8% perf-profile.self.cycles-pp.selinux_file_permission 2.02 ± 7% -1.3 0.68 ± 12% perf-profile.self.cycles-pp.fsnotify 2.46 ± 6% -1.3 1.20 ± 8% perf-profile.self.cycles-pp.__fget 1.75 ± 6% -1.1 0.64 ± 10% perf-profile.self.cycles-pp.avc_has_perm 1.83 ± 5% -1.1 0.75 ± 10% perf-profile.self.cycles-pp.___might_sleep 1.75 -1.0 0.77 ± 7% perf-profile.self.cycles-pp.pipe_write 1.20 ± 4% -0.7 0.46 ± 10% perf-profile.self.cycles-pp.__might_sleep 1.29 ± 11% -0.7 0.58 ± 10% perf-profile.self.cycles-pp.new_sync_read 1.63 ± 4% -0.7 0.97 ± 5% perf-profile.self.cycles-pp.pipe_read 1.15 ± 5% -0.7 0.49 ± 8% perf-profile.self.cycles-pp.__fsnotify_parent 0.89 ± 8% -0.6 0.27 ± 15% perf-profile.self.cycles-pp.copy_page_from_iter 0.89 ± 4% -0.6 0.29 ± 14% perf-profile.self.cycles-pp.security_file_permission 0.83 ± 5% -0.6 0.24 ± 21% perf-profile.self.cycles-pp.file_has_perm 0.90 ± 2% -0.5 0.41 ± 6% perf-profile.self.cycles-pp.new_sync_write 0.67 ± 10% -0.5 0.20 ± 15% perf-profile.self.cycles-pp.current_time 0.83 ± 4% -0.4 0.41 ± 10% perf-profile.self.cycles-pp.copy_page_to_iter 0.72 ± 7% -0.4 0.31 ± 9% perf-profile.self.cycles-pp.vfs_write 0.75 ± 4% -0.4 0.35 ± 9% perf-profile.self.cycles-pp.fput_many 1.01 ± 4% -0.3 0.69 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.45 ± 4% -0.3 0.14 ± 19% perf-profile.self.cycles-pp.inode_has_perm 0.54 ± 8% -0.3 0.23 ± 9% perf-profile.self.cycles-pp.file_update_time 0.41 ± 10% -0.3 0.11 ± 20% perf-profile.self.cycles-pp.iov_iter_init 0.49 ± 7% -0.3 0.19 ± 8% perf-profile.self.cycles-pp.atime_needs_update 0.44 ± 7% -0.3 0.15 ± 14% perf-profile.self.cycles-pp.ksys_read 0.44 ± 7% -0.3 0.17 ± 9% perf-profile.self.cycles-pp.__inode_security_revalidate 0.41 ± 2% -0.3 0.15 ± 8% perf-profile.self.cycles-pp.rcu_all_qs 0.43 ± 8% -0.3 0.17 ± 11% perf-profile.self.cycles-pp.ksys_write 0.37 ± 9% -0.3 0.11 ± 17% perf-profile.self.cycles-pp.timespec64_trunc 0.42 ± 14% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.__sb_start_write 0.39 ± 7% -0.2 0.17 ± 11% perf-profile.self.cycles-pp.__x64_sys_write 0.40 ± 8% -0.2 0.17 ± 8% perf-profile.self.cycles-pp.__x64_sys_read 0.41 ± 7% -0.2 0.20 ± 11% perf-profile.self.cycles-pp.__wake_up_common_lock 0.60 ± 4% -0.2 0.39 ± 7% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 1.25 -0.2 1.04 ± 2% perf-profile.self.cycles-pp.mutex_lock 0.30 ± 6% -0.2 0.12 ± 15% perf-profile.self.cycles-pp.__sb_end_write 0.35 ± 5% -0.2 0.18 ± 3% perf-profile.self.cycles-pp.rw_verify_area 0.34 ± 7% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.touch_atime 0.32 ± 4% -0.2 0.15 ± 14% perf-profile.self.cycles-pp.generic_pipe_buf_confirm 0.23 ± 16% -0.2 0.08 ± 11% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.74 ± 3% -0.2 0.59 ± 5% perf-profile.self.cycles-pp.vfs_read 0.60 ± 3% -0.2 0.45 ± 4% perf-profile.self.cycles-pp.__fget_light 0.27 ± 8% -0.1 0.12 ± 15% perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.18 ± 45% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.__vfs_read 0.18 ± 5% -0.1 0.06 ± 17% perf-profile.self.cycles-pp.__fdget_pos 0.25 ± 5% -0.1 0.14 ± 11% perf-profile.self.cycles-pp.__might_fault 0.07 ± 7% -0.0 0.04 ± 57% perf-profile.self.cycles-pp.kill_fasync 0.01 ±173% +0.1 0.06 ± 17% perf-profile.self.cycles-pp.sched_ttwu_pending 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.scheduler_ipi 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.rcu_note_context_switch 0.01 ±173% +0.1 0.07 ± 12% perf-profile.self.cycles-pp.generic_update_time 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.cpumask_next 0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.ttwu_do_activate 0.00 +0.1 0.06 perf-profile.self.cycles-pp.__mnt_want_write 0.06 ± 14% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.__mark_inode_dirty 0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.is_cpu_allowed 0.00 +0.1 0.07 ± 13% perf-profile.self.cycles-pp.sched_clock_cpu 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.sched_clock 0.03 ±100% +0.1 0.10 ± 11% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.00 +0.1 0.08 perf-profile.self.cycles-pp.native_load_tls 0.05 ± 60% +0.1 0.13 ± 5% perf-profile.self.cycles-pp.native_irq_return_iret 0.00 +0.1 0.09 ± 9% perf-profile.self.cycles-pp.ttwu_do_wakeup 0.00 +0.1 0.09 ± 11% perf-profile.self.cycles-pp.rb_next 0.00 +0.1 0.10 ± 8% perf-profile.self.cycles-pp.__list_add_valid 0.00 +0.1 0.10 ± 10% perf-profile.self.cycles-pp.rb_insert_color 0.00 +0.1 0.11 ± 9% perf-profile.self.cycles-pp.resched_curr 0.00 +0.1 0.11 ± 7% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.32 ± 7% +0.1 0.45 ± 5% perf-profile.self.cycles-pp.__wake_up_common 0.13 ± 10% +0.1 0.25 ± 5% perf-profile.self.cycles-pp.__list_del_entry_valid 0.00 +0.1 0.13 ± 5% perf-profile.self.cycles-pp.autoremove_wake_function 0.00 +0.1 0.13 ± 8% perf-profile.self.cycles-pp.check_preempt_curr 0.04 ± 58% +0.1 0.18 ± 10% perf-profile.self.cycles-pp.finish_wait 0.00 +0.1 0.14 ± 5% perf-profile.self.cycles-pp.set_next_buddy 0.00 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.put_prev_entity 0.00 +0.2 0.15 ± 5% perf-profile.self.cycles-pp.deactivate_task 0.05 ± 9% +0.2 0.21 ± 5% perf-profile.self.cycles-pp.rb_erase 0.00 +0.2 0.15 ± 10% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.00 +0.2 0.18 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.15 ± 7% +0.2 0.34 ± 2% perf-profile.self.cycles-pp.anon_pipe_buf_release 0.03 ±100% +0.2 0.21 ± 8% perf-profile.self.cycles-pp.pipe_wait 0.03 ±102% +0.2 0.24 ± 3% perf-profile.self.cycles-pp.activate_task 0.01 ±173% +0.2 0.23 ± 3% perf-profile.self.cycles-pp.prepare_to_wait 0.03 ±100% +0.2 0.25 ± 4% perf-profile.self.cycles-pp.set_next_entity 0.02 ±173% +0.2 0.26 ± 9% perf-profile.self.cycles-pp.dequeue_entity 0.04 ±100% +0.3 0.29 ± 5% perf-profile.self.cycles-pp.enqueue_entity 0.08 ± 31% +0.3 0.34 ± 2% perf-profile.self.cycles-pp.schedule 0.78 ± 3% +0.3 1.09 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.09 ± 24% +0.3 0.42 ± 7% perf-profile.self.cycles-pp.finish_task_switch 0.03 ±100% +0.3 0.37 ± 6% perf-profile.self.cycles-pp.update_min_vruntime 0.03 ±102% +0.4 0.38 perf-profile.self.cycles-pp.clear_buddies 0.03 ±100% +0.4 0.39 ± 12% perf-profile.self.cycles-pp.cpus_share_cache 0.07 ± 22% +0.4 0.44 ± 7% perf-profile.self.cycles-pp.check_preempt_wakeup 0.09 ± 27% +0.4 0.47 ± 4% perf-profile.self.cycles-pp.native_sched_clock 0.07 ± 26% +0.4 0.45 ± 4% perf-profile.self.cycles-pp.pick_next_entity 0.11 ± 15% +0.4 0.50 ± 3% perf-profile.self.cycles-pp.cpuacct_charge 0.07 ± 14% +0.5 0.53 ± 8% perf-profile.self.cycles-pp.account_entity_enqueue 0.10 ± 21% +0.5 0.56 ± 8% perf-profile.self.cycles-pp.try_to_wake_up 0.10 ± 18% +0.5 0.62 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair 0.14 ± 26% +0.6 0.75 ± 4% perf-profile.self.cycles-pp.__calc_delta 0.06 ± 13% +0.6 0.68 ± 4% perf-profile.self.cycles-pp.account_entity_dequeue 0.15 ± 27% +0.6 0.79 ± 6% perf-profile.self.cycles-pp.enqueue_task_fair 0.17 ± 30% +0.7 0.89 ± 4% perf-profile.self.cycles-pp.update_load_avg 0.15 ± 20% +0.7 0.88 ± 4% perf-profile.self.cycles-pp.reweight_entity 0.15 ± 23% +0.8 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair 0.24 ± 30% +0.9 1.11 perf-profile.self.cycles-pp.native_write_msr 0.20 ± 23% +0.9 1.07 ± 5% perf-profile.self.cycles-pp.select_task_rq_fair 0.20 ± 22% +0.9 1.07 ± 3% perf-profile.self.cycles-pp.find_next_bit 0.20 ± 31% +0.9 1.11 ± 3% perf-profile.self.cycles-pp.update_cfs_group 0.20 ± 28% +0.9 1.14 ± 2% perf-profile.self.cycles-pp.__switch_to 0.14 ± 37% +1.0 1.13 ± 5% perf-profile.self.cycles-pp.___perf_sw_event 0.21 ± 24% +1.0 1.22 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 0.24 ± 23% +1.1 1.31 ± 5% perf-profile.self.cycles-pp.cpumask_next_wrap 0.23 ± 32% +1.3 1.49 ± 5% perf-profile.self.cycles-pp.update_rq_clock 0.19 ± 28% +1.3 1.45 ± 6% perf-profile.self.cycles-pp.update_curr 0.23 ± 23% +1.3 1.55 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se 0.19 ± 10% +1.3 1.51 ± 10% perf-profile.self.cycles-pp.__enqueue_entity 0.28 ± 22% +1.3 1.63 ± 3% perf-profile.self.cycles-pp.switch_fpu_return 0.11 ± 19% +1.6 1.72 ± 5% perf-profile.self.cycles-pp._raw_spin_lock 0.42 ± 26% +1.7 2.12 ± 2% perf-profile.self.cycles-pp.__schedule 0.19 ± 20% +2.0 2.20 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.91 ± 29% +2.2 3.11 ± 12% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.32 ± 19% +7.7 8.05 ± 7% perf-profile.self.cycles-pp.select_idle_sibling 1.44 ± 25% +8.8 10.23 ± 7% perf-profile.self.cycles-pp.available_idle_cpu 439.00 +11.7% 490.50 interrupts.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59 436.25 +13.1% 493.25 interrupts.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60 436.25 +12.4% 490.50 interrupts.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61 436.25 +12.4% 490.50 interrupts.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62 509.75 ± 3% +545.1% 3288 ±120% interrupts.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3 474.00 ± 3% +49.7% 709.75 ± 24% interrupts.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7 588.00 ± 35% +196.0% 1740 ± 50% interrupts.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12 436.75 ± 2% +12.8% 492.50 interrupts.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16 438.00 +39.5% 611.00 ± 33% interrupts.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18 436.25 +13.8% 496.25 interrupts.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19 436.25 +14.3% 498.50 interrupts.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20 437.00 +12.2% 490.50 interrupts.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21 436.25 +12.4% 490.50 interrupts.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22 436.25 +12.4% 490.50 interrupts.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23 441.00 +11.8% 493.00 interrupts.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24 437.25 +13.3% 495.25 interrupts.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25 436.25 +12.4% 490.50 interrupts.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26 436.25 +12.4% 490.50 interrupts.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29 436.25 +12.4% 490.50 interrupts.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30 436.25 +12.4% 490.50 interrupts.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32 437.25 ± 2% +12.2% 490.50 interrupts.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33 436.25 +12.6% 491.25 interrupts.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34 440.75 +12.4% 495.25 ± 2% interrupts.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35 436.25 +12.6% 491.00 interrupts.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36 436.75 +13.1% 493.75 interrupts.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37 436.25 +12.8% 492.25 interrupts.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38 436.25 +12.8% 492.25 interrupts.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39 436.25 +12.4% 490.50 interrupts.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40 436.25 +12.4% 490.50 interrupts.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41 436.25 +12.4% 490.50 interrupts.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42 436.25 +14.6% 499.75 ± 2% interrupts.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43 436.25 +12.4% 490.50 interrupts.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44 436.50 +12.4% 490.50 interrupts.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45 436.25 +12.4% 490.50 interrupts.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46 436.25 +12.4% 490.50 interrupts.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47 436.25 +12.6% 491.00 interrupts.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48 441.25 ± 3% +14.1% 503.25 ± 3% interrupts.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49 436.25 +12.4% 490.50 interrupts.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50 436.25 +14.4% 499.25 ± 3% interrupts.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51 439.25 ± 2% +11.7% 490.50 interrupts.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53 437.75 +12.1% 490.50 interrupts.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54 436.50 +12.4% 490.50 interrupts.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55 436.25 +12.4% 490.50 interrupts.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56 439.00 ± 2% +11.7% 490.50 interrupts.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57 442.25 ± 3% +10.9% 490.50 interrupts.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58 1989 +12.1% 2230 interrupts.9:IR-IO-APIC.9-fasteoi.acpi 6596036 ± 10% -21.1% 5206443 ± 6% interrupts.CAL:Function_call_interrupts 86097 ± 13% -18.4% 70229 ± 7% interrupts.CPU0.CAL:Function_call_interrupts 1785369 +12.4% 2006294 interrupts.CPU0.LOC:Local_timer_interrupts 102823 ± 12% -18.3% 84031 ± 5% interrupts.CPU0.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496548 ± 4% interrupts.CPU0.TRM:Thermal_event_interrupts 1989 +12.1% 2230 interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi 90370 ± 8% -21.4% 70995 ± 6% interrupts.CPU1.CAL:Function_call_interrupts 1784834 +12.4% 2006046 interrupts.CPU1.LOC:Local_timer_interrupts 3088391 ± 15% -20.9% 2441882 ± 7% interrupts.CPU1.RES:Rescheduling_interrupts 105391 ± 8% -18.6% 85793 ± 6% interrupts.CPU1.TLB:TLB_shootdowns 388373 ± 12% +27.9% 496558 ± 4% interrupts.CPU1.TRM:Thermal_event_interrupts 88236 ± 11% -19.5% 71001 ± 7% interrupts.CPU10.CAL:Function_call_interrupts 1785471 +12.3% 2004740 interrupts.CPU10.LOC:Local_timer_interrupts 3040774 ± 4% -21.1% 2398396 ± 8% interrupts.CPU10.RES:Rescheduling_interrupts 104483 ± 10% -18.6% 85025 ± 5% interrupts.CPU10.TLB:TLB_shootdowns 388417 ± 12% +27.8% 496560 ± 4% interrupts.CPU10.TRM:Thermal_event_interrupts 1785039 +12.3% 2005274 interrupts.CPU11.LOC:Local_timer_interrupts 925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.NMI:Non-maskable_interrupts 925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.PMI:Performance_monitoring_interrupts 3094193 ± 14% -21.5% 2430228 ± 8% interrupts.CPU11.RES:Rescheduling_interrupts 102398 ± 10% -16.1% 85884 ± 5% interrupts.CPU11.TLB:TLB_shootdowns 388413 ± 12% +27.8% 496474 ± 4% interrupts.CPU11.TRM:Thermal_event_interrupts 588.00 ± 35% +196.0% 1740 ± 50% interrupts.CPU12.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12 87438 ± 9% -18.1% 71580 ± 7% interrupts.CPU12.CAL:Function_call_interrupts 1785376 +12.3% 2005068 interrupts.CPU12.LOC:Local_timer_interrupts 103057 ± 10% -16.4% 86186 ± 6% interrupts.CPU12.TLB:TLB_shootdowns 388416 ± 12% +27.8% 496490 ± 4% interrupts.CPU12.TRM:Thermal_event_interrupts 88735 ± 9% -19.4% 71496 ± 9% interrupts.CPU13.CAL:Function_call_interrupts 1785460 +12.3% 2005445 interrupts.CPU13.LOC:Local_timer_interrupts 104230 ± 9% -17.5% 86038 ± 6% interrupts.CPU13.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU13.TRM:Thermal_event_interrupts 88327 ± 12% -19.0% 71507 ± 7% interrupts.CPU14.CAL:Function_call_interrupts 1785094 +12.3% 2005441 interrupts.CPU14.LOC:Local_timer_interrupts 103317 ± 11% -18.1% 84581 ± 6% interrupts.CPU14.TLB:TLB_shootdowns 388412 ± 12% +27.8% 496564 ± 4% interrupts.CPU14.TRM:Thermal_event_interrupts 86972 ± 10% -17.3% 71898 ± 7% interrupts.CPU15.CAL:Function_call_interrupts 1784668 +12.4% 2005375 interrupts.CPU15.LOC:Local_timer_interrupts 3039799 ± 9% -21.1% 2397505 ± 9% interrupts.CPU15.RES:Rescheduling_interrupts 103367 ± 12% -16.9% 85892 ± 6% interrupts.CPU15.TLB:TLB_shootdowns 388342 ± 12% +27.9% 496559 ± 4% interrupts.CPU15.TRM:Thermal_event_interrupts 436.75 ± 2% +12.8% 492.50 interrupts.CPU16.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16 1784750 +12.4% 2005330 interrupts.CPU16.LOC:Local_timer_interrupts 55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.NMI:Non-maskable_interrupts 55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.PMI:Performance_monitoring_interrupts 2969996 ± 10% -18.7% 2413141 ± 9% interrupts.CPU16.RES:Rescheduling_interrupts 102367 ± 11% -13.7% 88305 ± 3% interrupts.CPU16.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496563 ± 4% interrupts.CPU16.TRM:Thermal_event_interrupts 1785363 +12.3% 2005849 interrupts.CPU17.LOC:Local_timer_interrupts 388418 ± 12% +27.8% 496560 ± 4% interrupts.CPU17.TRM:Thermal_event_interrupts 438.00 +39.5% 611.00 ± 33% interrupts.CPU18.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18 88992 ± 12% -22.3% 69145 ± 6% interrupts.CPU18.CAL:Function_call_interrupts 1785398 +12.1% 2001316 interrupts.CPU18.LOC:Local_timer_interrupts 2199105 ± 7% +115.2% 4731679 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts 108465 ± 8% -23.0% 83467 ± 5% interrupts.CPU18.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380131 ± 2% interrupts.CPU18.TRM:Thermal_event_interrupts 436.25 +13.8% 496.25 interrupts.CPU19.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19 92059 ± 6% -24.2% 69776 ± 5% interrupts.CPU19.CAL:Function_call_interrupts 1784665 +12.2% 2001579 interrupts.CPU19.LOC:Local_timer_interrupts 1284622 ± 7% +225.8% 4185208 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 108492 ± 7% -23.4% 83059 ± 4% interrupts.CPU19.TLB:TLB_shootdowns 329166 ± 3% +15.5% 380141 ± 2% interrupts.CPU19.TRM:Thermal_event_interrupts 1784512 +12.4% 2005572 interrupts.CPU2.LOC:Local_timer_interrupts 3091132 ± 8% -22.3% 2401342 ± 4% interrupts.CPU2.RES:Rescheduling_interrupts 102562 ± 9% -16.5% 85643 ± 5% interrupts.CPU2.TLB:TLB_shootdowns 388380 ± 12% +27.8% 496544 ± 4% interrupts.CPU2.TRM:Thermal_event_interrupts 436.25 +14.3% 498.50 interrupts.CPU20.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20 91785 ± 9% -23.9% 69853 ± 4% interrupts.CPU20.CAL:Function_call_interrupts 1785284 +12.1% 2001952 interrupts.CPU20.LOC:Local_timer_interrupts 1281621 ± 12% +203.6% 3890516 ± 7% interrupts.CPU20.RES:Rescheduling_interrupts 106297 ± 9% -19.9% 85180 ± 3% interrupts.CPU20.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU20.TRM:Thermal_event_interrupts 437.00 +12.2% 490.50 interrupts.CPU21.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21 92635 ± 7% -23.9% 70512 ± 6% interrupts.CPU21.CAL:Function_call_interrupts 1785480 +12.1% 2001723 interrupts.CPU21.LOC:Local_timer_interrupts 1595447 ± 6% +187.8% 4591004 interrupts.CPU21.RES:Rescheduling_interrupts 107575 ± 7% -21.6% 84296 ± 5% interrupts.CPU21.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380086 ± 2% interrupts.CPU21.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU22.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22 92674 ± 10% -24.4% 70047 ± 6% interrupts.CPU22.CAL:Function_call_interrupts 1784881 +12.1% 2001482 interrupts.CPU22.LOC:Local_timer_interrupts 1605143 ± 6% +179.5% 4486945 ± 5% interrupts.CPU22.RES:Rescheduling_interrupts 108749 ± 10% -23.3% 83409 ± 5% interrupts.CPU22.TLB:TLB_shootdowns 329295 ± 3% +15.4% 380136 ± 2% interrupts.CPU22.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU23.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23 90975 ± 11% -23.6% 69492 ± 5% interrupts.CPU23.CAL:Function_call_interrupts 1785236 +12.1% 2001811 interrupts.CPU23.LOC:Local_timer_interrupts 1600988 ± 2% +180.2% 4486667 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts 106819 ± 10% -22.3% 83007 ± 4% interrupts.CPU23.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380146 ± 2% interrupts.CPU23.TRM:Thermal_event_interrupts 441.00 +11.8% 493.00 interrupts.CPU24.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24 91919 ± 11% -23.1% 70689 ± 5% interrupts.CPU24.CAL:Function_call_interrupts 1784996 +12.1% 2000922 interrupts.CPU24.LOC:Local_timer_interrupts 1612237 ± 5% +188.4% 4649702 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts 108187 ± 8% -22.5% 83831 ± 3% interrupts.CPU24.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380122 ± 2% interrupts.CPU24.TRM:Thermal_event_interrupts 437.25 +13.3% 495.25 interrupts.CPU25.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25 88417 ± 12% -20.8% 70029 ± 4% interrupts.CPU25.CAL:Function_call_interrupts 1785404 +12.0% 1999622 interrupts.CPU25.LOC:Local_timer_interrupts 1612074 ± 2% +182.4% 4551699 interrupts.CPU25.RES:Rescheduling_interrupts 105324 ± 10% -20.1% 84201 ± 4% interrupts.CPU25.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU25.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU26.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26 92632 ± 12% -23.5% 70907 ± 5% interrupts.CPU26.CAL:Function_call_interrupts 1784730 +12.1% 2000264 interrupts.CPU26.LOC:Local_timer_interrupts 1639148 ± 6% +188.0% 4720149 ± 4% interrupts.CPU26.RES:Rescheduling_interrupts 108099 ± 11% -21.6% 84727 ± 4% interrupts.CPU26.TLB:TLB_shootdowns 329271 ± 3% +15.4% 379984 ± 2% interrupts.CPU26.TRM:Thermal_event_interrupts 91312 ± 12% -23.4% 69963 ± 6% interrupts.CPU27.CAL:Function_call_interrupts 1785171 +12.1% 2001743 interrupts.CPU27.LOC:Local_timer_interrupts 1704847 ± 11% +179.4% 4764019 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts 106515 ± 11% -21.6% 83491 ± 5% interrupts.CPU27.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380080 ± 2% interrupts.CPU27.TRM:Thermal_event_interrupts 89178 ± 8% -22.0% 69565 ± 6% interrupts.CPU28.CAL:Function_call_interrupts 1785194 +12.1% 2001836 interrupts.CPU28.LOC:Local_timer_interrupts 1593025 ± 8% +185.7% 4551782 ± 5% interrupts.CPU28.RES:Rescheduling_interrupts 104408 ± 10% -19.2% 84365 ± 6% interrupts.CPU28.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380142 ± 2% interrupts.CPU28.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU29.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29 90850 ± 12% -21.9% 70993 ± 6% interrupts.CPU29.CAL:Function_call_interrupts 1785398 +12.1% 2000545 interrupts.CPU29.LOC:Local_timer_interrupts 1657508 ± 5% +179.4% 4631109 ± 3% interrupts.CPU29.RES:Rescheduling_interrupts 106383 ± 13% -20.8% 84284 ± 4% interrupts.CPU29.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380133 ± 2% interrupts.CPU29.TRM:Thermal_event_interrupts 509.75 ± 3% +545.1% 3288 ±120% interrupts.CPU3.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3 87008 ± 11% -19.9% 69730 ± 5% interrupts.CPU3.CAL:Function_call_interrupts 1784446 +12.4% 2005368 interrupts.CPU3.LOC:Local_timer_interrupts 3169674 ± 9% -20.4% 2521963 ± 8% interrupts.CPU3.RES:Rescheduling_interrupts 103703 ± 12% -17.1% 85958 ± 7% interrupts.CPU3.TLB:TLB_shootdowns 388356 ± 12% +27.9% 496551 ± 4% interrupts.CPU3.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU30.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30 92043 ± 12% -24.2% 69765 ± 5% interrupts.CPU30.CAL:Function_call_interrupts 1785175 +12.1% 2001248 interrupts.CPU30.LOC:Local_timer_interrupts 1637465 ± 6% +191.3% 4769701 ± 3% interrupts.CPU30.RES:Rescheduling_interrupts 106836 ± 11% -22.5% 82810 ± 5% interrupts.CPU30.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU30.TRM:Thermal_event_interrupts 1785301 +12.0% 1999270 interrupts.CPU31.LOC:Local_timer_interrupts 62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.NMI:Non-maskable_interrupts 62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.PMI:Performance_monitoring_interrupts 1589500 ± 5% +184.1% 4515325 interrupts.CPU31.RES:Rescheduling_interrupts 106576 ± 11% -21.5% 83698 ± 5% interrupts.CPU31.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU31.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU32.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32 93348 ± 12% -24.4% 70595 ± 8% interrupts.CPU32.CAL:Function_call_interrupts 1785245 +12.1% 2001145 interrupts.CPU32.LOC:Local_timer_interrupts 1626883 ± 6% +176.3% 4494456 ± 5% interrupts.CPU32.RES:Rescheduling_interrupts 107151 ± 12% -21.4% 84236 ± 5% interrupts.CPU32.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380131 ± 2% interrupts.CPU32.TRM:Thermal_event_interrupts 437.25 ± 2% +12.2% 490.50 interrupts.CPU33.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33 91487 ± 12% -22.7% 70758 ± 3% interrupts.CPU33.CAL:Function_call_interrupts 1785371 +12.0% 1999987 interrupts.CPU33.LOC:Local_timer_interrupts 1578761 ± 6% +201.2% 4754762 ± 6% interrupts.CPU33.RES:Rescheduling_interrupts 106071 ± 12% -21.3% 83430 ± 2% interrupts.CPU33.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU33.TRM:Thermal_event_interrupts 436.25 +12.6% 491.25 interrupts.CPU34.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34 91116 ± 11% -24.3% 68943 ± 7% interrupts.CPU34.CAL:Function_call_interrupts 1785353 +12.1% 2001525 interrupts.CPU34.LOC:Local_timer_interrupts 1553828 ± 4% +186.0% 4444035 ± 3% interrupts.CPU34.RES:Rescheduling_interrupts 105575 ± 10% -20.7% 83762 ± 4% interrupts.CPU34.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380139 ± 2% interrupts.CPU34.TRM:Thermal_event_interrupts 440.75 +12.4% 495.25 ± 2% interrupts.CPU35.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35 91396 ± 11% -25.6% 67986 ± 11% interrupts.CPU35.CAL:Function_call_interrupts 1785237 +12.0% 2000288 interrupts.CPU35.LOC:Local_timer_interrupts 1602616 ± 10% +189.7% 4642115 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts 107669 ± 12% -21.3% 84736 ± 4% interrupts.CPU35.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380145 ± 2% interrupts.CPU35.TRM:Thermal_event_interrupts 436.25 +12.6% 491.00 interrupts.CPU36.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36 93553 ± 8% -19.8% 75044 ± 6% interrupts.CPU36.CAL:Function_call_interrupts 1784636 +12.4% 2005389 interrupts.CPU36.LOC:Local_timer_interrupts 2588984 ± 6% +25.8% 3256348 ± 6% interrupts.CPU36.RES:Rescheduling_interrupts 107053 ± 7% -19.0% 86754 ± 5% interrupts.CPU36.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496560 ± 4% interrupts.CPU36.TRM:Thermal_event_interrupts 436.75 +13.1% 493.75 interrupts.CPU37.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37 89518 ± 10% -18.9% 72619 ± 5% interrupts.CPU37.CAL:Function_call_interrupts 1784648 +12.4% 2005651 interrupts.CPU37.LOC:Local_timer_interrupts 3112476 ± 10% -20.8% 2464103 ± 5% interrupts.CPU37.RES:Rescheduling_interrupts 102576 ± 10% -14.4% 87794 ± 6% interrupts.CPU37.TLB:TLB_shootdowns 388412 ± 12% +27.8% 496563 ± 4% interrupts.CPU37.TRM:Thermal_event_interrupts 436.25 +12.8% 492.25 interrupts.CPU38.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38 92566 ± 10% -20.6% 73466 ± 6% interrupts.CPU38.CAL:Function_call_interrupts 1785132 +12.4% 2005883 interrupts.CPU38.LOC:Local_timer_interrupts 3163520 ± 10% -26.0% 2340820 ± 11% interrupts.CPU38.RES:Rescheduling_interrupts 105823 ± 10% -18.0% 86785 ± 4% interrupts.CPU38.TLB:TLB_shootdowns 388401 ± 12% +27.8% 496554 ± 4% interrupts.CPU38.TRM:Thermal_event_interrupts 436.25 +12.8% 492.25 interrupts.CPU39.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39 93197 ± 11% -21.8% 72854 ± 7% interrupts.CPU39.CAL:Function_call_interrupts 1784624 +12.4% 2005934 interrupts.CPU39.LOC:Local_timer_interrupts 105953 ± 11% -20.8% 83925 ± 6% interrupts.CPU39.TLB:TLB_shootdowns 388418 ± 12% +27.8% 496548 ± 4% interrupts.CPU39.TRM:Thermal_event_interrupts 86883 ± 13% -17.8% 71449 ± 7% interrupts.CPU4.CAL:Function_call_interrupts 1784433 +12.4% 2005462 interrupts.CPU4.LOC:Local_timer_interrupts 3119739 ± 9% -21.1% 2461192 ± 4% interrupts.CPU4.RES:Rescheduling_interrupts 104829 ± 11% -19.1% 84802 ± 5% interrupts.CPU4.TLB:TLB_shootdowns 388417 ± 12% +27.8% 496563 ± 4% interrupts.CPU4.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU40.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40 91881 ± 12% -21.4% 72188 ± 8% interrupts.CPU40.CAL:Function_call_interrupts 1784831 +12.4% 2005991 interrupts.CPU40.LOC:Local_timer_interrupts 3101288 ± 10% -17.2% 2569221 ± 7% interrupts.CPU40.RES:Rescheduling_interrupts 105884 ± 10% -19.0% 85772 ± 5% interrupts.CPU40.TLB:TLB_shootdowns 388415 ± 12% +27.8% 496564 ± 4% interrupts.CPU40.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU41.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41 91739 ± 12% -19.9% 73475 ± 7% interrupts.CPU41.CAL:Function_call_interrupts 1785231 +12.3% 2005398 interrupts.CPU41.LOC:Local_timer_interrupts 3223405 ± 11% -27.9% 2325498 ± 5% interrupts.CPU41.RES:Rescheduling_interrupts 104410 ± 11% -18.7% 84846 ± 6% interrupts.CPU41.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU41.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU42.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42 1785354 +12.4% 2006034 interrupts.CPU42.LOC:Local_timer_interrupts 3068304 ± 9% -22.1% 2390347 ± 6% interrupts.CPU42.RES:Rescheduling_interrupts 107102 ± 9% -20.7% 84890 ± 4% interrupts.CPU42.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496564 ± 4% interrupts.CPU42.TRM:Thermal_event_interrupts 436.25 +14.6% 499.75 ± 2% interrupts.CPU43.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43 92178 ± 9% -18.3% 75306 ± 7% interrupts.CPU43.CAL:Function_call_interrupts 1784759 +12.4% 2005986 interrupts.CPU43.LOC:Local_timer_interrupts 6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.NMI:Non-maskable_interrupts 6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.PMI:Performance_monitoring_interrupts 104773 ± 9% -17.6% 86369 ± 6% interrupts.CPU43.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496556 ± 4% interrupts.CPU43.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU44.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44 92387 ± 10% -21.9% 72174 ± 9% interrupts.CPU44.CAL:Function_call_interrupts 1784603 +12.4% 2006064 interrupts.CPU44.LOC:Local_timer_interrupts 106179 ± 9% -18.6% 86461 ± 6% interrupts.CPU44.TLB:TLB_shootdowns 388426 ± 12% +27.8% 496562 ± 4% interrupts.CPU44.TRM:Thermal_event_interrupts 436.50 +12.4% 490.50 interrupts.CPU45.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45 93641 ± 7% -21.9% 73155 ± 6% interrupts.CPU45.CAL:Function_call_interrupts 1784811 +12.4% 2005654 interrupts.CPU45.LOC:Local_timer_interrupts 3093195 ± 11% -15.6% 2609354 ± 4% interrupts.CPU45.RES:Rescheduling_interrupts 106463 ± 7% -19.6% 85593 ± 5% interrupts.CPU45.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496564 ± 4% interrupts.CPU45.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU46.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46 91739 ± 10% -19.0% 74279 ± 8% interrupts.CPU46.CAL:Function_call_interrupts 1784609 +12.4% 2005522 interrupts.CPU46.LOC:Local_timer_interrupts 2970471 ± 9% -21.0% 2345439 ± 8% interrupts.CPU46.RES:Rescheduling_interrupts 108158 ± 10% -19.8% 86743 ± 4% interrupts.CPU46.TLB:TLB_shootdowns 388368 ± 12% +27.8% 496469 ± 4% interrupts.CPU46.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU47.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47 92856 ± 9% -20.0% 74327 ± 8% interrupts.CPU47.CAL:Function_call_interrupts 1784835 +12.4% 2005700 interrupts.CPU47.LOC:Local_timer_interrupts 3116242 ± 10% -20.4% 2480398 ± 5% interrupts.CPU47.RES:Rescheduling_interrupts 104927 ± 9% -18.6% 85393 ± 6% interrupts.CPU47.TLB:TLB_shootdowns 388403 ± 12% +27.8% 496554 ± 4% interrupts.CPU47.TRM:Thermal_event_interrupts 436.25 +12.6% 491.00 interrupts.CPU48.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48 92889 ± 11% -21.5% 72890 ± 8% interrupts.CPU48.CAL:Function_call_interrupts 1784772 +12.4% 2006205 interrupts.CPU48.LOC:Local_timer_interrupts 3175210 ± 12% -19.7% 2548339 ± 13% interrupts.CPU48.RES:Rescheduling_interrupts 105183 ± 11% -18.6% 85618 ± 6% interrupts.CPU48.TLB:TLB_shootdowns 388426 ± 12% +27.8% 496563 ± 4% interrupts.CPU48.TRM:Thermal_event_interrupts 441.25 ± 3% +14.1% 503.25 ± 3% interrupts.CPU49.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49 90134 ± 10% -18.3% 73603 ± 6% interrupts.CPU49.CAL:Function_call_interrupts 1785145 +12.4% 2005932 interrupts.CPU49.LOC:Local_timer_interrupts 3185609 ± 11% -17.6% 2626502 ± 4% interrupts.CPU49.RES:Rescheduling_interrupts 102737 ± 10% -17.3% 84932 ± 5% interrupts.CPU49.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496560 ± 4% interrupts.CPU49.TRM:Thermal_event_interrupts 92114 ± 9% -23.2% 70721 ± 6% interrupts.CPU5.CAL:Function_call_interrupts 1785135 +12.3% 2005424 interrupts.CPU5.LOC:Local_timer_interrupts 3210646 ± 12% -24.8% 2415935 ± 4% interrupts.CPU5.RES:Rescheduling_interrupts 107349 ± 9% -18.9% 87034 ± 6% interrupts.CPU5.TLB:TLB_shootdowns 388418 ± 12% +27.8% 496564 ± 4% interrupts.CPU5.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU50.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50 92406 ± 10% -20.2% 73771 ± 8% interrupts.CPU50.CAL:Function_call_interrupts 1785200 +12.4% 2006488 interrupts.CPU50.LOC:Local_timer_interrupts 106635 ± 9% -19.9% 85431 ± 6% interrupts.CPU50.TLB:TLB_shootdowns 388419 ± 12% +27.8% 496564 ± 4% interrupts.CPU50.TRM:Thermal_event_interrupts 436.25 +14.4% 499.25 ± 3% interrupts.CPU51.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51 91491 ± 9% -20.8% 72473 ± 6% interrupts.CPU51.CAL:Function_call_interrupts 1784683 +12.4% 2005527 interrupts.CPU51.LOC:Local_timer_interrupts 3051494 ± 8% -24.6% 2300032 ± 10% interrupts.CPU51.RES:Rescheduling_interrupts 104954 ± 10% -20.2% 83775 ± 5% interrupts.CPU51.TLB:TLB_shootdowns 388264 ± 12% +27.9% 496462 ± 4% interrupts.CPU51.TRM:Thermal_event_interrupts 92938 ± 11% -19.7% 74594 ± 7% interrupts.CPU52.CAL:Function_call_interrupts 1785451 +12.4% 2006171 interrupts.CPU52.LOC:Local_timer_interrupts 3025681 ± 11% -17.1% 2506800 ± 10% interrupts.CPU52.RES:Rescheduling_interrupts 107100 ± 11% -19.6% 86156 ± 6% interrupts.CPU52.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU52.TRM:Thermal_event_interrupts 439.25 ± 2% +11.7% 490.50 interrupts.CPU53.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53 93000 ± 12% -20.9% 73549 ± 7% interrupts.CPU53.CAL:Function_call_interrupts 1785290 +12.3% 2005441 interrupts.CPU53.LOC:Local_timer_interrupts 3212623 ± 7% -18.2% 2629478 ± 10% interrupts.CPU53.RES:Rescheduling_interrupts 107303 ± 11% -17.5% 88567 ± 10% interrupts.CPU53.TLB:TLB_shootdowns 388411 ± 12% +27.8% 496557 ± 4% interrupts.CPU53.TRM:Thermal_event_interrupts 437.75 +12.1% 490.50 interrupts.CPU54.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54 1785170 +12.0% 2000090 interrupts.CPU54.LOC:Local_timer_interrupts 5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.NMI:Non-maskable_interrupts 5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.PMI:Performance_monitoring_interrupts 2066106 ± 9% +92.7% 3982033 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts 108150 ± 12% -21.2% 85173 ± 5% interrupts.CPU54.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380146 ± 2% interrupts.CPU54.TRM:Thermal_event_interrupts 436.50 +12.4% 490.50 interrupts.CPU55.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55 91164 ± 12% -19.9% 73014 ± 6% interrupts.CPU55.CAL:Function_call_interrupts 1785472 +12.0% 1999115 interrupts.CPU55.LOC:Local_timer_interrupts 1286654 ± 6% +228.2% 4223034 ± 5% interrupts.CPU55.RES:Rescheduling_interrupts 104943 ± 11% -19.5% 84466 ± 5% interrupts.CPU55.TLB:TLB_shootdowns 329292 ± 3% +15.4% 379918 ± 2% interrupts.CPU55.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU56.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56 96934 ± 13% -23.4% 74215 ± 6% interrupts.CPU56.CAL:Function_call_interrupts 1785010 +12.1% 2000169 interrupts.CPU56.LOC:Local_timer_interrupts 1312758 ± 17% +194.5% 3866522 ± 3% interrupts.CPU56.RES:Rescheduling_interrupts 109307 ± 12% -22.1% 85139 ± 4% interrupts.CPU56.TLB:TLB_shootdowns 329273 ± 3% +15.4% 380131 ± 2% interrupts.CPU56.TRM:Thermal_event_interrupts 439.00 ± 2% +11.7% 490.50 interrupts.CPU57.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57 94960 ± 11% -22.9% 73246 ± 5% interrupts.CPU57.CAL:Function_call_interrupts 1785297 +12.0% 2000019 interrupts.CPU57.LOC:Local_timer_interrupts 1666475 ± 10% +169.6% 4492937 interrupts.CPU57.RES:Rescheduling_interrupts 107534 ± 10% -22.0% 83871 ± 5% interrupts.CPU57.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380136 ± 2% interrupts.CPU57.TRM:Thermal_event_interrupts 442.25 ± 3% +10.9% 490.50 interrupts.CPU58.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58 92818 ± 12% -21.4% 72925 ± 5% interrupts.CPU58.CAL:Function_call_interrupts 1785429 +12.0% 1999528 interrupts.CPU58.LOC:Local_timer_interrupts 1606064 ± 5% +176.8% 4446179 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts 103696 ± 11% -19.5% 83458 ± 4% interrupts.CPU58.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380022 ± 2% interrupts.CPU58.TRM:Thermal_event_interrupts 439.00 +11.7% 490.50 interrupts.CPU59.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59 93876 ± 13% -20.5% 74669 ± 6% interrupts.CPU59.CAL:Function_call_interrupts 1785085 +12.0% 1999992 interrupts.CPU59.LOC:Local_timer_interrupts 1634812 ± 5% +174.5% 4487501 ± 4% interrupts.CPU59.RES:Rescheduling_interrupts 106929 ± 13% -21.0% 84522 ± 5% interrupts.CPU59.TLB:TLB_shootdowns 329291 ± 3% +15.4% 380111 ± 2% interrupts.CPU59.TRM:Thermal_event_interrupts 89930 ± 9% -20.5% 71520 ± 7% interrupts.CPU6.CAL:Function_call_interrupts 1785220 +12.3% 2005468 interrupts.CPU6.LOC:Local_timer_interrupts 106307 ± 7% -19.9% 85162 ± 6% interrupts.CPU6.TLB:TLB_shootdowns 388421 ± 12% +27.8% 496564 ± 4% interrupts.CPU6.TRM:Thermal_event_interrupts 436.25 +13.1% 493.25 interrupts.CPU60.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60 98595 ± 11% -25.6% 73379 ± 5% interrupts.CPU60.CAL:Function_call_interrupts 1785116 +12.1% 2000324 interrupts.CPU60.LOC:Local_timer_interrupts 1604314 ± 8% +186.8% 4601614 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts 110442 ± 10% -24.0% 83932 ± 5% interrupts.CPU60.TLB:TLB_shootdowns 329256 ± 3% +15.5% 380140 ± 2% interrupts.CPU60.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU61.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61 93035 ± 11% -21.4% 73124 ± 5% interrupts.CPU61.CAL:Function_call_interrupts 1785176 +12.1% 2000960 interrupts.CPU61.LOC:Local_timer_interrupts 4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.NMI:Non-maskable_interrupts 4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.PMI:Performance_monitoring_interrupts 1575186 ± 6% +194.9% 4644760 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts 103665 ± 11% -18.7% 84259 ± 5% interrupts.CPU61.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380127 ± 2% interrupts.CPU61.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU62.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62 95188 ± 10% -21.9% 74376 ± 6% interrupts.CPU62.CAL:Function_call_interrupts 1785028 +12.0% 1999262 interrupts.CPU62.LOC:Local_timer_interrupts 1613834 ± 8% +184.7% 4594499 ± 3% interrupts.CPU62.RES:Rescheduling_interrupts 106856 ± 10% -21.4% 83987 ± 5% interrupts.CPU62.TLB:TLB_shootdowns 329270 ± 3% +15.4% 380120 ± 2% interrupts.CPU62.TRM:Thermal_event_interrupts 94411 ± 11% -21.3% 74269 ± 5% interrupts.CPU63.CAL:Function_call_interrupts 1785322 +12.0% 1999237 interrupts.CPU63.LOC:Local_timer_interrupts 1669218 ± 6% +187.2% 4794684 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts 105402 ± 10% -18.3% 86076 ± 8% interrupts.CPU63.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU63.TRM:Thermal_event_interrupts 96060 ± 11% -22.6% 74364 ± 6% interrupts.CPU64.CAL:Function_call_interrupts 1785304 +12.0% 1999571 interrupts.CPU64.LOC:Local_timer_interrupts 1681501 ± 7% +164.0% 4438647 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts 108534 ± 10% -22.8% 83813 ± 5% interrupts.CPU64.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU64.TRM:Thermal_event_interrupts 94178 ± 9% -22.7% 72758 ± 8% interrupts.CPU65.CAL:Function_call_interrupts 1785338 +12.0% 1999408 interrupts.CPU65.LOC:Local_timer_interrupts 1674108 ± 4% +182.2% 4724262 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts 107519 ± 10% -22.6% 83259 ± 5% interrupts.CPU65.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380128 ± 2% interrupts.CPU65.TRM:Thermal_event_interrupts 95135 ± 8% -22.4% 73795 ± 5% interrupts.CPU66.CAL:Function_call_interrupts 1785413 +12.0% 2000352 interrupts.CPU66.LOC:Local_timer_interrupts 616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.NMI:Non-maskable_interrupts 616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.PMI:Performance_monitoring_interrupts 1646166 ± 6% +169.3% 4433267 ± 6% interrupts.CPU66.RES:Rescheduling_interrupts 107190 ± 7% -22.5% 83034 ± 4% interrupts.CPU66.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380118 ± 2% interrupts.CPU66.TRM:Thermal_event_interrupts 95627 ± 10% -22.4% 74249 ± 5% interrupts.CPU67.CAL:Function_call_interrupts 1785314 +12.1% 2001128 interrupts.CPU67.LOC:Local_timer_interrupts 1611146 ± 4% +183.8% 4572204 ± 7% interrupts.CPU67.RES:Rescheduling_interrupts 109283 ± 11% -22.5% 84676 ± 4% interrupts.CPU67.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU67.TRM:Thermal_event_interrupts 95479 ± 11% -20.6% 75791 ± 5% interrupts.CPU68.CAL:Function_call_interrupts 1785192 +12.0% 1999411 interrupts.CPU68.LOC:Local_timer_interrupts 1575394 ± 7% +189.9% 4567277 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts 106972 ± 12% -18.6% 87039 ± 5% interrupts.CPU68.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380129 ± 2% interrupts.CPU68.TRM:Thermal_event_interrupts 93067 ± 14% -19.9% 74538 ± 4% interrupts.CPU69.CAL:Function_call_interrupts 1785257 +12.1% 2000636 interrupts.CPU69.LOC:Local_timer_interrupts 1578663 ± 9% +192.9% 4624518 ± 3% interrupts.CPU69.RES:Rescheduling_interrupts 104271 ± 12% -18.5% 84989 ± 2% interrupts.CPU69.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380066 ± 2% interrupts.CPU69.TRM:Thermal_event_interrupts 474.00 ± 3% +49.7% 709.75 ± 24% interrupts.CPU7.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7 92062 ± 9% -19.7% 73884 ± 5% interrupts.CPU7.CAL:Function_call_interrupts 1784266 +12.4% 2004980 interrupts.CPU7.LOC:Local_timer_interrupts 2505703 ± 11% -19.0% 2029187 ± 5% interrupts.CPU7.RES:Rescheduling_interrupts 107698 ± 8% -18.4% 87903 ± 6% interrupts.CPU7.TLB:TLB_shootdowns 388281 ± 12% +27.8% 496347 ± 4% interrupts.CPU7.TRM:Thermal_event_interrupts 92337 ± 12% -17.5% 76148 ± 2% interrupts.CPU70.CAL:Function_call_interrupts 1785161 +12.0% 1999757 interrupts.CPU70.LOC:Local_timer_interrupts 1562360 ± 4% +170.8% 4231208 ± 9% interrupts.CPU70.RES:Rescheduling_interrupts 103862 ± 10% -18.8% 84371 ± 3% interrupts.CPU70.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU70.TRM:Thermal_event_interrupts 95829 ± 12% -21.2% 75509 ± 5% interrupts.CPU71.CAL:Function_call_interrupts 1784894 +12.1% 2001287 interrupts.CPU71.LOC:Local_timer_interrupts 1533567 ± 7% +196.9% 4553440 interrupts.CPU71.RES:Rescheduling_interrupts 105883 ± 11% -20.4% 84248 ± 6% interrupts.CPU71.TLB:TLB_shootdowns 329247 ± 3% +15.5% 380146 ± 2% interrupts.CPU71.TRM:Thermal_event_interrupts 88372 ± 11% -20.3% 70389 ± 7% interrupts.CPU8.CAL:Function_call_interrupts 1784452 +12.4% 2005305 interrupts.CPU8.LOC:Local_timer_interrupts 103992 ± 10% -17.4% 85887 ± 5% interrupts.CPU8.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496557 ± 4% interrupts.CPU8.TRM:Thermal_event_interrupts 89290 ± 10% -19.1% 72221 ± 6% interrupts.CPU9.CAL:Function_call_interrupts 1784716 +12.4% 2005710 interrupts.CPU9.LOC:Local_timer_interrupts 104340 ± 10% -16.0% 87647 ± 3% interrupts.CPU9.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU9.TRM:Thermal_event_interrupts 1.285e+08 +12.2% 1.442e+08 interrupts.LOC:Local_timer_interrupts 144.00 +50.0% 216.00 interrupts.MCP:Machine_check_polls 1.677e+08 ± 8% +50.4% 2.522e+08 ± 2% interrupts.RES:Rescheduling_interrupts 7624145 ± 10% -19.7% 6125835 ± 5% interrupts.TLB:TLB_shootdowns 25836833 ± 8% +22.2% 31559779 ± 2% interrupts.TRM:Thermal_event_interrupts *************************************************************************************************** lkp-bdw-ep6: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/process/50%/debian-x86_64-2019-11-14.cgz/lkp-bdw-ep6/hackbench/0xb000038 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 134091 -10.6% 119937 hackbench.throughput 3.352e+09 ± 2% +16.1% 3.892e+09 ± 5% hackbench.time.involuntary_context_switches 535571 ± 2% +10.6% 592420 ± 4% numa-meminfo.node0.FilePages 12614 ± 18% -12.2% 11078 ± 18% numa-meminfo.node1.Mapped 3472 ± 6% +18.0% 4096 slabinfo.skbuff_head_cache.active_objs 3568 ± 6% +15.7% 4128 slabinfo.skbuff_head_cache.num_objs 730128 ± 5% -52.2% 349073 ± 25% turbostat.C3 14.77 +5.9% 15.64 turbostat.RAMWatt 24613783 ± 5% -32.6% 16595193 ± 23% numa-numastat.node0.local_node 24632941 ± 5% -32.6% 16605181 ± 23% numa-numastat.node0.numa_hit 25827961 -29.9% 18097150 ± 19% numa-numastat.node1.local_node 25836948 -29.9% 18115676 ± 19% numa-numastat.node1.numa_hit 50467170 ± 2% -31.2% 34733456 ± 10% proc-vmstat.numa_hit 50439016 ± 2% -31.2% 34704933 ± 10% proc-vmstat.numa_local 50622650 ± 2% -31.1% 34896854 ± 10% proc-vmstat.pgalloc_normal 50548381 ± 2% -31.1% 34834391 ± 10% proc-vmstat.pgfree 79.00 +2.2% 80.75 vmstat.cpu.sy 19.00 -7.9% 17.50 ± 2% vmstat.cpu.us 13450755 +4.6% 14066265 vmstat.system.cs 1122772 -12.1% 986681 vmstat.system.in 133891 ± 2% +10.6% 148112 ± 4% numa-vmstat.node0.nr_file_pages 12301995 ± 4% -30.2% 8590153 ± 22% numa-vmstat.node0.numa_hit 12282910 ± 4% -30.1% 8580139 ± 22% numa-vmstat.node0.numa_local 12964354 ± 5% -31.2% 8914408 ± 18% numa-vmstat.node1.numa_hit 12806275 ± 5% -31.7% 8746928 ± 19% numa-vmstat.node1.numa_local 33816695 ± 10% +17.8% 39824104 ± 7% cpuidle.C1.time 21015951 ± 3% +19.6% 25127666 ± 7% cpuidle.C1E.time 730991 ± 5% -52.1% 350128 ± 25% cpuidle.C3.usage 1.497e+08 ± 28% +41.6% 2.119e+08 ± 15% cpuidle.C6.time 6771631 ± 4% -45.2% 3713804 ± 15% cpuidle.POLL.time 5209842 ± 4% -44.1% 2913245 ± 15% cpuidle.POLL.usage 26965506 +5.5% 28444381 ± 4% sched_debug.cfs_rq:/.min_vruntime.avg 28257967 +6.6% 30126596 ± 4% sched_debug.cfs_rq:/.min_vruntime.max 582203 ± 9% +37.0% 797353 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 4.65 ± 8% -10.7% 4.16 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.stddev 4620 ± 6% -10.7% 4125 ± 4% sched_debug.cfs_rq:/.runnable_weight.stddev 582831 ± 9% +36.6% 796226 ± 7% sched_debug.cfs_rq:/.spread0.stddev 154.73 ± 2% -14.7% 132.00 ± 3% sched_debug.cfs_rq:/.util_avg.stddev 120.93 ± 25% +43.5% 173.50 ± 19% sched_debug.cfs_rq:/.util_est_enqueued.min 184.93 ± 2% -11.0% 164.54 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev 182135 ± 18% +57.1% 286098 ± 6% sched_debug.cpu.avg_idle.avg 185926 ± 19% +49.0% 276991 ± 9% sched_debug.cpu.avg_idle.stddev 3.32 ± 4% -10.8% 2.96 ± 3% sched_debug.cpu.nr_running.stddev 46710201 +9.0% 50934866 ± 5% sched_debug.cpu.nr_switches.avg 50053735 +16.9% 58491253 ± 7% sched_debug.cpu.nr_switches.max 43751407 +6.7% 46698664 ± 4% sched_debug.cpu.nr_switches.min 1344975 ± 10% +78.9% 2406504 ± 16% sched_debug.cpu.nr_switches.stddev 1.63 ± 3% -32.8% 1.09 ± 11% sched_debug.cpu.nr_uninterruptible.avg 1017 ± 27% -59.5% 411.75 ± 16% interrupts.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6 6547829 -14.4% 5603910 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts 1017 ± 27% -59.5% 411.75 ± 16% interrupts.CPU19.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6 6801598 -14.7% 5801847 ± 5% interrupts.CPU19.RES:Rescheduling_interrupts 6609756 -13.7% 5703084 ± 8% interrupts.CPU20.RES:Rescheduling_interrupts 6446878 ± 2% -11.8% 5689145 ± 7% interrupts.CPU21.RES:Rescheduling_interrupts 6634825 -9.4% 6008203 ± 6% interrupts.CPU24.RES:Rescheduling_interrupts 6801182 -9.8% 6137528 ± 3% interrupts.CPU25.RES:Rescheduling_interrupts 6687252 -12.9% 5821967 ± 6% interrupts.CPU28.RES:Rescheduling_interrupts 6583387 -10.9% 5868659 ± 6% interrupts.CPU29.RES:Rescheduling_interrupts 6585631 -16.6% 5490726 ± 9% interrupts.CPU37.RES:Rescheduling_interrupts 6759973 -13.2% 5867800 ± 7% interrupts.CPU57.RES:Rescheduling_interrupts 6459616 ± 2% -10.5% 5783047 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts 6580818 ± 2% -16.9% 5468945 ± 3% interrupts.CPU60.RES:Rescheduling_interrupts 6723178 ± 2% -12.4% 5887778 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts 6480129 -16.2% 5427168 ± 7% interrupts.CPU64.RES:Rescheduling_interrupts 6439558 ± 2% -11.6% 5691160 ± 5% interrupts.CPU65.RES:Rescheduling_interrupts 6662822 -12.0% 5866390 ± 7% interrupts.CPU68.RES:Rescheduling_interrupts 6602178 -12.0% 5809760 ± 7% interrupts.CPU7.RES:Rescheduling_interrupts 6616172 -14.1% 5685889 ± 7% interrupts.CPU73.RES:Rescheduling_interrupts 6591477 -13.8% 5682655 ± 8% interrupts.CPU81.RES:Rescheduling_interrupts 13.97 +15.8% 16.18 perf-stat.i.MPKI 1.70 -0.1 1.65 perf-stat.i.branch-miss-rate% 3.607e+08 -1.6% 3.549e+08 perf-stat.i.branch-misses 1.22 ± 6% +0.8 2.01 ± 18% perf-stat.i.cache-miss-rate% 17650711 ± 6% +98.0% 34953649 ± 18% perf-stat.i.cache-misses 1.496e+09 +17.3% 1.755e+09 perf-stat.i.cache-references 13503062 +4.6% 14122231 perf-stat.i.context-switches 2946654 ± 2% +18.5% 3491269 ± 3% perf-stat.i.cpu-migrations 29365 ± 8% -30.6% 20387 ± 31% perf-stat.i.cycles-between-cache-misses 0.35 ± 2% +0.0 0.36 ± 2% perf-stat.i.dTLB-store-miss-rate% 2.048e+10 -2.1% 2.006e+10 perf-stat.i.dTLB-stores 55.17 -1.4 53.74 perf-stat.i.iTLB-load-miss-rate% 1.019e+08 +5.5% 1.074e+08 perf-stat.i.iTLB-loads 3068 -3.0% 2975 perf-stat.i.minor-faults 9195204 ± 6% +113.0% 19589023 ± 18% perf-stat.i.node-load-misses 28132 ± 6% +125.3% 63396 ± 13% perf-stat.i.node-loads 58.92 -9.8 49.16 ± 4% perf-stat.i.node-store-miss-rate% 3588196 ± 6% +60.5% 5760265 ± 19% perf-stat.i.node-store-misses 2415953 ± 5% +145.2% 5924042 ± 18% perf-stat.i.node-stores 3068 -3.0% 2975 perf-stat.i.page-faults 13.96 +16.0% 16.19 perf-stat.overall.MPKI 1.70 -0.1 1.65 perf-stat.overall.branch-miss-rate% 1.18 ± 6% +0.8 1.99 ± 18% perf-stat.overall.cache-miss-rate% 13823 ± 6% -47.8% 7221 ± 20% perf-stat.overall.cycles-between-cache-misses 0.35 ± 2% +0.0 0.36 ± 2% perf-stat.overall.dTLB-store-miss-rate% 55.11 -1.4 53.70 perf-stat.overall.iTLB-load-miss-rate% 59.75 -10.5 49.23 perf-stat.overall.node-store-miss-rate% 78192 +13.0% 88370 perf-stat.overall.path-length 3.601e+08 -1.6% 3.543e+08 perf-stat.ps.branch-misses 17622620 ± 6% +98.0% 34899005 ± 18% perf-stat.ps.cache-misses 1.494e+09 +17.3% 1.752e+09 perf-stat.ps.cache-references 13480657 +4.6% 14099389 perf-stat.ps.context-switches 2941749 ± 2% +18.5% 3485631 ± 3% perf-stat.ps.cpu-migrations 2.045e+10 -2.1% 2.003e+10 perf-stat.ps.dTLB-stores 1.017e+08 +5.5% 1.072e+08 perf-stat.ps.iTLB-loads 3064 -3.0% 2972 perf-stat.ps.minor-faults 9180406 ± 6% +113.0% 19558101 ± 18% perf-stat.ps.node-load-misses 28101 ± 6% +125.3% 63314 ± 13% perf-stat.ps.node-loads 3582467 ± 6% +60.5% 5751138 ± 19% perf-stat.ps.node-store-misses 2412086 ± 5% +145.2% 5914636 ± 18% perf-stat.ps.node-stores 3064 -3.0% 2972 perf-stat.ps.page-faults 223485 +9.1% 243918 ± 5% softirqs.CPU0.TIMER 11887 ± 11% -18.2% 9723 ± 6% softirqs.CPU1.SCHED 215411 +9.2% 235267 ± 6% softirqs.CPU1.TIMER 11293 ± 3% -14.3% 9678 ± 10% softirqs.CPU10.SCHED 213916 +9.8% 234908 ± 5% softirqs.CPU10.TIMER 212758 +7.7% 229141 ± 5% softirqs.CPU11.TIMER 11098 ± 2% -15.9% 9337 ± 10% softirqs.CPU12.SCHED 213397 +9.0% 232706 ± 6% softirqs.CPU12.TIMER 212419 +8.3% 230115 ± 6% softirqs.CPU13.TIMER 10777 ± 4% -12.5% 9426 ± 6% softirqs.CPU14.SCHED 11241 ± 3% -11.5% 9950 ± 6% softirqs.CPU15.SCHED 214604 +10.0% 236116 ± 8% softirqs.CPU15.TIMER 11147 -14.9% 9482 ± 8% softirqs.CPU16.SCHED 216343 +8.4% 234558 ± 6% softirqs.CPU16.TIMER 214680 +7.0% 229748 ± 6% softirqs.CPU17.TIMER 212791 +8.6% 230988 ± 6% softirqs.CPU19.TIMER 11211 ± 4% -10.4% 10047 ± 6% softirqs.CPU2.SCHED 219918 +9.8% 241513 ± 6% softirqs.CPU2.TIMER 10825 ± 4% -14.3% 9273 ± 7% softirqs.CPU20.SCHED 215209 +8.8% 234149 ± 6% softirqs.CPU20.TIMER 10869 ± 4% -11.4% 9626 ± 4% softirqs.CPU21.SCHED 216225 +8.7% 235105 ± 5% softirqs.CPU21.TIMER 13013 ± 7% -14.0% 11192 ± 3% softirqs.CPU22.SCHED 220406 +8.9% 239975 ± 4% softirqs.CPU22.TIMER 11925 -14.9% 10145 ± 3% softirqs.CPU23.SCHED 220231 +7.6% 237056 ± 5% softirqs.CPU23.TIMER 11140 ± 2% -11.5% 9861 ± 7% softirqs.CPU24.SCHED 216091 +8.9% 235338 ± 5% softirqs.CPU24.TIMER 11438 ± 3% -16.3% 9578 ± 2% softirqs.CPU25.SCHED 214880 +9.4% 235143 ± 5% softirqs.CPU25.TIMER 214888 +8.8% 233874 ± 5% softirqs.CPU26.TIMER 11410 ± 2% -13.7% 9844 ± 10% softirqs.CPU27.SCHED 215012 +8.7% 233790 ± 4% softirqs.CPU27.TIMER 220618 ± 2% +7.8% 237788 ± 5% softirqs.CPU28.TIMER 11096 ± 5% -14.3% 9513 ± 6% softirqs.CPU29.SCHED 217194 +8.2% 235108 ± 5% softirqs.CPU29.TIMER 11445 ± 2% -16.7% 9536 ± 10% softirqs.CPU3.SCHED 214692 +9.1% 234221 ± 5% softirqs.CPU3.TIMER 11127 ± 5% -10.5% 9954 ± 6% softirqs.CPU30.SCHED 214861 +8.2% 232532 ± 5% softirqs.CPU30.TIMER 217582 +7.8% 234493 ± 4% softirqs.CPU31.TIMER 214606 +8.8% 233473 ± 4% softirqs.CPU32.TIMER 11299 ± 2% -11.7% 9975 ± 6% softirqs.CPU33.SCHED 11503 ± 2% -16.0% 9658 ± 10% softirqs.CPU34.SCHED 213233 +10.9% 236580 ± 6% softirqs.CPU34.TIMER 11162 ± 2% -11.6% 9863 ± 7% softirqs.CPU35.SCHED 11241 -13.9% 9678 ± 6% softirqs.CPU36.SCHED 215681 +9.2% 235446 ± 4% softirqs.CPU36.TIMER 11358 -15.2% 9632 ± 6% softirqs.CPU37.SCHED 219691 ± 2% +18.5% 260407 ± 13% softirqs.CPU37.TIMER 11570 ± 2% -13.1% 10049 ± 6% softirqs.CPU38.SCHED 215559 +8.2% 233189 ± 4% softirqs.CPU38.TIMER 11295 ± 4% -13.1% 9819 ± 10% softirqs.CPU39.SCHED 215781 ± 2% +7.6% 232108 ± 5% softirqs.CPU39.TIMER 11506 ± 7% -13.0% 10006 ± 6% softirqs.CPU4.SCHED 214973 +8.9% 234140 ± 6% softirqs.CPU4.TIMER 11304 ± 2% -13.7% 9759 ± 10% softirqs.CPU40.SCHED 213827 +10.1% 235341 ± 4% softirqs.CPU40.TIMER 11462 ± 4% -13.5% 9913 ± 8% softirqs.CPU41.SCHED 215879 ± 2% +7.2% 231499 ± 5% softirqs.CPU41.TIMER 11418 ± 3% -14.0% 9819 ± 8% softirqs.CPU43.SCHED 214864 +8.3% 232698 ± 4% softirqs.CPU43.TIMER 10706 ± 5% -13.9% 9223 ± 11% softirqs.CPU44.SCHED 217386 +9.4% 237916 ± 5% softirqs.CPU44.TIMER 215191 +9.7% 236017 ± 5% softirqs.CPU45.TIMER 214214 +9.9% 235466 ± 6% softirqs.CPU46.TIMER 214471 +9.9% 235675 ± 5% softirqs.CPU47.TIMER 11251 ± 2% -13.0% 9785 ± 8% softirqs.CPU48.SCHED 212500 +9.9% 233451 ± 6% softirqs.CPU48.TIMER 213609 +9.9% 234657 ± 5% softirqs.CPU49.TIMER 216836 +8.5% 235225 ± 5% softirqs.CPU5.TIMER 215854 +9.0% 235383 ± 5% softirqs.CPU50.TIMER 11468 -14.4% 9820 ± 9% softirqs.CPU51.SCHED 215402 +8.8% 234360 ± 5% softirqs.CPU51.TIMER 214152 +8.8% 233038 ± 5% softirqs.CPU52.TIMER 212843 +8.4% 230716 ± 5% softirqs.CPU53.TIMER 11373 ± 2% -14.9% 9679 ± 7% softirqs.CPU54.SCHED 214008 +9.5% 234380 ± 5% softirqs.CPU54.TIMER 212619 +7.5% 228539 ± 5% softirqs.CPU55.TIMER 11136 ± 3% -16.4% 9312 ± 10% softirqs.CPU56.SCHED 213404 +9.0% 232542 ± 6% softirqs.CPU56.TIMER 11171 ± 4% -11.9% 9837 ± 7% softirqs.CPU57.SCHED 212283 +8.6% 230474 ± 6% softirqs.CPU57.TIMER 11137 ± 4% -14.0% 9578 ± 6% softirqs.CPU58.SCHED 11226 ± 4% -17.2% 9289 ± 4% softirqs.CPU59.SCHED 214193 +23.1% 263668 ± 22% softirqs.CPU59.TIMER 217195 +8.2% 234975 ± 5% softirqs.CPU6.TIMER 11051 ± 4% -14.4% 9460 ± 9% softirqs.CPU60.SCHED 217131 +7.9% 234363 ± 5% softirqs.CPU60.TIMER 11206 ± 3% -12.6% 9793 ± 9% softirqs.CPU61.SCHED 211791 +9.2% 231302 ± 5% softirqs.CPU61.TIMER 214956 ± 2% +8.8% 233770 ± 5% softirqs.CPU62.TIMER 11180 ± 3% -14.8% 9526 ± 8% softirqs.CPU63.SCHED 213413 +8.7% 232008 ± 5% softirqs.CPU63.TIMER 215674 +8.1% 233086 ± 6% softirqs.CPU64.TIMER 11032 ± 3% -15.7% 9302 ± 7% softirqs.CPU65.SCHED 215870 +9.0% 235386 ± 5% softirqs.CPU65.TIMER 11403 ± 3% -14.2% 9786 ± 6% softirqs.CPU66.SCHED 215968 +9.4% 236175 ± 5% softirqs.CPU66.TIMER 11098 ± 3% -7.4% 10281 ± 4% softirqs.CPU67.SCHED 217846 +8.3% 236023 ± 4% softirqs.CPU67.TIMER 11094 -12.2% 9740 ± 7% softirqs.CPU68.SCHED 215512 +9.4% 235796 ± 5% softirqs.CPU68.TIMER 11321 ± 3% -14.1% 9724 ± 3% softirqs.CPU69.SCHED 213564 +9.0% 232867 ± 5% softirqs.CPU69.TIMER 218822 +8.8% 238080 ± 6% softirqs.CPU7.TIMER 11507 ± 2% -12.9% 10026 ± 4% softirqs.CPU70.SCHED 213928 +9.4% 233986 ± 4% softirqs.CPU70.TIMER 213528 +8.5% 231730 ± 4% softirqs.CPU71.TIMER 11262 ± 2% -11.1% 10013 ± 5% softirqs.CPU72.SCHED 11145 ± 3% -14.5% 9533 ± 8% softirqs.CPU73.SCHED 216467 +9.2% 236409 ± 5% softirqs.CPU73.TIMER 11447 ± 4% -13.2% 9935 ± 4% softirqs.CPU74.SCHED 212578 +9.1% 232022 ± 5% softirqs.CPU74.TIMER 215256 +7.8% 232072 ± 5% softirqs.CPU75.TIMER 11324 ± 4% -11.8% 9985 ± 3% softirqs.CPU76.SCHED 214170 +8.6% 232516 ± 5% softirqs.CPU76.TIMER 11646 ± 5% -15.7% 9824 ± 7% softirqs.CPU77.SCHED 213706 +8.0% 230880 ± 5% softirqs.CPU77.TIMER 11397 ± 6% -13.9% 9810 ± 10% softirqs.CPU78.SCHED 11179 ± 3% -11.5% 9894 ± 5% softirqs.CPU79.SCHED 213215 +8.4% 231081 ± 5% softirqs.CPU79.TIMER 11165 ± 4% -16.2% 9352 ± 9% softirqs.CPU8.SCHED 216508 ± 2% +7.9% 233692 ± 6% softirqs.CPU8.TIMER 11203 ± 2% -13.9% 9643 ± 4% softirqs.CPU80.SCHED 216446 +8.3% 234469 ± 5% softirqs.CPU80.TIMER 11151 ± 2% -12.8% 9726 ± 6% softirqs.CPU81.SCHED 11373 ± 3% -13.0% 9891 ± 6% softirqs.CPU82.SCHED 213597 +8.9% 232608 ± 5% softirqs.CPU82.TIMER 214308 +7.9% 231148 ± 5% softirqs.CPU83.TIMER 214384 +20.9% 259149 ± 15% softirqs.CPU84.TIMER 11458 -13.7% 9887 ± 7% softirqs.CPU86.SCHED 215459 +8.2% 233058 ± 5% softirqs.CPU86.TIMER 11303 ± 4% -12.4% 9904 ± 8% softirqs.CPU87.SCHED 214185 +7.9% 231116 ± 5% softirqs.CPU87.TIMER 11030 ± 2% -13.0% 9593 ± 8% softirqs.CPU9.SCHED 213229 +8.7% 231788 ± 6% softirqs.CPU9.TIMER 995635 -12.9% 867688 ± 5% softirqs.SCHED 19052052 +8.6% 20690944 ± 5% softirqs.TIMER 31.32 -1.6 29.74 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.24 -1.5 28.76 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 26.06 -0.9 25.17 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.35 ± 5% -0.9 1.48 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 25.36 -0.8 24.54 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 2.75 ± 4% -0.7 2.08 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 3.84 ± 2% -0.6 3.23 ± 2% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 7.83 -0.5 7.30 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 1.68 ± 6% -0.5 1.17 ± 10% perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 8.02 -0.5 7.53 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 8.08 -0.5 7.60 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.63 ± 4% -0.4 1.19 ± 6% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write 1.71 ± 6% -0.4 1.28 ± 8% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait 2.60 -0.4 2.19 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.03 ± 2% -0.4 1.64 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.47 -0.4 3.08 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.95 ± 2% -0.3 1.61 ± 3% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 2.67 -0.3 2.35 ± 3% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 0.56 -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.22 ± 4% -0.3 0.94 ± 2% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 1.25 ± 2% -0.3 1.00 ± 5% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64 2.64 -0.2 2.41 ± 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.53 -0.2 1.34 ± 7% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.59 ± 4% -0.2 0.40 ± 57% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.34 -0.2 1.16 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.07 ± 2% -0.2 0.90 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 1.05 -0.2 0.88 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule 1.56 ± 2% -0.2 1.38 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read 1.17 ± 4% -0.2 1.01 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 0.98 ± 2% -0.2 0.82 ± 4% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.66 ± 2% -0.2 1.50 ± 5% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 0.73 ± 3% -0.2 0.57 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.96 ± 2% -0.1 0.82 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write 0.89 ± 2% -0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.66 ± 6% -0.1 0.54 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 2.42 -0.1 2.32 perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.pipe_wait.pipe_read 0.89 ± 4% -0.1 0.79 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write 0.74 ± 2% -0.1 0.66 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.84 -0.1 0.79 ± 3% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.56 -0.0 0.54 ± 2% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.77 ± 2% +0.1 0.82 perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.90 ± 3% +0.1 0.97 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr 0.69 ± 3% +0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.80 ± 4% +0.1 0.89 ± 2% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 1.10 +0.1 1.20 perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.73 ± 3% +0.1 0.83 ± 2% perf-profile.calltrace.cycles-pp.__switch_to 1.05 +0.1 1.16 perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.63 ± 4% +0.1 0.75 ± 2% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.99 ± 5% +0.1 1.12 ± 5% perf-profile.calltrace.cycles-pp.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.07 ± 3% +0.1 1.21 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 0.64 ± 6% +0.2 0.80 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read 1.41 ± 3% +0.2 1.59 ± 2% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.77 ± 5% +0.2 1.00 ± 5% perf-profile.calltrace.cycles-pp.load_new_mm_cr3.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop 1.73 ± 3% +0.3 2.06 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.14 ±173% +0.5 0.63 ± 5% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read 0.13 ±173% +0.5 0.64 ± 5% perf-profile.calltrace.cycles-pp.migrate_task_rq_fair.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common 1.85 ± 5% +0.5 2.38 ± 3% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.00 +0.6 0.59 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.00 +0.6 0.60 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +0.7 0.67 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.15 ±173% +1.0 1.14 ± 12% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 87.66 +1.1 88.78 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 86.87 +1.1 88.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.59 ± 4% +1.4 6.96 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.72 ± 4% +1.4 7.12 ± 3% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.87 ± 4% +1.4 7.28 ± 3% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.70 ± 30% +3.3 4.98 ± 13% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 35.54 +3.3 38.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 34.28 +3.5 37.82 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 31.12 +4.1 35.22 ± 2% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.49 +4.1 34.63 ± 2% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 22.54 ± 2% +6.0 28.53 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 20.58 ± 2% +6.2 26.76 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 19.97 ± 2% +6.2 26.16 ± 3% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 19.73 ± 3% +6.2 25.92 ± 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 3.39 ± 20% +7.0 10.39 ± 12% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.89 ± 15% +7.1 11.98 ± 10% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 31.34 -1.6 29.75 perf-profile.children.cycles-pp.ksys_read 30.28 -1.5 28.79 perf-profile.children.cycles-pp.vfs_read 4.14 ± 5% -1.3 2.85 ± 7% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 3.51 ± 6% -1.0 2.54 ± 8% perf-profile.children.cycles-pp.update_cfs_group 26.10 -0.9 25.20 perf-profile.children.cycles-pp.new_sync_read 25.42 -0.8 24.59 perf-profile.children.cycles-pp.pipe_read 4.67 -0.8 3.86 ± 4% perf-profile.children.cycles-pp.security_file_permission 4.33 ± 2% -0.7 3.65 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64 2.18 ± 4% -0.5 1.64 ± 6% perf-profile.children.cycles-pp.mutex_unlock 7.92 -0.5 7.40 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair 8.10 -0.5 7.61 ± 3% perf-profile.children.cycles-pp.activate_task 8.16 -0.5 7.67 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate 4.12 ± 4% -0.5 3.64 ± 5% perf-profile.children.cycles-pp._raw_spin_lock 2.83 -0.4 2.38 ± 6% perf-profile.children.cycles-pp.selinux_file_permission 3.53 -0.4 3.13 ± 4% perf-profile.children.cycles-pp.enqueue_entity 1.99 ± 2% -0.4 1.64 ± 3% perf-profile.children.cycles-pp.copy_page_from_iter 3.34 -0.4 2.99 ± 4% perf-profile.children.cycles-pp.update_load_avg 2.70 -0.3 2.39 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter 2.66 -0.3 2.35 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.93 ± 3% -0.3 0.64 ± 7% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 1.41 ± 4% -0.3 1.13 perf-profile.children.cycles-pp.file_has_perm 2.75 -0.3 2.49 ± 2% perf-profile.children.cycles-pp.dequeue_entity 2.29 ± 2% -0.3 2.04 perf-profile.children.cycles-pp.mutex_lock 0.85 ± 3% -0.2 0.61 ± 4% perf-profile.children.cycles-pp.__inode_security_revalidate 0.59 ± 3% -0.2 0.39 ± 6% perf-profile.children.cycles-pp.__mutex_lock 1.79 ± 2% -0.2 1.59 ± 4% perf-profile.children.cycles-pp.__fdget_pos 1.70 ± 2% -0.2 1.52 ± 3% perf-profile.children.cycles-pp.__fget_light 0.93 ± 3% -0.2 0.75 perf-profile.children.cycles-pp.avc_has_perm 1.09 ± 2% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.copyin 2.33 ± 2% -0.2 2.16 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.96 ± 2% -0.2 0.81 ± 2% perf-profile.children.cycles-pp.___might_sleep 1.67 ± 2% -0.2 1.52 ± 5% perf-profile.children.cycles-pp.copyout 0.83 ± 4% -0.1 0.68 ± 2% perf-profile.children.cycles-pp._cond_resched 0.33 ± 5% -0.1 0.19 ± 10% perf-profile.children.cycles-pp.preempt_schedule_common 0.42 ± 2% -0.1 0.28 ± 8% perf-profile.children.cycles-pp.wake_up_q 0.70 ± 3% -0.1 0.59 ± 3% perf-profile.children.cycles-pp.__might_sleep 0.65 ± 5% -0.1 0.54 ± 7% perf-profile.children.cycles-pp.__fsnotify_parent 0.52 ± 2% -0.1 0.43 perf-profile.children.cycles-pp.__might_fault 0.53 ± 5% -0.1 0.45 perf-profile.children.cycles-pp.current_time 0.84 ± 3% -0.1 0.77 perf-profile.children.cycles-pp.fsnotify 0.16 ± 7% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.15 ± 8% -0.1 0.08 ± 15% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.22 ± 3% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.wake_q_add 0.49 -0.1 0.44 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.26 ± 4% -0.1 0.21 ± 2% perf-profile.children.cycles-pp.rcu_all_qs 0.08 ± 14% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_read 0.25 ± 4% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.finish_wait 0.38 -0.0 0.34 ± 2% perf-profile.children.cycles-pp.__x64_sys_read 0.43 ± 3% -0.0 0.39 ± 2% perf-profile.children.cycles-pp.prepare_to_wait 0.14 ± 20% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.task_tick_fair 0.16 ± 19% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.scheduler_tick 0.10 ± 8% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.18 ± 8% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.inode_has_perm 0.21 ± 2% -0.0 0.18 ± 6% perf-profile.children.cycles-pp.deactivate_task 0.20 ± 2% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.generic_pipe_buf_confirm 0.13 ± 3% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__sb_end_write 0.14 ± 9% -0.0 0.12 perf-profile.children.cycles-pp.timespec64_trunc 0.14 ± 3% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.native_irq_return_iret 0.11 ± 4% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.17 ± 4% -0.0 0.15 perf-profile.children.cycles-pp.iov_iter_init 0.11 ± 4% -0.0 0.09 ± 10% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.09 ± 4% -0.0 0.07 perf-profile.children.cycles-pp.__x2apic_send_IPI_dest 0.15 -0.0 0.13 ± 3% perf-profile.children.cycles-pp.rb_insert_color 0.18 ± 2% -0.0 0.17 ± 2% perf-profile.children.cycles-pp.set_next_buddy 0.08 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.interrupt_entry 0.23 ± 2% +0.0 0.24 perf-profile.children.cycles-pp.cpumask_next 0.67 +0.0 0.69 perf-profile.children.cycles-pp.__calc_delta 0.15 ± 3% +0.0 0.18 ± 6% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.17 ± 6% +0.1 0.23 ± 3% perf-profile.children.cycles-pp.remove_entity_load_avg 0.14 ± 6% +0.1 0.19 ± 7% perf-profile.children.cycles-pp.attach_entity_load_avg 0.80 ± 3% +0.1 0.86 perf-profile.children.cycles-pp.check_preempt_wakeup 0.37 ± 3% +0.1 0.45 ± 4% perf-profile.children.cycles-pp.account_entity_enqueue 0.79 ± 4% +0.1 0.88 ± 5% perf-profile.children.cycles-pp.finish_task_switch 0.80 ± 2% +0.1 0.89 ± 2% perf-profile.children.cycles-pp.put_prev_entity 0.54 ± 2% +0.1 0.65 perf-profile.children.cycles-pp.account_entity_dequeue 1.01 ± 5% +0.1 1.13 ± 5% perf-profile.children.cycles-pp.set_task_cpu 0.49 ± 7% +0.2 0.65 ± 5% perf-profile.children.cycles-pp.migrate_task_rq_fair 1.23 ± 4% +0.2 1.40 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 1.15 ± 2% +0.2 1.33 ± 2% perf-profile.children.cycles-pp.__switch_to 1.42 ± 3% +0.2 1.59 ± 2% perf-profile.children.cycles-pp.switch_fpu_return 0.76 ± 3% +0.2 0.96 ± 3% perf-profile.children.cycles-pp.pick_next_entity 2.21 +0.2 2.42 perf-profile.children.cycles-pp.reweight_entity 2.20 ± 2% +0.2 2.43 perf-profile.children.cycles-pp.load_new_mm_cr3 1.24 ± 5% +0.2 1.48 ± 4% perf-profile.children.cycles-pp.update_rq_clock 1.43 ± 5% +0.3 1.73 ± 2% perf-profile.children.cycles-pp.___perf_sw_event 0.38 ± 15% +0.3 0.68 ± 9% perf-profile.children.cycles-pp.find_next_bit 3.96 +0.3 4.28 perf-profile.children.cycles-pp.pick_next_task_fair 4.43 ± 2% +0.4 4.82 perf-profile.children.cycles-pp.switch_mm_irqs_off 1.11 +0.7 1.83 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.46 ± 26% +0.7 1.20 ± 12% perf-profile.children.cycles-pp.cpumask_next_wrap 87.76 +1.1 88.86 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 86.98 +1.1 88.10 perf-profile.children.cycles-pp.do_syscall_64 21.71 +1.3 23.02 perf-profile.children.cycles-pp.__schedule 5.98 ± 4% +1.4 7.34 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop 21.59 +1.4 22.99 perf-profile.children.cycles-pp.schedule 1.73 ± 29% +3.3 5.03 ± 13% perf-profile.children.cycles-pp.available_idle_cpu 35.55 +3.3 38.88 perf-profile.children.cycles-pp.ksys_write 34.31 +3.5 37.84 perf-profile.children.cycles-pp.vfs_write 31.14 +4.1 35.23 ± 2% perf-profile.children.cycles-pp.new_sync_write 30.56 +4.2 34.74 ± 2% perf-profile.children.cycles-pp.pipe_write 22.96 ± 2% +6.0 28.93 ± 3% perf-profile.children.cycles-pp.__wake_up_common_lock 20.11 ± 2% +6.1 26.19 ± 3% perf-profile.children.cycles-pp.try_to_wake_up 20.66 ± 2% +6.2 26.82 ± 3% perf-profile.children.cycles-pp.__wake_up_common 20.01 ± 2% +6.2 26.19 ± 3% perf-profile.children.cycles-pp.autoremove_wake_function 3.46 ± 20% +7.0 10.48 ± 12% perf-profile.children.cycles-pp.select_idle_sibling 4.92 ± 15% +7.1 12.01 ± 10% perf-profile.children.cycles-pp.select_task_rq_fair 11.85 ± 3% -2.1 9.72 ± 2% perf-profile.self.cycles-pp.do_syscall_64 4.13 ± 5% -1.3 2.85 ± 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 3.49 ± 6% -1.0 2.52 ± 9% perf-profile.self.cycles-pp.update_cfs_group 4.33 ± 2% -0.7 3.64 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64 2.12 ± 4% -0.5 1.59 ± 6% perf-profile.self.cycles-pp.mutex_unlock 2.61 -0.3 2.31 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.60 ± 2% -0.3 1.30 ± 5% perf-profile.self.cycles-pp.update_load_avg 1.48 ± 3% -0.2 1.25 ± 2% perf-profile.self.cycles-pp.mutex_lock 1.95 -0.2 1.75 ± 7% perf-profile.self.cycles-pp.selinux_file_permission 1.06 ± 5% -0.2 0.86 ± 5% perf-profile.self.cycles-pp.pipe_write 1.46 ± 2% -0.2 1.28 ± 3% perf-profile.self.cycles-pp.pipe_read 1.65 ± 2% -0.2 1.49 ± 4% perf-profile.self.cycles-pp.__fget_light 0.91 ± 3% -0.2 0.74 perf-profile.self.cycles-pp.avc_has_perm 0.94 ± 2% -0.2 0.79 ± 2% perf-profile.self.cycles-pp.___might_sleep 1.84 -0.1 1.72 ± 2% perf-profile.self.cycles-pp.update_curr 1.05 -0.1 0.93 perf-profile.self.cycles-pp.enqueue_task_fair 0.63 ± 3% -0.1 0.52 ± 2% perf-profile.self.cycles-pp.__might_sleep 0.60 ± 6% -0.1 0.51 ± 7% perf-profile.self.cycles-pp.__fsnotify_parent 0.29 ± 3% -0.1 0.19 ± 5% perf-profile.self.cycles-pp.__mutex_lock 0.38 ± 4% -0.1 0.29 ± 4% perf-profile.self.cycles-pp.vfs_write 0.51 -0.1 0.43 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter 0.49 -0.1 0.41 ± 2% perf-profile.self.cycles-pp.new_sync_write 0.35 ± 4% -0.1 0.28 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter 0.81 ± 3% -0.1 0.75 perf-profile.self.cycles-pp.fsnotify 0.26 -0.1 0.19 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate 0.20 ± 4% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.__mutex_unlock_slowpath 0.29 ± 5% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.file_has_perm 0.22 -0.1 0.16 ± 8% perf-profile.self.cycles-pp.wake_q_add 0.57 ± 2% -0.1 0.52 perf-profile.self.cycles-pp.new_sync_read 0.34 ± 2% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.dequeue_entity 0.26 ± 3% -0.1 0.21 ± 3% perf-profile.self.cycles-pp.ksys_read 0.28 ± 2% -0.0 0.23 perf-profile.self.cycles-pp.security_file_permission 0.22 ± 3% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.finish_wait 0.25 -0.0 0.21 ± 4% perf-profile.self.cycles-pp.ksys_write 0.88 -0.0 0.83 perf-profile.self.cycles-pp.__lock_text_start 0.66 -0.0 0.61 perf-profile.self.cycles-pp.vfs_read 1.37 -0.0 1.32 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.29 ± 4% -0.0 0.25 perf-profile.self.cycles-pp.current_time 0.20 ± 3% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.rcu_all_qs 0.10 ± 8% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.42 -0.0 0.39 ± 2% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.34 -0.0 0.31 ± 2% perf-profile.self.cycles-pp.__x64_sys_read 0.09 ± 5% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.wake_up_q 0.15 ± 8% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.inode_has_perm 0.14 ± 5% -0.0 0.11 ± 9% perf-profile.self.cycles-pp.rb_next 0.20 ± 2% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.deactivate_task 0.11 ± 4% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.__fdget_pos 0.24 ± 2% -0.0 0.22 ± 3% perf-profile.self.cycles-pp.prepare_to_wait 0.14 ± 3% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.native_irq_return_iret 0.13 ± 7% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.timespec64_trunc 0.06 -0.0 0.04 ± 57% perf-profile.self.cycles-pp.copyout 0.11 ± 4% -0.0 0.09 ± 10% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.17 ± 4% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.generic_pipe_buf_confirm 0.18 ± 2% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.__might_fault 0.17 ± 2% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.set_next_buddy 0.15 ± 2% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.rb_insert_color 0.12 ± 3% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.__sb_end_write 0.10 ± 8% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.06 -0.0 0.05 perf-profile.self.cycles-pp.kill_fasync 0.14 ± 3% +0.0 0.16 ± 5% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.66 +0.0 0.68 perf-profile.self.cycles-pp.__calc_delta 0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.native_load_tls 0.14 ± 3% +0.0 0.17 ± 4% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.67 +0.0 0.70 perf-profile.self.cycles-pp.dequeue_task_fair 0.23 ± 3% +0.0 0.27 ± 3% perf-profile.self.cycles-pp._cond_resched 0.17 ± 4% +0.0 0.21 ± 6% perf-profile.self.cycles-pp.activate_task 0.35 ± 3% +0.0 0.40 perf-profile.self.cycles-pp.pick_next_entity 1.20 +0.1 1.25 perf-profile.self.cycles-pp.select_task_rq_fair 0.13 ± 6% +0.1 0.18 ± 6% perf-profile.self.cycles-pp.attach_entity_load_avg 0.87 ± 3% +0.1 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair 0.27 ± 6% +0.1 0.36 ± 7% perf-profile.self.cycles-pp.migrate_task_rq_fair 0.32 ± 2% +0.1 0.41 ± 4% perf-profile.self.cycles-pp.account_entity_enqueue 0.42 +0.1 0.56 ± 2% perf-profile.self.cycles-pp.account_entity_dequeue 2.24 ± 3% +0.1 2.39 ± 2% perf-profile.self.cycles-pp.switch_mm_irqs_off 1.09 ± 2% +0.2 1.26 ± 2% perf-profile.self.cycles-pp.__switch_to 1.20 ± 4% +0.2 1.37 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 1.40 ± 3% +0.2 1.58 ± 2% perf-profile.self.cycles-pp.switch_fpu_return 0.89 ± 6% +0.2 1.10 ± 5% perf-profile.self.cycles-pp.update_rq_clock 2.19 ± 2% +0.2 2.42 perf-profile.self.cycles-pp.load_new_mm_cr3 1.33 ± 4% +0.3 1.60 ± 3% perf-profile.self.cycles-pp.___perf_sw_event 0.34 ± 14% +0.3 0.66 ± 9% perf-profile.self.cycles-pp.find_next_bit 0.28 ± 23% +0.4 0.72 ± 12% perf-profile.self.cycles-pp.cpumask_next_wrap 0.89 +0.7 1.58 ± 4% perf-profile.self.cycles-pp._raw_spin_lock 1.04 +0.7 1.76 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.65 ± 12% +3.0 3.61 ± 13% perf-profile.self.cycles-pp.select_idle_sibling 1.71 ± 29% +3.3 4.97 ± 13% perf-profile.self.cycles-pp.available_idle_cpu *************************************************************************************************** lkp-cfl-e1: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-2018-04-03.cgz/lkp-cfl-e1/hackbench/0xb8 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:2 -50% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x 1:2 -50% :4 dmesg.WARNING:stack_recursion %stddev %change %stddev \ | \ 49845 -6.8% 46466 hackbench.throughput 8.776e+08 +13.0% 9.915e+08 hackbench.time.involuntary_context_switches 93623 -6.3% 87707 hackbench.time.minor_page_faults 740.08 -1.4% 729.54 hackbench.time.user_time 1.569e+09 +3.5% 1.623e+09 hackbench.time.voluntary_context_switches 3.072e+08 -6.2% 2.88e+08 hackbench.workload 21.45 -1.7% 21.09 boot-time.boot 1747212 ± 3% -31.0% 1205072 ± 14% cpuidle.POLL.time 1289000 ± 4% -36.6% 817407 ± 13% cpuidle.POLL.usage 4038486 +6.3% 4293331 vmstat.system.cs 394711 -10.2% 354353 ± 2% vmstat.system.in 10059 -15.4% 8506 ± 5% softirqs.CPU0.SCHED 309881 -27.3% 225159 ± 4% softirqs.CPU2.TIMER 220105 +20.8% 265807 ± 16% softirqs.CPU6.TIMER 120937 ± 3% -17.6% 99667 ± 2% softirqs.SCHED 18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.active_objs 18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.num_objs 8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.active_objs 8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.num_objs 5280 -8.8% 4814 slabinfo.task_delay_info.active_objs 5280 -8.8% 4814 slabinfo.task_delay_info.num_objs 65295 -1.9% 64034 proc-vmstat.nr_active_anon 60605 -1.5% 59671 proc-vmstat.nr_anon_pages 12935 -0.9% 12820 proc-vmstat.nr_slab_unreclaimable 65295 -1.9% 64034 proc-vmstat.nr_zone_active_anon 24774010 -16.6% 20673098 proc-vmstat.numa_hit 24774010 -16.6% 20673098 proc-vmstat.numa_local 24839243 -16.5% 20735391 proc-vmstat.pgalloc_normal 673024 -2.4% 657142 proc-vmstat.pgfault 24822112 -16.5% 20717243 proc-vmstat.pgfree 247.95 ± 14% -19.1% 200.59 ± 4% sched_debug.cfs_rq:/.load_avg.max 43.55 ± 3% -19.9% 34.86 ± 6% sched_debug.cfs_rq:/.load_avg.min 55.51 ± 6% -13.2% 48.19 ± 7% sched_debug.cfs_rq:/.load_avg.stddev 20116 ± 8% -12.8% 17531 ± 5% sched_debug.cfs_rq:/.runnable_weight.min 1288 +7.5% 1384 ± 3% sched_debug.cfs_rq:/.util_avg.max 54763 ± 9% +130.8% 126390 ± 25% sched_debug.cpu.avg_idle.avg 301502 ± 18% +84.9% 557406 ± 14% sched_debug.cpu.avg_idle.max 82233 ± 22% +122.7% 183153 ± 17% sched_debug.cpu.avg_idle.stddev 0.00 ± 12% -12.0% 0.00 ± 8% sched_debug.cpu.next_balance.stddev 16.92 ± 4% -7.5% 15.66 ± 3% sched_debug.cpu.nr_running.avg 2.92 ± 10% -20.9% 2.31 ± 16% sched_debug.cpu.nr_uninterruptible.avg 2977 ± 50% -73.0% 803.75 ± 43% interrupts.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1 546.00 ± 23% +119.5% 1198 ± 16% interrupts.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3 13831842 -11.5% 12245495 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts 45259 ± 5% +68.9% 76440 ± 15% interrupts.CPU0.TRM:Thermal_event_interrupts 2977 ± 50% -73.0% 803.75 ± 43% interrupts.CPU1.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1 45283 ± 5% +68.9% 76474 ± 15% interrupts.CPU1.TRM:Thermal_event_interrupts 13947887 -11.0% 12413074 interrupts.CPU10.RES:Rescheduling_interrupts 45275 ± 5% +68.9% 76468 ± 15% interrupts.CPU10.TRM:Thermal_event_interrupts 45286 ± 5% +68.9% 76468 ± 15% interrupts.CPU11.TRM:Thermal_event_interrupts 13579741 -11.8% 11974820 ± 5% interrupts.CPU12.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU12.TRM:Thermal_event_interrupts 13668073 -11.3% 12119346 interrupts.CPU13.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76474 ± 15% interrupts.CPU13.TRM:Thermal_event_interrupts 13578574 -7.8% 12518854 interrupts.CPU14.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76467 ± 15% interrupts.CPU14.TRM:Thermal_event_interrupts 13689117 -10.1% 12310290 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76473 ± 15% interrupts.CPU15.TRM:Thermal_event_interrupts 13924470 -10.5% 12461613 interrupts.CPU2.RES:Rescheduling_interrupts 45264 ± 5% +68.9% 76467 ± 15% interrupts.CPU2.TRM:Thermal_event_interrupts 546.00 ± 23% +119.5% 1198 ± 16% interrupts.CPU3.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3 13951884 -10.5% 12480739 interrupts.CPU3.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76469 ± 15% interrupts.CPU3.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76471 ± 15% interrupts.CPU4.TRM:Thermal_event_interrupts 14030690 -10.7% 12530992 ± 3% interrupts.CPU5.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU5.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76474 ± 15% interrupts.CPU6.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76472 ± 15% interrupts.CPU7.TRM:Thermal_event_interrupts 13319036 -10.2% 11966842 ± 2% interrupts.CPU8.RES:Rescheduling_interrupts 45280 ± 5% +68.7% 76405 ± 15% interrupts.CPU8.TRM:Thermal_event_interrupts 13401328 -11.8% 11816711 ± 3% interrupts.CPU9.RES:Rescheduling_interrupts 45279 ± 5% +68.9% 76467 ± 15% interrupts.CPU9.TRM:Thermal_event_interrupts 2.19e+08 -9.8% 1.974e+08 ± 2% interrupts.RES:Rescheduling_interrupts 724498 ± 5% +68.9% 1223435 ± 15% interrupts.TRM:Thermal_event_interrupts 53.44 +4.2% 55.68 perf-stat.i.MPKI 0.12 -0.0 0.10 ± 4% perf-stat.i.cache-miss-rate% 1764490 -13.1% 1533362 ± 3% perf-stat.i.cache-misses 1.767e+09 +3.6% 1.832e+09 perf-stat.i.cache-references 4055785 +6.2% 4306694 perf-stat.i.context-switches 336154 +21.5% 408363 perf-stat.i.cpu-migrations 34528 ± 5% +10.3% 38089 ± 3% perf-stat.i.cycles-between-cache-misses 0.01 ± 6% -0.0 0.01 ± 6% perf-stat.i.dTLB-load-miss-rate% 899577 ± 6% -15.2% 762553 ± 5% perf-stat.i.dTLB-load-misses 0.00 ± 7% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate% 22816 ± 7% -13.2% 19813 ± 4% perf-stat.i.dTLB-store-misses 6.044e+09 -1.2% 5.969e+09 perf-stat.i.dTLB-stores 31331199 -3.4% 30272777 perf-stat.i.iTLB-load-misses 1061 +3.1% 1094 perf-stat.i.instructions-per-iTLB-miss 1077 -2.9% 1046 perf-stat.i.minor-faults 73018 -10.9% 65028 ± 2% perf-stat.i.node-loads 122632 ± 2% -9.3% 111181 ± 4% perf-stat.i.node-stores 1077 -2.9% 1046 perf-stat.i.page-faults 53.29 +4.2% 55.53 perf-stat.overall.MPKI 0.10 -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate% 25264 +15.3% 29120 ± 3% perf-stat.overall.cycles-between-cache-misses 0.01 ± 6% -0.0 0.01 ± 6% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 7% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 1058 +2.9% 1089 perf-stat.overall.instructions-per-iTLB-miss 65158 +6.6% 69487 perf-stat.overall.path-length 1761814 -13.1% 1530883 ± 3% perf-stat.ps.cache-misses 1.765e+09 +3.7% 1.829e+09 perf-stat.ps.cache-references 4048770 +6.2% 4299578 perf-stat.ps.context-switches 335565 +21.5% 407697 perf-stat.ps.cpu-migrations 898100 ± 6% -15.2% 761293 ± 5% perf-stat.ps.dTLB-load-misses 22777 ± 7% -13.2% 19781 ± 4% perf-stat.ps.dTLB-store-misses 6.034e+09 -1.2% 5.96e+09 perf-stat.ps.dTLB-stores 31277565 -3.4% 30222833 perf-stat.ps.iTLB-load-misses 1075 -2.9% 1044 perf-stat.ps.minor-faults 72896 -10.9% 64922 ± 2% perf-stat.ps.node-loads 122424 ± 2% -9.3% 111005 ± 4% perf-stat.ps.node-stores 1075 -2.9% 1044 perf-stat.ps.page-faults 39.37 ±100% -36.4 2.98 ±173% perf-profile.calltrace.cycles-pp.start_thread 21.68 ±100% -20.0 1.64 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread 20.14 ±100% -18.6 1.52 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 20.00 ±100% -18.5 1.51 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 18.05 ±100% -16.7 1.39 ±173% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 16.75 ±100% -15.5 1.22 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread 16.70 ±100% -15.4 1.26 ±173% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 14.98 ±100% -13.9 1.09 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 14.82 ±100% -13.7 1.08 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 13.95 ±100% -12.9 1.03 ±173% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 12.97 ±100% -12.0 0.95 ±173% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 3.69 ± 6% -1.1 2.58 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.25 ± 6% -1.0 3.29 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.54 ± 10% -0.5 1.05 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 1.07 ± 16% -0.5 0.60 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write 1.14 ± 2% -0.3 0.80 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_wait 0.55 ± 4% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read 1.90 ± 9% -0.2 1.67 ± 7% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.71 ± 7% -0.2 1.48 ± 7% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64 0.68 ± 19% +0.2 0.86 perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.67 ± 16% +0.2 0.88 ± 4% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait 0.29 ±100% +0.4 0.64 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 0.31 ±100% +0.4 0.67 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.32 ±100% +0.4 0.68 ± 2% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.29 ±100% +0.4 0.68 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.35 ±100% +0.4 0.78 perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 1.23 ± 19% +0.4 1.68 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.29 ± 18% +0.5 1.76 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.update_load_avg.set_next_entity.pick_next_task_fair.__schedule.schedule 0.00 +0.6 0.59 ± 7% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule 8.82 +0.6 9.46 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 0.47 ±100% +0.7 1.14 ± 6% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 2.21 ± 10% +0.7 2.88 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 9.10 +0.7 9.79 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 9.04 +0.7 9.74 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.71 ±100% +0.9 1.61 ± 3% perf-profile.calltrace.cycles-pp.__switch_to 0.68 ± 99% +0.9 1.60 ± 3% perf-profile.calltrace.cycles-pp.native_write_msr 0.86 ±100% +0.9 1.79 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 6.54 +1.0 7.53 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 26.96 +1.0 27.95 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 1.42 ± 63% +1.0 2.47 ± 3% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.54 ± 8% +1.2 5.79 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.99 +1.3 16.27 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 15.35 +1.3 16.66 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 27.37 +1.3 28.70 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 17.81 +1.6 19.38 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 3.95 ± 11% +1.8 5.70 ± 6% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 23.38 +2.0 25.33 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 24.30 +2.0 26.27 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 23.73 +2.0 25.71 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 5.24 ± 10% +2.0 7.25 ± 6% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 3.23 ± 57% +2.7 5.98 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.34 ± 57% +2.8 6.18 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 39.37 ±100% -36.4 2.98 ±173% perf-profile.children.cycles-pp.start_thread 21.68 ±100% -20.0 1.67 ±173% perf-profile.children.cycles-pp.__GI___libc_write 16.99 ±100% -15.7 1.24 ±173% perf-profile.children.cycles-pp.__GI___libc_read 6.84 ± 10% -2.1 4.71 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 45.35 -1.0 44.33 perf-profile.children.cycles-pp.ksys_write 6.70 ± 5% -1.0 5.70 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 5.90 ± 4% -0.6 5.34 ± 6% perf-profile.children.cycles-pp.security_file_permission 3.35 ± 9% -0.6 2.80 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 42.14 -0.5 41.63 perf-profile.children.cycles-pp.vfs_write 3.41 -0.4 3.01 perf-profile.children.cycles-pp.__fdget_pos 3.30 -0.4 2.91 perf-profile.children.cycles-pp.__fget_light 2.71 ± 2% -0.4 2.32 perf-profile.children.cycles-pp.__fget 3.23 -0.3 2.93 ± 3% perf-profile.children.cycles-pp.mutex_lock 2.76 -0.3 2.47 ± 5% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 2.21 ± 3% -0.3 1.93 ± 10% perf-profile.children.cycles-pp.file_has_perm 3.14 -0.2 2.90 ± 4% perf-profile.children.cycles-pp.copy_page_to_iter 3.03 ± 5% -0.2 2.79 ± 5% perf-profile.children.cycles-pp.selinux_file_permission 2.10 ± 5% -0.2 1.86 ± 2% perf-profile.children.cycles-pp.mutex_unlock 1.50 ± 5% -0.2 1.27 ± 2% perf-profile.children.cycles-pp.fput_many 0.68 ± 17% -0.2 0.48 ± 4% perf-profile.children.cycles-pp.__mutex_lock 1.66 ± 2% -0.2 1.47 ± 4% perf-profile.children.cycles-pp.copyout 1.73 -0.2 1.57 ± 2% perf-profile.children.cycles-pp.update_rq_clock 1.30 -0.1 1.17 ± 6% perf-profile.children.cycles-pp.copyin 0.27 ± 11% -0.1 0.15 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common 1.25 ± 3% -0.1 1.15 ± 3% perf-profile.children.cycles-pp.fsnotify 0.79 -0.1 0.69 ± 7% perf-profile.children.cycles-pp.reschedule_interrupt 0.91 -0.1 0.82 ± 5% perf-profile.children.cycles-pp._cond_resched 0.80 ± 2% -0.1 0.71 ± 3% perf-profile.children.cycles-pp.file_update_time 0.29 ± 10% -0.1 0.24 ± 5% perf-profile.children.cycles-pp.rb_insert_color 0.26 -0.0 0.21 ± 2% perf-profile.children.cycles-pp.native_irq_return_iret 0.07 ± 14% -0.0 0.03 ±100% perf-profile.children.cycles-pp.scheduler_ipi 0.22 ± 6% -0.0 0.18 ± 4% perf-profile.children.cycles-pp.__x64_sys_read 0.25 ± 4% -0.0 0.21 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.27 -0.0 0.23 ± 9% perf-profile.children.cycles-pp.timespec64_trunc 0.23 ± 2% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.22 -0.0 0.20 ± 5% perf-profile.children.cycles-pp.__x64_sys_write 0.24 -0.0 0.22 ± 3% perf-profile.children.cycles-pp.__list_add_valid 0.14 -0.0 0.12 ± 10% perf-profile.children.cycles-pp.kill_fasync 0.11 ± 4% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.perf_exclude_event 0.10 +0.0 0.12 ± 8% perf-profile.children.cycles-pp.is_cpu_allowed 0.09 ± 11% +0.0 0.12 ± 7% perf-profile.children.cycles-pp.rcu_note_context_switch 0.07 +0.0 0.10 ± 10% perf-profile.children.cycles-pp.pkg_thermal_notify 0.17 ± 3% +0.0 0.20 ± 9% perf-profile.children.cycles-pp.deactivate_task 0.16 +0.0 0.21 ± 10% perf-profile.children.cycles-pp.update_cfs_rq_h_load 0.08 ± 6% +0.0 0.12 ± 10% perf-profile.children.cycles-pp.intel_thermal_interrupt 0.08 ± 6% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.smp_thermal_interrupt 0.08 +0.1 0.13 ± 9% perf-profile.children.cycles-pp.thermal_interrupt 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.default_wake_function 0.19 ± 5% +0.1 0.25 ± 5% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.19 ± 5% +0.1 0.27 ± 4% perf-profile.children.cycles-pp.finish_wait 0.89 ± 8% +0.1 0.98 perf-profile.children.cycles-pp.__calc_delta 0.63 ± 4% +0.1 0.74 ± 4% perf-profile.children.cycles-pp.native_sched_clock 0.33 ± 4% +0.1 0.44 ± 5% perf-profile.children.cycles-pp.set_next_buddy 1.23 ± 3% +0.1 1.34 ± 3% perf-profile.children.cycles-pp.check_preempt_wakeup 0.69 ± 4% +0.1 0.81 ± 3% perf-profile.children.cycles-pp.sched_clock 1.52 ± 3% +0.1 1.65 ± 2% perf-profile.children.cycles-pp.set_next_entity 0.73 ± 4% +0.1 0.88 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.57 ± 4% +0.2 0.74 ± 4% perf-profile.children.cycles-pp.account_entity_dequeue 1.21 ± 4% +0.2 1.39 ± 2% perf-profile.children.cycles-pp.__update_load_avg_se 1.99 ± 3% +0.2 2.20 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 1.64 ± 6% +0.2 1.85 perf-profile.children.cycles-pp.__switch_to_asm 1.60 ± 4% +0.2 1.85 ± 3% perf-profile.children.cycles-pp.update_cfs_group 3.91 +0.2 4.15 perf-profile.children.cycles-pp.update_curr 1.04 ± 5% +0.3 1.30 ± 4% perf-profile.children.cycles-pp.___perf_sw_event 1.05 ± 9% +0.3 1.33 ± 2% perf-profile.children.cycles-pp.pick_next_entity 1.57 ± 3% +0.3 1.89 perf-profile.children.cycles-pp.native_write_msr 1.01 ± 9% +0.3 1.33 ± 5% perf-profile.children.cycles-pp.put_prev_entity 2.17 ± 7% +0.3 2.51 ± 2% perf-profile.children.cycles-pp.switch_fpu_return 1.93 ± 7% +0.4 2.36 perf-profile.children.cycles-pp.__switch_to 2.99 ± 4% +0.6 3.57 ± 2% perf-profile.children.cycles-pp.reweight_entity 8.89 +0.6 9.50 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair 9.16 +0.7 9.82 perf-profile.children.cycles-pp.ttwu_do_activate 9.09 +0.7 9.76 ± 2% perf-profile.children.cycles-pp.activate_task 5.18 ± 6% +0.9 6.04 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair 6.71 +0.9 7.63 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair 28.01 +1.4 29.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common_lock 4.91 ± 7% +1.4 6.28 ± 4% perf-profile.children.cycles-pp.exit_to_usermode_loop 18.19 +1.4 19.56 perf-profile.children.cycles-pp.pipe_wait 4.03 ± 11% +1.7 5.75 ± 6% perf-profile.children.cycles-pp.select_idle_sibling 23.86 ± 2% +1.8 25.71 ± 2% perf-profile.children.cycles-pp.try_to_wake_up 24.44 +1.9 26.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common 23.84 +1.9 25.77 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function 5.29 ± 10% +2.0 7.29 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair 20.27 ± 2% +2.3 22.53 ± 2% perf-profile.children.cycles-pp.__schedule 20.36 ± 2% +2.5 22.82 ± 2% perf-profile.children.cycles-pp.schedule 6.82 ± 10% -2.1 4.69 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 2.67 ± 2% -0.4 2.29 perf-profile.self.cycles-pp.__fget 2.72 -0.3 2.44 ± 5% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.22 -0.2 0.98 ± 2% perf-profile.self.cycles-pp.update_rq_clock 1.36 ± 3% -0.2 1.13 ± 6% perf-profile.self.cycles-pp.pipe_write 1.48 ± 5% -0.2 1.24 ± 3% perf-profile.self.cycles-pp.fput_many 2.04 ± 5% -0.2 1.81 ± 2% perf-profile.self.cycles-pp.mutex_unlock 1.96 -0.2 1.73 ± 2% perf-profile.self.cycles-pp.mutex_lock 0.66 ± 12% -0.1 0.53 ± 10% perf-profile.self.cycles-pp.file_has_perm 0.38 ± 10% -0.1 0.28 ± 9% perf-profile.self.cycles-pp.ksys_read 0.34 ± 7% -0.1 0.25 ± 7% perf-profile.self.cycles-pp.__mutex_lock 1.21 ± 3% -0.1 1.11 ± 2% perf-profile.self.cycles-pp.fsnotify 1.00 -0.1 0.94 ± 5% perf-profile.self.cycles-pp.__might_sleep 0.60 ± 4% -0.1 0.55 ± 8% perf-profile.self.cycles-pp.vfs_write 0.29 ± 10% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.rb_insert_color 0.32 ± 6% -0.0 0.28 ± 6% perf-profile.self.cycles-pp.ksys_write 0.26 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret 0.07 ± 14% -0.0 0.03 ±100% perf-profile.self.cycles-pp.scheduler_ipi 0.36 ± 5% -0.0 0.32 ± 8% perf-profile.self.cycles-pp.__sb_start_write 0.29 -0.0 0.24 ± 4% perf-profile.self.cycles-pp.check_preempt_curr 0.22 ± 6% -0.0 0.17 ± 6% perf-profile.self.cycles-pp.__x64_sys_read 0.25 -0.0 0.21 ± 8% perf-profile.self.cycles-pp.timespec64_trunc 0.22 ± 4% -0.0 0.19 ± 11% perf-profile.self.cycles-pp.wake_q_add 0.21 ± 2% -0.0 0.18 ± 6% perf-profile.self.cycles-pp.__x64_sys_write 0.21 ± 2% -0.0 0.20 perf-profile.self.cycles-pp.__list_add_valid 0.11 ± 4% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.kill_fasync 0.06 ± 9% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.rcu_note_context_switch 0.15 +0.0 0.18 ± 6% perf-profile.self.cycles-pp.autoremove_wake_function 0.16 +0.0 0.19 ± 9% perf-profile.self.cycles-pp.deactivate_task 0.26 +0.0 0.29 ± 5% perf-profile.self.cycles-pp.rcu_all_qs 0.33 ± 3% +0.0 0.37 ± 4% perf-profile.self.cycles-pp.set_next_entity 0.18 ± 5% +0.0 0.22 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.03 ±100% +0.0 0.07 perf-profile.self.cycles-pp.check_cfs_rq_runtime 0.16 ± 6% +0.0 0.20 ± 6% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.16 +0.0 0.20 ± 12% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.20 ± 2% +0.1 0.25 ± 2% perf-profile.self.cycles-pp.activate_task 0.41 ± 6% +0.1 0.46 perf-profile.self.cycles-pp.schedule 1.96 +0.1 2.02 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.15 ± 6% +0.1 0.22 perf-profile.self.cycles-pp.finish_wait 0.41 ± 4% +0.1 0.48 ± 6% perf-profile.self.cycles-pp.account_entity_enqueue 0.61 ± 4% +0.1 0.71 ± 3% perf-profile.self.cycles-pp.native_sched_clock 0.83 ± 2% +0.1 0.93 perf-profile.self.cycles-pp.dequeue_task_fair 0.29 ± 3% +0.1 0.40 ± 5% perf-profile.self.cycles-pp.set_next_buddy 1.02 +0.1 1.15 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair 0.46 ± 12% +0.2 0.61 ± 3% perf-profile.self.cycles-pp.pick_next_entity 1.17 ± 4% +0.2 1.35 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se 0.42 ± 8% +0.2 0.61 ± 5% perf-profile.self.cycles-pp.account_entity_dequeue 1.64 ± 6% +0.2 1.85 perf-profile.self.cycles-pp.__switch_to_asm 1.23 ± 4% +0.2 1.46 ± 3% perf-profile.self.cycles-pp.reweight_entity 1.07 ± 5% +0.2 1.31 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair 0.90 ± 5% +0.2 1.14 ± 5% perf-profile.self.cycles-pp.___perf_sw_event 1.57 ± 4% +0.2 1.82 ± 2% perf-profile.self.cycles-pp.update_cfs_group 1.17 ± 3% +0.3 1.42 ± 7% perf-profile.self.cycles-pp.update_load_avg 1.56 ± 4% +0.3 1.88 perf-profile.self.cycles-pp.native_write_msr 2.16 ± 7% +0.3 2.50 ± 2% perf-profile.self.cycles-pp.switch_fpu_return 1.84 ± 8% +0.4 2.25 ± 2% perf-profile.self.cycles-pp.__switch_to 1.22 +0.5 1.73 ± 4% perf-profile.self.cycles-pp._raw_spin_lock 0.92 ± 11% +1.3 2.27 ± 6% perf-profile.self.cycles-pp.select_idle_sibling Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 6 months

4
6
0 / 0

[btrfs] c75e839414: aim7.jobs-per-min -9.1% regression

by kernel test robot

Greeting, FYI, we noticed a -9.1% regression of aim7.jobs-per-min due to commit: commit: c75e839414d3610e6487ae3145199c500d55f7f7 ("btrfs: kill the subvol_srcu") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: aim7 on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: disk: 4BRD_12G md: RAID0 fs: btrfs test: disk_wrt load: 1500 cpufreq_governor: performance ucode: 0x500002c test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/4BRD_12G/btrfs/x86_64-rhel-7.6/1500/RAID0/debian-x86_64-20191114.cgz/lkp-csl-2ap2/disk_wrt/aim7/0x500002c commit: efc3453494 ("btrfs: make btrfs_cleanup_fs_roots use the radix tree lock") c75e839414 ("btrfs: kill the subvol_srcu") efc3453494af7818 c75e839414d3610e6487ae31451 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 3:9 -33% :8 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x %stddev %change %stddev \ | \ 29509 ± 2% -9.1% 26837 ± 2% aim7.jobs-per-min 305.28 ± 2% +10.0% 335.72 ± 2% aim7.time.elapsed_time 305.28 ± 2% +10.0% 335.72 ± 2% aim7.time.elapsed_time.max 4883135 ± 10% +37.9% 6735464 ± 7% aim7.time.involuntary_context_switches 56288 ± 2% +10.5% 62202 ± 2% aim7.time.system_time 2344783 +6.5% 2497364 ± 2% aim7.time.voluntary_context_switches 62337721 ± 2% +9.8% 68456490 ± 2% turbostat.IRQ 431.56 ± 6% +22.3% 527.88 ± 4% vmstat.procs.r 27340 ± 2% +11.2% 30397 ± 2% vmstat.system.cs 226804 ± 6% +21.7% 276057 ± 4% meminfo.Active(file) 221309 ± 6% +22.3% 270668 ± 4% meminfo.Dirty 720.89 ±111% +49.3% 1076 ± 73% meminfo.Mlocked 14278 ± 2% -8.3% 13094 ± 2% meminfo.max_used_kB 57228 ± 6% +22.7% 70195 ± 5% numa-meminfo.node0.Active(file) 55433 ± 6% +21.6% 67431 ± 4% numa-meminfo.node0.Dirty 56152 ± 6% +21.4% 68180 ± 5% numa-meminfo.node1.Active(file) 55001 ± 6% +22.5% 67397 ± 4% numa-meminfo.node1.Dirty 56373 ± 6% +21.7% 68594 ± 4% numa-meminfo.node2.Active(file) 55222 ± 7% +22.6% 67726 ± 4% numa-meminfo.node2.Dirty 56671 ± 6% +20.5% 68317 ± 3% numa-meminfo.node3.Active(file) 55285 ± 6% +21.8% 67355 ± 4% numa-meminfo.node3.Dirty 56694 ± 6% +21.7% 69019 ± 4% proc-vmstat.nr_active_file 55342 ± 6% +22.3% 67662 ± 4% proc-vmstat.nr_dirty 402316 +2.1% 410951 proc-vmstat.nr_file_pages 180.22 ±111% +49.4% 269.25 ± 73% proc-vmstat.nr_mlock 56694 ± 6% +21.7% 69019 ± 4% proc-vmstat.nr_zone_active_file 54680 ± 6% +22.8% 67168 ± 4% proc-vmstat.nr_zone_write_pending 3144381 ± 2% +6.1% 3335275 proc-vmstat.pgactivate 1387558 ± 2% +7.9% 1496754 ± 2% proc-vmstat.pgfault 983.33 ± 4% +5.4% 1036 proc-vmstat.unevictable_pgs_culled 14331 ± 6% +22.6% 17566 ± 5% numa-vmstat.node0.nr_active_file 13884 ± 6% +21.6% 16884 ± 4% numa-vmstat.node0.nr_dirty 14330 ± 6% +22.6% 17566 ± 5% numa-vmstat.node0.nr_zone_active_file 13714 ± 6% +22.2% 16755 ± 4% numa-vmstat.node0.nr_zone_write_pending 14047 ± 6% +21.3% 17043 ± 4% numa-vmstat.node1.nr_active_file 13763 ± 6% +22.3% 16838 ± 4% numa-vmstat.node1.nr_dirty 14047 ± 6% +21.3% 17043 ± 4% numa-vmstat.node1.nr_zone_active_file 13599 ± 6% +23.0% 16726 ± 4% numa-vmstat.node1.nr_zone_write_pending 14074 ± 5% +21.7% 17130 ± 4% numa-vmstat.node2.nr_active_file 13786 ± 6% +22.6% 16906 ± 4% numa-vmstat.node2.nr_dirty 14074 ± 5% +21.7% 17130 ± 4% numa-vmstat.node2.nr_zone_active_file 13617 ± 7% +23.3% 16789 ± 4% numa-vmstat.node2.nr_zone_write_pending 14151 ± 6% +20.7% 17074 ± 3% numa-vmstat.node3.nr_active_file 13790 ± 6% +22.1% 16832 ± 4% numa-vmstat.node3.nr_dirty 14152 ± 6% +20.7% 17074 ± 3% numa-vmstat.node3.nr_zone_active_file 13627 ± 6% +22.5% 16696 ± 4% numa-vmstat.node3.nr_zone_write_pending 828631 ± 16% +49.9% 1241934 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 1.33 ± 30% +96.9% 2.61 ± 14% sched_debug.cfs_rq:/.nr_spread_over.avg 831152 ± 16% +49.7% 1244586 ± 8% sched_debug.cfs_rq:/.spread0.stddev 1.92 ± 14% +28.4% 2.47 ± 8% sched_debug.cpu.nr_running.avg 5.00 ± 9% +29.7% 6.48 ± 9% sched_debug.cpu.nr_running.max 0.90 ± 9% +16.0% 1.05 ± 3% sched_debug.cpu.nr_running.stddev 21790 ± 11% +16.7% 25440 ± 2% sched_debug.cpu.nr_switches.avg 19761 ± 11% +17.9% 23289 ± 2% sched_debug.cpu.nr_switches.min 20403 ± 12% +18.1% 24101 ± 2% sched_debug.cpu.sched_count.avg 27825 ± 10% +15.8% 32227 ± 4% sched_debug.cpu.sched_count.max 18995 ± 13% +18.3% 22463 ± 2% sched_debug.cpu.sched_count.min 1143 ± 11% +15.6% 1322 ± 4% sched_debug.cpu.sched_count.stddev 1404 ± 7% -31.0% 969.66 ± 11% sched_debug.cpu.sched_goidle.avg 1186 ± 8% -36.0% 759.25 ± 14% sched_debug.cpu.sched_goidle.min 16821 ± 14% +20.6% 20284 ± 6% sched_debug.cpu.ttwu_count.max 2790 ± 11% +21.7% 3395 ± 7% sched_debug.cpu.ttwu_count.stddev 398.19 ± 16% +28.3% 510.73 ± 10% sched_debug.cpu.ttwu_local.stddev 43238516 ± 5% +19.0% 51459028 ± 4% perf-stat.i.cache-misses 83870573 ± 4% +13.8% 95436555 ± 4% perf-stat.i.cache-references 27181 ± 2% +11.1% 30198 ± 2% perf-stat.i.context-switches 3008 ± 2% -7.1% 2794 perf-stat.i.cpu-migrations 12612 ± 5% -15.6% 10641 ± 3% perf-stat.i.cycles-between-cache-misses 1.1e+09 ± 2% -7.3% 1.019e+09 ± 2% perf-stat.i.dTLB-stores 10684629 ± 2% +10.8% 11842781 ± 2% perf-stat.i.node-load-misses 641429 ± 2% +24.4% 798190 ± 2% perf-stat.i.node-loads 5915955 ± 2% -6.3% 5543368 ± 2% perf-stat.i.node-store-misses 380971 -6.7% 355629 ± 2% perf-stat.i.node-stores 1.66 ± 4% +14.4% 1.90 ± 3% perf-stat.overall.MPKI 13064 ± 5% -15.8% 10999 ± 4% perf-stat.overall.cycles-between-cache-misses 43484558 ± 5% +19.2% 51837476 ± 4% perf-stat.ps.cache-misses 83278267 ± 4% +14.4% 95253129 ± 3% perf-stat.ps.cache-references 27414 ± 2% +11.2% 30486 ± 2% perf-stat.ps.context-switches 3028 ± 2% -7.0% 2817 perf-stat.ps.cpu-migrations 1.104e+09 ± 2% -7.3% 1.023e+09 ± 2% perf-stat.ps.dTLB-stores 4394 -1.8% 4315 perf-stat.ps.minor-faults 10766873 ± 2% +11.0% 11947268 ± 2% perf-stat.ps.node-load-misses 645940 ± 2% +24.7% 805392 ± 2% perf-stat.ps.node-loads 5965618 ± 2% -6.2% 5595562 ± 2% perf-stat.ps.node-store-misses 383092 -6.5% 358124 ± 2% perf-stat.ps.node-stores 4395 -1.8% 4315 perf-stat.ps.page-faults 1.538e+13 ± 2% +10.0% 1.692e+13 ± 2% perf-stat.total.instructions 27.39 -0.1 27.28 perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 35.13 +0.1 35.23 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 35.23 +0.1 35.34 perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 35.48 +0.1 35.60 perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 35.08 +0.1 35.22 perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter 34.97 +0.1 35.12 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write 35.57 +0.1 35.72 perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 35.10 +0.2 35.26 perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 35.11 +0.2 35.27 perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 98.71 +0.2 98.88 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.69 +0.2 98.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.74 +0.2 98.91 perf-profile.calltrace.cycles-pp.write 98.71 +0.2 98.89 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 98.68 +0.2 98.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.64 +0.2 98.82 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 98.63 +0.2 98.81 perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 98.58 +0.2 98.77 perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 27.39 -0.1 27.28 perf-profile.children.cycles-pp.btrfs_dirty_pages 0.56 ± 5% -0.1 0.46 ± 9% perf-profile.children.cycles-pp.btrfs_get_extent 0.51 ± 5% -0.1 0.41 ± 9% perf-profile.children.cycles-pp.btrfs_lookup_file_extent 0.52 ± 5% -0.1 0.43 ± 9% perf-profile.children.cycles-pp.btrfs_search_slot 0.44 ± 6% -0.1 0.35 ± 11% perf-profile.children.cycles-pp.btrfs_read_lock_root_node 0.43 ± 6% -0.1 0.34 ± 11% perf-profile.children.cycles-pp.btrfs_tree_read_lock 0.19 ± 15% -0.1 0.11 ± 27% perf-profile.children.cycles-pp.start_secondary 0.19 ± 16% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.cpuidle_enter 0.19 ± 16% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.cpuidle_enter_state 0.41 ± 7% -0.1 0.33 ± 8% perf-profile.children.cycles-pp.rwsem_optimistic_spin 0.46 ± 7% -0.1 0.38 ± 8% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.secondary_startup_64 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.cpu_startup_entry 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.do_idle 0.18 ± 15% -0.1 0.10 ± 24% perf-profile.children.cycles-pp.intel_idle 0.29 ± 9% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.osq_lock 0.19 ± 6% -0.0 0.15 ± 16% perf-profile.children.cycles-pp.finish_wait 0.39 ± 5% -0.0 0.35 ± 6% perf-profile.children.cycles-pp.do_unlinkat 0.39 ± 6% -0.0 0.35 ± 7% perf-profile.children.cycles-pp.unlink 0.26 ± 5% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_sys_open 0.26 ± 5% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_sys_openat2 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_filp_open 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.path_openat 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.creat 0.00 +0.0 0.05 perf-profile.children.cycles-pp.btrfs_calculate_inode_block_rsv_size 99.65 +0.1 99.73 perf-profile.children.cycles-pp.do_syscall_64 99.66 +0.1 99.74 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 35.62 +0.1 35.74 perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 35.57 +0.1 35.72 perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 97.55 +0.2 97.72 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 98.74 +0.2 98.92 perf-profile.children.cycles-pp.write 98.70 +0.2 98.88 perf-profile.children.cycles-pp.ksys_write 98.69 +0.2 98.87 perf-profile.children.cycles-pp.vfs_write 98.65 +0.2 98.83 perf-profile.children.cycles-pp.new_sync_write 98.63 +0.2 98.81 perf-profile.children.cycles-pp.btrfs_file_write_iter 98.59 +0.2 98.78 perf-profile.children.cycles-pp.btrfs_buffered_write 97.60 +0.2 97.79 perf-profile.children.cycles-pp._raw_spin_lock 0.18 ± 15% -0.1 0.10 ± 24% perf-profile.self.cycles-pp.intel_idle 0.28 ± 9% -0.1 0.21 ± 12% perf-profile.self.cycles-pp.osq_lock 0.45 ± 2% -0.0 0.41 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 135788 ± 2% +13.1% 153639 ± 3% softirqs.CPU0.RCU 115547 ± 3% +8.0% 124768 ± 4% softirqs.CPU0.TIMER 135243 ± 3% +14.0% 154188 ± 2% softirqs.CPU1.RCU 136056 ± 3% +12.4% 152960 ± 3% softirqs.CPU10.RCU 134253 ± 3% +14.2% 153260 ± 3% softirqs.CPU100.RCU 134668 ± 4% +13.7% 153157 ± 2% softirqs.CPU101.RCU 109995 ± 3% +7.9% 118637 ± 3% softirqs.CPU101.TIMER 134989 ± 4% +13.9% 153700 ± 2% softirqs.CPU102.RCU 109876 ± 3% +8.9% 119608 ± 2% softirqs.CPU102.TIMER 134868 ± 4% +13.8% 153427 ± 3% softirqs.CPU103.RCU 109933 ± 3% +8.2% 118946 ± 3% softirqs.CPU103.TIMER 134454 ± 3% +14.0% 153285 ± 3% softirqs.CPU104.RCU 109731 ± 2% +8.4% 118965 ± 3% softirqs.CPU104.TIMER 134878 ± 3% +13.6% 153283 ± 3% softirqs.CPU105.RCU 134096 ± 3% +14.1% 153022 ± 3% softirqs.CPU106.RCU 109821 ± 2% +8.2% 118845 ± 3% softirqs.CPU106.TIMER 133905 ± 3% +15.0% 154004 ± 3% softirqs.CPU107.RCU 134117 ± 3% +14.3% 153232 ± 3% softirqs.CPU108.RCU 109545 ± 2% +8.8% 119182 ± 3% softirqs.CPU108.TIMER 134574 ± 3% +13.8% 153202 ± 3% softirqs.CPU109.RCU 109804 ± 3% +8.2% 118760 ± 3% softirqs.CPU109.TIMER 135106 ± 3% +14.8% 155124 ± 3% softirqs.CPU11.RCU 111746 ± 4% +7.8% 120460 ± 3% softirqs.CPU11.TIMER 134482 ± 4% +14.3% 153687 ± 3% softirqs.CPU110.RCU 110314 ± 3% +8.1% 119241 ± 3% softirqs.CPU110.TIMER 134464 ± 3% +14.3% 153634 ± 3% softirqs.CPU111.RCU 110626 ± 3% +8.3% 119815 ± 3% softirqs.CPU111.TIMER 134895 ± 3% +14.4% 154257 ± 3% softirqs.CPU112.RCU 110040 ± 3% +14.6% 126058 ± 16% softirqs.CPU112.TIMER 134644 ± 3% +13.6% 152993 ± 3% softirqs.CPU113.RCU 109693 ± 3% +8.1% 118569 ± 3% softirqs.CPU113.TIMER 134313 ± 3% +13.9% 152993 ± 3% softirqs.CPU114.RCU 109711 ± 3% +8.6% 119143 ± 3% softirqs.CPU114.TIMER 134559 ± 3% +13.8% 153115 ± 3% softirqs.CPU115.RCU 110226 ± 3% +8.2% 119221 ± 3% softirqs.CPU115.TIMER 134087 ± 3% +14.2% 153077 ± 3% softirqs.CPU116.RCU 110225 ± 3% +9.0% 120184 ± 3% softirqs.CPU116.TIMER 134271 ± 3% +14.1% 153220 ± 3% softirqs.CPU117.RCU 109179 ± 2% +8.3% 118274 ± 3% softirqs.CPU117.TIMER 134281 ± 3% +14.1% 153230 ± 3% softirqs.CPU118.RCU 134415 ± 3% +14.0% 153215 ± 3% softirqs.CPU119.RCU 109514 ± 2% +8.3% 118643 ± 3% softirqs.CPU119.TIMER 135560 ± 3% +13.5% 153874 ± 3% softirqs.CPU12.RCU 110316 ± 2% +8.5% 119734 ± 3% softirqs.CPU12.TIMER 134177 ± 3% +14.6% 153815 ± 3% softirqs.CPU120.RCU 134367 ± 3% +15.0% 154493 ± 3% softirqs.CPU121.RCU 133972 ± 3% +14.3% 153173 ± 3% softirqs.CPU122.RCU 134801 ± 4% +13.6% 153136 ± 3% softirqs.CPU123.RCU 134709 ± 3% +13.9% 153440 ± 3% softirqs.CPU124.RCU 135185 ± 3% +13.1% 152925 ± 3% softirqs.CPU125.RCU 134977 ± 3% +13.8% 153608 ± 3% softirqs.CPU126.RCU 133862 ± 4% +14.7% 153558 ± 3% softirqs.CPU127.RCU 134554 ± 3% +13.9% 153237 ± 3% softirqs.CPU128.RCU 134254 ± 3% +14.1% 153130 ± 3% softirqs.CPU129.RCU 135133 ± 3% +14.0% 154103 ± 3% softirqs.CPU13.RCU 110389 ± 2% +8.1% 119382 ± 3% softirqs.CPU13.TIMER 134319 ± 3% +13.8% 152909 ± 3% softirqs.CPU130.RCU 133590 ± 3% +14.5% 152925 ± 3% softirqs.CPU131.RCU 134416 ± 4% +14.0% 153223 ± 3% softirqs.CPU132.RCU 133931 ± 3% +14.5% 153359 ± 3% softirqs.CPU133.RCU 134031 ± 3% +14.5% 153527 ± 3% softirqs.CPU134.RCU 133851 ± 4% +14.3% 153052 ± 3% softirqs.CPU135.RCU 134235 ± 4% +13.9% 152959 ± 3% softirqs.CPU136.RCU 134003 ± 3% +14.2% 153037 ± 3% softirqs.CPU137.RCU 133453 ± 3% +14.4% 152625 ± 3% softirqs.CPU138.RCU 133514 ± 3% +14.8% 153289 ± 3% softirqs.CPU139.RCU 134952 ± 3% +13.6% 153372 ± 3% softirqs.CPU14.RCU 134045 ± 3% +13.7% 152361 ± 3% softirqs.CPU140.RCU 133662 ± 3% +14.1% 152528 ± 3% softirqs.CPU141.RCU 133643 ± 3% +14.7% 153283 ± 2% softirqs.CPU142.RCU 133313 ± 3% +15.0% 153254 ± 3% softirqs.CPU143.RCU 133585 ± 3% +14.4% 152837 ± 3% softirqs.CPU144.RCU 108590 ± 3% +12.7% 122339 ± 5% softirqs.CPU144.TIMER 133471 ± 3% +14.5% 152883 ± 3% softirqs.CPU145.RCU 108095 ± 3% +14.5% 123726 ± 6% softirqs.CPU145.TIMER 133588 ± 3% +15.4% 154173 ± 3% softirqs.CPU146.RCU 108268 ± 3% +19.8% 129684 ± 16% softirqs.CPU146.TIMER 134060 ± 3% +14.3% 153270 ± 3% softirqs.CPU147.RCU 108731 ± 3% +13.0% 122902 ± 5% softirqs.CPU147.TIMER 133778 ± 3% +14.1% 152681 ± 3% softirqs.CPU148.RCU 108208 ± 3% +12.6% 121875 ± 5% softirqs.CPU148.TIMER 133514 ± 3% +14.2% 152460 ± 3% softirqs.CPU149.RCU 108054 ± 3% +15.6% 124938 ± 9% softirqs.CPU149.TIMER 134677 ± 3% +14.5% 154138 ± 2% softirqs.CPU15.RCU 133245 ± 3% +14.4% 152373 ± 3% softirqs.CPU150.RCU 107788 ± 3% +13.1% 121957 ± 5% softirqs.CPU150.TIMER 133747 ± 3% +14.0% 152473 ± 3% softirqs.CPU151.RCU 108258 ± 4% +12.4% 121662 ± 5% softirqs.CPU151.TIMER 133819 ± 3% +14.1% 152726 ± 3% softirqs.CPU152.RCU 108163 ± 3% +12.7% 121940 ± 5% softirqs.CPU152.TIMER 134057 ± 3% +14.0% 152844 ± 3% softirqs.CPU153.RCU 107680 ± 3% +13.1% 121787 ± 5% softirqs.CPU153.TIMER 133660 ± 3% +14.2% 152675 ± 3% softirqs.CPU154.RCU 107911 ± 3% +15.4% 124485 ± 9% softirqs.CPU154.TIMER 133349 ± 3% +14.6% 152811 ± 3% softirqs.CPU155.RCU 107513 ± 3% +15.1% 123737 ± 6% softirqs.CPU155.TIMER 133901 ± 3% +14.0% 152615 ± 3% softirqs.CPU156.RCU 107853 ± 3% +13.0% 121904 ± 5% softirqs.CPU156.TIMER 133621 ± 3% +14.2% 152615 ± 3% softirqs.CPU157.RCU 108040 ± 3% +12.6% 121684 ± 5% softirqs.CPU157.TIMER 133891 ± 3% +14.2% 152862 ± 3% softirqs.CPU158.RCU 108534 ± 3% +12.7% 122300 ± 5% softirqs.CPU158.TIMER 133925 ± 3% +14.5% 153347 ± 3% softirqs.CPU159.RCU 108099 ± 3% +12.9% 122015 ± 5% softirqs.CPU159.TIMER 134893 ± 3% +14.5% 154389 ± 3% softirqs.CPU16.RCU 134169 ± 3% +13.9% 152812 ± 3% softirqs.CPU160.RCU 108300 ± 3% +12.9% 122318 ± 5% softirqs.CPU160.TIMER 133818 ± 3% +15.1% 154048 ± 4% softirqs.CPU161.RCU 107919 ± 3% +13.6% 122606 ± 6% softirqs.CPU161.TIMER 133867 ± 3% +14.2% 152856 ± 3% softirqs.CPU162.RCU 107991 ± 3% +13.2% 122209 ± 5% softirqs.CPU162.TIMER 133579 ± 3% +14.3% 152654 ± 3% softirqs.CPU163.RCU 108317 ± 3% +12.9% 122271 ± 5% softirqs.CPU163.TIMER 133914 ± 3% +14.4% 153158 ± 3% softirqs.CPU164.RCU 133684 ± 3% +14.4% 152895 ± 3% softirqs.CPU165.RCU 107435 ± 3% +12.5% 120877 ± 5% softirqs.CPU165.TIMER 133983 ± 3% +13.9% 152582 ± 3% softirqs.CPU166.RCU 107432 ± 3% +13.2% 121641 ± 5% softirqs.CPU166.TIMER 133947 ± 3% +13.6% 152137 ± 3% softirqs.CPU167.RCU 134005 ± 3% +14.5% 153385 ± 4% softirqs.CPU168.RCU 115752 ± 4% +12.0% 129634 ± 4% softirqs.CPU168.TIMER 133557 ± 3% +15.2% 153865 ± 4% softirqs.CPU169.RCU 115884 ± 3% +11.7% 129481 ± 4% softirqs.CPU169.TIMER 135348 ± 3% +13.8% 153977 ± 3% softirqs.CPU17.RCU 110589 ± 3% +7.9% 119298 ± 3% softirqs.CPU17.TIMER 134463 ± 3% +14.0% 153229 ± 3% softirqs.CPU170.RCU 133740 ± 3% +14.1% 152630 ± 3% softirqs.CPU171.RCU 116524 ± 4% +10.9% 129237 ± 3% softirqs.CPU171.TIMER 133716 ± 3% +13.8% 152196 ± 3% softirqs.CPU172.RCU 116393 ± 4% +10.9% 129087 ± 4% softirqs.CPU172.TIMER 133564 ± 3% +13.9% 152165 ± 3% softirqs.CPU173.RCU 116237 ± 3% +11.4% 129460 ± 3% softirqs.CPU173.TIMER 133411 ± 3% +14.5% 152764 ± 3% softirqs.CPU174.RCU 116018 ± 3% +11.2% 129006 ± 4% softirqs.CPU174.TIMER 133147 ± 3% +15.0% 153104 ± 2% softirqs.CPU175.RCU 116052 ± 4% +11.7% 129664 ± 3% softirqs.CPU175.TIMER 133989 ± 3% +13.8% 152504 ± 3% softirqs.CPU176.RCU 116071 ± 4% +10.9% 128756 ± 3% softirqs.CPU176.TIMER 133902 ± 3% +14.1% 152811 ± 3% softirqs.CPU177.RCU 115994 ± 3% +11.1% 128849 ± 4% softirqs.CPU177.TIMER 133679 ± 3% +14.4% 152887 ± 3% softirqs.CPU178.RCU 115942 ± 4% +13.4% 131439 ± 5% softirqs.CPU178.TIMER 133985 ± 3% +14.0% 152762 ± 3% softirqs.CPU179.RCU 116347 ± 4% +11.2% 129422 ± 4% softirqs.CPU179.TIMER 134982 ± 3% +14.0% 153838 ± 3% softirqs.CPU18.RCU 110310 ± 3% +8.6% 119851 ± 3% softirqs.CPU18.TIMER 133236 ± 3% +14.7% 152870 ± 3% softirqs.CPU180.RCU 115901 ± 4% +11.3% 129054 ± 4% softirqs.CPU180.TIMER 133615 ± 3% +14.4% 152876 ± 3% softirqs.CPU181.RCU 115505 ± 4% +11.3% 128560 ± 3% softirqs.CPU181.TIMER 133171 ± 3% +14.7% 152810 ± 3% softirqs.CPU182.RCU 115955 ± 4% +11.0% 128687 ± 3% softirqs.CPU182.TIMER 133413 ± 3% +14.4% 152671 ± 3% softirqs.CPU183.RCU 116313 ± 4% +11.0% 129140 ± 3% softirqs.CPU183.TIMER 133598 ± 3% +14.8% 153388 ± 3% softirqs.CPU184.RCU 115462 ± 3% +13.0% 130430 ± 6% softirqs.CPU184.TIMER 133884 ± 3% +13.9% 152525 ± 3% softirqs.CPU185.RCU 115598 ± 4% +11.1% 128397 ± 4% softirqs.CPU185.TIMER 133262 ± 3% +14.5% 152646 ± 3% softirqs.CPU186.RCU 115745 ± 3% +11.1% 128612 ± 4% softirqs.CPU186.TIMER 133430 ± 3% +14.5% 152767 ± 3% softirqs.CPU187.RCU 115769 ± 4% +11.0% 128464 ± 3% softirqs.CPU187.TIMER 133259 ± 3% +14.8% 153036 ± 3% softirqs.CPU188.RCU 116108 ± 4% +11.3% 129232 ± 3% softirqs.CPU188.TIMER 133609 ± 3% +14.1% 152436 ± 3% softirqs.CPU189.RCU 115984 ± 4% +11.0% 128710 ± 3% softirqs.CPU189.TIMER 135090 ± 3% +13.9% 153802 ± 3% softirqs.CPU19.RCU 110865 ± 2% +8.2% 119936 ± 3% softirqs.CPU19.TIMER 133685 ± 3% +14.2% 152631 ± 3% softirqs.CPU190.RCU 115642 ± 3% +11.2% 128581 ± 4% softirqs.CPU190.TIMER 131031 ± 3% +14.6% 150126 ± 3% softirqs.CPU191.RCU 114275 ± 4% +11.4% 127249 ± 4% softirqs.CPU191.TIMER 134928 ± 3% +14.3% 154165 ± 3% softirqs.CPU2.RCU 135126 ± 3% +14.0% 154083 ± 3% softirqs.CPU20.RCU 111013 ± 3% +8.6% 120549 ± 3% softirqs.CPU20.TIMER 134663 ± 3% +14.5% 154251 ± 3% softirqs.CPU21.RCU 109671 ± 2% +8.2% 118708 ± 3% softirqs.CPU21.TIMER 135047 ± 3% +13.9% 153766 ± 3% softirqs.CPU22.RCU 134437 ± 3% +14.2% 153492 ± 3% softirqs.CPU23.RCU 109675 ± 2% +8.3% 118824 ± 3% softirqs.CPU23.TIMER 136114 ± 3% +13.3% 154277 ± 3% softirqs.CPU24.RCU 134825 ± 3% +14.2% 154007 ± 3% softirqs.CPU25.RCU 139234 ± 4% +13.8% 158515 ± 3% softirqs.CPU26.RCU 135056 ± 3% +14.1% 154062 ± 3% softirqs.CPU27.RCU 135307 ± 3% +13.7% 153897 ± 3% softirqs.CPU28.RCU 134686 ± 3% +14.7% 154497 ± 3% softirqs.CPU29.RCU 134688 ± 3% +14.4% 154115 ± 2% softirqs.CPU3.RCU 134642 ± 3% +14.5% 154226 ± 3% softirqs.CPU30.RCU 134773 ± 3% +14.2% 153887 ± 3% softirqs.CPU31.RCU 134523 ± 3% +14.3% 153733 ± 3% softirqs.CPU32.RCU 134488 ± 3% +14.4% 153801 ± 3% softirqs.CPU33.RCU 134483 ± 3% +14.1% 153380 ± 3% softirqs.CPU34.RCU 134886 ± 3% +13.6% 153180 ± 3% softirqs.CPU35.RCU 135021 ± 3% +13.6% 153340 ± 3% softirqs.CPU36.RCU 134064 ± 3% +15.1% 154277 ± 3% softirqs.CPU37.RCU 134166 ± 3% +14.8% 153967 ± 3% softirqs.CPU38.RCU 134488 ± 3% +14.4% 153871 ± 3% softirqs.CPU39.RCU 135294 ± 2% +14.6% 155025 ± 3% softirqs.CPU4.RCU 134598 ± 4% +14.3% 153848 ± 3% softirqs.CPU40.RCU 134255 ± 3% +14.8% 154099 ± 3% softirqs.CPU41.RCU 134334 ± 3% +14.3% 153491 ± 3% softirqs.CPU42.RCU 134201 ± 3% +14.5% 153654 ± 3% softirqs.CPU43.RCU 134260 ± 3% +14.2% 153359 ± 3% softirqs.CPU44.RCU 134005 ± 3% +14.4% 153367 ± 3% softirqs.CPU45.RCU 134236 ± 3% +14.3% 153419 ± 3% softirqs.CPU46.RCU 134552 ± 3% +13.8% 153056 ± 3% softirqs.CPU47.RCU 134623 ± 3% +14.4% 154028 ± 3% softirqs.CPU48.RCU 134348 ± 3% +14.7% 154080 ± 3% softirqs.CPU49.RCU 109020 ± 3% +18.6% 129279 ± 14% softirqs.CPU49.TIMER 134334 ± 3% +15.0% 154455 ± 3% softirqs.CPU5.RCU 110260 ± 2% +8.8% 119951 ± 4% softirqs.CPU5.TIMER 134377 ± 3% +13.8% 152954 ± 3% softirqs.CPU50.RCU 109169 ± 3% +15.0% 125548 ± 9% softirqs.CPU50.TIMER 134780 ± 3% +14.2% 153967 ± 3% softirqs.CPU51.RCU 109951 ± 3% +12.5% 123641 ± 5% softirqs.CPU51.TIMER 133929 ± 3% +15.0% 154012 ± 3% softirqs.CPU52.RCU 108665 ± 3% +12.8% 122613 ± 5% softirqs.CPU52.TIMER 133761 ± 3% +14.9% 153675 ± 3% softirqs.CPU53.RCU 108606 ± 3% +19.6% 129861 ± 16% softirqs.CPU53.TIMER 134369 ± 3% +14.1% 153255 ± 3% softirqs.CPU54.RCU 108917 ± 3% +12.5% 122585 ± 5% softirqs.CPU54.TIMER 133810 ± 3% +14.6% 153282 ± 3% softirqs.CPU55.RCU 108875 ± 3% +12.5% 122468 ± 5% softirqs.CPU55.TIMER 134038 ± 3% +14.3% 153248 ± 3% softirqs.CPU56.RCU 108569 ± 3% +12.8% 122502 ± 5% softirqs.CPU56.TIMER 133993 ± 4% +14.5% 153426 ± 3% softirqs.CPU57.RCU 108465 ± 3% +12.8% 122399 ± 5% softirqs.CPU57.TIMER 134049 ± 3% +14.7% 153708 ± 3% softirqs.CPU58.RCU 108634 ± 4% +18.8% 129107 ± 16% softirqs.CPU58.TIMER 134025 ± 3% +14.9% 154015 ± 3% softirqs.CPU59.RCU 108501 ± 3% +18.6% 128721 ± 13% softirqs.CPU59.TIMER 135247 ± 3% +14.5% 154804 ± 3% softirqs.CPU6.RCU 133745 ± 4% +14.7% 153452 ± 3% softirqs.CPU60.RCU 108526 ± 3% +13.0% 122688 ± 5% softirqs.CPU60.TIMER 133931 ± 3% +14.6% 153549 ± 3% softirqs.CPU61.RCU 108764 ± 3% +12.9% 122787 ± 5% softirqs.CPU61.TIMER 134682 ± 3% +13.8% 153266 ± 3% softirqs.CPU62.RCU 109188 ± 3% +12.5% 122806 ± 5% softirqs.CPU62.TIMER 134190 ± 3% +14.6% 153761 ± 2% softirqs.CPU63.RCU 108489 ± 3% +12.8% 122426 ± 5% softirqs.CPU63.TIMER 134280 ± 3% +14.3% 153538 ± 3% softirqs.CPU64.RCU 108493 ± 3% +13.3% 122953 ± 5% softirqs.CPU64.TIMER 134074 ± 3% +14.7% 153765 ± 3% softirqs.CPU65.RCU 108331 ± 3% +13.0% 122440 ± 5% softirqs.CPU65.TIMER 134424 ± 4% +14.3% 153692 ± 3% softirqs.CPU66.RCU 108874 ± 3% +12.5% 122490 ± 5% softirqs.CPU66.TIMER 133801 ± 3% +14.9% 153746 ± 3% softirqs.CPU67.RCU 108903 ± 3% +13.2% 123240 ± 5% softirqs.CPU67.TIMER 134085 ± 3% +14.4% 153371 ± 3% softirqs.CPU68.RCU 109310 ± 3% +12.3% 122810 ± 5% softirqs.CPU68.TIMER 134091 ± 3% +14.5% 153515 ± 3% softirqs.CPU69.RCU 108193 ± 3% +12.5% 121716 ± 5% softirqs.CPU69.TIMER 134883 ± 3% +13.6% 153181 ± 3% softirqs.CPU7.RCU 134392 ± 3% +14.8% 154240 ± 3% softirqs.CPU70.RCU 108056 ± 3% +13.4% 122578 ± 5% softirqs.CPU70.TIMER 133901 ± 3% +14.2% 152873 ± 3% softirqs.CPU71.RCU 134273 ± 3% +14.4% 153630 ± 3% softirqs.CPU72.RCU 116189 ± 3% +11.5% 129495 ± 3% softirqs.CPU72.TIMER 134231 ± 3% +14.5% 153682 ± 3% softirqs.CPU73.RCU 116348 ± 3% +11.3% 129453 ± 3% softirqs.CPU73.TIMER 134235 ± 3% +14.1% 153158 ± 3% softirqs.CPU74.RCU 133896 ± 3% +14.6% 153477 ± 3% softirqs.CPU75.RCU 116933 ± 4% +11.2% 129980 ± 3% softirqs.CPU75.TIMER 134014 ± 3% +14.3% 153198 ± 3% softirqs.CPU76.RCU 116889 ± 3% +11.0% 129768 ± 3% softirqs.CPU76.TIMER 134608 ± 3% +13.7% 152994 ± 3% softirqs.CPU77.RCU 117342 ± 3% +11.0% 130274 ± 3% softirqs.CPU77.TIMER 134551 ± 3% +13.9% 153253 ± 3% softirqs.CPU78.RCU 117229 ± 4% +10.5% 129549 ± 4% softirqs.CPU78.TIMER 134118 ± 3% +14.3% 153325 ± 3% softirqs.CPU79.RCU 116982 ± 3% +11.0% 129802 ± 3% softirqs.CPU79.TIMER 135303 ± 3% +13.9% 154129 ± 3% softirqs.CPU8.RCU 134699 ± 3% +13.6% 153047 ± 3% softirqs.CPU80.RCU 117058 ± 4% +10.6% 129470 ± 3% softirqs.CPU80.TIMER 134088 ± 3% +15.6% 155063 ± 3% softirqs.CPU81.RCU 116539 ± 4% +12.3% 130886 ± 5% softirqs.CPU81.TIMER 133989 ± 3% +14.6% 153526 ± 3% softirqs.CPU82.RCU 116614 ± 4% +16.8% 136238 ± 12% softirqs.CPU82.TIMER 133966 ± 3% +14.3% 153142 ± 3% softirqs.CPU83.RCU 117140 ± 3% +11.2% 130205 ± 4% softirqs.CPU83.TIMER 133561 ± 3% +15.0% 153661 ± 3% softirqs.CPU84.RCU 116648 ± 4% +11.3% 129844 ± 4% softirqs.CPU84.TIMER 134217 ± 3% +14.2% 153330 ± 3% softirqs.CPU85.RCU 116247 ± 4% +11.3% 129336 ± 3% softirqs.CPU85.TIMER 133956 ± 3% +14.5% 153338 ± 3% softirqs.CPU86.RCU 116654 ± 4% +11.1% 129560 ± 3% softirqs.CPU86.TIMER 133834 ± 3% +15.1% 154049 ± 2% softirqs.CPU87.RCU 116898 ± 4% +12.0% 130912 ± 3% softirqs.CPU87.TIMER 133951 ± 3% +15.1% 154211 ± 3% softirqs.CPU88.RCU 116139 ± 3% +16.0% 134676 ± 13% softirqs.CPU88.TIMER 134288 ± 3% +14.3% 153433 ± 2% softirqs.CPU89.RCU 116227 ± 3% +11.0% 129064 ± 3% softirqs.CPU89.TIMER 135289 ± 3% +13.6% 153744 ± 3% softirqs.CPU9.RCU 133598 ± 3% +14.5% 153019 ± 3% softirqs.CPU90.RCU 116395 ± 3% +10.9% 129084 ± 4% softirqs.CPU90.TIMER 133854 ± 4% +14.9% 153799 ± 3% softirqs.CPU91.RCU 116562 ± 4% +11.1% 129452 ± 3% softirqs.CPU91.TIMER 134107 ± 3% +14.1% 152958 ± 3% softirqs.CPU92.RCU 117107 ± 4% +10.6% 129559 ± 3% softirqs.CPU92.TIMER 133800 ± 3% +14.7% 153465 ± 3% softirqs.CPU93.RCU 116415 ± 3% +11.5% 129823 ± 4% softirqs.CPU93.TIMER 134092 ± 3% +14.0% 152835 ± 3% softirqs.CPU94.RCU 116063 ± 3% +11.4% 129252 ± 4% softirqs.CPU94.TIMER 134284 ± 3% +14.1% 153230 ± 4% softirqs.CPU95.RCU 117908 ± 3% +11.0% 130904 ± 4% softirqs.CPU95.TIMER 134264 ± 3% +14.3% 153455 ± 3% softirqs.CPU96.RCU 109819 ± 3% +8.6% 119258 ± 3% softirqs.CPU96.TIMER 135000 ± 3% +13.7% 153529 ± 3% softirqs.CPU97.RCU 134163 ± 3% +14.2% 153253 ± 3% softirqs.CPU98.RCU 109990 ± 3% +8.1% 118896 ± 3% softirqs.CPU98.TIMER 134966 ± 3% +13.5% 153245 ± 3% softirqs.CPU99.RCU 25776297 ± 3% +14.3% 29452390 ± 3% softirqs.RCU 21445010 ± 3% +9.5% 23488096 ± 3% softirqs.TIMER 614.00 ± 2% +9.9% 674.50 ± 2% interrupts.9:IO-APIC.9-fasteoi.acpi 611854 ± 2% +10.1% 673945 ± 2% interrupts.CPU0.LOC:Local_timer_interrupts 614.00 ± 2% +9.9% 674.50 ± 2% interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 611835 ± 2% +10.1% 673883 ± 2% interrupts.CPU1.LOC:Local_timer_interrupts 611859 ± 2% +10.2% 673975 ± 2% interrupts.CPU10.LOC:Local_timer_interrupts 7508 ± 5% +15.5% 8675 ± 4% interrupts.CPU10.RES:Rescheduling_interrupts 611802 ± 2% +10.2% 673904 ± 2% interrupts.CPU100.LOC:Local_timer_interrupts 7323 ± 5% +16.1% 8503 ± 3% interrupts.CPU100.RES:Rescheduling_interrupts 611804 ± 2% +10.2% 673917 ± 2% interrupts.CPU101.LOC:Local_timer_interrupts 7342 ± 3% +13.8% 8352 ± 3% interrupts.CPU101.RES:Rescheduling_interrupts 611780 ± 2% +10.2% 673877 ± 2% interrupts.CPU102.LOC:Local_timer_interrupts 7506 ± 5% +12.2% 8424 ± 4% interrupts.CPU102.RES:Rescheduling_interrupts 611734 ± 2% +10.2% 673851 ± 2% interrupts.CPU103.LOC:Local_timer_interrupts 7436 ± 3% +15.8% 8607 ± 4% interrupts.CPU103.RES:Rescheduling_interrupts 611800 ± 2% +10.2% 673908 ± 2% interrupts.CPU104.LOC:Local_timer_interrupts 7519 ± 5% +13.1% 8502 ± 2% interrupts.CPU104.RES:Rescheduling_interrupts 611790 ± 2% +10.1% 673864 ± 2% interrupts.CPU105.LOC:Local_timer_interrupts 7346 ± 5% +16.4% 8553 ± 2% interrupts.CPU105.RES:Rescheduling_interrupts 611801 ± 2% +10.1% 673859 ± 2% interrupts.CPU106.LOC:Local_timer_interrupts 7352 ± 5% +16.5% 8567 ± 2% interrupts.CPU106.RES:Rescheduling_interrupts 611797 ± 2% +10.1% 673890 ± 2% interrupts.CPU107.LOC:Local_timer_interrupts 7238 ± 6% +15.3% 8344 ± 4% interrupts.CPU107.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673882 ± 2% interrupts.CPU108.LOC:Local_timer_interrupts 7310 ± 5% +16.3% 8503 ± 5% interrupts.CPU108.RES:Rescheduling_interrupts 611788 ± 2% +10.1% 673881 ± 2% interrupts.CPU109.LOC:Local_timer_interrupts 7324 ± 6% +13.4% 8307 interrupts.CPU109.RES:Rescheduling_interrupts 611827 ± 2% +10.2% 673960 ± 2% interrupts.CPU11.LOC:Local_timer_interrupts 7598 ± 6% +12.6% 8557 ± 5% interrupts.CPU11.RES:Rescheduling_interrupts 611796 ± 2% +10.1% 673865 ± 2% interrupts.CPU110.LOC:Local_timer_interrupts 7326 ± 4% +16.2% 8513 ± 4% interrupts.CPU110.RES:Rescheduling_interrupts 611804 ± 2% +10.1% 673890 ± 2% interrupts.CPU111.LOC:Local_timer_interrupts 7312 ± 3% +14.4% 8365 ± 3% interrupts.CPU111.RES:Rescheduling_interrupts 611801 ± 2% +10.1% 673890 ± 2% interrupts.CPU112.LOC:Local_timer_interrupts 7277 ± 2% +14.6% 8337 ± 5% interrupts.CPU112.RES:Rescheduling_interrupts 611789 ± 2% +10.2% 673896 ± 2% interrupts.CPU113.LOC:Local_timer_interrupts 7346 ± 4% +15.2% 8461 ± 4% interrupts.CPU113.RES:Rescheduling_interrupts 611759 ± 2% +10.2% 673861 ± 2% interrupts.CPU114.LOC:Local_timer_interrupts 7290 ± 4% +17.9% 8594 ± 4% interrupts.CPU114.RES:Rescheduling_interrupts 611783 ± 2% +10.2% 673883 ± 2% interrupts.CPU115.LOC:Local_timer_interrupts 7470 ± 4% +13.6% 8489 ± 5% interrupts.CPU115.RES:Rescheduling_interrupts 611807 ± 2% +10.2% 673925 ± 2% interrupts.CPU116.LOC:Local_timer_interrupts 7336 ± 4% +15.3% 8455 ± 2% interrupts.CPU116.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673890 ± 2% interrupts.CPU117.LOC:Local_timer_interrupts 7326 ± 5% +15.9% 8493 ± 3% interrupts.CPU117.RES:Rescheduling_interrupts 611782 ± 2% +10.1% 673849 ± 2% interrupts.CPU118.LOC:Local_timer_interrupts 7239 ± 4% +18.4% 8574 ± 5% interrupts.CPU118.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673881 ± 2% interrupts.CPU119.LOC:Local_timer_interrupts 7327 ± 4% +13.7% 8335 ± 6% interrupts.CPU119.RES:Rescheduling_interrupts 611853 ± 2% +10.1% 673944 ± 2% interrupts.CPU12.LOC:Local_timer_interrupts 7516 ± 4% +17.0% 8798 ± 6% interrupts.CPU12.RES:Rescheduling_interrupts 611417 ± 2% +10.2% 673606 ± 2% interrupts.CPU120.LOC:Local_timer_interrupts 7392 ± 4% +14.2% 8444 ± 5% interrupts.CPU120.RES:Rescheduling_interrupts 611457 ± 2% +10.2% 673608 ± 2% interrupts.CPU121.LOC:Local_timer_interrupts 7376 ± 3% +15.6% 8525 ± 4% interrupts.CPU121.RES:Rescheduling_interrupts 611426 ± 2% +10.2% 673610 ± 2% interrupts.CPU122.LOC:Local_timer_interrupts 7466 ± 5% +11.7% 8336 ± 4% interrupts.CPU122.RES:Rescheduling_interrupts 611384 ± 2% +10.2% 673577 ± 2% interrupts.CPU123.LOC:Local_timer_interrupts 7240 ± 5% +18.2% 8559 ± 3% interrupts.CPU123.RES:Rescheduling_interrupts 611388 ± 2% +10.2% 673597 ± 2% interrupts.CPU124.LOC:Local_timer_interrupts 7245 ± 5% +18.0% 8548 ± 5% interrupts.CPU124.RES:Rescheduling_interrupts 611391 ± 2% +10.2% 673567 ± 2% interrupts.CPU125.LOC:Local_timer_interrupts 7318 ± 3% +15.3% 8436 ± 5% interrupts.CPU125.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673570 ± 2% interrupts.CPU126.LOC:Local_timer_interrupts 7258 ± 5% +16.4% 8448 ± 4% interrupts.CPU126.RES:Rescheduling_interrupts 611468 ± 2% +10.2% 673614 ± 2% interrupts.CPU127.LOC:Local_timer_interrupts 7276 ± 5% +16.9% 8508 ± 5% interrupts.CPU127.RES:Rescheduling_interrupts 611422 ± 2% +10.2% 673591 ± 2% interrupts.CPU128.LOC:Local_timer_interrupts 7278 ± 4% +17.0% 8518 ± 3% interrupts.CPU128.RES:Rescheduling_interrupts 611407 ± 2% +10.2% 673562 ± 2% interrupts.CPU129.LOC:Local_timer_interrupts 7284 ± 3% +17.0% 8519 ± 5% interrupts.CPU129.RES:Rescheduling_interrupts 611825 ± 2% +10.1% 673920 ± 2% interrupts.CPU13.LOC:Local_timer_interrupts 7550 ± 4% +16.2% 8771 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts 611428 ± 2% +10.2% 673571 ± 2% interrupts.CPU130.LOC:Local_timer_interrupts 7337 ± 5% +16.4% 8541 ± 5% interrupts.CPU130.RES:Rescheduling_interrupts 611406 ± 2% +10.2% 673582 ± 2% interrupts.CPU131.LOC:Local_timer_interrupts 7205 ± 5% +17.1% 8440 ± 6% interrupts.CPU131.RES:Rescheduling_interrupts 611449 ± 2% +10.2% 673586 ± 2% interrupts.CPU132.LOC:Local_timer_interrupts 7431 ± 4% +12.7% 8372 ± 3% interrupts.CPU132.RES:Rescheduling_interrupts 611464 ± 2% +10.2% 673588 ± 2% interrupts.CPU133.LOC:Local_timer_interrupts 7289 ± 4% +15.9% 8446 ± 3% interrupts.CPU133.RES:Rescheduling_interrupts 611437 ± 2% +10.2% 673583 ± 2% interrupts.CPU134.LOC:Local_timer_interrupts 7321 ± 4% +17.7% 8616 ± 4% interrupts.CPU134.RES:Rescheduling_interrupts 611401 ± 2% +10.2% 673595 ± 2% interrupts.CPU135.LOC:Local_timer_interrupts 7300 ± 4% +15.7% 8446 ± 4% interrupts.CPU135.RES:Rescheduling_interrupts 611462 ± 2% +10.2% 673602 ± 2% interrupts.CPU136.LOC:Local_timer_interrupts 7389 ± 4% +13.9% 8417 ± 5% interrupts.CPU136.RES:Rescheduling_interrupts 611424 ± 2% +10.2% 673588 ± 2% interrupts.CPU137.LOC:Local_timer_interrupts 7126 ± 4% +20.0% 8550 ± 5% interrupts.CPU137.RES:Rescheduling_interrupts 611383 ± 2% +10.2% 673556 ± 2% interrupts.CPU138.LOC:Local_timer_interrupts 7297 ± 5% +16.1% 8475 ± 5% interrupts.CPU138.RES:Rescheduling_interrupts 611422 ± 2% +10.2% 673603 ± 2% interrupts.CPU139.LOC:Local_timer_interrupts 7160 ± 4% +17.6% 8423 ± 4% interrupts.CPU139.RES:Rescheduling_interrupts 611815 ± 2% +10.1% 673901 ± 2% interrupts.CPU14.LOC:Local_timer_interrupts 7576 ± 5% +16.3% 8810 ± 3% interrupts.CPU14.RES:Rescheduling_interrupts 611388 ± 2% +10.2% 673577 ± 2% interrupts.CPU140.LOC:Local_timer_interrupts 7342 ± 6% +12.9% 8293 ± 4% interrupts.CPU140.RES:Rescheduling_interrupts 611394 ± 2% +10.2% 673596 ± 2% interrupts.CPU141.LOC:Local_timer_interrupts 7257 ± 4% +16.1% 8423 ± 5% interrupts.CPU141.RES:Rescheduling_interrupts 611411 ± 2% +10.2% 673590 ± 2% interrupts.CPU142.LOC:Local_timer_interrupts 7300 ± 6% +16.1% 8475 ± 3% interrupts.CPU142.RES:Rescheduling_interrupts 611396 ± 2% +10.2% 673622 ± 2% interrupts.CPU143.LOC:Local_timer_interrupts 7399 ± 5% +14.0% 8435 ± 4% interrupts.CPU143.RES:Rescheduling_interrupts 610929 ± 2% +10.3% 673686 ± 2% interrupts.CPU144.LOC:Local_timer_interrupts 7281 ± 5% +13.4% 8257 ± 5% interrupts.CPU144.RES:Rescheduling_interrupts 610918 ± 2% +10.3% 673755 ± 2% interrupts.CPU145.LOC:Local_timer_interrupts 7343 ± 5% +14.5% 8408 ± 4% interrupts.CPU145.RES:Rescheduling_interrupts 610910 ± 2% +10.3% 673705 ± 2% interrupts.CPU146.LOC:Local_timer_interrupts 7524 ± 4% +13.6% 8550 ± 6% interrupts.CPU146.RES:Rescheduling_interrupts 610858 ± 2% +10.3% 673769 ± 2% interrupts.CPU147.LOC:Local_timer_interrupts 7433 ± 4% +10.8% 8237 ± 3% interrupts.CPU147.RES:Rescheduling_interrupts 610872 ± 2% +10.3% 673727 ± 2% interrupts.CPU148.LOC:Local_timer_interrupts 7354 ± 3% +13.1% 8320 ± 4% interrupts.CPU148.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673750 ± 2% interrupts.CPU149.LOC:Local_timer_interrupts 7365 ± 3% +16.5% 8582 ± 3% interrupts.CPU149.RES:Rescheduling_interrupts 611818 ± 2% +10.2% 673937 ± 2% interrupts.CPU15.LOC:Local_timer_interrupts 7632 ± 4% +12.5% 8587 ± 4% interrupts.CPU15.RES:Rescheduling_interrupts 610898 ± 2% +10.3% 673770 ± 2% interrupts.CPU150.LOC:Local_timer_interrupts 7369 ± 4% +12.4% 8282 ± 3% interrupts.CPU150.RES:Rescheduling_interrupts 610887 ± 2% +10.3% 673754 ± 2% interrupts.CPU151.LOC:Local_timer_interrupts 7322 ± 4% +15.3% 8445 ± 3% interrupts.CPU151.RES:Rescheduling_interrupts 610849 ± 2% +10.3% 673762 ± 2% interrupts.CPU152.LOC:Local_timer_interrupts 7305 ± 4% +15.5% 8439 ± 5% interrupts.CPU152.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673759 ± 2% interrupts.CPU153.LOC:Local_timer_interrupts 7363 ± 6% +14.5% 8432 ± 2% interrupts.CPU153.RES:Rescheduling_interrupts 610878 ± 2% +10.3% 673730 ± 2% interrupts.CPU154.LOC:Local_timer_interrupts 7373 ± 4% +14.1% 8414 ± 4% interrupts.CPU154.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673720 ± 2% interrupts.CPU155.LOC:Local_timer_interrupts 7385 ± 5% +11.8% 8260 ± 4% interrupts.CPU155.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673753 ± 2% interrupts.CPU156.LOC:Local_timer_interrupts 7377 ± 5% +14.0% 8410 ± 5% interrupts.CPU156.RES:Rescheduling_interrupts 610886 ± 2% +10.3% 673754 ± 2% interrupts.CPU157.LOC:Local_timer_interrupts 7399 ± 5% +13.1% 8370 ± 4% interrupts.CPU157.RES:Rescheduling_interrupts 610854 ± 2% +10.3% 673699 ± 2% interrupts.CPU158.LOC:Local_timer_interrupts 7489 ± 4% +10.5% 8278 ± 2% interrupts.CPU158.RES:Rescheduling_interrupts 610921 ± 2% +10.3% 673726 ± 2% interrupts.CPU159.LOC:Local_timer_interrupts 7281 ± 5% +15.4% 8402 ± 4% interrupts.CPU159.RES:Rescheduling_interrupts 611836 ± 2% +10.1% 673889 ± 2% interrupts.CPU16.LOC:Local_timer_interrupts 7548 ± 4% +15.6% 8723 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts 610870 ± 2% +10.3% 673742 ± 2% interrupts.CPU160.LOC:Local_timer_interrupts 610856 ± 2% +10.3% 673722 ± 2% interrupts.CPU161.LOC:Local_timer_interrupts 7472 ± 5% +13.4% 8475 ± 5% interrupts.CPU161.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673708 ± 2% interrupts.CPU162.LOC:Local_timer_interrupts 7177 ± 5% +14.8% 8239 ± 5% interrupts.CPU162.RES:Rescheduling_interrupts 610864 ± 2% +10.3% 673696 ± 2% interrupts.CPU163.LOC:Local_timer_interrupts 7337 ± 5% +12.7% 8266 ± 4% interrupts.CPU163.RES:Rescheduling_interrupts 610858 ± 2% +10.3% 673724 ± 2% interrupts.CPU164.LOC:Local_timer_interrupts 7269 ± 5% +15.2% 8376 ± 4% interrupts.CPU164.RES:Rescheduling_interrupts 610861 ± 2% +10.3% 673711 ± 2% interrupts.CPU165.LOC:Local_timer_interrupts 7431 ± 4% +9.4% 8131 ± 4% interrupts.CPU165.RES:Rescheduling_interrupts 610867 ± 2% +10.3% 673684 ± 2% interrupts.CPU166.LOC:Local_timer_interrupts 7339 ± 2% +14.1% 8374 ± 4% interrupts.CPU166.RES:Rescheduling_interrupts 610913 ± 2% +10.3% 673730 ± 2% interrupts.CPU167.LOC:Local_timer_interrupts 7174 ± 3% +17.5% 8427 ± 4% interrupts.CPU167.RES:Rescheduling_interrupts 611289 ± 2% +10.2% 673888 ± 2% interrupts.CPU168.LOC:Local_timer_interrupts 7311 ± 5% +14.7% 8382 ± 3% interrupts.CPU168.RES:Rescheduling_interrupts 611333 ± 2% +10.2% 673887 ± 2% interrupts.CPU169.LOC:Local_timer_interrupts 7232 ± 2% +17.8% 8517 ± 5% interrupts.CPU169.RES:Rescheduling_interrupts 611838 ± 2% +10.1% 673879 ± 2% interrupts.CPU17.LOC:Local_timer_interrupts 7539 ± 2% +17.7% 8873 ± 10% interrupts.CPU17.RES:Rescheduling_interrupts 611349 ± 2% +10.2% 673889 ± 2% interrupts.CPU170.LOC:Local_timer_interrupts 7374 ± 2% +14.2% 8424 ± 5% interrupts.CPU170.RES:Rescheduling_interrupts 611297 ± 2% +10.2% 673873 ± 2% interrupts.CPU171.LOC:Local_timer_interrupts 7438 ± 4% +13.9% 8471 ± 5% interrupts.CPU171.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673845 ± 2% interrupts.CPU172.LOC:Local_timer_interrupts 7317 ± 3% +16.6% 8529 ± 5% interrupts.CPU172.RES:Rescheduling_interrupts 611297 ± 2% +10.2% 673887 ± 2% interrupts.CPU173.LOC:Local_timer_interrupts 611309 ± 2% +10.2% 673858 ± 2% interrupts.CPU174.LOC:Local_timer_interrupts 7369 ± 5% +15.8% 8531 ± 5% interrupts.CPU174.RES:Rescheduling_interrupts 611311 ± 2% +10.2% 673872 ± 2% interrupts.CPU175.LOC:Local_timer_interrupts 7352 ± 4% +15.1% 8465 ± 4% interrupts.CPU175.RES:Rescheduling_interrupts 611309 ± 2% +10.2% 673886 ± 2% interrupts.CPU176.LOC:Local_timer_interrupts 7395 ± 2% +14.0% 8434 ± 5% interrupts.CPU176.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673868 ± 2% interrupts.CPU177.LOC:Local_timer_interrupts 611248 ± 2% +10.2% 673805 ± 2% interrupts.CPU178.LOC:Local_timer_interrupts 7355 ± 5% +14.1% 8393 ± 4% interrupts.CPU178.RES:Rescheduling_interrupts 611319 ± 2% +10.2% 673906 ± 2% interrupts.CPU179.LOC:Local_timer_interrupts 7269 ± 4% +15.2% 8374 ± 6% interrupts.CPU179.RES:Rescheduling_interrupts 611799 ± 2% +10.1% 673864 ± 2% interrupts.CPU18.LOC:Local_timer_interrupts 7585 ± 4% +14.5% 8683 ± 4% interrupts.CPU18.RES:Rescheduling_interrupts 611330 ± 2% +10.2% 673869 ± 2% interrupts.CPU180.LOC:Local_timer_interrupts 7404 ± 4% +14.0% 8442 ± 5% interrupts.CPU180.RES:Rescheduling_interrupts 611324 ± 2% +10.2% 673889 ± 2% interrupts.CPU181.LOC:Local_timer_interrupts 7435 ± 2% +13.0% 8399 ± 3% interrupts.CPU181.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673874 ± 2% interrupts.CPU182.LOC:Local_timer_interrupts 7250 ± 4% +14.6% 8305 ± 4% interrupts.CPU182.RES:Rescheduling_interrupts 611307 ± 2% +10.2% 673852 ± 2% interrupts.CPU183.LOC:Local_timer_interrupts 7320 ± 4% +13.2% 8290 ± 5% interrupts.CPU183.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673877 ± 2% interrupts.CPU184.LOC:Local_timer_interrupts 7377 ± 4% +12.5% 8299 ± 5% interrupts.CPU184.RES:Rescheduling_interrupts 611312 ± 2% +10.2% 673866 ± 2% interrupts.CPU185.LOC:Local_timer_interrupts 7333 ± 4% +13.9% 8351 ± 5% interrupts.CPU185.RES:Rescheduling_interrupts 611290 ± 2% +10.2% 673868 ± 2% interrupts.CPU186.LOC:Local_timer_interrupts 7426 ± 4% +14.1% 8471 ± 3% interrupts.CPU186.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673853 ± 2% interrupts.CPU187.LOC:Local_timer_interrupts 7380 ± 3% +12.5% 8304 ± 5% interrupts.CPU187.RES:Rescheduling_interrupts 611307 ± 2% +10.2% 673881 ± 2% interrupts.CPU188.LOC:Local_timer_interrupts 7277 ± 5% +16.3% 8467 ± 6% interrupts.CPU188.RES:Rescheduling_interrupts 611299 ± 2% +10.2% 673879 ± 2% interrupts.CPU189.LOC:Local_timer_interrupts 7436 ± 3% +14.5% 8514 ± 6% interrupts.CPU189.RES:Rescheduling_interrupts 611761 ± 2% +10.2% 673928 ± 2% interrupts.CPU19.LOC:Local_timer_interrupts 7515 ± 6% +12.7% 8467 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 611321 ± 2% +10.2% 673856 ± 2% interrupts.CPU190.LOC:Local_timer_interrupts 7188 ± 3% +16.4% 8366 ± 5% interrupts.CPU190.RES:Rescheduling_interrupts 611544 ± 2% +10.2% 674120 ± 2% interrupts.CPU191.LOC:Local_timer_interrupts 7207 ± 3% +15.3% 8312 ± 3% interrupts.CPU191.RES:Rescheduling_interrupts 611805 ± 2% +10.2% 673920 ± 2% interrupts.CPU2.LOC:Local_timer_interrupts 611781 ± 2% +10.2% 673897 ± 2% interrupts.CPU20.LOC:Local_timer_interrupts 7531 ± 5% +16.4% 8768 ± 3% interrupts.CPU20.RES:Rescheduling_interrupts 611715 ± 2% +10.2% 673897 ± 2% interrupts.CPU21.LOC:Local_timer_interrupts 7467 ± 3% +14.9% 8577 ± 4% interrupts.CPU21.RES:Rescheduling_interrupts 611834 ± 2% +10.1% 673900 ± 2% interrupts.CPU22.LOC:Local_timer_interrupts 7427 ± 6% +17.9% 8754 ± 4% interrupts.CPU22.RES:Rescheduling_interrupts 611788 ± 2% +10.1% 673869 ± 2% interrupts.CPU23.LOC:Local_timer_interrupts 7592 ± 4% +12.2% 8518 ± 4% interrupts.CPU23.RES:Rescheduling_interrupts 611426 ± 2% +10.2% 673619 ± 2% interrupts.CPU24.LOC:Local_timer_interrupts 7457 ± 4% +14.7% 8551 ± 3% interrupts.CPU24.RES:Rescheduling_interrupts 611439 ± 2% +10.2% 673636 ± 2% interrupts.CPU25.LOC:Local_timer_interrupts 611402 ± 2% +10.2% 673620 ± 2% interrupts.CPU26.LOC:Local_timer_interrupts 7463 ± 3% +18.4% 8838 ± 7% interrupts.CPU26.RES:Rescheduling_interrupts 611413 ± 2% +10.2% 673615 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts 7412 ± 4% +16.1% 8607 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts 611409 ± 2% +10.2% 673625 ± 2% interrupts.CPU28.LOC:Local_timer_interrupts 7478 ± 5% +15.6% 8648 ± 3% interrupts.CPU28.RES:Rescheduling_interrupts 611415 ± 2% +10.2% 673620 ± 2% interrupts.CPU29.LOC:Local_timer_interrupts 7431 ± 2% +15.5% 8585 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts 611807 ± 2% +10.1% 673896 ± 2% interrupts.CPU3.LOC:Local_timer_interrupts 7659 ± 5% +20.9% 9264 ± 10% interrupts.CPU3.RES:Rescheduling_interrupts 611433 ± 2% +10.2% 673520 ± 2% interrupts.CPU30.LOC:Local_timer_interrupts 7432 ± 3% +16.5% 8659 ± 4% interrupts.CPU30.RES:Rescheduling_interrupts 611450 ± 2% +10.2% 673604 ± 2% interrupts.CPU31.LOC:Local_timer_interrupts 7490 ± 4% +14.6% 8585 ± 5% interrupts.CPU31.RES:Rescheduling_interrupts 611444 ± 2% +10.2% 673588 ± 2% interrupts.CPU32.LOC:Local_timer_interrupts 7393 ± 4% +19.0% 8799 ± 3% interrupts.CPU32.RES:Rescheduling_interrupts 611407 ± 2% +10.2% 673606 ± 2% interrupts.CPU33.LOC:Local_timer_interrupts 7398 ± 4% +16.8% 8644 ± 4% interrupts.CPU33.RES:Rescheduling_interrupts 611452 ± 2% +10.2% 673609 ± 2% interrupts.CPU34.LOC:Local_timer_interrupts 7414 ± 5% +18.8% 8806 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts 611433 ± 2% +10.2% 673588 ± 2% interrupts.CPU35.LOC:Local_timer_interrupts 7486 ± 5% +18.6% 8878 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts 611450 ± 2% +10.2% 673603 ± 2% interrupts.CPU36.LOC:Local_timer_interrupts 7503 ± 3% +15.0% 8627 ± 3% interrupts.CPU36.RES:Rescheduling_interrupts 611465 ± 2% +10.2% 673611 ± 2% interrupts.CPU37.LOC:Local_timer_interrupts 7416 ± 3% +17.6% 8719 ± 2% interrupts.CPU37.RES:Rescheduling_interrupts 611445 ± 2% +10.2% 673575 ± 2% interrupts.CPU38.LOC:Local_timer_interrupts 7477 ± 5% +14.2% 8540 ± 3% interrupts.CPU38.RES:Rescheduling_interrupts 611420 ± 2% +10.2% 673600 ± 2% interrupts.CPU39.LOC:Local_timer_interrupts 7562 ± 5% +14.0% 8620 ± 4% interrupts.CPU39.RES:Rescheduling_interrupts 611823 ± 2% +10.2% 673949 ± 2% interrupts.CPU4.LOC:Local_timer_interrupts 7579 ± 4% +14.9% 8708 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts 611390 ± 2% +10.2% 673618 ± 2% interrupts.CPU40.LOC:Local_timer_interrupts 7444 ± 4% +15.9% 8626 ± 3% interrupts.CPU40.RES:Rescheduling_interrupts 611429 ± 2% +10.2% 673612 ± 2% interrupts.CPU41.LOC:Local_timer_interrupts 7464 ± 5% +15.3% 8605 ± 2% interrupts.CPU41.RES:Rescheduling_interrupts 611418 ± 2% +10.2% 673597 ± 2% interrupts.CPU42.LOC:Local_timer_interrupts 7394 ± 4% +18.6% 8772 ± 5% interrupts.CPU42.RES:Rescheduling_interrupts 611364 ± 2% +10.2% 673601 ± 2% interrupts.CPU43.LOC:Local_timer_interrupts 7483 ± 5% +15.3% 8630 ± 3% interrupts.CPU43.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673593 ± 2% interrupts.CPU44.LOC:Local_timer_interrupts 7463 ± 2% +16.0% 8656 ± 4% interrupts.CPU44.RES:Rescheduling_interrupts 611423 ± 2% +10.2% 673593 ± 2% interrupts.CPU45.LOC:Local_timer_interrupts 7455 ± 4% +14.8% 8558 ± 3% interrupts.CPU45.RES:Rescheduling_interrupts 611333 ± 2% +10.2% 673616 ± 2% interrupts.CPU46.LOC:Local_timer_interrupts 7406 ± 3% +17.4% 8696 ± 3% interrupts.CPU46.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673605 ± 2% interrupts.CPU47.LOC:Local_timer_interrupts 7482 ± 5% +15.7% 8660 ± 2% interrupts.CPU47.RES:Rescheduling_interrupts 610910 ± 2% +10.3% 673737 ± 2% interrupts.CPU48.LOC:Local_timer_interrupts 7521 ± 6% +15.1% 8655 ± 4% interrupts.CPU48.RES:Rescheduling_interrupts 610921 ± 2% +10.3% 673770 ± 2% interrupts.CPU49.LOC:Local_timer_interrupts 7594 ± 5% +12.7% 8558 ± 3% interrupts.CPU49.RES:Rescheduling_interrupts 611796 ± 2% +10.1% 673831 ± 2% interrupts.CPU5.LOC:Local_timer_interrupts 610930 ± 2% +10.3% 673671 ± 2% interrupts.CPU50.LOC:Local_timer_interrupts 7504 ± 2% +17.6% 8824 ± 4% interrupts.CPU50.RES:Rescheduling_interrupts 610882 ± 2% +10.3% 673738 ± 2% interrupts.CPU51.LOC:Local_timer_interrupts 610884 ± 2% +10.3% 673731 ± 2% interrupts.CPU52.LOC:Local_timer_interrupts 7436 ± 4% +18.1% 8783 ± 3% interrupts.CPU52.RES:Rescheduling_interrupts 610885 ± 2% +10.3% 673774 ± 2% interrupts.CPU53.LOC:Local_timer_interrupts 7544 ± 4% +15.3% 8700 ± 4% interrupts.CPU53.RES:Rescheduling_interrupts 610902 ± 2% +10.3% 673763 ± 2% interrupts.CPU54.LOC:Local_timer_interrupts 7534 ± 4% +15.6% 8709 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts 610927 ± 2% +10.3% 673739 ± 2% interrupts.CPU55.LOC:Local_timer_interrupts 610883 ± 2% +10.3% 673719 ± 2% interrupts.CPU56.LOC:Local_timer_interrupts 610921 ± 2% +10.3% 673689 ± 2% interrupts.CPU57.LOC:Local_timer_interrupts 7473 ± 5% +14.8% 8577 ± 4% interrupts.CPU57.RES:Rescheduling_interrupts 610893 ± 2% +10.3% 673781 ± 2% interrupts.CPU58.LOC:Local_timer_interrupts 7407 ± 5% +16.6% 8635 ± 3% interrupts.CPU58.RES:Rescheduling_interrupts 610898 ± 2% +10.3% 673763 ± 2% interrupts.CPU59.LOC:Local_timer_interrupts 7642 ± 5% +13.1% 8643 ± 3% interrupts.CPU59.RES:Rescheduling_interrupts 611823 ± 2% +10.1% 673914 ± 2% interrupts.CPU6.LOC:Local_timer_interrupts 7542 ± 3% +17.3% 8845 ± 3% interrupts.CPU6.RES:Rescheduling_interrupts 610869 ± 2% +10.3% 673784 ± 2% interrupts.CPU60.LOC:Local_timer_interrupts 7535 ± 4% +14.8% 8647 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts 610906 ± 2% +10.3% 673701 ± 2% interrupts.CPU61.LOC:Local_timer_interrupts 7631 ± 4% +13.5% 8660 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts 610863 ± 2% +10.3% 673705 ± 2% interrupts.CPU62.LOC:Local_timer_interrupts 7634 ± 4% +11.8% 8532 ± 4% interrupts.CPU62.RES:Rescheduling_interrupts 610920 ± 2% +10.3% 673730 ± 2% interrupts.CPU63.LOC:Local_timer_interrupts 7549 ± 5% +14.8% 8663 ± 5% interrupts.CPU63.RES:Rescheduling_interrupts 610884 ± 2% +10.3% 673727 ± 2% interrupts.CPU64.LOC:Local_timer_interrupts 7644 ± 3% +13.4% 8672 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts 610853 ± 2% +10.3% 673708 ± 2% interrupts.CPU65.LOC:Local_timer_interrupts 7555 ± 5% +14.8% 8672 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts 610950 ± 2% +10.3% 673804 ± 2% interrupts.CPU66.LOC:Local_timer_interrupts 7457 ± 5% +16.8% 8710 ± 4% interrupts.CPU66.RES:Rescheduling_interrupts 610888 ± 2% +10.3% 673735 ± 2% interrupts.CPU67.LOC:Local_timer_interrupts 7550 ± 4% +11.5% 8415 ± 5% interrupts.CPU67.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673727 ± 2% interrupts.CPU68.LOC:Local_timer_interrupts 7612 ± 4% +12.1% 8534 ± 3% interrupts.CPU68.RES:Rescheduling_interrupts 610901 ± 2% +10.3% 673659 ± 2% interrupts.CPU69.LOC:Local_timer_interrupts 7493 ± 4% +14.8% 8601 ± 4% interrupts.CPU69.RES:Rescheduling_interrupts 611808 ± 2% +10.1% 673844 ± 2% interrupts.CPU7.LOC:Local_timer_interrupts 7673 ± 5% +15.0% 8821 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts 610906 ± 2% +10.3% 673733 ± 2% interrupts.CPU70.LOC:Local_timer_interrupts 7412 ± 3% +16.9% 8664 ± 4% interrupts.CPU70.RES:Rescheduling_interrupts 610871 ± 2% +10.3% 673671 ± 2% interrupts.CPU71.LOC:Local_timer_interrupts 7519 ± 4% +13.8% 8557 ± 5% interrupts.CPU71.RES:Rescheduling_interrupts 611325 ± 2% +10.2% 673906 ± 2% interrupts.CPU72.LOC:Local_timer_interrupts 7720 ± 4% +9.6% 8459 ± 4% interrupts.CPU72.RES:Rescheduling_interrupts 611328 ± 2% +10.2% 673908 ± 2% interrupts.CPU73.LOC:Local_timer_interrupts 7535 ± 4% +14.6% 8634 ± 4% interrupts.CPU73.RES:Rescheduling_interrupts 611311 ± 2% +10.2% 673883 ± 2% interrupts.CPU74.LOC:Local_timer_interrupts 7526 ± 5% +14.7% 8635 ± 6% interrupts.CPU74.RES:Rescheduling_interrupts 611315 ± 2% +10.2% 673868 ± 2% interrupts.CPU75.LOC:Local_timer_interrupts 7712 ± 5% +9.8% 8469 ± 4% interrupts.CPU75.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673860 ± 2% interrupts.CPU76.LOC:Local_timer_interrupts 7476 ± 5% +14.2% 8537 ± 5% interrupts.CPU76.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673834 ± 2% interrupts.CPU77.LOC:Local_timer_interrupts 7670 ± 3% +12.5% 8630 ± 5% interrupts.CPU77.RES:Rescheduling_interrupts 611326 ± 2% +10.2% 673851 ± 2% interrupts.CPU78.LOC:Local_timer_interrupts 7417 ± 3% +15.6% 8578 ± 4% interrupts.CPU78.RES:Rescheduling_interrupts 611328 ± 2% +10.2% 673820 ± 2% interrupts.CPU79.LOC:Local_timer_interrupts 7545 ± 4% +13.9% 8590 ± 4% interrupts.CPU79.RES:Rescheduling_interrupts 611842 ± 2% +10.1% 673928 ± 2% interrupts.CPU8.LOC:Local_timer_interrupts 7673 ± 3% +14.9% 8818 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts 611366 ± 2% +10.2% 673868 ± 2% interrupts.CPU80.LOC:Local_timer_interrupts 7513 ± 3% +16.1% 8723 ± 5% interrupts.CPU80.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673860 ± 2% interrupts.CPU81.LOC:Local_timer_interrupts 611299 ± 2% +10.2% 673868 ± 2% interrupts.CPU82.LOC:Local_timer_interrupts 7523 ± 4% +13.8% 8562 ± 4% interrupts.CPU82.RES:Rescheduling_interrupts 611320 ± 2% +10.2% 673727 ± 2% interrupts.CPU83.LOC:Local_timer_interrupts 7518 ± 5% +12.4% 8454 ± 5% interrupts.CPU83.RES:Rescheduling_interrupts 611326 ± 2% +10.2% 673899 ± 2% interrupts.CPU84.LOC:Local_timer_interrupts 7452 ± 6% +12.3% 8365 ± 5% interrupts.CPU84.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673885 ± 2% interrupts.CPU85.LOC:Local_timer_interrupts 7399 ± 5% +11.1% 8218 ± 5% interrupts.CPU85.RES:Rescheduling_interrupts 611278 ± 2% +10.2% 673875 ± 2% interrupts.CPU86.LOC:Local_timer_interrupts 7376 ± 4% +16.3% 8581 ± 3% interrupts.CPU86.RES:Rescheduling_interrupts 611313 ± 2% +10.2% 673874 ± 2% interrupts.CPU87.LOC:Local_timer_interrupts 7544 ± 4% +12.8% 8508 ± 4% interrupts.CPU87.RES:Rescheduling_interrupts 611351 ± 2% +10.2% 673928 ± 2% interrupts.CPU88.LOC:Local_timer_interrupts 7418 ± 6% +13.3% 8406 ± 4% interrupts.CPU88.RES:Rescheduling_interrupts 611324 ± 2% +10.2% 673795 ± 2% interrupts.CPU89.LOC:Local_timer_interrupts 7426 ± 4% +13.6% 8439 ± 5% interrupts.CPU89.RES:Rescheduling_interrupts 611832 ± 2% +10.2% 673968 ± 2% interrupts.CPU9.LOC:Local_timer_interrupts 7538 ± 3% +15.6% 8715 ± 4% interrupts.CPU9.RES:Rescheduling_interrupts 611298 ± 2% +10.2% 673878 ± 2% interrupts.CPU90.LOC:Local_timer_interrupts 7421 ± 3% +12.7% 8363 ± 4% interrupts.CPU90.RES:Rescheduling_interrupts 611305 ± 2% +10.2% 673874 ± 2% interrupts.CPU91.LOC:Local_timer_interrupts 7377 ± 5% +15.6% 8528 ± 5% interrupts.CPU91.RES:Rescheduling_interrupts 611288 ± 2% +10.2% 673876 ± 2% interrupts.CPU92.LOC:Local_timer_interrupts 7349 ± 4% +12.0% 8234 ± 4% interrupts.CPU92.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673857 ± 2% interrupts.CPU93.LOC:Local_timer_interrupts 7438 ± 3% +14.7% 8530 ± 4% interrupts.CPU93.RES:Rescheduling_interrupts 611276 ± 2% +10.2% 673866 ± 2% interrupts.CPU94.LOC:Local_timer_interrupts 7363 ± 4% +12.7% 8300 ± 5% interrupts.CPU94.RES:Rescheduling_interrupts 611381 ± 2% +10.2% 673922 ± 2% interrupts.CPU95.LOC:Local_timer_interrupts 10549 ± 3% +14.9% 12117 ± 4% interrupts.CPU95.RES:Rescheduling_interrupts 611824 ± 2% +10.1% 673847 ± 2% interrupts.CPU96.LOC:Local_timer_interrupts 7416 ± 3% +12.9% 8375 ± 5% interrupts.CPU96.RES:Rescheduling_interrupts 611751 ± 2% +10.2% 673870 ± 2% interrupts.CPU97.LOC:Local_timer_interrupts 7437 ± 4% +14.0% 8477 ± 4% interrupts.CPU97.RES:Rescheduling_interrupts 611787 ± 2% +10.1% 673877 ± 2% interrupts.CPU98.LOC:Local_timer_interrupts 7555 ± 4% +13.0% 8534 ± 3% interrupts.CPU98.RES:Rescheduling_interrupts 611783 ± 2% +10.1% 673872 ± 2% interrupts.CPU99.LOC:Local_timer_interrupts 7452 ± 3% +12.8% 8404 ± 5% interrupts.CPU99.RES:Rescheduling_interrupts 1.174e+08 ± 2% +10.2% 1.294e+08 ± 2% interrupts.LOC:Local_timer_interrupts 1433875 ± 3% +14.6% 1643307 ± 3% interrupts.RES:Rescheduling_interrupts aim7.jobs-per-min 30500 +-------------------------------------------------------------------+ | ..+.... .| 30000 |-+ ...... ... ... | 29500 |................ ...... .. ... | | +.. ... .. | 29000 |-+ ... ... | 28500 |-+ . ... | | +. | 28000 |-+ O O | 27500 |-+ | | | 27000 |-+ | 26500 |-+ | | O | 26000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 6 months

2
3
0 / 0

[sched] c3a340f7e7: invalid_opcode:#[##]

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-4.9): commit: c3a340f7e7eadac7662ab104ceb16432e5a4c6b2 ("sched: Have sched_class_highest define by vmlinux.lds.h") https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git sched/core in testcase: trinity with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------------+------------+------------+ | | 590d697963 | c3a340f7e7 | +------------------------------------------+------------+------------+ | boot_successes | 4 | 0 | | boot_failures | 0 | 4 | | invalid_opcode:#[##] | 0 | 4 | | EIP:sched_init | 0 | 4 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 4 | | kernel_BUG_at_kernel/sched/core.c | 0 | 4 | +------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 1.813939] ** If you see this message and you are not debugging ** [ 1.814781] ** the kernel, report this immediately to your vendor! ** [ 1.815661] ** ** [ 1.816509] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 1.817362] ********************************************************** [ 1.820914] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC [ 1.821634] CPU: 0 PID: 0 Comm: swapper Not tainted 5.8.0-rc1-00024-gc3a340f7e7ead #2 [ 1.822630] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 1.823710] EIP: sched_init+0x3d/0x2fb [ 1.824191] Code: 20 a6 e5 c1 3d 40 a6 e5 c1 0f 95 c0 08 c2 75 1a b8 a0 a6 e5 c1 3d c0 a6 e5 c1 75 0e b8 20 a7 e5 c1 3d 40 a7 e5 c1 74 04 0f 0b <0f> 0b 55 89 e5 57 56 53 83 ec 08 e8 67 04 00 00 8b 3d 80 ad ec c1 [ 1.826607] EAX: c1e5a601 EBX: c1dac484 ECX: 00000000 EDX: 00000001 [ 1.827434] ESI: 00000000 EDI: 000003f8 EBP: c1e93f9c ESP: c1e93f78 [ 1.828241] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00210002 [ 1.829112] CR0: 80050033 CR2: ffc70000 CR3: 021ce000 CR4: 00000690 [ 1.829919] Call Trace: [ 1.830240] ? start_kernel+0x43f/0x677 [ 1.830738] i386_start_kernel+0x48/0x4a [ 1.831261] startup_32_smp+0x15f/0x164 [ 1.831757] Modules linked in: [ 1.832165] ---[ end trace db9d28bea02c9e7f ]--- [ 1.832761] EIP: sched_init+0x3d/0x2fb [ 1.833243] Code: 20 a6 e5 c1 3d 40 a6 e5 c1 0f 95 c0 08 c2 75 1a b8 a0 a6 e5 c1 3d c0 a6 e5 c1 75 0e b8 20 a7 e5 c1 3d 40 a7 e5 c1 74 04 0f 0b <0f> 0b 55 89 e5 57 56 53 83 ec 08 e8 67 04 00 00 8b 3d 80 ad ec c1 [ 1.835656] EAX: c1e5a601 EBX: c1dac484 ECX: 00000000 EDX: 00000001 [ 1.836463] ESI: 00000000 EDI: 000003f8 EBP: c1e93f9c ESP: c1e93f78 [ 1.838603] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00210002 [ 1.839496] CR0: 80050033 CR2: ffc70000 CR3: 021ce000 CR4: 00000690 [ 1.840300] Kernel panic - not syncing: Fatal exception [ 1.840969] ------------[ cut here ]------------ [ 1.840970] kernel BUG at kernel/sched/core.c:6652! Elapsed time: 60 qemu-img create -f qcow2 disk-vm-snb-94-0 256G qemu-img create -f qcow2 disk-vm-snb-94-1 256G To reproduce: # build kernel cd linux cp config-5.8.0-rc1-00024-gc3a340f7e7ead .config make HOSTCC=gcc-4.9 CC=gcc-4.9 ARCH=i386 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

1 year, 6 months

6
12
0 / 0

[PATCH] syscalls/ptrace10: Add new regression test

by Cyril Hrubis

New regression test for a kernel commit: commit bd14406b78e6daa1ea3c1673bda1ffc9efdeead0 Author: Jiri Olsa <jolsa(a)kernel.org> Date: Mon Aug 27 11:12:25 2018 +0200 perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled set Signed-off-by: Cyril Hrubis <chrubis(a)suse.cz> CC: Andy Lutomirski <luto(a)kernel.org> CC: Peter Zijlstra <peterz(a)infradead.org> CC: Thomas Gleixner <tglx(a)linutronix.de> CC: Alexandre Chartre <alexandre.chartre(a)oracle.com> --- This is a follow up for the ptrace08 fixes. runtest/syscalls | 1 + testcases/kernel/syscalls/ptrace/.gitignore | 1 + testcases/kernel/syscalls/ptrace/ptrace10.c | 86 +++++++++++++++++++++ 3 files changed, 88 insertions(+) create mode 100644 testcases/kernel/syscalls/ptrace/ptrace10.c diff --git a/runtest/syscalls b/runtest/syscalls index 398145f65..163471bcd 100644 --- a/runtest/syscalls +++ b/runtest/syscalls @@ -993,6 +993,7 @@ ptrace05 ptrace05 ptrace07 ptrace07 ptrace08 ptrace08 ptrace09 ptrace09 +ptrace10 ptrace10 pwrite01 pwrite01 pwrite02 pwrite02 diff --git a/testcases/kernel/syscalls/ptrace/.gitignore b/testcases/kernel/syscalls/ptrace/.gitignore index 7639e1a9f..7ee3b3c47 100644 --- a/testcases/kernel/syscalls/ptrace/.gitignore +++ b/testcases/kernel/syscalls/ptrace/.gitignore @@ -5,3 +5,4 @@ /ptrace07 /ptrace08 /ptrace09 +/ptrace10 diff --git a/testcases/kernel/syscalls/ptrace/ptrace10.c b/testcases/kernel/syscalls/ptrace/ptrace10.c new file mode 100644 index 000000000..b5d6b9f8f --- /dev/null +++ b/testcases/kernel/syscalls/ptrace/ptrace10.c @@ -0,0 +1,86 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2020 Cyril Hrubis <chrbis(a)suse.cz> + * + * After fix for CVE-2018-1000199 (see ptrace08.c) subsequent calls to POKEUSER + * for x86 debug registers were ignored silently. + * + * This is a regression test for commit: + * + * commit bd14406b78e6daa1ea3c1673bda1ffc9efdeead0 + * Author: Jiri Olsa <jolsa(a)kernel.org> + * Date: Mon Aug 27 11:12:25 2018 +0200 + * + * perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled set + */ + +#include <stdlib.h> +#include <stdio.h> +#include <stddef.h> +#include <sys/ptrace.h> +#include <sys/user.h> +#include <signal.h> +#include "tst_test.h" + +#if defined(__i386__) || defined(__x86_64__) + +static pid_t child_pid; + +static void child_main(void) +{ + raise(SIGSTOP); + exit(0); +} + +static void run(void) +{ + int status; + unsigned long addr; + + child_pid = SAFE_FORK(); + + if (!child_pid) + child_main(); + + if (SAFE_WAITPID(child_pid, &status, WUNTRACED) != child_pid) + tst_brk(TBROK, "Received event from unexpected PID"); + + SAFE_PTRACE(PTRACE_ATTACH, child_pid, NULL, NULL); + SAFE_PTRACE(PTRACE_POKEUSER, child_pid, + (void *)offsetof(struct user, u_debugreg[0]), (void *)1); + SAFE_PTRACE(PTRACE_POKEUSER, child_pid, + (void *)offsetof(struct user, u_debugreg[0]), (void *)2); + + addr = ptrace(PTRACE_PEEKUSER, child_pid, + (void*)offsetof(struct user, u_debugreg[0]), NULL); + + if (addr == 2) + tst_res(TPASS, "The rd0 was set on second PTRACE_POKEUSR"); + else + tst_res(TFAIL, "The rd0 wasn't set on second PTRACE_POKEUSER"); + + SAFE_PTRACE(PTRACE_DETACH, child_pid, NULL, NULL); + SAFE_KILL(child_pid, SIGCONT); + child_pid = 0; + tst_reap_children(); +} + +static void cleanup(void) +{ + /* Main process terminated by tst_brk() with child still paused */ + if (child_pid) + SAFE_KILL(child_pid, SIGKILL); +} + +static struct tst_test test = { + .test_all = run, + .cleanup = cleanup, + .forks_child = 1, + .tags = (const struct tst_tag[]) { + {"linux-git", "bd14406b78e6"}, + {} + } +}; +#else +TST_TEST_TCONF("This test is only supported on x86 systems"); +#endif -- 2.26.2

1 year, 7 months

2
4
0 / 0

[net] d18d22ce8f: UBSAN:Undefined_behaviour_in_net/core/dev.c

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: d18d22ce8f62839365c984b1df474d3975ed4eb2 ("net: Fix packet reordering caused by GRO and listified RX cooperation") https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y in testcase: trinity version: trinity-i386-4d2343bd-1_20200320 with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------------------------------------+------------+------------+ | | 8bc3025206 | d18d22ce8f | +------------------------------------------------------------------+------------+------------+ | boot_successes | 0 | 0 | | boot_failures | 4 | 6 | | UBSAN:Undefined_behaviour_in_arch/x86/kernel.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbprint.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbutils.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbdata.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbxface.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_kernel/workqueue.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/nsaccess.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_lib/string.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_arch/x86/kernel/alternative.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_arch/x86/realmode/init.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_crypto/algapi.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_kernel/events/core.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_kernel/exit.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_kernel/sched/cputime.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_kernel/locking/lockdep.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbxfload.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/dsmthdat.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/psargs.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/dsinit.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/evgpeinit.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/evregion.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/nsinit.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/utpredef.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/sysfs.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rsutils.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rsaddr.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rscalc.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rsmisc.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rsxface.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_lib/zlib_inflate/inffast.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_crypto/api.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_drivers/base/devres.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/linux/compiler.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/uapi/linux/swab.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/linux/unaligned/access_ok.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv4/af_inet.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/net/ip.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv4/ipconfig.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv4/ip_input.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv4/udp.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_kernel/signal.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/linux/skbuff.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_kernel/cgroup/cgroup.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv6/ip6_offload.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv6/ip6_input.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/net/ipv6.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv6/route.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv6/icmp.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/net/ip6_checksum.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv6/ndisc.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv6/addrconf_core.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv6/addrconf.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv6/ip6_fib.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_fs/readdir.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/unix/garbage.c | 4 | 4 | | UBSAN:Undefined_behaviour_in_include/linux/etherdevice.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv4/tcp_offload.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv4/tcp_ipv4.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/linux/tcp.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_include/net/dsfield.h | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv4/tcp_input.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/ipv4/tcp_minisocks.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_net/sunrpc/xprtsock.c | 4 | 6 | | UBSAN:Undefined_behaviour_in_kernel/events/hw_breakpoint.c | 4 | 4 | | UBSAN:Undefined_behaviour_in_kernel/capability.c | 4 | 4 | | UBSAN:Undefined_behaviour_in_kernel/time/time.c | 3 | | | UBSAN:Undefined_behaviour_in_kernel/sys.c | 4 | 4 | | UBSAN:Undefined_behaviour_in_fs/utimes.c | 4 | 4 | | UBSAN:Undefined_behaviour_in_kernel/futex.c | 4 | 4 | | UBSAN:Undefined_behaviour_in_include/net/ndisc.h | 4 | 4 | | UBSAN:Undefined_behaviour_in_net/ipv6/udp.c | 4 | 4 | | UBSAN:Undefined_behaviour_in_include/linux/jhash.h | 4 | 4 | | UBSAN:Undefined_behaviour_in_include/net/neighbour.h | 4 | 4 | | EIP:check_poison_obj | 1 | 2 | | EIP:rcutorture_one_extend | 1 | | | UBSAN:Undefined_behaviour_in_kernel/sched/core.c | 2 | 4 | | EIP:arch_local_irq_enable | 1 | | | EIP:delay_tsc | 2 | 1 | | EIP:arch_local_irq_restore | 2 | 2 | | EIP:kmem_cache_alloc | 1 | | | UBSAN:Undefined_behaviour_in_fs/proc/array.c | 1 | | | EIP:__copy_user_ll | 1 | | | EIP:update_stack_state | 0 | 1 | | UBSAN:Undefined_behaviour_in_net/core/dev.c | 0 | 6 | | UBSAN:Undefined_behaviour_in_fs/ioctl.c | 0 | 2 | | EIP:copy_user_highpage | 0 | 1 | | EIP:do_fault | 0 | 1 | | EIP:native_safe_halt | 0 | 2 | +------------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> [ 439.724028] ================================================================================ [ 439.724993] UBSAN: Undefined behaviour in net/core/dev.c:5144:2 [ 439.725784] member access within misaligned address ba36ed7e for type 'struct sk_buff' [ 439.726595] which requires 8 byte alignment [ 439.727042] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1 [ 439.727807] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 439.728754] Call Trace: [ 439.729035] <SOFTIRQ> [ 439.729305] dump_stack+0x16/0x18 [ 439.729673] ubsan_epilogue+0x8/0x20 [ 439.730060] ubsan_type_mismatch_common+0x9f/0xed [ 439.730564] __ubsan_handle_type_mismatch_v1+0x2c/0x32 [ 439.731118] netif_receive_skb_list_internal+0xe9/0x3fe [ 439.731677] ? e1000_clean_rx_irq+0x758/0x94d [ 439.732145] gro_normal_list+0x43/0xbf [ 439.732597] napi_complete_done+0xbe/0x1c0 [ 439.733031] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72 [ 439.733563] e1000_clean+0xf32/0x126e [ 439.733952] ? __lock_is_held+0x57/0x9e [ 439.734361] net_rx_action+0x15e/0x448 [ 439.734773] __do_softirq+0x277/0x573 [ 439.735164] ? __irqentry_text_end+0x7/0x7 [ 439.735599] do_softirq_own_stack+0x33/0x3a [ 439.736043] </SOFTIRQ> [ 439.736314] irq_exit+0x68/0xf4 [ 439.736683] do_IRQ+0xda/0xed [ 439.737009] common_interrupt+0x11b/0x120 [ 439.737435] EIP: native_safe_halt+0x12/0x13 [ 439.737932] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0 [ 439.739875] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006 [ 439.740568] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34 [ 439.741226] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246 [ 439.741939] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.742381] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.742817] ? native_safe_halt+0x12/0x13 [ 439.743246] ? default_idle+0x2c/0x37 [ 439.743642] arch_cpu_idle+0x9/0xb [ 439.744007] default_idle_call+0x1d/0x1f [ 439.744430] do_idle+0x10d/0x296 [ 439.744807] ? schedule+0x14e/0x199 [ 439.745204] cpu_startup_entry+0xd/0xf [ 439.745631] rest_init+0xd2/0xd5 [ 439.745995] arch_call_rest_init+0x8/0xb [ 439.746431] start_kernel+0x3f4/0x3fc [ 439.746859] i386_start_kernel+0x42/0x44 [ 439.747277] startup_32_smp+0x164/0x168 [ 439.747689] ================================================================================ [ 439.750463] ================================================================================ [ 439.751352] UBSAN: Undefined behaviour in net/core/dev.c:5144:2 [ 439.752186] member access within misaligned address ba36ed7e for type 'struct sk_buff' [ 439.753066] which requires 8 byte alignment [ 439.753541] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1 [ 439.754353] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 439.755198] Call Trace: [ 439.755468] <SOFTIRQ> [ 439.755736] dump_stack+0x16/0x18 [ 439.756110] ubsan_epilogue+0x8/0x20 [ 439.756537] ubsan_type_mismatch_common+0x9f/0xed [ 439.757050] __ubsan_handle_type_mismatch_v1+0x2c/0x32 [ 439.757603] netif_receive_skb_list_internal+0x51/0x3fe [ 439.758227] ? e1000_clean_rx_irq+0x758/0x94d [ 439.758691] gro_normal_list+0x43/0xbf [ 439.759097] napi_complete_done+0xbe/0x1c0 [ 439.759530] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72 [ 439.760045] e1000_clean+0xf32/0x126e [ 439.760444] ? __lock_is_held+0x57/0x9e [ 439.760871] net_rx_action+0x15e/0x448 [ 439.761259] __do_softirq+0x277/0x573 [ 439.761648] ? __irqentry_text_end+0x7/0x7 [ 439.762077] do_softirq_own_stack+0x33/0x3a [ 439.762516] </SOFTIRQ> [ 439.762787] irq_exit+0x68/0xf4 [ 439.763124] do_IRQ+0xda/0xed [ 439.763441] common_interrupt+0x11b/0x120 [ 439.763881] EIP: native_safe_halt+0x12/0x13 [ 439.764301] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0 [ 439.766225] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006 [ 439.766879] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34 [ 439.767524] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246 [ 439.768297] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.768771] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.769212] ? native_safe_halt+0x12/0x13 [ 439.769649] ? default_idle+0x2c/0x37 [ 439.770043] arch_cpu_idle+0x9/0xb [ 439.770422] default_idle_call+0x1d/0x1f [ 439.770850] do_idle+0x10d/0x296 [ 439.771200] ? schedule+0x14e/0x199 [ 439.771571] cpu_startup_entry+0xd/0xf [ 439.771970] rest_init+0xd2/0xd5 [ 439.772321] arch_call_rest_init+0x8/0xb [ 439.772763] start_kernel+0x3f4/0x3fc [ 439.773148] i386_start_kernel+0x42/0x44 [ 439.773568] startup_32_smp+0x164/0x168 [ 439.773984] ================================================================================ [ 439.774851] ================================================================================ [ 439.775731] UBSAN: Undefined behaviour in net/core/dev.c:5054:2 [ 439.776563] member access within misaligned address ba36ed7e for type 'struct sk_buff' [ 439.777424] which requires 8 byte alignment [ 439.777924] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1 [ 439.778704] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 439.779548] Call Trace: [ 439.779833] <SOFTIRQ> [ 439.780098] dump_stack+0x16/0x18 [ 439.780492] ubsan_epilogue+0x8/0x20 [ 439.780897] ubsan_type_mismatch_common+0x9f/0xed [ 439.781389] __ubsan_handle_type_mismatch_v1+0x2c/0x32 [ 439.781932] netif_receive_skb_list_internal+0x39d/0x3fe [ 439.782487] gro_normal_list+0x43/0xbf [ 439.782897] napi_complete_done+0xbe/0x1c0 [ 439.783336] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72 [ 439.783853] e1000_clean+0xf32/0x126e [ 439.784249] ? __lock_is_held+0x57/0x9e [ 439.784705] net_rx_action+0x15e/0x448 [ 439.785089] __do_softirq+0x277/0x573 [ 439.785468] ? __irqentry_text_end+0x7/0x7 [ 439.785886] do_softirq_own_stack+0x33/0x3a [ 439.786330] </SOFTIRQ> [ 439.786606] irq_exit+0x68/0xf4 [ 439.786948] do_IRQ+0xda/0xed [ 439.787268] common_interrupt+0x11b/0x120 [ 439.787698] EIP: native_safe_halt+0x12/0x13 [ 439.788194] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0 [ 439.790264] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006 [ 439.790910] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34 [ 439.791554] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246 [ 439.792262] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.792718] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.793159] ? native_safe_halt+0x12/0x13 [ 439.793585] ? default_idle+0x2c/0x37 [ 439.793979] arch_cpu_idle+0x9/0xb [ 439.794345] default_idle_call+0x1d/0x1f [ 439.794763] do_idle+0x10d/0x296 [ 439.795112] ? schedule+0x14e/0x199 [ 439.795488] cpu_startup_entry+0xd/0xf [ 439.795884] rest_init+0xd2/0xd5 [ 439.796236] arch_call_rest_init+0x8/0xb [ 439.796682] start_kernel+0x3f4/0x3fc [ 439.797074] i386_start_kernel+0x42/0x44 [ 439.797487] startup_32_smp+0x164/0x168 [ 439.797957] ================================================================================ [ 439.798825] ================================================================================ [ 439.799688] UBSAN: Undefined behaviour in net/core/dev.c:5054:2 [ 439.800526] member access within misaligned address ba36ed7e for type 'struct sk_buff' [ 439.801371] which requires 8 byte alignment [ 439.801839] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1 [ 439.802643] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 439.803535] Call Trace: [ 439.803802] <SOFTIRQ> [ 439.804066] dump_stack+0x16/0x18 [ 439.804422] ubsan_epilogue+0x8/0x20 [ 439.804829] ubsan_type_mismatch_common+0x9f/0xed [ 439.805317] __ubsan_handle_type_mismatch_v1+0x2c/0x32 [ 439.805851] netif_receive_skb_list_internal+0x208/0x3fe [ 439.806405] gro_normal_list+0x43/0xbf [ 439.806804] napi_complete_done+0xbe/0x1c0 [ 439.807253] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72 [ 439.807782] e1000_clean+0xf32/0x126e [ 439.808244] ? __lock_is_held+0x57/0x9e [ 439.808702] net_rx_action+0x15e/0x448 [ 439.809125] __do_softirq+0x277/0x573 [ 439.809513] ? __irqentry_text_end+0x7/0x7 [ 439.809944] do_softirq_own_stack+0x33/0x3a [ 439.810387] </SOFTIRQ> [ 439.810656] irq_exit+0x68/0xf4 [ 439.810998] do_IRQ+0xda/0xed [ 439.811322] common_interrupt+0x11b/0x120 [ 439.811747] EIP: native_safe_halt+0x12/0x13 [ 439.812189] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0 [ 439.814134] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006 [ 439.814799] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34 [ 439.815463] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246 [ 439.816222] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.816679] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.817226] ? native_safe_halt+0x12/0x13 [ 439.817677] ? default_idle+0x2c/0x37 [ 439.818149] arch_cpu_idle+0x9/0xb [ 439.818521] default_idle_call+0x1d/0x1f [ 439.818938] do_idle+0x10d/0x296 [ 439.819282] ? schedule+0x14e/0x199 [ 439.819648] cpu_startup_entry+0xd/0xf [ 439.820050] rest_init+0xd2/0xd5 [ 439.820409] arch_call_rest_init+0x8/0xb [ 439.820853] start_kernel+0x3f4/0x3fc [ 439.821250] i386_start_kernel+0x42/0x44 [ 439.821663] startup_32_smp+0x164/0x168 [ 439.822067] ================================================================================ [ 439.822933] ================================================================================ [ 439.823800] UBSAN: Undefined behaviour in net/core/dev.c:5000:2 [ 439.824624] member access within misaligned address ba36ed7e for type 'struct sk_buff' [ 439.825443] which requires 8 byte alignment [ 439.825891] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1 [ 439.826650] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 439.827527] Call Trace: [ 439.827800] <SOFTIRQ> [ 439.828116] dump_stack+0x16/0x18 [ 439.828520] ubsan_epilogue+0x8/0x20 [ 439.828904] ubsan_type_mismatch_common+0x9f/0xed [ 439.829397] __ubsan_handle_type_mismatch_v1+0x2c/0x32 [ 439.829943] __netif_receive_skb_list_core+0x135/0x154 [ 439.830492] netif_receive_skb_list_internal+0x3e3/0x3fe [ 439.831044] gro_normal_list+0x43/0xbf [ 439.831440] napi_complete_done+0xbe/0x1c0 [ 439.831901] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72 [ 439.832427] e1000_clean+0xf32/0x126e [ 439.832892] ? __lock_is_held+0x57/0x9e [ 439.833324] net_rx_action+0x15e/0x448 [ 439.833726] __do_softirq+0x277/0x573 [ 439.834118] ? __irqentry_text_end+0x7/0x7 [ 439.834553] do_softirq_own_stack+0x33/0x3a [ 439.834996] </SOFTIRQ> [ 439.835282] irq_exit+0x68/0xf4 [ 439.835622] do_IRQ+0xda/0xed [ 439.835964] common_interrupt+0x11b/0x120 [ 439.836395] EIP: native_safe_halt+0x12/0x13 [ 439.837047] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0 [ 439.839039] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006 [ 439.839708] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34 [ 439.840369] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246 [ 439.841078] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.841489] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.841894] ? native_safe_halt+0x12/0x13 [ 439.842306] ? default_idle+0x2c/0x37 [ 439.842678] arch_cpu_idle+0x9/0xb [ 439.843028] default_idle_call+0x1d/0x1f [ 439.843458] do_idle+0x10d/0x296 [ 439.843812] ? schedule+0x14e/0x199 [ 439.844176] cpu_startup_entry+0xd/0xf [ 439.844605] rest_init+0xd2/0xd5 [ 439.844965] arch_call_rest_init+0x8/0xb [ 439.845387] start_kernel+0x3f4/0x3fc [ 439.845789] i386_start_kernel+0x42/0x44 [ 439.846203] startup_32_smp+0x164/0x168 [ 439.846610] ================================================================================ [ 439.847484] ================================================================================ [ 439.848401] UBSAN: Undefined behaviour in net/core/dev.c:5000:2 [ 439.849226] member access within misaligned address ba36ed7e for type 'struct sk_buff' [ 439.850048] which requires 8 byte alignment [ 439.850488] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1 [ 439.851283] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 439.852106] Call Trace: [ 439.852364] <SOFTIRQ> [ 439.852688] dump_stack+0x16/0x18 [ 439.853045] ubsan_epilogue+0x8/0x20 [ 439.853427] ubsan_type_mismatch_common+0x9f/0xed [ 439.853937] __ubsan_handle_type_mismatch_v1+0x2c/0x32 [ 439.854479] __netif_receive_skb_list_core+0x69/0x154 [ 439.855021] netif_receive_skb_list_internal+0x3e3/0x3fe [ 439.855574] gro_normal_list+0x43/0xbf [ 439.855976] napi_complete_done+0xbe/0x1c0 [ 439.856413] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72 [ 439.856956] e1000_clean+0xf32/0x126e [ 439.857345] ? __lock_is_held+0x57/0x9e [ 439.857752] net_rx_action+0x15e/0x448 [ 439.858215] __do_softirq+0x277/0x573 [ 439.858615] ? __irqentry_text_end+0x7/0x7 [ 439.859046] do_softirq_own_stack+0x33/0x3a [ 439.859485] </SOFTIRQ> [ 439.859755] irq_exit+0x68/0xf4 [ 439.860095] do_IRQ+0xda/0xed [ 439.860415] common_interrupt+0x11b/0x120 [ 439.860882] EIP: native_safe_halt+0x12/0x13 [ 439.861340] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0 [ 439.863238] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006 [ 439.863888] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34 [ 439.864589] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246 [ 439.865335] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.865760] ? rwsem_mark_wake+0x2a9/0x4c2 [ 439.866134] ? native_safe_halt+0x12/0x13 [ 439.866540] ? default_idle+0x2c/0x37 [ 439.866910] arch_cpu_idle+0x9/0xb [ 439.867259] default_idle_call+0x1d/0x1f [ 439.867660] do_idle+0x10d/0x296 [ 439.868064] ? schedule+0x14e/0x199 [ 439.868447] cpu_startup_entry+0xd/0xf [ 439.868878] rest_init+0xd2/0xd5 [ 439.869230] arch_call_rest_init+0x8/0xb [ 439.869645] start_kernel+0x3f4/0x3fc [ 439.870020] i386_start_kernel+0x42/0x44 [ 439.870409] startup_32_smp+0x164/0x168 [ 439.870802] ================================================================================ [ 442.758767] sysrq: Emergency Sync [ 442.759580] sysrq: Resetting Kboot worker: lkp-worker48 Elapsed time: 480 kvm=( qemu-system-i386 -enable-kvm -cpu SandyBridge -kernel $kernel -initrd initrd-vm-snb-i386-8.cgz -m 8192 -smp 2 -device e1000,netdev=net0 -netdev user,id=net0,hostfwd=tcp::32032-:22 -boot order=nc -no-reboot -watchdog i6300esb -watchdog-action debug -rtc base=localtime -serial stdio -display none -monitor null ) append=( ip=::::vm-snb-i386-8::dhcp root=/dev/ram0 user=lkp job=/job-script ARCH=i386 kconfig=i386-randconfig-a015-20200923 branch=linux-stable-rc/linux-5.4.y commit=d18d22ce8f62839365c984b1df474d3975ed4eb2 BOOT_IMAGE=/pkg/linux/i386-randconfig-a015-20200923/gcc-9/d18d22ce8f62839365c984b1df474d3975ed4eb2/vmlinuz-5.4.15-00023-gd18d22ce8f628 max_uptime=3600 RESULT_ROOT=/result/trinity/300s/vm-snb-i386/debian-i386-20191205.cgz/i386-randconfig-a015-20200923/gcc-9/d18d22ce8f62839365c984b1df474d3975ed4eb2/3 result_service=tmpfs selinux=0 debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 net.ifnames=0 printk.devkmsg=on panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 drbd.minor_count=8 systemd.log_level=err ignore_loglevel To reproduce: # build kernel cd linux cp config-5.4.15-00023-gd18d22ce8f628 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

1 year, 7 months

3
2
0 / 0

[hugetlbfs] c0d0381ade: vm-scalability.throughput -33.4% regression

by kernel test robot

Greeting, FYI, we noticed a -33.4% regression of vm-scalability.throughput due to commit: commit: c0d0381ade79885c04a04c303284b040616b116e ("hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: vm-scalability on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory with following parameters: runtime: 300s size: 8T test: anon-cow-seq-hugetlb cpufreq_governor: performance ucode: 0x11 test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us. test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-knm01/anon-cow-seq-hugetlb/vm-scalability/0x11 commit: 49aef7175c ("mm/memblock.c: remove redundant assignment to variable max_addr") c0d0381ade ("hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization") 49aef7175cc6eb70 c0d0381ade79885c04a04c30328 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x :4 25% 1:4 dmesg.WARNING:at_ip__mutex_lock/0x :4 25% 1:4 dmesg.WARNING:at_ip_perf_event_mmap_output/0x %stddev %change %stddev \ | \ 37643 -29.5% 26549 ± 4% vm-scalability.median 9.18 ± 16% +9.3 18.48 ± 21% vm-scalability.median_stddev% 12666048 -33.4% 8429873 ± 4% vm-scalability.throughput 1701516 -19.5% 1369683 ± 5% vm-scalability.time.minor_page_faults 13447 ± 3% -19.3% 10848 vm-scalability.time.percent_of_cpu_this_job_got 26620 ± 4% -10.5% 23830 vm-scalability.time.system_time 14297 ± 3% -35.3% 9255 ± 4% vm-scalability.time.user_time 125377 ± 2% +621.3% 904348 ± 4% vm-scalability.time.voluntary_context_switches 2.468e+09 -19.6% 1.984e+09 ± 5% vm-scalability.workload 3.807e+10 ± 4% +20.9% 4.605e+10 ± 2% cpuidle.C1.time 9524 ± 10% -17.3% 7873 ± 4% numa-vmstat.node0.nr_mapped 2494252 -12.4% 2184278 ± 3% numa-numastat.node0.local_node 2494194 -12.4% 2184223 ± 3% numa-numastat.node0.numa_hit 49.26 ± 3% +9.4 58.62 mpstat.cpu.all.idle% 32.89 ± 4% -3.2 29.70 mpstat.cpu.all.sys% 17.79 ± 3% -6.2 11.64 ± 4% mpstat.cpu.all.usr% 4856 +47.6% 7169 numa-meminfo.node0.HugePages_Surp 4856 +47.6% 7169 numa-meminfo.node0.HugePages_Total 37725 ± 10% -17.7% 31054 ± 4% numa-meminfo.node0.Mapped 12274212 +38.3% 16981169 numa-meminfo.node0.MemUsed 49.00 ± 3% +18.9% 58.25 vmstat.cpu.id 17.00 ± 4% -33.8% 11.25 ± 3% vmstat.cpu.us 138.25 ± 2% -19.5% 111.25 ± 2% vmstat.procs.r 5836 ± 2% +88.0% 10969 ± 3% vmstat.system.cs 4855 +48.0% 7186 meminfo.HugePages_Surp 4856 +48.0% 7186 meminfo.HugePages_Total 9946017 +48.0% 14718559 meminfo.Hugetlb 48593 ± 8% -13.9% 41859 ± 3% meminfo.Mapped 12877361 +36.8% 17619364 meminfo.Memused 81607 ± 2% +29.1% 105324 meminfo.max_used_kB 2890 ±100% +285.4% 11139 ± 3% perf-stat.i.context-switches 298.31 ±100% +243.7% 1025 ± 2% perf-stat.i.cpu-migrations 0.11 ±100% +119.3% 0.24 perf-stat.i.ipc 0.04 ±100% +129.7% 0.10 ± 4% perf-stat.i.metric.K/sec 0.11 ±100% +122.1% 0.24 perf-stat.overall.ipc 3107 ±100% +137.3% 7372 ± 3% perf-stat.overall.path-length 2836 ±100% +284.4% 10902 ± 3% perf-stat.ps.context-switches 275.01 ±100% +257.9% 984.31 ± 2% perf-stat.ps.cpu-migrations 111986 -11.7% 98924 ± 2% slabinfo.anon_vma.active_objs 2434 -11.7% 2150 ± 2% slabinfo.anon_vma.active_slabs 111986 -11.7% 98924 ± 2% slabinfo.anon_vma.num_objs 2434 -11.7% 2150 ± 2% slabinfo.anon_vma.num_slabs 258615 -9.1% 235038 ± 2% slabinfo.anon_vma_chain.active_objs 4041 -9.1% 3672 ± 2% slabinfo.anon_vma_chain.active_slabs 258670 -9.1% 235049 ± 2% slabinfo.anon_vma_chain.num_objs 4041 -9.1% 3672 ± 2% slabinfo.anon_vma_chain.num_slabs 1500 -17.8% 1232 ± 5% slabinfo.hugetlbfs_inode_cache.active_objs 1500 -17.8% 1232 ± 5% slabinfo.hugetlbfs_inode_cache.num_objs 682.00 ± 6% +15.9% 790.50 ± 6% slabinfo.numa_policy.active_objs 682.00 ± 6% +15.9% 790.50 ± 6% slabinfo.numa_policy.num_objs 1082230 -19.6% 869890 ± 5% proc-vmstat.htlb_buddy_alloc_success 1721553 -6.8% 1604364 proc-vmstat.nr_dirty_background_threshold 3447317 -6.8% 3212652 proc-vmstat.nr_dirty_threshold 17371789 -6.8% 16197198 proc-vmstat.nr_free_pages 351.00 +3.8% 364.25 proc-vmstat.nr_inactive_file 39705 +1.4% 40269 proc-vmstat.nr_kernel_stack 12252 ± 7% -13.7% 10570 ± 3% proc-vmstat.nr_mapped 2632 +8.8% 2863 proc-vmstat.nr_page_table_pages 98880 -1.6% 97341 proc-vmstat.nr_slab_unreclaimable 351.00 +3.8% 364.25 proc-vmstat.nr_zone_inactive_file 6285 ± 50% -90.8% 577.50 ± 47% proc-vmstat.numa_hint_faults 6285 ± 50% -90.8% 577.50 ± 47% proc-vmstat.numa_hint_faults_local 2514799 -12.3% 2205532 ± 3% proc-vmstat.numa_hit 2514798 -12.3% 2205531 ± 3% proc-vmstat.numa_local 5.557e+08 -19.6% 4.469e+08 ± 5% proc-vmstat.pgalloc_normal 2510883 -12.9% 2187221 ± 3% proc-vmstat.pgfault 5.553e+08 -19.5% 4.468e+08 ± 5% proc-vmstat.pgfree 60107 ± 2% -19.7% 48269 ± 17% sched_debug.cfs_rq:/.exec_clock.avg 71511 ± 2% -23.4% 54794 ± 16% sched_debug.cfs_rq:/.exec_clock.max 44464 ± 7% -16.9% 36954 ± 18% sched_debug.cfs_rq:/.exec_clock.min 4619 ± 4% -47.8% 2412 ± 11% sched_debug.cfs_rq:/.exec_clock.stddev 614.00 ± 10% +371.9% 2897 ±124% sched_debug.cfs_rq:/.load_avg.max 49.58 ± 11% +321.1% 208.79 ±123% sched_debug.cfs_rq:/.load_avg.stddev 11526482 ± 4% -37.6% 7189313 ± 17% sched_debug.cfs_rq:/.min_vruntime.avg 13399855 ± 4% -40.0% 8039878 ± 15% sched_debug.cfs_rq:/.min_vruntime.max 8425626 ± 8% -37.2% 5294770 ± 18% sched_debug.cfs_rq:/.min_vruntime.min 788640 ± 4% -56.4% 343805 ± 11% sched_debug.cfs_rq:/.min_vruntime.stddev 0.37 ± 8% -40.0% 0.22 ± 10% sched_debug.cfs_rq:/.nr_running.avg 21.17 ± 5% +134.9% 49.72 ± 21% sched_debug.cfs_rq:/.nr_spread_over.avg 56.15 ± 13% +60.7% 90.25 ± 19% sched_debug.cfs_rq:/.nr_spread_over.max 6.95 ± 7% +289.4% 27.06 ± 23% sched_debug.cfs_rq:/.nr_spread_over.min 7.32 ± 5% +39.4% 10.20 ± 19% sched_debug.cfs_rq:/.nr_spread_over.stddev 413.38 ± 11% -27.5% 299.85 ± 14% sched_debug.cfs_rq:/.runnable_avg.avg 834.44 ± 53% -65.7% 286.48 ± 5% sched_debug.cfs_rq:/.runnable_avg.stddev 2625208 ± 11% -29.7% 1845657 ± 13% sched_debug.cfs_rq:/.spread0.avg 4483653 ± 6% -39.8% 2699533 ± 11% sched_debug.cfs_rq:/.spread0.max -475138 -94.3% -26940 sched_debug.cfs_rq:/.spread0.min 801419 ± 4% -56.8% 345835 ± 11% sched_debug.cfs_rq:/.spread0.stddev 371.84 ± 8% -22.5% 288.10 ± 14% sched_debug.cfs_rq:/.util_avg.avg 382.64 ± 4% -32.5% 258.09 ± 4% sched_debug.cfs_rq:/.util_avg.stddev 330.75 ± 10% -54.8% 149.57 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.avg 358.21 ± 6% -27.6% 259.25 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev 4047967 ± 4% +17.6% 4760305 ± 4% sched_debug.cpu.avg_idle.avg 561715 ± 52% +116.4% 1215479 ± 7% sched_debug.cpu.avg_idle.min 731.80 ± 6% -38.6% 449.23 ± 8% sched_debug.cpu.clock.stddev 731.80 ± 6% -38.6% 449.23 ± 8% sched_debug.cpu.clock_task.stddev 4741 ± 13% -43.0% 2702 ± 18% sched_debug.cpu.curr->pid.avg 5519 ± 7% -15.8% 4650 ± 11% sched_debug.cpu.curr->pid.stddev 2119875 ± 5% +16.3% 2465873 ± 4% sched_debug.cpu.max_idle_balance_cost.avg 1107931 ± 2% +24.6% 1380330 ± 3% sched_debug.cpu.max_idle_balance_cost.min 581556 ± 5% +11.1% 646020 ± 5% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 ± 4% -30.8% 0.00 ± 11% sched_debug.cpu.next_balance.stddev 0.31 ± 12% -32.6% 0.21 ± 10% sched_debug.cpu.nr_running.avg 5116 +46.1% 7477 ± 11% sched_debug.cpu.nr_switches.avg 2301 ± 5% +85.0% 4257 ± 22% sched_debug.cpu.nr_switches.min 0.09 ± 44% +300.2% 0.34 ± 17% sched_debug.cpu.nr_uninterruptible.avg 296.40 ± 3% +24.1% 367.95 ± 8% sched_debug.cpu.nr_uninterruptible.max -50.40 +38.8% -69.98 sched_debug.cpu.nr_uninterruptible.min 23.04 ± 3% +53.3% 35.32 ± 12% sched_debug.cpu.nr_uninterruptible.stddev 2343 +98.2% 4646 ± 18% sched_debug.cpu.sched_count.avg 1110 ± 2% +193.0% 3252 ± 23% sched_debug.cpu.sched_count.min 913.60 ± 2% +125.6% 2060 ± 18% sched_debug.cpu.sched_goidle.avg 356.95 +296.7% 1416 ± 25% sched_debug.cpu.sched_goidle.min 1148 +102.6% 2326 ± 18% sched_debug.cpu.ttwu_count.avg 510.15 ± 5% +80.8% 922.26 ± 21% sched_debug.cpu.ttwu_count.min 148.25 ± 6% -11.3% 131.52 ± 6% sched_debug.cpu.ttwu_local.stddev 21.96 ± 6% -10.0 11.99 ± 11% perf-profile.calltrace.cycles-pp.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault 19.05 ± 5% -8.1 10.96 ± 11% perf-profile.calltrace.cycles-pp.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault 18.72 ± 5% -8.0 10.76 ± 11% perf-profile.calltrace.cycles-pp.copy_subpage.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault 17.82 ± 4% -7.6 10.17 ± 10% perf-profile.calltrace.cycles-pp.copy_page.copy_subpage.copy_user_huge_page.hugetlb_cow.hugetlb_fault 2.22 ± 44% -1.5 0.68 ± 23% perf-profile.calltrace.cycles-pp.alloc_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault 1.03 ± 22% -0.6 0.40 ±102% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.do_access 0.91 ± 23% -0.6 0.32 ±103% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.do_access 0.81 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit 0.81 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit 0.80 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput 0.79 ± 19% -0.2 0.59 ± 15% perf-profile.calltrace.cycles-pp.__free_huge_page.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap 1.44 ± 22% +0.6 2.00 ± 28% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 0.71 ± 80% +1.5 2.16 ± 33% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock 1.03 ± 55% +1.6 2.67 ± 32% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock.__mutex_lock 1.24 ± 57% +2.0 3.29 ± 26% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock.__mutex_lock.hugetlb_fault 1.38 ± 55% +2.3 3.66 ± 25% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.osq_lock.__mutex_lock.hugetlb_fault.handle_mm_fault 29.76 ± 21% +20.1 49.83 ± 13% perf-profile.calltrace.cycles-pp.__mutex_lock.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault 15.71 ± 27% +23.2 38.88 ± 18% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.hugetlb_fault.handle_mm_fault.do_page_fault 21.96 ± 6% -10.0 12.00 ± 11% perf-profile.children.cycles-pp.hugetlb_cow 19.36 ± 5% -8.2 11.11 ± 11% perf-profile.children.cycles-pp.copy_user_huge_page 18.74 ± 5% -8.0 10.77 ± 11% perf-profile.children.cycles-pp.copy_subpage 18.31 ± 5% -7.8 10.54 ± 11% perf-profile.children.cycles-pp.copy_page 16.95 ± 20% -7.0 10.00 ± 53% perf-profile.children.cycles-pp.do_rw_once 2.46 ± 41% -1.7 0.80 ± 14% perf-profile.children.cycles-pp.alloc_huge_page 3.11 ± 36% -1.6 1.48 ± 4% perf-profile.children.cycles-pp._raw_spin_lock 2.64 ± 44% -1.5 1.10 ± 6% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 1.60 ± 38% -1.0 0.57 ± 14% perf-profile.children.cycles-pp.alloc_surplus_huge_page 1.81 ± 12% -0.5 1.34 ± 4% perf-profile.children.cycles-pp.do_syscall_64 1.81 ± 12% -0.5 1.35 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.76 ± 28% -0.4 0.38 ± 12% perf-profile.children.cycles-pp.alloc_fresh_huge_page 1.21 ± 10% -0.4 0.85 ± 11% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.40 ± 52% -0.3 0.09 ± 20% perf-profile.children.cycles-pp.prep_new_huge_page 0.81 ± 19% -0.2 0.60 ± 14% perf-profile.children.cycles-pp.tlb_finish_mmu 0.81 ± 19% -0.2 0.60 ± 14% perf-profile.children.cycles-pp.tlb_flush_mmu 0.80 ± 18% -0.2 0.60 ± 15% perf-profile.children.cycles-pp.release_pages 0.79 ± 19% -0.2 0.59 ± 15% perf-profile.children.cycles-pp.__free_huge_page 0.44 ± 13% -0.2 0.26 ± 14% perf-profile.children.cycles-pp.___might_sleep 0.20 ± 14% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.update_and_free_page 0.32 ± 12% -0.1 0.21 ± 6% perf-profile.children.cycles-pp.ksys_read 0.29 ± 13% -0.1 0.18 ± 10% perf-profile.children.cycles-pp.read 0.31 ± 11% -0.1 0.20 ± 8% perf-profile.children.cycles-pp.vfs_read 0.18 ± 17% -0.1 0.09 ± 8% perf-profile.children.cycles-pp._cond_resched 0.16 ± 9% -0.1 0.09 ± 17% perf-profile.children.cycles-pp.__libc_start_main 0.16 ± 11% -0.1 0.08 ± 17% perf-profile.children.cycles-pp.main 0.19 ± 14% -0.1 0.12 ± 17% perf-profile.children.cycles-pp.x86_pmu_disable 0.26 ± 8% -0.1 0.19 ± 6% perf-profile.children.cycles-pp.ksys_write 0.26 ± 9% -0.1 0.19 ± 8% perf-profile.children.cycles-pp.vfs_write 0.15 ± 12% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.new_sync_read 0.15 ± 11% -0.1 0.09 ± 17% perf-profile.children.cycles-pp.pipe_read 0.24 ± 9% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.new_sync_write 0.08 ± 6% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__perf_sw_event 0.11 ± 14% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.__wake_up_common 0.12 ± 3% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.schedule_timeout 0.12 ± 13% -0.0 0.07 ± 17% perf-profile.children.cycles-pp.__wake_up_common_lock 0.10 ± 15% -0.0 0.06 ± 15% perf-profile.children.cycles-pp.autoremove_wake_function 0.11 ± 29% -0.0 0.07 ± 10% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.26 ± 6% -0.0 0.22 ± 5% perf-profile.children.cycles-pp.tick_program_event 0.09 ± 31% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.09 ± 9% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.rcu_gp_kthread 0.08 ± 10% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.cmd_stat 0.01 ±173% +0.0 0.06 ± 11% perf-profile.children.cycles-pp.select_task_rq_fair 0.11 ± 11% +0.0 0.16 ± 9% perf-profile.children.cycles-pp.finish_task_switch 0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.rwsem_wake 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.arch_stack_walk 0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.dequeue_task_fair 0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.stack_trace_save_tsk 0.13 ± 6% +0.1 0.21 ± 10% perf-profile.children.cycles-pp.schedule_idle 0.06 ± 11% +0.1 0.14 ± 10% perf-profile.children.cycles-pp.enqueue_entity 0.03 ±100% +0.1 0.11 ± 7% perf-profile.children.cycles-pp.update_load_avg 0.00 +0.1 0.09 ± 12% perf-profile.children.cycles-pp.__account_scheduler_latency 0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.activate_task 0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.ttwu_do_activate 0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.enqueue_task_fair 0.00 +0.1 0.09 ± 8% perf-profile.children.cycles-pp.wake_up_q 0.14 ± 24% +0.1 0.27 ± 13% perf-profile.children.cycles-pp.schedule_preempt_disabled 0.00 +0.1 0.14 ± 15% perf-profile.children.cycles-pp.sched_ttwu_pending 1.17 ± 13% +0.4 1.56 ± 8% perf-profile.children.cycles-pp.load_balance 0.99 ± 11% +0.4 1.41 ± 9% perf-profile.children.cycles-pp.find_busiest_group 0.96 ± 11% +0.4 1.38 ± 9% perf-profile.children.cycles-pp.update_sd_lb_stats 0.66 ± 9% +0.5 1.20 ± 11% perf-profile.children.cycles-pp.newidle_balance 0.53 ± 7% +0.6 1.11 ± 12% perf-profile.children.cycles-pp.pick_next_task_fair 0.71 ± 8% +0.6 1.35 ± 10% perf-profile.children.cycles-pp.schedule 0.00 +0.7 0.68 ± 17% perf-profile.children.cycles-pp.rwsem_down_read_slowpath 0.86 ± 7% +0.7 1.58 ± 9% perf-profile.children.cycles-pp.__schedule 52.14 ± 14% +10.7 62.86 ± 9% perf-profile.children.cycles-pp.page_fault 52.09 ± 14% +10.7 62.83 ± 9% perf-profile.children.cycles-pp.do_page_fault 51.96 ± 14% +10.8 62.74 ± 9% perf-profile.children.cycles-pp.handle_mm_fault 51.86 ± 14% +10.8 62.67 ± 9% perf-profile.children.cycles-pp.hugetlb_fault 29.77 ± 21% +20.1 49.84 ± 13% perf-profile.children.cycles-pp.__mutex_lock 15.83 ± 27% +23.4 39.18 ± 18% perf-profile.children.cycles-pp.osq_lock 16.53 ± 4% -7.1 9.48 ± 9% perf-profile.self.cycles-pp.copy_page 15.27 ± 22% -6.4 8.90 ± 53% perf-profile.self.cycles-pp.do_rw_once 2.38 ± 42% -1.4 1.02 ± 7% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.70 ± 9% -0.2 0.48 ± 14% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler 0.38 ± 2% -0.2 0.21 ± 11% perf-profile.self.cycles-pp.copy_subpage 0.33 ± 4% -0.2 0.17 ± 11% perf-profile.self.cycles-pp.copy_user_huge_page 0.18 ± 18% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.update_and_free_page 0.19 ± 14% -0.1 0.11 ± 18% perf-profile.self.cycles-pp.x86_pmu_disable 0.19 ± 6% -0.1 0.12 ± 10% perf-profile.self.cycles-pp.___might_sleep 0.35 ± 14% -0.1 0.28 ± 8% perf-profile.self.cycles-pp.run_timer_softirq 0.18 ± 10% -0.1 0.12 ± 6% perf-profile.self.cycles-pp.get_page_from_freelist 0.26 ± 6% -0.0 0.22 ± 5% perf-profile.self.cycles-pp.tick_program_event 0.07 -0.0 0.06 ± 7% perf-profile.self.cycles-pp.try_to_wake_up 0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.update_load_avg 0.73 ± 12% +0.3 1.06 ± 8% perf-profile.self.cycles-pp.update_sd_lb_stats 14.37 ± 25% +21.2 35.60 ± 18% perf-profile.self.cycles-pp.osq_lock 22773 ± 3% +11.6% 25420 ± 2% softirqs.CPU0.SCHED 15462 ± 6% +24.5% 19248 ± 5% softirqs.CPU1.SCHED 8331 ± 11% +27.1% 10590 ± 11% softirqs.CPU10.RCU 14783 ± 4% +16.0% 17145 ± 4% softirqs.CPU10.SCHED 14236 ± 3% +15.6% 16462 ± 2% softirqs.CPU100.SCHED 14117 ± 3% +18.2% 16683 ± 3% softirqs.CPU101.SCHED 13918 ± 4% +18.4% 16472 ± 5% softirqs.CPU102.SCHED 14183 ± 6% +18.0% 16742 ± 3% softirqs.CPU103.SCHED 13795 ± 5% +19.8% 16528 softirqs.CPU104.SCHED 14326 ± 6% +17.7% 16859 softirqs.CPU105.SCHED 13842 ± 5% +19.4% 16530 ± 2% softirqs.CPU106.SCHED 14155 ± 4% +20.0% 16983 ± 2% softirqs.CPU107.SCHED 13907 ± 3% +18.6% 16492 ± 2% softirqs.CPU108.SCHED 14411 ± 7% +16.1% 16726 ± 2% softirqs.CPU109.SCHED 14354 +19.0% 17085 ± 2% softirqs.CPU11.SCHED 13689 ± 6% +21.8% 16667 ± 3% softirqs.CPU110.SCHED 13461 ± 6% +25.6% 16907 ± 3% softirqs.CPU111.SCHED 13738 ± 9% +23.2% 16925 softirqs.CPU112.SCHED 14135 ± 3% +20.5% 17032 softirqs.CPU113.SCHED 14477 ± 4% +20.8% 17491 ± 4% softirqs.CPU114.SCHED 123003 ± 2% -7.8% 113426 ± 2% softirqs.CPU114.TIMER 14014 ± 4% +23.4% 17299 softirqs.CPU115.SCHED 13528 ± 6% +24.4% 16824 ± 2% softirqs.CPU116.SCHED 13530 ± 6% +24.5% 16840 ± 2% softirqs.CPU117.SCHED 14263 ± 3% +16.1% 16557 ± 2% softirqs.CPU119.SCHED 7902 ± 2% +22.1% 9648 ± 6% softirqs.CPU12.RCU 14332 ± 3% +19.5% 17130 ± 4% softirqs.CPU12.SCHED 13476 ± 5% +26.5% 17047 softirqs.CPU120.SCHED 14240 ± 6% +20.7% 17183 softirqs.CPU121.SCHED 13161 ± 5% +27.0% 16708 ± 2% softirqs.CPU122.SCHED 13037 ± 7% +28.8% 16786 softirqs.CPU123.SCHED 14229 ± 5% +16.5% 16577 ± 3% softirqs.CPU124.SCHED 14427 ± 4% +15.4% 16652 softirqs.CPU125.SCHED 14426 ± 4% +16.3% 16771 softirqs.CPU126.SCHED 6680 +39.3% 9305 ± 22% softirqs.CPU127.RCU 13659 ± 3% +23.7% 16890 ± 3% softirqs.CPU128.SCHED 14396 ± 5% +17.9% 16979 ± 3% softirqs.CPU129.SCHED 14183 ± 7% +20.2% 17048 ± 2% softirqs.CPU13.SCHED 6700 ± 2% +35.8% 9097 ± 17% softirqs.CPU130.RCU 13949 ± 4% +22.2% 17051 ± 4% softirqs.CPU130.SCHED 14271 ± 4% +21.8% 17379 ± 2% softirqs.CPU131.SCHED 129315 ± 3% -7.0% 120290 softirqs.CPU131.TIMER 14135 ± 2% +16.0% 16396 ± 2% softirqs.CPU132.SCHED 13567 ± 4% +22.5% 16617 ± 3% softirqs.CPU133.SCHED 13912 ± 5% +20.1% 16712 ± 2% softirqs.CPU134.SCHED 14309 ± 5% +18.9% 17016 ± 2% softirqs.CPU135.SCHED 13637 ± 4% +20.3% 16402 ± 5% softirqs.CPU136.SCHED 13843 ± 4% +19.3% 16518 ± 2% softirqs.CPU137.SCHED 14049 ± 5% +19.2% 16750 ± 2% softirqs.CPU138.SCHED 14244 ± 7% +18.1% 16816 softirqs.CPU139.SCHED 14139 +22.1% 17271 ± 2% softirqs.CPU14.SCHED 13460 ± 8% +22.9% 16546 softirqs.CPU140.SCHED 13645 ± 6% +21.7% 16601 ± 2% softirqs.CPU141.SCHED 13278 ± 6% +25.5% 16661 ± 3% softirqs.CPU142.SCHED 12487 ± 5% +29.2% 16136 ± 2% softirqs.CPU143.SCHED 13440 ± 9% +22.8% 16502 softirqs.CPU144.SCHED 126064 ± 2% -8.1% 115836 softirqs.CPU144.TIMER 13995 ± 6% +21.2% 16966 ± 3% softirqs.CPU145.SCHED 13834 ± 3% +14.8% 15883 ± 6% softirqs.CPU146.SCHED 14297 ± 9% +17.1% 16735 softirqs.CPU147.SCHED 14500 ± 3% +15.2% 16711 softirqs.CPU148.SCHED 14419 ± 6% +17.5% 16945 softirqs.CPU149.SCHED 8018 ± 6% +24.4% 9977 ± 7% softirqs.CPU15.RCU 13832 ± 5% +28.2% 17737 ± 5% softirqs.CPU15.SCHED 14538 ± 5% +15.0% 16723 ± 2% softirqs.CPU150.SCHED 15092 +9.5% 16526 ± 4% softirqs.CPU152.SCHED 14533 ± 2% +15.6% 16804 softirqs.CPU153.SCHED 14415 ± 2% +14.8% 16554 ± 2% softirqs.CPU154.SCHED 14583 ± 2% +15.4% 16833 ± 2% softirqs.CPU155.SCHED 14851 ± 2% +11.8% 16598 ± 2% softirqs.CPU156.SCHED 6747 +35.2% 9124 ± 19% softirqs.CPU158.RCU 13839 ± 4% +25.9% 17429 ± 4% softirqs.CPU158.SCHED 15020 ± 5% +11.8% 16785 ± 2% softirqs.CPU159.SCHED 13468 ± 5% +25.8% 16945 ± 2% softirqs.CPU16.SCHED 128305 ± 3% -8.1% 117855 softirqs.CPU160.TIMER 15111 ± 2% +13.7% 17184 softirqs.CPU162.SCHED 14788 ± 2% +12.8% 16685 ± 3% softirqs.CPU163.SCHED 13903 ± 5% +21.5% 16889 softirqs.CPU164.SCHED 14547 ± 5% +13.5% 16510 ± 2% softirqs.CPU165.SCHED 14440 ± 4% +13.3% 16354 ± 3% softirqs.CPU166.SCHED 14450 ± 5% +14.1% 16494 ± 2% softirqs.CPU167.SCHED 131454 ± 3% -6.9% 122396 ± 2% softirqs.CPU167.TIMER 13683 ± 6% +23.0% 16824 ± 3% softirqs.CPU168.SCHED 14468 ± 5% +18.1% 17085 ± 3% softirqs.CPU169.SCHED 7703 +22.5% 9437 ± 4% softirqs.CPU17.RCU 14392 +19.8% 17237 softirqs.CPU17.SCHED 14570 ± 6% +16.9% 17038 ± 3% softirqs.CPU170.SCHED 14172 ± 5% +19.3% 16905 softirqs.CPU171.SCHED 14176 ± 3% +17.9% 16709 softirqs.CPU172.SCHED 127159 ± 2% -8.4% 116497 ± 2% softirqs.CPU172.TIMER 15034 ± 2% +11.1% 16698 ± 2% softirqs.CPU174.SCHED 14923 ± 8% +17.0% 17458 ± 5% softirqs.CPU175.SCHED 14164 ± 5% +18.8% 16833 ± 2% softirqs.CPU176.SCHED 127027 ± 2% -8.4% 116378 ± 2% softirqs.CPU176.TIMER 14359 ± 3% +16.9% 16788 ± 4% softirqs.CPU177.SCHED 14466 ± 3% +18.7% 17169 softirqs.CPU178.SCHED 14632 ± 2% +15.1% 16837 softirqs.CPU179.SCHED 123562 ± 3% -8.2% 113392 ± 2% softirqs.CPU179.TIMER 7677 +27.6% 9794 ± 2% softirqs.CPU18.RCU 14177 ± 2% +21.6% 17233 softirqs.CPU18.SCHED 14429 ± 7% +15.8% 16707 softirqs.CPU180.SCHED 15194 ± 5% +12.4% 17084 softirqs.CPU181.SCHED 13917 ± 3% +20.2% 16723 ± 3% softirqs.CPU182.SCHED 14587 ± 2% +15.1% 16786 ± 3% softirqs.CPU183.SCHED 14266 ± 3% +18.0% 16827 ± 2% softirqs.CPU184.SCHED 14661 ± 3% +16.3% 17053 softirqs.CPU185.SCHED 14222 ± 5% +21.0% 17202 ± 3% softirqs.CPU186.SCHED 15059 ± 2% +13.0% 17022 ± 3% softirqs.CPU187.SCHED 125388 ± 2% -7.6% 115887 ± 2% softirqs.CPU187.TIMER 13435 ± 4% +25.0% 16801 ± 3% softirqs.CPU188.SCHED 13434 ± 2% +28.8% 17297 ± 2% softirqs.CPU189.SCHED 7719 +27.0% 9806 ± 9% softirqs.CPU19.RCU 14075 ± 2% +20.7% 16986 softirqs.CPU19.SCHED 13735 ± 3% +20.4% 16536 ± 2% softirqs.CPU190.SCHED 13772 ± 3% +19.8% 16494 softirqs.CPU192.SCHED 13911 ± 2% +17.4% 16327 ± 2% softirqs.CPU193.SCHED 6417 +34.3% 8616 ± 17% softirqs.CPU194.RCU 14008 +17.6% 16468 ± 8% softirqs.CPU194.SCHED 14094 ± 4% +19.9% 16905 softirqs.CPU195.SCHED 14387 ± 7% +16.1% 16698 ± 5% softirqs.CPU196.SCHED 13704 ± 3% +23.7% 16952 softirqs.CPU198.SCHED 8214 ± 10% +23.7% 10157 ± 7% softirqs.CPU2.RCU 14431 +17.4% 16946 ± 4% softirqs.CPU2.SCHED 7770 +30.4% 10135 ± 8% softirqs.CPU20.RCU 13996 ± 2% +23.7% 17320 softirqs.CPU20.SCHED 14611 ± 5% +12.7% 16463 ± 3% softirqs.CPU200.SCHED 14897 ± 3% +15.5% 17199 softirqs.CPU201.SCHED 14105 ± 6% +21.7% 17167 ± 2% softirqs.CPU202.SCHED 127698 ± 3% -7.1% 118685 softirqs.CPU202.TIMER 14369 ± 6% +18.6% 17048 ± 2% softirqs.CPU203.SCHED 126668 ± 3% -8.8% 115539 softirqs.CPU203.TIMER 14461 ± 4% +13.1% 16356 ± 3% softirqs.CPU205.SCHED 13139 ± 10% +26.6% 16634 ± 3% softirqs.CPU206.SCHED 14808 ± 2% +11.8% 16550 ± 2% softirqs.CPU207.SCHED 13744 ± 5% +21.0% 16624 ± 4% softirqs.CPU208.SCHED 14039 ± 5% +18.7% 16661 ± 2% softirqs.CPU209.SCHED 127488 ± 2% -8.2% 116998 softirqs.CPU209.TIMER 7755 ± 2% +30.5% 10122 ± 12% softirqs.CPU21.RCU 13731 ± 5% +25.2% 17185 ± 2% softirqs.CPU21.SCHED 14755 ± 6% +16.1% 17123 ± 4% softirqs.CPU211.SCHED 13920 ± 3% +18.5% 16501 ± 2% softirqs.CPU212.SCHED 14435 ± 4% +16.1% 16765 ± 3% softirqs.CPU213.SCHED 13857 ± 6% +21.6% 16844 ± 2% softirqs.CPU214.SCHED 13417 ± 8% +16.5% 15635 ± 2% softirqs.CPU215.SCHED 127636 ± 3% -7.4% 118224 softirqs.CPU215.TIMER 13715 ± 3% +21.7% 16686 ± 3% softirqs.CPU216.SCHED 126421 ± 2% -7.8% 116544 softirqs.CPU216.TIMER 13885 ± 5% +18.2% 16416 softirqs.CPU217.SCHED 14003 ± 8% +20.5% 16875 softirqs.CPU218.SCHED 13891 ± 4% +23.9% 17210 ± 4% softirqs.CPU22.SCHED 14901 ± 2% +12.0% 16694 ± 2% softirqs.CPU220.SCHED 14623 ± 6% +17.3% 17153 softirqs.CPU221.SCHED 14613 ± 8% +13.9% 16643 ± 2% softirqs.CPU224.SCHED 14745 ± 4% +12.9% 16643 softirqs.CPU225.SCHED 126232 -8.3% 115717 softirqs.CPU225.TIMER 14290 ± 2% +17.2% 16753 softirqs.CPU226.SCHED 15229 ± 3% +10.3% 16800 ± 3% softirqs.CPU228.SCHED 14282 ± 6% +15.4% 16479 ± 3% softirqs.CPU23.SCHED 13785 ± 9% +24.2% 17126 ± 4% softirqs.CPU230.SCHED 126791 ± 4% -9.4% 114889 ± 3% softirqs.CPU231.TIMER 14749 ± 3% +15.0% 16958 ± 3% softirqs.CPU232.SCHED 15099 ± 5% +11.9% 16893 ± 2% softirqs.CPU233.SCHED 14825 ± 6% +12.9% 16738 softirqs.CPU234.SCHED 14272 ± 5% +18.7% 16945 ± 2% softirqs.CPU236.SCHED 15187 ± 2% +10.9% 16846 softirqs.CPU237.SCHED 15267 ± 7% +11.1% 16955 ± 3% softirqs.CPU239.SCHED 14156 ± 4% +16.3% 16466 ± 3% softirqs.CPU24.SCHED 14671 ± 3% +13.9% 16711 ± 3% softirqs.CPU240.SCHED 127318 ± 2% -7.6% 117651 ± 2% softirqs.CPU240.TIMER 14218 ± 5% +20.2% 17097 softirqs.CPU241.SCHED 14554 ± 3% +15.2% 16771 softirqs.CPU242.SCHED 14003 +19.9% 16785 softirqs.CPU243.SCHED 13761 ± 6% +21.5% 16724 ± 2% softirqs.CPU244.SCHED 14418 ± 6% +16.4% 16781 softirqs.CPU245.SCHED 14760 ± 7% +14.8% 16940 ± 4% softirqs.CPU246.SCHED 14891 ± 3% +12.3% 16722 ± 4% softirqs.CPU249.SCHED 13823 ± 4% +24.1% 17148 softirqs.CPU25.SCHED 14201 ± 4% +18.1% 16774 ± 3% softirqs.CPU250.SCHED 14620 ± 10% +18.5% 17331 softirqs.CPU251.SCHED 14877 ± 3% +11.3% 16564 ± 2% softirqs.CPU252.SCHED 14517 ± 6% +16.3% 16877 ± 3% softirqs.CPU254.SCHED 14284 ± 7% +18.0% 16858 ± 4% softirqs.CPU255.SCHED 121538 ± 3% -7.5% 112473 softirqs.CPU255.TIMER 14582 +14.8% 16736 softirqs.CPU256.SCHED 14057 ± 3% +22.8% 17259 softirqs.CPU257.SCHED 15076 ± 3% +13.4% 17097 ± 3% softirqs.CPU258.SCHED 14356 ± 3% +20.2% 17251 ± 3% softirqs.CPU259.SCHED 13998 ± 3% +20.1% 16806 ± 2% softirqs.CPU26.SCHED 13899 ± 4% +19.0% 16534 ± 5% softirqs.CPU260.SCHED 128277 ± 4% -7.4% 118758 softirqs.CPU260.TIMER 14288 ± 5% +20.7% 17251 softirqs.CPU261.SCHED 14456 ± 5% +15.1% 16644 softirqs.CPU262.SCHED 14114 ± 2% +17.8% 16624 ± 4% softirqs.CPU263.SCHED 13328 ± 4% +25.0% 16657 softirqs.CPU264.SCHED 13328 ± 4% +25.1% 16676 ± 4% softirqs.CPU266.SCHED 14367 ± 3% +19.5% 17165 ± 2% softirqs.CPU267.SCHED 122663 ± 3% -7.6% 113293 ± 2% softirqs.CPU268.TIMER 13527 ± 6% +25.2% 16940 softirqs.CPU27.SCHED 14341 ± 5% +19.8% 17187 softirqs.CPU270.SCHED 14345 ± 5% +14.3% 16403 ± 5% softirqs.CPU271.SCHED 14990 ± 3% +14.4% 17152 ± 4% softirqs.CPU272.SCHED 15007 ± 2% +13.4% 17012 softirqs.CPU273.SCHED 14668 ± 8% +17.0% 17157 ± 4% softirqs.CPU274.SCHED 14287 ± 6% +17.2% 16749 ± 4% softirqs.CPU277.SCHED 6301 +41.8% 8938 ± 16% softirqs.CPU278.RCU 13980 ± 10% +24.6% 17424 ± 2% softirqs.CPU278.SCHED 14763 ± 7% +16.0% 17119 softirqs.CPU279.SCHED 13682 ± 5% +23.9% 16950 softirqs.CPU28.SCHED 14204 ± 3% +17.0% 16617 ± 3% softirqs.CPU280.SCHED 14202 ± 3% +14.4% 16240 ± 2% softirqs.CPU281.SCHED 14629 ± 7% +16.3% 17013 ± 3% softirqs.CPU283.SCHED 13692 ± 6% +22.7% 16795 softirqs.CPU284.SCHED 14194 ± 7% +17.5% 16679 ± 3% softirqs.CPU285.SCHED 123721 ± 3% -8.1% 113741 ± 3% softirqs.CPU285.TIMER 14757 ± 7% +15.5% 17050 softirqs.CPU286.SCHED 13560 ± 6% +17.2% 15894 softirqs.CPU287.SCHED 14201 ± 4% +19.9% 17020 ± 2% softirqs.CPU29.SCHED 14345 ± 3% +20.5% 17292 softirqs.CPU3.SCHED 13910 ± 4% +20.9% 16814 ± 4% softirqs.CPU30.SCHED 7587 +41.8% 10759 ± 23% softirqs.CPU31.RCU 13401 ± 4% +26.6% 16967 ± 2% softirqs.CPU31.SCHED 13593 +24.6% 16937 softirqs.CPU32.SCHED 13175 ± 2% +29.0% 16989 softirqs.CPU33.SCHED 13491 ± 5% +25.8% 16974 ± 3% softirqs.CPU34.SCHED 13520 ± 7% +25.9% 17018 ± 2% softirqs.CPU35.SCHED 13723 ± 6% +22.1% 16763 ± 2% softirqs.CPU36.SCHED 13449 ± 3% +27.1% 17091 softirqs.CPU37.SCHED 7405 +40.0% 10368 ± 17% softirqs.CPU38.RCU 14299 ± 6% +22.4% 17509 ± 3% softirqs.CPU38.SCHED 14269 ± 3% +18.6% 16918 softirqs.CPU39.SCHED 7977 ± 2% +23.6% 9862 ± 5% softirqs.CPU4.RCU 14055 ± 2% +23.4% 17349 ± 3% softirqs.CPU4.SCHED 14257 ± 2% +19.4% 17030 ± 2% softirqs.CPU40.SCHED 13774 ± 2% +24.1% 17090 softirqs.CPU41.SCHED 14425 ± 8% +17.7% 16976 softirqs.CPU42.SCHED 14314 ± 2% +21.2% 17345 ± 2% softirqs.CPU43.SCHED 13274 ± 4% +27.7% 16954 softirqs.CPU44.SCHED 135612 ± 10% -15.7% 114264 ± 3% softirqs.CPU44.TIMER 13919 ± 2% +20.9% 16829 softirqs.CPU45.SCHED 13156 +27.9% 16834 ± 2% softirqs.CPU46.SCHED 13617 ± 2% +22.0% 16614 softirqs.CPU47.SCHED 13565 ± 3% +25.7% 17054 ± 2% softirqs.CPU48.SCHED 13791 ± 4% +21.6% 16769 softirqs.CPU49.SCHED 14260 ± 3% +20.1% 17126 ± 4% softirqs.CPU5.SCHED 13510 ± 7% +25.3% 16934 softirqs.CPU50.SCHED 12790 ± 10% +30.3% 16669 softirqs.CPU51.SCHED 13989 +19.9% 16774 ± 3% softirqs.CPU52.SCHED 14279 ± 3% +17.1% 16722 ± 2% softirqs.CPU53.SCHED 7263 +36.9% 9940 ± 21% softirqs.CPU54.RCU 14088 ± 2% +20.1% 16918 softirqs.CPU54.SCHED 13508 ± 6% +23.1% 16629 softirqs.CPU55.SCHED 130302 ± 6% -9.4% 118014 ± 7% softirqs.CPU55.TIMER 13313 ± 3% +27.9% 17030 ± 2% softirqs.CPU56.SCHED 7360 ± 2% +33.2% 9803 ± 12% softirqs.CPU57.RCU 13818 ± 2% +21.3% 16768 ± 2% softirqs.CPU57.SCHED 7296 +27.1% 9274 ± 7% softirqs.CPU58.RCU 13922 ± 3% +23.4% 17181 ± 2% softirqs.CPU58.SCHED 7271 +30.3% 9476 ± 11% softirqs.CPU59.RCU 14169 ± 5% +19.7% 16956 ± 2% softirqs.CPU59.SCHED 8176 ± 7% +16.5% 9524 ± 3% softirqs.CPU6.RCU 13689 ± 3% +25.2% 17132 ± 5% softirqs.CPU6.SCHED 13645 ± 2% +23.3% 16825 softirqs.CPU60.SCHED 13441 ± 6% +24.9% 16786 ± 2% softirqs.CPU61.SCHED 13497 ± 5% +25.7% 16959 softirqs.CPU62.SCHED 13682 ± 4% +25.5% 17170 softirqs.CPU63.SCHED 13656 ± 6% +22.3% 16695 ± 3% softirqs.CPU64.SCHED 13704 ± 7% +22.7% 16819 softirqs.CPU65.SCHED 13683 ± 3% +23.0% 16826 softirqs.CPU66.SCHED 14076 ± 2% +19.3% 16797 ± 2% softirqs.CPU67.SCHED 13713 ± 4% +18.2% 16215 ± 2% softirqs.CPU68.SCHED 13396 ± 3% +25.1% 16754 ± 2% softirqs.CPU69.SCHED 8042 ± 3% +21.0% 9730 ± 7% softirqs.CPU7.RCU 13604 ± 3% +27.3% 17320 ± 5% softirqs.CPU7.SCHED 13023 ± 2% +29.2% 16828 ± 2% softirqs.CPU70.SCHED 12307 ± 3% +32.1% 16255 softirqs.CPU71.SCHED 7727 ± 22% +37.3% 10612 ± 22% softirqs.CPU72.RCU 13320 ± 3% +28.0% 17050 ± 2% softirqs.CPU72.SCHED 13838 ± 3% +22.0% 16883 ± 3% softirqs.CPU73.SCHED 14604 ± 2% +14.4% 16707 ± 4% softirqs.CPU74.SCHED 14184 ± 4% +19.1% 16892 ± 2% softirqs.CPU75.SCHED 14138 ± 4% +19.0% 16830 softirqs.CPU76.SCHED 7098 +28.1% 9090 ± 12% softirqs.CPU77.RCU 13082 ± 4% +28.0% 16741 softirqs.CPU77.SCHED 126355 ± 4% -10.8% 112765 ± 2% softirqs.CPU78.TIMER 14718 ± 5% +12.6% 16576 ± 3% softirqs.CPU79.SCHED 7959 ± 2% +27.1% 10119 ± 10% softirqs.CPU8.RCU 13775 ± 4% +25.1% 17232 ± 5% softirqs.CPU8.SCHED 14397 +13.6% 16355 ± 2% softirqs.CPU80.SCHED 13999 ± 6% +20.7% 16896 ± 2% softirqs.CPU81.SCHED 14129 ± 7% +18.2% 16706 softirqs.CPU82.SCHED 13931 +17.6% 16385 ± 3% softirqs.CPU83.SCHED 14455 ± 5% +16.1% 16776 ± 2% softirqs.CPU84.SCHED 14426 ± 5% +16.4% 16799 softirqs.CPU86.SCHED 7049 +28.2% 9036 ± 10% softirqs.CPU88.RCU 13901 ± 3% +20.1% 16697 ± 2% softirqs.CPU88.SCHED 127258 ± 2% -7.3% 118017 softirqs.CPU88.TIMER 14353 ± 5% +17.4% 16855 ± 2% softirqs.CPU89.SCHED 14823 ± 4% +14.7% 17004 softirqs.CPU9.SCHED 14278 ± 4% +18.4% 16908 softirqs.CPU90.SCHED 14225 ± 2% +18.8% 16894 ± 2% softirqs.CPU91.SCHED 7091 +26.7% 8987 ± 10% softirqs.CPU92.RCU 13726 ± 3% +24.0% 17014 ± 3% softirqs.CPU92.SCHED 14051 ± 5% +20.6% 16939 softirqs.CPU93.SCHED 14123 ± 4% +17.9% 16649 ± 3% softirqs.CPU94.SCHED 130109 ± 5% -8.2% 119400 softirqs.CPU95.TIMER 13638 ± 4% +23.5% 16847 ± 2% softirqs.CPU96.SCHED 13134 ± 2% +29.4% 17001 ± 3% softirqs.CPU97.SCHED 13357 ± 3% +25.7% 16791 ± 2% softirqs.CPU98.SCHED 14112 ± 6% +20.0% 16938 softirqs.CPU99.SCHED 1991578 +21.8% 2426642 ± 3% softirqs.RCU 4100803 +18.4% 4855612 softirqs.SCHED 454.00 ± 27% +675.8% 3522 ±139% interrupts.30:IR-PCI-MSI.2097153-edge.eth0-TxRx-0 384622 -1.6% 378485 interrupts.CAL:Function_call_interrupts 930.25 ± 8% -46.7% 496.00 ± 10% interrupts.CPU0.TLB:TLB_shootdowns 758.50 ± 9% -44.8% 418.75 ± 11% interrupts.CPU1.TLB:TLB_shootdowns 454.00 ± 27% +675.8% 3522 ±139% interrupts.CPU10.30:IR-PCI-MSI.2097153-edge.eth0-TxRx-0 886.75 ± 5% -39.4% 537.00 ± 3% interrupts.CPU10.TLB:TLB_shootdowns 169.50 ± 11% +67.0% 283.00 ± 9% interrupts.CPU100.TLB:TLB_shootdowns 178.50 ± 20% +44.0% 257.00 ± 9% interrupts.CPU101.TLB:TLB_shootdowns 169.00 ± 23% +62.7% 275.00 ± 20% interrupts.CPU104.TLB:TLB_shootdowns 1463 ± 28% +88.1% 2753 ± 22% interrupts.CPU105.NMI:Non-maskable_interrupts 1463 ± 28% +88.1% 2753 ± 22% interrupts.CPU105.PMI:Performance_monitoring_interrupts 158.75 ± 21% +73.4% 275.25 ± 4% interrupts.CPU106.TLB:TLB_shootdowns 157.50 ± 24% +67.3% 263.50 ± 13% interrupts.CPU109.TLB:TLB_shootdowns 2884 ± 27% -43.7% 1625 ± 6% interrupts.CPU11.NMI:Non-maskable_interrupts 2884 ± 27% -43.7% 1625 ± 6% interrupts.CPU11.PMI:Performance_monitoring_interrupts 713.50 ± 5% -38.8% 436.50 ± 13% interrupts.CPU11.TLB:TLB_shootdowns 146.00 ± 38% +79.3% 261.75 ± 14% interrupts.CPU110.TLB:TLB_shootdowns 162.00 ± 14% +82.6% 295.75 ± 12% interrupts.CPU112.TLB:TLB_shootdowns 445.75 ± 52% +170.1% 1204 ± 91% interrupts.CPU114.RES:Rescheduling_interrupts 160.25 ± 37% +70.8% 273.75 ± 14% interrupts.CPU114.TLB:TLB_shootdowns 1417 ± 17% +66.0% 2352 ± 34% interrupts.CPU115.NMI:Non-maskable_interrupts 1417 ± 17% +66.0% 2352 ± 34% interrupts.CPU115.PMI:Performance_monitoring_interrupts 380.75 ± 9% +37.9% 525.00 ± 15% interrupts.CPU116.RES:Rescheduling_interrupts 183.00 ± 9% +48.2% 271.25 ± 14% interrupts.CPU116.TLB:TLB_shootdowns 170.50 ± 15% +74.5% 297.50 ± 5% interrupts.CPU118.TLB:TLB_shootdowns 454.25 ± 38% +251.3% 1596 ±103% interrupts.CPU12.RES:Rescheduling_interrupts 847.00 ± 12% -36.9% 534.25 ± 10% interrupts.CPU12.TLB:TLB_shootdowns 173.25 ± 20% +65.7% 287.00 ± 13% interrupts.CPU122.TLB:TLB_shootdowns 420.50 ± 32% +80.5% 759.00 ± 37% interrupts.CPU123.RES:Rescheduling_interrupts 247.75 ± 22% +75.9% 435.75 ± 14% interrupts.CPU124.RES:Rescheduling_interrupts 181.75 ± 33% +58.7% 288.50 ± 15% interrupts.CPU126.TLB:TLB_shootdowns 364.75 ± 21% +39.1% 507.50 ± 11% interrupts.CPU128.RES:Rescheduling_interrupts 337.75 ± 24% +248.5% 1177 ± 74% interrupts.CPU13.RES:Rescheduling_interrupts 783.25 ± 14% -47.8% 409.00 ± 3% interrupts.CPU13.TLB:TLB_shootdowns 277.00 ± 14% +84.9% 512.25 ± 10% interrupts.CPU131.RES:Rescheduling_interrupts 155.75 ± 24% +71.7% 267.50 ± 21% interrupts.CPU132.TLB:TLB_shootdowns 174.25 ± 24% +50.4% 262.00 ± 19% interrupts.CPU133.TLB:TLB_shootdowns 137.00 ± 7% +116.4% 296.50 ± 11% interrupts.CPU134.TLB:TLB_shootdowns 152.00 ± 24% +86.3% 283.25 ± 17% interrupts.CPU138.TLB:TLB_shootdowns 820.00 ± 10% -32.5% 553.75 ± 3% interrupts.CPU14.TLB:TLB_shootdowns 277.75 ± 19% +81.1% 503.00 ± 7% interrupts.CPU140.RES:Rescheduling_interrupts 298.25 ± 31% +59.3% 475.00 ± 15% interrupts.CPU142.RES:Rescheduling_interrupts 192.50 ± 22% +38.6% 266.75 ± 4% interrupts.CPU143.TLB:TLB_shootdowns 290.00 ± 38% +103.4% 590.00 ± 18% interrupts.CPU145.RES:Rescheduling_interrupts 104.50 ± 34% +90.4% 199.00 ± 16% interrupts.CPU145.TLB:TLB_shootdowns 226.50 ± 10% +96.9% 446.00 ± 15% interrupts.CPU146.RES:Rescheduling_interrupts 95.75 ± 8% +145.4% 235.00 ± 12% interrupts.CPU146.TLB:TLB_shootdowns 145.00 ± 22% +65.2% 239.50 ± 16% interrupts.CPU147.TLB:TLB_shootdowns 90.25 ± 23% +212.2% 281.75 ± 19% interrupts.CPU148.TLB:TLB_shootdowns 104.25 ± 22% +142.7% 253.00 ± 11% interrupts.CPU149.TLB:TLB_shootdowns 2888 ± 16% -35.6% 1860 ± 30% interrupts.CPU15.NMI:Non-maskable_interrupts 2888 ± 16% -35.6% 1860 ± 30% interrupts.CPU15.PMI:Performance_monitoring_interrupts 472.25 ± 31% +275.2% 1771 ±106% interrupts.CPU15.RES:Rescheduling_interrupts 816.50 ± 16% -50.0% 408.50 ± 11% interrupts.CPU15.TLB:TLB_shootdowns 84.75 ± 24% +207.4% 260.50 ± 10% interrupts.CPU150.TLB:TLB_shootdowns 133.50 ± 16% +108.6% 278.50 ± 17% interrupts.CPU151.TLB:TLB_shootdowns 103.75 ± 25% +132.0% 240.75 ± 8% interrupts.CPU152.TLB:TLB_shootdowns 105.00 ± 60% +155.0% 267.75 ± 7% interrupts.CPU153.TLB:TLB_shootdowns 100.50 ± 28% +133.1% 234.25 ± 8% interrupts.CPU154.TLB:TLB_shootdowns 73.50 ± 36% +220.4% 235.50 ± 11% interrupts.CPU155.TLB:TLB_shootdowns 229.50 ± 18% +182.1% 647.50 ± 8% interrupts.CPU156.RES:Rescheduling_interrupts 114.50 ± 20% +110.9% 241.50 ± 29% interrupts.CPU156.TLB:TLB_shootdowns 125.75 ± 27% +79.7% 226.00 ± 8% interrupts.CPU157.TLB:TLB_shootdowns 97.50 ± 21% +143.1% 237.00 ± 16% interrupts.CPU158.TLB:TLB_shootdowns 94.75 ± 49% +136.4% 224.00 ± 21% interrupts.CPU159.TLB:TLB_shootdowns 452.00 ± 36% +89.2% 855.25 ± 39% interrupts.CPU16.RES:Rescheduling_interrupts 959.00 ± 16% -54.3% 438.25 ± 8% interrupts.CPU16.TLB:TLB_shootdowns 96.75 ± 9% +145.0% 237.00 ± 9% interrupts.CPU161.TLB:TLB_shootdowns 94.25 ± 24% +169.5% 254.00 ± 10% interrupts.CPU162.TLB:TLB_shootdowns 100.50 ± 8% +141.8% 243.00 ± 14% interrupts.CPU163.TLB:TLB_shootdowns 87.00 ± 42% +186.5% 249.25 ± 8% interrupts.CPU164.TLB:TLB_shootdowns 93.50 ± 29% +145.7% 229.75 ± 9% interrupts.CPU165.TLB:TLB_shootdowns 110.50 ± 33% +165.4% 293.25 ± 15% interrupts.CPU166.TLB:TLB_shootdowns 323.25 ± 37% +105.7% 665.00 ± 39% interrupts.CPU167.RES:Rescheduling_interrupts 96.75 ± 48% +140.3% 232.50 ± 14% interrupts.CPU167.TLB:TLB_shootdowns 224.00 ± 22% +140.8% 539.50 ± 40% interrupts.CPU168.RES:Rescheduling_interrupts 87.50 ± 26% +151.1% 219.75 ± 14% interrupts.CPU168.TLB:TLB_shootdowns 1120 ± 47% +79.9% 2015 ± 27% interrupts.CPU169.NMI:Non-maskable_interrupts 1120 ± 47% +79.9% 2015 ± 27% interrupts.CPU169.PMI:Performance_monitoring_interrupts 100.25 ± 22% +117.5% 218.00 ± 9% interrupts.CPU169.TLB:TLB_shootdowns 667.75 ± 12% -30.3% 465.50 ± 14% interrupts.CPU17.TLB:TLB_shootdowns 75.00 ± 28% +252.7% 264.50 ± 22% interrupts.CPU170.TLB:TLB_shootdowns 94.25 ± 19% +126.0% 213.00 ± 13% interrupts.CPU171.TLB:TLB_shootdowns 318.25 ± 28% +53.3% 488.00 ± 18% interrupts.CPU172.RES:Rescheduling_interrupts 85.25 ± 33% +221.1% 273.75 ± 19% interrupts.CPU172.TLB:TLB_shootdowns 82.25 ± 18% +130.7% 189.75 ± 20% interrupts.CPU173.TLB:TLB_shootdowns 67.75 ± 55% +261.6% 245.00 ± 12% interrupts.CPU174.TLB:TLB_shootdowns 227.75 ± 41% +588.6% 1568 ± 95% interrupts.CPU175.RES:Rescheduling_interrupts 91.50 ± 10% +150.8% 229.50 ± 10% interrupts.CPU175.TLB:TLB_shootdowns 99.50 ± 17% +147.2% 246.00 ± 22% interrupts.CPU176.TLB:TLB_shootdowns 79.25 ± 18% +200.0% 237.75 ± 27% interrupts.CPU177.TLB:TLB_shootdowns 87.50 ± 36% +129.7% 201.00 ± 4% interrupts.CPU178.TLB:TLB_shootdowns 127.25 ± 44% +82.7% 232.50 ± 15% interrupts.CPU179.TLB:TLB_shootdowns 1161 ± 15% +137.6% 2758 ± 27% interrupts.CPU18.NMI:Non-maskable_interrupts 1161 ± 15% +137.6% 2758 ± 27% interrupts.CPU18.PMI:Performance_monitoring_interrupts 425.75 ± 15% +149.5% 1062 ± 64% interrupts.CPU18.RES:Rescheduling_interrupts 985.50 ± 13% -50.1% 492.25 ± 13% interrupts.CPU18.TLB:TLB_shootdowns 112.25 ± 33% +120.7% 247.75 ± 8% interrupts.CPU180.TLB:TLB_shootdowns 126.25 ± 15% +96.0% 247.50 ± 9% interrupts.CPU181.TLB:TLB_shootdowns 229.25 ± 25% +74.6% 400.25 ± 7% interrupts.CPU182.RES:Rescheduling_interrupts 102.50 ± 41% +173.4% 280.25 ± 4% interrupts.CPU182.TLB:TLB_shootdowns 253.25 ± 13% +52.6% 386.50 ± 8% interrupts.CPU183.RES:Rescheduling_interrupts 97.00 ± 50% +153.4% 245.75 ± 4% interrupts.CPU183.TLB:TLB_shootdowns 99.00 ± 14% +143.7% 241.25 ± 18% interrupts.CPU184.TLB:TLB_shootdowns 287.50 ± 37% +57.0% 451.25 ± 32% interrupts.CPU185.RES:Rescheduling_interrupts 92.50 ± 27% +139.2% 221.25 ± 5% interrupts.CPU185.TLB:TLB_shootdowns 2302 ± 26% +37.4% 3162 ± 5% interrupts.CPU186.NMI:Non-maskable_interrupts 2302 ± 26% +37.4% 3162 ± 5% interrupts.CPU186.PMI:Performance_monitoring_interrupts 87.75 ± 16% +204.0% 266.75 ± 20% interrupts.CPU186.TLB:TLB_shootdowns 97.00 ± 31% +157.0% 249.25 ± 10% interrupts.CPU187.TLB:TLB_shootdowns 261.50 ± 10% +54.2% 403.25 ± 15% interrupts.CPU188.RES:Rescheduling_interrupts 108.50 ± 42% +103.7% 221.00 ± 26% interrupts.CPU188.TLB:TLB_shootdowns 223.75 ± 21% +88.4% 421.50 ± 18% interrupts.CPU189.RES:Rescheduling_interrupts 119.25 ± 11% +108.4% 248.50 ± 25% interrupts.CPU189.TLB:TLB_shootdowns 400.25 ± 27% +117.4% 870.25 ± 52% interrupts.CPU19.RES:Rescheduling_interrupts 699.50 ± 16% -31.8% 476.75 ± 10% interrupts.CPU19.TLB:TLB_shootdowns 99.75 ± 43% +148.6% 248.00 ± 12% interrupts.CPU190.TLB:TLB_shootdowns 98.25 ± 10% +146.6% 242.25 ± 17% interrupts.CPU191.TLB:TLB_shootdowns 105.25 ± 29% +145.4% 258.25 ± 19% interrupts.CPU192.TLB:TLB_shootdowns 89.50 ± 23% +191.3% 260.75 ± 17% interrupts.CPU193.TLB:TLB_shootdowns 66.25 ± 25% +285.7% 255.50 ± 17% interrupts.CPU194.TLB:TLB_shootdowns 278.75 ± 10% +84.9% 515.50 ± 32% interrupts.CPU195.RES:Rescheduling_interrupts 80.00 ± 18% +247.8% 278.25 ± 14% interrupts.CPU195.TLB:TLB_shootdowns 70.25 ± 12% +246.3% 243.25 ± 11% interrupts.CPU196.TLB:TLB_shootdowns 104.75 ± 26% +147.0% 258.75 ± 23% interrupts.CPU197.TLB:TLB_shootdowns 86.75 ± 62% +188.2% 250.00 ± 20% interrupts.CPU198.TLB:TLB_shootdowns 94.25 ± 17% +150.1% 235.75 ± 19% interrupts.CPU199.TLB:TLB_shootdowns 3084 ± 4% -52.9% 1453 ± 6% interrupts.CPU2.NMI:Non-maskable_interrupts 3084 ± 4% -52.9% 1453 ± 6% interrupts.CPU2.PMI:Performance_monitoring_interrupts 556.25 ± 46% +179.6% 1555 ± 83% interrupts.CPU2.RES:Rescheduling_interrupts 891.00 ± 15% -42.6% 511.50 ± 13% interrupts.CPU2.TLB:TLB_shootdowns 364.75 ± 16% +193.9% 1072 ± 40% interrupts.CPU20.RES:Rescheduling_interrupts 825.75 ± 13% -35.7% 530.75 ± 6% interrupts.CPU20.TLB:TLB_shootdowns 129.25 ± 17% +83.0% 236.50 ± 7% interrupts.CPU200.TLB:TLB_shootdowns 88.50 ± 58% +200.0% 265.50 ± 12% interrupts.CPU201.TLB:TLB_shootdowns 116.75 ± 24% +99.8% 233.25 ± 24% interrupts.CPU202.TLB:TLB_shootdowns 254.25 ± 35% +121.8% 564.00 ± 14% interrupts.CPU203.RES:Rescheduling_interrupts 83.25 ± 27% +203.0% 252.25 ± 18% interrupts.CPU203.TLB:TLB_shootdowns 110.25 ± 16% +195.2% 325.50 ± 35% interrupts.CPU204.TLB:TLB_shootdowns 332.00 ± 44% +60.5% 532.75 ± 15% interrupts.CPU205.RES:Rescheduling_interrupts 89.00 ± 32% +165.4% 236.25 ± 11% interrupts.CPU205.TLB:TLB_shootdowns 236.25 ± 23% +112.8% 502.75 ± 10% interrupts.CPU206.RES:Rescheduling_interrupts 82.75 ± 32% +195.2% 244.25 ± 13% interrupts.CPU206.TLB:TLB_shootdowns 121.25 ± 15% +86.6% 226.25 ± 18% interrupts.CPU207.TLB:TLB_shootdowns 83.75 ± 24% +170.7% 226.75 ± 5% interrupts.CPU208.TLB:TLB_shootdowns 330.25 ± 13% +131.5% 764.50 ± 44% interrupts.CPU209.RES:Rescheduling_interrupts 75.75 ± 20% +217.8% 240.75 ± 9% interrupts.CPU209.TLB:TLB_shootdowns 778.00 ± 16% -39.2% 473.00 ± 34% interrupts.CPU21.TLB:TLB_shootdowns 268.75 ± 27% +178.4% 748.25 ± 37% interrupts.CPU210.RES:Rescheduling_interrupts 114.75 ± 16% +100.0% 229.50 ± 13% interrupts.CPU210.TLB:TLB_shootdowns 112.00 ± 29% +101.1% 225.25 ± 15% interrupts.CPU211.TLB:TLB_shootdowns 97.00 ± 58% +159.3% 251.50 ± 21% interrupts.CPU212.TLB:TLB_shootdowns 263.75 ± 29% +80.3% 475.50 ± 15% interrupts.CPU213.RES:Rescheduling_interrupts 101.25 ± 25% +98.5% 201.00 ± 28% interrupts.CPU215.TLB:TLB_shootdowns 1275 ± 12% +70.6% 2177 ± 30% interrupts.CPU216.NMI:Non-maskable_interrupts 1275 ± 12% +70.6% 2177 ± 30% interrupts.CPU216.PMI:Performance_monitoring_interrupts 278.75 ± 15% +48.7% 414.50 ± 14% interrupts.CPU216.RES:Rescheduling_interrupts 95.50 ± 26% +118.3% 208.50 ± 11% interrupts.CPU216.TLB:TLB_shootdowns 1391 ± 30% +54.2% 2146 ± 33% interrupts.CPU217.NMI:Non-maskable_interrupts 1391 ± 30% +54.2% 2146 ± 33% interrupts.CPU217.PMI:Performance_monitoring_interrupts 114.50 ± 10% +84.9% 211.75 ± 16% interrupts.CPU217.TLB:TLB_shootdowns 77.00 ± 48% +308.4% 314.50 ± 55% interrupts.CPU218.TLB:TLB_shootdowns 70.75 ± 41% +259.0% 254.00 ± 21% interrupts.CPU219.TLB:TLB_shootdowns 884.75 ± 18% -36.6% 561.25 ± 12% interrupts.CPU22.TLB:TLB_shootdowns 58.25 ± 36% +315.9% 242.25 ± 13% interrupts.CPU220.TLB:TLB_shootdowns 230.50 ± 35% +103.9% 470.00 ± 18% interrupts.CPU221.RES:Rescheduling_interrupts 61.00 ± 40% +254.1% 216.00 ± 3% interrupts.CPU221.TLB:TLB_shootdowns 114.75 ± 47% +102.6% 232.50 ± 22% interrupts.CPU223.TLB:TLB_shootdowns 94.25 ± 19% +146.9% 232.75 ± 15% interrupts.CPU224.TLB:TLB_shootdowns 92.75 ± 47% +142.6% 225.00 ± 12% interrupts.CPU225.TLB:TLB_shootdowns 84.75 ± 19% +182.0% 239.00 ± 4% interrupts.CPU226.TLB:TLB_shootdowns 91.00 ± 34% +164.8% 241.00 ± 20% interrupts.CPU227.TLB:TLB_shootdowns 2345 ± 32% -40.0% 1407 ± 17% interrupts.CPU228.NMI:Non-maskable_interrupts 2345 ± 32% -40.0% 1407 ± 17% interrupts.CPU228.PMI:Performance_monitoring_interrupts 61.75 ± 24% +232.0% 205.00 ± 10% interrupts.CPU228.TLB:TLB_shootdowns 75.00 ± 49% +188.7% 216.50 ± 14% interrupts.CPU229.TLB:TLB_shootdowns 706.25 ± 17% -38.5% 434.00 ± 19% interrupts.CPU23.TLB:TLB_shootdowns 78.50 ± 20% +208.6% 242.25 ± 11% interrupts.CPU230.TLB:TLB_shootdowns 67.50 ± 35% +214.4% 212.25 ± 20% interrupts.CPU231.TLB:TLB_shootdowns 204.25 ± 15% +141.9% 494.00 ± 11% interrupts.CPU232.RES:Rescheduling_interrupts 65.25 ± 19% +254.0% 231.00 ± 13% interrupts.CPU232.TLB:TLB_shootdowns 104.75 ± 28% +106.9% 216.75 ± 26% interrupts.CPU233.TLB:TLB_shootdowns 65.00 ± 44% +226.9% 212.50 ± 8% interrupts.CPU234.TLB:TLB_shootdowns 72.00 ± 53% +186.8% 206.50 ± 20% interrupts.CPU235.TLB:TLB_shootdowns 262.50 ± 38% +75.3% 460.25 ± 13% interrupts.CPU236.RES:Rescheduling_interrupts 72.75 ± 61% +195.5% 215.00 ± 7% interrupts.CPU236.TLB:TLB_shootdowns 78.25 ± 21% +187.2% 224.75 ± 32% interrupts.CPU237.TLB:TLB_shootdowns 92.25 ± 27% +166.9% 246.25 ± 17% interrupts.CPU238.TLB:TLB_shootdowns 120.50 ± 37% +118.5% 263.25 ± 19% interrupts.CPU239.TLB:TLB_shootdowns 871.75 ± 7% -42.3% 502.75 ± 12% interrupts.CPU24.TLB:TLB_shootdowns 63.75 ± 34% +271.0% 236.50 ± 6% interrupts.CPU240.TLB:TLB_shootdowns 77.25 ± 29% +207.4% 237.50 ± 15% interrupts.CPU241.TLB:TLB_shootdowns 49.75 ± 29% +310.6% 204.25 ± 13% interrupts.CPU242.TLB:TLB_shootdowns 76.00 ± 19% +194.7% 224.00 ± 17% interrupts.CPU243.TLB:TLB_shootdowns 80.75 ± 10% +183.6% 229.00 ± 17% interrupts.CPU244.TLB:TLB_shootdowns 86.75 ± 57% +171.8% 235.75 ± 8% interrupts.CPU245.TLB:TLB_shootdowns 56.50 ± 51% +344.2% 251.00 ± 13% interrupts.CPU246.TLB:TLB_shootdowns 270.50 ± 24% +153.1% 684.75 ± 34% interrupts.CPU247.RES:Rescheduling_interrupts 88.25 ± 24% +180.2% 247.25 ± 18% interrupts.CPU247.TLB:TLB_shootdowns 81.25 ± 37% +167.1% 217.00 ± 20% interrupts.CPU248.TLB:TLB_shootdowns 67.00 ± 26% +216.4% 212.00 ± 11% interrupts.CPU249.TLB:TLB_shootdowns 740.75 ± 11% -36.9% 467.25 ± 7% interrupts.CPU25.TLB:TLB_shootdowns 67.50 ± 29% +247.0% 234.25 ± 11% interrupts.CPU250.TLB:TLB_shootdowns 92.00 ± 63% +111.1% 194.25 ± 24% interrupts.CPU251.TLB:TLB_shootdowns 72.25 ± 45% +223.5% 233.75 ± 16% interrupts.CPU252.TLB:TLB_shootdowns 78.75 ± 29% +178.7% 219.50 ± 12% interrupts.CPU253.TLB:TLB_shootdowns 64.25 ± 58% +258.4% 230.25 ± 11% interrupts.CPU254.TLB:TLB_shootdowns 267.00 ± 17% +44.3% 385.25 ± 13% interrupts.CPU255.RES:Rescheduling_interrupts 114.25 ± 36% +106.1% 235.50 ± 12% interrupts.CPU255.TLB:TLB_shootdowns 74.00 ± 33% +213.9% 232.25 ± 13% interrupts.CPU256.TLB:TLB_shootdowns 86.00 ± 81% +158.4% 222.25 ± 19% interrupts.CPU257.TLB:TLB_shootdowns 1565 ± 38% +54.4% 2417 ± 27% interrupts.CPU258.NMI:Non-maskable_interrupts 1565 ± 38% +54.4% 2417 ± 27% interrupts.CPU258.PMI:Performance_monitoring_interrupts 252.00 ± 25% +118.6% 550.75 ± 54% interrupts.CPU258.RES:Rescheduling_interrupts 81.75 ± 21% +192.7% 239.25 ± 9% interrupts.CPU258.TLB:TLB_shootdowns 106.75 ± 37% +109.1% 223.25 ± 20% interrupts.CPU259.TLB:TLB_shootdowns 899.75 ± 8% -42.9% 514.00 ± 6% interrupts.CPU26.TLB:TLB_shootdowns 67.25 ± 41% +291.8% 263.50 ± 22% interrupts.CPU260.TLB:TLB_shootdowns 59.25 ± 34% +289.0% 230.50 ± 8% interrupts.CPU261.TLB:TLB_shootdowns 292.75 ± 12% +61.0% 471.25 ± 16% interrupts.CPU262.RES:Rescheduling_interrupts 49.75 ± 45% +400.0% 248.75 ± 14% interrupts.CPU262.TLB:TLB_shootdowns 325.00 ± 17% +52.1% 494.25 ± 27% interrupts.CPU263.RES:Rescheduling_interrupts 87.00 ± 32% +143.1% 211.50 ± 11% interrupts.CPU263.TLB:TLB_shootdowns 218.25 ± 14% +71.7% 374.75 ± 5% interrupts.CPU264.RES:Rescheduling_interrupts 75.75 ± 28% +188.8% 218.75 ± 9% interrupts.CPU264.TLB:TLB_shootdowns 56.25 ± 55% +306.2% 228.50 ± 20% interrupts.CPU265.TLB:TLB_shootdowns 80.75 ± 53% +221.1% 259.25 ± 12% interrupts.CPU266.TLB:TLB_shootdowns 271.25 ± 27% +56.2% 423.75 ± 22% interrupts.CPU267.RES:Rescheduling_interrupts 60.25 ± 48% +352.7% 272.75 ± 16% interrupts.CPU267.TLB:TLB_shootdowns 56.00 ± 43% +295.5% 221.50 ± 20% interrupts.CPU268.TLB:TLB_shootdowns 97.25 ± 14% +151.2% 244.25 ± 17% interrupts.CPU269.TLB:TLB_shootdowns 688.00 ± 13% -32.2% 466.25 ± 11% interrupts.CPU27.TLB:TLB_shootdowns 208.00 ± 27% +93.6% 402.75 ± 8% interrupts.CPU270.RES:Rescheduling_interrupts 57.50 ± 30% +368.3% 269.25 ± 19% interrupts.CPU270.TLB:TLB_shootdowns 73.75 ± 34% +288.5% 286.50 ± 14% interrupts.CPU271.TLB:TLB_shootdowns 279.25 ± 32% +37.3% 383.50 ± 10% interrupts.CPU272.RES:Rescheduling_interrupts 79.25 ± 14% +189.6% 229.50 ± 28% interrupts.CPU272.TLB:TLB_shootdowns 216.50 ± 15% +201.8% 653.50 ± 66% interrupts.CPU273.RES:Rescheduling_interrupts 65.50 ± 22% +256.1% 233.25 ± 11% interrupts.CPU273.TLB:TLB_shootdowns 74.75 ± 68% +198.3% 223.00 ± 17% interrupts.CPU274.TLB:TLB_shootdowns 68.50 ± 42% +204.4% 208.50 ± 10% interrupts.CPU275.TLB:TLB_shootdowns 59.75 ± 35% +226.8% 195.25 ± 13% interrupts.CPU276.TLB:TLB_shootdowns 292.25 ± 35% +80.2% 526.75 ± 32% interrupts.CPU277.RES:Rescheduling_interrupts 73.25 ± 39% +233.8% 244.50 ± 14% interrupts.CPU277.TLB:TLB_shootdowns 61.50 ± 60% +248.0% 214.00 ± 9% interrupts.CPU278.TLB:TLB_shootdowns 163.25 ± 15% +188.1% 470.25 ± 18% interrupts.CPU279.RES:Rescheduling_interrupts 81.50 ± 10% +142.0% 197.25 ± 8% interrupts.CPU279.TLB:TLB_shootdowns 372.75 ± 20% +139.3% 892.00 ± 59% interrupts.CPU28.RES:Rescheduling_interrupts 891.00 ± 4% -41.6% 520.50 ± 10% interrupts.CPU28.TLB:TLB_shootdowns 75.75 ± 23% +223.8% 245.25 ± 12% interrupts.CPU280.TLB:TLB_shootdowns 66.25 ± 39% +234.0% 221.25 ± 12% interrupts.CPU281.TLB:TLB_shootdowns 239.25 ± 24% +90.9% 456.75 ± 15% interrupts.CPU282.RES:Rescheduling_interrupts 63.50 ± 36% +281.9% 242.50 ± 11% interrupts.CPU282.TLB:TLB_shootdowns 75.50 ± 39% +215.2% 238.00 ± 10% interrupts.CPU283.TLB:TLB_shootdowns 269.75 ± 41% +72.2% 464.50 ± 20% interrupts.CPU284.RES:Rescheduling_interrupts 66.25 ± 51% +251.7% 233.00 ± 22% interrupts.CPU284.TLB:TLB_shootdowns 103.75 ± 33% +117.3% 225.50 ± 6% interrupts.CPU285.TLB:TLB_shootdowns 274.75 ± 25% +42.6% 391.75 ± 9% interrupts.CPU286.RES:Rescheduling_interrupts 102.50 ± 22% +106.1% 211.25 ± 3% interrupts.CPU286.TLB:TLB_shootdowns 1604 ± 9% -17.7% 1320 interrupts.CPU287.CAL:Function_call_interrupts 1860 ± 18% -26.0% 1376 ± 16% interrupts.CPU287.NMI:Non-maskable_interrupts 1860 ± 18% -26.0% 1376 ± 16% interrupts.CPU287.PMI:Performance_monitoring_interrupts 245.50 ± 19% +54.3% 378.75 ± 8% interrupts.CPU287.RES:Rescheduling_interrupts 685.00 ± 13% -35.5% 441.75 ± 4% interrupts.CPU29.TLB:TLB_shootdowns 689.75 ± 18% -32.4% 466.00 ± 6% interrupts.CPU3.TLB:TLB_shootdowns 399.75 ± 15% +149.8% 998.75 ± 69% interrupts.CPU30.RES:Rescheduling_interrupts 793.00 ± 16% -36.1% 506.75 ± 7% interrupts.CPU30.TLB:TLB_shootdowns 3182 ± 24% -56.9% 1372 ± 6% interrupts.CPU31.NMI:Non-maskable_interrupts 3182 ± 24% -56.9% 1372 ± 6% interrupts.CPU31.PMI:Performance_monitoring_interrupts 283.00 ± 10% +134.5% 663.50 ± 31% interrupts.CPU31.RES:Rescheduling_interrupts 783.50 ± 20% -43.8% 440.25 ± 3% interrupts.CPU31.TLB:TLB_shootdowns 905.25 ± 8% -39.7% 546.25 ± 19% interrupts.CPU32.TLB:TLB_shootdowns 709.25 ± 4% -35.0% 461.00 ± 14% interrupts.CPU33.TLB:TLB_shootdowns 474.50 ± 31% +92.7% 914.50 ± 50% interrupts.CPU34.RES:Rescheduling_interrupts 879.50 ± 11% -43.0% 501.50 ± 9% interrupts.CPU34.TLB:TLB_shootdowns 750.00 ± 12% -42.5% 431.25 ± 10% interrupts.CPU35.TLB:TLB_shootdowns 889.00 ± 5% -45.4% 485.25 ± 6% interrupts.CPU36.TLB:TLB_shootdowns 415.25 ± 26% +34.3% 557.75 ± 9% interrupts.CPU37.RES:Rescheduling_interrupts 818.50 ± 14% -45.1% 449.25 ± 14% interrupts.CPU37.TLB:TLB_shootdowns 453.50 ± 19% +23.7% 561.00 ± 11% interrupts.CPU38.RES:Rescheduling_interrupts 899.00 ± 5% -37.8% 558.75 ± 7% interrupts.CPU38.TLB:TLB_shootdowns 752.25 ± 14% -41.3% 441.25 ± 11% interrupts.CPU39.TLB:TLB_shootdowns 1502 ± 7% +48.7% 2234 ± 24% interrupts.CPU4.NMI:Non-maskable_interrupts 1502 ± 7% +48.7% 2234 ± 24% interrupts.CPU4.PMI:Performance_monitoring_interrupts 1012 ± 26% -47.3% 534.00 ± 6% interrupts.CPU4.TLB:TLB_shootdowns 882.25 ± 14% -41.4% 517.00 ± 17% interrupts.CPU40.TLB:TLB_shootdowns 762.50 ± 13% -35.8% 489.25 ± 4% interrupts.CPU41.TLB:TLB_shootdowns 995.75 ± 12% -49.2% 506.00 ± 8% interrupts.CPU42.TLB:TLB_shootdowns 652.50 ± 12% -35.4% 421.25 ± 11% interrupts.CPU43.TLB:TLB_shootdowns 421.25 ± 18% +102.1% 851.25 ± 47% interrupts.CPU44.RES:Rescheduling_interrupts 819.00 ± 10% -39.3% 497.25 ± 7% interrupts.CPU44.TLB:TLB_shootdowns 369.00 ± 21% +79.1% 660.75 ± 22% interrupts.CPU45.RES:Rescheduling_interrupts 801.00 ± 7% -41.6% 467.75 ± 8% interrupts.CPU45.TLB:TLB_shootdowns 786.75 ± 15% -38.5% 483.75 ± 12% interrupts.CPU46.TLB:TLB_shootdowns 716.75 ± 13% -35.4% 463.25 ± 11% interrupts.CPU47.TLB:TLB_shootdowns 813.25 ± 9% -34.6% 531.75 ± 6% interrupts.CPU48.TLB:TLB_shootdowns 658.50 ± 6% -26.2% 486.25 ± 6% interrupts.CPU49.TLB:TLB_shootdowns 735.25 ± 20% -36.6% 466.00 ± 18% interrupts.CPU5.TLB:TLB_shootdowns 852.50 ± 17% -43.0% 485.75 ± 9% interrupts.CPU50.TLB:TLB_shootdowns 815.75 ± 8% -40.3% 487.25 ± 12% interrupts.CPU51.TLB:TLB_shootdowns 899.50 ± 9% -42.5% 517.50 ± 10% interrupts.CPU52.TLB:TLB_shootdowns 654.50 ± 9% -33.8% 433.00 ± 19% interrupts.CPU53.TLB:TLB_shootdowns 920.00 ± 8% -40.9% 543.50 ± 6% interrupts.CPU54.TLB:TLB_shootdowns 687.25 ± 19% -31.5% 470.75 ± 16% interrupts.CPU55.TLB:TLB_shootdowns 2467 ± 22% -41.5% 1442 ± 13% interrupts.CPU56.NMI:Non-maskable_interrupts 2467 ± 22% -41.5% 1442 ± 13% interrupts.CPU56.PMI:Performance_monitoring_interrupts 342.00 ± 7% +106.1% 705.00 ± 30% interrupts.CPU56.RES:Rescheduling_interrupts 884.75 ± 19% -45.2% 485.25 ± 8% interrupts.CPU56.TLB:TLB_shootdowns 772.00 ± 11% -43.0% 440.00 ± 11% interrupts.CPU57.TLB:TLB_shootdowns 942.50 ± 11% -46.3% 506.00 ± 10% interrupts.CPU58.TLB:TLB_shootdowns 754.25 ± 5% -38.0% 467.50 interrupts.CPU59.TLB:TLB_shootdowns 393.75 ± 9% +311.4% 1619 ±109% interrupts.CPU6.RES:Rescheduling_interrupts 860.25 ± 12% -46.6% 459.00 ± 7% interrupts.CPU6.TLB:TLB_shootdowns 894.50 ± 16% -43.4% 506.00 ± 13% interrupts.CPU60.TLB:TLB_shootdowns 1684 ± 32% +78.1% 2999 ± 14% interrupts.CPU61.NMI:Non-maskable_interrupts 1684 ± 32% +78.1% 2999 ± 14% interrupts.CPU61.PMI:Performance_monitoring_interrupts 693.50 ± 17% -34.9% 451.75 ± 9% interrupts.CPU61.TLB:TLB_shootdowns 936.25 ± 16% -49.3% 474.75 ± 8% interrupts.CPU62.TLB:TLB_shootdowns 421.25 ± 44% +112.9% 897.00 ± 11% interrupts.CPU63.RES:Rescheduling_interrupts 731.25 ± 8% -41.1% 431.00 ± 7% interrupts.CPU63.TLB:TLB_shootdowns 883.00 ± 15% -43.7% 496.75 ± 7% interrupts.CPU64.TLB:TLB_shootdowns 736.00 ± 11% -36.8% 465.50 ± 5% interrupts.CPU65.TLB:TLB_shootdowns 880.00 ± 7% -42.6% 505.25 ± 8% interrupts.CPU66.TLB:TLB_shootdowns 943.50 ± 11% -40.5% 561.75 ± 10% interrupts.CPU68.TLB:TLB_shootdowns 393.25 ± 36% +100.5% 788.50 ± 26% interrupts.CPU69.RES:Rescheduling_interrupts 790.25 ± 9% -38.3% 487.50 ± 8% interrupts.CPU69.TLB:TLB_shootdowns 415.50 ± 24% +407.0% 2106 ±123% interrupts.CPU7.RES:Rescheduling_interrupts 679.00 ± 14% -36.8% 429.25 interrupts.CPU7.TLB:TLB_shootdowns 350.25 ± 13% +92.1% 672.75 ± 12% interrupts.CPU70.RES:Rescheduling_interrupts 778.75 ± 3% -41.8% 453.50 ± 11% interrupts.CPU70.TLB:TLB_shootdowns 673.50 ± 8% -39.8% 405.75 ± 11% interrupts.CPU71.TLB:TLB_shootdowns 460.75 ± 74% +95.7% 901.75 ± 43% interrupts.CPU74.RES:Rescheduling_interrupts 160.75 ± 33% +82.1% 292.75 ± 15% interrupts.CPU76.TLB:TLB_shootdowns 198.00 ± 22% +45.8% 288.75 ± 9% interrupts.CPU77.TLB:TLB_shootdowns 2305 ± 26% -35.5% 1486 ± 12% interrupts.CPU79.NMI:Non-maskable_interrupts 2305 ± 26% -35.5% 1486 ± 12% interrupts.CPU79.PMI:Performance_monitoring_interrupts 155.25 ± 31% +72.9% 268.50 ± 8% interrupts.CPU79.TLB:TLB_shootdowns 887.25 ± 17% -45.4% 484.50 ± 10% interrupts.CPU8.TLB:TLB_shootdowns 192.00 ± 18% +32.9% 255.25 ± 12% interrupts.CPU80.TLB:TLB_shootdowns 158.75 ± 30% +95.1% 309.75 ± 12% interrupts.CPU82.TLB:TLB_shootdowns 265.75 ± 35% +120.8% 586.75 ± 30% interrupts.CPU84.RES:Rescheduling_interrupts 146.75 ± 29% +80.2% 264.50 ± 20% interrupts.CPU84.TLB:TLB_shootdowns 186.00 ± 31% +56.0% 290.25 ± 9% interrupts.CPU86.TLB:TLB_shootdowns 169.25 ± 15% +66.5% 281.75 ± 2% interrupts.CPU88.TLB:TLB_shootdowns 879.75 ± 10% -47.5% 462.00 ± 4% interrupts.CPU9.TLB:TLB_shootdowns 161.00 ± 20% +75.8% 283.00 ± 5% interrupts.CPU90.TLB:TLB_shootdowns 194.75 ± 12% +50.3% 292.75 ± 11% interrupts.CPU91.TLB:TLB_shootdowns 2722 ± 34% -37.8% 1692 ± 6% interrupts.CPU92.NMI:Non-maskable_interrupts 2722 ± 34% -37.8% 1692 ± 6% interrupts.CPU92.PMI:Performance_monitoring_interrupts 190.75 ± 14% +36.3% 260.00 ± 22% interrupts.CPU92.TLB:TLB_shootdowns 188.00 ± 15% +57.0% 295.25 ± 10% interrupts.CPU94.TLB:TLB_shootdowns 200.25 ± 11% +59.6% 319.50 ± 16% interrupts.CPU95.TLB:TLB_shootdowns 316.25 ± 55% +105.1% 648.50 ± 19% interrupts.CPU96.RES:Rescheduling_interrupts 143.75 ± 42% +91.0% 274.50 ± 8% interrupts.CPU96.TLB:TLB_shootdowns 2409 ± 41% -34.2% 1586 ± 46% interrupts.CPU97.NMI:Non-maskable_interrupts 2409 ± 41% -34.2% 1586 ± 46% interrupts.CPU97.PMI:Performance_monitoring_interrupts 186.00 ± 33% +56.0% 290.25 ± 16% interrupts.CPU98.TLB:TLB_shootdowns vm-scalability.time.user_time 16000 +-------------------------------------------------------------------+ | + | 15000 |-+ :+ + + + | 14000 |-+ .++. +. + + +.+ :+ .++ ++. +.++ + + :+ + +| |+.+++ ++ + + + ++.+++ +.+++.++.+ ++ + +.+ + :+ | 13000 |-+ + | | | 12000 |-+ | | | 11000 |-+ | 10000 |-+ | | OO O O | 9000 |-+ OO O OO O OO O | |O OOO O O OO | 8000 +-------------------------------------------------------------------+ vm-scalability.time.voluntary_context_switches 1e+06 +------------------------------------------------------------------+ | O O | 900000 |O+OOO OO O OOO OOO O O OO | 800000 |-+ | | | 700000 |-+ | 600000 |-+ | | | 500000 |-+ | 400000 |-+ | | | 300000 |-+ | 200000 |-+ | |+.+++.+++.+ + +.+++.++.+++.+++.+++.+++. .+ +. | 100000 +------------------------------------------------------------------+ vm-scalability.throughput 1.6e+07 +-----------------------------------------------------------------+ | | 1.5e+07 |-+ + + + | 1.4e+07 |+. + + +. + + +. + + | | +++ +.+ +.+++ .++ +++ +.++ +++ +.+ +. | 1.3e+07 |-+ + +.+++ +++.++ +++.+++.+| 1.2e+07 |-+ | | | 1.1e+07 |-+ | 1e+07 |-+ | | | 9e+06 |-+ O O O O | 8e+06 |-+ O OO O OO O OO OO O | |O O O O O | 7e+06 +-----------------------------------------------------------------+ vm-scalability.median 44000 +-------------------------------------------------------------------+ 42000 |-+ + + + | | : : : : :: | 40000 |+. : +. + ++.+ .++. : + .+++. : +. + | 38000 |-++++ + +.+ ++.+++ +++ + ++.+ + +. .+++. +. ++. | | +++ + + +| 36000 |-+ | 34000 |-+ | 32000 |-+ | | | 30000 |-+ | 28000 |-+ O O O | | O O O O | 26000 |O+OOO O O OO OOO O O O | 24000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 7 months

3
10
0 / 0

[fs] b6509f6a8c: will-it-scale.per_thread_ops -12.6% regression

by kernel test robot

Greeting, FYI, we noticed a -12.6% regression of will-it-scale.per_thread_ops due to commit: commit: b6509f6a8c4313c068c69785c001451415969e44 ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: nr_task: 100% mode: thread test: eventfd1 cpufreq_governor: performance ucode: 0x5002f01 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops -6.4% regression | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=unix1 | | | ucode=0x5002f01 | +------------------+---------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops -2.3% regression | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=100% | | | test=pipe1 | | | ucode=0x5002f01 | +------------------+---------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/eventfd1/will-it-scale/0x5002f01 commit: v5.8-rc3 b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"") v5.8-rc3 b6509f6a8c4313c068c69785c00 ---------------- --------------------------- %stddev %change %stddev \ | \ 1652352 -12.6% 1444002 ± 2% will-it-scale.per_thread_ops 3.173e+08 -12.6% 2.772e+08 ± 2% will-it-scale.workload 23.95 -3.1 20.90 ± 2% mpstat.cpu.all.usr% 23.00 -12.0% 20.25 ± 4% vmstat.cpu.us 578849 ± 4% +8.6% 628428 ± 4% sched_debug.cpu.max_idle_balance_cost.max 0.00 ± 9% -14.4% 0.00 ± 13% sched_debug.cpu.next_balance.stddev 2207 ± 24% +34.7% 2973 ± 20% numa-vmstat.node0.nr_mapped 6949 ± 22% +43.7% 9984 ± 10% numa-vmstat.node0.nr_slab_reclaimable 9739 ± 13% -29.0% 6913 ± 12% numa-vmstat.node1.nr_slab_reclaimable 7247 ± 13% -28.4% 5190 ± 19% slabinfo.eventpoll_pwq.active_objs 7247 ± 13% -28.4% 5190 ± 19% slabinfo.eventpoll_pwq.num_objs 960.00 ± 11% -40.0% 576.00 ± 26% slabinfo.kmalloc-rcl-128.active_objs 960.00 ± 11% -40.0% 576.00 ± 26% slabinfo.kmalloc-rcl-128.num_objs 27801 ± 22% +43.7% 39940 ± 10% numa-meminfo.node0.KReclaimable 8718 ± 25% +35.0% 11773 ± 20% numa-meminfo.node0.Mapped 27801 ± 22% +43.7% 39940 ± 10% numa-meminfo.node0.SReclaimable 38959 ± 13% -29.0% 27656 ± 12% numa-meminfo.node1.KReclaimable 38959 ± 13% -29.0% 27656 ± 12% numa-meminfo.node1.SReclaimable 107406 ± 2% -13.6% 92844 ± 6% numa-meminfo.node1.Slab 0.00 +1.1 1.09 ± 16% perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.3 1.26 ± 18% perf-profile.calltrace.cycles-pp.fsnotify_parent.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.00 +1.8 1.76 ± 21% perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +3.7 3.70 ± 20% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +3.0 3.02 ± 17% perf-profile.children.cycles-pp.fsnotify_parent 0.00 +5.5 5.55 ± 20% perf-profile.children.cycles-pp.fsnotify 0.00 +2.7 2.66 ± 17% perf-profile.self.cycles-pp.fsnotify_parent 0.00 +5.4 5.35 ± 20% perf-profile.self.cycles-pp.fsnotify 10890 ± 21% +53.7% 16738 ± 12% interrupts.CPU168.CAL:Function_call_interrupts 4602 ± 5% +22.7% 5649 ± 13% interrupts.CPU169.CAL:Function_call_interrupts 4920 ± 13% -21.3% 3874 ± 2% interrupts.CPU170.CAL:Function_call_interrupts 463.80 ± 11% -25.1% 347.50 ± 4% interrupts.CPU170.RES:Rescheduling_interrupts 3619 ± 2% +10.2% 3990 ± 9% interrupts.CPU36.CAL:Function_call_interrupts 5768 ± 34% +43.3% 8267 ± 2% interrupts.CPU74.NMI:Non-maskable_interrupts 5768 ± 34% +43.3% 8267 ± 2% interrupts.CPU74.PMI:Performance_monitoring_interrupts 5769 ± 34% +43.3% 8268 ± 2% interrupts.CPU75.NMI:Non-maskable_interrupts 5769 ± 34% +43.3% 8268 ± 2% interrupts.CPU75.PMI:Performance_monitoring_interrupts 5769 ± 34% +43.3% 8267 ± 2% interrupts.CPU76.NMI:Non-maskable_interrupts 5769 ± 34% +43.3% 8267 ± 2% interrupts.CPU76.PMI:Performance_monitoring_interrupts 5768 ± 34% +43.3% 8268 ± 2% interrupts.CPU77.NMI:Non-maskable_interrupts 5768 ± 34% +43.3% 8268 ± 2% interrupts.CPU77.PMI:Performance_monitoring_interrupts 3602 ± 2% +12.9% 4068 ± 13% interrupts.CPU81.CAL:Function_call_interrupts 0.05 ± 32% +121.5% 0.10 ± 39% perf-stat.i.MPKI 1.06 -0.1 0.94 perf-stat.i.branch-miss-rate% 1.107e+09 -13.3% 9.596e+08 ± 2% perf-stat.i.branch-misses 199835 -11.5% 176905 ± 2% perf-stat.i.dTLB-store-misses 9.558e+10 +9.4% 1.046e+11 ± 2% perf-stat.i.dTLB-stores 9.687e+08 -13.8% 8.345e+08 ± 2% perf-stat.i.iTLB-load-misses 535.53 +20.3% 644.34 perf-stat.i.instructions-per-iTLB-miss 1.10 ± 12% +79.6% 1.98 ± 2% perf-stat.i.metric.K/sec 1.05 -0.1 0.92 perf-stat.overall.branch-miss-rate% 0.00 ± 13% -0.0 0.00 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate% 528.82 +20.8% 638.59 perf-stat.overall.instructions-per-iTLB-miss 487060 +20.3% 586095 perf-stat.overall.path-length 1.103e+09 -13.3% 9.564e+08 ± 2% perf-stat.ps.branch-misses 199661 -11.5% 176727 ± 2% perf-stat.ps.dTLB-store-misses 9.525e+10 +9.5% 1.043e+11 ± 2% perf-stat.ps.dTLB-stores 9.67e+08 -13.9% 8.33e+08 ± 3% perf-stat.ps.iTLB-load-misses will-it-scale.per_thread_ops 1.7e+06 +----------------------------------------------------------------+ | .+.. .+.+.. .+.. .+ | 1.65e+06 |.+ .+.+ + +.+. .+.+ +. .+.+. | | +. +. + | 1.6e+06 |-+ | | | 1.55e+06 |-+ | | | 1.5e+06 |-+ | | O O O O O O | 1.45e+06 |-O O O O O O O O O O O O O O O O O O O O | | O | 1.4e+06 |-+ | | | 1.35e+06 +----------------------------------------------------------------+ will-it-scale.workload 3.3e+08 +-----------------------------------------------------------------+ | | 3.2e+08 |.+ .+.+..+.+..+.+.+.. .+..+.+. .+.+ | | +..+ +.+ +..+ | 3.1e+08 |-+ | | | 3e+08 |-+ | | | 2.9e+08 |-+ | | | 2.8e+08 |-O O O O O O O O O O O O O O O O O O O O O O O O | | O O O | 2.7e+08 |-+ | | | 2.6e+08 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-csl-2ap4: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/unix1/will-it-scale/0x5002f01 commit: v5.8-rc3 b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"") v5.8-rc3 b6509f6a8c4313c068c69785c00 ---------------- --------------------------- %stddev %change %stddev \ | \ 720286 -6.4% 674507 will-it-scale.per_process_ops 1.383e+08 -6.4% 1.295e+08 will-it-scale.workload 97977 +1.4% 99342 proc-vmstat.nr_anon_pages 8.31 ± 41% +48.5% 12.33 ± 3% sched_debug.cpu.sched_goidle.min 1097 ± 8% +19.8% 1315 ± 9% slabinfo.task_group.active_objs 1097 ± 8% +19.8% 1315 ± 9% slabinfo.task_group.num_objs 13194 ± 25% -29.6% 9295 ± 24% softirqs.CPU124.RCU 9178 ± 8% -7.8% 8467 ± 4% softirqs.CPU48.RCU 1229 ± 68% +156.3% 3151 ± 70% interrupts.CPU0.CAL:Function_call_interrupts 8259 -37.6% 5157 ± 34% interrupts.CPU163.NMI:Non-maskable_interrupts 8259 -37.6% 5157 ± 34% interrupts.CPU163.PMI:Performance_monitoring_interrupts 15337 ± 81% +146.4% 37788 ± 45% numa-vmstat.node2.nr_active_anon 15187 ± 81% +147.5% 37594 ± 45% numa-vmstat.node2.nr_anon_pages 15337 ± 81% +146.4% 37788 ± 45% numa-vmstat.node2.nr_zone_active_anon 127540 ± 6% -8.0% 117311 numa-vmstat.node2.numa_other 62308 ± 81% +144.0% 152033 ± 45% numa-meminfo.node2.Active 61336 ± 81% +146.7% 151288 ± 45% numa-meminfo.node2.Active(anon) 15647 ±195% +497.8% 93536 ± 57% numa-meminfo.node2.AnonHugePages 60733 ± 82% +147.8% 150509 ± 45% numa-meminfo.node2.AnonPages 803985 ± 8% +10.5% 888495 ± 9% numa-meminfo.node2.MemUsed 1.14 -0.1 1.07 perf-stat.i.branch-miss-rate% 1.223e+09 -6.3% 1.146e+09 perf-stat.i.branch-misses 0.99 -1.5% 0.98 perf-stat.i.cpi 1.619e+11 +1.4% 1.641e+11 perf-stat.i.dTLB-loads 1e+11 +4.0% 1.04e+11 perf-stat.i.dTLB-stores 1.06e+09 -7.8% 9.779e+08 perf-stat.i.iTLB-load-misses 5.531e+11 +1.3% 5.603e+11 perf-stat.i.instructions 527.69 +9.7% 578.78 perf-stat.i.instructions-per-iTLB-miss 1.01 +1.4% 1.02 perf-stat.i.ipc 1931 +1.4% 1958 perf-stat.i.metric.M/sec 1.12 -0.1 1.06 perf-stat.overall.branch-miss-rate% 0.99 -1.4% 0.97 perf-stat.overall.cpi 0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate% 521.71 +9.8% 572.92 perf-stat.overall.instructions-per-iTLB-miss 1.01 +1.4% 1.03 perf-stat.overall.ipc 1206072 +8.1% 1303735 perf-stat.overall.path-length 1.218e+09 -6.3% 1.142e+09 perf-stat.ps.branch-misses 1.613e+11 +1.4% 1.635e+11 perf-stat.ps.dTLB-loads 9.97e+10 +4.0% 1.036e+11 perf-stat.ps.dTLB-stores 1.057e+09 -7.8% 9.747e+08 perf-stat.ps.iTLB-load-misses 5.512e+11 +1.3% 5.584e+11 perf-stat.ps.instructions 1.668e+14 +1.2% 1.688e+14 perf-stat.total.instructions 38.39 -2.2 36.14 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 36.80 -2.1 34.66 perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 34.46 -2.0 32.44 perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write 33.29 -1.8 31.47 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.96 -1.8 29.19 perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write 31.72 -1.7 30.05 perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64 25.95 -1.5 24.46 perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read 53.97 -1.5 52.49 perf-profile.calltrace.cycles-pp.write 24.74 -1.4 23.30 perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read 47.12 -1.0 46.10 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 46.34 -1.0 45.38 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 44.29 -0.9 43.43 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 42.43 -0.8 41.64 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 13.05 -0.8 12.28 perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 11.31 -0.7 10.65 perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 10.31 -0.6 9.71 perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg 8.93 -0.5 8.39 perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 8.64 -0.5 8.14 perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter 8.31 -0.5 7.85 perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg 7.91 -0.5 7.45 perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 6.55 -0.4 6.19 perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 0.54 ± 2% -0.3 0.25 ±100% perf-profile.calltrace.cycles-pp.___might_sleep.__might_fault._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg 3.81 -0.2 3.58 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.write 3.98 -0.2 3.75 perf-profile.calltrace.cycles-pp.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg 3.63 -0.2 3.41 perf-profile.calltrace.cycles-pp._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 3.88 -0.2 3.65 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.read 3.82 -0.2 3.60 perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter 2.87 -0.2 2.66 perf-profile.calltrace.cycles-pp.security_socket_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write 3.71 -0.2 3.50 perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg 3.62 -0.2 3.42 perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic 3.91 -0.2 3.71 perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic 2.95 -0.2 2.76 perf-profile.calltrace.cycles-pp.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read.vfs_read 3.50 -0.2 3.31 perf-profile.calltrace.cycles-pp.__check_object_size.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 3.33 -0.2 3.15 perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb 3.44 -0.2 3.25 perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic 3.09 -0.2 2.91 perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg 3.65 -0.2 3.47 perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor 2.38 -0.2 2.21 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.08 -0.1 1.94 perf-profile.calltrace.cycles-pp.aa_sk_perm.security_socket_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 2.63 -0.1 2.50 perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb 3.44 -0.1 3.31 perf-profile.calltrace.cycles-pp.sock_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read 1.71 -0.1 1.59 perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 1.69 -0.1 1.58 perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 1.96 -0.1 1.85 perf-profile.calltrace.cycles-pp.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read 1.44 -0.1 1.34 perf-profile.calltrace.cycles-pp.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 1.57 -0.1 1.48 perf-profile.calltrace.cycles-pp.copyin._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg 1.47 -0.1 1.39 perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor 1.20 -0.1 1.13 perf-profile.calltrace.cycles-pp.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 0.70 -0.1 0.63 ± 4% perf-profile.calltrace.cycles-pp.__might_sleep.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb 1.10 ± 2% -0.1 1.03 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.98 -0.1 0.91 perf-profile.calltrace.cycles-pp.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 1.01 ± 2% -0.1 0.94 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.01 -0.1 0.94 perf-profile.calltrace.cycles-pp.__might_fault._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg 1.13 -0.1 1.07 perf-profile.calltrace.cycles-pp.unix_write_space.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all 1.05 -0.1 0.99 ± 3% perf-profile.calltrace.cycles-pp.__ksize.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg 0.96 -0.1 0.91 perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 0.96 -0.1 0.90 perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 0.98 -0.1 0.93 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.write 1.04 -0.1 0.98 perf-profile.calltrace.cycles-pp.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 1.00 -0.1 0.94 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.read 0.80 -0.1 0.75 perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 1.02 -0.0 0.97 perf-profile.calltrace.cycles-pp.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 0.85 -0.0 0.80 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter 0.68 ± 2% -0.0 0.64 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.71 -0.0 0.67 perf-profile.calltrace.cycles-pp.___might_sleep.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags 0.70 ± 2% -0.0 0.66 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyin._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg 0.85 -0.0 0.81 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 0.56 -0.0 0.52 ± 2% perf-profile.calltrace.cycles-pp.apparmor_socket_recvmsg.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read 0.57 -0.0 0.54 perf-profile.calltrace.cycles-pp.___might_sleep.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 0.68 -0.0 0.64 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter 0.57 -0.0 0.54 perf-profile.calltrace.cycles-pp.__might_sleep.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags 1.68 -0.0 1.64 perf-profile.calltrace.cycles-pp.mutex_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 0.65 -0.0 0.62 perf-profile.calltrace.cycles-pp.mutex_unlock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 0.00 +0.6 0.57 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.6 0.62 perf-profile.calltrace.cycles-pp.fsnotify_parent.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.00 +0.6 0.64 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.50 +0.7 3.20 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.9 0.93 perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 48.49 +1.3 49.77 perf-profile.calltrace.cycles-pp.read 41.56 +1.7 43.30 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 40.78 +1.8 42.56 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 38.81 +1.9 40.68 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 37.05 +1.9 38.94 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.00 +1.9 1.90 perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 38.49 -2.2 36.24 perf-profile.children.cycles-pp.new_sync_write 36.86 -2.1 34.71 perf-profile.children.cycles-pp.sock_write_iter 34.57 -2.0 32.53 perf-profile.children.cycles-pp.sock_sendmsg 33.36 -1.8 31.55 perf-profile.children.cycles-pp.new_sync_read 31.28 -1.8 29.50 perf-profile.children.cycles-pp.unix_stream_sendmsg 31.77 -1.7 30.10 perf-profile.children.cycles-pp.sock_read_iter 26.00 -1.5 24.50 perf-profile.children.cycles-pp.unix_stream_recvmsg 54.00 -1.5 52.53 perf-profile.children.cycles-pp.write 24.99 -1.5 23.53 perf-profile.children.cycles-pp.unix_stream_read_generic 44.34 -0.9 43.48 perf-profile.children.cycles-pp.ksys_write 13.14 -0.8 12.36 perf-profile.children.cycles-pp.sock_alloc_send_pskb 42.57 -0.7 41.85 perf-profile.children.cycles-pp.vfs_write 11.37 -0.7 10.71 perf-profile.children.cycles-pp.alloc_skb_with_frags 10.51 -0.6 9.90 perf-profile.children.cycles-pp.__alloc_skb 9.00 -0.5 8.46 perf-profile.children.cycles-pp.unix_stream_read_actor 8.72 -0.5 8.21 perf-profile.children.cycles-pp.skb_copy_datagram_iter 8.00 -0.5 7.53 perf-profile.children.cycles-pp.skb_copy_datagram_from_iter 8.38 -0.5 7.91 perf-profile.children.cycles-pp.__skb_datagram_iter 7.43 -0.4 7.04 perf-profile.children.cycles-pp.__check_object_size 6.57 -0.4 6.21 perf-profile.children.cycles-pp.consume_skb 5.14 -0.3 4.83 perf-profile.children.cycles-pp.entry_SYSCALL_64 4.60 -0.3 4.32 perf-profile.children.cycles-pp.syscall_return_via_sysret 4.16 -0.3 3.89 perf-profile.children.cycles-pp.aa_sk_perm 4.22 -0.3 3.96 perf-profile.children.cycles-pp.___might_sleep 3.50 -0.2 3.26 perf-profile.children.cycles-pp.common_file_perm 4.07 -0.2 3.83 perf-profile.children.cycles-pp.__kmalloc_reserve 3.71 -0.2 3.48 perf-profile.children.cycles-pp._copy_from_iter 3.85 -0.2 3.63 perf-profile.children.cycles-pp.skb_release_all 2.95 -0.2 2.73 perf-profile.children.cycles-pp.__might_sleep 2.91 -0.2 2.70 perf-profile.children.cycles-pp.security_socket_sendmsg 3.72 -0.2 3.51 perf-profile.children.cycles-pp._copy_to_iter 3.76 -0.2 3.55 perf-profile.children.cycles-pp.skb_release_head_state 3.01 -0.2 2.81 perf-profile.children.cycles-pp.security_socket_recvmsg 3.54 -0.2 3.33 perf-profile.children.cycles-pp.__kmalloc_node_track_caller 3.97 -0.2 3.77 perf-profile.children.cycles-pp.simple_copy_to_iter 3.28 -0.2 3.09 perf-profile.children.cycles-pp.kmem_cache_alloc_node 3.54 -0.2 3.35 perf-profile.children.cycles-pp.unix_destruct_scm 2.36 ± 2% -0.2 2.21 ± 2% perf-profile.children.cycles-pp.__fdget_pos 2.64 -0.1 2.50 perf-profile.children.cycles-pp.sock_wfree 3.50 -0.1 3.37 perf-profile.children.cycles-pp.sock_recvmsg 2.01 ± 2% -0.1 1.88 ± 3% perf-profile.children.cycles-pp.__fget_light 2.10 ± 2% -0.1 1.97 ± 2% perf-profile.children.cycles-pp.__might_fault 1.97 -0.1 1.85 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 1.54 -0.1 1.44 perf-profile.children.cycles-pp.wait_for_unix_gc 1.95 -0.1 1.85 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 1.43 -0.1 1.34 perf-profile.children.cycles-pp._cond_resched 1.60 -0.1 1.51 perf-profile.children.cycles-pp.copyin 1.52 -0.1 1.43 perf-profile.children.cycles-pp.__prepare_exit_to_usermode 1.67 ± 3% -0.1 1.58 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 1.64 -0.1 1.55 perf-profile.children.cycles-pp.copy_user_generic_unrolled 1.50 -0.1 1.42 perf-profile.children.cycles-pp.copyout 1.21 -0.1 1.13 perf-profile.children.cycles-pp.sock_def_readable 0.98 -0.1 0.91 perf-profile.children.cycles-pp.skb_set_owner_w 0.95 -0.1 0.89 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 1.05 -0.1 0.99 ± 2% perf-profile.children.cycles-pp.__ksize 0.84 -0.1 0.78 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 1.13 -0.1 1.08 perf-profile.children.cycles-pp.unix_write_space 1.05 -0.1 0.99 perf-profile.children.cycles-pp.skb_unlink 0.74 ± 2% -0.1 0.69 perf-profile.children.cycles-pp.apparmor_file_permission 1.02 -0.1 0.97 perf-profile.children.cycles-pp.skb_queue_tail 1.75 -0.0 1.70 perf-profile.children.cycles-pp.mutex_lock 0.56 -0.0 0.52 ± 2% perf-profile.children.cycles-pp.apparmor_socket_recvmsg 0.71 -0.0 0.67 perf-profile.children.cycles-pp.rcu_all_qs 0.67 -0.0 0.63 perf-profile.children.cycles-pp.__syscall_return_slowpath 0.67 -0.0 0.64 perf-profile.children.cycles-pp.check_stack_object 0.44 -0.0 0.41 perf-profile.children.cycles-pp.apparmor_socket_sendmsg 0.67 -0.0 0.65 perf-profile.children.cycles-pp.mutex_unlock 0.30 -0.0 0.28 ± 2% perf-profile.children.cycles-pp.put_pid 0.23 -0.0 0.21 ± 2% perf-profile.children.cycles-pp.skb_release_data 0.43 +0.0 0.46 perf-profile.children.cycles-pp.apparmor_socket_getpeersec_dgram 4.96 +0.6 5.57 perf-profile.children.cycles-pp.security_file_permission 88.79 +0.7 89.51 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 87.30 +0.8 88.11 perf-profile.children.cycles-pp.do_syscall_64 48.55 +1.3 49.85 perf-profile.children.cycles-pp.read 0.00 +1.9 1.87 perf-profile.children.cycles-pp.fsnotify_parent 38.85 +1.9 40.72 perf-profile.children.cycles-pp.ksys_read 37.17 +2.0 39.17 perf-profile.children.cycles-pp.vfs_read 0.00 +2.9 2.90 perf-profile.children.cycles-pp.fsnotify 4.56 -0.3 4.28 perf-profile.self.cycles-pp.syscall_return_via_sysret 4.10 -0.2 3.85 perf-profile.self.cycles-pp.___might_sleep 2.96 -0.2 2.75 perf-profile.self.cycles-pp.unix_stream_read_generic 2.65 -0.2 2.46 perf-profile.self.cycles-pp.__might_sleep 2.94 -0.2 2.75 perf-profile.self.cycles-pp.unix_stream_sendmsg 2.85 ± 2% -0.2 2.67 perf-profile.self.cycles-pp.common_file_perm 1.40 ± 4% -0.2 1.22 perf-profile.self.cycles-pp.new_sync_read 2.32 -0.2 2.17 perf-profile.self.cycles-pp.aa_sk_perm 2.56 -0.1 2.41 perf-profile.self.cycles-pp.entry_SYSCALL_64 1.90 ± 2% -0.1 1.77 ± 3% perf-profile.self.cycles-pp.__fget_light 2.13 -0.1 2.00 perf-profile.self.cycles-pp.__alloc_skb 2.29 -0.1 2.17 perf-profile.self.cycles-pp.sock_write_iter 1.93 -0.1 1.81 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 1.79 -0.1 1.67 perf-profile.self.cycles-pp.write 1.83 -0.1 1.72 perf-profile.self.cycles-pp.read 1.52 -0.1 1.42 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 1.71 -0.1 1.61 perf-profile.self.cycles-pp.__kmalloc_node_track_caller 1.93 -0.1 1.84 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 1.60 ± 3% -0.1 1.52 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 1.81 -0.1 1.73 perf-profile.self.cycles-pp.kmem_cache_alloc_node 1.17 -0.1 1.09 perf-profile.self.cycles-pp.sock_def_readable 1.55 -0.1 1.47 perf-profile.self.cycles-pp.copy_user_generic_unrolled 1.25 ± 2% -0.1 1.17 ± 2% perf-profile.self.cycles-pp._copy_to_iter 0.61 -0.1 0.54 perf-profile.self.cycles-pp.ksys_write 1.44 -0.1 1.37 perf-profile.self.cycles-pp.sock_wfree 0.95 -0.1 0.88 perf-profile.self.cycles-pp.skb_set_owner_w 1.06 -0.1 0.99 ± 2% perf-profile.self.cycles-pp._copy_from_iter 0.58 ± 5% -0.1 0.52 ± 5% perf-profile.self.cycles-pp.aa_file_perm 1.04 -0.1 0.97 ± 2% perf-profile.self.cycles-pp.__ksize 1.10 -0.1 1.04 perf-profile.self.cycles-pp.unix_write_space 0.68 ± 2% -0.1 0.62 perf-profile.self.cycles-pp.__skb_datagram_iter 0.93 -0.1 0.87 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.52 -0.1 0.47 perf-profile.self.cycles-pp.ksys_read 0.68 ± 2% -0.1 0.63 ± 2% perf-profile.self.cycles-pp.apparmor_file_permission 0.83 -0.0 0.78 perf-profile.self.cycles-pp.unix_destruct_scm 0.87 -0.0 0.82 perf-profile.self.cycles-pp.consume_skb 0.83 -0.0 0.79 perf-profile.self.cycles-pp.sock_alloc_send_pskb 0.71 -0.0 0.67 perf-profile.self.cycles-pp._cond_resched 0.82 -0.0 0.78 perf-profile.self.cycles-pp.alloc_skb_with_frags 0.29 ± 3% -0.0 0.25 perf-profile.self.cycles-pp.unix_stream_read_actor 0.66 -0.0 0.62 perf-profile.self.cycles-pp.skb_copy_datagram_from_iter 0.53 -0.0 0.49 ± 2% perf-profile.self.cycles-pp.apparmor_socket_recvmsg 0.64 -0.0 0.60 perf-profile.self.cycles-pp.__syscall_return_slowpath 0.60 ± 2% -0.0 0.56 ± 4% perf-profile.self.cycles-pp.__might_fault 0.50 -0.0 0.46 perf-profile.self.cycles-pp.__kmalloc_reserve 0.37 -0.0 0.34 perf-profile.self.cycles-pp.security_socket_recvmsg 2.25 -0.0 2.21 perf-profile.self.cycles-pp.sock_read_iter 0.54 -0.0 0.51 perf-profile.self.cycles-pp.rcu_all_qs 0.37 ± 5% -0.0 0.34 ± 4% perf-profile.self.cycles-pp.sock_sendmsg 0.43 -0.0 0.40 perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.98 -0.0 0.95 perf-profile.self.cycles-pp.unix_stream_recvmsg 0.35 -0.0 0.32 perf-profile.self.cycles-pp.skb_copy_datagram_iter 0.41 -0.0 0.38 perf-profile.self.cycles-pp.apparmor_socket_sendmsg 0.56 -0.0 0.54 perf-profile.self.cycles-pp.check_stack_object 0.44 -0.0 0.41 perf-profile.self.cycles-pp.wait_for_unix_gc 0.29 ± 3% -0.0 0.26 ± 2% perf-profile.self.cycles-pp.security_socket_sendmsg 0.42 -0.0 0.40 perf-profile.self.cycles-pp.security_socket_getpeersec_dgram 0.63 -0.0 0.61 perf-profile.self.cycles-pp.mutex_unlock 0.40 ± 2% -0.0 0.38 perf-profile.self.cycles-pp.__fdget_pos 0.31 -0.0 0.29 perf-profile.self.cycles-pp.copyin 0.23 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.skb_release_data 0.19 -0.0 0.17 ± 2% perf-profile.self.cycles-pp.skb_release_head_state 0.24 -0.0 0.23 perf-profile.self.cycles-pp.copyout 0.21 ± 2% -0.0 0.19 ± 2% perf-profile.self.cycles-pp.simple_copy_to_iter 0.26 -0.0 0.25 perf-profile.self.cycles-pp.kmalloc_slab 0.17 ± 2% +0.0 0.20 ± 2% perf-profile.self.cycles-pp.__vfs_read 0.40 +0.0 0.43 perf-profile.self.cycles-pp.apparmor_socket_getpeersec_dgram 0.73 +0.0 0.77 perf-profile.self.cycles-pp.mutex_lock 0.51 +0.1 0.57 perf-profile.self.cycles-pp.sock_recvmsg 1.33 +0.2 1.57 perf-profile.self.cycles-pp.vfs_write 0.75 ± 6% +0.4 1.10 ± 3% perf-profile.self.cycles-pp.security_file_permission 0.97 +0.5 1.49 ± 2% perf-profile.self.cycles-pp.vfs_read 0.00 +1.7 1.65 perf-profile.self.cycles-pp.fsnotify_parent 0.00 +2.8 2.80 perf-profile.self.cycles-pp.fsnotify *************************************************************************************************** lkp-csl-2ap4: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/pipe1/will-it-scale/0x5002f01 commit: v5.8-rc3 b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"") v5.8-rc3 b6509f6a8c4313c068c69785c00 ---------------- --------------------------- %stddev %change %stddev \ | \ 1045643 -2.3% 1021331 will-it-scale.per_thread_ops 2.008e+08 -2.3% 1.961e+08 will-it-scale.workload 29490176 ± 3% -9.8% 26607616 ± 3% meminfo.DirectMap2M 202721 ± 4% +11.5% 225943 ± 3% numa-numastat.node2.local_node 25892 ± 28% -75.5% 6341 ± 97% numa-numastat.node2.other_node 1843 +8.9% 2007 ± 5% numa-vmstat.node2.nr_mapped 130681 ± 5% -14.7% 111436 ± 5% numa-vmstat.node2.numa_other 9801 ± 5% +16.0% 11370 ± 7% sched_debug.cpu.ttwu_count.max 3052 ± 12% +23.6% 3771 ± 11% sched_debug.cpu.ttwu_local.max 136156 +1.4% 138002 proc-vmstat.nr_active_anon 109009 +1.1% 110229 proc-vmstat.nr_anon_pages 136156 +1.4% 138002 proc-vmstat.nr_zone_active_anon 1120468 +1.6% 1137854 proc-vmstat.pgalloc_normal 337.83 ± 61% +53.8% 519.75 ± 47% interrupts.39:PCI-MSI.524290-edge.eth0-TxRx-1 337.83 ± 61% +53.8% 519.75 ± 47% interrupts.CPU11.39:PCI-MSI.524290-edge.eth0-TxRx-1 303.33 +45.7% 442.00 ± 44% interrupts.CPU148.RES:Rescheduling_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU155.NMI:Non-maskable_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU155.PMI:Performance_monitoring_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU157.NMI:Non-maskable_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU157.PMI:Performance_monitoring_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU158.NMI:Non-maskable_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU158.PMI:Performance_monitoring_interrupts 15130 ± 14% -18.1% 12396 ± 8% interrupts.CPU168.CAL:Function_call_interrupts 4137 ± 15% +44.3% 5970 ± 26% interrupts.CPU48.CAL:Function_call_interrupts 336.00 ± 6% +60.0% 537.75 ± 31% interrupts.CPU48.RES:Rescheduling_interrupts 0.75 -0.0 0.73 perf-stat.i.branch-miss-rate% 8.192e+08 -2.2% 8.01e+08 perf-stat.i.branch-misses 0.98 -1.4% 0.97 perf-stat.i.cpi 1.7e+11 +1.5% 1.725e+11 perf-stat.i.dTLB-loads 1.055e+11 +2.7% 1.084e+11 perf-stat.i.dTLB-stores 7.304e+08 -2.5% 7.123e+08 perf-stat.i.iTLB-load-misses 5.6e+11 +1.6% 5.688e+11 perf-stat.i.instructions 775.04 +4.2% 807.76 perf-stat.i.instructions-per-iTLB-miss 1.02 +1.5% 1.04 perf-stat.i.ipc 2015 +1.5% 2046 perf-stat.i.metric.M/sec 0.73 -0.0 0.72 perf-stat.overall.branch-miss-rate% 0.98 -1.5% 0.96 perf-stat.overall.cpi 0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate% 764.55 +4.1% 796.24 perf-stat.overall.instructions-per-iTLB-miss 1.02 +1.5% 1.04 perf-stat.overall.ipc 841233 +4.0% 874479 perf-stat.overall.path-length 8.164e+08 -2.2% 7.983e+08 perf-stat.ps.branch-misses 1.695e+11 +1.5% 1.72e+11 perf-stat.ps.dTLB-loads 1.052e+11 +2.7% 1.08e+11 perf-stat.ps.dTLB-stores 7.301e+08 -2.5% 7.119e+08 perf-stat.ps.iTLB-load-misses 5.582e+11 +1.6% 5.668e+11 perf-stat.ps.instructions 1.689e+14 +1.5% 1.715e+14 perf-stat.total.instructions 50.20 -1.4 48.82 perf-profile.calltrace.cycles-pp.__GI___libc_read 40.68 -1.2 39.52 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read 39.53 -1.1 38.39 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 36.75 -1.1 35.70 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 32.27 -0.9 31.36 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 24.69 -0.6 24.14 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 22.23 -0.5 21.72 perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 21.04 -0.5 20.55 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.70 -0.4 18.27 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 6.80 -0.1 6.65 perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 5.39 -0.1 5.26 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__GI___libc_write 2.96 -0.1 2.84 ± 2% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.27 -0.1 2.15 perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 2.42 -0.1 2.31 ± 2% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 5.38 -0.1 5.28 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__GI___libc_read 2.61 -0.1 2.52 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read 3.46 -0.1 3.38 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 2.62 -0.1 2.55 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 2.08 -0.1 2.02 perf-profile.calltrace.cycles-pp.__pthread_disable_asynccancel 0.99 -0.1 0.94 perf-profile.calltrace.cycles-pp.fput_many.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 1.04 -0.0 1.00 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 0.97 -0.0 0.93 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyin.copy_page_from_iter.pipe_write.new_sync_write 1.23 -0.0 1.19 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 1.11 -0.0 1.08 perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyout.copy_page_to_iter.pipe_read.new_sync_read 1.20 -0.0 1.17 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.pipe_write.new_sync_write.vfs_write.ksys_write 1.01 -0.0 0.98 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 0.79 -0.0 0.76 perf-profile.calltrace.cycles-pp.___might_sleep.mutex_lock.pipe_read.new_sync_read.vfs_read 1.50 -0.0 1.47 perf-profile.calltrace.cycles-pp.__might_fault.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 1.13 -0.0 1.11 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.pipe_read.new_sync_read.vfs_read.ksys_read 0.79 -0.0 0.77 perf-profile.calltrace.cycles-pp.___might_sleep.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 0.72 -0.0 0.70 perf-profile.calltrace.cycles-pp.__sb_end_write.pipe_write.new_sync_write.vfs_write.ksys_write 0.99 -0.0 0.97 perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +0.9 0.92 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 49.09 +1.4 50.48 perf-profile.calltrace.cycles-pp.__GI___libc_write 0.00 +1.5 1.50 ± 5% perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 39.33 +1.6 40.95 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write 38.18 +1.7 39.84 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 35.19 +1.8 37.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 30.77 +1.8 32.61 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 50.18 -1.4 48.80 perf-profile.children.cycles-pp.__GI___libc_read 36.87 -1.1 35.81 perf-profile.children.cycles-pp.ksys_read 32.66 -0.9 31.73 perf-profile.children.cycles-pp.vfs_read 24.84 -0.6 24.29 perf-profile.children.cycles-pp.new_sync_write 22.51 -0.5 22.00 perf-profile.children.cycles-pp.pipe_write 21.18 -0.5 20.69 perf-profile.children.cycles-pp.new_sync_read 18.94 -0.4 18.50 perf-profile.children.cycles-pp.pipe_read 5.44 -0.2 5.27 perf-profile.children.cycles-pp.mutex_lock 7.06 -0.2 6.90 perf-profile.children.cycles-pp.copy_page_from_iter 6.34 -0.2 6.19 perf-profile.children.cycles-pp.syscall_return_via_sysret 7.28 -0.1 7.13 perf-profile.children.cycles-pp.entry_SYSCALL_64 2.33 -0.1 2.21 perf-profile.children.cycles-pp.copyin 7.14 -0.1 7.03 perf-profile.children.cycles-pp.copy_page_to_iter 2.48 -0.1 2.36 ± 2% perf-profile.children.cycles-pp.copyout 4.78 -0.1 4.67 perf-profile.children.cycles-pp.___might_sleep 2.48 -0.1 2.39 perf-profile.children.cycles-pp.copy_user_generic_unrolled 2.23 -0.1 2.14 perf-profile.children.cycles-pp.__prepare_exit_to_usermode 3.56 -0.1 3.48 perf-profile.children.cycles-pp.__wake_up_common_lock 2.69 -0.1 2.63 perf-profile.children.cycles-pp.__might_sleep 2.41 -0.1 2.35 perf-profile.children.cycles-pp._raw_spin_lock_irq 2.08 -0.1 2.02 perf-profile.children.cycles-pp.__pthread_disable_asynccancel 1.88 -0.1 1.82 perf-profile.children.cycles-pp.fput_many 1.40 -0.0 1.35 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.34 -0.0 0.29 perf-profile.children.cycles-pp.fput 1.38 -0.0 1.34 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.24 ± 7% -0.0 0.22 perf-profile.children.cycles-pp.tick_sched_handle 0.92 -0.0 0.90 perf-profile.children.cycles-pp.fpregs_assert_state_consistent 1.28 -0.0 1.26 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.75 -0.0 0.73 perf-profile.children.cycles-pp.__sb_end_write 80.17 +0.5 80.63 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 77.95 +0.5 78.47 perf-profile.children.cycles-pp.do_syscall_64 1.98 +0.9 2.88 perf-profile.children.cycles-pp.fsnotify_parent 3.04 +1.4 4.42 ± 2% perf-profile.children.cycles-pp.fsnotify 49.09 +1.4 50.48 perf-profile.children.cycles-pp.__GI___libc_write 35.45 +1.8 37.22 perf-profile.children.cycles-pp.ksys_write 30.96 +2.0 32.93 perf-profile.children.cycles-pp.vfs_write 6.28 -0.1 6.13 perf-profile.self.cycles-pp.syscall_return_via_sysret 4.68 -0.1 4.58 perf-profile.self.cycles-pp.___might_sleep 2.35 -0.1 2.26 perf-profile.self.cycles-pp.copy_user_generic_unrolled 2.26 -0.1 2.18 perf-profile.self.cycles-pp.mutex_lock 2.61 -0.1 2.54 perf-profile.self.cycles-pp.__GI___libc_write 3.09 -0.1 3.02 perf-profile.self.cycles-pp.pipe_write 2.57 -0.1 2.50 perf-profile.self.cycles-pp.__GI___libc_read 2.26 -0.1 2.19 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 3.74 -0.1 3.68 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.93 -0.1 0.87 perf-profile.self.cycles-pp.ksys_write 1.38 -0.1 1.32 perf-profile.self.cycles-pp.__prepare_exit_to_usermode 2.31 -0.1 2.25 perf-profile.self.cycles-pp._raw_spin_lock_irq 1.78 -0.1 1.72 perf-profile.self.cycles-pp.fput_many 1.99 -0.1 1.94 perf-profile.self.cycles-pp.__pthread_disable_asynccancel 2.37 -0.0 2.32 perf-profile.self.cycles-pp.__might_sleep 1.14 -0.0 1.10 perf-profile.self.cycles-pp.do_syscall_64 1.36 -0.0 1.31 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 2.22 -0.0 2.18 perf-profile.self.cycles-pp.__pthread_enable_asynccancel 1.87 -0.0 1.83 perf-profile.self.cycles-pp.mutex_unlock 1.34 -0.0 1.31 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.91 -0.0 0.89 perf-profile.self.cycles-pp.file_update_time 0.85 -0.0 0.82 perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.44 -0.0 0.41 ± 2% perf-profile.self.cycles-pp.touch_atime 0.73 -0.0 0.71 perf-profile.self.cycles-pp.__sb_end_write 1.23 -0.0 1.21 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.21 -0.0 0.20 perf-profile.self.cycles-pp.kill_fasync 2.08 ± 2% +0.4 2.49 ± 4% perf-profile.self.cycles-pp.vfs_write 1.71 +0.8 2.51 perf-profile.self.cycles-pp.fsnotify_parent 2.94 +1.3 4.25 ± 2% perf-profile.self.cycles-pp.fsnotify Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 7 months

3
3
0 / 0

[mm] 5ef64cc898: vm-scalability.throughput -20.6% regression

by kernel test robot

Greeting, FYI, we noticed a -20.6% regression of vm-scalability.throughput due to commit: commit: 5ef64cc8987a9211d3f3667331ba3411a94ddc79 ("mm: allow a controlled amount of unfairness in the page lock") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: vm-scalability on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory with following parameters: runtime: 300s size: 2T test: shm-xread-seq cpufreq_governor: performance ucode: 0xd6 test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us. test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/ In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------+ | testcase: change | unixbench: unixbench.score 5.4% improvement | | test machine | 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory | | test parameters | cpufreq_governor=performance | | | nr_task=30% | | | runtime=300s | | | test=shell8 | | | ucode=0xd6 | +------------------+---------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops 688.3% improvement | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=50% | | | test=readseek2 | | | ucode=0x5002f01 | +------------------+---------------------------------------------------------------------------+ | testcase: change | stress-ng: stress-ng.timerfd.ops_per_sec -3.2% regression | | test machine | 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory | | test parameters | class=interrupt | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | testtime=30s | | | ucode=0x5002f01 | +------------------+---------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/debian-10.4-x86_64-20200603.cgz/300s/2T/lkp-cfl-e1/shm-xread-seq/vm-scalability/0xd6 commit: 5925fa68fe (" perf tools fixes for v5.9: 3rd batch") 5ef64cc898 ("mm: allow a controlled amount of unfairness in the page lock") 5925fa68fe824465 5ef64cc8987a9211d3f3667331b ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 3:4 -34% 1:4 perf-profile.calltrace.cycles-pp.sync_regs.error_entry 3:4 -24% 2:4 perf-profile.calltrace.cycles-pp.error_entry 5:4 -30% 4:4 perf-profile.children.cycles-pp.error_entry 0:4 -4% 0:4 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 985306 -20.6% 782124 vm-scalability.median 5.42 ± 4% -2.3 3.15 ± 3% vm-scalability.median_stddev% 15764904 -20.6% 12513939 vm-scalability.throughput 173.16 +22.0% 211.25 vm-scalability.time.elapsed_time 173.16 +22.0% 211.25 vm-scalability.time.elapsed_time.max 15477 ± 4% +129.1% 35465 ± 3% vm-scalability.time.involuntary_context_switches 2.569e+08 +56.2% 4.013e+08 vm-scalability.time.minor_page_faults 1283 +12.8% 1447 vm-scalability.time.percent_of_cpu_this_job_got 971.71 +87.0% 1816 vm-scalability.time.system_time 56042863 -10.0% 50440135 vm-scalability.time.voluntary_context_switches 197.11 +19.3% 235.15 uptime.boot 1052993 -19.6% 846362 meminfo.Active 1052899 -19.6% 846266 meminfo.Active(anon) 105575 -17.9% 86662 meminfo.max_used_kB 20.13 -12.0 8.10 mpstat.cpu.all.idle% 0.61 ± 3% -0.6 0.01 ± 3% mpstat.cpu.all.iowait% 0.95 +0.6 1.53 mpstat.cpu.all.irq% 34.10 +19.4 53.49 mpstat.cpu.all.sys% 44.17 -7.4 36.81 mpstat.cpu.all.usr% 20.75 ± 2% -60.2% 8.25 ± 5% vmstat.cpu.id 34.25 +57.7% 54.00 vmstat.cpu.sy 43.25 -16.8% 36.00 vmstat.cpu.us 11.00 +27.3% 14.00 vmstat.procs.r 637460 -26.0% 471762 vmstat.system.cs 68148 +88.2% 128254 vmstat.system.in 41584830 ± 3% -86.3% 5680589 cpuidle.C1.time 7034552 -52.7% 3328990 cpuidle.C1.usage 166290 ± 42% -52.8% 78434 ± 30% cpuidle.C10.time 1.039e+08 -87.4% 13042277 ± 3% cpuidle.C1E.time 9612513 ± 2% -96.9% 294433 ± 3% cpuidle.C1E.usage 2.713e+08 -59.5% 1.1e+08 cpuidle.POLL.time 48132156 -40.9% 28439987 cpuidle.POLL.usage 264158 -19.1% 213736 proc-vmstat.nr_active_anon 2482625 -1.7% 2441126 proc-vmstat.nr_file_pages 2240838 -1.9% 2199269 proc-vmstat.nr_shmem 264158 -19.1% 213736 proc-vmstat.nr_zone_active_anon 40104796 +19.0% 47743573 proc-vmstat.numa_hit 40104796 +19.0% 47743573 proc-vmstat.numa_local 40189131 +19.0% 47826525 proc-vmstat.pgalloc_normal 2.572e+08 +56.2% 4.016e+08 proc-vmstat.pgfault 39751695 +17.5% 46709736 ± 2% proc-vmstat.pgfree 19586 ± 2% +25.4% 24554 proc-vmstat.pgreuse 12821 ± 11% +70.1% 21804 ± 7% softirqs.CPU0.RCU 55373 +58.2% 87598 softirqs.CPU0.SCHED 13132 ± 12% +56.6% 20564 ± 2% softirqs.CPU1.RCU 54173 ± 2% +63.0% 88299 softirqs.CPU1.SCHED 12063 ± 11% +56.0% 18815 ± 8% softirqs.CPU10.RCU 52997 ± 2% +63.7% 86776 softirqs.CPU10.SCHED 11146 ± 12% +73.5% 19343 ± 6% softirqs.CPU11.RCU 52858 +62.8% 86060 softirqs.CPU11.SCHED 12394 ± 20% +57.9% 19564 ± 6% softirqs.CPU12.RCU 52153 +66.2% 86659 softirqs.CPU12.SCHED 11714 ± 11% +58.9% 18619 ± 9% softirqs.CPU13.RCU 51630 +69.1% 87317 softirqs.CPU13.SCHED 11541 ± 15% +71.1% 19751 ± 10% softirqs.CPU14.RCU 52171 ± 2% +67.7% 87471 softirqs.CPU14.SCHED 12034 ± 13% +60.5% 19317 ± 8% softirqs.CPU15.RCU 52657 +65.2% 86972 softirqs.CPU15.SCHED 11964 ± 13% +75.0% 20934 ± 6% softirqs.CPU2.RCU 53736 +62.2% 87150 softirqs.CPU2.SCHED 11927 ± 11% +61.5% 19267 ± 9% softirqs.CPU3.RCU 52931 ± 2% +63.2% 86386 softirqs.CPU3.SCHED 12353 ± 15% +58.0% 19513 ± 6% softirqs.CPU4.RCU 51881 +68.0% 87174 softirqs.CPU4.SCHED 11154 ± 16% +72.4% 19234 ± 6% softirqs.CPU5.RCU 51954 +67.9% 87209 softirqs.CPU5.SCHED 11700 ± 12% +79.5% 21004 ± 11% softirqs.CPU6.RCU 51722 +68.9% 87358 softirqs.CPU6.SCHED 11795 ± 11% +79.5% 21169 ± 13% softirqs.CPU7.RCU 52777 +64.8% 86985 softirqs.CPU7.SCHED 11352 ± 16% +70.3% 19336 ± 7% softirqs.CPU8.RCU 53939 +61.1% 86897 softirqs.CPU8.SCHED 13223 ± 21% +54.5% 20423 ± 10% softirqs.CPU9.RCU 53570 ± 2% +61.4% 86435 softirqs.CPU9.SCHED 192323 ± 12% +65.7% 318668 ± 7% softirqs.RCU 846531 +64.5% 1392755 softirqs.SCHED 13340 ± 3% +28.3% 17120 ± 3% softirqs.TIMER 5616584 +255.3% 19953996 interrupts.CAL:Function_call_interrupts 403342 ± 4% +225.2% 1311736 interrupts.CPU0.CAL:Function_call_interrupts 348468 +21.9% 424744 interrupts.CPU0.LOC:Local_timer_interrupts 28665 ± 4% -21.7% 22431 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts 385359 +233.7% 1285782 interrupts.CPU1.CAL:Function_call_interrupts 348549 +21.8% 424706 interrupts.CPU1.LOC:Local_timer_interrupts 28794 ± 3% -17.9% 23633 interrupts.CPU1.RES:Rescheduling_interrupts 369509 ± 3% +240.9% 1259663 interrupts.CPU10.CAL:Function_call_interrupts 348480 +21.9% 424783 interrupts.CPU10.LOC:Local_timer_interrupts 30068 ± 3% -18.6% 24464 ± 2% interrupts.CPU10.RES:Rescheduling_interrupts 353201 ± 2% +246.8% 1225041 interrupts.CPU11.CAL:Function_call_interrupts 348488 +21.9% 424702 interrupts.CPU11.LOC:Local_timer_interrupts 30935 ± 3% -22.9% 23861 ± 5% interrupts.CPU11.RES:Rescheduling_interrupts 323460 ± 2% +276.6% 1218231 interrupts.CPU12.CAL:Function_call_interrupts 348410 +21.9% 424849 interrupts.CPU12.LOC:Local_timer_interrupts 31301 ± 3% -24.0% 23799 ± 4% interrupts.CPU12.RES:Rescheduling_interrupts 306249 +291.8% 1199851 interrupts.CPU13.CAL:Function_call_interrupts 348436 +21.9% 424721 interrupts.CPU13.LOC:Local_timer_interrupts 31048 ± 3% -19.3% 25040 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts 311947 ± 2% +291.1% 1220172 interrupts.CPU14.CAL:Function_call_interrupts 348554 +21.8% 424574 interrupts.CPU14.LOC:Local_timer_interrupts 30893 ± 4% -18.7% 25116 ± 4% interrupts.CPU14.RES:Rescheduling_interrupts 339980 ± 2% +259.0% 1220678 interrupts.CPU15.CAL:Function_call_interrupts 348418 +21.9% 424803 interrupts.CPU15.LOC:Local_timer_interrupts 29624 ± 3% -16.7% 24674 interrupts.CPU15.RES:Rescheduling_interrupts 369199 ± 2% +241.6% 1261224 interrupts.CPU2.CAL:Function_call_interrupts 348518 +21.9% 424863 interrupts.CPU2.LOC:Local_timer_interrupts 29702 ± 2% -21.3% 23377 ± 4% interrupts.CPU2.RES:Rescheduling_interrupts 352942 +253.9% 1249080 interrupts.CPU3.CAL:Function_call_interrupts 348266 +21.9% 424684 interrupts.CPU3.LOC:Local_timer_interrupts 31038 -25.3% 23178 ± 4% interrupts.CPU3.RES:Rescheduling_interrupts 327561 +272.0% 1218661 interrupts.CPU4.CAL:Function_call_interrupts 348447 +21.9% 424700 interrupts.CPU4.LOC:Local_timer_interrupts 30527 -22.9% 23534 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts 311428 ± 2% +286.3% 1202941 interrupts.CPU5.CAL:Function_call_interrupts 348533 +21.9% 424787 interrupts.CPU5.LOC:Local_timer_interrupts 30825 ± 4% -13.7% 26600 ± 5% interrupts.CPU5.RES:Rescheduling_interrupts 319963 ± 2% +287.0% 1238272 interrupts.CPU6.CAL:Function_call_interrupts 348246 +22.0% 424716 interrupts.CPU6.LOC:Local_timer_interrupts 29411 ± 4% -20.2% 23471 ± 2% interrupts.CPU6.RES:Rescheduling_interrupts 344529 ± 3% +263.6% 1252800 interrupts.CPU7.CAL:Function_call_interrupts 348501 +21.8% 424469 interrupts.CPU7.LOC:Local_timer_interrupts 30725 ± 3% -20.7% 24366 ± 5% interrupts.CPU7.RES:Rescheduling_interrupts 408919 ± 5% +219.7% 1307498 interrupts.CPU8.CAL:Function_call_interrupts 348306 +22.0% 424854 interrupts.CPU8.LOC:Local_timer_interrupts 27910 ± 4% -19.9% 22361 ± 2% interrupts.CPU8.RES:Rescheduling_interrupts 388993 ± 3% +229.7% 1282361 interrupts.CPU9.CAL:Function_call_interrupts 348529 +21.9% 424872 interrupts.CPU9.LOC:Local_timer_interrupts 28499 ± 2% -13.0% 24791 ± 4% interrupts.CPU9.RES:Rescheduling_interrupts 5575155 +21.9% 6795831 interrupts.LOC:Local_timer_interrupts 479969 ± 2% -19.8% 384701 interrupts.RES:Rescheduling_interrupts 239.00 ± 4% -44.9% 131.75 ± 11% interrupts.TLB:TLB_shootdowns 48085 +69.4% 81451 sched_debug.cfs_rq:/.exec_clock.avg 49258 +67.3% 82414 sched_debug.cfs_rq:/.exec_clock.max 46356 +73.3% 80327 sched_debug.cfs_rq:/.exec_clock.min 80499 ± 81% -60.4% 31857 ± 3% sched_debug.cfs_rq:/.load.stddev 127.01 ± 9% -16.2% 106.49 ± 13% sched_debug.cfs_rq:/.load_avg.avg 642.42 ± 23% -30.5% 446.56 ± 3% sched_debug.cfs_rq:/.load_avg.max 169.53 ± 17% -29.5% 119.49 ± 12% sched_debug.cfs_rq:/.load_avg.stddev 588910 +87.0% 1101108 sched_debug.cfs_rq:/.min_vruntime.avg 606363 +84.1% 1116408 sched_debug.cfs_rq:/.min_vruntime.max 561578 ± 2% +91.7% 1076386 sched_debug.cfs_rq:/.min_vruntime.min 0.71 ± 8% +24.8% 0.89 sched_debug.cfs_rq:/.nr_running.avg 0.45 ± 9% -47.2% 0.24 sched_debug.cfs_rq:/.nr_running.stddev 0.73 ± 18% +59.6% 1.16 ± 11% sched_debug.cfs_rq:/.nr_spread_over.avg 1559 ± 5% +9.0% 1699 ± 5% sched_debug.cfs_rq:/.runnable_avg.max 381.23 ± 4% +62.0% 617.48 ± 5% sched_debug.cfs_rq:/.util_est_enqueued.avg 1129 ± 8% +20.4% 1359 ± 6% sched_debug.cfs_rq:/.util_est_enqueued.max 77869 ± 6% -37.4% 48731 ± 8% sched_debug.cpu.avg_idle.avg 399988 ± 12% -30.9% 276240 ± 23% sched_debug.cpu.avg_idle.max 96550 ± 12% -34.3% 63478 ± 24% sched_debug.cpu.avg_idle.stddev 83560 +35.7% 113424 sched_debug.cpu.clock.avg 83561 +35.7% 113425 sched_debug.cpu.clock.max 83558 +35.7% 113423 sched_debug.cpu.clock.min 0.78 ± 9% -35.6% 0.50 ± 4% sched_debug.cpu.clock.stddev 82827 +35.1% 111904 sched_debug.cpu.clock_task.avg 82999 +35.0% 112089 sched_debug.cpu.clock_task.max 82262 +35.2% 111189 sched_debug.cpu.clock_task.min 1618 ± 6% +48.8% 2408 ± 2% sched_debug.cpu.curr->pid.avg 2493 +16.5% 2905 sched_debug.cpu.curr->pid.max 985.88 ± 8% -28.7% 703.01 ± 14% sched_debug.cpu.curr->pid.stddev 0.79 ± 5% +24.7% 0.98 sched_debug.cpu.nr_running.avg 0.54 ± 11% -19.9% 0.44 ± 3% sched_debug.cpu.nr_running.stddev 2444639 +11.6% 2728125 sched_debug.cpu.nr_switches.avg 2466248 +18.3% 2917708 sched_debug.cpu.nr_switches.max 13988 ± 27% +642.0% 103791 ± 3% sched_debug.cpu.nr_switches.stddev 0.21 ± 22% -75.6% 0.05 ± 76% sched_debug.cpu.nr_uninterruptible.avg 2440164 +11.6% 2723801 sched_debug.cpu.sched_count.avg 2457113 +18.4% 2909974 sched_debug.cpu.sched_count.max 12769 ± 28% +704.0% 102671 ± 3% sched_debug.cpu.sched_count.stddev 1219314 +11.5% 1359077 sched_debug.cpu.sched_goidle.avg 1227676 +18.3% 1452278 sched_debug.cpu.sched_goidle.max 6331 ± 28% +710.4% 51313 ± 3% sched_debug.cpu.sched_goidle.stddev 1220257 +11.7% 1363456 sched_debug.cpu.ttwu_count.avg 1251629 +11.3% 1392991 sched_debug.cpu.ttwu_count.max 1196282 +10.0% 1316458 sched_debug.cpu.ttwu_count.min 17387 ± 8% +16.8% 20308 ± 10% sched_debug.cpu.ttwu_count.stddev 790.29 ± 3% +163.0% 2078 ± 5% sched_debug.cpu.ttwu_local.avg 1612 ± 20% +87.9% 3028 ± 8% sched_debug.cpu.ttwu_local.max 531.25 ± 6% +148.4% 1319 ± 18% sched_debug.cpu.ttwu_local.min 283.96 ± 23% +49.2% 423.75 ± 12% sched_debug.cpu.ttwu_local.stddev 83560 +35.7% 113423 sched_debug.cpu_clk 82663 +36.2% 112557 sched_debug.ktime 84851 +35.2% 114688 sched_debug.sched_clk 32.08 +6.1% 34.03 perf-stat.i.MPKI 1.672e+10 -16.1% 1.403e+10 perf-stat.i.branch-instructions 0.26 +0.0 0.31 perf-stat.i.branch-miss-rate% 33778666 +9.6% 37021638 perf-stat.i.branch-misses 1.77 ± 2% -0.4 1.35 perf-stat.i.cache-miss-rate% 20527502 ± 2% -19.2% 16588407 perf-stat.i.cache-misses 649397 -26.3% 478422 perf-stat.i.context-switches 1.28 +13.5% 1.46 perf-stat.i.cpi 5.822e+10 +5.2% 6.125e+10 perf-stat.i.cpu-cycles 209.75 ± 6% +22.7% 257.35 ± 3% perf-stat.i.cpu-migrations 3159 +21.6% 3843 perf-stat.i.cycles-between-cache-misses 0.05 ± 5% +0.0 0.07 ± 6% perf-stat.i.dTLB-load-miss-rate% 8766755 ± 5% +14.0% 9996512 ± 5% perf-stat.i.dTLB-load-misses 1.572e+10 -15.2% 1.333e+10 perf-stat.i.dTLB-loads 0.05 +0.0 0.07 perf-stat.i.dTLB-store-miss-rate% 1534360 +27.2% 1951224 perf-stat.i.dTLB-store-misses 3.172e+09 -10.9% 2.826e+09 perf-stat.i.dTLB-stores 78.19 +15.6 93.74 perf-stat.i.iTLB-load-miss-rate% 6626547 -23.1% 5098140 perf-stat.i.iTLB-load-misses 1936185 ± 3% -84.8% 294324 perf-stat.i.iTLB-loads 5.451e+10 -15.1% 4.625e+10 perf-stat.i.instructions 0.94 -19.5% 0.76 perf-stat.i.ipc 3.64 +5.2% 3.83 perf-stat.i.metric.GHz 0.07 ± 8% -25.7% 0.06 ± 14% perf-stat.i.metric.K/sec 2310 -14.7% 1971 perf-stat.i.metric.M/sec 1488490 +27.9% 1903399 perf-stat.i.minor-faults 195100 ± 2% -26.6% 143181 perf-stat.i.node-loads 13149630 -18.3% 10746838 perf-stat.i.node-stores 1488491 +27.9% 1903399 perf-stat.i.page-faults 24.51 +17.8% 28.88 perf-stat.overall.MPKI 0.20 +0.1 0.26 perf-stat.overall.branch-miss-rate% 1.54 ± 2% -0.3 1.24 perf-stat.overall.cache-miss-rate% 1.07 +24.0% 1.32 perf-stat.overall.cpi 2837 ± 2% +30.1% 3691 perf-stat.overall.cycles-between-cache-misses 0.06 ± 5% +0.0 0.07 ± 5% perf-stat.overall.dTLB-load-miss-rate% 0.05 +0.0 0.07 perf-stat.overall.dTLB-store-miss-rate% 77.39 +17.1 94.54 perf-stat.overall.iTLB-load-miss-rate% 8229 ± 2% +10.3% 9073 perf-stat.overall.instructions-per-iTLB-miss 0.94 -19.3% 0.76 perf-stat.overall.ipc 3984 +3.7% 4129 perf-stat.overall.path-length 1.662e+10 -16.0% 1.396e+10 perf-stat.ps.branch-instructions 33589028 +9.7% 36852570 perf-stat.ps.branch-misses 20412439 ± 2% -19.1% 16512315 perf-stat.ps.cache-misses 645542 -26.3% 476080 perf-stat.ps.context-switches 5.788e+10 +5.3% 6.096e+10 perf-stat.ps.cpu-cycles 208.60 ± 5% +23.0% 256.52 ± 3% perf-stat.ps.cpu-migrations 8715455 ± 5% +14.1% 9947519 ± 5% perf-stat.ps.dTLB-load-misses 1.563e+10 -15.1% 1.327e+10 perf-stat.ps.dTLB-loads 1525309 +27.3% 1941693 perf-stat.ps.dTLB-store-misses 3.153e+09 -10.8% 2.812e+09 perf-stat.ps.dTLB-stores 6587329 -23.0% 5073245 perf-stat.ps.iTLB-load-misses 1924701 ± 3% -84.8% 292898 perf-stat.ps.iTLB-loads 5.419e+10 -15.1% 4.603e+10 perf-stat.ps.instructions 1479701 +28.0% 1894095 perf-stat.ps.minor-faults 194078 ± 2% -26.5% 142593 perf-stat.ps.node-loads 13072694 -18.2% 10694520 perf-stat.ps.node-stores 1479702 +28.0% 1894095 perf-stat.ps.page-faults 9.416e+12 +3.7% 9.76e+12 perf-stat.total.instructions 28.81 ± 37% -21.6 7.25 ± 2% perf-profile.calltrace.cycles-pp.secondary_startup_64 27.38 ± 37% -20.6 6.82 ± 2% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 27.38 ± 37% -20.6 6.82 ± 2% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 27.34 ± 37% -20.5 6.79 ± 2% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 23.22 ± 37% -18.7 4.48 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 23.20 ± 37% -18.7 4.47 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 15.78 ± 39% -12.4 3.42 ± 4% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 7.97 ± 37% -7.1 0.87 ± 15% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 5.85 ± 40% -5.0 0.82 ± 13% perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_page_function.__wake_up_common.wake_up_page_bit.do_fault 6.11 ± 40% -4.9 1.19 ± 12% perf-profile.calltrace.cycles-pp.__wake_up_common.wake_up_page_bit.do_fault.__handle_mm_fault.handle_mm_fault 6.02 ± 41% -4.9 1.12 ± 12% perf-profile.calltrace.cycles-pp.wake_page_function.__wake_up_common.wake_up_page_bit.do_fault.__handle_mm_fault 4.33 ± 40% -4.0 0.28 ±100% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.wake_page_function.__wake_up_common.wake_up_page_bit 4.31 ± 40% -4.0 0.28 ±100% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.wake_page_function.__wake_up_common 4.14 ± 40% -3.9 0.27 ±100% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.wake_page_function 2.32 ± 42% -1.6 0.74 ± 12% perf-profile.calltrace.cycles-pp.schedule.io_schedule.__lock_page.find_lock_entry.shmem_getpage_gfp 2.27 ± 42% -1.5 0.73 ± 12% perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.__lock_page.find_lock_entry 2.37 ± 42% -1.5 0.89 ± 18% perf-profile.calltrace.cycles-pp.io_schedule.__lock_page.find_lock_entry.shmem_getpage_gfp.shmem_fault 0.28 ±100% +0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.__sysvec_call_function_single.asm_call_on_stack.sysvec_call_function_single 0.27 ±100% +0.4 0.62 ± 4% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.__sysvec_call_function_single.asm_call_on_stack 0.30 ±100% +0.4 0.68 ± 4% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.__sysvec_call_function_single.asm_call_on_stack.sysvec_call_function_single.asm_sysvec_call_function_single 0.32 ±100% +0.4 0.74 ± 5% perf-profile.calltrace.cycles-pp.asm_call_on_stack.sysvec_call_function_single.asm_sysvec_call_function_single.finish_task_switch.__schedule 0.32 ±100% +0.4 0.74 ± 5% perf-profile.calltrace.cycles-pp.__sysvec_call_function_single.asm_call_on_stack.sysvec_call_function_single.asm_sysvec_call_function_single.finish_task_switch 0.14 ±173% +0.5 0.59 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.__sysvec_call_function_single 0.35 ±100% +0.5 0.81 ± 5% perf-profile.calltrace.cycles-pp.sysvec_call_function_single.asm_sysvec_call_function_single.finish_task_switch.__schedule.schedule_idle 0.40 ±100% +0.5 0.90 ± 6% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry 0.14 ±173% +0.5 0.65 ± 4% perf-profile.calltrace.cycles-pp.release_pages.pagevec_lru_move_fn.activate_page.mark_page_accessed.zap_pte_range 0.38 ±100% +0.5 0.89 ± 5% perf-profile.calltrace.cycles-pp.asm_sysvec_call_function_single.finish_task_switch.__schedule.schedule_idle.do_idle 0.18 ±173% +0.7 0.83 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlinkat 0.18 ±173% +0.7 0.83 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat 0.18 ±173% +0.7 0.83 ± 7% perf-profile.calltrace.cycles-pp.unlinkat 0.18 ±173% +0.7 0.83 ± 7% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat 0.18 ±173% +0.7 0.83 ± 7% perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat 0.18 ±173% +0.7 0.83 ± 7% perf-profile.calltrace.cycles-pp.shmem_evict_inode.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.18 ±173% +0.7 0.83 ± 7% perf-profile.calltrace.cycles-pp.shmem_truncate_range.shmem_evict_inode.evict.do_unlinkat.do_syscall_64 0.18 ±173% +0.7 0.83 ± 7% perf-profile.calltrace.cycles-pp.shmem_undo_range.shmem_truncate_range.shmem_evict_inode.evict.do_unlinkat 0.26 ±173% +0.9 1.13 ± 5% perf-profile.calltrace.cycles-pp.__activate_page.pagevec_lru_move_fn.activate_page.mark_page_accessed.zap_pte_range 0.28 ±173% +1.2 1.50 ± 4% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.zap_pte_range.unmap_page_range.unmap_vmas 0.37 ±173% +1.5 1.90 ± 5% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap 0.73 ±173% +2.2 2.89 ± 3% perf-profile.calltrace.cycles-pp.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap 0.76 ±173% +3.0 3.73 ± 3% perf-profile.calltrace.cycles-pp.workingset_age_nonresident.workingset_activation.mark_page_accessed.zap_pte_range.unmap_page_range 0.91 ±173% +3.4 4.34 ± 3% perf-profile.calltrace.cycles-pp.workingset_activation.mark_page_accessed.zap_pte_range.unmap_page_range.unmap_vmas 0.00 +4.7 4.65 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.wake_up_page_bit.do_fault.__handle_mm_fault 0.00 +4.8 4.78 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.wake_up_page_bit.do_fault.__handle_mm_fault.handle_mm_fault 1.56 ±173% +5.9 7.47 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.activate_page.mark_page_accessed 1.57 ±173% +6.0 7.53 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.activate_page.mark_page_accessed.zap_pte_range 0.00 +7.3 7.34 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.wake_up_page_bit.filemap_map_pages.do_fault 0.00 +7.5 7.46 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.wake_up_page_bit.filemap_map_pages.do_fault.__handle_mm_fault 2.07 ±173% +7.7 9.74 ± 3% perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.activate_page.mark_page_accessed.zap_pte_range.unmap_page_range 0.37 ±100% +8.2 8.52 ± 2% perf-profile.calltrace.cycles-pp.wake_up_page_bit.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault 2.41 ±173% +8.7 11.09 ± 3% perf-profile.calltrace.cycles-pp.activate_page.mark_page_accessed.zap_pte_range.unmap_page_range.unmap_vmas 18.70 ± 39% +11.2 29.95 ± 2% perf-profile.calltrace.cycles-pp.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 4.12 ±173% +14.6 18.74 ± 3% perf-profile.calltrace.cycles-pp.mark_page_accessed.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap 5.98 ±173% +20.5 26.51 ± 3% perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.exit_mmap.mmput 6.00 ±173% +20.6 26.57 ± 3% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.do_exit.do_group_exit 6.00 ±173% +20.6 26.57 ± 3% perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.mmput.do_exit 6.00 ±173% +20.6 26.60 ± 3% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 6.00 ±173% +20.6 26.60 ± 3% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group 6.00 ±173% +20.6 26.61 ± 3% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.00 ±173% +20.6 26.61 ± 3% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.00 ±173% +20.6 26.61 ± 3% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.00 ±173% +20.6 26.61 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 6.00 ±173% +20.6 26.61 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 28.81 ± 37% -21.6 7.25 ± 2% perf-profile.children.cycles-pp.secondary_startup_64 28.81 ± 37% -21.6 7.25 ± 2% perf-profile.children.cycles-pp.cpu_startup_entry 28.77 ± 37% -21.6 7.22 ± 2% perf-profile.children.cycles-pp.do_idle 27.38 ± 37% -20.6 6.82 ± 2% perf-profile.children.cycles-pp.start_secondary 24.40 ± 37% -19.6 4.76 ± 3% perf-profile.children.cycles-pp.cpuidle_enter 24.38 ± 37% -19.6 4.76 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state 15.86 ± 38% -12.2 3.63 ± 3% perf-profile.children.cycles-pp.poll_idle 8.17 ± 36% -7.3 0.92 ± 13% perf-profile.children.cycles-pp.intel_idle 6.35 ± 38% -4.3 2.06 ± 4% perf-profile.children.cycles-pp.wake_page_function 6.45 ± 38% -4.3 2.20 ± 4% perf-profile.children.cycles-pp.__wake_up_common 6.17 ± 38% -4.2 1.95 ± 4% perf-profile.children.cycles-pp.try_to_wake_up 5.04 ± 38% -3.0 2.06 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate 5.02 ± 38% -3.0 2.04 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair 4.84 ± 38% -2.9 1.97 ± 3% perf-profile.children.cycles-pp.enqueue_entity 4.02 ± 38% -2.4 1.62 ± 3% perf-profile.children.cycles-pp.__account_scheduler_latency 3.40 ± 38% -2.0 1.36 ± 2% perf-profile.children.cycles-pp.stack_trace_save_tsk 3.17 ± 39% -1.9 1.27 perf-profile.children.cycles-pp.arch_stack_walk 2.06 ± 38% -1.3 0.79 perf-profile.children.cycles-pp.unwind_next_frame 2.40 ± 40% -1.2 1.17 perf-profile.children.cycles-pp.io_schedule 2.36 ± 40% -1.2 1.15 perf-profile.children.cycles-pp.schedule 1.42 ± 52% -1.0 0.42 ± 15% perf-profile.children.cycles-pp.start_kernel 1.34 ± 38% -1.0 0.38 ± 4% perf-profile.children.cycles-pp.menu_select 1.22 ± 40% -0.7 0.57 perf-profile.children.cycles-pp.dequeue_task_fair 1.08 ± 41% -0.6 0.50 perf-profile.children.cycles-pp.dequeue_entity 0.95 ± 38% -0.5 0.40 ± 3% perf-profile.children.cycles-pp.pick_next_task_fair 0.80 ± 37% -0.4 0.39 ± 2% perf-profile.children.cycles-pp.update_load_avg 0.62 ± 39% -0.4 0.24 ± 6% perf-profile.children.cycles-pp.set_next_entity 0.50 ± 38% -0.3 0.16 ± 7% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.48 ± 38% -0.3 0.21 ± 7% perf-profile.children.cycles-pp.__orc_find 0.49 ± 44% -0.3 0.22 ± 3% perf-profile.children.cycles-pp.update_curr 0.44 ± 42% -0.3 0.18 ± 4% perf-profile.children.cycles-pp.stack_trace_consume_entry_nosched 0.36 ± 38% -0.2 0.13 ± 8% perf-profile.children.cycles-pp.select_task_rq_fair 0.44 ± 39% -0.2 0.21 ± 6% perf-profile.children.cycles-pp.unwind_get_return_address 0.37 ± 39% -0.2 0.17 ± 7% perf-profile.children.cycles-pp.__kernel_text_address 0.34 ± 36% -0.2 0.14 ± 5% perf-profile.children.cycles-pp.orc_find 0.33 ± 40% -0.2 0.17 ± 3% perf-profile.children.cycles-pp.update_rq_clock 0.26 ± 33% -0.2 0.09 ± 12% perf-profile.children.cycles-pp.tick_nohz_next_event 0.29 ± 36% -0.2 0.12 ± 4% perf-profile.children.cycles-pp.__switch_to_asm 0.30 ± 41% -0.2 0.14 ± 10% perf-profile.children.cycles-pp.kernel_text_address 0.28 ± 35% -0.1 0.14 ± 9% perf-profile.children.cycles-pp.sched_clock_cpu 0.24 ± 36% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.ktime_get 0.26 ± 36% -0.1 0.12 ± 8% perf-profile.children.cycles-pp.sched_clock 0.25 ± 35% -0.1 0.12 ± 11% perf-profile.children.cycles-pp.native_sched_clock 0.25 ± 41% -0.1 0.12 ± 5% perf-profile.children.cycles-pp.__switch_to 0.18 ± 35% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.18 ± 40% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.19 ± 43% -0.1 0.07 perf-profile.children.cycles-pp.in_sched_functions 0.16 ± 42% -0.1 0.04 ± 60% perf-profile.children.cycles-pp.update_cfs_rq_h_load 0.17 ± 39% -0.1 0.05 ± 58% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.22 ± 39% -0.1 0.11 ± 7% perf-profile.children.cycles-pp.__update_load_avg_se 0.18 ± 39% -0.1 0.07 ± 14% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.16 ± 41% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.check_preempt_curr 0.11 ± 39% -0.1 0.03 ±100% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.14 ± 40% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.stack_access_ok 0.14 ± 40% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.__unwind_start 0.12 ± 42% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.reweight_entity 0.15 ± 39% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.read_tsc 0.15 ± 32% -0.1 0.08 ± 16% perf-profile.children.cycles-pp.delayacct_end 0.10 ± 39% -0.1 0.04 ± 58% perf-profile.children.cycles-pp.account_entity_dequeue 0.03 ±100% +0.0 0.07 ± 7% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.01 ±173% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.xas_init_marks 0.03 ±100% +0.0 0.07 ± 11% perf-profile.children.cycles-pp.__unlock_page_memcg 0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.rcu_all_qs 0.06 ± 61% +0.1 0.12 ± 5% perf-profile.children.cycles-pp.__mod_memcg_state 0.07 ± 41% +0.1 0.13 ± 17% perf-profile.children.cycles-pp.xas_store 0.05 ± 64% +0.1 0.11 ± 4% perf-profile.children.cycles-pp.__mod_node_page_state 0.02 ±173% +0.1 0.09 perf-profile.children.cycles-pp.unaccount_page_cache_page 0.02 ±173% +0.1 0.09 ± 7% perf-profile.children.cycles-pp.__tlb_remove_page_size 0.07 ± 68% +0.1 0.17 ± 9% perf-profile.children.cycles-pp.__mod_lruvec_state 0.07 ± 50% +0.1 0.18 ± 10% perf-profile.children.cycles-pp.serial8250_console_write 0.02 ±173% +0.1 0.12 ± 12% perf-profile.children.cycles-pp.devkmsg_write.cold 0.02 ±173% +0.1 0.12 ± 12% perf-profile.children.cycles-pp.devkmsg_emit 0.03 ±173% +0.1 0.14 ± 11% perf-profile.children.cycles-pp.io_serial_in 0.04 ±118% +0.1 0.15 ± 9% perf-profile.children.cycles-pp.serial8250_console_putchar 0.03 ±102% +0.1 0.14 ± 13% perf-profile.children.cycles-pp.ksys_write 0.03 ±102% +0.1 0.14 ± 13% perf-profile.children.cycles-pp.vfs_write 0.03 ±102% +0.1 0.14 ± 13% perf-profile.children.cycles-pp.new_sync_write 0.04 ±118% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.wait_for_xmitr 0.02 ±173% +0.1 0.13 ± 11% perf-profile.children.cycles-pp.write 0.05 ±114% +0.1 0.17 ± 11% perf-profile.children.cycles-pp.uart_console_write 0.04 ±173% +0.1 0.15 ± 16% perf-profile.children.cycles-pp.process_one_work 0.03 ±173% +0.1 0.15 ± 16% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work 0.04 ±173% +0.1 0.16 ± 17% perf-profile.children.cycles-pp.worker_thread 0.03 ±173% +0.1 0.16 ± 14% perf-profile.children.cycles-pp.memcpy_erms 0.13 ± 37% +0.1 0.26 ± 4% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.04 ±173% +0.1 0.16 ± 15% perf-profile.children.cycles-pp.ret_from_fork 0.09 ± 67% +0.1 0.21 ± 12% perf-profile.children.cycles-pp.vprintk_emit 0.04 ±173% +0.1 0.16 ± 15% perf-profile.children.cycles-pp.kthread 0.03 ±173% +0.1 0.16 ± 8% perf-profile.children.cycles-pp.free_pcppages_bulk 0.08 ± 61% +0.1 0.21 ± 12% perf-profile.children.cycles-pp.console_unlock 0.06 ±129% +0.2 0.23 ± 6% perf-profile.children.cycles-pp.free_unref_page_list 0.07 ±118% +0.2 0.26 ± 7% perf-profile.children.cycles-pp.__delete_from_page_cache 0.09 ±127% +0.2 0.33 ± 8% perf-profile.children.cycles-pp.__pagevec_release 0.10 ±120% +0.2 0.34 ± 9% perf-profile.children.cycles-pp.delete_from_page_cache 0.06 ±173% +0.2 0.31 ± 9% perf-profile.children.cycles-pp.mem_cgroup_page_lruvec 0.10 ±120% +0.3 0.38 ± 7% perf-profile.children.cycles-pp.truncate_inode_page 0.08 ±173% +0.3 0.39 ± 8% perf-profile.children.cycles-pp.free_pages_and_swap_cache 0.34 ± 51% +0.3 0.69 ± 5% perf-profile.children.cycles-pp.lock_page_memcg 0.54 ± 46% +0.4 0.91 ± 3% perf-profile.children.cycles-pp.sched_ttwu_pending 0.25 ± 76% +0.4 0.62 ± 4% perf-profile.children.cycles-pp.unlock_page_memcg 0.55 ± 43% +0.4 0.93 ± 3% perf-profile.children.cycles-pp.find_get_entry 0.55 ± 45% +0.4 0.93 ± 3% perf-profile.children.cycles-pp.__sysvec_call_function_single 0.60 ± 44% +0.4 1.01 ± 4% perf-profile.children.cycles-pp.sysvec_call_function_single 0.86 ± 35% +0.4 1.28 ± 3% perf-profile.children.cycles-pp.asm_call_on_stack 0.66 ± 44% +0.5 1.12 ± 3% perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.75 ± 30% +0.5 1.30 ± 4% perf-profile.children.cycles-pp.PageHuge 0.23 ±123% +0.6 0.83 ± 7% perf-profile.children.cycles-pp.do_unlinkat 0.23 ±123% +0.6 0.83 ± 7% perf-profile.children.cycles-pp.evict 0.23 ±123% +0.6 0.83 ± 7% perf-profile.children.cycles-pp.shmem_evict_inode 0.23 ±123% +0.6 0.83 ± 7% perf-profile.children.cycles-pp.shmem_truncate_range 0.23 ±123% +0.6 0.83 ± 7% perf-profile.children.cycles-pp.shmem_undo_range 0.18 ±173% +0.7 0.83 ± 7% perf-profile.children.cycles-pp.unlinkat 0.26 ±173% +0.9 1.14 ± 5% perf-profile.children.cycles-pp.__activate_page 0.37 ±173% +1.6 1.93 ± 6% perf-profile.children.cycles-pp.tlb_flush_mmu 0.57 ±147% +2.0 2.54 ± 4% perf-profile.children.cycles-pp.release_pages 0.74 ±173% +2.2 2.90 ± 3% perf-profile.children.cycles-pp.page_remove_rmap 0.76 ±173% +3.0 3.74 ± 3% perf-profile.children.cycles-pp.workingset_age_nonresident 0.91 ±173% +3.4 4.34 ± 3% perf-profile.children.cycles-pp.workingset_activation 2.37 ±148% +7.6 9.94 ± 3% perf-profile.children.cycles-pp.pagevec_lru_move_fn 7.14 ± 38% +7.8 14.90 ± 3% perf-profile.children.cycles-pp.wake_up_page_bit 2.41 ±173% +8.7 11.09 ± 3% perf-profile.children.cycles-pp.activate_page 18.80 ± 39% +11.2 30.01 ± 2% perf-profile.children.cycles-pp.filemap_map_pages 4.13 ±173% +14.6 18.76 ± 3% perf-profile.children.cycles-pp.mark_page_accessed 4.09 ± 52% +16.8 20.87 ± 3% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 2.63 ± 93% +17.4 20.05 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 6.00 ±173% +20.6 26.56 ± 3% perf-profile.children.cycles-pp.zap_pte_range 6.00 ±173% +20.6 26.57 ± 3% perf-profile.children.cycles-pp.unmap_vmas 6.00 ±173% +20.6 26.57 ± 3% perf-profile.children.cycles-pp.unmap_page_range 6.00 ±173% +20.6 26.61 ± 3% perf-profile.children.cycles-pp.mmput 6.00 ±173% +20.6 26.61 ± 3% perf-profile.children.cycles-pp.exit_mmap 6.00 ±173% +20.6 26.61 ± 3% perf-profile.children.cycles-pp.__x64_sys_exit_group 6.00 ±173% +20.6 26.61 ± 3% perf-profile.children.cycles-pp.do_group_exit 6.00 ±173% +20.6 26.61 ± 3% perf-profile.children.cycles-pp.do_exit 6.33 ±168% +21.3 27.62 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 6.32 ±168% +21.3 27.62 ± 3% perf-profile.children.cycles-pp.do_syscall_64 15.70 ± 38% -12.1 3.56 ± 3% perf-profile.self.cycles-pp.poll_idle 8.17 ± 36% -7.3 0.92 ± 13% perf-profile.self.cycles-pp.intel_idle 1.06 ± 38% -0.7 0.38 ± 2% perf-profile.self.cycles-pp.unwind_next_frame 0.67 ± 38% -0.5 0.16 ± 9% perf-profile.self.cycles-pp.menu_select 0.60 ± 41% -0.3 0.30 ± 5% perf-profile.self.cycles-pp.__schedule 0.48 ± 38% -0.3 0.21 ± 7% perf-profile.self.cycles-pp.__orc_find 0.40 ± 40% -0.3 0.13 ± 9% perf-profile.self.cycles-pp.set_next_entity 0.46 ± 38% -0.3 0.20 ± 4% perf-profile.self.cycles-pp.__account_scheduler_latency 0.38 ± 38% -0.2 0.16 ± 8% perf-profile.self.cycles-pp.update_load_avg 0.36 ± 38% -0.2 0.17 ± 2% perf-profile.self.cycles-pp.try_to_wake_up 0.33 ± 37% -0.2 0.14 ± 7% perf-profile.self.cycles-pp.orc_find 0.33 ± 46% -0.2 0.15 ± 10% perf-profile.self.cycles-pp.update_curr 0.28 ± 39% -0.2 0.09 ± 8% perf-profile.self.cycles-pp.do_idle 0.29 ± 36% -0.2 0.12 ± 4% perf-profile.self.cycles-pp.__switch_to_asm 0.25 ± 42% -0.1 0.11 ± 6% perf-profile.self.cycles-pp.stack_trace_consume_entry_nosched 0.20 ± 37% -0.1 0.07 ± 15% perf-profile.self.cycles-pp.stack_trace_save_tsk 0.24 ± 36% -0.1 0.12 ± 7% perf-profile.self.cycles-pp.native_sched_clock 0.24 ± 39% -0.1 0.12 ± 7% perf-profile.self.cycles-pp.__switch_to 0.16 ± 42% -0.1 0.04 ± 60% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.22 ± 41% -0.1 0.11 ± 7% perf-profile.self.cycles-pp.__update_load_avg_se 0.20 ± 42% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.update_rq_clock 0.17 ± 39% -0.1 0.07 perf-profile.self.cycles-pp.enqueue_task_fair 0.16 ± 37% -0.1 0.06 ± 6% perf-profile.self.cycles-pp.enqueue_entity 0.12 ± 43% -0.1 0.03 ±100% perf-profile.self.cycles-pp.__unwind_start 0.15 ± 39% -0.1 0.07 ± 6% perf-profile.self.cycles-pp.read_tsc 0.19 ± 32% -0.1 0.11 ± 7% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.11 ± 36% -0.1 0.03 ±100% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.16 ± 38% -0.1 0.08 ± 8% perf-profile.self.cycles-pp.update_cfs_group 0.13 ± 40% -0.1 0.05 perf-profile.self.cycles-pp.stack_access_ok 0.10 ± 36% -0.1 0.03 ±100% perf-profile.self.cycles-pp.account_entity_dequeue 0.03 ±100% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.03 ±100% +0.0 0.07 ± 10% perf-profile.self.cycles-pp.__unlock_page_memcg 0.04 ± 59% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.__mod_node_page_state 0.00 +0.1 0.05 perf-profile.self.cycles-pp.__mod_zone_page_state 0.01 ±173% +0.1 0.07 ± 22% perf-profile.self.cycles-pp.__mod_lruvec_state 0.06 ± 63% +0.1 0.12 ± 7% perf-profile.self.cycles-pp.__mod_memcg_state 0.02 ±173% +0.1 0.08 ± 10% perf-profile.self.cycles-pp.vm_normal_page 0.01 ±173% +0.1 0.08 ± 6% perf-profile.self.cycles-pp.__tlb_remove_page_size 0.02 ±173% +0.1 0.10 ± 10% perf-profile.self.cycles-pp.free_pcppages_bulk 0.06 ± 71% +0.1 0.14 ± 9% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.03 ±173% +0.1 0.14 ± 11% perf-profile.self.cycles-pp.io_serial_in 0.03 ±173% +0.1 0.15 ± 16% perf-profile.self.cycles-pp.memcpy_erms 0.21 ± 42% +0.2 0.42 ± 5% perf-profile.self.cycles-pp.find_lock_entry 0.07 ±173% +0.2 0.30 ± 6% perf-profile.self.cycles-pp.pagevec_lru_move_fn 0.07 ± 64% +0.2 0.30 ± 2% perf-profile.self.cycles-pp.wake_up_page_bit 0.06 ±173% +0.2 0.30 ± 9% perf-profile.self.cycles-pp.mem_cgroup_page_lruvec 0.07 ±173% +0.3 0.36 ± 8% perf-profile.self.cycles-pp.free_pages_and_swap_cache 0.35 ± 43% +0.3 0.66 ± 3% perf-profile.self.cycles-pp.find_get_entry 0.33 ± 53% +0.3 0.68 ± 5% perf-profile.self.cycles-pp.lock_page_memcg 0.25 ± 76% +0.4 0.62 ± 3% perf-profile.self.cycles-pp.unlock_page_memcg 0.16 ±173% +0.5 0.67 ± 5% perf-profile.self.cycles-pp.__activate_page 0.75 ± 30% +0.5 1.28 ± 4% perf-profile.self.cycles-pp.PageHuge 0.49 ±150% +1.7 2.21 ± 4% perf-profile.self.cycles-pp.release_pages 0.57 ±173% +1.8 2.33 ± 3% perf-profile.self.cycles-pp.zap_pte_range 0.81 ±173% +2.5 3.32 ± 4% perf-profile.self.cycles-pp.mark_page_accessed 0.76 ±173% +3.0 3.72 ± 3% perf-profile.self.cycles-pp.workingset_age_nonresident 4.08 ± 52% +16.8 20.86 ± 3% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath vm-scalability.time.system_time 1900 +--------------------------------------------------------------------+ 1800 |-O O O O O O O O O O O O O O O O O O O O O O | | O O | 1700 |-+ | 1600 |-+ | | | 1500 |-+ | 1400 |-+ | 1300 |-+ | | | 1200 |-+ | 1100 |-+ | | | 1000 |.+..+.+..+.+.+..+.+.+..+.+..+.+.+..+.+.+..+.+..+.+.+..+.+.+..+.+..+.| 900 +--------------------------------------------------------------------+ vm-scalability.time.percent_of_cpu_this_job_got 1460 +--------------------------------------------------------------------+ 1440 |-O O O O O O O O O O O O O O O O O O O O O O O O | | | 1420 |-+ | 1400 |-+ | | | 1380 |-+ | 1360 |-+ | 1340 |-+ | | | 1320 |-+ | 1300 |-+ | | .+. .+.. .+.. .+. .+..+.+. .+.+..+.| 1280 |.+..+.+. +.+..+.+ +.+..+.+ +.+.+..+.+. + +. | 1260 +--------------------------------------------------------------------+ vm-scalability.time.elapsed_time 215 +---------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O O | 210 |-+ O O O O O O | 205 |-+ | | | 200 |-+ | 195 |-+ | | | 190 |-+ | 185 |-+ | | | 180 |-+ | 175 |-+ +. +.. + .+. | | .+.+.. + +..+. .+.+.. + .+..+. .. + .+.+.+. +..+.| 170 +---------------------------------------------------------------------+ vm-scalability.time.elapsed_time.max 215 +---------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O O | 210 |-+ O O O O O O | 205 |-+ | | | 200 |-+ | 195 |-+ | | | 190 |-+ | 185 |-+ | | | 180 |-+ | 175 |-+ +. +.. + .+. | | .+.+.. + +..+. .+.+.. + .+..+. .. + .+.+.+. +..+.| 170 +---------------------------------------------------------------------+ vm-scalability.time.minor_page_faults 4.2e+08 +-----------------------------------------------------------------+ | O O O O O O O O | 4e+08 |-O O O O O O O O O O O O O O O O | 3.8e+08 |-+ | | | 3.6e+08 |-+ | 3.4e+08 |-+ | | | 3.2e+08 |-+ | 3e+08 |-+ | | | 2.8e+08 |-+ | 2.6e+08 |-+ .+.. .+. .+. .+. | |.+..+.+.+ +.+.+.+..+.+.+..+.+ +..+.+.+..+ +.+..+.+ +..+.| 2.4e+08 +-----------------------------------------------------------------+ vm-scalability.time.voluntary_context_switches 5.7e+07 +-----------------------------------------------------------------+ | + + + .+. .+ | 5.6e+07 |-+ :+ +. :+ :+ .+ +. +| 5.5e+07 |.+ .+ : + + +. .+. .+ : +.. .+.. : +.+..+ | | +..+ + : + +. +.+. + : +.+ : | 5.4e+07 |-+ + + + | | | 5.3e+07 |-+ | | | 5.2e+07 |-+ | 5.1e+07 |-+ O | | O O O O O | 5e+07 |-+ O O O | | O O O O O O O O O O O | 4.9e+07 +-----------------------------------------------------------------+ vm-scalability.time.involuntary_context_switches 45000 +-------------------------------------------------------------------+ | O | 40000 |-+ O O | | O O O | 35000 |-+ O O O O O O O | | O O O O O O O O | 30000 |-+ O O O | | | 25000 |-+ | | | 20000 |-+ .+.. .+ .+ | |.+..+.+ .+. .+. .+.+.+. + .+.+..+ + .+. .+. | 15000 |-+ + +..+ +. +..+ +. +.+..+ +..+.| | | 10000 +-------------------------------------------------------------------+ vm-scalability.throughput 1.65e+07 +----------------------------------------------------------------+ |.+. .+. .+. | 1.6e+07 |-+ +..+.+. .+..+.+.+.+. +.+.+.. .+ +..+.+. .+..+.+.+. .+.+.| 1.55e+07 |-+ + + + +. | | | 1.5e+07 |-+ | 1.45e+07 |-+ | | | 1.4e+07 |-+ | 1.35e+07 |-+ | | | 1.3e+07 |-+ | 1.25e+07 |-O O O O O O O O O O O O O O O O O O O O O O | | O O | 1.2e+07 +----------------------------------------------------------------+ vm-scalability.median 1.05e+06 +----------------------------------------------------------------+ | | 1e+06 |.+. .+ .+.+.+..+. .+.. +.+. .+ +.. | | +..+ + .+..+ +.+ + +..+ + + +.+.+. .+.+.| | + + + +. | 950000 |-+ | | | 900000 |-+ | | | 850000 |-+ | | | | | 800000 |-O O O O O O O O O O O O O O O O O | | O O O O O O O | 750000 +----------------------------------------------------------------+ vm-scalability.median_stddev_ 6 +---------------------------------------------------------------------+ | +.. +..+. +. +. +.. .+. .+ | 5.5 |:+ + + +..+ + + : + + + +. : +.| |: : +.+ + + : +.+.. : + +..+ : .. | 5 |-+ : .. + : .. + + + + | | + + + | 4.5 |-+ | | | 4 |-+ | | | 3.5 |-+ O | | O O O O O | 3 |-+ O O O O O O O O O O O O O O O | | O O O | 2.5 +---------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-cfl-e1: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory ========================================================================================= compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/30%/debian-10.4-x86_64-20200603.cgz/300s/lkp-cfl-e1/shell8/unixbench/0xd6 commit: 5925fa68fe (" perf tools fixes for v5.9: 3rd batch") 5ef64cc898 ("mm: allow a controlled amount of unfairness in the page lock") 5925fa68fe824465 5ef64cc8987a9211d3f3667331b ---------------- --------------------------- %stddev %change %stddev \ | \ 21570 +5.4% 22733 unixbench.score 1054019 ± 4% +25.8% 1325439 ± 3% unixbench.time.involuntary_context_switches 299493 ± 2% -87.4% 37651 ± 3% unixbench.time.major_page_faults 1.034e+08 +5.6% 1.092e+08 unixbench.time.minor_page_faults 1286 +7.3% 1379 unixbench.time.percent_of_cpu_this_job_got 334.45 +11.0% 371.24 unixbench.time.system_time 477.46 +4.8% 500.52 unixbench.time.user_time 3401111 -7.4% 3148880 unixbench.time.voluntary_context_switches 815347 +5.4% 859335 unixbench.workload 5.09 ± 3% -5.1 0.00 ±100% mpstat.cpu.all.iowait% 1752 ± 16% -21.0% 1384 ± 13% slabinfo.skbuff_head_cache.active_objs 1752 ± 16% -19.2% 1416 ± 11% slabinfo.skbuff_head_cache.num_objs 10037 -16.6% 8374 ± 7% slabinfo.vmap_area.active_objs 10355 -11.8% 9132 ± 6% slabinfo.vmap_area.num_objs 48.00 +5.7% 50.75 vmstat.cpu.sy 31.25 +9.6% 34.25 vmstat.cpu.us 3.50 ± 24% -100.0% 0.00 vmstat.procs.b 18.75 ± 4% +17.3% 22.00 ± 3% vmstat.procs.r 111395 -9.5% 100851 vmstat.system.cs 36967 +3.2% 38149 vmstat.system.in 52268017 -42.8% 29897553 ± 3% cpuidle.C1.time 1117760 -25.5% 833099 cpuidle.C1.usage 50982561 ± 2% -61.2% 19764268 ± 6% cpuidle.C1E.time 769266 -43.4% 435332 ± 3% cpuidle.C1E.usage 74333 ± 3% -31.2% 51121 ± 16% cpuidle.C3.usage 5602742 ± 4% -97.7% 129968 ± 28% cpuidle.POLL.time 697760 ± 2% -98.4% 11332 ± 12% cpuidle.POLL.usage 16352 ± 3% -8.3% 14993 ± 2% softirqs.CPU1.SCHED 15561 -11.7% 13743 softirqs.CPU10.SCHED 15622 -10.2% 14029 softirqs.CPU13.SCHED 15418 -8.0% 14188 ± 2% softirqs.CPU14.SCHED 15598 -8.0% 14355 ± 2% softirqs.CPU15.SCHED 15749 -10.1% 14155 softirqs.CPU3.SCHED 15847 -10.0% 14262 softirqs.CPU4.SCHED 970.25 +2.3% 993.00 proc-vmstat.nr_page_table_pages 13460 +5.3% 14177 ± 3% proc-vmstat.nr_slab_unreclaimable 70057163 +4.5% 73199359 proc-vmstat.numa_hit 70057163 +4.5% 73199359 proc-vmstat.numa_local 100067 +21.2% 121296 proc-vmstat.pgactivate 74525416 +4.7% 78002806 proc-vmstat.pgalloc_normal 1.041e+08 +5.1% 1.094e+08 proc-vmstat.pgfault 74511080 +4.7% 77987815 proc-vmstat.pgfree 5126035 +8.5% 5561192 proc-vmstat.pgreuse 3425 +7.0% 3664 proc-vmstat.thp_fault_alloc 1450159 +5.4% 1528363 proc-vmstat.unevictable_pgs_culled 15.80 ±100% +200.5% 47.50 ± 32% sched_debug.cfs_rq:/.removed.load_avg.avg 61.21 ±100% +137.4% 145.31 ± 14% sched_debug.cfs_rq:/.removed.load_avg.stddev 1.85 ±130% +649.4% 13.88 ± 30% sched_debug.cfs_rq:/.removed.runnable_avg.avg 29.62 ±130% +512.2% 181.38 ± 14% sched_debug.cfs_rq:/.removed.runnable_avg.max 7.17 ±130% +531.8% 45.30 ± 17% sched_debug.cfs_rq:/.removed.runnable_avg.stddev 1.48 ±170% +815.3% 13.59 ± 29% sched_debug.cfs_rq:/.removed.util_avg.avg 23.75 ±170% +649.5% 178.00 ± 13% sched_debug.cfs_rq:/.removed.util_avg.max 5.75 ±170% +673.5% 44.47 ± 16% sched_debug.cfs_rq:/.removed.util_avg.stddev 124782 ± 12% +29.5% 161579 ± 9% sched_debug.cpu.avg_idle.avg 5321 ± 44% +152.1% 13417 ± 11% sched_debug.cpu.curr->pid.avg 7329 ± 36% +104.2% 14969 ± 8% sched_debug.cpu.curr->pid.max 0.94 ± 16% +33.3% 1.25 ± 4% sched_debug.cpu.nr_running.avg 213716 -9.1% 194283 sched_debug.cpu.nr_switches.min 219.88 ± 16% -69.1% 67.88 ± 25% sched_debug.cpu.nr_uninterruptible.max -212.88 -64.7% -75.25 sched_debug.cpu.nr_uninterruptible.min 116.88 ± 11% -68.1% 37.27 ± 11% sched_debug.cpu.nr_uninterruptible.stddev 215276 -9.2% 195388 sched_debug.cpu.sched_count.avg 219021 -9.4% 198334 sched_debug.cpu.sched_count.max 212018 -9.6% 191589 sched_debug.cpu.sched_count.min 63854 -33.3% 42613 ± 2% sched_debug.cpu.sched_goidle.avg 65292 -33.2% 43641 ± 2% sched_debug.cpu.sched_goidle.max 62644 -33.7% 41522 ± 2% sched_debug.cpu.sched_goidle.min 36980 ± 4% +15.4% 42667 ± 4% sched_debug.cpu.ttwu_local.avg 37905 ± 2% +14.8% 43522 ± 2% sched_debug.cpu.ttwu_local.max 36245 ± 5% +15.1% 41734 ± 5% sched_debug.cpu.ttwu_local.min 479.50 ±122% +208.1% 1477 ± 66% interrupts.132:IR-PCI-MSI.2097153-edge.eth1-TxRx-0 21616 +25.2% 27062 interrupts.CPU0.RES:Rescheduling_interrupts 1431 -65.8% 489.00 ± 4% interrupts.CPU0.TLB:TLB_shootdowns 479.50 ±122% +208.1% 1477 ± 66% interrupts.CPU1.132:IR-PCI-MSI.2097153-edge.eth1-TxRx-0 1687 ± 7% -12.9% 1469 ± 4% interrupts.CPU1.CAL:Function_call_interrupts 21784 +23.9% 26998 ± 2% interrupts.CPU1.RES:Rescheduling_interrupts 1399 ± 2% -64.9% 491.75 ± 7% interrupts.CPU1.TLB:TLB_shootdowns 21969 +22.0% 26811 interrupts.CPU10.RES:Rescheduling_interrupts 1388 ± 2% -64.0% 500.50 interrupts.CPU10.TLB:TLB_shootdowns 21807 +24.8% 27212 interrupts.CPU11.RES:Rescheduling_interrupts 1367 ± 2% -64.4% 487.25 ± 5% interrupts.CPU11.TLB:TLB_shootdowns 21738 ± 2% +26.4% 27485 interrupts.CPU12.RES:Rescheduling_interrupts 1376 -62.3% 518.50 ± 9% interrupts.CPU12.TLB:TLB_shootdowns 21766 +25.0% 27217 interrupts.CPU13.RES:Rescheduling_interrupts 1425 -65.6% 490.75 ± 9% interrupts.CPU13.TLB:TLB_shootdowns 1557 ± 4% -9.7% 1406 ± 6% interrupts.CPU14.CAL:Function_call_interrupts 21777 +26.0% 27447 interrupts.CPU14.RES:Rescheduling_interrupts 1426 ± 2% -64.9% 500.25 ± 7% interrupts.CPU14.TLB:TLB_shootdowns 21639 ± 2% +27.8% 27653 interrupts.CPU15.RES:Rescheduling_interrupts 1441 ± 3% -64.6% 510.75 ± 6% interrupts.CPU15.TLB:TLB_shootdowns 21908 +24.0% 27161 interrupts.CPU2.RES:Rescheduling_interrupts 1415 -64.5% 503.00 ± 2% interrupts.CPU2.TLB:TLB_shootdowns 21904 +22.4% 26804 interrupts.CPU3.RES:Rescheduling_interrupts 1474 -65.6% 507.25 ± 8% interrupts.CPU3.TLB:TLB_shootdowns 21534 ± 2% +26.2% 27181 interrupts.CPU4.RES:Rescheduling_interrupts 1356 -62.2% 513.00 ± 6% interrupts.CPU4.TLB:TLB_shootdowns 21836 +23.9% 27045 interrupts.CPU5.RES:Rescheduling_interrupts 1407 ± 2% -63.9% 508.50 ± 2% interrupts.CPU5.TLB:TLB_shootdowns 21999 +23.3% 27125 interrupts.CPU6.RES:Rescheduling_interrupts 1403 ± 4% -62.7% 524.00 ± 9% interrupts.CPU6.TLB:TLB_shootdowns 1557 ± 4% -11.0% 1385 ± 4% interrupts.CPU7.CAL:Function_call_interrupts 21508 ± 2% +26.1% 27111 interrupts.CPU7.RES:Rescheduling_interrupts 1417 ± 5% -64.1% 509.50 ± 6% interrupts.CPU7.TLB:TLB_shootdowns 21905 +25.2% 27429 interrupts.CPU8.RES:Rescheduling_interrupts 1411 ± 2% -65.4% 488.50 ± 8% interrupts.CPU8.TLB:TLB_shootdowns 21904 +24.7% 27307 interrupts.CPU9.RES:Rescheduling_interrupts 1388 ± 2% -63.7% 503.50 interrupts.CPU9.TLB:TLB_shootdowns 348600 +24.8% 435055 interrupts.RES:Rescheduling_interrupts 22530 -64.3% 8046 ± 4% interrupts.TLB:TLB_shootdowns 48.16 -1.7% 47.32 perf-stat.i.MPKI 9.687e+09 +5.1% 1.019e+10 perf-stat.i.branch-instructions 2.126e+08 +4.6% 2.224e+08 perf-stat.i.branch-misses 4.44 +0.2 4.60 perf-stat.i.cache-miss-rate% 94624278 +7.1% 1.013e+08 perf-stat.i.cache-misses 2.382e+09 +3.3% 2.461e+09 perf-stat.i.cache-references 114789 -9.1% 104363 perf-stat.i.context-switches 5.294e+10 +5.8% 5.601e+10 perf-stat.i.cpu-cycles 19853 -22.2% 15448 perf-stat.i.cpu-migrations 569.61 -1.9% 558.71 perf-stat.i.cycles-between-cache-misses 1.223e+10 +5.1% 1.285e+10 perf-stat.i.dTLB-loads 3655243 +4.1% 3805877 perf-stat.i.dTLB-store-misses 7.024e+09 +5.1% 7.383e+09 perf-stat.i.dTLB-stores 10344903 +3.7% 10732085 perf-stat.i.iTLB-load-misses 8031239 +1.9% 8185952 perf-stat.i.iTLB-loads 4.759e+10 +5.2% 5.006e+10 perf-stat.i.instructions 5236 +1.2% 5299 perf-stat.i.instructions-per-iTLB-miss 4684 ± 2% -87.4% 589.00 ± 3% perf-stat.i.major-faults 3.31 +5.8% 3.50 perf-stat.i.metric.GHz 1960 +5.0% 2058 perf-stat.i.metric.M/sec 1588100 +5.6% 1677762 perf-stat.i.minor-faults 5071078 +8.5% 5500804 perf-stat.i.node-loads 31239619 +7.2% 33482613 perf-stat.i.node-stores 1592784 +5.4% 1678351 perf-stat.i.page-faults 50.05 -1.8% 49.15 perf-stat.overall.MPKI 3.97 +0.1 4.12 perf-stat.overall.cache-miss-rate% 559.44 -1.2% 552.70 perf-stat.overall.cycles-between-cache-misses 4600 +1.4% 4664 perf-stat.overall.instructions-per-iTLB-miss 9.533e+09 +5.1% 1.002e+10 perf-stat.ps.branch-instructions 2.093e+08 +4.6% 2.189e+08 perf-stat.ps.branch-misses 93122368 +7.1% 99714732 perf-stat.ps.cache-misses 2.344e+09 +3.3% 2.421e+09 perf-stat.ps.cache-references 112962 -9.1% 102685 perf-stat.ps.context-switches 5.21e+10 +5.8% 5.511e+10 perf-stat.ps.cpu-cycles 19537 -22.2% 15200 perf-stat.ps.cpu-migrations 1.203e+10 +5.1% 1.264e+10 perf-stat.ps.dTLB-loads 3597096 +4.1% 3744712 perf-stat.ps.dTLB-store-misses 6.913e+09 +5.1% 7.264e+09 perf-stat.ps.dTLB-stores 10180397 +3.7% 10559643 perf-stat.ps.iTLB-load-misses 7903463 +1.9% 8054357 perf-stat.ps.iTLB-loads 4.684e+10 +5.2% 4.926e+10 perf-stat.ps.instructions 4609 ± 2% -87.4% 579.53 ± 3% perf-stat.ps.major-faults 1562827 +5.6% 1650780 perf-stat.ps.minor-faults 4990492 +8.5% 5412486 perf-stat.ps.node-loads 30742642 +7.2% 32944203 perf-stat.ps.node-stores 1567436 +5.4% 1651360 perf-stat.ps.page-faults 3.004e+12 +5.2% 3.159e+12 perf-stat.total.instructions 29.27 ± 57% -17.3 11.93 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 29.27 ± 57% -17.3 11.93 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 29.27 ± 57% -17.3 11.93 ±173% perf-profile.calltrace.cycles-pp.write 29.27 ± 57% -17.3 11.93 ±173% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 29.27 ± 57% -17.3 11.93 ±173% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 29.27 ± 57% -17.3 11.93 ±173% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 29.09 ± 57% -17.2 11.93 ±173% perf-profile.calltrace.cycles-pp.devkmsg_write.cold.new_sync_write.vfs_write.ksys_write.do_syscall_64 29.09 ± 57% -17.2 11.93 ±173% perf-profile.calltrace.cycles-pp.devkmsg_emit.devkmsg_write.cold.new_sync_write.vfs_write.ksys_write 29.09 ± 57% -17.2 11.93 ±173% perf-profile.calltrace.cycles-pp.vprintk_emit.devkmsg_emit.devkmsg_write.cold.new_sync_write.vfs_write 29.09 ± 57% -17.2 11.93 ±173% perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write.cold.new_sync_write 25.68 ± 57% -14.9 10.79 ±173% perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write.cold 24.54 ± 57% -14.0 10.51 ±173% perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit 15.00 ± 46% -11.4 3.59 ±106% perf-profile.calltrace.cycles-pp.do_group_exit.get_signal.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode 15.00 ± 46% -11.4 3.59 ±106% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.get_signal.arch_do_signal.exit_to_user_mode_prepare 15.00 ± 46% -11.4 3.59 ±106% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 15.00 ± 46% -11.4 3.59 ±106% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 15.00 ± 46% -11.4 3.59 ±106% perf-profile.calltrace.cycles-pp.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 15.00 ± 46% -11.4 3.59 ±106% perf-profile.calltrace.cycles-pp.get_signal.arch_do_signal.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 20.19 ± 58% -11.1 9.09 ±173% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock 20.19 ± 58% -11.1 9.09 ±173% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit 13.48 ± 48% -10.9 2.58 ±109% perf-profile.calltrace.cycles-pp.perf_release.__fput.task_work_run.do_exit.do_group_exit 13.48 ± 48% -10.9 2.58 ±109% perf-profile.calltrace.cycles-pp.perf_event_release_kernel.perf_release.__fput.task_work_run.do_exit 16.11 ± 40% -10.4 5.73 ±114% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 13.48 ± 48% -9.9 3.59 ±106% perf-profile.calltrace.cycles-pp.task_work_run.do_exit.do_group_exit.get_signal.arch_do_signal 13.48 ± 48% -9.9 3.59 ±106% perf-profile.calltrace.cycles-pp.__fput.task_work_run.do_exit.do_group_exit.get_signal 18.44 ± 58% -9.6 8.81 ±173% perf-profile.calltrace.cycles-pp.io_serial_in.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write 4.35 ± 76% -3.5 0.85 ±173% perf-profile.calltrace.cycles-pp.io_serial_out.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit 47.23 ± 27% -26.2 21.02 ± 78% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 29.27 ± 57% -17.3 11.93 ±173% perf-profile.children.cycles-pp.write 29.27 ± 57% -17.3 11.93 ±173% perf-profile.children.cycles-pp.ksys_write 29.27 ± 57% -17.3 11.93 ±173% perf-profile.children.cycles-pp.vfs_write 29.27 ± 57% -17.3 11.93 ±173% perf-profile.children.cycles-pp.new_sync_write 29.09 ± 57% -17.2 11.93 ±173% perf-profile.children.cycles-pp.devkmsg_write.cold 29.09 ± 57% -17.2 11.93 ±173% perf-profile.children.cycles-pp.devkmsg_emit 29.09 ± 57% -17.2 11.93 ±173% perf-profile.children.cycles-pp.vprintk_emit 29.09 ± 57% -17.2 11.93 ±173% perf-profile.children.cycles-pp.console_unlock 25.68 ± 57% -14.9 10.79 ±173% perf-profile.children.cycles-pp.serial8250_console_write 24.54 ± 57% -14.3 10.23 ±173% perf-profile.children.cycles-pp.uart_console_write 15.19 ± 44% -11.6 3.59 ±106% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 15.19 ± 44% -11.6 3.59 ±106% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 15.74 ± 41% -11.6 4.15 ±112% perf-profile.children.cycles-pp.do_group_exit 15.74 ± 41% -11.6 4.15 ±112% perf-profile.children.cycles-pp.do_exit 21.15 ± 58% -11.5 9.66 ±173% perf-profile.children.cycles-pp.wait_for_xmitr 15.00 ± 46% -11.4 3.59 ±106% perf-profile.children.cycles-pp.arch_do_signal 15.00 ± 46% -11.4 3.59 ±106% perf-profile.children.cycles-pp.get_signal 20.19 ± 58% -11.1 9.09 ±173% perf-profile.children.cycles-pp.serial8250_console_putchar 13.48 ± 48% -10.9 2.58 ±109% perf-profile.children.cycles-pp.perf_release 13.48 ± 48% -10.9 2.58 ±109% perf-profile.children.cycles-pp.perf_event_release_kernel 19.40 ± 58% -10.0 9.38 ±173% perf-profile.children.cycles-pp.io_serial_in 13.48 ± 48% -9.9 3.59 ±106% perf-profile.children.cycles-pp.task_work_run 13.48 ± 48% -9.9 3.59 ±106% perf-profile.children.cycles-pp.__fput 4.54 ± 79% -3.4 1.14 ±173% perf-profile.children.cycles-pp.io_serial_out 19.40 ± 58% -10.0 9.38 ±173% perf-profile.self.cycles-pp.io_serial_in 4.54 ± 79% -3.4 1.14 ±173% perf-profile.self.cycles-pp.io_serial_out *************************************************************************************************** lkp-csl-2ap3: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/thread/50%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap3/readseek2/will-it-scale/0x5002f01 commit: 5925fa68fe (" perf tools fixes for v5.9: 3rd batch") 5ef64cc898 ("mm: allow a controlled amount of unfairness in the page lock") 5925fa68fe824465 5ef64cc8987a9211d3f3667331b ---------------- --------------------------- %stddev %change %stddev \ | \ 1266 +688.3% 9979 ± 3% will-it-scale.per_thread_ops 121587 +688.0% 958052 ± 3% will-it-scale.workload 71.50 ± 5% -28.3% 51.25 vmstat.cpu.id 25.00 ± 18% -100.0% 0.00 vmstat.cpu.wa 1.00 +8975.0% 90.75 vmstat.procs.r 1.225e+09 ± 41% -63.7% 4.441e+08 ± 69% cpuidle.C1.time 1.272e+08 ± 18% -43.3% 72207404 ± 7% cpuidle.C1E.usage 26464356 ± 10% +69.2% 44765776 ± 10% cpuidle.POLL.time 2008351 ± 16% +641.3% 14887271 ± 10% cpuidle.POLL.usage 7829 ± 6% +1248.0% 105533 ± 5% meminfo.Active 7825 ± 6% +1248.6% 105529 ± 5% meminfo.Active(anon) 36955 +11.8% 41313 meminfo.Mapped 27224 +368.3% 127479 ± 4% meminfo.Shmem 31081 -59.8% 12502 ±104% numa-numastat.node1.other_node 220546 ± 7% -28.7% 157349 ± 3% numa-numastat.node2.local_node 251604 ± 6% -28.2% 180707 ± 7% numa-numastat.node2.numa_hit 7880 ±170% +294.3% 31072 numa-numastat.node3.other_node 71.78 ± 6% -20.4 51.40 mpstat.cpu.all.idle% 25.99 ± 16% -26.0 0.03 ±105% mpstat.cpu.all.iowait% 1.53 ± 3% -1.0 0.52 ± 2% mpstat.cpu.all.irq% 0.11 ± 2% -0.0 0.07 ± 3% mpstat.cpu.all.soft% 0.53 ± 2% +47.4 47.91 mpstat.cpu.all.sys% 0.05 +0.0 0.09 ± 22% mpstat.cpu.all.usr% 1988 ± 18% +31.3% 2610 ± 20% numa-vmstat.node0.nr_mapped 77149 ± 37% -63.9% 27823 ± 25% numa-vmstat.node0.numa_other 179.00 ±130% +232.8% 595.75 ± 56% numa-vmstat.node1.nr_active_anon 179.00 ±130% +232.8% 595.75 ± 56% numa-vmstat.node1.nr_zone_active_anon 1260 ± 23% +1900.9% 25226 ± 5% numa-vmstat.node3.nr_active_anon 71809 +34.9% 96880 ± 3% numa-vmstat.node3.nr_file_pages 1845 ± 34% +1367.1% 27068 ± 11% numa-vmstat.node3.nr_shmem 9936 ± 9% -42.1% 5758 ± 4% numa-vmstat.node3.nr_slab_reclaimable 1260 ± 23% +1900.8% 25225 ± 5% numa-vmstat.node3.nr_zone_active_anon 65717 ± 43% +87.3% 123087 numa-vmstat.node3.numa_other 1181 ± 13% +28.6% 1520 ± 5% slabinfo.dmaengine-unmap-16.active_objs 1181 ± 13% +28.6% 1520 ± 5% slabinfo.dmaengine-unmap-16.num_objs 6785 ± 9% +24.7% 8464 ± 2% slabinfo.files_cache.active_objs 6785 ± 9% +24.7% 8464 ± 2% slabinfo.files_cache.num_objs 3748 ± 5% +13.8% 4267 ± 2% slabinfo.sighand_cache.active_objs 3793 ± 5% +13.4% 4300 ± 2% slabinfo.sighand_cache.num_objs 829.50 ± 14% +23.8% 1027 ± 5% slabinfo.skbuff_fclone_cache.active_objs 829.50 ± 14% +23.8% 1027 ± 5% slabinfo.skbuff_fclone_cache.num_objs 25690 -17.0% 21326 slabinfo.vmap_area.active_objs 25696 -17.0% 21331 slabinfo.vmap_area.num_objs 7835 ± 19% +30.5% 10227 ± 22% numa-meminfo.node0.Mapped 718.00 ±129% +232.6% 2387 ± 56% numa-meminfo.node1.Active 718.00 ±129% +232.1% 2384 ± 56% numa-meminfo.node1.Active(anon) 5038 ± 23% +1904.1% 100980 ± 5% numa-meminfo.node3.Active 5035 ± 23% +1905.3% 100980 ± 5% numa-meminfo.node3.Active(anon) 287239 +34.9% 387535 ± 3% numa-meminfo.node3.FilePages 39748 ± 9% -42.0% 23046 ± 4% numa-meminfo.node3.KReclaimable 789152 ± 4% +7.8% 850631 ± 6% numa-meminfo.node3.MemUsed 39748 ± 9% -42.0% 23046 ± 4% numa-meminfo.node3.SReclaimable 7378 ± 34% +1367.6% 108288 ± 11% numa-meminfo.node3.Shmem 112975 ± 7% -22.4% 87674 ± 10% numa-meminfo.node3.Slab 1955 ± 6% +1249.2% 26376 ± 5% proc-vmstat.nr_active_anon 292548 +8.6% 317683 proc-vmstat.nr_file_pages 9365 +12.1% 10501 proc-vmstat.nr_mapped 6805 +368.3% 31870 ± 4% proc-vmstat.nr_shmem 31824 +1.8% 32389 proc-vmstat.nr_slab_reclaimable 1955 ± 6% +1249.2% 26376 ± 5% proc-vmstat.nr_zone_active_anon 2404 ± 12% +107.0% 4976 ± 49% proc-vmstat.numa_hint_faults_local 996995 +3.3% 1030379 proc-vmstat.numa_hit 903561 +3.7% 936976 proc-vmstat.numa_local 92254 ± 13% +44.3% 133088 ± 10% proc-vmstat.numa_pte_updates 1519 ± 24% +1609.0% 25963 ± 6% proc-vmstat.pgactivate 1084057 +3.1% 1117729 proc-vmstat.pgalloc_normal 1156577 +4.0% 1202453 proc-vmstat.pgfault 170.02 ± 57% +8564.4% 14731 ± 41% sched_debug.cfs_rq:/.MIN_vruntime.avg 16736 ± 45% +11393.3% 1923555 ± 36% sched_debug.cfs_rq:/.MIN_vruntime.max 1596 ± 49% +10399.2% 167614 ± 39% sched_debug.cfs_rq:/.MIN_vruntime.stddev 1242 ± 10% +5367.8% 67950 ± 8% sched_debug.cfs_rq:/.exec_clock.avg 3496 ± 16% +3299.1% 118860 ± 6% sched_debug.cfs_rq:/.exec_clock.max 307.99 ± 22% +5682.7% 17809 ± 31% sched_debug.cfs_rq:/.exec_clock.min 443.83 ± 8% +2973.4% 13640 ± 4% sched_debug.cfs_rq:/.exec_clock.stddev 466502 ± 18% -85.2% 69202 ±129% sched_debug.cfs_rq:/.load.max 56674 ± 17% -82.3% 10016 ± 67% sched_debug.cfs_rq:/.load.stddev 553.96 ± 7% -54.4% 252.48 ± 39% sched_debug.cfs_rq:/.load_avg.max 53.50 ± 10% -43.5% 30.25 ± 38% sched_debug.cfs_rq:/.load_avg.stddev 170.02 ± 57% +8564.4% 14731 ± 41% sched_debug.cfs_rq:/.max_vruntime.avg 16736 ± 45% +11393.3% 1923555 ± 36% sched_debug.cfs_rq:/.max_vruntime.max 1596 ± 49% +10399.2% 167614 ± 39% sched_debug.cfs_rq:/.max_vruntime.stddev 33466 ± 25% +17200.5% 5789901 ± 8% sched_debug.cfs_rq:/.min_vruntime.avg 60233 ± 19% +16685.1% 10110254 ± 6% sched_debug.cfs_rq:/.min_vruntime.max 20331 ± 26% +7420.2% 1528965 ± 32% sched_debug.cfs_rq:/.min_vruntime.min 4797 ± 11% +24047.1% 1158377 ± 4% sched_debug.cfs_rq:/.min_vruntime.stddev 0.03 ± 15% +1586.0% 0.53 ± 6% sched_debug.cfs_rq:/.nr_running.avg 0.17 ± 8% +155.9% 0.43 ± 2% sched_debug.cfs_rq:/.nr_running.stddev 0.13 ±103% +263.7% 0.49 ± 18% sched_debug.cfs_rq:/.nr_spread_over.avg 1.13 ± 63% +237.5% 3.82 ± 15% sched_debug.cfs_rq:/.nr_spread_over.max 0.17 ± 89% +337.2% 0.76 ± 11% sched_debug.cfs_rq:/.nr_spread_over.stddev 31.86 ± 14% +1420.5% 484.49 ± 7% sched_debug.cfs_rq:/.runnable_avg.avg 702.53 ± 4% +33.6% 938.62 ± 5% sched_debug.cfs_rq:/.runnable_avg.max 86.88 ± 7% +323.3% 367.76 ± 2% sched_debug.cfs_rq:/.runnable_avg.stddev -1332 -1.7e+05% 2293744 ± 23% sched_debug.cfs_rq:/.spread0.avg 25416 ± 45% +25922.7% 6614096 ± 10% sched_debug.cfs_rq:/.spread0.max -14452 +13512.9% -1967432 sched_debug.cfs_rq:/.spread0.min 4799 ± 11% +24038.1% 1158452 ± 4% sched_debug.cfs_rq:/.spread0.stddev 31.73 ± 14% +1425.9% 484.20 ± 7% sched_debug.cfs_rq:/.util_avg.avg 702.49 ± 4% +33.6% 938.58 ± 5% sched_debug.cfs_rq:/.util_avg.max 86.82 ± 7% +323.6% 367.72 ± 2% sched_debug.cfs_rq:/.util_avg.stddev 3.25 ± 12% +13255.4% 434.48 ± 6% sched_debug.cfs_rq:/.util_est_enqueued.avg 259.20 ± 5% +242.9% 888.80 sched_debug.cfs_rq:/.util_est_enqueued.max 24.35 ± 6% +1360.2% 355.60 sched_debug.cfs_rq:/.util_est_enqueued.stddev 853642 -37.8% 530714 sched_debug.cpu.avg_idle.avg 75902 ± 12% -80.9% 14497 ± 29% sched_debug.cpu.avg_idle.min 177509 ± 4% +147.6% 439550 sched_debug.cpu.avg_idle.stddev 96.07 ± 9% +1468.1% 1506 ± 2% sched_debug.cpu.curr->pid.avg 720.78 ± 4% +132.6% 1676 sched_debug.cpu.curr->pid.stddev 0.00 ± 20% +322.0% 0.00 ± 27% sched_debug.cpu.next_balance.stddev 0.02 ± 8% +1657.9% 0.41 sched_debug.cpu.nr_running.avg 0.15 ± 4% +207.4% 0.45 sched_debug.cpu.nr_running.stddev 49550 ± 6% -31.8% 33787 ± 10% sched_debug.cpu.nr_switches.stddev 0.40 -94.3% 0.02 ± 42% sched_debug.cpu.nr_uninterruptible.avg 49442 ± 6% -31.9% 33686 ± 10% sched_debug.cpu.sched_count.stddev 24716 ± 6% -31.9% 16830 ± 10% sched_debug.cpu.sched_goidle.stddev 24764 ± 6% -32.3% 16770 ± 10% sched_debug.cpu.ttwu_count.stddev 4432 ± 40% -60.8% 1735 ± 26% sched_debug.cpu.ttwu_local.max 337.31 ± 35% -54.2% 154.44 ± 23% sched_debug.cpu.ttwu_local.stddev 17.43 -85.9% 2.46 ± 7% perf-stat.i.MPKI 7.452e+08 +805.2% 6.745e+09 perf-stat.i.branch-instructions 1.69 -1.3 0.34 ± 3% perf-stat.i.branch-miss-rate% 13281209 +71.2% 22738819 ± 2% perf-stat.i.branch-misses 13.68 ± 6% +40.3 54.02 ± 2% perf-stat.i.cache-miss-rate% 8061015 ± 8% +357.2% 36853851 ± 9% perf-stat.i.cache-misses 59267552 ± 2% +15.1% 68218277 ± 7% perf-stat.i.cache-references 3.22 ± 2% +221.8% 10.36 perf-stat.i.cpi 1.103e+10 ± 2% +2512.2% 2.882e+11 perf-stat.i.cpu-cycles 223.55 +11.5% 249.23 perf-stat.i.cpu-migrations 1409 ± 5% +459.4% 7886 ± 10% perf-stat.i.cycles-between-cache-misses 0.00 ± 3% -0.0 0.00 ± 4% perf-stat.i.dTLB-load-miss-rate% 42216 ± 6% +374.4% 200286 ± 4% perf-stat.i.dTLB-load-misses 9.242e+08 +667.5% 7.093e+09 perf-stat.i.dTLB-loads 0.00 -0.0 0.00 ± 3% perf-stat.i.dTLB-store-miss-rate% 13269 ± 2% +23.8% 16421 perf-stat.i.dTLB-store-misses 5.132e+08 +64.4% 8.439e+08 ± 2% perf-stat.i.dTLB-stores 57.39 +14.0 71.43 perf-stat.i.iTLB-load-miss-rate% 7475200 ± 2% +51.8% 11347314 ± 4% perf-stat.i.iTLB-load-misses 5545950 -18.1% 4541031 ± 7% perf-stat.i.iTLB-loads 3.499e+09 +694.7% 2.781e+10 perf-stat.i.instructions 470.84 +421.6% 2455 ± 4% perf-stat.i.instructions-per-iTLB-miss 0.31 ± 2% -68.9% 0.10 perf-stat.i.ipc 0.69 ± 3% +55.3% 1.07 ± 5% perf-stat.i.major-faults 0.06 ± 2% +2510.5% 1.50 perf-stat.i.metric.GHz 0.44 ± 14% +128.0% 1.01 ± 2% perf-stat.i.metric.K/sec 11.72 +556.3% 76.89 perf-stat.i.metric.M/sec 3716 +4.5% 3884 perf-stat.i.minor-faults 96.24 +2.0 98.23 perf-stat.i.node-load-miss-rate% 1506224 ± 8% +563.7% 9996147 perf-stat.i.node-load-misses 59182 ± 20% +202.2% 178865 ± 2% perf-stat.i.node-loads 97.87 +1.9 99.75 perf-stat.i.node-store-miss-rate% 406316 ± 6% +1532.3% 6632290 perf-stat.i.node-store-misses 9574 ± 16% +26.7% 12131 ± 8% perf-stat.i.node-stores 3717 +4.5% 3885 perf-stat.i.page-faults 16.94 -85.5% 2.45 ± 7% perf-stat.overall.MPKI 1.78 -1.4 0.34 ± 3% perf-stat.overall.branch-miss-rate% 13.59 ± 6% +40.3 53.94 ± 2% perf-stat.overall.cache-miss-rate% 3.15 ± 2% +228.8% 10.37 perf-stat.overall.cpi 1374 ± 5% +474.4% 7895 ± 10% perf-stat.overall.cycles-between-cache-misses 0.00 ± 6% -0.0 0.00 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.00 -0.0 0.00 ± 3% perf-stat.overall.dTLB-store-miss-rate% 57.40 +14.1 71.46 perf-stat.overall.iTLB-load-miss-rate% 468.36 ± 2% +424.0% 2454 ± 4% perf-stat.overall.instructions-per-iTLB-miss 0.32 ± 2% -69.6% 0.10 perf-stat.overall.ipc 96.24 +2.0 98.24 perf-stat.overall.node-load-miss-rate% 97.70 +2.1 99.82 perf-stat.overall.node-store-miss-rate% 7.428e+08 +804.5% 6.719e+09 perf-stat.ps.branch-instructions 13241794 +70.8% 22612139 ± 2% perf-stat.ps.branch-misses 8035056 ± 8% +356.9% 36710582 ± 9% perf-stat.ps.cache-misses 59071874 ± 2% +15.0% 67947551 ± 7% perf-stat.ps.cache-references 1.1e+10 ± 2% +2510.8% 2.871e+11 perf-stat.ps.cpu-cycles 222.96 +11.4% 248.34 perf-stat.ps.cpu-migrations 42122 ± 6% +373.7% 199521 ± 4% perf-stat.ps.dTLB-load-misses 9.212e+08 +667.0% 7.065e+09 perf-stat.ps.dTLB-loads 13235 ± 2% +23.4% 16333 perf-stat.ps.dTLB-store-misses 5.116e+08 +64.3% 8.404e+08 ± 2% perf-stat.ps.dTLB-stores 7450273 ± 2% +51.7% 11304013 ± 4% perf-stat.ps.iTLB-load-misses 5527433 -18.2% 4522978 ± 7% perf-stat.ps.iTLB-loads 3.488e+09 +694.1% 2.77e+10 perf-stat.ps.instructions 0.69 ± 3% +53.2% 1.05 ± 5% perf-stat.ps.major-faults 3705 +4.3% 3862 perf-stat.ps.minor-faults 1501438 ± 8% +563.2% 9958261 perf-stat.ps.node-load-misses 59037 ± 20% +202.0% 178306 ± 2% perf-stat.ps.node-loads 405024 ± 6% +1531.3% 6607029 perf-stat.ps.node-store-misses 9563 ± 16% +26.0% 12047 ± 8% perf-stat.ps.node-stores 3705 +4.3% 3863 perf-stat.ps.page-faults 1.053e+12 +693.1% 8.355e+12 perf-stat.total.instructions 89.08 -56.7 32.42 ± 20% perf-profile.calltrace.cycles-pp.secondary_startup_64 88.49 -56.3 32.22 ± 20% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 88.48 -56.3 32.22 ± 20% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 88.43 -56.2 32.22 ± 20% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 71.64 -39.7 31.95 ± 20% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 67.10 -35.2 31.95 ± 20% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 25.60 ± 4% -25.6 0.00 perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 14.64 ± 7% -14.6 0.00 perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 45.20 -13.4 31.77 ± 20% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 12.89 ± 2% -12.9 0.00 perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 9.12 ± 6% -9.1 0.00 perf-profile.calltrace.cycles-pp.asm_call_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 9.08 ± 6% -9.1 0.00 perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 8.88 ± 6% -8.9 0.00 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 2.17 +25.8 27.93 ± 10% perf-profile.calltrace.cycles-pp.__lock_page.find_lock_entry.shmem_getpage_gfp.shmem_file_read_iter.new_sync_read 2.46 +26.5 28.95 ± 10% perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_file_read_iter.new_sync_read.vfs_read.ksys_read 2.42 +26.5 28.91 ± 10% perf-profile.calltrace.cycles-pp.find_lock_entry.shmem_getpage_gfp.shmem_file_read_iter.new_sync_read.vfs_read 0.00 +27.4 27.39 ± 10% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.__lock_page.find_lock_entry.shmem_getpage_gfp 0.00 +27.5 27.48 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.__lock_page.find_lock_entry.shmem_getpage_gfp.shmem_file_read_iter 4.69 ± 4% +32.8 37.48 ± 9% perf-profile.calltrace.cycles-pp.wake_up_page_bit.shmem_file_read_iter.new_sync_read.vfs_read.ksys_read 0.00 +36.9 36.91 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.wake_up_page_bit.shmem_file_read_iter.new_sync_read 0.00 +37.0 36.98 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.wake_up_page_bit.shmem_file_read_iter.new_sync_read.vfs_read 9.25 ± 2% +58.2 67.42 ± 9% perf-profile.calltrace.cycles-pp.__libc_read 8.75 ± 2% +58.6 67.37 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read 8.61 ± 2% +58.8 67.36 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 8.53 ± 2% +58.8 67.35 ± 9% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 8.33 ± 2% +59.0 67.30 ± 9% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 8.04 ± 2% +59.2 67.27 ± 9% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 7.99 ± 2% +59.3 67.26 ± 9% perf-profile.calltrace.cycles-pp.shmem_file_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64 89.08 -56.7 32.42 ± 20% perf-profile.children.cycles-pp.secondary_startup_64 89.08 -56.7 32.42 ± 20% perf-profile.children.cycles-pp.cpu_startup_entry 89.06 -56.6 32.42 ± 20% perf-profile.children.cycles-pp.do_idle 88.49 -56.3 32.22 ± 20% perf-profile.children.cycles-pp.start_secondary 72.17 -40.0 32.15 ± 20% perf-profile.children.cycles-pp.cpuidle_enter 72.13 -40.0 32.15 ± 20% perf-profile.children.cycles-pp.cpuidle_enter_state 20.61 ± 2% -20.3 0.29 ± 17% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 14.94 ± 7% -14.7 0.23 ± 19% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 45.50 -13.5 31.96 ± 20% perf-profile.children.cycles-pp.intel_idle 12.98 ± 2% -12.9 0.04 ± 59% perf-profile.children.cycles-pp.menu_select 11.71 ± 6% -11.5 0.22 ± 17% perf-profile.children.cycles-pp.asm_call_on_stack 9.28 ± 6% -9.1 0.16 ± 15% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 9.08 ± 6% -8.9 0.16 ± 15% perf-profile.children.cycles-pp.hrtimer_interrupt 4.91 ± 4% -4.8 0.11 ± 11% perf-profile.children.cycles-pp.__hrtimer_run_queues 4.24 ± 4% -4.1 0.18 ± 6% perf-profile.children.cycles-pp.__wake_up_common 4.10 ± 4% -4.0 0.15 ± 5% perf-profile.children.cycles-pp.wake_page_function 3.90 ± 4% -3.8 0.11 ± 4% perf-profile.children.cycles-pp.try_to_wake_up 3.39 ± 3% -3.3 0.13 ± 6% perf-profile.children.cycles-pp.ttwu_do_activate 3.36 ± 3% -3.2 0.13 ± 6% perf-profile.children.cycles-pp.enqueue_task_fair 3.19 ± 3% -3.1 0.13 ± 5% perf-profile.children.cycles-pp.enqueue_entity 3.09 ± 6% -3.0 0.08 ± 19% perf-profile.children.cycles-pp.tick_sched_timer 2.83 ± 2% -2.7 0.14 ± 5% perf-profile.children.cycles-pp.__schedule 2.67 ± 5% -2.6 0.03 ±105% perf-profile.children.cycles-pp.irq_exit_rcu 2.42 ± 6% -2.3 0.07 ± 15% perf-profile.children.cycles-pp.tick_sched_handle 2.40 ± 6% -2.3 0.07 ± 15% perf-profile.children.cycles-pp.update_process_times 2.33 ± 4% -2.2 0.10 ± 4% perf-profile.children.cycles-pp.__account_scheduler_latency 2.05 ± 5% -2.0 0.08 ± 5% perf-profile.children.cycles-pp.stack_trace_save_tsk 1.85 ± 4% -1.8 0.08 ± 10% perf-profile.children.cycles-pp.arch_stack_walk 1.74 -1.6 0.10 ± 4% perf-profile.children.cycles-pp.schedule 1.71 -1.6 0.10 ± 7% perf-profile.children.cycles-pp.io_schedule 1.20 ± 4% -1.1 0.06 ± 7% perf-profile.children.cycles-pp.unwind_next_frame 1.15 ± 6% -1.1 0.04 ± 58% perf-profile.children.cycles-pp.scheduler_tick 1.19 ± 18% -1.1 0.13 ± 6% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle 1.07 ± 18% -0.9 0.12 ± 6% perf-profile.children.cycles-pp.sched_ttwu_pending 0.65 ± 10% -0.5 0.14 ± 11% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.55 ± 17% -0.5 0.10 ± 16% perf-profile.children.cycles-pp.poll_idle 0.53 ± 7% -0.4 0.10 ± 10% perf-profile.children.cycles-pp.__libc_lseek64 0.59 ± 29% -0.4 0.20 ± 20% perf-profile.children.cycles-pp.start_kernel 0.50 ± 2% -0.4 0.12 ± 10% perf-profile.children.cycles-pp.copy_page_to_iter 0.25 ± 13% -0.2 0.03 ±100% perf-profile.children.cycles-pp.ksys_lseek 0.30 ± 6% -0.2 0.09 ± 9% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.29 ± 6% -0.2 0.09 ± 9% perf-profile.children.cycles-pp.copyout 0.25 ± 26% -0.2 0.05 ± 8% perf-profile.children.cycles-pp.__fdget_pos 0.00 +0.1 0.14 ± 10% perf-profile.children.cycles-pp.mark_page_accessed 0.07 ± 17% +0.5 0.54 ± 11% perf-profile.children.cycles-pp.find_get_entry 2.18 +25.8 27.95 ± 10% perf-profile.children.cycles-pp.__lock_page 2.46 +26.5 28.95 ± 10% perf-profile.children.cycles-pp.shmem_getpage_gfp 2.43 +26.5 28.92 ± 10% perf-profile.children.cycles-pp.find_lock_entry 0.23 ± 13% +27.3 27.49 ± 10% perf-profile.children.cycles-pp._raw_spin_lock_irq 4.69 ± 4% +32.8 37.48 ± 9% perf-profile.children.cycles-pp.wake_up_page_bit 1.40 ± 5% +35.6 37.03 ± 9% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 9.53 ± 2% +57.9 67.46 ± 9% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 9.32 ± 2% +58.1 67.43 ± 9% perf-profile.children.cycles-pp.__libc_read 9.33 ± 2% +58.1 67.44 ± 9% perf-profile.children.cycles-pp.do_syscall_64 8.61 ± 2% +58.7 67.35 ± 9% perf-profile.children.cycles-pp.ksys_read 8.42 ± 2% +58.9 67.30 ± 9% perf-profile.children.cycles-pp.vfs_read 8.05 ± 2% +59.2 67.27 ± 9% perf-profile.children.cycles-pp.new_sync_read 7.99 ± 2% +59.3 67.26 ± 9% perf-profile.children.cycles-pp.shmem_file_read_iter 0.34 ± 7% +64.0 64.31 ± 9% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 45.49 -13.5 31.96 ± 20% perf-profile.self.cycles-pp.intel_idle 9.33 ± 5% -9.3 0.00 perf-profile.self.cycles-pp.menu_select 9.05 ± 14% -9.1 0.00 perf-profile.self.cycles-pp.cpuidle_enter_state 1.15 ± 5% -1.0 0.11 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.49 ± 16% -0.4 0.10 ± 16% perf-profile.self.cycles-pp.poll_idle 0.29 ± 5% -0.2 0.09 ± 9% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.00 +0.1 0.14 ± 11% perf-profile.self.cycles-pp.mark_page_accessed 0.00 +0.2 0.18 ± 11% perf-profile.self.cycles-pp.wake_up_page_bit 0.19 ± 14% +0.3 0.47 ± 12% perf-profile.self.cycles-pp.shmem_file_read_iter 0.01 ±173% +0.3 0.33 ± 11% perf-profile.self.cycles-pp.find_lock_entry 0.06 ± 14% +0.5 0.51 ± 11% perf-profile.self.cycles-pp.find_get_entry 0.34 ± 7% +64.0 64.31 ± 9% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 40631 +54.3% 62702 ± 5% softirqs.CPU0.SCHED 39633 ± 5% +94.6% 77112 ± 7% softirqs.CPU1.SCHED 38496 +106.7% 79558 ± 4% softirqs.CPU10.SCHED 38753 +88.1% 72913 ± 7% softirqs.CPU100.SCHED 38787 +84.0% 71383 ± 12% softirqs.CPU101.SCHED 38756 +75.9% 68184 ± 7% softirqs.CPU102.SCHED 38730 +78.4% 69103 ± 6% softirqs.CPU103.SCHED 38745 +75.2% 67870 ± 5% softirqs.CPU104.SCHED 38709 +94.0% 75083 ± 8% softirqs.CPU105.SCHED 38983 +69.0% 65884 ± 10% softirqs.CPU106.SCHED 38712 +73.6% 67205 ± 6% softirqs.CPU107.SCHED 38657 +79.1% 69224 ± 6% softirqs.CPU108.SCHED 38755 +82.1% 70592 ± 9% softirqs.CPU109.SCHED 38376 +103.8% 78211 ± 7% softirqs.CPU11.SCHED 38830 +73.5% 67362 ± 8% softirqs.CPU110.SCHED 38711 +78.9% 69245 ± 12% softirqs.CPU111.SCHED 38752 +74.7% 67717 ± 6% softirqs.CPU112.SCHED 38878 +82.8% 71085 ± 8% softirqs.CPU113.SCHED 38720 +79.7% 69581 ± 9% softirqs.CPU114.SCHED 38626 +74.7% 67473 ± 7% softirqs.CPU115.SCHED 34760 ± 20% +102.9% 70520 ± 8% softirqs.CPU116.SCHED 38677 +76.2% 68131 ± 7% softirqs.CPU117.SCHED 35465 ± 21% +91.8% 68005 ± 7% softirqs.CPU118.SCHED 34519 ± 20% +103.1% 70108 ± 7% softirqs.CPU119.SCHED 38170 +100.7% 76597 ± 9% softirqs.CPU12.SCHED 39018 +83.8% 71712 ± 8% softirqs.CPU120.SCHED 38857 +75.0% 67994 ± 9% softirqs.CPU121.SCHED 38733 +76.0% 68183 ± 8% softirqs.CPU122.SCHED 38784 +78.3% 69142 ± 6% softirqs.CPU123.SCHED 38778 +74.8% 67770 ± 3% softirqs.CPU124.SCHED 39019 +74.5% 68088 ± 6% softirqs.CPU125.SCHED 38781 +74.9% 67826 ± 7% softirqs.CPU126.SCHED 38770 +71.9% 66651 ± 8% softirqs.CPU127.SCHED 38841 +81.5% 70505 ± 8% softirqs.CPU128.SCHED 38690 +75.1% 67746 ± 4% softirqs.CPU129.SCHED 38239 ± 2% +96.2% 75045 ± 6% softirqs.CPU13.SCHED 38875 +79.2% 69655 ± 8% softirqs.CPU130.SCHED 38866 +73.9% 67575 ± 7% softirqs.CPU131.SCHED 38723 +75.5% 67944 ± 7% softirqs.CPU132.SCHED 38643 +78.7% 69042 ± 9% softirqs.CPU133.SCHED 38669 +78.9% 69174 ± 9% softirqs.CPU134.SCHED 38658 +78.7% 69077 ± 9% softirqs.CPU135.SCHED 38795 +75.0% 67873 ± 12% softirqs.CPU136.SCHED 38817 +70.2% 66079 ± 8% softirqs.CPU137.SCHED 39032 +75.8% 68606 ± 6% softirqs.CPU138.SCHED 38864 +72.8% 67154 ± 10% softirqs.CPU139.SCHED 38779 ± 3% +102.2% 78423 ± 5% softirqs.CPU14.SCHED 38948 +81.9% 70830 ± 10% softirqs.CPU140.SCHED 38662 +83.5% 70926 ± 12% softirqs.CPU141.SCHED 35132 ± 19% +98.2% 69627 ± 8% softirqs.CPU142.SCHED 38825 +69.5% 65816 ± 5% softirqs.CPU143.SCHED 35007 ± 20% +99.3% 69785 ± 8% softirqs.CPU144.SCHED 38938 +76.3% 68648 ± 10% softirqs.CPU145.SCHED 38990 +67.6% 65363 ± 3% softirqs.CPU146.SCHED 39037 +74.1% 67972 ± 2% softirqs.CPU147.SCHED 38810 +76.8% 68600 ± 4% softirqs.CPU148.SCHED 38929 +76.2% 68578 ± 10% softirqs.CPU149.SCHED 38071 +101.6% 76734 ± 6% softirqs.CPU15.SCHED 38904 +81.0% 70427 ± 14% softirqs.CPU150.SCHED 38908 +72.7% 67194 ± 10% softirqs.CPU151.SCHED 38827 +75.7% 68201 ± 8% softirqs.CPU152.SCHED 38848 +70.5% 66256 ± 4% softirqs.CPU153.SCHED 38894 +75.4% 68216 ± 9% softirqs.CPU154.SCHED 38881 +79.8% 69913 ± 4% softirqs.CPU155.SCHED 38886 +73.5% 67482 ± 9% softirqs.CPU156.SCHED 38777 +73.2% 67168 ± 6% softirqs.CPU157.SCHED 38833 +76.4% 68520 ± 10% softirqs.CPU158.SCHED 38874 +71.4% 66620 ± 9% softirqs.CPU159.SCHED 38189 +104.4% 78046 ± 6% softirqs.CPU16.SCHED 38889 +71.9% 66832 ± 7% softirqs.CPU160.SCHED 38982 +80.9% 70533 ± 7% softirqs.CPU161.SCHED 38937 +71.4% 66745 ± 4% softirqs.CPU162.SCHED 34815 ± 20% +96.1% 68256 ± 6% softirqs.CPU163.SCHED 38870 +72.7% 67148 ± 9% softirqs.CPU164.SCHED 38827 +76.0% 68355 ± 6% softirqs.CPU165.SCHED 38850 +82.1% 70729 ± 10% softirqs.CPU166.SCHED 35197 ± 21% +100.0% 70389 ± 10% softirqs.CPU167.SCHED 39361 ± 2% +82.8% 71946 ± 6% softirqs.CPU168.SCHED 38813 +82.6% 70862 ± 7% softirqs.CPU169.SCHED 38121 +95.0% 74331 ± 7% softirqs.CPU17.SCHED 38956 +80.2% 70181 ± 6% softirqs.CPU170.SCHED 38916 +79.7% 69914 ± 6% softirqs.CPU171.SCHED 38859 +76.2% 68479 softirqs.CPU172.SCHED 39012 +78.8% 69749 ± 9% softirqs.CPU173.SCHED 38863 +71.7% 66736 ± 4% softirqs.CPU174.SCHED 38841 +81.3% 70431 ± 7% softirqs.CPU175.SCHED 38806 +78.9% 69422 ± 10% softirqs.CPU176.SCHED 38885 +71.6% 66722 ± 6% softirqs.CPU177.SCHED 38896 +79.5% 69803 ± 9% softirqs.CPU178.SCHED 38940 +72.3% 67090 ± 12% softirqs.CPU179.SCHED 37988 +101.2% 76416 ± 6% softirqs.CPU18.SCHED 38844 +74.8% 67888 ± 4% softirqs.CPU180.SCHED 38949 +70.0% 66213 ± 5% softirqs.CPU181.SCHED 39087 +70.4% 66587 ± 5% softirqs.CPU182.SCHED 38848 +70.4% 66185 ± 5% softirqs.CPU183.SCHED 38838 +78.3% 69242 ± 7% softirqs.CPU184.SCHED 38993 +72.7% 67340 ± 10% softirqs.CPU185.SCHED 38797 +67.6% 65035 ± 7% softirqs.CPU186.SCHED 38794 +69.4% 65713 ± 6% softirqs.CPU187.SCHED 38914 +62.2% 63112 ± 6% softirqs.CPU188.SCHED 38898 +71.8% 66822 ± 4% softirqs.CPU189.SCHED 38250 +102.0% 77270 ± 7% softirqs.CPU19.SCHED 38851 +63.0% 63339 ± 4% softirqs.CPU190.SCHED 38794 +23.5% 47899 ± 3% softirqs.CPU191.SCHED 38426 +97.9% 76043 ± 11% softirqs.CPU2.SCHED 38106 +97.4% 75204 ± 5% softirqs.CPU20.SCHED 39713 ± 7% +95.0% 77425 ± 10% softirqs.CPU21.SCHED 38374 +102.0% 77519 ± 5% softirqs.CPU22.SCHED 38131 +98.3% 75617 ± 6% softirqs.CPU23.SCHED 39241 ± 3% +83.5% 72016 ± 5% softirqs.CPU24.SCHED 30118 ± 27% +151.0% 75608 ± 6% softirqs.CPU25.SCHED 38278 +97.2% 75482 ± 6% softirqs.CPU26.SCHED 38217 +96.1% 74939 ± 6% softirqs.CPU27.SCHED 38037 +99.1% 75732 ± 9% softirqs.CPU28.SCHED 38193 +98.2% 75710 ± 6% softirqs.CPU29.SCHED 38287 +102.3% 77439 ± 6% softirqs.CPU3.SCHED 38154 +99.0% 75924 ± 7% softirqs.CPU30.SCHED 38073 +102.0% 76913 ± 6% softirqs.CPU31.SCHED 38084 +92.9% 73483 ± 6% softirqs.CPU32.SCHED 38042 +99.3% 75828 ± 10% softirqs.CPU33.SCHED 38193 +94.2% 74188 ± 5% softirqs.CPU34.SCHED 38243 +98.8% 76013 ± 5% softirqs.CPU35.SCHED 37961 +99.4% 75711 ± 5% softirqs.CPU36.SCHED 38135 +94.9% 74336 ± 6% softirqs.CPU37.SCHED 38335 +94.2% 74461 ± 3% softirqs.CPU38.SCHED 38342 +94.4% 74533 ± 8% softirqs.CPU39.SCHED 37627 ± 2% +94.6% 73233 ± 6% softirqs.CPU4.SCHED 38084 +98.8% 75711 ± 2% softirqs.CPU40.SCHED 38257 +102.8% 77575 ± 5% softirqs.CPU41.SCHED 37955 +97.2% 74843 ± 9% softirqs.CPU42.SCHED 38071 +100.0% 76133 ± 4% softirqs.CPU43.SCHED 38082 +92.2% 73183 ± 5% softirqs.CPU44.SCHED 38148 +90.7% 72759 ± 7% softirqs.CPU45.SCHED 34308 ± 20% +115.3% 73863 ± 7% softirqs.CPU46.SCHED 38238 +103.4% 77791 ± 9% softirqs.CPU47.SCHED 38665 +94.8% 75310 ± 5% softirqs.CPU48.SCHED 38415 +99.2% 76524 ± 5% softirqs.CPU49.SCHED 38158 +94.4% 74167 ± 6% softirqs.CPU5.SCHED 38443 +104.8% 78724 ± 7% softirqs.CPU50.SCHED 34425 ± 20% +125.0% 77470 ± 10% softirqs.CPU51.SCHED 38267 +99.8% 76461 ± 7% softirqs.CPU52.SCHED 38349 +99.2% 76392 ± 4% softirqs.CPU53.SCHED 38403 +94.9% 74848 ± 5% softirqs.CPU54.SCHED 38395 +101.7% 77431 ± 5% softirqs.CPU55.SCHED 38356 +100.3% 76846 ± 5% softirqs.CPU56.SCHED 38445 +105.2% 78902 ± 11% softirqs.CPU57.SCHED 38351 +100.3% 76801 ± 4% softirqs.CPU58.SCHED 38463 +96.3% 75505 ± 9% softirqs.CPU59.SCHED 38189 +102.9% 77480 ± 6% softirqs.CPU6.SCHED 38471 +101.3% 77442 ± 6% softirqs.CPU60.SCHED 38361 +102.9% 77834 ± 10% softirqs.CPU61.SCHED 38143 +98.5% 75709 ± 7% softirqs.CPU62.SCHED 38955 ± 2% +102.1% 78740 ± 6% softirqs.CPU63.SCHED 38370 +103.9% 78255 ± 6% softirqs.CPU64.SCHED 38401 +94.2% 74557 ± 5% softirqs.CPU65.SCHED 38433 +104.2% 78462 ± 7% softirqs.CPU66.SCHED 38339 +100.5% 76880 ± 6% softirqs.CPU67.SCHED 38398 +103.3% 78048 ± 4% softirqs.CPU68.SCHED 38319 +100.5% 76811 ± 8% softirqs.CPU69.SCHED 34179 ± 21% +123.9% 76523 ± 7% softirqs.CPU7.SCHED 38552 +93.5% 74604 ± 4% softirqs.CPU70.SCHED 38506 +93.4% 74456 ± 6% softirqs.CPU71.SCHED 38504 +92.3% 74035 ± 5% softirqs.CPU72.SCHED 38134 +96.9% 75089 ± 7% softirqs.CPU73.SCHED 38196 +99.0% 76000 ± 6% softirqs.CPU74.SCHED 38306 +99.1% 76276 ± 6% softirqs.CPU75.SCHED 38705 +100.5% 77595 ± 10% softirqs.CPU76.SCHED 38366 +99.0% 76349 ± 5% softirqs.CPU77.SCHED 38450 +106.6% 79426 ± 8% softirqs.CPU78.SCHED 38155 +98.8% 75834 ± 4% softirqs.CPU79.SCHED 37097 ± 2% +111.0% 78291 ± 8% softirqs.CPU8.SCHED 38264 +100.0% 76540 ± 3% softirqs.CPU80.SCHED 34260 ± 20% +130.9% 79108 ± 7% softirqs.CPU81.SCHED 38212 +99.3% 76155 ± 4% softirqs.CPU82.SCHED 38116 +106.6% 78750 softirqs.CPU83.SCHED 38164 +104.8% 78146 ± 7% softirqs.CPU84.SCHED 38154 +109.1% 79799 ± 7% softirqs.CPU85.SCHED 38363 +107.0% 79418 ± 9% softirqs.CPU86.SCHED 38224 +108.8% 79830 ± 7% softirqs.CPU87.SCHED 38587 +99.1% 76832 ± 6% softirqs.CPU88.SCHED 38495 +104.7% 78813 ± 3% softirqs.CPU89.SCHED 38367 ± 2% +83.6% 70426 ± 4% softirqs.CPU9.SCHED 34134 ± 19% +137.1% 80916 ± 10% softirqs.CPU90.SCHED 38192 +109.9% 80161 ± 7% softirqs.CPU91.SCHED 38336 +116.7% 83080 ± 7% softirqs.CPU92.SCHED 38193 +107.3% 79164 ± 8% softirqs.CPU93.SCHED 27437 ± 12% -25.5% 20438 ± 13% softirqs.CPU94.RCU 38299 +115.9% 82699 ± 8% softirqs.CPU94.SCHED 18235 ± 4% +202.2% 55116 ± 33% softirqs.CPU95.SCHED 36482 ± 13% +135.6% 85946 ± 9% softirqs.CPU96.SCHED 38587 +80.8% 69767 ± 4% softirqs.CPU97.SCHED 38930 +78.6% 69537 ± 4% softirqs.CPU98.SCHED 32564 ± 32% +111.7% 68941 ± 7% softirqs.CPU99.SCHED 7325109 +89.6% 13891305 ± 6% softirqs.SCHED 236548 ± 4% +1218.3% 3118323 ± 8% interrupts.CAL:Function_call_interrupts 1429 ± 10% +585.9% 9804 ± 24% interrupts.CPU0.CAL:Function_call_interrupts 231.75 ± 33% +2494.9% 6013 ± 13% interrupts.CPU0.NMI:Non-maskable_interrupts 231.75 ± 33% +2494.9% 6013 ± 13% interrupts.CPU0.PMI:Performance_monitoring_interrupts 6.00 ± 69% +6658.3% 405.50 ± 23% interrupts.CPU0.TLB:TLB_shootdowns 1364 ± 16% +1245.8% 18364 ± 4% interrupts.CPU1.CAL:Function_call_interrupts 203.00 +3411.8% 7129 ± 4% interrupts.CPU1.NMI:Non-maskable_interrupts 203.00 +3411.8% 7129 ± 4% interrupts.CPU1.PMI:Performance_monitoring_interrupts 1143 ± 15% -66.3% 385.25 ± 8% interrupts.CPU1.RES:Rescheduling_interrupts 21.00 ± 83% +3652.4% 788.00 ± 7% interrupts.CPU1.TLB:TLB_shootdowns 1110 ± 5% +1759.6% 20646 ± 12% interrupts.CPU10.CAL:Function_call_interrupts 208.00 +2277.9% 4946 ± 29% interrupts.CPU10.NMI:Non-maskable_interrupts 208.00 +2277.9% 4946 ± 29% interrupts.CPU10.PMI:Performance_monitoring_interrupts 830.50 ± 28% -51.5% 402.50 ± 13% interrupts.CPU10.RES:Rescheduling_interrupts 29.00 ±143% +2773.3% 833.25 ± 13% interrupts.CPU10.TLB:TLB_shootdowns 1131 ± 18% +1366.0% 16591 ± 7% interrupts.CPU100.CAL:Function_call_interrupts 204.00 ± 3% +2103.3% 4494 ± 20% interrupts.CPU100.NMI:Non-maskable_interrupts 204.00 ± 3% +2103.3% 4494 ± 20% interrupts.CPU100.PMI:Performance_monitoring_interrupts 1286 ± 8% -74.4% 329.50 ± 9% interrupts.CPU100.RES:Rescheduling_interrupts 9.00 ± 65% +8488.9% 773.00 ± 14% interrupts.CPU100.TLB:TLB_shootdowns 1189 ± 23% +1270.2% 16298 ± 12% interrupts.CPU101.CAL:Function_call_interrupts 202.00 ± 3% +3234.9% 6736 ± 14% interrupts.CPU101.NMI:Non-maskable_interrupts 202.00 ± 3% +3234.9% 6736 ± 14% interrupts.CPU101.PMI:Performance_monitoring_interrupts 1486 ± 20% -80.0% 297.50 ± 11% interrupts.CPU101.RES:Rescheduling_interrupts 17.75 ± 97% +3942.3% 717.50 ± 10% interrupts.CPU101.TLB:TLB_shootdowns 1066 ± 9% +1193.7% 13800 ± 7% interrupts.CPU102.CAL:Function_call_interrupts 201.75 ± 3% +2311.2% 4864 ± 29% interrupts.CPU102.NMI:Non-maskable_interrupts 201.75 ± 3% +2311.2% 4864 ± 29% interrupts.CPU102.PMI:Performance_monitoring_interrupts 1124 ± 17% -75.4% 276.25 ± 10% interrupts.CPU102.RES:Rescheduling_interrupts 8.00 ± 30% +7431.2% 602.50 ± 11% interrupts.CPU102.TLB:TLB_shootdowns 1151 ± 12% +1149.8% 14391 ± 5% interrupts.CPU103.CAL:Function_call_interrupts 180.75 ± 26% +2754.5% 5159 ± 23% interrupts.CPU103.NMI:Non-maskable_interrupts 180.75 ± 26% +2754.5% 5159 ± 23% interrupts.CPU103.PMI:Performance_monitoring_interrupts 1452 ± 9% -79.3% 301.25 ± 16% interrupts.CPU103.RES:Rescheduling_interrupts 22.00 ± 69% +2867.0% 652.75 ± 17% interrupts.CPU103.TLB:TLB_shootdowns 1373 ± 17% +954.6% 14487 ± 8% interrupts.CPU104.CAL:Function_call_interrupts 176.00 ± 25% +2905.3% 5289 ± 23% interrupts.CPU104.NMI:Non-maskable_interrupts 176.00 ± 25% +2905.3% 5289 ± 23% interrupts.CPU104.PMI:Performance_monitoring_interrupts 1366 ± 24% -80.1% 271.75 ± 18% interrupts.CPU104.RES:Rescheduling_interrupts 9.25 ± 15% +6373.0% 598.75 ± 14% interrupts.CPU104.TLB:TLB_shootdowns 1337 ± 8% +1221.1% 17663 ± 12% interrupts.CPU105.CAL:Function_call_interrupts 205.25 +2922.3% 6203 ± 26% interrupts.CPU105.NMI:Non-maskable_interrupts 205.25 +2922.3% 6203 ± 26% interrupts.CPU105.PMI:Performance_monitoring_interrupts 1466 ± 13% -77.0% 337.50 ± 11% interrupts.CPU105.RES:Rescheduling_interrupts 29.50 ± 66% +2657.6% 813.50 ± 9% interrupts.CPU105.TLB:TLB_shootdowns 1314 ± 10% +937.2% 13633 ± 17% interrupts.CPU106.CAL:Function_call_interrupts 179.00 ± 24% +2685.2% 4985 ± 31% interrupts.CPU106.NMI:Non-maskable_interrupts 179.00 ± 24% +2685.2% 4985 ± 31% interrupts.CPU106.PMI:Performance_monitoring_interrupts 1635 ± 9% -84.8% 248.00 ± 4% interrupts.CPU106.RES:Rescheduling_interrupts 11.25 ± 21% +5428.9% 622.00 ± 18% interrupts.CPU106.TLB:TLB_shootdowns 1110 ± 6% +1134.1% 13701 ± 2% interrupts.CPU107.CAL:Function_call_interrupts 174.75 ± 24% +2917.6% 5273 ± 28% interrupts.CPU107.NMI:Non-maskable_interrupts 174.75 ± 24% +2917.6% 5273 ± 28% interrupts.CPU107.PMI:Performance_monitoring_interrupts 1395 ± 27% -82.0% 250.75 ± 16% interrupts.CPU107.RES:Rescheduling_interrupts 8.75 ± 35% +7760.0% 687.75 ± 19% interrupts.CPU107.TLB:TLB_shootdowns 1461 ± 9% +997.1% 16036 ± 18% interrupts.CPU108.CAL:Function_call_interrupts 177.25 ± 25% +3711.1% 6755 ± 15% interrupts.CPU108.NMI:Non-maskable_interrupts 177.25 ± 25% +3711.1% 6755 ± 15% interrupts.CPU108.PMI:Performance_monitoring_interrupts 1525 ± 9% -81.6% 280.25 ± 19% interrupts.CPU108.RES:Rescheduling_interrupts 22.00 ± 60% +2913.6% 663.00 ± 14% interrupts.CPU108.TLB:TLB_shootdowns 1167 ± 10% +1162.5% 14736 ± 14% interrupts.CPU109.CAL:Function_call_interrupts 151.50 ± 31% +3887.3% 6040 ± 33% interrupts.CPU109.NMI:Non-maskable_interrupts 151.50 ± 31% +3887.3% 6040 ± 33% interrupts.CPU109.PMI:Performance_monitoring_interrupts 1331 ± 28% -76.7% 310.50 ± 14% interrupts.CPU109.RES:Rescheduling_interrupts 8.00 ± 39% +8687.5% 703.00 ± 14% interrupts.CPU109.TLB:TLB_shootdowns 1125 ± 13% +1642.3% 19605 ± 14% interrupts.CPU11.CAL:Function_call_interrupts 176.25 ± 23% +2172.6% 4005 ± 44% interrupts.CPU11.NMI:Non-maskable_interrupts 176.25 ± 23% +2172.6% 4005 ± 44% interrupts.CPU11.PMI:Performance_monitoring_interrupts 1199 ± 31% -67.0% 395.50 ± 13% interrupts.CPU11.RES:Rescheduling_interrupts 18.00 ± 74% +4216.7% 777.00 ± 15% interrupts.CPU11.TLB:TLB_shootdowns 1108 ± 10% +1073.9% 13009 ± 5% interrupts.CPU110.CAL:Function_call_interrupts 149.50 ± 31% +2692.1% 4174 ± 41% interrupts.CPU110.NMI:Non-maskable_interrupts 149.50 ± 31% +2692.1% 4174 ± 41% interrupts.CPU110.PMI:Performance_monitoring_interrupts 1303 ± 9% -80.7% 251.50 ± 12% interrupts.CPU110.RES:Rescheduling_interrupts 18.25 ± 79% +3406.8% 640.00 ± 17% interrupts.CPU110.TLB:TLB_shootdowns 1189 ± 21% +1172.3% 15130 ± 19% interrupts.CPU111.CAL:Function_call_interrupts 147.75 ± 33% +2802.4% 4288 ± 42% interrupts.CPU111.NMI:Non-maskable_interrupts 147.75 ± 33% +2802.4% 4288 ± 42% interrupts.CPU111.PMI:Performance_monitoring_interrupts 1547 ± 28% -82.1% 277.25 ± 21% interrupts.CPU111.RES:Rescheduling_interrupts 28.50 ± 99% +2150.9% 641.50 ± 29% interrupts.CPU111.TLB:TLB_shootdowns 1135 ± 12% +1154.3% 14236 ± 8% interrupts.CPU112.CAL:Function_call_interrupts 148.75 ± 32% +2586.7% 3996 ± 48% interrupts.CPU112.NMI:Non-maskable_interrupts 148.75 ± 32% +2586.7% 3996 ± 48% interrupts.CPU112.PMI:Performance_monitoring_interrupts 1316 ± 12% -79.5% 270.00 ± 13% interrupts.CPU112.RES:Rescheduling_interrupts 7.50 ± 46% +7680.0% 583.50 ± 12% interrupts.CPU112.TLB:TLB_shootdowns 1193 ± 16% +1226.6% 15833 ± 21% interrupts.CPU113.CAL:Function_call_interrupts 149.75 ± 32% +4233.9% 6490 ± 10% interrupts.CPU113.NMI:Non-maskable_interrupts 149.75 ± 32% +4233.9% 6490 ± 10% interrupts.CPU113.PMI:Performance_monitoring_interrupts 1572 ± 33% -81.0% 298.00 ± 11% interrupts.CPU113.RES:Rescheduling_interrupts 9.75 ± 61% +7625.6% 753.25 ± 10% interrupts.CPU113.TLB:TLB_shootdowns 1397 ± 15% +992.7% 15273 ± 9% interrupts.CPU114.CAL:Function_call_interrupts 148.00 ± 32% +3691.9% 5612 ± 39% interrupts.CPU114.NMI:Non-maskable_interrupts 148.00 ± 32% +3691.9% 5612 ± 39% interrupts.CPU114.PMI:Performance_monitoring_interrupts 1313 ± 13% -79.0% 276.00 ± 8% interrupts.CPU114.RES:Rescheduling_interrupts 28.75 ±101% +2879.1% 856.50 ± 18% interrupts.CPU114.TLB:TLB_shootdowns 1152 ± 11% +1054.8% 13309 ± 5% interrupts.CPU115.CAL:Function_call_interrupts 176.75 ± 25% +2775.8% 5083 ± 26% interrupts.CPU115.NMI:Non-maskable_interrupts 176.75 ± 25% +2775.8% 5083 ± 26% interrupts.CPU115.PMI:Performance_monitoring_interrupts 1278 ± 22% -78.1% 280.00 ± 12% interrupts.CPU115.RES:Rescheduling_interrupts 8.75 ± 31% +7614.3% 675.00 ± 11% interrupts.CPU115.TLB:TLB_shootdowns 1427 ± 24% +934.8% 14774 ± 15% interrupts.CPU116.CAL:Function_call_interrupts 203.00 ± 3% +2693.3% 5670 ± 25% interrupts.CPU116.NMI:Non-maskable_interrupts 203.00 ± 3% +2693.3% 5670 ± 25% interrupts.CPU116.PMI:Performance_monitoring_interrupts 1451 ± 35% -77.1% 331.75 ± 13% interrupts.CPU116.RES:Rescheduling_interrupts 11.50 ± 29% +4582.6% 538.50 ± 14% interrupts.CPU116.TLB:TLB_shootdowns 1225 ± 16% +1001.5% 13499 ± 21% interrupts.CPU117.CAL:Function_call_interrupts 176.50 ± 25% +2546.0% 4670 ± 23% interrupts.CPU117.NMI:Non-maskable_interrupts 176.50 ± 25% +2546.0% 4670 ± 23% interrupts.CPU117.PMI:Performance_monitoring_interrupts 1317 ± 28% -78.7% 281.25 ± 30% interrupts.CPU117.RES:Rescheduling_interrupts 8.25 ± 71% +7412.1% 619.75 ± 30% interrupts.CPU117.TLB:TLB_shootdowns 1216 ± 12% +1026.3% 13704 ± 6% interrupts.CPU118.CAL:Function_call_interrupts 204.75 ± 4% +2645.8% 5622 ± 29% interrupts.CPU118.NMI:Non-maskable_interrupts 204.75 ± 4% +2645.8% 5622 ± 29% interrupts.CPU118.PMI:Performance_monitoring_interrupts 1314 ± 20% -78.5% 283.00 ± 9% interrupts.CPU118.RES:Rescheduling_interrupts 19.25 ± 81% +3127.3% 621.25 ± 17% interrupts.CPU118.TLB:TLB_shootdowns 1140 ± 8% +1211.5% 14961 ± 4% interrupts.CPU119.CAL:Function_call_interrupts 202.75 ± 4% +2414.2% 5097 ± 29% interrupts.CPU119.NMI:Non-maskable_interrupts 202.75 ± 4% +2414.2% 5097 ± 29% interrupts.CPU119.PMI:Performance_monitoring_interrupts 1249 ± 11% -76.3% 296.00 ± 18% interrupts.CPU119.RES:Rescheduling_interrupts 10.00 ± 27% +6227.5% 632.75 ± 11% interrupts.CPU119.TLB:TLB_shootdowns 1262 ± 10% +1468.3% 19799 ± 8% interrupts.CPU12.CAL:Function_call_interrupts 207.25 ± 2% +1845.1% 4031 ± 23% interrupts.CPU12.NMI:Non-maskable_interrupts 207.25 ± 2% +1845.1% 4031 ± 23% interrupts.CPU12.PMI:Performance_monitoring_interrupts 810.50 ± 27% -52.0% 389.00 ± 10% interrupts.CPU12.RES:Rescheduling_interrupts 15.75 ± 90% +5019.0% 806.25 ± 8% interrupts.CPU12.TLB:TLB_shootdowns 1266 ± 4% +1199.9% 16462 ± 7% interrupts.CPU120.CAL:Function_call_interrupts 203.75 ± 3% +2367.0% 5026 ± 32% interrupts.CPU120.NMI:Non-maskable_interrupts 203.75 ± 3% +2367.0% 5026 ± 32% interrupts.CPU120.PMI:Performance_monitoring_interrupts 1049 ± 15% -66.2% 354.75 ± 17% interrupts.CPU120.RES:Rescheduling_interrupts 18.75 ± 84% +3682.7% 709.25 ± 12% interrupts.CPU120.TLB:TLB_shootdowns 1206 ± 15% +1076.3% 14194 ± 7% interrupts.CPU121.CAL:Function_call_interrupts 199.50 ± 4% +2815.2% 5815 ± 17% interrupts.CPU121.NMI:Non-maskable_interrupts 199.50 ± 4% +2815.2% 5815 ± 17% interrupts.CPU121.PMI:Performance_monitoring_interrupts 1507 ± 28% -81.1% 284.50 ± 18% interrupts.CPU121.RES:Rescheduling_interrupts 11.00 ± 22% +5811.4% 650.25 ± 20% interrupts.CPU121.TLB:TLB_shootdowns 1016 ± 5% +1261.5% 13839 ± 7% interrupts.CPU122.CAL:Function_call_interrupts 198.75 ± 4% +3276.7% 6711 ± 4% interrupts.CPU122.NMI:Non-maskable_interrupts 198.75 ± 4% +3276.7% 6711 ± 4% interrupts.CPU122.PMI:Performance_monitoring_interrupts 1169 ± 30% -73.5% 310.00 ± 2% interrupts.CPU122.RES:Rescheduling_interrupts 12.50 ± 32% +5146.0% 655.75 ± 20% interrupts.CPU122.TLB:TLB_shootdowns 1141 ± 5% +1093.9% 13625 ± 10% interrupts.CPU123.CAL:Function_call_interrupts 178.75 ± 27% +3893.4% 7138 ± 11% interrupts.CPU123.NMI:Non-maskable_interrupts 178.75 ± 27% +3893.4% 7138 ± 11% interrupts.CPU123.PMI:Performance_monitoring_interrupts 1197 ± 52% -73.1% 321.50 ± 16% interrupts.CPU123.RES:Rescheduling_interrupts 8.75 ± 75% +7505.7% 665.50 ± 24% interrupts.CPU123.TLB:TLB_shootdowns 1079 ± 8% +1259.3% 14673 ± 12% interrupts.CPU124.CAL:Function_call_interrupts 198.50 ± 4% +2452.9% 5067 ± 28% interrupts.CPU124.NMI:Non-maskable_interrupts 198.50 ± 4% +2452.9% 5067 ± 28% interrupts.CPU124.PMI:Performance_monitoring_interrupts 1326 ± 10% -78.6% 284.25 ± 27% interrupts.CPU124.RES:Rescheduling_interrupts 11.25 ± 14% +6237.8% 713.00 ± 14% interrupts.CPU124.TLB:TLB_shootdowns 1146 ± 8% +1212.8% 15051 ± 7% interrupts.CPU125.CAL:Function_call_interrupts 200.50 ± 3% +2483.8% 5180 ± 28% interrupts.CPU125.NMI:Non-maskable_interrupts 200.50 ± 3% +2483.8% 5180 ± 28% interrupts.CPU125.PMI:Performance_monitoring_interrupts 1121 ± 12% -75.7% 272.25 ± 11% interrupts.CPU125.RES:Rescheduling_interrupts 8.00 ± 46% +8106.2% 656.50 ± 6% interrupts.CPU125.TLB:TLB_shootdowns 1054 ± 6% +1175.3% 13451 ± 12% interrupts.CPU126.CAL:Function_call_interrupts 196.00 ± 3% +2189.5% 4487 ± 37% interrupts.CPU126.NMI:Non-maskable_interrupts 196.00 ± 3% +2189.5% 4487 ± 37% interrupts.CPU126.PMI:Performance_monitoring_interrupts 1300 ± 6% -77.5% 293.25 ± 13% interrupts.CPU126.RES:Rescheduling_interrupts 10.50 ± 44% +6485.7% 691.50 ± 8% interrupts.CPU126.TLB:TLB_shootdowns 1201 ± 16% +1013.0% 13375 ± 7% interrupts.CPU127.CAL:Function_call_interrupts 175.50 ± 25% +3272.9% 5919 ± 33% interrupts.CPU127.NMI:Non-maskable_interrupts 175.50 ± 25% +3272.9% 5919 ± 33% interrupts.CPU127.PMI:Performance_monitoring_interrupts 1281 ± 10% -75.4% 315.75 ± 19% interrupts.CPU127.RES:Rescheduling_interrupts 33.75 ± 81% +1794.8% 639.50 ± 3% interrupts.CPU127.TLB:TLB_shootdowns 1316 ± 10% +941.9% 13716 ± 22% interrupts.CPU128.CAL:Function_call_interrupts 171.50 ± 23% +2881.6% 5113 ± 31% interrupts.CPU128.NMI:Non-maskable_interrupts 171.50 ± 23% +2881.6% 5113 ± 31% interrupts.CPU128.PMI:Performance_monitoring_interrupts 1202 ± 9% -73.0% 324.25 ± 13% interrupts.CPU128.RES:Rescheduling_interrupts 14.50 ± 28% +4265.5% 633.00 ± 22% interrupts.CPU128.TLB:TLB_shootdowns 1198 ± 10% +1072.9% 14057 ± 19% interrupts.CPU129.CAL:Function_call_interrupts 173.00 ± 25% +2972.8% 5316 ± 41% interrupts.CPU129.NMI:Non-maskable_interrupts 173.00 ± 25% +2972.8% 5316 ± 41% interrupts.CPU129.PMI:Performance_monitoring_interrupts 1219 ± 38% -74.2% 314.75 ± 22% interrupts.CPU129.RES:Rescheduling_interrupts 15.25 ± 43% +4042.6% 631.75 ± 15% interrupts.CPU129.TLB:TLB_shootdowns 1569 ± 50% +997.0% 17214 ± 9% interrupts.CPU13.CAL:Function_call_interrupts 204.75 ± 5% +2106.8% 4518 ± 53% interrupts.CPU13.NMI:Non-maskable_interrupts 204.75 ± 5% +2106.8% 4518 ± 53% interrupts.CPU13.PMI:Performance_monitoring_interrupts 1204 ± 29% -71.1% 348.50 ± 27% interrupts.CPU13.RES:Rescheduling_interrupts 16.75 ± 87% +4329.9% 742.00 ± 14% interrupts.CPU13.TLB:TLB_shootdowns 1000 ± 4% +1382.7% 14830 ± 12% interrupts.CPU130.CAL:Function_call_interrupts 172.00 ± 24% +3391.7% 6005 ± 23% interrupts.CPU130.NMI:Non-maskable_interrupts 172.00 ± 24% +3391.7% 6005 ± 23% interrupts.CPU130.PMI:Performance_monitoring_interrupts 1174 ± 14% -73.3% 313.25 ± 19% interrupts.CPU130.RES:Rescheduling_interrupts 10.25 ± 32% +6546.3% 681.25 ± 4% interrupts.CPU130.TLB:TLB_shootdowns 1156 ± 9% +1093.9% 13804 ± 14% interrupts.CPU131.CAL:Function_call_interrupts 176.00 ± 28% +2972.2% 5407 ± 28% interrupts.CPU131.NMI:Non-maskable_interrupts 176.00 ± 28% +2972.2% 5407 ± 28% interrupts.CPU131.PMI:Performance_monitoring_interrupts 1370 ± 22% -76.7% 319.25 ± 14% interrupts.CPU131.RES:Rescheduling_interrupts 30.00 ± 52% +1986.7% 626.00 ± 19% interrupts.CPU131.TLB:TLB_shootdowns 1151 ± 9% +1135.4% 14222 ± 13% interrupts.CPU132.CAL:Function_call_interrupts 168.00 ± 23% +2673.5% 4659 ± 29% interrupts.CPU132.NMI:Non-maskable_interrupts 168.00 ± 23% +2673.5% 4659 ± 29% interrupts.CPU132.PMI:Performance_monitoring_interrupts 1643 ± 13% -79.8% 332.75 ± 16% interrupts.CPU132.RES:Rescheduling_interrupts 14.50 ± 36% +5208.6% 769.75 ± 11% interrupts.CPU132.TLB:TLB_shootdowns 1150 ± 9% +1148.8% 14370 ± 9% interrupts.CPU133.CAL:Function_call_interrupts 144.25 ± 34% +3515.1% 5214 ± 25% interrupts.CPU133.NMI:Non-maskable_interrupts 144.25 ± 34% +3515.1% 5214 ± 25% interrupts.CPU133.PMI:Performance_monitoring_interrupts 1200 ± 27% -74.7% 303.50 ± 14% interrupts.CPU133.RES:Rescheduling_interrupts 27.75 ± 77% +2288.3% 662.75 ± 13% interrupts.CPU133.TLB:TLB_shootdowns 1041 ± 11% +1120.7% 12707 ± 6% interrupts.CPU134.CAL:Function_call_interrupts 172.75 ± 27% +2867.6% 5126 ± 25% interrupts.CPU134.NMI:Non-maskable_interrupts 172.75 ± 27% +2867.6% 5126 ± 25% interrupts.CPU134.PMI:Performance_monitoring_interrupts 1147 ± 32% -70.3% 340.50 ± 22% interrupts.CPU134.RES:Rescheduling_interrupts 8.50 ± 37% +8305.9% 714.50 ± 23% interrupts.CPU134.TLB:TLB_shootdowns 1169 ± 8% +1200.7% 15208 ± 16% interrupts.CPU135.CAL:Function_call_interrupts 171.50 ± 27% +2944.2% 5220 ± 35% interrupts.CPU135.NMI:Non-maskable_interrupts 171.50 ± 27% +2944.2% 5220 ± 35% interrupts.CPU135.PMI:Performance_monitoring_interrupts 1110 ± 16% -68.3% 351.50 ± 21% interrupts.CPU135.RES:Rescheduling_interrupts 12.00 ± 33% +5641.7% 689.00 ± 21% interrupts.CPU135.TLB:TLB_shootdowns 1118 ± 7% +1085.7% 13265 ± 12% interrupts.CPU136.CAL:Function_call_interrupts 174.25 ± 27% +2698.6% 4876 ± 24% interrupts.CPU136.NMI:Non-maskable_interrupts 174.25 ± 27% +2698.6% 4876 ± 24% interrupts.CPU136.PMI:Performance_monitoring_interrupts 1392 ± 7% -79.5% 285.25 ± 12% interrupts.CPU136.RES:Rescheduling_interrupts 58.25 ±108% +987.1% 633.25 ± 17% interrupts.CPU136.TLB:TLB_shootdowns 1250 ± 29% +980.4% 13513 ± 14% interrupts.CPU137.CAL:Function_call_interrupts 199.25 ± 3% +2614.4% 5408 ± 24% interrupts.CPU137.NMI:Non-maskable_interrupts 199.25 ± 3% +2614.4% 5408 ± 24% interrupts.CPU137.PMI:Performance_monitoring_interrupts 1281 ± 10% -77.5% 288.75 ± 31% interrupts.CPU137.RES:Rescheduling_interrupts 21.75 ± 76% +2344.8% 531.75 ± 15% interrupts.CPU137.TLB:TLB_shootdowns 1161 ± 9% +1142.5% 14431 ± 10% interrupts.CPU138.CAL:Function_call_interrupts 204.75 ± 11% +2804.8% 5947 ± 19% interrupts.CPU138.NMI:Non-maskable_interrupts 204.75 ± 11% +2804.8% 5947 ± 19% interrupts.CPU138.PMI:Performance_monitoring_interrupts 1324 ± 19% -75.6% 322.75 ± 23% interrupts.CPU138.RES:Rescheduling_interrupts 12.25 ± 21% +5142.9% 642.25 ± 11% interrupts.CPU138.TLB:TLB_shootdowns 1089 ± 11% +1073.7% 12781 ± 9% interrupts.CPU139.CAL:Function_call_interrupts 174.00 ± 24% +3120.3% 5603 ± 31% interrupts.CPU139.NMI:Non-maskable_interrupts 174.00 ± 24% +3120.3% 5603 ± 31% interrupts.CPU139.PMI:Performance_monitoring_interrupts 1434 ± 19% -82.0% 258.00 ± 8% interrupts.CPU139.RES:Rescheduling_interrupts 30.75 ±119% +2056.9% 663.25 ± 14% interrupts.CPU139.TLB:TLB_shootdowns 1220 ± 14% +1438.2% 18769 ± 8% interrupts.CPU14.CAL:Function_call_interrupts 206.75 ± 4% +2813.7% 6024 ± 25% interrupts.CPU14.NMI:Non-maskable_interrupts 206.75 ± 4% +2813.7% 6024 ± 25% interrupts.CPU14.PMI:Performance_monitoring_interrupts 1255 ± 29% -67.5% 408.00 ± 23% interrupts.CPU14.RES:Rescheduling_interrupts 11.25 ± 22% +7417.8% 845.75 ± 11% interrupts.CPU14.TLB:TLB_shootdowns 1172 ± 9% +1141.5% 14556 ± 10% interrupts.CPU140.CAL:Function_call_interrupts 172.00 ± 23% +2840.4% 5057 ± 26% interrupts.CPU140.NMI:Non-maskable_interrupts 172.00 ± 23% +2840.4% 5057 ± 26% interrupts.CPU140.PMI:Performance_monitoring_interrupts 1538 ± 33% -77.5% 346.00 ± 19% interrupts.CPU140.RES:Rescheduling_interrupts 13.00 ± 56% +4734.6% 628.50 ± 25% interrupts.CPU140.TLB:TLB_shootdowns 1131 ± 11% +1224.3% 14981 ± 4% interrupts.CPU141.CAL:Function_call_interrupts 174.00 ± 25% +3593.5% 6426 ± 16% interrupts.CPU141.NMI:Non-maskable_interrupts 174.00 ± 25% +3593.5% 6426 ± 16% interrupts.CPU141.PMI:Performance_monitoring_interrupts 1099 ± 4% -73.7% 289.00 ± 12% interrupts.CPU141.RES:Rescheduling_interrupts 10.25 ± 17% +6936.6% 721.25 ± 21% interrupts.CPU141.TLB:TLB_shootdowns 1101 ± 13% +1300.5% 15422 ± 16% interrupts.CPU142.CAL:Function_call_interrupts 171.25 ± 23% +2942.3% 5210 ± 19% interrupts.CPU142.NMI:Non-maskable_interrupts 171.25 ± 23% +2942.3% 5210 ± 19% interrupts.CPU142.PMI:Performance_monitoring_interrupts 1238 ± 3% -75.2% 307.25 ± 19% interrupts.CPU142.RES:Rescheduling_interrupts 13.75 ± 17% +5021.8% 704.25 ± 21% interrupts.CPU142.TLB:TLB_shootdowns 1014 ± 12% +1164.9% 12826 ± 14% interrupts.CPU143.CAL:Function_call_interrupts 171.50 ± 24% +3326.4% 5876 ± 10% interrupts.CPU143.NMI:Non-maskable_interrupts 171.50 ± 24% +3326.4% 5876 ± 10% interrupts.CPU143.PMI:Performance_monitoring_interrupts 1299 ± 32% -75.4% 319.00 ± 21% interrupts.CPU143.RES:Rescheduling_interrupts 10.00 ± 37% +7027.5% 712.75 ± 15% interrupts.CPU143.TLB:TLB_shootdowns 1149 ± 4% +1225.5% 15239 ± 11% interrupts.CPU144.CAL:Function_call_interrupts 199.00 ± 31% +2607.5% 5388 ± 38% interrupts.CPU144.NMI:Non-maskable_interrupts 199.00 ± 31% +2607.5% 5388 ± 38% interrupts.CPU144.PMI:Performance_monitoring_interrupts 901.00 ± 17% -67.5% 292.75 ± 7% interrupts.CPU144.RES:Rescheduling_interrupts 29.75 ± 64% +2062.2% 643.25 ± 18% interrupts.CPU144.TLB:TLB_shootdowns 1240 ± 5% +1016.7% 13849 ± 6% interrupts.CPU145.CAL:Function_call_interrupts 170.25 ± 25% +3281.4% 5756 ± 7% interrupts.CPU145.NMI:Non-maskable_interrupts 170.25 ± 25% +3281.4% 5756 ± 7% interrupts.CPU145.PMI:Performance_monitoring_interrupts 1430 ± 17% -81.6% 263.75 ± 22% interrupts.CPU145.RES:Rescheduling_interrupts 14.75 ± 10% +4771.2% 718.50 ± 16% interrupts.CPU145.TLB:TLB_shootdowns 1360 ± 33% +901.6% 13626 ± 13% interrupts.CPU146.CAL:Function_call_interrupts 196.75 ± 2% +2932.3% 5966 ± 27% interrupts.CPU146.NMI:Non-maskable_interrupts 196.75 ± 2% +2932.3% 5966 ± 27% interrupts.CPU146.PMI:Performance_monitoring_interrupts 1173 ± 14% -77.6% 263.00 ± 21% interrupts.CPU146.RES:Rescheduling_interrupts 19.00 ± 82% +2542.1% 502.00 ± 35% interrupts.CPU146.TLB:TLB_shootdowns 1161 ± 27% +1038.3% 13224 ± 21% interrupts.CPU147.CAL:Function_call_interrupts 197.25 ± 2% +2361.7% 4855 ± 39% interrupts.CPU147.NMI:Non-maskable_interrupts 197.25 ± 2% +2361.7% 4855 ± 39% interrupts.CPU147.PMI:Performance_monitoring_interrupts 1253 ± 27% -76.8% 291.25 ± 23% interrupts.CPU147.RES:Rescheduling_interrupts 12.25 ± 63% +4467.3% 559.50 ± 29% interrupts.CPU147.TLB:TLB_shootdowns 1127 ± 13% +1099.5% 13527 ± 8% interrupts.CPU148.CAL:Function_call_interrupts 200.75 ± 3% +2706.2% 5633 ± 37% interrupts.CPU148.NMI:Non-maskable_interrupts 200.75 ± 3% +2706.2% 5633 ± 37% interrupts.CPU148.PMI:Performance_monitoring_interrupts 1375 ± 15% -79.6% 280.25 ± 12% interrupts.CPU148.RES:Rescheduling_interrupts 16.50 ± 29% +3775.8% 639.50 ± 10% interrupts.CPU148.TLB:TLB_shootdowns 1132 ± 13% +1194.9% 14664 ± 19% interrupts.CPU149.CAL:Function_call_interrupts 197.50 ± 2% +2371.9% 4882 ± 33% interrupts.CPU149.NMI:Non-maskable_interrupts 197.50 ± 2% +2371.9% 4882 ± 33% interrupts.CPU149.PMI:Performance_monitoring_interrupts 1546 ± 22% -81.7% 283.50 ± 12% interrupts.CPU149.RES:Rescheduling_interrupts 34.75 ± 42% +1830.2% 670.75 ± 17% interrupts.CPU149.TLB:TLB_shootdowns 1273 ± 32% +1440.2% 19606 ± 13% interrupts.CPU15.CAL:Function_call_interrupts 201.00 ± 2% +2902.0% 6034 ± 26% interrupts.CPU15.NMI:Non-maskable_interrupts 201.00 ± 2% +2902.0% 6034 ± 26% interrupts.CPU15.PMI:Performance_monitoring_interrupts 7.25 ± 54% +11358.6% 830.75 ± 21% interrupts.CPU15.TLB:TLB_shootdowns 1135 ± 14% +1192.5% 14676 ± 21% interrupts.CPU150.CAL:Function_call_interrupts 172.25 ± 25% +2381.3% 4274 ± 34% interrupts.CPU150.NMI:Non-maskable_interrupts 172.25 ± 25% +2381.3% 4274 ± 34% interrupts.CPU150.PMI:Performance_monitoring_interrupts 1234 ± 23% -76.3% 292.00 ± 14% interrupts.CPU150.RES:Rescheduling_interrupts 12.25 ± 38% +5240.8% 654.25 ± 23% interrupts.CPU150.TLB:TLB_shootdowns 1328 ± 22% +941.4% 13832 ± 15% interrupts.CPU151.CAL:Function_call_interrupts 171.25 ± 25% +2801.0% 4968 ± 33% interrupts.CPU151.NMI:Non-maskable_interrupts 171.25 ± 25% +2801.0% 4968 ± 33% interrupts.CPU151.PMI:Performance_monitoring_interrupts 1492 ± 11% -82.1% 266.75 ± 15% interrupts.CPU151.RES:Rescheduling_interrupts 18.25 ± 35% +2826.0% 534.00 ± 30% interrupts.CPU151.TLB:TLB_shootdowns 1203 ± 6% +1112.1% 14588 ± 18% interrupts.CPU152.CAL:Function_call_interrupts 171.50 ± 25% +2989.7% 5298 ± 26% interrupts.CPU152.NMI:Non-maskable_interrupts 171.50 ± 25% +2989.7% 5298 ± 26% interrupts.CPU152.PMI:Performance_monitoring_interrupts 1402 ± 26% -79.6% 285.50 ± 12% interrupts.CPU152.RES:Rescheduling_interrupts 12.50 ± 47% +4642.0% 592.75 ± 15% interrupts.CPU152.TLB:TLB_shootdowns 1075 ± 5% +1003.9% 11870 ± 26% interrupts.CPU153.CAL:Function_call_interrupts 178.50 ± 27% +3121.7% 5750 ± 17% interrupts.CPU153.NMI:Non-maskable_interrupts 178.50 ± 27% +3121.7% 5750 ± 17% interrupts.CPU153.PMI:Performance_monitoring_interrupts 1309 ± 26% -79.7% 265.25 ± 24% interrupts.CPU153.RES:Rescheduling_interrupts 33.00 ±119% +1569.7% 551.00 ± 32% interrupts.CPU153.TLB:TLB_shootdowns 1275 ± 18% +953.4% 13433 ± 13% interrupts.CPU154.CAL:Function_call_interrupts 174.25 ± 25% +3646.2% 6527 ± 22% interrupts.CPU154.NMI:Non-maskable_interrupts 174.25 ± 25% +3646.2% 6527 ± 22% interrupts.CPU154.PMI:Performance_monitoring_interrupts 1363 ± 24% -79.2% 283.75 ± 12% interrupts.CPU154.RES:Rescheduling_interrupts 19.75 ± 27% +2900.0% 592.50 ± 15% interrupts.CPU154.TLB:TLB_shootdowns 1179 ± 10% +1137.7% 14592 ± 20% interrupts.CPU155.CAL:Function_call_interrupts 172.50 ± 25% +4136.8% 7308 ± 15% interrupts.CPU155.NMI:Non-maskable_interrupts 172.50 ± 25% +4136.8% 7308 ± 15% interrupts.CPU155.PMI:Performance_monitoring_interrupts 1368 ± 17% -78.3% 297.25 ± 18% interrupts.CPU155.RES:Rescheduling_interrupts 17.00 ± 39% +3847.1% 671.00 ± 8% interrupts.CPU155.TLB:TLB_shootdowns 1130 ± 15% +1116.7% 13754 ± 8% interrupts.CPU156.CAL:Function_call_interrupts 171.00 ± 23% +3322.1% 5851 ± 16% interrupts.CPU156.NMI:Non-maskable_interrupts 171.00 ± 23% +3322.1% 5851 ± 16% interrupts.CPU156.PMI:Performance_monitoring_interrupts 1229 ± 24% -78.5% 264.25 ± 18% interrupts.CPU156.RES:Rescheduling_interrupts 19.00 ± 69% +3373.7% 660.00 ± 10% interrupts.CPU156.TLB:TLB_shootdowns 1025 ± 7% +1280.2% 14154 ± 26% interrupts.CPU157.CAL:Function_call_interrupts 170.50 ± 25% +3323.3% 5836 ± 12% interrupts.CPU157.NMI:Non-maskable_interrupts 170.50 ± 25% +3323.3% 5836 ± 12% interrupts.CPU157.PMI:Performance_monitoring_interrupts 1303 ± 12% -79.6% 265.50 ± 27% interrupts.CPU157.RES:Rescheduling_interrupts 20.50 ±100% +2854.9% 605.75 ± 5% interrupts.CPU157.TLB:TLB_shootdowns 1248 ± 13% +1136.5% 15432 ± 6% interrupts.CPU158.CAL:Function_call_interrupts 146.25 ± 35% +3346.5% 5040 ± 26% interrupts.CPU158.NMI:Non-maskable_interrupts 146.25 ± 35% +3346.5% 5040 ± 26% interrupts.CPU158.PMI:Performance_monitoring_interrupts 1532 ± 23% -78.8% 324.75 ± 18% interrupts.CPU158.RES:Rescheduling_interrupts 15.00 ± 37% +4298.3% 659.75 ± 34% interrupts.CPU158.TLB:TLB_shootdowns 1242 ± 7% +942.0% 12942 ± 12% interrupts.CPU159.CAL:Function_call_interrupts 148.50 ± 36% +3184.8% 4878 ± 21% interrupts.CPU159.NMI:Non-maskable_interrupts 148.50 ± 36% +3184.8% 4878 ± 21% interrupts.CPU159.PMI:Performance_monitoring_interrupts 1268 ± 21% -79.8% 256.75 ± 20% interrupts.CPU159.RES:Rescheduling_interrupts 32.50 ± 90% +1569.2% 542.50 ± 16% interrupts.CPU159.TLB:TLB_shootdowns 1106 ± 8% +1676.8% 19659 ± 12% interrupts.CPU16.CAL:Function_call_interrupts 205.75 ± 3% +2565.1% 5483 ± 36% interrupts.CPU16.NMI:Non-maskable_interrupts 205.75 ± 3% +2565.1% 5483 ± 36% interrupts.CPU16.PMI:Performance_monitoring_interrupts 1281 ± 30% -67.6% 414.50 ± 13% interrupts.CPU16.RES:Rescheduling_interrupts 21.50 ± 55% +4008.1% 883.25 ± 6% interrupts.CPU16.TLB:TLB_shootdowns 1168 ± 12% +1036.9% 13284 ± 14% interrupts.CPU160.CAL:Function_call_interrupts 144.25 ± 34% +4039.7% 5971 ± 30% interrupts.CPU160.NMI:Non-maskable_interrupts 144.25 ± 34% +4039.7% 5971 ± 30% interrupts.CPU160.PMI:Performance_monitoring_interrupts 1048 ± 27% -76.3% 249.00 ± 16% interrupts.CPU160.RES:Rescheduling_interrupts 11.00 ± 38% +6231.8% 696.50 ± 11% interrupts.CPU160.TLB:TLB_shootdowns 1106 ± 14% +1200.8% 14396 ± 18% interrupts.CPU161.CAL:Function_call_interrupts 147.25 ± 33% +3577.9% 5415 ± 14% interrupts.CPU161.NMI:Non-maskable_interrupts 147.25 ± 33% +3577.9% 5415 ± 14% interrupts.CPU161.PMI:Performance_monitoring_interrupts 1495 ± 15% -80.4% 293.25 ± 17% interrupts.CPU161.RES:Rescheduling_interrupts 18.25 ± 5% +3780.8% 708.25 ± 8% interrupts.CPU161.TLB:TLB_shootdowns 1217 ± 23% +998.1% 13369 ± 14% interrupts.CPU162.CAL:Function_call_interrupts 153.25 ± 33% +3249.8% 5133 ± 37% interrupts.CPU162.NMI:Non-maskable_interrupts 153.25 ± 33% +3249.8% 5133 ± 37% interrupts.CPU162.PMI:Performance_monitoring_interrupts 1321 ± 29% -81.0% 251.50 ± 14% interrupts.CPU162.RES:Rescheduling_interrupts 30.50 ± 62% +1946.7% 624.25 ± 8% interrupts.CPU162.TLB:TLB_shootdowns 1071 ± 4% +1210.4% 14044 ± 12% interrupts.CPU163.CAL:Function_call_interrupts 146.50 ± 34% +3657.0% 5504 ± 31% interrupts.CPU163.NMI:Non-maskable_interrupts 146.50 ± 34% +3657.0% 5504 ± 31% interrupts.CPU163.PMI:Performance_monitoring_interrupts 1582 ± 14% -82.1% 283.00 ± 11% interrupts.CPU163.RES:Rescheduling_interrupts 11.75 ± 21% +5280.9% 632.25 ± 14% interrupts.CPU163.TLB:TLB_shootdowns 1089 ± 10% +1103.2% 13105 ± 10% interrupts.CPU164.CAL:Function_call_interrupts 147.75 ± 36% +3304.1% 5029 ± 51% interrupts.CPU164.NMI:Non-maskable_interrupts 147.75 ± 36% +3304.1% 5029 ± 51% interrupts.CPU164.PMI:Performance_monitoring_interrupts 1245 ± 28% -79.0% 261.25 ± 13% interrupts.CPU164.RES:Rescheduling_interrupts 24.00 ± 80% +2418.8% 604.50 ± 14% interrupts.CPU164.TLB:TLB_shootdowns 1067 ± 6% +1151.9% 13361 ± 13% interrupts.CPU165.CAL:Function_call_interrupts 147.50 ± 34% +4051.4% 6123 ± 28% interrupts.CPU165.NMI:Non-maskable_interrupts 147.50 ± 34% +4051.4% 6123 ± 28% interrupts.CPU165.PMI:Performance_monitoring_interrupts 1398 ± 30% -79.5% 286.25 ± 18% interrupts.CPU165.RES:Rescheduling_interrupts 22.00 ± 73% +2510.2% 574.25 ± 24% interrupts.CPU165.TLB:TLB_shootdowns 1009 ± 8% +1390.8% 15053 ± 11% interrupts.CPU166.CAL:Function_call_interrupts 152.00 ± 36% +3428.5% 5363 ± 26% interrupts.CPU166.NMI:Non-maskable_interrupts 152.00 ± 36% +3428.5% 5363 ± 26% interrupts.CPU166.PMI:Performance_monitoring_interrupts 1299 ± 12% -76.7% 302.75 ± 12% interrupts.CPU166.RES:Rescheduling_interrupts 9.50 ± 39% +6428.9% 620.25 ± 13% interrupts.CPU166.TLB:TLB_shootdowns 1049 ± 12% +1315.0% 14850 ± 8% interrupts.CPU167.CAL:Function_call_interrupts 157.00 ± 40% +3769.9% 6075 ± 41% interrupts.CPU167.NMI:Non-maskable_interrupts 157.00 ± 40% +3769.9% 6075 ± 41% interrupts.CPU167.PMI:Performance_monitoring_interrupts 1629 ± 6% -81.5% 302.00 ± 19% interrupts.CPU167.RES:Rescheduling_interrupts 14.50 ± 25% +4098.3% 608.75 ± 20% interrupts.CPU167.TLB:TLB_shootdowns 1799 ± 40% +813.1% 16428 ± 3% interrupts.CPU168.CAL:Function_call_interrupts 181.00 ± 24% +3204.0% 5980 ± 34% interrupts.CPU168.NMI:Non-maskable_interrupts 181.00 ± 24% +3204.0% 5980 ± 34% interrupts.CPU168.PMI:Performance_monitoring_interrupts 987.50 ± 10% -70.2% 294.50 ± 12% interrupts.CPU168.RES:Rescheduling_interrupts 20.75 ± 81% +3185.5% 681.75 ± 19% interrupts.CPU168.TLB:TLB_shootdowns 1475 ± 21% +927.6% 15162 ± 10% interrupts.CPU169.CAL:Function_call_interrupts 178.25 ± 22% +3114.6% 5730 ± 22% interrupts.CPU169.NMI:Non-maskable_interrupts 178.25 ± 22% +3114.6% 5730 ± 22% interrupts.CPU169.PMI:Performance_monitoring_interrupts 1203 ± 19% -75.2% 298.00 ± 19% interrupts.CPU169.RES:Rescheduling_interrupts 46.00 ± 95% +1379.9% 680.75 ± 19% interrupts.CPU169.TLB:TLB_shootdowns 1073 ± 12% +1504.9% 17232 ± 11% interrupts.CPU17.CAL:Function_call_interrupts 203.75 ± 2% +1949.0% 4174 ± 15% interrupts.CPU17.NMI:Non-maskable_interrupts 203.75 ± 2% +1949.0% 4174 ± 15% interrupts.CPU17.PMI:Performance_monitoring_interrupts 1024 ± 30% -66.9% 339.25 ± 16% interrupts.CPU17.RES:Rescheduling_interrupts 16.75 ±105% +4129.9% 708.50 ± 12% interrupts.CPU17.TLB:TLB_shootdowns 1363 ± 13% +986.6% 14813 ± 10% interrupts.CPU170.CAL:Function_call_interrupts 175.50 ± 26% +3093.4% 5604 ± 28% interrupts.CPU170.NMI:Non-maskable_interrupts 175.50 ± 26% +3093.4% 5604 ± 28% interrupts.CPU170.PMI:Performance_monitoring_interrupts 1143 ± 37% -76.2% 272.75 ± 21% interrupts.CPU170.RES:Rescheduling_interrupts 23.25 ±111% +2730.1% 658.00 ± 8% interrupts.CPU170.TLB:TLB_shootdowns 1430 ± 9% +994.8% 15658 ± 9% interrupts.CPU171.CAL:Function_call_interrupts 188.50 ± 30% +2864.6% 5588 ± 24% interrupts.CPU171.NMI:Non-maskable_interrupts 188.50 ± 30% +2864.6% 5588 ± 24% interrupts.CPU171.PMI:Performance_monitoring_interrupts 1123 ± 33% -76.3% 266.25 ± 12% interrupts.CPU171.RES:Rescheduling_interrupts 10.25 ± 24% +6714.6% 698.50 ± 12% interrupts.CPU171.TLB:TLB_shootdowns 1565 ± 49% +811.5% 14268 ± 15% interrupts.CPU172.CAL:Function_call_interrupts 208.25 ± 5% +2480.3% 5373 ± 31% interrupts.CPU172.NMI:Non-maskable_interrupts 208.25 ± 5% +2480.3% 5373 ± 31% interrupts.CPU172.PMI:Performance_monitoring_interrupts 35.25 ±128% +1641.1% 613.75 ± 11% interrupts.CPU172.TLB:TLB_shootdowns 1228 ± 15% +1209.1% 16085 ± 12% interrupts.CPU173.CAL:Function_call_interrupts 203.00 ± 4% +3154.1% 6605 ± 13% interrupts.CPU173.NMI:Non-maskable_interrupts 203.00 ± 4% +3154.1% 6605 ± 13% interrupts.CPU173.PMI:Performance_monitoring_interrupts 1248 ± 26% -78.0% 274.50 ± 21% interrupts.CPU173.RES:Rescheduling_interrupts 33.50 ± 62% +1862.7% 657.50 ± 27% interrupts.CPU173.TLB:TLB_shootdowns 1070 ± 8% +1239.8% 14338 ± 11% interrupts.CPU174.CAL:Function_call_interrupts 203.50 ± 4% +2676.9% 5651 ± 18% interrupts.CPU174.NMI:Non-maskable_interrupts 203.50 ± 4% +2676.9% 5651 ± 18% interrupts.CPU174.PMI:Performance_monitoring_interrupts 19.50 ± 70% +3064.1% 617.00 ± 9% interrupts.CPU174.TLB:TLB_shootdowns 1468 ± 19% +1011.8% 16329 ± 3% interrupts.CPU175.CAL:Function_call_interrupts 200.50 ± 3% +2875.7% 5966 ± 16% interrupts.CPU175.NMI:Non-maskable_interrupts 200.50 ± 3% +2875.7% 5966 ± 16% interrupts.CPU175.PMI:Performance_monitoring_interrupts 1106 ± 31% -75.1% 275.25 ± 7% interrupts.CPU175.RES:Rescheduling_interrupts 28.75 ± 60% +2479.1% 741.50 ± 14% interrupts.CPU175.TLB:TLB_shootdowns 1393 ± 8% +876.9% 13611 ± 8% interrupts.CPU176.CAL:Function_call_interrupts 200.00 ± 3% +3190.5% 6581 ± 13% interrupts.CPU176.NMI:Non-maskable_interrupts 200.00 ± 3% +3190.5% 6581 ± 13% interrupts.CPU176.PMI:Performance_monitoring_interrupts 971.50 ± 11% -71.1% 280.75 ± 7% interrupts.CPU176.RES:Rescheduling_interrupts 14.00 ± 49% +5275.0% 752.50 ± 6% interrupts.CPU176.TLB:TLB_shootdowns 1394 ± 12% +803.9% 12600 ± 6% interrupts.CPU177.CAL:Function_call_interrupts 202.25 ± 2% +2877.6% 6022 ± 13% interrupts.CPU177.NMI:Non-maskable_interrupts 202.25 ± 2% +2877.6% 6022 ± 13% interrupts.CPU177.PMI:Performance_monitoring_interrupts 1117 ± 18% -76.0% 268.00 ± 21% interrupts.CPU177.RES:Rescheduling_interrupts 14.00 ± 26% +4117.9% 590.50 ± 14% interrupts.CPU177.TLB:TLB_shootdowns 1430 ± 15% +968.2% 15281 ± 4% interrupts.CPU178.CAL:Function_call_interrupts 207.25 ± 9% +2723.2% 5851 ± 21% interrupts.CPU178.NMI:Non-maskable_interrupts 207.25 ± 9% +2723.2% 5851 ± 21% interrupts.CPU178.PMI:Performance_monitoring_interrupts 916.50 ± 21% -74.1% 237.25 ± 6% interrupts.CPU178.RES:Rescheduling_interrupts 20.50 ± 69% +2967.1% 628.75 ± 12% interrupts.CPU178.TLB:TLB_shootdowns 1350 ± 12% +866.1% 13042 ± 16% interrupts.CPU179.CAL:Function_call_interrupts 203.25 ± 3% +2323.0% 4924 ± 28% interrupts.CPU179.NMI:Non-maskable_interrupts 203.25 ± 3% +2323.0% 4924 ± 28% interrupts.CPU179.PMI:Performance_monitoring_interrupts 1495 ± 14% -83.6% 245.75 ± 12% interrupts.CPU179.RES:Rescheduling_interrupts 23.75 ± 76% +2553.7% 630.25 ± 17% interrupts.CPU179.TLB:TLB_shootdowns 1155 ± 9% +1583.0% 19451 ± 15% interrupts.CPU18.CAL:Function_call_interrupts 202.50 ± 3% +2122.7% 4501 ± 44% interrupts.CPU18.NMI:Non-maskable_interrupts 202.50 ± 3% +2122.7% 4501 ± 44% interrupts.CPU18.PMI:Performance_monitoring_interrupts 1107 ± 20% -67.0% 365.50 ± 24% interrupts.CPU18.RES:Rescheduling_interrupts 5.50 ± 32% +14081.8% 780.00 ± 15% interrupts.CPU18.TLB:TLB_shootdowns 1327 ± 4% +1054.4% 15324 ± 6% interrupts.CPU180.CAL:Function_call_interrupts 179.25 ± 26% +3402.6% 6278 ± 12% interrupts.CPU180.NMI:Non-maskable_interrupts 179.25 ± 26% +3402.6% 6278 ± 12% interrupts.CPU180.PMI:Performance_monitoring_interrupts 1134 ± 19% -78.7% 241.25 ± 21% interrupts.CPU180.RES:Rescheduling_interrupts 19.75 ± 59% +3427.8% 696.75 ± 11% interrupts.CPU180.TLB:TLB_shootdowns 1279 ± 5% +947.7% 13403 ± 8% interrupts.CPU181.CAL:Function_call_interrupts 184.50 ± 25% +3133.9% 5966 ± 15% interrupts.CPU181.NMI:Non-maskable_interrupts 184.50 ± 25% +3133.9% 5966 ± 15% interrupts.CPU181.PMI:Performance_monitoring_interrupts 1399 ± 8% -83.7% 228.75 ± 21% interrupts.CPU181.RES:Rescheduling_interrupts 17.25 ± 49% +3360.9% 597.00 ± 14% interrupts.CPU181.TLB:TLB_shootdowns 1176 ± 11% +971.9% 12610 ± 12% interrupts.CPU182.CAL:Function_call_interrupts 179.75 ± 27% +2885.3% 5366 ± 25% interrupts.CPU182.NMI:Non-maskable_interrupts 179.75 ± 27% +2885.3% 5366 ± 25% interrupts.CPU182.PMI:Performance_monitoring_interrupts 813.25 ± 28% -70.0% 244.25 ± 24% interrupts.CPU182.RES:Rescheduling_interrupts 9.25 ± 55% +7583.8% 710.75 ± 18% interrupts.CPU182.TLB:TLB_shootdowns 1458 ± 24% +788.6% 12958 ± 14% interrupts.CPU183.CAL:Function_call_interrupts 177.00 ± 27% +3059.6% 5592 ± 8% interrupts.CPU183.NMI:Non-maskable_interrupts 177.00 ± 27% +3059.6% 5592 ± 8% interrupts.CPU183.PMI:Performance_monitoring_interrupts 1085 ± 28% -78.8% 229.75 ± 12% interrupts.CPU183.RES:Rescheduling_interrupts 22.00 ± 71% +2365.9% 542.50 ± 23% interrupts.CPU183.TLB:TLB_shootdowns 1143 ± 6% +1166.9% 14480 ± 6% interrupts.CPU184.CAL:Function_call_interrupts 178.00 ± 27% +3791.3% 6926 ± 14% interrupts.CPU184.NMI:Non-maskable_interrupts 178.00 ± 27% +3791.3% 6926 ± 14% interrupts.CPU184.PMI:Performance_monitoring_interrupts 1061 ± 17% -73.8% 278.25 ± 19% interrupts.CPU184.RES:Rescheduling_interrupts 11.00 ± 43% +5834.1% 652.75 ± 26% interrupts.CPU184.TLB:TLB_shootdowns 1274 ± 11% +969.3% 13628 ± 4% interrupts.CPU185.CAL:Function_call_interrupts 181.75 ± 27% +3123.4% 5858 ± 34% interrupts.CPU185.NMI:Non-maskable_interrupts 181.75 ± 27% +3123.4% 5858 ± 34% interrupts.CPU185.PMI:Performance_monitoring_interrupts 1332 ± 19% -82.9% 227.50 ± 5% interrupts.CPU185.RES:Rescheduling_interrupts 15.00 ± 30% +3793.3% 584.00 ± 10% interrupts.CPU185.TLB:TLB_shootdowns 1452 ± 24% +830.7% 13514 ± 13% interrupts.CPU186.CAL:Function_call_interrupts 175.75 ± 26% +2956.8% 5372 ± 35% interrupts.CPU186.NMI:Non-maskable_interrupts 175.75 ± 26% +2956.8% 5372 ± 35% interrupts.CPU186.PMI:Performance_monitoring_interrupts 1349 ± 18% -83.2% 226.75 ± 14% interrupts.CPU186.RES:Rescheduling_interrupts 21.50 ± 44% +2820.9% 628.00 ± 15% interrupts.CPU186.TLB:TLB_shootdowns 1241 ± 14% +969.0% 13269 ± 6% interrupts.CPU187.CAL:Function_call_interrupts 177.25 ± 26% +2643.6% 4863 ± 33% interrupts.CPU187.NMI:Non-maskable_interrupts 177.25 ± 26% +2643.6% 4863 ± 33% interrupts.CPU187.PMI:Performance_monitoring_interrupts 1196 ± 46% -79.0% 251.25 ± 23% interrupts.CPU187.RES:Rescheduling_interrupts 21.25 ± 42% +2525.9% 558.00 ± 23% interrupts.CPU187.TLB:TLB_shootdowns 1401 ± 16% +720.2% 11493 ± 12% interrupts.CPU188.CAL:Function_call_interrupts 182.50 ± 27% +2534.8% 4808 ± 31% interrupts.CPU188.NMI:Non-maskable_interrupts 182.50 ± 27% +2534.8% 4808 ± 31% interrupts.CPU188.PMI:Performance_monitoring_interrupts 1079 ± 23% -80.9% 206.50 ± 17% interrupts.CPU188.RES:Rescheduling_interrupts 13.75 ± 13% +3150.9% 447.00 ± 23% interrupts.CPU188.TLB:TLB_shootdowns 1352 ± 11% +881.9% 13277 ± 16% interrupts.CPU189.CAL:Function_call_interrupts 179.50 ± 26% +2773.8% 5158 ± 37% interrupts.CPU189.NMI:Non-maskable_interrupts 179.50 ± 26% +2773.8% 5158 ± 37% interrupts.CPU189.PMI:Performance_monitoring_interrupts 1375 ± 28% -82.5% 240.25 ± 17% interrupts.CPU189.RES:Rescheduling_interrupts 35.25 ± 48% +1895.7% 703.50 ± 19% interrupts.CPU189.TLB:TLB_shootdowns 1156 ± 4% +1495.6% 18453 ± 14% interrupts.CPU19.CAL:Function_call_interrupts 204.50 +2211.9% 4727 ± 23% interrupts.CPU19.NMI:Non-maskable_interrupts 204.50 +2211.9% 4727 ± 23% interrupts.CPU19.PMI:Performance_monitoring_interrupts 1344 ± 21% -69.5% 410.50 ± 15% interrupts.CPU19.RES:Rescheduling_interrupts 12.25 ± 15% +6300.0% 784.00 ± 7% interrupts.CPU19.TLB:TLB_shootdowns 1154 ± 6% +947.2% 12092 ± 11% interrupts.CPU190.CAL:Function_call_interrupts 176.75 ± 26% +2962.0% 5412 ± 25% interrupts.CPU190.NMI:Non-maskable_interrupts 176.75 ± 26% +2962.0% 5412 ± 25% interrupts.CPU190.PMI:Performance_monitoring_interrupts 1021 ± 17% -79.4% 210.25 ± 23% interrupts.CPU190.RES:Rescheduling_interrupts 18.00 ± 25% +2488.9% 466.00 ± 17% interrupts.CPU190.TLB:TLB_shootdowns 1060 ± 12% +354.3% 4815 ± 10% interrupts.CPU191.CAL:Function_call_interrupts 183.25 ± 25% +2279.8% 4361 ± 29% interrupts.CPU191.NMI:Non-maskable_interrupts 183.25 ± 25% +2279.8% 4361 ± 29% interrupts.CPU191.PMI:Performance_monitoring_interrupts 642.50 ± 26% -85.0% 96.50 ± 11% interrupts.CPU191.RES:Rescheduling_interrupts 1533 ± 33% +1016.4% 17117 ± 4% interrupts.CPU2.CAL:Function_call_interrupts 211.75 ± 8% +3240.4% 7073 ± 16% interrupts.CPU2.NMI:Non-maskable_interrupts 211.75 ± 8% +3240.4% 7073 ± 16% interrupts.CPU2.PMI:Performance_monitoring_interrupts 1065 ± 7% -64.9% 374.25 ± 10% interrupts.CPU2.RES:Rescheduling_interrupts 9.75 ± 48% +8120.5% 801.50 ± 17% interrupts.CPU2.TLB:TLB_shootdowns 1158 ± 9% +1451.8% 17977 ± 18% interrupts.CPU20.CAL:Function_call_interrupts 206.50 ± 4% +1652.7% 3619 ± 28% interrupts.CPU20.NMI:Non-maskable_interrupts 206.50 ± 4% +1652.7% 3619 ± 28% interrupts.CPU20.PMI:Performance_monitoring_interrupts 1099 ± 49% -65.9% 375.00 ± 16% interrupts.CPU20.RES:Rescheduling_interrupts 20.75 ± 86% +4350.6% 923.50 ± 7% interrupts.CPU20.TLB:TLB_shootdowns 4622 ±128% +288.3% 17949 ± 16% interrupts.CPU21.CAL:Function_call_interrupts 175.25 ± 24% +2803.9% 5089 ± 22% interrupts.CPU21.NMI:Non-maskable_interrupts 175.25 ± 24% +2803.9% 5089 ± 22% interrupts.CPU21.PMI:Performance_monitoring_interrupts 1170 ± 33% -65.3% 405.50 ± 16% interrupts.CPU21.RES:Rescheduling_interrupts 9.75 ± 8% +8459.0% 834.50 ± 23% interrupts.CPU21.TLB:TLB_shootdowns 1130 ± 8% +1540.3% 18539 ± 19% interrupts.CPU22.CAL:Function_call_interrupts 181.75 ± 26% +2270.2% 4307 ± 15% interrupts.CPU22.NMI:Non-maskable_interrupts 181.75 ± 26% +2270.2% 4307 ± 15% interrupts.CPU22.PMI:Performance_monitoring_interrupts 1330 ± 25% -70.1% 398.00 ± 20% interrupts.CPU22.RES:Rescheduling_interrupts 10.25 ± 54% +8246.3% 855.50 ± 15% interrupts.CPU22.TLB:TLB_shootdowns 1130 ± 12% +1531.3% 18446 ± 13% interrupts.CPU23.CAL:Function_call_interrupts 177.50 ± 23% +2556.3% 4715 ± 31% interrupts.CPU23.NMI:Non-maskable_interrupts 177.50 ± 23% +2556.3% 4715 ± 31% interrupts.CPU23.PMI:Performance_monitoring_interrupts 1240 ± 6% -68.6% 389.75 ± 15% interrupts.CPU23.RES:Rescheduling_interrupts 33.00 ± 84% +2371.2% 815.50 ± 6% interrupts.CPU23.TLB:TLB_shootdowns 1247 ± 11% +1214.2% 16394 ± 13% interrupts.CPU24.CAL:Function_call_interrupts 180.25 ± 27% +3176.8% 5906 ± 31% interrupts.CPU24.NMI:Non-maskable_interrupts 180.25 ± 27% +3176.8% 5906 ± 31% interrupts.CPU24.PMI:Performance_monitoring_interrupts 934.00 ± 6% -59.9% 374.25 ± 17% interrupts.CPU24.RES:Rescheduling_interrupts 12.50 ± 36% +6030.0% 766.25 ± 12% interrupts.CPU24.TLB:TLB_shootdowns 1104 ± 13% +1607.5% 18855 ± 21% interrupts.CPU25.CAL:Function_call_interrupts 199.75 ± 3% +2961.6% 6115 ± 38% interrupts.CPU25.NMI:Non-maskable_interrupts 199.75 ± 3% +2961.6% 6115 ± 38% interrupts.CPU25.PMI:Performance_monitoring_interrupts 1100 ± 41% -59.5% 445.25 ± 14% interrupts.CPU25.RES:Rescheduling_interrupts 9.00 ± 47% +9119.4% 829.75 ± 13% interrupts.CPU25.TLB:TLB_shootdowns 1463 ± 17% +1091.0% 17433 ± 4% interrupts.CPU26.CAL:Function_call_interrupts 199.25 ± 4% +2527.7% 5235 ± 37% interrupts.CPU26.NMI:Non-maskable_interrupts 199.25 ± 4% +2527.7% 5235 ± 37% interrupts.CPU26.PMI:Performance_monitoring_interrupts 1474 ± 18% -73.8% 386.25 ± 25% interrupts.CPU26.RES:Rescheduling_interrupts 11.75 ± 38% +6944.7% 827.75 ± 14% interrupts.CPU26.TLB:TLB_shootdowns 1148 ± 17% +1236.0% 15346 ± 6% interrupts.CPU27.CAL:Function_call_interrupts 204.50 ± 4% +2298.5% 4905 ± 40% interrupts.CPU27.NMI:Non-maskable_interrupts 204.50 ± 4% +2298.5% 4905 ± 40% interrupts.CPU27.PMI:Performance_monitoring_interrupts 1265 ± 31% -69.6% 384.50 ± 20% interrupts.CPU27.RES:Rescheduling_interrupts 24.00 ± 68% +3257.3% 805.75 ± 20% interrupts.CPU27.TLB:TLB_shootdowns 1024 ± 5% +1668.9% 18118 ± 7% interrupts.CPU28.CAL:Function_call_interrupts 198.25 ± 3% +3294.2% 6729 ± 16% interrupts.CPU28.NMI:Non-maskable_interrupts 198.25 ± 3% +3294.2% 6729 ± 16% interrupts.CPU28.PMI:Performance_monitoring_interrupts 1301 ± 8% -69.6% 395.00 ± 6% interrupts.CPU28.RES:Rescheduling_interrupts 32.25 ± 61% +2263.6% 762.25 ± 11% interrupts.CPU28.TLB:TLB_shootdowns 1072 ± 7% +1619.3% 18435 ± 11% interrupts.CPU29.CAL:Function_call_interrupts 200.00 ± 3% +3180.0% 6560 ± 13% interrupts.CPU29.NMI:Non-maskable_interrupts 200.00 ± 3% +3180.0% 6560 ± 13% interrupts.CPU29.PMI:Performance_monitoring_interrupts 1335 ± 15% -68.9% 415.75 ± 11% interrupts.CPU29.RES:Rescheduling_interrupts 23.50 ± 56% +3357.4% 812.50 ± 7% interrupts.CPU29.TLB:TLB_shootdowns 1328 ± 33% +1287.1% 18424 ± 17% interrupts.CPU3.CAL:Function_call_interrupts 204.00 ± 3% +3221.1% 6775 ± 15% interrupts.CPU3.NMI:Non-maskable_interrupts 204.00 ± 3% +3221.1% 6775 ± 15% interrupts.CPU3.PMI:Performance_monitoring_interrupts 1256 ± 9% -70.6% 369.25 ± 13% interrupts.CPU3.RES:Rescheduling_interrupts 11.75 ± 46% +6491.5% 774.50 ± 10% interrupts.CPU3.TLB:TLB_shootdowns 1028 ± 3% +1513.6% 16588 ± 12% interrupts.CPU30.CAL:Function_call_interrupts 194.50 ± 2% +3653.2% 7300 ± 20% interrupts.CPU30.NMI:Non-maskable_interrupts 194.50 ± 2% +3653.2% 7300 ± 20% interrupts.CPU30.PMI:Performance_monitoring_interrupts 1336 ± 11% -67.4% 435.25 ± 23% interrupts.CPU30.RES:Rescheduling_interrupts 40.50 ±122% +1838.9% 785.25 ± 7% interrupts.CPU30.TLB:TLB_shootdowns 1140 ± 6% +1545.2% 18768 ± 8% interrupts.CPU31.CAL:Function_call_interrupts 200.25 ± 4% +2843.4% 5894 ± 32% interrupts.CPU31.NMI:Non-maskable_interrupts 200.25 ± 4% +2843.4% 5894 ± 32% interrupts.CPU31.PMI:Performance_monitoring_interrupts 1193 ± 27% -65.4% 413.25 ± 11% interrupts.CPU31.RES:Rescheduling_interrupts 34.50 ± 85% +2294.9% 826.25 ± 5% interrupts.CPU31.TLB:TLB_shootdowns 1205 ± 9% +1171.7% 15324 ± 11% interrupts.CPU32.CAL:Function_call_interrupts 197.75 ± 5% +3269.5% 6663 ± 14% interrupts.CPU32.NMI:Non-maskable_interrupts 197.75 ± 5% +3269.5% 6663 ± 14% interrupts.CPU32.PMI:Performance_monitoring_interrupts 1052 ± 15% -63.8% 380.75 ± 14% interrupts.CPU32.RES:Rescheduling_interrupts 7.50 ± 39% +11026.7% 834.50 ± 17% interrupts.CPU32.TLB:TLB_shootdowns 1199 ± 15% +1305.8% 16855 ± 6% interrupts.CPU33.CAL:Function_call_interrupts 176.00 ± 26% +3479.7% 6300 ± 32% interrupts.CPU33.NMI:Non-maskable_interrupts 176.00 ± 26% +3479.7% 6300 ± 32% interrupts.CPU33.PMI:Performance_monitoring_interrupts 1174 ± 39% -66.5% 393.25 ± 11% interrupts.CPU33.RES:Rescheduling_interrupts 21.75 ± 89% +3767.8% 841.25 ± 11% interrupts.CPU33.TLB:TLB_shootdowns 1103 ± 5% +1436.9% 16955 ± 11% interrupts.CPU34.CAL:Function_call_interrupts 170.75 ± 24% +3065.3% 5404 ± 23% interrupts.CPU34.NMI:Non-maskable_interrupts 170.75 ± 24% +3065.3% 5404 ± 23% interrupts.CPU34.PMI:Performance_monitoring_interrupts 1522 ± 14% -76.6% 355.75 ± 14% interrupts.CPU34.RES:Rescheduling_interrupts 41.50 ±119% +1819.3% 796.50 ± 4% interrupts.CPU34.TLB:TLB_shootdowns 1107 ± 12% +1574.7% 18539 ± 15% interrupts.CPU35.CAL:Function_call_interrupts 200.75 ± 6% +2967.6% 6158 ± 23% interrupts.CPU35.NMI:Non-maskable_interrupts 200.75 ± 6% +2967.6% 6158 ± 23% interrupts.CPU35.PMI:Performance_monitoring_interrupts 1132 ± 20% -63.5% 413.00 ± 8% interrupts.CPU35.RES:Rescheduling_interrupts 9.00 ± 30% +9308.3% 846.75 ± 16% interrupts.CPU35.TLB:TLB_shootdowns 1011 ± 5% +1621.5% 17417 ± 8% interrupts.CPU36.CAL:Function_call_interrupts 194.75 ± 3% +3467.9% 6948 ± 7% interrupts.CPU36.NMI:Non-maskable_interrupts 194.75 ± 3% +3467.9% 6948 ± 7% interrupts.CPU36.PMI:Performance_monitoring_interrupts 983.00 ± 32% -58.4% 409.25 ± 22% interrupts.CPU36.RES:Rescheduling_interrupts 21.75 ± 75% +3134.5% 703.50 ± 10% interrupts.CPU36.TLB:TLB_shootdowns 1136 ± 9% +1481.5% 17970 ± 15% interrupts.CPU37.CAL:Function_call_interrupts 192.75 ± 3% +3192.0% 6345 ± 12% interrupts.CPU37.NMI:Non-maskable_interrupts 192.75 ± 3% +3192.0% 6345 ± 12% interrupts.CPU37.PMI:Performance_monitoring_interrupts 1269 ± 40% -70.2% 379.00 ± 22% interrupts.CPU37.RES:Rescheduling_interrupts 16.50 ± 21% +5768.2% 968.25 ± 32% interrupts.CPU37.TLB:TLB_shootdowns 1168 ± 11% +1204.1% 15238 ± 13% interrupts.CPU38.CAL:Function_call_interrupts 195.25 ± 3% +3163.8% 6372 ± 13% interrupts.CPU38.NMI:Non-maskable_interrupts 195.25 ± 3% +3163.8% 6372 ± 13% interrupts.CPU38.PMI:Performance_monitoring_interrupts 1412 ± 26% -71.1% 408.00 ± 21% interrupts.CPU38.RES:Rescheduling_interrupts 27.00 ±104% +2736.1% 765.75 ± 20% interrupts.CPU38.TLB:TLB_shootdowns 1159 ± 2% +1433.1% 17769 ± 19% interrupts.CPU39.CAL:Function_call_interrupts 170.50 ± 26% +3568.5% 6254 ± 25% interrupts.CPU39.NMI:Non-maskable_interrupts 170.50 ± 26% +3568.5% 6254 ± 25% interrupts.CPU39.PMI:Performance_monitoring_interrupts 1395 ± 22% -72.4% 385.50 ± 17% interrupts.CPU39.RES:Rescheduling_interrupts 33.00 ± 79% +2272.0% 782.75 ± 16% interrupts.CPU39.TLB:TLB_shootdowns 1093 ± 8% +1469.4% 17154 ± 11% interrupts.CPU4.CAL:Function_call_interrupts 205.50 ± 3% +2958.8% 6285 ± 27% interrupts.CPU4.NMI:Non-maskable_interrupts 205.50 ± 3% +2958.8% 6285 ± 27% interrupts.CPU4.PMI:Performance_monitoring_interrupts 1344 ± 24% -73.6% 354.75 ± 11% interrupts.CPU4.RES:Rescheduling_interrupts 21.00 ± 71% +3188.1% 690.50 ± 14% interrupts.CPU4.TLB:TLB_shootdowns 1058 ± 9% +1555.4% 17518 ± 16% interrupts.CPU40.CAL:Function_call_interrupts 147.00 ± 34% +4442.0% 6676 ± 10% interrupts.CPU40.NMI:Non-maskable_interrupts 147.00 ± 34% +4442.0% 6676 ± 10% interrupts.CPU40.PMI:Performance_monitoring_interrupts 1115 ± 11% -63.5% 407.75 ± 18% interrupts.CPU40.RES:Rescheduling_interrupts 33.50 ± 74% +2391.8% 834.75 ± 16% interrupts.CPU40.TLB:TLB_shootdowns 1037 ± 9% +1763.6% 19334 ± 8% interrupts.CPU41.CAL:Function_call_interrupts 148.00 ± 34% +4094.8% 6208 ± 6% interrupts.CPU41.NMI:Non-maskable_interrupts 148.00 ± 34% +4094.8% 6208 ± 6% interrupts.CPU41.PMI:Performance_monitoring_interrupts 1286 ± 16% -69.1% 397.25 ± 11% interrupts.CPU41.RES:Rescheduling_interrupts 8.00 ± 23% +11725.0% 946.00 ± 6% interrupts.CPU41.TLB:TLB_shootdowns 1202 ± 7% +1323.9% 17118 ± 16% interrupts.CPU42.CAL:Function_call_interrupts 170.75 ± 25% +3722.4% 6526 ± 13% interrupts.CPU42.NMI:Non-maskable_interrupts 170.75 ± 25% +3722.4% 6526 ± 13% interrupts.CPU42.PMI:Performance_monitoring_interrupts 1201 ± 21% -68.9% 373.00 ± 8% interrupts.CPU42.RES:Rescheduling_interrupts 27.50 ±115% +2939.1% 835.75 ± 5% interrupts.CPU42.TLB:TLB_shootdowns 1114 ± 8% +1456.3% 17337 ± 15% interrupts.CPU43.CAL:Function_call_interrupts 151.00 ± 37% +4446.0% 6864 ± 11% interrupts.CPU43.NMI:Non-maskable_interrupts 151.00 ± 37% +4446.0% 6864 ± 11% interrupts.CPU43.PMI:Performance_monitoring_interrupts 1182 ± 22% -65.1% 412.50 ± 12% interrupts.CPU43.RES:Rescheduling_interrupts 20.50 ± 72% +3782.9% 796.00 ± 7% interrupts.CPU43.TLB:TLB_shootdowns 1072 ± 13% +1418.1% 16273 ± 19% interrupts.CPU44.CAL:Function_call_interrupts 147.00 ± 33% +4449.7% 6688 ± 13% interrupts.CPU44.NMI:Non-maskable_interrupts 147.00 ± 33% +4449.7% 6688 ± 13% interrupts.CPU44.PMI:Performance_monitoring_interrupts 1036 ± 47% -62.0% 393.75 ± 23% interrupts.CPU44.RES:Rescheduling_interrupts 19.00 ± 78% +4377.6% 850.75 ± 16% interrupts.CPU44.TLB:TLB_shootdowns 1324 ± 20% +1200.4% 17217 ± 25% interrupts.CPU45.CAL:Function_call_interrupts 146.50 ± 34% +4031.6% 6052 ± 9% interrupts.CPU45.NMI:Non-maskable_interrupts 146.50 ± 34% +4031.6% 6052 ± 9% interrupts.CPU45.PMI:Performance_monitoring_interrupts 1421 ± 17% -69.5% 433.50 ± 23% interrupts.CPU45.RES:Rescheduling_interrupts 12.75 ± 32% +5831.4% 756.25 ± 18% interrupts.CPU45.TLB:TLB_shootdowns 1117 ± 8% +1500.1% 17885 ± 6% interrupts.CPU46.CAL:Function_call_interrupts 147.00 ± 35% +4182.7% 6295 ± 26% interrupts.CPU46.NMI:Non-maskable_interrupts 147.00 ± 35% +4182.7% 6295 ± 26% interrupts.CPU46.PMI:Performance_monitoring_interrupts 1382 ± 19% -72.6% 378.75 ± 18% interrupts.CPU46.RES:Rescheduling_interrupts 17.25 ± 82% +4456.5% 786.00 ± 14% interrupts.CPU46.TLB:TLB_shootdowns 991.00 ± 3% +1739.2% 18226 ± 21% interrupts.CPU47.CAL:Function_call_interrupts 146.50 ± 34% +4033.8% 6056 ± 38% interrupts.CPU47.NMI:Non-maskable_interrupts 146.50 ± 34% +4033.8% 6056 ± 38% interrupts.CPU47.PMI:Performance_monitoring_interrupts 1355 ± 27% -71.1% 392.25 ± 7% interrupts.CPU47.RES:Rescheduling_interrupts 9.75 ± 36% +7759.0% 766.25 ± 13% interrupts.CPU47.TLB:TLB_shootdowns 1255 ± 4% +1345.2% 18141 ± 12% interrupts.CPU48.CAL:Function_call_interrupts 185.25 ± 28% +3158.0% 6035 ± 20% interrupts.CPU48.NMI:Non-maskable_interrupts 185.25 ± 28% +3158.0% 6035 ± 20% interrupts.CPU48.PMI:Performance_monitoring_interrupts 1187 ± 9% -63.4% 434.50 ± 11% interrupts.CPU48.RES:Rescheduling_interrupts 10.75 ± 27% +7479.1% 814.75 ± 14% interrupts.CPU48.TLB:TLB_shootdowns 1049 +1531.7% 17116 ± 14% interrupts.CPU49.CAL:Function_call_interrupts 193.50 ± 3% +2870.8% 5748 ± 24% interrupts.CPU49.NMI:Non-maskable_interrupts 193.50 ± 3% +2870.8% 5748 ± 24% interrupts.CPU49.PMI:Performance_monitoring_interrupts 1169 ± 24% -65.7% 400.75 ± 15% interrupts.CPU49.RES:Rescheduling_interrupts 12.50 ± 41% +5830.0% 741.25 ± 15% interrupts.CPU49.TLB:TLB_shootdowns 1230 ± 22% +1391.4% 18352 ± 14% interrupts.CPU5.CAL:Function_call_interrupts 179.00 ± 26% +2636.3% 4898 ± 21% interrupts.CPU5.NMI:Non-maskable_interrupts 179.00 ± 26% +2636.3% 4898 ± 21% interrupts.CPU5.PMI:Performance_monitoring_interrupts 1262 ± 25% -72.4% 349.00 ± 27% interrupts.CPU5.RES:Rescheduling_interrupts 8.50 ± 39% +8964.7% 770.50 ± 6% interrupts.CPU5.TLB:TLB_shootdowns 1181 ± 2% +1606.9% 20171 ± 8% interrupts.CPU50.CAL:Function_call_interrupts 195.50 ± 3% +2789.4% 5648 ± 24% interrupts.CPU50.NMI:Non-maskable_interrupts 195.50 ± 3% +2789.4% 5648 ± 24% interrupts.CPU50.PMI:Performance_monitoring_interrupts 1315 ± 15% -68.4% 416.25 ± 14% interrupts.CPU50.RES:Rescheduling_interrupts 25.50 ± 78% +3609.8% 946.00 ± 19% interrupts.CPU50.TLB:TLB_shootdowns 1153 ± 16% +1384.8% 17131 ± 4% interrupts.CPU51.CAL:Function_call_interrupts 197.00 ± 2% +2581.5% 5282 ± 26% interrupts.CPU51.NMI:Non-maskable_interrupts 197.00 ± 2% +2581.5% 5282 ± 26% interrupts.CPU51.PMI:Performance_monitoring_interrupts 1266 ± 14% -65.7% 433.75 ± 9% interrupts.CPU51.RES:Rescheduling_interrupts 14.00 ± 39% +6333.9% 900.75 ± 19% interrupts.CPU51.TLB:TLB_shootdowns 1022 ± 4% +1549.3% 16868 ± 11% interrupts.CPU52.CAL:Function_call_interrupts 199.75 ± 3% +2957.8% 6108 ± 27% interrupts.CPU52.NMI:Non-maskable_interrupts 199.75 ± 3% +2957.8% 6108 ± 27% interrupts.CPU52.PMI:Performance_monitoring_interrupts 1245 ± 30% -69.1% 385.00 ± 10% interrupts.CPU52.RES:Rescheduling_interrupts 12.00 ± 45% +6639.6% 808.75 ± 4% interrupts.CPU52.TLB:TLB_shootdowns 1071 ± 10% +1656.0% 18811 ± 15% interrupts.CPU53.CAL:Function_call_interrupts 198.75 ± 3% +2980.1% 6121 ± 29% interrupts.CPU53.NMI:Non-maskable_interrupts 198.75 ± 3% +2980.1% 6121 ± 29% interrupts.CPU53.PMI:Performance_monitoring_interrupts 1074 ± 34% -63.5% 392.50 ± 13% interrupts.CPU53.RES:Rescheduling_interrupts 15.00 ±105% +5255.0% 803.25 ± 14% interrupts.CPU53.TLB:TLB_shootdowns 1127 ± 21% +1410.0% 17017 ± 21% interrupts.CPU54.CAL:Function_call_interrupts 169.50 ± 24% +3912.4% 6801 ± 27% interrupts.CPU54.NMI:Non-maskable_interrupts 169.50 ± 24% +3912.4% 6801 ± 27% interrupts.CPU54.PMI:Performance_monitoring_interrupts 1311 ± 26% -71.2% 377.25 ± 23% interrupts.CPU54.RES:Rescheduling_interrupts 13.00 ± 21% +6096.2% 805.50 ± 18% interrupts.CPU54.TLB:TLB_shootdowns 1322 ± 31% +1308.5% 18620 ± 15% interrupts.CPU55.CAL:Function_call_interrupts 147.00 ± 35% +3924.8% 5916 ± 29% interrupts.CPU55.NMI:Non-maskable_interrupts 147.00 ± 35% +3924.8% 5916 ± 29% interrupts.CPU55.PMI:Performance_monitoring_interrupts 1107 ± 16% -65.8% 379.00 ± 19% interrupts.CPU55.RES:Rescheduling_interrupts 24.25 ±114% +3718.6% 926.00 ± 19% interrupts.CPU55.TLB:TLB_shootdowns 1116 ± 4% +1521.7% 18110 ± 14% interrupts.CPU56.CAL:Function_call_interrupts 173.25 ± 26% +3283.4% 5861 ± 39% interrupts.CPU56.NMI:Non-maskable_interrupts 173.25 ± 26% +3283.4% 5861 ± 39% interrupts.CPU56.PMI:Performance_monitoring_interrupts 1150 ± 23% -68.1% 366.50 ± 20% interrupts.CPU56.RES:Rescheduling_interrupts 10.50 ± 39% +8031.0% 853.75 ± 7% interrupts.CPU56.TLB:TLB_shootdowns 1202 ± 21% +1357.6% 17531 ± 21% interrupts.CPU57.CAL:Function_call_interrupts 188.50 ± 31% +3120.7% 6071 ± 34% interrupts.CPU57.NMI:Non-maskable_interrupts 188.50 ± 31% +3120.7% 6071 ± 34% interrupts.CPU57.PMI:Performance_monitoring_interrupts 1343 ± 16% -70.7% 393.50 ± 10% interrupts.CPU57.RES:Rescheduling_interrupts 12.75 ± 24% +7007.8% 906.25 ± 19% interrupts.CPU57.TLB:TLB_shootdowns 1054 ± 9% +1572.7% 17643 ± 15% interrupts.CPU58.CAL:Function_call_interrupts 173.00 ± 26% +2821.1% 5053 ± 23% interrupts.CPU58.NMI:Non-maskable_interrupts 173.00 ± 26% +2821.1% 5053 ± 23% interrupts.CPU58.PMI:Performance_monitoring_interrupts 1287 ± 19% -68.4% 407.00 ± 17% interrupts.CPU58.RES:Rescheduling_interrupts 16.00 ± 98% +5365.6% 874.50 ± 11% interrupts.CPU58.TLB:TLB_shootdowns 1035 ± 11% +1525.8% 16831 ± 19% interrupts.CPU59.CAL:Function_call_interrupts 171.25 ± 26% +2705.4% 4804 ± 56% interrupts.CPU59.NMI:Non-maskable_interrupts 171.25 ± 26% +2705.4% 4804 ± 56% interrupts.CPU59.PMI:Performance_monitoring_interrupts 1214 ± 30% -71.0% 352.50 ± 5% interrupts.CPU59.RES:Rescheduling_interrupts 27.50 ±116% +3414.5% 966.50 ± 27% interrupts.CPU59.TLB:TLB_shootdowns 1158 ± 11% +1497.6% 18500 ± 7% interrupts.CPU6.CAL:Function_call_interrupts 176.75 ± 25% +3246.7% 5915 ± 28% interrupts.CPU6.NMI:Non-maskable_interrupts 176.75 ± 25% +3246.7% 5915 ± 28% interrupts.CPU6.PMI:Performance_monitoring_interrupts 1568 ± 19% -75.1% 391.00 ± 17% interrupts.CPU6.RES:Rescheduling_interrupts 9.00 ± 40% +9402.8% 855.25 ± 6% interrupts.CPU6.TLB:TLB_shootdowns 1036 ± 7% +1762.4% 19294 ± 24% interrupts.CPU60.CAL:Function_call_interrupts 172.25 ± 26% +3278.2% 5819 ± 27% interrupts.CPU60.NMI:Non-maskable_interrupts 172.25 ± 26% +3278.2% 5819 ± 27% interrupts.CPU60.PMI:Performance_monitoring_interrupts 1405 ± 24% -71.3% 404.00 ± 14% interrupts.CPU60.RES:Rescheduling_interrupts 24.75 ± 80% +3187.9% 813.75 ± 8% interrupts.CPU60.TLB:TLB_shootdowns 1071 ± 7% +1622.3% 18449 ± 12% interrupts.CPU61.CAL:Function_call_interrupts 170.75 ± 27% +3293.9% 5795 ± 30% interrupts.CPU61.NMI:Non-maskable_interrupts 170.75 ± 27% +3293.9% 5795 ± 30% interrupts.CPU61.PMI:Performance_monitoring_interrupts 1413 ± 13% -71.7% 400.00 ± 5% interrupts.CPU61.RES:Rescheduling_interrupts 13.75 ± 22% +6080.0% 849.75 ± 4% interrupts.CPU61.TLB:TLB_shootdowns 1145 ± 13% +1577.6% 19212 ± 26% interrupts.CPU62.CAL:Function_call_interrupts 170.75 ± 27% +3811.3% 6678 ± 32% interrupts.CPU62.NMI:Non-maskable_interrupts 170.75 ± 27% +3811.3% 6678 ± 32% interrupts.CPU62.PMI:Performance_monitoring_interrupts 1050 ± 33% -64.0% 378.25 ± 18% interrupts.CPU62.RES:Rescheduling_interrupts 9.25 ± 59% +8451.4% 791.00 ± 30% interrupts.CPU62.TLB:TLB_shootdowns 2207 ± 90% +761.8% 19021 ± 12% interrupts.CPU63.CAL:Function_call_interrupts 172.25 ± 27% +3726.7% 6591 ± 23% interrupts.CPU63.NMI:Non-maskable_interrupts 172.25 ± 27% +3726.7% 6591 ± 23% interrupts.CPU63.PMI:Performance_monitoring_interrupts 1324 ± 27% -70.2% 395.00 ± 18% interrupts.CPU63.RES:Rescheduling_interrupts 22.00 ± 50% +4025.0% 907.50 ± 10% interrupts.CPU63.TLB:TLB_shootdowns 1163 ± 5% +1450.6% 18045 ± 10% interrupts.CPU64.CAL:Function_call_interrupts 192.50 ± 2% +2521.7% 5046 ± 33% interrupts.CPU64.NMI:Non-maskable_interrupts 192.50 ± 2% +2521.7% 5046 ± 33% interrupts.CPU64.PMI:Performance_monitoring_interrupts 1376 ± 28% -71.2% 395.75 ± 12% interrupts.CPU64.RES:Rescheduling_interrupts 21.00 ± 27% +3470.2% 749.75 ± 8% interrupts.CPU64.TLB:TLB_shootdowns 1069 ± 9% +1462.5% 16710 ± 11% interrupts.CPU65.CAL:Function_call_interrupts 193.00 ± 2% +3553.5% 7051 ± 13% interrupts.CPU65.NMI:Non-maskable_interrupts 193.00 ± 2% +3553.5% 7051 ± 13% interrupts.CPU65.PMI:Performance_monitoring_interrupts 1199 ± 24% -68.8% 374.00 ± 12% interrupts.CPU65.RES:Rescheduling_interrupts 11.75 ± 62% +6238.3% 744.75 ± 9% interrupts.CPU65.TLB:TLB_shootdowns 1055 ± 7% +1710.1% 19105 ± 13% interrupts.CPU66.CAL:Function_call_interrupts 204.25 ± 7% +3271.5% 6886 ± 16% interrupts.CPU66.NMI:Non-maskable_interrupts 204.25 ± 7% +3271.5% 6886 ± 16% interrupts.CPU66.PMI:Performance_monitoring_interrupts 1186 ± 43% -65.7% 406.75 ± 5% interrupts.CPU66.RES:Rescheduling_interrupts 17.00 ± 89% +4783.8% 830.25 ± 8% interrupts.CPU66.TLB:TLB_shootdowns 1190 ± 16% +1446.7% 18409 ± 12% interrupts.CPU67.CAL:Function_call_interrupts 193.50 ± 3% +3171.6% 6330 ± 15% interrupts.CPU67.NMI:Non-maskable_interrupts 193.50 ± 3% +3171.6% 6330 ± 15% interrupts.CPU67.PMI:Performance_monitoring_interrupts 1108 ± 17% -64.6% 392.00 ± 14% interrupts.CPU67.RES:Rescheduling_interrupts 16.75 ± 68% +4800.0% 820.75 ± 10% interrupts.CPU67.TLB:TLB_shootdowns 1058 ± 3% +1616.3% 18158 ± 16% interrupts.CPU68.CAL:Function_call_interrupts 194.75 ± 3% +3460.2% 6933 ± 13% interrupts.CPU68.NMI:Non-maskable_interrupts 194.75 ± 3% +3460.2% 6933 ± 13% interrupts.CPU68.PMI:Performance_monitoring_interrupts 1436 ± 22% -72.7% 392.50 ± 17% interrupts.CPU68.RES:Rescheduling_interrupts 15.50 ± 39% +5325.8% 841.00 ± 11% interrupts.CPU68.TLB:TLB_shootdowns 1025 ± 7% +1608.1% 17512 ± 20% interrupts.CPU69.CAL:Function_call_interrupts 193.25 ± 3% +2644.6% 5304 ± 28% interrupts.CPU69.NMI:Non-maskable_interrupts 193.25 ± 3% +2644.6% 5304 ± 28% interrupts.CPU69.PMI:Performance_monitoring_interrupts 1345 ± 30% -72.5% 369.75 ± 15% interrupts.CPU69.RES:Rescheduling_interrupts 20.25 ± 41% +4295.1% 890.00 ± 15% interrupts.CPU69.TLB:TLB_shootdowns 1217 ± 11% +1464.7% 19046 ± 14% interrupts.CPU7.CAL:Function_call_interrupts 208.50 ± 6% +2593.2% 5615 ± 29% interrupts.CPU7.NMI:Non-maskable_interrupts 208.50 ± 6% +2593.2% 5615 ± 29% interrupts.CPU7.PMI:Performance_monitoring_interrupts 1075 ± 19% -64.9% 377.00 ± 9% interrupts.CPU7.RES:Rescheduling_interrupts 7.75 ± 27% +10390.3% 813.00 ± 16% interrupts.CPU7.TLB:TLB_shootdowns 1022 ± 7% +1610.7% 17483 ± 8% interrupts.CPU70.CAL:Function_call_interrupts 199.25 ± 3% +3039.6% 6255 ± 16% interrupts.CPU70.NMI:Non-maskable_interrupts 199.25 ± 3% +3039.6% 6255 ± 16% interrupts.CPU70.PMI:Performance_monitoring_interrupts 1400 ± 10% -71.7% 396.75 ± 17% interrupts.CPU70.RES:Rescheduling_interrupts 9.75 ± 24% +8520.5% 840.50 ± 10% interrupts.CPU70.TLB:TLB_shootdowns 946.25 ± 2% +1795.8% 17939 ± 20% interrupts.CPU71.CAL:Function_call_interrupts 199.50 ± 8% +2880.2% 5945 ± 28% interrupts.CPU71.NMI:Non-maskable_interrupts 199.50 ± 8% +2880.2% 5945 ± 28% interrupts.CPU71.PMI:Performance_monitoring_interrupts 1114 ± 24% -68.6% 350.00 ± 22% interrupts.CPU71.RES:Rescheduling_interrupts 20.75 ±115% +3991.6% 849.00 ± 14% interrupts.CPU71.TLB:TLB_shootdowns 1275 ± 6% +1258.0% 17321 ± 5% interrupts.CPU72.CAL:Function_call_interrupts 217.50 ± 9% +2579.8% 5828 ± 19% interrupts.CPU72.NMI:Non-maskable_interrupts 217.50 ± 9% +2579.8% 5828 ± 19% interrupts.CPU72.PMI:Performance_monitoring_interrupts 989.25 ± 7% -62.1% 375.00 ± 17% interrupts.CPU72.RES:Rescheduling_interrupts 12.25 ± 40% +6334.7% 788.25 ± 18% interrupts.CPU72.TLB:TLB_shootdowns 947.00 ± 14% +1776.1% 17766 ± 12% interrupts.CPU73.CAL:Function_call_interrupts 210.00 ± 7% +2687.9% 5854 ± 21% interrupts.CPU73.NMI:Non-maskable_interrupts 210.00 ± 7% +2687.9% 5854 ± 21% interrupts.CPU73.PMI:Performance_monitoring_interrupts 972.75 ± 14% -61.3% 376.00 ± 18% interrupts.CPU73.RES:Rescheduling_interrupts 13.00 ± 47% +5917.3% 782.25 ± 13% interrupts.CPU73.TLB:TLB_shootdowns 1414 ± 15% +1142.3% 17569 ± 8% interrupts.CPU74.CAL:Function_call_interrupts 199.75 ± 3% +2971.5% 6135 ± 14% interrupts.CPU74.NMI:Non-maskable_interrupts 199.75 ± 3% +2971.5% 6135 ± 14% interrupts.CPU74.PMI:Performance_monitoring_interrupts 1032 ± 22% -64.1% 370.50 ± 12% interrupts.CPU74.RES:Rescheduling_interrupts 12.75 ± 15% +6227.5% 806.75 ± 10% interrupts.CPU74.TLB:TLB_shootdowns 1299 ± 9% +1348.1% 18810 ± 13% interrupts.CPU75.CAL:Function_call_interrupts 203.50 ± 3% +2811.8% 5925 ± 22% interrupts.CPU75.NMI:Non-maskable_interrupts 203.50 ± 3% +2811.8% 5925 ± 22% interrupts.CPU75.PMI:Performance_monitoring_interrupts 1076 ± 11% -65.2% 374.00 ± 15% interrupts.CPU75.RES:Rescheduling_interrupts 10.50 ± 39% +7202.4% 766.75 ± 14% interrupts.CPU75.TLB:TLB_shootdowns 1345 ± 17% +1239.1% 18014 ± 10% interrupts.CPU76.CAL:Function_call_interrupts 206.50 ± 7% +2614.3% 5605 ± 24% interrupts.CPU76.NMI:Non-maskable_interrupts 206.50 ± 7% +2614.3% 5605 ± 24% interrupts.CPU76.PMI:Performance_monitoring_interrupts 1611 ± 33% -77.8% 357.75 ± 6% interrupts.CPU76.RES:Rescheduling_interrupts 28.25 ± 59% +2906.2% 849.25 ± 6% interrupts.CPU76.TLB:TLB_shootdowns 1213 ± 13% +1514.4% 19595 ± 17% interrupts.CPU77.CAL:Function_call_interrupts 201.50 ± 4% +2808.8% 5861 ± 16% interrupts.CPU77.NMI:Non-maskable_interrupts 201.50 ± 4% +2808.8% 5861 ± 16% interrupts.CPU77.PMI:Performance_monitoring_interrupts 1088 ± 13% -67.2% 356.50 ± 21% interrupts.CPU77.RES:Rescheduling_interrupts 20.00 ± 82% +3843.8% 788.75 ± 21% interrupts.CPU77.TLB:TLB_shootdowns 1274 ± 15% +1467.5% 19973 ± 15% interrupts.CPU78.CAL:Function_call_interrupts 203.00 ± 6% +2826.2% 5940 ± 26% interrupts.CPU78.NMI:Non-maskable_interrupts 203.00 ± 6% +2826.2% 5940 ± 26% interrupts.CPU78.PMI:Performance_monitoring_interrupts 1368 ± 23% -72.7% 374.00 ± 12% interrupts.CPU78.RES:Rescheduling_interrupts 15.00 ± 35% +5416.7% 827.50 ± 8% interrupts.CPU78.TLB:TLB_shootdowns 1218 ± 10% +1509.9% 19608 ± 11% interrupts.CPU79.CAL:Function_call_interrupts 198.00 ± 3% +2778.9% 5700 ± 34% interrupts.CPU79.NMI:Non-maskable_interrupts 198.00 ± 3% +2778.9% 5700 ± 34% interrupts.CPU79.PMI:Performance_monitoring_interrupts 994.00 ± 53% -63.5% 362.50 ± 16% interrupts.CPU79.RES:Rescheduling_interrupts 12.50 ± 37% +5876.0% 747.00 ± 19% interrupts.CPU79.TLB:TLB_shootdowns 1216 ± 12% +1528.0% 19804 ± 9% interrupts.CPU8.CAL:Function_call_interrupts 204.25 ± 2% +2584.2% 5482 ± 23% interrupts.CPU8.NMI:Non-maskable_interrupts 204.25 ± 2% +2584.2% 5482 ± 23% interrupts.CPU8.PMI:Performance_monitoring_interrupts 908.00 ± 25% -57.7% 384.50 ± 15% interrupts.CPU8.RES:Rescheduling_interrupts 17.75 ± 98% +4738.0% 858.75 ± 7% interrupts.CPU8.TLB:TLB_shootdowns 1431 ± 5% +1096.4% 17127 ± 19% interrupts.CPU80.CAL:Function_call_interrupts 198.50 ± 3% +2388.4% 4939 ± 36% interrupts.CPU80.NMI:Non-maskable_interrupts 198.50 ± 3% +2388.4% 4939 ± 36% interrupts.CPU80.PMI:Performance_monitoring_interrupts 990.75 ± 20% -62.3% 373.25 ± 17% interrupts.CPU80.RES:Rescheduling_interrupts 15.00 ± 12% +4636.7% 710.50 ± 9% interrupts.CPU80.TLB:TLB_shootdowns 1297 ± 6% +1303.6% 18211 ± 13% interrupts.CPU81.CAL:Function_call_interrupts 199.75 ± 3% +2650.4% 5494 ± 30% interrupts.CPU81.NMI:Non-maskable_interrupts 199.75 ± 3% +2650.4% 5494 ± 30% interrupts.CPU81.PMI:Performance_monitoring_interrupts 1206 ± 21% -67.1% 397.00 ± 15% interrupts.CPU81.RES:Rescheduling_interrupts 9.00 ± 45% +9580.6% 871.25 ± 10% interrupts.CPU81.TLB:TLB_shootdowns 1290 ± 16% +1374.2% 19024 ± 17% interrupts.CPU82.CAL:Function_call_interrupts 213.75 ± 16% +2576.6% 5721 ± 24% interrupts.CPU82.NMI:Non-maskable_interrupts 213.75 ± 16% +2576.6% 5721 ± 24% interrupts.CPU82.PMI:Performance_monitoring_interrupts 1347 ± 19% -73.0% 363.75 ± 19% interrupts.CPU82.RES:Rescheduling_interrupts 16.50 ± 17% +4842.4% 815.50 ± 13% interrupts.CPU82.TLB:TLB_shootdowns 1438 ± 49% +1202.3% 18734 ± 11% interrupts.CPU83.CAL:Function_call_interrupts 202.00 ± 2% +3182.2% 6630 ± 27% interrupts.CPU83.NMI:Non-maskable_interrupts 202.00 ± 2% +3182.2% 6630 ± 27% interrupts.CPU83.PMI:Performance_monitoring_interrupts 972.00 ± 34% -56.0% 427.50 ± 28% interrupts.CPU83.RES:Rescheduling_interrupts 10.25 ± 4% +8290.2% 860.00 ± 9% interrupts.CPU83.TLB:TLB_shootdowns 1247 ± 8% +1491.5% 19857 ± 3% interrupts.CPU84.CAL:Function_call_interrupts 204.25 ± 6% +2474.4% 5258 ± 20% interrupts.CPU84.NMI:Non-maskable_interrupts 204.25 ± 6% +2474.4% 5258 ± 20% interrupts.CPU84.PMI:Performance_monitoring_interrupts 1083 ± 20% -64.8% 381.00 ± 11% interrupts.CPU84.RES:Rescheduling_interrupts 28.75 ± 70% +2530.4% 756.25 ± 8% interrupts.CPU84.TLB:TLB_shootdowns 1190 ± 9% +1535.3% 19468 ± 10% interrupts.CPU85.CAL:Function_call_interrupts 204.25 ± 5% +2661.3% 5640 ± 21% interrupts.CPU85.NMI:Non-maskable_interrupts 204.25 ± 5% +2661.3% 5640 ± 21% interrupts.CPU85.PMI:Performance_monitoring_interrupts 879.50 ± 5% -56.1% 386.25 ± 11% interrupts.CPU85.RES:Rescheduling_interrupts 11.75 ± 38% +7053.2% 840.50 ± 10% interrupts.CPU85.TLB:TLB_shootdowns 1306 ± 11% +1246.6% 17593 ± 11% interrupts.CPU86.CAL:Function_call_interrupts 205.50 ± 6% +2968.5% 6305 ± 20% interrupts.CPU86.NMI:Non-maskable_interrupts 205.50 ± 6% +2968.5% 6305 ± 20% interrupts.CPU86.PMI:Performance_monitoring_interrupts 1465 ± 22% -72.2% 406.75 ± 14% interrupts.CPU86.RES:Rescheduling_interrupts 21.00 ± 22% +3439.3% 743.25 ± 14% interrupts.CPU86.TLB:TLB_shootdowns 1271 ± 16% +1446.3% 19653 ± 14% interrupts.CPU87.CAL:Function_call_interrupts 170.00 ± 21% +3408.2% 5964 ± 28% interrupts.CPU87.NMI:Non-maskable_interrupts 170.00 ± 21% +3408.2% 5964 ± 28% interrupts.CPU87.PMI:Performance_monitoring_interrupts 1211 ± 33% -66.4% 407.00 ± 15% interrupts.CPU87.RES:Rescheduling_interrupts 36.75 ± 66% +2429.3% 929.50 ± 15% interrupts.CPU87.TLB:TLB_shootdowns 1240 ± 8% +1407.9% 18705 ± 8% interrupts.CPU88.CAL:Function_call_interrupts 171.25 ± 22% +2680.7% 4762 ± 26% interrupts.CPU88.NMI:Non-maskable_interrupts 171.25 ± 22% +2680.7% 4762 ± 26% interrupts.CPU88.PMI:Performance_monitoring_interrupts 1250 ± 19% -71.0% 363.00 ± 13% interrupts.CPU88.RES:Rescheduling_interrupts 19.00 ± 59% +4122.4% 802.25 ± 18% interrupts.CPU88.TLB:TLB_shootdowns 1401 ± 39% +1330.7% 20051 ± 20% interrupts.CPU89.CAL:Function_call_interrupts 174.50 ± 19% +2893.8% 5224 ± 34% interrupts.CPU89.NMI:Non-maskable_interrupts 174.50 ± 19% +2893.8% 5224 ± 34% interrupts.CPU89.PMI:Performance_monitoring_interrupts 1102 ± 28% -63.8% 399.50 ± 18% interrupts.CPU89.RES:Rescheduling_interrupts 14.50 ± 44% +6006.9% 885.50 ± 10% interrupts.CPU89.TLB:TLB_shootdowns 1137 ± 6% +1300.8% 15931 ± 9% interrupts.CPU9.CAL:Function_call_interrupts 209.25 +2083.9% 4569 ± 34% interrupts.CPU9.NMI:Non-maskable_interrupts 209.25 +2083.9% 4569 ± 34% interrupts.CPU9.PMI:Performance_monitoring_interrupts 868.25 ± 9% -64.5% 308.25 ± 22% interrupts.CPU9.RES:Rescheduling_interrupts 16.25 ± 99% +3958.5% 659.50 ± 12% interrupts.CPU9.TLB:TLB_shootdowns 1735 ± 24% +1116.8% 21117 ± 12% interrupts.CPU90.CAL:Function_call_interrupts 172.25 ± 22% +3146.7% 5592 ± 22% interrupts.CPU90.NMI:Non-maskable_interrupts 172.25 ± 22% +3146.7% 5592 ± 22% interrupts.CPU90.PMI:Performance_monitoring_interrupts 11.00 ± 51% +7452.3% 830.75 ± 12% interrupts.CPU90.TLB:TLB_shootdowns 1216 ± 9% +1593.4% 20592 ± 9% interrupts.CPU91.CAL:Function_call_interrupts 171.50 ± 24% +2807.9% 4987 ± 37% interrupts.CPU91.NMI:Non-maskable_interrupts 171.50 ± 24% +2807.9% 4987 ± 37% interrupts.CPU91.PMI:Performance_monitoring_interrupts 10.00 ± 33% +9285.0% 938.50 ± 3% interrupts.CPU91.TLB:TLB_shootdowns 1601 ± 31% +1199.7% 20807 ± 5% interrupts.CPU92.CAL:Function_call_interrupts 176.25 ± 22% +2839.7% 5181 ± 26% interrupts.CPU92.NMI:Non-maskable_interrupts 176.25 ± 22% +2839.7% 5181 ± 26% interrupts.CPU92.PMI:Performance_monitoring_interrupts 980.00 ± 29% -53.0% 461.00 ± 16% interrupts.CPU92.RES:Rescheduling_interrupts 27.50 ± 89% +3560.9% 1006 ± 10% interrupts.CPU92.TLB:TLB_shootdowns 1250 ± 11% +1394.2% 18678 ± 11% interrupts.CPU93.CAL:Function_call_interrupts 173.00 ± 23% +3162.4% 5644 ± 41% interrupts.CPU93.NMI:Non-maskable_interrupts 173.00 ± 23% +3162.4% 5644 ± 41% interrupts.CPU93.PMI:Performance_monitoring_interrupts 860.00 ± 44% -54.3% 393.00 ± 7% interrupts.CPU93.RES:Rescheduling_interrupts 5.25 ± 47% +14252.4% 753.50 ± 16% interrupts.CPU93.TLB:TLB_shootdowns 1246 ± 10% +1684.5% 22235 ± 11% interrupts.CPU94.CAL:Function_call_interrupts 171.25 ± 22% +2938.8% 5204 ± 34% interrupts.CPU94.NMI:Non-maskable_interrupts 171.25 ± 22% +2938.8% 5204 ± 34% interrupts.CPU94.PMI:Performance_monitoring_interrupts 1248 ± 28% -65.6% 429.50 ± 11% interrupts.CPU94.RES:Rescheduling_interrupts 15.50 ± 18% +6246.8% 983.75 ± 7% interrupts.CPU94.TLB:TLB_shootdowns 1366 ± 11% +1942.5% 27906 ± 11% interrupts.CPU95.CAL:Function_call_interrupts 178.50 ± 21% +3917.6% 7171 ± 24% interrupts.CPU95.NMI:Non-maskable_interrupts 178.50 ± 21% +3917.6% 7171 ± 24% interrupts.CPU95.PMI:Performance_monitoring_interrupts 1552 ± 14% -65.5% 535.75 ± 12% interrupts.CPU95.RES:Rescheduling_interrupts 24.75 ± 11% +4877.8% 1232 ± 4% interrupts.CPU95.TLB:TLB_shootdowns 1383 ± 18% +1511.1% 22285 ± 7% interrupts.CPU96.CAL:Function_call_interrupts 189.75 ± 24% +2856.4% 5609 ± 22% interrupts.CPU96.NMI:Non-maskable_interrupts 189.75 ± 24% +2856.4% 5609 ± 22% interrupts.CPU96.PMI:Performance_monitoring_interrupts 2279 ± 13% -80.7% 439.25 ± 13% interrupts.CPU96.RES:Rescheduling_interrupts 18.25 ± 19% +5732.9% 1064 ± 8% interrupts.CPU96.TLB:TLB_shootdowns 1344 ± 21% +988.4% 14633 ± 11% interrupts.CPU97.CAL:Function_call_interrupts 175.25 ± 24% +2505.1% 4565 ± 24% interrupts.CPU97.NMI:Non-maskable_interrupts 175.25 ± 24% +2505.1% 4565 ± 24% interrupts.CPU97.PMI:Performance_monitoring_interrupts 1252 ± 35% -75.2% 311.25 ± 7% interrupts.CPU97.RES:Rescheduling_interrupts 23.25 ±106% +2841.9% 684.00 ± 11% interrupts.CPU97.TLB:TLB_shootdowns 1460 ± 19% +897.0% 14556 ± 16% interrupts.CPU98.CAL:Function_call_interrupts 208.00 ± 5% +2507.9% 5424 ± 33% interrupts.CPU98.NMI:Non-maskable_interrupts 208.00 ± 5% +2507.9% 5424 ± 33% interrupts.CPU98.PMI:Performance_monitoring_interrupts 1478 ± 21% -80.3% 291.25 ± 24% interrupts.CPU98.RES:Rescheduling_interrupts 14.00 ± 44% +4632.1% 662.50 ± 24% interrupts.CPU98.TLB:TLB_shootdowns 1310 ± 32% +1008.5% 14523 ± 11% interrupts.CPU99.CAL:Function_call_interrupts 203.00 ± 3% +2390.0% 5054 ± 28% interrupts.CPU99.NMI:Non-maskable_interrupts 203.00 ± 3% +2390.0% 5054 ± 28% interrupts.CPU99.PMI:Performance_monitoring_interrupts 1333 ± 13% -77.4% 300.75 ± 17% interrupts.CPU99.RES:Rescheduling_interrupts 15.25 ± 37% +4483.6% 699.00 ± 12% interrupts.CPU99.TLB:TLB_shootdowns 3.25 ± 13% +6584.6% 217.25 ± 8% interrupts.IWI:IRQ_work_interrupts 35365 ± 2% +2967.1% 1084710 ± 7% interrupts.NMI:Non-maskable_interrupts 35365 ± 2% +2967.1% 1084710 ± 7% interrupts.PMI:Performance_monitoring_interrupts 237203 ± 2% -72.7% 64704 ± 12% interrupts.RES:Rescheduling_interrupts 3697 ± 12% +3721.5% 141290 interrupts.TLB:TLB_shootdowns *************************************************************************************************** lkp-csl-2sp5: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode: interrupt/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp5/stress-ng/30s/0x5002f01 commit: 5925fa68fe (" perf tools fixes for v5.9: 3rd batch") 5ef64cc898 ("mm: allow a controlled amount of unfairness in the page lock") 5925fa68fe824465 5ef64cc8987a9211d3f3667331b ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 50% 2:4 kmsg.BTRFS_error(device_sda5):bdev/dev/sda5_errs:wr#,rd#,flush#,corrupt#,gen %stddev %change %stddev \ | \ 1543392 ± 2% -62.7% 575755 ± 90% stress-ng.sigchld.ops 51445 ± 2% -62.7% 19191 ± 90% stress-ng.sigchld.ops_per_sec 2.397e+08 ± 46% -90.9% 21848257 ± 2% stress-ng.sigrt.ops 7970038 ± 46% -90.9% 727138 ± 2% stress-ng.sigrt.ops_per_sec 6635 +3.0% 6831 stress-ng.time.percent_of_cpu_this_job_got 42339 +4.7% 44340 stress-ng.time.system_time 4468 ± 2% -15.0% 3800 ± 7% stress-ng.time.user_time 1.629e+09 -3.2% 1.576e+09 ± 2% stress-ng.timerfd.ops 54286374 -3.2% 52528279 ± 2% stress-ng.timerfd.ops_per_sec 0.75 ±173% +33333.3% 250.75 ±149% interrupts.104:PCI-MSI.31981637-edge.i40e-eth0-TxRx-68 0.75 ±173% +33266.7% 250.25 ±150% interrupts.CPU68.104:PCI-MSI.31981637-edge.i40e-eth0-TxRx-68 15.59 ± 2% -1.8 13.80 ± 3% mpstat.cpu.all.idle% 3.17 ± 4% -0.7 2.48 ± 13% mpstat.cpu.all.iowait% 46863 ± 29% -21.0% 37005 ± 30% proc-vmstat.nr_kernel_stack 6181473 ± 3% -60.5% 2442698 ± 88% proc-vmstat.pgreuse 6.755e+08 ± 8% -27.7% 4.883e+08 ± 16% cpuidle.C1.time 1.461e+08 ± 42% -50.0% 73023084 ± 7% cpuidle.POLL.time 12133129 ± 45% -87.8% 1477722 ± 8% cpuidle.POLL.usage 15.92 ± 2% -11.7% 14.06 ± 3% iostat.cpu.idle 3.18 ± 4% -22.0% 2.48 ± 12% iostat.cpu.iowait 76.37 +3.6% 79.12 iostat.cpu.system 15.25 ± 2% -11.5% 13.50 ± 3% vmstat.cpu.id 76.25 +3.0% 78.50 vmstat.cpu.sy 129.50 ± 59% -96.9% 4.00 ± 25% vmstat.procs.b 77813 ± 2% -10.5% 69667 ± 7% softirqs.CPU25.SCHED 81392 ± 7% -15.5% 68815 ± 8% softirqs.CPU41.SCHED 78946 ± 2% -10.3% 70795 ± 6% softirqs.CPU76.SCHED 78599 ± 2% -12.4% 68835 ± 8% softirqs.CPU77.SCHED 81487 ± 6% -14.0% 70042 ± 9% softirqs.CPU83.SCHED 67827 ± 16% -31.1% 46745 ± 2% slabinfo.Acpi-State.active_objs 1370 ± 17% -32.8% 920.25 ± 2% slabinfo.Acpi-State.active_slabs 69916 ± 17% -32.8% 46969 ± 2% slabinfo.Acpi-State.num_objs 1370 ± 17% -32.8% 920.25 ± 2% slabinfo.Acpi-State.num_slabs 810.00 ± 6% +17.8% 954.00 ± 3% slabinfo.kmem_cache_node.active_objs 847.50 ± 6% +17.0% 991.50 ± 3% slabinfo.kmem_cache_node.num_objs 5228 -16.6% 4359 ± 9% slabinfo.sighand_cache.active_objs 5314 -15.0% 4516 ± 8% slabinfo.sighand_cache.num_objs 50134 ± 42% -70.9% 14578 ± 46% sched_debug.cfs_rq:/.MIN_vruntime.avg 2621056 ± 41% -46.6% 1399499 ± 46% sched_debug.cfs_rq:/.MIN_vruntime.max 345309 ± 38% -58.9% 142092 ± 46% sched_debug.cfs_rq:/.MIN_vruntime.stddev 50134 ± 42% -70.9% 14578 ± 46% sched_debug.cfs_rq:/.max_vruntime.avg 2621056 ± 41% -46.6% 1399499 ± 46% sched_debug.cfs_rq:/.max_vruntime.max 345309 ± 38% -58.9% 142092 ± 46% sched_debug.cfs_rq:/.max_vruntime.stddev 139637 ± 19% +35.2% 188810 ± 14% sched_debug.cpu.avg_idle.min 5696 ± 33% -57.4% 2425 ± 58% sched_debug.cpu.max_idle_balance_cost.stddev 575.75 -83.2% 96.71 ± 16% sched_debug.cpu.nr_uninterruptible.max -567.52 -73.4% -150.73 sched_debug.cpu.nr_uninterruptible.min 304.20 ± 8% -88.5% 35.11 ± 18% sched_debug.cpu.nr_uninterruptible.stddev 286124 ± 3% -13.0% 248853 ± 9% sched_debug.cpu.sched_goidle.avg 250037 ± 2% -14.0% 214914 ± 8% sched_debug.cpu.sched_goidle.min 1.051e+08 -6.7% 98127607 ± 4% perf-stat.i.branch-misses 63185666 ± 7% -16.5% 52728923 ± 15% perf-stat.i.cache-misses 2.133e+11 +3.0% 2.196e+11 perf-stat.i.cpu-cycles 5687177 ± 2% -5.4% 5377921 ± 3% perf-stat.i.dTLB-store-misses 8.732e+09 -4.0% 8.379e+09 perf-stat.i.dTLB-stores 73450472 -4.3% 70273743 perf-stat.i.iTLB-load-misses 0.26 -5.7% 0.25 ± 2% perf-stat.i.ipc 2.21 +3.0% 2.28 perf-stat.i.metric.GHz 532333 -9.8% 480107 ± 6% perf-stat.i.page-faults 5.10 ± 4% -13.4% 4.42 ± 11% perf-stat.overall.MPKI 0.94 -0.0 0.92 perf-stat.overall.branch-miss-rate% 70.77 -6.0 64.73 ± 4% perf-stat.overall.node-load-miss-rate% 83.74 -1.1 82.62 perf-stat.overall.node-store-miss-rate% 1.018e+08 -6.6% 95063940 ± 4% perf-stat.ps.branch-misses 71426753 ± 6% -13.6% 61704660 ± 12% perf-stat.ps.cache-misses 2.156e+11 +2.8% 2.216e+11 perf-stat.ps.cpu-cycles 8.315e+09 -4.1% 7.972e+09 ± 2% perf-stat.ps.dTLB-stores 69988088 -4.5% 66854914 perf-stat.ps.iTLB-load-misses Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 7 months

3
2
0 / 0

[drm/dp] 6509ca051a: PANIC:double_fault

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 6509ca051abf4ff60d63732badcb2173a715f741 ("drm/dp: Add drm_dp_downstream_{min,max}_tmds_clock()") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: boot on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-------------------------------------------------------+------------+------------+ | | fe7cf496e5 | 6509ca051a | +-------------------------------------------------------+------------+------------+ | boot_successes | 4 | 0 | | boot_failures | 0 | 7 | | PANIC:double_fault | 0 | 7 | | double_fault:#[##] | 0 | 7 | | EIP:handle_exception | 0 | 7 | | Kernel_panic-not_syncing:Fatal_exception_in_interrupt | 0 | 7 | +-------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 15.006925] Applicom driver: $Id: ac.c,v 1.30 2000/03/22 16:03:57 dwmw2 Exp $ [ 15.007450] ac.o: No PCI boards found. [ 15.007720] ac.o: For an ISA board you must supply memory and irq parameters. [ 15.008278] toshiba: not a supported Toshiba laptop [ 15.010558] random: get_random_u32 called from arch_rnd+0x1c/0x40 with crng_init=0 [ 15.011048] traps: PANIC: double fault, error_code: 0x0 [ 15.011048] double fault: 0000 [#1] PREEMPT SMP PTI [ 15.011048] CPU: 1 PID: 147 Comm: modprobe Not tainted 5.9.0-rc4-00881-g6509ca051abf4 #1 [ 15.011048] EIP: handle_exception+0x0/0x128 [ 15.011048] Code: 24 10 36 89 48 fc 8b 4c 24 0c 81 e1 ff ff 00 00 36 89 48 f8 8b 4c 24 08 36 89 48 f4 8b 4c 24 04 36 89 48 f0 59 8d 60 f0 58 cf <fc> 0f a0 50 b8 d8 00 00 00 8e e0 58 81 64 24 10 ff ff 00 00 f7 44 [ 15.011048] EAX: 2262d000 EBX: ffa2efbc ECX: 00000000 EDX: c28dacf6 [ 15.011048] ESI: e2629ff0 EDI: ffa2f000 EBP: 00000000 ESP: ffa2e008 [ 15.011048] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010006 [ 15.011048] CR0: 80050033 CR2: ffa2dffc CR3: 02db2000 CR4: 000406b0 [ 15.011048] Call Trace: [ 15.011048] <ENTRY_TRAMPOLINE> [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80/0x80 [ 15.011048] ? sysvec_kvm_asyncpf_interrupt+0x50/0x50 [ 15.011048] ? restore_all_switch_stack+0x80 [ 15.011048] Lost 217 message(s)! [ 15.011048] ---[ end trace 94527bd98e02a7a3 ]--- [ 15.011048] EIP: handle_exception+0x0/0x128 [ 15.011048] Code: 24 10 36 89 48 fc 8b 4c 24 0c 81 e1 ff ff 00 00 36 89 48 f8 8b 4c 24 08 36 89 48 f4 8b 4c 24 04 36 89 48 f0 59 8d 60 f0 58 cf <fc> 0f a0 50 b8 d8 00 00 00 8e e0 58 81 64 24 10 ff ff 00 00 f7 44 [ 15.011048] EAX: 2262d000 EBX: ffa2efbc ECX: 00000000 EDX: c28dacf6 [ 15.011048] ESI: e2629ff0 EDI: ffa2f000 EBP: 00000000 ESP: ffa2e008 [ 15.011048] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010006 [ 15.011048] CR0: 80050033 CR2: ffa2dffc CR3: 02db2000 CR4: 000406b0 [ 15.011048] Kernel panic - not syncing: Fatal exception in interrupt [ 15.011048] Kernel Offset: disabled Kboot worker: lkp-worker30 Elapsed time: 60 To reproduce: # build kernel cd linux cp config-5.9.0-rc4-00881-g6509ca051abf4 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

1 year, 7 months

3
3
0 / 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP September 2020