November 2021 - LKP - Ml01.01.Org

[drm] d1af5cd869: BUG:kernel_NULL_pointer_dereference,address

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: d1af5cd86997d53c140a5abdced40c5e45d68e34 ("[PATCH] drm: get rid of DRM_DEBUG_* log calls in drm core, files drm_a*.c") url: https://github.com/0day-ci/linux/commits/Claudio-Suarez/drm-get-rid-of-DR... base: git://anongit.freedesktop.org/drm/drm drm-next patch link: https://lore.kernel.org/dri-devel/YaC7zXW119tlzfVh@gineta.localdomain in testcase: boot on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 4G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +---------------------------------------------+------------+------------+ | | c18c889111 | d1af5cd869 | +---------------------------------------------+------------+------------+ | boot_successes | 13 | 0 | | boot_failures | 0 | 15 | | BUG:kernel_NULL_pointer_dereference,address | 0 | 15 | | Oops:#[##] | 0 | 15 | | EIP:drm_atomic_helper_check_plane_state | 0 | 15 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 15 | +---------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 125.561383][ T1] BUG: kernel NULL pointer dereference, address: 00000010 [ 125.562724][ T1] #PF: supervisor read access in kernel mode [ 125.563784][ T1] #PF: error_code(0x0000) - not-present page [ 125.564418][ T1] *pde = 00000000 [ 125.564418][ T1] Oops: 0000 [#1] PREEMPT SMP [ 125.564418][ T1] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.16.0-rc2-00259-gd1af5cd86997 #1 [ 125.564418][ T1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 125.564418][ T1] EIP: drm_atomic_helper_check_plane_state (drivers/gpu/drm/drm_atomic_helper.c:867) [ 125.564418][ T1] Code: 45 d4 50 89 f0 e8 c6 8a 00 00 5a 80 7b 6c 00 74 6f 80 7d cc 00 75 69 8b 45 e4 39 43 5c 75 08 8b 45 ec 39 43 64 74 49 8b 43 04 <8b> 00 85 c0 74 03 8b 40 08 68 d3 54 71 c2 6a 04 50 e8 50 0d 04 00 All code ======== 0: 45 d4 rex.RB (bad) 2: 50 push %rax 3: 89 f0 mov %esi,%eax 5: e8 c6 8a 00 00 callq 0x8ad0 a: 5a pop %rdx b: 80 7b 6c 00 cmpb $0x0,0x6c(%rbx) f: 74 6f je 0x80 11: 80 7d cc 00 cmpb $0x0,-0x34(%rbp) 15: 75 69 jne 0x80 17: 8b 45 e4 mov -0x1c(%rbp),%eax 1a: 39 43 5c cmp %eax,0x5c(%rbx) 1d: 75 08 jne 0x27 1f: 8b 45 ec mov -0x14(%rbp),%eax 22: 39 43 64 cmp %eax,0x64(%rbx) 25: 74 49 je 0x70 27: 8b 43 04 mov 0x4(%rbx),%eax 2a:* 8b 00 mov (%rax),%eax <-- trapping instruction 2c: 85 c0 test %eax,%eax 2e: 74 03 je 0x33 30: 8b 40 08 mov 0x8(%rax),%eax 33: 68 d3 54 71 c2 pushq $0xffffffffc27154d3 38: 6a 04 pushq $0x4 3a: 50 push %rax 3b: e8 50 0d 04 00 callq 0x40d90 Code starting with the faulting instruction =========================================== 0: 8b 00 mov (%rax),%eax 2: 85 c0 test %eax,%eax 4: 74 03 je 0x9 6: 8b 40 08 mov 0x8(%rax),%eax 9: 68 d3 54 71 c2 pushq $0xffffffffc27154d3 e: 6a 04 pushq $0x4 10: 50 push %rax 11: e8 50 0d 04 00 callq 0x40d66 [ 125.564418][ T1] EAX: 00000010 EBX: c036fce8 ECX: 08000000 EDX: 00000001 [ 125.564418][ T1] ESI: c036fd34 EDI: 00010000 EBP: c036fcd4 ESP: c036fca0 [ 125.564418][ T1] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00010287 [ 125.564418][ T1] CR0: 80050033 CR2: 00000010 CR3: 02f31000 CR4: 000406d0 [ 125.564418][ T1] Call Trace: [ 125.564418][ T1] igt_check_plane_state (drivers/gpu/drm/selftests/test-drm_plane_helper.c:131 (discriminator 2)) [ 125.564418][ T1] ? test_drm_mm_init (drivers/gpu/drm/selftests/test-drm_modeset_common.c:16) [ 125.564418][ T1] test_drm_modeset_init (drivers/gpu/drm/selftests/drm_selftest.c:77 drivers/gpu/drm/selftests/test-drm_modeset_common.c:19) [ 125.564418][ T1] do_one_initcall (init/main.c:1297) [ 125.564418][ T1] ? __this_cpu_preempt_check (lib/smp_processor_id.c:67) [ 125.564418][ T1] ? lock_is_held_type (kernel/locking/lockdep.c:438 kernel/locking/lockdep.c:5681) [ 125.564418][ T1] kernel_init_freeable (init/main.c:1369 init/main.c:1386 init/main.c:1405 init/main.c:1610) [ 125.564418][ T1] ? rest_init (init/main.c:1491) [ 125.564418][ T1] kernel_init (init/main.c:1501) [ 125.564418][ T1] ret_from_fork (arch/x86/entry/entry_32.S:775) [ 125.564418][ T1] Modules linked in: [ 125.564418][ T1] CR2: 0000000000000010 [ 125.564418][ T1] random: get_random_bytes called from print_oops_end_marker+0x2c/0x80 with crng_init=0 [ 125.564418][ T1] ---[ end trace 9f868d4c92c9c57f ]--- [ 125.564418][ T1] EIP: drm_atomic_helper_check_plane_state (drivers/gpu/drm/drm_atomic_helper.c:867) [ 125.564418][ T1] Code: 45 d4 50 89 f0 e8 c6 8a 00 00 5a 80 7b 6c 00 74 6f 80 7d cc 00 75 69 8b 45 e4 39 43 5c 75 08 8b 45 ec 39 43 64 74 49 8b 43 04 <8b> 00 85 c0 74 03 8b 40 08 68 d3 54 71 c2 6a 04 50 e8 50 0d 04 00 All code ======== 0: 45 d4 rex.RB (bad) 2: 50 push %rax 3: 89 f0 mov %esi,%eax 5: e8 c6 8a 00 00 callq 0x8ad0 a: 5a pop %rdx b: 80 7b 6c 00 cmpb $0x0,0x6c(%rbx) f: 74 6f je 0x80 11: 80 7d cc 00 cmpb $0x0,-0x34(%rbp) 15: 75 69 jne 0x80 17: 8b 45 e4 mov -0x1c(%rbp),%eax 1a: 39 43 5c cmp %eax,0x5c(%rbx) 1d: 75 08 jne 0x27 1f: 8b 45 ec mov -0x14(%rbp),%eax 22: 39 43 64 cmp %eax,0x64(%rbx) 25: 74 49 je 0x70 27: 8b 43 04 mov 0x4(%rbx),%eax 2a:* 8b 00 mov (%rax),%eax <-- trapping instruction 2c: 85 c0 test %eax,%eax 2e: 74 03 je 0x33 30: 8b 40 08 mov 0x8(%rax),%eax 33: 68 d3 54 71 c2 pushq $0xffffffffc27154d3 38: 6a 04 pushq $0x4 3a: 50 push %rax 3b: e8 50 0d 04 00 callq 0x40d90 Code starting with the faulting instruction =========================================== 0: 8b 00 mov (%rax),%eax 2: 85 c0 test %eax,%eax 4: 74 03 je 0x9 6: 8b 40 08 mov 0x8(%rax),%eax 9: 68 d3 54 71 c2 pushq $0xffffffffc27154d3 e: 6a 04 pushq $0x4 10: 50 push %rax 11: e8 50 0d 04 00 callq 0x40d66 To reproduce: # build kernel cd linux cp config-5.16.0-rc2-00259-gd1af5cd86997 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month

3
8
0 / 0

[ucounts] dc7e5f9d41: UBSAN:array-index-out-of-bounds_in_kernel/ucount.c

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: dc7e5f9d419cb31b7751e87cf576f23a0153147c ("[PATCH v1 2/2] ucounts: Move rlimit max values from ucounts max") url: https://github.com/0day-ci/linux/commits/Alexey-Gladkov/ucounts-Fix-rlimi... base: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git 136057256686de39cc3a07c2e39ef6bc43003ff6 in testcase: boot on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 4G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +--------------------------------------------------------------------+------------+------------+ | | 98e4b47106 | dc7e5f9d41 | +--------------------------------------------------------------------+------------+------------+ | boot_successes | 32 | 0 | | boot_failures | 20 | 55 | | UBSAN:array-index-out-of-bounds_in_kernel/ucount.c | 0 | 36 | +--------------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 2.559359][ T1] UBSAN: array-index-out-of-bounds in kernel/ucount.c:109:33 [ 2.561796][ T1] index 13 is out of range for type 'long int [12]' [ 2.563347][ T1] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.16.0-rc2-00002-gdc7e5f9d419c #1 [ 2.565651][ T1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 2.567267][ T1] Call Trace: [ 2.567267][ T1] dump_stack_lvl (lib/dump_stack.c:107) [ 2.567267][ T1] dump_stack (lib/dump_stack.c:114) [ 2.567267][ T1] ubsan_epilogue (lib/ubsan.c:152) [ 2.567267][ T1] __ubsan_handle_out_of_bounds.cold (lib/ubsan.c:291 lib/ubsan.c:278) [ 2.567267][ T1] ? kmemdup (mm/util.c:132) [ 2.567267][ T1] setup_userns_sysctls (kernel/ucount.c:109) [ 2.567267][ T1] ? idle_threads_init (kernel/ucount.c:359) [ 2.567267][ T1] user_namespace_sysctl_init (kernel/ucount.c:371 (discriminator 2)) [ 2.567267][ T1] ? idle_threads_init (kernel/ucount.c:359) [ 2.567267][ T1] do_one_initcall (init/main.c:1297) [ 2.567267][ T1] ? __this_cpu_preempt_check (lib/smp_processor_id.c:67) [ 2.567267][ T1] ? lock_is_held_type (kernel/locking/lockdep.c:438 kernel/locking/lockdep.c:5681) [ 2.567267][ T1] ? rcu_read_lock_sched_held (include/linux/lockdep.h:283 kernel/rcu/update.c:125) [ 2.567267][ T1] kernel_init_freeable (init/main.c:1369 init/main.c:1386 init/main.c:1405 init/main.c:1610) [ 2.567267][ T1] ? rest_init (init/main.c:1491) [ 2.567267][ T1] kernel_init (init/main.c:1501) [ 2.567267][ T1] ret_from_fork (arch/x86/entry/entry_32.S:775) [ 2.567347][ T1] ================================================================================ [ 2.570153][ T1] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 2.571601][ T1] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 2.579937][ T1] ACPI: Added _OSI(Module Device) [ 2.581413][ T1] ACPI: Added _OSI(Processor Device) [ 2.582830][ T1] ACPI: Added _OSI(3.0 _SCP Extensions) [ 2.583285][ T1] ACPI: Added _OSI(Processor Aggregator Device) [ 2.584940][ T1] ACPI: Added _OSI(Linux-Dell-Video) [ 2.586372][ T1] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 2.587290][ T1] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) [ 2.596108][ T1] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 2.599386][ T9] Callback from call_rcu_tasks() invoked. [ 2.605503][ T1] ACPI: Interpreter enabled [ 2.606945][ T1] ACPI: PM: (supports S0 S3 S4 S5) [ 2.607284][ T1] ACPI: Using IOAPIC for interrupt routing [ 2.609116][ T1] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 2.612350][ T1] ACPI: Enabled 2 GPEs in block 00 to 0F [ 2.643022][ T1] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 2.643298][ T1] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 2.645668][ T1] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 2.647392][ T1] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 2.654525][ T1] acpiphp: Slot [3] registered [ 2.655430][ T1] acpiphp: Slot [4] registered [ 2.656809][ T1] acpiphp: Slot [5] registered [ 2.658207][ T1] acpiphp: Slot [6] registered [ 2.659411][ T1] acpiphp: Slot [7] registered [ 2.660914][ T1] acpiphp: Slot [8] registered [ 2.662430][ T1] acpiphp: Slot [9] registered [ 2.663430][ T1] acpiphp: Slot [10] registered [ 2.664892][ T1] acpiphp: Slot [11] registered [ 2.666269][ T1] acpiphp: Slot [12] registered [ 2.667416][ T1] acpiphp: Slot [13] registered [ 2.669191][ T1] acpiphp: Slot [14] registered [ 2.670666][ T1] acpiphp: Slot [15] registered [ 2.671408][ T1] acpiphp: Slot [16] registered [ 2.672829][ T1] acpiphp: Slot [17] registered [ 2.674311][ T1] acpiphp: Slot [18] registered [ 2.675427][ T1] acpiphp: Slot [19] registered [ 2.676842][ T1] acpiphp: Slot [20] registered [ 2.678323][ T1] acpiphp: Slot [21] registered [ 2.679419][ T1] acpiphp: Slot [22] registered [ 2.680822][ T1] acpiphp: Slot [23] registered [ 2.682305][ T1] acpiphp: Slot [24] registered [ 2.683418][ T1] acpiphp: Slot [25] registered [ 2.684928][ T1] acpiphp: Slot [26] registered [ 2.686387][ T1] acpiphp: Slot [27] registered [ 2.687409][ T1] acpiphp: Slot [28] registered [ 2.688886][ T1] acpiphp: Slot [29] registered [ 2.690319][ T1] acpiphp: Slot [30] registered [ 2.691400][ T1] acpiphp: Slot [31] registered [ 2.692745][ T1] PCI host bridge to bus 0000:00 [ 2.694068][ T1] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 2.695285][ T1] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 2.697287][ T1] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 2.699286][ T1] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 2.701613][ T1] pci_bus 0000:00: root bus resource [mem 0x140000000-0x1bfffffff window] [ 2.703362][ T1] pci_bus 0000:00: root bus resource [bus 00-ff] [ 2.705158][ T1] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 2.708313][ T1] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 2.711484][ T1] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 2.718382][ T1] pci 0000:00:01.1: reg 0x20: [io 0xc040-0xc04f] [ 2.721586][ T1] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 2.723286][ T1] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 2.725270][ T1] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 2.727282][ T1] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 2.729813][ T1] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 2.731875][ T1] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 2.735311][ T1] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 2.738460][ T1] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 [ 2.742150][ T1] pci 0000:00:02.0: reg 0x10: [mem 0xfd000000-0xfdffffff pref] [ 2.747291][ T1] pci 0000:00:02.0: reg 0x18: [mem 0xfebf0000-0xfebf0fff] [ 2.757450][ T1] pci 0000:00:02.0: reg 0x30: [mem 0xfebe0000-0xfebeffff pref] [ 2.760228][ T1] pci 0000:00:03.0: [8086:100e] type 00 class 0x020000 [ 2.763282][ T1] pci 0000:00:03.0: reg 0x10: [mem 0xfebc0000-0xfebdffff] [ 2.766577][ T1] pci 0000:00:03.0: reg 0x14: [io 0xc000-0xc03f] [ 2.776442][ T1] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 2.780153][ T1] pci 0000:00:04.0: [8086:25ab] type 00 class 0x088000 [ 2.782788][ T1] pci 0000:00:04.0: reg 0x10: [mem 0xfebf1000-0xfebf100f] [ 2.789260][ T1] pci_bus 0000:00: on NUMA node 0 To reproduce: # build kernel cd linux cp config-5.16.0-rc2-00002-gdc7e5f9d419c .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month

3
6
0 / 0

[security] d3b04a4398: WARNING:at_crypto/kdf_sp800108.c:#crypto_kdf108_init

by kernel test robot

Greeting, FYI, we noticed the following commit (built with clang-14): commit: d3b04a4398fe8022c9ca4b5ac6ab08059334b180 ("security: DH - use KDF implementation from crypto API") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: boot on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 4G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------------------------+------------+------------+ | | d792134423 | d3b04a4398 | +------------------------------------------------------+------------+------------+ | boot_successes | 14 | 0 | | boot_failures | 0 | 16 | | WARNING:at_crypto/kdf_sp800108.c:#crypto_kdf108_init | 0 | 16 | | EIP:crypto_kdf108_init | 0 | 16 | +------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 18.648980][ T1] WARNING: CPU: 0 PID: 1 at crypto/kdf_sp800108.c:138 crypto_kdf108_init (crypto/kdf_sp800108.c:136) [ 18.649900][ T1] Modules linked in: [ 18.650289][ T1] CPU: 0 PID: 1 Comm: swapper Not tainted 5.16.0-rc1-00049-gd3b04a4398fe #1 [ 18.651136][ T1] EIP: crypto_kdf108_init (crypto/kdf_sp800108.c:136) [ 18.651676][ T1] Code: ff ff 0f 44 f8 89 da 83 c2 08 89 d8 e8 cf 72 fa fe 89 f0 e8 08 70 eb fe 85 ff 74 39 57 68 39 26 d3 b1 e8 e9 4c d8 fe 83 c4 08 <0f> 0b 89 f8 5e 5f 5b 5d c3 68 85 74 d6 b1 68 85 17 d6 b1 e8 85 15 All code ======== 0: ff (bad) 1: ff 0f decl (%rdi) 3: 44 f8 rex.R clc 5: 89 da mov %ebx,%edx 7: 83 c2 08 add $0x8,%edx a: 89 d8 mov %ebx,%eax c: e8 cf 72 fa fe callq 0xfffffffffefa72e0 11: 89 f0 mov %esi,%eax 13: e8 08 70 eb fe callq 0xfffffffffeeb7020 18: 85 ff test %edi,%edi 1a: 74 39 je 0x55 1c: 57 push %rdi 1d: 68 39 26 d3 b1 pushq $0xffffffffb1d32639 22: e8 e9 4c d8 fe callq 0xfffffffffed84d10 27: 83 c4 08 add $0x8,%esp 2a:* 0f 0b ud2 <-- trapping instruction 2c: 89 f8 mov %edi,%eax 2e: 5e pop %rsi 2f: 5f pop %rdi 30: 5b pop %rbx 31: 5d pop %rbp 32: c3 retq 33: 68 85 74 d6 b1 pushq $0xffffffffb1d67485 38: 68 85 17 d6 b1 pushq $0xffffffffb1d61785 3d: e8 .byte 0xe8 3e: 85 .byte 0x85 3f: 15 .byte 0x15 Code starting with the faulting instruction =========================================== 0: 0f 0b ud2 2: 89 f8 mov %edi,%eax 4: 5e pop %rsi 5: 5f pop %rdi 6: 5b pop %rbx 7: 5d pop %rbp 8: c3 retq 9: 68 85 74 d6 b1 pushq $0xffffffffb1d67485 e: 68 85 17 d6 b1 pushq $0xffffffffb1d61785 13: e8 .byte 0xe8 14: 85 .byte 0x85 15: 15 .byte 0x15 [ 18.653613][ T1] EAX: 0000003a EBX: 80000000 ECX: 00000000 EDX: b2cebcf0 [ 18.654334][ T1] ESI: b61c9310 EDI: fffffff4 EBP: b2cebdf0 ESP: b2cebde4 [ 18.655050][ T1] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00010296 [ 18.655814][ T1] CR0: 80050033 CR2: 00000000 CR3: 02341000 CR4: 00040690 [ 18.656516][ T1] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 [ 18.657216][ T1] DR6: fffe0ff0 DR7: 00000400 [ 18.657668][ T1] Call Trace: [ 18.658019][ T1] ? __initstub__kmod_pkcs8_key_parser__180_176_pkcs8_key_init6 (crypto/kdf_sp800108.c:148) [ 18.658857][ T1] __initstub__kmod_kdf_sp800108__96_148_crypto_kdf108_init6 (crypto/kdf_sp800108.c:148) [ 18.659980][ T1] do_one_initcall (init/main.c:1297) [ 18.660507][ T1] ? __initstub__kmod_pkcs8_key_parser__180_176_pkcs8_key_init6 (crypto/kdf_sp800108.c:148) [ 18.661375][ T1] do_initcall_level (init/main.c:1369) [ 18.661842][ T1] ? rest_init (init/main.c:1491) [ 18.662268][ T1] do_initcalls (init/main.c:1383) [ 18.662691][ T1] do_basic_setup (init/main.c:1406) [ 18.663144][ T1] kernel_init_freeable (init/main.c:1614) [ 18.663653][ T1] kernel_init (kernel/async.c:257 kernel/async.c:244 init/main.c:1501) [ 18.664089][ T1] ret_from_fork (??:?) [ 18.664544][ T1] irq event stamp: 175171 [ 18.664971][ T1] hardirqs last enabled at (175179): __up_console_sem (arch/x86/include/asm/irqflags.h:45 arch/x86/include/asm/irqflags.h:80 arch/x86/include/asm/irqflags.h:140 kernel/printk/printk.c:255) [ 18.665846][ T1] hardirqs last disabled at (175188): __up_console_sem (kernel/printk/printk.c:253) [ 18.666710][ T1] softirqs last enabled at (173350): do_softirq_own_stack (arch/x86/kernel/irq_32.c:60 arch/x86/kernel/irq_32.c:150) [ 18.667665][ T1] softirqs last disabled at (173341): do_softirq_own_stack (arch/x86/kernel/irq_32.c:60 arch/x86/kernel/irq_32.c:150) [ 18.668625][ T1] random: get_random_bytes called from __warn+0xc1/0x140 with crng_init=0 [ 18.668633][ T1] ---[ end trace b93351358662f7fd ]--- [ 18.670099][ T1] start plist test [ 18.671957][ T1] end plist test [ 18.673358][ T1] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 18.674227][ T1] ACPI: button: Power Button [PWRF] [ 18.818315][ T1] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled [ 18.819246][ T1] serial 00:05: GPIO lookup for consumer rs485-term [ 18.819988][ T1] serial 00:05: using ACPI for GPIO lookup [ 18.820589][ T1] acpi PNP0501:00: GPIO: looking up rs485-term-gpios [ 18.821240][ T1] acpi PNP0501:00: GPIO: looking up rs485-term-gpio [ 18.821876][ T1] serial 00:05: using lookup tables for GPIO lookup [ 18.822547][ T1] serial 00:05: No GPIO consumer rs485-term found [ 18.823340][ T1] 00:05: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 18.825623][ T1] serial 00:06: GPIO lookup for consumer rs485-term [ 18.826321][ T1] serial 00:06: using ACPI for GPIO lookup [ 18.826881][ T1] acpi PNP0501:01: GPIO: looking up rs485-term-gpios [ 18.827549][ T1] acpi PNP0501:01: GPIO: looking up rs485-term-gpio [ 18.828236][ T1] serial 00:06: using lookup tables for GPIO lookup [ 18.828923][ T1] serial 00:06: No GPIO consumer rs485-term found [ 18.829702][ T1] 00:06: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 18.836645][ T1] platform pc8736x_gpio.0: NatSemi pc8736x GPIO Driver Initializing [ 18.837502][ T1] platform pc8736x_gpio.0: no device found [ 18.838140][ T1] nsc_gpio initializing [ 18.845377][ T1] e1000: Intel(R) PRO/1000 Network Driver [ 18.846008][ T1] e1000: Copyright (c) 1999-2006 Intel Corporation. [ 19.127869][ T1] ACPI: _SB_.LNKC: Enabled at IRQ 11 [ 19.466405][ T1] e1000 0000:00:03.0 eth0: (PCI:33MHz:32-bit) 52:54:00:12:34:56 [ 19.467380][ T1] e1000 0000:00:03.0 eth0: Intel(R) PRO/1000 Network Connection [ 19.468499][ T1] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 19.475314][ T1] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 19.476051][ T1] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 19.487285][ T1] i2c_dev: i2c /dev entries driver [ 19.494003][ T3] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 19.497043][ T1] Driver for 1-wire Dallas network protocol. [ 19.511630][ T1] Driver 'memconsole' was unable to register with bus_type 'coreboot' because the bus was not initialized. [ 19.512985][ T1] SPI driver xlnx-slave-spi has no spi_device_id for xlnx,fpga-slave-serial [ 19.517806][ T1] NET: Registered PF_INET6 protocol family [ 19.518593][ T1] random: get_random_u32 called from neigh_hash_alloc+0x54/0xd0 with crng_init=0 [ 19.526205][ T1] Segment Routing with IPv6 [ 19.526785][ T1] In-situ OAM (IOAM) with IPv6 [ 19.529269][ T1] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver [ 19.536497][ T1] NET: Registered PF_PACKET protocol family [ 19.537223][ T1] 9pnet: Installing 9P2000 support [ 19.538234][ T1] ... APIC ID: 00000000 (0) [ 19.538743][ T1] ... APIC VERSION: 00050014 [ 19.539125][ T1] 0000000000000000000000000000000000000000000000000000000000000000 [ 19.539125][ T1] 0000000000000000000000000000000000000000000000000000000000001000 [ 19.539125][ T1] [ 19.542112][ T1] number of MP IRQ sources: 15. [ 19.542621][ T1] number of IO-APIC #0 registers: 24. [ 19.543182][ T1] testing the IO APIC....................... [ 19.544281][ T1] IO APIC #0...... [ 19.544687][ T1] .... register #00: 00000000 [ 19.545208][ T1] ....... : physical APIC id: 00 [ 19.545730][ T1] ....... : Delivery Type: 0 [ 19.546212][ T1] ....... : LTS : 0 [ 19.546682][ T1] .... register #01: 00170011 [ 19.547164][ T1] ....... : max redirection entries: 17 [ 19.547775][ T1] ....... : PRQ implemented: 0 [ 19.548308][ T1] ....... : IO APIC version: 11 [ 19.548840][ T1] .... register #02: 00000000 [ 19.549320][ T1] ....... : arbitration: 00 [ 19.549817][ T1] .... IRQ redirection table: [ 19.550293][ T1] IOAPIC 0: [ 19.550616][ T1] pin00, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.551513][ T1] pin01, enabled , edge , high, V(22), IRR(0), S(0), logical , D(0001), M(0) [ 19.552435][ T1] pin02, enabled , edge , high, V(30), IRR(0), S(0), logical , D(0001), M(0) [ 19.553416][ T1] pin03, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.554350][ T1] pin04, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.555264][ T1] pin05, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.556199][ T1] pin06, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.557158][ T1] pin07, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.558041][ T1] pin08, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.558974][ T1] pin09, enabled , level, high, V(20), IRR(0), S(0), logical , D(0001), M(0) [ 19.559951][ T1] pin0a, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.560874][ T1] pin0b, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.561787][ T1] pin0c, enabled , edge , high, V(21), IRR(0), S(0), logical , D(0001), M(0) [ 19.562724][ T1] pin0d, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) [ 19.563702][ T1] pin0e, disabled, edge , high, V(00), IRR(0), S(0), physical, D(0000), M(0) To reproduce: # build kernel cd linux cp config-5.16.0-rc1-00049-gd3b04a4398fe .config make HOSTCC=clang-14 CC=clang-14 ARCH=i386 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=clang-14 CC=clang-14 ARCH=i386 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month, 1 week

5
4
0 / 0

[x86/mm/tlb] 2f4305b19f: will-it-scale.per_thread_ops 23.3% improvement

by kernel test robot

Greeting, FYI, we noticed a 23.3% improvement of will-it-scale.per_thread_ops due to commit: commit: 2f4305b19fe6a2a261d76c21856c5598f7d878fe ("x86/mm/tlb: Privatize cpu_tlbstate") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 144 threads 4 sockets Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory with following parameters: nr_task: 100% mode: thread test: tlb_flush3 cpufreq_governor: performance ucode: 0x16 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/thread/100%/debian-10.4-x86_64-20200603.cgz/lkp-hsw-4ex1/tlb_flush3/will-it-scale/0x16 commit: 4ce94eabac ("x86/mm/tlb: Flush remote and local TLBs concurrently") 2f4305b19f ("x86/mm/tlb: Privatize cpu_tlbstate") 4ce94eabac16b1d2 2f4305b19fe6a2a261d76c21856 ---------------- --------------------------- %stddev %change %stddev \ | \ 402682 +23.3% 496636 will-it-scale.144.threads 1.80 +11.3% 2.00 ± 3% will-it-scale.144.threads_idle 2796 +23.3% 3448 will-it-scale.per_thread_ops 402682 +23.3% 496636 will-it-scale.workload 9.557e+08 +10.8% 1.059e+09 ± 3% cpuidle..time 5063246 +16.3% 5887664 ± 3% cpuidle..usage 8369 +18.1% 9888 vmstat.system.cs 5731465 +20.1% 6881040 vmstat.system.in 560168 +24.4% 696788 ± 15% numa-meminfo.node2.Unevictable 262269 ± 30% -31.6% 179352 ± 26% numa-meminfo.node3.Inactive 262269 ± 30% -31.6% 179352 ± 26% numa-meminfo.node3.Inactive(anon) 837035 -19.2% 676664 syscalls.sys_madvise.med 5.449e+09 -1.1e+09 4.393e+09 ± 7% syscalls.sys_madvise.noise.2% 5.214e+09 -1e+09 4.188e+09 ± 8% syscalls.sys_madvise.noise.5% 2.09 +0.3 2.35 ± 5% mpstat.cpu.all.idle% 27.65 +3.7 31.31 mpstat.cpu.all.irq% 0.02 +0.0 0.02 ± 2% mpstat.cpu.all.soft% 1.13 -0.1 1.01 ± 2% mpstat.cpu.all.usr% 1932284 ± 15% +42.9% 2761167 ± 12% sched_debug.cfs_rq:/.spread0.max -4519665 -31.5% -3095288 sched_debug.cfs_rq:/.spread0.min 11234 +14.5% 12860 sched_debug.cpu.nr_switches.avg 7851 +22.2% 9591 sched_debug.cpu.nr_switches.min 11.17 ± 6% +16.5% 13.01 ± 6% sched_debug.cpu.nr_uninterruptible.stddev 1026202 ± 2% +24.8% 1280794 ± 7% turbostat.C1 0.14 +0.0 0.18 ± 10% turbostat.C1% 3.618e+10 +23.1% 4.455e+10 turbostat.IRQ 533.78 +1.6% 542.52 turbostat.PkgWatt 91.85 +10.4% 101.40 turbostat.RAMWatt 140041 +24.4% 174196 ± 15% numa-vmstat.node2.nr_unevictable 140041 +24.4% 174196 ± 15% numa-vmstat.node2.nr_zone_unevictable 1855459 ± 6% +11.7% 2071737 ± 10% numa-vmstat.node2.numa_hit 1624284 ± 7% +15.8% 1881173 ± 12% numa-vmstat.node2.numa_local 65628 ± 30% -31.6% 44870 ± 26% numa-vmstat.node3.nr_inactive_anon 65628 ± 30% -31.6% 44870 ± 26% numa-vmstat.node3.nr_zone_inactive_anon 208421 -8.5% 190704 proc-vmstat.nr_inactive_anon 208421 -8.5% 190704 proc-vmstat.nr_zone_inactive_anon 6652316 +1.8% 6774779 proc-vmstat.numa_hit 6418896 +1.9% 6542164 proc-vmstat.numa_local 6769479 +1.9% 6900686 proc-vmstat.pgalloc_normal 1.391e+08 +21.5% 1.69e+08 proc-vmstat.pgfault 0.01 ± 10% -35.4% 0.01 ± 13% perf-sched.sch_delay.avg.ms.__traceiter_sched_switch.__traceiter_sched_switch.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop 0.21 ± 81% -66.6% 0.07 ± 5% perf-sched.sch_delay.avg.ms.__traceiter_sched_switch.__traceiter_sched_switch.schedule_timeout.kcompactd.kthread 0.02 ± 17% -24.5% 0.02 ± 2% perf-sched.sch_delay.avg.ms.__traceiter_sched_switch.__traceiter_sched_switch.smpboot_thread_fn.kthread.ret_from_fork 0.22 ± 16% -24.6% 0.17 ± 16% perf-sched.sch_delay.max.ms.__traceiter_sched_switch.__traceiter_sched_switch.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop 3.29 ±143% -93.8% 0.21 ± 8% perf-sched.sch_delay.max.ms.__traceiter_sched_switch.__traceiter_sched_switch.schedule_timeout.kcompactd.kthread 3.97 ±208% -94.9% 0.20 ± 4% perf-sched.sch_delay.max.ms.__traceiter_sched_switch.__traceiter_sched_switch.smpboot_thread_fn.kthread.ret_from_fork 0.49 ± 4% -15.8% 0.41 ± 16% perf-sched.wait_time.avg.ms.__traceiter_sched_switch.__traceiter_sched_switch.rwsem_down_write_slowpath.down_write_killable.__vm_munmap 2.34 ± 82% +124.8% 5.27 ± 9% perf-sched.wait_time.avg.ms.__traceiter_sched_switch.__traceiter_sched_switch.schedule_timeout.khugepaged.kthread 2.34 ± 82% +124.8% 5.27 ± 9% perf-sched.wait_time.max.ms.__traceiter_sched_switch.__traceiter_sched_switch.schedule_timeout.khugepaged.kthread 16.65 +4.9% 17.47 perf-stat.i.MPKI 1.297e+10 +13.6% 1.473e+10 perf-stat.i.branch-instructions 1.56 +0.1 1.68 perf-stat.i.branch-miss-rate% 2.009e+08 +22.3% 2.457e+08 perf-stat.i.branch-misses 35.84 -2.0 33.82 perf-stat.i.cache-miss-rate% 3.259e+08 +12.9% 3.679e+08 perf-stat.i.cache-misses 9.054e+08 +19.5% 1.082e+09 perf-stat.i.cache-references 8402 +17.6% 9881 perf-stat.i.context-switches 7.45 -12.5% 6.52 perf-stat.i.cpi 190.55 +2.6% 195.51 perf-stat.i.cpu-migrations 1262 -11.1% 1122 perf-stat.i.cycles-between-cache-misses 1.258e+08 ± 20% +36.6% 1.718e+08 ± 24% perf-stat.i.dTLB-load-misses 1.383e+10 +14.4% 1.583e+10 perf-stat.i.dTLB-loads 12368871 ± 12% +37.3% 16985856 ± 17% perf-stat.i.dTLB-store-misses 4.345e+09 +22.2% 5.31e+09 perf-stat.i.dTLB-stores 50649234 +31.9% 66823313 perf-stat.i.iTLB-load-misses 3363217 +24.8% 4197710 perf-stat.i.iTLB-loads 5.435e+10 +13.9% 6.192e+10 perf-stat.i.instructions 1167 ± 15% -19.1% 944.66 perf-stat.i.instructions-per-iTLB-miss 0.14 +13.7% 0.16 perf-stat.i.ipc 582.91 +11.8% 651.90 ± 4% perf-stat.i.metric.K/sec 224.35 +15.3% 258.60 perf-stat.i.metric.M/sec 427345 +23.1% 525869 perf-stat.i.minor-faults 2.58e+08 +14.6% 2.956e+08 perf-stat.i.node-load-misses 1722038 ± 2% -6.6% 1607885 ± 3% perf-stat.i.node-loads 71.40 +8.9 80.32 perf-stat.i.node-store-miss-rate% 46474868 +20.2% 55872747 perf-stat.i.node-store-misses 18493804 -26.7% 13564558 ± 2% perf-stat.i.node-stores 442923 +22.2% 541358 perf-stat.i.page-faults 16.66 +4.9% 17.48 perf-stat.overall.MPKI 1.55 +0.1 1.67 perf-stat.overall.branch-miss-rate% 35.99 -2.0 33.99 perf-stat.overall.cache-miss-rate% 7.47 -12.4% 6.54 perf-stat.overall.cpi 1246 -11.6% 1101 perf-stat.overall.cycles-between-cache-misses 1073 -13.6% 926.95 perf-stat.overall.instructions-per-iTLB-miss 0.13 +14.2% 0.15 perf-stat.overall.ipc 71.53 +8.9 80.47 perf-stat.overall.node-store-miss-rate% 40918438 -7.6% 37809036 perf-stat.overall.path-length 1.294e+10 +13.5% 1.469e+10 perf-stat.ps.branch-instructions 2.003e+08 +22.2% 2.449e+08 perf-stat.ps.branch-misses 3.25e+08 +12.8% 3.667e+08 perf-stat.ps.cache-misses 9.03e+08 +19.5% 1.079e+09 perf-stat.ps.cache-references 8318 +18.3% 9841 perf-stat.ps.context-switches 189.75 +2.7% 194.85 perf-stat.ps.cpu-migrations 1.255e+08 ± 20% +36.5% 1.713e+08 ± 24% perf-stat.ps.dTLB-load-misses 1.379e+10 +14.4% 1.578e+10 perf-stat.ps.dTLB-loads 12333464 ± 12% +37.3% 16931060 ± 17% perf-stat.ps.dTLB-store-misses 4.332e+09 +22.2% 5.293e+09 perf-stat.ps.dTLB-stores 50510702 +31.9% 66610583 perf-stat.ps.iTLB-load-misses 3350681 +24.9% 4183591 perf-stat.ps.iTLB-loads 5.42e+10 +13.9% 6.173e+10 perf-stat.ps.instructions 426148 +23.0% 524191 perf-stat.ps.minor-faults 2.573e+08 +14.6% 2.947e+08 perf-stat.ps.node-load-misses 1719740 ± 2% -6.5% 1607172 ± 3% perf-stat.ps.node-loads 46348066 +20.2% 55695238 perf-stat.ps.node-store-misses 18444055 -26.7% 13522869 ± 2% perf-stat.ps.node-stores 441672 +22.2% 539706 perf-stat.ps.page-faults 1.648e+13 +13.9% 1.878e+13 perf-stat.total.instructions 13523 ± 5% +10.4% 14924 ± 4% softirqs.CPU0.SCHED 8763 ± 2% +13.5% 9942 ± 11% softirqs.CPU101.SCHED 8942 ± 5% +9.4% 9782 ± 2% softirqs.CPU106.SCHED 9208 ± 5% +11.7% 10286 ± 7% softirqs.CPU110.RCU 8875 ± 4% +17.9% 10467 ± 11% softirqs.CPU111.RCU 8731 ± 2% +15.8% 10114 ± 6% softirqs.CPU111.SCHED 8749 +13.5% 9931 ± 4% softirqs.CPU112.SCHED 8753 +17.4% 10278 ± 5% softirqs.CPU113.SCHED 8801 +13.3% 9975 ± 3% softirqs.CPU114.SCHED 10306 ± 3% +9.4% 11279 ± 5% softirqs.CPU115.RCU 8758 ± 2% +13.0% 9898 ± 4% softirqs.CPU115.SCHED 10118 ± 6% +11.3% 11257 ± 5% softirqs.CPU116.RCU 8859 ± 4% +12.4% 9957 ± 3% softirqs.CPU116.SCHED 8776 +14.0% 10001 ± 3% softirqs.CPU117.SCHED 10271 ± 2% +11.2% 11418 ± 6% softirqs.CPU118.RCU 8758 +14.0% 9981 ± 4% softirqs.CPU118.SCHED 8816 ± 2% +13.0% 9959 ± 3% softirqs.CPU119.SCHED 8751 ± 2% +12.7% 9865 ± 3% softirqs.CPU120.SCHED 8705 +13.3% 9866 ± 2% softirqs.CPU122.SCHED 9937 ± 7% +10.5% 10984 ± 5% softirqs.CPU123.RCU 9962 ± 2% +12.2% 11174 ± 9% softirqs.CPU124.RCU 8761 +12.7% 9874 ± 3% softirqs.CPU124.SCHED 8827 +12.5% 9927 ± 6% softirqs.CPU125.SCHED 8933 ± 4% +11.4% 9954 ± 4% softirqs.CPU129.SCHED 8750 +10.9% 9706 ± 3% softirqs.CPU13.SCHED 8903 ± 6% +11.8% 9950 ± 3% softirqs.CPU135.SCHED 9873 ± 3% +7.6% 10619 ± 3% softirqs.CPU137.RCU 8913 ± 4% +10.1% 9811 ± 2% softirqs.CPU138.SCHED 8847 ± 3% +10.6% 9788 ± 2% softirqs.CPU139.SCHED 8824 +10.0% 9704 ± 3% softirqs.CPU19.SCHED 9041 +16.0% 10485 ± 8% softirqs.CPU2.SCHED 8724 +11.2% 9702 ± 2% softirqs.CPU22.SCHED 9026 ± 4% +10.7% 9993 ± 7% softirqs.CPU23.SCHED 8771 ± 2% +10.3% 9677 ± 2% softirqs.CPU28.SCHED 8075 ± 7% +16.3% 9395 ± 10% softirqs.CPU3.SCHED 8845 +10.4% 9761 ± 2% softirqs.CPU30.SCHED 8775 ± 2% +12.7% 9886 ± 2% softirqs.CPU31.SCHED 8915 ± 3% +11.5% 9940 ± 4% softirqs.CPU32.SCHED 8684 +11.1% 9651 ± 2% softirqs.CPU33.SCHED 8811 ± 2% +12.2% 9882 ± 2% softirqs.CPU35.SCHED 8907 ± 3% +11.3% 9915 ± 4% softirqs.CPU36.SCHED 8684 +13.7% 9871 ± 3% softirqs.CPU37.SCHED 8694 +13.0% 9823 ± 4% softirqs.CPU38.SCHED 8697 +12.5% 9783 ± 2% softirqs.CPU39.SCHED 8378 ± 6% +17.1% 9807 softirqs.CPU4.SCHED 10390 ± 4% +26.3% 13118 ± 29% softirqs.CPU43.RCU 8643 +15.8% 10007 ± 4% softirqs.CPU43.SCHED 10129 ± 7% +13.3% 11472 ± 5% softirqs.CPU44.RCU 10393 ± 3% +12.1% 11654 ± 8% softirqs.CPU45.RCU 8584 +17.2% 10060 ± 7% softirqs.CPU45.SCHED 8637 +14.5% 9888 ± 2% softirqs.CPU46.SCHED 8698 ± 3% +11.7% 9719 softirqs.CPU47.SCHED 8628 +13.9% 9831 ± 2% softirqs.CPU48.SCHED 8656 +11.5% 9656 ± 6% softirqs.CPU49.SCHED 9616 ± 2% +9.5% 10527 ± 4% softirqs.CPU50.RCU 8671 +11.7% 9683 ± 2% softirqs.CPU50.SCHED 8649 +11.4% 9633 softirqs.CPU51.SCHED 8635 ± 2% +13.1% 9767 ± 2% softirqs.CPU53.SCHED 11056 ± 3% +10.5% 12213 ± 5% softirqs.CPU55.RCU 8703 ± 2% +13.4% 9865 ± 5% softirqs.CPU55.SCHED 8918 ± 3% +9.8% 9793 ± 4% softirqs.CPU56.SCHED 8738 ± 2% +11.7% 9758 ± 2% softirqs.CPU58.SCHED 10208 ± 3% +10.3% 11264 ± 4% softirqs.CPU59.RCU 8755 ± 3% +9.9% 9623 ± 2% softirqs.CPU61.SCHED 9001 ± 3% +8.2% 9741 ± 2% softirqs.CPU63.SCHED 9794 ± 4% +10.4% 10811 ± 4% softirqs.CPU64.RCU 8824 ± 3% +10.0% 9706 ± 4% softirqs.CPU64.SCHED 9930 ± 5% +9.0% 10824 ± 3% softirqs.CPU65.RCU 8772 +10.8% 9722 ± 3% softirqs.CPU65.SCHED 9913 ± 2% +9.1% 10816 ± 3% softirqs.CPU66.RCU 8660 ± 4% +12.1% 9709 ± 2% softirqs.CPU66.SCHED 9909 ± 5% +8.8% 10783 ± 3% softirqs.CPU67.RCU 9763 ± 5% +9.3% 10673 ± 2% softirqs.CPU68.RCU 8797 ± 2% +9.9% 9671 ± 2% softirqs.CPU69.SCHED 8409 ± 9% +16.3% 9775 ± 3% softirqs.CPU71.SCHED 8700 ± 2% +13.6% 9883 ± 4% softirqs.CPU72.SCHED 8719 ± 3% +9.5% 9543 ± 3% softirqs.CPU75.SCHED 8803 ± 2% +10.2% 9697 ± 2% softirqs.CPU78.SCHED 8665 +15.5% 10011 ± 5% softirqs.CPU8.SCHED 8791 ± 2% +10.5% 9718 ± 2% softirqs.CPU84.SCHED 8789 ± 2% +12.4% 9877 ± 8% softirqs.CPU86.SCHED 8977 ± 3% +13.4% 10177 ± 7% softirqs.CPU92.SCHED 8839 +12.5% 9943 ± 3% softirqs.CPU93.SCHED 8779 ± 2% +11.2% 9766 ± 2% softirqs.CPU95.SCHED 8779 ± 3% +11.3% 9770 ± 5% softirqs.CPU96.SCHED 1280353 +10.4% 1412876 softirqs.SCHED 5.34 -3.7 1.66 ± 5% perf-profile.calltrace.cycles-pp.native_flush_tlb_multi.flush_tlb_mm_range.zap_pte_range.unmap_page_range.zap_page_range 49.36 -1.6 47.74 perf-profile.calltrace.cycles-pp.flush_tlb_mm_range.zap_pte_range.unmap_page_range.zap_page_range.do_madvise 49.76 -1.5 48.25 perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.zap_page_range.do_madvise.__x64_sys_madvise 49.84 -1.5 48.36 perf-profile.calltrace.cycles-pp.unmap_page_range.zap_page_range.do_madvise.__x64_sys_madvise.do_syscall_64 96.32 -0.3 96.01 perf-profile.calltrace.cycles-pp.__madvise 0.56 ± 2% +0.1 0.67 perf-profile.calltrace.cycles-pp.default_send_IPI_mask_sequence_phys.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.tlb_flush_mmu 0.60 +0.1 0.71 ± 2% perf-profile.calltrace.cycles-pp.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 0.65 +0.1 0.76 perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault 0.72 +0.1 0.84 ± 2% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 0.82 +0.1 0.94 ± 3% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.testcase 0.42 ± 44% +0.2 0.62 ± 2% perf-profile.calltrace.cycles-pp.default_send_IPI_mask_sequence_phys.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.zap_pte_range 0.00 +0.6 0.58 ± 2% perf-profile.calltrace.cycles-pp.cpumask_next.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.tlb_flush_mmu 0.00 +0.6 0.60 perf-profile.calltrace.cycles-pp.cpumask_next.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.zap_pte_range 21.53 +0.7 22.18 perf-profile.calltrace.cycles-pp.llist_add_batch.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.zap_pte_range 43.74 +0.9 44.63 perf-profile.calltrace.cycles-pp.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.tlb_flush_mmu.tlb_finish_mmu 43.92 +0.9 44.86 perf-profile.calltrace.cycles-pp.on_each_cpu_mask.flush_tlb_mm_range.tlb_flush_mmu.tlb_finish_mmu.zap_page_range 44.30 +1.1 45.38 perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.zap_page_range.do_madvise.__x64_sys_madvise 44.28 +1.1 45.36 perf-profile.calltrace.cycles-pp.flush_tlb_mm_range.tlb_flush_mmu.tlb_finish_mmu.zap_page_range.do_madvise 44.57 +1.1 45.70 perf-profile.calltrace.cycles-pp.tlb_finish_mmu.zap_page_range.do_madvise.__x64_sys_madvise.do_syscall_64 5.66 ± 2% +1.2 6.83 ± 2% perf-profile.calltrace.cycles-pp.asm_sysvec_call_function.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.zap_pte_range 5.48 +1.2 6.66 ± 3% perf-profile.calltrace.cycles-pp.asm_sysvec_call_function.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.tlb_flush_mmu 12.08 ± 3% +1.3 13.36 ± 3% perf-profile.calltrace.cycles-pp.flush_tlb_func.flush_smp_call_function_queue.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function 8.78 +1.6 10.36 perf-profile.calltrace.cycles-pp.llist_reverse_order.flush_smp_call_function_queue.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function 11.72 +1.7 13.38 perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function.llist_add_batch 11.80 +1.7 13.49 perf-profile.calltrace.cycles-pp.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function.llist_add_batch.smp_call_function_many_cond 12.54 +1.7 14.26 perf-profile.calltrace.cycles-pp.asm_sysvec_call_function.llist_add_batch.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range 11.96 +1.7 13.69 perf-profile.calltrace.cycles-pp.sysvec_call_function.asm_sysvec_call_function.llist_add_batch.smp_call_function_many_cond.on_each_cpu_mask 43.42 +1.9 45.28 perf-profile.calltrace.cycles-pp.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range.zap_pte_range.unmap_page_range 43.61 +1.9 45.50 perf-profile.calltrace.cycles-pp.on_each_cpu_mask.flush_tlb_mm_range.zap_pte_range.unmap_page_range.zap_page_range 10.64 +2.2 12.87 ± 3% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function.smp_call_function_many_cond 10.70 +2.3 12.96 ± 3% perf-profile.calltrace.cycles-pp.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function.smp_call_function_many_cond.on_each_cpu_mask 10.85 +2.3 13.14 ± 3% perf-profile.calltrace.cycles-pp.sysvec_call_function.asm_sysvec_call_function.smp_call_function_many_cond.on_each_cpu_mask.flush_tlb_mm_range 5.41 -3.7 1.71 ± 5% perf-profile.children.cycles-pp.native_flush_tlb_multi 49.82 -1.5 48.32 perf-profile.children.cycles-pp.zap_pte_range 49.91 -1.5 48.42 perf-profile.children.cycles-pp.unmap_page_range 3.06 -1.1 1.93 perf-profile.children.cycles-pp.native_flush_tlb_local 93.66 -0.5 93.12 perf-profile.children.cycles-pp.flush_tlb_mm_range 0.14 ± 4% -0.1 0.02 ± 99% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.11 ± 4% -0.1 0.05 perf-profile.children.cycles-pp.__irqentry_text_end 0.26 -0.0 0.23 ± 4% perf-profile.children.cycles-pp.irqentry_exit_to_user_mode 0.32 ± 2% -0.0 0.30 ± 2% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.10 +0.0 0.11 ± 3% perf-profile.children.cycles-pp.syscall_trace_enter 0.10 ± 6% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.native_sched_clock 0.07 +0.0 0.09 perf-profile.children.cycles-pp.unlock_page 0.10 ± 7% +0.0 0.13 ± 2% perf-profile.children.cycles-pp.native_apic_mem_write 0.11 ± 7% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.sched_clock 0.32 ± 2% +0.0 0.34 ± 3% perf-profile.children.cycles-pp.sysvec_call_function_single 0.31 ± 3% +0.0 0.34 ± 2% perf-profile.children.cycles-pp.__sysvec_call_function_single 0.33 ± 2% +0.0 0.36 ± 2% perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.14 ± 4% +0.0 0.17 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.25 ± 3% +0.0 0.29 perf-profile.children.cycles-pp.irq_exit_rcu 0.27 ± 2% +0.0 0.32 ± 3% perf-profile.children.cycles-pp.next_uptodate_page 0.21 +0.0 0.26 perf-profile.children.cycles-pp.find_next_bit 0.21 ± 3% +0.0 0.25 perf-profile.children.cycles-pp.irqtime_account_irq 0.01 ±223% +0.1 0.06 ± 16% perf-profile.children.cycles-pp.flush_tlb_batched_pending 0.17 ± 2% +0.1 0.24 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.30 ± 2% +0.1 0.40 perf-profile.children.cycles-pp.__x86_retpoline_rax 0.61 +0.1 0.72 ± 2% perf-profile.children.cycles-pp.filemap_map_pages 0.65 +0.1 0.76 ± 2% perf-profile.children.cycles-pp.do_fault 0.73 +0.1 0.85 ± 2% perf-profile.children.cycles-pp.__handle_mm_fault 0.84 +0.1 0.96 ± 3% perf-profile.children.cycles-pp.handle_mm_fault 0.59 +0.1 0.72 perf-profile.children.cycles-pp.__default_send_IPI_dest_field 1.08 +0.2 1.31 perf-profile.children.cycles-pp.default_send_IPI_mask_sequence_phys 0.86 ± 5% +0.3 1.15 ± 9% perf-profile.children.cycles-pp._find_next_bit 3.09 ± 2% +0.3 3.40 ± 7% perf-profile.children.cycles-pp.testcase 1.31 ± 3% +0.4 1.72 ± 4% perf-profile.children.cycles-pp.cpumask_next 44.37 +1.1 45.44 perf-profile.children.cycles-pp.tlb_flush_mmu 44.58 +1.1 45.71 perf-profile.children.cycles-pp.tlb_finish_mmu 10.14 +1.6 11.69 perf-profile.children.cycles-pp.llist_reverse_order 14.14 ± 3% +1.9 16.04 ± 2% perf-profile.children.cycles-pp.flush_tlb_func 87.54 +2.8 90.36 perf-profile.children.cycles-pp.on_each_cpu_mask 87.54 +2.8 90.36 perf-profile.children.cycles-pp.smp_call_function_many_cond 26.48 +3.7 30.20 perf-profile.children.cycles-pp.flush_smp_call_function_queue 26.31 +3.7 30.04 perf-profile.children.cycles-pp.__sysvec_call_function 26.70 +3.8 30.50 perf-profile.children.cycles-pp.sysvec_call_function 27.75 +3.8 31.59 perf-profile.children.cycles-pp.asm_sysvec_call_function 3.65 -2.8 0.90 perf-profile.self.cycles-pp.native_flush_tlb_multi 31.33 -1.4 29.96 perf-profile.self.cycles-pp.llist_add_batch 3.04 -1.1 1.90 perf-profile.self.cycles-pp.native_flush_tlb_local 0.11 ± 4% -0.1 0.05 perf-profile.self.cycles-pp.__irqentry_text_end 0.26 -0.0 0.23 ± 4% perf-profile.self.cycles-pp.irqentry_exit_to_user_mode 0.21 ± 2% -0.0 0.18 ± 2% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.10 +0.0 0.11 ± 3% perf-profile.self.cycles-pp.error_entry 0.09 +0.0 0.10 ± 4% perf-profile.self.cycles-pp.sysvec_call_function 0.07 ± 6% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.irqtime_account_irq 0.09 +0.0 0.11 ± 6% perf-profile.self.cycles-pp.filemap_map_pages 0.19 +0.0 0.21 ± 3% perf-profile.self.cycles-pp.next_uptodate_page 0.10 ± 5% +0.0 0.12 ± 4% perf-profile.self.cycles-pp.native_sched_clock 0.10 ± 5% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.native_apic_mem_write 0.16 ± 2% +0.0 0.20 ± 2% perf-profile.self.cycles-pp.find_next_bit 0.12 ± 5% +0.0 0.16 ± 7% perf-profile.self.cycles-pp.zap_pte_range 0.16 ± 2% +0.0 0.20 ± 2% perf-profile.self.cycles-pp.__x86_retpoline_rax 0.18 ± 2% +0.1 0.24 perf-profile.self.cycles-pp.cpumask_next 0.13 ± 2% +0.1 0.19 perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.42 ± 2% +0.1 0.48 ± 2% perf-profile.self.cycles-pp.testcase 0.59 +0.1 0.72 perf-profile.self.cycles-pp.__default_send_IPI_dest_field 2.43 +0.2 2.58 perf-profile.self.cycles-pp.flush_smp_call_function_queue 0.54 ± 6% +0.2 0.70 ± 10% perf-profile.self.cycles-pp._find_next_bit 0.48 ± 6% +0.2 0.67 ± 4% perf-profile.self.cycles-pp.flush_tlb_mm_range 10.13 +1.6 11.69 perf-profile.self.cycles-pp.llist_reverse_order 11.02 ± 3% +3.0 14.03 ± 2% perf-profile.self.cycles-pp.flush_tlb_func 1.664e+09 +21.2% 2.017e+09 interrupts.CAL:Function_call_interrupts 3711325 +20.1% 4455927 interrupts.CPU0.CAL:Function_call_interrupts 2.386e+08 +23.5% 2.947e+08 interrupts.CPU0.TLB:TLB_shootdowns 3744563 +19.9% 4489768 interrupts.CPU1.CAL:Function_call_interrupts 2.388e+08 +23.2% 2.942e+08 interrupts.CPU1.TLB:TLB_shootdowns 4014451 +20.6% 4842627 interrupts.CPU10.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.949e+08 interrupts.CPU10.TLB:TLB_shootdowns 11478870 +20.9% 13882134 interrupts.CPU100.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.949e+08 interrupts.CPU100.TLB:TLB_shootdowns 11701370 +20.8% 14131979 interrupts.CPU101.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU101.TLB:TLB_shootdowns 11923501 +20.8% 14397900 interrupts.CPU102.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU102.TLB:TLB_shootdowns 12163111 +20.7% 14685189 interrupts.CPU103.CAL:Function_call_interrupts 2.391e+08 +23.4% 2.95e+08 interrupts.CPU103.TLB:TLB_shootdowns 12447537 +20.4% 14986287 interrupts.CPU104.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU104.TLB:TLB_shootdowns 12722214 +20.2% 15297802 interrupts.CPU105.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.949e+08 interrupts.CPU105.TLB:TLB_shootdowns 12990125 +20.3% 15627081 interrupts.CPU106.CAL:Function_call_interrupts 2.391e+08 +23.4% 2.949e+08 interrupts.CPU106.TLB:TLB_shootdowns 13296299 +20.2% 15982711 interrupts.CPU107.CAL:Function_call_interrupts 2.39e+08 +23.4% 2.95e+08 interrupts.CPU107.TLB:TLB_shootdowns 13603578 +19.6% 16267405 interrupts.CPU108.CAL:Function_call_interrupts 2.388e+08 +23.3% 2.945e+08 interrupts.CPU108.TLB:TLB_shootdowns 13908753 +20.0% 16687726 interrupts.CPU109.CAL:Function_call_interrupts 2.39e+08 +23.4% 2.948e+08 interrupts.CPU109.TLB:TLB_shootdowns 4042050 +20.5% 4872043 interrupts.CPU11.CAL:Function_call_interrupts 2.39e+08 +23.2% 2.945e+08 interrupts.CPU11.TLB:TLB_shootdowns 14229005 +20.2% 17101290 interrupts.CPU110.CAL:Function_call_interrupts 2.389e+08 +23.4% 2.949e+08 interrupts.CPU110.TLB:TLB_shootdowns 14535972 +20.5% 17511646 interrupts.CPU111.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU111.TLB:TLB_shootdowns 14884780 +20.4% 17918975 interrupts.CPU112.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.95e+08 interrupts.CPU112.TLB:TLB_shootdowns 15237079 +20.4% 18344915 interrupts.CPU113.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.949e+08 interrupts.CPU113.TLB:TLB_shootdowns 15595868 +20.7% 18820635 interrupts.CPU114.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.948e+08 interrupts.CPU114.TLB:TLB_shootdowns 15991331 +20.9% 19334777 interrupts.CPU115.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU115.TLB:TLB_shootdowns 16381106 +21.4% 19889187 interrupts.CPU116.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.95e+08 interrupts.CPU116.TLB:TLB_shootdowns 16807315 +21.7% 20462755 interrupts.CPU117.CAL:Function_call_interrupts 2.391e+08 +23.4% 2.95e+08 interrupts.CPU117.TLB:TLB_shootdowns 17279160 +21.8% 21046648 interrupts.CPU118.CAL:Function_call_interrupts 2.393e+08 +23.2% 2.949e+08 interrupts.CPU118.TLB:TLB_shootdowns 17773272 +22.0% 21677864 interrupts.CPU119.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.95e+08 interrupts.CPU119.TLB:TLB_shootdowns 4078269 +20.5% 4914003 interrupts.CPU12.CAL:Function_call_interrupts 2.392e+08 +23.1% 2.946e+08 interrupts.CPU12.TLB:TLB_shootdowns 18284395 +22.1% 22325427 interrupts.CPU120.CAL:Function_call_interrupts 2.393e+08 +23.2% 2.949e+08 interrupts.CPU120.TLB:TLB_shootdowns 18842127 +22.3% 23046843 interrupts.CPU121.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.95e+08 interrupts.CPU121.TLB:TLB_shootdowns 19479272 +22.5% 23863390 interrupts.CPU122.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.95e+08 interrupts.CPU122.TLB:TLB_shootdowns 20057248 +22.9% 24656059 interrupts.CPU123.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.95e+08 interrupts.CPU123.TLB:TLB_shootdowns 20754006 +22.9% 25509076 interrupts.CPU124.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.95e+08 interrupts.CPU124.TLB:TLB_shootdowns 21464324 +23.3% 26457683 interrupts.CPU125.CAL:Function_call_interrupts 2.391e+08 +23.4% 2.95e+08 interrupts.CPU125.TLB:TLB_shootdowns 22340213 +23.7% 27645975 interrupts.CPU126.CAL:Function_call_interrupts 2.389e+08 +23.4% 2.949e+08 interrupts.CPU126.TLB:TLB_shootdowns 23306152 +23.8% 28854758 interrupts.CPU127.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.95e+08 interrupts.CPU127.TLB:TLB_shootdowns 24363030 +23.3% 30028045 interrupts.CPU128.CAL:Function_call_interrupts 2.393e+08 +23.1% 2.947e+08 interrupts.CPU128.TLB:TLB_shootdowns 25354003 +23.5% 31316735 interrupts.CPU129.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.95e+08 interrupts.CPU129.TLB:TLB_shootdowns 4106417 +20.8% 4960674 interrupts.CPU13.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU13.TLB:TLB_shootdowns 26490591 +23.3% 32658486 interrupts.CPU130.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.95e+08 interrupts.CPU130.TLB:TLB_shootdowns 27647753 +23.4% 34104474 interrupts.CPU131.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU131.TLB:TLB_shootdowns 28961846 +23.6% 35787877 interrupts.CPU132.CAL:Function_call_interrupts 2.394e+08 +23.2% 2.95e+08 interrupts.CPU132.TLB:TLB_shootdowns 30413765 +23.7% 37610502 interrupts.CPU133.CAL:Function_call_interrupts 2.393e+08 +23.2% 2.948e+08 interrupts.CPU133.TLB:TLB_shootdowns 32023851 +23.2% 39446337 interrupts.CPU134.CAL:Function_call_interrupts 2.393e+08 +23.2% 2.949e+08 interrupts.CPU134.TLB:TLB_shootdowns 33865881 +22.7% 41547963 interrupts.CPU135.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.949e+08 interrupts.CPU135.TLB:TLB_shootdowns 35817363 +22.6% 43897151 interrupts.CPU136.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.948e+08 interrupts.CPU136.TLB:TLB_shootdowns 38025094 +22.8% 46678700 ± 2% interrupts.CPU137.CAL:Function_call_interrupts 2.392e+08 +23.4% 2.95e+08 interrupts.CPU137.TLB:TLB_shootdowns 40500703 +22.3% 49544961 ± 2% interrupts.CPU138.CAL:Function_call_interrupts 2.395e+08 +23.2% 2.951e+08 interrupts.CPU138.TLB:TLB_shootdowns 43303639 +21.8% 52741872 ± 2% interrupts.CPU139.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.95e+08 interrupts.CPU139.TLB:TLB_shootdowns 4138660 +20.9% 5004748 interrupts.CPU14.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.948e+08 interrupts.CPU14.TLB:TLB_shootdowns 46594989 +21.3% 56502005 ± 2% interrupts.CPU140.CAL:Function_call_interrupts 2.394e+08 +23.2% 2.95e+08 interrupts.CPU140.TLB:TLB_shootdowns 50218642 +20.8% 60644028 ± 2% interrupts.CPU141.CAL:Function_call_interrupts 2.393e+08 +23.3% 2.951e+08 interrupts.CPU141.TLB:TLB_shootdowns 54307694 +20.0% 65173883 ± 3% interrupts.CPU142.CAL:Function_call_interrupts 2.395e+08 +23.3% 2.952e+08 interrupts.CPU142.TLB:TLB_shootdowns 57982307 +19.2% 69104544 ± 3% interrupts.CPU143.CAL:Function_call_interrupts 2.36e+08 +23.6% 2.917e+08 interrupts.CPU143.TLB:TLB_shootdowns 4177402 +20.8% 5046895 interrupts.CPU15.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU15.TLB:TLB_shootdowns 4205664 +21.0% 5088293 interrupts.CPU16.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.948e+08 interrupts.CPU16.TLB:TLB_shootdowns 4242322 +21.0% 5131393 interrupts.CPU17.CAL:Function_call_interrupts 5303 ± 35% +24.7% 6613 ± 27% interrupts.CPU17.NMI:Non-maskable_interrupts 5303 ± 35% +24.7% 6613 ± 27% interrupts.CPU17.PMI:Performance_monitoring_interrupts 2.39e+08 +23.3% 2.947e+08 interrupts.CPU17.TLB:TLB_shootdowns 4272394 +20.8% 5159009 interrupts.CPU18.CAL:Function_call_interrupts 2.385e+08 +23.2% 2.939e+08 interrupts.CPU18.TLB:TLB_shootdowns 4305056 +20.9% 5203542 interrupts.CPU19.CAL:Function_call_interrupts 2.384e+08 +23.5% 2.943e+08 interrupts.CPU19.TLB:TLB_shootdowns 3780317 +20.0% 4536165 interrupts.CPU2.CAL:Function_call_interrupts 2.39e+08 +23.2% 2.945e+08 interrupts.CPU2.TLB:TLB_shootdowns 4342053 +20.9% 5248451 interrupts.CPU20.CAL:Function_call_interrupts 2.387e+08 +23.4% 2.945e+08 interrupts.CPU20.TLB:TLB_shootdowns 4380955 +20.6% 5285264 interrupts.CPU21.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.947e+08 interrupts.CPU21.TLB:TLB_shootdowns 4410600 +20.7% 5325050 interrupts.CPU22.CAL:Function_call_interrupts 2.389e+08 +23.4% 2.948e+08 interrupts.CPU22.TLB:TLB_shootdowns 4450114 +20.5% 5362898 interrupts.CPU23.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU23.TLB:TLB_shootdowns 4480845 +20.5% 5399425 interrupts.CPU24.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.948e+08 interrupts.CPU24.TLB:TLB_shootdowns 4519452 +20.3% 5435546 interrupts.CPU25.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.948e+08 interrupts.CPU25.TLB:TLB_shootdowns 4550723 +20.4% 5477766 interrupts.CPU26.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.947e+08 interrupts.CPU26.TLB:TLB_shootdowns 4586663 +20.2% 5512162 interrupts.CPU27.CAL:Function_call_interrupts 2.392e+08 +23.1% 2.945e+08 interrupts.CPU27.TLB:TLB_shootdowns 4616641 +20.6% 5565409 interrupts.CPU28.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU28.TLB:TLB_shootdowns 4653394 +20.4% 5603550 interrupts.CPU29.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.949e+08 interrupts.CPU29.TLB:TLB_shootdowns 3804402 +20.3% 4576241 interrupts.CPU3.CAL:Function_call_interrupts 2.388e+08 +23.4% 2.945e+08 interrupts.CPU3.TLB:TLB_shootdowns 4688361 +20.4% 5642583 interrupts.CPU30.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.948e+08 interrupts.CPU30.TLB:TLB_shootdowns 4719191 +20.6% 5689475 interrupts.CPU31.CAL:Function_call_interrupts 2.389e+08 +23.4% 2.949e+08 interrupts.CPU31.TLB:TLB_shootdowns 4764766 +20.2% 5728455 interrupts.CPU32.CAL:Function_call_interrupts 2.391e+08 +23.2% 2.947e+08 interrupts.CPU32.TLB:TLB_shootdowns 4805352 +20.2% 5777904 interrupts.CPU33.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.948e+08 interrupts.CPU33.TLB:TLB_shootdowns 4840779 +20.4% 5827446 interrupts.CPU34.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.949e+08 interrupts.CPU34.TLB:TLB_shootdowns 4881838 +20.3% 5874604 interrupts.CPU35.CAL:Function_call_interrupts 2.39e+08 +23.4% 2.948e+08 interrupts.CPU35.TLB:TLB_shootdowns 4921551 +19.8% 5898107 interrupts.CPU36.CAL:Function_call_interrupts 2.387e+08 +23.0% 2.937e+08 interrupts.CPU36.TLB:TLB_shootdowns 4960027 +20.2% 5962825 interrupts.CPU37.CAL:Function_call_interrupts 2.387e+08 +23.3% 2.943e+08 interrupts.CPU37.TLB:TLB_shootdowns 4998377 +20.5% 6021718 interrupts.CPU38.CAL:Function_call_interrupts 2.386e+08 +23.5% 2.946e+08 interrupts.CPU38.TLB:TLB_shootdowns 5049137 +20.3% 6074766 interrupts.CPU39.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.947e+08 interrupts.CPU39.TLB:TLB_shootdowns 3836174 +20.3% 4615967 interrupts.CPU4.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.947e+08 interrupts.CPU4.TLB:TLB_shootdowns 5092462 +20.4% 6131543 interrupts.CPU40.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.948e+08 interrupts.CPU40.TLB:TLB_shootdowns 5139841 +20.3% 6181435 interrupts.CPU41.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU41.TLB:TLB_shootdowns 5177860 +20.2% 6224596 interrupts.CPU42.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.946e+08 interrupts.CPU42.TLB:TLB_shootdowns 5224032 +20.3% 6286501 interrupts.CPU43.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU43.TLB:TLB_shootdowns 5262525 +20.6% 6347167 interrupts.CPU44.CAL:Function_call_interrupts 2.39e+08 +23.4% 2.949e+08 interrupts.CPU44.TLB:TLB_shootdowns 5314673 +20.5% 6403698 interrupts.CPU45.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU45.TLB:TLB_shootdowns 5360050 +20.6% 6464077 interrupts.CPU46.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.95e+08 interrupts.CPU46.TLB:TLB_shootdowns 5409053 +20.4% 6512964 interrupts.CPU47.CAL:Function_call_interrupts 2.393e+08 +23.2% 2.947e+08 interrupts.CPU47.TLB:TLB_shootdowns 5453237 +20.6% 6574075 interrupts.CPU48.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.947e+08 interrupts.CPU48.TLB:TLB_shootdowns 5504995 +20.5% 6631328 interrupts.CPU49.CAL:Function_call_interrupts 2.393e+08 +23.2% 2.947e+08 interrupts.CPU49.TLB:TLB_shootdowns 3862397 +20.4% 4650979 interrupts.CPU5.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.947e+08 interrupts.CPU5.TLB:TLB_shootdowns 5551632 +20.8% 6704398 interrupts.CPU50.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.948e+08 interrupts.CPU50.TLB:TLB_shootdowns 5607515 +20.7% 6769364 interrupts.CPU51.CAL:Function_call_interrupts 2.393e+08 +23.2% 2.949e+08 interrupts.CPU51.TLB:TLB_shootdowns 5655712 +20.9% 6838736 interrupts.CPU52.CAL:Function_call_interrupts 2.39e+08 +23.4% 2.949e+08 interrupts.CPU52.TLB:TLB_shootdowns 5716824 +20.9% 6910425 interrupts.CPU53.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.949e+08 interrupts.CPU53.TLB:TLB_shootdowns 5756291 +21.1% 6972273 interrupts.CPU54.CAL:Function_call_interrupts 2.383e+08 +23.5% 2.943e+08 interrupts.CPU54.TLB:TLB_shootdowns 5843081 +20.7% 7051100 interrupts.CPU55.CAL:Function_call_interrupts 2.386e+08 +23.4% 2.944e+08 interrupts.CPU55.TLB:TLB_shootdowns 5929863 +20.3% 7134897 interrupts.CPU56.CAL:Function_call_interrupts 332.33 ± 3% +12.7% 374.50 ± 4% interrupts.CPU56.RES:Rescheduling_interrupts 2.39e+08 +23.2% 2.944e+08 interrupts.CPU56.TLB:TLB_shootdowns 5999925 +20.4% 7222940 interrupts.CPU57.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.947e+08 interrupts.CPU57.TLB:TLB_shootdowns 6072900 +20.4% 7309901 interrupts.CPU58.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.948e+08 interrupts.CPU58.TLB:TLB_shootdowns 6138769 +20.4% 7390234 interrupts.CPU59.CAL:Function_call_interrupts 2.388e+08 +23.4% 2.948e+08 interrupts.CPU59.TLB:TLB_shootdowns 3892231 +20.4% 4686172 interrupts.CPU6.CAL:Function_call_interrupts 2.389e+08 +23.4% 2.947e+08 interrupts.CPU6.TLB:TLB_shootdowns 6220137 +20.2% 7477269 interrupts.CPU60.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.948e+08 interrupts.CPU60.TLB:TLB_shootdowns 6308903 +19.8% 7556048 interrupts.CPU61.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.947e+08 interrupts.CPU61.TLB:TLB_shootdowns 6374058 +19.9% 7641746 interrupts.CPU62.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.947e+08 interrupts.CPU62.TLB:TLB_shootdowns 6456179 +19.8% 7732626 interrupts.CPU63.CAL:Function_call_interrupts 267.00 ± 3% +28.4% 342.83 ± 13% interrupts.CPU63.RES:Rescheduling_interrupts 2.392e+08 +23.3% 2.948e+08 interrupts.CPU63.TLB:TLB_shootdowns 6541366 +19.6% 7826548 interrupts.CPU64.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.947e+08 interrupts.CPU64.TLB:TLB_shootdowns 6630748 +19.5% 7925912 interrupts.CPU65.CAL:Function_call_interrupts 256.83 ± 11% +26.6% 325.17 ± 8% interrupts.CPU65.RES:Rescheduling_interrupts 2.391e+08 +23.3% 2.948e+08 interrupts.CPU65.TLB:TLB_shootdowns 6725901 +19.3% 8021772 interrupts.CPU66.CAL:Function_call_interrupts 2.391e+08 +23.2% 2.947e+08 interrupts.CPU66.TLB:TLB_shootdowns 6821826 +19.2% 8128703 interrupts.CPU67.CAL:Function_call_interrupts 213.50 ± 7% +47.2% 314.33 ± 3% interrupts.CPU67.RES:Rescheduling_interrupts 2.392e+08 +23.2% 2.948e+08 interrupts.CPU67.TLB:TLB_shootdowns 6914584 +19.0% 8228597 interrupts.CPU68.CAL:Function_call_interrupts 193.50 ± 6% +68.6% 326.17 ± 14% interrupts.CPU68.RES:Rescheduling_interrupts 2.391e+08 +23.2% 2.946e+08 interrupts.CPU68.TLB:TLB_shootdowns 7003298 +19.0% 8335060 interrupts.CPU69.CAL:Function_call_interrupts 174.33 ± 7% +82.3% 317.83 ± 3% interrupts.CPU69.RES:Rescheduling_interrupts 2.388e+08 +23.4% 2.948e+08 interrupts.CPU69.TLB:TLB_shootdowns 3928467 +20.2% 4722635 interrupts.CPU7.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.947e+08 interrupts.CPU7.TLB:TLB_shootdowns 7115855 +18.7% 8446872 interrupts.CPU70.CAL:Function_call_interrupts 149.17 ± 4% +102.0% 301.33 ± 3% interrupts.CPU70.RES:Rescheduling_interrupts 2.392e+08 +23.3% 2.948e+08 interrupts.CPU70.TLB:TLB_shootdowns 7213939 +18.7% 8561752 interrupts.CPU71.CAL:Function_call_interrupts 2.387e+08 +23.4% 2.946e+08 interrupts.CPU71.TLB:TLB_shootdowns 7290699 +18.2% 8618307 interrupts.CPU72.CAL:Function_call_interrupts 215.33 ± 16% +81.1% 390.00 ± 6% interrupts.CPU72.RES:Rescheduling_interrupts 2.384e+08 +23.4% 2.941e+08 interrupts.CPU72.TLB:TLB_shootdowns 7399375 +18.2% 8743431 interrupts.CPU73.CAL:Function_call_interrupts 171.00 ± 18% +110.1% 359.33 ± 4% interrupts.CPU73.RES:Rescheduling_interrupts 2.388e+08 +23.2% 2.943e+08 interrupts.CPU73.TLB:TLB_shootdowns 7506122 +18.3% 8880787 interrupts.CPU74.CAL:Function_call_interrupts 129.17 ± 16% +161.3% 337.50 ± 8% interrupts.CPU74.RES:Rescheduling_interrupts 2.389e+08 +23.2% 2.944e+08 interrupts.CPU74.TLB:TLB_shootdowns 7606538 +18.5% 9014921 interrupts.CPU75.CAL:Function_call_interrupts 83.33 ± 15% +283.6% 319.67 ± 6% interrupts.CPU75.RES:Rescheduling_interrupts 2.39e+08 +23.3% 2.948e+08 interrupts.CPU75.TLB:TLB_shootdowns 7706473 +18.8% 9155542 interrupts.CPU76.CAL:Function_call_interrupts 101.33 ± 47% +215.5% 319.67 ± 4% interrupts.CPU76.RES:Rescheduling_interrupts 2.388e+08 +23.4% 2.948e+08 interrupts.CPU76.TLB:TLB_shootdowns 7812143 +18.9% 9289239 interrupts.CPU77.CAL:Function_call_interrupts 76.33 ± 21% +316.2% 317.67 ± 5% interrupts.CPU77.RES:Rescheduling_interrupts 2.391e+08 +23.2% 2.946e+08 interrupts.CPU77.TLB:TLB_shootdowns 7920128 +19.2% 9436961 interrupts.CPU78.CAL:Function_call_interrupts 83.33 ± 16% +286.4% 322.00 ± 7% interrupts.CPU78.RES:Rescheduling_interrupts 2.389e+08 +23.3% 2.947e+08 interrupts.CPU78.TLB:TLB_shootdowns 8046697 +19.2% 9593342 interrupts.CPU79.CAL:Function_call_interrupts 105.33 ± 76% +193.8% 309.50 ± 8% interrupts.CPU79.RES:Rescheduling_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU79.TLB:TLB_shootdowns 3958005 +20.4% 4765279 interrupts.CPU8.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.948e+08 interrupts.CPU8.TLB:TLB_shootdowns 8170020 +19.3% 9749056 interrupts.CPU80.CAL:Function_call_interrupts 63.33 ± 25% +371.1% 298.33 ± 5% interrupts.CPU80.RES:Rescheduling_interrupts 2.392e+08 +23.3% 2.948e+08 interrupts.CPU80.TLB:TLB_shootdowns 8299618 +19.5% 9915165 interrupts.CPU81.CAL:Function_call_interrupts 55.17 ± 34% +426.6% 290.50 ± 4% interrupts.CPU81.RES:Rescheduling_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU81.TLB:TLB_shootdowns 8423644 +19.7% 10083774 interrupts.CPU82.CAL:Function_call_interrupts 90.33 ± 82% +251.7% 317.67 ± 24% interrupts.CPU82.RES:Rescheduling_interrupts 2.391e+08 +23.3% 2.948e+08 interrupts.CPU82.TLB:TLB_shootdowns 8559491 +19.8% 10257100 interrupts.CPU83.CAL:Function_call_interrupts 61.67 ± 53% +340.3% 271.50 ± 7% interrupts.CPU83.RES:Rescheduling_interrupts 2.392e+08 +23.2% 2.947e+08 interrupts.CPU83.TLB:TLB_shootdowns 8692527 +20.1% 10435645 interrupts.CPU84.CAL:Function_call_interrupts 68.33 ± 61% +262.4% 247.67 ± 7% interrupts.CPU84.RES:Rescheduling_interrupts 2.392e+08 +23.2% 2.946e+08 interrupts.CPU84.TLB:TLB_shootdowns 8834576 +20.4% 10633976 interrupts.CPU85.CAL:Function_call_interrupts 66.17 ± 41% +258.4% 237.17 ± 16% interrupts.CPU85.RES:Rescheduling_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU85.TLB:TLB_shootdowns 8993131 +20.5% 10837698 interrupts.CPU86.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.948e+08 interrupts.CPU86.TLB:TLB_shootdowns 9154455 +20.8% 11054644 interrupts.CPU87.CAL:Function_call_interrupts 47.33 ± 28% +293.0% 186.00 ± 12% interrupts.CPU87.RES:Rescheduling_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU87.TLB:TLB_shootdowns 9317071 +20.9% 11263700 interrupts.CPU88.CAL:Function_call_interrupts 57.50 ± 16% +195.9% 170.17 ± 10% interrupts.CPU88.RES:Rescheduling_interrupts 2.392e+08 +23.2% 2.947e+08 interrupts.CPU88.TLB:TLB_shootdowns 9491277 +21.3% 11513887 interrupts.CPU89.CAL:Function_call_interrupts 53.00 ± 23% +167.3% 141.67 ± 23% interrupts.CPU89.RES:Rescheduling_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU89.TLB:TLB_shootdowns 3982893 +20.4% 4795936 interrupts.CPU9.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.947e+08 interrupts.CPU9.TLB:TLB_shootdowns 9665176 +21.1% 11706642 interrupts.CPU90.CAL:Function_call_interrupts 90.50 ± 16% +89.1% 171.17 ± 15% interrupts.CPU90.RES:Rescheduling_interrupts 2.388e+08 +23.2% 2.942e+08 interrupts.CPU90.TLB:TLB_shootdowns 9818237 +21.3% 11912938 interrupts.CPU91.CAL:Function_call_interrupts 78.17 ± 24% +81.0% 141.50 ± 19% interrupts.CPU91.RES:Rescheduling_interrupts 2.387e+08 +23.5% 2.947e+08 interrupts.CPU91.TLB:TLB_shootdowns 10007640 +21.1% 12115427 interrupts.CPU92.CAL:Function_call_interrupts 2.39e+08 +23.3% 2.948e+08 interrupts.CPU92.TLB:TLB_shootdowns 10173238 +20.9% 12294991 interrupts.CPU93.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.947e+08 interrupts.CPU93.TLB:TLB_shootdowns 10331943 +21.1% 12509293 interrupts.CPU94.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.948e+08 interrupts.CPU94.TLB:TLB_shootdowns 10503418 +21.1% 12718159 interrupts.CPU95.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU95.TLB:TLB_shootdowns 10682794 +21.0% 12921155 interrupts.CPU96.CAL:Function_call_interrupts 2.392e+08 +23.3% 2.949e+08 interrupts.CPU96.TLB:TLB_shootdowns 10873785 +20.9% 13145051 interrupts.CPU97.CAL:Function_call_interrupts 2.393e+08 +23.2% 2.948e+08 interrupts.CPU97.TLB:TLB_shootdowns 11064879 +20.9% 13375375 interrupts.CPU98.CAL:Function_call_interrupts 2.391e+08 +23.3% 2.947e+08 interrupts.CPU98.TLB:TLB_shootdowns 11267475 +20.9% 13617022 interrupts.CPU99.CAL:Function_call_interrupts 2.392e+08 +23.2% 2.946e+08 interrupts.CPU99.TLB:TLB_shootdowns 29153 +20.0% 34985 interrupts.RES:Rescheduling_interrupts 3.443e+10 +23.3% 4.244e+10 interrupts.TLB:TLB_shootdowns will-it-scale.per_thread_ops 4000 +--------------------------------------------------------------------+ | | 3500 |-O O O O O O O O O O O O O O OO O O O O O O O O O O O O O O O O | 3000 |-+ .+. .+. | |.+.+.+.+.+.+ +.+.+.+.+.+.+.+ ++ +.+.+.+.+.+.+.+.+.+.+.+ | 2500 |-+ : : | | : : | 2000 |-+ : : | | : : | 1500 |-+ : : | 1000 |-+ : : | | : : | 500 |-+ : | | : | 0 +--------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month, 2 weeks

3
5
0 / 0

[xfs] bad77c375e: stress-ng.fallocate.ops_per_sec -10.0% regression

by kernel test robot

Greeting, FYI, we noticed a -10.0% regression of stress-ng.fallocate.ops_per_sec due to commit: commit: bad77c375e8de6c776c848e443f7dc2d0d909be5 ("xfs: CIL checkpoint flushes caches unconditionally") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: stress-ng on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory with following parameters: nr_threads: 10% disk: 1HDD testtime: 60s fs: xfs class: filesystem test: fallocate cpufreq_governor: performance ucode: 0x5003006 In addition to that, the commit also has significant impact on the following tests: If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run bin/lkp run generated-yaml-file ========================================================================================= class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode: filesystem/gcc-9/performance/1HDD/xfs/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/fallocate/stress-ng/60s/0x5003006 commit: 0431d926b3 ("xfs: async blkdev cache flush") bad77c375e ("xfs: CIL checkpoint flushes caches unconditionally") 0431d926b399d74f bad77c375e8de6c776c848e443f ---------------- --------------------------- %stddev %change %stddev \ | \ 618.67 -10.0% 556.67 ± 2% stress-ng.fallocate.ops 10.31 -10.0% 9.28 ± 2% stress-ng.fallocate.ops_per_sec 111618 -5.7% 105270 stress-ng.time.voluntary_context_switches 502.67 ± 2% +52.0% 763.83 ± 47% interrupts.CPU44.CAL:Function_call_interrupts 2.10 ± 2% -25.1% 1.57 iostat.cpu.iowait 2.16 ± 2% -0.5 1.62 mpstat.cpu.all.iowait% 0.09 ± 11% +45.0% 0.14 ± 34% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork 151.00 ± 19% +20.2% 181.50 ± 7% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork 6749 +6.4% 7178 perf-stat.i.context-switches 6640 +6.4% 7063 perf-stat.ps.context-switches 353877 -1.4% 348924 proc-vmstat.pgfree 39122 -15.1% 33221 proc-vmstat.pgpgout 1225 ± 14% -9.0% 1115 turbostat.Bzy_MHz 86.82 ± 17% -10.4% 77.83 turbostat.PkgWatt 602.83 -15.1% 511.83 vmstat.io.bo 6953 +6.1% 7376 vmstat.system.cs 1.33 ± 6% +0.4 1.70 ± 13% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt 0.08 ± 22% +0.1 0.17 ± 23% perf-profile.children.cycles-pp.wait_for_completion 0.07 ± 17% +0.1 0.19 ± 64% perf-profile.children.cycles-pp.submit_bio 0.07 ± 17% +0.1 0.19 ± 64% perf-profile.children.cycles-pp.submit_bio_noacct 1.40 ± 8% +0.3 1.74 ± 12% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler stress-ng.fallocate.ops_per_sec 12 +----------------------------------------------------------------------+ | | 10 |.+.++.+.+.+ +.+.+.++.+.+.++.+.+.+.++.+.+.++.+.+.++.+.+.+.++.+.+.++.+.| | O : : O O OO O O O OO | | OO O :O : O O O O | 8 |-+ O : : O O OO O O O O O O OO O | | : : | 6 |-+ : : | | :: | 4 |-+ :: | | :: | | :: | 2 |-+ : | | : | 0 +----------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month, 3 weeks

3
3
0 / 0

[btrfs] 675923ac06: BUG:KASAN:use-after-free_in__blk_queue_split

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 675923ac0693d080b3b769813cf9ca7f73fc0322 ("[PATCH RFC 09/11] btrfs: remove bio split operations in btrfs_submit_direct()") url: https://github.com/0day-ci/linux/commits/Qu-Wenruo/btrfs-split-bio-at-btr... base: https://git.kernel.org/cgit/linux/kernel/git/kdave/linux.git for-next patch link: https://lore.kernel.org/linux-btrfs/20211128055259.39249-10-wqu@suse.com in testcase: xfstests version: xfstests-x86_64-99bc497-1_20211129 with following parameters: disk: 6HDD fs: btrfs test: btrfs-group-14 ucode: 0x28 test-description: xfstests is a regression test suite for xfs and other files ystems. test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git on test machine: 8 threads 1 sockets Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 8G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 44.231297][ T118] BUG: KASAN: use-after-free in __blk_queue_split (block/blk-merge.c:289 block/blk-merge.c:357) [ 44.231305][ T118] Read of size 8 at addr ffff88810b4f8100 by task kworker/u17:0/118 [ 44.231308][ T118] [ 44.231310][ T118] CPU: 1 PID: 118 Comm: kworker/u17:0 Not tainted 5.16.0-rc1-00057-g675923ac0693 #1 [ 44.231314][ T118] Hardware name: Dell Inc. OptiPlex 9020/0DNKMN, BIOS A05 12/05/2013 [ 44.239216][ T335] [ 44.242067][ T118] Workqueue: btrfs-worker-high btrfs_work_helper [btrfs] [ 44.249883][ T335] ID SIZE PATH [ 44.257166][ T118] [ 44.257169][ T118] Call Trace: [ 44.257172][ T118] <TASK> [ 44.257173][ T118] dump_stack_lvl (lib/dump_stack.c:107) [ 44.259361][ T335] [ 44.268531][ T118] print_address_description+0x21/0x140 [ 44.268542][ T118] ? __blk_queue_split (block/blk-merge.c:289 block/blk-merge.c:357) [ 44.277053][ T335] 1 300.00GiB /dev/sda2 [ 44.278613][ T118] kasan_report.cold (mm/kasan/report.c:434 mm/kasan/report.c:450) [ 44.285479][ T335] [ 44.289630][ T118] ? __blk_queue_split (block/blk-merge.c:289 block/blk-merge.c:357) [ 44.289637][ T118] __blk_queue_split (block/blk-merge.c:289 block/blk-merge.c:357) [ 44.289641][ T118] ? __blk_mq_delay_run_hw_queue (block/blk-mq.c:1806 (discriminator 1) block/blk-mq.c:1869 (discriminator 1)) [ 44.291854][ T335] [ 44.294947][ T118] ? bio_attempt_discard_merge (block/blk-merge.c:340) [ 44.294955][ T118] ? deadline_dispatch2_stop (block/mq-deadline.c:691) [ 44.297748][ T335] [ 44.302077][ T118] ? update_io_ticks (block/blk-core.c:1224) [ 44.302087][ T118] blk_mq_submit_bio (block/blk-mq.c:2626) [ 44.304635][ T335] btrfs-progs v5.15 [ 44.310680][ T118] ? mempool_alloc (mm/mempool.c:392) [ 44.310689][ T118] ? blk_mq_try_issue_list_directly (block/blk-mq.c:2612) [ 44.315664][ T335] [ 44.320231][ T118] ? mempool_destroy (mm/mempool.c:375) [ 44.325975][ T335] See http://btrfs.wiki.kernel.org for more information. [ 44.327097][ T118] ? bio_alloc_bioset (block/bio.c:497) [ 44.332050][ T335] [ 44.336812][ T118] submit_bio_noacct (include/linux/bio.h:567 block/blk-core.c:925 block/blk-core.c:950 block/blk-core.c:939) [ 44.336819][ T118] ? ktime_get (kernel/time/timekeeping.c:290 kernel/time/timekeeping.c:386 kernel/time/timekeeping.c:829 kernel/time/timekeeping.c:817) [ 44.342571][ T335] [ 44.344717][ T118] ? __submit_bio (block/blk-core.c:940) [ 44.344725][ T118] ? __bio_clone_fast (include/linux/blk_types.h:225 include/linux/blk-cgroup.h:531 block/bio.c:734) [ 44.344731][ T118] btrfs_map_bio (fs/btrfs/volumes.c:6850 fs/btrfs/volumes.c:6917) btrfs [ 44.350377][ T335] [ 44.355656][ T118] ? update_load_avg (kernel/sched/fair.c:3579 kernel/sched/fair.c:3816) [ 44.355664][ T118] ? update_curr (arch/x86/include/asm/preempt.h:80 include/linux/rcupdate.h:68 include/linux/rcupdate.h:686 include/linux/cgroup.h:794 kernel/sched/fair.c:877) [ 44.358854][ T335] Performing full device TRIM /dev/sda5 (300.00GiB) ... [ 44.362519][ T118] ? btrfs_map_sblock (fs/btrfs/volumes.c:6881) btrfs [ 44.367386][ T335] [ 44.371023][ T118] ? _raw_spin_lock_irqsave (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:513 include/asm-generic/qspinlock.h:82 include/linux/spinlock.h:185 include/linux/spinlock_api_smp.h:111 kernel/locking/spinlock.c:162) [ 44.371030][ T118] run_one_async_done (fs/btrfs/disk-io.c:831) btrfs [ 44.377101][ T335] NOTE: several default settings have changed in version 5.15, please make sure [ 44.381705][ T118] ? run_one_async_start (fs/btrfs/disk-io.c:796) btrfs [ 44.383894][ T335] [ 44.388476][ T118] btrfs_work_helper (fs/btrfs/async-thread.c:258 fs/btrfs/async-thread.c:335) btrfs [ 44.396171][ T335] this does not affect your deployments: [ 44.400189][ T118] process_one_work (arch/x86/include/asm/jump_label.h:27 include/linux/jump_label.h:212 include/trace/events/workqueue.h:108 kernel/workqueue.c:2303) [ 44.402373][ T335] [ 44.407135][ T118] worker_thread (include/linux/list.h:284 kernel/workqueue.c:2446) [ 44.407141][ T118] ? process_one_work (kernel/workqueue.c:2388) [ 44.411923][ T335] - DUP for metadata (-m dup) [ 44.413480][ T118] kthread (kernel/kthread.c:327) [ 44.418000][ T335] [ 44.422844][ T118] ? set_kthread_struct (kernel/kthread.c:272) [ 44.422850][ T118] ret_from_fork (arch/x86/entry/entry_64.S:301) [ 44.428781][ T335] - enabled no-holes (-O no-holes) [ 44.430229][ T118] </TASK> [ 44.430231][ T118] [ 44.430232][ T118] Allocated by task 2368: [ 44.435095][ T335] [ 44.439510][ T118] kasan_save_stack (mm/kasan/common.c:38) [ 44.439516][ T118] __kasan_slab_alloc (mm/kasan/common.c:46 mm/kasan/common.c:434 mm/kasan/common.c:467) [ 44.447235][ T335] - enabled free-space-tree (-R free-space-tree) [ 44.451652][ T118] kmem_cache_alloc (mm/slab.h:520 mm/slub.c:3234 mm/slub.c:3242 mm/slub.c:3247) [ 44.453841][ T335] [ 44.459120][ T118] bvec_alloc (block/bio.c:201) [ 44.459124][ T118] bio_alloc_bioset (block/bio.c:481) [ 44.459126][ T118] iomap_dio_bio_iter (include/linux/bio.h:371 fs/iomap/direct-io.c:311) [ 44.464702][ T335] [ 44.473508][ T118] __iomap_dio_rw (fs/iomap/direct-io.c:433 fs/iomap/direct-io.c:589) [ 44.473513][ T118] iomap_dio_rw (fs/iomap/direct-io.c:679) [ 44.473517][ T118] btrfs_file_write_iter (fs/btrfs/file.c:1999 fs/btrfs/file.c:2093) btrfs [ 44.479252][ T335] [ 44.481419][ T118] new_sync_write (fs/read_write.c:504 (discriminator 1)) [ 44.481423][ T118] vfs_write (fs/read_write.c:590) [ 44.481426][ T118] ksys_pwrite64 (fs/read_write.c:697) [ 44.481429][ T118] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) [ 44.487379][ T335] Label: (null) [ 44.492868][ T118] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:113) [ 44.492874][ T118] [ 44.492875][ T118] Freed by task 0: [ 44.492878][ T118] kasan_save_stack (mm/kasan/common.c:38) [ 44.497663][ T335] [ 44.499827][ T118] kasan_set_track (mm/kasan/common.c:46) [ 44.499832][ T118] kasan_set_free_info (mm/kasan/generic.c:372) [ 44.499834][ T118] __kasan_slab_free (mm/kasan/common.c:368 mm/kasan/common.c:328 mm/kasan/common.c:374) [ 44.505205][ T335] UUID: efe0957a-b115-4c51-b8cd-42bfeb15a87a [ 44.509199][ T118] kmem_cache_free (mm/slub.c:1749 mm/slub.c:3513 mm/slub.c:3529) [ 44.509204][ T118] bio_free (block/bio.c:238) [ 44.514240][ T335] [ 44.518134][ T118] btrfs_dio_private_put (fs/btrfs/inode.c:8091 fs/btrfs/inode.c:8070) btrfs [ 44.520811][ T335] Node size: 16384 [ 44.525342][ T118] split_bio_endio (fs/btrfs/extent_io.c:3210) btrfs [ 44.529603][ T335] [ 44.535053][ T118] btrfs_end_bio (fs/btrfs/volumes.c:5946 fs/btrfs/volumes.c:6666 fs/btrfs/volumes.c:6718) btrfs [ 44.538367][ T335] Sector size: 4096 [ 44.540102][ T118] blk_update_request (block/blk-mq.c:744) [ 44.540108][ T118] scsi_end_request (drivers/scsi/scsi_lib.c:543) [ 44.544287][ T335] [ 44.546451][ T118] scsi_io_completion (drivers/scsi/scsi_lib.c:939) [ 44.546457][ T118] blk_complete_reqs (block/blk-mq.c:890 (discriminator 3)) [ 44.551490][ T335] Filesystem size: 300.00GiB [ 44.555649][ T118] __do_softirq (arch/x86/include/asm/jump_label.h:27 include/linux/jump_label.h:212 include/trace/events/irq.h:142 kernel/softirq.c:559) [ 44.555654][ T118] [ 44.555655][ T118] The buggy address belongs to the object at ffff88810b4f8000 [ 44.555655][ T118] which belongs to the cache biovec-64 of size 1024 [ 44.555658][ T118] The buggy address is located 256 bytes inside of [ 44.555658][ T118] 1024-byte region [ffff88810b4f8000, ffff88810b4f8400) [ 44.562340][ T335] [ 44.567009][ T118] The buggy address belongs to the page: [ 44.567010][ T118] page:000000000e7fa944 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x10b4f8 [ 44.567014][ T118] head:000000000e7fa944 order:3 compound_mapcount:0 compound_pincount:0 [ 44.567017][ T118] flags: 0x17ffffc0010200(slab|head|node=0|zone=2|lastcpupid=0x1fffff) [ 44.569626][ T335] Block group profiles: [ 44.573269][ T118] raw: 0017ffffc0010200 0000000000000000 dead000000000122 ffff888100e12780 [ 44.573272][ T118] raw: 0000000000000000 00000000801c001c 00000001ffffffff 0000000000000000 [ 44.573273][ T118] page dumped because: kasan: bad access detected [ 44.573275][ T118] [ 44.573276][ T118] Memory state around the buggy address: [ 44.573278][ T118] ffff88810b4f8000: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 44.577971][ T335] [ 44.582902][ T118] ffff88810b4f8080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 44.582904][ T118] >ffff88810b4f8100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 44.582905][ T118] ^ [ 44.582907][ T118] ffff88810b4f8180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 44.582909][ T118] ffff88810b4f8200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 44.582911][ T118] ================================================================== [ 44.585982][ T335] Data: single 8.00MiB [ 44.589677][ T118] Disabling lock debugging due to kernel taint [ 44.590244][ C5] ------------[ cut here ]------------ [ 44.593963][ T335] [ 44.599569][ C5] refcount_t: underflow; use-after-free. To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month, 3 weeks

1
0
0 / 0

[fs] 23a831cdec: WARNING:at_include/linux/thread_info.h:#do_getxattr

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 23a831cdecdff03c5a70316f516be2f072e7356d ("[PATCH v1 3/5] fs: split off the vfs_getxattr from getxattr") url: https://github.com/0day-ci/linux/commits/Stefan-Roesch/io_uring-add-xattr... patch link: https://lore.kernel.org/io-uring/20211129221257.2536146-4-shr@fb.com in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-----------------------------------------------------+------------+------------+ | | f42638af1a | 23a831cdec | +-----------------------------------------------------+------------+------------+ | boot_successes | 7 | 0 | | boot_failures | 0 | 7 | | WARNING:at_include/linux/thread_info.h:#do_getxattr | 0 | 7 | | EIP:do_getxattr | 0 | 7 | +-----------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 48.539269][ T175] WARNING: CPU: 0 PID: 175 at include/linux/thread_info.h:216 do_getxattr (include/linux/thread_info.h:216 include/linux/uaccess.h:199 fs/xattr.c:689) [ 48.540381][ T175] Modules linked in: [ 48.540785][ T175] CPU: 0 PID: 175 Comm: systemd-tmpfile Not tainted 5.16.0-rc3-00093-g23a831cdecdf #1 [ 48.541790][ T175] EIP: do_getxattr (include/linux/thread_info.h:216 include/linux/uaccess.h:199 fs/xattr.c:689) [ 48.542321][ T175] Code: ff ff 0f 45 f8 eb a8 8d 76 00 8b 45 f0 ba fb 5d 0c c2 e8 8b 65 14 00 85 c0 74 c0 eb ca bf f4 ff ff ff eb 92 8d b6 00 00 00 00 <0f> 0b bf f2 ff ff ff e9 79 ff ff ff 8d b4 26 00 00 00 00 8d 74 26 All code ======== 0: ff (bad) 1: ff 0f decl (%rdi) 3: 45 f8 rex.RB clc 5: eb a8 jmp 0xffffffffffffffaf 7: 8d 76 00 lea 0x0(%rsi),%esi a: 8b 45 f0 mov -0x10(%rbp),%eax d: ba fb 5d 0c c2 mov $0xc20c5dfb,%edx 12: e8 8b 65 14 00 callq 0x1465a2 17: 85 c0 test %eax,%eax 19: 74 c0 je 0xffffffffffffffdb 1b: eb ca jmp 0xffffffffffffffe7 1d: bf f4 ff ff ff mov $0xfffffff4,%edi 22: eb 92 jmp 0xffffffffffffffb6 24: 8d b6 00 00 00 00 lea 0x0(%rsi),%esi 2a:* 0f 0b ud2 <-- trapping instruction 2c: bf f2 ff ff ff mov $0xfffffff2,%edi 31: e9 79 ff ff ff jmpq 0xffffffffffffffaf 36: 8d b4 26 00 00 00 00 lea 0x0(%rsi,%riz,1),%esi 3d: 8d .byte 0x8d 3e: 74 26 je 0x66 Code starting with the faulting instruction =========================================== 0: 0f 0b ud2 2: bf f2 ff ff ff mov $0xfffffff2,%edi 7: e9 79 ff ff ff jmpq 0xffffffffffffff85 c: 8d b4 26 00 00 00 00 lea 0x0(%rsi,%riz,1),%esi 13: 8d .byte 0x8d 14: 74 26 je 0x3c [ 48.544484][ T175] EAX: c2221740 EBX: 00000084 ECX: ffffffa1 EDX: eff39900 [ 48.545385][ T175] ESI: eff39900 EDI: ffffffa1 EBP: f5453d9c ESP: f5453d88 [ 48.546305][ T175] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068 EFLAGS: 00010282 [ 48.547289][ T175] CR0: 80050033 CR2: b7e0e1b0 CR3: 2ff11000 CR4: 00040690 [ 48.548217][ T175] Call Trace: [ 48.548641][ T175] getxattr (fs/xattr.c:715) [ 48.549219][ T175] ? mntput (fs/namespace.c:1235) [ 48.549808][ T175] ? terminate_walk (fs/namei.c:677) [ 48.550404][ T175] ? path_lookupat+0x6f/0x110 [ 48.551075][ T175] ? filename_lookup (fs/namei.c:2489) [ 48.551715][ T175] ? __this_cpu_preempt_check (lib/smp_processor_id.c:67) [ 48.552403][ T175] ? __this_cpu_preempt_check (lib/smp_processor_id.c:67) [ 48.553145][ T175] ? lockdep_hardirqs_on (kernel/locking/lockdep.c:4356) [ 48.553702][ T175] ? kmem_cache_free (mm/slab.c:3742 (discriminator 1)) [ 48.554328][ T175] ? putname (fs/namei.c:271) [ 48.554761][ T175] ? trace_hardirqs_on (kernel/trace/trace_preemptirq.c:51 (discriminator 22)) [ 48.555364][ T175] ? putname (fs/namei.c:271) [ 48.555823][ T175] ? putname (fs/namei.c:271) [ 48.556339][ T175] path_getxattr (fs/xattr.c:728) [ 48.556824][ T175] __ia32_sys_getxattr (fs/xattr.c:737) [ 48.557425][ T175] __do_fast_syscall_32 (arch/x86/entry/common.c:112 arch/x86/entry/common.c:178) [ 48.558024][ T175] ? sched_clock (arch/x86/kernel/tsc.c:254) [ 48.558549][ T175] ? sched_clock_cpu (kernel/sched/clock.c:371) [ 48.559316][ T175] ? __this_cpu_preempt_check (lib/smp_processor_id.c:67) [ 48.559975][ T175] ? __this_cpu_preempt_check (lib/smp_processor_id.c:67) [ 48.560682][ T175] ? __this_cpu_preempt_check (lib/smp_processor_id.c:67) [ 48.561337][ T175] ? lockdep_hardirqs_on (kernel/locking/lockdep.c:4356) [ 48.561964][ T175] ? irqentry_exit_to_user_mode (kernel/entry/common.c:316) [ 48.562723][ T175] do_fast_syscall_32 (arch/x86/entry/common.c:203) [ 48.563296][ T175] do_SYSENTER_32 (arch/x86/entry/common.c:247) [ 48.563781][ T175] entry_SYSENTER_32 (arch/x86/entry/entry_32.S:872) [ 48.564372][ T175] EIP: 0xb7f8f549 [ 48.564790][ T175] Code: 03 74 c0 01 10 05 03 74 b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10 08 03 74 d8 01 00 00 00 00 00 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 8d 76 00 58 b8 77 00 00 00 cd 80 90 8d 76 All code ======== 0: 03 74 c0 01 add 0x1(%rax,%rax,8),%esi 4: 10 05 03 74 b8 01 adc %al,0x1b87403(%rip) # 0x1b8740d a: 10 06 adc %al,(%rsi) c: 03 74 b4 01 add 0x1(%rsp,%rsi,4),%esi 10: 10 07 adc %al,(%rdi) 12: 03 74 b0 01 add 0x1(%rax,%rsi,4),%esi 16: 10 08 adc %cl,(%rax) 18: 03 74 d8 01 add 0x1(%rax,%rbx,8),%esi 1c: 00 00 add %al,(%rax) 1e: 00 00 add %al,(%rax) 20: 00 51 52 add %dl,0x52(%rcx) 23: 55 push %rbp 24: 89 e5 mov %esp,%ebp 26: 0f 34 sysenter 28: cd 80 int $0x80 2a:* 5d pop %rbp <-- trapping instruction 2b: 5a pop %rdx 2c: 59 pop %rcx 2d: c3 retq 2e: 90 nop 2f: 90 nop 30: 90 nop 31: 90 nop 32: 8d 76 00 lea 0x0(%rsi),%esi 35: 58 pop %rax 36: b8 77 00 00 00 mov $0x77,%eax 3b: cd 80 int $0x80 3d: 90 nop 3e: 8d .byte 0x8d 3f: 76 .byte 0x76 Code starting with the faulting instruction =========================================== 0: 5d pop %rbp 1: 5a pop %rdx 2: 59 pop %rcx 3: c3 retq 4: 90 nop 5: 90 nop 6: 90 nop 7: 90 nop 8: 8d 76 00 lea 0x0(%rsi),%esi b: 58 pop %rax c: b8 77 00 00 00 mov $0x77,%eax 11: cd 80 int $0x80 13: 90 nop 14: 8d .byte 0x8d 15: 76 .byte 0x76 To reproduce: # build kernel cd linux cp config-5.16.0-rc3-00093-g23a831cdecdf .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage modules make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 INSTALL_MOD_PATH=<mod-install-dir> modules_install cd <mod-install-dir> find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month, 3 weeks

1
0
0 / 0

[sched/fair] 5235451f72: phoronix-test-suite.neatbench.CPU.fps 19.7% improvement

by kernel test robot

Greeting, FYI, we noticed a 19.7% improvement of phoronix-test-suite.neatbench.CPU.fps due to commit: commit: 5235451f72e3500897c10e239c9c1e37daf8af05 ("sched/fair: Adjust the allowed NUMA imbalance when SD_NUMA spans multiple LLCs") https://git.kernel.org/cgit/linux/kernel/git/mel/linux.git sched-numaimb-v2r5 in testcase: phoronix-test-suite on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 128G memory with following parameters: test: neatbench-1.0.4 option_a: CPU cpufreq_governor: performance ucode: 0x5003006 test-description: The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. test-url: http://www.phoronix-test-suite.com/ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. ========================================================================================= compiler/cpufreq_governor/kconfig/option_a/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/CPU/debian-x86_64-phoronix/lkp-csl-2sp8/neatbench-1.0.4/phoronix-test-suite/0x5003006 commit: ddceaec491 ("sched/fair: Use weight of SD_NUMA domain in find_busiest_group") 5235451f72 ("sched/fair: Adjust the allowed NUMA imbalance when SD_NUMA spans multiple LLCs") ddceaec4917d67a4 5235451f72e3500897c10e239c9 ---------------- --------------------------- %stddev %change %stddev \ | \ 22.63 +19.7% 27.09 phoronix-test-suite.neatbench.CPU.fps 88721 ± 7% -10.3% 79589 meminfo.Mapped 55876 ± 15% -74.3% 14367 ±107% numa-numastat.node1.other_node 22185 ± 7% -10.3% 19896 proc-vmstat.nr_mapped 1.91 ± 11% -0.3 1.60 ± 7% mpstat.cpu.all.irq% 0.50 ± 4% -0.0 0.46 ± 3% mpstat.cpu.all.sys% 173.96 +4.7% 182.12 turbostat.PkgWatt 79.46 +3.1% 81.96 turbostat.RAMWatt 15733 ± 2% -7.6% 14535 ± 3% slabinfo.filp.num_objs 1274 ± 6% +15.2% 1468 ± 3% slabinfo.kmalloc-cg-2k.active_objs 1274 ± 6% +15.2% 1468 ± 3% slabinfo.kmalloc-cg-2k.num_objs 5188 ± 4% +14.1% 5917 ± 2% slabinfo.kmalloc-cg-64.active_objs 5188 ± 4% +14.1% 5917 ± 2% slabinfo.kmalloc-cg-64.num_objs 3275 ± 6% +17.2% 3837 ± 3% slabinfo.kmalloc-cg-96.active_objs 3275 ± 6% +17.2% 3837 ± 3% slabinfo.kmalloc-cg-96.num_objs 25751 ± 21% -47.1% 13621 ± 35% numa-vmstat.node0.nr_active_file 143853 ± 16% -36.6% 91242 ± 16% numa-vmstat.node0.nr_anon_pages 484824 ± 20% -42.2% 280168 ± 23% numa-vmstat.node0.nr_file_pages 287252 ± 8% -52.1% 137475 ± 72% numa-vmstat.node0.nr_inactive_file 12530 ± 3% -8.1% 11513 ± 5% numa-vmstat.node0.nr_kernel_stack 18149 ± 18% -42.8% 10376 ± 30% numa-vmstat.node0.nr_mapped 1271 ± 10% -38.7% 779.00 ± 9% numa-vmstat.node0.nr_page_table_pages 171802 ± 44% -24.9% 129011 ± 54% numa-vmstat.node0.nr_shmem 19997 ± 12% -33.7% 13261 ± 11% numa-vmstat.node0.nr_slab_reclaimable 25751 ± 21% -47.1% 13621 ± 35% numa-vmstat.node0.nr_zone_active_file 287251 ± 8% -52.1% 137475 ± 72% numa-vmstat.node0.nr_zone_inactive_file 14994 ± 65% +281.4% 57183 ± 26% numa-vmstat.node0.numa_other 38335 ± 58% +149.5% 95642 ± 9% numa-vmstat.node1.nr_anon_pages 27.17 ± 89% +226.6% 88.71 ± 19% numa-vmstat.node1.nr_anon_transparent_hugepages 98914 ± 98% +208.3% 304907 ± 23% numa-vmstat.node1.nr_file_pages 14116 ±170% +1051.8% 162591 ± 61% numa-vmstat.node1.nr_inactive_file 4120 ± 69% +131.4% 9535 ± 31% numa-vmstat.node1.nr_mapped 435.67 ± 30% +108.6% 908.71 ± 7% numa-vmstat.node1.nr_page_table_pages 6585 ± 40% +107.6% 13670 ± 9% numa-vmstat.node1.nr_slab_reclaimable 14116 ±170% +1051.8% 162591 ± 61% numa-vmstat.node1.nr_zone_inactive_file 80193 ± 12% -51.8% 38628 ± 39% numa-vmstat.node1.numa_other 2.729e+09 ± 2% +6.4% 2.904e+09 perf-stat.i.branch-instructions 505.40 ± 5% +15.9% 585.66 ± 5% perf-stat.i.cpu-migrations 3.396e+09 ± 4% +9.3% 3.713e+09 ± 3% perf-stat.i.dTLB-stores 1405668 -4.1% 1348471 perf-stat.i.iTLB-load-misses 3.235e+10 ± 3% +8.2% 3.502e+10 ± 2% perf-stat.i.instructions 1.705e+08 ± 2% +7.6% 1.835e+08 ± 2% perf-stat.i.metric.M/sec 28.41 ± 5% +31.1 59.50 ± 2% perf-stat.i.node-load-miss-rate% 5828324 ± 7% +182.2% 16448667 ± 3% perf-stat.i.node-load-misses 20465765 ± 3% -34.7% 13359732 ± 5% perf-stat.i.node-loads 28.94 ± 13% +16.1 45.04 ± 4% perf-stat.i.node-store-miss-rate% 10431914 ± 19% +40.9% 14703306 ± 4% perf-stat.i.node-store-misses 27569324 ± 4% -20.4% 21952235 ± 4% perf-stat.i.node-stores 23020 ± 4% +12.8% 25974 ± 3% perf-stat.overall.instructions-per-iTLB-miss 22.15 ± 4% +33.0 55.19 ± 2% perf-stat.overall.node-load-miss-rate% 27.33 ± 17% +12.8 40.12 ± 2% perf-stat.overall.node-store-miss-rate% 2.709e+09 ± 2% +6.7% 2.89e+09 ± 2% perf-stat.ps.branch-instructions 502.85 ± 5% +16.2% 584.29 ± 5% perf-stat.ps.cpu-migrations 3.371e+09 ± 4% +9.6% 3.695e+09 ± 4% perf-stat.ps.dTLB-stores 1395438 -3.8% 1341938 perf-stat.ps.iTLB-load-misses 3.211e+10 ± 3% +8.5% 3.485e+10 ± 3% perf-stat.ps.instructions 5789322 ± 7% +182.8% 16371874 ± 3% perf-stat.ps.node-load-misses 20322823 ± 3% -34.5% 13305599 ± 5% perf-stat.ps.node-loads 10365664 ± 19% +41.4% 14656403 ± 4% perf-stat.ps.node-store-misses 27391826 ± 4% -20.1% 21876735 ± 4% perf-stat.ps.node-stores 127314 ± 28% -48.0% 66146 ± 34% numa-meminfo.node0.Active 103005 ± 21% -47.1% 54486 ± 35% numa-meminfo.node0.Active(file) 575287 ± 16% -36.6% 364942 ± 16% numa-meminfo.node0.AnonPages 1939219 ± 20% -42.2% 1120673 ± 23% numa-meminfo.node0.FilePages 2390610 ± 19% -40.6% 1419336 ± 14% numa-meminfo.node0.Inactive 1149006 ± 8% -52.1% 549901 ± 72% numa-meminfo.node0.Inactive(file) 79990 ± 12% -33.7% 53046 ± 11% numa-meminfo.node0.KReclaimable 12525 ± 3% -8.1% 11512 ± 5% numa-meminfo.node0.KernelStack 72557 ± 18% -42.8% 41505 ± 30% numa-meminfo.node0.Mapped 3332861 ± 16% -34.7% 2176379 ± 11% numa-meminfo.node0.MemUsed 5087 ± 10% -38.8% 3114 ± 9% numa-meminfo.node0.PageTables 79990 ± 12% -33.7% 53046 ± 11% numa-meminfo.node0.SReclaimable 687133 ± 44% -24.9% 516044 ± 54% numa-meminfo.node0.Shmem 204809 ± 12% -17.7% 168628 ± 5% numa-meminfo.node0.Slab 56726 ± 87% +222.0% 182665 ± 18% numa-meminfo.node1.AnonHugePages 153183 ± 58% +149.7% 382447 ± 9% numa-meminfo.node1.AnonPages 395648 ± 98% +208.3% 1219628 ± 23% numa-meminfo.node1.FilePages 519470 ± 85% +188.0% 1495904 ± 15% numa-meminfo.node1.Inactive 56466 ±170% +1051.8% 650365 ± 61% numa-meminfo.node1.Inactive(file) 26342 ± 40% +107.6% 54684 ± 9% numa-meminfo.node1.KReclaimable 16427 ± 69% +132.2% 38138 ± 31% numa-meminfo.node1.Mapped 1185301 ± 46% +89.2% 2242025 ± 7% numa-meminfo.node1.MemUsed 1750 ± 30% +107.7% 3635 ± 7% numa-meminfo.node1.PageTables 26342 ± 40% +107.6% 54684 ± 9% numa-meminfo.node1.SReclaimable 121213 ± 23% +32.7% 160815 ± 5% numa-meminfo.node1.Slab 0.43 ± 20% -34.3% 0.28 ± 43% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown] 0.26 ± 97% -97.0% 0.01 ±244% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.down_read.do_user_addr_fault.exc_page_fault 0.00 ± 45% +193.4% 0.01 ± 30% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 2.51 ± 54% -73.3% 0.67 ± 69% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.__alloc_pages.alloc_pages_vma.do_anonymous_page 0.89 ± 99% -99.0% 0.01 ±244% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.down_read.do_user_addr_fault.exc_page_fault 84857 ± 29% -55.3% 37936 ± 48% perf-sched.total_wait_and_delay.count.ms 0.84 ± 17% -70.9% 0.25 ±120% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 20.69 ± 9% +82.7% 37.80 ± 28% perf-sched.wait_and_delay.avg.ms.futex_wait_queue.futex_wait.do_futex.__x64_sys_futex 41.86 ± 6% +161.8% 109.59 ± 91% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork 19148 ± 30% -62.5% 7189 ± 52% perf-sched.wait_and_delay.count.do_task_dead.do_exit.__x64_sys_exit.do_syscall_64.entry_SYSCALL_64_after_hwframe 3032 ± 35% -58.3% 1264 ± 50% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 674.17 ± 34% -80.7% 130.29 ±118% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 4787 ± 34% -68.2% 1522 ± 57% perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.down_write_killable.do_mprotect_pkey.__x64_sys_mprotect 2403 ± 35% -65.7% 823.14 ± 65% perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.down_write_killable.vm_mmap_pgoff.do_syscall_64 18832 ± 29% -60.4% 7456 ± 50% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork 59.29 ± 35% -54.4% 27.01 ± 86% perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 15.92 ± 18% -73.3% 4.25 ±120% perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 2178 ± 96% -51.7% 1051 ± 12% perf-sched.wait_and_delay.max.ms.futex_wait_queue.futex_wait.do_futex.__x64_sys_futex 20.68 ± 9% +82.7% 37.79 ± 28% perf-sched.wait_time.avg.ms.futex_wait_queue.futex_wait.do_futex.__x64_sys_futex 2.77 ± 22% +50.3% 4.17 ± 7% perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork 41.85 ± 6% +161.8% 109.58 ± 91% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork 59.28 ± 35% -54.5% 26.95 ± 87% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 2.71 ± 68% -78.0% 0.60 ±154% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function.[unknown] 14.90 ± 21% -52.9% 7.02 ± 53% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 2178 ± 96% -51.7% 1051 ± 12% perf-sched.wait_time.max.ms.futex_wait_queue.futex_wait.do_futex.__x64_sys_futex 16.26 ± 25% -45.5% 8.86 ± 50% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.do_exit.__x64_sys_exit.do_syscall_64 13.65 ± 5% -29.1% 9.68 ± 47% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.down_write_killable.do_mprotect_pkey.__x64_sys_mprotect 1664 ± 89% -75.5% 408.00 ± 97% interrupts.CPU27.NMI:Non-maskable_interrupts 1664 ± 89% -75.5% 408.00 ± 97% interrupts.CPU27.PMI:Performance_monitoring_interrupts 1614 ± 89% -75.9% 388.57 ± 85% interrupts.CPU30.NMI:Non-maskable_interrupts 1614 ± 89% -75.9% 388.57 ± 85% interrupts.CPU30.PMI:Performance_monitoring_interrupts 1634 ± 87% -73.3% 435.57 ± 91% interrupts.CPU34.NMI:Non-maskable_interrupts 1634 ± 87% -73.3% 435.57 ± 91% interrupts.CPU34.PMI:Performance_monitoring_interrupts 1631 ± 86% -73.8% 428.14 ± 86% interrupts.CPU35.NMI:Non-maskable_interrupts 1631 ± 86% -73.8% 428.14 ± 86% interrupts.CPU35.PMI:Performance_monitoring_interrupts 1591 ± 89% -74.3% 409.43 ± 86% interrupts.CPU36.NMI:Non-maskable_interrupts 1591 ± 89% -74.3% 409.43 ± 86% interrupts.CPU36.PMI:Performance_monitoring_interrupts 1608 ± 87% -75.2% 399.71 ± 88% interrupts.CPU38.NMI:Non-maskable_interrupts 1608 ± 87% -75.2% 399.71 ± 88% interrupts.CPU38.PMI:Performance_monitoring_interrupts 1606 ± 84% -76.5% 377.14 ± 91% interrupts.CPU39.NMI:Non-maskable_interrupts 1606 ± 84% -76.5% 377.14 ± 91% interrupts.CPU39.PMI:Performance_monitoring_interrupts 1564 ± 87% -74.1% 405.71 ± 93% interrupts.CPU41.NMI:Non-maskable_interrupts 1564 ± 87% -74.1% 405.71 ± 93% interrupts.CPU41.PMI:Performance_monitoring_interrupts 1595 ± 86% -76.4% 376.86 ± 92% interrupts.CPU42.NMI:Non-maskable_interrupts 1595 ± 86% -76.4% 376.86 ± 92% interrupts.CPU42.PMI:Performance_monitoring_interrupts 2699 ± 65% -69.5% 822.43 ± 69% interrupts.CPU61.NMI:Non-maskable_interrupts 2699 ± 65% -69.5% 822.43 ± 69% interrupts.CPU61.PMI:Performance_monitoring_interrupts 1511 ± 97% -75.8% 365.43 ± 96% interrupts.CPU88.NMI:Non-maskable_interrupts 1511 ± 97% -75.8% 365.43 ± 96% interrupts.CPU88.PMI:Performance_monitoring_interrupts 1492 ± 97% -74.0% 388.57 ± 92% interrupts.CPU89.NMI:Non-maskable_interrupts 1492 ± 97% -74.0% 388.57 ± 92% interrupts.CPU89.PMI:Performance_monitoring_interrupts 1532 ±102% -80.2% 303.57 ± 74% interrupts.CPU93.NMI:Non-maskable_interrupts 1532 ±102% -80.2% 303.57 ± 74% interrupts.CPU93.PMI:Performance_monitoring_interrupts 1577 ± 98% -74.8% 397.86 ± 84% interrupts.CPU94.NMI:Non-maskable_interrupts 1577 ± 98% -74.8% 397.86 ± 84% interrupts.CPU94.PMI:Performance_monitoring_interrupts 1598 ± 91% -80.5% 312.14 ± 80% interrupts.CPU95.NMI:Non-maskable_interrupts 1598 ± 91% -80.5% 312.14 ± 80% interrupts.CPU95.PMI:Performance_monitoring_interrupts 193554 ± 42% -68.3% 61401 ± 60% interrupts.NMI:Non-maskable_interrupts 193554 ± 42% -68.3% 61401 ± 60% interrupts.PMI:Performance_monitoring_interrupts 11.62 ± 26% -6.4 5.19 ± 45% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt 11.28 ± 27% -6.3 4.98 ± 45% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 10.27 ± 27% -5.8 4.48 ± 45% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 10.19 ± 27% -5.7 4.44 ± 45% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 5.68 ± 29% -3.3 2.42 ± 45% perf-profile.calltrace.cycles-pp.update_curr.task_tick_fair.scheduler_tick.update_process_times.tick_sched_handle 5.24 ± 16% -3.0 2.22 ± 51% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr 5.16 ± 16% -3.0 2.19 ± 52% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime 5.14 ± 29% -2.9 2.20 ± 45% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.task_tick_fair.scheduler_tick.update_process_times 4.55 ± 29% -2.6 1.93 ± 45% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.task_tick_fair 4.35 ± 22% -2.5 1.86 ± 51% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event 3.89 ± 21% -2.2 1.68 ± 52% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow 3.81 ± 21% -2.2 1.64 ± 52% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow 2.29 ± 16% -1.1 1.19 ± 44% perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.29 ± 16% -1.1 1.19 ± 44% perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.86 ± 19% -0.9 1.01 ± 43% perf-profile.calltrace.cycles-pp.do_anonymous_page.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault 1.36 ± 15% -0.7 0.68 ± 45% perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.97 ± 16% -0.5 0.42 ± 64% perf-profile.calltrace.cycles-pp.dup_task_struct.copy_process.kernel_clone.__do_sys_clone.do_syscall_64 0.96 ± 14% -0.5 0.45 ± 63% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.96 ± 14% -0.5 0.45 ± 63% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary 0.84 ± 19% -0.5 0.33 ± 86% perf-profile.calltrace.cycles-pp.wake_up_new_task.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.77 ± 22% -0.5 0.32 ± 87% perf-profile.calltrace.cycles-pp.__sysvec_call_function.sysvec_call_function.asm_sysvec_call_function 0.87 ± 21% -0.4 0.44 ± 64% perf-profile.calltrace.cycles-pp.sysvec_call_function.asm_sysvec_call_function 0.81 ± 16% -0.4 0.43 ± 87% perf-profile.calltrace.cycles-pp.schedule.rwsem_down_read_slowpath.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 0.81 ± 16% -0.4 0.43 ± 87% perf-profile.calltrace.cycles-pp.__schedule.schedule.rwsem_down_read_slowpath.do_user_addr_fault.exc_page_fault 0.62 ± 19% +0.9 1.51 ± 46% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.37 ±100% +1.1 1.45 ± 61% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.74 ± 18% +1.1 1.83 ± 44% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.44 ±100% +1.3 1.75 ± 57% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 0.57 ± 82% +1.3 1.91 ± 67% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 1.16 ± 24% +2.2 3.32 ± 49% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 2.44 ± 26% +4.1 6.50 ± 46% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 2.48 ± 26% +4.2 6.67 ± 46% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 3.78 ± 24% +6.3 10.08 ± 46% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 4.20 ± 23% +8.2 12.44 ± 48% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 8.22 ± 15% -4.1 4.10 ± 40% perf-profile.children.cycles-pp.perf_tp_event 7.88 ± 17% -4.1 3.76 ± 45% perf-profile.children.cycles-pp.update_curr 7.66 ± 15% -3.9 3.78 ± 41% perf-profile.children.cycles-pp.perf_swevent_overflow 7.62 ± 15% -3.9 3.76 ± 40% perf-profile.children.cycles-pp.__perf_event_overflow 7.52 ± 14% -3.8 3.71 ± 40% perf-profile.children.cycles-pp.perf_event_output_forward 7.20 ± 17% -3.8 3.43 ± 45% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime 6.38 ± 15% -3.2 3.20 ± 40% perf-profile.children.cycles-pp.perf_prepare_sample 5.70 ± 15% -2.8 2.90 ± 40% perf-profile.children.cycles-pp.perf_callchain 5.59 ± 14% -2.7 2.84 ± 40% perf-profile.children.cycles-pp.get_perf_callchain 3.97 ± 15% -1.9 2.08 ± 39% perf-profile.children.cycles-pp.perf_callchain_kernel 2.85 ± 14% -1.3 1.55 ± 40% perf-profile.children.cycles-pp.unwind_next_frame 2.37 ± 15% -1.0 1.38 ± 28% perf-profile.children.cycles-pp.__do_sys_clone 2.37 ± 15% -1.0 1.38 ± 28% perf-profile.children.cycles-pp.kernel_clone 1.88 ± 19% -0.8 1.05 ± 37% perf-profile.children.cycles-pp.do_anonymous_page 1.52 ± 14% -0.8 0.70 ± 45% perf-profile.children.cycles-pp.perf_callchain_user 1.73 ± 13% -0.8 0.91 ± 40% perf-profile.children.cycles-pp.__unwind_start 1.44 ± 14% -0.8 0.67 ± 44% perf-profile.children.cycles-pp.__get_user_nocheck_8 1.44 ± 15% -0.6 0.81 ± 43% perf-profile.children.cycles-pp.clear_page_erms 1.94 ± 18% -0.6 1.34 ± 10% perf-profile.children.cycles-pp.native_irq_return_iret 1.42 ± 13% -0.6 0.83 ± 26% perf-profile.children.cycles-pp.copy_process 1.11 ± 18% -0.5 0.58 ± 30% perf-profile.children.cycles-pp.__alloc_pages 1.03 ± 19% -0.5 0.51 ± 30% perf-profile.children.cycles-pp.alloc_pages_vma 0.90 ± 20% -0.5 0.41 ± 39% perf-profile.children.cycles-pp.kernel_init_free_pages 1.02 ± 18% -0.5 0.52 ± 29% perf-profile.children.cycles-pp.get_page_from_freelist 0.98 ± 16% -0.5 0.50 ± 37% perf-profile.children.cycles-pp.dup_task_struct 0.94 ± 17% -0.5 0.47 ± 47% perf-profile.children.cycles-pp.__orc_find 0.73 ± 38% -0.5 0.28 ± 57% perf-profile.children.cycles-pp.ksys_write 0.73 ± 38% -0.4 0.28 ± 57% perf-profile.children.cycles-pp.vfs_write 0.73 ± 38% -0.4 0.28 ± 56% perf-profile.children.cycles-pp.new_sync_write 0.98 ± 14% -0.4 0.57 ± 26% perf-profile.children.cycles-pp.schedule_idle 0.73 ± 22% -0.4 0.33 ± 45% perf-profile.children.cycles-pp.unwind_get_return_address 0.73 ± 19% -0.4 0.34 ± 39% perf-profile.children.cycles-pp.prep_new_page 0.55 ± 22% -0.4 0.17 ± 70% perf-profile.children.cycles-pp.__libc_write 0.76 ± 21% -0.4 0.38 ± 39% perf-profile.children.cycles-pp.orc_find 0.55 ± 22% -0.4 0.17 ± 70% perf-profile.children.cycles-pp.__generic_file_write_iter 0.55 ± 22% -0.4 0.17 ± 71% perf-profile.children.cycles-pp.generic_file_write_iter 0.55 ± 22% -0.4 0.17 ± 70% perf-profile.children.cycles-pp.generic_perform_write 0.68 ± 22% -0.4 0.31 ± 46% perf-profile.children.cycles-pp.__kernel_text_address 0.75 ± 26% -0.4 0.37 ± 47% perf-profile.children.cycles-pp.update_load_avg 0.84 ± 18% -0.4 0.48 ± 39% perf-profile.children.cycles-pp.wake_up_new_task 0.63 ± 24% -0.4 0.28 ± 44% perf-profile.children.cycles-pp.kernel_text_address 0.69 ± 8% -0.4 0.34 ± 48% perf-profile.children.cycles-pp.perf_output_sample 0.75 ± 11% -0.3 0.41 ± 34% perf-profile.children.cycles-pp.perf_trace_sched_switch 1.03 ± 11% -0.3 0.71 ± 46% perf-profile.children.cycles-pp.rwsem_down_read_slowpath 0.65 ± 16% -0.3 0.34 ± 42% perf-profile.children.cycles-pp.__vmalloc_node_range 0.68 ± 17% -0.3 0.40 ± 39% perf-profile.children.cycles-pp.select_task_rq_fair 0.63 ± 12% -0.3 0.37 ± 42% perf-profile.children.cycles-pp.finish_task_switch 0.42 ± 28% -0.2 0.18 ± 52% perf-profile.children.cycles-pp.account_user_time 0.55 ± 22% -0.2 0.32 ± 44% perf-profile.children.cycles-pp.find_idlest_group 0.80 ± 10% -0.2 0.57 ± 39% perf-profile.children.cycles-pp.newidle_balance 0.42 ± 8% -0.2 0.21 ± 50% perf-profile.children.cycles-pp.perf_output_copy 0.38 ± 17% -0.2 0.17 ± 46% perf-profile.children.cycles-pp.__perf_event_header__init_id 0.45 ± 10% -0.2 0.25 ± 50% perf-profile.children.cycles-pp.do_task_dead 0.51 ± 16% -0.2 0.32 ± 30% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template 0.26 ± 21% -0.2 0.08 ± 89% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.26 ± 22% -0.2 0.07 ± 86% perf-profile.children.cycles-pp.copy_page_from_iter_atomic 0.32 ± 22% -0.2 0.14 ± 49% perf-profile.children.cycles-pp.perf_event_pid_type 0.25 ± 21% -0.2 0.07 ± 86% perf-profile.children.cycles-pp.copyin 0.37 ± 14% -0.2 0.20 ± 49% perf-profile.children.cycles-pp.alloc_vmap_area 0.38 ± 15% -0.2 0.21 ± 48% perf-profile.children.cycles-pp.__get_vm_area_node 0.26 ± 25% -0.2 0.09 ± 48% perf-profile.children.cycles-pp.perf_output_begin_forward 0.27 ± 22% -0.1 0.13 ± 35% perf-profile.children.cycles-pp.__vmalloc_area_node 0.28 ± 22% -0.1 0.14 ± 53% perf-profile.children.cycles-pp.queue_work_on 0.24 ± 21% -0.1 0.11 ± 32% perf-profile.children.cycles-pp.__alloc_pages_bulk 0.29 ± 19% -0.1 0.16 ± 40% perf-profile.children.cycles-pp.__queue_work 0.20 ± 24% -0.1 0.07 ± 69% perf-profile.children.cycles-pp.is_bpf_text_address 0.18 ± 23% -0.1 0.04 ± 88% perf-profile.children.cycles-pp.fault_in_iov_iter_readable 0.18 ± 23% -0.1 0.04 ± 88% perf-profile.children.cycles-pp.fault_in_readable 0.17 ± 24% -0.1 0.04 ± 88% perf-profile.children.cycles-pp.__get_user_nocheck_1 0.36 ± 13% -0.1 0.23 ± 30% perf-profile.children.cycles-pp.rcu_do_batch 0.27 ± 8% -0.1 0.14 ± 30% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.27 ± 7% -0.1 0.14 ± 29% perf-profile.children.cycles-pp.down_write_killable 0.22 ± 27% -0.1 0.10 ± 60% perf-profile.children.cycles-pp.fixup_exception 0.24 ± 24% -0.1 0.12 ± 57% perf-profile.children.cycles-pp.kernelmode_fixup_or_oops 0.16 ± 31% -0.1 0.04 ± 64% perf-profile.children.cycles-pp.is_ftrace_trampoline 0.28 ± 13% -0.1 0.16 ± 43% perf-profile.children.cycles-pp.enqueue_task_fair 0.17 ± 24% -0.1 0.06 ± 70% perf-profile.children.cycles-pp.bpf_ksym_find 0.24 ± 23% -0.1 0.12 ± 44% perf-profile.children.cycles-pp.perf_tp_event_match 0.30 ± 16% -0.1 0.19 ± 44% perf-profile.children.cycles-pp.sched_move_task 0.23 ± 29% -0.1 0.12 ± 32% perf-profile.children.cycles-pp.___perf_sw_event 0.25 ± 22% -0.1 0.15 ± 44% perf-profile.children.cycles-pp.free_work 0.35 ± 13% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.vm_mmap_pgoff 0.24 ± 21% -0.1 0.14 ± 45% perf-profile.children.cycles-pp.__vunmap 0.48 ± 16% -0.1 0.37 ± 13% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.19 ± 25% -0.1 0.09 ± 62% perf-profile.children.cycles-pp.search_exception_tables 0.19 ± 20% -0.1 0.09 ± 45% perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.16 ± 15% -0.1 0.07 ± 87% perf-profile.children.cycles-pp.perf_output_end 0.16 ± 34% -0.1 0.06 ± 72% perf-profile.children.cycles-pp.__cgroup_account_cputime_field 0.13 ± 28% -0.1 0.04 ± 64% perf-profile.children.cycles-pp.ftrace_ops_trampoline 0.18 ± 22% -0.1 0.09 ± 58% perf-profile.children.cycles-pp.search_extable 0.16 ± 26% -0.1 0.07 ± 55% perf-profile.children.cycles-pp.__update_load_avg_se 0.26 ± 18% -0.1 0.17 ± 26% perf-profile.children.cycles-pp.rmqueue 0.16 ± 19% -0.1 0.07 ± 65% perf-profile.children.cycles-pp.__sysvec_call_function_single 0.22 ± 12% -0.1 0.14 ± 44% perf-profile.children.cycles-pp.enqueue_entity 0.14 ± 19% -0.1 0.05 ± 70% perf-profile.children.cycles-pp.arch_dup_task_struct 0.16 ± 19% -0.1 0.08 ± 49% perf-profile.children.cycles-pp.sysvec_call_function_single 0.16 ± 18% -0.1 0.08 ± 49% perf-profile.children.cycles-pp.perf_iterate_sb 0.16 ± 23% -0.1 0.08 ± 57% perf-profile.children.cycles-pp.bsearch 0.13 ± 11% -0.1 0.05 ± 93% perf-profile.children.cycles-pp.perf_trace_buf_alloc 0.17 ± 15% -0.1 0.09 ± 49% perf-profile.children.cycles-pp.sched_ttwu_pending 0.14 ± 23% -0.1 0.07 ± 52% perf-profile.children.cycles-pp.perf_event_task 0.22 ± 8% -0.1 0.14 ± 32% perf-profile.children.cycles-pp.kmem_cache_free 0.13 ± 29% -0.1 0.06 ± 72% perf-profile.children.cycles-pp.cmp_ex_search 0.12 ± 25% -0.1 0.05 ± 66% perf-profile.children.cycles-pp.cpuacct_charge 0.16 ± 22% -0.1 0.09 ± 29% perf-profile.children.cycles-pp.__perf_sw_event 0.11 ± 26% -0.1 0.05 ± 65% perf-profile.children.cycles-pp.update_min_vruntime 0.19 ± 14% -0.1 0.12 ± 42% perf-profile.children.cycles-pp.do_futex 0.10 ± 6% -0.1 0.04 ± 89% perf-profile.children.cycles-pp.perf_swevent_get_recursion_context 0.13 ± 18% -0.1 0.07 ± 14% perf-profile.children.cycles-pp.lru_add_drain_cpu 0.10 ± 24% -0.1 0.04 ± 66% perf-profile.children.cycles-pp.perf_event_task_output 0.13 ± 28% -0.1 0.07 ± 43% perf-profile.children.cycles-pp.sched_post_fork 0.13 ± 20% -0.1 0.07 ± 13% perf-profile.children.cycles-pp.lru_add_drain 0.15 ± 17% -0.1 0.09 ± 48% perf-profile.children.cycles-pp.remove_vm_area 0.10 ± 25% -0.1 0.04 ± 66% perf-profile.children.cycles-pp.update_cfs_group 0.12 ± 11% -0.1 0.07 ± 24% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.11 ± 16% -0.1 0.06 ± 67% perf-profile.children.cycles-pp.futex_wait 0.09 ± 21% -0.1 0.04 ± 64% perf-profile.children.cycles-pp.kmem_cache_alloc_node 0.12 ± 31% -0.1 0.07 ± 43% perf-profile.children.cycles-pp.task_fork_fair 0.11 ± 16% -0.1 0.06 ± 66% perf-profile.children.cycles-pp.__x64_sys_futex 0.11 ± 15% -0.1 0.06 ± 66% perf-profile.children.cycles-pp.free_vmap_area_noflush 0.08 ± 17% -0.0 0.04 ± 63% perf-profile.children.cycles-pp.__slab_alloc 0.10 ± 15% -0.0 0.06 ± 66% perf-profile.children.cycles-pp.futex_wait_queue 0.08 ± 20% -0.0 0.04 ± 63% perf-profile.children.cycles-pp.___slab_alloc 0.08 ± 17% -0.0 0.03 ± 88% perf-profile.children.cycles-pp.__switch_to_asm 0.09 ± 11% -0.0 0.05 ± 44% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm 0.06 ± 47% +0.1 0.12 ± 17% perf-profile.children.cycles-pp.__hrtimer_get_next_event 0.00 +0.1 0.07 ± 24% perf-profile.children.cycles-pp.tick_nohz_tick_stopped 0.04 ± 72% +0.1 0.12 ± 34% perf-profile.children.cycles-pp.rcu_dynticks_inc 0.19 ± 11% +0.1 0.28 ± 39% perf-profile.children.cycles-pp.arch_scale_freq_tick 0.00 +0.1 0.10 ± 53% perf-profile.children.cycles-pp.native_apic_mem_write 0.01 ±223% +0.1 0.11 ± 57% perf-profile.children.cycles-pp.hrtimer_get_next_event 0.36 ± 11% +0.1 0.49 ± 17% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.05 ± 73% +0.1 0.19 ± 40% perf-profile.children.cycles-pp.rcu_eqs_exit 0.07 ± 27% +0.2 0.26 ± 72% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.21 ± 18% +0.2 0.42 ± 51% perf-profile.children.cycles-pp.read_tsc 0.07 ± 49% +0.2 0.28 ± 57% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.29 ± 9% +0.2 0.52 ± 39% perf-profile.children.cycles-pp.native_sched_clock 0.36 ± 8% +0.3 0.62 ± 38% perf-profile.children.cycles-pp.sched_clock_cpu 0.18 ± 21% +0.4 0.58 ± 35% perf-profile.children.cycles-pp.tick_irq_enter 0.36 ± 9% +0.4 0.77 ± 51% perf-profile.children.cycles-pp.lapic_next_deadline 0.24 ± 17% +0.4 0.66 ± 36% perf-profile.children.cycles-pp.irq_enter_rcu 0.33 ± 11% +0.5 0.79 ± 49% perf-profile.children.cycles-pp.rebalance_domains 0.56 ± 35% +0.9 1.49 ± 60% perf-profile.children.cycles-pp.tick_nohz_next_event 1.02 ± 24% +1.1 2.14 ± 59% perf-profile.children.cycles-pp.clockevents_program_event 0.66 ± 34% +1.1 1.80 ± 58% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 1.18 ± 23% +2.2 3.38 ± 49% perf-profile.children.cycles-pp.menu_select 1.40 ± 14% -0.6 0.79 ± 43% perf-profile.self.cycles-pp.clear_page_erms 1.94 ± 18% -0.6 1.34 ± 9% perf-profile.self.cycles-pp.native_irq_return_iret 0.94 ± 17% -0.5 0.47 ± 47% perf-profile.self.cycles-pp.__orc_find 0.75 ± 21% -0.4 0.37 ± 39% perf-profile.self.cycles-pp.orc_find 0.89 ± 12% -0.4 0.53 ± 43% perf-profile.self.cycles-pp.unwind_next_frame 0.41 ± 25% -0.2 0.20 ± 53% perf-profile.self.cycles-pp.update_curr 0.26 ± 21% -0.2 0.08 ± 89% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.27 ± 23% -0.2 0.10 ± 73% perf-profile.self.cycles-pp.perf_output_sample 0.37 ± 30% -0.2 0.20 ± 52% perf-profile.self.cycles-pp.update_load_avg 0.26 ± 25% -0.2 0.09 ± 48% perf-profile.self.cycles-pp.perf_output_begin_forward 0.24 ± 21% -0.1 0.10 ± 46% perf-profile.self.cycles-pp.__unwind_start 0.21 ± 31% -0.1 0.08 ± 46% perf-profile.self.cycles-pp.account_user_time 0.30 ± 11% -0.1 0.17 ± 45% perf-profile.self.cycles-pp.alloc_vmap_area 0.17 ± 24% -0.1 0.06 ± 70% perf-profile.self.cycles-pp.bpf_ksym_find 0.24 ± 23% -0.1 0.12 ± 44% perf-profile.self.cycles-pp.perf_tp_event_match 0.15 ± 40% -0.1 0.04 ±101% perf-profile.self.cycles-pp.task_tick_fair 0.19 ± 27% -0.1 0.09 ± 64% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime 0.18 ± 10% -0.1 0.09 ± 46% perf-profile.self.cycles-pp.__schedule 0.12 ± 30% -0.1 0.04 ± 64% perf-profile.self.cycles-pp.ftrace_ops_trampoline 0.14 ± 27% -0.1 0.06 ± 50% perf-profile.self.cycles-pp.__update_load_avg_se 0.12 ± 26% -0.1 0.04 ± 86% perf-profile.self.cycles-pp.cpuacct_charge 0.11 ± 21% -0.1 0.04 ± 65% perf-profile.self.cycles-pp.perf_output_copy 0.16 ± 29% -0.1 0.10 ± 44% perf-profile.self.cycles-pp.___perf_sw_event 0.12 ± 33% -0.1 0.06 ± 72% perf-profile.self.cycles-pp.cmp_ex_search 0.11 ± 25% -0.1 0.04 ± 67% perf-profile.self.cycles-pp.update_min_vruntime 0.10 ± 9% -0.1 0.04 ± 89% perf-profile.self.cycles-pp.perf_swevent_get_recursion_context 0.10 ± 11% -0.1 0.04 ± 65% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.10 ± 25% -0.1 0.04 ± 66% perf-profile.self.cycles-pp.update_cfs_group 0.08 ± 17% -0.0 0.03 ± 88% perf-profile.self.cycles-pp.__switch_to_asm 0.09 ± 11% -0.0 0.05 ± 65% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm 0.09 ± 17% -0.0 0.06 ± 12% perf-profile.self.cycles-pp.rwsem_down_write_slowpath 0.00 +0.1 0.06 ± 26% perf-profile.self.cycles-pp.tick_nohz_tick_stopped 0.04 ± 71% +0.1 0.12 ± 35% perf-profile.self.cycles-pp.rcu_dynticks_inc 0.19 ± 11% +0.1 0.28 ± 39% perf-profile.self.cycles-pp.arch_scale_freq_tick 0.04 ± 71% +0.1 0.16 ± 91% perf-profile.self.cycles-pp.sysvec_apic_timer_interrupt 0.30 ± 9% +0.1 0.43 ± 17% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.09 ± 19% +0.2 0.26 ± 31% perf-profile.self.cycles-pp.do_idle 0.21 ± 17% +0.2 0.42 ± 52% perf-profile.self.cycles-pp.read_tsc 0.28 ± 10% +0.2 0.51 ± 41% perf-profile.self.cycles-pp.native_sched_clock 0.36 ± 9% +0.4 0.77 ± 51% perf-profile.self.cycles-pp.lapic_next_deadline 1.02 ± 29% +0.9 1.96 ± 54% perf-profile.self.cycles-pp.ktime_get 0.43 ± 21% +0.9 1.38 ± 44% perf-profile.self.cycles-pp.menu_select 0.53 ± 12% +1.1 1.65 ± 50% perf-profile.self.cycles-pp.cpuidle_enter_state phoronix-test-suite.neatbench.CPU.fps 30 +----------------------------------------------------------------------+ | O O O O O O O O O O OO O O O O O O O O O O | 25 |-+ O O O O O O O O O O O O O | | .+.+.+.+.+.+.+. .+ .+.+ .+.+.+. .+.+.+ | | +.+ +.+.+ : +.+ : +.+ +.+ | 20 |-: : : : : | | : : : : : | 15 |:+ : : : : | |: : : : : | 10 |:+ : : :: | |: : : :: | |: : : :: | 5 |-+ : : | | : : | 0 +----------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month, 3 weeks

1
0
0 / 0

[mm] f73a14220a: xfstests.btrfs.101.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: f73a14220ada7779b997116bd5c096ffb854123a ("mm: Probe for sub-page faults in fault_in_writeable()") https://git.kernel.org/cgit/linux/kernel/git/arm64/linux.git devel/btrfs-live-lock-fix in testcase: xfstests version: xfstests-x86_64-99bc497-1_20211129 with following parameters: disk: 6HDD fs: btrfs test: btrfs-group-10 ucode: 0x28 test-description: xfstests is a regression test suite for xfs and other files ystems. test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git on test machine: 8 threads 1 sockets Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 8G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> 2021-11-29 20:29:13 export TEST_DIR=/fs/sda1 2021-11-29 20:29:13 export TEST_DEV=/dev/sda1 2021-11-29 20:29:13 export FSTYP=btrfs 2021-11-29 20:29:13 export SCRATCH_MNT=/fs/scratch 2021-11-29 20:29:13 mkdir /fs/scratch -p 2021-11-29 20:29:13 export SCRATCH_DEV_POOL="/dev/sda2 /dev/sda3 /dev/sda4 /dev/sda5 /dev/sda6" 2021-11-29 20:29:13 sed "s:^:btrfs/:" //lkp/benchmarks/xfstests/tests/btrfs-group-10 2021-11-29 20:29:13 ./check btrfs/100 btrfs/101 btrfs/102 btrfs/103 btrfs/104 btrfs/105 btrfs/106 btrfs/107 btrfs/108 btrfs/109 FSTYP -- btrfs PLATFORM -- Linux/x86_64 lkp-hsw-d01 5.16.0-rc2-00002-gf73a14220ada #1 SMP Sun Nov 28 21:10:09 CST 2021 MKFS_OPTIONS -- /dev/sda2 MOUNT_OPTIONS -- /dev/sda2 /fs/scratch btrfs/100 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/100.out.bad) --- tests/btrfs/100.out 2021-11-29 16:37:46.000000000 +0000 +++ /lkp/benchmarks/xfstests/results//btrfs/100.out.bad 2021-11-29 20:29:15.865210474 +0000 @@ -1,11 +1,3 @@ QA output created by 100 -Label: none uuid: <UUID> - Total devices <NUM> FS bytes used <SIZE> - devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV - devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test - -Label: none uuid: <UUID> ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/100.out /lkp/benchmarks/xfstests/results//btrfs/100.out.bad' to see the entire diff) btrfs/101 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/101.out.bad) --- tests/btrfs/101.out 2021-11-29 16:37:46.000000000 +0000 +++ /lkp/benchmarks/xfstests/results//btrfs/101.out.bad 2021-11-29 20:29:17.421210372 +0000 @@ -1,11 +1,3 @@ QA output created by 101 -Label: none uuid: <UUID> - Total devices <NUM> FS bytes used <SIZE> - devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV - devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test - -Label: none uuid: <UUID> ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/101.out /lkp/benchmarks/xfstests/results//btrfs/101.out.bad' to see the entire diff) btrfs/102 1s btrfs/103 2s btrfs/104 1s btrfs/105 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/105.out.bad) --- tests/btrfs/105.out 2021-11-29 16:37:46.000000000 +0000 +++ /lkp/benchmarks/xfstests/results//btrfs/105.out.bad 2021-11-29 20:29:22.532210037 +0000 @@ -5,7 +5,5 @@ 9802287a6faa01a1fd0e01732b732fca SCRATCH_MNT/mysnap1/foo/bar wrote 98304/98304 bytes at offset 0 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -File digest in the original filesystem after being replaced: -de277dfac706c359613033120349cf88 SCRATCH_MNT/mysnap2_ro/foo/bar -File digest in the new filesystem: -de277dfac706c359613033120349cf88 SCRATCH_MNT/mysnap2_ro/foo/bar ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/105.out /lkp/benchmarks/xfstests/results//btrfs/105.out.bad' to see the entire diff) btrfs/106 1s btrfs/107 1s btrfs/108 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/108.out.bad) --- tests/btrfs/108.out 2021-11-29 16:37:46.000000000 +0000 +++ /lkp/benchmarks/xfstests/results//btrfs/108.out.bad 2021-11-29 20:29:26.064209806 +0000 @@ -8,6 +8,5 @@ File digests in the original filesystem: fbf36a062ffcbd644b5739c4d683ccc7 SCRATCH_MNT/snap/foo 5d2c92827a70aad932cfe7363105c55e SCRATCH_MNT/snap/bar -File digests in the new filesystem: -fbf36a062ffcbd644b5739c4d683ccc7 SCRATCH_MNT/snap/foo -5d2c92827a70aad932cfe7363105c55e SCRATCH_MNT/snap/bar +failed: '/bin/btrfs send -f /fs/sda1/btrfs-test-108/1.snap /fs/scratch/snap' ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/108.out /lkp/benchmarks/xfstests/results//btrfs/108.out.bad' to see the entire diff) btrfs/109 [failed, exit status 1]- output mismatch (see /lkp/benchmarks/xfstests/results//btrfs/109.out.bad) --- tests/btrfs/109.out 2021-11-29 16:37:46.000000000 +0000 +++ /lkp/benchmarks/xfstests/results//btrfs/109.out.bad 2021-11-29 20:29:27.005209745 +0000 @@ -8,6 +8,5 @@ File digests in the original filesystem: 253f558dbd25727d7d2fdb77f9ad2590 SCRATCH_MNT/snap/foo 253f558dbd25727d7d2fdb77f9ad2590 SCRATCH_MNT/snap/bar -File digests in the new filesystem: -253f558dbd25727d7d2fdb77f9ad2590 SCRATCH_MNT/snap/foo -253f558dbd25727d7d2fdb77f9ad2590 SCRATCH_MNT/snap/bar +failed: '/bin/btrfs send -f /fs/sda1/btrfs-test-109/1.snap /fs/scratch/snap' ... (Run 'diff -u /lkp/benchmarks/xfstests/tests/btrfs/109.out /lkp/benchmarks/xfstests/results//btrfs/109.out.bad' to see the entire diff) Ran: btrfs/100 btrfs/101 btrfs/102 btrfs/103 btrfs/104 btrfs/105 btrfs/106 btrfs/107 btrfs/108 btrfs/109 Failures: btrfs/100 btrfs/101 btrfs/105 btrfs/108 btrfs/109 Failed 5 of 10 tests To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month, 3 weeks

1
0
0 / 0

[loop] c895b784c6: INFO:rcu_sched_self-detected_stall_on_CPU

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: c895b784c699224d690c7dfbdcff309df82366e3 ("loop: don't hold lo_mutex during __loop_clr_fd()") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: stress-ng version: stress-ng-x86_64-0.11-06_20211126 with following parameters: nr_threads: 100% disk: 1HDD testtime: 60s class: device test: loop cpufreq_governor: performance ucode: 0xb000280 on test machine: 96 threads 2 sockets Ice Lake with 256G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 194.222111][ C71] rcu: INFO: rcu_sched self-detected stall on CPU [ 194.222114][ C71] rcu: 71-....: (14353 ticks this GP) idle=dfb/1/0x4000000000000000 softirq=9274/9274 fqs=21431 [ 194.222119][ C71] (t=100001 jiffies g=12029 q=232127) [ 194.222123][ C71] NMI backtrace for cpu 71 [ 194.222125][ C71] CPU: 71 PID: 5857 Comm: stress-ng Not tainted 5.16.0-rc2-00042-gc895b784c699 #1 [ 194.222127][ C71] Call Trace: [ 194.222128][ C71] <IRQ> [ 194.222129][ C71] dump_stack_lvl (lib/dump_stack.c:107) [ 194.222137][ C71] nmi_cpu_backtrace.cold (lib/nmi_backtrace.c:113) [ 194.222139][ C71] ? lapic_can_unplug_cpu (arch/x86/kernel/apic/hw_nmi.c:33) [ 194.222143][ C71] nmi_trigger_cpumask_backtrace (lib/nmi_backtrace.c:62) [ 194.222147][ C71] rcu_dump_cpu_stacks (kernel/rcu/tree_stall.h:339 (discriminator 5)) [ 194.222151][ C71] rcu_sched_clock_irq.cold (kernel/rcu/tree_stall.h:629 kernel/rcu/tree_stall.h:711 kernel/rcu/tree.c:3878 kernel/rcu/tree.c:2597) [ 194.222153][ C71] update_process_times (arch/x86/include/asm/preempt.h:27 kernel/time/timer.c:1787) [ 194.222158][ C71] tick_sched_handle+0x1f/0x80 [ 194.222162][ C71] tick_sched_timer (kernel/time/tick-sched.c:1426) [ 194.222163][ C71] ? tick_sched_do_timer (kernel/time/tick-sched.c:1408) [ 194.222164][ C71] __hrtimer_run_queues (kernel/time/hrtimer.c:1685 kernel/time/hrtimer.c:1749) [ 194.222166][ C71] hrtimer_interrupt (kernel/time/hrtimer.c:1814) [ 194.222168][ C71] __sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1086 arch/x86/kernel/apic/apic.c:1103) [ 194.222172][ C71] sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1097 (discriminator 14)) [ 194.222176][ C71] </IRQ> [ 194.222177][ C71] <TASK> [ 194.222178][ C71] asm_sysvec_apic_timer_interrupt (arch/x86/include/asm/idtentry.h:638) [ 194.222183][ C71] RIP: 0010:console_unlock (kernel/printk/printk.c:2719) [ 194.222187][ C71] Code: 00 44 89 e6 48 c7 c7 40 72 c3 82 e8 99 3c 01 00 65 ff 0d d2 f9 eb 7e e9 dd fd ff ff e8 88 1b 00 00 4d 85 ed 0f 85 e3 00 00 00 <8b> 44 24 0c 85 c0 0f 84 e4 fc ff ff 31 d2 be a0 0a 00 00 48 c7 c7 All code ======== 0: 00 44 89 e6 add %al,-0x1a(%rcx,%rcx,4) 4: 48 c7 c7 40 72 c3 82 mov $0xffffffff82c37240,%rdi b: e8 99 3c 01 00 callq 0x13ca9 10: 65 ff 0d d2 f9 eb 7e decl %gs:0x7eebf9d2(%rip) # 0x7eebf9e9 17: e9 dd fd ff ff jmpq 0xfffffffffffffdf9 1c: e8 88 1b 00 00 callq 0x1ba9 21: 4d 85 ed test %r13,%r13 24: 0f 85 e3 00 00 00 jne 0x10d 2a:* 8b 44 24 0c mov 0xc(%rsp),%eax <-- trapping instruction 2e: 85 c0 test %eax,%eax 30: 0f 84 e4 fc ff ff je 0xfffffffffffffd1a 36: 31 d2 xor %edx,%edx 38: be a0 0a 00 00 mov $0xaa0,%esi 3d: 48 rex.W 3e: c7 .byte 0xc7 3f: c7 .byte 0xc7 Code starting with the faulting instruction =========================================== 0: 8b 44 24 0c mov 0xc(%rsp),%eax 4: 85 c0 test %eax,%eax 6: 0f 84 e4 fc ff ff je 0xfffffffffffffcf0 c: 31 d2 xor %edx,%edx e: be a0 0a 00 00 mov $0xaa0,%esi 13: 48 rex.W 14: c7 .byte 0xc7 15: c7 .byte 0xc7 [ 194.222189][ C71] RSP: 0018:ffa00000222dbb50 EFLAGS: 00000206 [ 194.222191][ C71] RAX: 0000000000000000 RBX: ffffffff8357d8b8 RCX: 0000000000000008 [ 194.222192][ C71] RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffffffff8357d8b8 [ 194.222193][ C71] RBP: 000000000000004d R08: 0000000000000300 R09: 0000000000000000 [ 194.222193][ C71] R10: 0000000000000001 R11: ffffffffc0598080 R12: 0000000000000000 [ 194.222194][ C71] R13: 0000000000000200 R14: 0000000000000000 R15: 0000000000000000 [ 194.222197][ C71] vprintk_emit (arch/x86/include/asm/preempt.h:85 kernel/printk/printk.c:2246) [ 194.222200][ C71] ? kernfs_add_one (fs/kernfs/dir.c:766) [ 194.222202][ C71] _printk (kernel/printk/printk.c:2270) [ 194.222204][ C71] set_capacity_and_notify.cold (block/genhd.c:96) [ 194.222207][ C71] loop_set_size+0x11/0x40 loop [ 194.222211][ C71] loop_configure (drivers/block/loop.c:1057) loop [ 194.222213][ C71] lo_ioctl (drivers/block/loop.c:1546) loop [ 194.222216][ C71] blkdev_ioctl (block/ioctl.c:588) [ 194.222221][ C71] ? do_sys_openat2 (fs/open.c:1222) [ 194.222225][ C71] __x64_sys_ioctl (fs/ioctl.c:51 fs/ioctl.c:874 fs/ioctl.c:860 fs/ioctl.c:860) [ 194.222230][ C71] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) [ 194.222231][ C71] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:113) [ 194.222233][ C71] RIP: 0033:0x7f40e173b427 [ 194.222235][ C71] Code: 00 00 90 48 8b 05 69 aa 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 0c 00 f7 d8 64 89 01 48 All code ======== 0: 00 00 add %al,(%rax) 2: 90 nop 3: 48 8b 05 69 aa 0c 00 mov 0xcaa69(%rip),%rax # 0xcaa73 a: 64 c7 00 26 00 00 00 movl $0x26,%fs:(%rax) 11: 48 c7 c0 ff ff ff ff mov $0xffffffffffffffff,%rax 18: c3 retq 19: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1) 20: 00 00 00 23: b8 10 00 00 00 mov $0x10,%eax 28: 0f 05 syscall 2a:* 48 3d 01 f0 ff ff cmp $0xfffffffffffff001,%rax <-- trapping instruction 30: 73 01 jae 0x33 32: c3 retq 33: 48 8b 0d 39 aa 0c 00 mov 0xcaa39(%rip),%rcx # 0xcaa73 3a: f7 d8 neg %eax 3c: 64 89 01 mov %eax,%fs:(%rcx) 3f: 48 rex.W Code starting with the faulting instruction =========================================== 0: 48 3d 01 f0 ff ff cmp $0xfffffffffffff001,%rax 6: 73 01 jae 0x9 8: c3 retq 9: 48 8b 0d 39 aa 0c 00 mov 0xcaa39(%rip),%rcx # 0xcaa49 10: f7 d8 neg %eax 12: 64 89 01 mov %eax,%fs:(%rcx) 15: 48 rex.W [ 194.222236][ C71] RSP: 002b:00007ffdaea025a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 194.222238][ C71] RAX: ffffffffffffffda RBX: 0000000000000028 RCX: 00007f40e173b427 [ 194.222238][ C71] RDX: 0000000000000005 RSI: 0000000000004c00 RDI: 0000000000000007 [ 194.222239][ C71] RBP: 0000000000000006 R08: 0000000000000000 R09: 00007ffdaea02316 [ 194.222240][ C71] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000007 [ 194.222240][ C71] R13: 00000000000003e8 R14: 00007ffdaea06980 R15: 00007ffdaea038b0 [ 194.222242][ C71] </TASK> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang

1 month, 3 weeks

4
3
0 / 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP November 2021