Mel Gorman 2a791f4412 mm/page_alloc: do not prefetch buddies during bulk free ... free_pcppages_bulk() has taken two passes through the pcp lists since commit 0a5f4e5b45 ("mm/free_pcppages_bulk: do not hold lock when picking pages to free") due to deferring the cost of selecting PCP lists until the zone lock is held. As the list processing now takes place under the zone lock, it's less clear that this will always benefit for two reasons. 1. There is a guaranteed cost to calculating the buddy which definitely has to be calculated again. However, as the zone lock is held and there is no deferring of buddy merging, there is no guarantee that the prefetch will have completed when the second buddy calculation takes place and buddies are being merged. With or without the prefetch, there may be further stalls depending on how many pages get merged. In other words, a stall due to merging is inevitable and at best only one stall might be avoided at the cost of calculating the buddy location twice. 2. As the zone lock is held, prefetch_nr makes less sense as once prefetch_nr expires, the cache lines of interest have already been merged. The main concern is that there is a definite cost to calculating the buddy location early for the prefetch and it is a "maybe win" depending on whether the CPU prefetch logic and memory is fast enough. Remove the prefetch logic on the basis that reduced instructions in a path is always a saving where as the prefetch might save one memory stall depending on the CPU and memory. In most cases, this has marginal benefit as the calculations are a small part of the overall freeing of pages. However, it was detectable on at least one machine. 5.17.0-rc3 5.17.0-rc3 mm-highpcplimit-v2r1 mm-noprefetch-v1r1 Min elapsed 630.00 ( 0.00%) 610.00 ( 3.17%) Amean elapsed 639.00 ( 0.00%) 623.00 * 2.50%* Max elapsed 660.00 ( 0.00%) 660.00 ( 0.00%) Link: https://lkml.kernel.org/r/20220221094119.15282-2-mgorman@techsingularity.net Signed-off-by: Mel Gorman <mgorman@techsingularity.net> Suggested-by: Aaron Lu <aaron.lu@intel.com> Reviewed-by: Vlastimil Babka <vbabka@suse.cz> Reviewed-by: Aaron Lu <aaron.lu@intel.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Michal Hocko <mhocko@kernel.org> Cc: Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		2022-03-22 15:57:06 -07:00
..
damon	mm/damon: hide kernel pointer from tracepoint event	2022-01-15 16:30:33 +02:00
kasan	lib/stackdepot: always do filter_irq_stacks() in stack_depot_save()	2022-01-22 08:33:38 +02:00
kfence	kfence: make test case compatible with run time set sample interval	2022-02-11 17:55:00 -08:00
backing-dev.c	remove congestion tracking framework	2022-03-22 15:57:01 -07:00
balloon_compaction.c
bootmem_info.c	bootmem: Use page->index instead of page->freelist	2022-01-06 12:27:03 +01:00
cma.c	cma: factor out minimum alignment requirement	2022-03-22 15:57:05 -07:00
cma.h
cma_debug.c
cma_sysfs.c
compaction.c	mm: compaction: fix the migration stats in trace_mm_compaction_migratepages()	2022-01-15 16:30:30 +02:00
debug.c	mm,fs: split dump_mapping() out from dump_page()	2022-01-15 16:30:26 +02:00
debug_page_ref.c
debug_vm_pgtable.c	mm/debug_vm_pgtable: remove pte entry from the page table	2022-02-04 09:25:04 -08:00
dmapool.c	mm/dmapool.c: revert "make dma pool to use kmalloc_node"	2022-01-15 16:30:28 +02:00
early_ioremap.c	mm/early_ioremap.c: remove redundant early_ioremap_shutdown()	2021-09-08 11:50:24 -07:00
fadvise.c	remove inode_congested()	2022-03-22 15:57:01 -07:00
failslab.c
filemap.c	tmpfs: do not allocate pages on read	2022-03-22 15:57:02 -07:00
folio-compat.c	filemap: Add filemap_release_folio()	2022-01-04 13:15:34 -05:00
frontswap.c	frontswap: remove support for multiple ops	2022-01-22 08:33:38 +02:00
gup.c	mm/gup: remove unused get_user_pages_locked()	2022-03-22 15:57:01 -07:00
gup_test.c
gup_test.h
highmem.c	Fixes for 5.16 folios:	2021-11-25 10:13:56 -08:00
hmm.c	mm/hmm.c: allow VM_MIXEDMAP to work with hmm_range_fault	2022-01-15 16:30:31 +02:00
huge_memory.c	mm: thp: fix wrong cache flush in remove_migration_pmd()	2022-03-22 15:57:04 -07:00
hugetlb.c	mm: hugetlb: fix missing cache flush in hugetlb_mcopy_atomic_pte()	2022-03-22 15:57:04 -07:00
hugetlb_cgroup.c	hugetlb: add hugetlb.*.numa_stat file	2022-01-15 16:30:29 +02:00
hugetlb_vmemmap.c	mm: hugetlb: introduce CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON	2021-06-30 20:47:26 -07:00
hugetlb_vmemmap.h	mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate	2021-06-30 20:47:25 -07:00
hwpoison-inject.c	mm: hwpoison: don't drop slab caches for offlining non-LRU page	2021-09-03 09:58:15 -07:00
init-mm.c	mm: add setup_initial_init_mm() helper	2021-07-08 11:48:21 -07:00
internal.h	mm/sparse: make mminit_validate_memmodel_limits() static	2022-03-22 15:57:05 -07:00
interval_tree.c
io-mapping.c
ioremap.c	mm: move ioremap_page_range to vmalloc.c	2021-09-08 11:50:24 -07:00
Kconfig	mm: enforce pageblock_order < MAX_ORDER	2022-03-22 15:57:06 -07:00
Kconfig.debug	mm: page table check	2022-01-15 16:30:28 +02:00
khugepaged.c	mm/page_table_check: check entries at pmd levels	2022-02-04 09:25:04 -08:00
kmemleak.c	mm/kmemleak: avoid scanning potential huge holes	2022-02-04 09:25:05 -08:00
ksm.c	mm: ksm: fix use-after-free kasan report in ksm_might_need_to_copy	2022-01-15 16:30:31 +02:00
list_lru.c	mm: memcontrol: rename memcg_cache_id to memcg_kmem_id	2022-03-22 15:57:04 -07:00
maccess.c	ARM: 9115/1: mm/maccess: fix unaligned copy_{from,to}_kernel_nofault	2021-08-20 11:39:25 +01:00
madvise.c	mm: fix use-after-free when anon vma name is used after vma is freed	2022-03-05 11:08:32 -08:00
Makefile	mm: remove cleancache	2022-01-22 08:33:38 +02:00
mapping_dirty_helpers.c	mm: move tlb_flush_pending inline helpers to mm_inline.h	2022-01-15 16:30:27 +02:00
memblock.c	memblock: use kfree() to release kmalloced memblock regions	2022-02-20 08:45:39 +02:00
memcontrol.c	mm: memcontrol: fix cannot alloc the maximum memcg ID	2022-03-22 15:57:03 -07:00
memfd.c	memfd: fix F_SEAL_WRITE after shmem huge page allocated	2022-03-05 11:08:32 -08:00
memory-failure.c	memory-failure: fetch compound_head after pgmap_pfn_valid()	2022-01-30 09:56:58 +02:00
memory.c	mm/memory.c: use helper macro min and max in unmap_mapping_range_tree()	2022-03-22 15:57:05 -07:00
memory_hotplug.c	treewide: Add missing includes masked by cgroup -> bpf dependency	2021-12-03 10:58:13 -08:00
mempolicy.c	mm: change lookup_node() to use get_user_pages_fast()	2022-03-22 15:57:01 -07:00
mempool.c	mm: remove spurious blkdev.h includes	2021-10-18 06:17:01 -06:00
memremap.c	mm/memremap: avoid calling kasan_remove_zero_shadow() for device private memory	2022-03-22 15:57:01 -07:00
memtest.c
migrate.c	mm: replace multiple dcache flush with flush_dcache_folio()	2022-03-22 15:57:04 -07:00
mincore.c
mlock.c	mm: refactor vm_area_struct::anon_vma_name usage code	2022-03-05 11:08:32 -08:00
mm_init.c
mmap.c	mm/mmap: remove obsolete comment in ksys_mmap_pgoff	2022-03-22 15:57:05 -07:00
mmap_lock.c	mm: mmap_lock: fix disabling preemption directly	2021-07-23 17:43:28 -07:00
mmu_gather.c	mm: move tlb_flush_pending inline helpers to mm_inline.h	2022-01-15 16:30:27 +02:00
mmu_notifier.c
mmzone.c	mm/mmzone.c: use try_cmpxchg() in page_cpupid_xchg_last()	2022-03-22 15:57:05 -07:00
mprotect.c	mm: refactor vm_area_struct::anon_vma_name usage code	2022-03-05 11:08:32 -08:00
mremap.c	mm/mremap:: use vma_lookup() instead of find_vma()	2022-03-22 15:57:05 -07:00
msync.c
nommu.c	Merge branch 'akpm' (patches from Andrew)	2021-11-06 14:08:17 -07:00
oom_kill.c	Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2022-01-17 05:49:30 +02:00
page-writeback.c	mm/writeback: minor clean up for highmem_dirtyable_memory	2022-03-22 15:57:01 -07:00
page_alloc.c	mm/page_alloc: do not prefetch buddies during bulk free	2022-03-22 15:57:06 -07:00
page_counter.c	mm/page_counter: remove an incorrect call to propagate_protected_usage()	2022-01-15 16:30:27 +02:00
page_ext.c	mm: make some vars and functions static or __init	2022-01-15 16:30:31 +02:00
page_idle.c	mm/idle_page_tracking: make PG_idle reusable	2021-09-08 11:50:24 -07:00
page_io.c	delayacct: support swapin delay accounting for swapping without blkio	2022-01-20 08:52:55 +02:00
page_isolation.c	Revert "mm/page_isolation: unset migratetype directly for non Buddy page"	2022-02-04 09:25:04 -08:00
page_owner.c	lib/stackdepot: allow optional init and stack_table allocation by kvmalloc()	2022-01-22 08:33:37 +02:00
page_poison.c
page_reporting.c
page_reporting.h
page_table_check.c	mm/page_table_check: check entries at pmd levels	2022-02-04 09:25:04 -08:00
page_vma_mapped.c	mm: device exclusive memory access	2021-07-01 11:06:03 -07:00
pagewalk.c
percpu-internal.h	mm: memcg/percpu: account extra objcg space to memory cgroups	2022-01-15 16:30:31 +02:00
percpu-km.c	percpu: flush tlb in pcpu_reclaim_populated()	2021-07-04 18:30:17 +00:00
percpu-stats.c
percpu-vm.c	percpu: flush tlb in pcpu_reclaim_populated()	2021-07-04 18:30:17 +00:00
percpu.c	bitmap patches for 5.17-rc1	2022-01-23 06:20:44 +02:00
pgalloc-track.h
pgtable-generic.c	mm: move tlb_flush_pending inline helpers to mm_inline.h	2022-01-15 16:30:27 +02:00
process_vm_access.c
ptdump.c
readahead.c	remove inode_congested()	2022-03-22 15:57:01 -07:00
rmap.c	mm/rmap: fix potential batched TLB flush race	2022-01-15 16:30:31 +02:00
rodata_test.c
secretmem.c	mm/secretmem: avoid letting secretmem_users drop to zero	2021-10-28 17:18:55 -07:00
shmem.c	mm: shmem: fix missing cache flush in shmem_mfill_atomic_pte()	2022-03-22 15:57:04 -07:00
shuffle.c
shuffle.h
slab.c	mm: introduce kmem_cache_alloc_lru	2022-03-22 15:57:03 -07:00
slab.h	mm: introduce kmem_cache_alloc_lru	2022-03-22 15:57:03 -07:00
slab_common.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
slob.c	mm: introduce kmem_cache_alloc_lru	2022-03-22 15:57:03 -07:00
slub.c	mm: introduce kmem_cache_alloc_lru	2022-03-22 15:57:03 -07:00
sparse-vmemmap.c	mm: remove redundant smp_wmb()	2021-11-06 13:30:36 -07:00
sparse.c	mm/sparse: make mminit_validate_memmodel_limits() static	2022-03-22 15:57:05 -07:00
swap.c	mm/swap: fix confusing comment in folio_mark_accessed	2022-03-22 15:57:01 -07:00
swap_cgroup.c
swap_slots.c	treewide: Add missing includes masked by cgroup -> bpf dependency	2021-12-03 10:58:13 -08:00
swap_state.c	mm: swap: get rid of livelock in swapin readahead	2022-03-17 11:02:13 -07:00
swapfile.c	mm: mark swap_lock and swap_active_head static	2022-01-22 08:33:38 +02:00
truncate.c	mm: remove cleancache	2022-01-22 08:33:38 +02:00
usercopy.c	mm: Convert check_heap_object() to use struct slab	2022-01-06 12:25:51 +01:00
userfaultfd.c	mm: userfaultfd: fix missing cache flush in mcopy_atomic_pte() and __mcopy_atomic()	2022-03-22 15:57:04 -07:00
util.c	mm: Consider __GFP_NOWARN flag for oversized kvmalloc() calls	2022-03-04 10:00:37 -08:00
vmacache.c
vmalloc.c	mm/vmalloc.c: fix "unused function" warning	2022-03-22 15:57:05 -07:00
vmpressure.c	mm/vmpressure: fix data-race with memcg->socket_pressure	2021-11-06 13:30:40 -07:00
vmscan.c	remove bdi_congested() and wb_congested() and related functions	2022-03-22 15:57:01 -07:00
vmstat.c	mm/vmstat: add events for THP max_ptes_* exceeds	2022-01-15 16:30:29 +02:00
workingset.c	xarray: use kmem_cache_alloc_lru to allocate xa_node	2022-03-22 15:57:03 -07:00
z3fold.c	mm/z3fold: add kerneldoc fields for z3fold_pool	2021-07-01 11:06:03 -07:00
zbud.c	mm/zbud: add kerneldoc fields for zbud_pool	2021-07-01 11:06:03 -07:00
zpool.c	zpool: remove the list of pools_head	2022-01-15 16:30:31 +02:00
zsmalloc.c	zsmalloc: replace get_cpu_var with local_lock	2022-01-22 08:33:37 +02:00
zswap.c	frontswap: remove support for multiple ops	2022-01-22 08:33:38 +02:00