kernel_optimize_test/mm
Mel Gorman 72853e2991 mm: page allocator: update free page counters after pages are placed on the free list
When allocating a page, the system uses NR_FREE_PAGES counters to
determine if watermarks would remain intact after the allocation was made.
This check is made without interrupts disabled or the zone lock held and
so is race-prone by nature.  Unfortunately, when pages are being freed in
batch, the counters are updated before the pages are added on the list.
During this window, the counters are misleading as the pages do not exist
yet.  When under significant pressure on systems with large numbers of
CPUs, it's possible for processes to make progress even though they should
have been stalled.  This is particularly problematic if a number of the
processes are using GFP_ATOMIC as the min watermark can be accidentally
breached and in extreme cases, the system can livelock.

This patch updates the counters after the pages have been added to the
list.  This makes the allocator more cautious with respect to preserving
the watermarks and mitigates livelock possibilities.

[akpm@linux-foundation.org: avoid modifying incoming args]
Signed-off-by: Mel Gorman <mel@csn.ul.ie>
Reviewed-by: Rik van Riel <riel@redhat.com>
Reviewed-by: Minchan Kim <minchan.kim@gmail.com>
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Reviewed-by: Christoph Lameter <cl@linux.com>
Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2010-09-09 18:57:25 -07:00
..
backing-dev.c writeback: avoid unnecessary calculation of bdi dirty thresholds 2010-08-12 08:43:29 -07:00
bootmem.c x86,nobootmem: make alloc_bootmem_node fall back to other node when 32bit numa is used 2010-07-20 16:25:40 -07:00
bounce.c bounce: call flush_dcache_page() after bounce_copy_vec() 2010-09-09 18:57:25 -07:00
compaction.c mm: compaction: handle active and inactive fairly in too_many_isolated 2010-09-09 18:57:24 -07:00
debug-pagealloc.c
dmapool.c
fadvise.c readahead: introduce FMODE_RANDOM for POSIX_FADV_RANDOM 2010-03-06 11:26:25 -08:00
failslab.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
filemap_xip.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
filemap.c gcc-4.6: mm: fix unused but set warnings 2010-08-09 20:44:58 -07:00
fremap.c mm: clean up mm_counter 2010-03-06 11:26:23 -08:00
highmem.c mm,kdb,kgdb: Add a debug reference for the kdb kmap usage 2010-08-05 09:22:24 -05:00
hugetlb.c Merge branch 'hwpoison' of git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6 2010-08-12 10:15:10 -07:00
hwpoison-inject.c HWPOISON, hugetlb: support hwpoison injection for hugepage 2010-08-11 09:23:11 +02:00
init-mm.c mm: provide init_mm mm_context initializer 2010-08-09 20:44:54 -07:00
internal.h
Kconfig mm: avoid warning when COMPACTION is selected 2010-09-09 18:57:24 -07:00
Kconfig.debug
kmemcheck.c kmemcheck: Fix build errors due to missing slab.h 2010-03-30 22:02:32 +09:00
kmemleak-test.c
kmemleak.c kmemleak: Fix typo in the comment 2010-08-08 21:57:23 +01:00
ksm.c mm: fix swapin race condition 2010-09-09 18:57:24 -07:00
maccess.c
madvise.c
Makefile lmb: rename to memblock 2010-07-14 17:14:00 +10:00
memblock.c memblock: Fix memblock_is_region_reserved() to return a boolean 2010-08-09 11:21:38 +10:00
memcontrol.c memcg: convert to use zone_to_nid() from bare zone->zone_pgdat->node_id 2010-08-11 08:59:19 -07:00
memory_hotplug.c memory hotplug: fix next block calculation in is_removable 2010-09-09 18:57:24 -07:00
memory-failure.c Merge branch 'hwpoison' of git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6 2010-08-12 10:15:10 -07:00
memory.c mm: fix swapin race condition 2010-09-09 18:57:24 -07:00
mempolicy.c mempolicy: reduce stack size of migrate_pages() 2010-08-09 20:44:58 -07:00
mempool.c
migrate.c mm: extend KSM refcounts to the anon_vma root 2010-08-09 20:44:55 -07:00
mincore.c mincore: do nested page table walks 2010-05-25 08:06:58 -07:00
mlock.c mm: Move vma_stack_continue into mm.h 2010-09-09 09:05:06 -07:00
mm_init.c
mmap.c guard page for stacks that grow upwards 2010-08-24 12:13:20 -07:00
mmu_context.c exit: fix oops in sync_mm_rss 2010-03-24 16:31:21 -07:00
mmu_notifier.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
mmzone.c
mprotect.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
mremap.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
msync.c sanitize vfs_fsync calling conventions 2010-05-21 18:31:21 -04:00
nommu.c mm: make the vma list be doubly linked 2010-08-21 08:49:21 -07:00
oom_kill.c oom: __task_cred() need rcu_read_lock() 2010-08-20 09:34:55 -07:00
page_alloc.c mm: page allocator: update free page counters after pages are placed on the free list 2010-09-09 18:57:25 -07:00
page_cgroup.c kmemleak: Annotate false positive in init_section_page_cgroup() 2010-07-19 11:54:14 +01:00
page_io.c block: unify flags for struct bio and struct request 2010-08-07 18:20:39 +02:00
page_isolation.c
page-writeback.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client 2010-08-28 14:07:20 -07:00
pagewalk.c pagemap: fix pfn calculation for hugepage 2010-04-07 08:38:04 -07:00
percpu_up.c percpu: add __percpu notations to UP allocator 2010-08-07 14:20:53 +02:00
percpu-km.c percpu: implement kernel memory based chunk allocation 2010-05-01 08:30:50 +02:00
percpu-vm.c percpu: move vmalloc based chunk management into percpu-vm.c 2010-05-01 08:30:50 +02:00
percpu.c percpu: fix a mismatch between code and comment 2010-08-27 11:36:19 +02:00
prio_tree.c
quicklist.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
readahead.c readahead.c: fix comment 2010-05-25 08:07:00 -07:00
rmap.c mm: fix hang on anon_vma->root->lock 2010-08-28 13:54:12 -07:00
shmem.c shmem: put_super must percpu_counter_destroy 2010-08-17 18:33:11 -07:00
slab.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/slab-2.6 2010-08-22 10:08:52 -07:00
slob.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/slab-2.6 2010-08-06 11:44:08 -07:00
slub.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/slab-2.6 2010-08-06 11:44:08 -07:00
sparse-vmemmap.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sparse.c sparsemem: on no vmemmap path put mem_map on node high too 2010-05-25 08:06:56 -07:00
swap_state.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
swap.c mm: export lru_cache_add_*() to modules 2010-05-25 15:06:06 +02:00
swapfile.c swap: discard while swapping only if SWAP_FLAG_DISCARD 2010-09-09 18:57:25 -07:00
thrash.c
truncate.c check ATTR_SIZE contraints in inode_change_ok 2010-08-09 16:47:39 -04:00
util.c mm: use memdup_user 2010-08-09 20:44:54 -07:00
vmalloc.c Merge branch 'stable/xen-swiotlb-0.8.6' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen 2010-08-12 09:09:41 -07:00
vmscan.c memcg: remove nid and zid argument from mem_cgroup_soft_limit_reclaim() 2010-08-11 08:59:19 -07:00
vmstat.c vmstat: update zone stat threshold when onlining a cpu 2010-09-09 18:57:25 -07:00