kernel_optimize_test/mm
Johannes Weiner 527a5ec9a5 mm: memcg: per-priority per-zone hierarchy scan generations
Memory cgroup limit reclaim currently picks one memory cgroup out of the
target hierarchy, remembers it as the last scanned child, and reclaims
all zones in it with decreasing priority levels.

The new hierarchy reclaim code will pick memory cgroups from the same
hierarchy concurrently from different zones and priority levels, it
becomes necessary that hierarchy roots not only remember the last
scanned child, but do so for each zone and priority level.

Until now, we reclaimed memcgs like this:

    mem = mem_cgroup_iter(root)
    for each priority level:
      for each zone in zonelist:
        reclaim(mem, zone)

But subsequent patches will move the memcg iteration inside the loop
over the zones:

    for each priority level:
      for each zone in zonelist:
        mem = mem_cgroup_iter(root)
        reclaim(mem, zone)

And to keep with the original scan order - memcg -> priority -> zone -
the last scanned memcg has to be remembered per zone and per priority
level.

Furthermore, global reclaim will be switched to the hierarchy walk as
well.  Different from limit reclaim, which can just recheck the limit
after some reclaim progress, its target is to scan all memcgs for the
desired zone pages, proportional to the memcg size, and so reliably
detecting a full hierarchy round-trip will become crucial.

Currently, the code relies on one reclaimer encountering the same memcg
twice, but that is error-prone with concurrent reclaimers.  Instead, use
a generation counter that is increased every time the child with the
highest ID has been visited, so that reclaimers can stop when the
generation changes.

Signed-off-by: Johannes Weiner <jweiner@redhat.com>
Reviewed-by: Kirill A. Shutemov <kirill@shutemov.name>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: Ying Han <yinghan@google.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Michel Lespinasse <walken@google.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-01-12 20:13:04 -08:00
..
backing-dev.c freezer: implement and use kthread_freezable_should_stop() 2011-11-21 12:32:23 -08:00
bootmem.c mm: bootmem: try harder to free pages in bulk 2012-01-10 16:30:45 -08:00
bounce.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
cleancache.c
compaction.c mm: compaction: push isolate search base of compact control one pfn ahead 2012-01-10 16:30:44 -08:00
debug-pagealloc.c mm, x86: Remove debug_pagealloc_enabled 2011-12-06 09:24:07 +01:00
dmapool.c mm: fix implicit stat.h usage in dmapool.c 2011-10-31 09:20:12 -04:00
fadvise.c fadvise: only initiate writeback for specified range with FADV_DONTNEED 2012-01-10 16:30:43 -08:00
failslab.c switch debugfs to umode_t 2012-01-03 22:54:56 -05:00
filemap_xip.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
filemap.c memcg: add mem_cgroup_replace_page_cache() to fix LRU issue 2012-01-12 20:13:04 -08:00
fremap.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
highmem.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
huge_memory.c thp: reduce khugepaged freezing latency 2011-12-09 07:50:28 -08:00
hugetlb.c mm/hugetlb.c: avoid bogus counter of surplus huge page 2012-01-10 16:30:45 -08:00
hwpoison-inject.c
init-mm.c
internal.h mm: thp: tail page refcounting fix 2011-11-02 16:06:57 -07:00
Kconfig Merge branch 'master' into x86/memblock 2011-11-28 09:46:22 -08:00
Kconfig.debug mm: more intensive memory corruption debugging 2012-01-10 16:30:42 -08:00
kmemcheck.c
kmemleak-test.c
kmemleak.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
ksm.c oom: fix race while temporarily setting current's oom_score_adj 2011-10-31 17:30:45 -07:00
maccess.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
madvise.c
Makefile Cross Memory Attach 2011-10-31 17:30:44 -07:00
memblock.c memblock: Reimplement memblock allocation using reverse free area iterator 2011-12-08 10:22:09 -08:00
memcontrol.c mm: memcg: per-priority per-zone hierarchy scan generations 2012-01-12 20:13:04 -08:00
memory_hotplug.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
memory-failure.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
memory.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
mempolicy.c mm/mempolicy.c: mpol_equal(): use bool 2012-01-10 16:30:45 -08:00
mempool.c mempool: fix first round failure behavior 2012-01-10 16:30:45 -08:00
migrate.c mm/migrate.c: remove the unused macro lru_to_page 2012-01-10 16:30:46 -08:00
mincore.c mm: clarify the radix_tree exceptional cases 2011-08-03 14:25:24 -10:00
mlock.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
mm_init.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
mmap.c mm: simplify find_vma_prev() 2012-01-10 16:30:44 -08:00
mmu_context.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
mmu_notifier.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
mmzone.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
mprotect.c
mremap.c mremap: enforce rmap src/dst vma ordering in case of vma_merge() succeeding in copy_vma() 2012-01-10 16:30:44 -08:00
msync.c
nobootmem.c Merge branch 'master' into x86/memblock 2011-11-28 09:46:22 -08:00
nommu.c xen: map foreign pages for shared rings by updating the PTEs directly 2011-11-16 12:13:08 -05:00
oom_kill.c tracepoint: add tracepoints for debugging oom_score_adj 2012-01-10 16:30:44 -08:00
page_alloc.c mm: page_alloc: generalize order handling in __free_pages_bootmem() 2012-01-10 16:30:44 -08:00
page_cgroup.c mm/page_cgroup.c: quiet sparse noise 2011-11-02 16:07:00 -07:00
page_io.c
page_isolation.c
page-writeback.c Merge branch 'writeback-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/wfg/linux 2012-01-10 16:59:59 -08:00
pagewalk.c
percpu-km.c
percpu-vm.c percpu: fix chunk range calculation 2011-11-22 08:09:46 -08:00
percpu.c percpu: fix per_cpu_ptr_to_phys() handling of non-page-aligned addresses 2011-12-15 11:41:40 -08:00
pgtable-generic.c
prio_tree.c
process_vm_access.c Cross Memory Attach 2011-10-31 17:30:44 -07:00
quicklist.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
readahead.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
rmap.c mremap: enforce rmap src/dst vma ordering in case of vma_merge() succeeding in copy_vma() 2012-01-10 16:30:44 -08:00
shmem.c vfs: switch ->show_options() to struct dentry * 2012-01-06 23:19:54 -05:00
slab.c Merge branch 'slab/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/linux 2012-01-11 18:52:23 -08:00
slob.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
slub.c mm,x86,um: move CMPXCHG_DOUBLE config option 2012-01-12 20:13:03 -08:00
sparse-vmemmap.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
sparse.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
swap_state.c fs: move code out of buffer.c 2012-01-03 22:54:07 -05:00
swap.c mm: add free_hot_cold_page_list() helper 2012-01-10 16:30:41 -08:00
swapfile.c mm: avoid livelock on !__GFP_FS allocations 2012-01-10 16:30:42 -08:00
thrash.c mm/thrash.c: quiet sparse noise 2011-10-31 17:30:50 -07:00
truncate.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
util.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
vmalloc.c mm/vmalloc.c: change void* into explict vm_struct* 2012-01-10 16:30:46 -08:00
vmscan.c mm: vmscan: distinguish between memcg triggering reclaim and memcg being scanned 2012-01-12 20:13:04 -08:00
vmstat.c mm,x86,um: move CMPXCHG_LOCAL config option 2012-01-12 20:13:03 -08:00