kernel_optimize_test/mm
Michel Lespinasse fed067da46 mlock: only hold mmap_sem in shared mode when faulting in pages
Currently mlock() holds mmap_sem in exclusive mode while the pages get
faulted in.  In the case of a large mlock, this can potentially take a
very long time, during which various commands such as 'ps auxw' will
block.  This makes sysadmins unhappy:

real    14m36.232s
user    0m0.003s
sys     0m0.015s
(output from 'time ps auxw' while a 20GB file was being mlocked without
being previously preloaded into page cache)

I propose that mlock() could release mmap_sem after the VM_LOCKED bits
have been set in all appropriate VMAs.  Then a second pass could be done
to actually mlock the pages, in small batches, releasing mmap_sem when we
block on disk access or when we detect some contention.

This patch:

Before this change, mlock() holds mmap_sem in exclusive mode while the
pages get faulted in.  In the case of a large mlock, this can potentially
take a very long time.  Various things will block while mmap_sem is held,
including 'ps auxw'.  This can make sysadmins angry.

I propose that mlock() could release mmap_sem after the VM_LOCKED bits
have been set in all appropriate VMAs.  Then a second pass could be done
to actually mlock the pages with mmap_sem held for reads only.  We need to
recheck the vma flags after we re-acquire mmap_sem, but this is easy.

In the case where a vma has been munlocked before mlock completes, pages
that were already marked as PageMlocked() are handled by the munlock()
call, and mlock() is careful to not mark new page batches as PageMlocked()
after the munlock() call has cleared the VM_LOCKED vma flags.  So, the end
result will be identical to what'd happen if munlock() had executed after
the mlock() call.

In a later change, I will allow the second pass to release mmap_sem when
blocking on disk accesses or when it is otherwise contended, so that it
won't be held for long periods of time even in shared mode.

Signed-off-by: Michel Lespinasse <walken@google.com>
Tested-by: Valdis Kletnieks <Valdis.Kletnieks@vt.edu>
Cc: Hugh Dickins <hughd@google.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Nick Piggin <npiggin@kernel.dk>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2011-01-13 17:32:35 -08:00
..
backing-dev.c
bootmem.c
bounce.c
compaction.c mm: compaction: perform a faster migration scan when migrating asynchronously 2011-01-13 17:32:34 -08:00
debug-pagealloc.c
dmapool.c
fadvise.c
failslab.c
filemap_xip.c
filemap.c mm: clear PageError bit in msync & fsync 2011-01-13 17:32:35 -08:00
fremap.c
highmem.c
hugetlb.c
hwpoison-inject.c
init-mm.c
internal.h
Kconfig
Kconfig.debug
kmemcheck.c
kmemleak-test.c
kmemleak.c
ksm.c
maccess.c
madvise.c
Makefile
memblock.c
memcontrol.c memcg: fix wrong VM_BUG_ON() in try_charge()'s mm->owner check 2010-12-30 10:07:06 -08:00
memory_hotplug.c mm: migration: cleanup migrate_pages API by matching types for offlining and sync 2011-01-13 17:32:34 -08:00
memory-failure.c mm: migration: allow migration to operate asynchronously and avoid synchronous compaction in the faster path 2011-01-13 17:32:34 -08:00
memory.c mlock: avoid dirtying pages and triggering writeback 2011-01-13 17:32:35 -08:00
mempolicy.c mm: migration: cleanup migrate_pages API by matching types for offlining and sync 2011-01-13 17:32:34 -08:00
mempool.c
migrate.c mm: migration: cleanup migrate_pages API by matching types for offlining and sync 2011-01-13 17:32:34 -08:00
mincore.c
mlock.c mlock: only hold mmap_sem in shared mode when faulting in pages 2011-01-13 17:32:35 -08:00
mm_init.c
mmap.c
mmu_context.c
mmu_notifier.c
mmzone.c mm: page allocator: adjust the per-cpu counter threshold when memory is low 2011-01-13 17:32:31 -08:00
mprotect.c
mremap.c
msync.c
nommu.c
oom_kill.c
page_alloc.c mm: migration: allow migration to operate asynchronously and avoid synchronous compaction in the faster path 2011-01-13 17:32:34 -08:00
page_cgroup.c
page_io.c
page_isolation.c
page-writeback.c mm/page-writeback.c: fix __set_page_dirty_no_writeback() return value 2011-01-13 17:32:32 -08:00
pagewalk.c
percpu-km.c
percpu-vm.c mm: remove gfp mask from pcpu_get_vm_areas 2011-01-13 17:32:34 -08:00
percpu.c Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2011-01-13 10:05:56 -08:00
prio_tree.c
quicklist.c
readahead.c
rmap.c
shmem.c fs: icache RCU free inodes 2011-01-07 17:50:26 +11:00
slab.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/slab-2.6 2011-01-10 08:38:01 -08:00
slob.c kernel: kmem_ptr_validate considered harmful 2011-01-07 17:50:16 +11:00
slub.c mm: convert sprintf_symbol to %pS 2011-01-13 17:32:33 -08:00
sparse-vmemmap.c
sparse.c
swap_state.c
swap.c
swapfile.c
thrash.c
truncate.c
util.c kernel: kmem_ptr_validate considered harmful 2011-01-07 17:50:16 +11:00
vmalloc.c mm: unify module_alloc code for vmalloc 2011-01-13 17:32:34 -08:00
vmscan.c mm: vmscan: rename lumpy_mode to reclaim_mode 2011-01-13 17:32:34 -08:00
vmstat.c mm: vmstat: use a single setter function and callback for adjusting percpu thresholds 2011-01-13 17:32:31 -08:00