kernel_optimize_test/mm
Jan Beulich cdcd629869 x86: Fix and improve cmpxchg_double{,_local}()
Just like the per-CPU ones they had several
problems/shortcomings:

Only the first memory operand was mentioned in the asm()
operands, and the 2x64-bit version didn't have a memory clobber
while the 2x32-bit one did. The former allowed the compiler to
not recognize the need to re-load the data in case it had it
cached in some register, while the latter was overly
destructive.

The types of the local copies of the old and new values were
incorrect (the types of the pointed-to variables should be used
here, to make sure the respective old/new variable types are
compatible).

The __dummy/__junk variables were pointless, given that local
copies of the inputs already existed (and can hence be used for
discarded outputs).

The 32-bit variant of cmpxchg_double_local() referenced
cmpxchg16b_local().

At once also:

 - change the return value type to what it really is: 'bool'
 - unify 32- and 64-bit variants
 - abstract out the common part of the 'normal' and 'local' variants

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/4F01F12A020000780006A19B@nat28.tlf.novell.com
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2012-01-04 15:01:54 +01:00
..
backing-dev.c backing-dev: ensure wakeup_timer is deleted 2011-11-11 13:29:04 +01:00
bootmem.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
bounce.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
cleancache.c
compaction.c mm: compaction: make compact_zone_order() static 2011-10-31 17:30:49 -07:00
debug-pagealloc.c debug-pagealloc: add support for highmem pages 2011-10-31 17:30:48 -07:00
dmapool.c mm: fix implicit stat.h usage in dmapool.c 2011-10-31 09:20:12 -04:00
fadvise.c
failslab.c fault-injection: add ability to export fault_attr in arbitrary directory 2011-08-03 14:25:20 -10:00
filemap_xip.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
filemap.c vfs: __read_cache_page should use gfp argument rather than GFP_KERNEL 2011-12-21 17:02:46 -08:00
fremap.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
highmem.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
huge_memory.c thp: reduce khugepaged freezing latency 2011-12-09 07:50:28 -08:00
hugetlb.c thp: set compound tail page _count to zero 2011-12-09 07:50:28 -08:00
hwpoison-inject.c
init-mm.c atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
internal.h mm: thp: tail page refcounting fix 2011-11-02 16:06:57 -07:00
Kconfig memblock: add NO_BOOTMEM config symbol 2011-10-31 17:30:47 -07:00
Kconfig.debug
kmemcheck.c
kmemleak-test.c
kmemleak.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
ksm.c oom: fix race while temporarily setting current's oom_score_adj 2011-10-31 17:30:45 -07:00
maccess.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
madvise.c
Makefile Cross Memory Attach 2011-10-31 17:30:44 -07:00
memblock.c mm/memblock.c: quiet sparse noise 2011-10-31 17:30:50 -07:00
memcontrol.c memcg: keep root group unchanged if creation fails 2011-12-20 10:25:04 -08:00
memory_hotplug.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
memory-failure.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
memory.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
mempolicy.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
mempool.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
migrate.c mm/migrate.c: pair unlock_page() and lock_page() when migrating huge pages 2011-12-09 07:50:28 -08:00
mincore.c mm: clarify the radix_tree exceptional cases 2011-08-03 14:25:24 -10:00
mlock.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
mm_init.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
mmap.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
mmu_context.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
mmu_notifier.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
mmzone.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
mprotect.c
mremap.c thp: mremap support and TLB optimization 2011-10-31 17:30:48 -07:00
msync.c
nobootmem.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
nommu.c xen: map foreign pages for shared rings by updating the PTEs directly 2011-11-16 12:13:08 -05:00
oom_kill.c oom: fix integer overflow of points in oom_badness 2011-12-20 10:25:04 -08:00
page_alloc.c mm: Ensure that pfn_valid() is called once per pageblock when reserving pageblocks 2011-12-09 07:50:28 -08:00
page_cgroup.c mm/page_cgroup.c: quiet sparse noise 2011-11-02 16:07:00 -07:00
page_io.c
page_isolation.c
page-writeback.c writeback: set max_pause to lowest value on zero bdi_dirty 2011-12-08 10:49:29 +08:00
pagewalk.c
percpu-km.c
percpu-vm.c percpu: fix chunk range calculation 2011-11-22 08:09:46 -08:00
percpu.c percpu: fix per_cpu_ptr_to_phys() handling of non-page-aligned addresses 2011-12-15 11:41:40 -08:00
pgtable-generic.c
prio_tree.c
process_vm_access.c Cross Memory Attach 2011-10-31 17:30:44 -07:00
quicklist.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
readahead.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
rmap.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
shmem.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
slab.c slab, lockdep: Fix silly bug 2011-12-05 09:44:00 +01:00
slob.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
slub.c x86: Fix and improve cmpxchg_double{,_local}() 2012-01-04 15:01:54 +01:00
sparse-vmemmap.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
sparse.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
swap_state.c mm: delete various needless include <linux/module.h> 2011-10-31 09:20:11 -04:00
swap.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
swapfile.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
thrash.c mm/thrash.c: quiet sparse noise 2011-10-31 17:30:50 -07:00
truncate.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
util.c mm: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
vmalloc.c mm/vmalloc.c: remove static declaration of va from __get_vm_area_node 2011-12-20 10:25:04 -08:00
vmscan.c vmscan: use atomic-long for shrinker batching 2011-12-09 07:50:27 -08:00
vmstat.c mm/vmstat.c: cache align vm_stat 2011-10-31 17:30:51 -07:00