kernel_optimize_test/mm
Christoph Lameter e20b8cca76 PAGEFLAGS_EXTENDED and separate page flags for Head and Tail
Having separate page flags for the head and the tail of a compound page allows
the compiler to use bitops instead of operations on a word to check for a tail
page.  That is f.e.  important for virt_to_head_page() which is used in
various critical code paths (kfree for example):

Code for PageTail(page)

Before:

 mov    (%rdi),%rdx		page->flags
 mov    %rdx,%rax		3 bytes
 and    $0x12000,%eax		5 bytes
 cmp    $0x12000,%rax		6 bytes
 je     897 <kfree+0xa7>

After:

 mov    (%rdi),%rax
 test   $0x40,%ah			(3 bytes)
 jne    887 <kfree+0x97>

So we go from 14 bytes to 3 bytes and from 3 instructions to one.  From the
use of 2 registers we go to none.

We can only use page flags for this if we have page flags available.  This
patch introduces CONFIG_PAGEFLAGS_EXTENDED that is set if pageflags are not
scarce due to SPARSEMEM using page flags for its sectionid on 32 bit NUMA
platforms.

Additional page flag definitions can be added to the CONFIG_PAGEFLAGS_EXTENDED
section in page-flags.h if the functionality depends on PAGEFLAGS_EXTENDED or
if more page flag overlapping tricks are used for the !PAGEFLAGS_EXTENDED
fallback (the upcoming virtual compound patch may hook in here and Rik's/Lee's
additional page flags to solve the reclaim issues could also be added there
[hint...  hint...  where are these patchsets?]).

Avoiding the overlaying of Pg_reclaim also clears the way for possible use of
compound pages for the pagecache or on the LRU.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-04-28 08:58:22 -07:00
..
allocpercpu.c
backing-dev.c
bootmem.c mm: allow reserve_bootmem() cross nodes 2008-04-26 22:51:08 +02:00
bounce.c
dmapool.c dmapool: enable debugging for CONFIG_SLUB_DEBUG_ON too 2008-04-28 08:58:20 -07:00
fadvise.c
filemap_xip.c
filemap.c mm: rotate_reclaimable_page() cleanup 2008-04-28 08:58:20 -07:00
fremap.c
highmem.c
hugetlb.c hugetlb: decrease hugetlb_lock cycling in gather_surplus_huge_pages 2008-04-28 08:58:19 -07:00
internal.h
Kconfig PAGEFLAGS_EXTENDED and separate page flags for Head and Tail 2008-04-28 08:58:22 -07:00
maccess.c
madvise.c
Makefile
memcontrol.c
memory_hotplug.c hotplug-memory: make online_page() common 2008-04-28 08:58:17 -07:00
memory.c mm: remove nopage 2008-04-28 08:58:18 -07:00
mempolicy.c mempolicy: disallow static or relative flags for local preferred mode 2008-04-28 08:58:20 -07:00
mempool.c
migrate.c
mincore.c mm: remove nopage 2008-04-28 08:58:18 -07:00
mlock.c
mmap.c mmap_region: cleanup the final vma_merge() related code 2008-04-28 08:58:18 -07:00
mmzone.c mm: filter based on a nodemask as well as a gfp_mask 2008-04-28 08:58:19 -07:00
mprotect.c
mremap.c
msync.c
nommu.c
oom_kill.c mm: have zonelist contains structs with both a zone pointer and zone_idx 2008-04-28 08:58:18 -07:00
page_alloc.c pageflags: eliminate PG_xxx aliases 2008-04-28 08:58:22 -07:00
page_io.c
page_isolation.c
page-writeback.c
pagewalk.c mm: fix possible off-by-one in walk_pte_range() 2008-04-28 08:58:16 -07:00
pdflush.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/juhl/trivial 2008-04-21 16:36:46 -07:00
prio_tree.c
quicklist.c
readahead.c
rmap.c mm: remove nopage 2008-04-28 08:58:18 -07:00
shmem_acl.c
shmem.c mempolicy: fix parsing of tmpfs mpol mount option 2008-04-28 08:58:20 -07:00
slab.c mm: move cache_line_size() to <linux/cache.h> 2008-04-28 08:58:19 -07:00
slob.c
slub.c mm: move cache_line_size() to <linux/cache.h> 2008-04-28 08:58:19 -07:00
sparse-vmemmap.c
sparse.c hotplug memory remove: generic __remove_pages() support 2008-04-28 08:58:17 -07:00
swap_state.c
swap.c mm: rotate_reclaimable_page() cleanup 2008-04-28 08:58:20 -07:00
swapfile.c mm: try both endianess when checking for endianess 2008-04-28 08:58:19 -07:00
thrash.c
tiny-shmem.c
truncate.c fix invalidate_inode_pages2_range() to not clear ret 2008-04-28 08:58:18 -07:00
util.c
vmalloc.c vmallocinfo: add caller information 2008-04-28 08:58:21 -07:00
vmscan.c mm: have zonelist contains structs with both a zone pointer and zone_idx 2008-04-28 08:58:18 -07:00
vmstat.c mm: remember what the preferred zone is for zone_statistics 2008-04-28 08:58:18 -07:00