| 89cbc76768c2fa4ed95545bf961f3a14ddfeed21 |
|
17-Aug-2014 |
Christoph Lameter <cl@linux.com> |
x86: Replace __get_cpu_var uses
__get_cpu_var() is used for multiple purposes in the kernel source. One of
them is address calculation via the form &__get_cpu_var(x). This calculates
the address for the instance of the percpu variable of the current processor
based on an offset.
Other use cases are for storing and retrieving data from the current
processors percpu area. __get_cpu_var() can be used as an lvalue when
writing data or on the right side of an assignment.
__get_cpu_var() is defined as :
#define __get_cpu_var(var) (*this_cpu_ptr(&(var)))
__get_cpu_var() always only does an address determination. However, store
and retrieve operations could use a segment prefix (or global register on
other platforms) to avoid the address calculation.
this_cpu_write() and this_cpu_read() can directly take an offset into a
percpu area and use optimized assembly code to read and write per cpu
variables.
This patch converts __get_cpu_var into either an explicit address
calculation using this_cpu_ptr() or into a use of this_cpu operations that
use the offset. Thereby address calculations are avoided and less registers
are used when code is generated.
Transformations done to __get_cpu_var()
1. Determine the address of the percpu instance of the current processor.
DEFINE_PER_CPU(int, y);
int *x = &__get_cpu_var(y);
Converts to
int *x = this_cpu_ptr(&y);
2. Same as #1 but this time an array structure is involved.
DEFINE_PER_CPU(int, y[20]);
int *x = __get_cpu_var(y);
Converts to
int *x = this_cpu_ptr(y);
3. Retrieve the content of the current processors instance of a per cpu
variable.
DEFINE_PER_CPU(int, y);
int x = __get_cpu_var(y)
Converts to
int x = __this_cpu_read(y);
4. Retrieve the content of a percpu struct
DEFINE_PER_CPU(struct mystruct, y);
struct mystruct x = __get_cpu_var(y);
Converts to
memcpy(&x, this_cpu_ptr(&y), sizeof(x));
5. Assignment to a per cpu variable
DEFINE_PER_CPU(int, y)
__get_cpu_var(y) = x;
Converts to
__this_cpu_write(y, x);
6. Increment/Decrement etc of a per cpu variable
DEFINE_PER_CPU(int, y);
__get_cpu_var(y)++
Converts to
__this_cpu_inc(y)
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: x86@kernel.org
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Christoph Lameter <cl@linux.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
| 2a6f6d095509c7dc6e9ff8d9fd9fba0b730ecce3 |
|
18-Dec-2010 |
Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> |
xen/multicall: move *idx fields to start of mc_buffer
The CPU would prefer small offsets.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
|
| eac303bf2ef63e7f7b6971badea0ff7bf08a2b22 |
|
18-Dec-2010 |
Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> |
xen/multicall: special-case singleton hypercalls
Singleton calls seem to end up being pretty common, so just
directly call the hypercall rather than going via multicall.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
|
| 4a7b005dbfa554e7cc7fbc08e0299a9b7a91ef3b |
|
18-Dec-2010 |
Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> |
xen/multicalls: add unlikely around slowpath in __xen_mc_entry()
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
|
| ffc78767f2ac5b8007aeb366e748c8dd8b8bc3a3 |
|
18-Dec-2010 |
Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> |
xen/multicalls: disable MC_DEBUG
It's useful - and probably should be a config - but its very heavyweight,
especially with the tracing stuff to help sort out problems.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
|
| c796f213a6934712ede728d9b53ef0e5066db23a |
|
16-Dec-2010 |
Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> |
xen/trace: add multicall tracing
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
|
| 84cdee76b15f3669f012d5916287d124c805ef2f |
|
16-Dec-2010 |
Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> |
xen/multicalls: remove debugfs stats
Remove debugfs stats to make way for tracing.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
|
| f124c6ae59e193705c9ddac57684d50006d710e6 |
|
03-Jun-2011 |
Dan Carpenter <error27@gmail.com> |
xen: off by one errors in multicalls.c
b->args[] has MC_ARGS elements, so the comparison here should be
">=" instead of ">". Otherwise we read past the end of the array
one space.
CC: stable@kernel.org
Signed-off-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
|
| c99608637eac8834d830496c462c054137772122 |
|
06-Feb-2009 |
Jeremy Fitzhardinge <jeremy@goop.org> |
x86, xen: do multicall callbacks with interrupts disabled
We can't call the callbacks after enabling interrupts, as we may get a
nested multicall call, which would cause a great deal of havok.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
| 3d39e9d07b576ee72f2c94cfad9e618fe21763b2 |
|
06-Feb-2009 |
Jeremy Fitzhardinge <jeremy@goop.org> |
x86, xen: degrade BUG to WARN when multicall fails
If one of the components of a multicall fails, WARN rather than BUG,
to help with debugging.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
| b93d51dc62a41b5c2d6f32a0870709dc0cc55545 |
|
06-Feb-2009 |
Ian Campbell <ian.campbell@citrix.com> |
x86, xen: record and display initiator of each multicall when debugging
Store the caller for each multicall so we can report it on failure.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
| f63c2f248959366cd11bfa476f866737047cf663 |
|
16-Dec-2008 |
Tej <bewith.tej@gmail.com> |
xen: whitespace/checkpatch cleanup
Impact: cleanup
Signed-off-by: Tej <bewith.tej@gmail.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
| 994025caba3e6beade9bde84dd1b70d9d250f27b |
|
21-Aug-2008 |
Jeremy Fitzhardinge <jeremy@goop.org> |
xen: add debugfs support
Add support for exporting statistics on mmu updates, multicall
batching and pv spinlocks into debugfs. The base path is xen/ and
each subsystem adds its own directory: mmu, multicalls, spinlocks.
In each directory, writing 1 to "zero_stats" will cause the
corresponding stats to be zeroed the next time they're updated.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Acked-by: Jan Beulich <jbeulich@novell.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
| 8ba6c2b0958c332d2f3336f4ca9c116ed81f38e9 |
|
09-Jul-2008 |
Jeremy Fitzhardinge <jeremy@goop.org> |
xen: print backtrace on multicall failure
Print a backtrace if a multicall fails, to help with debugging.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Cc: Stephen Tweedie <sct@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
| 400d34944c4ad82a817c06e570bc93b1114aa596 |
|
16-Jun-2008 |
Jeremy Fitzhardinge <jeremy@goop.org> |
xen: add mechanism to extend existing multicalls
Some Xen hypercalls accept an array of operations to work on. In
general this is because its more efficient for the hypercall to the
work all at once rather than as separate hypercalls (even batched as a
multicall).
This patch adds a mechanism (xen_mc_extend_args()) to allocate more
argument space to the last-issued multicall, in order to extend its
argument list.
The user of this mechanism is xen/mmu.c, which uses it to extend the
args array of mmu_update. This is particularly valuable when doing
the update for a large mprotect, which goes via
ptep_modify_prot_commit(), but it also manages to batch updates to
pgd/pmds as well.
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Acked-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
| 7ebed39ff7eec204850736a662828da0b942b8c0 |
|
22-Feb-2008 |
Paolo Ciarrocchi <paolo.ciarrocchi@gmail.com> |
x86: coding style fixes to arch/x86/xen/multicalls.c
Before:
total: 2 errors, 2 warnings, 138 lines checked
After:
total: 0 errors, 2 warnings, 138 lines checked
No code changed:
arch/x86/xen/multicalls.o:
text data bss dec hex filename
887 2832 0 3719 e87 multicalls.o.before
887 2832 0 3719 e87 multicalls.o.after
md5:
cf6d72d9db6dc5a3ebe01eec9f05e95f multicalls.o.before.asm
cf6d72d9db6dc5a3ebe01eec9f05e95f multicalls.o.after.asm
Signed-off-by: Paolo Ciarrocchi <paolo.ciarrocchi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
| a122d6230e8d8ac7cffdf0bc9cc4b256b928fe49 |
|
16-Oct-2007 |
Jeremy Fitzhardinge <jeremy@xensource.com> |
xen: add some debug output for failed multicalls
Multicalls are expected to never fail, and the normal response to a
failed multicall is very terse. In the interests of better
debuggability, add some more verbose output. It may be worth turning
this off once it all seems more tested.
Signed-off-by: Jeremy Fitzhardinge <jeremy@xensource.com>
|
| 91e0c5f3dad47838cb2ecc1865ce789a0b7182b1 |
|
16-Oct-2007 |
Jeremy Fitzhardinge <jeremy@xensource.com> |
xen: add batch completion callbacks
This adds a mechanism to register a callback function to be called once
a batch of hypercalls has been issued. This is typically used to unlock
things which must remain locked until the hypercall has taken place.
[ Stable folks: pre-req for 2.6.23 bugfix "xen: deal with stale cr3
values when unpinning pagetables" ]
Signed-off-by: Jeremy Fitzhardinge <jeremy@xensource.com>
Cc: Stable Kernel <stable@kernel.org>
|
| 9702785a747aa27baf46ff504beab6528f21f2dd |
|
11-Oct-2007 |
Thomas Gleixner <tglx@linutronix.de> |
i386: move xen
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|