| 4e857c58efeb99393cba5a5d0d8ec7117183137c |
|
17-Mar-2014 |
Peter Zijlstra <peterz@infradead.org> |
arch: Mass conversion of smp_mb__*()
Mostly scripted conversion of the smp_mb__* barriers.
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/n/tip-55dhyhocezdw1dg7u19hmh1u@git.kernel.org
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: linux-arch@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
| c196403b79aa241c3fefb3ee5bb328aa7c5cc860 |
|
16-Jan-2014 |
Gerald Schaefer <gerald.schaefer@de.ibm.com> |
net: rds: fix per-cpu helper usage
commit ae4b46e9d "net: rds: use this_cpu_* per-cpu helper" broke per-cpu
handling for rds. chpfirst is the result of __this_cpu_read(), so it is
an absolute pointer and not __percpu. Therefore, __this_cpu_write()
should not operate on chpfirst, but rather on cache->percpu->first, just
like __this_cpu_read() did before.
Cc: <stable@vger.kernel.org> # 3.8+
Signed-off-byd Gerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| f2e9bd70327d788011cf787a51ceba5925bbc63a |
|
21-Dec-2012 |
Marciniszyn, Mike <mike.marciniszyn@intel.com> |
IB/rds: Correct ib_api use with gs_dma_address/sg_dma_len
0b088e00 ("RDS: Use page_remainder_alloc() for recv bufs")
added uses of sg_dma_len() and sg_dma_address(). This makes
RDS DOA with the qib driver.
IB ulps should use ib_sg_dma_len() and ib_sg_dma_address
respectively since some HCAs overload ib_sg_dma* operations.
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| ae4b46e9d7128d2d76e6857fe0b9fc240e8ac695 |
|
12-Nov-2012 |
Shan Wei <davidshan@tencent.com> |
net: rds: use this_cpu_* per-cpu helper
Signed-off-by: Shan Wei <davidshan@tencent.com>
Reviewed-by: Christoph Lameter <cl@linux.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| 6114eab535ab49239e0a6ce08eb9243664aef993 |
|
25-Nov-2011 |
Cong Wang <amwang@redhat.com> |
rds: remove the second argument of k[un]map_atomic()
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Cong Wang <amwang@redhat.com>
|
| 5fd5c44d3f27c93685d4a036565245f3cdb8c033 |
|
09-Feb-2012 |
Masanari Iida <standby24x7@gmail.com> |
rds: Fix typo in iw_recv.c and ib_recv.c
Correct spelling "inclue" to "include" in
net/rds/iw_recv.c and net/rds/ib_recv.c
Signed-off-by: Masanari Iida <standby24x7@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
|
| 59f740a6aeb2cde2f79fe0df38262d4c1ef35cd8 |
|
03-Aug-2010 |
Zach Brown <zach.brown@oracle.com> |
RDS/IB: print string constants in more places
This prints the constant identifier for work completion status and rdma
cm event types, like we already do for IB event types.
A core string array helper is added that each string type uses.
Signed-off-by: Zach Brown <zach.brown@oracle.com>
|
| b4e1da3c9a0ac9b01f45a8578b7347e3a31f9fb8 |
|
20-Jul-2010 |
Chris Mason <chris.mason@oracle.com> |
RDS: properly use sg_init_table
This is only needed to keep debugging code from bugging.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
|
| ef87b7ea39a91906218a262686bcb8bad8b6b46e |
|
09-Jul-2010 |
Zach Brown <zach.brown@oracle.com> |
RDS: remove __init and __exit annotation
The trivial amount of memory saved isn't worth the cost of dealing with section
mismatches.
Signed-off-by: Zach Brown <zach.brown@oracle.com>
|
| c20f5b9633bb0953bd2422f0f1430a2028cdbd0a |
|
08-Jul-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB: Use SLAB_HWCACHE_ALIGN flag for kmem_cache_create()
We are *definitely* counting cycles as closely as DaveM, so
ensure hwcache alignment for our recv ring control structs.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| d455ab64096b9a86849c7315c53e595330842db6 |
|
07-Jul-2010 |
Zach Brown <zach.brown@oracle.com> |
RDS/IB: always process recv completions
The recv refill path was leaking fragments because the recv event handler had
marked a ring element as free without freeing its frag. This was happening
because it wasn't processing receives when the conn wasn't marked up or
connecting, as can be the case if it races with rmmod.
Two observations support always processing receives in the callback.
First, buildup should only post receives, thus triggering recv event handler
calls, once it has built up all the state to handle them. Teardown should
destroy the CQ and drain the ring before tearing down the state needed to
process recvs. Both appear to be true today.
Second, this test was fundamentally racy. There is nothing to stop rmmod and
connection destruction from swooping in the moment after the conn state was
sampled but before real receive procesing starts.
Signed-off-by: Zach Brown <zach.brown@oracle.com>
|
| b6fb0df12db6c8b6bbcc7b5c9459b3bbf5f0cee6 |
|
24-Jun-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB: Make ib_recv_refill return void
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| 037f18a3074753991656189a091a5fa371999107 |
|
27-May-2010 |
Chris Mason <chris.mason@oracle.com> |
RDS: use friendly gfp masks for prefill
When prefilling the rds frags, we end up doing a lot of allocations.
We're not in atomic context here, and so there's no reason to dip into
atomic reserves. This changes the prefills to use masks that allow
waiting.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
|
| 33244125871734ebc0d8d147680a0d7e99385e0b |
|
27-May-2010 |
Chris Mason <chris.mason@oracle.com> |
RDS/IB: Add caching of frags and incs
This patch is based heavily on an initial patch by Chris Mason.
Instead of freeing slab memory and pages, it keeps them, and
funnels them back to be reused.
The lock minimization strategy uses xchg and cmpxchg atomic ops
for manipulation of pointers to list heads. We anchor the lists with a
pointer to a list_head struct instead of a static list_head struct.
We just have to carefully use the existing primitives with
the difference between a pointer and a static head struct.
For example, 'list_empty()' means that our anchor pointer points to a list with
a single item instead of meaning that our static head element doesn't point to
any list items.
Original patch by Chris, with significant mods and fixes by Andy and Zach.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: Zach Brown <zach.brown@oracle.com>
|
| fc24f78085e8771670af42f2b8929b16a0c98a22 |
|
25-May-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB: Remove ib_recv_unmap_page()
All it does is call unmap_sg(), so just call that directly.
The comment above unmap_page also may be incorrect, so we
shouldn't hold on to it, either.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| 3427e854e1a0e76be8b3d75fc0fa878f59b43693 |
|
25-May-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS: Assume recv->r_frag is always NULL in refill_one()
refill_one() should never be called on a recv struct that
doesn't need a new r_frag allocated. Add a WARN and remove
conditional around r_frag alloc code.
Also, add a comment to explain why r_ibinc may or may not
need refilling.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| 0b088e003ccf316a76c51be5dec2d70b93be3be8 |
|
25-May-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS: Use page_remainder_alloc() for recv bufs
Instead of splitting up a page into RDS_FRAG_SIZE chunks
ourselves, ask rds_page_remainder_alloc() to do it. While it
is possible PAGE_SIZE > FRAG_SIZE, on x86en it isn't, so having
duplicate "carve up a page into buffers" code seems excessive.
The other modification this spawns is the use of a single
struct scatterlist in rds_page_frag instead of a bare page ptr.
This causes verbosity to increase in some places, and decrease
in others.
Finally, I decided to unify the lifetimes and alloc/free of
rds_page_frag and its page. This is a nice simplification in itself,
but will be extra-nice once we come to adding cmason's recycling
patch.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| 7e3f2952eeb1a0fe2aa9882fd1705a88f9d89b35 |
|
12-May-2010 |
Chris Mason <chris.mason@oracle.com> |
rds: don't let RDS shutdown a connection while senders are present
This is the first in a long line of patches that tries to fix races
between RDS connection shutdown and RDS traffic.
Here we are maintaining a count of active senders to make sure
the connection doesn't go away while they are using it.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
|
| f17a1a55fb672d7f64be7f2e940ef5669e5efa0a |
|
19-Mar-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS: Refill recv ring directly from tasklet
Performance is better if we use allocations that don't block
to refill the receive ring. Since the whole reason we were
kicking out to the worker thread was so we could do blocking
allocs, we no longer need to do this.
Remove gfp params from rds_ib_recv_refill(); we always use
GFP_NOWAIT.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| 919ced4ce7d6ac62dd5be62d8993fe22a527d53a |
|
14-Jan-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB: Remove ib_[header/data]_sge() functions
These functions were to cope with differently ordered
sg entries depending on RDS 3.0 or 3.1+. Now that
we've dropped 3.0 compatibility we no longer need them.
Also, modify usage sites for these to refer to sge[0] or [1]
directly. Reorder code to initialize header sgs first.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| f147dd9ecabf23fd63d2562ffe64252a0453ecde |
|
14-Jan-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB: Disallow connections less than RDS 3.1
RDS 3.0 connections (in OFED 1.3 and earlier) put the
header at the end. 3.1 connections put it at the head.
The code has significant added complexity in order to
handle both configurations. In OFED 1.6 we can
drop this and simplify the code by only supporting
"header-first" configuration.
This patch checks the protocol version, and if prior
to 3.1, does not complete the connection.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| 809fa148a29467954280fe8b7f97c92403f6293c |
|
12-Jan-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS: inc_purge() transport function unused - remove it
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| 8690bfa17aea4c42da1bcf90a7af93d161eca624 |
|
12-Jan-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS: cleanup: remove "== NULL"s and "!= NULL"s in ptr comparisons
Favor "if (foo)" style over "if (foo != NULL)".
Signed-off-by: Andy Grover <andy.grover@oracle.com>
|
| 5a0e3ad6af8660be21ca98a971cd00f331318c05 |
|
24-Mar-2010 |
Tejun Heo <tj@kernel.org> |
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
|
| 735f61e62611161588123930823af6e6a9fd5c2c |
|
11-Mar-2010 |
Andy Grover <andy.grover@oracle.com> |
RDS: Do not BUG() on error returned from ib_post_send
BUGging on a runtime error code should be avoided. This
patch also eliminates all other BUG()s that have no real
reason to exist.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| f64f9e719261a87818dd192a3a2352e5b20fbd0f |
|
30-Nov-2009 |
Joe Perches <joe@perches.com> |
net: Move && and || to end of previous line
Not including net/atm/
Compiled tested x86 allyesconfig only
Added a > 80 column line or two, which I ignored.
Existing checkpatch plaints willfully, cheerfully ignored.
Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| d521b63b27e3a397e0ef7ca86b6e813861083c83 |
|
30-Oct-2009 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB+IW: Move recv processing to a tasklet
Move receive processing from event handler to a tasklet.
This should help prevent hangcheck timer from going off
when RDS is under heavy load.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| 86357b19bcabd9355937f3fb84f90ba9fe76a5d3 |
|
30-Oct-2009 |
Andy Grover <andy.grover@oracle.com> |
RDS: Fix potential race around rds_i[bw]_allocation
"At rds_ib_recv_refill_one(), it first executes atomic_read(&rds_ib_allocation)
for if-condition checking,
and then executes atomic_inc(&rds_ib_allocation) if the condition was
not satisfied.
However, if any other code which updates rds_ib_allocation executes
between these two atomic operation executions,
it seems that it may result race condition. (especially when
rds_ib_allocation + 1 == rds_ib_sysctl_max_recv_allocation)"
This patch fixes this by using atomic_inc_unless to eliminate the
possibility of allocating more than rds_ib_sysctl_max_recv_allocation
and then decrementing the count if the allocation fails. It also
makes an identical change to the iwarp transport.
Reported-by: Shin Hong <hongshin@gmail.com>
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| 597ddd50e1c07ac55ac7742442690efcf16a37f5 |
|
17-Jul-2009 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB: Rename byte_len to data_len to enhance readability
Of course len is in bytes. Calling it data_len hopefully indicates
a little better what the variable is actually for.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| 02a6a2592e41d27644d647f3bce23598649961bc |
|
17-Jul-2009 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB: Handle connections using RDS 3.0 wire protocol
The big differences between RDS 3.0 and 3.1 are protocol-level
flow control, and with 3.1 the header is in front of the data. The header
always ends up in the header buffer, and the data goes in the data page.
In 3.0 our "header" is a trailer, and will end up either in the data
page, the header buffer, or split across the two. Since 3.1 is backwards-
compatible with 3.0, we need to continue to support these cases. This
patch does that -- if using RDS 3.0 wire protocol, it will copy the header
from wherever it ended up into the header buffer.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| 7b70d0336da777c00395cc7a503497c2cdabd1a8 |
|
09-Apr-2009 |
Steve Wise <swise@opengridcomputing.com> |
RDS/IW+IB: Allow max credit advertise window.
Fix hack that restricts the credit advertisement to 127.
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| 8cbd9606a6367c221a7bbcc47f3ab1a8c31b6437 |
|
01-Apr-2009 |
Andy Grover <andy.grover@oracle.com> |
RDS: Use spinlock to protect 64b value update on 32b archs
We have a 64bit value that needs to be set atomically.
This is easy and quick on all 64bit archs, and can also be done
on x86/32 with set_64bit() (uses cmpxchg8b). However other
32b archs don't have this.
I actually changed this to the current state in preparation for
mainline because the old way (using a spinlock on 32b) resulted in
unsightly #ifdefs in the code. But obviously, being correct takes
precedence.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| 1e23b3ee0e943060a2e677cd20ed666b6060e12d |
|
24-Feb-2009 |
Andy Grover <andy.grover@oracle.com> |
RDS/IB: Receive datagrams via IB
Header parsing, ring refill. It puts the incoming data into an
rds_incoming struct, which is passed up to rds-core.
Signed-off-by: Andy Grover <andy.grover@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|