History log of /drivers/gpu/drm/ttm/ttm_bo.c
Revision Date Author Comments
e82c95f9f0cb025da8e1af6df1c29080a78d0f84 12-Jun-2012 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fix buffer object metadata accounting regression v2

commit a393c730ab69617c3291a3b0b2a228c9be2fc28c upstream.

A regression was introduced in the 3.3 rc series, commit
"drm/ttm: simplify memory accounting for ttm user v2",
causing the metadata of buffer objects created using the ttm_bo_create()
function to be accounted twice.
That causes massive leaks with the vmwgfx driver running for example
SpecViewperf Catia-03 test 2, eventually killing the app.

Furthermore, the same commit introduces a regression where
metadata accounting is leaked if a buffer object is
initialized with an illegal size. This is also fixed with this commit.

v2: Fixed an error path and removed an unused variable.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
71a47b9b18398fb55ab48865c0886a5b0ded6253 01-Jun-2012 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fix spinlock imbalance

commit a8ff3ee211fccf708e1911bbc096625453ebf759 upstream.

This imbalance may cause hangs when TTM is trying to swap out a buffer
that is already on the delayed delete list.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jakob Bornecrantz <jakob@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
25d0479a5925562fbf999afb5a8daa3f501c729d 17-Mar-2012 Joe Perches <joe@perches.com> drm/ttm: Use pr_fmt and pr_<level>

Use the more current logging style.

Add pr_fmt and remove the TTM_PFX uses.
Coalesce formats and align arguments.

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
9f1feed2e16652a6e599ed4a73b4c501bb3d4568 25-Jan-2012 Ben Skeggs <bskeggs@redhat.com> drm/ttm: fix two regressions since move_notify changes

Both changes in dc97b3409a790d2a21aac6e5cdb99558b5944119 cause serious
regressions in the nouveau driver.

move_notify() was originally able to presume that bo->mem is the old node,
and new_mem is the new node. The above commit moves the call to
move_notify() to after move() has been done, which means that now, sometimes,
new_mem isn't the new node at all, bo->mem is, and new_mem points at a
stale, possibly-just-been-killed-by-move node.

This is clearly not a good situation. This patch reverts this change, and
replaces it with a cleanup in the move() failure path instead.

The second issue is that the call to move_notify() from cleanup_memtype_use()
causes the TTM ghost objects to get passed into the driver. This is clearly
bad as the driver knows nothing about these "fake" TTM BOs, and ends up
accessing uninitialised memory.

I worked around this in nouveau's move_notify() hook by ensuring the BO
destructor was nouveau's. I don't particularly like this solution, and
would rather TTM never pass the driver these objects. However, I don't
clearly understand the reason why we're calling move_notify() here anyway
and am happy to work around the problem in nouveau instead of breaking the
behaviour expected by other drivers.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Cc: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
dc97b3409a790d2a21aac6e5cdb99558b5944119 18-Nov-2011 Jerome Glisse <jglisse@redhat.com> drm/ttm: callback move_notify any time bo placement change v4

Previously we were calling back move_notify in error path when the
bo is returned to it's original position or when destroy the bo.
When destroying the bo set the new mem placement as NULL when calling
back in the driver.

Updating nouveau to deal with NULL placement properly.

v2: reserve the object before calling move_notify in bo destroy path
at that point ttm should be the only piece of code interacting
with the object so atomic_set is safe here.
v3: callback move notify only once the bo is in its new position
call move notify want swaping out the buffer
v4:- don't call move_notify when swapin out bo, assume driver should
do what is appropriate in swap notify
- move move_notify call back to ttm_bo_cleanup_memtype_use for
destroy path

Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
57de4ba959b290f0b8cf36ecd5e7f1b29d4b8a12 11-Nov-2011 Jerome Glisse <jglisse@redhat.com> drm/ttm: simplify memory accounting for ttm user v2

Provide helper function to compute the kernel memory size needed
for each buffer object. Move all the accounting inside ttm, simplifying
driver and avoiding code duplication accross them.

v2 fix accounting of ghost object, one would have thought that i
would have run into the issue since a longtime but it seems
ghost object are rare when you have plenty of vram ;)

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
649bf3ca77343e3be1e0af8e21356fa569b1abd9 02-Nov-2011 Jerome Glisse <jglisse@redhat.com> drm/ttm: merge ttm_backend and ttm_tt V5

ttm_backend will only exist with a ttm_tt, and ttm_tt
will only be of interest when bound to a backend. Merge them
to avoid code and data duplication.

V2 Rebase on top of memory accounting overhaul
V3 Rebase on top of more memory accounting changes
V4 Rebase on top of no memory account changes (where/when is my
delorean when i need it ?)
V5 make sure ttm is unbound before destroying, change commit
message on suggestion from Tormod Volden

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
3316497bcd73dcad971d79bed32571ed785a8c01 01-Nov-2011 Jerome Glisse <jglisse@redhat.com> drm/ttm: remove userspace backed ttm object support

This was never use in none of the driver, properly using userspace
page for bo would need more code (vma interaction mostly). Removing
this dead code in preparation of ttm_tt & backend merge.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
26cc40a83384178bfaefbcfb4786591498f3e190 21-Nov-2011 Thomas Hellstrom <thellstrom@vmware.com> ttm: Don't return the bo reserved on error path

An unlikely race could case a bo to be returned reserved on an error path.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
1717c0e23f411147490c7a3312b894f0ea9a5fb1 27-Oct-2011 Dave Airlie <airlied@redhat.com> Revert "drm/ttm: add a way to bo_wait for either the last read or last write"

This reverts commit dfadbbdb57b3f2bb33e14f129a43047c6f0caefa.

Further upstream discussion between Marek and Thomas decided this wasn't
fully baked and needed further work, so revert it before it hits mainline.

Signed-off-by: Dave Airlie <airlied@redhat.com>
4d798937889706629ceb2019a844cf81e598312b 04-Oct-2011 Thomas Hellstrom <thellstrom@vmware.com> ttm: export ttm_bo_create

Used by the vmwgfx driver.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
ff02b13f6867af72682d7a9bb9bd705f9af2bab0 13-Sep-2011 Ben Skeggs <bskeggs@redhat.com> drm/ttm: request zeroed system memory pages for new TT buffer objects

Fixes an information leak to userspace, we were handing out un-zeroed pages
for any newly created TTM_PL_TT buffer.

Reported-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Tested-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Cc: stable@kernel.org
Signed-off-by: Dave Airlie <airlied@redhat.com>
dfadbbdb57b3f2bb33e14f129a43047c6f0caefa 13-Aug-2011 Marek Olšák <maraeo@gmail.com> drm/ttm: add a way to bo_wait for either the last read or last write

Sometimes we want to know whether a buffer is busy and wait for it (bo_wait).
However, sometimes it would be more useful to be able to query whether
a buffer is busy and being either read or written, and wait until it's stopped
being either read or written. The point of this is to be able to avoid
unnecessary waiting, e.g. if a GPU has written something to a buffer and is now
reading that buffer, and a CPU wants to map that buffer for read, it needs to
only wait for the last write. If there were no write, there wouldn't be any
waiting needed.

This, or course, requires user space drivers to send read/write flags
with each relocation (like we have read/write domains in radeon, so we can
actually use those for something useful now).

Now how this patch works:

The read/write flags should passed to ttm_validate_buffer. TTM maintains
separate sync objects of the last read and write for each buffer, in addition
to the sync object of the last use of a buffer. ttm_bo_wait then operates
with one the sync objects.

Signed-off-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
8d3bb23609d4ae22803a15d232289fc09a7b61c4 22-Aug-2011 Ben Skeggs <bskeggs@redhat.com> drm/ttm: ensure ttm for new node is bound before calling move_notify()

This was true for new TTM_PL_SYSTEM and new TTM_PL_TT cases, but wasn't
the case on TTM_PL_SYSTEM<->TTM_PL_TT moves, which causes trouble on some
paths as nouveau's move_notify() hook requires that the dma addresses be
valid at this point.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
7c4c3960dff109bc5db4c35da481c212dadb5eb5 22-Aug-2011 Marcin Slusarz <marcin.slusarz@gmail.com> drm/ttm: fix ttm_bo_add_ttm(user) failure path

ttm_tt_destroy kfrees passed object, so we need to nullify
a reference to it.

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Cc: stable@kernel.org
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
60063497a95e716c9a689af3be2687d261f115b4 27-Jul-2011 Arun Sharma <asharma@fb.com> atomic: use <linux/atomic.h>

This allows us to move duplicated code in <asm/atomic.h>
(atomic_inc_not_zero() for now) to <linux/atomic.h>

Signed-off-by: Arun Sharma <asharma@fb.com>
Reviewed-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: David Miller <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Acked-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
5df23979bc628934febe02e80f9644ec67603ee8 04-Apr-2011 Jan Engelhardt <jengelh@medozas.de> drm: fix "persistant" typo

Signed-off-by: Jan Engelhardt <jengelh@medozas.de>
Signed-off-by: Dave Airlie <airlied@redhat.com>
82ef594efb16d1747c55dc449a8eeb38ea97ef80 02-Feb-2011 Ben Skeggs <bskeggs@redhat.com> drm/ttm: call driver move_notify() when doing system->tt bo moves

Nouveau doesn't have enough information at ttm_backend_func.bind() time
to implement things like tiled GART, or to keep a buffer at a constant
address in the GPU virtual address space no matter where in physical
memory it's placed.

To resolve this, nouveau will handle binding of all buffers to the GPU
itself from the move_notify() hook. This commit ensures it's called
for all buffer moves.

Talked to Dave about the impact on radeon, which uses move_notify, it
doesn't look like anything should break there.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Thomas Hellstrom <thomas@shipmail.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>
f094cfc6c382cec7b2c77dd7798576684153acbb 24-Dec-2010 Tejun Heo <tj@kernel.org> drm/ttm: use cancel_delayed_work_sync() in ttm_bo

Make ttm_bo::ttm_bo_device_release call cancel_delayed_work_sync()
instead of calling cancel_delayed_work() followed by
flush_scheduled_work().

This is to prepare for the deprecation and removal of
flush_scheduled_work().

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc:: Thomas Hellstrom <thellstrom@vmware.com>
Cc:: Dave Airlie <airlied@redhat.com>
eba67093f535322cb4f1c4b737319c0907a0c81d 11-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fix up io_mem_reserve / io_mem_free calling

This patch attempts to fix up shortcomings with the current calling
sequences.

1) There's a fastpath where no locking occurs and only io_mem_reserved is
called to obtain needed info for mapping. The fastpath is set per
memory type manager.
2) If the fastpath is disabled, io_mem_reserve and io_mem_free will be exactly
balanced and not called recursively for the same struct ttm_mem_reg.
3) Optionally the driver can choose to enable a per memory type manager LRU
eviction mechanism that, when io_mem_reserve returns -EAGAIN will attempt
to kill user-space mappings of memory in that manager to free up needed
resources

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
65705962025df490d13df59ec57c5329d1bd0a16 17-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm/vmwgfx: Have TTM manage the validation sequence.

Rather than having the driver supply the validation sequence, leave that
responsibility to TTM. This saves some confusion and a function argument.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
95762c2b34069bf4adb7929969f1f5f5fc8a38df 17-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Improved fencing of buffer object lists

Drastically reduce the number of spin lock / unlock operations by performing
unreserving and fencing under global locks.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <j.glisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
702adba22433c175e8429a47760f35ca16caf1cd 17-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm/radeon/nouveau: Kill the bo lock in favour of a bo device fence_lock

The bo lock used only to protect the bo sync object members, and since it
is a per bo lock, fencing a buffer list will see a lot of locks and unlocks.
Replace it with a per-device lock that protects the sync object members on
*all* bos. Reading and setting these members will always be very quick, so
the risc of heavy lock contention is microscopic. Note that waiting for
sync objects will always take place outside of this lock.

The bo device fence lock will eventually be replaced with a seqlock /
rcu mechanism so we can determine that a bo is idle under a
rcu / read seqlock.

However this change will allow us to batch fencing and unreserving of
buffers with a minimal amount of locking.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
96726fe50feae74812a2ccf5d5da23cb01c0a413 17-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Don't deadlock on recursive multi-bo reservations

Add an aid for the driver to detect deadlocks on multi-bo reservations
Update documentation.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
2357cbe5f4ca8a52329c2c2a26b68839870d5d43 16-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Use kref_sub instead of repeatedly calling kref_put

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
d6ea88865d3e5b0c62040531310c1f2c6a994f46 22-Nov-2010 Dave Airlie <airlied@redhat.com> drm/ttm: Add a bo list reserve fastpath (v2)

Makes it possible to reserve a list of buffer objects with a single
spin lock / unlock if there is no contention.
Should improve cpu usage on SMP kernels.

v2: Initialize private list members on reserve and don't call
ttm_bo_list_ref_sub() with zero put_count.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
95ccb0f3bdc4438bf470c25450e6a8a6090ca253 11-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fix up a theoretical deadlock

A process suspended waiting for a higher sequence or no sequence to unreserve,
a bo may be beaten to the reservation by a process with a lower sequence.
In that case the first process should give up trying to reserve and
return -EAGAIN. In order for that to happen, we must wake waiting processes
when we change sequence, so that they have a chance to detect the new
sequence.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
7dfbbdcffebc41441e64278961f57d2840a76259 09-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Be consistent on ttm_bo_init() failures

Call destroy() on _all_ ttm_bo_init() failures, and make sure that
behavior is documented in the function description.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
29e190e049168b01dc5fa26d577ef99cafd753ee 02-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Remove the CAP_SYS_ADMIN requirement for bo pinning

This breaks vmwgfx non-root EGL clients and is a remnant from the
TTM user-space interface. This test should be done in the driver.
Replace the remaining placement test with a BUG_ON, since triggering
it is a driver bug.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
aa123268c2623c62e33248dafc0572f091689e86 02-Nov-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Make sure a sync object doesn't disappear while we use it

The sync object may disappear as soon as we release the bo::lock, so
take a reference on it while we use it.
One option would be to call sync_object_flush() before releasing the bo::lock,
but that would put an atomic requirement on that function.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
06fba6d4168069d818593e2fcc2d4bd0f888e97b 29-Oct-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Add a barrier when unreserving

Since we're doing this outside of a spinlock to provide the necessary
barriers, add an explicit barrier.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
dbc4a5b83585e89a6bce650d32426f61c8d4bca5 29-Oct-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Remove mm init error printouts and checks

Replace with BUG_ON(). These error messages remained from the time
when TTM was initialized from user-space. Nowadays hitting one of those
is really a kernel bug.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
6e4c55db120b03d411e0eff7cd35f3edabbefe14 29-Oct-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Remove pointless list_empty check

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
3205bc242b5e3950c808049dbf199fca91f2c844 29-Oct-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Documentation update

Remove an obsolete comment about mm nodes.
Document the new bo range manager interface.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
e1efc9b6ac22c605fd326b3f6af9b393325d43b4 19-Oct-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Optimize delayed buffer destruction

This commit replaces the ttm_bo_cleanup_ref function with two new functions.
One for the case where the bo is not yet on the delayed destroy list, and
one for the case where the bo was on the delayed destroy list, at least at
the time of call. This makes it possible to optimize the two cases somewhat.

It also enables the possibility to directly destroy buffers on the
delayed delete list when they are about to be evicted or swapped out.
Currently they were only evicted / swapped and destruction was left for the
delayed buffer destruction thread.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
40d857bba2915a4e8d82f44744a186bfdd1a46ea 19-Oct-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Avoid using the ttm_mem_type_manager::put_locked function

Release the lru spinlock early.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
965d38074e6eae71757a8baf9a348139e1e6894d 09-Oct-2010 Jean Delvare <khali@linux-fr.org> drm/ttm: Simplify ttm_bo_wait_unreserved

Function ttm_bo_wait_unreserved can be slightly simplified.

Signed-off-by: Jean Delvare <khali@linux-fr.org>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
c9220b0f7cbd1d2272426aa81a72ae2f6582bb71 08-Oct-2010 Dave Airlie <airlied@redhat.com> drm/ttm: add unlocked variant of new manager put node.

We need the unlocked variant for the new codepath introduced to fix the
race condition in master recently.

Signed-off-by: Dave Airlie <airlied@redhat.com>
1df6a2ebd75067aefbdf07482bf8e3d0584e04ee 30-Sep-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fix two race conditions + fix busy codepaths

This fixes a race pointed out by Dave Airlie where we don't take a buffer
object about to be destroyed off the LRU lists properly. It also fixes a rare
case where a buffer object could be destroyed in the middle of an
accelerated eviction.

The patch also adds a utility function that can be used to prematurely
release GPU memory space usage of an object waiting to be destroyed.
For example during eviction or swapout.

The above mentioned commit didn't queue the buffer on the delayed destroy
list under some rare circumstances. It also didn't completely honor the
remove_all parameter.

Fixes:
https://bugzilla.redhat.com/show_bug.cgi?id=615505
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=591061

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
d961db75ce86a84f1f04e91ad1014653ed7d9f46 05-Aug-2010 Ben Skeggs <bskeggs@redhat.com> drm/ttm: restructure to allow driver to plug in alternate memory manager

Nouveau will need this on GeForce 8 and up to account for the GPU
reordering physical VRAM for some memory types.

Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellström <thellstrom@vmware.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
42311ff90dc8746bd81427b2ed6efda9af791b77 03-Aug-2010 Ben Skeggs <bskeggs@redhat.com> drm/ttm: introduce utility function to free an allocated memory node

Existing core code/drivers call drm_mm_put_block on ttm_mem_reg.mm_node
directly. Future patches will modify TTM behaviour in such a way that
ttm_mem_reg.mm_node doesn't necessarily belong to drm_mm.

Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellström <thellstrom@vmware.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
ba4420c224c2808f2661cf8428f43ceef7a73a4a 09-Mar-2010 Dave Airlie <airlied@redhat.com> drm: move ttm global code to core drm

I wrote this for the prime sharing work, but I also noticed other external
non-upstream drivers from a large company carrying a similiar patch, so I
may as well ship it in master.

Signed-off-by: Dave Airlie <airlied@redhat.com>
db3307a9f7b8078c654021e3b35354a2b09a8e67 02-Jul-2010 Daniel Vetter <daniel.vetter@ffwll.ch> drm: kill drm_mm_node->private

Only ever assigned, never used.

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
[glisse: I will re-add if needed for range-restricted allocations]
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Dave Airlie <airlied@redhat.com>
7c5ee5366f79f53de2a11e73953daee6d58df124 26-Apr-2010 Matthew Garrett <mjg@redhat.com> ttm: Provide an API for starting and stopping the delayed workqueue

We want to be able to prevent the delayed workqueue from changing state
while we're reclocking, so add an API to block and unblock it.

Signed-off-by: Matthew Garrett <mjg@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
8cfe92d683a0041ac8e016a0b0a487c99a78f6c1 28-Apr-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Remove the ttm_bo_block_reservation() function.

It's unused and buggy in its current form, since it can place a bo
in the reserved state without removing it from lru lists.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
0c321c79627189204d7d0bf65ab19f5ac419abed 07-Apr-2010 Jerome Glisse <jglisse@redhat.com> drm/ttm: remove io_ field from TTM V6

All TTM driver have been converted to new io_mem_reserve/free
interface which allow driver to choose and return proper io
base, offset to core TTM for ioremapping if necessary. This
patch remove what is now deadcode.

V2 adapt to match with change in first patch of the patchset
V3 update after io_mem_reserve/io_mem_free callback balancing
V4 adjust to minor cleanup
V5 remove the needs ioremap flag
V6 keep the ioremapping facility in TTM

[airlied- squashed driver removals in here also]

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
82c5da6bf8b55a931b042fb531083863d26c8020 09-Apr-2010 Jerome Glisse <jglisse@redhat.com> drm/ttm: ttm_fault callback to allow driver to handle bo placement V6

On fault the driver is given the opportunity to perform any operation
it sees fit in order to place the buffer into a CPU visible area of
memory. This patch doesn't break TTM users, nouveau, vmwgfx and radeon
should keep working properly. Future patch will take advantage of this
infrastructure and remove the old path from TTM once driver are
converted.

V2 return VM_FAULT_NOPAGE if callback return -EBUSY or -ERESTARTSYS
V3 balance io_mem_reserve and io_mem_free call, fault_reserve_notify
is responsible to perform any necessary task for mapping to succeed
V4 minor cleanup, atomic_t -> bool as member is protected by reserve
mecanism from concurent access
V5 the callback is now responsible for iomapping the bo and providing
a virtual address this simplify TTM and will allow to get rid of
TTM_MEMTYPE_FLAG_NEEDS_IOREMAP
V6 use the bus addr data to decide to ioremap or this isn't needed
but we don't necesarily need to ioremap in the callback but still
allow driver to use static mapping

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
9d87fa2138d06ff400551800d67d522625033e35 07-Apr-2010 Jerome Glisse <jglisse@redhat.com> drm/ttm: split no_wait argument in 2 GPU or reserve wait

There is case where we want to be able to wait only for the
GPU while not waiting for other buffer to be unreserved. This
patch split the no_wait argument all the way down in the whole
ttm path so that upper level can decide on what to wait on or
not.

[airlied: squashed these 4 for bisectability reasons.]
drm/radeon/kms: update to TTM no_wait splitted argument
drm/nouveau: update to TTM no_wait splitted argument
drm/vmwgfx: update to TTM no_wait splitted argument
[vmwgfx patch: Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>]

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
b642ed06f2fccf62534f5269358776e0cba28f3c 13-Mar-2010 Robert P. J. Day <rpjday@crashcourse.ca> drm: "kobject_init/kobject_add" -> "kobject_init_and_add".

Replace sequential calls to kobject_init() and kobject_add() with the
combo wrapper kobject_init_and_add(), which provides the same
semantics.

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Dave Airlie <airlied@redhat.com>
52cf25d0ab7f78eeecc59ac652ed5090f69b619e 19-Jan-2010 Emese Revfy <re.emese@gmail.com> Driver core: Constify struct sysfs_ops in struct kobj_type

Constify struct sysfs_ops.

This is part of the ops structure constification
effort started by Arjan van de Ven et al.

Benefits of this constification:

* prevents modification of data that is shared
(referenced) by many other structure instances
at runtime

* detects/prevents accidental (but not intentional)
modification attempts on archs that enforce
read-only kernel data at runtime

* potentially better optimized code as the compiler
can assume that the const data cannot be changed

* the compiler/linker move const data into .rodata
and therefore exclude them from false sharing

Signed-off-by: Emese Revfy <re.emese@gmail.com>
Acked-by: David Teigland <teigland@redhat.com>
Acked-by: Matt Domsch <Matt_Domsch@dell.com>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Acked-by: Hans J. Koch <hjk@linutronix.de>
Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>
Acked-by: Jens Axboe <jens.axboe@oracle.com>
Acked-by: Stephen Hemminger <shemminger@vyatta.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
e22238ea37a870f70e34668a4992bde0c92bba8d 12-Feb-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fix a bug occuring when validating a buffer object in a range.

If the buffer object was already in the requested memory type, but
outside of the requested range it was never moved into the requested range.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
0eaddb28d3460f13102cd82cf58af875bfb1bd53 16-Jan-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Allow system memory as a busy placement.

This is needed to fix a vmwgfx memory usage bug.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
1a961ce09fe39df9a1b796df98794fd32c76c413 20-Jan-2010 Luca Barbieri <luca@luca-barbieri.com> drm/ttm: Fix race condition in ttm_bo_delayed_delete (v3, final)

Resending this with Thomas Hellstrom's signoff for merging into 2.6.33

ttm_bo_delayed_delete has a race condition, because after we do:
kref_put(&nentry->list_kref, ttm_bo_release_list);

we are not holding the list lock and not holding any reference to
objects, and thus every bo in the list can be removed and freed at
this point.

However, we then use the next pointer we stored, which is not guaranteed
to be valid.

This was apparently the cause of some Nouveau oopses I experienced.

This patch rewrites the function so that it keeps the reference to nentry
until nentry itself is freed and we already got a reference to nentry->next.

v2 updated by me according to Thomas Hellstrom's feedback.
v3 proposed by Thomas Hellstrom. Commit comment updated by me.

Both updates fixed minor efficiency/style issues only and all three versions
should be correct.

Signed-off-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
354fb52cb6138de0e6cf84a0f6a7f3467586e390 13-Jan-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Make sure system buffer objects has offset == 0.

This is a convention that the vmwgfx driver has come to rely on.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
e99e1e7893ac80fe769477cb1ddd4b482cef8902 13-Jan-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Export symbols needed for vmwgfx suspend / resume operations.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
3f09ea4ecdcbcea05541f83e557d6ce2e56626d8 13-Jan-2010 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Add a swap_notify callback.

This is needed for a bugfix in the vmwgfx driver.
Drivers may have GPU bindings on buffers that core TTM is not aware of,
and TTM may view those buffers as ordinary system memory buffers.
Add a notifier to such drivers when TTM is about to move the buffer
contents out to swappable memory. The driver must then release any
private GPU bindings on those buffers.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
5012f5063f41ca01240e5983c3b2cceb9aafc7a4 10-Dec-2009 Jerome Glisse <jglisse@redhat.com> drm/ttm: Fix memory type manager debug information printing

System memory type doesn't have a drm_mm manager associated to
it. This patch avoid trying to call drm_mm_debug on unitialized
drm_mm when printing debug info on the system memory manager.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
eb6d2c39dbe46ea1c4f3da4eac5728c73f109ea1 10-Dec-2009 Jerome Glisse <jglisse@redhat.com> drm/ttm: Fix printk format & compute bo->mem.size at bo initialization

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
9c51ba1db37cab780f38b2210913959f22d7b830 02-Dec-2009 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fix potential ttm_mem_evict_first races.

1) The function was previously called with a potentially empty
LRU list which would have lead to an OOPS or servere corruption.
2) In rare cases, after reservation has succeeded, another process may
already have evicted it or even pinned it. We must revalidate the
buffer status after releasing the lru lock.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
aaa207369436d04bb85382ddbb688a5b9461fd21 02-Dec-2009 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Delayed delete fixes.

1) Remove from lru before reserving so we avoid competing with
evicting processes.
2) Avoid calling kref_put() on bo::list_kref while spinlocked.
3) Additional refcounting bug-checking.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
b663752627e7c6b4bc414684d71c6adce5719fce 14-Dec-2009 Dave Airlie <airlied@redhat.com> drm/ttm: fix two bugs in new placement routines.

a) the loops were going to <= not <, leading to illegal memory access
b) the busy placement checks were using the placement arrays not the
busy placement ones.

Acked-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
d1ede145cea25c5b6d2ebb19b167af14e374bb45 11-Dec-2009 Ben Skeggs <bskeggs@redhat.com> drm/ttm: export some functions useful to drivers using ttm

These are functions required by nouveau which will be merged later.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
09855acb1c2e3779f25317ec9a8ffe1b1784a4a8 10-Dec-2009 Jerome Glisse <jglisse@redhat.com> drm/ttm: Convert ttm_buffer_object_init to use ttm_placement

Convert ttm_buffer_object_init to use struct ttm_placement and
rename to ttm_bo_init for consistency with function naming. This
allow to give more complex placement at buffer creation. For
instance you ask to allocate bo into vram first but if there is
not enough vram you can give system as a second possible
placement. It also allow to create buffer in a specific range.

Also rename ttm_buffer_object_validate to ttm_bo_validate.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
fb53f8621a3fab88776ae2450a1f3afc7920231b 09-Dec-2009 Jerome Glisse <jglisse@redhat.com> drm/ttm: Print debug information on memory manager when eviction fails

This add helper function to print information on eviction placements
and memory manager status when eviction fails to allocate memory
space.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
7cb7d1d7b650c9764c8a1b00e2b43d932acde779 09-Dec-2009 Jerome Glisse <jglisse@redhat.com> drm/ttm: Initialize eviction placement in case the driver callback doesn't

This would allow to catch driver callback error of not properly
setting the eviction placement structure.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@linux.ie>
98ffc4158e12008102cb6ae242a7fc46f9243f0d 07-Dec-2009 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Have the TTM code return -ERESTARTSYS instead of -ERESTART.

Return -ERESTARTSYS instead of -ERESTART when interrupted by a signal.
The -ERESTARTSYS is converted to an -EINTR by the kernel signal layer
before returned to user-space.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
ca262a9998d46196750bb19a9dc4bd465b170ff7 08-Dec-2009 Jerome Glisse <jglisse@redhat.com> drm/ttm: Rework validation & memory space allocation (V3)

This change allow driver to pass sorted memory placement,
from most prefered placement to least prefered placement.
In order to avoid long function prototype a structure is
used to gather memory placement informations such as range
restriction (if you need a buffer to be in given range).
Range restriction is determined by fpfn & lpfn which are
the first page and last page number btw which allocation
can happen. If those fields are set to 0 ttm will assume
buffer can be put anywhere in the address space (thus it
avoids putting a burden on the driver to always properly
set those fields).

This patch also factor few functions like evicting first
entry of lru list or getting a memory space. This avoid
code duplication.

V2: Change API to use placement flags and array instead
of packing placement order into a quadword.
V3: Make sure we set the appropriate mem.placement flag
when validating or allocation memory space.

[Pending Thomas Hellstrom further review but okay
from preliminary review so far].

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
447aeb907e417e0e837b4a4026d5081c88b6e8ca 08-Dec-2009 Dave Airlie <airlied@redhat.com> drm/ttm: fix unreachable code.

None of the in-tree drivers use user objects yet so this wasn't hitting
us.

Stanse found unreachable code in ttm_bo_add_ttm:
http://decibel.fi.muni.cz/~xslaby/stanse/error.cgi?db=32&id=714#l238

Reported-by: Jiri Slaby <jirislaby@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
7f5f4db2d50ea1af8f160686d2e97bbfa5102b4f 20-Aug-2009 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fixes for "Make parts of a struct ttm_bo_device global"

ttm:
Remove a stray debug printout.
Remove a re-init of the lru spinlock at device init.

radeon:
Fix the size of the bo_global allocation.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
a987fcaa805fcb24ba885c2e29fd4fdb6816f08f 18-Aug-2009 Thomas Hellstrom <thellstrom@vmware.com> ttm: Make parts of a struct ttm_bo_device global.

Common resources, like memory accounting and swap lists should be
global and not per device. Introduce a struct ttm_bo_global to
accomodate this, and register it with sysfs. Add a small sysfs interface
to return the number of active buffer objects.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@linux.ie>
5fd9cbad3a4ae82c83c55b9c621d156c326724ef 17-Aug-2009 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Memory accounting rework.

Use inclusive zones to simplify accounting and its sysfs representation.
Use DMA32 accounting where applicable.

Add a sysfs interface to make the heuristically determined limits
readable and configurable.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@linux.ie>
c96e7c7a3a79931446ecf9494a8415e4d164ebd8 03-Aug-2009 Roel Kluin <roel.kluin@gmail.com> drm/ttm: Read buffer overflow

Check whether index is within bounds before grabbing the element.

Signed-off-by: Roel Kluin <roel.kluin@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
fee280d3fd9bc5247bef9f4ab35a4693bfffdcfd 03-Aug-2009 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: Fix a sync object leak.

If there are multiple simultaneous waiters for the same buffer object,
a temporary reference to its sync object may be leaked.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
e024e11070a0a0dc7163ce1ec2da354a638bdbed 24-Jun-2009 Dave Airlie <airlied@redhat.com> drm/radeon/kms: add initial colortiling support.

This adds new set/get tiling interfaces where the pitch
and macro/micro tiling enables can be set. Along with
a flag to decide if this object should have a surface when mapped.

The only thing we need to allocate with a mapped surface should be
the frontbuffer. Note rotate scanout shouldn't require one, and
back/depth shouldn't either, though mesa needs some fixes.

It fixes the TTM interfaces along Thomas's suggestions, and I've tested
the surface stealing code with two X servers and not seen any lockdep issues.

I've stopped tiling the fbcon frontbuffer, as I don't see there being
any advantage other than testing, I've left the testing commands in there,
just flip the fb_tiled to true in radeon_fb.c

Open: Can we integrate endian swapping in with this?

Future features:
texture tiling - need to relocate texture registers TXOFFSET* with tiling info.

This also merges Michel's cleanup surfaces regs at init time patch
even though it makes sense on its own, this patch really relies on it.

Some PowerMac firmwares set up a tiling surface at the beginning of VRAM
which messes us up otherwise.
that patch is:
Signed-off-by: Michel Dänzer <daenzer@vmware.com>

Signed-off-by: Dave Airlie <airlied@redhat.com>
ad49f501867cba87e1e45e5ebae0b12435d68bf1 10-Jul-2009 Dave Airlie <airlied@linux.ie> drm/ttm/radeon: add dma32 support.

This add support for using dma32 memory on gpus that really need it.

Currently IGPs are left without DMA32 but we might need to change
that unless we can fix rs690.

Signed-off-by: Dave Airlie <airlied@redhat.com>
ae3e8122cbf8f9301369f276f4179aa6ec1b5b9c 24-Jun-2009 Thomas Hellstrom <thellstrom@vmware.com> ttm: Fix caching mode selection.

A bug caused a new caching state to be selected on each buffer object
validation regardless of the current caching state.
Moreover, a caching state could be selected that wasn't supported by
the memory type.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
87ef92092fd092936535ba057ee19b97bb6a709a 17-Jun-2009 Thomas Hellstrom <thellstrom@vmware.com> drm/ttm: fix an error path to exit function correctly

Just a goto instead of a direct exit.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
78ecf091aa592a9e160ebbbfa5873c2bb2e2d0f8 17-Jun-2009 Thomas Hellstrom <thellstrom@vmware.com> ttm: Return -ERESTART when a signal interrupts bo eviction.

A bug caused the ttm code to just terminate the wait when a signal
was received while waiting for the GPU to release a buffer object that
was to be evicted.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
ba4e7d973dd09b66912ac4c0856add8b0703a997 10-Jun-2009 Thomas Hellstrom <thellstrom@vmware.com> drm: Add the TTM GPU memory manager subsystem.

TTM is a GPU memory manager subsystem designed for use with GPU
devices with various memory types (On-card VRAM, AGP,
PCI apertures etc.). It's essentially a helper library that assists
the DRM driver in creating and managing persistent buffer objects.

TTM manages placement of data and CPU map setup and teardown on
data movement. It can also optionally manage synchronization of
data on a per-buffer-object level.

TTM takes care to provide an always valid virtual user-space address
to a buffer object which makes user-space sub-allocation of
big buffer objects feasible.

TTM uses a fine-grained per buffer-object locking scheme, taking
care to release all relevant locks when waiting for the GPU.
Although this implies some locking overhead, it's probably a big
win for devices with multiple command submission mechanisms, since
the lock contention will be minimal.

TTM can be used with whatever user-space interface the driver
chooses, including GEM. It's used by the upcoming Radeon KMS DRM driver
and is also the GPU memory management core of various new experimental
DRM drivers.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>