| d19a46fedbf054bc58a47738ae5d54704653f24b |
|
03-Mar-2015 |
Jens Axboe <axboe@fb.com> |
smalloc: bump initial size to 8 pools
Commit 23bd40f944b7 shrank our pool size from 128 pools at 8MB max,
to one static 16MB pool. Christian reports:
"For our tests with about 250k files we found the smalloc pool being
depleted. Now for us values of 3-4 would be enough, but since it is a
compile time switch I'd like to make it safe for everybody and set 8."
Bump the pool alloc to 8 pools again, retaining the 16MB size. That's
still substantially less than before, but should be enough for most
cases. Allocate the pools at init time, to avoid the issue that the
original commit fixed.
Also fix a bug where we failed to iterate some pools when restarting
the allocation scan, resulting in less-than-optimal exhaustion of all
pools before smalloc() gave up and returned NULL.
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/smalloc.c
|
| eb433fd28ce3ce814fabfb945bf330faf3193f1d |
|
03-Mar-2015 |
Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com> |
fio: fix smalloc strdop allocation failure
smalloc_strdup didn't check for allocation success and thereby ran into
segfaults if the single pool went out of memory.
Now with this patch applied it is still failing, but in a more consistent
way than segfaulting.
You still get a bad allocation, but it looks like this now:
fio: smalloc OOM
fio: filesetup.c:1495: dup_files: Assertion `0' failed.
Aborted
In fact the upper layers expected smalloc_strdup to retrun NULL on failure.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/smalloc.c
|
| 016f32fc265166737684ef6dbdf42dffea69057b |
|
14-Jan-2015 |
Jens Axboe <axboe@fb.com> |
smalloc: limit to 1 pool, and bump size to 16MB
We can easily add new pools, but if we do this after jobs have been
forked off, then it wont really be shared storage as the new mmap
mappings will only apply to the process that instantiated them.
Since we moved the randommap mappings to private storage, we should
not need a lot of room for shared allocations. Hence I hope the 16MB
will suffice.
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/smalloc.c
|
| dc068d5d5159be1a061cbcfe95d9d8149e61e034 |
|
06-Nov-2014 |
Jens Axboe <axboe@fb.com> |
smalloc: add zeroing scalloc() variant
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/smalloc.c
|
| f5c3fce95c366a60f21a4f4f9ac0e697c051608c |
|
13-Oct-2014 |
Jens Axboe <axboe@fb.com> |
smalloc: use log_err() for errors
If this happens at runtime, stderr/stdout may have been closed
and used for server/client connections.
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/smalloc.c
|
| ef7035a9b2c7af1e745b7cd2448685527ef7eeb0 |
|
19-Jun-2014 |
Jens Axboe <axboe@fb.com> |
Add support for compiling for ESX
With contributions from Ryan Haynes <rhaynes@fusionio.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/smalloc.c
|
| 10aa136bddbaa7c845ab4eacb4a9a4a88d6657a3 |
|
02-Apr-2014 |
Jens Axboe <axboe@fb.com> |
Cleanup symbols that should be static
Run analysis on symbols not used outside of their current file,
turn them into statics.
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/smalloc.c
|
| d7df1d133b0c3daad4ae4c731e0dae7b0181fd62 |
|
21-Mar-2013 |
Jens Axboe <axboe@kernel.dk> |
Rework lockfile= file lock handling
Get rid of the hand rolled rw semaphores, just use pthread
rwlocks instead. Kill the batching too, it was broken by
default, so nobody could have been using it.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/smalloc.c
|
| a3ebe7e079847413458b9d80bd7a4acc924d108b |
|
02-Nov-2012 |
Jens Axboe <axboe@kernel.dk> |
smalloc: move to size_t for allocations
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/smalloc.c
|
| 7982aa7da64b68f38c7ec685a428334e3af4e340 |
|
02-Nov-2012 |
Jens Axboe <axboe@kernel.dk> |
smalloc: fix int truncation issue
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/smalloc.c
|
| 521da527743088a9bd2ab882f8b64799d49d5848 |
|
02-Aug-2012 |
Jens Axboe <axboe@kernel.dk> |
mutex: make 0/1 FIO_MUTEX_LOCKED and FIO_MUTEX_UNLOCKED
Makes the API look cleaner.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/smalloc.c
|
| e43606c2b13ad7fc1af2bbe4a61cf8480ee3a532 |
|
20-Feb-2012 |
Bruce Cran <bruce@cran.org.uk> |
Fix assumption that pointers fits in a 'long'
Windows uses LLP64 model so [u]intptr_t is more correct.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/smalloc.c
|
| 271067a6e278dafd62649257fe99ce536acfe7bb |
|
15-Aug-2011 |
Jiri Horky <jiri.horky@cesnet.cz> |
Fix bug in smalloc size calculation
Jiri reports:
I think I found a bug. If the device (file) used in fio has "wrong" size, and is used with "wrong" block size, the fio fails to allocate memory for randommap.
Following conditions must be true, considering just one thread for simplicity:
1) number of blocks on the device are too big to fill in the default's pool
2) number of blocks on the device must be "right", so the size of newly allocated pool by smalloc.c:add_pool is exactly as big as requested size (there are some roundings in place, so not every size triggers this error)
When this is true, function smalloc.c:__smalloc_pool fails to use the newly created (exactly big enough) pool, because of this line:
idx = find_next_zero(pool->bitmap[i], last_idx);
which always returns non-zero value (the minimum returned value is last_idx +1), and so in our case the idx is 1 even though, the block number 0 is free. As the direct consequence, the function doesn't find enough free space in the given pool and returns NULL pointer. This leads to another try to allocate enough memory in smalloc.c:smalloc_real function, and so on...
The behavior could be reprocuded using both stable 1.57 and git version of the tool. The values that triggers the error are:
init_random_map - real_file_size: 21999995584512, o.rw_min_bs: 4096, blocks: 5371092672, num_maps: 83923323, alloc size: 671386584
whereas this block device is fine:
init_random_map - real_file_size: 19999995985920, o.rw_min_bs: 4096, blocks: 4882811520, num_maps: 76293930, alloc size: 610351440
the configuration file used:
[global]
description=CESNET_test
[cesnet]
filename=/dev/sdd
rw=randread
size=10000g
ioengine=libaio
iodepth=4
bs=4k
runtime=10m
time_based
numjobs=1
group_reporting
Get rid of the +1, it looks like a remnant from when the pools
had a header associated with them.
Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/smalloc.c
|
| 8d5844e9df308cc06d86a83d6bf28a29db46b6a9 |
|
29-Jun-2011 |
Jens Axboe <jaxboe@fusionio.com> |
Make smalloc use anonymous memory mmaps
Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/smalloc.c
|
| 9b8365618309572d8fd2579c8ea3132db89f843f |
|
08-Jan-2011 |
Bruce Cran <bruce@cran.org.uk> |
Various fixes/updates
- Update HOWTO to note that directio and fallocate don't work with ZFS
on Solaris.
Refactor the Makefile's to add CPPFLAGS and LIBS.
- Add -fno-omit-frame-pointer from Linux Makefile to every platform
- Change undefined $(ALL_CFLAGS) to $(CFLAGS)
- Pass -std=gnu99, without which OS X fails to build.
- Add -D__EXTENSIONS__ on Solaris since some functions we need are
behind it.
- Pull in <limits.h> in fio.c to get PTHREAD_STACK_MIN.
- NetBSD doesn't define PTHREAD_STACK_MIN so set it to 4k in
os-netbsd.h
- If we have posix_fallocate don't error out if it fails during mutex
and malloc operations since it will fail on Solaris with a ZFS
filesystem. As I understand it these aren't performance-critical
operations so do they need to be considered critical?
- Remove fio_unused from os-* files since it's defined in fio.h and we
don't really need it.
- FreeBSD has an idprio command but not the API so don't claim it does.
- OS X doesn't have the timer_* API so emulate it using
setitimer/sigaction.
- NetBSD and Solaris don't support CLOCK_MONOTONIC in timer_create so
remove FIO_HAVE_CLOCK_MONOTONIC from their os-* files.
I've noticed that a change I made a while ago to use fmin/fmax could
cause issues on older OSes - I had a CD with NetBSD 5.0.2 and found
they had only been implemented in 5.1 so I'm not sure if I should
revert it.
Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/smalloc.c
|
| c0e75abff75f2fd11107c9da7c34f9265b6303e1 |
|
14-Oct-2010 |
Jens Axboe <jaxboe@fusionio.com> |
Fix compile warning on platforms with posix_fallocate
Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/smalloc.c
|
| 3a8600b4ae9027d02aca7eb1990e5cda4e9f423a |
|
25-Jun-2010 |
Greg Edwards <greg.edwards@hp.com> |
gracefully handle full /tmp file system
If /tmp happens to be full, fio gives you a very unhelpful error:
# ./fio rand-write-256k-q256.job
Bus error
Use posix_fallocate() to gracefully handle this condition.
Signed-off-by: Greg Edwards <greg.edwards@hp.com>
Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/smalloc.c
|
| 0b9d69ecb14045cf3b2622ae922756b9889b25e6 |
|
11-Sep-2009 |
Jens Axboe <jens.axboe@oracle.com> |
Silence the style/whitespace police
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 5e012980ff5f8396a3e3ebc432e1dd32cebedaa1 |
|
31-Jul-2009 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: enlarge pool size
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 122426daab0fbd2fce244ac018d737a8f302f974 |
|
07-Jul-2009 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: just round alloc_size to int aligned for post redzone pointer
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| cf98708da292cb9e77bb6aff4eda7652103f4f77 |
|
07-Jul-2009 |
Jens Axboe <jens.axboe@oracle.com> |
Make smalloc redzone pointer properly aligned
It would SIGBUS on archs that require proper alignment, and work
suboptimally on others. Real fix is pending, making sure that
we always return naturally aligned pointers. But this at least
makes it work on sparc64.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 2e3e31e36b3565dcc5e36ba3842e7b207799601f |
|
06-Jan-2009 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: get rid of MP_SAFE define and lock checks
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| b8a6582e89999f88c574b905b89743762d8080df |
|
04-Dec-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: pool->file is only used in add_pool(), so make it local
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 443bb114d963a99082eef916025268a5a107092b |
|
04-Dec-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: unlink pool file in add_pool()
Don't defer until cleanup time, since we may not clean up if fio
crashes or is killed in other ways.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| d1271dc19c335c39298e16b825f84fdd8650132e |
|
10-Sep-2008 |
Shaozhi Shawn Ye <yeshao@google.com> |
smalloc: Remove read/write lock race condition which will result in segmentation fault
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 6885768609eafda8648bf50b05115d784cb04fb9 |
|
31-Jul-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: default to 128 pools instead of 4
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 6548f47f241013c7f6e1bb4b8a341f7d7a78764f |
|
13-Jun-2008 |
Jens Axboe <jens.axboe@oracle.com> |
Revert "smalloc: convert to spinlocks"
This reverts commit 3f1959101ed6a935ff3e2aec0fe7f5828f23fbcb.
/external/fio/smalloc.c
|
| 3f1959101ed6a935ff3e2aec0fe7f5828f23fbcb |
|
12-Jun-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: convert to spinlocks
Makes the allocator 10-20% faster.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| b3268b928ec7596a1d294a46a2771c3da789e7d3 |
|
02-Jun-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: use optimized ffz()
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 697a606cc0af1c39ac18998c344a522199accb36 |
|
31-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
Add sparc and sparc64 support
Also get rid of the arch private ffz functions, the generic
one is just fine.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 89da54e865169abcf0a99c6e7ee12ec2c8ddd8b9 |
|
27-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: final cleanups
Passes all regressions and tests, must be Bug Free.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| dcb69098630845b53ebb8034014d44b409e16f9e |
|
27-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: fix map overflow
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 1601cc905c8b42d8b1735e0767ac753df8805ad9 |
|
27-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: fix off-by-one in ptr_valid()
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 076b17cea9c9d6c86b926a166ef58f02334c9bdc |
|
27-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: kill another debug printf()
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 808e9ea8dd3dc3d631a7982fa0c4b01547cb811a |
|
27-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: fix compile error
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 179446e0122d04ddbeb3d353ff68350490b0841c |
|
27-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: cleanup and remove debug printf()
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 1f37f63654d36bb4d07180a4b622beb79ba2c3e3 |
|
27-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: turn on the thread safe flag
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| ec996e9c8f3baacababec55b3be746e035099c14 |
|
27-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: change to a bitmap allocator
It's a lot cleaner and easier to understand than the previous one,
even if it does waste some more memory.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 55f6491de3d5f734a1cbe3e7deed87a382d6c863 |
|
26-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: add pre and post redzone checks to pointers
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 868eb3cc388f4f34a68a633a1cf8b036c4dcc47a |
|
23-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: debug
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| c5dda9e387101dba4a654f6304553509d73dc298 |
|
23-May-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: increase alloc size
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 136f6b79180f6bd3f3e9ec8a97b9636a1bb71e15 |
|
17-Apr-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: remove pool resize support
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 8e5732e558509fc0f4ccdeb1e4d01ad038aead06 |
|
16-Apr-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: cleanups and allow sfree(NULL)
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| c08e194db676bd9dcd0f43bacf2051c7c91a62df |
|
16-Apr-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: remember to account for sizeof block header
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| adf57099338307b3a4a129475a12aad31e35ade3 |
|
16-Apr-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: when adding a new pool, make it big enough to hold the failing alloc
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 2b386d2569c9078ca9790c4e6d318ec3835b8739 |
|
26-Mar-2008 |
Jens Axboe <jens.axboe@oracle.com> |
Add softrandommap and --alloc-size options
The softrandommap job option allows fio to continue if it cannot
allocate from the smalloc backing for the random map. Or the user
can specify --alloc-size with a size larger than the default of 1024k
to increase the size of the shared allocation pool.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 85f467031d14d98940f56941016faa481a628c0f |
|
26-Mar-2008 |
Jens Axboe <jens.axboe@oracle.com> |
smalloc: increase initial pool size from 64k to 1m
We allocate randommaps from that pool as well and they can grow
quite large.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 5ec10eaad3b09875b91e19a20bbdfa06f2117562 |
|
06-Mar-2008 |
Jens Axboe <jens.axboe@oracle.com> |
Style fixups
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 4d4e80f2b4260f2c8b37a8612ce655502a799f7a |
|
04-Mar-2008 |
Jens Axboe <jens.axboe@oracle.com> |
Revamp file locking
Get rid of the semaphore implementation, no need to carry both.
Add different locking modes (exclusive and readwrite) to enable
a wider range of testing. Also combine lockfile and lockfile_batch,
the latter is now a postfix option to the former.
So to enable readers-excluding-writers locking mode with a lock batch
count of 4, you would write:
lockfile=readwrite:4
instead.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 9c5b52907227f8f50dc472d11601869f75b8d663 |
|
03-Mar-2008 |
Jens Axboe <jens.axboe@oracle.com> |
Add rw mutex init helper
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| 65864cf7c951a79c02b1d1f358f0eb5d25a9aff5 |
|
03-Mar-2008 |
Jens Axboe <jens.axboe@oracle.com> |
Use read-writer locks in smalloc
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|
| d24c33a479fcd68debad128da057814495f65e20 |
|
01-Mar-2008 |
Jens Axboe <jens.axboe@oracle.com> |
Add a really simple allocator, backed with mmap'ed memory
We can use this simple allocator to get memory that can be safely
shared across processes or threads.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/smalloc.c
|