History log of /external/fio/filesetup.c
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
eda3a60699e1d96bb68875ef2169ca819eb8f4f9 20-May-2017 Elliott Hughes <enh@google.com> Update to fio-2.20.

Bug: N/A
Test: builds, seems to work
Change-Id: I455dcc898f00dc50ae5cb65d1b381982df5a51e0
/external/fio/filesetup.c
d55dd0413b5f2df5e637c9c6e752b4272e4a1e5f 15-Dec-2014 Jens Axboe <axboe@fb.com> file: unionize lfsr/randommap

We only use one of them, add file flags to distinguish between the
two.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
eea6bedef6357ff84b878f989f758f9a8e441839 15-Dec-2014 Jens Axboe <axboe@fb.com> file: move mmap related data to engines/mmap.c where it belongs

We should not keep this in the generic file structure, it belongs
in file engine data for the specific engine.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
08a99bed133fdd5dd6ef87c8638b45b135b10a8e 15-Dec-2014 Jens Axboe <axboe@fb.com> Fix read/write mix and different levels of randomness

Stephen reports:

When using fio configuration below:

[global]
ioengine=libaio
direct=1
runtime=600
bs=32k
iodepth=8
rw=randrw
rwmixread=80
percentage_random=100,0

[drive1]
filename=/dev/sda

I am expecting to see 80% reads, 20% writes where all reads are random
and all writes are sequential. I captured a bus trace of traffic to the
disk and the bus trace reflected as much with one issue. The write
commands are essentially random. Each write begins at a new random LBA.
If 2 or more writes occur in a row, the LBA's are sequential based on
the block size BUT I feel the heart of this feature would be to emulate
a large file write during random access. With that in mind would it be
possible for sequential reads or writes within mixed sequential/random
workload to remember the last LBA accessed? In this scenario the writes
would still only take up 20% of the workload but when a write did occur
it should be the next sequential step from the last write.

-----

This is due to the fact that fio tracks last start/end on a unified
basis, not per read/write/trim direction. Fix that.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
4d6922b77db7109369f0499506cf2cf7a62aba45 12-Nov-2014 Jens Axboe <axboe@fb.com> Fix some shadow warnings for some gcc variants

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
f678701783f1c200ce574215feeb7ff801c4ff72 06-Nov-2014 Jens Axboe <axboe@fb.com> Get rid of __ prefix for internal frand state

It made (some) sense when we had both OS and frand in anon unions,
but now lets just kill it.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
559073f664a872f32632892c560667097b29e963 06-Nov-2014 Jens Axboe <axboe@fb.com> Remove use of OS provided random functions

We added the internal random generator a long time ago, and kept
the OS variant around as an opt-in feature with using use_os_rand=1.
We defaulted to using the fio provided one, and I doubt that
anyone has used the option.

The time has come to kill it.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
a85066dcaf01cd3611b4c7c987357650cd7f08a6 30-Sep-2014 Jens Axboe <axboe@fb.com> filesetup: fix a bug where we overwrite the set size

For the null io engine, we don't set file sizes. And we should
not overwrite a potentially legitimate o->size, if total_size
is zero.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
9187a269bf4539fa4b5b97a9b7439ba408d6b872 19-Aug-2014 Castor Fu <castor@alumni.caltech.edu> Add unlink hook to ioengine API, gluster ioengine

fio would just call unlink even with engines that are not using the
operating systems file namespace... This provides a hook to allow
overriding that, with a default handler, and implements it for the
gluster ioengine.

There are others which it'd probably make sense I'm sure.

Huamin Chen looked over my changes to the gluster code earlier...
>I like this unlink idea, it would be great if you can also make unlink optional (if my coding reading is correct). This looks a great pull request candidate to fio. Please ping Axboe after you are done. He is not actively watching pull requests.

>Please also feel free to augment gluster code and pull me for review if necessary.

-castor

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
5a65b4e4ce51be2144abaf47635e20a0ebf684ec 25-Jul-2014 Jiri Horky <jiri.horky@gmail.com> Allow reset of offset_increment counter

Reset offset_increment only within a sub-group of jobs, eg when
numjobs=x is used to automatically group jobs together.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
34329ca61aaf3be246a4a9d69412a1f3cdccee42 09-Jul-2014 Andreas Gruenbacher <andreas.gruenbacher@gmail.com> Fix two error paths

In generic_open_file(), when an error is detected, return 1 to indicate
the failure.

In get_file_sizes(), when an error other than ENOENT is detected, fail
immediately and don't clear the error.
/external/fio/filesetup.c
ef7035a9b2c7af1e745b7cd2448685527ef7eeb0 19-Jun-2014 Jens Axboe <axboe@fb.com> Add support for compiling for ESX

With contributions from Ryan Haynes <rhaynes@fusionio.com>

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
4c8e9f3a32ad0a1bc8bbc85b0cfe43e73a120753 10-Jun-2014 Jens Axboe <axboe@fb.com> Reset file count when freeing files

This fixes and oops with iolog replay, and a manually specified
filename=

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
1be9f219c5f7d218a6c5cced3ee93c7174a032da 20-May-2014 Jens Axboe <axboe@fb.com> Add ->invalidate() IO engine ops

Allow IO engines to plug in their own invalidate cache handler.
This fixes an issue on rbd, where we attempt to invalidate the
cache, but fail because it's not a valid file descriptor.

Reported-by: xan.peng@gmail.com
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
9c0f3f3279985b724c2f1358bb05cc2c795f2265 28-Apr-2014 Bruce Cran <bcran@fusionio.com> Windows: always open files in binary mode

On Windows, open() defaults to text mode, causing control characters
to be interpreted. Avoid this by passing the _O_BINARY flag.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
77731b29c35286a43ca6109ebf13389169ec20e9 28-Apr-2014 Jens Axboe <axboe@fb.com> Add option for io_limit

'size' denotes both the size of the region for IO, as well as the
amount of IO to transfer. Sometimes it's useful to be able to set
them separately instead. With this patch, you can do:

size=100G
io_limit=10G

and have fio do 10G of IO between 0..100G of the address space.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
d1faa06dc74fcdcae02e70260c90121502ef01cf 16-Apr-2014 Jens Axboe <axboe@fb.com> Error out if the set blocksize is larger than the file size

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
3660ceae229f08b4086279be7c82e86926f0304b 15-Apr-2014 Jens Axboe <axboe@fb.com> strcpy: kill last of the suspect ones

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
9824f73be5a03e683a236eb1ebe18f4656ac06b8 14-Apr-2014 Jens Axboe <axboe@fb.com> filesetup: silence bogus warning on close(-1)

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
71b84caab6ef5ddbc7893e3268887b8ff0516f75 14-Apr-2014 Jens Axboe <axboe@fb.com> td_verror() needs errno style error

put_file_log() could pass in -1, we need the actual error instead.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
da27a4bf0f9753fcc9da7b63f1be482fc0349907 14-Apr-2014 Jens Axboe <axboe@fb.com> file: fix potential buffer overrun in get_fs_free_counts()

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
f3e1eb23a6c900921caa58c6cad66d3b60b0b6d7 11-Apr-2014 Jens Axboe <axboe@fb.com> Fix leak on error path in extend_file()

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
68ace9e00ca481d101f9937461402b83e8895ba2 11-Apr-2014 Jens Axboe <axboe@fb.com> Fix directory leak in recurse_dir() on error

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
dfe11fd1e8249a52c42f34c9e9e6b910ab6b05c5 10-Apr-2014 Jens Axboe <axboe@fb.com> Make cache invalidation a soft error

On some platforms, we don't have reliable means of flushing
eg block device caches. Just warn of this fact instead of
erroring out, since the user can't do anything to stop this.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
3b5fea35f499c68cc56d03178a275656f2b4dc1b 09-Apr-2014 Jens Axboe <axboe@fb.com> Revert "Fixup ->open_files if not given"

This reverts commit bcd27f7ae1ccebd2ac1778752bf8f13fa99600e9.

The commit causes us to default to 1 file for cases where
openfiles= isn't set, where before we would default to
nr_files.

Signed-off-by: Jens Axboe <axboe@fb.com>

Conflicts:
filesetup.c
/external/fio/filesetup.c
d1e78e6b0429bac92794fb6f1c24b5078c68da6d 06-Apr-2014 Jens Axboe <axboe@fb.com> Fix crash in setup_files() if no files are defined

Commit 002fe73409d1e moved the division by o->nr_files outside
of the file loop, so for error cases we could end up with
doing the division without having files defined. Crash ensues.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
ef799a64618cbd53adb50d615666cd7afe9f0a8f 02-Apr-2014 Jens Axboe <axboe@fb.com> File pre-read lseek error check

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
7b5cb700485518c2e7f2f49548319c89ba907195 02-Apr-2014 Jens Axboe <axboe@fb.com> Add alloc_new_file()

Hopefully avoid a bug like commit 4db205dc fixed by abstracting
out the allocate-and-initialize part of fio_file allocation.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
90426237af4f3c9c7628aebfd4421fbe43d68c2a 01-Apr-2014 Jens Axboe <axboe@fb.com> Properly protect already-allocated file list

We need proper locking around it for thread based setups.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
4db205dc7c0195cacc949cfbc461350ca6ec74b0 01-Apr-2014 Jens Axboe <axboe@fb.com> Properly initialize shadow_fd to -1

We forget that in one case, causing all sorts of issues when
we end up closing this bogus (STDIN) descriptor.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
190b8f0c9c1c1a99ed26fd953f7af9023f9533ee 20-Mar-2014 Castor Fu <castor@alumni.caltech.edu> Pass -Wstrict-prototypes -Wold-style-definition, whitespace

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
1417daeb92c44632f2e1e376d736a0f198d7af8d 20-Mar-2014 Jens Axboe <axboe@fb.com> Don't re-create/unlink write file for append workload

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
b5f4d8baefc6eb3e13235b7d9042b7ffecdb23dd 20-Mar-2014 Jens Axboe <axboe@fb.com> Unlink after file close

Sébastien reports:

I'm having an issue on windows : unlink=1 is not working (temp file are
still there) and it's working fine on unix

Looking at the code in close_and_free_files function :

[...]
if (td->o.unlink && f->filetype == FIO_TYPE_FILE) {
dprint(FD_FILE, "free unlink %s\n", f->file_name);
unlink(f->file_name);
}
[...]

Unlink() fails because the file is still open : On Solaris, truss shows
the following :

[...]
write(3, "\0\0\0\0\0 ;80\0\0\0\0\0".., 32768) = 32768
unlink("/data/fio/random_rw.0.0") = 0
close(3) = 0
[...]

So unlink is called first.

I would put this unlinking phase AFTER the remove_file_hash call.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
bedc9dc24223bb33be4120f4a57718bc54888ca5 17-Mar-2014 Jens Axboe <axboe@fb.com> Add support for file appends

Add option file_append (bool). If set, fio will append to a file
instead of operate within the size of it. This is similar to
setting offset= to the size of each file.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
8edd973d57a311d4c590d7385796bbdf111ed04c 01-Mar-2014 Jens Axboe <axboe@fb.com> Add runstate swap helpers

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
225ba9e3433cf27d8ff7b213d9f78b7ef2776c70 26-Feb-2014 Jens Axboe <axboe@fb.com> Branch and cache miss speedups

Just some low hanging fruit.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
5903e7b7907854014478b6febfc5645a203ff59e 26-Feb-2014 Jens Axboe <axboe@fb.com> blktrace: load improvements

- Set the state explicitly to SETTING_UP when loading the
traces. This could take a while, so tell the status thread
that we are setting up so it knows how to display us
correctly.

- Fix a bug with multiple devices where we didn't cache the
major/minor properly. This made blktrace spend tons of CPU
cycles looking up /sys.

- Handle multiple devices correctly when adding files.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
bcd27f7ae1ccebd2ac1778752bf8f13fa99600e9 25-Feb-2014 Jens Axboe <axboe@fb.com> Fixup ->open_files if not given

For engines/users that add files with add_file() after the
job has been initialized, ensure that we bump the allowed
number of open files if it wasn't explicitly given as
an option.

This unbreaks the net engine after commit 002fe73409d1e made
fio more strict wrt open files.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
bcbfeefa7bce8383cf85fe59ced91f54821dfbd2 20-Feb-2014 Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com> fio: add multi directory support

This patch adds support for ':' seperated multiple directories at the
directory config statement in order to achieve an automatic distribution
of job clones (numjob) across directories.

That way people can distribute a load across these directories (usually
mount points of disks) automatically - changing numjob will be
sufficient to get all job clones evenly (optimal if dirs % numjobs = 0,
otherwise as good as possible) distributed at all times.

To avoid confused users old config Files will behave like they always
did, old fio binaries using new config files won't abort but just use
the first specified dir. If one specifies an explcit (non generated)
filename the distribution to many directories is also deactivated.

It also fixes an issue of events seeming out of order like when running
with --debug=file seeing the "..." message meaning "I created the
clones" prior to the last clone activities. Now the clones are called
with index N-1 .. 1, zero being the base thread as before.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
f2a2803a4982a1516eb75d9f98de68cf21d1ae7d 11-Feb-2014 Jens Axboe <axboe@fb.com> Enfore that lockfile= must precede filename=

The way fio parses options, we need to have the file locking
defined before we start adding files. Normally we can handle
this with option priorities, but that doesn't work across
job section boundaries. So catch this case and error out,
otherwise fio will segfault when it attempts to unlock files
at the end of a job.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
002fe73409d1e3d5e7dfe2885f75885bfaf506bc 11-Feb-2014 Jens Axboe <axboe@fb.com> Fix issue with openfiles= and file sizing

Fill the given size as well as we can, given the size and nr
of files.

Fix an issue where we don't properly honor how many files to
keep open at any point in time.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
67ad92428bb19008354ca20e614050241e9f17e7 07-Feb-2014 Jens Axboe <axboe@fb.com> Fix crash with file locking and dup'ed files

Ensure that we properly inherit the file locks when
duplicating files.

Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/filesetup.c
27ddbfa09b22911d5cefb42a44f894e6eb48216e 23-Jan-2014 Jens Axboe <axboe@kernel.dk> Fix potential oops in file unlocking at free time

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
3cd4c66f35fad0df64e4e6dfeddc5bdfd5206e0c 30-Dec-2013 John <j@bitminer.ca> Add OpenBSD support

I managed to make a clean compile and a few test runs using the
attached patches.

These work on OpenBSD 5.4 at least. I used the 2.1.4 release of
fio as a basis.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
d01612f3ae2515eb035d0c4ce954d8cb167a0a61 15-Nov-2013 Chris Mason <chris.mason@fusionio.com> Add support for O_ATOMIC

O_ATOMIC makes writes atomic, meaning that they are fully stable
on media (in the event of a power cut) when acknowledged by the
device and OS.

This only truly works on Linux with the pending patches to
add O_ATOMIC.

Updated by Jens to:

- Add man page and HOWTO description of the option
- Make O_ATOMIC imply O_DIRECT, so that it actually works if you
don't set O_DIRECT manually.
- Add the option to the conversion list so it works for
client/server.
- Error handling so that if atomic=1 is set and the OS does not
support it, error out instead of just pretending it works.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
de8f6de97438d5664cd8765e60102b9109a273e2 27-Sep-2013 Anatol Pomozov <anatol.pomozov@gmail.com> Fix misspellings using 'codespell' tool

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
57e54e087e051186d18cdedad20a2460463d4d30 20-Sep-2013 Juan Casse <jcasse@chromium.org> Fixes bug: stale LAST_POS(f) is not being reset

Problem:
When running fio with ioengine=sync, LAST_POS(f) state is not
reset after the file is closed. This causes workloads with
readwrite=randread and loops > 1 to fail verification if the
state of LAST_POS(f) at the beginning of the next loop is the
same as the io_u->offset. If that is the case, lseek in not
invoked, but the file position is at 0 and not at io_u->offset.

Proposed Solution:
Other ioengines, such as binject and fusion-aw, set f->engine_data
to 0 when closing the file. The sync ioengin uses f->engine_data to
store the LAST_POS(f) state. The proposed solution is to set
f->engine_data = 0 when closing a file in generic_close_file().

Signed-off-by: Juan Casse <jcasse@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
3e10fb832645e3ab3ef006f589f0459dc567cb53 09-Aug-2013 Jens Axboe <axboe@kernel.dk> verify: fix problem with hole punching on newer Linux kernels

Jeff Moyer writes:

Our QE team noticed fio failures on recent kernels. I simplified the
job file to the following:

[global]
name=fio-mmap
rw=write
bs=4K
direct=1
end_fsync=1
verify=crc32

[file3]
size=100M
ioengine=mmap
mem=malloc
direct=1

After fio completes (and returns verify errors), the file is completely
full of zeroes.

Here is what the verify logic is doing:

static void do_verify(struct thread_data *td, uint64_t verify_bytes)
{
...
/*
* sync io first and invalidate cache, to make sure we really
* read from disk.
*/
for_each_file(td, f, i) {
if (!fio_file_open(f))
continue;
if (fio_io_sync(td, f))
break;
if (file_invalidate_cache(td, f)) <--------
break;
}

That invalidate cache call looks like so:

static int __file_invalidate_cache(struct thread_data *td, struct fio_file *f,
unsigned long long off,
unsigned long long len)
{
...
/*
* FIXME: add blockdev flushing too
*/
if (f->mmap_ptr) {
ret = posix_madvise(f->mmap_ptr, f->mmap_sz, POSIX_MADV_DONTNEED);
(void) posix_madvise(f->mmap_ptr, f->mmap_sz, FIO_MADV_FREE); <-------

FIO_MADV_FREE can be defined as MADV_REMOVE, which will actually punch a
hole in the file (a hole the size of the entire file, btw). Of course,
unallocated blocks are returned as zeroes by the file system, so that
explains that!

---

For now, only do the MADV_FREE on non-files to avoid punching holes.
Further investigation is needed on the blockdev side, but at least
this should cure the immediate problem.

Reported-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
cc86c395fd9dd2002ec1edc0967b7c9453debdfb 03-May-2013 Jens Axboe <axboe@kernel.dk> Honor random/zero/compressible settings for filling an initial file

Right now we only do that for when the job writes, do it for the
initial setup as well. Otherwise you just get zero filled files
initially, which might not be what you want.

Also fixes a bug with compression if compress_chunk isn't set.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
4e0a8fa2593006505b7f4e18931a201d221b49e9 15-Apr-2013 Jens Axboe <axboe@kernel.dk> Fixup bad logging types

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
e90391eb56d81a2cc175c50caf6b4e5abbdb6142 13-Apr-2013 Jens Axboe <axboe@kernel.dk> filesetup: set TD_SETTING_UP runstate when laying out files

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
888677a428eb6ae6b54195e99734d6d6b3ee55e7 10-Apr-2013 Jens Axboe <axboe@kernel.dk> Merge in zipf differences from gfio

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
de98bd30b02bd89a78059d162b2c8426e889703d 05-Apr-2013 Jens Axboe <axboe@kernel.dk> Add filename_format option

Allows the user to set how fio should generate the file names
used by jobs. The default is $jobname.$jobnumber.$filenumber
as it always has been, but now it's configurable as well. See
the included documentation.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
d7df1d133b0c3daad4ae4c731e0dae7b0181fd62 21-Mar-2013 Jens Axboe <axboe@kernel.dk> Rework lockfile= file lock handling

Get rid of the hand rolled rw semaphores, just use pthread
rwlocks instead. Kill the batching too, it was broken by
default, so nobody could have been using it.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
d474cbc9ba33448848b50cc697622a402e91e33e 08-Mar-2013 Alex Pyrgiotis <apyrgio@grnet.gr> Improve LFSR implementation

Changes:

1. Use Galois LFSR instead of Fibonacci LFSR.
2. Use XNOR gates instead of XOR gates.
3. Add tap sizes for LFSRs ranging from 3-bits to 15-bits.
4. Add spin parameter.

Rationale:

1. Fibonacci LFSRs have the following drawbacks:

a. Their next state can not be computed in one cycle, since the input
bit must be propagated serially through the XOR gates. Galois
LFSRs however, can be computed instantly with XOR-masks.

b. Their state changes cannot be considered "irregular", even by I/O
standards. Actually, if the current state of an n-bit LFSR is x,
then the next will either be (x >> 1) or (2^n + (x >> 1)). Galois
LFSRs have instead their XOR gates interleaved with their bits,
which means that the inner bits are changed as well, besides of
the shifting. If the number of taps is z, this means that the
different outcomes are 2^(z + 1).

2. An LFSR with XOR gates has the all-zeroes state as illegal. Since
zero is valid for most I/O operations, it would be more intuitive to
use XNOR gates, that have as the all-ones state as illegal.

3. Allow smaller I/O benchmarks to use LFSRs.

4. The spin parameter follows the same rationale as in 1b. To make the
LFSR outcome "appear" less predictable, we can spin internally the
LFSR state and produce the i-th number. To understand the spin
parameter, consider the following state sequence of a 3-bit LFSR:

0, 2, 3, 5, 6, 1, 4
Same LFSR, spin value 2: 0, 3, 6, 4, 2, 5, 1
But what is the benefit from using spin? Well, the benefits are
two-fold:

a. For the same I/O size, we can create a different I/O sequences.

b. Following the rationale of 1b, we can have more variable outcomes.
If the spin value is "i" and the taps are "z", the number of
different outcomes become i * 2^(z + 1).

Signed-off-by: Alex Pyrgiotis <apyrgio@grnet.gr>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
4b91ee8fd12c72bd76ce9f5ff9116626b48566a0 25-Feb-2013 Jens Axboe <axboe@kernel.dk> Fixup wrong types for dprint()

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
53737ae01ca65f88089d3b8639df7be5b2f7ef0e 21-Feb-2013 Jens Axboe <axboe@kernel.dk> Fix problem with terminating on unaligned sizes

Three separate little issues:

- Don't round up the number of blocks needed. That means we end
up with potential partial blocks, which we can never do IO to.

- Fix an axmap bug where we only check against the specific bit
in the mask, but we really want to check for "this bit or any
higher bit". This makes axmap_next_free() behave more like it
should, instead of failing way too often.

- For the almost-full case, we can return a next available bit
that is just outside the allowed range. Punt to first-free for
that case.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
98ffb8f3ecebed9984d1744f142eb8be10c14dbd 30-Jan-2013 Ken Raeburn <raeburn@permabit.com> Fix bugs in [v]snprintf usage

When calling snprintf, supply the full buffer size instead of
one byte less.

When using the returned length from vsnprintf for logging, don't write
more than the actual buffer size (minus one for the trailing \0), in
case the formatted string was truncated.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
97ac992c99d89eed8cdf9249bb038ac4d7b709b9 24-Jan-2013 Jens Axboe <axboe@kernel.dk> Correct ifdef checks for posix and linux fallocate

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
33c48814e08cf961801bf37f759da2748eb3431b 21-Jan-2013 Jens Axboe <axboe@kernel.dk> Re-seed random generator when a file is reset

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
38f30c81d8d194da047950cbbda3896bd35cd98c 11-Jan-2013 Jens Axboe <axboe@kernel.dk> filesetup: limit rand map to the actual IO size, if needed

No need allocating a larger map than we need.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
d537c08b0eacdeb05835ee8c5f269b872239713a 11-Jan-2013 Jianpeng Ma <majianpeng@gmail.com> Add a check avoid segfault

If new_size was zero or realloc failed, it would be segment fault.
So add a check.

Signed-off-by: Jianpeng Ma <majianpeng@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
1983e3271c20a5dca84ac1ad872e44dd9e62e6b8 10-Jan-2013 Jianpeng Ma <majianpeng@gmail.com> Fix a potential integer overflow

Because nr_files and new_size are not same types.It can cause new_size
less than zero.Then realloc will be failed.

Signed-off-by: Jianpeng Ma <majianpeng@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
293b8c1f23bbe4c735cd00fbb7cca1121bf18016 04-Jan-2013 Jens Axboe <axboe@kernel.dk> Start of transition from unsigned long long to uint64_t

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
21415db9f1689a5c47fda5e710879c4739707442 04-Jan-2013 Jens Axboe <axboe@kernel.dk> zipf/pareto: use min of file size and io size for init

It is valid to use size=foo, where foo is larger than the size
of the file or device. For that case, we don't want offsets
larger than the size of the file. So use the minimum of the
real or given size.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
898cf6c7c1c5bceb57a15e339beb343bec5478e7 04-Jan-2013 Jens Axboe <axboe@kernel.dk> zipf/pareto: use size= if given, not always device size

We used the full device/file size for the zipf or pareto init,
this breaks if size= is specifically given. This causes fio
to generate offsets outside the specified size= range, and fio
will then just exit.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
e6c4d732fc99070091367d0ce41fe2cf1ddd1dc9 12-Dec-2012 Jens Axboe <axboe@kernel.dk> file hash: don't close file fd on lookup/add race

On Linux, udev often has a rule that triggers blkid to read data off
the device, if the device has been opened for writing. This triggers
when the device is closed.

Fio has an internal file hash for lookup of files. When threads start,
it can happen that one thread does a hash lookup without finding the
file, but when it has opened the file and attempts to insert it into
the hash, another thread has already completed that operation. When
that race happens, fio closes the file handle and does the lookup
again. That then triggers blkid to read pages off the device. As data
in the cache is invalidated on open of the device, we know have page
cache entries for the device again.

This is a problem for unbuffered workloads, where the existance of
page cache pages slows it down due to having to check for invalidation.
The user observed problem is that fio exhibits bi-modal performance
results, depending on whether the file hash race was hit during setup
or not.

Fix this by NOT closing the file if we hit this race, but instead wait
until the file is closed after the run.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
82af46be1fa0c0e188bbb6723008fa33a510606f 04-Dec-2012 Jens Axboe <axboe@kernel.dk> lfsr: ensure that the cycle follows the randrepeat= setting

Use the regular block offset seed to "seed" the lfsr generator,
so that it obeys randrepeat=0/1 as well.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
1cad7121e8e6c59440ae43545be05fa302e4110d 29-Nov-2012 Jens Axboe <axboe@kernel.dk> Fix broken 'norandommap'

commit 8055e41d broke the option, always falling through to
the failure case.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
7ebd796f4e50c21d652e62bf1e112755b0f338a8 28-Nov-2012 Jens Axboe <axboe@kernel.dk> Rename the bitmap to axmap

It's not really a bitmap, it's a bitmap of bitmaps. Now named.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
3831a843c289ab6a42a19752733d0fca4250882a 26-Nov-2012 Jens Axboe <axboe@kernel.dk> Ensure that lfsr_init() is always called

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
8055e41d0ecc54770a2653427532b3e2c5fabdad 26-Nov-2012 Jens Axboe <axboe@kernel.dk> Add LFSR generator

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
51ede0b1e9c9b570b942b50b44d0455183a0d5ec 22-Nov-2012 Jens Axboe <axboe@kernel.dk> Rework file random map

Fio slows down at the end of a random IO run, when the random
map is used and it gets fuller. This causes slowdowns in
IOPS. This is largely due to the file random map being an
array of bits, and with random access to single bits of the
array at the time, locality is awful. The effect is observable
throughout a run, though, where it gradually gets slower and
slower. It just becomes more apparent at near the end of the
run, where the last 10% are fairly bad. This is even with
doing a bunch of tricks to reduce that cost.

Implement an N-level bitmap, where layer N uses a single bit
to represent 32/64-bits at layer N-1. The number of layers
depends on the number of blocks.

This has slightly higher overhead initially in theory, in
practice it performs about the same. As a bonus, the throughput
remains constant during the run, and even becomes faster
near the end.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
8425687edef1b2961a17bd58341686f7b598cf28 17-Nov-2012 Jens Axboe <axboe@kernel.dk> Use unique seed for zipf/pareto init if rand_repeat is not set

By default, fio uses repeatable random sequences for anything.
This makes it easy to run the same job again. But if this is
disabled, ensure that the zipf/pareto sequences are also
randomized. Re-use the normal random io offset seed for
the random distribution too.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
2316296a514711bb388d87b34742c04bb561d986 07-Nov-2012 Jens Axboe <axboe@kernel.dk> zipf: seed zipf/pareto rand with filename hash and job id

We don't want 4 jobs operating on the same file to use the
same hot spots.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
9c6f63166eaecc13e4b2ca1d80cc1b5e6185fd43 07-Nov-2012 Jens Axboe <axboe@kernel.dk> Make the zipf/pareto state per file

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
4d002569dc503474a53e824e3f72b10a09c9a2b5 21-Sep-2012 Shaohua Li <shli@fusionio.com> Correctly apply size percentage for file iosize

The size percentage should be applied first when calculating file iosize,
otherwise it's a nop.

Signed-off-by: Shaohua Li <shli@fusionio.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
1ccc6dc75b28ef70cd7a8c39ac8c1cb68c720a65 19-Sep-2012 Dmitry Monakhov <dmonakhov@openvz.org> export file_lookup_open

generic_open_file is not always appropriate method, but other modules
may want to implement similar callback and use hash optimization.

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
f3afa57e36550288340f1b6c694f354ae72654b9 17-Sep-2012 Jens Axboe <axboe@kernel.dk> Add --output-format command line option

Right now we have normal, terse, and json output. Lets add a parameter
that controls how to output results, instead of having options for
both terse and json outputs. If we are going to add a 4th output
type in the future, it's only going to get more messy.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
6eaf09d6e9ca1f8accb057cdb18620b7e53ae33f 14-Sep-2012 Shaohua Li <shli@fusionio.com> Add support for trim as a workload type

This only works on Linux so far, and it's always sync given what
the interface to the kernel looks like. Also restricted to pure
block devices.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
ce95d651568bd5494e4c4ddc95832715c6760d56 16-Aug-2012 Dan Ehrenberg <dehrenberg@google.com> Fixing wraparound behavior for time-based sequential read jobs

Previously, a buggy patch made sequential jobs wrap around when they
reached an offset of 'size'. However, the appropriate wraparound is at
size + initial offset. This patch restores the previous behavior.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
521da527743088a9bd2ab882f8b64799d49d5848 02-Aug-2012 Jens Axboe <axboe@kernel.dk> mutex: make 0/1 FIO_MUTEX_LOCKED and FIO_MUTEX_UNLOCKED

Makes the API look cleaner.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
89ac1d48971578ccb0645c292d4a058340aeb909 11-Jun-2012 Shaohua Li <ShLi@fusionio.com> Fix "fio: file hash not empty on exit" with blktrace replay

There are two problems:
1. blktrace data hasn't file close action
2. redirect file fileno isn't correct

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
a93c5f049da63a60f9962177d8ada50094d234de 11-Jun-2012 Jens Axboe <axboe@kernel.dk> Add a more verbose/immediate warning if we fail open with O_DIRECT

If we get EINVAL with O_DIRECT open of the file, then that means
that the file system does not support direct IO. Warn as such
from fio.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
25460cf6350fb90b8a7943bb649a51d81bac13a4 02-May-2012 Jens Axboe <axboe@kernel.dk> Add create_only option

If this option is set, then fio will only run the preparation
phase of a job. This means it will create all the files it
needs, but exit after that.

Suggested-by: Georg Schönberger <gschoenberger@thomas-krenn.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
591e9e0653dd8f5d8464b2896434a01ab742a3b1 15-Mar-2012 Jens Axboe <axboe@kernel.dk> Fix offset_increment option and add man page entry for it

td->thread_number starts at 1, not 0. So we need to subtract
one before multiplying with the offset.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
214ac7e009897f8f82ab9e21aff9bc86d33bb470 15-Mar-2012 Dan Ehrenberg <dehrenberg@google.com> New offset_increment option

This patch adds a new option to fio job files. It is described
in the HOWTO as follows:

offset_increment=int If this is provided, then the real offset becomes
the offset + offset_increment * thread_number, where the
thread number is a counter that starts at 0 and is incremented
for each job. This option is useful if there are several jobs
which are intended to operate on a file in parallel in disjoint
segments, with even spacing between the starting points.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
3892182aff21486d9c38a346fe8681c9098ecb0c 22-Feb-2012 Bruce Cran <bruce@cran.org.uk> stat() doesn't work with devices on Windows, so move check for '\\.\' earlier.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
b9fd788f0e8adacc33316107594e9eb0463743d7 20-Feb-2012 Bruce Cran <bruce@cran.org.uk> Add OS_PATH_SEPARATOR for platforms which have different path separator characters (e.g. Windows and 'classic' MacOS).

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
ceadd59ef93421001530aa765d928e9e8c26b32e 02-Feb-2012 Jens Axboe <axboe@kernel.dk> Make random map allocation more resilient

Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/filesetup.c
7bb591020669b7266c57108f2a68b48a03ae72ee 12-Jul-2011 Jens Axboe <jaxboe@fusionio.com> Allow percentage setting for size=

Sometimes it's useful to set this to a particular size of a device.
Allowing percentages makes this more easy, as one does not have to
do the math outside of fio and pass in as environment variables
or custom job files.

To use, simply add a

size=15%

or whatever is needed, then fio will use 15% of the total size of
the device(s) or file(s) given. Percentage can be from 1 to 100.

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
5c909c91a2f58848b485523920e2278f8d63ecd4 11-Jul-2011 Jens Axboe <jaxboe@fusionio.com> Remove old debug printf()

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
a596f047e2b3d447ccca76bd075f05473a1f8d1c 17-Jun-2011 Eric Gouriou <egouriou@google.com> Add ability to invoke fallocate() FALLOC_FL_KEEP_SIZE.

Linux offers fallocate() and the FALLOC_FL_KEEP_SIZE option as
an alternative to posix_fallocate(). When FALLOC_FL_KEEP_SIZE is
specified for an falloc request going beyond the end of the file,
the requested blocks get preallocated without changing the apparent
size of the file. This is is a commonly recommended use of fallocate()
for workloads performing append writes.

This patch modifies the fallocate option from a boolean option
to a string option accepting none/posix/keep/0/1. 'keep' is only
made available on systems where FALLOC_FL_KEEP_SIZE is available
(i.e., Linux at this time). If specified, fallocate() is used
with FALLOC_FL_KEEP_SIZE set. 'none' disables pre-allocation while
'posix' uses posix_fallocate(). The default behavior remains unchaned,
i.e., invoking posix_fallocate. The settings '0'/'1' are there to
provide backward compatibility for users who had explicitly set the
boolean option.

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
3f0ca9b914e6d0db8c5c457713c277ce458cbc02 24-May-2011 Jens Axboe <jaxboe@fusionio.com> Silence gcc 4.6 warnings

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
1294c3ec7a02d20a98b105c1c41b23358afc67e4 11-May-2011 Jens Axboe <jaxboe@fusionio.com> Fix signed int/long truncation on 32-bit architectures

With using our internal random generator, we generate
unsigned ints. But by mistake this was assigned to
a signed long, that doesn't work so well on 32-bit machines.

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
4c07ad86c6cc6e158c70a5c625f60144db03f20d 28-Mar-2011 Jens Axboe <jaxboe@fusionio.com> Switch all random generators to be decided by use_os_rand

This includes file sizing, next file, block size selection,
trim state, and verify state.

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
0ce8b119b65849e537cab628a176a0ec4238aab0 27-Jan-2011 Jens Axboe <jaxboe@fusionio.com> Fix end-of-job slowdown for random IO with a random map

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
9b8365618309572d8fd2579c8ea3132db89f843f 08-Jan-2011 Bruce Cran <bruce@cran.org.uk> Various fixes/updates

- Update HOWTO to note that directio and fallocate don't work with ZFS
on Solaris.

Refactor the Makefile's to add CPPFLAGS and LIBS.
- Add -fno-omit-frame-pointer from Linux Makefile to every platform
- Change undefined $(ALL_CFLAGS) to $(CFLAGS)
- Pass -std=gnu99, without which OS X fails to build.
- Add -D__EXTENSIONS__ on Solaris since some functions we need are
behind it.

- Pull in <limits.h> in fio.c to get PTHREAD_STACK_MIN.
- NetBSD doesn't define PTHREAD_STACK_MIN so set it to 4k in
os-netbsd.h

- If we have posix_fallocate don't error out if it fails during mutex
and malloc operations since it will fail on Solaris with a ZFS
filesystem. As I understand it these aren't performance-critical
operations so do they need to be considered critical?

- Remove fio_unused from os-* files since it's defined in fio.h and we
don't really need it.

- FreeBSD has an idprio command but not the API so don't claim it does.

- OS X doesn't have the timer_* API so emulate it using
setitimer/sigaction.

- NetBSD and Solaris don't support CLOCK_MONOTONIC in timer_create so
remove FIO_HAVE_CLOCK_MONOTONIC from their os-* files.

I've noticed that a change I made a while ago to use fmin/fmax could
cause issues on older OSes - I had a CD with NetBSD 5.0.2 and found
they had only been implemented in 5.1 so I'm not sure if I should
revert it.

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
ecc314ba7c5f02b7e90ac1dfbce1a74cd4e6d6fe 04-Jan-2011 Bruce Cran <bruce@cran.org.uk> FIO Windows update

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
03e20d687566753b90383571e5e152c5142bdffd 02-Jan-2011 Bruce Cran <bruce@cran.org.uk> First snapshot of FIO for Windows

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
0f2152c19ed12fc2280c3d475973112f18e48cd5 15-Dec-2010 Bruce Cran <bruce@cran.org.uk> FreeBSD build broken due to missing -lrt

The latest fio fails to link on FreeBSD because timer_create is in
librt but the -lrt flag is missing from Makefile.FreeBSD.

I also noticed a couple of compiler warnings when building with clang -
a format of "%Lu" is used but 'L' doesn't have any effect with 'u' - I
guess %llu is needed since the types are 'unsigned long long'?

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
0e238572c4e63b0675fd9cbbf168d19ad8d60464 08-Oct-2010 Jens Axboe <jaxboe@fusionio.com> binject: setup and teardown mappings internally

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
49ffb4a2e1ac3026d77d9e1c03edc2753fcec41b 24-Aug-2010 Jens Axboe <jaxboe@fusionio.com> Fix blktrace replay

We would add duplicate files, if the open event comes before
the add file event. Make sure we return the current index file
instead of adding a duplicate, this causes bad file lookups.

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
38dad62d5154ffaad445bd0231b271b9a46a5190 20-Jul-2010 Jens Axboe <jaxboe@fusionio.com> Add rw_sequencer option

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
10accd7cf68af21ef4831528626a75ba7a06ce81 25-Jun-2010 Greg Edwards <greg.edwards@hp.com> fix posix_fallocate() return value usage

posix_fallocate() returns the error number on failure. Fix the return value
usage.

Signed-off-by: Greg Edwards <greg.edwards@hp.com>
Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
4ccdccd149d82c94ec6ccdb4118d9e27479b8516 24-Jun-2010 Jens Axboe <jaxboe@fusionio.com> Add chardev size getting

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
68b0316f581c3c881d7dbf7c9dafef4a10100a66 24-Jun-2010 Jens Axboe <jaxboe@fusionio.com> Make fill_device detect bdev sizes too

Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
d74ac843d82345862070400cb5fe408524c613f3 09-Jun-2010 YAMAMOTO Takashi <yamamoto@valinux.co.jp> include libgen.h for basename

Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
Signed-off-by: Jens Axboe <jaxboe@fusionio.com>
/external/fio/filesetup.c
2e3bd4c21cc239fbda992a4ede89ebb85f550920 17-May-2010 Jens Axboe <jens.axboe@oracle.com> Improve ETA for fill_device based job files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
61eb313e28c0f0ba8eb144c5b5f331b6b74c4fc8 31-Mar-2010 Jens Axboe <jens.axboe@oracle.com> sfree(NULL) is ok

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b30d395ec288508f9c787af11d0f52e9df26e39f 06-Feb-2010 Jens Axboe <jens.axboe@oracle.com> get_file_type() should use stat(), not lstat()

We care about the device, not the potential link we are passed.

Reported-by: "H. Tolley" <h_tolley@hotmail.com>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
7bc8c2cf02fbd538ca388618ff2f4618787a86e0 28-Jan-2010 Jens Axboe <jens.axboe@oracle.com> Add specific knob for controlling fallocate() usage

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
0b9d69ecb14045cf3b2622ae922756b9889b25e6 11-Sep-2009 Jens Axboe <jens.axboe@oracle.com> Silence the style/whitespace police

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
a1c58075279454a91ec43366846b93e8dcf9753c 04-Aug-2009 Jens Axboe <jens.axboe@oracle.com> Add strong madvise() hint for cache pruning

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b22989b9f9349b3c1d1c41846ab27ff0914bd6de 17-Jul-2009 Jens Axboe <jens.axboe@oracle.com> Get rid of KiB vs KB distinction

Confuses more than it does good, drop it and default to just using KB,
MB, etc.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
9c0d224129b0c59698e4c77e7fed00dc8cbb50e1 01-Jul-2009 Jens Axboe <jens.axboe@oracle.com> Don't allow pre_read on IO engines that cannot seek

We cannot pre-read files, if we cannot seek back and read the same
data again. So just disable it with a warning to the user.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
2be3eec330360e6735fb4e7d70907e11eaa346f8 10-Jun-2009 Jens Axboe <jens.axboe@oracle.com> File close fixes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
835d9b9ed0e5c2591b8a3eacbd5f4ab6bdab466c 09-Jun-2009 Jens Axboe <jens.axboe@oracle.com> Use __e for errno

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
22a57ba81de2c3f458797e9158da760c9e0ea435 09-Jun-2009 Jens Axboe <jens.axboe@oracle.com> Fix problem with too many opened files

If using raw block devices, we would leak the fd from the bdev_size()
function. Also add some debug triggers to catch this in the future.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
0d1cd207e409a36313784cb9596990c819775f77 05-Jun-2009 Jens Axboe <jens.axboe@oracle.com> Fix fill_device with read/randread

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
c592b9fe12d4739d99d5bece517e304804876df6 03-Jun-2009 Jens Axboe <jens.axboe@oracle.com> More fio.h cleanups

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
d6aed795f2e3e403828abf60874dd2d6e8342a1b 03-Jun-2009 Jens Axboe <jens.axboe@oracle.com> Clean up file flags

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
5e0074c2bfa46deddbad3b524c253a63dba05e63 02-Jun-2009 Jens Axboe <jens.axboe@oracle.com> Fix problem with fill_device and read workloads

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ac89311241264280574c81e6400c9a0c59615082 02-Jun-2009 Jens Axboe <jens.axboe@oracle.com> mmap engine: allow large files on 32-bit archs

Map chunks of 2GB at the time in total, remapping as we go
along.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
58726a1006aa33303e41951faac1ceca2bf09ec5 02-Jun-2009 Jens Axboe <jens.axboe@oracle.com> Fix problem with fadvise() and fill_device option

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
55c39e2764f4b2ab2f66ab495126b821c568c032 20-May-2009 Jens Axboe <jens.axboe@oracle.com> Fix old printf() in pre_read()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b0f65863844b4de92d10fefaabde80ea5bc3e5cc 20-May-2009 Jens Axboe <jens.axboe@oracle.com> pre_read fixes

1) Add a specific runstate for pre-read, so we can see that this
is what the job is currently doing.
2) open/close files for pre-read

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
afad68f778a764cbe57d4a5d54cbea32444aaa45 20-May-2009 Zhang, Yanmin <yanmin_zhang@linux.intel.com> Add pre_read option

With this option set, files will be pre-read into memory before
starting the given IO operation(s).

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
4b341fca0dab0caa41d000e8db36f1530c26616f 20-Apr-2009 Carl Henrik Lunde <chlunde@ping.uio.no> Fix crash with multiple files

There's an off-by-one in add_file() when it calculates the size
for the file array.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
99a47c6905731e53dd4d8f7ea6501f36d9329e7b 07-Apr-2009 Jens Axboe <jens.axboe@oracle.com> Use td_io_get_file_size() instead of calling ->get_file_size()

It may be NULL on some engines, causing a crash.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
df9c26b10275a631e83e7cc92d5f7384998b2c49 05-Mar-2009 Jens Axboe <jens.axboe@oracle.com> Avoid opening files until they are used

Fio still opens and creates all files, just to check the size of them. Add
a specialized IO engine op for getting the size of a file and use that
instead.

This also cleans a lot of things up. Note that the IO engine version is now
bumped to 10, meaning that external engines will have to separate the
file open from the size checking.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
8bb7679e73d3086a01b9d21a650b0d7a859412b6 04-Mar-2009 Jens Axboe <jens.axboe@oracle.com> Botch edit in files setup

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
fc99bc0d82371cc704395a7718e78abd0e0d2709 04-Mar-2009 Jens Axboe <jens.axboe@oracle.com> Realloc td->files to full size immediately

For adding many files, we do as many reallocs() as we have files. This is
slow, so do full allocs instead.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
814452bd47e0f2bb4825b8042b664f32de0aff8d 04-Mar-2009 Jens Axboe <jens.axboe@oracle.com> Add create_on_open option

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>

Please enter the commit message for your changes. Lines starting
/external/fio/filesetup.c
7ab077ab119fd28b6ccf5705ea8453e414f574c1 02-Feb-2009 Jens Axboe <jens.axboe@oracle.com> Check for a valid block device size

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b9fbcf2104ab79127825e0c23ba0a147ec076087 10-Sep-2008 Shaozhi Shawn Ye <yeshao@google.com> filehash: Filehash should be removed after the file is closed

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
a7ba8c5f4c0f9fe117d529c1ad6686b2070bae51 10-Sep-2008 Shaozhi Shawn Ye <yeshao@google.com> filesetup: When terse_output is set, The 'laying out' message should not be displayed.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
c48c0be79173897824d8f439c39374f2588931dc 31-Jul-2008 Jens Axboe <jens.axboe@oracle.com> Detect and complain on smalloc failures

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
bc3456fa06cc7777c4764da8c7b1c0fd0a3c3772 25-Jun-2008 Aaron Carroll <aaronc@gelato.unsw.edu.au> Prevent filetype disappearing

fio_file->filetype is set in add_file(), but for duplicated files
fio-file->file_name is already set so add_file() is not called.
This patch copies the filetype at the same time as file_name so
it remains valid for dup'd files.

Signed-off-by: Aaron Carroll <aaronc@gelato.unsw.edu.au>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
dc873b6f4a536c332b72cce268d5a7ccd356a891 04-Jun-2008 Jens Axboe <jens.axboe@oracle.com> Add OS agnostic RAND_MAX

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
fffca02dfd1b700903ba23b67fab7b59bf460ec2 02-Jun-2008 Jens Axboe <jens.axboe@oracle.com> Guard posix_fallocate() behind OS define

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
659424767f62f46c429eb66bf46b3e836e3bc0ad 02-Jun-2008 Jens Axboe <jens.axboe@oracle.com> Don't make posix_fallocate() set td->verror

It's supposed to be an error log, not stop the job.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
de605666880852ba9d1a0469723126b09782c520 31-May-2008 Jens Axboe <jens.axboe@oracle.com> ffz() takes ints, so change the file_map to being int based

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
739097e6e145afb98057846c419c60c2cc2dd9c0 30-May-2008 Jens Axboe <jens.axboe@oracle.com> Fixup posix_fallocate() usage

It returns the error directly, not through errno. Also make
it non-fatal, Solaris seems to complain about it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
5921e80c5dfc9f96d2f21da6ae58f2b5d3a0b373 30-May-2008 Jens Axboe <jens.axboe@oracle.com> Make it work on opensolaris

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
a5fb461fdff86c442daeebbd28e07984be9f3807 28-May-2008 Jens Axboe <jens.axboe@oracle.com> Fix bug in put_file() and __file_invalidate_cache()

- put_file() has bad logic in detecting fsync() error, only set
ret to f_ret if f_ret indicates failure.
- Check for open file in __file_invalidate_cache()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
47d85e2bd4763204a983c7bf4833c0d466dc36ca 04-Apr-2008 Jens Axboe <jens.axboe@oracle.com> Merge branch 'master' of ssh://git.kernel.dk/data/git/fio
3baddf2450d811be658fb47ef883ee84478dc352 04-Apr-2008 Jens Axboe <jens.axboe@oracle.com> If a file is extended, drop extended length in cache

Similar to how the cache state would be the next time the job is run,
good for consistency.

Reported and suggested by "Zhang, Yanmin" <yanmin_zhang@linux.intel.com>.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
2b386d2569c9078ca9790c4e6d318ec3835b8739 26-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Add softrandommap and --alloc-size options

The softrandommap job option allows fio to continue if it cannot
allocate from the smalloc backing for the random map. Or the user
can specify --alloc-size with a size larger than the default of 1024k
to increase the size of the shared allocation pool.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
303032ae4e13c7ac10b8198c40df98d95a4524a6 26-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Allow randommap allocation to fail

It's not that critical, so just print a warning and continue
without randommap.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
bd199f2b98eeb9101795e40fdef5889c630178c1 26-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Fix badly merged dprint()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ffdfabd48d0edfba9cb53cfd27a2f12ad887c211 26-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Make sure that file is created for FIO_NOEXTEND engines

This fixes a SIGBUS with the mmap engine. Reported by
"Zhang, Yanmin" <yanmin_zhang@linux.intel.com>

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
691c8fb014da9dd82e999a90b5511423f4eee188 07-Mar-2008 Jens Axboe <jens.axboe@oracle.com> blktrace improvements and fixups

Remove iolog version 1 support, it's just too limited. Version 2 and
blktrace manage file events (add,open,close,tc) on their own and we
can cleanup the file creation if we kill it.

Fixup for_each_file() for no allocated files.

Start of handling notify events for blktrace. We don't see program
notifications, need to look into that.

Properly account io bytes when adding blktrace events, so that the ETA
and io count is correct on replay.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
173081581f94dc4f9f203eb7a7922fc843bfafde 07-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Only set O_RDWR for open() when readonly isn't set

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
5ec10eaad3b09875b91e19a20bbdfa06f2117562 06-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Style fixups

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e8670ef8b26039573fbf835b4b95ba45a1039d83 06-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Always make job use open() instead of dup(), even for shared files

The fd may be opened after other jobs are forked, thus they wont
share fd space (unless the 'thread' option is used).

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
eda41004e84015059bb5ab8a688d3012e8a519a2 06-Mar-2008 Jens Axboe <jens.axboe@oracle.com> File setup open can fail in multiple functions, not just open()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
98e1ac4e1a2c7386a1064277f7e67c4736fb6e65 06-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Be more careful about catching fsync() errors

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
9efef3c4bd6a032a2a9f7a9aab1c29436525d781 06-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Multiple file handling fixes

Some of that got broken with the dynamically allocated file changes.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
4d4e80f2b4260f2c8b37a8612ce655502a799f7a 04-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Revamp file locking

Get rid of the semaphore implementation, no need to carry both.
Add different locking modes (exclusive and readwrite) to enable
a wider range of testing. Also combine lockfile and lockfile_batch,
the latter is now a postfix option to the former.

So to enable readers-excluding-writers locking mode with a lock batch
count of 4, you would write:

lockfile=readwrite:4

instead.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
29c1349f1840c3f60434c9da602074bc8fde4afe 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Add the file sharing bits

When you use the same filename for several jobs now, they will share
the same file structure. Enable locking through two new options:

- lockfile. If set, a semaphore is associated with the file and it is
held from ->prep() to ->queue() has done its work.
- lockfile_batch. This controls how many IOs the job gets to do per
semaphore acqusition.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
4906e0b57f2b8cbad02ef2428522281cf6038bdd 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Hook up the file hashing

Add on file open, remove on close.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
78d99e6a2695d3a7936c5dd02f996f13a30309b1 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Remember to free the file structure as well

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f17c43928c202e557f2f325272c60d9050ee528e 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Allocate file structure from shared process pool

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b0fe421a6a0487ff71f6f903908f666d9ca1b327 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Missed update of dup_files() for dynamically allocated files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b2bd2bd96a09540b3add0ec74db2cdb1c145ca33 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Add file locking hooks

Does nothing so far, but adds locking calls that cover from ->prep()
to after ->queue(). That is the period where we do IO.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
126d65c6fc97d6acdc568aa5a969c012018daf15 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Make file structures dynamically allocated

Current td->files is an array of files, make it an array of pointers
instead and allocate individual file structures on the fly. This is
a preparation patch for file sharing.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
6977bcd0e4ee3faa7ffd8f208e4031bdf906ed88 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Update close file handler to return potential error

Filesystems like NFS do return errors on close(), up until now we
have been ignoring them. Fix that. Adjust io_ops engine version
to 9, since this is an API change.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
24ffd2c2bca53f4de85ab6defd52a812be1f2429 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Add close_files() that closes all files

Renames the current implementation to close_and_free_files(), since
that is what it actually does.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
de8dd119d7d869dd9c90c6d62bf4bededd10642e 01-Mar-2008 Jens Axboe <jens.axboe@oracle.com> Only initialize random map if we are actually going to use it

We alloc the randommap even for sequential workloads right now,
if the 'norandommap' option isn't given. Fix that, check for
td_random() before continuing.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
982016d689dc7fedbb09646ba0df7e72a781b164 26-Feb-2008 Zhang, Yanmin <yanmin_zhang@linux.intel.com> extend_file need continue when the file exists

If some testing files exist and some others don't exist, fio will
report failure on the no-exist file. Here is a patch to fix it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
6ae1f57f2cb8661b97b770372eeb3694f6d5a744 21-Feb-2008 Jens Axboe <jens.axboe@oracle.com> Remove dead FIO_FILE_EXISTS flag

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
8172fe972f3a79d42f34d7f6e259ed3f7f800316 01-Feb-2008 Jens Axboe <jens.axboe@oracle.com> A few debug debug log fixes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ee56ad500f6692381e131cc37299d23fa910a24a 01-Feb-2008 Jens Axboe <jens.axboe@oracle.com> Add --debug for enabling internal dumps on various actions

A little weak currently, when it's fully integrated everywhere in
fio it'll help find fio problems that I can't trigger.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
273f8c912d981439049d9b21aa048aaeaa323c5d 25-Jan-2008 Jens Axboe <jens.axboe@oracle.com> Random IO fixes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
65bdb10a09222d8b5d213de74824be775772ea8f 24-Jan-2008 Jens Axboe <jens.axboe@oracle.com> More file extend fixes

Simplify the logic a bit, hope this works...

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
587af925762ff4bdf6a54965a35b2da2637f2f6e 24-Jan-2008 Gurudas Pai <gurudas.pai@oracle.com> file extent fix

I ran a test with latest fio using jobfile,

[global]
size=10m
[job1]
ioengine=sync
rw=write
filename=testfile

And second time,
[global]
size=30m --> size increased.
[job1]
ioengine=sync
rw=write
filename=testfile

fio did not extend the file to 30m.

In case of f->file_offset=0, f->io_size equals to f->real_file_size, hence no extend.
Following patch worked for me. But this patch will not handle the case where we have
to extend a file and with offset :(

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
fc74ac1d749c977308016eb22c8658a9a5ac7989 11-Jan-2008 Shawn Lewis <shawnlewis@google.com> Don't truncate files if we don't need to extend them

Don't truncate files if we don't need to extend them.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
aa31f1f108348ba2a8749c73fa8c25f3d4630a7c 11-Jan-2008 Shawn Lewis <shawnlewis@google.com> Add fill_device option

Add fill_device option. Causes fio to write until ENOSPC occurs (assuming
rw=write).

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
8f9cc4a9928d057945becd351c858c3433934465 21-Dec-2007 Gurudas Pai <gurudas.pai@oracle.com> fio: Fix for new file creation

fio fails to create new file and just comes out without doing anything..
This happens only when you are creating file freshly.

./fio ../jobfile/jobfile.test
job1: (g=0): rw=randrw, bs=8K-8K/8K-8K, ioengine=sync, iodepth=1
Starting 1 process

Run status group 0 (all jobs):

Disk stats (read/write):
sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%

jobfile:

[global]
bs=8k
iodepth_batch=1
randrepeat=1
size=100m
[job1]
ioengine=sync
rw=randrw
filename=newfile

Manually bisected it to commit 509eab12448823a8eefbe925804b5308ee63bf5e
Here is the patch which fixed the issue for me. Please review and apply.
In case of new file "f->io_size > f->real_file_size" becomes true, since
f->real_file_size is 0.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
509eab12448823a8eefbe925804b5308ee63bf5e 14-Dec-2007 Jens Axboe <jens.axboe@oracle.com> Overflow bugs in random map

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
056f3459149a7670257350d84b49858718e4a0e6 26-Nov-2007 Aaron Carroll <aaronc@gelato.unsw.edu.au> Fix failure in opening files with O_NOATIME

If we try to open a file with O_NOATIME and it fails, turn off
O_NOATIME and try again. This occurs if the user does not own
the file and is unprivileged.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
1f809d151ddc4a4c348c2e4f6007db510e3e08a2 25-Oct-2007 Jens Axboe <jens.axboe@oracle.com> Fixes for CPU burn engine

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ad92396cc21055cee93470dcf65e82ccde03aac5 11-Oct-2007 Jens Axboe <jens.axboe@oracle.com> Set O_NOATIME on open for non-regular files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
4241ea8fb0606f9bd5b7ed1278d3b7825883acf6 12-Sep-2007 Jens Axboe <jens.axboe@oracle.com> Extend --readonly

- Never allow open of a data file with writeable bits sets, and that
includes extend as well.

- Skip any writes in replay logs (iolog/iolog2/blktrace)

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ea966f8188915643670fd0a13d1bf37d45936da5 03-Sep-2007 Jens Axboe <jens.axboe@oracle.com> Fix ETA display for iolog replay

Valerie Henson reported (and supplied a patch) to fix the issue with
ETA display not being anywhere near correct when replaying an iolog.
Fix this by ensuring that td->total_io_size is always set correctly.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f627d8ae4dca9de8a72928a7928cf2903e81ad41 24-Aug-2007 Jens Axboe <jens.axboe@oracle.com> Adjust file layout printf

Make it file(s) not files, there may be 1 or many.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
d11a531f38b9b612dfb3bd09c89d011dc786f81d 27-Jul-2007 ljzhang,Yaxin Hu,Jianchao Tang <nonggia@sjtu.edu.cn> [PATCH] Fix file_size_high

It is generating a size between

[file_size_low, file_size_low+file_size_high]

where it should be

[file_size_low, file_size_high].

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
bcdedd0ac6e9413258b608ecb3511867b1a9c534 27-Jul-2007 ljzhang,Yaxin Hu,Jianchao Tang <nonggia@sjtu.edu.cn> [PATCH] Fix fileoffset option

1. The job file we prepared:
----------offset-----------------
[global]
filename=./temp/HOWTO
nrfiles=1
rw=read
size=8K
offset=16k
bs=1k
thread
loops=10
[offset]
description="Option 'offset' doesn't work."
---------------------------------
Besides, we copied the file HOWTO to the directory ./temp/.
So we thought fio would read 8K text sequentially from the point 16K off
the beginning of HOWTO.

we run fio with gdb to see what was read out after the first io, and we
got this:
---------------------------------
... ...
(gdb) bt
from /lib/tls/i686/cmov/libpthread.so.0
(gdb) x/4w io_u->xfer_buf
0x8073f08: 0x6c626154 0x666f2065 0x6e6f6320
0x746e6574
(gdb)
---------------------------------
The above was performed after the first io finished by td_io_queue.
The contents read out as we can see was the ASCII codes for 'Table of
content', which was right from the beginning of file HOWTO.
That means fio read from the beginning but not the point 16K off the
beginning.Is that right?

2. Reason for the problem:
It seems the offset is saved in td->o.start_offset. But it isn't passed
to f->file_offset.And the setting up of f->io_size doesn't refer to
f->file_offset.

--------------------------------------------------------

And now we can check it with gdb:
--------------------------------------------------------
... ...
(gdb) bt
from /lib/tls/i686/cmov/libpthread.so.0
(gdb) x/4w io_u->xfer_buf
0x8073f08: 0x206b636f 0x7420666f 0x66206568
0x20656c69
(gdb)
---------------------------------------------------------
After the first io, I got '0x206b636f 0x7420666f 0x66206568 0x20656c69'
read out.And they represent the string 'ock of the file '.Then i search
the string in HOWTO and got only one line containing the string:
'norandommap Normally fio will cover every block of the file when doing'
I removed the text following that line in HOWTO, and then I found that
the size of the rest text in HOWTO was 16K, which equaled to the offset
we set in job file.
So I think the option 'offset' does work after the changes.

In fact, I am not sure of the fix, but i do hope it can help.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
2298290eabbe1017421a6ba0f5de93d8c2b048ad 27-Jul-2007 ljzhang,Yaxin Hu,Jianchao Tang <nonggia@sjtu.edu.cn> [PATCH] Fix size given larger than filesize given

1. The mission is defined like this:
----------filesize----------------
[global]
directory=./temp
nrfiles=2
rw=randread
filesize=32k
size=96k
thread
[filesize]
description="Filesize less than size."
----------------------------------

This job file caused another non-stop thread, which was asked to read
96K but only provided two files of 32K each. The running information
kept looking like this:
---------------------------------
nonggia@nonggia-desktop:~/fio$ fio --version
fio 1.16.9
nonggia@nonggia-desktop:~/fio$ fio filesize
filesize: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
Starting 1 thread
Jobs: 1 (f=2): [r] [66.7% done] [ 0/ 0 kb/s] [eta 00m:51s]
---------------------------------

2. The reason:
After having covered both the two files, the thread still had 32K to
issued, while there were already no available files on hand.
I agree that it is far from a bug.Maybe it was the job file that we
should blame. It was wrong written from the beginning.But why not make
it more gentle if we can?

3. Fix suggestion:
So what about change it like this?
Just limit the max td->o.size with the total_size that all the files can
provide.
---------------------------------
diff -Nraup fio-7.25/filesetup.c fio-7.25-filesize/filesetup.c
--- fio-7.25/filesetup.c 2007-07-25 14:25:05.000000000 +0800
+++ fio-7.25-filesize/filesetup.c 2007-07-27 14:17:39.000000000 +0800
@@ -414,7 +414,7 @@ int setup_files(struct thread_data *td)
}
}

- if (!td->o.size)
+ if (!td->o.size || td->o.size > total_size)
td->o.size = total_size;

/*
---------------------------------

Now it behaves like this, just finishes as much job as it can and comes
to the end:
---------------------------------
nonggia@nonggia-desktop:~/fio$ ./filesize_fio filesize
filesize: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
Starting 1 thread

filesize: (groupid=0, jobs=1): err= 0: pid=8045
Description : ["Filesize less than size."]
read : io=64KiB, bw=32768KiB/s, iops=8000, runt= 2msec
clat (usec): min= 72, max= 907, avg=167.56, stdev=210.90
cpu : usr=0.00%, sys=0.00%, ctx=1
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
issued r/w: total=16/0, short=0/0
lat (usec): 100=62.50%, 250=25.00%, 500=6.25%, 1000=6.25%

Run status group 0 (all jobs):
READ: io=64KiB, aggrb=32768KiB/s, minb=32768KiB/s, maxb=32768KiB/s,
mint=2msec, maxt=2msec

Disk stats (read/write):
sda: ios=16/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
nonggia@nonggia-desktop:~/fio$
---------------------------------

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
7172cfe8e0e918cc279eb501986f6ec78cc7aad9 23-Jul-2007 Jens Axboe <jens.axboe@oracle.com> Only do the root warning once per thread

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f29b25a370598d387e539c3dcae126274c6cbf4d 23-Jul-2007 Jens Axboe <jens.axboe@oracle.com> Add version 2 of the iolog format

The old format had some short comings - notably not supporting > 1
file workloads. So add a new format that defines adding/open/close
of files. Fio will still load the older format iologs, but the new
ones wont work on older fio versions.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
97af62cec418cd722fb43c6010d6430534e12353 22-May-2007 Jens Axboe <jens.axboe@oracle.com> IO verification needs to open and grab a reference to a file

Otherwise it can disappear in midair.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
a9b7b3052b35c20e82b8fc909cd546a5fe1ff293 19-Apr-2007 Jens Axboe <jens.axboe@oracle.com> Fix file unlinking

We never set FIO_FILE_UNLINK. Just unlink if it's a regular
file and unlink=1 was given.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
660a1cb5fb9843ec09a04337714e78d63cd557e7 17-Apr-2007 Jens Axboe <jens.axboe@oracle.com> Reduntant write bit checks

No point in checking for td_rw(td) || td_write(), since td_write()
will also be true for td_rw().

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ce98fa663bb7c157b0db45791731d4795d6208a5 17-Apr-2007 Jens Axboe <jens.axboe@oracle.com> If writing to stdout, move stat output to stderr

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
d424d4dd657eaf4aeb8ffb07a550a5f64940f41a 17-Apr-2007 Jens Axboe <jens.axboe@oracle.com> No need to check for type before doing fsync()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
661598287ecc3b8987f312cf8403936552ce686a 16-Apr-2007 Jens Axboe <jens.axboe@oracle.com> Add support for using '-' as filename for stdin/stdout

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b5605e9ded4188ed91faae1a43ecd333a9856f62 16-Apr-2007 Jens Axboe <jens.axboe@oracle.com> Add check for pipe/fifo files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
07eb79dfa0e3801875d1c2907cfdb5da7c9d2cb3 12-Apr-2007 Jens Axboe <jens.axboe@oracle.com> Put the ->real_file_size handling into fio

Then we can remove it from the io engines, where the disk-less IO
engines provided a ->setup() hook just to set that.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
40b44f4aac3fa9c00ef1573ad73bfc62f760cc2c 11-Apr-2007 Jens Axboe <jens.axboe@oracle.com> Fix problem with treating ENOENT as an error

It's just a zero-sized file, as far as get_file_sizes() is concerned.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
317b95d07d4921d2594a1be6e014c9c2d062fe75 02-Apr-2007 Jens Axboe <jens.axboe@oracle.com> Move os/arch/compiler headers into directories

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
bab3fd58530027a67df288783f4a70759b3c84b7 02-Apr-2007 Jens Axboe <jens.axboe@oracle.com> Log and return error in early file open checks

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ea443657ce85860b7a0dd2f39f2572d4a09fc885 29-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Unlink a writeable file, if overwrite isn't set

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
1020a139beff50faf1bb18f761b024b664a09e14 29-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Fix loop with multiple files

The random lookup would loop infinitely, if no files were
open. So check for that and check for number of files
done.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e8be2ad711db813537a1f2a6a6939c42909ba5cc 29-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Only use posix_fallocate(), if we are overwriting

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
409b3417e46cc3ce7041f042bcc26c3feb46e9e9 28-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Fix size setting on raw devices

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
541d66d70b8fbe9860b31307c0760e6d5f4c78a8 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Use clear_error() instead of clearing the error manually

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
9bf27b4530fca07088ab978aea47be79a3d48879 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Auto limit open files if we get EMFILE

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
0ddb270c21adffa45cd9d215f29aafd8a9c5b15a 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> 'opendir' fixes

- Don't dive into a directory if it isn't a directory
- Dump info on how many files added

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
000b080395e45cbe0ce54e75f0e993addbdc8676 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> No need to return an error from get_file_sizes()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
507a702f6a61fa461c2566d01dcc46380ee7dffa 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Only overwrite in layout if needed

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
413d669320995eaef092c58e67cdb7b500134551 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Do the invalidate/advise hinting in td_open_file()

Then we only do it once per file, not twice.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
7bb48f84ac78cac1f90e3e04d0220d90d6a64a6b 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Revamp the file creation code

This was long overdue, it's much simpler now and hopefully
less bug prone. Probably a few stray bugs here and there
that needs straightening out, but the end result should
be a lot better.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
148cf6a95f738d6d9afd62bc63ba40b64bd647bd 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Cleanup generic_file_open()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
7e0e25c924d51f23196b3e0ea83b79178fc46823 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Don't completely fail for block device flushing failure

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
1d56dfee7857b91d1805d90396f36d62ad30e186 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Make sure to set io size on non-files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
160b966d83adace2629de10f85ed269ab2e587f5 27-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Improve random verify block sorting

Add an option to control it and also check the nosort flag in file
layout to further catch a case where sorting isn't needed.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
cade3ef44669c5c962d7ed18ca0e361116ee44ad 23-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Make sure the ->files array is job private

Otherwise we introduce differences between threads and processes,
we don't want to do that.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
fa1da8653e2177b3cc31a91e963c389f079d4172 23-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Fix file unlinking

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
bd0ee748e96338ace7b22b6f0ac1f8934d3385c0 23-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Fix bug with numjobs > 1, directory and filename given

Reported by gurudas pai <gurudas.pai@oracle.com>

If numjobs > 1, add_job() will append the directory name
several times. Fix this by doing the directory append in
add_file() instead, it cleans up the code as well.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
d2f3ac3592a5674fefa082cb439cc72355d2fed0 22-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Add option to disable fadvise() hints

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e4e33258637e55550a0fac59eb3350e8bf06f8e6 21-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Dump actual filename in failure to open

Helpful for debugging.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
3f344202f614ef3275e7000738d998ca85ebb9cf 20-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Fix divide-by-zero

Another file creation crap out, this really needs a rewrite and
soon.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
c343981b9874179009a92e4d29eae95cf341843a 20-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Clear ->file_map after free()

Just in case...

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ee3dcd9519760e0f8fb1488fdbb2b6d00af638df 20-Mar-2007 Jens Axboe <jens.axboe@oracle.com> File creation fix

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
687270767de17dbad53c50e03e5d73d52505405c 15-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Untangle the file creation mess

Still needs a lot of work, will do a rewrite of this soonish. At
least it should work now.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
2dc1bbeb58edc85f2829eed6729862c438ea2353 15-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Move thread options into a seperate structure

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
467d1b6b3ebbfcb7cc0545a882ba8a93ea66752b 15-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Style cleanup

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e8be395bcbc7428eea88d0db0d2144b8596d3511 15-Mar-2007 Jens Axboe <jens.axboe@oracle.com> More file creation improvements

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f4c4f4d74bf101a685f3f9839144c52c4ed29e30 15-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Remember to account for existing files in size setup

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
6d86144dd10b05e8b82e9b895c35dd778e5e71ab 15-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Use log_info() throughout

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
9c60ce649d3f976d196709d4399bb7c540ce97b5 15-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Add 'filesize' option

Allows the user to define the range of file sizes generated.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b6b37d7f60e29fbccc9a03dafac04645579f80b9 14-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Don't free ->files

It may be shared.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e407bec29fa8252f7809b2e9f88b95e619cd468b 14-Mar-2007 Jens Axboe <jens.axboe@oracle.com> New/old file mix fix

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e85b2b83e39b64ba5cb73fbfddbf3902b6e84925 14-Mar-2007 Jens Axboe <jens.axboe@oracle.com> recurse_dir(): Move . and .. check earlier

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
96d32d518a0743a6c050057bc1562e4883e51c5d 14-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Better handling of file creation vs existing files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
8ab538730857ce3a29c5f00fb55b967468751ec9 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Print informative error when we hit the max number of files open

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
bbf6b5401cdc0b20c405a064356ab0abb27c52d0 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Add 'opendir' option

This option adds all files from a directory and downward in the
filesystem hierarchy.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ebb1415f729c123b8a13bcbd667bf4b4cc95b4d4 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Add 'fsync_on_close' option

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
cbcd1062e2351b87321f91f1f266ba2c8d9c90a7 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Total file size handling fix

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
7b4e4fe5e6fa26f82f9169c18ec70c08d0805ca9 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Init stat for all files, not just current range

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
1549441cdfecb2c1a4a8b8d494253859d67b4344 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Normal vs special files fixups

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
1315a68a091816d284c5f46b28e419349aa821d8 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> One more f->file_name typo

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e3bab463d4397e3378d8c7437e31de8aaf861a4f 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> td->filename vs f->filename typo

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
cae61953cd7ee0dfc0d8b736553979f10ede7cd9 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Don't stack allocate file name

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
0ad920e7f85db1fdc26649be6bc7e584e8c7fdc9 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Add file reference counting

We must not close a file, while io is in progress to it. That
will make the queuing engines barf.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f11bd94d50d5995a64682f74b5f0f7509bf2c550 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Turn file ->open and ->unlink into flags

We'll need more flags in the next commits, so do this as a
preparatory patch.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
af52b3455ad892322aab2791282b6bd4efdfdbf3 13-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Allow explicit setting of a number of files

We currently only allow filename=foo for one file, add the possibility
to specify any number of files by seperating with a colon.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
cf3d609747c3e070001173b17ea95e9d6f46b15c 12-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Overwrite fix

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
0263882a95b9213d3afe285ae8adb5d0310d5039 12-Mar-2007 Jens Axboe <jens.axboe@oracle.com> File truncation and extend fixes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b3dc7f075b85e004f8c681ebb8566475017696f3 09-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Fix ->file_map leak

Just to appease valgrind.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
a978ba684deb758465a0ccb18a008797636e8054 08-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Get rid of reopen_files()

Move the full file state clear into td_io_open_file(), so a
reopen is a plain close/open of that file.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
0f14fef318f38a285a04097636a65a997596e821 08-Mar-2007 Jens Axboe <jens.axboe@oracle.com> openfiles fix

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
da86ccb633a8eac058c7f1a74d2d74a073e51b41 08-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Fix invalidate cache typo

The odirect check was reversed.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b5af82930ccfd7dda6a1b11794efb452eb76d8dc 08-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Revamp file open/close handling

Some IO engines need special handling for opening and closing
files, and this has complicated the fio filesetup.c file. Instead
have the io engine provide hooks for file open/close. This also
greatly cleans up the flags (we can get rid of SELFOPEN and MMAPIO)
and moves private knowledge into the engines where it belongs.

This potentially destabilizes fio somewhat, so testing is needed.
The new openfiles option that is introduced with this change isn't
verified working yet, hence it isn't documented.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
2fd233b7302603fb6fd9b7098994d2a155cf7a22 02-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Convert null io engine to use ->setup()

Then we can get rid of the NULLIO flags hack

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
bdb4e2e99d4a87e5d3677cc09aa1ce92135125a9 01-Mar-2007 Jens Axboe <jens.axboe@oracle.com> Better management of open files

This is a first step to properly supporting dynamic open/close of
new files, differently sized files, etc.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
a9defc9abb4a3013335d3aeeb132fd90900cac85 01-Mar-2007 Joel Becker <joel.becker@oracle.com> [PATCH] ioengine flags: Replace FIO_NETIO with real flags that map the
behavior

The FIO_NETIO ioengine flag created two behaviors. First, open_files()
would not open any files - this was handled in engines/net:setup().
Second, init_disk_util() would ignore the files and not try to locate
their disk.

Other engines might want one of these behaviors but not the other. If
an engine opens its own files, it doesn't want open_files() to do anything,
even if the files are eventually disk-based. An engine might want to
prevent init_disk_util() from running.

The FIO_NETIO flag is split into two flags.

FIO_SELFOPEN states that the engine will do its own open(), whether it
is a real open(2) or something internal to the engine.

FIO_DISKLESSIO states that init_disk_util() is should not be run. fio
should treat the engine's files as if they are not part of a disk.

Finally, engines/net.c is changed to use these two flags.

Signed-off-by: Joel Becker <joel.becker@oracle.com>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ad2da605a62faf16887970618b434db19594e17b 26-Feb-2007 Jens Axboe <jens.axboe@oracle.com> No need to do fadvise() on O_DIRECT/raw IO

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
413dd459a7710ba421061e840dd9ac3161c70f20 23-Feb-2007 Jens Axboe <jens.axboe@oracle.com> Streamline thread_data data direction setting and checking

Currently it's a mess of ->ddir, ->iomix and ->sequential. Add
a TD_DDIR_* for each of these, so we can store them as one value.

A prerequisite for the next parsing cleanup.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
9d80e114104d43103d6acc920bce69296b647a53 22-Feb-2007 Jens Axboe <jens.axboe@oracle.com> Correct spelling error

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e1161c325f7866bae879e686d1c673ca32ab09ae 22-Feb-2007 Jens Axboe <jens.axboe@oracle.com> Add more context to the error messages

Errors like:

fio: pid=0, err=22/file:filesetup.c:380, error=Invalid argument

do not give a lot of clue as to what is wrong, unless you
have a matching source. So add a context relevant info
message as well.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
fa01d139c558a3788154c5b0f094bfdb5325728f 22-Feb-2007 Jens Axboe <jens.axboe@oracle.com> Don't check st_size for special files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f4ee22ca72da53e0e891693d16358296b703a684 10-Feb-2007 Jens Axboe <jens.axboe@oracle.com> [PATCH] File size fix on NULL io engines

Just set the file sizes to the size given, it's pretend anyway.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
4d9345ae968ca92ace3dab06c25a9b73159cb329 07-Feb-2007 Jens Axboe <jens.axboe@oracle.com> [PATCH] Don't create files for engines that don't need them

Introduce FIO_NULLIO as a way to control that from the io engine.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ed92ac0ce9ce1cc64697272d307d4fa7d18ed64c 06-Feb-2007 Jens Axboe <jens.axboe@oracle.com> [PATCH] Simple support for networked IO

Adds a new ioengine, net. Use with ioengine=net, it supports only strict
reading or writing (no mixed reads/writes) to/from a single host.
The filename given must contain the host and port to connect to (or
listen from).

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
3404cefcb23db812b3f3a164ef773b37d53355ab 24-Jan-2007 Jens Axboe <jens.axboe@oracle.com> [PATCH] Fix typo

WRITE -> DDIR_WRITE

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
eff6861d99c1e20ef24d5ab9eef1ac0bbfa655d3 14-Jan-2007 Jens Axboe <jens.axboe@oracle.com> [PATCH] Unlink write file if !overwrite and it exists

Repeatability across runs, unlink the file so that the job
runs like it wasn't there to begin with.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
745508ddfcb7b89b65d99a9dabc6b957aafb9a57 08-Jan-2007 Jens Axboe <jens.axboe@oracle.com> [PATCH] Non-overwrite random write fix

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
ee88470c001012d08ae37bfc31741a935be2e7a1 20-Dec-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Must use ->real_file_size

->file_size is the size of the file from offset, while ->real_file_size
is the total size of the file. So when comparing EOF, we should look
at ->real_file_size against the offset.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
9b031dc82192e884c1d6da9c86f047ecb9ef5c3a 15-Dec-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Only unlink for created files

If the file was already there, don't unlink it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
1e97cce9f5a87a67293a05ec4533ed6968698b2e 05-Dec-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Fix warnings from icc

icc spewed a bunch of warnings on building fio, but it did actually build
and work. Some of them are real bugs, most are just "helpful" warnings.

icc doesn't like pointer arithmetic, however these are not fixed up. It
works as-is, just ignore those class of warnings.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
02bcaa8c31feb93c61b701d143a7eea3efd2124d 24-Nov-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Time and seek optimizations

We did too many gettimeofday() calls, this patch cuts the number by
40%. Use clock_gettime() MONOTONIC instead, it is faster on my system
at least.

This patch also optimizes calling lseek() only when necessary for the
sync io engine.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
7abf833d25ab6cc866308fe37483dfe2fde28efc 23-Nov-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Close files on error

The error path doesn't close already opened files. Also set
O_SYNC regardless of file type.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
21972cdef7db6b557c2d56138f8434658a9d1e49 23-Nov-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Always open the files from the job itself

This way we don't fork or spawn threads and make them inherit the open
file.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
132ad46d22bcf12da95ad69e03c1b6f23f5e60a9 23-Nov-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] File close fix

We should unlink and invalidate a closed file, to less impact on
other threads or pending jobs.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f697125ab9003d358a37186d610e09799afd190f 15-Nov-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] nrfiles > 1 random fix

We should only generate a block within the specific file size, not
the total io size.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
a00735e66f9ec42549da94eba3170e543b542904 03-Nov-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Add seperate read/write block size options

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
40f8298cd252157fa15d93f8eb470116a94069a0 25-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Use posix_fallocate() to force file layout

Dunno how well this works on all filesystems, but at least it's
a good hint to send wrt fragmentation and layout.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
c21ac0fff997a382929d1a53cf32d841fc35549f 24-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Only unlink the same file once

If filename= is used, make sure we only do the unlink() once.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
13f8e2d2e3e5ec7d8c18b70fb2a2e2a026190020 24-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Add option to specify the exact file used

Fio typically just makes up a filename for a job, but sometimes you want
to explicitly reuse the same file for multiple jobs. Add a filename=
option to cater to that.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b4a6a59a939f0e6554632de311e7d1d1b7633ccf 20-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Be nicer about cleaning up allocated memory

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
0a7eb121ecb2e4279e9eb8178aa1e554dda40191 20-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] More size fixes

Clean it up so that the create/non-create/bdev paths are the same
to avoid further problems in this area.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f102706350093e60617cc271b128e56c57ab3ce7 20-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Fixup io size on block devices

Still leftover bugs from the > 1 file per job changes.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
25205e975e6dce6079a4b94d656724011f1aabd0 19-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] File creation and info fixes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
74939e38e5164682a57cbe2dfdced09b35cd0259 19-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Total io size / eta fix

Inadvertently the files were counted twice, hence ETA ended at 50%.
This was also the real bug for the verify ETA fixed a few days ago,
so reinstate the proper check when doing verifies as well.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
7313b48c3196b4750f4103e8622126039e2231c6 19-Oct-2006 Jens Axboe <axboe@nelson.home.kernel.dk> Merge branch 'master' of ssh://router/data/git/fio
0ab8db8943acc6b9ea778735563bfad2a79e8ddd 18-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Make fio -W clean again

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
e5b401d4bf67a1704f28872d3abe09eaf65cdabe 18-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Sync and invalidate cache prior to running verify

Make sure we re-read the contents from disk, not from memory.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
f6cbb269c54ff40f3922818d42117f5a8026df36 18-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] Add support for unlinking io files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
b2a151925a91f38aeb298d693687a47269ad4e94 18-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH Various fixes

- Multiple files fixes
- Fix for unaligned io issued for raw io

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c
53cdc6864f7471b28cc9b40a5314ab43e5b1cb5e 18-Oct-2006 Jens Axboe <jens.axboe@oracle.com> [PATCH] First cut at supporting > 1 file per job

This is likely very buggy, a simple test works though.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
/external/fio/filesetup.c