c38e38f16f77567a423f75c26e1929cbe4902e24 |
|
10-Nov-2014 |
Vasily Tarasov <tarasov@vasily.name> |
Fix for a race when fio prints I/O statistics periodically Below is the demonstration for the latest code in git: Job-file: [job] filename=/dev/sda rw=randread blocksize=4096 direct=1 time_based=1 runtime=300 numjobs=16 Command line: fio --output-format=json --status-interval=10 /tmp/job.file | tee /tmp/fio.out After parsing json and printing per-job "read->runtime" and "read->iops" in two columns, one can see this: runtime iops 9854 4762 19861 4724 29867 4711 39873 4705 49880 4701 59886 4698 69894 4696 79900 4694 89907 4693 99913 4692 109919 4692 119926 4691 129933 4691 139940 4691 149946 4691 159953 4690 169959 4690 179966 4690 189972 4690 199978 4690 209984 4690 219990 4690 229996 4690 240002 4690 250009 4690 260015 4690 270021 4690 280027 4690 290033 4690 600043 2344 <--- You can see a spurious jump in runtime here 300001 4690 The problem is two-fold, IMHO: 1) td->ts.runtime[DDIR_READ/DDIR_WRITE/DDIR_TRIM] are set in microseconds instead of milliseconds in thread_main() function. The values are later converted to milliseconds (after exiting from the loop in thread_main()), but it is too late for the out-of-band access by the helper_thread (the one that prints periodic statistics). 2) The helper_thread in __show_running_run_stats() temporarily adds to td->ts.runtime[DDIR_READ/DDIR_WRITE/DDIR_TRIM] the time that has passed since the beginning of the loop (td->start). However, the same addition happens in thread_main() by the job thread and td->start is not reset atomically with this addition. As a result, the runtime can be added twice. (That's why in the output we see 600043, which is approximately twice larger than the previous reading). My proposal is to make the job threads updating both td->ts.runtime and td->start atomically by using an already existing stat_mutex. __show_running_run_stats() already takes this mutex where needed. Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
4ffeb0addbb090160f2d4eb1e98a1f184ed9ac66 |
|
26-Oct-2014 |
Gwendal Grignou <gwendal@chromium.org> |
fio: Fix padding properly Completely fix padding: - use anonymous union for padding. - move existing padding in thread_stat. - add alignment checks. Signed-off-by: Gwendal Grignou <gwendal@chromium.org> Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
f5a94181a29ed2ee0d69a2cd0bb38a890e8f66ba |
|
24-Oct-2014 |
Jens Axboe <axboe@fb.com> |
Fix fp alignment Commit a5bbbd8a324a fixed alignment earlier up, but then screwed it up further down. Fix it. Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
98d53159f44f24cae23c0ce9d4674b23fd2b2706 |
|
24-Oct-2014 |
Gwendal Grignou <gwendal@chromium.org> |
fio: fix alignement to prevent bus error on ARM Add a filler field to be ensure 64bit alignment. Otherwise, we would trigger SIGBUS error in sum_stat() Signed-off-by: Gwendal Grignou <gwendal@chromium.org> Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
7fe36313273ef051670d16aa27953699fd5cdf06 |
|
24-Oct-2014 |
Jens Axboe <axboe@fb.com> |
Consolidate disk util, eta, and status check thread We don't need two, we can just have one do everything. Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
fdb0da8028e156c0da43aca18e1423d1b300bdad |
|
23-Oct-2014 |
Jens Axboe <axboe@fb.com> |
Start and stop status interval (or file) thread separately This gets rid of the need to create threads on the fly for this, and we can manage it and stop it before we do real stats at the end. Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
72d83a3352d9fb0f7b3ffa5b4a92de1f9fb02c2c |
|
20-Oct-2014 |
Jens Axboe <axboe@fb.com> |
stat: ensure that the fp fields are 64-bit aligned Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
67e149c4c2b580089287813246344908770c3be9 |
|
09-Oct-2014 |
Jens Axboe <axboe@fb.com> |
stat: add dropped ios to the standard output Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
2e627243f7877cefb3606ef99ec24debe32ac98c |
|
25-Jul-2014 |
Jens Axboe <axboe@fb.com> |
Don't grab stat mutex for final stat output We should not need it, and there's a bug with getting stuck on it that hasn't been fully debugged/understood yet. So for now, lets just forget about the mutex for final output. Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
eb663201be7f38339e9177fbbd2b99da30f19e61 |
|
30-Jun-2014 |
Jens Axboe <axboe@fb.com> |
server: pack over-the-wire structs Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
ce8ea6e7f0a08f282d06dd3f0890fe7128146f44 |
|
27-Jun-2014 |
Jens Axboe <axboe@fb.com> |
Fix struct jobs_eta packing on 32 vs 64-bit archs Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
61f6cced3dcde7e4a6fc9fca01783ec80ff269a4 |
|
24-Jun-2014 |
Jens Axboe <axboe@fb.com> |
Add a get_jobs_eta() to return jobs eta information We duplicate this code in eta.c and server.c, consolidate it. Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
1814876d9e82b2b03b042581b5d2c743638726e5 |
|
24-Jun-2014 |
Jens Axboe <axboe@fb.com> |
Ensure we have enough room for the ETA runstr The calculation of 1:4 is wrong, it's 1:5. And we missed the similar change in server.c. Reported-by: Andreas Grünbacher <andreas.gruenbacher@gmail.com> Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
4e59d0f3a3ff569bddb31fe6927a9faf204ae9bf |
|
14-Mar-2014 |
Jens Axboe <axboe@fb.com> |
Bump length of description field to 256 chars Ensure that string always end up \0 terminated, too. Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
6bb58215842760895071d9f331da4dc2dfc16f30 |
|
21-Feb-2014 |
Jens Axboe <axboe@fb.com> |
Improve latency_target runs Reset the stats when we have found our target, and then do another latency_window run with those settings so that the final results reflect the probed values. Signed-off-by: Jens Axboe <axboe@fb.com>
/external/fio/stat.h
|
3e260a46ea9a8de224c3d0a29a608da3440f284a |
|
09-Dec-2013 |
Jens Axboe <axboe@kernel.dk> |
Add options to have fio latency profile a device This adds three new options: - latency_target. This defines a specific latency target, in usec. - latency_window. This defines the period over which fio samples. - latency_percentile. This defines the percentage of IOs that must meet the criteria specified by latency_target/latency_window. With these options set, fio will run the described workload and vary the queue depth between 1 and iodepth= to find the best performing spot that meets the criteria specified by the three options. A sample job file is also added to demonstrate how to use this. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
952b05e00103bf45576e1860cde0626bd42ed52a |
|
31-Oct-2013 |
Castor Fu <castor@alumni.caltech.edu> |
Add json output for client/server mode In client/server mode, this adds support for json mode. Each job's details are gradually accumulated and upon completion the client dumps the full json tree. The tree is annotated with the server host and port. Signed-off-by: Castor Fu <castor@alumni.caltech.edu> Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
cef9175e52e8a240117b9f45b228fdaa0f1c0572 |
|
27-Apr-2013 |
Jens Axboe <axboe@kernel.dk> |
Ensure that stat outputs are serialized Two issues: - We don't want potentially two stat outputs running at the same time. This could potentially happen if USR1 or interval stats race with the thread exit. - We could potentially crash if the interval or USR1 signaled stat output trigger when fio is tearing down. Let the main thread wait for any in-progress output in that case. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
06464907159baf7a1eeac654a023743e33f28d86 |
|
25-Apr-2013 |
Jens Axboe <axboe@kernel.dk> |
Add support for other ways of triggering intermediate result outputs Fio has support for using the USR1 signal to dump the current results while continuing to run. Add a --status-interval parameter to allow this to be configured to automatically happen every X seconds. There's also support for signaling fio through the file system. To do that, simply touch /tmp/fio-dump-status. When fio sees this file, it will unlink and dump the current results. A small tweak is needed for the file approach to work in Windows. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
714e85f340194409d0abcf643cd2f154a7380a74 |
|
15-Apr-2013 |
Jens Axboe <axboe@kernel.dk> |
eta: improve output for threads sitting in TD_SETTING_UP Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
22f80458a520d0c5371c64bf91d24cdd21dff825 |
|
09-Apr-2013 |
Jens Axboe <axboe@kernel.dk> |
Merge branch 'master' into gfio Conflicts: eta.c fio.h init.c options.c stat.c Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
ad705bcb7e79a7cdb9891db17b4c40b13b6c30c3 |
|
09-Apr-2013 |
Steven Noonan <steven@uplinklabs.net> |
implement 'unit_base' option to select between KB and Kbit et. al. With network testing, it's often desirable to measure in terms of kilobits/megabits rather than kilobytes/megabytes. This adds an option named 'unit_base' which can be set to either '1' or '8', where '1' means represent in terms of bits and '8' means to represent rate in terms of bytes. Signed-off-by: Steven Noonan <steven@uplinklabs.net> Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
9db01ef976006c002b05fa4e4ec589eb029aac5b |
|
07-Feb-2013 |
Jens Axboe <axboe@kernel.dk> |
Merge branch 'master' into gfio Conflicts: Makefile configure fio.h Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
435d195a9da120c5a618129cdb73418f4748c20a |
|
06-Feb-2013 |
Vincent Kang Fu <VFu@fusionio.com> |
Handle percentile lists with higher precision that 2 digits We cap the output at %2.2f right now, that's not always enough. Make the parser check and store the precision required to output the list correctly. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
95820b6e6c92025df8d89c0bf39b174e53137c41 |
|
31-Jan-2013 |
Jens Axboe <axboe@kernel.dk> |
Merge branch 'master' into gfio Conflicts: Makefile client.c configure fio.c fio.h server.c server.h Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
771e58befea806d2d881953050c4e65329eee382 |
|
30-Jan-2013 |
Jens Axboe <axboe@kernel.dk> |
Add unified_rw_reporting option If this is set, then fio will sum and display just a single set of statistics for any IO type. By default, fio accounts and reports each data direction separately. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
92a0daf8163eaa8d117d1e6e4fdc11a8dddac2d2 |
|
09-Jan-2013 |
Jens Axboe <axboe@kernel.dk> |
Merge branch 'master' into gfio Conflicts: client.c server.h Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
372aecb9049618c1279a946b66f6b29066730951 |
|
08-Jan-2013 |
Jens Axboe <axboe@kernel.dk> |
Use variable[] instead of GCC variable[0] syntax Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
d79db1222039e906dd49ae290daa59701f4e2385 |
|
24-Sep-2012 |
Jens Axboe <axboe@kernel.dk> |
Merge branch 'master' into gfio Conflicts: Makefile backend.c client.c fio.h init.c io_ddir.h options.c server.h stat.c stat.h Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
6eaf09d6e9ca1f8accb057cdb18620b7e53ae33f |
|
14-Sep-2012 |
Shaohua Li <shli@fusionio.com> |
Add support for trim as a workload type This only works on Linux so far, and it's always sync given what the interface to the kernel looks like. Also restricted to pure block devices. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
c41a9d06a1957ba7c5a019e3d6088a5e8e8e0a47 |
|
11-May-2012 |
Jens Axboe <axboe@kernel.dk> |
Merge branch 'master' into gfio Conflicts: client.c eta.c server.c server.h Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
b7f05eb03c84bdc1259d1bb1c348328b16164430 |
|
11-May-2012 |
Jens Axboe <axboe@kernel.dk> |
eta: ETA bandwidth was off by 1.024 Same fix as 033bbb51 essentially. Since mtime is a 1000th of a second, for the usual option of having 1024 be the KB base, we end up being off by 1.024 if we don't multiply by 1000 before dividing by the runtime (and then dividing by 1024 to get to next power-of-2). Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
4c6d91e8808fd11f3cee663e42db813b0934c8bb |
|
30-Mar-2012 |
Jens Axboe <axboe@kernel.dk> |
Define SIGUSR1 to inform fio to dump run stats while continuing to run For long running jobs, it may be interesting to see the stats without having to terminate the job. Set up SIGUSR1 so that it does that, gives you the stats at that point in time. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
b852e7cf097cec1c7fb92305f9c3c6b394bb39ad |
|
30-Mar-2012 |
Jens Axboe <axboe@kernel.dk> |
Define SIGUSR1 to inform fio to dump run stats while continuing to run For long running jobs, it may be interesting to see the stats without having to terminate the job. Set up SIGUSR1 so that it does that, gives you the stats at that point in time. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
2f122b135b7319ff8dd04dadf31ff28b301051a3 |
|
15-Mar-2012 |
Jens Axboe <axboe@kernel.dk> |
Add thread number ID to appropriate network commands The client doesn't necessarily have a 1:1 mapping between jobs and its internal job representation, so allow it to tell the various jobs apart. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
ec41265e81c70d8573d1359e27876c37c30c7d9d |
|
08-Mar-2012 |
Jens Axboe <axboe@kernel.dk> |
Abstract out the thread_options structure We need to prepare to have an on-disk/net version of the same. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
2e33101f218a1603eeecca969f5b7a0e98696a01 |
|
05-Mar-2012 |
Jens Axboe <axboe@kernel.dk> |
gfio: add os resource usage and io depth display IO depths is a bit of a hack, needs work. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
e5bd13470beaeed9c4a6835b7b92265fb94173a9 |
|
05-Mar-2012 |
Jens Axboe <axboe@kernel.dk> |
gfio: add latency bucket display Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
a269790cd25788dd4226641a3ceab1b3c8fda14b |
|
05-Mar-2012 |
Jens Axboe <axboe@kernel.dk> |
Add completion percentiles to results output Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
b29ad56266faa33326de00e315d2b34b735fb028 |
|
05-Mar-2012 |
Jens Axboe <axboe@kernel.dk> |
Unify cli latency display And export usec_to_msec(). Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
3e47bd250cac5fb81a5c0ad578dfbe90c6ddf6de |
|
29-Feb-2012 |
Jens Axboe <axboe@kernel.dk> |
Update GUI to attempt to graphically handle ETA output The whole layout will probably be changed, but for now it demonstrates how to properly integrate with the net client to handle the data and output it. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
37f0c1ae23ad1716403d3d113c3dfdf41c47e329 |
|
11-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
client: display summed total of all clients when all stats have been received Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
5b9babb7fb9ac46c0e960ccd88c2d85ba3065c01 |
|
10-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
Abstract out and export summation of thread_stats Client side will need it. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
af9c9fb34e420fc4d9cf317aa0f3cf6795a5a07f |
|
09-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
client/server: request ETA instead of having the server send it automatically Also changes the 'serial' of the command to a tag, that's passed back and forth for commands that need to use it. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
c8eeb9df1f52f28567a5937e141decc6a26ec30b |
|
05-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
Add support for write_iops_log Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
802ad4a83e92a30b5fdccf117d59fbb69068c054 |
|
05-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
Change network transmitted doubles to fio_fp64_t IEEE 754 type Encode and decode when transferring over the network. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
1d1f45aec4bd2c1168ed5539174a821555db6f65 |
|
03-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
server: send network copy of run_str[] Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
cf451d1ede3bbbd2fed3619eb43def054d5b5a5a |
|
03-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
server: add ETA as a specific command No more text passing for ETA. Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
b75a394fd0e9455ab71912d4462aaae7e6e4366b |
|
03-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
Abstract out calculation of ETA from display of ETA Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
ddcc0b69aa4ed04c8681f447a1a6274bb8837a14 |
|
03-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
Add type checking to 16/32/64 endianness converters Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|
a64e88dad0c0e4a510ae8ab54cde1a20b99c59d1 |
|
03-Oct-2011 |
Jens Axboe <axboe@kernel.dk> |
server: transmit status as structures, not text Signed-off-by: Jens Axboe <axboe@kernel.dk>
/external/fio/stat.h
|