1NetPIPE Network Protocol Independent Performance Evaluator, Release 2.4
2Copyright 1997, 1998, 1999 Iowa State University Research Foundation, Inc.
3
4$Id: README,v 1.1 2003/03/28 18:14:44 robbiew Exp $
5
6This program is free software; you can redistribute it and/or modify
7it under the terms of the GNU General Public License as published by
8the Free Software Foundation. You should have received a copy of the
9GNU General Public License along with this program; if not, write to the
10Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
11
12The URL for this document:
13
14ftp://ftp.scl.ameslab.gov/pub/netpipe/README
15
16Getting NetPIPE
17---------------
18
19The NetPIPE implementation in C can be found at:
20
21ftp://ftp.scl.ameslab.gov/pub/netpipe/netpipe-2.4.tar.gz
22
23The source code for NetPIPE 2.4 is provided as a gzipped tar archive,
24which can be uncompressed with "gunzip netpipe-2.4.tar.gz" (or "gzip
25-d netpipe-2.4.tar.gz"), and then extracted from the uncompressed
26archive with the command "tar xvf netpipe-2.4.tar". If you do not
27have the gzip program, it can be obtained as:
28
29ftp://prep.ai.mit.edu/pub/gnu/gzip-1.2.4.tar
30
31Similarly, the NetPIPE implementation in Java can be found at:
32
33ftp://ftp.scl.ameslab.gov/pub/netpipe-Java-1.0.tar.gz
34
35The instructions that follow apply to the C implementation of
36NetPIPE.
37
38What is NetPIPE?
39----------------
40
41NetPIPE is a protocol independent performance tool that encapsulates
42the best of ttcp and netperf and visually represents the network
43performance under a variety of conditions. By taking the end-to-end
44application view of a network, NetPIPE clearly shows the overhead
45associated with different protocol layers. Netpipe answers such
46questions as: how soon will a given data block of size k arrive at its
47destination? Which network and protocol will transmit size k blocks
48the fastest? What is a given network's effective maximum throughput
49and saturation level? Does there exist a block size k for which the
50throughput is maximized? How much communication overhead is due to the
51network communication protocol layer(s)? How quickly will a small (< 1
52kbyte) control message arrive, and which network and protocol are best
53for this purpose?
54
55For a paper fully describing NetPIPE and sample investigation of
56network performance issues using NetPIPE, see
57http://www.scl.ameslab.gov/netpipe/paper/full.html.
58
59Building NetPIPE
60----------------
61
62NetPIPE is provided with protocol-specific shims for TCP (using the
63Berkeley sockets interface), MPI, and PVM. If you do not have MPI or
64PVM, don't worry; TCP is the typical shim used. It should be easy to
65write new interfaces for other protocols based on the examples shown
66by the TCP, MPI and PVM interfaces.
67
68NetPIPE requires an ANSI C compiler.
69
70Review the provided Makefile and change any necessary settings, such
71as the CFLAGS compiler flags, required extra libraries, and MPI or PVM
72library & include file pathnames if you have these communication
73libraries. If you want to turn on getrusage calls to get CPU time
74required for communication, add "-DHAVE_GETRUSAGE" to the CFLAGS line
75in the Makefile.
76
77Compile NetPIPE with the desired communication interface by using the
78command "make TCP", "make MPI", or "make PVM" as appropriate,
79corresponding to the executable files NPtcp, NPmpi, or NPpvm
80respectively.
81
82Consult the appropriate section below for details on running NetPIPE
83over TCP, MPI, or PVM, and the following section on interpreting the
84results.
85
86Running NPtcp
87-------------
88
89For TCP, run a NetPIPE receiver on one computer by issuing the command
90"NPtcp -r". Run a NetPIPE sender on another computer by issuing the
91command "NPtcp -t -h <receiver's address> -o <output file> -P" and any
92other options as appropriate (each option affects only the process on
93which it is specified -- options are not negotiated between the
94transmitter and the receiver):
95
96 -A: specify buffers alignment e.g. "-A 4096"
97
98 -a: asynchronous receive (a.k.a. preposted receive)
99 This option currently has no effect on TCP
100
101 -b: specify send and receive TCP buffer sizes e.g. "-b 32768"
102
103 -h: specify hostname of receiver e.g. "-h mumblehost"
104
105 -i: specify increment step size e.g. "-i 64"
106 Default is exponential increment calculated at runtime
107
108 -l: lower bound (start value for block size) e.g. "-l 1"
109
110 -O: specify buffer offset e.g. "-O 127"
111
112 -o: specify output filename e.g. "-o output.txt"
113
114 -P: print real-time results on stdout
115
116 -p: specify port e.g. "-p 5150"
117
118 -s: stream option (default mode is "ping pong")
119 If this option is used, it must be specified on both
120 the sending and receiving processes
121
122 -u: upper bound (stop value for block size) e.g. "-u 1048576"
123
124Running NPmpi
125-------------
126
127For MPI, how you run NPmpi may depend on the MPI implementation you
128are using. Assuming you are using the "p4" device (for a cluster of
129individual systems interconnected using TCP/IP) in the Argonne MPICH
130implementation, you could run NPmpi one of two ways.
131
132 If your system's default machine file begins with the two
133 names of the systems you want to test, use "mpirun -np 2
134 NPmpi", followed by any of the NetPIPE options listed below.
135
136 Otherwise, create a file that contains the host names of the
137 two systems you want to include in the test, one host name on
138 each line (assume the file is named "machines.p4"). Then, use
139 the command "mpirun -machinefile machines.p4 -np 2 NPmpi",
140 followed by any of the NetPIPE options listed below.
141
142To find out how to run NPmpi using any other implementation of MPI,
143please consult the implementation's documentation.
144
145The NetPIPE options for MPI are:
146
147 -A: specify buffers alignment e.g. "-A 4096"
148
149 -a: asynchronous receive (a.k.a. preposted receive)
150 May not have any effect, depending on your MPI
151 implementation
152
153 -i: specify increment step size e.g. "-i 64"
154 Default is exponential increment calculated at runtime
155
156 -l: lower bound (start value for block size) e.g. "-l 1"
157
158 -O: specify buffer offset e.g. "-O 127"
159
160 -o: specify output filename e.g. "-o output.txt"
161
162 -P: print real-time results on stdout
163
164 -s: stream option (default mode is "ping pong")
165 If this option is used, it must be specified on both
166 the sending and receiving processes
167
168 -u: upper bound (stop value for block size) e.g. "-u 1048576"
169
170Running NPpvm
171-------------
172
173First, start PVM with the command "pvm" on one machine and a second
174machine with the PVM command "add <othermachine>", where
175<othermachine> is the name of the other computer to include in the
176test. Exit the PVM command line interface. Start the receiver
177process on one of the machines with the command "NPpvm -r". Finally,
178start the transmitter process on the other machine with the command
179"NPpvm -t -o <output file> -P" and any other options as appropriate
180(each option affects only the process on which it is specified --
181options are not negotiated between the transmitter and the receiver):
182
183 -A: specify buffers alignment e.g. "-A 4096"
184
185 -a: asynchronous receive (a.k.a. preposted receive)
186 This option has no effect on PVM
187
188 -i: specify increment step size e.g. "-i 64"
189 Default is exponential increment calculated at runtime
190
191 -l: lower bound (start value for block size) e.g. "-l 1"
192
193 -O: specify buffer offset e.g. "-O 127"
194
195 -o: specify output filename e.g. "-o output.txt"
196
197 -P: print real-time results on stdout
198
199 -s: stream option (default mode is "ping pong")
200 If this option is used, it must be specified on both
201 the sending and receiving processes
202
203 -u: upper bound (stop value for block size) e.g. "-u 1048576"
204
205
206Interpreting the Results
207------------------------
208
209NetPIPE's output file contains five columns: time to transfer the block,
210bits per second, bits in block, bytes in block, and variance. These
211columns may be graphed to represent and compare the network's
212performance. For example, the "network signature" graph can be
213created by graphing the time column versus the bits per second column
214(see the NetPIPE report at the URL above for the details why this
215graph is important and how to interpret it). The more traditional
216"throughput versus block size" graph can be created by
217graphing the bytes column versus the bits per second column.
218
219See http://www.scl.ameslab.gov/Projects/ClusterCookbook/nprun.html for
220a detailed tutorial on running NetPIPE and graphing the results.
221
222Help
223----
224
225NetPIPE is currently maintained by Guy Helmer. Email
226"ghelmer@scl.ameslab.gov" or call 515-294-9469 for help or
227suggestions.
228
229Changes
230-------
231
232version 2.4 (12/16/99)
233 * Add getrusage calls to get CPU time used by communication if
234 HAVE_GETRUSAGE is defined (be aware that no studies have been
235 conducted to test the accuracy of results across different systems)
236 * Use "unsigned int" instead of "unsigned long" to communicate 32-bit
237 integers in TCP.c (this solves interoperability problems between
238 Compaq/DEC Alphas and most other systems)
239 * Add dummy "echo" commands after TCP, MPI, and PVM targets in the
240 Makefile. Some implementations of make(1) (such as those found
241 in Linux distributions) interpret the targets with no following
242 statements as a rule to do something silly like
243 "cc -O -o TCP TCP.c" after the dependency is satisfied.
244
245version 2.3 (9/24/98)
246 * Add PVM interface contributed by Clark E. Dorman <dorman@s3i.com>
247
248 * Revamp README file with instructions for NPmpi and NPpvm, and
249 clarify some instructions for NPtcp
250
251version 2.2 (8/21/98):
252 * Carefully check all return values from write(2) and read(2)
253 system calls in TCP.c. Handle short reads properly. Make the Sync()
254 function transmit and receive a useful string which can be
255 checked for validity.
256
257 * Correct the overloading of SendTime() and RecvTime() functions
258 by breaking out SendRepeat() and RecvRepeat() as separate
259 functions.
260
261 * Handle systems whose accept(2) system call does not carry socket
262 options over from the listening socket. In particular, set the
263 TCP_NODELAY flag and socket buffers on an accepted socket.
264