• Home
  • History
  • Annotate
  • only in /external/chromium_org/v8/tools/testrunner/
NameDateSize

..03-Jan-20144 KiB

__init__.py03-Jan-20141.5 KiB

local/03-Jan-20144 KiB

network/03-Jan-20144 KiB

objects/03-Jan-20144 KiB

README03-Jan-20146.7 KiB

server/03-Jan-20144 KiB

README

1Test suite runner for V8, including support for distributed running.
2====================================================================
3
4
5Local usage instructions:
6=========================
7
8Run the main script with --help to get detailed usage instructions:
9
10$ tools/run-tests.py --help
11
12The interface is mostly the same as it was for the old test runner.
13You'll likely want something like this:
14
15$ tools/run-tests.py --nonetwork --arch ia32 --mode release
16
17--nonetwork is the default on Mac and Windows. If you don't specify --arch
18and/or --mode, all available values will be used and run in turn (e.g.,
19omitting --mode from the above example will run ia32 in both Release and Debug
20modes).
21
22
23Networked usage instructions:
24=============================
25
26Networked running is only supported on Linux currently. Make sure that all
27machines participating in the cluster are binary-compatible (e.g. mixing
28Ubuntu Lucid and Precise doesn't work).
29
30Setup:
31------
32
331.) Copy tools/test-server.py to a new empty directory anywhere on your hard
34    drive (preferably not inside your V8 checkout just to keep things clean).
35    Please do create a copy, not just a symlink.
36
372.) Navigate to the new directory and let the server setup itself:
38
39$ ./test-server.py setup
40
41    This will install PIP and UltraJSON, create a V8 working directory, and
42    generate a keypair.
43
443.) Swap public keys with someone who's already part of the networked cluster.
45
46$ cp trusted/`cat data/mypubkey`.pem /where/peers/can/see/it/myname.pem
47$ ./test-server.py approve /wherever/they/put/it/yourname.pem
48
49
50Usage:
51------
52
531.) Start your server:
54
55$ ./test-server.py start
56
572.) (Optionally) inspect the server's status:
58
59$ ./test-server.py status
60
613.) From your regular V8 working directory, run tests:
62
63$ tool/run-tests.py --arch ia32 --mode debug
64
654.) (Optionally) enjoy the speeeeeeeeeeeeeeeed
66
67
68Architecture overview:
69======================
70
71Code organization:
72------------------
73
74This section is written from the point of view of the tools/ directory.
75
76./run-tests.py:
77  Main script. Parses command-line options and drives the test execution
78  procedure from a high level. Imports the actual implementation of all
79  steps from the testrunner/ directory.
80
81./test-server.py:
82  Interface to interact with the server. Contains code to setup the server's
83  working environment and can start and stop server daemon processes.
84  Imports some stuff from the testrunner/server/ directory.
85
86./testrunner/local/*:
87  Implementation needed to run tests locally. Used by run-tests.py. Inspired by
88  (and partly copied verbatim from) the original test.py script.
89
90./testrunner/local/old_statusfile.py:
91  Provides functionality to read an old-style <testsuite>.status file and
92  convert it to new-style syntax. This can be removed once the new-style
93  syntax becomes authoritative (and old-style syntax is no longer supported).
94  ./status-file-converter.py provides a stand-alone interface to this.
95
96./testrunner/objects/*:
97  A bunch of data container classes, used by the scripts in the various other
98  directories; serializable for transmission over the network.
99
100./testrunner/network/*:
101  Equivalents and extensions of some of the functionality in ./testrunner/local/
102  as required when dispatching tests to peers on the network.
103
104./testrunner/network/network_execution.py:
105  Drop-in replacement for ./testrunner/local/execution that distributes
106  test jobs to network peers instead of running them locally.
107
108./testrunner/network/endpoint.py:
109  Receiving end of a network distributed job, uses the implementation
110  in ./testrunner/local/execution.py for actually running the tests.
111
112./testrunner/server/*:
113  Implementation of the daemon that accepts and runs test execution jobs from
114  peers on the network. Should ideally have no dependencies on any of the other
115  directories, but that turned out to be impractical, so there are a few
116  exceptions.
117
118./testrunner/server/compression.py:
119  Defines a wrapper around Python TCP sockets that provides JSON based
120  serialization, gzip based compression, and ensures message completeness.
121
122
123Networking architecture:
124------------------------
125
126The distribution stuff is designed to be a layer between deciding which tests
127to run on the one side, and actually running them on the other. The frontend
128that the user interacts with is the same for local and networked execution,
129and the actual test execution and result gathering code is the same too.
130
131The server daemon starts four separate servers, each listening on another port:
132- "Local": Communication with a run-tests.py script running on the same host.
133  The test driving script e.g. needs to ask for available peers. It then talks
134  to those peers directly (one of them will be the locally running server).
135- "Work": Listens for test job requests from run-tests.py scripts on the network
136  (including localhost). Accepts an arbitrary number of connections at the
137  same time, but only works on them in a serialized fashion.
138- "Status": Used for communication with other servers on the network, e.g. for
139  exchanging trusted public keys to create the transitive trust closure.
140- "Discovery": Used to detect presence of other peers on the network.
141  In contrast to the other three, this uses UDP (as opposed to TCP).
142
143
144Give us a diagram! We love diagrams!
145------------------------------------
146                                     .
147                         Machine A   .  Machine B
148                                     .
149+------------------------------+     .
150|        run-tests.py          |     .
151|         with flag:           |     .
152|--nonetwork   --network       |     .
153|   |          /    |          |     .
154|   |         /     |          |     .
155|   v        /      v          |     .
156|BACKEND    /   distribution   |     .
157+--------- / --------| \ ------+     .
158          /          |  \_____________________
159         /           |               .        \
160        /            |               .         \
161+----- v ----------- v --------+     .    +---- v -----------------------+
162| LocalHandler | WorkHandler   |     .    | WorkHandler   | LocalHandler |
163|              |     |         |     .    |     |         |              |
164|              |     v         |     .    |     v         |              |
165|              |  BACKEND      |     .    |  BACKEND      |              |
166|------------- +---------------|     .    |---------------+--------------|
167| Discovery    | StatusHandler <----------> StatusHandler | Discovery    |
168+---- ^ -----------------------+     .    +-------------------- ^ -------+
169      |                              .                          |
170      +---------------------------------------------------------+
171
172Note that the three occurrences of "BACKEND" are the same code
173(testrunner/local/execution.py and its imports), but running from three
174distinct directories (and on two different machines).
175