1402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> 2402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll 3402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<html> 4402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<head> 5402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> 6402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <title>Components view</title> 7402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</head> 8402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll 9402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<body> 10402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h1>Components view</h1> 11402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p> 12402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollThis view shows the performance results in similar way than the one used to generate 13402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollthe performance results, hence make it easy to match the numbers in the corresponding 14402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollpage HTML page. 15402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p><p> 16402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollWhen starting the tool for the first time, this view is empty as no data has been 17402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollpopulated, neither from the <a href="local_data.html">local data</a> files 18402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollnor from the performance results database. 19402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p> 20402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll 21402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll 22402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h2>Hierarchical tree</h2> 23402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p> 24402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollTypically, the Eclipse builder runs performance tests for each component after 25402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollthe build is made on several performance test machines. Each component defines one 26402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollor several specific performance test suites made of several test (aka scenario). 27402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p><p> 28402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollSeveral performance numbers (e.g. Elapsed Process Time and CPU Time) are stored 29402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollfor each scenario and all build results are available in the performance results 30402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Molldatabase. 31402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p><p> 32402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollHence the tree structure is made as follow: 33402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<pre> 34402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollComponent 35402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll + Scenario 36402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll + Test machine 37402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll + Build 38402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll + Performance numbers 39402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</pre> 40402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Molland may look as follow: 41402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p><img src="images/components.png" alt="Components view"/></p> 42402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll 43402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h2>Icons</h2> 44402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p> 45402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollSeveral icons are displayed on tree element, here are their meaning. 46402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p><p> 47402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollThe red cross means that there's at least one scenario on one machine for 48402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollthe last build with a failure (i.e. a regression over 10%). 49402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p><p> 50402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollThe warning icon means that some warnings occur for some results. The current possible warning are: 51402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<ul> 52402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>error over the 3% threshold on test(s)</li> 53402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>unreliable test(s): the deviation through the test(s) history is over 20%</li> 54402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>unstable test(s): the deviation through the test(s) history is between 10 and 20%</li> 55402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>no baseline for test(s)</li> 56402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>only one run on test(s)</li> 57402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</ul> 58402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p><p> 59402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollThe information icon gives some other interesting information: 60402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<ul> 61402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>the Student T-test fails on test(s)</li> 62402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>the test(s) value or its delta is less than 100ms</li> 63402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</ul> 64402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p><p> 65402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollNote that for component and scenario level, the status is the aggregation of 66402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollthe children status. That means that as soon as one scenario is in error then 67402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollthe component is also flagged in error. And of course the higher severity is 68402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Molldisplayed masking lower possible icons. 69402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p> 70402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll 71402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h2>Filters</h2> 72402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p> 73402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollThere are several possible filters in this view: 74402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll 75402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h3>Builds filters</h3> 76402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<ul> 77402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>Baseline: hide the baselines (starting with R-3.x)</li> 78402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>Nightly: hide the nightly builds (starting with 'N')</li> 79402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>Old: hide all builds before last milestone except earlier milestones</li> 80402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</ul> 81402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll 82402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h3>Scenarios filter</h3> 83402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<ul> 84402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>Advanced scenarios: hide the scenarios which are not in the fingerprints</li> 85402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</ul> 86402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p> 87402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollAs baselines results are not really useful for the survey, the filter is activated 88402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollby default in this view. Currently the survey only concerns the fingerprints 89402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollscenario, hence the corresponding filter is also activated by default. 90402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p> 91402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</body> 92402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</html>
