1402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<html><head><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> 2402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<style type="text/css">p, table, td, th { font-family: arial, helvetica, geneva; font-size: 10pt} 3402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollpre { font-family: "Courier New", Courier, mono; font-size: 10pt} 4402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollh2 { font-family: arial, helvetica, geneva; font-size: 18pt; font-weight: bold ; line-height: 14px} 5402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollcode { font-family: "Courier New", Courier, mono; font-size: 10pt} 6402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollsup { font-family: arial,helvetica,geneva; font-size: 10px} 7402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollh3 { font-family: arial, helvetica, geneva; font-size: 14pt; font-weight: bold} 8402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollli { font-family: arial, helvetica, geneva; font-size: 10pt} 9402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollh1 { font-family: arial, helvetica, geneva; font-size: 28px; font-weight: bold} 10402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollbody { font-family: arial, helvetica, geneva; font-size: 10pt; clip: rect( ); margin-top: 5mm; margin-left: 3mm} 11402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll.indextop { font-size: x-large;; font-family: Verdana, Arial, Helvetica, sans-serif; font-weight: bold} 12402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll.indexsub { font-size: xx-small;; font-family: Arial, Helvetica, sans-serif; color: #8080FF} 13402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</style> 14402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<body> 15402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p> 16402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h1>Table of contents</h1> 17402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<ul> 18402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li><a href="#scales">Scales</a> 19402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<ul> 20402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li><a href="#percent">Percentage</a></li> 21402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li><a href="#time">Time linear</a></li> 22402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li><a href="#time">Time logarithmic</a></li> 23402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</ul> 24402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</li> 25402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li><a href="#tips">Tips for time scales</a></li> 26402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</ul> 27402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p> 28402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h1><a name="scales">Scales</h1> 29402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollSince 3.5, it's possible to see results in fingerprints with three different 30402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Mollkinds of scale. 31402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h3><a name="percent">Percentage scale</h3> 32402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p>The X axis represents percentage of the variation vs. the given baseline</p> 33402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p>This is the way fingerprints were displayed since the beginning: 34402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p><img src="images/percentage.png"> 35402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<ul> 36402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>Red bar means a regression, even if it's less than the 10% threshold.</li> 37402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>Green bar means an improvement</li> 38402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<li>Gray bar means an 'explained' regression.</li> 39402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</ul> 40402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</p> 41402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h3><a name="time">Linear and logarithmic time scales</h3> 42402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p>For these scales, the X axis represents the duration time of the test.<br> 43402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollThe colors meanings are the same than for the percentage scale.<br> 44402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollThese kind of graphs give a better idea of time duration for each test.</p> 45402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p>Typically use linear scale if you want to see the tests relativeness for all the component tests:</p> 46402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p><img src="images/linear.png"> 47402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p>But the logarithmic scale is more appropriate when there are a strong duration differences between tests, hence makes short duration tests easier to survey:</p> 48402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p><img src="images/log.png"> 49402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p>Each test have two bars: the former is white and shows the baseline result, the latter is colored (red, green or gray) and shows the current build result.<br> 50402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael MollThe variation between the baseline and the build is displayed as a percentage on top of both bars.</p> 51402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<h1><a name="tips">Tips for time scales</h1> 52402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<p>Tips are almost the same for linear and logarithmic scales:</p> 53402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll<table border="0"> 54402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 55402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td valign="top"><img src="/images/light.gif"></td> 56402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><b>Flying over a bar displays its time value<b>:</td></tr> 57402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 58402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 59402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td></td> 60402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><img src="images/help_time_baseline.png"></td> 61402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 62402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 63402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td></td> 64402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><img src="images/help_time_current.png"></td> 65402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 66402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr><td><br></td></tr> 67402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 68402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td valign="top"><img src="/images/light.gif"></td> 69402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><b>For <u>linear scale only</u>, when the error on the time result is noticeable, 70402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll then the measurement uncertainty is shown in yellow at the end of the bar<b>:</td></tr> 71402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 72402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 73402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td></td> 74402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><img src="images/help_time_error.png"></td> 75402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 76402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr><td><br></td></tr> 77402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 78402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td valign="top"><img src="/images/light.gif"></td> 79402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><b>A performance regression may sometimes have a known good reason</b>.<br> 80402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll In this case, the current build bar is grayed and flying over it also shows the given explanation:</td></tr> 81402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 82402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 83402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td></td> 84402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><img src="images/help_explained_regression.png"></td> 85402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 86402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr><td><br></td></tr> 87402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 88402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td valign="top"><img src="/images/light.gif"></td> 89402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><b>Test result may have big error which can make the test result not fully reliable</b>.<br> 90402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll In this case, a warning icon is shown after the variation value and flying over it gives the offending error value:</td></tr> 91402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 92402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 93402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td></td> 94402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><img src="images/help_error_warning.png"></td> 95402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 96402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr><td><br></td></tr> 97402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 98402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td valign="top"><img src="/images/light.gif"></td> 99402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><b>Test may have no result for the used baseline, hence the first available build is used as a reference</b>.<br> 100402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll In this case, a warning icon is shown after the scenario title and flying over it gives the build ID used to compute the variation:</td></tr> 101402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 102402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <tr> 103402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td></td> 104402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll <td><img src="images/help_no_baseline.png"></td> 105402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll </tr> 106402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</table> 107402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</body> 108402794e73aed8611d62eb4b01cd155e2d76fcb87Raphael Moll</html> 109
