1cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)This file contains high-level info about how to use password manager tests and 2cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)how to create new ones. 3cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 4cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)The password manager tests purpose is to allow automatic password manager 5cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)checking and avoiding to do so manually. 6cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)The tests are written in python using selenium Webdriver library. 7cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 8cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 9cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)=====Getting started===== 10cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 11cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Build ChromeDriver by building the 'chromedriver' target. This will 12cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)create an executable binary in the build folder named 'chromedriver[.exe]'. 13cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 14cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Build chrome too by building the 'chrome' target. This will 15cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)create an executable binary in the build folder named 'chrome[.exe]'. 16cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 17cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Install Selenium (the version tested was 2.41.0): 18cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)pip install -U selenium 19cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 20cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 21cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)For security reasons, we didn't publish the passwords and the usernames we 22cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)used to test. So we put them to an xml file (websites.xml). The structure of 23cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)the file is the following: 24cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)<websites> 25cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) <website name = "website name"> 26cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) <username>username</username> 27cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) <password>password</password> 28cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) </website> 29cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)<websites> 30cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)You can ask someone to give you the websites.xml file and put it in the same 31cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)folder as the tests. You can also create your own websites.xml with your 32cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)personal accounts. 33cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)WARNING: All the content of the PROFILEPATH is going to be deleted. 34cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Show the help: 35cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)python tests.py --help 36cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Run all the working tests tests by executing: 37cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)python tests.py --chrome-path CHROMEPATH --chromedriver-path CHROMEDRIVERPATH 38cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)--profile-path PROFILEPATH [--passwords_path PASSWORDSPATH] 39cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 40cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Run all the tests by executing: 41cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)python tests.py --all --chrome-path CHROMEPATH --chromedriver-path 42cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)CHROMEDRIVERPATH --profile-path PROFILEPATH [--passwords_path PASSWORDSPATH] 43cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 44cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Run one or many tests by executing: 45cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)python tests.py google --chrome-path CHROMEPATH --chromedriver-path 46cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)CHROMEDRIVERPATH profile-path PROFILEPATH [--passwords_path PASSWORDSPATH] 47cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 48cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)python tests.py google facebook --chrome-path CHROMEPATH --chromedriver-path 49cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)CHROMEDRIVERPATH --profile-path PROFILEPATH [--passwords_path PASSWORDSPATH] 50cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 51cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)python tests.py google facebook amazon --chrome-path CHROMEPATH 52cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)--chromedriver-path CHROMEDRIVERPATH --profile-path PROFILEPATH 53cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)[--passwords_path PASSWORDSPATH] 54cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 55cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)To display the debugging messages on the screen, use: 56cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)python tests.py --log DEBUG|INFO|WARNING|ERROR|CRITICAL --log-screen 57cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)To save debugging messages into a file, use: 58cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)python tests.py --log DEBUG|INFO|WARNING|ERROR|CRITICAL --log-file LOG_FILE 59cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 60116680a4aac90f2aa7413d9095a592090648e557Ben MurdochTo save the result of the tests as an xml file, use: 61116680a4aac90f2aa7413d9095a592090648e557Ben Murdochpython tests.py --save-path SAVERESULTPATH 62cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 63cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)=====Creating new test===== 64cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 65cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)1) Open tests.py. 66cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 67cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)2) Add tests like this: 68cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 69cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)class NewWebsiteTest(WebsiteTest): 70cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 71cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) def Login(self): 72cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) # Add login steps for the website, for example: 73cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) self.GoTo("http://url") 74cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) self.FillUsernameInto("Username CSS selector") 75cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) self.FillPasswordInto("Password CSS selector") 76cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) self.Submit("Password CSS selector") 77cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 78cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) def Logout(self): 79cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) # Add logout steps for the website, for example: 80cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) self.Click("Logout button CSS selector") 81cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 82cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Then, to create the new test, you need just to add: 83cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 84cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)environment.AddWebsiteTest(NewWebsiteTest("website name")) 85cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 86cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* For security reasons, passwords and usernames need to be supplied in a 87cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)separate XML file and never checked in to the repository. The XML file should 88cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)contain data structured like this: 89cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 90cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)<website name = "website name"> 91cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) <username>username</username> 92cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) <password>password</password> 93cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)</website> 94cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 95cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)The "website name" is only used to find the username and password in the xml 96cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)file. 97cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 98cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 99cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Use the flowing methods to perform the login and logout: 100cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)The methods that you can use are: 101cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 102cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* Click: find an element using CSS Selector and click on it. Throw an 103cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)exception if the element is not visible. 104cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.Click("css_selector") 105cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* ClickIfClickable: find an element using CSS Selector and click on it if it's 106cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)possible to do that. 107cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.ClickIfClickable("css_selector") 108cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* GoTo: navigate the main frame to a url. 109cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.GoTo("url") 110cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* HoverOver: find an element using CSS Selector and hover over it. 111cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.HoverOver("css_selector") 112cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* SendEnterTo: find an element using CSS Selector and send enter to it. 113cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.SendEnterTo("css_selector") 114cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 115cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* IsDisplayed: check if an element is displayed. 116cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.IsDisplayed("css_selector") 117cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* Wait: wait for some amount of time in seconds. 118cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.Wait(10) 119cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* WaitUntilDisplayed: wait for an element defined using CSS Selector to be 120cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)displayed. 121cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.WaitUntilDisplayed("css_selector") 122cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 123cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* FillPasswordInto: find an input element using CSS Selector and fill it with 124cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)the password. 125cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.FillPasswordInto("css_selector") 126cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* FillUsernameInto: find an input element using CSS Selector and fill it with 127cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)the username. 128cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.FillUsernameInto("css_selector") 129cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* Submit: find an element using CSS Selector and call its submit() handler. 130cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)self.Submit("css_selector") 131cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 132cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 133cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)=====Files structure===== 134cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 135cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Classes: 136cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* environment.py: the definition the tests Environment. 137cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* websitetest.py: WebsiteTest is defined here. You need to create an instance 138cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)of this class for each website you want to test. 139cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 140cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Tests: 141cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* tests.py: the tests setup and the configuration for each website happens 142cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)here. This file contain 3 separate kinds of tests: 143cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 144cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)1) working tests: tests that are supposed to work. If you have a problem with 145cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)one of them, rerun it again. Or try using the Known Issues section to fix it. 146cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)2) tests that can cause a crash (the cause of the crash is not related to the 147cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)password manager): This means that this set is expected to become a working 148cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)test or failing test when the issue that causes the crash now is solved. 149cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)3) failing tests: tests that fail for known bug related to the password 150cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)manager. When this bug is solved, all the tests that were failing because of 151cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)it are going to be moved to working tests. 152cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 153cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)Other files: 154cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* websites.xml : a private file where you can find all the passwords. You can 155cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)ask someone to give it to you or just create your own with your personal 156cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)accounts. 157cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)<websites> 158cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) <website name = "website name"> 159cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) <username>username</username> 160cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) <password>password</password> 161cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) </website> 162cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)</websites> 163cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 164cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 165cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)=====Known Issues===== 166cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles) 167cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)The tests are very fragile. Here are some suggestions for solving most of the 168cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)problems: 169cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* Restart the tests. 170cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* Remove the profile if the tests fail at the beginning for unknown reason. 171cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* If tests fail, isolate the one that causes problem, read debugging messages 172cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)and keep your eyes on the browser window to understand its causes: 173cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)a) In the tests, we often need to wait for a menu to appear ... If the 174cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)menu takes more time to appear than expected, the tests are going to fail. 175cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)b) The websites change very often. And even if they are not changed, they some 176cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)time show a popup that broke the tests. In the case you need to login manually 177cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)to the website, close all popup and logout. 178cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)* If you are logged in when the tests crashes, don't forget to log out before 179cedac228d2dd51db4b79ea1e72c7f249408ee061Torne (Richard Coles)running the tests a second time. 180