14b6829f0d28990dd645e16386eb226d0f10c8731shiqian#!/usr/bin/env python 24b6829f0d28990dd645e16386eb226d0f10c8731shiqian# 36d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan# Copyright 2005 Google Inc. All Rights Reserved. 44b6829f0d28990dd645e16386eb226d0f10c8731shiqian# 54b6829f0d28990dd645e16386eb226d0f10c8731shiqian# Redistribution and use in source and binary forms, with or without 64b6829f0d28990dd645e16386eb226d0f10c8731shiqian# modification, are permitted provided that the following conditions are 74b6829f0d28990dd645e16386eb226d0f10c8731shiqian# met: 84b6829f0d28990dd645e16386eb226d0f10c8731shiqian# 94b6829f0d28990dd645e16386eb226d0f10c8731shiqian# * Redistributions of source code must retain the above copyright 104b6829f0d28990dd645e16386eb226d0f10c8731shiqian# notice, this list of conditions and the following disclaimer. 114b6829f0d28990dd645e16386eb226d0f10c8731shiqian# * Redistributions in binary form must reproduce the above 124b6829f0d28990dd645e16386eb226d0f10c8731shiqian# copyright notice, this list of conditions and the following disclaimer 134b6829f0d28990dd645e16386eb226d0f10c8731shiqian# in the documentation and/or other materials provided with the 144b6829f0d28990dd645e16386eb226d0f10c8731shiqian# distribution. 154b6829f0d28990dd645e16386eb226d0f10c8731shiqian# * Neither the name of Google Inc. nor the names of its 164b6829f0d28990dd645e16386eb226d0f10c8731shiqian# contributors may be used to endorse or promote products derived from 174b6829f0d28990dd645e16386eb226d0f10c8731shiqian# this software without specific prior written permission. 184b6829f0d28990dd645e16386eb226d0f10c8731shiqian# 194b6829f0d28990dd645e16386eb226d0f10c8731shiqian# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 204b6829f0d28990dd645e16386eb226d0f10c8731shiqian# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 214b6829f0d28990dd645e16386eb226d0f10c8731shiqian# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 224b6829f0d28990dd645e16386eb226d0f10c8731shiqian# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 234b6829f0d28990dd645e16386eb226d0f10c8731shiqian# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 244b6829f0d28990dd645e16386eb226d0f10c8731shiqian# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 254b6829f0d28990dd645e16386eb226d0f10c8731shiqian# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 264b6829f0d28990dd645e16386eb226d0f10c8731shiqian# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 274b6829f0d28990dd645e16386eb226d0f10c8731shiqian# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 284b6829f0d28990dd645e16386eb226d0f10c8731shiqian# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 294b6829f0d28990dd645e16386eb226d0f10c8731shiqian# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 304b6829f0d28990dd645e16386eb226d0f10c8731shiqian 314b6829f0d28990dd645e16386eb226d0f10c8731shiqian"""Unit test for Google Test test filters. 324b6829f0d28990dd645e16386eb226d0f10c8731shiqian 334b6829f0d28990dd645e16386eb226d0f10c8731shiqianA user can specify which test(s) in a Google Test program to run via either 344b6829f0d28990dd645e16386eb226d0f10c8731shiqianthe GTEST_FILTER environment variable or the --gtest_filter flag. 354b6829f0d28990dd645e16386eb226d0f10c8731shiqianThis script tests such functionality by invoking 364b6829f0d28990dd645e16386eb226d0f10c8731shiqiangtest_filter_unittest_ (a program written with Google Test) with different 374b6829f0d28990dd645e16386eb226d0f10c8731shiqianenvironments and command line flags. 38905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 39905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wanNote that test sharding may also influence which tests are filtered. Therefore, 40905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wanwe test that here also. 414b6829f0d28990dd645e16386eb226d0f10c8731shiqian""" 424b6829f0d28990dd645e16386eb226d0f10c8731shiqian 434b6829f0d28990dd645e16386eb226d0f10c8731shiqian__author__ = 'wan@google.com (Zhanyong Wan)' 444b6829f0d28990dd645e16386eb226d0f10c8731shiqian 454b6829f0d28990dd645e16386eb226d0f10c8731shiqianimport os 464b6829f0d28990dd645e16386eb226d0f10c8731shiqianimport re 47f904a612d9444ab36c07a8e619c113432e046f49vladlosevimport sets 48e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevimport sys 49e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev 50ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqianimport gtest_test_utils 514b6829f0d28990dd645e16386eb226d0f10c8731shiqian 524b6829f0d28990dd645e16386eb226d0f10c8731shiqian# Constants. 534b6829f0d28990dd645e16386eb226d0f10c8731shiqian 54e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# Checks if this platform can pass empty environment variables to child 55e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# processes. We set an env variable to an empty string and invoke a python 56e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# script in a subprocess to print whether the variable is STILL in 57e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# os.environ. We then use 'eval' to parse the child's output so that an 58e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# exception is thrown if the input is anything other than 'True' nor 'False'. 59e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevos.environ['EMPTY_VAR'] = '' 60e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevchild = gtest_test_utils.Subprocess( 61e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev [sys.executable, '-c', 'import os; print \'EMPTY_VAR\' in os.environ']) 62e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevCAN_PASS_EMPTY_ENV = eval(child.output) 63e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev 64e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev 65e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# Check if this platform can unset environment variables in child processes. 66e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# We set an env variable to a non-empty string, unset it, and invoke 67e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# a python script in a subprocess to print whether the variable 68e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# is NO LONGER in os.environ. 69e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# We use 'eval' to parse the child's output so that an exception 70e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# is thrown if the input is neither 'True' nor 'False'. 71e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevos.environ['UNSET_VAR'] = 'X' 72e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevdel os.environ['UNSET_VAR'] 73e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevchild = gtest_test_utils.Subprocess( 74e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev [sys.executable, '-c', 'import os; print \'UNSET_VAR\' not in os.environ']) 75e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevCAN_UNSET_ENV = eval(child.output) 76e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev 77e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev 78e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# Checks if we should test with an empty filter. This doesn't 79e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# make sense on platforms that cannot pass empty env variables (Win32) 80e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# and on platforms that cannot unset variables (since we cannot tell 81e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev# the difference between "" and NULL -- Borland and Solaris < 5.10) 82e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevCAN_TEST_EMPTY_FILTER = (CAN_PASS_EMPTY_ENV and CAN_UNSET_ENV) 83e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev 84767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan 854b6829f0d28990dd645e16386eb226d0f10c8731shiqian# The environment variable for specifying the test filters. 864b6829f0d28990dd645e16386eb226d0f10c8731shiqianFILTER_ENV_VAR = 'GTEST_FILTER' 874b6829f0d28990dd645e16386eb226d0f10c8731shiqian 88905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan# The environment variables for test sharding. 89905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wanTOTAL_SHARDS_ENV_VAR = 'GTEST_TOTAL_SHARDS' 90905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wanSHARD_INDEX_ENV_VAR = 'GTEST_SHARD_INDEX' 91905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wanSHARD_STATUS_FILE_ENV_VAR = 'GTEST_SHARD_STATUS_FILE' 92905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 934b6829f0d28990dd645e16386eb226d0f10c8731shiqian# The command line flag for specifying the test filters. 944b6829f0d28990dd645e16386eb226d0f10c8731shiqianFILTER_FLAG = 'gtest_filter' 954b6829f0d28990dd645e16386eb226d0f10c8731shiqian 96ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian# The command line flag for including disabled tests. 97ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqianALSO_RUN_DISABED_TESTS_FLAG = 'gtest_also_run_disabled_tests' 98ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 994b6829f0d28990dd645e16386eb226d0f10c8731shiqian# Command to run the gtest_filter_unittest_ program. 100767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wanCOMMAND = gtest_test_utils.GetTestExecutablePath('gtest_filter_unittest_') 1014b6829f0d28990dd645e16386eb226d0f10c8731shiqian 102f904a612d9444ab36c07a8e619c113432e046f49vladlosev# Regex for determining whether parameterized tests are enabled in the binary. 103f904a612d9444ab36c07a8e619c113432e046f49vladlosevPARAM_TEST_REGEX = re.compile(r'/ParamTest') 104f904a612d9444ab36c07a8e619c113432e046f49vladlosev 1054b6829f0d28990dd645e16386eb226d0f10c8731shiqian# Regex for parsing test case names from Google Test's output. 106f904a612d9444ab36c07a8e619c113432e046f49vladlosevTEST_CASE_REGEX = re.compile(r'^\[\-+\] \d+ tests? from (\w+(/\w+)?)') 1074b6829f0d28990dd645e16386eb226d0f10c8731shiqian 1084b6829f0d28990dd645e16386eb226d0f10c8731shiqian# Regex for parsing test names from Google Test's output. 109f904a612d9444ab36c07a8e619c113432e046f49vladlosevTEST_REGEX = re.compile(r'^\[\s*RUN\s*\].*\.(\w+(/\w+)?)') 1104b6829f0d28990dd645e16386eb226d0f10c8731shiqian 111a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev# The command line flag to tell Google Test to output the list of tests it 112a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev# will run. 113a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosevLIST_TESTS_FLAG = '--gtest_list_tests' 114a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 115a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev# Indicates whether Google Test supports death tests. 116a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosevSUPPORTS_DEATH_TESTS = 'HasDeathTest' in gtest_test_utils.Subprocess( 117a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev [COMMAND, LIST_TESTS_FLAG]).output 118a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 1194b6829f0d28990dd645e16386eb226d0f10c8731shiqian# Full names of all tests in gtest_filter_unittests_. 120f904a612d9444ab36c07a8e619c113432e046f49vladlosevPARAM_TESTS = [ 121f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestX/0', 122f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestX/1', 123f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestY/0', 124f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestY/1', 125f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqQ/ParamTest.TestX/0', 126f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqQ/ParamTest.TestX/1', 127f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqQ/ParamTest.TestY/0', 128f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqQ/ParamTest.TestY/1', 129f904a612d9444ab36c07a8e619c113432e046f49vladlosev ] 130f904a612d9444ab36c07a8e619c113432e046f49vladlosev 131ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqianDISABLED_TESTS = [ 132ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'BarTest.DISABLED_TestFour', 133ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'BarTest.DISABLED_TestFive', 134ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'BazTest.DISABLED_TestC', 135ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'DISABLED_FoobarTest.Test1', 136ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'DISABLED_FoobarTest.DISABLED_Test2', 137ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'DISABLED_FoobarbazTest.TestA', 138ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian ] 139ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 140a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosevif SUPPORTS_DEATH_TESTS: 141a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev DEATH_TESTS = [ 142a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'HasDeathTest.Test1', 143a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'HasDeathTest.Test2', 144a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev ] 145a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosevelse: 146a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev DEATH_TESTS = [] 147a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 148ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian# All the non-disabled tests. 149ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqianACTIVE_TESTS = [ 1504b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Abc', 1514b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Xyz', 1524b6829f0d28990dd645e16386eb226d0f10c8731shiqian 153f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestOne', 154f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestTwo', 155f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestThree', 1564b6829f0d28990dd645e16386eb226d0f10c8731shiqian 157f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BazTest.TestOne', 1584b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestA', 1594b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestB', 160a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev ] + DEATH_TESTS + PARAM_TESTS 1614b6829f0d28990dd645e16386eb226d0f10c8731shiqian 162f904a612d9444ab36c07a8e619c113432e046f49vladlosevparam_tests_present = None 1634b6829f0d28990dd645e16386eb226d0f10c8731shiqian 1644b6829f0d28990dd645e16386eb226d0f10c8731shiqian# Utilities. 1654b6829f0d28990dd645e16386eb226d0f10c8731shiqian 166e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosevenviron = os.environ.copy() 167e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev 1684b6829f0d28990dd645e16386eb226d0f10c8731shiqian 1694b6829f0d28990dd645e16386eb226d0f10c8731shiqiandef SetEnvVar(env_var, value): 1704b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Sets the env variable to 'value'; unsets it when 'value' is None.""" 1714b6829f0d28990dd645e16386eb226d0f10c8731shiqian 1724b6829f0d28990dd645e16386eb226d0f10c8731shiqian if value is not None: 173e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev environ[env_var] = value 174e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev elif env_var in environ: 175e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev del environ[env_var] 1764b6829f0d28990dd645e16386eb226d0f10c8731shiqian 1774b6829f0d28990dd645e16386eb226d0f10c8731shiqian 178b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wandef RunAndReturnOutput(args = None): 179b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan """Runs the test program and returns its output.""" 1804b6829f0d28990dd645e16386eb226d0f10c8731shiqian 181e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev return gtest_test_utils.Subprocess([COMMAND] + (args or []), 182e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev env=environ).output 183b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan 184b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan 185b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wandef RunAndExtractTestList(args = None): 186b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan """Runs the test program and returns its exit code and a list of tests run.""" 187b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan 188e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev p = gtest_test_utils.Subprocess([COMMAND] + (args or []), env=environ) 1894b6829f0d28990dd645e16386eb226d0f10c8731shiqian tests_run = [] 1904b6829f0d28990dd645e16386eb226d0f10c8731shiqian test_case = '' 1914b6829f0d28990dd645e16386eb226d0f10c8731shiqian test = '' 192b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan for line in p.output.split('\n'): 1934b6829f0d28990dd645e16386eb226d0f10c8731shiqian match = TEST_CASE_REGEX.match(line) 1944b6829f0d28990dd645e16386eb226d0f10c8731shiqian if match is not None: 1954b6829f0d28990dd645e16386eb226d0f10c8731shiqian test_case = match.group(1) 1964b6829f0d28990dd645e16386eb226d0f10c8731shiqian else: 1974b6829f0d28990dd645e16386eb226d0f10c8731shiqian match = TEST_REGEX.match(line) 1984b6829f0d28990dd645e16386eb226d0f10c8731shiqian if match is not None: 1994b6829f0d28990dd645e16386eb226d0f10c8731shiqian test = match.group(1) 200b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan tests_run.append(test_case + '.' + test) 201b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan return (tests_run, p.exit_code) 202905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 203905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 204905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wandef InvokeWithModifiedEnv(extra_env, function, *args, **kwargs): 205905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan """Runs the given function and arguments in a modified environment.""" 206905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan try: 207e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev original_env = environ.copy() 208e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev environ.update(extra_env) 209905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan return function(*args, **kwargs) 210905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan finally: 211e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev environ.clear() 212e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev environ.update(original_env) 213905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 214905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 215905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wandef RunWithSharding(total_shards, shard_index, command): 2166d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan """Runs a test program shard and returns exit code and a list of tests run.""" 2174b6829f0d28990dd645e16386eb226d0f10c8731shiqian 218905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan extra_env = {SHARD_INDEX_ENV_VAR: str(shard_index), 219905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan TOTAL_SHARDS_ENV_VAR: str(total_shards)} 220b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan return InvokeWithModifiedEnv(extra_env, RunAndExtractTestList, command) 2214b6829f0d28990dd645e16386eb226d0f10c8731shiqian 2224b6829f0d28990dd645e16386eb226d0f10c8731shiqian# The unit test. 2234b6829f0d28990dd645e16386eb226d0f10c8731shiqian 2244b6829f0d28990dd645e16386eb226d0f10c8731shiqian 2256d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wanclass GTestFilterUnitTest(gtest_test_utils.TestCase): 226a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev """Tests the env variable or the command line flag to filter tests.""" 2274b6829f0d28990dd645e16386eb226d0f10c8731shiqian 2284b6829f0d28990dd645e16386eb226d0f10c8731shiqian # Utilities. 2294b6829f0d28990dd645e16386eb226d0f10c8731shiqian 2304b6829f0d28990dd645e16386eb226d0f10c8731shiqian def AssertSetEqual(self, lhs, rhs): 2314b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Asserts that two sets are equal.""" 2324b6829f0d28990dd645e16386eb226d0f10c8731shiqian 2334b6829f0d28990dd645e16386eb226d0f10c8731shiqian for elem in lhs: 2344b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.assert_(elem in rhs, '%s in %s' % (elem, rhs)) 2354b6829f0d28990dd645e16386eb226d0f10c8731shiqian 2364b6829f0d28990dd645e16386eb226d0f10c8731shiqian for elem in rhs: 2374b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.assert_(elem in lhs, '%s in %s' % (elem, lhs)) 2384b6829f0d28990dd645e16386eb226d0f10c8731shiqian 239905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan def AssertPartitionIsValid(self, set_var, list_of_sets): 240905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan """Asserts that list_of_sets is a valid partition of set_var.""" 241905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 242905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan full_partition = [] 243905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan for slice_var in list_of_sets: 244905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan full_partition.extend(slice_var) 245905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.assertEqual(len(set_var), len(full_partition)) 246f61494217ff1ae294447ebc30679f5d07fba96e1zhanyong.wan self.assertEqual(sets.Set(set_var), sets.Set(full_partition)) 247905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 248767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan def AdjustForParameterizedTests(self, tests_to_run): 2496d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan """Adjust tests_to_run in case value parameterized tests are disabled.""" 2506d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 251767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan global param_tests_present 252767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan if not param_tests_present: 253767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan return list(sets.Set(tests_to_run) - sets.Set(PARAM_TESTS)) 254767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan else: 255767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan return tests_to_run 256767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan 2574b6829f0d28990dd645e16386eb226d0f10c8731shiqian def RunAndVerify(self, gtest_filter, tests_to_run): 258a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev """Checks that the binary runs correct set of tests for a given filter.""" 2596d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 260767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan tests_to_run = self.AdjustForParameterizedTests(tests_to_run) 2614b6829f0d28990dd645e16386eb226d0f10c8731shiqian 262a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # First, tests using the environment variable. 2634b6829f0d28990dd645e16386eb226d0f10c8731shiqian 264767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan # Windows removes empty variables from the environment when passing it 265a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # to a new process. This means it is impossible to pass an empty filter 266a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # into a process using the environment variable. However, we can still 267a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # test the case when the variable is not supplied (i.e., gtest_filter is 268a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # None). 269767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan # pylint: disable-msg=C6403 270e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev if CAN_TEST_EMPTY_FILTER or gtest_filter != '': 271767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan SetEnvVar(FILTER_ENV_VAR, gtest_filter) 272b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan tests_run = RunAndExtractTestList()[0] 273767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan SetEnvVar(FILTER_ENV_VAR, None) 274767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan self.AssertSetEqual(tests_run, tests_to_run) 275767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan # pylint: enable-msg=C6403 2764b6829f0d28990dd645e16386eb226d0f10c8731shiqian 277a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # Next, tests using the command line flag. 2784b6829f0d28990dd645e16386eb226d0f10c8731shiqian 2794b6829f0d28990dd645e16386eb226d0f10c8731shiqian if gtest_filter is None: 280b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan args = [] 2814b6829f0d28990dd645e16386eb226d0f10c8731shiqian else: 282b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan args = ['--%s=%s' % (FILTER_FLAG, gtest_filter)] 2834b6829f0d28990dd645e16386eb226d0f10c8731shiqian 284b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan tests_run = RunAndExtractTestList(args)[0] 2854b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.AssertSetEqual(tests_run, tests_to_run) 2864b6829f0d28990dd645e16386eb226d0f10c8731shiqian 287905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan def RunAndVerifyWithSharding(self, gtest_filter, total_shards, tests_to_run, 288b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan args=None, check_exit_0=False): 2896d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan """Checks that binary runs correct tests for the given filter and shard. 2906d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 2916d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan Runs all shards of gtest_filter_unittest_ with the given filter, and 292905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan verifies that the right set of tests were run. The union of tests run 293905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan on each shard should be identical to tests_to_run, without duplicates. 2946d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 2956d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan Args: 2966d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan gtest_filter: A filter to apply to the tests. 2976d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan total_shards: A total number of shards to split test run into. 2986d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan tests_to_run: A set of tests expected to run. 299b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan args : Arguments to pass to the to the test binary. 3006d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan check_exit_0: When set to a true value, make sure that all shards 3016d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan return 0. 302905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan """ 3036d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 304767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan tests_to_run = self.AdjustForParameterizedTests(tests_to_run) 305767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan 306767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan # Windows removes empty variables from the environment when passing it 307a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # to a new process. This means it is impossible to pass an empty filter 308a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # into a process using the environment variable. However, we can still 309a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # test the case when the variable is not supplied (i.e., gtest_filter is 310a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # None). 311767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan # pylint: disable-msg=C6403 312e7dfd7ed918c9714f096c873544ed11e91ff8106vladlosev if CAN_TEST_EMPTY_FILTER or gtest_filter != '': 313767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan SetEnvVar(FILTER_ENV_VAR, gtest_filter) 314767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan partition = [] 315767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan for i in range(0, total_shards): 316b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan (tests_run, exit_code) = RunWithSharding(total_shards, i, args) 317767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan if check_exit_0: 318b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan self.assertEqual(0, exit_code) 319767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan partition.append(tests_run) 320767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan 321767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan self.AssertPartitionIsValid(tests_to_run, partition) 322767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan SetEnvVar(FILTER_ENV_VAR, None) 323767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan # pylint: enable-msg=C6403 324905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 325ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian def RunAndVerifyAllowingDisabled(self, gtest_filter, tests_to_run): 3266d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan """Checks that the binary runs correct set of tests for the given filter. 3276d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 3286d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan Runs gtest_filter_unittest_ with the given filter, and enables 329ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian disabled tests. Verifies that the right set of tests were run. 3306d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 3316d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan Args: 3326d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan gtest_filter: A filter to apply to the tests. 3336d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan tests_to_run: A set of tests expected to run. 334ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian """ 3356d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 336767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan tests_to_run = self.AdjustForParameterizedTests(tests_to_run) 337767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan 338ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian # Construct the command line. 339b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan args = ['--%s' % ALSO_RUN_DISABED_TESTS_FLAG] 340ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian if gtest_filter is not None: 341b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan args.append('--%s=%s' % (FILTER_FLAG, gtest_filter)) 342ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 343b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan tests_run = RunAndExtractTestList(args)[0] 344ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.AssertSetEqual(tests_run, tests_to_run) 345ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 346f904a612d9444ab36c07a8e619c113432e046f49vladlosev def setUp(self): 347767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan """Sets up test case. 348767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan 349767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan Determines whether value-parameterized tests are enabled in the binary and 350767998dfadaf87c1193d51cf5847a7f147c6e1f7zhanyong.wan sets the flags accordingly. 351f904a612d9444ab36c07a8e619c113432e046f49vladlosev """ 3526d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 353f904a612d9444ab36c07a8e619c113432e046f49vladlosev global param_tests_present 354f904a612d9444ab36c07a8e619c113432e046f49vladlosev if param_tests_present is None: 355f904a612d9444ab36c07a8e619c113432e046f49vladlosev param_tests_present = PARAM_TEST_REGEX.search( 356b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan RunAndReturnOutput()) is not None 357f904a612d9444ab36c07a8e619c113432e046f49vladlosev 3584b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testDefaultBehavior(self): 3594b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests the behavior of not specifying the filter.""" 3604b6829f0d28990dd645e16386eb226d0f10c8731shiqian 361ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerify(None, ACTIVE_TESTS) 3624b6829f0d28990dd645e16386eb226d0f10c8731shiqian 363905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan def testDefaultBehaviorWithShards(self): 3646d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan """Tests the behavior without the filter, with sharding enabled.""" 3656d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 366905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding(None, 1, ACTIVE_TESTS) 367905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding(None, 2, ACTIVE_TESTS) 368905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding(None, len(ACTIVE_TESTS) - 1, ACTIVE_TESTS) 369905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding(None, len(ACTIVE_TESTS), ACTIVE_TESTS) 370905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding(None, len(ACTIVE_TESTS) + 1, ACTIVE_TESTS) 371905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 3724b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testEmptyFilter(self): 3734b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests an empty filter.""" 3744b6829f0d28990dd645e16386eb226d0f10c8731shiqian 3754b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('', []) 376905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding('', 1, []) 377905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding('', 2, []) 3784b6829f0d28990dd645e16386eb226d0f10c8731shiqian 3794b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testBadFilter(self): 3804b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests a filter that matches nothing.""" 3814b6829f0d28990dd645e16386eb226d0f10c8731shiqian 3824b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('BadFilter', []) 383ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('BadFilter', []) 3844b6829f0d28990dd645e16386eb226d0f10c8731shiqian 3854b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testFullName(self): 3864b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests filtering by full name.""" 3874b6829f0d28990dd645e16386eb226d0f10c8731shiqian 3884b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('FooTest.Xyz', ['FooTest.Xyz']) 389ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('FooTest.Xyz', ['FooTest.Xyz']) 390905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding('FooTest.Xyz', 5, ['FooTest.Xyz']) 3914b6829f0d28990dd645e16386eb226d0f10c8731shiqian 3924b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testUniversalFilters(self): 3934b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests filters that match everything.""" 3944b6829f0d28990dd645e16386eb226d0f10c8731shiqian 395ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerify('*', ACTIVE_TESTS) 396ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerify('*.*', ACTIVE_TESTS) 397905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.RunAndVerifyWithSharding('*.*', len(ACTIVE_TESTS) - 3, ACTIVE_TESTS) 398ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('*', ACTIVE_TESTS + DISABLED_TESTS) 399ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('*.*', ACTIVE_TESTS + DISABLED_TESTS) 4004b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4014b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testFilterByTestCase(self): 4024b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests filtering by test case name.""" 4034b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4044b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('FooTest.*', ['FooTest.Abc', 'FooTest.Xyz']) 4054b6829f0d28990dd645e16386eb226d0f10c8731shiqian 406ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian BAZ_TESTS = ['BazTest.TestOne', 'BazTest.TestA', 'BazTest.TestB'] 407ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerify('BazTest.*', BAZ_TESTS) 408ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('BazTest.*', 409ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian BAZ_TESTS + ['BazTest.DISABLED_TestC']) 410ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 4114b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testFilterByTest(self): 4124b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests filtering by test name.""" 4134b6829f0d28990dd645e16386eb226d0f10c8731shiqian 414f904a612d9444ab36c07a8e619c113432e046f49vladlosev self.RunAndVerify('*.TestOne', ['BarTest.TestOne', 'BazTest.TestOne']) 4154b6829f0d28990dd645e16386eb226d0f10c8731shiqian 416ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian def testFilterDisabledTests(self): 417ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian """Select only the disabled tests to run.""" 418ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 419ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerify('DISABLED_FoobarTest.Test1', []) 420ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('DISABLED_FoobarTest.Test1', 421ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian ['DISABLED_FoobarTest.Test1']) 422ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 423ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerify('*DISABLED_*', []) 424ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('*DISABLED_*', DISABLED_TESTS) 425ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 426ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerify('*.DISABLED_*', []) 427ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('*.DISABLED_*', [ 428ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'BarTest.DISABLED_TestFour', 429ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'BarTest.DISABLED_TestFive', 430ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'BazTest.DISABLED_TestC', 431ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'DISABLED_FoobarTest.DISABLED_Test2', 432ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian ]) 433ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 434ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerify('DISABLED_*', []) 435ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian self.RunAndVerifyAllowingDisabled('DISABLED_*', [ 436ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'DISABLED_FoobarTest.Test1', 437ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'DISABLED_FoobarTest.DISABLED_Test2', 438ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 'DISABLED_FoobarbazTest.TestA', 439ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian ]) 440ca6949fd2a0a317fd7c7c48aa86670aac50a48c6shiqian 4414b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testWildcardInTestCaseName(self): 4424b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests using wildcard in the test case name.""" 4434b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4444b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('*a*.*', [ 445f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestOne', 446f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestTwo', 447f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestThree', 4484b6829f0d28990dd645e16386eb226d0f10c8731shiqian 449f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BazTest.TestOne', 4504b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestA', 451a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'BazTest.TestB', ] + DEATH_TESTS + PARAM_TESTS) 4524b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4534b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testWildcardInTestName(self): 4544b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests using wildcard in the test name.""" 4554b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4564b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('*.*A*', ['FooTest.Abc', 'BazTest.TestA']) 4574b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4584b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testFilterWithoutDot(self): 4594b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests a filter that has no '.' in it.""" 4604b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4614b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('*z*', [ 4624b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Xyz', 4634b6829f0d28990dd645e16386eb226d0f10c8731shiqian 464f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BazTest.TestOne', 4654b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestA', 4664b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestB', 4674b6829f0d28990dd645e16386eb226d0f10c8731shiqian ]) 4684b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4694b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testTwoPatterns(self): 4704b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests filters that consist of two patterns.""" 4714b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4724b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('Foo*.*:*A*', [ 4734b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Abc', 4744b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Xyz', 4754b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4764b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestA', 4774b6829f0d28990dd645e16386eb226d0f10c8731shiqian ]) 4784b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4794b6829f0d28990dd645e16386eb226d0f10c8731shiqian # An empty pattern + a non-empty one 4804b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify(':*A*', ['FooTest.Abc', 'BazTest.TestA']) 4814b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4824b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testThreePatterns(self): 4834b6829f0d28990dd645e16386eb226d0f10c8731shiqian """Tests filters that consist of three patterns.""" 4844b6829f0d28990dd645e16386eb226d0f10c8731shiqian 485f904a612d9444ab36c07a8e619c113432e046f49vladlosev self.RunAndVerify('*oo*:*A*:*One', [ 4864b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Abc', 4874b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Xyz', 4884b6829f0d28990dd645e16386eb226d0f10c8731shiqian 489f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestOne', 4904b6829f0d28990dd645e16386eb226d0f10c8731shiqian 491f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BazTest.TestOne', 4924b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestA', 4934b6829f0d28990dd645e16386eb226d0f10c8731shiqian ]) 4944b6829f0d28990dd645e16386eb226d0f10c8731shiqian 4954b6829f0d28990dd645e16386eb226d0f10c8731shiqian # The 2nd pattern is empty. 496f904a612d9444ab36c07a8e619c113432e046f49vladlosev self.RunAndVerify('*oo*::*One', [ 4974b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Abc', 4984b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Xyz', 4994b6829f0d28990dd645e16386eb226d0f10c8731shiqian 500f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestOne', 5014b6829f0d28990dd645e16386eb226d0f10c8731shiqian 502f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BazTest.TestOne', 5034b6829f0d28990dd645e16386eb226d0f10c8731shiqian ]) 5044b6829f0d28990dd645e16386eb226d0f10c8731shiqian 5054b6829f0d28990dd645e16386eb226d0f10c8731shiqian # The last 2 patterns are empty. 5064b6829f0d28990dd645e16386eb226d0f10c8731shiqian self.RunAndVerify('*oo*::', [ 5074b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Abc', 5084b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Xyz', 5094b6829f0d28990dd645e16386eb226d0f10c8731shiqian ]) 5104b6829f0d28990dd645e16386eb226d0f10c8731shiqian 5114b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testNegativeFilters(self): 512a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev self.RunAndVerify('*-BazTest.TestOne', [ 513905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 'FooTest.Abc', 5144b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Xyz', 5154b6829f0d28990dd645e16386eb226d0f10c8731shiqian 516f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestOne', 517f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestTwo', 518f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestThree', 5194b6829f0d28990dd645e16386eb226d0f10c8731shiqian 5204b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestA', 5214b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'BazTest.TestB', 522a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev ] + DEATH_TESTS + PARAM_TESTS) 523905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 524a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev self.RunAndVerify('*-FooTest.Abc:BazTest.*', [ 5254b6829f0d28990dd645e16386eb226d0f10c8731shiqian 'FooTest.Xyz', 5264b6829f0d28990dd645e16386eb226d0f10c8731shiqian 527f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestOne', 528f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestTwo', 529f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestThree', 530a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev ] + DEATH_TESTS + PARAM_TESTS) 5314b6829f0d28990dd645e16386eb226d0f10c8731shiqian 532f904a612d9444ab36c07a8e619c113432e046f49vladlosev self.RunAndVerify('BarTest.*-BarTest.TestOne', [ 533f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestTwo', 534f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestThree', 5354b6829f0d28990dd645e16386eb226d0f10c8731shiqian ]) 5364b6829f0d28990dd645e16386eb226d0f10c8731shiqian 5374b6829f0d28990dd645e16386eb226d0f10c8731shiqian # Tests without leading '*'. 538a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev self.RunAndVerify('-FooTest.Abc:FooTest.Xyz:BazTest.*', [ 539f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestOne', 540f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestTwo', 541f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'BarTest.TestThree', 542a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev ] + DEATH_TESTS + PARAM_TESTS) 543f904a612d9444ab36c07a8e619c113432e046f49vladlosev 544f904a612d9444ab36c07a8e619c113432e046f49vladlosev # Value parameterized tests. 545f904a612d9444ab36c07a8e619c113432e046f49vladlosev self.RunAndVerify('*/*', PARAM_TESTS) 546f904a612d9444ab36c07a8e619c113432e046f49vladlosev 547f904a612d9444ab36c07a8e619c113432e046f49vladlosev # Value parameterized tests filtering by the sequence name. 548f904a612d9444ab36c07a8e619c113432e046f49vladlosev self.RunAndVerify('SeqP/*', [ 549f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestX/0', 550f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestX/1', 551f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestY/0', 552f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestY/1', 553f904a612d9444ab36c07a8e619c113432e046f49vladlosev ]) 554f904a612d9444ab36c07a8e619c113432e046f49vladlosev 555f904a612d9444ab36c07a8e619c113432e046f49vladlosev # Value parameterized tests filtering by the test name. 556f904a612d9444ab36c07a8e619c113432e046f49vladlosev self.RunAndVerify('*/0', [ 557f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestX/0', 558f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqP/ParamTest.TestY/0', 559f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqQ/ParamTest.TestX/0', 560f904a612d9444ab36c07a8e619c113432e046f49vladlosev 'SeqQ/ParamTest.TestY/0', 5614b6829f0d28990dd645e16386eb226d0f10c8731shiqian ]) 5624b6829f0d28990dd645e16386eb226d0f10c8731shiqian 5634b6829f0d28990dd645e16386eb226d0f10c8731shiqian def testFlagOverridesEnvVar(self): 5646d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan """Tests that the filter flag overrides the filtering env. variable.""" 5654b6829f0d28990dd645e16386eb226d0f10c8731shiqian 5664b6829f0d28990dd645e16386eb226d0f10c8731shiqian SetEnvVar(FILTER_ENV_VAR, 'Foo*') 567b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan args = ['--%s=%s' % (FILTER_FLAG, '*One')] 568b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan tests_run = RunAndExtractTestList(args)[0] 5694b6829f0d28990dd645e16386eb226d0f10c8731shiqian SetEnvVar(FILTER_ENV_VAR, None) 5704b6829f0d28990dd645e16386eb226d0f10c8731shiqian 571f904a612d9444ab36c07a8e619c113432e046f49vladlosev self.AssertSetEqual(tests_run, ['BarTest.TestOne', 'BazTest.TestOne']) 5724b6829f0d28990dd645e16386eb226d0f10c8731shiqian 573905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan def testShardStatusFileIsCreated(self): 574905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan """Tests that the shard file is created if specified in the environment.""" 575905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 5766d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan shard_status_file = os.path.join(gtest_test_utils.GetTempDir(), 5776d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 'shard_status_file') 578905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.assert_(not os.path.exists(shard_status_file)) 579905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 580905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan extra_env = {SHARD_STATUS_FILE_ENV_VAR: shard_status_file} 581905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan try: 582b7ec0f7b3b21338babc9a6ab5a593a40634a8062zhanyong.wan InvokeWithModifiedEnv(extra_env, RunAndReturnOutput) 583905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan finally: 584905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.assert_(os.path.exists(shard_status_file)) 585905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan os.remove(shard_status_file) 586905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 587905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan def testShardStatusFileIsCreatedWithListTests(self): 588a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev """Tests that the shard file is created with the "list_tests" flag.""" 589905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 5906d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan shard_status_file = os.path.join(gtest_test_utils.GetTempDir(), 5916d63ee6720540cadb8919037c5d41a6413cc9101zhanyong.wan 'shard_status_file2') 592905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.assert_(not os.path.exists(shard_status_file)) 593905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 594905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan extra_env = {SHARD_STATUS_FILE_ENV_VAR: shard_status_file} 595905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan try: 596a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev output = InvokeWithModifiedEnv(extra_env, 597a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev RunAndReturnOutput, 598a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev [LIST_TESTS_FLAG]) 599905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan finally: 600a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # This assertion ensures that Google Test enumerated the tests as 601a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev # opposed to running them. 602a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev self.assert_('[==========]' not in output, 603a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'Unexpected output during test enumeration.\n' 604a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'Please ensure that LIST_TESTS_FLAG is assigned the\n' 605a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'correct flag value for listing Google Test tests.') 606a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 607905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan self.assert_(os.path.exists(shard_status_file)) 608905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan os.remove(shard_status_file) 609905074c42ba4d0c5b472730b00e43a01d6a50134zhanyong.wan 610a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev if SUPPORTS_DEATH_TESTS: 611a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev def testShardingWorksWithDeathTests(self): 612a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev """Tests integration with death tests and sharding.""" 613a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 614a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev gtest_filter = 'HasDeathTest.*:SeqP/*' 615a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev expected_tests = [ 616a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'HasDeathTest.Test1', 617a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'HasDeathTest.Test2', 618a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 619a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'SeqP/ParamTest.TestX/0', 620a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'SeqP/ParamTest.TestX/1', 621a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'SeqP/ParamTest.TestY/0', 622a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 'SeqP/ParamTest.TestY/1', 623a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev ] 624a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev 625a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev for flag in ['--gtest_death_test_style=threadsafe', 626a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev '--gtest_death_test_style=fast']: 627a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev self.RunAndVerifyWithSharding(gtest_filter, 3, expected_tests, 628a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev check_exit_0=True, args=[flag]) 629a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev self.RunAndVerifyWithSharding(gtest_filter, 5, expected_tests, 630a232a5cee9c506514ba88d551aa4c0a11d2f3c4bvladlosev check_exit_0=True, args=[flag]) 6314b6829f0d28990dd645e16386eb226d0f10c8731shiqian 6324b6829f0d28990dd645e16386eb226d0f10c8731shiqianif __name__ == '__main__': 6334b6829f0d28990dd645e16386eb226d0f10c8731shiqian gtest_test_utils.Main() 634