11be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania#!/usr/bin/env python 21be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# 31be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Copyright 2005, Google Inc. 41be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# All rights reserved. 51be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# 61be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Redistribution and use in source and binary forms, with or without 71be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# modification, are permitted provided that the following conditions are 81be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# met: 91be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# 101be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# * Redistributions of source code must retain the above copyright 111be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# notice, this list of conditions and the following disclaimer. 121be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# * Redistributions in binary form must reproduce the above 131be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# copyright notice, this list of conditions and the following disclaimer 141be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# in the documentation and/or other materials provided with the 151be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# distribution. 161be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# * Neither the name of Google Inc. nor the names of its 171be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# contributors may be used to endorse or promote products derived from 181be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# this software without specific prior written permission. 191be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# 201be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 211be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 221be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 231be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 241be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 251be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 261be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 271be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 281be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 291be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 301be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 311be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 321be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania"""Unit test for Google Test test filters. 331be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 341be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaA user can specify which test(s) in a Google Test program to run via either 351be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniathe GTEST_FILTER environment variable or the --gtest_filter flag. 361be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaThis script tests such functionality by invoking 371be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniagtest_filter_unittest_ (a program written with Google Test) with different 381be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaenvironments and command line flags. 391be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 401be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaNote that test sharding may also influence which tests are filtered. Therefore, 411be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniawe test that here also. 421be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania""" 431be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 441be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania__author__ = 'wan@google.com (Zhanyong Wan)' 451be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 461be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaimport os 471be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaimport re 481be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaimport sets 491be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaimport tempfile 501be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaimport unittest 511be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaimport gtest_test_utils 521be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 531be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Constants. 541be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 551be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# The environment variable for specifying the test filters. 561be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaFILTER_ENV_VAR = 'GTEST_FILTER' 571be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 581be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# The environment variables for test sharding. 591be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaTOTAL_SHARDS_ENV_VAR = 'GTEST_TOTAL_SHARDS' 601be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaSHARD_INDEX_ENV_VAR = 'GTEST_SHARD_INDEX' 611be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaSHARD_STATUS_FILE_ENV_VAR = 'GTEST_SHARD_STATUS_FILE' 621be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 631be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# The command line flag for specifying the test filters. 641be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaFILTER_FLAG = 'gtest_filter' 651be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 661be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# The command line flag for including disabled tests. 671be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaALSO_RUN_DISABED_TESTS_FLAG = 'gtest_also_run_disabled_tests' 681be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 691be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Command to run the gtest_filter_unittest_ program. 701be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaCOMMAND = os.path.join(gtest_test_utils.GetBuildDir(), 711be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'gtest_filter_unittest_') 721be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 731be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Regex for determining whether parameterized tests are enabled in the binary. 741be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaPARAM_TEST_REGEX = re.compile(r'/ParamTest') 751be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 761be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Regex for parsing test case names from Google Test's output. 771be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaTEST_CASE_REGEX = re.compile(r'^\[\-+\] \d+ tests? from (\w+(/\w+)?)') 781be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 791be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Regex for parsing test names from Google Test's output. 801be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaTEST_REGEX = re.compile(r'^\[\s*RUN\s*\].*\.(\w+(/\w+)?)') 811be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 821be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Full names of all tests in gtest_filter_unittests_. 831be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaPARAM_TESTS = [ 841be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestX/0', 851be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestX/1', 861be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestY/0', 871be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestY/1', 881be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqQ/ParamTest.TestX/0', 891be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqQ/ParamTest.TestX/1', 901be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqQ/ParamTest.TestY/0', 911be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqQ/ParamTest.TestY/1', 921be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ] 931be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 941be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaDISABLED_TESTS = [ 951be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.DISABLED_TestFour', 961be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.DISABLED_TestFive', 971be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.DISABLED_TestC', 981be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'DISABLED_FoobarTest.Test1', 991be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'DISABLED_FoobarTest.DISABLED_Test2', 1001be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'DISABLED_FoobarbazTest.TestA', 1011be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ] 1021be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1031be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# All the non-disabled tests. 1041be2c9def7187e4e643c00a31dd9986395795d7dNicolas CataniaACTIVE_TESTS = [ 1051be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Abc', 1061be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Xyz', 1071be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1081be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestOne', 1091be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestTwo', 1101be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestThree', 1111be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1121be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestOne', 1131be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestA', 1141be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestB', 1151be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1161be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'HasDeathTest.Test1', 1171be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'HasDeathTest.Test2', 1181be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ] + PARAM_TESTS 1191be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1201be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaparam_tests_present = None 1211be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1221be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# Utilities. 1231be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1241be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1251be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniadef SetEnvVar(env_var, value): 1261be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Sets the env variable to 'value'; unsets it when 'value' is None.""" 1271be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1281be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if value is not None: 1291be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania os.environ[env_var] = value 1301be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania elif env_var in os.environ: 1311be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania del os.environ[env_var] 1321be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1331be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1341be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniadef Run(command): 1351be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Runs a Google Test program and returns a list of full names of the 1361be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests that were run along with the test exit code. 1371be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 1381be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1391be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania stdout_file = os.popen(command, 'r') 1401be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests_run = [] 1411be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania test_case = '' 1421be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania test = '' 1431be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania for line in stdout_file: 1441be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania match = TEST_CASE_REGEX.match(line) 1451be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if match is not None: 1461be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania test_case = match.group(1) 1471be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania else: 1481be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania match = TEST_REGEX.match(line) 1491be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if match is not None: 1501be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania test = match.group(1) 1511be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests_run += [test_case + '.' + test] 1521be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania exit_code = stdout_file.close() 1531be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania return (tests_run, exit_code) 1541be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1551be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1561be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniadef InvokeWithModifiedEnv(extra_env, function, *args, **kwargs): 1571be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Runs the given function and arguments in a modified environment.""" 1581be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania try: 1591be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania original_env = os.environ.copy() 1601be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania os.environ.update(extra_env) 1611be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania return function(*args, **kwargs) 1621be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania finally: 1631be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania for key in extra_env.iterkeys(): 1641be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if key in original_env: 1651be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania os.environ[key] = original_env[key] 1661be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania else: 1671be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania del os.environ[key] 1681be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1691be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1701be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniadef RunWithSharding(total_shards, shard_index, command): 1711be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Runs the Google Test program shard and returns a list of full names of the 1721be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests that were run along with the exit code. 1731be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 1741be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1751be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania extra_env = {SHARD_INDEX_ENV_VAR: str(shard_index), 1761be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania TOTAL_SHARDS_ENV_VAR: str(total_shards)} 1771be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania return InvokeWithModifiedEnv(extra_env, Run, command) 1781be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1791be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania# The unit test. 1801be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1811be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1821be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaclass GTestFilterUnitTest(unittest.TestCase): 1831be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests using the GTEST_FILTER environment variable or the 1841be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania --gtest_filter flag to filter tests. 1851be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 1861be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1871be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # Utilities. 1881be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1891be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def AssertSetEqual(self, lhs, rhs): 1901be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Asserts that two sets are equal.""" 1911be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1921be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania for elem in lhs: 1931be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assert_(elem in rhs, '%s in %s' % (elem, rhs)) 1941be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1951be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania for elem in rhs: 1961be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assert_(elem in lhs, '%s in %s' % (elem, lhs)) 1971be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 1981be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def AssertPartitionIsValid(self, set_var, list_of_sets): 1991be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Asserts that list_of_sets is a valid partition of set_var.""" 2001be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2011be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania full_partition = [] 2021be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania for slice_var in list_of_sets: 2031be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania full_partition.extend(slice_var) 2041be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assertEqual(len(set_var), len(full_partition)) 2051be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assertEqual(sets.Set(set_var), sets.Set(full_partition)) 2061be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2071be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def RunAndVerify(self, gtest_filter, tests_to_run): 2081be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Runs gtest_flag_unittest_ with the given filter, and verifies 2091be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania that the right set of tests were run. 2101be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 2111be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # Adjust tests_to_run in case value parameterized tests are disabled 2121be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # in the binary. 2131be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania global param_tests_present 2141be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if not param_tests_present: 2151be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests_to_run = list(sets.Set(tests_to_run) - sets.Set(PARAM_TESTS)) 2161be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2171be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # First, tests using GTEST_FILTER. 2181be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2191be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania SetEnvVar(FILTER_ENV_VAR, gtest_filter) 2201be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests_run = Run(COMMAND)[0] 2211be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania SetEnvVar(FILTER_ENV_VAR, None) 2221be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2231be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.AssertSetEqual(tests_run, tests_to_run) 2241be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2251be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # Next, tests using --gtest_filter. 2261be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2271be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if gtest_filter is None: 2281be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania command = COMMAND 2291be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania else: 2301be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania command = '%s --%s=%s' % (COMMAND, FILTER_FLAG, gtest_filter) 2311be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2321be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests_run = Run(command)[0] 2331be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.AssertSetEqual(tests_run, tests_to_run) 2341be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2351be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def RunAndVerifyWithSharding(self, gtest_filter, total_shards, tests_to_run, 2361be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania command=COMMAND, check_exit_0=False): 2371be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Runs all shards of gtest_flag_unittest_ with the given filter, and 2381be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania verifies that the right set of tests were run. The union of tests run 2391be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania on each shard should be identical to tests_to_run, without duplicates. 2401be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania If check_exit_0, make sure that all shards returned 0. 2411be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 2421be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania SetEnvVar(FILTER_ENV_VAR, gtest_filter) 2431be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania partition = [] 2441be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania for i in range(0, total_shards): 2451be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania (tests_run, exit_code) = RunWithSharding(total_shards, i, command) 2461be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if check_exit_0: 2471be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assert_(exit_code is None) 2481be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania partition.append(tests_run) 2491be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2501be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.AssertPartitionIsValid(tests_to_run, partition) 2511be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania SetEnvVar(FILTER_ENV_VAR, None) 2521be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2531be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def RunAndVerifyAllowingDisabled(self, gtest_filter, tests_to_run): 2541be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Runs gtest_flag_unittest_ with the given filter, and enables 2551be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania disabled tests. Verifies that the right set of tests were run. 2561be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 2571be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # Construct the command line. 2581be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania command = '%s --%s' % (COMMAND, ALSO_RUN_DISABED_TESTS_FLAG) 2591be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if gtest_filter is not None: 2601be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania command = '%s --%s=%s' % (command, FILTER_FLAG, gtest_filter) 2611be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2621be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests_run = Run(command)[0] 2631be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.AssertSetEqual(tests_run, tests_to_run) 2641be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2651be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def setUp(self): 2661be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Sets up test case. Determines whether value-parameterized tests are 2671be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania enabled in the binary and sets flags accordingly. 2681be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 2691be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania global param_tests_present 2701be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania if param_tests_present is None: 2711be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania param_tests_present = PARAM_TEST_REGEX.search( 2721be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania '\n'.join(os.popen(COMMAND, 'r').readlines())) is not None 2731be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2741be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testDefaultBehavior(self): 2751be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests the behavior of not specifying the filter.""" 2761be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2771be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify(None, ACTIVE_TESTS) 2781be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2791be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testDefaultBehaviorWithShards(self): 2801be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests the behavior of not specifying the filter, with sharding 2811be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania enabled. 2821be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 2831be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding(None, 1, ACTIVE_TESTS) 2841be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding(None, 2, ACTIVE_TESTS) 2851be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding(None, len(ACTIVE_TESTS) - 1, ACTIVE_TESTS) 2861be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding(None, len(ACTIVE_TESTS), ACTIVE_TESTS) 2871be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding(None, len(ACTIVE_TESTS) + 1, ACTIVE_TESTS) 2881be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2891be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testEmptyFilter(self): 2901be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests an empty filter.""" 2911be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2921be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('', []) 2931be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding('', 1, []) 2941be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding('', 2, []) 2951be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2961be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testBadFilter(self): 2971be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests a filter that matches nothing.""" 2981be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 2991be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('BadFilter', []) 3001be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('BadFilter', []) 3011be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3021be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testFullName(self): 3031be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests filtering by full name.""" 3041be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3051be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('FooTest.Xyz', ['FooTest.Xyz']) 3061be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('FooTest.Xyz', ['FooTest.Xyz']) 3071be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding('FooTest.Xyz', 5, ['FooTest.Xyz']) 3081be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3091be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testUniversalFilters(self): 3101be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests filters that match everything.""" 3111be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3121be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*', ACTIVE_TESTS) 3131be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*.*', ACTIVE_TESTS) 3141be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding('*.*', len(ACTIVE_TESTS) - 3, ACTIVE_TESTS) 3151be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('*', ACTIVE_TESTS + DISABLED_TESTS) 3161be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('*.*', ACTIVE_TESTS + DISABLED_TESTS) 3171be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3181be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testFilterByTestCase(self): 3191be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests filtering by test case name.""" 3201be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3211be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('FooTest.*', ['FooTest.Abc', 'FooTest.Xyz']) 3221be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3231be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania BAZ_TESTS = ['BazTest.TestOne', 'BazTest.TestA', 'BazTest.TestB'] 3241be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('BazTest.*', BAZ_TESTS) 3251be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('BazTest.*', 3261be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania BAZ_TESTS + ['BazTest.DISABLED_TestC']) 3271be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3281be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testFilterByTest(self): 3291be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests filtering by test name.""" 3301be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3311be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*.TestOne', ['BarTest.TestOne', 'BazTest.TestOne']) 3321be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3331be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testFilterDisabledTests(self): 3341be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Select only the disabled tests to run.""" 3351be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3361be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('DISABLED_FoobarTest.Test1', []) 3371be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('DISABLED_FoobarTest.Test1', 3381be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ['DISABLED_FoobarTest.Test1']) 3391be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3401be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*DISABLED_*', []) 3411be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('*DISABLED_*', DISABLED_TESTS) 3421be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3431be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*.DISABLED_*', []) 3441be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('*.DISABLED_*', [ 3451be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.DISABLED_TestFour', 3461be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.DISABLED_TestFive', 3471be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.DISABLED_TestC', 3481be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'DISABLED_FoobarTest.DISABLED_Test2', 3491be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 3501be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3511be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('DISABLED_*', []) 3521be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyAllowingDisabled('DISABLED_*', [ 3531be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'DISABLED_FoobarTest.Test1', 3541be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'DISABLED_FoobarTest.DISABLED_Test2', 3551be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'DISABLED_FoobarbazTest.TestA', 3561be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 3571be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3581be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testWildcardInTestCaseName(self): 3591be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests using wildcard in the test case name.""" 3601be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3611be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*a*.*', [ 3621be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestOne', 3631be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestTwo', 3641be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestThree', 3651be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3661be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestOne', 3671be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestA', 3681be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestB', 3691be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3701be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'HasDeathTest.Test1', 3711be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'HasDeathTest.Test2', ] + PARAM_TESTS) 3721be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3731be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testWildcardInTestName(self): 3741be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests using wildcard in the test name.""" 3751be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3761be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*.*A*', ['FooTest.Abc', 'BazTest.TestA']) 3771be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3781be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testFilterWithoutDot(self): 3791be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests a filter that has no '.' in it.""" 3801be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3811be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*z*', [ 3821be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Xyz', 3831be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3841be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestOne', 3851be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestA', 3861be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestB', 3871be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 3881be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3891be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testTwoPatterns(self): 3901be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests filters that consist of two patterns.""" 3911be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3921be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('Foo*.*:*A*', [ 3931be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Abc', 3941be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Xyz', 3951be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3961be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestA', 3971be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 3981be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 3991be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # An empty pattern + a non-empty one 4001be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify(':*A*', ['FooTest.Abc', 'BazTest.TestA']) 4011be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4021be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testThreePatterns(self): 4031be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests filters that consist of three patterns.""" 4041be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4051be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*oo*:*A*:*One', [ 4061be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Abc', 4071be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Xyz', 4081be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4091be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestOne', 4101be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4111be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestOne', 4121be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestA', 4131be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 4141be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4151be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # The 2nd pattern is empty. 4161be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*oo*::*One', [ 4171be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Abc', 4181be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Xyz', 4191be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4201be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestOne', 4211be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4221be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestOne', 4231be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 4241be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4251be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # The last 2 patterns are empty. 4261be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*oo*::', [ 4271be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Abc', 4281be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Xyz', 4291be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 4301be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4311be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testNegativeFilters(self): 4321be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*-HasDeathTest.Test1', [ 4331be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Abc', 4341be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Xyz', 4351be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4361be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestOne', 4371be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestTwo', 4381be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestThree', 4391be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4401be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestOne', 4411be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestA', 4421be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestB', 4431be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4441be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'HasDeathTest.Test2', 4451be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ] + PARAM_TESTS) 4461be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4471be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*-FooTest.Abc:HasDeathTest.*', [ 4481be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'FooTest.Xyz', 4491be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4501be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestOne', 4511be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestTwo', 4521be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestThree', 4531be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4541be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestOne', 4551be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestA', 4561be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestB', 4571be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ] + PARAM_TESTS) 4581be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4591be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('BarTest.*-BarTest.TestOne', [ 4601be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestTwo', 4611be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestThree', 4621be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 4631be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4641be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # Tests without leading '*'. 4651be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('-FooTest.Abc:FooTest.Xyz:HasDeathTest.*', [ 4661be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestOne', 4671be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestTwo', 4681be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BarTest.TestThree', 4691be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4701be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestOne', 4711be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestA', 4721be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'BazTest.TestB', 4731be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ] + PARAM_TESTS) 4741be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4751be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # Value parameterized tests. 4761be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*/*', PARAM_TESTS) 4771be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4781be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # Value parameterized tests filtering by the sequence name. 4791be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('SeqP/*', [ 4801be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestX/0', 4811be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestX/1', 4821be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestY/0', 4831be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestY/1', 4841be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 4851be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4861be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania # Value parameterized tests filtering by the test name. 4871be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerify('*/0', [ 4881be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestX/0', 4891be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestY/0', 4901be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqQ/ParamTest.TestX/0', 4911be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqQ/ParamTest.TestY/0', 4921be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ]) 4931be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4941be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testFlagOverridesEnvVar(self): 4951be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests that the --gtest_filter flag overrides the GTEST_FILTER 4961be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania environment variable. 4971be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """ 4981be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 4991be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania SetEnvVar(FILTER_ENV_VAR, 'Foo*') 5001be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania command = '%s --%s=%s' % (COMMAND, FILTER_FLAG, '*One') 5011be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania tests_run = Run(command)[0] 5021be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania SetEnvVar(FILTER_ENV_VAR, None) 5031be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5041be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.AssertSetEqual(tests_run, ['BarTest.TestOne', 'BazTest.TestOne']) 5051be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5061be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testShardStatusFileIsCreated(self): 5071be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests that the shard file is created if specified in the environment.""" 5081be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5091be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania test_tmpdir = tempfile.mkdtemp() 5101be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania shard_status_file = os.path.join(test_tmpdir, 'shard_status_file') 5111be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assert_(not os.path.exists(shard_status_file)) 5121be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5131be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania extra_env = {SHARD_STATUS_FILE_ENV_VAR: shard_status_file} 5141be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania stdout_file = InvokeWithModifiedEnv(extra_env, os.popen, COMMAND, 'r') 5151be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania try: 5161be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania stdout_file.readlines() 5171be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania finally: 5181be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania stdout_file.close() 5191be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assert_(os.path.exists(shard_status_file)) 5201be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania os.remove(shard_status_file) 5211be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania os.removedirs(test_tmpdir) 5221be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5231be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testShardStatusFileIsCreatedWithListTests(self): 5241be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests that the shard file is created with --gtest_list_tests.""" 5251be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5261be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania test_tmpdir = tempfile.mkdtemp() 5271be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania shard_status_file = os.path.join(test_tmpdir, 'shard_status_file2') 5281be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assert_(not os.path.exists(shard_status_file)) 5291be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5301be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania extra_env = {SHARD_STATUS_FILE_ENV_VAR: shard_status_file} 5311be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania stdout_file = InvokeWithModifiedEnv(extra_env, os.popen, 5321be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania '%s --gtest_list_tests' % COMMAND, 'r') 5331be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania try: 5341be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania stdout_file.readlines() 5351be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania finally: 5361be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania stdout_file.close() 5371be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.assert_(os.path.exists(shard_status_file)) 5381be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania os.remove(shard_status_file) 5391be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania os.removedirs(test_tmpdir) 5401be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5411be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania def testShardingWorksWithDeathTests(self): 5421be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania """Tests integration with death tests and sharding.""" 5431be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania gtest_filter = 'HasDeathTest.*:SeqP/*' 5441be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania expected_tests = [ 5451be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'HasDeathTest.Test1', 5461be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'HasDeathTest.Test2', 5471be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5481be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestX/0', 5491be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestX/1', 5501be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestY/0', 5511be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 'SeqP/ParamTest.TestY/1', 5521be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania ] 5531be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5541be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania for command in (COMMAND + ' --gtest_death_test_style=threadsafe', 5551be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania COMMAND + ' --gtest_death_test_style=fast'): 5561be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding(gtest_filter, 3, expected_tests, 5571be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania check_exit_0=True, command=command) 5581be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania self.RunAndVerifyWithSharding(gtest_filter, 5, expected_tests, 5591be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania check_exit_0=True, command=command) 5601be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania 5611be2c9def7187e4e643c00a31dd9986395795d7dNicolas Cataniaif __name__ == '__main__': 5621be2c9def7187e4e643c00a31dd9986395795d7dNicolas Catania gtest_test_utils.Main() 563