14967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar# -*- coding: utf-8 -*- 24967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar# The LLVM Compiler Infrastructure 34967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar# 44967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar# This file is distributed under the University of Illinois Open Source 54967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar# License. See LICENSE.TXT for details. 64967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar""" This module is responsible to capture the compiler invocation of any 74967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarbuild process. The result of that should be a compilation database. 84967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 94967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarThis implementation is using the LD_PRELOAD or DYLD_INSERT_LIBRARIES 104967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarmechanisms provided by the dynamic linker. The related library is implemented 114967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarin C language and can be found under 'libear' directory. 124967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 134967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarThe 'libear' library is capturing all child process creation and logging the 144967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarrelevant information about it into separate files in a specified directory. 154967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarThe parameter of this process is the output directory name, where the report 164967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarfiles shall be placed. This parameter is passed as an environment variable. 174967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 184967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarThe module also implements compiler wrappers to intercept the compiler calls. 194967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 204967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarThe module implements the build command execution and the post-processing of 214967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarthe output files, which will condensates into a compilation database. """ 224967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 234967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport sys 244967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport os 254967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport os.path 264967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport re 274967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport itertools 284967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport json 294967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport glob 304967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport argparse 314967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport logging 324967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarimport subprocess 334967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarfrom libear import build_libear, TemporaryDirectory 344967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarfrom libscanbuild import command_entry_point 354967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarfrom libscanbuild import duplicate_check, tempdir, initialize_logging 364967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarfrom libscanbuild.compilation import split_command 374967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainarfrom libscanbuild.shell import encode, decode 384967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 394967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar__all__ = ['capture', 'intercept_build_main', 'intercept_build_wrapper'] 404967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 414967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarGS = chr(0x1d) 424967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarRS = chr(0x1e) 434967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarUS = chr(0x1f) 444967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 454967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarCOMPILER_WRAPPER_CC = 'intercept-cc' 464967a710c84587c654b56c828382219c3937dacbPirama Arumuga NainarCOMPILER_WRAPPER_CXX = 'intercept-c++' 474967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 484967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 494967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar@command_entry_point 504967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef intercept_build_main(bin_dir): 514967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Entry point for 'intercept-build' command. """ 524967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 534967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar parser = create_parser() 544967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar args = parser.parse_args() 554967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 564967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar initialize_logging(args.verbose) 574967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('Parsed arguments: %s', args) 584967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 594967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar if not args.build: 604967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar parser.print_help() 614967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return 0 624967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 634967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return capture(args, bin_dir) 644967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 654967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 664967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef capture(args, bin_dir): 674967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ The entry point of build command interception. """ 684967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 694967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar def post_processing(commands): 704967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ To make a compilation database, it needs to filter out commands 714967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar which are not compiler calls. Needs to find the source file name 724967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar from the arguments. And do shell escaping on the command. 734967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 744967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar To support incremental builds, it is desired to read elements from 754967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar an existing compilation database from a previous run. These elements 764967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar shall be merged with the new elements. """ 774967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 784967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # create entries from the current run 794967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar current = itertools.chain.from_iterable( 804967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # creates a sequence of entry generators from an exec, 814967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar format_entry(command) for command in commands) 824967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # read entries from previous run 834967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar if 'append' in args and args.append and os.path.isfile(args.cdb): 844967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar with open(args.cdb) as handle: 854967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar previous = iter(json.load(handle)) 864967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar else: 874967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar previous = iter([]) 884967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # filter out duplicate entries from both 894967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar duplicate = duplicate_check(entry_hash) 904967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return (entry 914967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar for entry in itertools.chain(previous, current) 924967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar if os.path.exists(entry['file']) and not duplicate(entry)) 934967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 944967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar with TemporaryDirectory(prefix='intercept-', dir=tempdir()) as tmp_dir: 954967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # run the build command 964967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar environment = setup_environment(args, tmp_dir, bin_dir) 974967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('run build in environment: %s', environment) 984967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar exit_code = subprocess.call(args.build, env=environment) 994967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.info('build finished with exit code: %d', exit_code) 1004967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # read the intercepted exec calls 1014967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar exec_traces = itertools.chain.from_iterable( 1024967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar parse_exec_trace(os.path.join(tmp_dir, filename)) 1034967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar for filename in sorted(glob.iglob(os.path.join(tmp_dir, '*.cmd')))) 1044967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # do post processing only if that was requested 1054967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar if 'raw_entries' not in args or not args.raw_entries: 1064967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar entries = post_processing(exec_traces) 1074967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar else: 1084967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar entries = exec_traces 1094967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # dump the compilation database 1104967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar with open(args.cdb, 'w+') as handle: 1114967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar json.dump(list(entries), handle, sort_keys=True, indent=4) 1124967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return exit_code 1134967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1144967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1154967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef setup_environment(args, destination, bin_dir): 1164967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Sets up the environment for the build command. 1174967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1184967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar It sets the required environment variables and execute the given command. 1194967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar The exec calls will be logged by the 'libear' preloaded library or by the 1204967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'wrapper' programs. """ 1214967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1224967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar c_compiler = args.cc if 'cc' in args else 'cc' 1234967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar cxx_compiler = args.cxx if 'cxx' in args else 'c++' 1244967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1254967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar libear_path = None if args.override_compiler or is_preload_disabled( 1264967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar sys.platform) else build_libear(c_compiler, destination) 1274967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1284967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar environment = dict(os.environ) 1294967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar environment.update({'INTERCEPT_BUILD_TARGET_DIR': destination}) 1304967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1314967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar if not libear_path: 1324967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('intercept gonna use compiler wrappers') 1334967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar environment.update({ 1344967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'CC': os.path.join(bin_dir, COMPILER_WRAPPER_CC), 1354967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'CXX': os.path.join(bin_dir, COMPILER_WRAPPER_CXX), 1364967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'INTERCEPT_BUILD_CC': c_compiler, 1374967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'INTERCEPT_BUILD_CXX': cxx_compiler, 1384967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'INTERCEPT_BUILD_VERBOSE': 'DEBUG' if args.verbose > 2 else 'INFO' 1394967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar }) 1404967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar elif sys.platform == 'darwin': 1414967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('intercept gonna preload libear on OSX') 1424967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar environment.update({ 1434967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'DYLD_INSERT_LIBRARIES': libear_path, 1444967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'DYLD_FORCE_FLAT_NAMESPACE': '1' 1454967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar }) 1464967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar else: 1474967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('intercept gonna preload libear on UNIX') 1484967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar environment.update({'LD_PRELOAD': libear_path}) 1494967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1504967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return environment 1514967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1524967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1534967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef intercept_build_wrapper(cplusplus): 1544967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Entry point for `intercept-cc` and `intercept-c++` compiler wrappers. 1554967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1564967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar It does generate execution report into target directory. And execute 1574967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar the wrapped compilation with the real compiler. The parameters for 1584967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar report and execution are from environment variables. 1594967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1604967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar Those parameters which for 'libear' library can't have meaningful 1614967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar values are faked. """ 1624967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1634967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # initialize wrapper logging 1644967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.basicConfig(format='intercept: %(levelname)s: %(message)s', 1654967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar level=os.getenv('INTERCEPT_BUILD_VERBOSE', 'INFO')) 1664967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # write report 1674967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar try: 1684967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar target_dir = os.getenv('INTERCEPT_BUILD_TARGET_DIR') 1694967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar if not target_dir: 1704967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar raise UserWarning('exec report target directory not found') 1714967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar pid = str(os.getpid()) 1724967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar target_file = os.path.join(target_dir, pid + '.cmd') 1734967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('writing exec report to: %s', target_file) 1744967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar with open(target_file, 'ab') as handler: 1754967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar working_dir = os.getcwd() 1764967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar command = US.join(sys.argv) + US 1774967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar content = RS.join([pid, pid, 'wrapper', working_dir, command]) + GS 1784967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar handler.write(content.encode('utf-8')) 1794967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar except IOError: 1804967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.exception('writing exec report failed') 1814967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar except UserWarning as warning: 1824967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.warning(warning) 1834967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # execute with real compiler 1844967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar compiler = os.getenv('INTERCEPT_BUILD_CXX', 'c++') if cplusplus \ 1854967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar else os.getenv('INTERCEPT_BUILD_CC', 'cc') 1864967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar compilation = [compiler] + sys.argv[1:] 1874967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('execute compiler: %s', compilation) 1884967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return subprocess.call(compilation) 1894967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1904967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1914967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef parse_exec_trace(filename): 1924967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Parse the file generated by the 'libear' preloaded library. 1934967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1944967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar Given filename points to a file which contains the basic report 1954967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar generated by the interception library or wrapper command. A single 1964967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar report file _might_ contain multiple process creation info. """ 1974967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 1984967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('parse exec trace file: %s', filename) 1994967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar with open(filename, 'r') as handler: 2004967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar content = handler.read() 2014967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar for group in filter(bool, content.split(GS)): 2024967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar records = group.split(RS) 2034967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar yield { 2044967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'pid': records[0], 2054967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'ppid': records[1], 2064967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'function': records[2], 2074967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'directory': records[3], 2084967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'command': records[4].split(US)[:-1] 2094967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar } 2104967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2114967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2124967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef format_entry(exec_trace): 2134967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Generate the desired fields for compilation database entries. """ 2144967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2154967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar def abspath(cwd, name): 2164967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Create normalized absolute path from input filename. """ 2174967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar fullname = name if os.path.isabs(name) else os.path.join(cwd, name) 2184967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return os.path.normpath(fullname) 2194967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2204967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('format this command: %s', exec_trace['command']) 2214967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar compilation = split_command(exec_trace['command']) 2224967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar if compilation: 2234967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar for source in compilation.files: 2244967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar compiler = 'c++' if compilation.compiler == 'c++' else 'cc' 2254967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar command = [compiler, '-c'] + compilation.flags + [source] 2264967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logging.debug('formated as: %s', command) 2274967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar yield { 2284967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'directory': exec_trace['directory'], 2294967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'command': encode(command), 2304967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'file': abspath(exec_trace['directory'], source) 2314967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar } 2324967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2334967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2344967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef is_preload_disabled(platform): 2354967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Library-based interposition will fail silently if SIP is enabled, 2364967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar so this should be detected. You can detect whether SIP is enabled on 2374967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar Darwin by checking whether (1) there is a binary called 'csrutil' in 2384967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar the path and, if so, (2) whether the output of executing 'csrutil status' 2394967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar contains 'System Integrity Protection status: enabled'. 2404967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2414967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar Same problem on linux when SELinux is enabled. The status query program 2424967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 'sestatus' and the output when it's enabled 'SELinux status: enabled'. """ 2434967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2444967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar if platform == 'darwin': 2454967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar pattern = re.compile(r'System Integrity Protection status:\s+enabled') 2464967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar command = ['csrutil', 'status'] 2474967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar elif platform in {'linux', 'linux2'}: 2484967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar pattern = re.compile(r'SELinux status:\s+enabled') 2494967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar command = ['sestatus'] 2504967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar else: 2514967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return False 2524967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2534967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar try: 2544967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar lines = subprocess.check_output(command).decode('utf-8') 2554967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return any((pattern.match(line) for line in lines.splitlines())) 2564967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar except: 2574967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return False 2584967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2594967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2604967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef entry_hash(entry): 2614967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Implement unique hash method for compilation database entries. """ 2624967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2634967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # For faster lookup in set filename is reverted 2644967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar filename = entry['file'][::-1] 2654967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # For faster lookup in set directory is reverted 2664967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar directory = entry['directory'][::-1] 2674967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # On OS X the 'cc' and 'c++' compilers are wrappers for 2684967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # 'clang' therefore both call would be logged. To avoid 2694967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # this the hash does not contain the first word of the 2704967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar # command. 2714967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar command = ' '.join(decode(entry['command'])[1:]) 2724967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2734967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return '<>'.join([filename, directory, command]) 2744967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2754967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2764967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainardef create_parser(): 2774967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar """ Command line argument parser factory method. """ 2784967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2794967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar parser = argparse.ArgumentParser( 2804967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar formatter_class=argparse.ArgumentDefaultsHelpFormatter) 2814967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 2824967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar parser.add_argument( 2834967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar '--verbose', '-v', 2844967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar action='count', 2854967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar default=0, 2864967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar help="""Enable verbose output from '%(prog)s'. A second and third 2874967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar flag increases verbosity.""") 2884967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar parser.add_argument( 2894967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar '--cdb', 2904967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar metavar='<file>', 2914967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar default="compile_commands.json", 2924967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar help="""The JSON compilation database.""") 2934967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar group = parser.add_mutually_exclusive_group() 2944967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar group.add_argument( 2954967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar '--append', 2964967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar action='store_true', 2974967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar help="""Append new entries to existing compilation database.""") 2984967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar group.add_argument( 2994967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar '--disable-filter', '-n', 3004967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar dest='raw_entries', 3014967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar action='store_true', 3024967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar help="""Intercepted child process creation calls (exec calls) are all 3034967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar logged to the output. The output is not a compilation database. 3044967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar This flag is for debug purposes.""") 3054967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 3064967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar advanced = parser.add_argument_group('advanced options') 3074967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar advanced.add_argument( 3084967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar '--override-compiler', 3094967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar action='store_true', 3104967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar help="""Always resort to the compiler wrapper even when better 3114967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar intercept methods are available.""") 3124967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar advanced.add_argument( 3134967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar '--use-cc', 3144967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar metavar='<path>', 3154967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar dest='cc', 3164967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar default='cc', 3174967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar help="""When '%(prog)s' analyzes a project by interposing a compiler 3184967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar wrapper, which executes a real compiler for compilation and 3194967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar do other tasks (record the compiler invocation). Because of 3204967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar this interposing, '%(prog)s' does not know what compiler your 3214967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar project normally uses. Instead, it simply overrides the CC 3224967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar environment variable, and guesses your default compiler. 3234967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 3244967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar If you need '%(prog)s' to use a specific compiler for 3254967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar *compilation* then you can use this option to specify a path 3264967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar to that compiler.""") 3274967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar advanced.add_argument( 3284967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar '--use-c++', 3294967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar metavar='<path>', 3304967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar dest='cxx', 3314967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar default='c++', 3324967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar help="""This is the same as "--use-cc" but for C++ code.""") 3334967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 3344967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar parser.add_argument( 3354967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar dest='build', 3364967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar nargs=argparse.REMAINDER, 3374967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar help="""Command to run.""") 3384967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar 3394967a710c84587c654b56c828382219c3937dacbPirama Arumuga Nainar return parser 340