1645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez# Copyright (c) 2012 The Chromium Authors. All rights reserved. 2645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez# Use of this source code is governed by a BSD-style license that can be 3645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez# found in the LICENSE file. 4645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 5645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez"""A wrapper for subprocess to make calling shell commands easier.""" 6645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 7645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport logging 8645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport os 9645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport pipes 10645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport select 11645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport signal 12645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport string 13645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport StringIO 14645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport subprocess 15645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezimport time 16645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 17645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez# fcntl is not available on Windows. 18645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chaveztry: 19645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez import fcntl 20645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezexcept ImportError: 21645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez fcntl = None 22645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 23645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez_SafeShellChars = frozenset(string.ascii_letters + string.digits + '@%_-+=:,./') 24645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 25645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 26645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef SingleQuote(s): 27645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Return an shell-escaped version of the string using single quotes. 28645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 29645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Reliably quote a string which may contain unsafe characters (e.g. space, 30645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez quote, or other special characters such as '$'). 31645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 32645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The returned value can be used in a shell command line as one token that gets 33645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez to be interpreted literally. 34645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 35645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 36645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez s: The string to quote. 37645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 38645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Return: 39645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The string quoted using single quotes. 40645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 41645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return pipes.quote(s) 42645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 43645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 44645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef DoubleQuote(s): 45645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Return an shell-escaped version of the string using double quotes. 46645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 47645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Reliably quote a string which may contain unsafe characters (e.g. space 48645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez or quote characters), while retaining some shell features such as variable 49645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez interpolation. 50645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 51645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The returned value can be used in a shell command line as one token that gets 52645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez to be further interpreted by the shell. 53645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 54645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The set of characters that retain their special meaning may depend on the 55645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez shell implementation. This set usually includes: '$', '`', '\', '!', '*', 56645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez and '@'. 57645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 58645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 59645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez s: The string to quote. 60645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 61645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Return: 62645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The string quoted using double quotes. 63645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 64645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if not s: 65645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return '""' 66645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez elif all(c in _SafeShellChars for c in s): 67645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return s 68645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez else: 69645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return '"' + s.replace('"', '\\"') + '"' 70645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 71645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 72645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef ShrinkToSnippet(cmd_parts, var_name, var_value): 73645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Constructs a shell snippet for a command using a variable to shrink it. 74645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 75645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Takes into account all quoting that needs to happen. 76645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 77645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 78645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cmd_parts: A list of command arguments. 79645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez var_name: The variable that holds var_value. 80645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez var_value: The string to replace in cmd_parts with $var_name 81645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 82645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Returns: 83645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez A shell snippet that does not include setting the variable. 84645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 85645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez def shrink(value): 86645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez parts = (x and SingleQuote(x) for x in value.split(var_value)) 87645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez with_substitutions = ('"$%s"' % var_name).join(parts) 88645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return with_substitutions or "''" 89645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 90645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return ' '.join(shrink(part) for part in cmd_parts) 91645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 92645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 93645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef Popen(args, stdout=None, stderr=None, shell=None, cwd=None, env=None): 94645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return subprocess.Popen( 95645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args=args, cwd=cwd, stdout=stdout, stderr=stderr, 96645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez shell=shell, close_fds=True, env=env, 97645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez preexec_fn=lambda: signal.signal(signal.SIGPIPE, signal.SIG_DFL)) 98645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 99645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 100645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef Call(args, stdout=None, stderr=None, shell=None, cwd=None, env=None): 101645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez pipe = Popen(args, stdout=stdout, stderr=stderr, shell=shell, cwd=cwd, 102645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez env=env) 103645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez pipe.communicate() 104645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return pipe.wait() 105645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 106645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 107645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef RunCmd(args, cwd=None): 108645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Opens a subprocess to execute a program and returns its return value. 109645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 110645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 111645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args: A string or a sequence of program arguments. The program to execute is 112645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez the string or the first item in the args sequence. 113645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cwd: If not None, the subprocess's current directory will be changed to 114645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez |cwd| before it's executed. 115645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 116645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Returns: 117645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Return code from the command execution. 118645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 119645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez logging.info(str(args) + ' ' + (cwd or '')) 120645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return Call(args, cwd=cwd) 121645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 122645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 123645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef GetCmdOutput(args, cwd=None, shell=False): 124645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Open a subprocess to execute a program and returns its output. 125645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 126645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 127645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args: A string or a sequence of program arguments. The program to execute is 128645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez the string or the first item in the args sequence. 129645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cwd: If not None, the subprocess's current directory will be changed to 130645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez |cwd| before it's executed. 131645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez shell: Whether to execute args as a shell command. 132645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 133645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Returns: 134645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Captures and returns the command's stdout. 135645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Prints the command's stderr to logger (which defaults to stdout). 136645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 137645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez (_, output) = GetCmdStatusAndOutput(args, cwd, shell) 138645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return output 139645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 140645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 141645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef _ValidateAndLogCommand(args, cwd, shell): 142645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if isinstance(args, basestring): 143645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if not shell: 144645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez raise Exception('string args must be run with shell=True') 145645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez else: 146645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if shell: 147645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez raise Exception('array args must be run with shell=False') 148645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args = ' '.join(SingleQuote(c) for c in args) 149645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if cwd is None: 150645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cwd = '' 151645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez else: 152645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cwd = ':' + cwd 153645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez logging.info('[host]%s> %s', cwd, args) 154645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return args 155645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 156645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 157645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef GetCmdStatusAndOutput(args, cwd=None, shell=False): 158645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Executes a subprocess and returns its exit code and output. 159645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 160645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 161645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args: A string or a sequence of program arguments. The program to execute is 162645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez the string or the first item in the args sequence. 163645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cwd: If not None, the subprocess's current directory will be changed to 164645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez |cwd| before it's executed. 165645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez shell: Whether to execute args as a shell command. Must be True if args 166645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez is a string and False if args is a sequence. 167645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 168645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Returns: 169645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The 2-tuple (exit code, output). 170645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 171645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez status, stdout, stderr = GetCmdStatusOutputAndError( 172645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args, cwd=cwd, shell=shell) 173645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 174645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if stderr: 175645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez logging.critical('STDERR: %s', stderr) 176645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez logging.debug('STDOUT: %s%s', stdout[:4096].rstrip(), 177645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez '<truncated>' if len(stdout) > 4096 else '') 178645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return (status, stdout) 179645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 180645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 181645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef GetCmdStatusOutputAndError(args, cwd=None, shell=False): 182645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Executes a subprocess and returns its exit code, output, and errors. 183645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 184645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 185645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args: A string or a sequence of program arguments. The program to execute is 186645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez the string or the first item in the args sequence. 187645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cwd: If not None, the subprocess's current directory will be changed to 188645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez |cwd| before it's executed. 189645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez shell: Whether to execute args as a shell command. Must be True if args 190645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez is a string and False if args is a sequence. 191645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 192645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Returns: 193645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The 2-tuple (exit code, output). 194645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 195645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez _ValidateAndLogCommand(args, cwd, shell) 196645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez pipe = Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, 197645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez shell=shell, cwd=cwd) 198645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez stdout, stderr = pipe.communicate() 199645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return (pipe.returncode, stdout, stderr) 200645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 201645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 202645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezclass TimeoutError(Exception): 203645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Module-specific timeout exception.""" 204645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 205645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez def __init__(self, output=None): 206645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez super(TimeoutError, self).__init__() 207645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez self._output = output 208645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 209645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez @property 210645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez def output(self): 211645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return self._output 212645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 213645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 214645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef _IterProcessStdout(process, timeout=None, buffer_size=4096, 215645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez poll_interval=1): 216645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez assert fcntl, 'fcntl module is required' 217645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez try: 218645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez # Enable non-blocking reads from the child's stdout. 219645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez child_fd = process.stdout.fileno() 220645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez fl = fcntl.fcntl(child_fd, fcntl.F_GETFL) 221645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez fcntl.fcntl(child_fd, fcntl.F_SETFL, fl | os.O_NONBLOCK) 222645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 223645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez end_time = (time.time() + timeout) if timeout else None 224645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez while True: 225645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if end_time and time.time() > end_time: 226645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez raise TimeoutError() 227645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez read_fds, _, _ = select.select([child_fd], [], [], poll_interval) 228645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if child_fd in read_fds: 229645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez data = os.read(child_fd, buffer_size) 230645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if not data: 231645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez break 232645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez yield data 233645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if process.poll() is not None: 234645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez break 235645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez finally: 236645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez try: 237645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez # Make sure the process doesn't stick around if we fail with an 238645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez # exception. 239645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez process.kill() 240645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez except OSError: 241645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez pass 242645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez process.wait() 243645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 244645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 245645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef GetCmdStatusAndOutputWithTimeout(args, timeout, cwd=None, shell=False, 246645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez logfile=None): 247645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Executes a subprocess with a timeout. 248645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 249645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 250645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args: List of arguments to the program, the program to execute is the first 251645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez element. 252645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez timeout: the timeout in seconds or None to wait forever. 253645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cwd: If not None, the subprocess's current directory will be changed to 254645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez |cwd| before it's executed. 255645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez shell: Whether to execute args as a shell command. Must be True if args 256645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez is a string and False if args is a sequence. 257645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez logfile: Optional file-like object that will receive output from the 258645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez command as it is running. 259645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 260645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Returns: 261645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The 2-tuple (exit code, output). 262645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 263645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez _ValidateAndLogCommand(args, cwd, shell) 264645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez output = StringIO.StringIO() 265645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez process = Popen(args, cwd=cwd, shell=shell, stdout=subprocess.PIPE, 266645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez stderr=subprocess.STDOUT) 267645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez try: 268645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez for data in _IterProcessStdout(process, timeout=timeout): 269645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if logfile: 270645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez logfile.write(data) 271645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez output.write(data) 272645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez except TimeoutError: 273645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez raise TimeoutError(output.getvalue()) 274645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 275645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez str_output = output.getvalue() 276645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez logging.debug('STDOUT+STDERR: %s%s', str_output[:4096].rstrip(), 277645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez '<truncated>' if len(str_output) > 4096 else '') 278645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez return process.returncode, str_output 279645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 280645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 281645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavezdef IterCmdOutputLines(args, timeout=None, cwd=None, shell=False, 282645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez check_status=True): 283645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """Executes a subprocess and continuously yields lines from its output. 284645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 285645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Args: 286645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez args: List of arguments to the program, the program to execute is the first 287645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez element. 288645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cwd: If not None, the subprocess's current directory will be changed to 289645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez |cwd| before it's executed. 290645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez shell: Whether to execute args as a shell command. Must be True if args 291645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez is a string and False if args is a sequence. 292645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez check_status: A boolean indicating whether to check the exit status of the 293645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez process after all output has been read. 294645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 295645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Yields: 296645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez The output of the subprocess, line by line. 297645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez 298645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez Raises: 299645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez CalledProcessError if check_status is True and the process exited with a 300645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez non-zero exit status. 301645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez """ 302645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez cmd = _ValidateAndLogCommand(args, cwd, shell) 303645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez process = Popen(args, cwd=cwd, shell=shell, stdout=subprocess.PIPE, 304645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez stderr=subprocess.STDOUT) 305645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez buffer_output = '' 306645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez for data in _IterProcessStdout(process, timeout=timeout): 307645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez buffer_output += data 308645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez has_incomplete_line = buffer_output[-1] not in '\r\n' 309645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez lines = buffer_output.splitlines() 310645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez buffer_output = lines.pop() if has_incomplete_line else '' 311645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez for line in lines: 312645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez yield line 313645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if buffer_output: 314645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez yield buffer_output 315645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez if check_status and process.returncode: 316645501c2ab19a559ce82a1d5a29ced159a4c30fbLuis Hector Chavez raise subprocess.CalledProcessError(process.returncode, cmd) 317