1efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang# Copyright 2016 The Chromium OS Authors. All rights reserved. 2efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang# Use of this source code is governed by a BSD-style license that can be 3efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang# found in the LICENSE file. 4efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 5efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang"""This module provides an object to record the output of command-line program. 6efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang""" 7efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 8efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangimport fcntl 9efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangimport logging 10efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangimport os 11efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangimport pty 12efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangimport re 13efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangimport subprocess 14efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangimport threading 15efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangimport time 16efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 17efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 18efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangclass OutputRecorderError(Exception): 19efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """An exception class for output_recorder module.""" 20efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang pass 21efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 22efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 23efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangclass OutputRecorder(object): 24efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """A class used to record the output of command line program. 25efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 26efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang A thread is dedicated to performing non-blocking reading of the 27efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang command outpt in this class. Other possible approaches include 28efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 1. using gobject.io_add_watch() to register a callback and 29efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang reading the output when available, or 30efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 2. using select.select() with a short timeout, and reading 31efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang the output if available. 32efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang However, the above two approaches are not very reliable. Hence, 33efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang this approach using non-blocking reading is adopted. 34efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 35efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang To prevent the block buffering of the command output, a pseudo 36efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang terminal is created through pty.openpty(). This forces the 37efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang line output. 38efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 39efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang This class saves the output in self.contents so that it is 40efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang easy to perform regular expression search(). The output is 41efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang also saved in a file. 42efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 43efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """ 44efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 45efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang DEFAULT_OPEN_MODE = 'a' 465c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang START_DELAY_SECS = 1 # Delay after starting recording. 47efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang STOP_DELAY_SECS = 1 # Delay before stopping recording. 48efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang POLLING_DELAY_SECS = 0.1 # Delay before next polling. 49efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang TMP_FILE = '/tmp/output_recorder.dat' 50efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 51efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang def __init__(self, cmd, open_mode=DEFAULT_OPEN_MODE, 525c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang start_delay_secs=START_DELAY_SECS, 53efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang stop_delay_secs=STOP_DELAY_SECS, save_file=TMP_FILE): 54efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """Construction of output recorder. 55efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 56efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang @param cmd: the command of which the output is to record. 57efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang @param open_mode: the open mode for writing output to save_file. 58efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang Could be either 'w' or 'a'. 59efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang @param stop_delay_secs: the delay time before stopping the cmd. 60efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang @param save_file: the file to save the output. 61efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 62efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """ 63efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.cmd = cmd 64efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.open_mode = open_mode 655c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang self.start_delay_secs = start_delay_secs 66efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.stop_delay_secs = stop_delay_secs 67efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.save_file = save_file 68efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.contents = [] 69efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 70efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # Create a thread dedicated to record the output. 71efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._recording_thread = None 72efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._stop_recording_thread_event = threading.Event() 73efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 74efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # Use pseudo terminal to prevent buffering of the program output. 75efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._master, self._slave = pty.openpty() 76efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._output = os.fdopen(self._master) 77efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 78efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # Set non-blocking flag. 79efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang fcntl.fcntl(self._output, fcntl.F_SETFL, os.O_NONBLOCK) 80efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 81efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 82efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang def record(self): 83efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """Record the output of the cmd.""" 84efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang logging.info('Recording output of "%s".', self.cmd) 85efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang try: 86efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._recorder = subprocess.Popen( 87efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.cmd, stdout=self._slave, stderr=self._slave) 88efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang except: 89efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang raise OutputRecorderError('Failed to run "%s"' % self.cmd) 90efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 91efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang with open(self.save_file, self.open_mode) as output_f: 92efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang output_f.write(os.linesep + '*' * 80 + os.linesep) 93efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang while True: 94efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang try: 95efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # Perform non-blocking read. 96efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang line = self._output.readline() 97efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang except: 98efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # Set empty string if nothing to read. 99efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang line = '' 100efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 101efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang if line: 102efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang output_f.write(line) 103efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang output_f.flush() 104efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # The output, e.g. the output of btmon, may contain some 105efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # special unicode such that we would like to escape. 106efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # In this way, regular expression search could be conducted 107efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # properly. 108efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.contents.append(line.encode('unicode-escape')) 109efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang elif self._stop_recording_thread_event.is_set(): 110efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._stop_recording_thread_event.clear() 111efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang break 112efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang else: 113efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # Sleep a while if nothing to read yet. 114efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang time.sleep(self.POLLING_DELAY_SECS) 115efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 116efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 117efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang def start(self): 118efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """Start the recording thread.""" 119efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang logging.info('Start recording thread.') 120efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.clear_contents() 121efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._recording_thread = threading.Thread(target=self.record) 122efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._recording_thread.start() 1235c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang time.sleep(self.start_delay_secs) 124efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 125efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 126efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang def stop(self): 127efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """Stop the recording thread.""" 128efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang logging.info('Stop recording thread.') 129efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang time.sleep(self.stop_delay_secs) 130efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._stop_recording_thread_event.set() 131efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._recording_thread.join() 132efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 133efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # Kill the process. 134efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._recorder.terminate() 135efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self._recorder.kill() 136efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 137efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 138efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang def clear_contents(self): 139efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """Clear the contents.""" 140efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang self.contents = [] 141efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 142efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 1435c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang def get_contents(self, search_str='', start_str=''): 1445c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang """Get the (filtered) contents. 1455c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang 1465c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang @param search_str: only lines with search_str would be kept. 1475c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang @param start_str: all lines before the occurrence of start_str would be 1485c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang filtered. 149efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 1505c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang @returns: the (filtered) contents. 151efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 152efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """ 1535c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang search_pattern = re.compile(search_str) if search_str else None 1545c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang start_pattern = re.compile(start_str) if start_str else None 1555c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang 1565c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang # Just returns the original contents if no filtered conditions are 1575c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang # specified. 1585c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang if not search_pattern and not start_pattern: 1595c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang return self.contents 1605c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang 1615c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang contents = [] 1625c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang start_flag = not bool(start_pattern) 1635c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang for line in self.contents: 1645c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang if start_flag: 1655c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang if search_pattern.search(line): 1665c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang contents.append(line.strip()) 1675c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang elif start_pattern.search(line): 1685c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang start_flag = True 1695c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang contents.append(line.strip()) 1705c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang 1715c692b8b7a63a4c40f7ff990514567463cb24658Joseph Hwang return contents 172efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 173efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 174efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang def find(self, pattern_str, flags=re.I): 175efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """Find a pattern string in the contents. 176efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 177efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang Note that the pattern_str is considered as an arbitrary literal string 178efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang that might contain re meta-characters, e.g., '(' or ')'. Hence, 179efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang re.escape() is applied before using re.compile. 180efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 181efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang @param pattern_str: the pattern string to search. 182efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang @param flags: the flags of the pattern expression behavior. 183efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 184efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang @returns: True if found. False otherwise. 185efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 186efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang """ 187efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang pattern = re.compile(re.escape(pattern_str), flags) 188efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang for line in self.contents: 189efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang result = pattern.search(line) 190efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang if result: 191efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang return True 192efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang return False 193efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 194efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 195efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwangif __name__ == '__main__': 196efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # A demo using btmon tool to monitor bluetoohd activity. 197efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang cmd = 'btmon' 198efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang recorder = OutputRecorder(cmd) 199efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 200efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang if True: 201efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang recorder.start() 202efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang # Perform some bluetooth activities here in another terminal. 203efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang time.sleep(recorder.stop_delay_secs) 204efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang recorder.stop() 205efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang 206efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang for line in recorder.get_contents(): 207efaf035c4211b878a9e95e3b06b2eaa04899d6a0Joseph Hwang print line 208