10a8c90248264a8b26970b4473770bcc3df8515fJosh Gao"""Helper class to quickly write a loop over all standard input files. 20a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 30a8c90248264a8b26970b4473770bcc3df8515fJosh GaoTypical use is: 40a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 50a8c90248264a8b26970b4473770bcc3df8515fJosh Gao import fileinput 60a8c90248264a8b26970b4473770bcc3df8515fJosh Gao for line in fileinput.input(): 70a8c90248264a8b26970b4473770bcc3df8515fJosh Gao process(line) 80a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 90a8c90248264a8b26970b4473770bcc3df8515fJosh GaoThis iterates over the lines of all files listed in sys.argv[1:], 100a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodefaulting to sys.stdin if the list is empty. If a filename is '-' it 110a8c90248264a8b26970b4473770bcc3df8515fJosh Gaois also replaced by sys.stdin. To specify an alternative list of 120a8c90248264a8b26970b4473770bcc3df8515fJosh Gaofilenames, pass it as the argument to input(). A single file name is 130a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoalso allowed. 140a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 150a8c90248264a8b26970b4473770bcc3df8515fJosh GaoFunctions filename(), lineno() return the filename and cumulative line 160a8c90248264a8b26970b4473770bcc3df8515fJosh Gaonumber of the line that has just been read; filelineno() returns its 170a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoline number in the current file; isfirstline() returns true iff the 180a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoline just read is the first line of its file; isstdin() returns true 190a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoiff the line was read from sys.stdin. Function nextfile() closes the 200a8c90248264a8b26970b4473770bcc3df8515fJosh Gaocurrent file so that the next iteration will read the first line from 210a8c90248264a8b26970b4473770bcc3df8515fJosh Gaothe next file (if any); lines not read from the file will not count 220a8c90248264a8b26970b4473770bcc3df8515fJosh Gaotowards the cumulative line count; the filename is not changed until 230a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoafter the first line of the next file has been read. Function close() 240a8c90248264a8b26970b4473770bcc3df8515fJosh Gaocloses the sequence. 250a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 260a8c90248264a8b26970b4473770bcc3df8515fJosh GaoBefore any lines have been read, filename() returns None and both line 270a8c90248264a8b26970b4473770bcc3df8515fJosh Gaonumbers are zero; nextfile() has no effect. After all lines have been 280a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoread, filename() and the line number functions return the values 290a8c90248264a8b26970b4473770bcc3df8515fJosh Gaopertaining to the last line read; nextfile() has no effect. 300a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 310a8c90248264a8b26970b4473770bcc3df8515fJosh GaoAll files are opened in text mode by default, you can override this by 320a8c90248264a8b26970b4473770bcc3df8515fJosh Gaosetting the mode parameter to input() or FileInput.__init__(). 330a8c90248264a8b26970b4473770bcc3df8515fJosh GaoIf an I/O error occurs during opening or reading a file, the IOError 340a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoexception is raised. 350a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 360a8c90248264a8b26970b4473770bcc3df8515fJosh GaoIf sys.stdin is used more than once, the second and further use will 370a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoreturn no lines, except perhaps for interactive use, or if it has been 380a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoexplicitly reset (e.g. using sys.stdin.seek(0)). 390a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 400a8c90248264a8b26970b4473770bcc3df8515fJosh GaoEmpty files are opened and immediately closed; the only time their 410a8c90248264a8b26970b4473770bcc3df8515fJosh Gaopresence in the list of filenames is noticeable at all is when the 420a8c90248264a8b26970b4473770bcc3df8515fJosh Gaolast file opened is empty. 430a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 440a8c90248264a8b26970b4473770bcc3df8515fJosh GaoIt is possible that the last line of a file doesn't end in a newline 450a8c90248264a8b26970b4473770bcc3df8515fJosh Gaocharacter; otherwise lines are returned including the trailing 460a8c90248264a8b26970b4473770bcc3df8515fJosh Gaonewline. 470a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 480a8c90248264a8b26970b4473770bcc3df8515fJosh GaoClass FileInput is the implementation; its methods filename(), 490a8c90248264a8b26970b4473770bcc3df8515fJosh Gaolineno(), fileline(), isfirstline(), isstdin(), nextfile() and close() 500a8c90248264a8b26970b4473770bcc3df8515fJosh Gaocorrespond to the functions in the module. In addition it has a 510a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoreadline() method which returns the next input line, and a 520a8c90248264a8b26970b4473770bcc3df8515fJosh Gao__getitem__() method which implements the sequence behavior. The 530a8c90248264a8b26970b4473770bcc3df8515fJosh Gaosequence must be accessed in strictly sequential order; sequence 540a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoaccess and readline() cannot be mixed. 550a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 560a8c90248264a8b26970b4473770bcc3df8515fJosh GaoOptional in-place filtering: if the keyword argument inplace=1 is 570a8c90248264a8b26970b4473770bcc3df8515fJosh Gaopassed to input() or to the FileInput constructor, the file is moved 580a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoto a backup file and standard output is directed to the input file. 590a8c90248264a8b26970b4473770bcc3df8515fJosh GaoThis makes it possible to write a filter that rewrites its input file 600a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoin place. If the keyword argument backup=".<some extension>" is also 610a8c90248264a8b26970b4473770bcc3df8515fJosh Gaogiven, it specifies the extension for the backup file, and the backup 620a8c90248264a8b26970b4473770bcc3df8515fJosh Gaofile remains around; by default, the extension is ".bak" and it is 630a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodeleted when the output file is closed. In-place filtering is 640a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodisabled when standard input is read. XXX The current implementation 650a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodoes not work for MS-DOS 8+3 filesystems. 660a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 670a8c90248264a8b26970b4473770bcc3df8515fJosh GaoPerformance: this module is unfortunately one of the slower ways of 680a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoprocessing large numbers of input lines. Nevertheless, a significant 690a8c90248264a8b26970b4473770bcc3df8515fJosh Gaospeed-up has been obtained by using readlines(bufsize) instead of 700a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoreadline(). A new keyword argument, bufsize=N, is present on the 710a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoinput() function and the FileInput() class to override the default 720a8c90248264a8b26970b4473770bcc3df8515fJosh Gaobuffer size. 730a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 740a8c90248264a8b26970b4473770bcc3df8515fJosh GaoXXX Possible additions: 750a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 760a8c90248264a8b26970b4473770bcc3df8515fJosh Gao- optional getopt argument processing 770a8c90248264a8b26970b4473770bcc3df8515fJosh Gao- isatty() 780a8c90248264a8b26970b4473770bcc3df8515fJosh Gao- read(), read(size), even readlines() 790a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 800a8c90248264a8b26970b4473770bcc3df8515fJosh Gao""" 810a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 820a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoimport sys, os 830a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 840a8c90248264a8b26970b4473770bcc3df8515fJosh Gao__all__ = ["input","close","nextfile","filename","lineno","filelineno", 850a8c90248264a8b26970b4473770bcc3df8515fJosh Gao "isfirstline","isstdin","FileInput"] 860a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 870a8c90248264a8b26970b4473770bcc3df8515fJosh Gao_state = None 880a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 890a8c90248264a8b26970b4473770bcc3df8515fJosh GaoDEFAULT_BUFSIZE = 8*1024 900a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 910a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef input(files=None, inplace=0, backup="", bufsize=0, 920a8c90248264a8b26970b4473770bcc3df8515fJosh Gao mode="r", openhook=None): 930a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """input([files[, inplace[, backup[, mode[, openhook]]]]]) 940a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 950a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Create an instance of the FileInput class. The instance will be used 960a8c90248264a8b26970b4473770bcc3df8515fJosh Gao as global state for the functions of this module, and is also returned 970a8c90248264a8b26970b4473770bcc3df8515fJosh Gao to use during iteration. The parameters to this function will be passed 980a8c90248264a8b26970b4473770bcc3df8515fJosh Gao along to the constructor of the FileInput class. 990a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1000a8c90248264a8b26970b4473770bcc3df8515fJosh Gao global _state 1010a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if _state and _state._file: 1020a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "input() already active" 1030a8c90248264a8b26970b4473770bcc3df8515fJosh Gao _state = FileInput(files, inplace, backup, bufsize, mode, openhook) 1040a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return _state 1050a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1060a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef close(): 1070a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """Close the sequence.""" 1080a8c90248264a8b26970b4473770bcc3df8515fJosh Gao global _state 1090a8c90248264a8b26970b4473770bcc3df8515fJosh Gao state = _state 1100a8c90248264a8b26970b4473770bcc3df8515fJosh Gao _state = None 1110a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if state: 1120a8c90248264a8b26970b4473770bcc3df8515fJosh Gao state.close() 1130a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1140a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef nextfile(): 1150a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1160a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Close the current file so that the next iteration will read the first 1170a8c90248264a8b26970b4473770bcc3df8515fJosh Gao line from the next file (if any); lines not read from the file will 1180a8c90248264a8b26970b4473770bcc3df8515fJosh Gao not count towards the cumulative line count. The filename is not 1190a8c90248264a8b26970b4473770bcc3df8515fJosh Gao changed until after the first line of the next file has been read. 1200a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Before the first line has been read, this function has no effect; 1210a8c90248264a8b26970b4473770bcc3df8515fJosh Gao it cannot be used to skip the first file. After the last line of the 1220a8c90248264a8b26970b4473770bcc3df8515fJosh Gao last file has been read, this function has no effect. 1230a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1240a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not _state: 1250a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "no active input()" 1260a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return _state.nextfile() 1270a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1280a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef filename(): 1290a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1300a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Return the name of the file currently being read. 1310a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Before the first line has been read, returns None. 1320a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1330a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not _state: 1340a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "no active input()" 1350a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return _state.filename() 1360a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1370a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef lineno(): 1380a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1390a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Return the cumulative line number of the line that has just been read. 1400a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Before the first line has been read, returns 0. After the last line 1410a8c90248264a8b26970b4473770bcc3df8515fJosh Gao of the last file has been read, returns the line number of that line. 1420a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1430a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not _state: 1440a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "no active input()" 1450a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return _state.lineno() 1460a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1470a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef filelineno(): 1480a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1490a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Return the line number in the current file. Before the first line 1500a8c90248264a8b26970b4473770bcc3df8515fJosh Gao has been read, returns 0. After the last line of the last file has 1510a8c90248264a8b26970b4473770bcc3df8515fJosh Gao been read, returns the line number of that line within the file. 1520a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1530a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not _state: 1540a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "no active input()" 1550a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return _state.filelineno() 1560a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1570a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef fileno(): 1580a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1590a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Return the file number of the current file. When no file is currently 1600a8c90248264a8b26970b4473770bcc3df8515fJosh Gao opened, returns -1. 1610a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1620a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not _state: 1630a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "no active input()" 1640a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return _state.fileno() 1650a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1660a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef isfirstline(): 1670a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1680a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Returns true the line just read is the first line of its file, 1690a8c90248264a8b26970b4473770bcc3df8515fJosh Gao otherwise returns false. 1700a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1710a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not _state: 1720a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "no active input()" 1730a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return _state.isfirstline() 1740a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1750a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef isstdin(): 1760a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1770a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Returns true if the last line was read from sys.stdin, 1780a8c90248264a8b26970b4473770bcc3df8515fJosh Gao otherwise returns false. 1790a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1800a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not _state: 1810a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "no active input()" 1820a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return _state.isstdin() 1830a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1840a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoclass FileInput: 1850a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """class FileInput([files[, inplace[, backup[, mode[, openhook]]]]]) 1860a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1870a8c90248264a8b26970b4473770bcc3df8515fJosh Gao Class FileInput is the implementation of the module; its methods 1880a8c90248264a8b26970b4473770bcc3df8515fJosh Gao filename(), lineno(), fileline(), isfirstline(), isstdin(), fileno(), 1890a8c90248264a8b26970b4473770bcc3df8515fJosh Gao nextfile() and close() correspond to the functions of the same name 1900a8c90248264a8b26970b4473770bcc3df8515fJosh Gao in the module. 1910a8c90248264a8b26970b4473770bcc3df8515fJosh Gao In addition it has a readline() method which returns the next 1920a8c90248264a8b26970b4473770bcc3df8515fJosh Gao input line, and a __getitem__() method which implements the 1930a8c90248264a8b26970b4473770bcc3df8515fJosh Gao sequence behavior. The sequence must be accessed in strictly 1940a8c90248264a8b26970b4473770bcc3df8515fJosh Gao sequential order; random access and readline() cannot be mixed. 1950a8c90248264a8b26970b4473770bcc3df8515fJosh Gao """ 1960a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 1970a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def __init__(self, files=None, inplace=0, backup="", bufsize=0, 1980a8c90248264a8b26970b4473770bcc3df8515fJosh Gao mode="r", openhook=None): 1990a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if isinstance(files, basestring): 2000a8c90248264a8b26970b4473770bcc3df8515fJosh Gao files = (files,) 2010a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 2020a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if files is None: 2030a8c90248264a8b26970b4473770bcc3df8515fJosh Gao files = sys.argv[1:] 2040a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not files: 2050a8c90248264a8b26970b4473770bcc3df8515fJosh Gao files = ('-',) 2060a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 2070a8c90248264a8b26970b4473770bcc3df8515fJosh Gao files = tuple(files) 2080a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._files = files 2090a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._inplace = inplace 2100a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._backup = backup 2110a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._bufsize = bufsize or DEFAULT_BUFSIZE 2120a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._savestdout = None 2130a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._output = None 2140a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._filename = None 2150a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._lineno = 0 2160a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._filelineno = 0 2170a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._file = None 2180a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._isstdin = False 2190a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._backupfilename = None 2200a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._buffer = [] 2210a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._bufindex = 0 2220a8c90248264a8b26970b4473770bcc3df8515fJosh Gao # restrict mode argument to reading modes 2230a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if mode not in ('r', 'rU', 'U', 'rb'): 2240a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise ValueError("FileInput opening mode must be one of " 2250a8c90248264a8b26970b4473770bcc3df8515fJosh Gao "'r', 'rU', 'U' and 'rb'") 2260a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._mode = mode 2270a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if inplace and openhook: 2280a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise ValueError("FileInput cannot use an opening hook in inplace mode") 2290a8c90248264a8b26970b4473770bcc3df8515fJosh Gao elif openhook and not hasattr(openhook, '__call__'): 2300a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise ValueError("FileInput openhook must be callable") 2310a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._openhook = openhook 2320a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2330a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def __del__(self): 2340a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self.close() 2350a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2360a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def close(self): 2370a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self.nextfile() 2380a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._files = () 2390a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2400a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def __iter__(self): 2410a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self 2420a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2430a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def next(self): 2440a8c90248264a8b26970b4473770bcc3df8515fJosh Gao try: 2450a8c90248264a8b26970b4473770bcc3df8515fJosh Gao line = self._buffer[self._bufindex] 2460a8c90248264a8b26970b4473770bcc3df8515fJosh Gao except IndexError: 2470a8c90248264a8b26970b4473770bcc3df8515fJosh Gao pass 2480a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 2490a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._bufindex += 1 2500a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._lineno += 1 2510a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._filelineno += 1 2520a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return line 2530a8c90248264a8b26970b4473770bcc3df8515fJosh Gao line = self.readline() 2540a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not line: 2550a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise StopIteration 2560a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return line 2570a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2580a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def __getitem__(self, i): 2590a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if i != self._lineno: 2600a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise RuntimeError, "accessing lines out of order" 2610a8c90248264a8b26970b4473770bcc3df8515fJosh Gao try: 2620a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self.next() 2630a8c90248264a8b26970b4473770bcc3df8515fJosh Gao except StopIteration: 2640a8c90248264a8b26970b4473770bcc3df8515fJosh Gao raise IndexError, "end of input reached" 2650a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2660a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def nextfile(self): 2670a8c90248264a8b26970b4473770bcc3df8515fJosh Gao savestdout = self._savestdout 2680a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._savestdout = 0 2690a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if savestdout: 2700a8c90248264a8b26970b4473770bcc3df8515fJosh Gao sys.stdout = savestdout 2710a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2720a8c90248264a8b26970b4473770bcc3df8515fJosh Gao output = self._output 2730a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._output = 0 2740a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if output: 2750a8c90248264a8b26970b4473770bcc3df8515fJosh Gao output.close() 2760a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2770a8c90248264a8b26970b4473770bcc3df8515fJosh Gao file = self._file 2780a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._file = 0 2790a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if file and not self._isstdin: 2800a8c90248264a8b26970b4473770bcc3df8515fJosh Gao file.close() 2810a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2820a8c90248264a8b26970b4473770bcc3df8515fJosh Gao backupfilename = self._backupfilename 2830a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._backupfilename = 0 2840a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if backupfilename and not self._backup: 2850a8c90248264a8b26970b4473770bcc3df8515fJosh Gao try: os.unlink(backupfilename) 2860a8c90248264a8b26970b4473770bcc3df8515fJosh Gao except OSError: pass 2870a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2880a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._isstdin = False 2890a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._buffer = [] 2900a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._bufindex = 0 2910a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 2920a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def readline(self): 2930a8c90248264a8b26970b4473770bcc3df8515fJosh Gao try: 2940a8c90248264a8b26970b4473770bcc3df8515fJosh Gao line = self._buffer[self._bufindex] 2950a8c90248264a8b26970b4473770bcc3df8515fJosh Gao except IndexError: 2960a8c90248264a8b26970b4473770bcc3df8515fJosh Gao pass 2970a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 2980a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._bufindex += 1 2990a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._lineno += 1 3000a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._filelineno += 1 3010a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return line 3020a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not self._file: 3030a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not self._files: 3040a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return "" 3050a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._filename = self._files[0] 3060a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._files = self._files[1:] 3070a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._filelineno = 0 3080a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._file = None 3090a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._isstdin = False 3100a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._backupfilename = 0 3110a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if self._filename == '-': 3120a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._filename = '<stdin>' 3130a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._file = sys.stdin 3140a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._isstdin = True 3150a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 3160a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if self._inplace: 3170a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._backupfilename = ( 3180a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._filename + (self._backup or os.extsep+"bak")) 3190a8c90248264a8b26970b4473770bcc3df8515fJosh Gao try: os.unlink(self._backupfilename) 3200a8c90248264a8b26970b4473770bcc3df8515fJosh Gao except os.error: pass 3210a8c90248264a8b26970b4473770bcc3df8515fJosh Gao # The next few lines may raise IOError 3220a8c90248264a8b26970b4473770bcc3df8515fJosh Gao os.rename(self._filename, self._backupfilename) 3230a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._file = open(self._backupfilename, self._mode) 3240a8c90248264a8b26970b4473770bcc3df8515fJosh Gao try: 3250a8c90248264a8b26970b4473770bcc3df8515fJosh Gao perm = os.fstat(self._file.fileno()).st_mode 3260a8c90248264a8b26970b4473770bcc3df8515fJosh Gao except OSError: 3270a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._output = open(self._filename, "w") 3280a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 3290a8c90248264a8b26970b4473770bcc3df8515fJosh Gao fd = os.open(self._filename, 3300a8c90248264a8b26970b4473770bcc3df8515fJosh Gao os.O_CREAT | os.O_WRONLY | os.O_TRUNC, 3310a8c90248264a8b26970b4473770bcc3df8515fJosh Gao perm) 3320a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._output = os.fdopen(fd, "w") 3330a8c90248264a8b26970b4473770bcc3df8515fJosh Gao try: 3340a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if hasattr(os, 'chmod'): 3350a8c90248264a8b26970b4473770bcc3df8515fJosh Gao os.chmod(self._filename, perm) 3360a8c90248264a8b26970b4473770bcc3df8515fJosh Gao except OSError: 3370a8c90248264a8b26970b4473770bcc3df8515fJosh Gao pass 3380a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._savestdout = sys.stdout 3390a8c90248264a8b26970b4473770bcc3df8515fJosh Gao sys.stdout = self._output 3400a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 3410a8c90248264a8b26970b4473770bcc3df8515fJosh Gao # This may raise IOError 3420a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if self._openhook: 3430a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._file = self._openhook(self._filename, self._mode) 3440a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 3450a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._file = open(self._filename, self._mode) 3460a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._buffer = self._file.readlines(self._bufsize) 3470a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self._bufindex = 0 3480a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if not self._buffer: 3490a8c90248264a8b26970b4473770bcc3df8515fJosh Gao self.nextfile() 3500a8c90248264a8b26970b4473770bcc3df8515fJosh Gao # Recursive call 3510a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self.readline() 3520a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3530a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def filename(self): 3540a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self._filename 3550a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3560a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def lineno(self): 3570a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self._lineno 3580a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3590a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def filelineno(self): 3600a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self._filelineno 3610a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3620a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def fileno(self): 3630a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if self._file: 3640a8c90248264a8b26970b4473770bcc3df8515fJosh Gao try: 3650a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self._file.fileno() 3660a8c90248264a8b26970b4473770bcc3df8515fJosh Gao except ValueError: 3670a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return -1 3680a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 3690a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return -1 3700a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3710a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def isfirstline(self): 3720a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self._filelineno == 1 3730a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3740a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def isstdin(self): 3750a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return self._isstdin 3760a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3770a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3780a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef hook_compressed(filename, mode): 3790a8c90248264a8b26970b4473770bcc3df8515fJosh Gao ext = os.path.splitext(filename)[1] 3800a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if ext == '.gz': 3810a8c90248264a8b26970b4473770bcc3df8515fJosh Gao import gzip 3820a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return gzip.open(filename, mode) 3830a8c90248264a8b26970b4473770bcc3df8515fJosh Gao elif ext == '.bz2': 3840a8c90248264a8b26970b4473770bcc3df8515fJosh Gao import bz2 3850a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return bz2.BZ2File(filename, mode) 3860a8c90248264a8b26970b4473770bcc3df8515fJosh Gao else: 3870a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return open(filename, mode) 3880a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3890a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3900a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef hook_encoded(encoding): 3910a8c90248264a8b26970b4473770bcc3df8515fJosh Gao import codecs 3920a8c90248264a8b26970b4473770bcc3df8515fJosh Gao def openhook(filename, mode): 3930a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return codecs.open(filename, mode, encoding) 3940a8c90248264a8b26970b4473770bcc3df8515fJosh Gao return openhook 3950a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3960a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 3970a8c90248264a8b26970b4473770bcc3df8515fJosh Gaodef _test(): 3980a8c90248264a8b26970b4473770bcc3df8515fJosh Gao import getopt 3990a8c90248264a8b26970b4473770bcc3df8515fJosh Gao inplace = 0 4000a8c90248264a8b26970b4473770bcc3df8515fJosh Gao backup = 0 4010a8c90248264a8b26970b4473770bcc3df8515fJosh Gao opts, args = getopt.getopt(sys.argv[1:], "ib:") 4020a8c90248264a8b26970b4473770bcc3df8515fJosh Gao for o, a in opts: 4030a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if o == '-i': inplace = 1 4040a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if o == '-b': backup = a 4050a8c90248264a8b26970b4473770bcc3df8515fJosh Gao for line in input(args, inplace=inplace, backup=backup): 4060a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if line[-1:] == '\n': line = line[:-1] 4070a8c90248264a8b26970b4473770bcc3df8515fJosh Gao if line[-1:] == '\r': line = line[:-1] 4080a8c90248264a8b26970b4473770bcc3df8515fJosh Gao print "%d: %s[%d]%s %s" % (lineno(), filename(), filelineno(), 4090a8c90248264a8b26970b4473770bcc3df8515fJosh Gao isfirstline() and "*" or "", line) 4100a8c90248264a8b26970b4473770bcc3df8515fJosh Gao print "%d: %s[%d]" % (lineno(), filename(), filelineno()) 4110a8c90248264a8b26970b4473770bcc3df8515fJosh Gao 4120a8c90248264a8b26970b4473770bcc3df8515fJosh Gaoif __name__ == '__main__': 4130a8c90248264a8b26970b4473770bcc3df8515fJosh Gao _test() 414