1edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep"""Helper class to quickly write a loop over all standard input files. 2edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 3edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepTypical use is: 4edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 5edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep import fileinput 6edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep for line in fileinput.input(): 7edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep process(line) 8edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 9edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepThis iterates over the lines of all files listed in sys.argv[1:], 10edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdefaulting to sys.stdin if the list is empty. If a filename is '-' it 11edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepis also replaced by sys.stdin. To specify an alternative list of 12edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepfilenames, pass it as the argument to input(). A single file name is 13edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepalso allowed. 14edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 15edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepFunctions filename(), lineno() return the filename and cumulative line 16edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepnumber of the line that has just been read; filelineno() returns its 17edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepline number in the current file; isfirstline() returns true iff the 18edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepline just read is the first line of its file; isstdin() returns true 19edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepiff the line was read from sys.stdin. Function nextfile() closes the 20edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepcurrent file so that the next iteration will read the first line from 21edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepthe next file (if any); lines not read from the file will not count 22edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoeptowards the cumulative line count; the filename is not changed until 23edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepafter the first line of the next file has been read. Function close() 24edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepcloses the sequence. 25edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 26edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepBefore any lines have been read, filename() returns None and both line 27edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepnumbers are zero; nextfile() has no effect. After all lines have been 28edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepread, filename() and the line number functions return the values 29edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoeppertaining to the last line read; nextfile() has no effect. 30edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 31edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepAll files are opened in text mode by default, you can override this by 32edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepsetting the mode parameter to input() or FileInput.__init__(). 33edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepIf an I/O error occurs during opening or reading a file, the IOError 34edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepexception is raised. 35edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 36edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepIf sys.stdin is used more than once, the second and further use will 37edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepreturn no lines, except perhaps for interactive use, or if it has been 38edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepexplicitly reset (e.g. using sys.stdin.seek(0)). 39edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 40edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepEmpty files are opened and immediately closed; the only time their 41edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoeppresence in the list of filenames is noticeable at all is when the 42edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoeplast file opened is empty. 43edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 44edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepIt is possible that the last line of a file doesn't end in a newline 45edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepcharacter; otherwise lines are returned including the trailing 46edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepnewline. 47edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 48edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepClass FileInput is the implementation; its methods filename(), 49edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoeplineno(), fileline(), isfirstline(), isstdin(), nextfile() and close() 50edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepcorrespond to the functions in the module. In addition it has a 51edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepreadline() method which returns the next input line, and a 52edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep__getitem__() method which implements the sequence behavior. The 53edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepsequence must be accessed in strictly sequential order; sequence 54edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepaccess and readline() cannot be mixed. 55edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 56edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepOptional in-place filtering: if the keyword argument inplace=1 is 57edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoeppassed to input() or to the FileInput constructor, the file is moved 58edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepto a backup file and standard output is directed to the input file. 59edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepThis makes it possible to write a filter that rewrites its input file 60edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepin place. If the keyword argument backup=".<some extension>" is also 61edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepgiven, it specifies the extension for the backup file, and the backup 62edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepfile remains around; by default, the extension is ".bak" and it is 63edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdeleted when the output file is closed. In-place filtering is 64edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdisabled when standard input is read. XXX The current implementation 65edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdoes not work for MS-DOS 8+3 filesystems. 66edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 67edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepPerformance: this module is unfortunately one of the slower ways of 68edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepprocessing large numbers of input lines. Nevertheless, a significant 69edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepspeed-up has been obtained by using readlines(bufsize) instead of 70edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepreadline(). A new keyword argument, bufsize=N, is present on the 71edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepinput() function and the FileInput() class to override the default 72edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepbuffer size. 73edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 74edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepXXX Possible additions: 75edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 76edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep- optional getopt argument processing 77edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep- isatty() 78edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep- read(), read(size), even readlines() 79edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 80edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep""" 81edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 82edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepimport sys, os 83edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 84edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep__all__ = ["input","close","nextfile","filename","lineno","filelineno", 85edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep "isfirstline","isstdin","FileInput"] 86edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 87edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep_state = None 88edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 89edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander StoepDEFAULT_BUFSIZE = 8*1024 90edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 91edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef input(files=None, inplace=0, backup="", bufsize=0, 92edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep mode="r", openhook=None): 93edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """input([files[, inplace[, backup[, mode[, openhook]]]]]) 94edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 95edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Create an instance of the FileInput class. The instance will be used 96edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep as global state for the functions of this module, and is also returned 97edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep to use during iteration. The parameters to this function will be passed 98edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep along to the constructor of the FileInput class. 99edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 100edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep global _state 101edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if _state and _state._file: 102edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "input() already active" 103edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep _state = FileInput(files, inplace, backup, bufsize, mode, openhook) 104edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return _state 105edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 106edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef close(): 107edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """Close the sequence.""" 108edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep global _state 109edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep state = _state 110edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep _state = None 111edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if state: 112edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep state.close() 113edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 114edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef nextfile(): 115edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 116edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Close the current file so that the next iteration will read the first 117edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep line from the next file (if any); lines not read from the file will 118edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep not count towards the cumulative line count. The filename is not 119edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep changed until after the first line of the next file has been read. 120edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Before the first line has been read, this function has no effect; 121edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep it cannot be used to skip the first file. After the last line of the 122edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep last file has been read, this function has no effect. 123edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 124edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not _state: 125edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "no active input()" 126edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return _state.nextfile() 127edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 128edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef filename(): 129edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 130edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Return the name of the file currently being read. 131edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Before the first line has been read, returns None. 132edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 133edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not _state: 134edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "no active input()" 135edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return _state.filename() 136edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 137edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef lineno(): 138edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 139edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Return the cumulative line number of the line that has just been read. 140edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Before the first line has been read, returns 0. After the last line 141edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep of the last file has been read, returns the line number of that line. 142edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 143edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not _state: 144edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "no active input()" 145edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return _state.lineno() 146edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 147edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef filelineno(): 148edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 149edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Return the line number in the current file. Before the first line 150edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep has been read, returns 0. After the last line of the last file has 151edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep been read, returns the line number of that line within the file. 152edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 153edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not _state: 154edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "no active input()" 155edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return _state.filelineno() 156edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 157edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef fileno(): 158edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 159edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Return the file number of the current file. When no file is currently 160edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep opened, returns -1. 161edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 162edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not _state: 163edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "no active input()" 164edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return _state.fileno() 165edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 166edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef isfirstline(): 167edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 168edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Returns true the line just read is the first line of its file, 169edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep otherwise returns false. 170edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 171edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not _state: 172edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "no active input()" 173edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return _state.isfirstline() 174edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 175edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef isstdin(): 176edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 177edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Returns true if the last line was read from sys.stdin, 178edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep otherwise returns false. 179edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 180edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not _state: 181edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "no active input()" 182edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return _state.isstdin() 183edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 184edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepclass FileInput: 185edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """class FileInput([files[, inplace[, backup[, mode[, openhook]]]]]) 186edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 187edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep Class FileInput is the implementation of the module; its methods 188edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep filename(), lineno(), fileline(), isfirstline(), isstdin(), fileno(), 189edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep nextfile() and close() correspond to the functions of the same name 190edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep in the module. 191edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep In addition it has a readline() method which returns the next 192edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep input line, and a __getitem__() method which implements the 193edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep sequence behavior. The sequence must be accessed in strictly 194edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep sequential order; random access and readline() cannot be mixed. 195edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep """ 196edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 197edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def __init__(self, files=None, inplace=0, backup="", bufsize=0, 198edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep mode="r", openhook=None): 199edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if isinstance(files, basestring): 200edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep files = (files,) 201edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 202edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if files is None: 203edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep files = sys.argv[1:] 204edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not files: 205edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep files = ('-',) 206edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 207edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep files = tuple(files) 208edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._files = files 209edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._inplace = inplace 210edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._backup = backup 211edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._bufsize = bufsize or DEFAULT_BUFSIZE 212edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._savestdout = None 213edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._output = None 214edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._filename = None 215edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._lineno = 0 216edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._filelineno = 0 217edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._file = None 218edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._isstdin = False 219edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._backupfilename = None 220edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._buffer = [] 221edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._bufindex = 0 222edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep # restrict mode argument to reading modes 223edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if mode not in ('r', 'rU', 'U', 'rb'): 224edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise ValueError("FileInput opening mode must be one of " 225edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep "'r', 'rU', 'U' and 'rb'") 226edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._mode = mode 227edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if inplace and openhook: 228edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise ValueError("FileInput cannot use an opening hook in inplace mode") 229edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep elif openhook and not hasattr(openhook, '__call__'): 230edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise ValueError("FileInput openhook must be callable") 231edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._openhook = openhook 232edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 233edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def __del__(self): 234edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self.close() 235edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 236edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def close(self): 237edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self.nextfile() 238edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._files = () 239edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 240edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def __iter__(self): 241edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self 242edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 243edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def next(self): 244edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep try: 245edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep line = self._buffer[self._bufindex] 246edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep except IndexError: 247edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep pass 248edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 249edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._bufindex += 1 250edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._lineno += 1 251edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._filelineno += 1 252edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return line 253edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep line = self.readline() 254edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not line: 255edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise StopIteration 256edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return line 257edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 258edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def __getitem__(self, i): 259edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if i != self._lineno: 260edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise RuntimeError, "accessing lines out of order" 261edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep try: 262edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self.next() 263edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep except StopIteration: 264edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep raise IndexError, "end of input reached" 265edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 266edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def nextfile(self): 267edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep savestdout = self._savestdout 268edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._savestdout = 0 269edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if savestdout: 270edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep sys.stdout = savestdout 271edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 272edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep output = self._output 273edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._output = 0 274edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if output: 275edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep output.close() 276edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 277edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep file = self._file 278edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._file = 0 279edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if file and not self._isstdin: 280edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep file.close() 281edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 282edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep backupfilename = self._backupfilename 283edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._backupfilename = 0 284edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if backupfilename and not self._backup: 285edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep try: os.unlink(backupfilename) 286edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep except OSError: pass 287edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 288edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._isstdin = False 289edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._buffer = [] 290edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._bufindex = 0 291edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 292edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def readline(self): 293edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep try: 294edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep line = self._buffer[self._bufindex] 295edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep except IndexError: 296edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep pass 297edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 298edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._bufindex += 1 299edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._lineno += 1 300edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._filelineno += 1 301edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return line 302edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not self._file: 303edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not self._files: 304edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return "" 305edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._filename = self._files[0] 306edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._files = self._files[1:] 307edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._filelineno = 0 308edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._file = None 309edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._isstdin = False 310edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._backupfilename = 0 311edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if self._filename == '-': 312edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._filename = '<stdin>' 313edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._file = sys.stdin 314edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._isstdin = True 315edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 316edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if self._inplace: 317edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._backupfilename = ( 318edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._filename + (self._backup or os.extsep+"bak")) 319edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep try: os.unlink(self._backupfilename) 320edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep except os.error: pass 321edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep # The next few lines may raise IOError 322edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep os.rename(self._filename, self._backupfilename) 323edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._file = open(self._backupfilename, self._mode) 324edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep try: 325edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep perm = os.fstat(self._file.fileno()).st_mode 326edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep except OSError: 327edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._output = open(self._filename, "w") 328edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 329edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep fd = os.open(self._filename, 330edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep os.O_CREAT | os.O_WRONLY | os.O_TRUNC, 331edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep perm) 332edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._output = os.fdopen(fd, "w") 333edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep try: 334edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if hasattr(os, 'chmod'): 335edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep os.chmod(self._filename, perm) 336edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep except OSError: 337edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep pass 338edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._savestdout = sys.stdout 339edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep sys.stdout = self._output 340edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 341edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep # This may raise IOError 342edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if self._openhook: 343edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._file = self._openhook(self._filename, self._mode) 344edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 345edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._file = open(self._filename, self._mode) 346edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._buffer = self._file.readlines(self._bufsize) 347edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self._bufindex = 0 348edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if not self._buffer: 349edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep self.nextfile() 350edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep # Recursive call 351edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self.readline() 352edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 353edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def filename(self): 354edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self._filename 355edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 356edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def lineno(self): 357edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self._lineno 358edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 359edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def filelineno(self): 360edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self._filelineno 361edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 362edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def fileno(self): 363edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if self._file: 364edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep try: 365edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self._file.fileno() 366edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep except ValueError: 367edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return -1 368edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 369edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return -1 370edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 371edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def isfirstline(self): 372edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self._filelineno == 1 373edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 374edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def isstdin(self): 375edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return self._isstdin 376edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 377edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 378edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef hook_compressed(filename, mode): 379edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep ext = os.path.splitext(filename)[1] 380edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if ext == '.gz': 381edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep import gzip 382edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return gzip.open(filename, mode) 383edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep elif ext == '.bz2': 384edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep import bz2 385edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return bz2.BZ2File(filename, mode) 386edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep else: 387edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return open(filename, mode) 388edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 389edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 390edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef hook_encoded(encoding): 391edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep import codecs 392edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep def openhook(filename, mode): 393edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return codecs.open(filename, mode, encoding) 394edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep return openhook 395edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 396edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 397edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepdef _test(): 398edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep import getopt 399edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep inplace = 0 400edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep backup = 0 401edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep opts, args = getopt.getopt(sys.argv[1:], "ib:") 402edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep for o, a in opts: 403edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if o == '-i': inplace = 1 404edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if o == '-b': backup = a 405edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep for line in input(args, inplace=inplace, backup=backup): 406edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if line[-1:] == '\n': line = line[:-1] 407edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep if line[-1:] == '\r': line = line[:-1] 408edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep print "%d: %s[%d]%s %s" % (lineno(), filename(), filelineno(), 409edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep isfirstline() and "*" or "", line) 410edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep print "%d: %s[%d]" % (lineno(), filename(), filelineno()) 411edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep 412edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoepif __name__ == '__main__': 413edbb763a2b63074cd468a5d33a17908b2cc0654Jeff Vander Stoep _test() 414