1ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# -*- Mode: Python; tab-width: 4 -*- 2ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Id: asynchat.py,v 2.26 2000/09/07 22:29:26 rushing Exp 3ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Author: Sam Rushing <rushing@nightmare.com> 4ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 5ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# ====================================================================== 6ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Copyright 1996 by Sam Rushing 7ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# 8ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# All Rights Reserved 9ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# 10ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Permission to use, copy, modify, and distribute this software and 11ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# its documentation for any purpose and without fee is hereby 12ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# granted, provided that the above copyright notice appear in all 13ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# copies and that both that copyright notice and this permission 14ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# notice appear in supporting documentation, and that the name of Sam 15ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Rushing not be used in advertising or publicity pertaining to 16ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# distribution of the software without specific, written prior 17ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# permission. 18ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# 19ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, 20ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN 21ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR 22ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS 23ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, 24ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN 25ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 26ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# ====================================================================== 27ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 28ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotr"""A class supporting chat-style (command/response) protocols. 29ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 30ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team RobotThis class adds support for 'chat' style protocols - where one side 31ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotsends a 'command', and the other sends a response (examples would be 32ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotthe common internet protocols - smtp, nntp, ftp, etc..). 33ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 34ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team RobotThe handle_read() method looks at the input stream for the current 35ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot'terminator' (usually '\r\n' for single-line responses, '\r\n.\r\n' 36ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfor multi-line output), calling self.found_terminator() on its 37ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotreceipt. 38ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 39ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfor example: 40ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team RobotSay you build an async nntp client using this class. At the start 41ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotof the connection, you'll have self.terminator set to '\r\n', in 42ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotorder to process the single-line greeting. Just before issuing a 43ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot'LIST' command you'll set it to '\r\n.\r\n'. The output of the LIST 44ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotcommand will be accumulated (using your own 'collect_incoming_data' 45ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotmethod) up to the terminator, and then control will be returned to 46ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotyou - by calling your self.found_terminator() method. 47ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot""" 48ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 49ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotimport socket 50ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotimport asyncore 51ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfrom collections import deque 52ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfrom sys import py3kwarning 53ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfrom warnings import filterwarnings, catch_warnings 54ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 55ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotclass async_chat (asyncore.dispatcher): 56ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot """This is an abstract class. You must derive from this class, and add 57ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot the two methods collect_incoming_data() and found_terminator()""" 58ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 59ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # these are overridable defaults 60ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 61ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot ac_in_buffer_size = 4096 62ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot ac_out_buffer_size = 4096 63ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 64ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def __init__ (self, sock=None, map=None): 65ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # for string terminator matching 66ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = '' 67ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 68ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # we use a list here rather than cStringIO for a few reasons... 69ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # del lst[:] is faster than sio.truncate(0) 70ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # lst = [] is faster than sio.truncate(0) 71ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # cStringIO will be gaining unicode support in py3k, which 72ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # will negatively affect the performance of bytes compared to 73ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # a ''.join() equivalent 74ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.incoming = [] 75ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 76ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # we toss the use of the "simple producer" and replace it with 77ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # a pure deque, which the original fifo was a wrapping of 78ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.producer_fifo = deque() 79ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot asyncore.dispatcher.__init__ (self, sock, map) 80ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 81ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def collect_incoming_data(self, data): 82ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot raise NotImplementedError("must be implemented in subclass") 83ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 84ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def _collect_incoming_data(self, data): 85ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.incoming.append(data) 86ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 87ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def _get_data(self): 88ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot d = ''.join(self.incoming) 89ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot del self.incoming[:] 90ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return d 91ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 92ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def found_terminator(self): 93ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot raise NotImplementedError("must be implemented in subclass") 94ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 95ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def set_terminator (self, term): 96ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot "Set the input delimiter. Can be a fixed string of any length, an integer, or None" 97ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.terminator = term 98ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 99ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def get_terminator (self): 100ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return self.terminator 101ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 102ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # grab some more data from the socket, 103ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # throw it to the collector method, 104ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # check for the terminator, 105ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # if found, transition to the next state. 106ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 107ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def handle_read (self): 108ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 109ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot try: 110ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot data = self.recv (self.ac_in_buffer_size) 111ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot except socket.error, why: 112ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.handle_error() 113ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return 114ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 115ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = self.ac_in_buffer + data 116ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 117ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # Continue to search for self.terminator in self.ac_in_buffer, 118ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # while calling self.collect_incoming_data. The while loop 119ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # is necessary because we might read several data+terminator 120ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # combos with a single recv(4096). 121ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 122ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot while self.ac_in_buffer: 123ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot lb = len(self.ac_in_buffer) 124ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot terminator = self.get_terminator() 125ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if not terminator: 126ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # no terminator, collect it all 127ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.collect_incoming_data (self.ac_in_buffer) 128ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = '' 129ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot elif isinstance(terminator, int) or isinstance(terminator, long): 130ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # numeric terminator 131ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot n = terminator 132ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if lb < n: 133ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.collect_incoming_data (self.ac_in_buffer) 134ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = '' 135ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.terminator = self.terminator - lb 136ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 137ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.collect_incoming_data (self.ac_in_buffer[:n]) 138ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = self.ac_in_buffer[n:] 139ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.terminator = 0 140ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.found_terminator() 141ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 142ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # 3 cases: 143ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # 1) end of buffer matches terminator exactly: 144ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # collect data, transition 145ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # 2) end of buffer matches some prefix: 146ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # collect data to the prefix 147ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # 3) end of buffer does not match any prefix: 148ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # collect data 149ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot terminator_len = len(terminator) 150ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot index = self.ac_in_buffer.find(terminator) 151ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if index != -1: 152ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # we found the terminator 153ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if index > 0: 154ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # don't bother reporting the empty string (source of subtle bugs) 155ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.collect_incoming_data (self.ac_in_buffer[:index]) 156ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = self.ac_in_buffer[index+terminator_len:] 157ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # This does the Right Thing if the terminator is changed here. 158ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.found_terminator() 159ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 160ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # check for a prefix of the terminator 161ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot index = find_prefix_at_end (self.ac_in_buffer, terminator) 162ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if index: 163ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if index != lb: 164ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # we found a prefix, collect up to the prefix 165ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.collect_incoming_data (self.ac_in_buffer[:-index]) 166ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = self.ac_in_buffer[-index:] 167ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot break 168ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 169ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # no prefix, collect it all 170ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.collect_incoming_data (self.ac_in_buffer) 171ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = '' 172ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 173ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def handle_write (self): 174ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.initiate_send() 175ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 176ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def handle_close (self): 177ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.close() 178ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 179ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def push (self, data): 180ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot sabs = self.ac_out_buffer_size 181ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if len(data) > sabs: 182ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot for i in xrange(0, len(data), sabs): 183ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.producer_fifo.append(data[i:i+sabs]) 184ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 185ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.producer_fifo.append(data) 186ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.initiate_send() 187ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 188ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def push_with_producer (self, producer): 189ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.producer_fifo.append(producer) 190ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.initiate_send() 191ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 192ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def readable (self): 193ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot "predicate for inclusion in the readable for select()" 194ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # cannot use the old predicate, it violates the claim of the 195ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # set_terminator method. 196ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 197ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # return (len(self.ac_in_buffer) <= self.ac_in_buffer_size) 198ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return 1 199ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 200ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def writable (self): 201ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot "predicate for inclusion in the writable for select()" 202ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return self.producer_fifo or (not self.connected) 203ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 204ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def close_when_done (self): 205ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot "automatically close this channel once the outgoing queue is empty" 206ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.producer_fifo.append(None) 207ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 208ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def initiate_send(self): 209ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot while self.producer_fifo and self.connected: 210ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot first = self.producer_fifo[0] 211ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # handle empty string/buffer or None entry 212ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if not first: 213ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot del self.producer_fifo[0] 214ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if first is None: 215ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.handle_close() 216ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return 217ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 218ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # handle classic producer behavior 219ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot obs = self.ac_out_buffer_size 220ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot try: 221ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot with catch_warnings(): 222ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if py3kwarning: 223ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot filterwarnings("ignore", ".*buffer", DeprecationWarning) 224ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot data = buffer(first, 0, obs) 225ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot except TypeError: 226ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot data = first.more() 227ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if data: 228ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.producer_fifo.appendleft(data) 229ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 230ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot del self.producer_fifo[0] 231ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot continue 232ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 233ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # send the data 234ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot try: 235ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot num_sent = self.send(data) 236ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot except socket.error: 237ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.handle_error() 238ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return 239ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 240ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if num_sent: 241ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if num_sent < len(data) or obs < len(first): 242ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.producer_fifo[0] = first[num_sent:] 243ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 244ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot del self.producer_fifo[0] 245ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # we tried to send some actual data 246ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return 247ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 248ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def discard_buffers (self): 249ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot # Emergencies only! 250ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.ac_in_buffer = '' 251ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot del self.incoming[:] 252ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.producer_fifo.clear() 253ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 254ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotclass simple_producer: 255ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 256ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def __init__ (self, data, buffer_size=512): 257ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.data = data 258ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.buffer_size = buffer_size 259ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 260ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def more (self): 261ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if len (self.data) > self.buffer_size: 262ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot result = self.data[:self.buffer_size] 263ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.data = self.data[self.buffer_size:] 264ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return result 265ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 266ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot result = self.data 267ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.data = '' 268ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return result 269ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 270ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotclass fifo: 271ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def __init__ (self, list=None): 272ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if not list: 273ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.list = deque() 274ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 275ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.list = deque(list) 276ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 277ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def __len__ (self): 278ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return len(self.list) 279ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 280ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def is_empty (self): 281ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return not self.list 282ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 283ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def first (self): 284ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return self.list[0] 285ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 286ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def push (self, data): 287ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot self.list.append(data) 288ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 289ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot def pop (self): 290ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot if self.list: 291ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return (1, self.list.popleft()) 292ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot else: 293ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return (0, None) 294ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 295ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Given 'haystack', see if any prefix of 'needle' is at its end. This 296ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# assumes an exact match has already been checked. Return the number of 297ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# characters matched. 298ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# for example: 299ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# f_p_a_e ("qwerty\r", "\r\n") => 1 300ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# f_p_a_e ("qwertydkjf", "\r\n") => 0 301ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# f_p_a_e ("qwerty\r\n", "\r\n") => <undefined> 302ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 303ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# this could maybe be made faster with a computed regex? 304ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# [answer: no; circa Python-2.0, Jan 2001] 305ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# new python: 28961/s 306ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# old python: 18307/s 307ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# re: 12820/s 308ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# regex: 14035/s 309ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot 310ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotdef find_prefix_at_end (haystack, needle): 311ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot l = len(needle) - 1 312ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot while l and not haystack.endswith(needle[:l]): 313ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot l -= 1 314ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot return l 315