1ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# -*- Mode: Python; tab-width: 4 -*-
2ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot#       Id: asynchat.py,v 2.26 2000/09/07 22:29:26 rushing Exp
3ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot#       Author: Sam Rushing <rushing@nightmare.com>
4ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
5ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# ======================================================================
6ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Copyright 1996 by Sam Rushing
7ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot#
8ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot#                         All Rights Reserved
9ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot#
10ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Permission to use, copy, modify, and distribute this software and
11ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# its documentation for any purpose and without fee is hereby
12ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# granted, provided that the above copyright notice appear in all
13ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# copies and that both that copyright notice and this permission
14ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# notice appear in supporting documentation, and that the name of Sam
15ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Rushing not be used in advertising or publicity pertaining to
16ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# distribution of the software without specific, written prior
17ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# permission.
18ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot#
19ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
20ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN
21ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR
22ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
23ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
24ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
25ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
26ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# ======================================================================
27ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
28ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotr"""A class supporting chat-style (command/response) protocols.
29ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
30ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team RobotThis class adds support for 'chat' style protocols - where one side
31ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotsends a 'command', and the other sends a response (examples would be
32ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotthe common internet protocols - smtp, nntp, ftp, etc..).
33ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
34ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team RobotThe handle_read() method looks at the input stream for the current
35ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot'terminator' (usually '\r\n' for single-line responses, '\r\n.\r\n'
36ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfor multi-line output), calling self.found_terminator() on its
37ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotreceipt.
38ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
39ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfor example:
40ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team RobotSay you build an async nntp client using this class.  At the start
41ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotof the connection, you'll have self.terminator set to '\r\n', in
42ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotorder to process the single-line greeting.  Just before issuing a
43ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot'LIST' command you'll set it to '\r\n.\r\n'.  The output of the LIST
44ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotcommand will be accumulated (using your own 'collect_incoming_data'
45ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotmethod) up to the terminator, and then control will be returned to
46ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotyou - by calling your self.found_terminator() method.
47ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot"""
48ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
49ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotimport socket
50ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotimport asyncore
51ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfrom collections import deque
52ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfrom sys import py3kwarning
53ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotfrom warnings import filterwarnings, catch_warnings
54ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
55ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotclass async_chat (asyncore.dispatcher):
56ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    """This is an abstract class.  You must derive from this class, and add
57ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    the two methods collect_incoming_data() and found_terminator()"""
58ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
59ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    # these are overridable defaults
60ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
61ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    ac_in_buffer_size       = 4096
62ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    ac_out_buffer_size      = 4096
63ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
64ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def __init__ (self, sock=None, map=None):
65ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # for string terminator matching
66ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.ac_in_buffer = ''
67ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
68ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # we use a list here rather than cStringIO for a few reasons...
69ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # del lst[:] is faster than sio.truncate(0)
70ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # lst = [] is faster than sio.truncate(0)
71ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # cStringIO will be gaining unicode support in py3k, which
72ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # will negatively affect the performance of bytes compared to
73ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # a ''.join() equivalent
74ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.incoming = []
75ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
76ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # we toss the use of the "simple producer" and replace it with
77ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # a pure deque, which the original fifo was a wrapping of
78ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.producer_fifo = deque()
79ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        asyncore.dispatcher.__init__ (self, sock, map)
80ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
81ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def collect_incoming_data(self, data):
82ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        raise NotImplementedError("must be implemented in subclass")
83ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
84ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def _collect_incoming_data(self, data):
85ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.incoming.append(data)
86ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
87ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def _get_data(self):
88ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        d = ''.join(self.incoming)
89ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        del self.incoming[:]
90ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        return d
91ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
92ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def found_terminator(self):
93ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        raise NotImplementedError("must be implemented in subclass")
94ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
95ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def set_terminator (self, term):
96ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        "Set the input delimiter.  Can be a fixed string of any length, an integer, or None"
97ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.terminator = term
98ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
99ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def get_terminator (self):
100ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        return self.terminator
101ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
102ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    # grab some more data from the socket,
103ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    # throw it to the collector method,
104ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    # check for the terminator,
105ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    # if found, transition to the next state.
106ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
107ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def handle_read (self):
108ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
109ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        try:
110ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            data = self.recv (self.ac_in_buffer_size)
111ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        except socket.error, why:
112ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            self.handle_error()
113ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            return
114ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
115ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.ac_in_buffer = self.ac_in_buffer + data
116ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
117ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # Continue to search for self.terminator in self.ac_in_buffer,
118ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # while calling self.collect_incoming_data.  The while loop
119ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # is necessary because we might read several data+terminator
120ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # combos with a single recv(4096).
121ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
122ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        while self.ac_in_buffer:
123ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            lb = len(self.ac_in_buffer)
124ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            terminator = self.get_terminator()
125ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            if not terminator:
126ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                # no terminator, collect it all
127ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                self.collect_incoming_data (self.ac_in_buffer)
128ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                self.ac_in_buffer = ''
129ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            elif isinstance(terminator, int) or isinstance(terminator, long):
130ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                # numeric terminator
131ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                n = terminator
132ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                if lb < n:
133ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.collect_incoming_data (self.ac_in_buffer)
134ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.ac_in_buffer = ''
135ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.terminator = self.terminator - lb
136ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                else:
137ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.collect_incoming_data (self.ac_in_buffer[:n])
138ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.ac_in_buffer = self.ac_in_buffer[n:]
139ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.terminator = 0
140ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.found_terminator()
141ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            else:
142ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                # 3 cases:
143ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                # 1) end of buffer matches terminator exactly:
144ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                #    collect data, transition
145ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                # 2) end of buffer matches some prefix:
146ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                #    collect data to the prefix
147ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                # 3) end of buffer does not match any prefix:
148ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                #    collect data
149ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                terminator_len = len(terminator)
150ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                index = self.ac_in_buffer.find(terminator)
151ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                if index != -1:
152ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    # we found the terminator
153ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    if index > 0:
154ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                        # don't bother reporting the empty string (source of subtle bugs)
155ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                        self.collect_incoming_data (self.ac_in_buffer[:index])
156ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.ac_in_buffer = self.ac_in_buffer[index+terminator_len:]
157ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    # This does the Right Thing if the terminator is changed here.
158ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.found_terminator()
159ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                else:
160ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    # check for a prefix of the terminator
161ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    index = find_prefix_at_end (self.ac_in_buffer, terminator)
162ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    if index:
163ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                        if index != lb:
164ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                            # we found a prefix, collect up to the prefix
165ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                            self.collect_incoming_data (self.ac_in_buffer[:-index])
166ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                            self.ac_in_buffer = self.ac_in_buffer[-index:]
167ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                        break
168ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    else:
169ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                        # no prefix, collect it all
170ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                        self.collect_incoming_data (self.ac_in_buffer)
171ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                        self.ac_in_buffer = ''
172ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
173ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def handle_write (self):
174ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.initiate_send()
175ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
176ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def handle_close (self):
177ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.close()
178ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
179ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def push (self, data):
180ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        sabs = self.ac_out_buffer_size
181ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        if len(data) > sabs:
182ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            for i in xrange(0, len(data), sabs):
183ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                self.producer_fifo.append(data[i:i+sabs])
184ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        else:
185ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            self.producer_fifo.append(data)
186ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.initiate_send()
187ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
188ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def push_with_producer (self, producer):
189ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.producer_fifo.append(producer)
190ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.initiate_send()
191ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
192ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def readable (self):
193ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        "predicate for inclusion in the readable for select()"
194ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # cannot use the old predicate, it violates the claim of the
195ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # set_terminator method.
196ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
197ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # return (len(self.ac_in_buffer) <= self.ac_in_buffer_size)
198ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        return 1
199ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
200ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def writable (self):
201ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        "predicate for inclusion in the writable for select()"
202ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        return self.producer_fifo or (not self.connected)
203ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
204ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def close_when_done (self):
205ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        "automatically close this channel once the outgoing queue is empty"
206ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.producer_fifo.append(None)
207ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
208ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def initiate_send(self):
209ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        while self.producer_fifo and self.connected:
210ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            first = self.producer_fifo[0]
211ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            # handle empty string/buffer or None entry
212ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            if not first:
213ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                del self.producer_fifo[0]
214ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                if first is None:
215ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.handle_close()
216ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    return
217ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
218ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            # handle classic producer behavior
219ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            obs = self.ac_out_buffer_size
220ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            try:
221ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                with catch_warnings():
222ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    if py3kwarning:
223ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                        filterwarnings("ignore", ".*buffer", DeprecationWarning)
224ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    data = buffer(first, 0, obs)
225ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            except TypeError:
226ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                data = first.more()
227ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                if data:
228ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.producer_fifo.appendleft(data)
229ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                else:
230ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    del self.producer_fifo[0]
231ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                continue
232ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
233ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            # send the data
234ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            try:
235ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                num_sent = self.send(data)
236ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            except socket.error:
237ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                self.handle_error()
238ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                return
239ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
240ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            if num_sent:
241ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                if num_sent < len(data) or obs < len(first):
242ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    self.producer_fifo[0] = first[num_sent:]
243ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                else:
244ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot                    del self.producer_fifo[0]
245ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            # we tried to send some actual data
246ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            return
247ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
248ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def discard_buffers (self):
249ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        # Emergencies only!
250ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.ac_in_buffer = ''
251ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        del self.incoming[:]
252ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.producer_fifo.clear()
253ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
254ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotclass simple_producer:
255ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
256ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def __init__ (self, data, buffer_size=512):
257ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.data = data
258ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.buffer_size = buffer_size
259ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
260ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def more (self):
261ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        if len (self.data) > self.buffer_size:
262ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            result = self.data[:self.buffer_size]
263ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            self.data = self.data[self.buffer_size:]
264ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            return result
265ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        else:
266ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            result = self.data
267ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            self.data = ''
268ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            return result
269ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
270ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotclass fifo:
271ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def __init__ (self, list=None):
272ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        if not list:
273ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            self.list = deque()
274ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        else:
275ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            self.list = deque(list)
276ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
277ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def __len__ (self):
278ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        return len(self.list)
279ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
280ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def is_empty (self):
281ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        return not self.list
282ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
283ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def first (self):
284ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        return self.list[0]
285ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
286ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def push (self, data):
287ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        self.list.append(data)
288ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
289ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    def pop (self):
290ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        if self.list:
291ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            return (1, self.list.popleft())
292ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        else:
293ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot            return (0, None)
294ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
295ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# Given 'haystack', see if any prefix of 'needle' is at its end.  This
296ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# assumes an exact match has already been checked.  Return the number of
297ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# characters matched.
298ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# for example:
299ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# f_p_a_e ("qwerty\r", "\r\n") => 1
300ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# f_p_a_e ("qwertydkjf", "\r\n") => 0
301ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# f_p_a_e ("qwerty\r\n", "\r\n") => <undefined>
302ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
303ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# this could maybe be made faster with a computed regex?
304ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# [answer: no; circa Python-2.0, Jan 2001]
305ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# new python:   28961/s
306ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# old python:   18307/s
307ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# re:        12820/s
308ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot# regex:     14035/s
309ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot
310ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robotdef find_prefix_at_end (haystack, needle):
311ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    l = len(needle) - 1
312ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    while l and not haystack.endswith(needle[:l]):
313ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot        l -= 1
314ec1a0b3abe08fb9a3952e8f48231cda1f6d9b1fandroid-build-team Robot    return l
315