183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehr"""HTTP/1.1 client library 283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh<intro stuff goes here> 483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh<other stuff, too> 583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehHTTPConnection goes through a number of "states", which define when a client 783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehmay legally make another request or fetch the response for a particular 883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehrequest. This diagram details these state transitions: 983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 1083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh (null) 1183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | 1283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | HTTPConnection() 1383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh v 1483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Idle 1583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | 1683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | putrequest() 1783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh v 1883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Request-started 1983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | 2083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | ( putheader() )* endheaders() 2183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh v 2283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Request-sent 2383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | 2483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | response = getresponse() 2583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh v 2683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Unread-response [Response-headers-read] 2783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh |\____________________ 2883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | | 2983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | response.read() | putrequest() 3083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh v v 3183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Idle Req-started-unread-response 3283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh ______/| 3383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh / | 3483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh response.read() | | ( putheader() )* endheaders() 3583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh v v 3683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Request-started Req-sent-unread-response 3783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | 3883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh | response.read() 3983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh v 4083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Request-sent 4183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 4283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehThis diagram presents the following rules: 4383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh -- a second request may not be started until {response-headers-read} 4483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh -- a response [object] cannot be retrieved until {request-sent} 4583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh -- there is no differentiation between an unread response body and a 4683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh partially read response body 4783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 4883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehNote: this enforcement is applied by the HTTPConnection class. The 4983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh HTTPResponse class does not enforce this state machine, which 5083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh implies sophisticated clients may accelerate the request/response 5183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pipeline. Caution should be taken, though: accelerating the states 5283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh beyond the above pattern may imply knowledge of the server's 5383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh connection-close behavior for certain requests. For example, it 5483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh is impossible to tell whether the server will close the connection 5583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh UNTIL the response headers have been read; this means that further 5683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh requests cannot be placed into the pipeline until it is known that 5783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh the server will NOT be closing the connection. 5883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 5983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehLogical State __state __response 6083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh------------- ------- ---------- 6183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehIdle _CS_IDLE None 6283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehRequest-started _CS_REQ_STARTED None 6383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehRequest-sent _CS_REQ_SENT None 6483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehUnread-response _CS_IDLE <response_class> 6583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehReq-started-unread-response _CS_REQ_STARTED <response_class> 6683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehReq-sent-unread-response _CS_REQ_SENT <response_class> 6783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh""" 6883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 6983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehfrom array import array 7083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehimport os 7183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehimport socket 7283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehfrom sys import py3kwarning 7383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehfrom urlparse import urlsplit 7483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehimport warnings 7583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehwith warnings.catch_warnings(): 7683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if py3kwarning: 7783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh warnings.filterwarnings("ignore", ".*mimetools has been removed", 7883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh DeprecationWarning) 7983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh import mimetools 8083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 8183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehtry: 8283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh from cStringIO import StringIO 8383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehexcept ImportError: 8483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh from StringIO import StringIO 8583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 8683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh__all__ = ["HTTP", "HTTPResponse", "HTTPConnection", 8783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "HTTPException", "NotConnected", "UnknownProtocol", 8883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "UnknownTransferEncoding", "UnimplementedFileMode", 8983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "IncompleteRead", "InvalidURL", "ImproperConnectionState", 9083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "CannotSendRequest", "CannotSendHeader", "ResponseNotReady", 9183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "BadStatusLine", "error", "responses"] 9283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 9383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehHTTP_PORT = 80 9483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehHTTPS_PORT = 443 9583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 9683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh_UNKNOWN = 'UNKNOWN' 9783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 9883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# connection states 9983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh_CS_IDLE = 'Idle' 10083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh_CS_REQ_STARTED = 'Request-started' 10183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh_CS_REQ_SENT = 'Request-sent' 10283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 10383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# status codes 10483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# informational 10583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehCONTINUE = 100 10683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehSWITCHING_PROTOCOLS = 101 10783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehPROCESSING = 102 10883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 10983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# successful 11083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehOK = 200 11183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehCREATED = 201 11283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehACCEPTED = 202 11383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehNON_AUTHORITATIVE_INFORMATION = 203 11483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehNO_CONTENT = 204 11583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehRESET_CONTENT = 205 11683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehPARTIAL_CONTENT = 206 11783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehMULTI_STATUS = 207 11883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehIM_USED = 226 11983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 12083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# redirection 12183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehMULTIPLE_CHOICES = 300 12283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehMOVED_PERMANENTLY = 301 12383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehFOUND = 302 12483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehSEE_OTHER = 303 12583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehNOT_MODIFIED = 304 12683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehUSE_PROXY = 305 12783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehTEMPORARY_REDIRECT = 307 12883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 12983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# client error 13083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehBAD_REQUEST = 400 13183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehUNAUTHORIZED = 401 13283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehPAYMENT_REQUIRED = 402 13383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehFORBIDDEN = 403 13483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehNOT_FOUND = 404 13583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehMETHOD_NOT_ALLOWED = 405 13683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehNOT_ACCEPTABLE = 406 13783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehPROXY_AUTHENTICATION_REQUIRED = 407 13883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehREQUEST_TIMEOUT = 408 13983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehCONFLICT = 409 14083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehGONE = 410 14183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehLENGTH_REQUIRED = 411 14283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehPRECONDITION_FAILED = 412 14383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehREQUEST_ENTITY_TOO_LARGE = 413 14483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehREQUEST_URI_TOO_LONG = 414 14583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehUNSUPPORTED_MEDIA_TYPE = 415 14683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehREQUESTED_RANGE_NOT_SATISFIABLE = 416 14783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehEXPECTATION_FAILED = 417 14883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehUNPROCESSABLE_ENTITY = 422 14983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehLOCKED = 423 15083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehFAILED_DEPENDENCY = 424 15183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehUPGRADE_REQUIRED = 426 15283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 15383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# server error 15483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehINTERNAL_SERVER_ERROR = 500 15583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehNOT_IMPLEMENTED = 501 15683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehBAD_GATEWAY = 502 15783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehSERVICE_UNAVAILABLE = 503 15883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehGATEWAY_TIMEOUT = 504 15983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehHTTP_VERSION_NOT_SUPPORTED = 505 16083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehINSUFFICIENT_STORAGE = 507 16183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehNOT_EXTENDED = 510 16283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 16383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# Mapping status codes to official W3C names 16483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehresponses = { 16583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 100: 'Continue', 16683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 101: 'Switching Protocols', 16783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 16883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 200: 'OK', 16983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 201: 'Created', 17083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 202: 'Accepted', 17183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 203: 'Non-Authoritative Information', 17283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 204: 'No Content', 17383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 205: 'Reset Content', 17483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 206: 'Partial Content', 17583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 17683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 300: 'Multiple Choices', 17783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 301: 'Moved Permanently', 17883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 302: 'Found', 17983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 303: 'See Other', 18083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 304: 'Not Modified', 18183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 305: 'Use Proxy', 18283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 306: '(Unused)', 18383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 307: 'Temporary Redirect', 18483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 18583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 400: 'Bad Request', 18683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 401: 'Unauthorized', 18783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 402: 'Payment Required', 18883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 403: 'Forbidden', 18983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 404: 'Not Found', 19083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 405: 'Method Not Allowed', 19183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 406: 'Not Acceptable', 19283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 407: 'Proxy Authentication Required', 19383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 408: 'Request Timeout', 19483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 409: 'Conflict', 19583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 410: 'Gone', 19683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 411: 'Length Required', 19783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 412: 'Precondition Failed', 19883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 413: 'Request Entity Too Large', 19983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 414: 'Request-URI Too Long', 20083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 415: 'Unsupported Media Type', 20183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 416: 'Requested Range Not Satisfiable', 20283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 417: 'Expectation Failed', 20383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 20483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 500: 'Internal Server Error', 20583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 501: 'Not Implemented', 20683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 502: 'Bad Gateway', 20783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 503: 'Service Unavailable', 20883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 504: 'Gateway Timeout', 20983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 505: 'HTTP Version Not Supported', 21083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh} 21183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 21283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# maximal amount of data to read at one time in _safe_read 21383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew HsiehMAXAMOUNT = 1048576 21483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 21583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# maximal line length when calling readline(). 21683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh_MAXLINE = 65536 21783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 21883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass HTTPMessage(mimetools.Message): 21983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 22083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def addheader(self, key, value): 22183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Add header for field key handling repeats.""" 22283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh prev = self.dict.get(key) 22383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if prev is None: 22483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.dict[key] = value 22583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 22683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh combined = ", ".join((prev, value)) 22783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.dict[key] = combined 22883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 22983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def addcontinue(self, key, more): 23083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Add more field data from a continuation line.""" 23183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh prev = self.dict[key] 23283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.dict[key] = prev + "\n " + more 23383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 23483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def readheaders(self): 23583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Read header lines. 23683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 23783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Read header lines up to the entirely blank line that terminates them. 23883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh The (normally blank) line that ends the headers is skipped, but not 23983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh included in the returned list. If a non-header line ends the headers, 24083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh (which is an error), an attempt is made to backspace over it; it is 24183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh never included in the returned list. 24283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 24383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh The variable self.status is set to the empty string if all went well, 24483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh otherwise it is an error message. The variable self.headers is a 24583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh completely uninterpreted list of lines contained in the header (so 24683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh printing them will reproduce the header exactly as it appears in the 24783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh file). 24883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 24983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh If multiple header fields with the same name occur, they are combined 25083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh according to the rules in RFC 2616 sec 4.2: 25183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 25283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Appending each subsequent field-value to the first, each separated 25383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh by a comma. The order in which header fields with the same field-name 25483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh are received is significant to the interpretation of the combined 25583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh field value. 25683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 25783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # XXX The implementation overrides the readheaders() method of 25883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # rfc822.Message. The base class design isn't amenable to 25983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # customized behavior here so the method here is a copy of the 26083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # base class code with a few small changes. 26183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 26283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.dict = {} 26383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.unixfrom = '' 26483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.headers = hlist = [] 26583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.status = '' 26683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh headerseen = "" 26783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh firstline = 1 26883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh startofline = unread = tell = None 26983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if hasattr(self.fp, 'unread'): 27083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh unread = self.fp.unread 27183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh elif self.seekable: 27283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh tell = self.fp.tell 27383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh while True: 27483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if tell: 27583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 27683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh startofline = tell() 27783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except IOError: 27883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh startofline = tell = None 27983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.seekable = 0 28083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh line = self.fp.readline(_MAXLINE + 1) 28183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if len(line) > _MAXLINE: 28283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise LineTooLong("header line") 28383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not line: 28483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.status = 'EOF in headers' 28583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 28683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Skip unix From name time lines 28783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if firstline and line.startswith('From '): 28883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.unixfrom = self.unixfrom + line 28983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh continue 29083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh firstline = 0 29183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if headerseen and line[0] in ' \t': 29283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # XXX Not sure if continuation lines are handled properly 29383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # for http and/or for repeating headers 29483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # It's a continuation line. 29583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh hlist.append(line) 29683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.addcontinue(headerseen, line.strip()) 29783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh continue 29883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh elif self.iscomment(line): 29983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # It's a comment. Ignore it. 30083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh continue 30183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh elif self.islast(line): 30283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Note! No pushback here! The delimiter line gets eaten. 30383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 30483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh headerseen = self.isheader(line) 30583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if headerseen: 30683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # It's a legal header line, save it. 30783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh hlist.append(line) 30883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.addheader(headerseen, line[len(headerseen)+1:].strip()) 30983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh continue 31083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 31183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # It's not a header line; throw it back and stop here. 31283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not self.dict: 31383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.status = 'No headers' 31483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 31583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.status = 'Non-header line where header expected' 31683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Try to undo the read. 31783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if unread: 31883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh unread(line) 31983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh elif tell: 32083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.fp.seek(startofline) 32183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 32283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.status = self.status + '; bad seek' 32383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 32483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 32583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass HTTPResponse: 32683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 32783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # strict: If true, raise BadStatusLine if the status line can't be 32883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # parsed as a valid HTTP/1.0 or 1.1 status line. By default it is 32983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # false because it prevents clients from talking to HTTP/0.9 33083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # servers. Note that a response with a sufficiently corrupted 33183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # status line will look like an HTTP/0.9 response. 33283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 33383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # See RFC 2616 sec 19.6 and RFC 1945 sec 6 for details. 33483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 33583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, sock, debuglevel=0, strict=0, method=None, buffering=False): 33683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if buffering: 33783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # The caller won't be using any sock.recv() calls, so buffering 33883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # is fine and recommended for performance. 33983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.fp = sock.makefile('rb') 34083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 34183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # The buffer size is specified as zero, because the headers of 34283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # the response are read with readline(). If the reads were 34383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # buffered the readline() calls could consume some of the 34483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # response, which make be read via a recv() on the underlying 34583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # socket. 34683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.fp = sock.makefile('rb', 0) 34783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.debuglevel = debuglevel 34883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.strict = strict 34983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._method = method 35083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 35183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.msg = None 35283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 35383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # from the Status-Line of the response 35483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.version = _UNKNOWN # HTTP-Version 35583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.status = _UNKNOWN # Status-Code 35683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.reason = _UNKNOWN # Reason-Phrase 35783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 35883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.chunked = _UNKNOWN # is "chunked" being used? 35983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.chunk_left = _UNKNOWN # bytes left to read in current chunk 36083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length = _UNKNOWN # number of bytes left in response 36183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.will_close = _UNKNOWN # conn will close at end of response 36283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 36383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _read_status(self): 36483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Initialize with Simple-Response defaults 36583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh line = self.fp.readline(_MAXLINE + 1) 36683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if len(line) > _MAXLINE: 36783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise LineTooLong("header line") 36883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.debuglevel > 0: 36983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh print "reply:", repr(line) 37083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not line: 37183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Presumably, the server closed the connection before 37283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # sending a valid response. 37383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise BadStatusLine(line) 37483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 37583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh [version, status, reason] = line.split(None, 2) 37683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except ValueError: 37783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 37883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh [version, status] = line.split(None, 1) 37983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh reason = "" 38083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except ValueError: 38183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # empty version will cause next test to fail and status 38283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # will be treated as 0.9 response. 38383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh version = "" 38483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not version.startswith('HTTP/'): 38583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.strict: 38683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 38783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise BadStatusLine(line) 38883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 38983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # assume it's a Simple-Response from an 0.9 server 39083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.fp = LineAndFileWrapper(line, self.fp) 39183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return "HTTP/0.9", 200, "" 39283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 39383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # The status code is a three-digit number 39483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 39583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh status = int(status) 39683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if status < 100 or status > 999: 39783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise BadStatusLine(line) 39883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except ValueError: 39983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise BadStatusLine(line) 40083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return version, status, reason 40183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 40283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def begin(self): 40383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.msg is not None: 40483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # we've already started reading the response 40583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return 40683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 40783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # read until we get a non-100 response 40883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh while True: 40983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh version, status, reason = self._read_status() 41083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if status != CONTINUE: 41183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 41283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # skip the header from the 100 response 41383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh while True: 41483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh skip = self.fp.readline(_MAXLINE + 1) 41583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if len(skip) > _MAXLINE: 41683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise LineTooLong("header line") 41783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh skip = skip.strip() 41883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not skip: 41983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 42083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.debuglevel > 0: 42183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh print "header:", skip 42283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 42383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.status = status 42483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.reason = reason.strip() 42583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if version == 'HTTP/1.0': 42683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.version = 10 42783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh elif version.startswith('HTTP/1.'): 42883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.version = 11 # use HTTP/1.1 code for HTTP/1.x where x>=1 42983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh elif version == 'HTTP/0.9': 43083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.version = 9 43183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 43283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise UnknownProtocol(version) 43383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 43483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.version == 9: 43583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length = None 43683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.chunked = 0 43783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.will_close = 1 43883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.msg = HTTPMessage(StringIO()) 43983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return 44083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 44183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.msg = HTTPMessage(self.fp, 0) 44283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.debuglevel > 0: 44383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh for hdr in self.msg.headers: 44483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh print "header:", hdr, 44583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 44683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # don't let the msg keep an fp 44783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.msg.fp = None 44883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 44983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # are we using the chunked-style of transfer encoding? 45083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh tr_enc = self.msg.getheader('transfer-encoding') 45183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if tr_enc and tr_enc.lower() == "chunked": 45283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.chunked = 1 45383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.chunk_left = None 45483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 45583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.chunked = 0 45683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 45783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # will the connection close at the end of the response? 45883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.will_close = self._check_close() 45983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 46083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # do we have a Content-Length? 46183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked" 46283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh length = self.msg.getheader('content-length') 46383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if length and not self.chunked: 46483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 46583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length = int(length) 46683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except ValueError: 46783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length = None 46883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 46983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.length < 0: # ignore nonsensical negative lengths 47083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length = None 47183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 47283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length = None 47383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 47483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # does the body have a fixed length? (of zero) 47583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if (status == NO_CONTENT or status == NOT_MODIFIED or 47683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 100 <= status < 200 or # 1xx codes 47783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._method == 'HEAD'): 47883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length = 0 47983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 48083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # if the connection remains open, and we aren't using chunked, and 48183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # a content-length was not provided, then assume that the connection 48283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # WILL close. 48383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not self.will_close and \ 48483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh not self.chunked and \ 48583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length is None: 48683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.will_close = 1 48783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 48883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _check_close(self): 48983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh conn = self.msg.getheader('connection') 49083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.version == 11: 49183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # An HTTP/1.1 proxy is assumed to stay open unless 49283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # explicitly closed. 49383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh conn = self.msg.getheader('connection') 49483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if conn and "close" in conn.lower(): 49583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return True 49683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return False 49783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 49883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Some HTTP/1.0 implementations have support for persistent 49983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # connections, using rules different than HTTP/1.1. 50083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 50183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # For older HTTP, Keep-Alive indicates persistent connection. 50283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.msg.getheader('keep-alive'): 50383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return False 50483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 50583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # At least Akamai returns a "Connection: Keep-Alive" header, 50683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # which was supposed to be sent by the client. 50783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if conn and "keep-alive" in conn.lower(): 50883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return False 50983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 51083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Proxy-Connection is a netscape hack. 51183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pconn = self.msg.getheader('proxy-connection') 51283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if pconn and "keep-alive" in pconn.lower(): 51383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return False 51483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 51583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # otherwise, assume it will close 51683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return True 51783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 51883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def close(self): 51983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.fp: 52083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.fp.close() 52183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.fp = None 52283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 52383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def isclosed(self): 52483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # NOTE: it is possible that we will not ever call self.close(). This 52583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # case occurs when will_close is TRUE, length is None, and we 52683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # read up to the last byte, but NOT past it. 52783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 52883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # IMPLIES: if will_close is FALSE, then self.close() will ALWAYS be 52983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # called, meaning self.isclosed() is meaningful. 53083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self.fp is None 53183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 53283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # XXX It would be nice to have readline and __iter__ for this, too. 53383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 53483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def read(self, amt=None): 53583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.fp is None: 53683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return '' 53783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 53883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self._method == 'HEAD': 53983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 54083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return '' 54183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 54283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.chunked: 54383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self._read_chunked(amt) 54483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 54583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if amt is None: 54683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # unbounded read 54783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.length is None: 54883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh s = self.fp.read() 54983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 55083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 55183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh s = self._safe_read(self.length) 55283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except IncompleteRead: 55383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 55483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise 55583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length = 0 55683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() # we read everything 55783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return s 55883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 55983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.length is not None: 56083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if amt > self.length: 56183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # clip the read to the "end of response" 56283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh amt = self.length 56383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 56483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # we do not use _safe_read() here because this may be a .will_close 56583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # connection, and the user is reading more bytes than will be provided 56683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # (for example, reading in 1k chunks) 56783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh s = self.fp.read(amt) 56883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not s: 56983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Ideally, we would raise IncompleteRead if the content-length 57083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # wasn't satisfied, but it might break compatibility. 57183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 57283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.length is not None: 57383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.length -= len(s) 57483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not self.length: 57583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 57683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 57783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return s 57883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 57983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _read_chunked(self, amt): 58083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh assert self.chunked != _UNKNOWN 58183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh chunk_left = self.chunk_left 58283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh value = [] 58383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh while True: 58483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if chunk_left is None: 58583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh line = self.fp.readline(_MAXLINE + 1) 58683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if len(line) > _MAXLINE: 58783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise LineTooLong("chunk size") 58883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh i = line.find(';') 58983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if i >= 0: 59083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh line = line[:i] # strip chunk-extensions 59183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 59283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh chunk_left = int(line, 16) 59383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except ValueError: 59483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # close the connection as protocol synchronisation is 59583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # probably lost 59683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 59783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise IncompleteRead(''.join(value)) 59883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if chunk_left == 0: 59983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 60083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if amt is None: 60183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh value.append(self._safe_read(chunk_left)) 60283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh elif amt < chunk_left: 60383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh value.append(self._safe_read(amt)) 60483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.chunk_left = chunk_left - amt 60583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return ''.join(value) 60683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh elif amt == chunk_left: 60783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh value.append(self._safe_read(amt)) 60883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._safe_read(2) # toss the CRLF at the end of the chunk 60983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.chunk_left = None 61083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return ''.join(value) 61183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 61283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh value.append(self._safe_read(chunk_left)) 61383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh amt -= chunk_left 61483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 61583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # we read the whole chunk, get another 61683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._safe_read(2) # toss the CRLF at the end of the chunk 61783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh chunk_left = None 61883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 61983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # read and discard trailer up to the CRLF terminator 62083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh ### note: we shouldn't have any trailers! 62183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh while True: 62283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh line = self.fp.readline(_MAXLINE + 1) 62383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if len(line) > _MAXLINE: 62483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise LineTooLong("trailer line") 62583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not line: 62683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # a vanishingly small number of sites EOF without 62783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # sending the trailer 62883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 62983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if line == '\r\n': 63083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 63183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 63283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # we read everything; close the "file" 63383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 63483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 63583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return ''.join(value) 63683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 63783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _safe_read(self, amt): 63883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Read the number of bytes requested, compensating for partial reads. 63983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 64083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Normally, we have a blocking socket, but a read() can be interrupted 64183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh by a signal (resulting in a partial read). 64283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 64383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Note that we cannot distinguish between EOF and an interrupt when zero 64483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh bytes have been read. IncompleteRead() will be raised in this 64583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh situation. 64683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 64783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh This function should be used when <amt> bytes "should" be present for 64883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh reading. If the bytes are truly not available (due to EOF), then the 64983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh IncompleteRead exception can be used to detect the problem. 65083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 65183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # NOTE(gps): As of svn r74426 socket._fileobject.read(x) will never 65283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # return less than x bytes unless EOF is encountered. It now handles 65383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # signal interruptions (socket.error EINTR) internally. This code 65483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # never caught that exception anyways. It seems largely pointless. 65583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # self.fp.read(amt) will work fine. 65683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh s = [] 65783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh while amt > 0: 65883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh chunk = self.fp.read(min(amt, MAXAMOUNT)) 65983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not chunk: 66083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise IncompleteRead(''.join(s), amt) 66183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh s.append(chunk) 66283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh amt -= len(chunk) 66383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return ''.join(s) 66483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 66583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def fileno(self): 66683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self.fp.fileno() 66783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 66883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def getheader(self, name, default=None): 66983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.msg is None: 67083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise ResponseNotReady() 67183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self.msg.getheader(name, default) 67283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 67383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def getheaders(self): 67483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Return list of (header, value) tuples.""" 67583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.msg is None: 67683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise ResponseNotReady() 67783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self.msg.items() 67883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 67983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 68083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass HTTPConnection: 68183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 68283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh _http_vsn = 11 68383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh _http_vsn_str = 'HTTP/1.1' 68483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 68583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh response_class = HTTPResponse 68683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh default_port = HTTP_PORT 68783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh auto_open = 1 68883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh debuglevel = 0 68983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh strict = 0 69083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 69183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, host, port=None, strict=None, 69283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh timeout=socket._GLOBAL_DEFAULT_TIMEOUT, source_address=None): 69383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.timeout = timeout 69483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.source_address = source_address 69583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.sock = None 69683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._buffer = [] 69783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__response = None 69883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__state = _CS_IDLE 69983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._method = None 70083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel_host = None 70183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel_port = None 70283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel_headers = {} 70383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 70483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._set_hostport(host, port) 70583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if strict is not None: 70683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.strict = strict 70783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 70883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def set_tunnel(self, host, port=None, headers=None): 70983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ Sets up the host and the port for the HTTP CONNECT Tunnelling. 71083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 71183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh The headers argument should be a mapping of extra HTTP headers 71283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh to send with the CONNECT request. 71383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 71483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel_host = host 71583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel_port = port 71683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if headers: 71783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel_headers = headers 71883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 71983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel_headers.clear() 72083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 72183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _set_hostport(self, host, port): 72283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if port is None: 72383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh i = host.rfind(':') 72483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh j = host.rfind(']') # ipv6 addresses have [...] 72583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if i > j: 72683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 72783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh port = int(host[i+1:]) 72883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except ValueError: 72983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if host[i+1:] == "": # http://foo.com:/ == http://foo.com/ 73083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh port = self.default_port 73183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 73283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise InvalidURL("nonnumeric port: '%s'" % host[i+1:]) 73383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh host = host[:i] 73483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 73583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh port = self.default_port 73683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if host and host[0] == '[' and host[-1] == ']': 73783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh host = host[1:-1] 73883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.host = host 73983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.port = port 74083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 74183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def set_debuglevel(self, level): 74283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.debuglevel = level 74383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 74483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _tunnel(self): 74583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._set_hostport(self._tunnel_host, self._tunnel_port) 74683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.send("CONNECT %s:%d HTTP/1.0\r\n" % (self.host, self.port)) 74783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh for header, value in self._tunnel_headers.iteritems(): 74883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.send("%s: %s\r\n" % (header, value)) 74983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.send("\r\n") 75083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh response = self.response_class(self.sock, strict = self.strict, 75183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh method = self._method) 75283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh (version, code, message) = response._read_status() 75383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 75483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if code != 200: 75583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 75683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise socket.error("Tunnel connection failed: %d %s" % (code, 75783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh message.strip())) 75883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh while True: 75983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh line = response.fp.readline(_MAXLINE + 1) 76083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if len(line) > _MAXLINE: 76183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise LineTooLong("header line") 76283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not line: 76383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # for sites which EOF without sending trailer 76483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 76583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if line == '\r\n': 76683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh break 76783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 76883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 76983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def connect(self): 77083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Connect to the host and port specified in __init__.""" 77183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.sock = socket.create_connection((self.host,self.port), 77283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.timeout, self.source_address) 77383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 77483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self._tunnel_host: 77583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel() 77683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 77783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def close(self): 77883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Close the connection to the HTTP server.""" 77983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.sock: 78083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.sock.close() # close it manually... there may be other refs 78183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.sock = None 78283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.__response: 78383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__response.close() 78483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__response = None 78583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__state = _CS_IDLE 78683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 78783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def send(self, data): 78883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Send `data' to the server.""" 78983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.sock is None: 79083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.auto_open: 79183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.connect() 79283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 79383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise NotConnected() 79483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 79583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.debuglevel > 0: 79683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh print "send:", repr(data) 79783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh blocksize = 8192 79883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if hasattr(data,'read') and not isinstance(data, array): 79983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.debuglevel > 0: print "sendIng a read()able" 80083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh datablock = data.read(blocksize) 80183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh while datablock: 80283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.sock.sendall(datablock) 80383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh datablock = data.read(blocksize) 80483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 80583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.sock.sendall(data) 80683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 80783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _output(self, s): 80883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Add a line of output to the current request buffer. 80983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 81083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Assumes that the line does *not* end with \\r\\n. 81183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 81283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._buffer.append(s) 81383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 81483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _send_output(self, message_body=None): 81583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Send the currently buffered request and clear the buffer. 81683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 81783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Appends an extra \\r\\n to the buffer. 81883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh A message_body may be specified, to be appended to the request. 81983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 82083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._buffer.extend(("", "")) 82183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh msg = "\r\n".join(self._buffer) 82283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh del self._buffer[:] 82383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # If msg and message_body are sent in a single send() call, 82483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # it will avoid performance problems caused by the interaction 82583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # between delayed ack and the Nagle algorithm. 82683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if isinstance(message_body, str): 82783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh msg += message_body 82883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh message_body = None 82983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.send(msg) 83083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if message_body is not None: 83183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh #message_body was not a string (i.e. it is a file) and 83283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh #we must run the risk of Nagle 83383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.send(message_body) 83483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 83583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0): 83683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Send a request to the server. 83783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 83883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh `method' specifies an HTTP request method, e.g. 'GET'. 83983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh `url' specifies the object being requested, e.g. '/index.html'. 84083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh `skip_host' if True does not add automatically a 'Host:' header 84183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh `skip_accept_encoding' if True does not add automatically an 84283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 'Accept-Encoding:' header 84383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 84483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 84583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # if a prior response has been completed, then forget about it. 84683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.__response and self.__response.isclosed(): 84783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__response = None 84883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 84983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 85083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # in certain cases, we cannot issue another request on this connection. 85183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # this occurs when: 85283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 1) we are in the process of sending a request. (_CS_REQ_STARTED) 85383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 2) a response to a previous request has signalled that it is going 85483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # to close the connection upon completion. 85583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 3) the headers for the previous response have not been read, thus 85683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # we cannot determine whether point (2) is true. (_CS_REQ_SENT) 85783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 85883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # if there is no prior response, then we can request at will. 85983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 86083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # if point (2) is true, then we will have passed the socket to the 86183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # response (effectively meaning, "there is no prior response"), and 86283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # will open a new one when a new request is made. 86383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 86483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Note: if a prior response exists, then we *can* start a new request. 86583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # We are not allowed to begin fetching the response to this new 86683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # request, however, until that prior response is complete. 86783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 86883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.__state == _CS_IDLE: 86983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__state = _CS_REQ_STARTED 87083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 87183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise CannotSendRequest() 87283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 87383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Save the method we use, we need it later in the response phase 87483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._method = method 87583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not url: 87683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh url = '/' 87783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh hdr = '%s %s %s' % (method, url, self._http_vsn_str) 87883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 87983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._output(hdr) 88083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 88183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self._http_vsn == 11: 88283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Issue some standard headers for better HTTP/1.1 compliance 88383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 88483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not skip_host: 88583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # this header is issued *only* for HTTP/1.1 88683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # connections. more specifically, this means it is 88783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # only issued when the client uses the new 88883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # HTTPConnection() class. backwards-compat clients 88983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # will be using HTTP/1.0 and those clients may be 89083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # issuing this header themselves. we should NOT issue 89183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # it twice; some web servers (such as Apache) barf 89283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # when they see two Host: headers 89383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 89483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # If we need a non-standard port,include it in the 89583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # header. If the request is going through a proxy, 89683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # but the host of the actual URL, not the host of the 89783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # proxy. 89883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 89983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh netloc = '' 90083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if url.startswith('http'): 90183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh nil, netloc, nil, nil, nil = urlsplit(url) 90283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 90383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if netloc: 90483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 90583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh netloc_enc = netloc.encode("ascii") 90683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except UnicodeEncodeError: 90783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh netloc_enc = netloc.encode("idna") 90883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putheader('Host', netloc_enc) 90983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 91083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 91183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh host_enc = self.host.encode("ascii") 91283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except UnicodeEncodeError: 91383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh host_enc = self.host.encode("idna") 91483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Wrap the IPv6 Host Header with [] (RFC 2732) 91583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if host_enc.find(':') >= 0: 91683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh host_enc = "[" + host_enc + "]" 91783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.port == self.default_port: 91883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putheader('Host', host_enc) 91983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 92083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putheader('Host', "%s:%s" % (host_enc, self.port)) 92183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 92283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # note: we are assuming that clients will not attempt to set these 92383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # headers since *this* library must deal with the 92483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # consequences. this also means that when the supporting 92583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # libraries are updated to recognize other forms, then this 92683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # code should be changed (removed or updated). 92783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 92883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # we only want a Content-Encoding of "identity" since we don't 92983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # support encodings such as x-gzip or x-deflate. 93083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not skip_accept_encoding: 93183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putheader('Accept-Encoding', 'identity') 93283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 93383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # we can accept "chunked" Transfer-Encodings, but no others 93483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # NOTE: no TE header implies *only* "chunked" 93583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh #self.putheader('TE', 'chunked') 93683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 93783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # if TE is supplied in the header, then it must appear in a 93883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Connection header. 93983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh #self.putheader('Connection', 'TE') 94083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 94183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 94283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # For HTTP/1.0, the server will assume "not chunked" 94383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 94483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 94583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def putheader(self, header, *values): 94683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Send a request header line to the server. 94783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 94883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh For example: h.putheader('Accept', 'text/html') 94983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 95083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.__state != _CS_REQ_STARTED: 95183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise CannotSendHeader() 95283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 95383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh hdr = '%s: %s' % (header, '\r\n\t'.join([str(v) for v in values])) 95483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._output(hdr) 95583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 95683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def endheaders(self, message_body=None): 95783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Indicate that the last header line has been sent to the server. 95883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 95983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh This method sends the request to the server. The optional 96083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh message_body argument can be used to pass a message body 96183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh associated with the request. The message body will be sent in 96283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh the same packet as the message headers if it is string, otherwise it is 96383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh sent as a separate packet. 96483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 96583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.__state == _CS_REQ_STARTED: 96683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__state = _CS_REQ_SENT 96783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 96883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise CannotSendHeader() 96983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._send_output(message_body) 97083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 97183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def request(self, method, url, body=None, headers={}): 97283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Send a complete request to the server.""" 97383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._send_request(method, url, body, headers) 97483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 97583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _set_content_length(self, body): 97683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Set the content-length based on the body. 97783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh thelen = None 97883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 97983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh thelen = str(len(body)) 98083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except TypeError, te: 98183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # If this is a file-like object, try to 98283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # fstat its file descriptor 98383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 98483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh thelen = str(os.fstat(body.fileno()).st_size) 98583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except (AttributeError, OSError): 98683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Don't send a length if this failed 98783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.debuglevel > 0: print "Cannot stat!!" 98883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 98983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if thelen is not None: 99083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putheader('Content-Length', thelen) 99183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 99283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _send_request(self, method, url, body, headers): 99383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Honor explicitly requested Host: and Accept-Encoding: headers. 99483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh header_names = dict.fromkeys([k.lower() for k in headers]) 99583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh skips = {} 99683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if 'host' in header_names: 99783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh skips['skip_host'] = 1 99883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if 'accept-encoding' in header_names: 99983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh skips['skip_accept_encoding'] = 1 100083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 100183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putrequest(method, url, **skips) 100283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 100383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if body is not None and 'content-length' not in header_names: 100483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._set_content_length(body) 100583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh for hdr, value in headers.iteritems(): 100683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putheader(hdr, value) 100783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.endheaders(body) 100883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 100983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def getresponse(self, buffering=False): 101083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "Get the response from the server." 101183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 101283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # if a prior response has been completed, then forget about it. 101383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.__response and self.__response.isclosed(): 101483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__response = None 101583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 101683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 101783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # if a prior response exists, then it must be completed (otherwise, we 101883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # cannot read this response's header to determine the connection-close 101983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # behavior) 102083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 102183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # note: if a prior response existed, but was connection-close, then the 102283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # socket and response were made independent of this HTTPConnection 102383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # object since a new request requires that we open a whole new 102483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # connection 102583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 102683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # this means the prior response had one of two states: 102783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 1) will_close: this connection was reset and the prior socket and 102883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # response operate independently 102983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 2) persistent: the response was retained and we await its 103083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # isclosed() status to become true. 103183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # 103283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.__state != _CS_REQ_SENT or self.__response: 103383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh raise ResponseNotReady() 103483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 103583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh args = (self.sock,) 103683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh kwds = {"strict":self.strict, "method":self._method} 103783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.debuglevel > 0: 103883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh args += (self.debuglevel,) 103983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if buffering: 104083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh #only add this keyword if non-default, for compatibility with 104183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh #other response_classes. 104283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh kwds["buffering"] = True; 104383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh response = self.response_class(*args, **kwds) 104483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 104583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh response.begin() 104683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh assert response.will_close != _UNKNOWN 104783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__state = _CS_IDLE 104883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 104983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if response.will_close: 105083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # this effectively passes the connection to the response 105183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 105283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 105383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # remember this, so we can tell when it is complete 105483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.__response = response 105583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 105683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return response 105783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 105883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 105983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass HTTP: 106083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "Compatibility class with httplib.py from 1.5." 106183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 106283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh _http_vsn = 10 106383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh _http_vsn_str = 'HTTP/1.0' 106483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 106583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh debuglevel = 0 106683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 106783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh _connection_class = HTTPConnection 106883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 106983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, host='', port=None, strict=None): 107083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "Provide a default host, since the superclass requires one." 107183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 107283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # some joker passed 0 explicitly, meaning default port 107383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if port == 0: 107483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh port = None 107583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 107683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Note that we may pass an empty string as the host; this will raise 107783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # an error when we attempt to connect. Presumably, the client code 107883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # will call connect before then, with a proper host. 107983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._setup(self._connection_class(host, port, strict)) 108083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 108183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _setup(self, conn): 108283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._conn = conn 108383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 108483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # set up delegation to flesh out interface 108583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.send = conn.send 108683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putrequest = conn.putrequest 108783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.putheader = conn.putheader 108883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.endheaders = conn.endheaders 108983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.set_debuglevel = conn.set_debuglevel 109083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 109183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh conn._http_vsn = self._http_vsn 109283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh conn._http_vsn_str = self._http_vsn_str 109383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 109483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.file = None 109583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 109683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def connect(self, host=None, port=None): 109783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "Accept arguments to set the host/port, since the superclass doesn't." 109883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 109983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if host is not None: 110083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._conn._set_hostport(host, port) 110183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._conn.connect() 110283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 110383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def getfile(self): 110483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "Provide a getfile, since the superclass' does not use this concept." 110583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self.file 110683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 110783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def getreply(self, buffering=False): 110883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Compat definition since superclass does not define it. 110983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 111083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Returns a tuple consisting of: 111183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh - server status code (e.g. '200' if all goes well) 111283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh - server "reason" corresponding to status code 111383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh - any RFC822 headers in the response from the server 111483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 111583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh try: 111683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not buffering: 111783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh response = self._conn.getresponse() 111883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 111983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh #only add this keyword if non-default for compatibility 112083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh #with other connection classes 112183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh response = self._conn.getresponse(buffering) 112283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh except BadStatusLine, e: 112383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh ### hmm. if getresponse() ever closes the socket on a bad request, 112483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh ### then we are going to have problems with self.sock 112583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 112683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh ### should we keep this behavior? do people use it? 112783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # keep the socket open (as a file), and return it 112883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.file = self._conn.sock.makefile('rb', 0) 112983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 113083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # close our socket -- we want to restart after any protocol error 113183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.close() 113283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 113383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.headers = None 113483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return -1, e.line, None 113583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 113683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.headers = response.msg 113783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.file = response.fp 113883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return response.status, response.reason, response.msg 113983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 114083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def close(self): 114183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._conn.close() 114283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 114383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # note that self.file == response.fp, which gets closed by the 114483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # superclass. just clear the object ref here. 114583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh ### hmm. messy. if status==-1, then self.file is owned by us. 114683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh ### well... we aren't explicitly closing, but losing this ref will 114783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh ### do it 114883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.file = None 114983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 115083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehtry: 115183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh import ssl 115283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehexcept ImportError: 115383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 115483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehelse: 115583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh class HTTPSConnection(HTTPConnection): 115683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "This class allows communication via SSL." 115783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 115883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh default_port = HTTPS_PORT 115983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 116083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, host, port=None, key_file=None, cert_file=None, 116183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, 116283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh source_address=None): 116383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh HTTPConnection.__init__(self, host, port, strict, timeout, 116483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh source_address) 116583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.key_file = key_file 116683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.cert_file = cert_file 116783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 116883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def connect(self): 116983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "Connect to a host on a given (SSL) port." 117083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 117183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh sock = socket.create_connection((self.host, self.port), 117283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.timeout, self.source_address) 117383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self._tunnel_host: 117483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.sock = sock 117583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._tunnel() 117683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file) 117783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 117883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh __all__.append("HTTPSConnection") 117983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 118083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh class HTTPS(HTTP): 118183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """Compatibility with 1.5 httplib interface 118283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 118383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh Python 1.5.2 did not have an HTTPS class, but it defined an 118483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh interface for sending http requests that is also useful for 118583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh https. 118683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """ 118783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 118883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh _connection_class = HTTPSConnection 118983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 119083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, host='', port=None, key_file=None, cert_file=None, 119183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh strict=None): 119283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # provide a default host, pass the X509 cert info 119383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 119483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # urf. compensate for bad input. 119583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if port == 0: 119683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh port = None 119783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._setup(self._connection_class(host, port, key_file, 119883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh cert_file, strict)) 119983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 120083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # we never actually use these for anything, but we keep them 120183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # here for compatibility with post-1.5.2 CVS. 120283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.key_file = key_file 120383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.cert_file = cert_file 120483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 120583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 120683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def FakeSocket (sock, sslobj): 120783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh warnings.warn("FakeSocket is deprecated, and won't be in 3.x. " + 120883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh "Use the result of ssl.wrap_socket() directly instead.", 120983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh DeprecationWarning, stacklevel=2) 121083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return sslobj 121183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 121283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 121383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass HTTPException(Exception): 121483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # Subclasses that define an __init__ must call Exception.__init__ 121583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # or define self.args. Otherwise, str() will fail. 121683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 121783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 121883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass NotConnected(HTTPException): 121983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 122083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 122183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass InvalidURL(HTTPException): 122283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 122383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 122483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass UnknownProtocol(HTTPException): 122583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, version): 122683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.args = version, 122783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.version = version 122883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 122983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass UnknownTransferEncoding(HTTPException): 123083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 123183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 123283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass UnimplementedFileMode(HTTPException): 123383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 123483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 123583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass IncompleteRead(HTTPException): 123683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, partial, expected=None): 123783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.args = partial, 123883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.partial = partial 123983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.expected = expected 124083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __repr__(self): 124183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self.expected is not None: 124283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh e = ', %i more expected' % self.expected 124383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 124483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh e = '' 124583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return 'IncompleteRead(%i bytes read%s)' % (len(self.partial), e) 124683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __str__(self): 124783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return repr(self) 124883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 124983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass ImproperConnectionState(HTTPException): 125083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 125183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 125283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass CannotSendRequest(ImproperConnectionState): 125383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 125483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 125583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass CannotSendHeader(ImproperConnectionState): 125683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 125783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 125883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass ResponseNotReady(ImproperConnectionState): 125983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh pass 126083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 126183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass BadStatusLine(HTTPException): 126283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, line): 126383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if not line: 126483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh line = repr(line) 126583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.args = line, 126683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.line = line 126783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 126883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass LineTooLong(HTTPException): 126983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, line_type): 127083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh HTTPException.__init__(self, "got more than %d bytes when reading %s" 127183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh % (_MAXLINE, line_type)) 127283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 127383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh# for backwards compatibility 127483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieherror = HTTPException 127583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 127683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsiehclass LineAndFileWrapper: 127783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh """A limited file-like object for HTTP/0.9 responses.""" 127883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 127983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # The status-line parsing code calls readline(), which normally 128083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # get the HTTP status line. For a 0.9 response, however, this is 128183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # actually the first line of the body! Clients need to get a 128283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # readable file object that contains that line. 128383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 128483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __init__(self, line, file): 128583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._line = line 128683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._file = file 128783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._line_consumed = 0 128883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._line_offset = 0 128983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._line_left = len(line) 129083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 129183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def __getattr__(self, attr): 129283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return getattr(self._file, attr) 129383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 129483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def _done(self): 129583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # called when the last byte is read from the line. After the 129683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # call, all read methods are delegated to the underlying file 129783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh # object. 129883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._line_consumed = 1 129983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.read = self._file.read 130083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.readline = self._file.readline 130183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self.readlines = self._file.readlines 130283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 130383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def read(self, amt=None): 130483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self._line_consumed: 130583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self._file.read(amt) 130683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh assert self._line_left 130783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if amt is None or amt > self._line_left: 130883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh s = self._line[self._line_offset:] 130983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._done() 131083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if amt is None: 131183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return s + self._file.read() 131283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 131383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return s + self._file.read(amt - len(s)) 131483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 131583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh assert amt <= self._line_left 131683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh i = self._line_offset 131783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh j = i + amt 131883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh s = self._line[i:j] 131983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._line_offset = j 132083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._line_left -= amt 132183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self._line_left == 0: 132283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._done() 132383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return s 132483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 132583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def readline(self): 132683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self._line_consumed: 132783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self._file.readline() 132883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh assert self._line_left 132983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh s = self._line[self._line_offset:] 133083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._done() 133183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return s 133283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh 133383760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh def readlines(self, size=None): 133483760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if self._line_consumed: 133583760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return self._file.readlines(size) 133683760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh assert self._line_left 133783760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh L = [self._line[self._line_offset:]] 133883760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh self._done() 133983760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh if size is None: 134083760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return L + self._file.readlines() 134183760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh else: 134283760d213fb3bec7b4117d266fcfbf6fe2ba14abAndrew Hsieh return L + self._file.readlines(size) 1343