1b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# Copyright 2012 Google Inc. All Rights Reserved. 2b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# 3b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# Licensed under the Apache License, Version 2.0 (the "License"); 4b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# you may not use this file except in compliance with the License. 5b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# You may obtain a copy of the License at 6b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# 7b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# http://www.apache.org/licenses/LICENSE-2.0 8b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# 9b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# Unless required by applicable law or agreed to in writing, 10b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# software distributed under the License is distributed on an 11b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, 12b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# either express or implied. See the License for the specific 13b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik# language governing permissions and limitations under the License. 14b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 15b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik"""Google Cloud Storage specific Files API calls.""" 16b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 17b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 18b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 19b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 20b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 21b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik__all__ = ['AuthorizationError', 22b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'check_status', 23b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'Error', 24b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'FatalError', 25b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'FileClosedError', 26b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'ForbiddenError', 27b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'InvalidRange', 28b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'NotFoundError', 29b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'ServerError', 30b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'TimeoutError', 31b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'TransientError', 32b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik ] 33b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 34b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikimport httplib 35b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 36b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 37b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass Error(Exception): 38b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """Base error for all gcs operations. 39b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 40b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik Error can happen on GAE side or GCS server side. 41b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik For details on a particular GCS HTTP response code, see 42b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik https://developers.google.com/storage/docs/reference-status#standardcodes 43b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """ 44b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 45b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 46b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass TransientError(Error): 47b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """TransientError could be retried.""" 48b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 49b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 50b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass TimeoutError(TransientError): 51b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """HTTP 408 timeout.""" 52b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 53b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 54b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass FatalError(Error): 55b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """FatalError shouldn't be retried.""" 56b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 57b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 58b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass FileClosedError(FatalError): 59b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """File is already closed. 60b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 61b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik This can happen when the upload has finished but 'write' is called on 62b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik a stale upload handle. 63b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """ 64b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 65b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 66b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass NotFoundError(FatalError): 67b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """HTTP 404 resource not found.""" 68b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 69b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 70b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass ForbiddenError(FatalError): 71b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """HTTP 403 Forbidden. 72b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 73b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik While GCS replies with a 403 error for many reasons, the most common one 74b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik is due to bucket permission not correctly setup for your app to access. 75b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """ 76b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 77b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 78b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass AuthorizationError(FatalError): 79b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """HTTP 401 authentication required. 80b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 81b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik Unauthorized request has been received by GCS. 82b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 83b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik This error is mostly handled by GCS client. GCS client will request 84b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik a new access token and retry the request. 85b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """ 86b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 87b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 88b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass InvalidRange(FatalError): 89b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """HTTP 416 RequestRangeNotSatifiable.""" 90b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 91b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 92b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikclass ServerError(TransientError): 93b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """HTTP >= 500 server side error.""" 94b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 95b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 96b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craikdef check_status(status, expected, path, headers=None, 97b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik resp_headers=None, body=None, extras=None): 98b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """Check HTTP response status is expected. 99b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 100b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik Args: 101b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik status: HTTP response status. int. 102b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik expected: a list of expected statuses. A list of ints. 103b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik path: filename or a path prefix. 104b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik headers: HTTP request headers. 105b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik resp_headers: HTTP response headers. 106b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik body: HTTP response body. 107b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik extras: extra info to be logged verbatim if error occurs. 108b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 109b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik Raises: 110b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik AuthorizationError: if authorization failed. 111b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik NotFoundError: if an object that's expected to exist doesn't. 112b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik TimeoutError: if HTTP request timed out. 113b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik ServerError: if server experienced some errors. 114b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik FatalError: if any other unexpected errors occurred. 115b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik """ 116b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik if status in expected: 117b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik return 118b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 119b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik msg = ('Expect status %r from Google Storage. But got status %d.\n' 120b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'Path: %r.\n' 121b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'Request headers: %r.\n' 122b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'Response headers: %r.\n' 123b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'Body: %r.\n' 124b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 'Extra info: %r.\n' % 125b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik (expected, status, path, headers, resp_headers, body, extras)) 126b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik 127b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik if status == httplib.UNAUTHORIZED: 128b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik raise AuthorizationError(msg) 129b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik elif status == httplib.FORBIDDEN: 130b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik raise ForbiddenError(msg) 131b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik elif status == httplib.NOT_FOUND: 132b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik raise NotFoundError(msg) 133b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik elif status == httplib.REQUEST_TIMEOUT: 134b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik raise TimeoutError(msg) 135b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik elif status == httplib.REQUESTED_RANGE_NOT_SATISFIABLE: 136b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik raise InvalidRange(msg) 137b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik elif (status == httplib.OK and 308 in expected and 138b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik httplib.OK not in expected): 139b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik raise FileClosedError(msg) 140b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik elif status >= 500: 141b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik raise ServerError(msg) 142b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik else: 143b2cbf1594f8d6e4ba32d384cf379f62a74ed7654Chris Craik raise FatalError(msg) 144