18d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# -*- coding: utf-8 -*-
28d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# Copyright 2013 Google Inc. All Rights Reserved.
38d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi#
48d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# Licensed under the Apache License, Version 2.0 (the "License");
58d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# you may not use this file except in compliance with the License.
68d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# You may obtain a copy of the License at
78d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi#
88d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi#     http://www.apache.org/licenses/LICENSE-2.0
98d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi#
108d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# Unless required by applicable law or agreed to in writing, software
118d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# distributed under the License is distributed on an "AS IS" BASIS,
128d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
138d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# See the License for the specific language governing permissions and
148d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi# limitations under the License.
158d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi"""Additional help about types of credentials and authentication."""
168d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
178d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoifrom __future__ import absolute_import
188d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
198d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoifrom gslib.help_provider import HelpProvider
208d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
218d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi_DETAILED_HELP_TEXT = ("""
228d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi<B>OVERVIEW</B>
238d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  gsutil currently supports several types of credentials/authentication, as
248d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  well as the ability to access public data anonymously (see "gsutil help anon"
258d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  for more on anonymous access). Each of these type of credentials is discussed
268d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  in more detail below, along with information about configuring and using
278d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  credentials via either the Cloud SDK or standalone installations of gsutil.
288d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
298d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
308d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi<B>Configuring/Using Credentials via Cloud SDK Distribution of gsutil</B>
318d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  When gsutil is installed/used via the Cloud SDK ("gcloud"), credentials are
328d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  stored by Cloud SDK in a non-user-editable file located under
338d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  ~/.config/gcloud (any manipulation of credentials should be done via the
348d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  gcloud auth command). If you need to set up multiple credentials (e.g., one
358d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  for an individual user account and a second for a service account), the
368d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  gcloud auth command manages the credentials for you, and you switch between
378d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  credentials using the gcloud auth command as well (for more details see
388d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  https://developers.google.com/cloud/sdk/gcloud/#gcloud.auth).
398d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
408d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  Once credentials have been configured via gcloud auth, those credentials will
418d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  be used regardless of whether the user has any boto configuration files (which
428d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  are located at ~/.boto unless a different path is specified in the BOTO_CONFIG
438d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  environment variable). However, gsutil will still look for credentials in the
448d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  boto config file if a type of credential is needed that's not stored in the
458d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  gcloud credential store (e.g., an HMAC credential for an S3 account).
468d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
478d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
488d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi<B>Configuring/Using Credentials via Standalone gsutil Distribution</B>
498d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  If you installed a standalone distribution of gsutil (downloaded from
508d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  https://pub.storage.googleapis.com/gsutil.tar.gz,
518d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  https://pub.storage.googleapis.com/gsutil.zip, or PyPi), credentials are
528d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  configured using the gsutil config command, and are stored in the
538d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  user-editable boto config file (located at ~/.boto unless a different path is
548d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  specified in the BOTO_CONFIG environment). In this case if you want to set up
558d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  multiple credentials (e.g., one for an individual user account and a second
568d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  for a service account), you run gsutil config once for each credential, and
578d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  save each of the generated boto config files (e.g., renaming one to
588d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  ~/.boto_user_account and the second to ~/.boto_service_account), and you
598d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  switch between the credentials using the BOTO_CONFIG environment variable
608d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  (e.g., by running BOTO_CONFIG=~/.boto_user_account gsutil ls).
618d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
628d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  Note that when using the standalone version of gsutil with the JSON API you
638d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  can configure at most one of the following types of GCS credentials in a
648d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  single boto config file: OAuth2 User Account, OAuth2 Service Account. In
658d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  addition to these, you may also have S3 HMAC credentials (necessary for using
668d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  s3:// URLs) and GCE Internal Service Account credentials. GCE Internal Service
678d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  Account credentials are used only when OAuth2 credentials are not present.
688d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
698d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
708d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi<B>SUPPORTED CREDENTIAL TYPES</B>
718d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  gsutil supports several types of credentials (the specific subset depends on
728d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  which distribution of gsutil you are using; see above discussion).
738d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
748d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  OAuth2 User Account:
758d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    This is the preferred type of credentials for authenticating requests on
768d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    behalf of a specific user (which is probably the most common use of gsutil).
778d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    This is the default type of credential that will be created when you run
788d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    "gsutil config".
798d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    For more details about OAuth2 authentication, see:
808d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    https://developers.google.com/accounts/docs/OAuth2#scenarios
818d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
828d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  HMAC:
838d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    This type of credential can be used by programs that are implemented using
848d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    HMAC authentication, which is an authentication mechanism supported by
858d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    certain other cloud storage service providers. This type of credential can
868d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    also be used for interactive use when moving data to/from service providers
878d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    that support HMAC credentials. This is the type of credential that will be
888d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    created when you run "gsutil config -a".
898d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
908d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    Note that it's possible to set up HMAC credentials for both Google Cloud
918d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    Storage and another service provider; or to set up OAuth2 user account
928d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    credentials for Google Cloud Storage and HMAC credentials for another
938d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    service provider. To do so, after you run the gsutil config command, you
948d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    can edit the generated ~/.boto config file and look for comments for where
958d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    other credentials can be added.
968d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
978d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    For more details about HMAC authentication, see:
988d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      https://developers.google.com/storage/docs/reference/v1/getting-startedv1#keys
998d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1008d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  OAuth2 Service Account:
1018d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    This is the preferred type of credential to use when authenticating on
1028d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    behalf of a service or application (as opposed to a user). For example, if
1038d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    you will run gsutil out of a nightly cron job to upload/download data,
1048d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    using a service account allows the cron job not to depend on credentials of
1058d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    an individual employee at your company. This is the type of credential that
1068d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    will be configured when you run "gsutil config -e".
1078d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1088d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    It is important to note that a service account is considered an Editor by
1098d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    default for the purposes of API access, rather than an Owner. In particular,
1108d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    the fact that Editors have OWNER access in the default object and
1118d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    bucket ACLs, but the canned ACL options remove OWNER access from
1128d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    Editors, can lead to unexpected results. The solution to this problem is to
1138d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    add the email address for your service account as a project editor. To find
1148d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    the email address, visit the
1158d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    `Google Developers Console <https://cloud.google.com/console#/project>`_,
1168d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    click on the project you're using, click "APIs & auth", and click
1178d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    "Credentials".
1188d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1198d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    To create a service account, visit the Google Developers Console and then:
1208d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1218d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi       - Click "APIs & auth" in the left sidebar.
1228d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1238d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi       - Click "Credentials".
1248d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1258d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi       - Click "Create New Client ID".
1268d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1278d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi       - Select "Service Account" as your application type.
1288d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1298d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi       - Save the JSON private key or the .p12 private key and password
1308d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi         provided.
1318d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1328d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    For further information about account roles, see:
1338d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      https://developers.google.com/console/help/#DifferentRoles
1348d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1358d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    For more details about OAuth2 service accounts, see:
1368d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      https://developers.google.com/accounts/docs/OAuth2ServiceAccount
1378d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1388d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  GCE Internal Service Account:
1398d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    This is the type of service account used for accounts hosted by App Engine
1408d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    or GCE. Such credentials are created automatically for you on GCE when you
1418d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    run the gcutil addinstance command with the --service_account flag.
1428d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1438d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    For more details about GCE service accounts, see:
1448d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      https://developers.google.com/compute/docs/authentication;
1458d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1468d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi    For more details about App Engine service accounts, see:
1478d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      https://developers.google.com/appengine/docs/python/appidentity/overview
1488d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi""")
1498d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1508d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1518d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoiclass CommandOptions(HelpProvider):
1528d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  """Additional help about types of credentials and authentication."""
1538d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi
1548d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  # Help specification. See help_provider.py for documentation.
1558d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  help_spec = HelpProvider.HelpSpec(
1568d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      help_name='creds',
1578d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      help_name_aliases=['credentials', 'authentication', 'auth', 'gcloud'],
1588d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      help_type='additional_help',
1598d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      help_one_line_summary='Credential Types Supporting Various Use Cases',
1608d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      help_text=_DETAILED_HELP_TEXT,
1618d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi      subcommand_help_text={},
1628d2b206a675ec20ea07100c35df34e65ee1e45e8Ruchi Kandoi  )
163