Google Cloud Bigtable Python Library (Alpha)

An extension to gcloud-python

This library supports RPC requests to the Google Cloud Bigtable API over
HTTP/2. In order to support this, we'll rely on grpc. Unfortunately,
the install story of gRPC is still developing. We are working with the
gRPC team to rapidly make the install story more user-friendly.

The Cloud Bigtable API is supported via JSON over HTTP/1.1 (see the
gcloud CLI use of the API). However, features of HTTP/2 such
as streaming are used by the Bigtable API and are not possible
to support via HTTP/1.1.

Installing gRPC

Make sure you have downloaded homebrew on OS X or
linuxbrew on Linux. (On Linux, also be sure to
add brew to ${PATH} as instructed.)

First, install the gRPC core (C/C++) library

curl -fsSL https://goo.gl/getgrpc | bash

Since this uses brew to install, this cannot be run as
root (via sudo).

Next, install the Python grpcio library via:

BREW_PREFIX=$(brew --prefix)
[sudo] CFLAGS=-I${BREW_PREFIX}/include LDFLAGS=-L${BREW_PREFIX}/lib \
pip install --upgrade grpcio

You may wish to run this as root (via sudo) so it can be included with
your machine's Python libraries. If not, you'll need to use a Python
virtual environment so that non-privileged (i.e. non-sudo) installs
are allowed.

Finally, you can install this library via

[sudo] pip install -e git+https://github.com/dhermes/gcloud-python-bigtable#egg=gcloud-bigtable

Again, you may wish to install as root or in a virtual environment.

Running gcloud_bigtable code

Since the gRPC core is installed via brew, the system libraries
are not in a place that Python can readily find them.

In order to run code that uses gcloud_bigtable with these
libraries, you'll need to set the LD_LIBRARY_PATH environment
variable:

BREW_PREFIX=$(brew --prefix)
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${BREW_PREFIX}/lib

Authorization

You can make requests with your own Google account by
using the gcloud CLI tool. You can create an access token via

gcloud login

and then from there, the token created will be picked up automatically
when you create an object which requires authentication:

from gcloud_bigtable.client import Client
client = Client()

If instead you'd like to use a service account, you can set an
environment variable to the path containing the service account
credentials JSON file:

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json"

If you are familiar with the oauth2client library,
you can create a credentials object and pass it directly:

from gcloud_bigtable.client import Client
client = Client(credentials=credentials)

Enabling the Bigtable API

1. Visit Google Cloud Console

2. Either create a new project or visit an existing one

3. In the project, click "APIs & auth > APIs". The URI
   should be of the form

   https://console.developers.google.com/project/{project-id}/apiui/apis/library
   

4. On this page, search for bigtable, and click both Cloud Bigtable API
   and Cloud Bigtable Table Admin API.

5. For each API, click "Enable API" (if not already enabled)

Getting a Service Account Keyfile

1. Visit Google Cloud Console

2. Either create a new project or visit an existing one

3. In the project, click "APIs & auth > Credentials". The URI
   should be of the form

   https://console.developers.google.com/project/{project-id}/apiui/credential
   

4. On this page, click "Create new Client ID", select "Service account" as
   your "Application type" and then download the JSON key provided.

After downloading, you can use this file as your
GOOGLE_APPLICATION_CREDENTIALS.

Creating a Cluster in the UI

1. Visit Google Cloud Console

2. Either create a new project or visit an existing one

3. In the project, click "Storage > Cloud Bigtable". The URI
   should be of the form

   https://console.developers.google.com/project/{project-id}/bigtable/clusters
   

4. On this page, click Create a cluster and take note of the "Cluster ID"
   and "Zone" you use when creating it.

Error Messages

Unfortunately, the gRPC Python library does not surface
exceptions returned from the API. An issue has been
filed with the gRPC team about this problem.

Development

See CONTRIBUTING.md for instructions on development.