Skip to content

Add cap billing walkthrough tutorial #4494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Aug 17, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
/appengine/**/*.py @engelke @GoogleCloudPlatform/python-samples-owners
/auth/**/*.py @busunkim96 @GoogleCloudPlatform/python-samples-owners
/automl/**/*.py @telpirion @sirtorry @GoogleCloudPlatform/python-samples-owners
/billing/**/*.py @GoogleCloudPlatform/billing-samples-maintainers @GoogleCloudPlatform/python-samples-owners
/bigquery/**/*.py @shollyman @GoogleCloudPlatform/python-samples-owners
/bigquery_storage/**/*.py @shollyman @GoogleCloudPlatform/python-samples-owners
/bigtable/**/*.py @GoogleCloudPlatform/bigtable-dpe @GoogleCloudPlatform/python-samples-owners
Expand Down
9 changes: 9 additions & 0 deletions billing/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Cap Billing Example

If you are cost conscious and need to control your environment relative to your budget, then you can use programmatic budget notifications to automate your cost control response based on the budget notification.

For an overview of automated cost control responses, see https://cloud.google.com/billing/docs/how-to/notify.

## Run the walkthrough tutorial

[![Open in Cloud Shell](http://gstatic.com/cloudssh/images/open-btn.svg)](https://console.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fpython-docs-samples%2Fbilling&cloudshell_tutorial=walkthroughtutorial.md)
156 changes: 156 additions & 0 deletions billing/cap-billing-walkthrough.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Automating cost controls by capping billing

## Overview

In this tutorial, you will learn to automate cost controls by setting up programmatic budget notifications.
Programmatic budget notifications can be used to disable billing, which will stop the usage of paid services for your project.

You may choose to disable billing if you have a hard limit on how much money you can spend on Google Cloud.
This may be the case for students, researchers, or developers working in sandbox environments.

As you complete this guide, you will learn the following skills:
+ Creating a budget
+ Setting up a Pub/Sub topic
+ Connecting a billing account to a Pub/Sub topic
+ Deploying a function

**Time to complete:** About 10 minutes

## Getting started

### Before you begin

Before you attempt this tutorial, you will need:
+ An [active Cloud billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account#create_a_new_billing_account), where you are a billing admin, or you have been granted the correct level of permissions to complete the steps in this tutorial.

## Understand best practices

We recommend that you configure a separate, single Google Cloud project to contain all of your billing administration needs, including your Cloud Billing-related Pub/Sub topics. Your billing administration Google Cloud project can also be used for things like Cloud Billing Budget API access, Cloud Billing Account API access, Cloud Billing exported data, and so on.

## Select a test project

For this tutorial, select or create a test project. The function will be acting on this project, not the billing administration project.

**Caution:** Using the cap billing example will remove Cloud Billing from your project, shutting down all resources. This may result in resources being irretrievably deleted, with no option to recover services. You can re-enable Cloud Billing, but there is no guarantee of service recovery and manual configuration is required.

<walkthrough-project-setup></walkthrough-project-setup>

```sh
export GOOGLE_CLOUD_PROJECT={{project_id}}
```

## Select a billing administration project

For this tutorial, create a new billing administration project.

<walkthrough-project-setup></walkthrough-project-setup>

## Setup

Set up a default project ID so that you do not need to provide them in commands where those values are required.

```sh
gcloud config set project {{project-id}}
```

Enable the Billing Budgets, Cloud Functions, Cloud Billing, and Cloud Build APIs, which you will need for this tutorial.
```sh
gcloud services enable billingbudgets.googleapis.com cloudfunctions.googleapis.com cloudbilling.googleapis.com cloudbuild.googleapis.com
```

Set up environment variables for your budget, Pub/Sub topic, and function.
```sh
export BUDGET_NAME=billing_cap_budget
export TOPIC_NAME=budget-notification
export FUNCTION_NAME=stop_billing
```

**Next: Learn how to set up programmatic budget notifications**

## Set up programmatic notifications

To set up **programmatic budget notifications**, you must create a Pub/Sub topic, create a Cloud Billing budget, and connect the Cloud Billing budget to the Pub/Sub topic.

### Create a Pub/Sub topic

Create a Pub/Sub topic so that Cloud Billing can publish budget alerts to the topic.
```sh
gcloud pubsub topics create ${TOPIC_NAME}
```

### Connect a billing account

Find your project’s billing account ID with the following command. Copy the billing account ID.

**Note:** If you don’t see a billing account ID, make sure your project is attached to a billing account.
```sh
gcloud beta billing projects describe {{project-id}} | grep billingAccountName
```

Replace <BILLING_ID> with your project’s billing account ID.
```sh
export BILLING_ACCOUNT=<BILLING_ID>
```
**Next: Learn how to create a budget**

## Create a budget

Create a test budget of $100 that is associated with your project’s billing account. This command also specifies the Pub/Sub topic where budget related messages will be sent.
```sh
gcloud alpha billing budgets create \
--billing-account=${BILLING_ACCOUNT} \
--display-name=${BUDGET_NAME} \
--budget-amount=100 \
--all-updates-rule-pubsub-topic="projects/${GOOGLE_CLOUD_PROJECT}/topics/${TOPIC_NAME}"
```

**Next: Learn more about the cap billing function and how to deploy it**

## Deploy the function

This function will remove the billing account associated with the project if the cost amount is higher than the budget amount.
```sh
gcloud functions deploy ${FUNCTION_NAME} \
--runtime=python37 \
--source=./sample_code \
--trigger-topic=${TOPIC_NAME}
```

**Next: Learn about service account permissions and how to configure them**

## Configure service account permissions

During the creation, updating, or deletion of a function, the Cloud Functions service uses the Google Cloud Functions service agent service account. You must grant the service account the proper permissions so that it can disable billing, such as the Billing Admin role.
```sh
gcloud projects add-iam-policy-binding \
${GOOGLE_CLOUD_PROJECT} \
--member='serviceAccount:'${GOOGLE_CLOUD_PROJECT}'@appspot.gserviceaccount.com' \
--role='roles/owner'
```
**Next: Verify that Cloud Billing is disabled**

## Verify that Cloud Billing is disabled

To disable Cloud Billing on your project, publish a sample message in Pub/Sub with the test message below. If successful, the project will no longer be visible under the billing account and resources in the project will be disabled.
```sh
gcloud pubsub topics publish ${TOPIC_NAME} --message='{"costAmount": 100.01,"budgetAmount": 100.00}'
```

Check that your billing account has been removed with this command. If the output is blank, then you have successfully disabled Cloud Billing.
```sh
gcloud beta billing projects describe {{project-id}} | grep billingAccountName
```

**Next: Wrapping up**

## Congratulations!

<walkthrough-conclusion-trophy></walkthrough-conclusion-trophy>

You’ve completed the Cap Billing walkthrough!

**What's Next**

Here are some areas to explore to learn more about automating cost controls:
+ [Sending notifications to Slack](https://cloud.google.com/billing/docs/how-to/notify#send_notifications_to_slack)
+ [Selectively controlling usage of resources](https://cloud.google.com/billing/docs/how-to/notify#selectively_control_usage)
63 changes: 63 additions & 0 deletions billing/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import base64
import json
import os
from googleapiclient import discovery
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with the GAPIC libraries. What is the benefit or reason for using them versus the one currently being used?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Among other things, they're generally cleaner/easier to use from a syntactical perspective.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This page explains the main differences. (GAPIC == Google Cloud Client Library)

PROJECT_ID = os.getenv('GCP_PROJECT')
PROJECT_NAME = f'projects/{PROJECT_ID}'
def stop_billing(data, context):
pubsub_data = base64.b64decode(data['data']).decode('utf-8')
pubsub_json = json.loads(pubsub_data)
cost_amount = pubsub_json['costAmount']
budget_amount = pubsub_json['budgetAmount']
if cost_amount <= budget_amount:
print(f'No action necessary. (Current cost: {cost_amount})')
return

if PROJECT_ID is None:
print('No project specified with environment variable')
return

billing = discovery.build(
'cloudbilling',
'v1',
cache_discovery=False,
)

projects = billing.projects()

billing_enabled = __is_billing_enabled(PROJECT_NAME, projects)

if billing_enabled:
__disable_billing_for_project(PROJECT_NAME, projects)
else:
print('Billing already disabled')


def __is_billing_enabled(project_name, projects):
"""
Determine whether billing is enabled for a project
@param {string} project_name Name of project to check if billing is enabled
@return {bool} Whether project has billing enabled or not
"""
try:
res = projects.getBillingInfo(name=project_name).execute()
return res['billingEnabled']
except KeyError:
# If billingEnabled isn't part of the return, billing is not enabled
return False
except Exception:
print('Unable to determine if billing is enabled on specified project, assuming billing is enabled')
return True


def __disable_billing_for_project(project_name, projects):
"""
Disable billing for a project by removing its billing account
@param {string} project_name Name of project disable billing on
"""
body = {'billingAccountName': ''} # Disable billing
try:
res = projects.updateBillingInfo(name=project_name, body=body).execute()
print(f'Billing disabled: {json.dumps(res)}')
except Exception:
print('Failed to disable billing, possibly check permissions')