-
Notifications
You must be signed in to change notification settings - Fork 6.5k
Add cap billing walkthrough tutorial #4494
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
busunkim96
merged 17 commits into
GoogleCloudPlatform:master
from
hujessica:capbillingtutorial
Aug 17, 2020
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
dd9e43d
Added Cloud Shell tutorial, sample code, README
61dabd6
Changed Cloud Shell tutorial initiation link
cdc1bfa
Edited spacing
2e0ba77
Fixed spacing
ae527bd
Moved tutorial to billing/tutorial
95ec28a
Added CODEOWNERS
f3c4b4f
Added newlines to main.py and CODEOWNERS
701e57b
Moved files into new billing folder
81f3da2
Split up flags in Deploy the function gcloud command
247533c
Edited GOOGLE_CLOUD_PLATFORM env variable
667f784
Added CODEOWNERS to .github folder
4a9621f
Merge branch 'master' into capbillingtutorial
30392f9
Added CODEOWNERS to .github folder
1bd89fe
Added CODEOWNERS of the billing folder
94a7aab
Fix CODEOWNERS
874ef6c
Use dedicated GitHub team in CODEOWNERS
8541b56
Merge branch 'master' into capbillingtutorial
busunkim96 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# Cap Billing Example | ||
|
||
If you are cost conscious and need to control your environment relative to your budget, then you can use programmatic budget notifications to automate your cost control response based on the budget notification. | ||
|
||
For an overview of automated cost control responses, see https://cloud.google.com/billing/docs/how-to/notify. | ||
|
||
## Run the walkthrough tutorial | ||
|
||
[](https://console.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fpython-docs-samples%2Fbilling&cloudshell_tutorial=walkthroughtutorial.md) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,156 @@ | ||
# Automating cost controls by capping billing | ||
|
||
## Overview | ||
|
||
In this tutorial, you will learn to automate cost controls by setting up programmatic budget notifications. | ||
Programmatic budget notifications can be used to disable billing, which will stop the usage of paid services for your project. | ||
|
||
You may choose to disable billing if you have a hard limit on how much money you can spend on Google Cloud. | ||
This may be the case for students, researchers, or developers working in sandbox environments. | ||
|
||
As you complete this guide, you will learn the following skills: | ||
+ Creating a budget | ||
+ Setting up a Pub/Sub topic | ||
+ Connecting a billing account to a Pub/Sub topic | ||
+ Deploying a function | ||
|
||
**Time to complete:** About 10 minutes | ||
|
||
## Getting started | ||
|
||
### Before you begin | ||
|
||
Before you attempt this tutorial, you will need: | ||
+ An [active Cloud billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account#create_a_new_billing_account), where you are a billing admin, or you have been granted the correct level of permissions to complete the steps in this tutorial. | ||
|
||
## Understand best practices | ||
|
||
We recommend that you configure a separate, single Google Cloud project to contain all of your billing administration needs, including your Cloud Billing-related Pub/Sub topics. Your billing administration Google Cloud project can also be used for things like Cloud Billing Budget API access, Cloud Billing Account API access, Cloud Billing exported data, and so on. | ||
|
||
## Select a test project | ||
|
||
For this tutorial, select or create a test project. The function will be acting on this project, not the billing administration project. | ||
|
||
**Caution:** Using the cap billing example will remove Cloud Billing from your project, shutting down all resources. This may result in resources being irretrievably deleted, with no option to recover services. You can re-enable Cloud Billing, but there is no guarantee of service recovery and manual configuration is required. | ||
|
||
<walkthrough-project-setup></walkthrough-project-setup> | ||
|
||
```sh | ||
export GOOGLE_CLOUD_PROJECT={{project_id}} | ||
``` | ||
|
||
## Select a billing administration project | ||
|
||
For this tutorial, create a new billing administration project. | ||
|
||
<walkthrough-project-setup></walkthrough-project-setup> | ||
|
||
## Setup | ||
|
||
Set up a default project ID so that you do not need to provide them in commands where those values are required. | ||
|
||
```sh | ||
gcloud config set project {{project-id}} | ||
``` | ||
|
||
Enable the Billing Budgets, Cloud Functions, Cloud Billing, and Cloud Build APIs, which you will need for this tutorial. | ||
```sh | ||
gcloud services enable billingbudgets.googleapis.com cloudfunctions.googleapis.com cloudbilling.googleapis.com cloudbuild.googleapis.com | ||
``` | ||
|
||
Set up environment variables for your budget, Pub/Sub topic, and function. | ||
```sh | ||
export BUDGET_NAME=billing_cap_budget | ||
export TOPIC_NAME=budget-notification | ||
export FUNCTION_NAME=stop_billing | ||
``` | ||
|
||
**Next: Learn how to set up programmatic budget notifications** | ||
|
||
## Set up programmatic notifications | ||
|
||
To set up **programmatic budget notifications**, you must create a Pub/Sub topic, create a Cloud Billing budget, and connect the Cloud Billing budget to the Pub/Sub topic. | ||
|
||
### Create a Pub/Sub topic | ||
|
||
Create a Pub/Sub topic so that Cloud Billing can publish budget alerts to the topic. | ||
```sh | ||
gcloud pubsub topics create ${TOPIC_NAME} | ||
``` | ||
|
||
### Connect a billing account | ||
|
||
Find your project’s billing account ID with the following command. Copy the billing account ID. | ||
|
||
**Note:** If you don’t see a billing account ID, make sure your project is attached to a billing account. | ||
```sh | ||
gcloud beta billing projects describe {{project-id}} | grep billingAccountName | ||
``` | ||
|
||
Replace <BILLING_ID> with your project’s billing account ID. | ||
```sh | ||
export BILLING_ACCOUNT=<BILLING_ID> | ||
``` | ||
**Next: Learn how to create a budget** | ||
|
||
## Create a budget | ||
|
||
Create a test budget of $100 that is associated with your project’s billing account. This command also specifies the Pub/Sub topic where budget related messages will be sent. | ||
```sh | ||
gcloud alpha billing budgets create \ | ||
--billing-account=${BILLING_ACCOUNT} \ | ||
--display-name=${BUDGET_NAME} \ | ||
--budget-amount=100 \ | ||
--all-updates-rule-pubsub-topic="projects/${GOOGLE_CLOUD_PROJECT}/topics/${TOPIC_NAME}" | ||
``` | ||
|
||
**Next: Learn more about the cap billing function and how to deploy it** | ||
|
||
## Deploy the function | ||
|
||
This function will remove the billing account associated with the project if the cost amount is higher than the budget amount. | ||
```sh | ||
gcloud functions deploy ${FUNCTION_NAME} \ | ||
--runtime=python37 \ | ||
--source=./sample_code \ | ||
--trigger-topic=${TOPIC_NAME} | ||
``` | ||
|
||
**Next: Learn about service account permissions and how to configure them** | ||
|
||
## Configure service account permissions | ||
|
||
During the creation, updating, or deletion of a function, the Cloud Functions service uses the Google Cloud Functions service agent service account. You must grant the service account the proper permissions so that it can disable billing, such as the Billing Admin role. | ||
```sh | ||
gcloud projects add-iam-policy-binding \ | ||
${GOOGLE_CLOUD_PROJECT} \ | ||
--member='serviceAccount:'${GOOGLE_CLOUD_PROJECT}'@appspot.gserviceaccount.com' \ | ||
--role='roles/owner' | ||
``` | ||
**Next: Verify that Cloud Billing is disabled** | ||
|
||
## Verify that Cloud Billing is disabled | ||
|
||
To disable Cloud Billing on your project, publish a sample message in Pub/Sub with the test message below. If successful, the project will no longer be visible under the billing account and resources in the project will be disabled. | ||
```sh | ||
gcloud pubsub topics publish ${TOPIC_NAME} --message='{"costAmount": 100.01,"budgetAmount": 100.00}' | ||
``` | ||
|
||
Check that your billing account has been removed with this command. If the output is blank, then you have successfully disabled Cloud Billing. | ||
```sh | ||
gcloud beta billing projects describe {{project-id}} | grep billingAccountName | ||
``` | ||
|
||
**Next: Wrapping up** | ||
|
||
## Congratulations! | ||
|
||
<walkthrough-conclusion-trophy></walkthrough-conclusion-trophy> | ||
|
||
You’ve completed the Cap Billing walkthrough! | ||
|
||
**What's Next** | ||
|
||
Here are some areas to explore to learn more about automating cost controls: | ||
+ [Sending notifications to Slack](https://cloud.google.com/billing/docs/how-to/notify#send_notifications_to_slack) | ||
+ [Selectively controlling usage of resources](https://cloud.google.com/billing/docs/how-to/notify#selectively_control_usage) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
import base64 | ||
import json | ||
import os | ||
from googleapiclient import discovery | ||
PROJECT_ID = os.getenv('GCP_PROJECT') | ||
PROJECT_NAME = f'projects/{PROJECT_ID}' | ||
def stop_billing(data, context): | ||
pubsub_data = base64.b64decode(data['data']).decode('utf-8') | ||
pubsub_json = json.loads(pubsub_data) | ||
cost_amount = pubsub_json['costAmount'] | ||
budget_amount = pubsub_json['budgetAmount'] | ||
if cost_amount <= budget_amount: | ||
print(f'No action necessary. (Current cost: {cost_amount})') | ||
return | ||
|
||
if PROJECT_ID is None: | ||
print('No project specified with environment variable') | ||
return | ||
|
||
billing = discovery.build( | ||
'cloudbilling', | ||
'v1', | ||
cache_discovery=False, | ||
) | ||
|
||
projects = billing.projects() | ||
|
||
billing_enabled = __is_billing_enabled(PROJECT_NAME, projects) | ||
|
||
if billing_enabled: | ||
__disable_billing_for_project(PROJECT_NAME, projects) | ||
else: | ||
print('Billing already disabled') | ||
|
||
|
||
def __is_billing_enabled(project_name, projects): | ||
""" | ||
Determine whether billing is enabled for a project | ||
@param {string} project_name Name of project to check if billing is enabled | ||
@return {bool} Whether project has billing enabled or not | ||
""" | ||
try: | ||
res = projects.getBillingInfo(name=project_name).execute() | ||
return res['billingEnabled'] | ||
except KeyError: | ||
# If billingEnabled isn't part of the return, billing is not enabled | ||
return False | ||
except Exception: | ||
print('Unable to determine if billing is enabled on specified project, assuming billing is enabled') | ||
return True | ||
|
||
|
||
def __disable_billing_for_project(project_name, projects): | ||
""" | ||
Disable billing for a project by removing its billing account | ||
@param {string} project_name Name of project disable billing on | ||
""" | ||
body = {'billingAccountName': ''} # Disable billing | ||
try: | ||
res = projects.updateBillingInfo(name=project_name, body=body).execute() | ||
print(f'Billing disabled: {json.dumps(res)}') | ||
except Exception: | ||
print('Failed to disable billing, possibly check permissions') |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can the GAPIC libraries be used instead? https://github.com/googleapis/python-billing and https://github.com/googleapis/python-billingbudgets? (not sure which is applicable here).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not familiar with the GAPIC libraries. What is the benefit or reason for using them versus the one currently being used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Among other things, they're generally cleaner/easier to use from a syntactical perspective.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This page explains the main differences. (GAPIC == Google Cloud Client Library)