Creating a Kubernetes Cluster with Digital Ocean and Terraform


Warning! Provisioning infrastructure on Digital Ocean can cost you money! Make sure to destroy infrastructure after you’re done playing with it!

Introduction

To get started with Kubernetes, you need a Kubernetes cluster.1 There are many ways to get one:

  • install a Kubernetes cluster on your laptop using MiniKube
  • use a cloud platform’s user interface to provision a cluster
  • hand-build your own cluster (see Kubernetes the Hard Way)
  • use Terraform to manage the creation of a cluster on a cloud platform

This post will cover the last option; we’ll use Terraform to easily create a cluster on Digital Ocean (they have a free $100 credit for new users! 😎).

TLDR; get a Digital Ocean token, then run:

$ git clone git@github.com:ponderosa-io/tf-digital-ocean-cluster.git
$ export TF_VAR_do_token=<your_do_token>
$ export TF_VAR_do_cluster_name=<your_cluster_name>
$ cd tf-digital-ocean-cluster
$ terraform plan
$ terraform apply
$ ./get_config
$ export KUBECONFIG=$(pwd)/config

Getting and storing your Digital Ocean token

To interact with Digital Ocean from the command line, you need a token so that Digital Ocean’s servers know that you have permission to access your account. Follow the steps in their docs and copy the token to your clipboard.2 This token is the key to your account, so don’t share it with anyone!

Once you have copied your Digital Ocean token, open up your terminal and store it as an environment variable: export TF_VAR_do_token=your-token-goes-here.3

Writing your Terraform Configuration

Terraform is a tool that allows you to manage your infrastructure using declarative configuration (so you don’t have to go into the Digital Ocean interface and click around to find your cluster). We’re going to use it to manage our cluster. Once you have your Digital Ocean token set as an environment variable, it’s time to write your Terraform config. When finished, it looks like this:

variable "do_token" {}
variable "do_cluster_name" {}

provider "digitalocean" {
  token = "${var.do_token}"
}

resource "digitalocean_kubernetes_cluster" "my_digital_ocean_cluster" {
  name    = "${var.do_cluster_name}"
  region  = "nyc1"
  version = "1.13.1-do.2"

  node_pool {
    name       = "worker-pool"
    size       = "s-2vcpu-2gb"
    node_count = 2
  }
}

output "cluster-id" {
  value = "${digitalocean_kubernetes_cluster.my_digital_ocean_cluster.id}"
}

Let’s break it down, stanza by stanza:

Setting the variables

variable "do_token" {}
variable "do_cluster_name" {}

Here, we tell Terraform to expect two variables: do_token and do_cluster_name. If you set TF_VAR_do_token and TF_VAR_cluster_name, these variables will be automatically read in by Terraform. If you don’t set them, you will be prompted for them when you run Terraform commands.

Configuring Terraform to talk to Digital Ocean

provider "digitalocean" {
  token = "${var.do_token}"
}

Digital Ocean is a Terraform provider, which means that it talks to something outside Terraform (namely: the Digital Ocean API) to create and provision infrastructure. This stanza configures the provider to use the do_token Terraform variable, which, in turn, will look at the TF_VAR_do_token environment variable.

Declaring the Digital Ocean Kubernetes cluster

resource "digitalocean_kubernetes_cluster" "my_digital_ocean_cluster" {
  name    = "${var.do_cluster_name}"
  region  = "nyc1"
  version = "1.13.1-do.2"

  node_pool {
    name       = "worker-pool"
    size       = "s-2vcpu-2gb"
    node_count = 2
  }
}

This stanza tells Terraform that we expect a digitalocean_kubernetes_cluster resource with the name do_cluster_name, in the nyc1 region, at version 1.13.1-do.2, and so on.

Configuring Terraform outputs

output "cluster-id" {
  value = "${digitalocean_kubernetes_cluster.my_digital_ocean_cluster.id}"
}

At this point, we’re telling Terraform that we want it to report back specific information when we run terraform apply. Specifically, we want to know the id of the resource digitalocean_kubernetes_cluster.my_digital_ocean_cluster, which is specified in the stanza above. We’ll use this id to get the configuration file for the cluster later on in the post.

And that’s it! All the Terraform you need.

Running your Terraform

If you’ve made your own Terraform file as you read the last section, that’s great. Otherwise, no worries, just git clone git@github.com:ponderosa-io/tf-digital-ocean-cluster.git and cd into it.4

Make sure that you have your environment variables set:

export TF_VAR_do_token=your_digital_ocean_token
export TF_VAR_do_cluster_name=your_cluster_name

Now run terraform plan to see what Terraform will do if you run terraform apply. It should say that it’s planning to create a cluster:

terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + digitalocean_kubernetes_cluster.my_digital_ocean_cluster
      id:                     <computed>
      cluster_subnet:         <computed>
      created_at:             <computed>
      endpoint:               <computed>
      ipv4_address:           <computed>
      kube_config.#:          <computed>
      name:                   "mycoolcluster"
      node_pool.#:            "1"
      node_pool.0.id:         <computed>
      node_pool.0.name:       "worker-pool"
      node_pool.0.node_count: "2"
      node_pool.0.nodes.#:    <computed>
      node_pool.0.size:       "s-2vcpu-2gb"
      region:                 "nyc1"
      service_subnet:         <computed>
      status:                 <computed>
      updated_at:             <computed>
      version:                "1.13.1-do.2"


Plan: 1 to add, 0 to change, 0 to destroy.

As you can see, the name specified in the cluster is what you set in your TF_CLUSTER_NAME environment variable.

Assuming this looks good (remember, creating infrastructure can cost money!), run terraform apply. This can take 5 or 10 minutes to complete (they’re building a cluster!!!). When the terraform apply is complete, it will print your cluster’s ID, which we’ll need for the next step.

Accessing your cluster

To interact with your Kubernetes cluster, you need a kubeconfig file. This file contains information about how to access your cluster and keys to authenticate to the cluster so you can control it. Digital Ocean’s /v2/kubernetes/clusters/$K8S_CLUSTER_ID/kubeconfig API endpiont will supply the config we need, but we need to figure out our cluster ID first (the $K8S_CLUSTER_ID in the above URL; if we didn’t supply it, how would it know which cluster to give us access to?).

Programmatically getting the cluster id

To get our cluster id, we could just copy and paste the cluster-id output from terraform apply, but we can programmatically access this value with terraform output, which provides an interface to the output stanzas in our Terraform config. Run CLUSTER_ID=$(terraform output cluster-id) to set CLUSTER_ID to this value.5 Then, run echo $CLUSTER_ID to make sure it worked.

Once we have the CLUSTER_ID variable set, we can use curl to hit the Digital Ocean config endpoint and redirect the output to a file called config. Note that we are using the TF_VAR_do_token to authorize our request and our CLUSTER_ID variable to form the URL.

curl -X GET \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${TF_VAR_do_token}" \
"https://api.digitalocean.com/v2/kubernetes/clusters/$CLUSTER_ID/kubeconfig" \
> config

Setting the KUBECONFIG environment variable

kubectl is the standard tool for interacting with Kubernetes clusters (installation instructions are here). When you run kubectl commands, it looks for a config file at the location specified in the KUBECONFIG environment variable (by default, it’s at ~/.kube/config).

To configure kubectl to use the config you just created, either move the new file to ~/.kube/config or run export KUBECONFIG="$(pwd)/config".

Confirming you have cluster access

Run kubectl cluster-info to check that you can access the cluster you just created.

Destroying your cluster

One nice thing about Terraform is that it makes destroying infrastructure easy. To bring down your cluster, just run terraform destroy.

Footnotes

  1. A cluster is one or more nodes acting as a single running instance of the Kubernetes platform. 

  2. Go to your Digital Ocean dashboard, expand the “manage” menu, click on the “API” menu item, and click “Generate token.” 

  3. We’re using the TF_VAR_ prefix because Terraform automatically reads in environment variables that follow this convention. 

  4. If you haven’t already, install Terraform (brew install terraform will work if you have Homebrew). 

  5. The VARIABLE=$() syntax saves the output of the command inside the parentheses to the variable on the left.