Secrets

The secrets package provides access to key management services in a portable way. This guide shows how to work with secrets in the GDK.

Cloud applications frequently need to store sensitive information like web API credentials or encryption keys in a medium that is not fully secure. For example, an application that interacts with GitHub needs to store its OAuth2 client secret and use it when obtaining end-user credentials. If this information was compromised, it could allow someone else to impersonate the application. In order to keep such information secret and secure, you can encrypt the data, but then you need to worry about rotating the encryption keys and distributing them securely to all of your application servers. Most Cloud providers include a key management service to perform these tasks, usually with hardware-level security and audit logging.

The secrets package supports encryption and decryption operations.

Subpackages contain driver implementations of secrets for various services, including Cloud and on-prem solutions. You can develop your application locally using localsecrets, then deploy it to multiple Cloud providers with minimal initialization reconfiguration.

Opening a SecretsKeeper🔗

The first step in working with your secrets is to instantiate a portable *secrets.Keeper for your service.

The easiest way to do so is to use secrets.OpenKeeper and a service-specific URL pointing to the keeper, making sure you “blank import” the driver package to link it in.

import (
	"github.com/sraphs/gdk/secrets"
	_ "github.com/sraphs/gdk/secrets/<driver>"
)
...
keeper, err := secrets.OpenKeeper(context.Background(), "<driver-url>")
if err != nil {
    return fmt.Errorf("could not open keeper: %v", err)
}
defer keeper.Close()
// keeper is a *secrets.Keeper; see usage below
...

See Concepts: URLs for general background and the guide below for URL usage for each supported service.

Alternatively, if you need fine-grained control over the connection settings, you can call the constructor function in the driver package directly (like awskms.OpenKeeper).

import "github.com/sraphs/gdk/secrets/<driver>"
...
keeper, err := <driver>.OpenKeeper(...)
...

You may find the wire package useful for managing your initialization code when switching between different backing services.

See the guide below for constructor usage for each supported service.

Using a SecretsKeeper🔗

Once you have opened a secrets keeper for the secrets provider you want, you can encrypt and decrypt small messages using the keeper.

Encrypting Data🔗

To encrypt data with a keeper, you call Encrypt with the byte slice you want to encrypt.

plainText := []byte("Secrets secrets...")
cipherText, err := keeper.Encrypt(ctx, plainText)
if err != nil {
	return err
}

Decrypting Data🔗

To decrypt data with a keeper, you call Decrypt with the byte slice you want to decrypt. This should be data that you obtained from a previous call to Encrypt with a keeper that uses the same secret material (e.g. two AWS KMS keepers created with the same customer master key ID). The Decrypt method will return an error if the input data is corrupted.

var cipherText []byte // obtained from elsewhere and random-looking
plainText, err := keeper.Decrypt(ctx, cipherText)
if err != nil {
	return err
}

Large Messages🔗

The secrets keeper API is designed to work with small messages (i.e. <10 KiB in length.) Cloud key management services are high latency; using them for encrypting or decrypting large amounts of data is prohibitively slow (and in some providers not permitted). If you need your application to encrypt or decrypt large amounts of data, you should:

1. Generate a key for the encryption algorithm (16KiB chunks with secretbox is a reasonable approach).
2. Encrypt the key with secret keeper.
3. Store the encrypted key somewhere accessible to the application.

When your application needs to encrypt or decrypt a large message:

1. Decrypt the key from storage using the secret keeper
2. Use the decrypted key to encrypt or decrypt the message inside your application.

Keep Secrets in Configuration🔗

Once you have opened a secrets keeper for the secrets provider you want, you can use a secrets keeper to access sensitive configuration stored in an encrypted runtimevar.

First, you create a *runtimevar.Decoder configured to use your secrets keeper using runtimevar.DecryptDecode. In this example, we assume the data is a plain string, but the configuration could be a more structured type.

decodeFunc := runtimevar.DecryptDecode(keeper, runtimevar.StringDecode)
decoder := runtimevar.NewDecoder("", decodeFunc)

Then you can pass the decoder to the runtime configuration provider of your choice. See the Runtime Configuration How-To Guide for more on how to set up runtime configuration.

Other Usage Samples🔗

• CLI Sample
• Secrets package examples

Supported Services🔗

HashiCorp Vault🔗

The GDK can use the transit secrets engine in Vault to keep information secret. Vault URLs only specify the key ID. The Vault server endpoint and authentication token are specified using the environment variables VAULT_SERVER_URL and VAULT_SERVER_TOKEN, respectively.

import (
	"context"

	"github.com/sraphs/gdk/secrets"
	_ "github.com/sraphs/gdk/secrets/hashivault"
)

keeper, err := secrets.OpenKeeper(ctx, "hashivault://mykey")
if err != nil {
	return err
}
defer keeper.Close()

HashiCorp Vault Constructor🔗

The hashivault.OpenKeeper constructor opens a transit secrets engine key. You must first connect to your Vault instance.

import (
	"context"

	"github.com/hashicorp/vault/api"
	"github.com/sraphs/gdk/secrets/hashivault"
)

// Get a client to use with the Vault API.
client, err := hashivault.Dial(ctx, &hashivault.Config{
	Token: "CLIENT_TOKEN",
	APIConfig: api.Config{
		Address: "http://127.0.0.1:8200",
	},
})
if err != nil {
	return err
}

// Construct a *secrets.Keeper.
keeper := hashivault.OpenKeeper(client, "my-key", nil)
defer keeper.Close()

Local Secrets🔗

The GDK can use local encryption for keeping secrets. Internally, it uses the NaCl secret box algorithm to perform encryption and authentication.

import (
	"context"

	"github.com/sraphs/gdk/secrets"
	_ "github.com/sraphs/gdk/secrets/localsecrets"
)

// Using "base64key://", a new random key will be generated.
randomKeyKeeper, err := secrets.OpenKeeper(ctx, "base64key://")
if err != nil {
	return err
}
defer randomKeyKeeper.Close()

// Otherwise, the URL hostname must be a base64-encoded key, of length 32 bytes when decoded.
// Note that base64.URLEncode should be used, to avoid URL-unsafe characters.
savedKeyKeeper, err := secrets.OpenKeeper(ctx, "base64key://smGbjm71Nxd1Ig5FS0wj9SlbzAIrnolCz9bQQ6uAhl4=")
if err != nil {
	return err
}
defer savedKeyKeeper.Close()

Local Secrets Constructor🔗

The localsecrets.NewKeeper constructor takes in its secret material as a []byte.

import "github.com/sraphs/gdk/secrets/localsecrets"

secretKey, err := localsecrets.NewRandomKey()
if err != nil {
	return err
}
keeper := localsecrets.NewKeeper(secretKey)
defer keeper.Close()