Merge pull request #18 from gene1wood/add-encryption-context

Adding support for KMS "encryption context"

Author: a5an0

README.md

@@ -21,6 +21,8 @@ Whenever you want to store/share a credential, such as a database password, you
 
 When you want to fetch the credential, for example as part of the bootstrap process on your web-server, you simply do `credstash get [credential-name]`. For example, `export DB_PASSWORD=$(credstash get myapp.db.prod)`. When you run `get`, credstash will go and fetch the encrypted credential and the wrapped encryption key from the credential store (DynamoDB). It will then send the wrapped encryption key to KMS, where it is decrypted with the master key. credstash then uses the decrypted data encryption key to decrypt the credential. The credential is printed to `stdout`, so you can use it in scripts or assign environment variables to it.
 
+Optionally you can include any number of [Encryption Context](http://docs.aws.amazon.com/kms/latest/developerguide/encrypt-context.html) key value pairs to associate with the credential. The exact set of encryption context key value pairs that were associated with the credential when it was `put` in DynamoDB must be provided in the `get` request to successfully decrypt the credential. These encryption context key value pairs are useful to provide auditing context to the encryption and decryption operations in your CloudTrail logs. They are also useful for constraining access to a given credstash stored credential by using KMS Key Policy conditions and KMS Grant conditions. Doing so allows you to, for example, make sure that your database servers and web-servers can read the web-server DB user password but your database servers can not read your web-servers TLS/SSL certificate's private key. A `put` request with encryption context would look like `credstash put myapp.db.prod supersecretpassword1234 app.tier=db environment=prod`. In order for your web-servers to read that same credential they would execute a `get` call like `export DB_PASSWORD=$(credstash get myapp.db.prod environment=prod app.tier=db)`
+
 Credentials stored in the credential-store are versioned and immutable. That is, if you `put` a credential called `foo` with a version of `1` and a value of `bar`, then foo version 1 will always have a value of bar, and there is no way in `credstash` to change its value (although you could go fiddle with the bits in DDB, but you shouldn't do that). Credential rotation is handed through versions. Suppose you do `credstash put foo bar`, and then decide later to rotate `foo`, you can put version 2 of `foo` by doing `credstash put foo baz -v `. The next time you do `credstash get foo`, it will return `baz`. You can get specific credential versions as well (with the same `-v` flag). You can fetch a list of all credentials in the credential-store and their versions with the `list` command.
 
 ## Dependencies
@@ -64,43 +66,70 @@ Once credentials are in place, run `credstash setup`. This will create the DDB t
 
 ## Usage
 ```
-usage: credstash [-h] [-i INFILE] [-k KEY] [-n] [-r REGION] [-t TABLE]
-                 [-v VERSION]
-                 {delete,get,list,put,setup} [credential] [value]
+usage: credstash [-h] [-r REGION] [-t TABLE] {delete,get,list,put,setup} ...
 
 A credential/secret storage system
 
-positional arguments:
-  {delete,get,list,put,setup}
-                        Put, Get, or Delete a credential from the store, list
-                        credentials and their versions, or setup the
-                        credential store
-  credential            the name of the credential to store/get
-  value                 the value of the credential to put (ignored if action
-                        is 'get')
+delete
+    usage: credstash delete [-h] [-r REGION] [-t TABLE] credential
+
+    positional arguments:
+      credential  the name of the credential to delete
+
+get
+    usage: credstash get [-h] [-r REGION] [-t TABLE] [-k KEY] [-n] [-v VERSION]
+                         credential [context [context ...]]
+
+    positional arguments:
+      credential            the name of the credential to get
+      context               encryption context key/value pairs associated with the
+                            credential in the form of "key=value"
+
+    optional arguments:
+      -k KEY, --key KEY     the KMS key-id of the master key to use. See the
+                            README for more information. Defaults to
+                            alias/credstash
+      -n, --noline          Don't append newline to returned value (useful in
+                            scripts or with binary files)
+      -v VERSION, --version VERSION
+                            Get a specific version of the credential (defaults to
+                            the latest version).
+
+list
+    usage: credstash list [-h] [-r REGION] [-t TABLE]
+
+put
+    usage: credstash put [-h] [-r REGION] [-t TABLE] [-i INFILE] [-k KEY] [-v VERSION]
+                         credential value [context [context ...]]
+
+    positional arguments:
+      credential            the name of the credential to store
+      value                 the value of the credential to store
+      context               encryption context key/value pairs associated with the
+                            credential in the form of "key=value"
+
+    optional arguments:
+      -i INFILE, --infile INFILE
+                            store the contents of `infile` rather than provide a
+                            value on the command line
+      -k KEY, --key KEY     the KMS key-id of the master key to use. See the
+                            README for more information. Defaults to
+                            alias/credstash
+      -v VERSION, --version VERSION
+                            Put a specific version of the credential (update the
+                            credential; defaults to version `1`).
+
+setup
+    usage: credstash setup [-h] [-r REGION] [-t TABLE]
 
 optional arguments:
-  -h, --help            show this help message and exit
-  -i INFILE, --infile INFILE
-                        store the contents of `infile` rather than provide a
-                        value on the command line
-  -k KEY, --key KEY     the KMS key-id of the master key to use. See the
-                        README for more information. Defaults to
-                        alias/credstash
-  -n, --noline          Don't append newline to returned value (useful in
-                        scripts or with binary files)
   -r REGION, --region REGION
                         the AWS region in which to operate. If a region is not
                         specified, credstash will use the value of the
                         AWS_DEFAULT_REGION env variable, or if that is not
                         set, us-east-1
   -t TABLE, --table TABLE
                         DynamoDB table to use for credential storage
-  -v VERSION, --version VERSION
-                        If doing a `put`, put a specific version of the
-                        credential (update the credential; defaults to version
-                        `1`). If doing a `get`, get a specific version of the
-                        credential (defaults to the latest version).
 ```
 
 ## Security Notes

credstash.py

@@ -18,6 +18,7 @@
 import boto.kms
 import operator
 import os
+import os.path
 import sys
 import time
 
@@ -45,11 +46,23 @@ def __init__(self, value=""):
     def __str__(self):
         return self.value
 
+class KeyValueToDictionary(argparse.Action):
+    def __call__(self, parser, namespace, values, option_string=None):
+        setattr(namespace, 
+                self.dest, 
+                {x[0] : x[1] for x in values} if len(values) > 0 else None)
+
 
 def printStdErr(s):
     sys.stderr.write(str(s))
     sys.stderr.write("\n")
 
+def is_key_value_pair(string):
+    output = string.split('=')
+    if len(output) != 2:
+        msg = "%r is not the form of \"key=value\"" % string
+        raise argparse.ArgumentTypeError(msg)
+    return output
 
 def listSecrets(region="us-east-1", table="credential-store"):
     '''
@@ -59,14 +72,14 @@ def listSecrets(region="us-east-1", table="credential-store"):
     rs = secretStore.scan(attributes=("name", "version"))
     return [secret for secret in rs]
 
-def putSecret(name, secret, version, kms_key="alias/credstash", region="us-east-1", table="credential-store"):
+def putSecret(name, secret, version, kms_key="alias/credstash", region="us-east-1", table="credential-store", context=None):
     '''
     put a secret called `name` into the secret-store, protected by the key kms_key
     '''
     kms = boto.kms.connect_to_region(region)
     # generate a a 64 byte key. Half will be for data encryption, the other half for HMAC
     try:
-        kms_response = kms.generate_data_key(kms_key, number_of_bytes=64)
+        kms_response = kms.generate_data_key(kms_key, context, 64)
     except:
         raise KmsError("Could not generate key using KMS key %s" % kms_key)
     data_key = kms_response['Plaintext'][:32]
@@ -91,7 +104,7 @@ def putSecret(name, secret, version, kms_key="alias/credstash", region="us-east-
     data['hmac'] = b64hmac
     return secretStore.put_item(data=data)
 
-def getSecret(name, version="", region="us-east-1", table="credential-store"):
+def getSecret(name, version="", region="us-east-1", table="credential-store", context=None):
     '''
     fetch and decrypt the secret called `name`
     '''
@@ -108,9 +121,19 @@ def getSecret(name, version="", region="us-east-1", table="credential-store"):
     kms = boto.kms.connect_to_region(region)
     # Check the HMAC before we decrypt to verify ciphertext integrity
     try:
-        kms_response = kms.decrypt(b64decode(material['key']))
-    except:
-        raise KmsError("Could not decrypt hmac key with KMS")
+        kms_response = kms.decrypt(b64decode(material['key']), context)
+    except boto.kms.exceptions.InvalidCiphertextException:
+        if context is None:
+            msg = ("Could not decrypt hmac key with KMS. The credential may "
+                   "require that an encryption context be provided to decrypt "
+                   "it.")
+        else:
+            msg = ("Could not decrypt hmac key with KMS. The encryption "
+                   "context provided may not match the one used when the "
+                   "credential was stored.")
+        raise KmsError(msg)
+    except Exception as e:
+        raise KmsError("Decryption error %s" % e)
     key = kms_response['Plaintext'][:32]
     hmac_key = kms_response['Plaintext'][32:]
     hmac = HMAC(hmac_key, msg=b64decode(material['contents']), digestmod=SHA256)
@@ -153,21 +176,46 @@ def createDdbTable(region="us-east-1", table="credential-store"):
 
 
 def main():
-    parser = argparse.ArgumentParser(description="A credential/secret storage system")
+    parsers = {}
+    parsers['super'] = argparse.ArgumentParser(description="A credential/secret storage system")
 
-    parser.add_argument("action", type=str, choices=["delete", "get", "list", "put", "setup"], help="Put, Get, or Delete a credential from the store, list credentials and their versions, or setup the credential store")
-    parser.add_argument("credential", type=str, help="the name of the credential to store/get", nargs='?')
-    parser.add_argument("value", type=str, help="the value of the credential to put (ignored if action is 'get')", nargs='?', default="")
+    parsers['super'].add_argument("-r", "--region",  help="the AWS region in which to operate. If a region is not specified, credstash will use the value of the AWS_DEFAULT_REGION env variable, or if that is not set, us-east-1")
+    parsers['super'].add_argument("-t", "--table", default="credential-store", help="DynamoDB table to use for credential storage")
+    subparsers = parsers['super'].add_subparsers(help='Try commands like "{name} get -h" or "{name} put --help" to get each sub command\'s options'.format(name=os.path.basename(__file__)))
+
+    action = 'delete'
+    parsers[action] = subparsers.add_parser(action, help='Delete a credential from the store')
+    parsers[action].add_argument("credential", type=str, help="the name of the credential to delete")
+    parsers[action].set_defaults(action=action)
+
+    action = 'get'
+    parsers[action] = subparsers.add_parser(action, help='Get a credential from the store')
+    parsers[action].add_argument("credential", type=str, help="the name of the credential to get")
+    parsers[action].add_argument("context", type=is_key_value_pair, action=KeyValueToDictionary, nargs='*', help="encryption context key/value pairs associated with the credential in the form of \"key=value\"")
+    parsers[action].add_argument("-k", "--key", default="alias/credstash", help="the KMS key-id of the master key to use. See the README for more information. Defaults to alias/credstash")
+    parsers[action].add_argument("-n", "--noline", action="store_true", help="Don't append newline to returned value (useful in scripts or with binary files)")
+    parsers[action].add_argument("-v", "--version", default="", help="Get a specific version of the credential (defaults to the latest version).")
+    parsers[action].set_defaults(action=action)
+
+    action = 'list'
+    parsers[action] = subparsers.add_parser(action, help='list credentials and their versions')
+    parsers[action].set_defaults(action=action)
 
-    parser.add_argument("-i", "--infile", default="", help="store the contents of `infile` rather than provide a value on the command line")
-    parser.add_argument("-k", "--key", default="alias/credstash", help="the KMS key-id of the master key to use. See the README for more information. Defaults to alias/credstash")
-    parser.add_argument("-n", "--noline", action="store_true", help="Don't append newline to returned value (useful in scripts or with binary files)")
-    parser.add_argument("-r", "--region",  help="the AWS region in which to operate. If a region is not specified, credstash will use the value of the AWS_DEFAULT_REGION env variable, or if that is not set, us-east-1")
-    parser.add_argument("-t", "--table", default="credential-store", help="DynamoDB table to use for credential storage")
-    parser.add_argument("-v", "--version", default="", help="If doing a `put`, put a specific version of the credential (update the credential; defaults to version `1`). If doing a `get`, get a specific version of the credential (defaults to the latest version).")
+    action = 'put'
+    parsers[action] = subparsers.add_parser(action, help='Put a credential into the store')
+    parsers[action].add_argument("credential", type=str, help="the name of the credential to store")
+    parsers[action].add_argument("value", type=str, help="the value of the credential to store", default="")
+    parsers[action].add_argument("context", type=is_key_value_pair, action=KeyValueToDictionary, nargs='*', help="encryption context key/value pairs associated with the credential in the form of \"key=value\"")
+    parsers[action].add_argument("-i", "--infile", default="", help="store the contents of `infile` rather than provide a value on the command line")
+    parsers[action].add_argument("-k", "--key", default="alias/credstash", help="the KMS key-id of the master key to use. See the README for more information. Defaults to alias/credstash")
+    parsers[action].add_argument("-v", "--version", default="", help="Put a specific version of the credential (update the credential; defaults to version `1`).")
+    parsers[action].set_defaults(action=action)
 
+    action = 'setup'
+    parsers[action] = subparsers.add_parser(action, help='setup the credential store')
+    parsers[action].set_defaults(action=action)
 
-    args = parser.parse_args()
+    args = parsers['super'].parse_args()
     region = os.getenv("AWS_DEFAULT_REGION", DEFAULT_REGION) if not args.region else args.region
     if args.action == "delete":
         deleteSecrets(args.credential, region=region, table=args.table)
@@ -189,7 +237,7 @@ def main():
         else:
             value_to_put = args.value
         try:
-            if putSecret(args.credential, value_to_put, args.version, kms_key=args.key, region=region, table=args.table):
+            if putSecret(args.credential, value_to_put, args.version, kms_key=args.key, region=region, table=args.table, context=args.context):
                 print("{0} has been stored".format(args.credential))
         except KmsError as e:
             printStdErr(e)
@@ -198,7 +246,7 @@ def main():
         return 
     if args.action == "get":
         try:
-            sys.stdout.write(getSecret(args.credential, args.version, region=region, table=args.table))
+            sys.stdout.write(getSecret(args.credential, args.version, region=region, table=args.table, context=args.context))
             if not args.noline:
                 sys.stdout.write("\n")
         except ItemNotFound as e:

setup.py

@@ -2,7 +2,7 @@
 
 setup(
     name='credstash',
-    version='1.2',
+    version='1.3',
     description='A utility for managing secrets in the cloud using AWS KMS and DynamoDB',
     license='Apache2',
     url='https://github.com/LuminalOSS/credstash',