Deploy private CAs with CloudFormation
TL;DR deploy this demo 🤓
If you ever find yourself need to programatically issue X.509 certificates to private servers without having to manage your own PKI infrstructure, you probably looked at the AWS option called Certificate Manager Private Certificate Authority. It was released circa 2018, but as of the time of writing (September 2019), it has not ascended to the level of CloudFormation support.
We've developed a template to roll out a subordinate CA and all the dependent resources, including CloudFront and S3 for CRL using CloudFormation, without any manual intervention. Although it is not currently possible to roll out a Root CA, since AWS haven't implemented a step to import the Root CA certificate in their API/SDK, this method works for a subordinate CA deployment with signing key securely kept in SSM.
Our solution uses the generic-custom-resource-provider as well as a helpful secret-provider resource to generate and store the RSA signing key in the AWS SSM Parameter Store, so that the key never leaves CFN/AWS.
Most of the secret sauce is in the pca.yml
nested template, which implements this common flow:
- generate self-signed certificate using private key in SSM (
GenerateSelfSignedCertV1
) - create a subordinate CA (
PCASubordinateS1
) - retrieve the CSR (
CSRSubordinateS1
) - sign the CSR using self-signed certificate (
SignCSRSubordinateS1
) - import the signed certificate and activate the CA (
ImportSubordinateCertificateS1
)
Note: the S1
notation stands for Server CA v1
Since all the methods dealing with certficate data require bytes
encoding, we can not just use the standard CFN/boto proxy implemented in generic-custom-resource-provider
. Therefore we had to extend our resource provider with custom
plugin architecture and then implemented our own handlers in acm_pca.yml
(e.g.):
#!/usr/bin/env python
import os
import sys
import re
import random
import boto3
from OpenSSL import crypto
from OpenSSL import SSL
from datetime import datetime
from base64 import b64decode
class ACM_PCA:
authorityKeyIdentifier = False
def __init__(self, *args, **kwargs):
self.verbose = bool(int(os.getenv('VERBOSE', 0)))
if self.verbose: print(
'args: {}, kwargs: {}'.format(args, kwargs),
file=sys.stderr
)
def sign_csr(self, *args, **kwargs):
if self.verbose: print(
'args: {}, kwargs: {}'.format(args, kwargs),
file=sys.stderr
)
private_key = self.load_private_key(
self.get_private_key_pem(kwargs['PrivateKey'])
)
try:
csr_pem = b64decode(kwargs['Csr']).decode()
except:
csr_pem = kwargs['Csr']
try:
csr_payload = re.search(
'-----BEGIN CERTIFICATE REQUEST-----(.*)-----END CERTIFICATE REQUEST-----',
csr_pem,
re.IGNORECASE
)
assert csr_payload
csr_formatted = '{}{}{}'.format(
'-----BEGIN CERTIFICATE REQUEST-----',
csr_payload.group(1).replace(' ', '\n').replace('\\n', '\n'),
'-----END CERTIFICATE REQUEST-----'
)
csr_pem = csr_formatted
except:
pass
csr = crypto.load_certificate_request(
crypto.FILETYPE_PEM,
csr_pem
)
if self.verbose: print(
'csr: {}'.format(
crypto.dump_certificate_request(crypto.FILETYPE_TEXT, csr).decode()
),
file=sys.stderr
)
try:
cert_pem = b64decode(kwargs['CACert']).decode()
except:
cert_pem = kwargs['CACert']
ca_cert = crypto.load_certificate(
crypto.FILETYPE_PEM,
cert_pem
)
if self.verbose: print(
'ca_cert: {}'.format(
crypto.dump_certificate(crypto.FILETYPE_TEXT, ca_cert).decode()
),
file=sys.stderr
)
cert = crypto.X509()
cert.set_version(2)
cert.set_serial_number(random.randint(
1000000000000000000000000000000000000,
9999999999999999999999999999999999999
))
cert.set_issuer(ca_cert.get_subject())
cert.set_subject(csr.get_subject())
cert.set_pubkey(csr.get_pubkey())
cert.add_extensions([
crypto.X509Extension(
b'basicConstraints',
True,
b'CA:TRUE'
),
crypto.X509Extension(
b'subjectKeyIdentifier',
False,
b'hash',
subject=cert
),
crypto.X509Extension(
b'keyUsage',
True,
b'digitalSignature, cRLSign, keyCertSign'
)
])
if self.authorityKeyIdentifier:
cert.add_extensions([
crypto.X509Extension(
b'authorityKeyIdentifier',
False,
b'keyid:always,issuer',
issuer=ca_cert
)
])
cert.gmtime_adj_notBefore(0)
cert.gmtime_adj_notAfter(int(kwargs['ValidityInSeconds']))
cert.sign(private_key, kwargs['Digest'])
if self.verbose: print(
'cert: {}'.format(
crypto.dump_certificate(crypto.FILETYPE_TEXT, cert).decode()
),
file=sys.stderr
)
return {
'Certificate': crypto.dump_certificate(crypto.FILETYPE_PEM, cert).decode()
}
...
The CFN custom resource which calls the above method looks like this:
SignCSRSubordinateS1:
Type: 'Custom::SignCSRSubordinateS1'
Condition: HasSubordinateCSR
Version: 1.0
Properties:
ServiceToken: !Sub 'arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:generic-custom-resource-provider'
Version: R1
# custom module: acm_pca.py
AgentService: acm_pca
AgentType: custom
AgentCreateMethod: sign_csr
AgentCreateArgs: !Join
- ''
- - !Sub '{ "PrivateKey": "/rsa-private-keys/${NameTag}/key_pair",'
- '"CACert": "'
- Fn::Base64:
!Sub '${GenerateSelfSignedCertV1.Certificate}'
- '",'
- '"Csr": "'
- Fn::Base64:
!Sub '${CSRSubordinateS1.Csr}'
- '",'
- '"ValidityInSeconds": 315360000,'
- '"Digest": "sha256" }
To get out of \n
escape hell, we base64
encode certificate and CSR data using internal CFN function before passing them on. In order to pass interger values, we need to specify AgentCreateArgs
as properly formatted JSON, so since there is no short-hand notation for Fn::Base64
it looks a bit messy, but is functional.
Since the custom plugin requires access to crypto functions, we build all the dependencies in a Docker container using lambci/lambda:build-python3.7
image and package using make
. The corresponding Makefile
looks like this:
all: clean compile-libs debug
compile-libs:
@docker run --rm \
-v `pwd`:/src \
-w /src \
lambci/lambda:build-python3.7 \
bash -c '''yum update -y\
&& yum groupinstall "Development Tools" -y\
&& yum install -y ibffi openssl-devel\
&& pip3 install virtualenv\
&& export VIRTUAL_ENV=/src/venv\
&& python3 -m venv $${VIRTUAL_ENV}\
&& export PATH="$${VIRTUAL_ENV}/bin:$${PATH}"\
&& pip3 install --upgrade pip\
&& pip3 install --upgrade --force -r ./requirements.txt -t .''' \
&& rm -rf venv
clean:
@rm -rf enum*; find . -name '*.so' -delete
debug:
@find . -name '*.so'
.PHONY: all
Once your subordinate private CA is up and running in ACTIVE
state, you could issue_certificate
to any server with access to openssl
and curl
, by generating a CSR and then chaining aws acm-pca issue-certificate ...
followed by aws acm-pca certificate-issued ...
and lastly aws acm-pca get-certificate ...
.
After the certificate is installed in your web server/application, all that is left to do is to make sure your clients trust your root CA, by importing its certificate into the corresponding certificate store on each client.
A demo stack is available as a starting point and can be extended with additional custom resources to issue certificates, etc.
--belodetek 😬