Securely on-board Nginx application on Kubernetes using Vault
This write-up is not about what is vault and how it works. Click here to know how it works. This write-up demonstrate secure on-boarding of an application to a platform.
The idea is to authenticate each on-boarding step with vault.
Consider 3 entities as follows:
- Platform (Kubernetes)
- Application (Nginx)
- Vault
Prepare platform: Creating RBAC (Role Back Access Controls) on Kubernetes
Click here to know about RBAC in detail.
## Create a service account for vault and kubernetes cluster authentication.kind: ServiceAccount
apiVersion: v1
metadata:
name: vault-auth
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:auth-delegator
metadata:
name: role-token-review-binding
subjects:
- kind: ServiceAccount
name: vault-auth
namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: example
rules:
- verbs: ["*"]
apiGroups: [""]
resources: ["secrets"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: example-role-binding
subjects:
- kind: ServiceAccount
name: vault-auth
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: example
system:auth-delegator is a default cluster role and it allows delegated authentication and authorization checks. This is commonly used by add-on API servers for unified authentication and authorization.
Prepare Vault: Deploying vault on kubernetes.
Apply following YAML to spin vault.
## Deploy a vault in kubernetes cluster. Currently it is deployed in DEV mode.kind: Deployment
apiVersion: apps/v1
metadata:
name: vault
spec:
selector:
matchLabels:
app: vault
template:
metadata:
labels:
app: vault
spec:
containers:
- name: vault
image: vault
env:
- name: VAULT_DEV_ROOT_TOKEN_ID
value: myroot
- name: VAULT_DEV_LISTEN_ADDRESS
value: 0.0.0.0:8200
- name: VAULT_ADDR
value: http://127.0.0.1:8200
- name: VAULT_TOKEN
value: myroot
ports:
- containerPort: 8200
name: http
- containerPort: 1234
name: discovery
---
kind: Service
apiVersion: v1
metadata:
name: vault
spec:
selector:
app: vault
ports:
- port: 8200
name: http
- port: 1234
name: discovery
It also exposes a dns with the name vault for intra-cluster communication.
Following steps need to be executed inside vault container.
Create vault policy for the secret myapp. This should be executed from container lifecycle post start placeholder.
cat <<EOF | vault policy write myapp-kv-ro -
path "secret/myapp/*" { capabilities = ["read", "list"] }
path "secret/data/myapp/*" { capabilities = ["read", "list"] }
EOF Enable userpass auth mode for a test-user. This test-user will be use to put secrets in vault. These secrets will only be accessible to users with policy myapp-kv-ro
vault auth enable userpass
vault write auth/userpass/users/test-user \
password=training \
policies=myapp-kv-ro
vault login -method=userpass \
username=test-user \
password=trainingvault kv put secret/myapp/config username=us3rn@m3 password=s3cr3t
vault kv get secret/myapp/config
Get out of vault and retrieve Kubernetes cert using service account role vault-auth.
export auth_name=$(kubectl get sa vault-auth -o jsonpath="{.secrets[*]['name']}") export auth_cert=$(kubectl get secret $auth_name -o jsonpath="{.data['ca\.crt']}" | base64 --decode; echo)
Again get inside vault container enable Kubernetes authentication on vault using the cert obtained in previous step. The value of cert can be obtained from the variable auth_cert from previous step.
vault auth enable kubernetes
vault write auth/kubernetes/config kubernetes_host=https://192.168.99.100:8443 kubernetes_ca_cert="$auth_cert"Specify the role ‘example’ that will issue auth token to every agent.
vault write auth/kubernetes/role/example \
bound_service_account_names=vault-auth \
bound_service_account_namespaces=default \
policies=myapp-kv-ro \
ttl=24hTest the Kubernetes auth method to ensure that you can authenticate with Vault.
kubectl run --generator=run-pod/v1 tmp --rm -i --tty --serviceaccount=vault-auth --image alpine:3.7Get inside alpine pod and execute following scripts to login the vault using platform JWT.
apk update
apk add curl jq
VAULT_ADDR=http://vault:8200
curl -s $VAULT_ADDR/v1/sys/health | jq
KUBE_TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)echo $KUBE_TOKEN
curl --request POST \
--data '{"jwt": "'"$KUBE_TOKEN"'", "role": "example"}' \
$VAULT_ADDR/v1/auth/kubernetes/login | jq
Prepare Application: Deploy Nginx on Kubernetes and access vault secrets using platform role. Role name is “example”.
Apply following YAML to spin Nginx. Observe ConfigMap that has vault agent template which will execute inside init-container to obtain JWT from platform.
## for Maintaining nginx and vault configurations.
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-config
data:
index.html: |
<html>
<body>
<p>Some secrets:</p>
<ul>
<li><pre>username: .Data.username </pre></li>
<li><pre>password: .Data.password </pre></li>
</ul>
</body>
</html>
---
kind: ConfigMap
apiVersion: v1
metadata:
name: vault-agent-config
data:
vault-agent-config.hcl: |
exit_after_auth = true
pid_file = "/home/vault/pidfile"
auto_auth {
method "kubernetes" {
mount_path = "auth/kubernetes"
config = {
role = "example"
}
}
sink "file" {
config = {
path = "/home/vault/.vault-token"
}
}
}
---## Nginx as a microservice deployment. It queries vault for retrieving secrets.kind: Pod
apiVersion: v1
metadata:
name: vault-agent-example
spec:
serviceAccountName: vault-auth
restartPolicy: Never
volumes:
- name: temp
emptyDir: {}
- name: vault-token
emptyDir:
medium: Memory
- name: nginx-config
configMap:
name: nginx-config
defaultMode: 0777
items:
- key: index.html
path: index.html
- name: config
configMap:
name: vault-agent-config
items:
- key: vault-agent-config.hcl
path: vault-agent-config.hcl
initContainers:
- name: vault-agent-auth
image: vault
env:
- name: VAULT_ADDR
value: http://vault:8200
args: [ "agent", "-config=/etc/vault/vault-agent-config.hcl", "-log-level=debug" ]
volumeMounts:
- mountPath: /etc/vault
name: config
- mountPath: /home/vault
name: vault-token
- name: inject-secrets
image: gempesaw/curl-jq
volumeMounts:
- mountPath: /home/vault
name: vault-token
- mountPath: /etc/secrets
name: nginx-config
readOnly: false
- mountPath: /usr/secret
name: temp
readOnly: false
command:
- /bin/sh
- -c
args:
- >
curl --header "X-VAULT-TOKEN: `cat /home/vault/.vault-token`" http://vault:8200/v1/secret/data/myapp/config > /usr/secret/data.json;
export username=$(cat /usr/secret/data.json | jq -r .data.data.username) && cat /etc/secrets/index.html | sed "s/.Data.username/$username/g" > /usr/secret/index.html;
export password=$(cat /usr/secret/data.json | jq -r .data.data.password) && sed -i "s/.Data.password/$password/g" /usr/secret/index.html;
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- mountPath: /usr/share/nginx/html
name: temp
Access the Nginx container at port 80 to view the secrets stored in secret/myapp/config.
Summary
I’ve shared a concept of 3-step authentication to on-board an application to your platform.
- Vault authenticate platform.
- Platform authenticate application.
- Application complete the cycle by authentication from vault.
