diff --git a/docs/documentation/platform/kms/hsm-integration.mdx b/docs/documentation/platform/kms/hsm-integration.mdx index a0677f48a..b5dcbf1c3 100644 --- a/docs/documentation/platform/kms/hsm-integration.mdx +++ b/docs/documentation/platform/kms/hsm-integration.mdx @@ -999,157 +999,135 @@ Enabling HSM encryption has a set of key benefits: - - Before using the CloudHSM client with Kubernetes, it must be configured properly so Infisical can use it for cryptographic operations. - - **1. Download and configure the AWS CloudHSM client** - - You can download the AWS CloudHSM client from [the AWS documentation](https://docs.aws.amazon.com/cloudhsm/latest/userguide/pkcs11-library-install.html). - - - Note that the AWS CloudHSM client is only available for Linux and Windows. - If you're on a different operating system, you'll need to access a Linux machine to configure the client, such as an AWS EC2 Debian instance. - - - After installing the CloudHSM client, you should see all related files in the `/opt/cloudhsm/` directory on your machine. - - You need to run the `configure-pkcs11` binary which will configure the client to connect with your AWS CloudHSM cluster. Depending on if you have multiple HSM's inside your cluster, you'll need to run the command with different arguments. Below you'll find the appropriate command for your use case: - - - - - ```bash - sudo /opt/cloudhsm/bin/configure-pkcs11 -a --disable-key-availability-check - ``` - - - To use a single HSM, you must first manage client key durability settings by setting `disable_key_availability_check` to true by passing the `--disable-key-availability-check` flag. For more information read the [Key Synchronization](https://docs.aws.amazon.com/cloudhsm/latest/userguide/manage-key-sync.html) section in the AWS CloudHSM documentation. - - - - - ```bash - sudo /opt/cloudhsm/bin/configure-pkcs11 -a ... --disable-key-availability-check - ``` - - - - At this point you should have: - 1. [Activated the CloudHSM cluster](https://docs.aws.amazon.com/cloudhsm/latest/userguide/activate-cluster.html) - 2. [Created a Crypto User HSM user](https://docs.aws.amazon.com/cloudhsm/latest/userguide/cloudhsm_cli-user-create.html) - 3. Downloaded and configured the CloudHSM client as described in the previous steps. - - **2. Verify the configuration file** - - After configuring the CloudHSM client, you should notice that the PKCS11 configuration file has been updated to include the HSM's ENI IP address. You can find this file in the `/opt/cloudhsm/etc/cloudhsm-pkcs11.cfg` directory, and it should look like this: - - ```json cloudhsm-pkcs11.cfg - { - "clusters": [ - { - "type": "hsm1", - "cluster": { - // Your issuing CA certificate. - // As per AWS documentation, this defaults to `/opt/cloudhsm/etc/customerCA.crt`. - "hsm_ca_file": "/opt/cloudhsm/etc/customerCA.crt", - "servers": [ - { - "hostname": "", - "port": 2223, - "enable": true - }, - { - "hostname": "", - "port": 2223, - "enable": true - } - ], - // Only relevant if you passed the --disable-key-availability-check flag - "options": { - "disable_key_availability_check": true - } - } - } - ], - "logging": { - "log_type": "file", - "log_file": "/opt/cloudhsm/run/cloudhsm-pkcs11.log", - "log_level": "info", - "log_interval": "daily" - } - } - ``` - - **3. Copy the CloudHSM client files to a staging directory** - - Create a directory to stage the CloudHSM client files for Kubernetes: - - ```bash - mkdir -p /etc/cloudhsm-k8s - ``` - - Copy the entire `/opt/cloudhsm` directory to your staging location: - - ```bash - cp -r /opt/cloudhsm/* /etc/cloudhsm-k8s/ - ``` - - Ensure the configuration file paths are correct. The `cloudhsm-pkcs11.cfg` file should reference `/opt/cloudhsm/etc/customerCA.crt` as shown above, since this is where files will be mounted inside the Kubernetes container. - - - - - On the same machine that you configured the CloudHSM client, you can use `pkcs11-tool` to find the HSM slot number and to verify that the client is working correctly. - - First, install the `pkcs11-tool` package: - - ```bash - sudo apt-get install opensc -y - ``` - - Then, run the following command to find the HSM slot number: - - ```bash - pkcs11-tool --module /opt/cloudhsm/lib/libcloudhsm_pkcs11.so --list-slots --login - ``` - - It'll prompt you to log in with your PIN, which is your username and password separated by a colon. Example: `testUser:testPassword`. - - This will output the HSM slot number like so: - - ```bash - ubuntu@ec-2:~$ pkcs11-tool --module /opt/cloudhsm/lib/libcloudhsm_pkcs11.so --list-slots - Available slots: - Slot 0 (0x2000000000000001): hsm1 - token label : hsm1 - token manufacturer : Marvell Semiconductors, Inc. - token model : LS2 - token flags : login required, rng, token initialized - hardware version : 66.48 - firmware version : 10.2 - serial num : - pin min/max : 8/32 - ``` - - In this case we see that the HSM has a slot in the position of `0`. This slot number will be used in the later steps to set the `HSM_SLOT` environment variable. - - - - When you initialized your HSM, you were prompted to download the cluster CSR and sign it. - In order to use the HSM with Infisical, you need to obtain the issuer CA certificate that was used to sign the cluster CSR. - - If you followed [the official AWS documentation](https://docs.aws.amazon.com/cloudhsm/latest/userguide/initialize-cluster.html), you should have a CA certificate called `customerCA.crt`. - - Copy the CA certificate to your staging directory: - - ```bash - cp /path/to/customerCA.crt /etc/cloudhsm-k8s/etc/customerCA.crt - ``` - - Ensure the file is at `/etc/cloudhsm-k8s/etc/customerCA.crt` as this is what the configuration file expects. - - + + If you're using AWS EKS, you need to specify a storage class for the PVC and ensure that the EBS CSI Driver is installed and running. + + By default, EKS exposes `gp2` as the default storage class. Below are the steps required for setting the default storage class and ensuring the EBS CSI Driver is installed and running: + + + + + + Enable OIDC authentication for the EKS cluster: + ```bash + eksctl utils associate-iam-oidc-provider \ + --region \ + --cluster \ + --approve + ``` + + * Replace `` with your AWS region. + * Replace `` with your cluster name. + + + + + + 1. Check if EBS CSI Driver is installed and running by running the following command: + + ```bash + kubectl get pods -n kube-system | grep ebs-csi + ``` + + If you see no pods, you need to install the EBS CSI Driver as seen in the next step. + + + + Create a new IAM service account for the EBS CSI Driver: + + ```bash + eksctl create iamserviceaccount \ + --name ebs-csi-controller-sa \ + --namespace kube-system \ + --region \ + --cluster \ + --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \ + --approve \ + --role-only \ + --role-name AmazonEKS_EBS_CSI_DriverRole + ``` + + * Replace `` with your cluster name. + * Replace `` with your AWS region. + + Install the EBS CSI Driver: + + ```bash + eksctl create addon \ + --name aws-ebs-csi-driver \ + --cluster \ + --region \ + --service-account-role-arn arn:aws:iam:::role/AmazonEKS_EBS_CSI_DriverRole \ + --force + ``` + + * Replace `` with your cluster name. + * Replace `` with your AWS region. + * Replace `` with your actual account ID. Can be obtained by running `aws sts get-caller-identity --query Account --output text`. + + + + Verify the EBS CSI Driver is installed and running by running the following command: + + ```bash + kubectl get pods -n kube-system | grep ebs-csi + ``` + + You should see an output like this: + + ```bash + kubectl get pods -n kube-system | grep ebs-csi + ebs-csi-controller-6b6bbf996-rvf8r 6/6 Running 0 21s + ebs-csi-controller-6b6bbf996-vk4ng 6/6 Running 0 21s + ebs-csi-node-c6vbb 3/3 Running 0 21s + ebs-csi-node-s9zlr 3/3 Running 0 21s + ``` + + + + You can find the enabled storage class by running the following command: + + ```bash + kubectl get storageclass + ``` + + You should see an output like this: + + ```bash + $ kubectl get storageclass + + NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE + gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 65m + ``` + + In this case, the enabled storage class is `gp2`. + + + + You can set the default PVC storage class by patching the storage class with the following command: + + ```bash + kubectl patch storageclass gp2 -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' + ``` + + This will set the `gp2` storage class as the default storage class. + + + Now when you run `kubectl get storageclass`, you should see that `gp2` is the default storage class. + + ```bash + $ kubectl get storageclass + + NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE + gp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 68m + ``` + + Notice the `(default)` next to the `gp2` storage class. + + + + You need to create a Persistent Volume Claim (PVC) to mount the HSM client files to the Infisical deployment. ```bash @@ -1179,8 +1157,8 @@ Enabling HSM encryption has a set of key benefits: spec: containers: - name: setup - image: busybox - command: ["/bin/sh", "-c", "sleep 3600"] + image: debian:bookworm + command: ["/bin/sh", "-c", "sleep 7200"] volumeMounts: - name: cloudhsm-data mountPath: /data @@ -1191,27 +1169,226 @@ Enabling HSM encryption has a set of key benefits: EOF ``` - The above command will create a pod named `cloudhsm-setup-pod` with a busybox image. The pod will sleep for 3600 seconds _(one hour)_, which is enough time to upload the HSM client files to the PVC. + The above command will create a pod named `cloudhsm-setup-pod` with a Debian image. The pod will sleep for 7200 seconds _(two hours)_, which is enough time to set up the PVC and configure the HSM client. Ensure that the pod is running and is healthy by running the following command: ```bash - kubectl wait --for=condition=Ready pod/cloudhsm-setup-pod --timeout=60s + kubectl wait --for=condition=Ready pod/cloudhsm-setup-pod --timeout=120s ``` + - Next we need to copy the HSM client files into the PVC. + - ```bash - kubectl exec cloudhsm-setup-pod -- mkdir -p /data/ # Create the data directory - kubectl cp /etc/cloudhsm-k8s/. cloudhsm-setup-pod:/data/ # Copy the HSM client files into the PVC - kubectl exec cloudhsm-setup-pod -- chmod -R 755 /data/ # Set the correct permissions for the HSM client files - ``` + We need to configure the PVC to work with the CloudHSM, so Infiscial can consume the HSM client files. - Finally, we are ready to delete the temporary pod, as we have successfully uploaded the HSM client files to the PVC. This step may take a few minutes to complete. + **2.1. Start a shell in the PVC pod:** + + This will allow us to run commands directly within the setup pod. We'll use this to configure the CloudHSM client and to validate that it's working correctly. + + ```bash + kubectl exec -it cloudhsm-setup-pod -- /bin/sh + ``` + + **2.2. Create the data directory:** + + This will create a directory called `/data/` within the setup pod. This directory will remain even after the setup pod is deleted, as it's mounted as a volume from the PVC. + + ```bash + mkdir -p /data/ + ``` + + **2.3. Install the necessary packages:** + + This will install the necessary packages to allow us to test and install the CloudHSM client. + + ```bash + apt-get update -y + apt-get install opensc telnet wget -y + ``` + + **2.4. Try to reach the HSM device:** + + We need to validate that we're able to reach the HSM device from within Kubernetes. You can use telnet to ping the HSM device like so: + + ```bash + telnet 2223 + ``` + + You should see an output like this: + ```bash + $ telnet 2223 + Trying ... + Connected to . + ``` + + If it gets stuck on `Trying ....`, you may have configured your HSM client's security group incorrectly. Make sure you configure the security group to allow traffic from EKS on port 2223-2225. + + **2.5. Install the AWS CloudHSM client:** + + The Infisical images run on Debian, so we need to install a Debian-compatible version of the AWS CloudHSM client. + + ```bash + wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/Jammy/cloudhsm-pkcs11_latest_u22.04_amd64.deb + apt-get install ./cloudhsm-pkcs11_latest_u22.04_amd64.deb -y + ``` + + **2.6. Configure the CloudHSM client:** + + After installing the CloudHSM client, you should see all related files in the `/opt/cloudhsm/` directory on the CloudHSM setup pod. + + You need to run the `configure-pkcs11` binary which will configure the client to connect with your AWS CloudHSM cluster. Depending on if you have multiple HSM's inside your cluster, you'll need to run the command with different arguments. Below you'll find the appropriate command for your use case: + + + + + ```bash + /opt/cloudhsm/bin/configure-pkcs11 -a --disable-key-availability-check + ``` + + + To use a single HSM, you must first manage client key durability settings by setting `disable_key_availability_check` to true by passing the `--disable-key-availability-check` flag. For more information read the [Key Synchronization](https://docs.aws.amazon.com/cloudhsm/latest/userguide/manage-key-sync.html) section in the AWS CloudHSM documentation. + + + + + ```bash + /opt/cloudhsm/bin/configure-pkcs11 -a ... --disable-key-availability-check + ``` + + + + **2.7. Verify the CloudHSM client is configured correctly:** + + You can verify the CloudHSM client is configured correctly by running the following command: + ```bash + cat /opt/cloudhsm/etc/cloudhsm-pkcs11.cfg + ``` + + You should see an output like this: + + ```json + { + "clusters": [ + { + "type": "hsm1", + "cluster": { + "hsm_ca_file": "/opt/cloudhsm/etc/customerCA.crt", + "servers": [ + { + "hostname": "172.31.39.155", + "port": 2223, + "enable": true + } + ], + "options": { + "disable_key_availability_check": true + } + } + } + ], + "logging": { + "log_type": "file", + "log_file": "/opt/cloudhsm/run/cloudhsm-pkcs11.log", + "log_level": "info", + "log_interval": "daily" + } + } + ``` + + **2.8. Exit the pod:** + + Exit the pod by running the following command: + ```bash + exit + ``` + + **2.9. Copy your issuer CA certificate to the PVC:** + + When you initialized your HSM, you were prompted to download the cluster CSR and sign it. + In order to use the HSM with Infisical, you need to obtain the issuer CA certificate that was used to sign the cluster CSR. + + If you followed [the official AWS documentation](https://docs.aws.amazon.com/cloudhsm/latest/userguide/initialize-cluster.html), you should have a CA certificate called `customerCA.crt`. + + Copy the CA certificate from your local machine to the setup pod: + + ```bash + kubectl cp /path/to/customerCA.crt cloudhsm-setup-pod:/opt/cloudhsm/etc/customerCA.crt + ``` + + Ensure that the file is at `/opt/cloudhsm/etc/customerCA.crt` inside the setup pod by running the following command: + ```bash + kubectl exec -it cloudhsm-setup-pod -- cat /opt/cloudhsm/etc/customerCA.crt + ``` + + **2.10. Test the HSM client:** + + Finally, after we're done configuring the HSM client, we need to test it to ensure that it's working correctly. + + First, start a new shell into the setup pod by running the same shell command as before: + ```bash + kubectl exec -it cloudhsm-setup-pod -- /bin/sh + ``` + + Next, try generating a random 32 bytes long string by running the following command: + ```bash + pkcs11-tool --module /opt/cloudhsm/lib/libcloudhsm_pkcs11.so \ + --login --pin : \ + --generate-random 32 | base64 + ``` + + You should see an output like this: + ```bash + Using slot 0 with a present token (0x2000000000000001) + av1dlhVEsssjpcTNS+ysGUoKWH6+/PCaEDIdal5oQc0= + ``` + + + + **2.12. Copy the configured client to the PVC:** + + Copy from the HSM files into the `/data` directory in the PVC, which is what will be mounted for the Infisical deployment. + ```bash + cp -r /opt/cloudhsm/. /data/ + ``` + + Verify the files were copied correctly by running the following command: + ```bash + ls -la /data/ + ``` + + You should see an output like this: + ```bash + drwxr-xr-x. 8 root root 4096 Oct 13 18:50 . + drwxr-xr-x. 1 root root 131 Oct 13 18:29 .. + drwxr-xr-x. 2 root root 4096 Oct 13 18:50 bin + drwxr-xr-x. 3 root root 4096 Oct 13 18:50 doc + drwxr-xr-x. 2 root root 4096 Oct 13 18:50 etc + drwxr-xr-x. 3 root root 4096 Oct 13 18:50 include + drwxr-xr-x. 2 root root 4096 Oct 13 18:50 lib + drwxr-xr-t. 2 root root 4096 Oct 13 18:50 run + ``` + + **2.13. Set the correct permissions for the HSM client files:** + + ```bash + chmod -R 755 /data/ + ``` + + **2.14. Exit the pod:** + + Exit the pod by running the following command: + ```bash + exit + ``` + + **2.15. Delete the setup pod:** + + Delete the setup pod by running the following command: + ```bash + kubectl delete pod cloudhsm-setup-pod + ``` - ```bash - kubectl delete pod cloudhsm-setup-pod - ``` @@ -1304,4 +1481,16 @@ To disable HSM encryption, navigate to Infisical's Server Admin Console and set In order to disable HSM encryption, the Infisical instance must be able to access the HSM device. If the HSM device is no longer accessible, you will not be able to disable HSM encryption. - \ No newline at end of file + + + + +## Troubleshooting + + + + If you're using EKS, you may encounter an error like the following: `no persistent volumes available for this claim and no storage class is set` + + + + \ No newline at end of file