HTTPS/TLS Setup on Kloudfuse Stack
- Harold Lim
- Swapnil Kulkarni
- Pankaj Thakkar
- 1 Instructions for Setting HTTPS/TLS on the Kloudfuse Ingress
- 1.1 Prerequisites
- 1.2 Helm Values Configuration
- 1.2.1 GCP/Azure
- 1.2.2 AWS
- 1.2.3 Disable HTTP traffic
- 2 Instructions to Configure Kloudfuse stack to work with TLS Termination
The Kloudfuse helm chart supports 2 options for setting up HTTPS/TLS on the Kloudfuse Stack: 1. Configuring HTTPS/TLS on the Kubernetes Ingress rules. 2. TLS termination before traffic hits the Kloudfuse stack.
Note that, if TLS is enabled, the Grafana settings for alerts need to be updated. Instructions is at the end of this document.
Instructions for Setting HTTPS/TLS on the Kloudfuse Ingress
Note. If using AWS Certificate Manager (ACM) to terminate the tls on the AWS NLB, skip to the next section for instructions.
This section describes setting up HTTPS/TLS on the Ingress rules. The Kloudfuse helm chart supports creation of the certificates, using cert-manager and lets-encrypt or use an existing certificate.
Prerequisites
A DNS host name is required.
A static external IP address associated with the DNS host name that can be used by the ingress load balancer.
For GKE, refer to https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address for reserving a static external IP.
For AWS, refer to https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html for allocating IP addresses from AWS Elastic IP.
Step 3.a. is only needed if SSL certificate will be created as part of Kfuse installation. Certificate will be created by the cert-manager using lets-encrypt.
3. a. cert-manager must already be installed in the kubernetes cluster. Refer to https://cert-manager.io/docs/installation/ or follow the instructions below.
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager --create-namespace --namespace kfuse --version v1.7.1 --set installCRDs=true
Step 3.b. is needed if existing SSL certificate will be used.
3.b. Create a Kubernetes secret of type TLS with the server.crt and server.key files (SSL certificates).
kubectl create secret tls kfuse-tls --namespace kfuse --key server.key --cert server.crtHelm Values Configuration
Add the following entries in the custom_values.yaml depending on the cloud provider.
GCP/Azure
tls:
enabled: true
host: <REPLACE DNS HOST HERE>
email: <REPLACE EMAIL>
# If Step 3.a.
clusterIssuer: <PROVIDE A CLUSTER ISSUER NAME. ensure that this is unique within the cluster, leave blank if providing own ssl certificate>
# If Step 3.b.
secretName: <PROVIDE secret name containing SSL certificate, e.g., kfuse-tls in Step 3.b. example command>
ingress-nginx:
controller:
ingressClassResource:
enabled: true
name: kfuse-ingress
ingressClass: kfuse-ingress
watchIngressWithoutClass: false
service:
loadBalancerIP: <REPLACE PRE-ALLOCATED STATIC IP HERE>
external:
enabled: trueAWS
For AWS, instead of directly specifying the IP address, in the ingress loadbalancerIP, assignment is done through
service annotations.
Note that the number of Elastic IP allocations must match the number of subnets the AWS NLB uses (defaults to 3).
Refer to Route TCP and UDP traffic with Network Load Balancers - Amazon EKS and
Annotations - AWS Load Balancer Controller
Note: on some older cluster of EKS, the following annotation:
may need to be replaced with
Disable HTTP traffic
If you want to disable HTTP traffic completely, add the following to ingress-nginxsection in values.yaml
Note that this will disable HTTP traffic on both internal and external ingress. So use this config if you want to disable HTTP, and you’re not using internal ingress.
Instructions to Configure Kloudfuse stack to work with TLS Termination
The Kloudfuse stack can be configured to work on clusters that have TLS termination (e.g., using OKTA, AWS ACM).
Minimal instructions
Note: Skip to the next section for more detailed instructions if using AWS Certificate Manager and AWS NLB for TLS termination.
At the minimum, add the following entries in the custom_values.yaml.
The third-party service that does the TLS termination will need to be updated to point to the external ip of the kfuse-ingress-controller service.
Instructions to configure TLS termination on AWS NLB using AWS Certificate Manager
AWS supports configuring the TLS termination directly on the AWS NLB created that backs the Kfuse ingress service.
Prerequisites
A DNS host name is required.
A certificate associated with the above DNS host name issued by the AWS Certificate Manager.
Elastic IP(s) associated with the DNS host name that can be used by the ingress load balancer.
Refer to Elastic IP addresses - Amazon Elastic Compute Cloud for allocating IP addresses from AWS Elastic IP.
Helm Values Configuration
Following are the example configurations for nginx-ingress with their use-cases
Minimal configuration of nginx-ingress controller with ACM
Configuration of nginx-ingress controller, enabling host based routing (Only External Load Balancer)
This will restrict the Kfuse access only to the host specified in the configuration.
Configuration of nginx-ingress controller, enabling host based routing (Both Internal and External Load Balancer)
This will restrict the Kfuse access only to the host (internal & external) specified in the configuration. In this configuration the the internal load balancer can only be used for ingest path i.e. ingesting MELT data and query path is only available via external load balancer.
Configure internal load balancer for sending agent data on internal network
Ensure that the certificate created for external load balancer has been updated to contain the domain-name for the internal load balancer. Update the DNS entries as required. ACM only allows immediate subdomains (*.domain.com) or sibling domains (internal.domain.com). For example; if the kfuse stack is hosted at observability.domain.com; internal traffic can be routed to internal.observability.domain.com; For this the certificate can contain the two domain names. Once the ACM certificate has been resolved; update the custom_values.yaml ingress section for internal controller.