ExternalDNS & Cert Manager
ExternalDNS and Cert Manager are two well known integrations within the Kubernetes ecosystem that can be used in conjunction to automate the creation of TLS certificates.
Before you begin
-
Follow the Get started guide to install K8sGateway, set up a gateway resource, and deploy the httpbin sample app.
-
Get the external address of the gateway and save it in an environment variable.
export INGRESS_GW_ADDRESS=$(kubectl get svc -n gloo-system gloo-proxy-http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}") echo $INGRESS_GW_ADDRESS
kubectl port-forward deployment/gloo-proxy-http -n gloo-system 8080:8080
Set up ExternalDNS
ExternalDNS is used to dynamically set up and control DNS records for discovered gateway and HTTP resources. When you create a gateway or HTTP resource, and you define a hostname, External DNS uses the external address that is assigned to the gateway’s load balancer service that serves this hostname, and uses this information to create a DNS record in the DNS provider that you configured.
You can later use Cert Manager to create TLS certificates for this hostname so that you can serve HTTPS traffic on your gateway.
-
Create an HTTP route resource to expose httpbin on your domain. Replace
<my-domain.com>
with your domain. Note that you must own the domain to enable ExternalDNS to create DNS records on your behalf.kubectl apply -f- <<EOF apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: httpbin namespace: httpbin labels: example: httpbin-route spec: parentRefs: - name: http namespace: gloo-system hostnames: - "<my-domain.com>" rules: - backendRefs: - name: httpbin port: 8000 EOF
-
Deploy the ExternalDNS components. The following example configures ExternalDNS to monitor gateway and HTTP route resources to determine the list of DNS records that must be created or changed. DNS records are set up in DigitalOcean. To find the ExternalDNS configuration for your DNS provider, see the Kubernetes documentation.
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: ServiceAccount metadata: name: external-dns namespace: default --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: external-dns rules: - apiGroups: [""] resources: ["namespaces"] verbs: ["get","watch","list"] - apiGroups: ["gateway.networking.k8s.io"] resources: ["gateways","httproutes","grpcroutes","tlsroutes","tcproutes","udproutes"] verbs: ["get","watch","list"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: external-dns roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: external-dns subjects: - kind: ServiceAccount name: external-dns namespace: default --- apiVersion: apps/v1 kind: Deployment metadata: name: external-dns spec: replicas: 1 selector: matchLabels: app: external-dns strategy: type: Recreate template: metadata: labels: app: external-dns spec: serviceAccountName: external-dns containers: - name: external-dns image: registry.k8s.io/external-dns/external-dns:v0.13.5 args: - --source=gateway-httproute - --provider=digitalocean - --log-level=debug env: - name: DO_TOKEN value: "<my-token>" EOF
-
Wait for the DNS entry to get created. Note that depending on the DNS provider that you use, this process can take some time to complete. To verify that the DNS record is created, use the
dig
command as shown in the following example.dig <my-domain.com>
Example output for a successfully created DNS record:
;; ANSWER SECTION: <my-domain.com> 300 IN A 164.90.241.80
Set up Cert Manager
Cert Manager is a Kubernetes controller that helps you automate the process of obtaining and renewing certificates from various PKI providers, such as AWS Private CA, Google Cloud CA, or Vault. In this example, you learn how to install Cert Manager by using Helm and how to configure it to obtain TLS certificates for your domain from Let’s Encrypt.
--feature-gates=ExperimentalGatewayAPISupport=true
during the Helm installation.
-
Install Cert Manager.
- Add the Jetstack Helm repository.
helm repo add jetstack https://charts.jetstack.io --force-update
- Install Cert Manager in your cluster.
helm upgrade --install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace \ --set "extraArgs={--feature-gates=ExperimentalGatewayAPISupport=true}" --set installCRDs=true
- Add the Jetstack Helm repository.
-
Create an issuer resource that represents the Certificate Authority (CA) that you want to use to issue the TLS certificates for your domain. In this example, you configure Cert Manager to obtain a Let’s Encrypt certificate by using the ACME protocol. To automate domain validation and certificate issuance, you use the
http01
challenge. Thehttp01
challenge is designed to prove that you have control over your domain by requiring you to store a challenge token in your cluster so that Let’s Encrypt can validate it. For more information about this challenge, see the Let’s Encrypt documentation.kubectl apply -f- <<EOF apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: letsencrypt-http namespace: gloo-system spec: acme: email: [email protected] #You can switch to live URL if you are brave #server: https://acme-v02.api.letsencrypt.org/directory server: https://acme-staging-v02.api.letsencrypt.org/directory privateKeySecretRef: name: letsencrypt-http-issuer-account-key solvers: - http01: gatewayHTTPRoute: parentRefs: - name: http namespace: gloo-system kind: Gateway EOF
-
Verify that your TLS certificates are created successfully. Note that depending on the CA that you use, this process might take a while to complete.
kubectl get issuer letsencrypt-http -n gloo-system
Example output for successfully issued TLS certificates:
Status: Acme: Conditions: Last Transition Time: 2023-11-09T16:03:58Z Message: The ACME account was registered with the ACME server Observed Generation: 1 Reason: ACMEAccountRegistered Status: True Type: Ready
-
Verify that the TLS certificate was added to the secret that you configured in the Cert Manager issuer resource.
kubectl get secret letsencrypt-http-issuer-account-key -n gloo-system -o yaml
Configure an HTTPS listener on your gateway
-
Add an HTTPS listener to the gateway that you set up as part of the Get started guide. Replace
<my-domain.com>
with your domain.kubectl apply -f- <<EOF apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: http annotations: cert-manager.io/issuer: letsencrypt-http namespace: gloo-system spec: gatewayClassName: gloo-gateway listeners: - allowedRoutes: namespaces: from: All name: http port: 80 protocol: HTTP - allowedRoutes: namespaces: from: All hostname: <my-hostname.com> name: https port: 443 protocol: HTTPS tls: mode: Terminate certificateRefs: - name: letsencrypt-http-issuer-account-key kind: Secret EOF
-
Verify that the gateway is configured successfully. You can also review the external address that is assigned to the gateway.
kubectl get gateway http -n gloo-system
Example output for an AWS EKS cluster:
NAME CLASS ADDRESS PROGRAMMED AGE http gloo-gateway a3a6c06e2f4154185bf3f8af46abf22e-139567718.us-east-2.elb.amazonaws.com True 93s
Test your HTTPS listener
With the TLS certificate in place, you can now test your HTTPS listener.
Send a curl request to the httpbin app on the domain that you configured.
curl -vik https://<my-domain.com>/status/200
Example output:
* Trying 164.90.241.80:443...
* Connected to <my-domain.com> (164.90.241.80) port 443 (#0)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
* ALPN: server accepted h2
* Server certificate:
* subject: CN=<my-domain.com>
* start date: Nov 9 15:32:59 2023 GMT
* expire date: Feb 7 15:32:58 2024 GMT
* issuer: C=US; O=Let's Encrypt; CN=R3
* SSL certificate verify ok.
* using HTTP/2
* h2h3 [:method: GET]
* h2h3 [:path: /status/200]
* h2h3 [:scheme: https]
* h2h3 [:authority: <my-domain.com>]
* h2h3 [user-agent: curl/7.88.1]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x12c812800)
> GET /status/200 HTTP/2
> Host: <my-domain.com>
> user-agent: curl/7.88.1
> accept: */*
>
< HTTP/2 200
HTTP/2 200
< access-control-allow-credentials: true
access-control-allow-credentials: true
< access-control-allow-origin: *
access-control-allow-origin: *
< date: Thu, 09 Nov 2023 17:28:17 GMT
date: Thu, 09 Nov 2023 17:28:17 GMT
< content-length: 0
content-length: 0
< x-envoy-upstream-service-time: 2
x-envoy-upstream-service-time: 2
< server: envoy
server: envoy
<
* Connection #0 to host <my-domain.com> left intact