Doing things on the internet.

How to set up a custom domain name and TLS certificate for Plex running on Kubernetes

The holy grail: Plex playing music to a remote IPv6 client via a secure connection
The holy grail: Plex playing music to a remote IPv6 client via a secure connection

My Plex server’s remote access setup finally reached a state that I’m happy with, and I want to share that here. The idea here is: I want to expose the server outside of my home network so that I can listen to my music on the go, and do so seamlessly and via TLS encrypted IPv4 and IPv6 connections. Especially the latter is an issue that Plex has not solved yet; the official Remote Access functionality in Plex only supports IPv4. But I got there, and this is roughly how I did it:

With a Fritz!Box as a router and its MyFRITZ!Net service acting as a dynamic DNS service, exposing Plex via IPv4 and IPv6 _outside of the official Remote Access functionality is actually pretty straight-forward. The MyFRITZ!Net service gives every router a unique DNS name à la asdf1234.myfritz.net that receives updated A and AAAA records, every time the Fritz!Box receives a new IP address assignment. That way services such as DuckDNS are not necessary; all I need to do is create the correct port-forwardings, add the DNS name as a custom server access URL in Plex and presto, the server is exposed to the outside world.

But this does not solve TLS encryption for me. With MyFRITZ!Net there is no way I can present a valid TLS certificate for asdf1234.myfritz.net from my homeserver. It’s not my domain after all. Hence, all connections towards the server, while working, will be “downgraded” to unencrypted due to the certificate mismatch. Now, to get around that, I created a CNAME record pointing to asdf1234.myfritz.net. on one of the domains that I own, allowing me to have TLS certificates issued for it by Let’s Encrypt (LE). This way, I just need to give Plex the LE certificate to present.

Kubernetes setup

This is where it gets interesting though, because my homeserver is running Kubernetes, using the lightweight distro k3s. Within Kubernetes, I’m running cert-manager (with a ClusterIssuer configured for Let’s Encrypt) to make the certificate juggling possible.

Please note that the following manifest snippets show only the relevant portions required to follow along. The full manifests can be found at the end of the article:

We start off by creating a Certificate resource for the DNS name:

 1apiVersion: cert-manager.io/v1
 2kind: Certificate
 4  name: plex-janw-xyz-tls
 5  namespace: default
 7  dnsNames:
 8    - "plex.janw.xyz"
 9  issuerRef:
10    name: le-issuer-dns
11    kind: ClusterIssuer
12  secretName: plex-janw-xyz-tls

With that in place, I can now pass the certificate to Plex. Note here, that cert-manager currently does not generate PKCS#12-format certificate files but PEMs, while Plex expects PKCS#12 and does not work with PEMs. Therefore just mounting the cert’s Secret into the Plex container is not enough. We first must convert the format, and we do so in an initContainer:

 2  - name: convert-cert
 3    image: plexinc/pms-docker
 4    command: [bash]
 5    args:
 6      - c
 7      - |
 8        openssl \
 9          pkcs12 \
10          -export \
11          -out /tls-pkcs/tls.pfx \
12          -in /tls-cert/tls.crt \
13          -inkey /tls-cert/tls.key \
14          -password pass:SOMESECRET \
15          -name plex.janw.xyz \
16          -certpbe AES-256-CBC \
17          -keypbe AES-256-CBC \
18          -macalg SHA256
20        chmod 755 /tls-pkcs/tls.pfx
21        ls -lh /tls-pkcs/        
22    volumeMounts:
23      - mountPath: /tls-cert
24        name: tls-cert
25      - mountPath: /tls-pkcs
26        name: tls-pkcs
27  # …
29  # …
30  - name: tls-cert
31    secret:
32      secretName: myplex-tls
33  - name: tls-pkcs
34    emptyDir: {}

The script in the initContainer will convert the certificate and place it in the tls-pkcs emptyDir volume. This volume can now be passed to the Plex container itself:

 2  - name: plex
 3    image: plexinc/pms-docker
 4    volumeMounts:
 5      # …
 6      - mountPath: /tls-pkcs
 7        name: tls-pkcs
 8    ports:
 9      - name: pms
10        containerPort: 32400
12  # …
13  - name: tls-cert
14    secret:
15      secretName: plex-janw-xyz-tls
16  - name: tls-pkcs
17    emptyDir: {}

Plex settings UI

Now the certificate is available to Plex at /tls-pkcs/tls.pfx, and encrypted with the password SOMESECRET. Those details can be entered in the Plex server network settings now:

Custom certificate configuration in Plex’s network settings
Custom certificate configuration in Plex’s network settings

Further down on the settings page the custom server access URL must also be adjusted. Here we need the full external URL, including the port used in the forwarding setup:

Custom server access URL configuration in Plex’s network settings
Custom server access URL configuration in Plex’s network settings

Finally we’ll have to restart Plex, because it will load the certificate only on startup. After a rollout the certificate will be picked up though, and Plex should reachable via the new external hostname.

Notes on certificate renewal

You might have noticed the statefulset’s reloader.stakater.com/auto: "true" annotation. It belongs to Stakater Reloader, a light-weight controller that automatically performs rolling updates on workloads when the ConfigMaps or Secrets they reference are updated. It is particularly useful, for example, when you are doing configuration changes in a ConfigMap but otherwise don’t modify the Deployment that references it. Kubernetes would normally not perform a rollout because the workload did not change, and if there is no dedicated mechanism to reload the config, the workload continues running with the previous configuration. Reloader practically replaces a dedicated config reload mechanism by watching the ConfigMap itself, and doing a rolling update when it changes, thus ensuring the workload adopts the new configuration.

In the setup described above I’m relying on Reloader to be a crude “certificate reloader” for the time when the Let’s Encrypt certificate is renewed. Unfortunately Plex does not support reloading its custom certificate, it is only loaded once at startup time. With the Reloader in place, the renewal will cause the certificate’s Secret to be updated, which in turn causes a new rollout of Plex, now with the renewed certificate. This is far from ideal, because the renewal can happen at any time, while you’re watching something, and that would cause the stream to be interrupted. But in my opinion it is still better than manually restarting the server whenever the renewal occurs.

Full manifests

Tap/click here to expand the full manifests
 2apiVersion: cert-manager.io/v1
 3kind: Certificate
 5  name: plex-janw-xyz-tls
 6  namespace: default
 8  dnsNames:
 9    - "plex.janw.xyz"
10  issuerRef:
11    name: letsencrypt-issuer
12    kind: ClusterIssuer
13  secretName: plex-janw-xyz-tls
16kind: StatefulSet
17apiVersion: apps/v1
19  name: plex
20  namespace: default
21  annotations:
22    reloader.stakater.com/auto: "true"
24  replicas: 1
25  serviceName: plex
26  selector:
27    matchLabels:
28      app: plex
29  template:
30    metadata:
31      labels:
32        app: plex
33        app.kubernetes.io/name: plex
34    spec:
35      hostNetwork: true
36      initContainers:
37        - name: convert-cert
38          image: plexinc/pms-docker
39          command: [bash]
40          args:
41            - -c
42            - |
43              openssl \
44                pkcs12 \
45                -export \
46                -out /tls-pkcs/tls.pfx \
47                -in /tls-cert/tls.crt \
48                -inkey /tls-cert/tls.key \
49                -password pass:SOMESECRET \
50                -name plex.janw.xyz \
51                -certpbe AES-256-CBC \
52                -keypbe AES-256-CBC \
53                -macalg SHA256
55              chmod 755 /tls-pkcs/tls.pfx
56              ls -lh /tls-pkcs/              
58          volumeMounts:
59            - mountPath: /tls-cert
60              name: tls-cert
61            - mountPath: /tls-pkcs
62              name: tls-pkcs
63      containers:
64        - name: plex
65          image: plexinc/pms-docker
66          resources:
67          volumeMounts:
68            - mountPath: /media
69              name: media-vol
70              readOnly: true
71            - mountPath: /transcode
72              name: transcode-vol
73            - mountPath: /config
74              name: config-vol
75            - mountPath: /tls-pkcs
76              name: tls-pkcs
77          ports:
78            - name: pms
79              containerPort: 32400
80          livenessProbe:
81            httpGet:
82              path: /identity
83              port: 32400
84      volumes:
85        - name: media-vol
86          hostPath:
87            path: /mnt/media
88        - name: transcode-vol
89          hostPath:
90            path: /mnt/cache
91        - name: config-vol
92          hostPath:
93            path: /mnt/config
94        - name: tls-cert
95          secret:
96            secretName: plex-janw-xyz-tls
97        - name: tls-pkcs
98          emptyDir: {}