Features in Depth

This page explains how Plethos main features work internally. Knowing these mechanisms helps you understand what you see in your cluster and gives you a starting point for debugging when a vCluster doesn’t behave as expected.

Sleeping and waking

Setting spec.state to Sleeping frees the cluster’s resources without deleting it:

1. The controller scales the vCluster StatefulSet to 0 replicas and stores the previous replica count in the plethos.cs.hm.edu/scaled-down-replicas annotation.
2. It deletes the workload pods that the vCluster synced into the host namespace (identified by the vcluster.loft.sh/managed-by label). The control plane is stopped first, otherwise it would simply recreate them.
   Note: Currently, there might happen a race where the control plane recreates workloads before being fully scaled down. These workloads will then removed on the next reconcilation in 3 hours.
3. Waking restores the replica count and the claim goes through Waking back to Running.

Caution

Pods that aren’t managed by a StatefulSet, ReplicaSet or similar controller can’t be recreated after sleeping and are lost. Bare pods are a Kubernetes anti-pattern, so this is accepted behavior. Also, don’t use the vcluster CLI’s pause and resume on these clusters. A resumed cluster is put back to sleep on the next reconciliation.

Public app exposure

If you set a exposedAppsBaseDomain for your Plethos instance, users can make a web app in their vCluster publicly reachable. They need to deploy a Service named http in the vCluster’s default namespace on port 80. Only HTTP is supported.

When a vCluster is first provisioned, the controller creates an Ingress for <claim-name>.<exposedAppsBaseDomain>, whether or not the Service exists yet. This works because the Ingress points at the Service as vCluster syncs it to the host cluster, and that name is predictable: http-x-default-x-<claim-name>. Until the owner deploys the Service, the URL doesn’t serve anything.

For example, the claim u9f4c1e7a2b8d0356-test under apps.example.com is served at https://u9f4c1e7a2b8d0356-test.apps.example.com.

Deleting

Deletion of a vCluster follows the standard Kubernetes pattern. Deleting a vCluster happens by deleting its VClusterClaim. It makes no difference whether this happens in the portal, through the expiry job or with kubectl delete vcc.

The claim has a finalizer, so it isn’t removed immediately. The controller first runs a Helm uninstall Job to tear the vCluster down, then removes the finalizer and the claim disappears. This means deletion needs a running controller. When a claim is deleted while the controller is down, it will delete its vCluster resources when it starts up next time.

Deletion is permanent. The portal only lets the owner or an admin delete a cluster, but anyone with kubectl access to the claims namespace can delete claims directly.

Cluster expiry

The CronJob runs to check if a vCluster is about to or already expired. Time is measured from the renewed-at annotation, or the creation time if it was never renewed:

1. 7 days before expiry: the owner gets a warning email with a keep-alive link.
2. At expiry (90 days by default): the cluster is put to sleep and the owner is told.
3. 7 days before the end of the grace period (60 days by default): a second warning announces the deletion date.
4. End of the grace period: the claim, and therefore the cluster, is deleted.

Only clicking the keep-alive link renews a claim, other actions like waking don’t. A claim is never deleted without a warning naming the exact deadline having been sent first.

Hinweis

These periods are configured through environment variables of the portal image (EXPIRY_PERIOD_DAYS, GRACE_PERIOD_DAYS and WARNING_LEAD_DAYS) and are not currently exposed as Helm values.