PaaS CloudTechnical Reference
Known Issues
Known bugs, limitations, and workarounds affecting PaaS Cloud — both upstream Cozystack v1.4 issues and specifics of the e-INFRA CZ deployment.
This page documents known issues, edge cases, and limitations affecting PaaS Cloud. It covers both upstream bugs in Cozystack v1.4 (the version this platform runs) and behavior specific to the e-INFRA CZ deployment — such as the Perun/Keycloak identity integration. Please review these items before deploying production workloads or executing cluster-level upgrades.
Storage & Virtualization
Tenant Volume Expansion Failure
- Symptom: Attempting to resize a
PersistentVolumeClaim(PVC) inside a tenant cluster fails. The system returns aspec is immutableerror. - Cause: The cluster synchronization controller incorrectly attempts to modify restricted fields (such as
accessModes) during a resize event, instead of isolating updates to.storageand.volumeAttributesClassName. - Workaround: Manually patch the resource or recreate the volume with the desired capacity if scaling becomes blocked.
VM Disk From a Golden Image Never Binds in a Different StorageClass
- Symptom: Creating a
vm-diskfrom a golden image into a StorageClass different from the one the golden images live in never completes. The disk stays pending indefinitely with no error surfaced; behind the scenes a temporary clone PVC (tmp-pvc-…) incozy-publicremains unbound and the CSI driver logscontext deadline exceeded. - Cause: The golden images live in a replicated StorageClass, but the clone into a different target class (for example a node-local
localclass) is attempted as a CSI clone across storage backends, which cannot complete. Reported on v1.4.2 with LINSTOR storage. Tracked upstream in cozystack#2908; not yet fixed in the v1.4 release line. - Workaround: Create disks sourced from a golden image in the same StorageClass as the images (the default
replicatedclass). If you need the data in another class, create the disk inreplicatedfirst and migrate afterwards.
Cannot Add a Disk on the VM Create Form Until Refresh
- Symptom: On the dashboard’s VM create form, the Add disk control does nothing on a freshly opened page. Adding a disk only works after reloading the page once.
- Cause: A dashboard form-state bug in the VM disk widget shipped in v1.4.2. Tracked upstream in cozystack#3109; the disk form has since been reworked on newer releases.
- Workaround: Reload the VM create page once before adding disks.
Networking & Multus Integration
Stale Sandbox Reservations
- Symptom: Terminated VM sandboxes fail to clear from the cluster. This leaks internal network resources and causes node clutter.
- Cause: A race condition exists between Multus and the
kube-ovnintegration layer. Because Multus lacks a localized sandbox network cache, it rejects the deletion commands during network tear-down phases. - Workaround: Administrators must manually clean up leaked network namespaces and stale ovn ports on the affected host node.
TCP LoadBalancer Fails to Start on HAProxy 3.3+
- Symptom: A
TCPLoadBalancer(thetcp-balancerapp) never becomes ready; its HAProxy pod crashes on startup with a config error such as:backend 'http' has the same name as frontend 'http' ... This is no longer supported as of 3.3. Please rename one or the other. - Cause: The
tcp-balancerchart references the HAProxy image ashaproxy:latest. HAProxy 3.3 removed support for a frontend and backend sharing the same name, but the generatedhaproxy.cfgstill declares matching pairs (http/http,https/https,kubernetes/kubernetes,talos/talos). Because the image tag is:latest, aTCPLoadBalancercan start failing without any platform upgrade — as soon as a node pulls the newer HAProxy image. Tracked upstream in cozystack#1966; the fix (#2321) is not yet included in the v1.4 release line. - Workaround: Pin the HAProxy image to a pre-3.3 tag (for example
haproxy:3.2) until the fix is released. If you cannot override the image yourself, contact us at cloud@metacentrum.cz.
Managed Applications
MariaDB Volumes Are Not Deleted With the Application
- Symptom: After deleting a MariaDB application instance, its
PersistentVolumeClaims remain in the tenant namespace and keep consuming storage quota. - Cause: The chart’s post-delete cleanup hook looks for PVCs labelled
app.kubernetes.io/instance=<release>, but the MariaDB operator creates them via StatefulSets without that label, so the hook matches nothing and deletes nothing. Tracked upstream in cozystack#2350; not yet fixed in the v1.4 release line. - Workaround: After deleting a MariaDB instance, remove its leftover PVCs manually. List them with
kubectl -n <tenant-namespace> get pvc, identify the ones belonging to the deleted instance (their names include the instance name), and delete them withkubectl -n <tenant-namespace> delete pvc <name>.
Identity & Access
Generated Kubeconfig Uses the Wrong OIDC Issuer URL
- Symptom: After downloading a tenant
kubeconfigfrom the dashboard,kubectlcommands fail with an OIDC error such asinvalid client_id,issuer did not match, or a redirect to a non-existent Keycloak host. - Cause: The
infoHelm chart in this environment populates the OIDC issuer using_namespace.host(e.g.,keycloak.cloud.chloe.platform.cloud.e-infra.cz) instead of_cluster.root-host(keycloak.chloe.platform.cloud.e-infra.cz). - Workaround: Edit the
kubeconfigand replace the incorrect issuer URL with the correct one, or use thesedcommand shown in Using kubectl.
Dashboard Login Fails for Users in Many Groups (Large OIDC Token)
- Symptom: The first OIDC sign-in to the dashboard fails — you are redirected back with a 403 and an error such as
Login Failed: Unable to find a valid CSRF token. Please try again.(accompanied by aRequest ID), or requests fail with400 Bad Request: request header or cookie too large. Clicking Sign in a second time usually succeeds. - Cause: Because Keycloak federates identities from Perun, a user who belongs to a large number of groups receives a correspondingly large OIDC access token. Carried in cookies and headers, it exceeds the default NGINX
client-header-buffer-size. The ingress rejects the oversized request, and the dashboard’s authentication proxy then loses its CSRF/state cookie, so the login round-trip fails. Tracked upstream in cozystack#2853; not yet fixed in the v1.4 release line. - Workaround: Retry the sign-in — the second attempt reuses a smaller session cookie and usually passes. A durable fix requires raising
client-header-buffer-size(for example to16k) on the tenant-root ingress-nginx controller and theincloud-web-nginx-configconfigmap; if the problem persists, contact us at cloud@metacentrum.cz.
