SUSE Rancher Prime & RKE2: Enterprise Reference Architecture and OpenShift Migration Guide

This architectural and implementation guide provides technical blueprints, comparative analyses, security compliance instructions, and operational models for deploying SUSE Rancher Prime and RKE2 (Rancher Kubernetes Engine 2) in enterprise environments. It serves as an authoritative guide for architects, security administrators, and systems engineers migrating from Red Hat OpenShift (OCP) to a standards-compliant, modular, and highly secure Kubernetes ecosystem.

Table of Contents

1. Enterprise Reference Architecture
2. High Availability Control Plane & Endpoint API Setup
3. OS Prerequisites & CIS Hardening Compliance
4. Hardened Node Addition & CIS Troubleshooting
5. CNI Selection: Cilium vs. Canal (Default)
6. Red Hat OpenShift to RKE2/Rancher Migration Assessment
7. Production Lifecycle, GitOps, and Automation
8. Edge, OS, and Virtualization
9. Production Validation Checklist

1. Enterprise Reference Architecture

Modern, mission-critical applications require a Kubernetes cluster topology designed for resilience, security, and low-latency communication. The following architecture outlines the recommended setup for downstream RKE2 clusters managed by a centralized, high-availability SUSE Rancher Prime management cluster.

1.1 Cluster Topology Diagram

The following blueprint illustrates traffic flowing from external corporate client networks through a secure edge layer into highly available, multi-role control plane nodes, with integrated platform services attached to the worker node pool.

1.2 Recommended Infrastructure Components

For enterprise-grade production clusters, the platform is structured on upstream-compliant, highly secure integrations:

• Kubernetes Engine: RKE2 (government-grade security, FIPS 140-3 compliant builds, and built-in CIS benchmark profiles).
• Central Management: SUSE Rancher Prime (multi-cluster lifecycle management, centralized RBAC, and policy orchestration).
• Load Balancing: High-availability external hardware/software load balancers (e.g., A10, F5) or integrated virtual IPs via Kube-VIP.
• Ingress Controller: NGINX Ingress Controller (included by default, highly customizable, and standard for enterprise workloads).
• Persistent Storage: SUSE Longhorn (cloud-native distributed block storage) or enterprise CSI-compliant external SAN/NAS storage (such as NetApp Trident or Ceph).
• Container Registry: SUSE Private Registry (with Trivy vulnerability scanning and Cosign image digital signature validation).
• Observability: SUSE Observability (centralized logging, metrics, traces, events, topology, and change tracking) coupled with Rancher Prime Monitoring (Prometheus, Grafana, and Alertmanager).
• Policy Enforcement: SUSE Kubewarden (using WebAssembly to enforce admission control rules and replace OpenShift SCCs) alongside Kubernetes Pod Security Admission (PSA).
• Secrets Management: HashiCorp Vault integrated with External Secrets Operator (ESO), alongside RKE2 local Secrets Encryption.
• Continuous Delivery & GitOps: SUSE Fleet (built-in GitOps engine scaling to tens of thousands of clusters) or Argo CD / Flux.
• Backup and Recovery: Velero (for resource definition and persistent volume migration) paired with RKE2 etcd automated snapshots and Longhorn backups.

2. High Availability Control Plane & Endpoint API Setup

To deploy an enterprise-ready High Availability (HA) control plane cluster, RKE2 requires a dedicated load balancer or virtual IP configuration to distribute API traffic and handle node registration.

2.1 API and Registration Port Routing

• TCP 6443 (Kubernetes API Server): Used by administrators, CI/CD tools, kubectl, and the Rancher Prime controller to interact with the downstream cluster.
• TCP 9345 (RKE2 Node Registration Port): Used internally by RKE2 to coordinate node bootstrapping, certificate distribution, and etcd cluster state. This must be accessible from all nodes to the load balancer/fixed-registration address.

The load balancer must be configured in Layer 4 TCP pass-through mode for both ports to preserve TLS client certificates, which are used for Kubernetes authentication.

2.2 RKE2 HA Deployment Process

Step 1: Configure the Fixed Registration Address

Ensure your DNS or external load balancer has a record (e.g., fixed-registration-address.com) resolving to your load balancer VIP, which forwards 6443 and 9345 to all RKE2 control plane nodes.

Step 2: Configure the First Control Plane Server Node

Create the configuration directory and file:

sudo mkdir -p /etc/rancher/rke2/

Create /etc/rancher/rke2/config.yaml on the first server node:

# /etc/rancher/rke2/config.yaml (Server 1 - Cluster Initiator)
token: "my-secure-shared-bootstrap-token"
tls-san:
  - "fixed-registration-address.com"
  - "fixed-registration-IP-Address"
profile: "cis-1.23" # Adjust based on target Kubernetes version

Start and enable the RKE2 server service:

sudo systemctl enable --now rke2-server

Step 3: Join Additional Control Plane Server Nodes

On Server 2 and Server 3, create /etc/rancher/rke2/config.yaml:

# /etc/rancher/rke2/config.yaml (Server 2 and 3)
server: "https://fixed-registration-address.com:9345"
token: "my-secure-shared-bootstrap-token"
tls-san:
  - "fixed-registration-address.com"
  - "fixed-registration-IP-Address"
profile: "cis-1.23"

Start the service:

sudo systemctl enable --now rke2-server

Step 4: Join Worker (Agent) Nodes

On worker nodes, create the agent config file /etc/rancher/rke2/config.yaml:

# /etc/rancher/rke2/config.yaml (Workers / Agents)
server: "https://fixed-registration-address.com:9345"
token: "my-secure-shared-bootstrap-token"

Start the RKE2 agent service:

sudo systemctl enable --now rke2-agent

2.3 Verification & Diagnostics

To verify the control plane API endpoint, execute the following commands:

1. Check API Server socket binding:

   ss -tulpn | grep 6443
   # Expected output: LISTEN 0 128 *:6443
   

2. Verify HTTPS Connectivity and API Response:

   curl -k https://fixed-registration-address.com:6443/version
   

3. Confirm Cluster Health via kubectl:

   export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
   /var/lib/rancher/rke2/bin/kubectl cluster-info
   /var/lib/rancher/rke2/bin/kubectl get nodes -o wide
   

3. OS Prerequisites & CIS Hardening Compliance

RKE2 is engineered for compliance with Center for Internet Security (CIS) Kubernetes Benchmarks out of the box. However, executing a hardened profile requires specific host-level configuration before starting the RKE2 systemd services.

3.1 Kernel Parameters Configuration (60-rke2-cis.conf)

When the CIS profile is enabled (profile: "cis" or equivalent in config.yaml), the Kubelet enforces protect-kernel-defaults=true. This causes the Kubelet to immediately fail if the host OS kernel parameters are not set correctly.

To comply, copy or create the sysctl file in your configuration directory:

# Path: /etc/sysctl.d/60-rke2-cis.conf
# Enable IP forwarding (mandatory for Kubernetes networking)
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1

# Enable bridge netfilter parameters for iptables
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

# Kernel panic parameters for node reliability
kernel.panic = 10
kernel.panic_on_oops = 1

# Memory management constraints
vm.overcommit_memory = 1
vm.panic_on_oom = 0

# Filesystem protections to prevent privilege escalation
fs.protected_hardlinks = 1
fs.protected_symlinks = 1

# Access restrictions to kernel diagnostics
kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1

Apply the configurations immediately:

sudo systemctl restart systemd-sysctl

3.2 Operating System Baseline Requirements

To pass a production CIS audit, ensure the following host configuration is applied:

1. Mandatory Host User and Group for etcd:

   By default, RKE2 runs the etcd service under a low-privileged system account. You must manually provision the etcd user and group prior to installation:

   sudo groupadd -r etcd
   sudo useradd -r -c "etcd-user" -s /sbin/nologin -M -g etcd etcd
   

2. Audit Daemon (auditd):

   Ensure the audit daemon is active and configured. RKE2 audits API access and stores host-level journals, which require auditd to be active on the host OS:

   sudo systemctl enable --now auditd
   

3. Time Synchronization (chrony):

   Clock drift is a primary cause of TLS handshake and token expiration failures across distributed control planes.

   sudo systemctl enable --now chronyd
   

4. SELinux Hardening:

   RKE2 supports SELinux in Enforcing mode. Ensure the rke2-selinux RPM policy package is installed:

   sudo yum install -y rke2-selinux
   # Verify SELinux is enforcing
   getenforce # Should return "Enforcing"
   

5. Disable SWAP:

   Kubernetes does not support running on nodes with active SWAP partitions under default configurations due to performance unpredictability.

   sudo swapoff -a
   # Comment out any swap mounts in /etc/fstab to persist after reboot
   sudo sed -i '/swap/s/^/#/' /etc/fstab
   

6. SSH Security Hardening (Recommended):

   • Disable Root SSH Login: Set PermitRootLogin no in /etc/ssh/sshd_config.
   • Disable Password Authentication: Set PasswordAuthentication no in /etc/ssh/sshd_config.

4. Hardened Node Addition & CIS Troubleshooting

When scaling up a cluster with a strict CIS profile enabled, node addition frequently fails if the target host does not meet all security prerequisites.

4.1 Common Root Causes & Remediation

4.2 Recommended Cluster Build Order

To prevent troubleshooting cycles, adopt this strict procedural build order:

5. CNI Selection: Cilium vs. Canal (Default)

Choosing the Container Network Interface (CNI) is a critical architectural decision that dictates cluster performance, security posture, and network observability.

5.1 Cilium (Recommended for Enterprise Platforms)

Cilium utilizes eBPF (Extended Berkeley Packet Filter) in the Linux kernel to bypass traditional IPtables rules, offering unmatched network performance, deep observability, and native API-aware security.

• eBPF Data Path: Circumvents the Linux network stack's IPtables processing, providing flat routing performance regardless of cluster scale (O(1) lookup vs O(N) IPtables lookup).
• Hubble Observability: Provides deep, flow-level visibility of network traffic (HTTP, gRPC, TCP, UDP) with native integration into Grafana.
• Layer 7 Network Policies: Enforces security rules not only on IP/Port but also on application layer protocols (e.g., restrict HTTP methods like GET vs. POST, or validate API paths).
• Built-in Encryption: Provides native, transparent data-path encryption via WireGuard or IPsec.
• Operational Trade-off: Requires advanced kernel version support (Linux Kernel >= 4.19) and a more advanced engineering skillset to operate and troubleshoot.

5.2 Canal (Default CNI)

Canal is a combination of Flannel (for VXLAN overlay data-path routing) and Calico (for network policy enforcement).

• Simplicity and Stability: Highly tested, reliable, and runs on almost any standard Linux distribution with standard kernel configurations.
• Low Operational Overhead: Simple to run, configure, and maintain.
• IPtables Limitation: Relies heavily on traditional Linux IPtables, which can degrade packet processing speeds in massive environments with tens of thousands of active Network Policies or Services.
• Observability: Basic packet-level counters; lacks application-level flow analytics out of the box.

5.3 CNI & SDN Feature Comparison Matrix

The table below contrasts the native capabilities of OpenShift SDN / OVN-Kubernetes against RKE2 integrated CNI options:

6. Red Hat OpenShift to RKE2/Rancher Migration Assessment

Migrating from an opinionated, proprietary platform like Red Hat OpenShift (OCP) to a standards-compliant, modular distribution like RKE2 and Rancher Prime eliminates vendor lock-in, reduces licensing overhead, and improves architectural flexibility.

6.1 Strategic Architectural Differences

6.2 Component Feature Mapping

To ensure continuity of security and operations, translate OCP-specific API resources to standard Kubernetes and SUSE alternatives during your migration:

6.3 Deep-Dive Migration Replacement Strategies

1. Security Context Constraints (SCC) to PSA & Kubewarden

OpenShift uses SCCs to control pod privileges (e.g., preventing running as root, limiting host path mounts). In RKE2, this is addressed via:

• Pod Security Admission (PSA): Built-in Kubernetes mechanism that enforces three security levels (privileged, baseline, restricted) at the namespace level via annotations:

  apiVersion: v1
  kind: Namespace
  metadata:
    name: my-app-prod
    labels:
      pod-security.kubernetes.io/enforce: restricted
  

• Kubewarden: For advanced, granular controls (like enforcing image registry origins, restricting specific host volumes, or validating digital signatures), deploy Kubewarden. Kubewarden policies are written in WebAssembly (Wasm), allowing execution of security logic compiled from Go, Rust, or Rego.

2. OpenShift Routes to Standard Ingress

OpenShift Route resources must be rewritten into standard Kubernetes Ingress resources.

• OpenShift Route (Legacy):

  apiVersion: route.openshift.io/v1
  kind: Route
  metadata:
    name: frontend
  spec:
    to:
      kind: Service
      name: frontend-service
    tls:
      termination: edge
  

• Standard Kubernetes Ingress (Target RKE2 NGINX):

  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    name: frontend-ingress
    annotations:
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      cert-manager.io/cluster-issuer: "letsencrypt-prod"
  spec:
    ingressClassName: nginx
    tls:
    - hosts:
      - app.my-domain.com
      secretName: app-tls-secret
    rules:
    - host: app.my-domain.com
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: frontend-service
              port:
                number: 80
  

3. Integrated OAuth to Centralized IdP Integration

OpenShift forces integration into its internal OAuth system. Rancher Prime provides direct, native integrations with enterprise identity engines without requiring intermediate synchronization CRDs:

• Authentication: Directly integrate with LDAP, Active Directory, Okta, Keycloak, or Microsoft Entra ID (Azure AD).
• Authorization Mapping: Map corporate group claims to Kubernetes ClusterRoles or custom Rancher Projects. This allows developers to use single sign-on (SSO) and maintains RBAC guardrails without manual user synchronization.

4. Pipelines & S2I to External CI

S2I (Source-to-Image) builds inside OpenShift consume cluster control plane resources and create platform lock-in.

• Strategy: Migrate pipelines out of Kubernetes to dedicated CI systems (e.g., GitLab CI, GitHub Actions, Tekton, or Jenkins).
• Process: Build the Dockerfile, push the standardized OCI image to the SUSE Private Registry, verify the build via Trivy scanning, and execute GitOps-driven deployment manifests to RKE2 via Fleet.

7. Production Lifecycle, GitOps, and Automation

Operating an enterprise-scale Kubernetes fleet requires automation of cluster upgrades, policy validation, and disaster recovery.

7.1 Production Upgrade Strategy

Upgrading clusters in production requires careful sequencing to prevent application downtime.

1. Support Matrix Check: Prior to initiating upgrades, cross-reference your Rancher Prime and RKE2 versions with the official SUSE Support Matrix. Rancher Prime must always be upgraded to the target version before upgrading the managed downstream clusters.

2. Rancher-Managed Automated Upgrades: Leverage Rancher Prime's native UI or API to execute canary rolling upgrades.

   • Recommended Config settings:

     • Drain Nodes = Yes (safely evicts workloads before taking a node offline).
     • Timeout = 600 (provides 10 minutes for slow-draining workloads, failing safe if exceeded).

3. Mandatory Backup Validation: Always trigger an automated etcd snapshot via the Rancher UI/CLI prior to upgrading:

   • Rancher UI allows immediate on-demand etcd snapshot generation to S3 or local disk.
   • For virtualization/cloud environments, capture an external VM-level snapshot of control plane servers before running upgrades.

7.2 SUSE Fleet GitOps Architecture

SUSE Fleet is the native GitOps engine included within Rancher Prime. Unlike traditional GitOps controllers that pull manifests into a single cluster, Fleet is engineered for massive scale, orchestrating resource delivery across up to a million downstream clusters.

• Drift Management: Set custom drift rules. For third-party operators, ensure drift-management is configured to ignore rules for dynamic resource paths to avoid loops.
• Target Customization: Group clusters using Rancher labels (e.g., env: production, region: APAC). Fleet reads target groups and automatically customizes variables (using Helm values) before pushing manifests to target environments.

8. Edge, OS, and Virtualization

Enterprise environments frequently scale across centralized bare-metal datacenters, public clouds, and geographically distributed edge environments.

8.1 SUSE Elemental (Image-Based OS Operations)

SUSE Elemental is not an operating system itself; rather, it is an image-based OS delivery method designed to treat the underlying node OS similarly to a standard container.

• Immutable Core: The OS is deployed as a read-only immutable system image based on Sle-Micro or SLES.
• Zero-Patch Philosophy: Instead of executing manual patch commands (zypper update or apt-get upgrade) on thousands of edge servers, Elemental updates the node by completely replacing the underlying OS image with a validated newer version.
• Edge Optimization: Ideal for remote edge sites and bare-metal environments where traditional OS configuration tools are slow or unreliable. Updates are coordinated directly through the central Kubernetes control plane.

8.2 Rancher Virtualization (VMs on Kubernetes)

Using standard open-source virtualization standards (KubeVirt), Rancher Virtualization allows teams to run legacy Virtual Machines directly inside standard Kubernetes pods alongside modern containers on the same physical infrastructure.

• Scheduling: VMs use standard Kubernetes scheduling features, including node affinity/anti-affinity, tolerations, and resource quotas.
• Operational Capabilities: Fully supports VM live migration across bare-metal hosts, automated VM backups and restorations, snapshotting, and cloning.
• Hardware Access: Supports dedicated GPU pass-through and shared GPU capabilities (using NVIDIA MIG) directly inside legacy virtual machines for AI/ML inference workloads.

9. Production Validation Checklist

Before declaring the RKE2 production platform ready for customer workloads, run the following validation testing phases:

Phase 1: Infrastructure & Resiliency

• High Availability Control Plane: Successfully shut down Server Node 1 and verified the Kubernetes API remained responsive via Server Node 2 and 3.
• Load Balancer Failover: Tested load balancer active-passive failover and verified zero loss of connection to port 6443.
• etcd Quorum Recovery: Verified etcd continued processing reads and writes during a single control plane node failure.
• Storage CSI Resilience: Disrupted a physical storage path and verified that SUSE Longhorn re-routed block storage without pod interruption.

Phase 2: Security & Hardening

• CIS Compliance Audit: Executed a CIS benchmark scan on all nodes and verified a "PASS" status.
• Host Hardening Execution: Validated /etc/sysctl.d/60-rke2-cis.conf values are loaded (sysctl --system).
• SELinux Mode Validation: Ran getenforce on all nodes and confirmed status is Enforcing.
• Pod Security Admission (PSA): Verified that namespaces labeled restricted prevent pods from launching with host root access.
• Secret Encryption Verification: Confirmed secrets are encrypted on etcd disks by pulling a raw database string and checking for the k8s:enc:aescbc:v1 prefix.

Phase 3: Operational Readiness

• Automated etcd Backup: Verified scheduled etcd snapshots are successfully uploading to offsite S3/MinIO storage.
• Complete Disaster Restore: Successfully rebuilt a downstream cluster from scratch using only a backup etcd snapshot.
• SSO / IdP Integration: Confirmed corporate developers can authenticate via Microsoft Entra ID / Keycloak and receive correct RBAC roles.
• Observability Verification: Confirmed that SUSE Observability dashboard is successfully capturing API latencies, pod logging streams, and node metrics.
• Upgrade Simulation: Successfully executed a dry-run cluster upgrade on a staging cluster using Rancher's rolling update UI.