diff options
Diffstat (limited to 'ISO-BUILDER.md')
| -rw-r--r-- | ISO-BUILDER.md | 849 |
1 files changed, 849 insertions, 0 deletions
diff --git a/ISO-BUILDER.md b/ISO-BUILDER.md new file mode 100644 index 0000000..a1c5ebd --- /dev/null +++ b/ISO-BUILDER.md @@ -0,0 +1,849 @@ +# ISO Builder + +## Overview + +This document specifies how to build a **bootable ISO image** that contains the base OS, all binaries, configuration files, systemd units, and scripts required for cluster-from-systemd. + +## Design Decision: Kickstart + Live Image + +**Approach**: Create a Rocky Linux 9 live ISO with embedded kickstart for automated installation. + +**Rationale**: +- **Kickstart**: Automates installation (partitioning, package selection, post-install scripts) +- **Live CD tooling**: Well-established RHEL ecosystem tools (lorax, livecd-tools) +- **Single ISO**: All nodes boot from the same image +- **Reproducible**: Version-controlled build process + +**Alternative approaches considered**: +- **cloud-init/ignition**: Better for cloud, but requires external metadata service +- **PXE boot**: Network dependency, more complex infrastructure +- **Container-based build (mkosi)**: Bleeding edge, less RHEL-native + +## Build System Requirements + +**Host OS**: Fedora 39 or Rocky Linux 9 (build environment) + +**Required packages**: +```bash +dnf install -y \ + lorax \ + anaconda \ + pykickstart \ + createrepo_c \ + genisoimage \ + isomd5sum \ + syslinux \ + git +``` + +**Disk space**: ~30 GB (for build artifacts, repos, and final ISO) + +## Directory Structure + +``` +iso-build/ +├── kickstart/ +│ └── cluster-node.ks # Main kickstart file +├── repo/ +│ ├── binaries/ # Downloaded binaries (k8s, etcd, etc.) +│ ├── rpms/ # Custom RPM packages +│ └── repodata/ # RPM repository metadata +├── overlay/ +│ ├── etc/ +│ │ ├── cluster-config/ # Cluster YAML configs +│ │ │ ├── cluster.yaml +│ │ │ ├── services/ +│ │ │ └── nodes/ +│ │ ├── systemd/system/ # Systemd units +│ │ └── kubernetes/ +│ │ ├── pki/ # Pre-generated certificates +│ │ └── manifests/ +│ ├── usr/local/bin/ # Cluster scripts +│ └── var/www/html/ # Join info distribution +├── output/ +│ └── cluster-node.iso # Final bootable ISO +└── build.sh # Main build script +``` + +## Kickstart File + +**File**: `iso-build/kickstart/cluster-node.ks` + +```kickstart +#version=RHEL9 + +# Install mode (not upgrade) +install + +# Use CDROM installation media +cdrom + +# Text mode installation (no GUI) +text + +# System language +lang en_US.UTF-8 + +# Keyboard layouts +keyboard us + +# Network configuration (temporary DHCP, will be reconfigured on boot) +network --bootproto=dhcp --device=link --activate + +# Root password (change this!) +rootpw --plaintext cluster-password + +# System authorization +authselect select sssd + +# SELinux mode +selinux --enforcing + +# Firewall configuration +firewall --enabled --service=ssh + +# System timezone +timezone America/New_York --utc + +# Disk partitioning +# WARNING: This will ERASE the entire disk! +ignoredisk --only-use=sda +clearpart --all --initlabel --drives=sda +part /boot --fstype=xfs --size=1024 +part / --fstype=xfs --size=20480 --grow +part swap --fstype=swap --size=4096 + +# Bootloader +bootloader --location=mbr --boot-drive=sda + +# Reboot after installation +reboot + +# Package selection +%packages +@core +@standard +kernel +systemd +systemd-networkd +NetworkManager +vim +tmux +htop +curl +wget +jq +python3 +python3-pip +python3-pyyaml +openssl +iproute +iputils +bind-utils +iptables +nftables +%end + +# Pre-installation script +%pre --log=/tmp/ks-pre.log +#!/bin/bash +echo "==> Starting pre-installation" +# Could add disk detection logic here +%end + +# Post-installation script +%post --log=/root/ks-post.log --erroronfail +#!/bin/bash +set -x + +echo "==> Starting post-installation configuration" + +# 1. Copy cluster configuration files +echo "==> Installing cluster configuration..." +mkdir -p /etc/cluster-config/{services,nodes,environment} +cp -r /mnt/install/repo/overlay/etc/cluster-config/* /etc/cluster-config/ + +# 2. Copy systemd units +echo "==> Installing systemd units..." +cp /mnt/install/repo/overlay/etc/systemd/system/* /etc/systemd/system/ +systemctl daemon-reload + +# 3. Install cluster scripts +echo "==> Installing cluster scripts..." +cp /mnt/install/repo/overlay/usr/local/bin/* /usr/local/bin/ +chmod +x /usr/local/bin/*.sh + +# 4. Install binaries +echo "==> Installing Kubernetes binaries..." +BINARIES="kubelet kubectl kubeadm kube-apiserver kube-controller-manager kube-scheduler" +for binary in $BINARIES; do + if [[ -f /mnt/install/repo/binaries/$binary ]]; then + cp /mnt/install/repo/binaries/$binary /usr/local/bin/ + chmod +x /usr/local/bin/$binary + fi +done + +echo "==> Installing etcd..." +if [[ -f /mnt/install/repo/binaries/etcd ]]; then + cp /mnt/install/repo/binaries/etcd /usr/local/bin/ + cp /mnt/install/repo/binaries/etcdctl /usr/local/bin/ + chmod +x /usr/local/bin/etcd /usr/local/bin/etcdctl +fi + +echo "==> Installing CoreDNS..." +if [[ -f /mnt/install/repo/binaries/coredns ]]; then + cp /mnt/install/repo/binaries/coredns /usr/local/bin/ + chmod +x /usr/local/bin/coredns +fi + +echo "==> Installing CNI plugins..." +if [[ -d /mnt/install/repo/binaries/cni ]]; then + mkdir -p /opt/cni/bin + cp -r /mnt/install/repo/binaries/cni/* /opt/cni/bin/ + chmod +x /opt/cni/bin/* +fi + +echo "==> Installing Kafka..." +if [[ -d /mnt/install/repo/binaries/kafka ]]; then + mkdir -p /opt + cp -r /mnt/install/repo/binaries/kafka /opt/ + ln -sf /opt/kafka /opt/kafka-current +fi + +# 5. Install certificates +echo "==> Installing certificates..." +mkdir -p /etc/kubernetes/pki/etcd +if [[ -d /mnt/install/repo/overlay/etc/kubernetes/pki ]]; then + cp -r /mnt/install/repo/overlay/etc/kubernetes/pki/* /etc/kubernetes/pki/ + chmod 600 /etc/kubernetes/pki/*.key + chmod 600 /etc/kubernetes/pki/etcd/*.key +fi + +# 6. Create data directories +echo "==> Creating data directories..." +mkdir -p /var/lib/{kubelet,etcd,kafka,ceph/{mon,osd},mosquitto} +mkdir -p /var/lib/cluster-state + +# 7. Install container runtime (containerd) +echo "==> Installing containerd..." +dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo +dnf install -y containerd.io runc + +mkdir -p /etc/containerd +containerd config default > /etc/containerd/config.toml +sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml + +systemctl enable containerd + +# 8. Install Ceph packages +echo "==> Installing Ceph packages..." +cat > /etc/yum.repos.d/ceph.repo <<'CEPH_REPO' +[ceph] +name=Ceph packages for $basearch +baseurl=https://download.ceph.com/rpm-reef/el9/$basearch +enabled=1 +gpgcheck=1 +gpgkey=https://download.ceph.com/keys/release.asc + +[ceph-noarch] +name=Ceph noarch packages +baseurl=https://download.ceph.com/rpm-reef/el9/noarch +enabled=1 +gpgcheck=1 +gpgkey=https://download.ceph.com/keys/release.asc +CEPH_REPO + +dnf install -y ceph-common ceph-mon ceph-osd ceph-mds ceph-mgr + +# 9. Install Mosquitto +echo "==> Installing Mosquitto..." +dnf install -y epel-release +dnf install -y mosquitto mosquitto-clients + +# 10. Install Java (for Kafka) +echo "==> Installing Java..." +dnf install -y java-17-openjdk-headless + +# 11. Enable cluster-detect service +echo "==> Enabling cluster services..." +systemctl enable cluster-detect.service +systemctl enable cluster-bootstrap.service + +# 12. Disable unwanted services +systemctl disable firewalld +systemctl mask firewalld + +# 13. Configure sysctl for Kubernetes +cat >> /etc/sysctl.d/99-kubernetes.conf <<'SYSCTL' +net.bridge.bridge-nf-call-iptables = 1 +net.bridge.bridge-nf-call-ip6tables = 1 +net.ipv4.ip_forward = 1 +SYSCTL + +# 14. Load required kernel modules +cat > /etc/modules-load.d/kubernetes.conf <<'MODULES' +overlay +br_netfilter +MODULES + +# 15. Install yq (YAML processor) +echo "==> Installing yq..." +YQ_VERSION="v4.40.5" +curl -L "https://github.com/mikefarah/yq/releases/download/${YQ_VERSION}/yq_linux_amd64" \ + -o /usr/local/bin/yq +chmod +x /usr/local/bin/yq + +# 16. Create version info +cat > /etc/cluster-version <<VERSION +CLUSTER_ISO_VERSION=0.1.0 +BUILD_DATE=$(date -Iseconds) +KUBERNETES_VERSION=1.29.1 +CEPH_VERSION=18.2.1 +KAFKA_VERSION=3.6.1 +VERSION + +echo "==> Post-installation complete!" +echo "==> System will reboot and detect node identity on first boot" + +%end +``` + +## Build Script + +**File**: `iso-build/build.sh` + +```bash +#!/bin/bash +# Main ISO build script + +set -euo pipefail + +BUILD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +OUTPUT_DIR="$BUILD_DIR/output" +REPO_DIR="$BUILD_DIR/repo" +OVERLAY_DIR="$BUILD_DIR/overlay" +KICKSTART_FILE="$BUILD_DIR/kickstart/cluster-node.ks" + +ISO_NAME="cluster-node.iso" +ISO_LABEL="ClusterNode" +ISO_VERSION="0.1.0" + +echo "==> Cluster-from-SystemD ISO Builder v$ISO_VERSION" +echo "==> Build directory: $BUILD_DIR" + +# Check prerequisites +command -v lorax >/dev/null || { echo "ERROR: lorax not found. Install with: dnf install lorax"; exit 1; } +command -v createrepo_c >/dev/null || { echo "ERROR: createrepo_c not found"; exit 1; } + +# Create directories +mkdir -p "$OUTPUT_DIR" "$REPO_DIR"/{binaries,rpms} "$OVERLAY_DIR" + +# Step 1: Download binaries +echo "" +echo "==> Step 1: Downloading binaries..." +bash "$BUILD_DIR/../tools/download-kubernetes.sh" "$REPO_DIR/binaries" +bash "$BUILD_DIR/../tools/download-etcd.sh" "$REPO_DIR/binaries" +bash "$BUILD_DIR/../tools/download-cni-plugins.sh" "$REPO_DIR/binaries" +bash "$BUILD_DIR/../tools/download-coredns.sh" "$REPO_DIR/binaries" +bash "$BUILD_DIR/../tools/install-kafka.sh" "$REPO_DIR/binaries" + +# Step 2: Copy overlay files +echo "" +echo "==> Step 2: Copying overlay files..." + +# Copy configs +mkdir -p "$OVERLAY_DIR/etc/cluster-config" +cp -r "$BUILD_DIR/../configs"/* "$OVERLAY_DIR/etc/cluster-config/" + +# Copy systemd units +mkdir -p "$OVERLAY_DIR/etc/systemd/system" +cp "$BUILD_DIR/../systemd"/*.service "$OVERLAY_DIR/etc/systemd/system/" +cp "$BUILD_DIR/../systemd"/*.target "$OVERLAY_DIR/etc/systemd/system/" + +# Copy scripts +mkdir -p "$OVERLAY_DIR/usr/local/bin" +cp "$BUILD_DIR/../tools"/*.sh "$OVERLAY_DIR/usr/local/bin/" +cp "$BUILD_DIR/../tools"/*.py "$OVERLAY_DIR/usr/local/bin/" + +# Step 3: Generate certificates +echo "" +echo "==> Step 3: Generating certificates..." +bash "$BUILD_DIR/../tools/generate-build-certs.sh" "$OVERLAY_DIR/etc/kubernetes/pki" + +# Step 4: Create local repository +echo "" +echo "==> Step 4: Creating local repository..." +createrepo_c "$REPO_DIR/rpms" + +# Step 5: Build ISO using lorax +echo "" +echo "==> Step 5: Building ISO with lorax..." + +# Download Rocky Linux boot files +ROCKY_VERSION="9.3" +ROCKY_BOOT_ISO="Rocky-${ROCKY_VERSION}-x86_64-boot.iso" +ROCKY_URL="https://download.rockylinux.org/pub/rocky/${ROCKY_VERSION}/isos/x86_64/${ROCKY_BOOT_ISO}" + +if [[ ! -f "$REPO_DIR/$ROCKY_BOOT_ISO" ]]; then + echo "==> Downloading Rocky Linux boot ISO..." + curl -L "$ROCKY_URL" -o "$REPO_DIR/$ROCKY_BOOT_ISO" +fi + +# Mount boot ISO to extract vmlinuz and initrd +MOUNT_DIR="/tmp/rocky-mount-$$" +mkdir -p "$MOUNT_DIR" +mount -o loop "$REPO_DIR/$ROCKY_BOOT_ISO" "$MOUNT_DIR" + +# Create working directory for ISO build +ISO_WORK_DIR="/tmp/iso-build-$$" +mkdir -p "$ISO_WORK_DIR"/{isolinux,images,LiveOS} + +# Copy boot files +cp "$MOUNT_DIR/isolinux/vmlinuz" "$ISO_WORK_DIR/isolinux/" +cp "$MOUNT_DIR/isolinux/initrd.img" "$ISO_WORK_DIR/isolinux/" +cp "$MOUNT_DIR/isolinux/isolinux.bin" "$ISO_WORK_DIR/isolinux/" +cp "$MOUNT_DIR/isolinux/ldlinux.c32" "$ISO_WORK_DIR/isolinux/" +cp "$MOUNT_DIR/isolinux/libcom32.c32" "$ISO_WORK_DIR/isolinux/" +cp "$MOUNT_DIR/isolinux/libutil.c32" "$ISO_WORK_DIR/isolinux/" + +umount "$MOUNT_DIR" +rmdir "$MOUNT_DIR" + +# Create isolinux.cfg with kickstart +cat > "$ISO_WORK_DIR/isolinux/isolinux.cfg" <<'ISOLINUX' +default vesamenu.c32 +timeout 100 + +display boot.msg + +label install + menu label ^Install Cluster Node + kernel vmlinuz + append initrd=initrd.img inst.stage2=hd:LABEL=ClusterNode inst.ks=hd:LABEL=ClusterNode:/ks.cfg + +label rescue + menu label ^Rescue installed system + kernel vmlinuz + append initrd=initrd.img inst.stage2=hd:LABEL=ClusterNode rescue +ISOLINUX + +# Copy kickstart file to ISO root +cp "$KICKSTART_FILE" "$ISO_WORK_DIR/ks.cfg" + +# Copy overlay and repo to ISO +cp -r "$OVERLAY_DIR" "$ISO_WORK_DIR/overlay" +cp -r "$REPO_DIR" "$ISO_WORK_DIR/repo" + +# Create ISO +echo "==> Creating bootable ISO..." +genisoimage \ + -o "$OUTPUT_DIR/$ISO_NAME" \ + -b isolinux/isolinux.bin \ + -c isolinux/boot.cat \ + -no-emul-boot \ + -boot-load-size 4 \ + -boot-info-table \ + -J -R -v \ + -V "$ISO_LABEL" \ + "$ISO_WORK_DIR" + +# Make ISO bootable +isohybrid "$OUTPUT_DIR/$ISO_NAME" + +# Add MD5 checksum +implantisomd5 "$OUTPUT_DIR/$ISO_NAME" + +# Cleanup +rm -rf "$ISO_WORK_DIR" + +echo "" +echo "==> ISO build complete!" +echo "==> Output: $OUTPUT_DIR/$ISO_NAME" +echo "==> Size: $(du -h "$OUTPUT_DIR/$ISO_NAME" | cut -f1)" +echo "" +echo "Test with:" +echo " qemu-system-x86_64 -cdrom $OUTPUT_DIR/$ISO_NAME -m 4096 -smp 2" +``` + +## Simplified Build Script (Alternative: mkosi) + +For a more modern, declarative approach: + +**File**: `iso-build/mkosi.conf` + +```ini +[Output] +ImageId=cluster-node +ImageVersion=0.1.0 +Format=disk + +[Distribution] +Distribution=rocky +Release=9 + +[Content] +Packages= + systemd + kubernetes-kubeadm + etcd + vim + tmux + +ExtraTree=/path/to/overlay/ + +[Host] +QemuHeadless=yes +``` + +**Note**: mkosi is cleaner but less mature for RHEL-based distros. Recommend starting with lorax/kickstart. + +## Modified Download Scripts for Build + +Update download scripts to accept output directory: + +**Example**: `tools/download-kubernetes.sh` + +```bash +#!/bin/bash +# Modified to accept output directory for ISO build + +KUBE_VERSION="v1.29.1" +ARCH="amd64" +BASE_URL="https://dl.k8s.io/release/${KUBE_VERSION}/bin/linux/${ARCH}" +OUTPUT_DIR="${1:-/usr/local/bin}" + +mkdir -p "$OUTPUT_DIR" + +BINARIES=( + "kubeadm" + "kubectl" + "kubelet" + "kube-apiserver" + "kube-controller-manager" + "kube-scheduler" +) + +for binary in "${BINARIES[@]}"; do + echo "Downloading $binary to $OUTPUT_DIR..." + curl -L "${BASE_URL}/${binary}" -o "$OUTPUT_DIR/${binary}" + chmod +x "$OUTPUT_DIR/${binary}" +done + +echo "==> Kubernetes binaries downloaded to $OUTPUT_DIR" +``` + +## Testing the ISO + +### QEMU Testing + +```bash +#!/bin/bash +# tools/test-iso-qemu.sh + +ISO_PATH="iso-build/output/cluster-node.iso" +DISK_IMG="test-disk.qcow2" + +# Create virtual disk +qemu-img create -f qcow2 "$DISK_IMG" 40G + +# Boot from ISO +qemu-system-x86_64 \ + -cdrom "$ISO_PATH" \ + -hda "$DISK_IMG" \ + -m 4096 \ + -smp 2 \ + -boot d \ + -netdev user,id=net0 \ + -device virtio-net-pci,netdev=net0 \ + -serial stdio \ + -display none + +# After installation, boot from disk +# qemu-system-x86_64 -hda "$DISK_IMG" -m 4096 -smp 2 -boot c +``` + +### VirtualBox Testing + +```bash +# Create VM +VBoxManage createvm --name "cluster-test" --ostype RedHat_64 --register + +# Configure VM +VBoxManage modifyvm "cluster-test" \ + --memory 4096 \ + --cpus 2 \ + --nic1 bridged \ + --bridgeadapter1 eth0 + +# Create disk +VBoxManage createhd --filename cluster-test.vdi --size 40960 + +# Attach storage +VBoxManage storagectl "cluster-test" --name "SATA" --add sata --controller IntelAhci +VBoxManage storageattach "cluster-test" --storagectl "SATA" --port 0 --device 0 --type hdd --medium cluster-test.vdi +VBoxManage storageattach "cluster-test" --storagectl "SATA" --port 1 --device 0 --type dvddrive --medium cluster-node.iso + +# Start VM +VBoxManage startvm "cluster-test" +``` + +### Multi-Node Test + +```bash +#!/bin/bash +# tools/test-multi-node.sh +# Create 3 VMs for full cluster test + +for i in 1 2 3; do + DISK="test-node-${i}.qcow2" + qemu-img create -f qcow2 "$DISK" 40G + + qemu-system-x86_64 \ + -name "node-${i}" \ + -cdrom iso-build/output/cluster-node.iso \ + -hda "$DISK" \ + -m 4096 \ + -smp 2 \ + -boot d \ + -netdev tap,id=net0,ifname=tap${i},script=no \ + -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:1${i} \ + -daemonize \ + -vnc :${i} + + echo "Node $i started on VNC port 590${i}" +done +``` + +## Build Optimization + +### Caching Downloaded Files + +```bash +# Use a persistent cache directory +CACHE_DIR="$HOME/.cache/cluster-iso-build" +mkdir -p "$CACHE_DIR"/{binaries,rpms} + +# In build.sh, check cache before downloading +if [[ ! -f "$CACHE_DIR/binaries/kubelet" ]]; then + bash tools/download-kubernetes.sh "$CACHE_DIR/binaries" +fi + +cp -r "$CACHE_DIR/binaries"/* "$REPO_DIR/binaries/" +``` + +### Parallel Downloads + +```bash +# Download binaries in parallel +bash tools/download-kubernetes.sh "$REPO_DIR/binaries" & +bash tools/download-etcd.sh "$REPO_DIR/binaries" & +bash tools/download-coredns.sh "$REPO_DIR/binaries" & +wait + +echo "==> All downloads complete" +``` + +## Reproducible Builds + +### Version Lock File + +**File**: `iso-build/versions.lock` + +```yaml +kubernetes: + version: "1.29.1" + sha256: + kubelet: "abc123..." + kubectl: "def456..." + +etcd: + version: "3.5.11" + sha256: "789ghi..." + +ceph: + version: "18.2.1" + +kafka: + version: "3.6.1" + scala_version: "2.13" + +base_iso: + name: "Rocky-9.3-x86_64-boot.iso" + sha256: "..." +``` + +### Checksum Verification + +```bash +#!/bin/bash +# tools/verify-downloads.sh + +set -euo pipefail + +VERSIONS_FILE="iso-build/versions.lock" +BINARIES_DIR="$1" + +while read -r binary expected_sha; do + actual_sha=$(sha256sum "$BINARIES_DIR/$binary" | awk '{print $1}') + + if [[ "$actual_sha" != "$expected_sha" ]]; then + echo "ERROR: Checksum mismatch for $binary" + echo " Expected: $expected_sha" + echo " Actual: $actual_sha" + exit 1 + fi + + echo "✓ $binary: checksum OK" +done < <(yq eval '.kubernetes.sha256 | to_entries | .[] | .key + " " + .value' "$VERSIONS_FILE") +``` + +## CI/CD Integration + +### GitHub Actions Example + +```yaml +# .github/workflows/build-iso.yml +name: Build ISO + +on: + push: + tags: + - 'v*' + +jobs: + build: + runs-on: ubuntu-latest + container: + image: rockylinux:9 + + steps: + - name: Checkout code + uses: actions/checkout@v3 + + - name: Install build dependencies + run: | + dnf install -y lorax createrepo_c genisoimage isomd5sum + + - name: Run build + run: | + bash iso-build/build.sh + + - name: Upload ISO artifact + uses: actions/upload-artifact@v3 + with: + name: cluster-node-iso + path: iso-build/output/cluster-node.iso + + - name: Create release + uses: softprops/action-gh-release@v1 + with: + files: iso-build/output/cluster-node.iso +``` + +## Troubleshooting + +### Build Fails: "lorax command not found" + +```bash +# On Fedora +dnf install lorax anaconda-tui + +# On Rocky Linux +dnf install epel-release +dnf install lorax +``` + +### ISO Won't Boot: "Missing operating system" + +```bash +# Ensure isohybrid was run +isohybrid output/cluster-node.iso + +# Verify boot sector +file output/cluster-node.iso +# Should show: "DOS/MBR boot sector" +``` + +### Kickstart Fails During Post-Install + +```bash +# Boot ISO in rescue mode +# Check logs +cat /tmp/ks-post.log +cat /tmp/anaconda.log + +# Common issues: +# - Missing files in overlay +# - Incorrect paths in kickstart +# - Network not available during post-install +``` + +## Next Steps + +1. Create `iso-build/` directory structure +2. Implement build.sh script +3. Test ISO build on clean Fedora VM +4. Test installation in QEMU +5. Verify cluster-detect runs on first boot +6. Test multi-node cluster formation +7. Document customization procedures + +## Customization Guide + +### Adding Custom Packages + +Edit kickstart file: +```kickstart +%packages +@core +my-custom-package +%end +``` + +### Changing Root Password + +In kickstart: +```kickstart +rootpw --iscrypted $6$rounds=4096$salt$hash +``` + +Generate hash: +```bash +python3 -c 'import crypt; print(crypt.crypt("your-password", crypt.mkmeth(crypt.METHOD_SHA512)))' +``` + +### Embedding Custom Files + +Add to overlay directory: +```bash +mkdir -p overlay/etc/my-app/ +cp my-config.yaml overlay/etc/my-app/ +``` + +## Maintenance + +### Updating Base OS + +```bash +# Update ROCKY_VERSION in build.sh +ROCKY_VERSION="9.4" # When Rocky 9.4 is released + +# Re-download boot ISO +rm repo/Rocky-*.iso +bash iso-build/build.sh +``` + +### Updating Kubernetes Version + +```bash +# Edit versions.lock +kubernetes: + version: "1.30.0" + +# Re-run build +bash iso-build/build.sh +``` + +--- + +**P.S.** This is the glue that makes everything real—from pile of configs to bootable artifact. Kickstart is ugly but battle-tested; lorax is RHEL-native. The build script is bash-heavy but transparent. You can see exactly what's happening, which beats "magic container build tool" when debugging at 2 AM. Test in QEMU first, VirtualBox second, bare metal third. |