Proxmox VE Kernel Upgrade Guide and Troubleshooting

Understanding Proxmox VE Kernels

Proxmox VE ships its own patched kernel packages based on Ubuntu's kernel with additional enterprise-grade features. These kernels include ZFS modules, hardware drivers for server-class equipment, and patches for KVM/QEMU performance. Kernel packages follow the naming convention pve-kernel-X.Y.Z-N-pve and are installed alongside existing kernels, so you always have previous versions to fall back on.

When you run apt dist-upgrade, new kernel packages are automatically installed. However, the new kernel only becomes active after a reboot. This gives you a window to verify the package installed correctly before committing to it.

How proxmox-boot-tool Works

Proxmox uses proxmox-boot-tool to manage kernel boot entries, especially on systems using systemd-boot (common with ZFS root) or GRUB with EFI. The tool reads the list of boot-capable partitions from /etc/kernel/proxmox-boot-uuids and synchronizes kernel images to each ESP (EFI System Partition).

# View the currently configured boot partitions
proxmox-boot-tool status

# List contents of the UUID configuration
cat /etc/kernel/proxmox-boot-uuids

# Refresh boot entries after kernel changes
proxmox-boot-tool refresh

# Manually add a new ESP partition
proxmox-boot-tool init /dev/sda2

# Clean up old kernel entries from the ESP
proxmox-boot-tool clean

If /etc/kernel/proxmox-boot-uuids is empty or contains invalid UUIDs, the boot tool will fail to synchronize kernels, which can leave your system unbootable after a kernel upgrade. Always verify this file has correct entries before upgrading.

Installing a Specific Kernel Version

You can list all available Proxmox kernel packages and install a specific version:

# List all available PVE kernels
apt list pve-kernel-* 2>/dev/null | grep pve-kernel

# List installed kernels
dpkg -l | grep pve-kernel

# Install a specific kernel version
apt install pve-kernel-6.8.12-1-pve

# Check which kernel is currently running
uname -r

# Check which kernels are available at boot
proxmox-boot-tool kernel list

Pinning a Kernel Version with APT

If you need to prevent automatic kernel upgrades (for example, because a newer kernel causes issues with your hardware), you can pin the current kernel version using APT preferences:

# Create a pin file to hold the current kernel
cat > /etc/apt/preferences.d/proxmox-kernel-pin << 'EOF'
Package: pve-kernel-*
Pin: version 6.8.12-1-pve
Pin-Priority: 1001

Package: proxmox-kernel-*
Pin: version 6.8.12-1-pve
Pin-Priority: 1001
EOF

# Verify the pin is active
apt-cache policy pve-kernel-6.8.12-1-pve

Remember to remove or update this pin file when you are ready to accept newer kernels. Running a pinned kernel indefinitely means missing security patches and bug fixes.

Recovering from a Kernel Panic After Update

A kernel panic after a kernel upgrade is alarming but usually recoverable. The most common causes are ZFS module version mismatches, incompatible hardware drivers, and corrupted initramfs images.

Step 1: Boot the previous kernel. At the GRUB menu (hold Shift during POST for BIOS, or use the boot menu for EFI), select "Advanced options for Proxmox VE" and choose the previous kernel version.

Step 2: Rebuild the initramfs for the new kernel. Sometimes the initramfs was not built correctly during installation:

# Boot the old kernel first, then rebuild initramfs for the problematic kernel
update-initramfs -u -k 6.8.12-1-pve

# Refresh boot entries
proxmox-boot-tool refresh

# Try rebooting into the new kernel again
reboot

Step 3: If the new kernel still panics, remove it and stay on the working version:

# Remove the problematic kernel
apt remove pve-kernel-6.8.12-1-pve

# Clean up boot entries
proxmox-boot-tool refresh

# Update GRUB if using legacy boot
update-grub

ZFS Module Mismatch Issues

The most frequent kernel upgrade issue on Proxmox is a ZFS module version mismatch. The ZFS kernel module must match the version compiled for the running kernel. If the module fails to load, ZFS pools will not import and the system may not boot (if root is on ZFS).

# Check ZFS module version
dkms status | grep zfs
modinfo zfs | grep version

# Force reinstall ZFS modules for the current kernel
apt install --reinstall zfs-dkms
# Or for the pre-built module:
apt install --reinstall pve-kernel-$(uname -r)

# Rebuild initramfs to include the correct ZFS module
update-initramfs -u -k all
proxmox-boot-tool refresh

Managing Multiple Installed Kernels

Over time, kernel packages accumulate. Proxmox keeps old kernels installed for safety, but they consume disk space on the boot partition. Clean up old kernels while keeping at least two (the current and one previous):

# List all installed PVE kernels sorted by version
dpkg -l 'pve-kernel-*' | grep '^ii'

# Remove a specific old kernel
apt remove pve-kernel-6.5.13-3-pve

# After removing old kernels, refresh boot entries
proxmox-boot-tool refresh

# Verify remaining kernels
proxmox-boot-tool kernel list

Keeping kernel management under control is an important part of maintaining a healthy Proxmox environment. If you manage multiple nodes, ProxmoxR can help you track which kernel version each node in your cluster is running, making it easier to spot inconsistencies before they cause problems during upgrades or failovers.