Proxmox / High Availability

A Two-Node Proxmox Cluster Cannot Survive One Node Rebooting: You Need a QDevice

Buying a second Proxmox node to remove a single point of failure is the right instinct. Then you cluster them, reboot one for updates, and discover the survivor will not start guests. Two nodes means two votes. Majority of two is two. Losing either one leaves you below it, and a Proxmox node without quorum refuses to act. You have to solve this before forming the cluster, not after.

Why the survivor deliberately refuses

This is not a bug and it is not conservatism. It is the only safe behaviour available.

A node that has lost contact with its peer cannot tell the difference between two situations: the peer is down, or the peer is alive and the network between them broke. In the second case, both nodes are sitting there deciding whether to start the same guests off the same shared storage. Two nodes running the same VM against one disk destroys the filesystem in seconds.

Requiring a majority makes that impossible, because both halves of a split cannot both hold one. With only two votes in the cluster there is no majority to be had by one node, so the survivor stops. Correct, and not what you wanted.

pvecm status
# Expected votes: 2
# Highest expected: 2
# Total votes: 1          <-- one node gone
# Quorate: No             <-- nothing will start

The fix is a third voter that is not a third node

A QDevice is an external tiebreaker running corosync-qnetd. It holds a vote, it does not run guests, and it does not need to be anything like a Proxmox host. Three votes means one node plus the QDevice is two of three, a real majority, so the survivor keeps running.

# on the external voter
apt install corosync-qnetd

# on both cluster nodes
apt install corosync-qdevice

# from one node
pvecm qdevice setup <qdevice-ip>
pvecm status      # Expected votes: 3

BE WARNED: the QDevice cannot live on either node. Obvious once said, and easy to get wrong when you are picking a host. Any VM or container running on a cluster node dies with that node, and takes its vote with it, which leaves you exactly where you started. In this fleet both the backup server and the GPU box were disqualified for precisely that reason, since both are guests on the node in question. Candidates are a cheap VPS, a Raspberry Pi, a NAS, or any small always-on machine outside the cluster.

Three things that do not transfer to a second node

Adding a node quietly invalidates work you already did, because several settings are per-node rather than cluster-wide:

Startup order and boot delays are per-node. A carefully tiered boot sequence on the first host means nothing to guests that end up on the second. Re-tier after any migration, or the ordering you designed silently stops applying to half the fleet.
Any watchdog script you wrote is probably single-node. Anything enumerating pct list and qm list sees the local node only. It needs to become cluster-aware or be deployed on both.
PCIe passthrough blocks live migration. A guest with a passed-through GPU cannot be live-migrated, which constrains every "drain a node before maintenance" plan you were about to make. Find out which guests are pinned before you rely on being able to empty a host.

HA and a watchdog solve different problems

Once real quorum exists, Proxmox HA can restart guests elsewhere when a node fails. That supersedes a homegrown watchdog for node failure, and it is worth having.

It does not cover the other case, which is more common: a guest that is stopped while its node is perfectly healthy. HA sees a healthy node and does nothing. Something crashed, something was stopped by hand and never restarted, an update left it down. Keep a watchdog for that, and see a guest watchdog with an escape hatch.

If you already have two nodes and no QDevice

There is a documented escape hatch, and it is a genuine footgun:

pvecm expected 1     # temporarily lower the expected vote count

That tells the survivor to consider itself quorate alone. It gets your guests running now, and it disables the exact protection that prevents both nodes writing to the same disk. Use it to recover a real outage, undo it the moment the peer is back, and never leave it in place as a configuration. If you find yourself reaching for it routinely, you needed a QDevice.

When this isn't your problem

You have three or more nodes. Odd numbers give you a natural majority and no QDevice is needed.
You are not clustering at all. Two standalone hosts have no quorum concept. You lose central management and live migration, and you gain complete independence, which is a perfectly reasonable trade for a homelab.
No shared storage. The split-brain risk is smaller, but the quorum behaviour is identical, so the survivor still stops. The QDevice is still the answer.