Our Genome Scan sender went down because its login token lived in session storage that vanished. The readouts were safe in the queue, but nothing could send. The rule we set from that failure is simple: every token for a service we always use gets the longest life the provider offers, and its storage has to survive a restart. If the auth can break, the watchdog has to be able to find the break and restart the login itself.
Keeping the readouts in the queue is what made this a delay instead of a loss. The watchdog line is the one I'd keep, and one check pairs well with it: an alert on the age of the oldest queued readout, which still fires if the login check is the part that broke.
commentslikes