Remotely rebooting Linux (Debian in specific)

			Davor Ocelic, docelic+spinlocksolutions.com, Jun 16, 2004.
			Aug 8, 2004: add lilo -R step, and fix some formatting issues
			Jan 4, 2005: posted article to www.debian-administration.org site

		

To perform successful remote reboot (to upgrade the kernel usually), few things need to be taken into account:

• Adding a special bootloader entry (LILO example):

  			image=/vmlinuz.new
  				label=Newkernel
  				append="panic=5 newkernel"
  				read-only
  				optional
  			

  This will create a special image name to test the new kernel. panic=5 makes sure the kernel autoreboots on panic in 5 secods, and "newkernel" is an arbitrary "newinit"-like name we choose, so we can later check if we're in test phase or normal run. (If you didn't understand the last sentence, don't worry - just use the name "newkernel" as shown in the example).
  
  
  
• Put the testnet script in /etc/init.d/ and activate it:

  				update-rc.d testnet start 40 S .
  			

  This will install a script that will test network connectivity once you reboot. This will handle cases where the kernel does not panic (it mounts the root filesystem at least), but then something goes wrong and it doesnt start up the network properly.
  
  
  
• If you installed the kernel image from the .deb package, make sure the /vmlinuz link still points to the old kernel, and /vmlinuz.new to the new one.
  
  
  
• Also make sure there's an account left open for the colo facility personnel to access the system if you mess it up. Use adduser to create the account "support", then add them to sudoers file without a password:

  			echo "support ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
  			

  Run this little script to generate a random password for the account and print login/password info on the local console (you could copy it to a file and then run the file if you think it's more convenient):

  				makepasswd --crypt| perl -ne 'BEGIN{ ($p,$c) = split /\s+/, <STDIN>;
  				`usermod -p $c support;
  				echo ACCOUNT login/pwd is support/$p >> /etc/issue`;
  				print "Temporary password for the support account is: $p/$c\n"; exit }'
  			

  
  (Don't exactly follow this advice and put the login/password information in /etc/issue unless you're aware of the security implications).
  
  
• Write the new lilo.conf (which we modified in step 1), and make the test kernel (newkernel image) the default for just the next boot:

  				lilo
  				lilo -R newkernel
  			

• Reboot and see how it plays out. If you get into the new kernel, adjust /vmlinuz and /vmlinuz.old symlinks appropriately, re-run lilo, and reboot once again to the new image (which is now the default (so no lilo -R)).
  
  
  
• When the new system comes up the second time, remove the login/pwd info from /etc/issue and disable 'support' account.