mypapers/200706-bootseq/200706-bootseq.html

   1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
   2 <html>
   3   <head>
   4     <link rel="stylesheet" href="../mrtg-td/slides.css" type="text/css">
   5     <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   6     <meta name="Language" content="en">
   7     <meta name="Author" content="Petter Reinholdtsen">
   8 <style type="text/css">
   9 PRE {
  10   font-size: smaller
  11 }
  12 </style>
  13
  14   </head>
  15   <body>
  16
  17 <h1>Dependency based boot sequence</h1>
  18
  19 <p>Reordering the boot sequence for correctness and speed</p>
  20
  21 <p>There are several boot options available in Debian, and the default
  22    SysV-derived one can be tuned and corrected. The talk will present
  23    the available options, and go into details on some of the pitfalls
  24    and advantages of the default boot system. It will present some
  25    results based on the Google summer of code project working on the
  26    boot system.</p>
  27
  28 <div class="presenter">Petter Reinholdtsen
  29    <br>pere@hungry.com
  30    <br>Debconf 2007 Edinburgh, 2007-06-17</div>
  31
  32 <h2>About the speaker</h2>
  33
  34 <ul>
  35
  36   <li>Using Linux since 1993
  37   <li>Debian Developer since 2002
  38
  39   <li>Educated at University of Tromsø, Western Australia and Oslo
  40
  41   <li>Employed at University of Oslo to develop system administration tools
  42   <li>Leader of the Norwegian Unix User Group
  43
  44   <li>Co-maintaining sysvinit, maintaining insserv
  45   <li>and a few other (Debian Edu, Debian GIS, Debian
  46       Java, Debian Boot, etc)
  47
  48 </ul>
  49
  50 <h2>Summary of this talk</h2>
  51
  52 <ul>
  53   <li>Quick overview of the boot system alternatives in Debian
  54   <li>Some problems with the current system
  55   <li>How to improve the boot sequence ordering
  56   <li>how to prepare packages for this
  57 </ul>
  58
  59 <p>Based on work done by Carlos Villegas during Google Summer of Code
  60 2006, available from
  61 <a href="http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/">http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/</a>.</p>
  62
  63 <h2>Debian include several boot system options</h2>
  64
  65 <p>SysV init is only the most used boot system in Debian, not the only
  66    one.</p>
  67
  68 <ul>
  69
  70   <li>file-rc
  71   <li>upstart
  72   <li>init-ng
  73   <li>minit
  74   <li>runit
  75   <li>metainit initiative for init.d scripts
  76
  77 </ul>
  78
  79 <h2>The SysV boot system</h2>
  80
  81 <ul>
  82
  83 <li>Start first all scripts in /etc/rcS.d/
  84
  85 <li>For the runlevel 2, first all K##scripts in /etc/rc2.d/, next all
  86   S##script.
  87
  88 <li>All .sh scripts are sourced, the rest are exec-ed
  89
  90 </ul>
  91
  92 <a href="bootchart-sid.png"><img src="bootchart-sid.png" width="100%"></a>
  93
  94 <h2>Current sid boot sequence (base install)</h2>
  95
  96 <pre>
  97 /etc/rcS.d:
  98 README                  S30checkfs.sh
  99 S01glibc.sh             S30procps.sh
 100 S02hostname.sh          S35mountall.sh
 101 S02mountkernfs.sh       S36mountall-bootclean.sh
 102 S04mountdevsubfs.sh     S39ifupdown
 103 S05bootlogd             S40networking
 104 S10checkroot.sh         S45mountnfs.sh
 105 S11hwclock.sh           S46mountnfs-bootclean.sh
 106 S12mtab.sh              S55bootmisc.sh
 107 S18ifupdown-clean       S55urandom
 108 S20module-init-tools    S99stop-bootlogd-single
 109 S25libdevmapper1.02.1
 110
 111 /etc/rc2.d:
 112 README                  S89cron
 113 S10sysklogd             S99rc.local
 114 S11klogd                S99rmnologin
 115 S20makedev              S99stop-bootlogd
 116 S20openbsd-inetd
 117 </pre>
 118
 119 <h2>Current sid shutdown sequence (base install)</h2>
 120 <pre>
 121 /etc/rc6.d:
 122 K11cron                 S30urandom
 123 K20makedev              S31umountnfs.sh
 124 K20openbsd-inetd        S35networking
 125 K25hwclock.sh           S36ifupdown
 126 K89klogd                S40umountfs
 127 K90sysklogd             S60umountroot
 128 README                  S90reboot
 129 S20sendsigs
 130 </pre>
 131
 132 <p>But is it correct?
 133
 134 <h2>Problems with the current boot system</h2>
 135
 136 <ul>
 137   <li>Slightly incorrect ordering during startup and shutdown
 138   <li>Hard to pick correct sequence number
 139   <li>The halt/reboot runlevels are inconsistent/confusing (or wrong,
 140       depending on your view)
 141
 142   <li>Impossible to parallelise
 143 </ul>
 144
 145 <h2>How can we verify the boot sequence correctness?</h2>
 146
 147 <p><a href="depgraph-base.png"><img align="right" src="depgraph-base.png" width="100%"></a>
 148
 149 <ul>
 150   <li>Document the dependencies of each script
 151   <li>Verify the scripts are started in  dependency order
 152
 153 </ul></p>
 154
 155 <p>Example header:</p>
 156
 157 <pre>
 158 #!/bin/sh -e
 159 ### BEGIN INIT INFO
 160 # Provides:          networking
 161 # Required-Start:    mountkernfs ifupdown $local_fs
 162 # Required-Stop:     ifupdown $local_fs
 163 # X-Start-Before:    nis
 164 # X-Stop-After:      nis
 165 # Default-Start:     S
 166 # Default-Stop:      0 6
 167 # Short-Description: Raise network interfaces.
 168 ### END INIT INFO
 169 </pre>
 170
 171 <h2>Verifying the current sequence</h2>
 172
 173 <p>Using these headers and a script from the insserv package, we can
 174 verify the boot order:
 175
 176 <pre>
 177 # /usr/share/insserv/check-initd-order   -o
 178 LSB header missing in /etc/rcS.d/S11hwclock.sh
 179 LSB header missing in /etc/rcS.d/S25libdevmapper1.02.1
 180 Incorrect order modutils@? > checkfs.sh@30
 181 Incorrect order $all@? > stop-bootlogd-single@99
 182 Incorrect order $time@? > sysklogd@10
 183 Incorrect order $time@? > cron@89
 184 Incorrect order $all@? > rmnologin@99
 185 #
 186 </pre>
 187
 188 <p>For the packages without dependency information, override files can be
 189 read from <tt>/usr/share/insserv/overrides/</tt> and
 190 <tt>/etc/insserv/overrides/</tt>.
 191
 192 <h2>Verifying the current sequence II</h2>
 193 <p>Current status (base task install), with override files.</p>
 194 <pre>
 195 # /usr/share/insserv/check-initd-order
 196 Incorrect order $local_fs@? > hwclock.sh@11
 197 Incorrect order $remote_fs@? > hwclock.sh@11
 198 Incorrect order modutils@? > libdevmapper1.02.1@25
 199 Incorrect order modutils@? > checkfs.sh@30
 200 Incorrect order $all@? > stop-bootlogd-single@99
 201 Incorrect order $all@? > rmnologin@99
 202 #
 203 </pre>
 204
 205 <h2>Verifying the current sequence III</h2>
 206 <p>Current status (desktop task install)</p>
 207 <pre>
 208 # /usr/share/insserv/check-initd-order -o
 209 LSB header missing in /etc/rcS.d/S11hwclock.sh
 210 LSB header missing in /etc/rcS.d/S25libdevmapper1.02.1
 211 Incorrect order modutils@? > checkfs.sh@30
 212 Incorrect order $all@? > stop-bootlogd-single@99
 213 Incorrect order $time@? > sysklogd@10
 214 LSB header missing in /etc/rc2.d/S19hplip
 215 LSB header missing in /etc/rc2.d/S20cupsys
 216 LSB header missing in /etc/rc2.d/S20hotkey-setup
 217 LSB header missing in /etc/rc2.d/S89anacron
 218 Incorrect order $time@? > cron@89
 219 Incorrect order $all@? > rmnologin@99
 220 #
 221 </pre>
 222
 223 <h2>Reordering based on dependency information</h2>
 224
 225 <p>Advantages:
 226
 227 <ul>
 228
 229 <li>No need to replace existing init.d scripts
 230 <li>Activated by adding headers to the scripts or use override files
 231 <li>Experimental code to enable it is in the insserv package
 232
 233 </ul>
 234
 235 <p>How to enable this by default
 236
 237 <ul>
 238
 239 <li>Install the insserv package
 240 <li>Make all start-symlinks in rc0 and rc6 to stop symlinks
 241 <li>Replace update-rc.d
 242 <li>Reorder rc*.d/*
 243 </ul>
 244
 245 <h2>Status of dependency based sysv boot sequence</h2>
 246
 247 <p><a href="insserv-enable.png"><img align="right" src="insserv-enable.png" width="100%"></a>
 248 <ul>
 249   <li>Majority of common packages provides dependency information
 250   <li>Correct startup sequence for base and desktop tasks when enabled.
 251   <li>Fairly correct shutdown sequence.
 252   <li>Some packages fail to install because of incorrect init.d script
 253       registration order
 254
 255 </ul>
 256
 257 <p>
 258 To enable: <tt>BAD_INSSERV_HACKER=true dpkg-reconfigure insserv</tt>
 259 <p>Testrun:
 260 <pre>
 261 insserv:/var/tmp/chroot-sid-base-reorder# BAD_INSSERV_HACKER=true dpkg-reconfigure i
 262 info: Backing up existing boot scripts in /var/lib/insserv/bootscripts-20070615T1617.tar.gz
 263 info: Reordering boot system, log to /var/lib/insserv/run-20070615T1617.log
 264 info: Recording new boot sequence in /var/lib/insserv/bootscripts-20070615T1617-after.list
 265 info: Use '/usr/sbin/update-bootsystem-insserv restore' to restore the old boot sequence.
 266 success: Boot system successfully converted
 267 Adding `diversion of /usr/sbin/update-rc.d to /usr/sbin/update-rc.d.distrib by insserv'
 268 #
 269 </pre>
 270
 271 <h2>Similar boot graph</h2>
 272
 273 <a href="bootchart-reordered.png"><img src="bootchart-reordered.png" width="100%"></a>
 274
 275 <h2>Reordered sid boot sequence (base install)</h2>
 276 <pre>
 277 /etc/rcS.d:
 278 README                  S05mtab.sh
 279 S01glibc.sh             S06mountall.sh
 280 S01libdevmapper1.02.1   S07ifupdown
 281 S02hostname.sh          S07mountall-bootclean.sh
 282 S02mountkernfs.sh       S07urandom
 283 S03mountdevsubfs.sh     S08networking
 284 S03procps.sh            S09mountnfs.sh
 285 S04bootlogd             S10mountnfs-bootclean.sh
 286 S04checkroot.sh         S11bootmisc.sh
 287 S05checkfs.sh           S11hwclock.sh
 288 S05ifupdown-clean       S11stop-bootlogd-single
 289 S05module-init-tools
 290 /etc/rc2.d:
 291 README                  S11openbsd-inetd
 292 S07makedev              S11rc.local
 293 S07sysklogd             S12stop-bootlogd
 294 S08cron                 S13rmnologin
 295 S08klogd
 296 </pre>
 297
 298 <h2>Reordered sid shutdown sequence (base install)</h2>
 299
 300 <pre>
 301 /etc/rc6.d:
 302 K03openbsd-inetd        K31umountnfs.sh
 303 K06cron                 K35networking
 304 K06klogd                K36ifupdown
 305 K07sysklogd             K40umountfs
 306 K20makedev              K60umountroot
 307 K20sendsigs             K90reboot
 308 K25hwclock.sh           README
 309 </pre>
 310
 311 <h2>I'm not brave enough, get me out of here</h2>
 312
 313 <p>It is possible to automatically undo the reordering, <em>once</em>,
 314 by reconfiguring insserv and disabling the dependency based boot.</p>
 315
 316 <pre>
 317 # BAD_INSSERV_HACKER=true dpkg-reconfigure i
 318 info: Disabling dependency based boot system
 319 Removing `diversion of /usr/sbin/update-rc.d to /usr/sbin/update-rc.d.distrib by insserv'
 320 info: Restoring using backed up copy of init.d/ and rc*.d/.
 321 info: successfully restored backup of init.d scripts
 322 #
 323 </pre>
 324
 325 <h2>How can package maintainers assist</h2>
 326
 327 <ul>
 328
 329 <li>Update their init.d scripts to include correct dependency
 330     information using the LSB headers defined for this purpose,
 331     <a href="http://wiki.debian.org/LSBInitScripts">http://wiki.debian.org/LSBInitScripts</a>
 332 <li>Make sure their init.d scripts are registered in dependency order
 333 <li>Start using insserv to detect bugs in the packages
 334 </ul>
 335
 336 <h2>Even running boot scripts in parallel is possible</h2>
 337
 338 <pre>echo CONCURRENCY=shell >> /etc/default/rcS</pre>
 339 <a href="bootchart-reord-concur.png"><img src="bootchart-reord-concur.png" width="100%"></a>
 340
 341
 342 <h2>Thank you very much</h2>
 343
 344 <h3>Questions?</h3>
 345
 346 <p><a href="http://www.hungry.com/~pere/mypapers/200706-bootseq/">http://www.hungry.com/~pere/mypapers/200706-bootseq/</a></p>
 347
 348   </body>
 349 </html>