mypapers/200802-bootsequence/200802-bootsequence.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
   2 <html>
   3   <head>
   4   <title>Talk: Reordering the Debian boot sequence for correctness and speed</title>
   5     <link rel="stylesheet" href="../mrtg-td/slides.css" type="text/css">
   6     <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   7     <meta name="Language" content="en">
   8     <meta name="Author" content="Petter Reinholdtsen">
   9   </head>
  10   <body>
  11
  12 <a href="../200706-bootseq/200706-bootseq.html">last related talk</a>
  13
  14 <h1><A href="http://www.fosdem.org/2008/schedule/events/debian_boot">Reordering the Debian boot sequence for correctness and speed</a></h1>
  15
  16 <p>There are subtle bugs in the Debian boot and shutdown sequence.
  17 They are hard to find, as they normally only affect rare combination
  18 of packages, and harder to fix, as they normally require the combined
  19 work of several maintainers and changes in several packages.  This
  20 talk is about the release goal for Lenny to solve them, and gain a few
  21 advantages on the way.</p>
  22
  23 <div class="presenter">Petter Reinholdtsen - one of the sysvinit maintainers
  24   <br>pere@hungry.com
  25   <br>FOSDEM 2008, 2008-02-26</div>
  26
  27 <!--
  28
  29 There are subtle bugs in the debian boot and shutdown sequence. They
  30 are hard to find, as they normally only affect rare combination of
  31 packages, and harder to fix, as they normally require the combined
  32 work of several maintainers and changes in several packages.
  33
  34 One way to find these bugs is to document the dependencies of all
  35 init.d scripts, and use this information to check that the ordering is
  36 correct. When such information is available, it is also possible to
  37 reorder the boot and shutdown sequence to make sure all dependencies
  38 are fulfilled.
  39
  40 It is also possible to run scripts in parallel, to speed up the boot,
  41 when the order they need to run in is known.
  42
  43 This talk is about how all of this can be done with Debian.
  44 -->
  45
  46 <h2>Summary</h2>
  47
  48 <!--
  49
  50  - how do sysvinit boot work
  51    - runlevels
  52    - inittab
  53    - rc*.d/
  54  - ordering problem
  55  - how does it work in debian
  56  - effect on the boot
  57  - insserv
  58  - speed change?
  59  - concurrency
  60
  61  - how to write lsb headers
  62    - provides
  63    - deps
  64    - runlevel list
  65
  66  - status in debian
  67    - need better doc
  68    - fix insserv bugs
  69
  70  - what is needed to convert
  71    - add LSB header to packages and get the change into testing
  72    - update policy
  73    - more users to test headers and detect errors
  74    - install and activate insserv by default
  75
  76 -->
  77
  78 <h2>SysV init in Debian - Booting</h2>
  79
  80 <p>Note, this is the stuff going on after the initrd part is done.
  81 The very early boot is done before hard drive partitions is mounted.</p>
  82
  83 <ol>
  84
  85 <li>/sbin/init start, which look at /etc/inittab and decides what to
  86   do.</li>
  87
  88 <li>The scripts in /etc/rcS.d/ is executed in sequence by
  89   /etc/init.d/rc, with <tt>start</tt> as the argument.</li>
  90
  91 <li>Depending on the runlevel, the scripts for the given runlevel is
  92   executed (normally the ones in /etc/rc2.d/) are executed in
  93   sequence, first the stop scripts with <tt>stop</tt> as their
  94   argument, next the start scripts with <tt>start</tt> as their
  95   argument.  The rc*.d/ directories contain symlinks the files in to
  96   /etc/init.d/.</li>
  97
  98 <li>The ordering is important.</li>
  99
 100 <li>The single-user runlevel will present a login prompt after rcS.d/
 101   was executed.  Runlevel 1 is not the single user runlevel, but it
 102   behaves as a better single user.</li>
 103
 104 </ol>
 105
 106 <h2>SysV init in Debian - Switching runlevels</h2>
 107
 108 <ul>
 109
 110 <li>Call <tt>telinit X</tt> where X is the new runlevel, one of S, 0,
 111   1, 2, 3, 4, 5, 6.</li>
 112
 113 <li>init runs in sequence all stop scripts in /etc/rcX.d/ that also
 114    has a start symlink in the previous runlevel /etc/rcY.d/, with
 115    <tt>stop</tt> as their argument.</li>
 116
 117 <li>init then run in sequence all start scripts in /etc/rcX.d/, with
 118   <tt>start</tt> as their argument.</li>
 119
 120 </ul>
 121
 122 <p>Note that switching to runlevel S will not run the scripts in
 123 /etc/rcS.d/.  To get a similar effect after boot, switch to runlevel
 124 1.  It will (should) kill all services and prepare the machine for
 125 maintenance.</p>
 126
 127 <h2>SysV init in Debian - Shutting down</h2>
 128
 129 <p>This is equivalent to switching to runlevel 0 (halt) or 6 (reboot),
 130   with a minor exception.  All scripts (both start and stop) are
 131   executed with the <tt>stop</tt> argument.
 132
 133 <h2>The ordering problem</h2>
 134
 135 <p>Script ordering is vital for this to work.  And how are the scripts
 136 ordered?  By numbers.
 137
 138 <p>And the numbers are picked using skills, knowledge and negotiation.
 139 Getting it right is often hard.
 140
 141 <p>The current Debian default is wrong.  Stop sequence should by
 142 default be the reverse of the start sequence.  It isn't.
 143
 144 <p>Reordering is hard and require cooperation between maintainers of
 145 all packages involved.  Given two packages with two scripts inserted
 146 with the default settings in Debian:
 147
 148 <p>Package A: script_a sequence 20 (start and stop)
 149 <br>Package B: script_b sequence 20 (start and stop)
 150
 151 <p>Along come script C, which should run before script_a and after
 152 script_b.  Current solution is to change packages A and C or packages
 153 B and C to get something like this:
 154
 155 <p>Package A: script_a start seq. 22, stop seq. 18
 156 <br>Package B: script_b sequence 20 (start and stop)
 157 <br>Package C: script_c start seq 21, stop seq 19
 158
 159 <p>If other scripts depend on the old order of script_a, they will
 160 have to change their sequence number too.  Only way to discover this
 161 is by a lot of testing, or documenting script dependencies.
 162
 163 <h2>A ordering solution</h2>
 164
 165 <p>Let each script document its dependency, and generate sequence
 166 numbers using this dependency information.  Example:
 167
 168 <p>Package A: script_a depend on nothing
 169 <br>Package B: script_b depend on nothing
 170 <br>Package C: script_c depend on script_b, a dependency of script_a
 171
 172 <p>Generated sequence:
 173
 174 <p>script_b start seq 1, stop seq 3
 175 <br>script_c start seq 2, stop seq 2
 176 <br>script_a start seq 3, stop seq 1
 177
 178 <p>An implementation of this system is the dependency based boot
 179 sequencing, provided in the insserv package.</p>
 180
 181
 182 <h2>LSB headers for insserv</h2>
 183
 184 <ul>
 185   <li>"Provides" header, list the Facility/service provided by the
 186     script.  Do not list virtual facilities (like $time)</li>
 187
 188   <li>Runlevel entries (Default-Start and Default-Stop headers), list
 189     what runlevels to activate for this script
 190
 191   <li>Dependency entries (Required-Start, Required-Stop, Should-Start,
 192     Should-Stop, X-Start-Before, X-Stop-After), list the
 193     facilities/services needed by this script.  It will start after
 194     its start dependencies and stop before its stop dependencies.  The
 195     X-* entreis are reverse dependencies.  Required-* are hard
 196     dependencies (will install even if they are missing), while
 197     Should-* and X-* are soft dependencies (only taken into account if
 198     scripts providing them are present).</li>
 199
 200 </ul>
 201
 202 <h2>What to list as dependencies I</h2>
 203
 204 <p>If your package used the default update-rc.d settings before, this
 205 is the header for you:</p>
 206 <pre>
 207 ### BEGIN INIT INFO
 208 # Provides:          scriptname
 209 # Required-Start:    $remote_fs $syslog
 210 # Required-Stop:     $remote_fs $syslog
 211 # Default-Start:     2 3 4 5
 212 # Default-Stop:      0 1 6
 213 ### END INIT INFO
 214 </pre>
 215
 216 <p>$remote_fs is needed by all scripts using files in /usr/.  $syslog
 217 is needed only by scripts starting services logging to syslog.</p>
 218
 219 <h2>What to list as dependencies II</h2>
 220
 221 <p>In the common case, the start and stop dependencies are identical.
 222
 223 <p>Prefer virtual dependencies over specific dependencies
 224
 225 <h2>Virtual facilities</h2>
 226
 227 <p>Linux Software Base version 3.2 define these virtual facilities:
 228
 229 <dl>
 230
 231 <dt>$local_fs
 232 <dd>all local file systems are mounted.
 233
 234 <dt>$network
 235 <dd>basic networking support is available. Example: a server program
 236 could listen on a socket.
 237
 238 <dt>$portmap
 239 <dd>daemons providing SunRPC/ONCRPC portmapping service as defined in
 240 RFC 1833: Binding Protocols for ONC RPC Version 2 (if present) are
 241 running.
 242
 243 <dt>$remote_fs
 244 <dd>all remote file systems are available. In some configurations, file
 245 systems such as /usr may be remote. Many applications that require
 246 $local_fs will probably also require $remote_fs.
 247
 248 <dt>$time
 249 <dd>the system time has been set, for example by using a network-based
 250 time program such as ntp or rdate, or via the hardware Real Time
 251 Clock.
 252
 253 <dt>$syslog
 254 <dd>system logger is operational.
 255
 256 <dt>$named
 257 <dd>IP name-to-address translation, using the interfaces described in
 258 this specification, are available to the level the system normally
 259 provides them. Example: if a DNS query daemon normally provides this
 260 facility, then that daemon has been started.
 261
 262 </dl>
 263
 264 <p>All of these represent points in time during boot and shutdown.
 265
 266 <h2>Status of dependency based boot</h2>
 267
 268 <img alt="LSB header progress graph" src="lsb-header-progress.png" width="50%" align="right">
 269
 270 <p>Release goal for Debian Lenny.
 271
 272 <p>Packages with LSB header: 654 of 866 (76%)
 273 <br>Unsolved BTS reports: XXX
 274 <br>Packages without BTS reports: ~150
 275 <br>Last package projected fixed 2008-07-19 with the current rate
 276
 277 <p>Need better documentation
 278
 279 <p>Need debian policy updates
 280
 281 <p>Fix insserv bugs
 282
 283 <p>
 284
 285 <h2>References</h2>
 286
 287 http://wiki.debian.org/LSBInitScripts
 288 http://wiki.debian.org/LSBInitScripts/DependencyBasedBoot
 289
 290     <h2>Thank you very much</h2>
 291
 292     <h3>Questions?</h3>
 293
 294     <p><a href="http://www.hungry.com/~pere/mypapers/200802-bootsequence/200802-bootsequence.html">http://www.hungry.com/~pere/mypapers/200802-bootsequence/200802-bootsequence.html</a></p>
 295
 296   </body>
 297 </html>