mypapers/200802-bootsequence/200802-bootsequence.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
   2 <html>
   3   <head>
   4   <title>Talk: Reordering the Debian boot sequence for correctness and speed</title>
   5     <link rel="stylesheet" href="../mrtg-td/slides.css" type="text/css">
   6     <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   7     <meta name="Language" content="en">
   8     <meta name="Author" content="Petter Reinholdtsen">
   9   </head>
  10   <body>
  11
  12 <a href="../200706-bootseq/200706-bootseq.html">last related talk</a>
  13
  14 <h1><A href="http://www.fosdem.org/2008/schedule/events/debian_boot">Reordering the Debian boot sequence for correctness and speed</a></h1>
  15
  16 <p>There are subtle bugs in the Debian boot and shutdown sequence.
  17 They are hard to find, as they normally only affect rare combination
  18 of packages, and harder to fix, as they normally require the combined
  19 work of several maintainers and changes in several packages.  This
  20 talk is about the release goal for Lenny to solve them, and gain a few
  21 advantages on the way.</p>
  22
  23 <div class="presenter">Petter Reinholdtsen - one of the sysvinit maintainers
  24   <br>pere@hungry.com
  25   <br>FOSDEM 2008, 2008-02-26</div>
  26
  27 <!--
  28
  29 There are subtle bugs in the debian boot and shutdown sequence. They
  30 are hard to find, as they normally only affect rare combination of
  31 packages, and harder to fix, as they normally require the combined
  32 work of several maintainers and changes in several packages.
  33
  34 One way to find these bugs is to document the dependencies of all
  35 init.d scripts, and use this information to check that the ordering is
  36 correct. When such information is available, it is also possible to
  37 reorder the boot and shutdown sequence to make sure all dependencies
  38 are fulfilled.
  39
  40 It is also possible to run scripts in parallel, to speed up the boot,
  41 when the order they need to run in is known.
  42
  43 This talk is about how all of this can be done with Debian.
  44 -->
  45
  46 <h2>Outline</h2>
  47
  48 <!--
  49
  50  - how do sysvinit boot work
  51    - runlevels
  52    - inittab
  53    - rc*.d/
  54  - ordering problem
  55  - how does it work in debian
  56  - effect on the boot
  57  - insserv
  58    - how to test it
  59  - speed change?
  60  - concurrency
  61
  62  - how to write lsb headers
  63    - provides
  64    - deps
  65    - runlevel list
  66
  67  - status in debian
  68    - need better doc
  69    - fix insserv bugs
  70
  71  - what is needed to convert
  72    - add LSB header to packages and get the change into testing
  73    - update policy
  74    - more users to test headers and detect errors
  75    - install and activate insserv by default
  76
  77 -->
  78
  79 <h2>SysV init in Debian - Booting</h2>
  80
  81 <p>Note, this is the stuff going on after the initrd part is done.
  82 The very early boot is done before hard drive partitions is mounted.</p>
  83
  84 <ol>
  85
  86 <li>/sbin/init start, which look at /etc/inittab and decides what to
  87   do.</li>
  88
  89 <li>The scripts in /etc/rcS.d/ is executed in sequence by
  90   /etc/init.d/rc, with <tt>start</tt> as the argument.</li>
  91
  92 <li>Depending on the runlevel, the scripts for the given runlevel is
  93   executed (normally the ones in /etc/rc2.d/) are executed in
  94   sequence, first the stop scripts with <tt>stop</tt> as their
  95   argument, next the start scripts with <tt>start</tt> as their
  96   argument.  The rc*.d/ directories contain symlinks the files in to
  97   /etc/init.d/.</li>
  98
  99 <li>The ordering is important.</li>
 100
 101 <li>The single-user runlevel will present a login prompt after rcS.d/
 102   was executed.  Runlevel 1 is not the single user runlevel, but it
 103   behaves as a better single user.</li>
 104
 105 </ol>
 106
 107 <h2>SysV init in Debian - Switching runlevels</h2>
 108
 109 <ul>
 110
 111 <li>Call <tt>telinit X</tt> where X is the new runlevel, one of S, 0,
 112   1, 2, 3, 4, 5, 6.</li>
 113
 114 <li>init runs in sequence all stop scripts in /etc/rcX.d/ that also
 115    has a start symlink in the previous runlevel /etc/rcY.d/, with
 116    <tt>stop</tt> as their argument.</li>
 117
 118 <li>init then run in sequence all start scripts in /etc/rcX.d/, with
 119   <tt>start</tt> as their argument.</li>
 120
 121 </ul>
 122
 123 <p>Note that switching to runlevel S will not run the scripts in
 124 /etc/rcS.d/.  To get a similar effect after boot, switch to runlevel
 125 1.  It will (should) kill all services and prepare the machine for
 126 maintenance.</p>
 127
 128 <h2>SysV init in Debian - Shutting down</h2>
 129
 130 <p>This is equivalent to switching to runlevel 0 (halt) or 6 (reboot),
 131   with a minor exception.  All scripts (both start and stop) are
 132   executed with the <tt>stop</tt> argument.
 133
 134 <h2>The ordering problem</h2>
 135
 136 <p>Script ordering is vital for this to work.  And how are the scripts
 137 ordered?  By numbers.
 138
 139 <p>And the numbers are picked using skills, knowledge and negotiation.
 140 Getting it right is often hard.
 141
 142 <p>The current Debian default is wrong.  Stop sequence should by
 143 default be the reverse of the start sequence.  It isn't.
 144
 145 <p>Reordering is hard and require cooperation between maintainers of
 146 all packages involved.  Given two packages with two scripts inserted
 147 with the default settings in Debian:
 148
 149 <p>Package A: script_a sequence 20 (start and stop)
 150 <br>Package B: script_b sequence 20 (start and stop)
 151
 152 <p>Along come script C, which should run before script_a and after
 153 script_b.  Current solution is to change packages A and C or packages
 154 B and C to get something like this:
 155
 156 <p>Package A: script_a start seq. 22, stop seq. 18
 157 <br>Package B: script_b sequence 20 (start and stop)
 158 <br>Package C: script_c start seq 21, stop seq 19
 159
 160 <p>If other scripts depend on the old order of script_a, they will
 161 have to change their sequence number too.  Only way to discover this
 162 is by a lot of testing, or documenting script dependencies.
 163
 164 <h2>A ordering solution</h2>
 165
 166 <p>Let each script document its dependency, and generate sequence
 167 numbers using this dependency information.  Example:
 168
 169 <p>Package A: script_a depend on nothing
 170 <br>Package B: script_b depend on nothing
 171 <br>Package C: script_c depend on script_b, a dependency of script_a
 172
 173 <p>Generated sequence:
 174
 175 <p>script_b start seq 1, stop seq 3
 176 <br>script_c start seq 2, stop seq 2
 177 <br>script_a start seq 3, stop seq 1
 178
 179 <p>An implementation of this system is the dependency based boot
 180 sequencing, provided in the insserv package.</p>
 181
 182
 183 <h2>LSB headers for insserv</h2>
 184
 185 <ul>
 186   <li>"Provides" header, list the Facility/service provided by the
 187     script.  Do not list virtual facilities (like $time)</li>
 188
 189   <li>Runlevel entries (Default-Start and Default-Stop headers), list
 190     what runlevels to activate for this script
 191
 192   <li>Dependency entries (Required-Start, Required-Stop, Should-Start,
 193     Should-Stop, X-Start-Before, X-Stop-After), list the
 194     facilities/services needed by this script.  It will start after
 195     its start dependencies and stop before its stop dependencies.  The
 196     X-* entreis are reverse dependencies.  Required-* are hard
 197     dependencies (will install even if they are missing), while
 198     Should-* and X-* are soft dependencies (only taken into account if
 199     scripts providing them are present).</li>
 200
 201 </ul>
 202
 203 <h2>What to list as dependencies I</h2>
 204
 205 <p>If your package used the default update-rc.d settings before, this
 206 is the header for you:</p>
 207 <pre>
 208 ### BEGIN INIT INFO
 209 # Provides:          scriptname
 210 # Required-Start:    $remote_fs $syslog
 211 # Required-Stop:     $remote_fs $syslog
 212 # Default-Start:     2 3 4 5
 213 # Default-Stop:      0 1 6
 214 ### END INIT INFO
 215 </pre>
 216
 217 <p>$remote_fs is needed by all scripts using files in /usr/.  $syslog
 218 is needed only by scripts starting services logging to syslog.</p>
 219
 220 <h2>What to list as dependencies II</h2>
 221
 222 <p>In the common case, the start and stop dependencies are identical.
 223
 224 <p>Prefer virtual dependencies over specific dependencies
 225
 226 <h2>Virtual facilities</h2>
 227
 228 <p>Linux Software Base version 3.2 define these virtual facilities:
 229
 230 <dl>
 231
 232 <dt>$local_fs
 233 <dd>all local file systems are mounted.
 234
 235 <dt>$network
 236 <dd>basic networking support is available. Example: a server program
 237 could listen on a socket.
 238
 239 <dt>$portmap
 240 <dd>daemons providing SunRPC/ONCRPC portmapping service as defined in
 241 RFC 1833: Binding Protocols for ONC RPC Version 2 (if present) are
 242 running.
 243
 244 <dt>$remote_fs
 245 <dd>all remote file systems are available. In some configurations, file
 246 systems such as /usr may be remote. Many applications that require
 247 $local_fs will probably also require $remote_fs.
 248
 249 <dt>$time
 250 <dd>the system time has been set, for example by using a network-based
 251 time program such as ntp or rdate, or via the hardware Real Time
 252 Clock.
 253
 254 <dt>$syslog
 255 <dd>system logger is operational.
 256
 257 <dt>$named
 258 <dd>IP name-to-address translation, using the interfaces described in
 259 this specification, are available to the level the system normally
 260 provides them. Example: if a DNS query daemon normally provides this
 261 facility, then that daemon has been started.
 262
 263 </dl>
 264
 265 <p>All of these represent points in time during boot and shutdown.
 266
 267 <h2>Status of dependency based boot</h2>
 268
 269 <img alt="LSB header progress graph" src="lsb-header-progress.png" width="50%" align="right">
 270
 271 <p>Release goal for Debian Lenny.
 272
 273 <p>Packages with LSB header: 654 of 866 (76%)
 274 <br>Unsolved BTS reports: XXX
 275 <br>Packages without BTS reports: ~150
 276 <br>Last package projected fixed 2008-07-19 with the current rate
 277
 278 <p>Need better documentation
 279
 280 <p>Need debian policy updates
 281
 282 <p>Fix insserv bugs
 283
 284 <p>
 285
 286 <h2>References</h2>
 287
 288 http://wiki.debian.org/LSBInitScripts
 289 http://wiki.debian.org/LSBInitScripts/DependencyBasedBoot
 290
 291     <h2>Thank you very much</h2>
 292
 293     <h3>Questions?</h3>
 294
 295     <p><a href="http://www.hungry.com/~pere/mypapers/200802-bootsequence/200802-bootsequence.html">http://www.hungry.com/~pere/mypapers/200802-bootsequence/200802-bootsequence.html</a></p>
 296
 297   </body>
 298 </html>