blog/Some_notes_on_fault_tolerant_storage_systems.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
   2           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml" dir="ltr">
   4   <head>
   5     <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
   6     <title>Petter Reinholdtsen: Some notes on fault tolerant storage systems</title>
   7     <link rel="stylesheet" type="text/css" media="screen" href="http://people.skolelinux.org/pere/blog/style.css" />
   8     <link rel="stylesheet" type="text/css" media="screen" href="http://people.skolelinux.org/pere/blog/vim.css" />
   9
  10
  11   </head>
  12   <body>
  13     <div class="title">
  14  <h1>
  15      <a href="http://people.skolelinux.org/pere/blog/">Petter Reinholdtsen</a>
  16
  17  </h1>
  18
  19 </div>
  20
  21
  22     <div class="entry">
  23       <div class="title">Some notes on fault tolerant storage systems</div>
  24       <div class="date"> 1st November 2017</div>
  25       <div class="body"><p>If you care about how fault tolerant your storage is, you might
  26 find these articles and papers interesting.  They have formed how I
  27 think of when designing a storage system.</p>
  28
  29 <ul>
  30
  31 <li>USENIX :login; <a
  32 href="https://www.usenix.org/publications/login/summer2017/ganesan">Redundancy
  33 Does Not Imply Fault Tolerance.  Analysis of Distributed Storage
  34 Reactions to Single Errors and Corruptions</a> by Aishwarya Ganesan,
  35 Ramnatthan Alagappan, Andrea C. Arpaci-Dusseau, and Remzi
  36 H. Arpaci-Dusseau</li>
  37
  38 <li>ZDNet
  39 <a href="http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/">Why
  40 RAID 5 stops working in 2009</a> by Robin Harris</li>
  41
  42 <li>ZDNet
  43 <a href="http://www.zdnet.com/article/why-raid-6-stops-working-in-2019/">Why
  44 RAID 6 stops working in 2019</a> by Robin Harris</li>
  45
  46 <li>USENIX FAST'07
  47 <a href="http://research.google.com/archive/disk_failures.pdf">Failure
  48 Trends in a Large Disk Drive Population</a> by Eduardo Pinheiro,
  49 Wolf-Dietrich Weber and Luiz André Barroso</li>
  50
  51 <li>USENIX ;login: <a
  52 href="https://www.usenix.org/system/files/login/articles/hughes12-04.pdf">Data
  53 Integrity.  Finding Truth in a World of Guesses and Lies</a> by Doug
  54 Hughes</li>
  55
  56 <li>USENIX FAST'08
  57 <a href="https://www.usenix.org/events/fast08/tech/full_papers/bairavasundaram/bairavasundaram_html/">An
  58 Analysis of Data Corruption in the Storage Stack</a> by
  59 L. N. Bairavasundaram, G. R. Goodson, B. Schroeder, A. C.
  60 Arpaci-Dusseau, and R. H. Arpaci-Dusseau</li>
  61
  62 <li>USENIX FAST'07 <a
  63 href="https://www.usenix.org/legacy/events/fast07/tech/schroeder/schroeder_html/">Disk
  64 failures in the real world: what does an MTTF of 1,000,000 hours mean
  65 to you?</a> by B. Schroeder and G. A. Gibson.</li>
  66
  67 <li>USENIX ;login: <a
  68 href="https://www.usenix.org/events/fast08/tech/full_papers/jiang/jiang_html/">Are
  69 Disks the Dominant Contributor for Storage Failures?  A Comprehensive
  70 Study of Storage Subsystem Failure Characteristics</a> by Weihang
  71 Jiang, Chongfeng Hu, Yuanyuan Zhou, and Arkady Kanevsky</li>
  72
  73 <li>SIGMETRICS 2007
  74 <a href="http://research.cs.wisc.edu/adsl/Publications/latent-sigmetrics07.pdf">An
  75 analysis of latent sector errors in disk drives</a> by
  76 L. N. Bairavasundaram, G. R. Goodson, S. Pasupathy, and J. Schindler</li>
  77
  78 </ul>
  79
  80 <p>Several of these research papers are based on data collected from
  81 hundred thousands or millions of disk, and their findings are eye
  82 opening.  The short story is simply do not implicitly trust RAID or
  83 redundant storage systems.  Details matter.  And unfortunately there
  84 are few options on Linux addressing all the identified issues.  Both
  85 ZFS and Btrfs are doing a fairly good job, but have legal and
  86 practical issues on their own.  I wonder how cluster file systems like
  87 Ceph do in this regard.  After all, there is an old saying, you know
  88 you have a distributed system when the crash of a compyter you have
  89 never heard of stops you from getting any work done.  The same holds
  90 true if fault tolerance do not work.</p>
  91
  92 <p>Just remember, in the end, it do not matter how redundant, or how
  93 fault tolerant your storage is, if you do not continuously monitor its
  94 status to detect and replace failed disks.</p>
  95 </div>
  96
  97       <div class="tags">Tags: <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/raid">raid</a>, <a href="http://people.skolelinux.org/pere/blog/tags/sysadmin">sysadmin</a>.</div>
  98
  99
 100     </div>
 101
 102
 103
 104
 105     <div id="sidebar">
 106
 107
 108
 109 <h2>Archive</h2>
 110 <ul>
 111
 112 <li>2017
 113 <ul>
 114
 115 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/01/">January (4)</a></li>
 116
 117 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/02/">February (3)</a></li>
 118
 119 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/03/">March (5)</a></li>
 120
 121 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/04/">April (2)</a></li>
 122
 123 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/06/">June (5)</a></li>
 124
 125 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/07/">July (1)</a></li>
 126
 127 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/08/">August (1)</a></li>
 128
 129 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/09/">September (3)</a></li>
 130
 131 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/10/">October (5)</a></li>
 132
 133 <li><a href="http://people.skolelinux.org/pere/blog/archive/2017/11/">November (1)</a></li>
 134
 135 </ul></li>
 136
 137 <li>2016
 138 <ul>
 139
 140 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/01/">January (3)</a></li>
 141
 142 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/02/">February (2)</a></li>
 143
 144 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/03/">March (3)</a></li>
 145
 146 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/04/">April (8)</a></li>
 147
 148 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/05/">May (8)</a></li>
 149
 150 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/06/">June (2)</a></li>
 151
 152 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/07/">July (2)</a></li>
 153
 154 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/08/">August (5)</a></li>
 155
 156 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/09/">September (2)</a></li>
 157
 158 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/10/">October (3)</a></li>
 159
 160 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/11/">November (8)</a></li>
 161
 162 <li><a href="http://people.skolelinux.org/pere/blog/archive/2016/12/">December (5)</a></li>
 163
 164 </ul></li>
 165
 166 <li>2015
 167 <ul>
 168
 169 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/01/">January (7)</a></li>
 170
 171 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/02/">February (6)</a></li>
 172
 173 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/03/">March (1)</a></li>
 174
 175 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/04/">April (4)</a></li>
 176
 177 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/05/">May (3)</a></li>
 178
 179 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/06/">June (4)</a></li>
 180
 181 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/07/">July (6)</a></li>
 182
 183 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/08/">August (2)</a></li>
 184
 185 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/09/">September (2)</a></li>
 186
 187 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/10/">October (9)</a></li>
 188
 189 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/11/">November (6)</a></li>
 190
 191 <li><a href="http://people.skolelinux.org/pere/blog/archive/2015/12/">December (3)</a></li>
 192
 193 </ul></li>
 194
 195 <li>2014
 196 <ul>
 197
 198 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/01/">January (2)</a></li>
 199
 200 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/02/">February (3)</a></li>
 201
 202 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/03/">March (8)</a></li>
 203
 204 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/04/">April (7)</a></li>
 205
 206 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/05/">May (1)</a></li>
 207
 208 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/06/">June (2)</a></li>
 209
 210 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/07/">July (2)</a></li>
 211
 212 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/08/">August (2)</a></li>
 213
 214 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/09/">September (5)</a></li>
 215
 216 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/10/">October (6)</a></li>
 217
 218 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/11/">November (3)</a></li>
 219
 220 <li><a href="http://people.skolelinux.org/pere/blog/archive/2014/12/">December (5)</a></li>
 221
 222 </ul></li>
 223
 224 <li>2013
 225 <ul>
 226
 227 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/01/">January (11)</a></li>
 228
 229 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/02/">February (9)</a></li>
 230
 231 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/03/">March (9)</a></li>
 232
 233 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/04/">April (6)</a></li>
 234
 235 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/05/">May (9)</a></li>
 236
 237 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/06/">June (10)</a></li>
 238
 239 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/07/">July (7)</a></li>
 240
 241 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/08/">August (3)</a></li>
 242
 243 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/09/">September (5)</a></li>
 244
 245 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/10/">October (7)</a></li>
 246
 247 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/11/">November (9)</a></li>
 248
 249 <li><a href="http://people.skolelinux.org/pere/blog/archive/2013/12/">December (3)</a></li>
 250
 251 </ul></li>
 252
 253 <li>2012
 254 <ul>
 255
 256 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/01/">January (7)</a></li>
 257
 258 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/02/">February (10)</a></li>
 259
 260 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/03/">March (17)</a></li>
 261
 262 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/04/">April (12)</a></li>
 263
 264 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/05/">May (12)</a></li>
 265
 266 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/06/">June (20)</a></li>
 267
 268 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/07/">July (17)</a></li>
 269
 270 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/08/">August (6)</a></li>
 271
 272 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/09/">September (9)</a></li>
 273
 274 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/10/">October (17)</a></li>
 275
 276 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/11/">November (10)</a></li>
 277
 278 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/12/">December (7)</a></li>
 279
 280 </ul></li>
 281
 282 <li>2011
 283 <ul>
 284
 285 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/01/">January (16)</a></li>
 286
 287 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/02/">February (6)</a></li>
 288
 289 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/03/">March (6)</a></li>
 290
 291 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/04/">April (7)</a></li>
 292
 293 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/05/">May (3)</a></li>
 294
 295 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/06/">June (2)</a></li>
 296
 297 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/07/">July (7)</a></li>
 298
 299 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/08/">August (6)</a></li>
 300
 301 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/09/">September (4)</a></li>
 302
 303 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/10/">October (2)</a></li>
 304
 305 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/11/">November (3)</a></li>
 306
 307 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/12/">December (1)</a></li>
 308
 309 </ul></li>
 310
 311 <li>2010
 312 <ul>
 313
 314 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/01/">January (2)</a></li>
 315
 316 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/02/">February (1)</a></li>
 317
 318 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/03/">March (3)</a></li>
 319
 320 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/04/">April (3)</a></li>
 321
 322 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/05/">May (9)</a></li>
 323
 324 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/06/">June (14)</a></li>
 325
 326 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/07/">July (12)</a></li>
 327
 328 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/08/">August (13)</a></li>
 329
 330 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/09/">September (7)</a></li>
 331
 332 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/10/">October (9)</a></li>
 333
 334 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/11/">November (13)</a></li>
 335
 336 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/12/">December (12)</a></li>
 337
 338 </ul></li>
 339
 340 <li>2009
 341 <ul>
 342
 343 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/01/">January (8)</a></li>
 344
 345 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/02/">February (8)</a></li>
 346
 347 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/03/">March (12)</a></li>
 348
 349 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/04/">April (10)</a></li>
 350
 351 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/05/">May (9)</a></li>
 352
 353 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/06/">June (3)</a></li>
 354
 355 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/07/">July (4)</a></li>
 356
 357 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/08/">August (3)</a></li>
 358
 359 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/09/">September (1)</a></li>
 360
 361 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/10/">October (2)</a></li>
 362
 363 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/11/">November (3)</a></li>
 364
 365 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/12/">December (3)</a></li>
 366
 367 </ul></li>
 368
 369 <li>2008
 370 <ul>
 371
 372 <li><a href="http://people.skolelinux.org/pere/blog/archive/2008/11/">November (5)</a></li>
 373
 374 <li><a href="http://people.skolelinux.org/pere/blog/archive/2008/12/">December (7)</a></li>
 375
 376 </ul></li>
 377
 378 </ul>
 379
 380
 381
 382 <h2>Tags</h2>
 383 <ul>
 384
 385  <li><a href="http://people.skolelinux.org/pere/blog/tags/3d-printer">3d-printer (14)</a></li>
 386
 387  <li><a href="http://people.skolelinux.org/pere/blog/tags/amiga">amiga (1)</a></li>
 388
 389  <li><a href="http://people.skolelinux.org/pere/blog/tags/aros">aros (1)</a></li>
 390
 391  <li><a href="http://people.skolelinux.org/pere/blog/tags/bankid">bankid (4)</a></li>
 392
 393  <li><a href="http://people.skolelinux.org/pere/blog/tags/bitcoin">bitcoin (9)</a></li>
 394
 395  <li><a href="http://people.skolelinux.org/pere/blog/tags/bootsystem">bootsystem (16)</a></li>
 396
 397  <li><a href="http://people.skolelinux.org/pere/blog/tags/bsa">bsa (2)</a></li>
 398
 399  <li><a href="http://people.skolelinux.org/pere/blog/tags/chrpath">chrpath (2)</a></li>
 400
 401  <li><a href="http://people.skolelinux.org/pere/blog/tags/debian">debian (154)</a></li>
 402
 403  <li><a href="http://people.skolelinux.org/pere/blog/tags/debian edu">debian edu (158)</a></li>
 404
 405  <li><a href="http://people.skolelinux.org/pere/blog/tags/debian-handbook">debian-handbook (4)</a></li>
 406
 407  <li><a href="http://people.skolelinux.org/pere/blog/tags/digistan">digistan (10)</a></li>
 408
 409  <li><a href="http://people.skolelinux.org/pere/blog/tags/dld">dld (17)</a></li>
 410
 411  <li><a href="http://people.skolelinux.org/pere/blog/tags/docbook">docbook (24)</a></li>
 412
 413  <li><a href="http://people.skolelinux.org/pere/blog/tags/drivstoffpriser">drivstoffpriser (4)</a></li>
 414
 415  <li><a href="http://people.skolelinux.org/pere/blog/tags/english">english (358)</a></li>
 416
 417  <li><a href="http://people.skolelinux.org/pere/blog/tags/fiksgatami">fiksgatami (23)</a></li>
 418
 419  <li><a href="http://people.skolelinux.org/pere/blog/tags/fildeling">fildeling (12)</a></li>
 420
 421  <li><a href="http://people.skolelinux.org/pere/blog/tags/freeculture">freeculture (30)</a></li>
 422
 423  <li><a href="http://people.skolelinux.org/pere/blog/tags/freedombox">freedombox (9)</a></li>
 424
 425  <li><a href="http://people.skolelinux.org/pere/blog/tags/frikanalen">frikanalen (18)</a></li>
 426
 427  <li><a href="http://people.skolelinux.org/pere/blog/tags/h264">h264 (20)</a></li>
 428
 429  <li><a href="http://people.skolelinux.org/pere/blog/tags/intervju">intervju (42)</a></li>
 430
 431  <li><a href="http://people.skolelinux.org/pere/blog/tags/isenkram">isenkram (15)</a></li>
 432
 433  <li><a href="http://people.skolelinux.org/pere/blog/tags/kart">kart (20)</a></li>
 434
 435  <li><a href="http://people.skolelinux.org/pere/blog/tags/ldap">ldap (9)</a></li>
 436
 437  <li><a href="http://people.skolelinux.org/pere/blog/tags/lego">lego (4)</a></li>
 438
 439  <li><a href="http://people.skolelinux.org/pere/blog/tags/lenker">lenker (8)</a></li>
 440
 441  <li><a href="http://people.skolelinux.org/pere/blog/tags/lsdvd">lsdvd (2)</a></li>
 442
 443  <li><a href="http://people.skolelinux.org/pere/blog/tags/ltsp">ltsp (1)</a></li>
 444
 445  <li><a href="http://people.skolelinux.org/pere/blog/tags/mesh network">mesh network (8)</a></li>
 446
 447  <li><a href="http://people.skolelinux.org/pere/blog/tags/multimedia">multimedia (39)</a></li>
 448
 449  <li><a href="http://people.skolelinux.org/pere/blog/tags/nice free software">nice free software (9)</a></li>
 450
 451  <li><a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk (293)</a></li>
 452
 453  <li><a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug (189)</a></li>
 454
 455  <li><a href="http://people.skolelinux.org/pere/blog/tags/offentlig innsyn">offentlig innsyn (33)</a></li>
 456
 457  <li><a href="http://people.skolelinux.org/pere/blog/tags/open311">open311 (2)</a></li>
 458
 459  <li><a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett (65)</a></li>
 460
 461  <li><a href="http://people.skolelinux.org/pere/blog/tags/personvern">personvern (104)</a></li>
 462
 463  <li><a href="http://people.skolelinux.org/pere/blog/tags/raid">raid (2)</a></li>
 464
 465  <li><a href="http://people.skolelinux.org/pere/blog/tags/reactos">reactos (1)</a></li>
 466
 467  <li><a href="http://people.skolelinux.org/pere/blog/tags/reprap">reprap (11)</a></li>
 468
 469  <li><a href="http://people.skolelinux.org/pere/blog/tags/rfid">rfid (3)</a></li>
 470
 471  <li><a href="http://people.skolelinux.org/pere/blog/tags/robot">robot (10)</a></li>
 472
 473  <li><a href="http://people.skolelinux.org/pere/blog/tags/rss">rss (1)</a></li>
 474
 475  <li><a href="http://people.skolelinux.org/pere/blog/tags/ruter">ruter (5)</a></li>
 476
 477  <li><a href="http://people.skolelinux.org/pere/blog/tags/scraperwiki">scraperwiki (2)</a></li>
 478
 479  <li><a href="http://people.skolelinux.org/pere/blog/tags/sikkerhet">sikkerhet (53)</a></li>
 480
 481  <li><a href="http://people.skolelinux.org/pere/blog/tags/sitesummary">sitesummary (4)</a></li>
 482
 483  <li><a href="http://people.skolelinux.org/pere/blog/tags/skepsis">skepsis (5)</a></li>
 484
 485  <li><a href="http://people.skolelinux.org/pere/blog/tags/standard">standard (55)</a></li>
 486
 487  <li><a href="http://people.skolelinux.org/pere/blog/tags/stavekontroll">stavekontroll (6)</a></li>
 488
 489  <li><a href="http://people.skolelinux.org/pere/blog/tags/stortinget">stortinget (12)</a></li>
 490
 491  <li><a href="http://people.skolelinux.org/pere/blog/tags/surveillance">surveillance (52)</a></li>
 492
 493  <li><a href="http://people.skolelinux.org/pere/blog/tags/sysadmin">sysadmin (4)</a></li>
 494
 495  <li><a href="http://people.skolelinux.org/pere/blog/tags/usenix">usenix (2)</a></li>
 496
 497  <li><a href="http://people.skolelinux.org/pere/blog/tags/valg">valg (9)</a></li>
 498
 499  <li><a href="http://people.skolelinux.org/pere/blog/tags/video">video (59)</a></li>
 500
 501  <li><a href="http://people.skolelinux.org/pere/blog/tags/vitenskap">vitenskap (4)</a></li>
 502
 503  <li><a href="http://people.skolelinux.org/pere/blog/tags/web">web (40)</a></li>
 504
 505 </ul>
 506
 507
 508     </div>
 509     <p style="text-align: right">
 510  Created by <a href="http://steve.org.uk/Software/chronicle">Chronicle v4.6</a>
 511 </p>
 512
 513   </body>
 514 </html>