blog/Testing_if_a_file_system_can_be_used_for_home_directories___.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
   2           "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml" dir="ltr">
   4   <head>
   5     <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
   6     <title>Petter Reinholdtsen: Testing if a file system can be used for home directories...</title>
   7     <link rel="stylesheet" type="text/css" media="screen" href="http://people.skolelinux.org/pere/blog/style.css" />
   8     <link rel="stylesheet" type="text/css" media="screen" href="http://people.skolelinux.org/pere/blog/vim.css" />
   9   </head>
  10   <body>
  11     <div class="title">
  12  <h1>
  13      <a href="http://people.skolelinux.org/pere/blog/">Petter Reinholdtsen</a>
  14
  15  </h1>
  16
  17 </div>
  18
  19
  20     <div class="entry">
  21       <div class="title">Testing if a file system can be used for home directories...</div>
  22       <div class="date"> 8th August 2010</div>
  23       <div class="body"><p>A few years ago, I was involved in a project planning to use
  24 Windows file servers as home directory servers for Debian
  25 Edu/Skolelinux machines.  This was thought to be no problem, as the
  26 access would be through the SMB network file system protocol, and we
  27 knew other sites used SMB with unix and samba as the file server to
  28 mount home directories without any problems.  But, after months of
  29 struggling, we had to conclude that our goal was impossible.</p>
  30
  31 <p>The reason is simply that while SMB can be used for home
  32 directories when the file server is Samba running on Unix, this only
  33 work because of Samba have some extensions and the fact that the
  34 underlying file system is a unix file system.  When using a Windows
  35 file server, the underlying file system do not have POSIX semantics,
  36 and several programs will fail if the users home directory where they
  37 want to store their configuration lack POSIX semantics.</p>
  38
  39 <p>As part of this work, I wrote a small C program I want to share
  40 with you all, to replicate a few of the problematic applications (like
  41 OpenOffice.org and GCompris) and see if the file system was working as
  42 it should.  If you find yourself in spooky file system land, it might
  43 help you find your way out again.  This is the fs-test.c source:</p>
  44
  45 <pre>
  46 /*
  47  * Some tests to check the file system sematics.  Used to verify that
  48  * CIFS from a windows server do not work properly as a linux home
  49  * directory.
  50  * License: GPL v2 or later
  51  *
  52  * needs libsqlite3-dev and build-essential installed
  53  * compile with: gcc -Wall -lsqlite3 -DTEST_SQLITE fs-test.c -o fs-test
  54 */
  55
  56 #define _FILE_OFFSET_BITS 64
  57 #define _LARGEFILE_SOURCE 1
  58 #define _LARGEFILE64_SOURCE 1
  59
  60 #define _GNU_SOURCE /* for asprintf() */
  61
  62 #include &lt;errno.h>
  63 #include &lt;fcntl.h>
  64 #include &lt;stdio.h>
  65 #include &lt;string.h>
  66 #include &lt;stdlib.h>
  67 #include &lt;sys/file.h>
  68 #include &lt;sys/stat.h>
  69 #include &lt;sys/types.h>
  70 #include &lt;unistd.h>
  71
  72 #ifdef TEST_SQLITE
  73 /*
  74  * Test sqlite open, as done by gcompris require the libsqlite3-dev
  75  * package and linking with -lsqlite3.  A more low level test is
  76  * below.
  77  * See also &lt;URL: http://www.sqlite.org./faq.html#q5 >.
  78  */
  79 #include &lt;sqlite3.h>
  80 #define CREATE_TABLE_USERS                                              \
  81   "CREATE TABLE users (user_id INT UNIQUE, login TEXT, lastname TEXT, firstname TEXT, birthdate TEXT, class_id INT ); "
  82 int test_sqlite_open(void) {
  83   char *zErrMsg;
  84   char *name = "testsqlite.db";
  85   sqlite3 *db=NULL;
  86   unlink(name);
  87   int rc = sqlite3_open(name, &db);
  88   if( rc ){
  89     printf("error: sqlite open of %s failed: %s\n", name, sqlite3_errmsg(db));
  90     sqlite3_close(db);
  91     return -1;
  92   }
  93
  94   /* create tables */
  95   rc = sqlite3_exec(db,CREATE_TABLE_USERS, NULL,  0, &zErrMsg);
  96   if( rc != SQLITE_OK ){
  97     printf("error: sqlite table create failed: %s\n", zErrMsg);
  98     sqlite3_close(db);
  99     return -1;
 100   }
 101   printf("info: sqlite worked\n");
 102   sqlite3_close(db);
 103   return 0;
 104 }
 105 #endif /* TEST_SQLITE */
 106
 107 /*
 108  * Demonstrate locking issue found in gcompris using sqlite3.  This
 109  * work with ext3, but not with cifs server on Windows 2003.  This is
 110  * done in the sqlite3 library.
 111  * See also
 112  * &lt;URL:http://www.cygwin.com/ml/cygwin/2001-08/msg00854.html> and the
 113  * POSIX specification
 114  * &lt;URL:http://www.opengroup.org/onlinepubs/009695399/functions/fcntl.html>.
 115  */
 116 int test_gcompris_locking(void) {
 117   struct flock fl;
 118   char *name = "testsqlite.db";
 119   unlink(name);
 120   int fd = open(name, O_RDWR|O_CREAT|O_LARGEFILE, 0644);
 121   printf("info: testing fcntl locking\n");
 122
 123   fl.l_whence = SEEK_SET;
 124   fl.l_pid    = getpid();
 125   printf("  Read-locking 1 byte from 1073741824");
 126   fl.l_start  = 1073741824;
 127   fl.l_len    = 1;
 128   fl.l_type   = F_RDLCK;
 129   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 130
 131   printf("  Read-locking 510 byte from 1073741826");
 132   fl.l_start  = 1073741826;
 133   fl.l_len    = 510;
 134   fl.l_type   = F_RDLCK;
 135   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 136
 137   printf("  Unlocking 1 byte from 1073741824");
 138   fl.l_start  = 1073741824;
 139   fl.l_len    = 1;
 140   fl.l_type   = F_UNLCK;
 141   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 142
 143   printf("  Write-locking 1 byte from 1073741824");
 144   fl.l_start  = 1073741824;
 145   fl.l_len    = 1;
 146   fl.l_type   = F_WRLCK;
 147   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 148
 149   printf("  Write-locking 510 byte from 1073741826");
 150   fl.l_start  = 1073741826;
 151   fl.l_len    = 510;
 152   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 153
 154   printf("  Unlocking 2 byte from 1073741824");
 155   fl.l_start  = 1073741824;
 156   fl.l_len    = 2;
 157   fl.l_type   = F_UNLCK;
 158   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 159
 160   close(fd);
 161   return 0;
 162 }
 163
 164 /*
 165  * Test if permissions of freshly created directories allow entries
 166  * below them.  This was a problem with OpenOffice.org and gcompris.
 167  * Mounting with option 'sync' seem to solve this problem while
 168  * slowing down file operations.
 169  */
 170 int test_subdirectory_creation(void) {
 171 #define LEVELS 5
 172   char *path = strdup("test");
 173   char *dirs[LEVELS];
 174   int level;
 175   printf("info: testing subdirectory creation\n");
 176   for (level = 0; level &lt; LEVELS; level++) {
 177     char *newpath = NULL;
 178     if (-1 == mkdir(path, 0777)) {
 179       printf("  error: Unable to create directory '%s': %s\n",
 180              path, strerror(errno));
 181       break;
 182     }
 183     asprintf(&newpath, "%s/%s", path, "test");
 184     free(path);
 185     path = newpath;
 186   }
 187   return 0;
 188 }
 189
 190 /*
 191  * Test if symlinks can be created.  This was a problem detected with
 192  * KDE.
 193  */
 194 int test_symlinks(void) {
 195   printf("info: testing symlink creation\n");
 196   unlink("symlink");
 197   if (-1 == symlink("file", "symlink"))
 198     printf("  error: Unable to create symlink\n");
 199   return 0;
 200 }
 201
 202 int main(int argc, char **argv) {
 203   printf("Testing POSIX/Unix sematics on file system\n");
 204   test_symlinks();
 205   test_subdirectory_creation();
 206 #ifdef TEST_SQLITE
 207   test_sqlite_open();
 208 #endif /* TEST_SQLITE */
 209   test_gcompris_locking();
 210   return 0;
 211 }
 212 </pre>
 213
 214 <p>When everything is working, it should print something like
 215 this:</p>
 216
 217 <pre>
 218 Testing POSIX/Unix sematics on file system
 219 info: testing symlink creation
 220 info: testing subdirectory creation
 221 info: sqlite worked
 222 info: testing fcntl locking
 223   Read-locking 1 byte from 1073741824
 224   Read-locking 510 byte from 1073741826
 225   Unlocking 1 byte from 1073741824
 226   Write-locking 1 byte from 1073741824
 227   Write-locking 510 byte from 1073741826
 228   Unlocking 2 byte from 1073741824
 229 </pre>
 230
 231 <p>I do not remember the exact details of the problems we saw, but one
 232 of them was with locking, where if I remember correctly, POSIX allow a
 233 read-only lock to be upgraded to a read-write lock without unlocking
 234 the read-only lock (while Windows do not).  Another was a bug in the
 235 CIFS/SMB client implementation in the Linux kernel where directory
 236 meta information would be wrong for a fraction of a second, making
 237 OpenOffice.org fail to create its deep directory tree because it was
 238 not allowed to create files in its freshly created directory.</p>
 239
 240 <p>Anyway, here is a nice tool for your tool box, might you never need
 241 it. :)</p>
 242
 243 <p>Update 2010-08-27: Michael Gebetsroither report that he found the
 244 script so useful that he created a GIT repository and stored it in
 245 <a href="http://github.com/gebi/fs-test">http://github.com/gebi/fs-test</a>.</p>
 246 </div>
 247
 248       <div class="tags">Tags: <a href="http://people.skolelinux.org/pere/blog/tags/debian edu">debian edu</a>, <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug</a>.</div>
 249
 250
 251     </div>
 252
 253
 254
 255
 256     <div id="sidebar">
 257
 258
 259
 260 <h2>Archive</h2>
 261 <ul>
 262
 263 <li>2012
 264 <ul>
 265
 266 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/01/">January (7)</a></li>
 267
 268 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/02/">February (10)</a></li>
 269
 270 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/03/">March (17)</a></li>
 271
 272 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/04/">April (12)</a></li>
 273
 274 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/05/">May (12)</a></li>
 275
 276 <li><a href="http://people.skolelinux.org/pere/blog/archive/2012/06/">June (8)</a></li>
 277
 278 </ul></li>
 279
 280 <li>2011
 281 <ul>
 282
 283 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/01/">January (16)</a></li>
 284
 285 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/02/">February (6)</a></li>
 286
 287 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/03/">March (6)</a></li>
 288
 289 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/04/">April (7)</a></li>
 290
 291 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/05/">May (3)</a></li>
 292
 293 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/06/">June (2)</a></li>
 294
 295 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/07/">July (7)</a></li>
 296
 297 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/08/">August (6)</a></li>
 298
 299 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/09/">September (4)</a></li>
 300
 301 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/10/">October (2)</a></li>
 302
 303 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/11/">November (3)</a></li>
 304
 305 <li><a href="http://people.skolelinux.org/pere/blog/archive/2011/12/">December (1)</a></li>
 306
 307 </ul></li>
 308
 309 <li>2010
 310 <ul>
 311
 312 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/01/">January (2)</a></li>
 313
 314 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/02/">February (1)</a></li>
 315
 316 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/03/">March (3)</a></li>
 317
 318 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/04/">April (3)</a></li>
 319
 320 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/05/">May (9)</a></li>
 321
 322 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/06/">June (14)</a></li>
 323
 324 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/07/">July (12)</a></li>
 325
 326 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/08/">August (13)</a></li>
 327
 328 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/09/">September (7)</a></li>
 329
 330 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/10/">October (9)</a></li>
 331
 332 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/11/">November (13)</a></li>
 333
 334 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/12/">December (12)</a></li>
 335
 336 </ul></li>
 337
 338 <li>2009
 339 <ul>
 340
 341 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/01/">January (8)</a></li>
 342
 343 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/02/">February (8)</a></li>
 344
 345 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/03/">March (12)</a></li>
 346
 347 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/04/">April (10)</a></li>
 348
 349 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/05/">May (9)</a></li>
 350
 351 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/06/">June (3)</a></li>
 352
 353 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/07/">July (4)</a></li>
 354
 355 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/08/">August (3)</a></li>
 356
 357 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/09/">September (1)</a></li>
 358
 359 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/10/">October (2)</a></li>
 360
 361 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/11/">November (3)</a></li>
 362
 363 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/12/">December (3)</a></li>
 364
 365 </ul></li>
 366
 367 <li>2008
 368 <ul>
 369
 370 <li><a href="http://people.skolelinux.org/pere/blog/archive/2008/11/">November (5)</a></li>
 371
 372 <li><a href="http://people.skolelinux.org/pere/blog/archive/2008/12/">December (7)</a></li>
 373
 374 </ul></li>
 375
 376 </ul>
 377
 378
 379
 380 <h2>Tags</h2>
 381 <ul>
 382
 383  <li><a href="http://people.skolelinux.org/pere/blog/tags/3d-printer">3d-printer (13)</a></li>
 384
 385  <li><a href="http://people.skolelinux.org/pere/blog/tags/amiga">amiga (1)</a></li>
 386
 387  <li><a href="http://people.skolelinux.org/pere/blog/tags/aros">aros (1)</a></li>
 388
 389  <li><a href="http://people.skolelinux.org/pere/blog/tags/bitcoin">bitcoin (2)</a></li>
 390
 391  <li><a href="http://people.skolelinux.org/pere/blog/tags/bootsystem">bootsystem (12)</a></li>
 392
 393  <li><a href="http://people.skolelinux.org/pere/blog/tags/bsa">bsa (2)</a></li>
 394
 395  <li><a href="http://people.skolelinux.org/pere/blog/tags/debian">debian (54)</a></li>
 396
 397  <li><a href="http://people.skolelinux.org/pere/blog/tags/debian edu">debian edu (104)</a></li>
 398
 399  <li><a href="http://people.skolelinux.org/pere/blog/tags/digistan">digistan (8)</a></li>
 400
 401  <li><a href="http://people.skolelinux.org/pere/blog/tags/drivstoffpriser">drivstoffpriser (3)</a></li>
 402
 403  <li><a href="http://people.skolelinux.org/pere/blog/tags/english">english (135)</a></li>
 404
 405  <li><a href="http://people.skolelinux.org/pere/blog/tags/fiksgatami">fiksgatami (16)</a></li>
 406
 407  <li><a href="http://people.skolelinux.org/pere/blog/tags/fildeling">fildeling (12)</a></li>
 408
 409  <li><a href="http://people.skolelinux.org/pere/blog/tags/intervju">intervju (27)</a></li>
 410
 411  <li><a href="http://people.skolelinux.org/pere/blog/tags/kart">kart (16)</a></li>
 412
 413  <li><a href="http://people.skolelinux.org/pere/blog/tags/ldap">ldap (8)</a></li>
 414
 415  <li><a href="http://people.skolelinux.org/pere/blog/tags/lenker">lenker (4)</a></li>
 416
 417  <li><a href="http://people.skolelinux.org/pere/blog/tags/ltsp">ltsp (1)</a></li>
 418
 419  <li><a href="http://people.skolelinux.org/pere/blog/tags/multimedia">multimedia (16)</a></li>
 420
 421  <li><a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk (170)</a></li>
 422
 423  <li><a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug (130)</a></li>
 424
 425  <li><a href="http://people.skolelinux.org/pere/blog/tags/offentlig innsyn">offentlig innsyn (1)</a></li>
 426
 427  <li><a href="http://people.skolelinux.org/pere/blog/tags/open311">open311 (2)</a></li>
 428
 429  <li><a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett (25)</a></li>
 430
 431  <li><a href="http://people.skolelinux.org/pere/blog/tags/personvern">personvern (47)</a></li>
 432
 433  <li><a href="http://people.skolelinux.org/pere/blog/tags/raid">raid (1)</a></li>
 434
 435  <li><a href="http://people.skolelinux.org/pere/blog/tags/reprap">reprap (11)</a></li>
 436
 437  <li><a href="http://people.skolelinux.org/pere/blog/tags/rfid">rfid (2)</a></li>
 438
 439  <li><a href="http://people.skolelinux.org/pere/blog/tags/robot">robot (4)</a></li>
 440
 441  <li><a href="http://people.skolelinux.org/pere/blog/tags/rss">rss (1)</a></li>
 442
 443  <li><a href="http://people.skolelinux.org/pere/blog/tags/ruter">ruter (4)</a></li>
 444
 445  <li><a href="http://people.skolelinux.org/pere/blog/tags/scraperwiki">scraperwiki (1)</a></li>
 446
 447  <li><a href="http://people.skolelinux.org/pere/blog/tags/sikkerhet">sikkerhet (23)</a></li>
 448
 449  <li><a href="http://people.skolelinux.org/pere/blog/tags/sitesummary">sitesummary (4)</a></li>
 450
 451  <li><a href="http://people.skolelinux.org/pere/blog/tags/standard">standard (29)</a></li>
 452
 453  <li><a href="http://people.skolelinux.org/pere/blog/tags/stavekontroll">stavekontroll (1)</a></li>
 454
 455  <li><a href="http://people.skolelinux.org/pere/blog/tags/stortinget">stortinget (4)</a></li>
 456
 457  <li><a href="http://people.skolelinux.org/pere/blog/tags/surveillance">surveillance (10)</a></li>
 458
 459  <li><a href="http://people.skolelinux.org/pere/blog/tags/valg">valg (6)</a></li>
 460
 461  <li><a href="http://people.skolelinux.org/pere/blog/tags/video">video (25)</a></li>
 462
 463  <li><a href="http://people.skolelinux.org/pere/blog/tags/vitenskap">vitenskap (1)</a></li>
 464
 465  <li><a href="http://people.skolelinux.org/pere/blog/tags/web">web (20)</a></li>
 466
 467 </ul>
 468
 469
 470     </div>
 471     <p style="text-align: right">
 472  Created by <a href="http://steve.org.uk/Software/chronicle">Chronicle v4.4</a>
 473 </p>
 474
 475   </body>
 476 </html>