blog/Testing_if_a_file_system_can_be_used_for_home_directories___.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
   2         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html>
   4  <head>
   5   <title>Petter Reinholdtsen: Testing if a file system can be used for home directories...</title>
   6   <link rel="stylesheet" type="text/css" media="screen" href="http://people.skolelinux.org/pere/blog/style.css">
   7  </head>
   8  <body>
   9
  10  <div class="title">
  11   <h1>
  12        <a href="http://people.skolelinux.org/pere/blog/">Petter Reinholdtsen</a>
  13
  14   </h1>
  15
  16  </div>
  17
  18
  19  <div class="entry">
  20   <div class="title">Testing if a file system can be used for home directories...</div>
  21   <div class="date">2010-08-08 21:20</div>
  22   <div class="body">
  23 <p>A few years ago, I was involved in a project planning to use
  24 Windows file servers as home directory servers for Debian
  25 Edu/Skolelinux machines.  This was thought to be no problem, as the
  26 access would be through the SMB network file system protocol, and we
  27 knew other sites used SMB with unix and samba as the file server to
  28 mount home directories without any problems.  But, after months of
  29 struggling, we had to conclude that our goal was impossible.</p>
  30
  31 <p>The reason is simply that while SMB can be used for home
  32 directories when the file server is Samba running on Unix, this only
  33 work because of Samba have some extensions and the fact that the
  34 underlying file system is a unix file system.  When using a Windows
  35 file server, the underlying file system do not have POSIX semantics,
  36 and several programs will fail if the users home directory where they
  37 want to store their configuration lack POSIX semantics.</p>
  38
  39 <p>As part of this work, I wrote a small C program I want to share
  40 with you all, to replicate a few of the problematic applications (like
  41 OpenOffice.org and GCompris) and see if the file system was working as
  42 it should.  If you find yourself in spooky file system land, it might
  43 help you find your way out again.  This is the fs-test.c source:</p>
  44
  45 <pre>
  46 /*
  47  * Some tests to check the file system sematics.  Used to verify that
  48  * CIFS from a windows server do not work properly as a linux home
  49  * directory.
  50  * License: GPL v2 or later
  51  *
  52  * needs libsqlite3-dev and build-essential installed
  53  * compile with: gcc -Wall -lsqlite3 -DTEST_SQLITE fs-test.c -o fs-test
  54 */
  55
  56 #define _FILE_OFFSET_BITS 64
  57 #define _LARGEFILE_SOURCE 1
  58 #define _LARGEFILE64_SOURCE 1
  59
  60 #define _GNU_SOURCE /* for asprintf() */
  61
  62 #include &lt;errno.h>
  63 #include &lt;fcntl.h>
  64 #include &lt;stdio.h>
  65 #include &lt;string.h>
  66 #include &lt;stdlib.h>
  67 #include &lt;sys/file.h>
  68 #include &lt;sys/stat.h>
  69 #include &lt;sys/types.h>
  70 #include &lt;unistd.h>
  71
  72 #ifdef TEST_SQLITE
  73 /*
  74  * Test sqlite open, as done by gcompris require the libsqlite3-dev
  75  * package and linking with -lsqlite3.  A more low level test is
  76  * below.
  77  * See also &lt;URL: http://www.sqlite.org./faq.html#q5 >.
  78  */
  79 #include &lt;sqlite3.h>
  80 #define CREATE_TABLE_USERS                                              \
  81   "CREATE TABLE users (user_id INT UNIQUE, login TEXT, lastname TEXT, firstname TEXT, birthdate TEXT, class_id INT ); "
  82 int test_sqlite_open(void) {
  83   char *zErrMsg;
  84   char *name = "testsqlite.db";
  85   sqlite3 *db=NULL;
  86   unlink(name);
  87   int rc = sqlite3_open(name, &db);
  88   if( rc ){
  89     printf("error: sqlite open of %s failed: %s\n", name, sqlite3_errmsg(db));
  90     sqlite3_close(db);
  91     return -1;
  92   }
  93
  94   /* create tables */
  95   rc = sqlite3_exec(db,CREATE_TABLE_USERS, NULL,  0, &zErrMsg);
  96   if( rc != SQLITE_OK ){
  97     printf("error: sqlite table create failed: %s\n", zErrMsg);
  98     sqlite3_close(db);
  99     return -1;
 100   }
 101   printf("info: sqlite worked\n");
 102   sqlite3_close(db);
 103   return 0;
 104 }
 105 #endif /* TEST_SQLITE */
 106
 107 /*
 108  * Demonstrate locking issue found in gcompris using sqlite3.  This
 109  * work with ext3, but not with cifs server on Windows 2003.  This is
 110  * done in the sqlite3 library.
 111  * See also
 112  * &lt;URL:http://www.cygwin.com/ml/cygwin/2001-08/msg00854.html> and the
 113  * POSIX specification
 114  * &lt;URL:http://www.opengroup.org/onlinepubs/009695399/functions/fcntl.html>.
 115  */
 116 int test_gcompris_locking(void) {
 117   struct flock fl;
 118   char *name = "testsqlite.db";
 119   unlink(name);
 120   int fd = open(name, O_RDWR|O_CREAT|O_LARGEFILE, 0644);
 121   printf("info: testing fcntl locking\n");
 122
 123   fl.l_whence = SEEK_SET;
 124   fl.l_pid    = getpid();
 125   printf("  Read-locking 1 byte from 1073741824");
 126   fl.l_start  = 1073741824;
 127   fl.l_len    = 1;
 128   fl.l_type   = F_RDLCK;
 129   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 130
 131   printf("  Read-locking 510 byte from 1073741826");
 132   fl.l_start  = 1073741826;
 133   fl.l_len    = 510;
 134   fl.l_type   = F_RDLCK;
 135   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 136
 137   printf("  Unlocking 1 byte from 1073741824");
 138   fl.l_start  = 1073741824;
 139   fl.l_len    = 1;
 140   fl.l_type   = F_UNLCK;
 141   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 142
 143   printf("  Write-locking 1 byte from 1073741824");
 144   fl.l_start  = 1073741824;
 145   fl.l_len    = 1;
 146   fl.l_type   = F_WRLCK;
 147   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 148
 149   printf("  Write-locking 510 byte from 1073741826");
 150   fl.l_start  = 1073741826;
 151   fl.l_len    = 510;
 152   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 153
 154   printf("  Unlocking 2 byte from 1073741824");
 155   fl.l_start  = 1073741824;
 156   fl.l_len    = 2;
 157   fl.l_type   = F_UNLCK;
 158   if (0 != fcntl(fd, F_SETLK, &fl) ) printf(" - error!\n"); else printf("\n");
 159
 160   close(fd);
 161   return 0;
 162 }
 163
 164 /*
 165  * Test if permissions of freshly created directories allow entries
 166  * below them.  This was a problem with OpenOffice.org and gcompris.
 167  * Mounting with option 'sync' seem to solve this problem while
 168  * slowing down file operations.
 169  */
 170 int test_subdirectory_creation(void) {
 171 #define LEVELS 5
 172   char *path = strdup("test");
 173   char *dirs[LEVELS];
 174   int level;
 175   printf("info: testing subdirectory creation\n");
 176   for (level = 0; level &lt; LEVELS; level++) {
 177     char *newpath = NULL;
 178     if (-1 == mkdir(path, 0777)) {
 179       printf("  error: Unable to create directory '%s': %s\n",
 180              path, strerror(errno));
 181       break;
 182     }
 183     asprintf(&newpath, "%s/%s", path, "test");
 184     free(path);
 185     path = newpath;
 186   }
 187   return 0;
 188 }
 189
 190 /*
 191  * Test if symlinks can be created.  This was a problem detected with
 192  * KDE.
 193  */
 194 int test_symlinks(void) {
 195   printf("info: testing symlink creation\n");
 196   unlink("symlink");
 197   if (-1 == symlink("file", "symlink"))
 198     printf("  error: Unable to create symlink\n");
 199   return 0;
 200 }
 201
 202 int main(int argc, char **argv) {
 203   printf("Testing POSIX/Unix sematics on file system\n");
 204   test_symlinks();
 205   test_subdirectory_creation();
 206 #ifdef TEST_SQLITE
 207   test_sqlite_open();
 208 #endif /* TEST_SQLITE */
 209   test_gcompris_locking();
 210   return 0;
 211 }
 212 </pre>
 213
 214 <p>When everything is working, it should print something like
 215 this:</p>
 216
 217 <pre>
 218 Testing POSIX/Unix sematics on file system
 219 info: testing symlink creation
 220 info: testing subdirectory creation
 221 info: sqlite worked
 222 info: testing fcntl locking
 223   Read-locking 1 byte from 1073741824
 224   Read-locking 510 byte from 1073741826
 225   Unlocking 1 byte from 1073741824
 226   Write-locking 1 byte from 1073741824
 227   Write-locking 510 byte from 1073741826
 228   Unlocking 2 byte from 1073741824
 229 </pre>
 230
 231 <p>I do not remember the exact details of the problems we saw, but one
 232 of them was with locking, where if I remember correctly, POSIX allow a
 233 read-only lock to be upgraded to a read-write lock without unlocking
 234 the read-only lock (while Windows do not).  Another was a bug in the
 235 CIFS/SMB client implementation in the Linux kernel where directory
 236 meta information would be wrong for a fraction of a second, making
 237 OpenOffice.org fail to create its deep directory tree because it was
 238 not allowed to create files in its freshly created directory.</p>
 239
 240 <p>Anyway, here is a nice tool for your tool box, might you never need
 241 it. :)</p>
 242
 243 <p>Update 2010-08-27: Michael Gebetsroither report that he found the
 244 script so useful that he created a GIT repository and stored it in
 245 <a href="http://github.com/gebi/fs-test">http://github.com/gebi/fs-test</a>.</p>
 246 </div>
 247
 248   <div class="tags">Tags: <a href="http://people.skolelinux.org/pere/blog/tags/debian edu">debian edu</a>, <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>, <a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug</a>.</div>
 249
 250   </div>
 251
 252
 253
 254
 255
 256
 257 <div id="sidebar">
 258
 259 <h2>Archive</h2>
 260 <ul>
 261
 262 <li>2010
 263 <ul>
 264
 265 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/01/">January (2)</a></li>
 266
 267 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/02/">February (1)</a></li>
 268
 269 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/03/">March (3)</a></li>
 270
 271 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/04/">April (3)</a></li>
 272
 273 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/05/">May (9)</a></li>
 274
 275 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/06/">June (14)</a></li>
 276
 277 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/07/">July (12)</a></li>
 278
 279 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/08/">August (13)</a></li>
 280
 281 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/09/">September (7)</a></li>
 282
 283 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/10/">October (9)</a></li>
 284
 285 <li><a href="http://people.skolelinux.org/pere/blog/archive/2010/11/">November (12)</a></li>
 286
 287 </ul></li>
 288
 289 <li>2009
 290 <ul>
 291
 292 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/01/">January (8)</a></li>
 293
 294 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/02/">February (8)</a></li>
 295
 296 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/03/">March (12)</a></li>
 297
 298 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/04/">April (10)</a></li>
 299
 300 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/05/">May (9)</a></li>
 301
 302 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/06/">June (3)</a></li>
 303
 304 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/07/">July (4)</a></li>
 305
 306 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/08/">August (3)</a></li>
 307
 308 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/09/">September (1)</a></li>
 309
 310 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/10/">October (2)</a></li>
 311
 312 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/11/">November (3)</a></li>
 313
 314 <li><a href="http://people.skolelinux.org/pere/blog/archive/2009/12/">December (3)</a></li>
 315
 316 </ul></li>
 317
 318 <li>2008
 319 <ul>
 320
 321 <li><a href="http://people.skolelinux.org/pere/blog/archive/2008/11/">November (5)</a></li>
 322
 323 <li><a href="http://people.skolelinux.org/pere/blog/archive/2008/12/">December (7)</a></li>
 324
 325 </ul></li>
 326
 327 </ul>
 328
 329
 330
 331 <h2>Tags</h2>
 332 <ul>
 333
 334  <li><a href="http://people.skolelinux.org/pere/blog/tags/3d-printer">3d-printer (12)</a></li>
 335
 336  <li><a href="http://people.skolelinux.org/pere/blog/tags/amiga">amiga (1)</a></li>
 337
 338  <li><a href="http://people.skolelinux.org/pere/blog/tags/aros">aros (1)</a></li>
 339
 340  <li><a href="http://people.skolelinux.org/pere/blog/tags/bootsystem">bootsystem (10)</a></li>
 341
 342  <li><a href="http://people.skolelinux.org/pere/blog/tags/debian">debian (43)</a></li>
 343
 344  <li><a href="http://people.skolelinux.org/pere/blog/tags/debian edu">debian edu (51)</a></li>
 345
 346  <li><a href="http://people.skolelinux.org/pere/blog/tags/english">english (72)</a></li>
 347
 348  <li><a href="http://people.skolelinux.org/pere/blog/tags/fiksgatami">fiksgatami (1)</a></li>
 349
 350  <li><a href="http://people.skolelinux.org/pere/blog/tags/fildeling">fildeling (11)</a></li>
 351
 352  <li><a href="http://people.skolelinux.org/pere/blog/tags/kart">kart (5)</a></li>
 353
 354  <li><a href="http://people.skolelinux.org/pere/blog/tags/ldap">ldap (8)</a></li>
 355
 356  <li><a href="http://people.skolelinux.org/pere/blog/tags/lenker">lenker (4)</a></li>
 357
 358  <li><a href="http://people.skolelinux.org/pere/blog/tags/ltsp">ltsp (1)</a></li>
 359
 360  <li><a href="http://people.skolelinux.org/pere/blog/tags/multimedia">multimedia (11)</a></li>
 361
 362  <li><a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk (91)</a></li>
 363
 364  <li><a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug (113)</a></li>
 365
 366  <li><a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett (18)</a></li>
 367
 368  <li><a href="http://people.skolelinux.org/pere/blog/tags/personvern">personvern (26)</a></li>
 369
 370  <li><a href="http://people.skolelinux.org/pere/blog/tags/reprap">reprap (10)</a></li>
 371
 372  <li><a href="http://people.skolelinux.org/pere/blog/tags/robot">robot (4)</a></li>
 373
 374  <li><a href="http://people.skolelinux.org/pere/blog/tags/rss">rss (1)</a></li>
 375
 376  <li><a href="http://people.skolelinux.org/pere/blog/tags/sikkerhet">sikkerhet (19)</a></li>
 377
 378  <li><a href="http://people.skolelinux.org/pere/blog/tags/sitesummary">sitesummary (3)</a></li>
 379
 380  <li><a href="http://people.skolelinux.org/pere/blog/tags/standard">standard (16)</a></li>
 381
 382  <li><a href="http://people.skolelinux.org/pere/blog/tags/stavekontroll">stavekontroll (1)</a></li>
 383
 384  <li><a href="http://people.skolelinux.org/pere/blog/tags/video">video (16)</a></li>
 385
 386  <li><a href="http://people.skolelinux.org/pere/blog/tags/vitenskap">vitenskap (1)</a></li>
 387
 388  <li><a href="http://people.skolelinux.org/pere/blog/tags/web">web (14)</a></li>
 389
 390 </ul>
 391
 392 </div>
 393  </body>
 394 </html>