- <title>Debian init.d boot script example for rsyslog</title>
- <link>http://people.skolelinux.org/pere/blog/Debian_init_d_boot_script_example_for_rsyslog.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Debian_init_d_boot_script_example_for_rsyslog.html</guid>
- <pubDate>Sat, 2 Nov 2013 22:40:00 +0100</pubDate>
- <description><p>If one of the points of switching to a new init system in Debian is
-<a href="http://thomas.goirand.fr/blog/?p=147">to get rid of huge
-init.d scripts</a>, I doubt we need to switch away from sysvinit and
-init.d scripts at all. Here is an example init.d script, ie a rewrite
-of /etc/init.d/rsyslog:</p>
-
-<p><pre>
-#!/lib/init/init-d-script
-### BEGIN INIT INFO
-# Provides: rsyslog
-# Required-Start: $remote_fs $time
-# Required-Stop: umountnfs $time
-# X-Stop-After: sendsigs
-# Default-Start: 2 3 4 5
-# Default-Stop: 0 1 6
-# Short-Description: enhanced syslogd
-# Description: Rsyslog is an enhanced multi-threaded syslogd.
-# It is quite compatible to stock sysklogd and can be
-# used as a drop-in replacement.
-### END INIT INFO
-DESC="enhanced syslogd"
-DAEMON=/usr/sbin/rsyslogd
-</pre></p>
-
-<p>Pretty minimalistic to me... For the record, the original sysv-rc
-script was 137 lines, and the above is just 15 lines, most of it meta
-info/comments.</p>
-
-<p>How to do this, you ask? Well, one create a new script
-/lib/init/init-d-script looking something like this:
-
-<p><pre>
-#!/bin/sh
-
-# Define LSB log_* functions.
-# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
-# and status_of_proc is working.
-. /lib/lsb/init-functions
-
+ <title>S3QL, a locally mounted cloud file system - nice free software</title>
+ <link>http://people.skolelinux.org/pere/blog/S3QL__a_locally_mounted_cloud_file_system___nice_free_software.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/S3QL__a_locally_mounted_cloud_file_system___nice_free_software.html</guid>
+ <pubDate>Wed, 9 Apr 2014 11:30:00 +0200</pubDate>
+ <description><p>For a while now, I have been looking for a sensible offsite backup
+solution for use at home. My requirements are simple, it must be
+cheap and locally encrypted (in other words, I keep the encryption
+keys, the storage provider do not have access to my private files).
+One idea me and my friends had many years ago, before the cloud
+storage providers showed up, was to use Google mail as storage,
+writing a Linux block device storing blocks as emails in the mail
+service provided by Google, and thus get heaps of free space. On top
+of this one can add encryption, RAID and volume management to have
+lots of (fairly slow, I admit that) cheap and encrypted storage. But
+I never found time to implement such system. But the last few weeks I
+have looked at a system called
+<a href="https://bitbucket.org/nikratio/s3ql/">S3QL</a>, a locally
+mounted network backed file system with the features I need.</p>
+
+<p>S3QL is a fuse file system with a local cache and cloud storage,
+handling several different storage providers, any with Amazon S3,
+Google Drive or OpenStack API. There are heaps of such storage
+providers. S3QL can also use a local directory as storage, which
+combined with sshfs allow for file storage on any ssh server. S3QL
+include support for encryption, compression, de-duplication, snapshots
+and immutable file systems, allowing me to mount the remote storage as
+a local mount point, look at and use the files as if they were local,
+while the content is stored in the cloud as well. This allow me to
+have a backup that should survive fire. The file system can not be
+shared between several machines at the same time, as only one can
+mount it at the time, but any machine with the encryption key and
+access to the storage service can mount it if it is unmounted.</p>
+
+<p>It is simple to use. I'm using it on Debian Wheezy, where the
+package is included already. So to get started, run <tt>apt-get
+install s3ql</tt>. Next, pick a storage provider. I ended up picking
+Greenqloud, after reading their nice recipe on
+<a href="https://greenqloud.zendesk.com/entries/44611757-How-To-Use-S3QL-to-mount-a-StorageQloud-bucket-on-Debian-Wheezy">how
+to use S3QL with their Amazon S3 service</a>, because I trust the laws
+in Iceland more than those in USA when it come to keeping my personal
+data safe and private, and thus would rather spend money on a company
+in Iceland. Another nice recipe is available from the article
+<a href="http://www.admin-magazine.com/HPC/Articles/HPC-Cloud-Storage">S3QL
+Filesystem for HPC Storage</a> by Jeff Layton in the HPC section of
+Admin magazine. When the provider is picked, figure out how to get
+the API key needed to connect to the storage API. With Greencloud,
+the key did not show up until I had added payment details to my
+account.</p>
+
+<p>Armed with the API access details, it is time to create the file
+system. First, create a new bucket in the cloud. This bucket is the
+file system storage area. I picked a bucket name reflecting the
+machine that was going to store data there, but any name will do.
+I'll refer to it as <tt>bucket-name</tt> below. In addition, one need
+the API login and password, and a locally created password. Store it
+all in ~root/.s3ql/authinfo2 like this:
+
+<p><blockquote><pre>
+[s3c]
+storage-url: s3c://s.greenqloud.com:443/bucket-name
+backend-login: API-login
+backend-password: API-password
+fs-passphrase: local-password
+</pre></blockquote></p>
+
+<p>I create my local passphrase using <tt>pwget 50</tt> or similar,
+but any sensible way to create a fairly random password should do it.
+Armed with these details, it is now time to run mkfs, entering the API
+details and password to create it:</p>
+
+<p><blockquote><pre>
+# mkdir -m 700 /var/lib/s3ql-cache
+# mkfs.s3ql --cachedir /var/lib/s3ql-cache --authfile /root/.s3ql/authinfo2 \
+ --ssl s3c://s.greenqloud.com:443/bucket-name
+Enter backend login:
+Enter backend password:
+Before using S3QL, make sure to read the user's guide, especially
+the 'Important Rules to Avoid Loosing Data' section.
+Enter encryption password:
+Confirm encryption password:
+Generating random encryption key...
+Creating metadata tables...
+Dumping metadata...
+..objects..
+..blocks..
+..inodes..
+..inode_blocks..
+..symlink_targets..
+..names..
+..contents..
+..ext_attributes..
+Compressing and uploading metadata...
+Wrote 0.00 MB of compressed metadata.
+# </pre></blockquote></p>
+
+<p>The next step is mounting the file system to make the storage available.
+
+<p><blockquote><pre>
+# mount.s3ql --cachedir /var/lib/s3ql-cache --authfile /root/.s3ql/authinfo2 \
+ --ssl --allow-root s3c://s.greenqloud.com:443/bucket-name /s3ql
+Using 4 upload threads.
+Downloading and decompressing metadata...
+Reading metadata...
+..objects..
+..blocks..
+..inodes..
+..inode_blocks..
+..symlink_targets..
+..names..
+..contents..
+..ext_attributes..
+Mounting filesystem...
+# df -h /s3ql
+Filesystem Size Used Avail Use% Mounted on
+s3c://s.greenqloud.com:443/bucket-name 1.0T 0 1.0T 0% /s3ql