- <title>Full battery stats collector is now available in Debian</title>
- <link>http://people.skolelinux.org/pere/blog/Full_battery_stats_collector_is_now_available_in_Debian.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Full_battery_stats_collector_is_now_available_in_Debian.html</guid>
- <pubDate>Wed, 23 Mar 2016 22:10:00 +0100</pubDate>
- <description><p>Since this morning, the battery-stats package in Debian include an
-extended collector that will collect the complete battery history for
-later processing and graphing. The original collector store the
-battery level as percentage of last full level, while the new
-collector also record battery vendor, model, serial number, design
-full level, last full level and current battery level. This make it
-possible to predict the lifetime of the battery as well as visualise
-the energy flow when the battery is charging or discharging.</p>
-
-<p>The new tools are available in <tt>/usr/share/battery-stats/</tt>
-in the version 0.5.1 package in unstable. Get the new battery level graph
-and lifetime prediction by running:
-
-<p><pre>
-/usr/share/battery-stats/battery-stats-graph /var/log/battery-stats.csv
-</pre></p>
-
-<p>Or select the 'Battery Level Graph' from your application menu.</p>
-
-<p>The flow in/out of the battery can be seen by running (no menu
-entry yet):</p>
-
-<p><pre>
-/usr/share/battery-stats/battery-stats-graph-flow
-</pre></p>
-
-<p>I'm not quite happy with the way the data is visualised, at least
-when there are few data points. The graphs look a bit better with a
-few years of data.</p>
-
-<p>A while back one important feature I use in the battery stats
-collector broke in Debian. The scripts in
-<tt>/usr/lib/pm-utils/power.d/</tt> were no longer executed. I
-suspect it happened when Jessie started using systemd, but I do not
-know. The issue is reported as
-<a href="https://bugs.debian.org/818649">bug #818649</a> against
-pm-utils. I managed to work around it by adding an udev rule to call
-the collector script every time the power connector is connected and
-disconnected. With this fix in place it was finally time to make a
-new release of the package, and get it into Debian.</p>
-
-<p>If you are interested in how your laptop battery is doing, please
-check out the
-<a href="https://tracker.debian.org/pkg/battery-stats">battery-stats</a>
-in Debian unstable, or rebuild it on Jessie to get it working on
-Debian stable. :) The upstream source is available from
-<a href="https://github.com/petterreinholdtsen/battery-stats">github</a>.
-As always, patches are very welcome.</p>
-</description>
- </item>
-
- <item>
- <title>UsingQR - "Electronic" paper invoices using JSON and QR codes</title>
- <link>http://people.skolelinux.org/pere/blog/UsingQR____Electronic__paper_invoices_using_JSON_and_QR_codes.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/UsingQR____Electronic__paper_invoices_using_JSON_and_QR_codes.html</guid>
- <pubDate>Sat, 19 Mar 2016 09:40:00 +0100</pubDate>
- <description><p>Back in 2013 I proposed
-<a href="http://people.skolelinux.org/pere/blog/_Electronic__paper_invoices___using_vCard_in_a_QR_code.html">a
-way to make paper and PDF invoices easier to process electronically by
-adding a QR code with the key information about the invoice</a>. I
-suggested using vCard field definition, to get some standard format
-for name and address, but any format would work. I did not do
-anything about the proposal, but hoped someone one day would make
-something like it. It would make it possible to efficiently send
-machine readable invoices directly between seller and buyer.</p>
-
-<p>This was the background when I came across a proposal and
-specification from the web based accounting and invoicing supplier
-<a href="http://www.visma.com/">Visma</a> in Sweden called
-<a href="http://usingqr.com/">UsingQR</a>. Their PDF invoices contain
-a QR code with the key information of the invoice in JSON format.
-This is the typical content of a QR code following the UsingQR
-specification (based on a real world example, some numbers replaced to
-get a more bogus entry). I've reformatted the JSON to make it easier
-to read. Normally this is all on one long line:</p>
-
-<p><img src="http://people.skolelinux.org/pere/blog/images/2016-03-19-qr-invoice.png" align="right"><pre>
-{
- "vh":500.00,
- "vm":0,
- "vl":0,
- "uqr":1,
- "tp":1,
- "nme":"Din Leverandør",
- "cc":"NO",
- "cid":"997912345 MVA",
- "iref":"12300001",
- "idt":"20151022",
- "ddt":"20151105",
- "due":2500.0000,
- "cur":"NOK",
- "pt":"BBAN",
- "acc":"17202612345",
- "bc":"BIENNOK1",
- "adr":"0313 OSLO"
-}
-</pre></p>
-
-</p>The interpretation of the fields can be found in the
-<a href="http://usingqr.com/wp-content/uploads/2014/06/UsingQR_specification1.pdf">format
-specification</a> (revision 2 from june 2014). The format seem to
-have most of the information needed to handle accounting and payment
-of invoices, at least the fields I have needed so far here in
-Norway.</p>
-
-<p>Unfortunately, the site and document do not mention anything about
-the patent, trademark and copyright status of the format and the
-specification. Because of this, I asked the people behind it back in
-November to clarify. Ann-Christine Savlid (ann-christine.savlid (at)
-visma.com) replied that Visma had not applied for patent or trademark
-protection for this format, and that there were no copyright based
-usage limitations for the format. I urged her to make sure this was
-explicitly written on the web pages and in the specification, but
-unfortunately this has not happened yet. So I guess if there is
-submarine patents, hidden trademarks or a will to sue for copyright
-infringements, those starting to use the UsingQR format might be at
-risk, but if this happen there is some legal defense in the fact that
-the people behind the format claimed it was safe to do so. At least
-with patents, there is always
-<a href="http://www.paperspecs.com/paper-news/beware-the-qr-code-patent-trap/">a
-chance of getting sued...</a></p>
-
-<p>I also asked if they planned to maintain the format in an
-independent standard organization to give others more confidence that
-they would participate in the standardization process on equal terms
-with Visma, but they had no immediate plans for this. Their plan was
-to work with banks to try to get more users of the format, and
-evaluate the way forward if the format proved to be popular. I hope
-they conclude that using an open standard organisation like
-<a href="http://www.ietf.org/">IETF</a> is the correct place to
-maintain such specification.</p>
-
-<p><strong>Update 2016-03-20</strong>: Via Twitter I became aware of
-<a href="https://news.ycombinator.com/item?id=11319492">some comments
-about this blog post</a> that had several useful links and references to
-similar systems. In the Czech republic, the Czech Banking Association
-standard #26, with short name SPAYD, uses QR codes with payment
-information. More information is available from the Wikipedia page on
-<a href="https://en.wikipedia.org/wiki/Short_Payment_Descriptor">Short
-Payment Descriptor</a>. And in Germany, there is a system named
-<a href="http://www.bezahlcode.de/">BezahlCode</a>,
-(<a href="http://www.bezahlcode.de/wp-content/uploads/BezahlCode_TechDok.pdf">specification
-v1.8 2013-12-05 available as PDF</a>), which uses QR codes with
-URL-like formatting using "bank:" as the URI schema/protocol to
-provide the payment information. There is also the
-<a href="http://www.ferd-net.de/front_content.php?idcat=231">ZUGFeRD</a>
-file format that perhaps could be transfered using QR codes, but I am
-not sure if it is done already. Last, in Bolivia there are reports
-that tax information since november 2014 need to be printed in QR
-format on invoices. I have not been able to track down a
-specification for this format, because of my limited language skill
-sets.</p>
+ <title>Detecting NFS hangs on Linux without hanging yourself...</title>
+ <link>http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html</guid>
+ <pubDate>Thu, 9 Mar 2017 15:20:00 +0100</pubDate>
+ <description><p>Over the years, administrating thousand of NFS mounting linux
+computers at the time, I often needed a way to detect if the machine
+was experiencing NFS hang. If you try to use <tt>df</tt> or look at a
+file or directory affected by the hang, the process (and possibly the
+shell) will hang too. So you want to be able to detect this without
+risking the detection process getting stuck too. It has not been
+obvious how to do this. When the hang has lasted a while, it is
+possible to find messages like these in dmesg:</p>
+
+<p><blockquote>
+nfs: server nfsserver not responding, still trying
+<br>nfs: server nfsserver OK
+</blockquote></p>
+
+<p>It is hard to know if the hang is still going on, and it is hard to
+be sure looking in dmesg is going to work. If there are lots of other
+messages in dmesg the lines might have rotated out of site before they
+are noticed.</p>
+
+<p>While reading through the nfs client implementation in linux kernel
+code, I came across some statistics that seem to give a way to detect
+it. The om_timeouts sunrpc value in the kernel will increase every
+time the above log entry is inserted into dmesg. And after digging a
+bit further, I discovered that this value show up in
+/proc/self/mountstats on Linux.</p>
+
+<p>The mountstats content seem to be shared between files using the
+same file system context, so it is enough to check one of the
+mountstats files to get the state of the mount point for the machine.
+I assume this will not show lazy umounted NFS points, nor NFS mount
+points in a different process context (ie with a different filesystem
+view), but that does not worry me.</p>
+
+<p>The content for a NFS mount point look similar to this:</p>
+
+<p><blockquote><pre>
+[...]
+device /dev/mapper/Debian-var mounted on /var with fstype ext3
+device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
+ opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
+ age: 7863311
+ caps: caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
+ sec: flavor=1,pseudoflavor=1
+ events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0
+ bytes: 166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809
+ RPC iostats version: 1.0 p/v: 100003/3 (nfs)
+ xprt: tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
+ per-op statistics
+ NULL: 0 0 0 0 0 0 0 0
+ GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
+ SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
+ LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
+ ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
+ READLINK: 125 125 0 20472 18620 0 1112 1118
+ READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
+ WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
+ CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
+ MKDIR: 3680 3680 0 773980 993920 26 23990 24245
+ SYMLINK: 903 903 0 233428 245488 6 5865 5917
+ MKNOD: 80 80 0 20148 21760 0 299 304
+ REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
+ RMDIR: 3367 3367 0 645112 484848 22 5782 6002
+ RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
+ LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
+ READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
+ READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
+ FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
+ FSINFO: 2 2 0 232 328 0 1 1
+ PATHCONF: 1 1 0 116 140 0 0 0
+ COMMIT: 0 0 0 0 0 0 0 0
+
+device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
+[...]
+</pre></blockquote></p>
+
+<p>The key number to look at is the third number in the per-op list.
+It is the number of NFS timeouts experiences per file system
+operation. Here 22 write timeouts and 5 access timeouts. If these
+numbers are increasing, I believe the machine is experiencing NFS
+hang. Unfortunately the timeout value do not start to increase right
+away. The NFS operations need to time out first, and this can take a
+while. The exact timeout value depend on the setup. For example the
+defaults for TCP and UDP mount points are quite different, and the
+timeout value is affected by the soft, hard, timeo and retrans NFS
+mount options.</p>
+
+<p>The only way I have been able to get working on Debian and RedHat
+Enterprise Linux for getting the timeout count is to peek in /proc/.
+But according to
+<ahref="http://docs.oracle.com/cd/E19253-01/816-4555/netmonitor-12/index.html">Solaris
+10 System Administration Guide: Network Services</a>, the 'nfsstat -c'
+command can be used to get these timeout values. But this do not work
+on Linux, as far as I can tell. I
+<ahref="http://bugs.debian.org/857043">asked Debian about this</a>,
+but have not seen any replies yet.</p>
+
+<p>Is there a better way to figure out if a Linux NFS client is
+experiencing NFS hangs? Is there a way to detect which processes are
+affected? Is there a way to get the NFS mount going quickly once the
+network problem causing the NFS hang has been cleared? I would very
+much welcome some clues, as we regularly run into NFS hangs.</p>