- <title>Detecting NFS hangs on Linux without hanging yourself...</title>
- <link>http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html</link>
- <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Detecting_NFS_hangs_on_Linux_without_hanging_yourself___.html</guid>
- <pubDate>Thu, 9 Mar 2017 15:20:00 +0100</pubDate>
- <description><p>Over the years, administrating thousand of NFS mounting linux
-computers at the time, I often needed a way to detect if the machine
-was experiencing NFS hang. If you try to use <tt>df</tt> or look at a
-file or directory affected by the hang, the process (and possibly the
-shell) will hang too. So you want to be able to detect this without
-risking the detection process getting stuck too. It has not been
-obvious how to do this. When the hang has lasted a while, it is
-possible to find messages like these in dmesg:</p>
-
-<p><blockquote>
-nfs: server nfsserver not responding, still trying
-<br>nfs: server nfsserver OK
-</blockquote></p>
-
-<p>It is hard to know if the hang is still going on, and it is hard to
-be sure looking in dmesg is going to work. If there are lots of other
-messages in dmesg the lines might have rotated out of site before they
-are noticed.</p>
-
-<p>While reading through the nfs client implementation in linux kernel
-code, I came across some statistics that seem to give a way to detect
-it. The om_timeouts sunrpc value in the kernel will increase every
-time the above log entry is inserted into dmesg. And after digging a
-bit further, I discovered that this value show up in
-/proc/self/mountstats on Linux.</p>
-
-<p>The mountstats content seem to be shared between files using the
-same file system context, so it is enough to check one of the
-mountstats files to get the state of the mount point for the machine.
-I assume this will not show lazy umounted NFS points, nor NFS mount
-points in a different process context (ie with a different filesystem
-view), but that does not worry me.</p>
-
-<p>The content for a NFS mount point look similar to this:</p>
-
-<p><blockquote><pre>
-[...]
-device /dev/mapper/Debian-var mounted on /var with fstype ext3
-device nfsserver:/mnt/nfsserver/home0 mounted on /mnt/nfsserver/home0 with fstype nfs statvers=1.1
- opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=129.240.3.145,mountvers=3,mountport=4048,mountproto=udp,local_lock=all
- age: 7863311
- caps: caps=0x3fe7,wtmult=4096,dtsize=8192,bsize=0,namlen=255
- sec: flavor=1,pseudoflavor=1
- events: 61063112 732346265 1028140 35486205 16220064 8162542 761447191 71714012 37189 3891185 45561809 110486139 4850138 420353 15449177 296502 52736725 13523379 0 52182 9016896 1231 0 0 0 0 0
- bytes: 166253035039 219519120027 0 0 40783504807 185466229638 11677877 45561809
- RPC iostats version: 1.0 p/v: 100003/3 (nfs)
- xprt: tcp 925 1 6810 0 0 111505412 111480497 109 2672418560317 0 248 53869103 22481820
- per-op statistics
- NULL: 0 0 0 0 0 0 0 0
- GETATTR: 61063106 61063108 0 9621383060 6839064400 453650 77291321 78926132
- SETATTR: 463469 463470 0 92005440 66739536 63787 603235 687943
- LOOKUP: 17021657 17021657 0 3354097764 4013442928 57216 35125459 35566511
- ACCESS: 14281703 14290009 5 2318400592 1713803640 1709282 4865144 7130140
- READLINK: 125 125 0 20472 18620 0 1112 1118
- READ: 4214236 4214237 0 715608524 41328653212 89884 22622768 22806693
- WRITE: 8479010 8494376 22 187695798568 1356087148 178264904 51506907 231671771
- CREATE: 171708 171708 0 38084748 46702272 873 1041833 1050398
- MKDIR: 3680 3680 0 773980 993920 26 23990 24245
- SYMLINK: 903 903 0 233428 245488 6 5865 5917
- MKNOD: 80 80 0 20148 21760 0 299 304
- REMOVE: 429921 429921 0 79796004 61908192 3313 2710416 2741636
- RMDIR: 3367 3367 0 645112 484848 22 5782 6002
- RENAME: 466201 466201 0 130026184 121212260 7075 5935207 5961288
- LINK: 289155 289155 0 72775556 67083960 2199 2565060 2585579
- READDIR: 2933237 2933237 0 516506204 13973833412 10385 3190199 3297917
- READDIRPLUS: 1652839 1652839 0 298640972 6895997744 84735 14307895 14448937
- FSSTAT: 6144 6144 0 1010516 1032192 51 9654 10022
- FSINFO: 2 2 0 232 328 0 1 1
- PATHCONF: 1 1 0 116 140 0 0 0
- COMMIT: 0 0 0 0 0 0 0 0
-
-device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
-[...]
-</pre></blockquote></p>
-
-<p>The key number to look at is the third number in the per-op list.
-It is the number of NFS timeouts experiences per file system
-operation. Here 22 write timeouts and 5 access timeouts. If these
-numbers are increasing, I believe the machine is experiencing NFS
-hang. Unfortunately the timeout value do not start to increase right
-away. The NFS operations need to time out first, and this can take a
-while. The exact timeout value depend on the setup. For example the
-defaults for TCP and UDP mount points are quite different, and the
-timeout value is affected by the soft, hard, timeo and retrans NFS
-mount options.</p>
-
-<p>The only way I have been able to get working on Debian and RedHat
-Enterprise Linux for getting the timeout count is to peek in /proc/.
-But according to
-<ahref="http://docs.oracle.com/cd/E19253-01/816-4555/netmonitor-12/index.html">Solaris
-10 System Administration Guide: Network Services</a>, the 'nfsstat -c'
-command can be used to get these timeout values. But this do not work
-on Linux, as far as I can tell. I
-<ahref="http://bugs.debian.org/857043">asked Debian about this</a>,
-but have not seen any replies yet.</p>
-
-<p>Is there a better way to figure out if a Linux NFS client is
-experiencing NFS hangs? Is there a way to detect which processes are
-affected? Is there a way to get the NFS mount going quickly once the
-network problem causing the NFS hang has been cleared? I would very
-much welcome some clues, as we regularly run into NFS hangs.</p>
+ <title>Easier recipe to observe the cell phones around you</title>
+ <link>http://people.skolelinux.org/pere/blog/Easier_recipe_to_observe_the_cell_phones_around_you.html</link>
+ <guid isPermaLink="true">http://people.skolelinux.org/pere/blog/Easier_recipe_to_observe_the_cell_phones_around_you.html</guid>
+ <pubDate>Sun, 24 Sep 2017 08:30:00 +0200</pubDate>
+ <description><p>A little more than a month ago I wrote
+<a href="http://people.skolelinux.org/pere/blog/Simpler_recipe_on_how_to_make_a_simple__7_IMSI_Catcher_using_Debian.html">how
+to observe the SIM card ID (aka IMSI number) of mobile phones talking
+to nearby mobile phone base stations using Debian GNU/Linux and a
+cheap USB software defined radio</a>, and thus being able to pinpoint
+the location of people and equipment (like cars and trains) with an
+accuracy of a few kilometer. Since then we have worked to make the
+procedure even simpler, and it is now possible to do this without any
+manual frequency tuning and without building your own packages.</p>
+
+<p>The <a href="https://tracker.debian.org/pkg/gr-gsm">gr-gsm</a>
+package is now included in Debian testing and unstable, and the
+IMSI-catcher code no longer require root access to fetch and decode
+the GSM data collected using gr-gsm.</p>
+
+<p>Here is an updated recipe, using packages built by Debian and a git
+clone of two python scripts:</p>
+
+<ol>
+
+<li>Start with a Debian machine running the Buster version (aka
+ testing).</li>
+
+<li>Run '<tt>apt install gr-gsm python-numpy python-scipy
+ python-scapy</tt>' as root to install required packages.</li>
+
+<li>Fetch the code decoding GSM packages using '<tt>git clone
+ github.com/Oros42/IMSI-catcher.git</tt>'.</li>
+
+<li>Insert USB software defined radio supported by GNU Radio.</li>
+
+<li>Enter the IMSI-catcher directory and run '<tt>python
+ scan-and-livemon</tt>' to locate the frequency of nearby base
+ stations and start listening for GSM packages on one of them.</li>
+
+<li>Enter the IMSI-catcher directory and run '<tt>python
+ simple_IMSI-catcher.py</tt>' to display the collected information.</li>
+
+</ol>
+
+<p>Note, due to a bug somewhere the scan-and-livemon program (actually
+<a href="https://github.com/ptrkrysik/gr-gsm/issues/336">its underlying
+program grgsm_scanner</a>) do not work with the HackRF radio. It does
+work with RTL 8232 and other similar USB radio receivers you can get
+very cheaply
+(<a href="https://www.ebay.com/sch/items/?_nkw=rtl+2832">for example
+from ebay</a>), so for now the solution is to scan using the RTL radio
+and only use HackRF for fetching GSM data.</p>
+
+<p>As far as I can tell, a cell phone only show up on one of the
+frequencies at the time, so if you are going to track and count every
+cell phone around you, you need to listen to all the frequencies used.
+To listen to several frequencies, use the --numrecv argument to
+scan-and-livemon to use several receivers. Further, I am not sure if
+phones using 3G or 4G will show as talking GSM to base stations, so
+this approach might not see all phones around you. I typically see
+0-400 IMSI numbers an hour when looking around where I live.</p>
+
+<p>I've tried to run the scanner on a
+<a href="https://wiki.debian.org/RaspberryPi">Raspberry Pi 2 and 3
+running Debian Buster</a>, but the grgsm_livemon_headless process seem
+to be too CPU intensive to keep up. When GNU Radio print 'O' to
+stdout, I am told there it is caused by a buffer overflow between the
+radio and GNU Radio, caused by the program being unable to read the
+GSM data fast enough. If you see a stream of 'O's from the terminal
+where you started scan-and-livemon, you need a give the process more
+CPU power. Perhaps someone are able to optimize the code to a point
+where it become possible to set up RPi3 based GSM sniffers? I tried
+using Raspbian instead of Debian, but there seem to be something wrong
+with GNU Radio on raspbian, causing glibc to abort().</p>