1 <?xml version=
"1.0" encoding=
"utf-8"?>
2 <rss version='
2.0' xmlns:lj='http://www.livejournal.org/rss/lj/
1.0/'
>
4 <title>Petter Reinholdtsen - Entries tagged raid
</title>
5 <description>Entries tagged raid
</description>
6 <link>http://people.skolelinux.org/pere/blog/
</link>
10 <title>Some notes on fault tolerant storage systems
</title>
11 <link>http://people.skolelinux.org/pere/blog/Some_notes_on_fault_tolerant_storage_systems.html
</link>
12 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/Some_notes_on_fault_tolerant_storage_systems.html
</guid>
13 <pubDate>Wed,
1 Nov
2017 15:
35:
00 +
0100</pubDate>
14 <description><p
>If you care about how fault tolerant your storage is, you might
15 find these articles and papers interesting. They have formed how I
16 think of when designing a storage system.
</p
>
20 <li
>USENIX :login;
<a
21 href=
"https://www.usenix.org/publications/login/summer2017/ganesan
">Redundancy
22 Does Not Imply Fault Tolerance. Analysis of Distributed Storage
23 Reactions to Single Errors and Corruptions
</a
> by Aishwarya Ganesan,
24 Ramnatthan Alagappan, Andrea C. Arpaci-Dusseau, and Remzi
25 H. Arpaci-Dusseau
</li
>
28 <a href=
"http://www.zdnet.com/article/why-raid-
5-stops-working-in-
2009/
">Why
29 RAID
5 stops working in
2009</a
> by Robin Harris
</li
>
32 <a href=
"http://www.zdnet.com/article/why-raid-
6-stops-working-in-
2019/
">Why
33 RAID
6 stops working in
2019</a
> by Robin Harris
</li
>
35 <li
>USENIX FAST
'07
36 <a href=
"http://research.google.com/archive/disk_failures.pdf
">Failure
37 Trends in a Large Disk Drive Population
</a
> by Eduardo Pinheiro,
38 Wolf-Dietrich Weber and Luiz AndreĢ Barroso
</li
>
40 <li
>USENIX ;login:
<a
41 href=
"https://www.usenix.org/system/files/login/articles/hughes12-
04.pdf
">Data
42 Integrity. Finding Truth in a World of Guesses and Lies
</a
> by Doug
45 <li
>USENIX FAST
'08
46 <a href=
"https://www.usenix.org/events/fast08/tech/full_papers/bairavasundaram/bairavasundaram_html/
">An
47 cAnalysis of Data Corruption in the Storage Stack
</a
> -
48 L. N. Bairavasundaram, G. R. Goodson, B. Schroeder, A. C.
49 Arpaci-Dusseau, and R. H. Arpaci-Dusseau
</li
>
51 <li
>USENIX FAST
'07 <a
52 href=
"https://www.usenix.org/legacy/events/fast07/tech/schroeder/schroeder_html/
">Disk
53 failures in the real world: what does an MTTF of
1,
000,
000 hours mean
54 to you?
</a
> by B. Schroeder and G. A. Gibson.
</li
>
56 <li
>USENIX ;login:
<a
57 href=
"https://www.usenix.org/events/fast08/tech/full_papers/jiang/jiang_html/
">Are
58 Disks the Dominant Contributor for Storage Failures? A Comprehensive
59 Study of Storage Subsystem Failure Characteristics
</a
> by Weihang
60 Jiang, Chongfeng Hu, Yuanyuan Zhou, and Arkady Kanevsky
</li
>
62 <li
>SIGMETRICS
2007
63 <a href=
"http://research.cs.wisc.edu/adsl/Publications/latent-sigmetrics07.pdf
">An
64 analysis of latent sector errors in disk drives
</a
> -
65 L. N. Bairavasundaram, G. R. Goodson, S. Pasupathy, and J. Schindler
</li
>
69 <p
>Several of these research papers are based on data collected from
70 hundred thousands or millions of disk, and their findings are eye
71 opening. The short story is simply do not implicitly trust RAID or
72 redundant storage systems. Details matter. And unfortunately there
73 are few options on Linux addressing all the identified issues. Both
74 ZFS and Btrfs are doing a fairly good job, but have legal and
75 practical issues on their own. I wonder how cluster file systems like
76 Ceph do in this regard.
</p
>
78 <p
>Just remember, in the end, it do not matter how redundant, or how
79 fault tolerant your storage is, if you do not continuously monitor its
80 status to detect and replace failed disks.
</p
>
85 <title>How to figure out which RAID disk to replace when it fail
</title>
86 <link>http://people.skolelinux.org/pere/blog/How_to_figure_out_which_RAID_disk_to_replace_when_it_fail.html
</link>
87 <guid isPermaLink=
"true">http://people.skolelinux.org/pere/blog/How_to_figure_out_which_RAID_disk_to_replace_when_it_fail.html
</guid>
88 <pubDate>Tue,
14 Feb
2012 21:
25:
00 +
0100</pubDate>
89 <description><p
>Once in a while my home server have disk problems. Thanks to Linux
90 Software RAID, I have not lost data yet (but
91 <a href=
"http://comments.gmane.org/gmane.linux.raid/
34532">I was
92 close
</a
> this summer :). But once a disk is starting to behave
93 funny, a practical problem present itself. How to get from the Linux
94 device name (like /dev/sdd) to something that can be used to identify
95 the disk when the computer is turned off? In my case I have SATA
96 disks with a unique ID printed on the label. All I need is a way to
97 figure out how to query the disk to get the ID out.
</p
>
99 <p
>After fumbling a bit, I
100 <a href=
"http://www.cyberciti.biz/faq/linux-getting-scsi-ide-harddisk-information/
">found
101 that hdparm -I
</a
> will report the disk serial number, which is
102 printed on the disk label. The following (almost) one-liner can be
103 used to look up the ID of all the failed disks:
</p
>
105 <blockquote
><pre
>
106 for d in $(cat /proc/mdstat |grep
'(F)
'|tr
' ' "\n
"|grep
'(F)
'|cut -d\[ -f1|sort -u);
108 printf
"Failed disk $d:
"
109 hdparm -I /dev/$d |grep
'Serial Num
'
111 </blockquote
></pre
>
113 <p
>Putting it here to make sure I do not have to search for it the
114 next time, and in case other find it useful.
</p
>
116 <p
>At the moment I have two failing disk. :(
</p
>
118 <blockquote
><pre
>
119 Failed disk sdd1: Serial Number: WD-WCASJ1860823
120 Failed disk sdd2: Serial Number: WD-WCASJ1860823
121 Failed disk sde2: Serial Number: WD-WCASJ1840589
122 </blockquote
></pre
>
124 <p
>The last time I had failing disks, I added the serial number on
125 labels I printed and stuck on the short sides of each disk, to be able
126 to figure out which disk to take out of the box without having to
127 remove each disk to look at the physical vendor label. The vendor
128 label is at the top of the disk, which is hidden when the disks are
129 mounted inside my box.
</p
>
131 <p
>I really wish the check_linux_raid Nagios plugin for checking Linux
133 <a href=
"http://packages.qa.debian.org/n/nagios-plugins.html
">nagios-plugins-standard
</a
>
134 debian package would look up this value automatically, as it would
135 make the plugin a lot more useful when my disks fail. At the moment
136 it only report a failure when there are no more spares left (it really
137 should warn as soon as a disk is failing), and it do not tell me which
138 disk(s) is failing when the RAID is running short on disks.
</p
>
projects / homepage.git / blob