+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr">
+ <head>
+ <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
+ <title>Petter Reinholdtsen: Automatically upgrading server firmware on Dell PowerEdge</title>
+ <link rel="stylesheet" type="text/css" media="screen" href="http://people.skolelinux.org/pere/blog/style.css" />
+ <link rel="stylesheet" type="text/css" media="screen" href="http://people.skolelinux.org/pere/blog/vim.css" />
+ </head>
+ <body>
+ <div class="title">
+ <h1>
+ <a href="http://people.skolelinux.org/pere/blog/">Petter Reinholdtsen</a>
+
+ </h1>
+
+</div>
+
+
+ <div class="entry">
+ <div class="title">Automatically upgrading server firmware on Dell PowerEdge</div>
+ <div class="date">21st November 2011</div>
+ <div class="body"><p>At work we have heaps of servers. I believe the total count is
+around 1000 at the moment. To be able to get help from the vendors
+when something go wrong, we want to keep the firmware on the servers
+up to date. If the firmware isn't the latest and greatest, the
+vendors typically refuse to start debugging any problems until the
+firmware is upgraded. So before every reboot, we want to upgrade the
+firmware, and we would really like everyone handling servers at the
+university to do this themselves when they plan to reboot a machine.
+For that to happen we at the unix server admin group need to provide
+the tools to do so.</p>
+
+<p>To make firmware upgrading easier, I am working on a script to
+fetch and install the latest firmware for the servers we got. Most of
+our hardware are from Dell and HP, so I have focused on these servers
+so far. This blog post is about the Dell part.</P>
+
+<p>On the Dell FTP site I was lucky enough to find
+<a href="ftp://ftp.us.dell.com/catalog/Catalog.xml.gz">an XML file</a>
+with firmware information for all 11th generation servers, listing
+which firmware should be used on a given model and where on the FTP
+site I can find it. Using a simple perl XML parser I can then
+download the shell scripts Dell provides to do firmware upgrades from
+within Linux and reboot when all the firmware is primed and ready to
+be activated on the first reboot.</p>
+
+<p>This is the Dell related fragment of the perl code I am working on.
+Are there anyone working on similar tools for firmware upgrading all
+servers at a site? Please get in touch and lets share resources.</p>
+
+<p><pre>
+#!/usr/bin/perl
+use strict;
+use warnings;
+use File::Temp qw(tempdir);
+BEGIN {
+ # Install needed RHEL packages if missing
+ my %rhelmodules = (
+ 'XML::Simple' => 'perl-XML-Simple',
+ );
+ for my $module (keys %rhelmodules) {
+ eval "use $module;";
+ if ($@) {
+ my $pkg = $rhelmodules{$module};
+ system("yum install -y $pkg");
+ eval "use $module;";
+ }
+ }
+}
+my $errorsto = 'pere@hungry.com';
+
+upgrade_dell();
+
+exit 0;
+
+sub run_firmware_script {
+ my ($opts, $script) = @_;
+ unless ($script) {
+ print STDERR "fail: missing script name\n";
+ exit 1
+ }
+ print STDERR "Running $script\n\n";
+
+ if (0 == system("sh $script $opts")) { # FIXME correct exit code handling
+ print STDERR "success: firmware script ran succcessfully\n";
+ } else {
+ print STDERR "fail: firmware script returned error\n";
+ }
+}
+
+sub run_firmware_scripts {
+ my ($opts, @dirs) = @_;
+ # Run firmware packages
+ for my $dir (@dirs) {
+ print STDERR "info: Running scripts in $dir\n";
+ opendir(my $dh, $dir) or die "Unable to open directory $dir: $!";
+ while (my $s = readdir $dh) {
+ next if $s =~ m/^\.\.?/;
+ run_firmware_script($opts, "$dir/$s");
+ }
+ closedir $dh;
+ }
+}
+
+sub download {
+ my $url = shift;
+ print STDERR "info: Downloading $url\n";
+ system("wget --quiet \"$url\"");
+}
+
+sub upgrade_dell {
+ my @dirs;
+ my $product = `dmidecode -s system-product-name`;
+ chomp $product;
+
+ if ($product =~ m/PowerEdge/) {
+
+ # on RHEL, these pacakges are needed by the firwmare upgrade scripts
+ system('yum install -y compat-libstdc++-33.i686 libstdc++.i686 libxml2.i686 procmail');
+
+ my $tmpdir = tempdir(
+ CLEANUP => 1
+ );
+ chdir($tmpdir);
+ fetch_dell_fw('catalog/Catalog.xml.gz');
+ system('gunzip Catalog.xml.gz');
+ my @paths = fetch_dell_fw_list('Catalog.xml');
+ # -q is quiet, disabling interactivity and reducing console output
+ my $fwopts = "-q";
+ if (@paths) {
+ for my $url (@paths) {
+ fetch_dell_fw($url);
+ }
+ run_firmware_scripts($fwopts, $tmpdir);
+ } else {
+ print STDERR "error: Unsupported Dell model '$product'.\n";
+ print STDERR "error: Please report to $errorsto.\n";
+ }
+ chdir('/');
+ } else {
+ print STDERR "error: Unsupported Dell model '$product'.\n";
+ print STDERR "error: Please report to $errorsto.\n";
+ }
+}
+
+sub fetch_dell_fw {
+ my $path = shift;
+ my $url = "ftp://ftp.us.dell.com/$path";
+ download($url);
+}
+
+# Using ftp://ftp.us.dell.com/catalog/Catalog.xml.gz, figure out which
+# firmware packages to download from Dell. Only work for Linux
+# machines and 11th generation Dell servers.
+sub fetch_dell_fw_list {
+ my $filename = shift;
+
+ my $product = `dmidecode -s system-product-name`;
+ chomp $product;
+ my ($mybrand, $mymodel) = split(/\s+/, $product);
+
+ print STDERR "Finding firmware bundles for $mybrand $mymodel\n";
+
+ my $xml = XMLin($filename);
+ my @paths;
+ for my $bundle (@{$xml->{SoftwareBundle}}) {
+ my $brand = $bundle->{TargetSystems}->{Brand}->{Display}->{content};
+ my $model = $bundle->{TargetSystems}->{Brand}->{Model}->{Display}->{content};
+ my $oscode;
+ if ("ARRAY" eq ref $bundle->{TargetOSes}->{OperatingSystem}) {
+ $oscode = $bundle->{TargetOSes}->{OperatingSystem}[0]->{osCode};
+ } else {
+ $oscode = $bundle->{TargetOSes}->{OperatingSystem}->{osCode};
+ }
+ if ($mybrand eq $brand && $mymodel eq $model && "LIN" eq $oscode)
+ {
+ @paths = map { $_->{path} } @{$bundle->{Contents}->{Package}};
+ }
+ }
+ for my $component (@{$xml->{SoftwareComponent}}) {
+ my $componenttype = $component->{ComponentType}->{value};
+
+ # Drop application packages, only firmware and BIOS
+ next if 'APAC' eq $componenttype;
+
+ my $cpath = $component->{path};
+ for my $path (@paths) {
+ if ($cpath =~ m%/$path$%) {
+ push(@paths, $cpath);
+ }
+ }
+ }
+ return @paths;
+}
+</pre>
+
+<p>The code is only tested on RedHat Enterprise Linux, but I suspect
+it could work on other platforms with some tweaking. Anyone know a
+index like Catalog.xml is available from HP for HP servers? At the
+moment I maintain a similar list manually and it is quickly getting
+outdated.</p>
+</div>
+
+ <div class="tags">Tags: <a href="http://people.skolelinux.org/pere/blog/tags/debian">debian</a>, <a href="http://people.skolelinux.org/pere/blog/tags/english">english</a>.</div>
+
+
+ </div>
+
+
+
+
+ <div id="sidebar">
+
+
+
+<h2>Archive</h2>
+<ul>
+
+<li>2011
+<ul>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/01/">January (16)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/02/">February (6)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/03/">March (6)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/04/">April (7)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/05/">May (3)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/06/">June (2)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/07/">July (7)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/08/">August (6)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/09/">September (4)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/10/">October (2)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2011/11/">November (3)</a></li>
+
+</ul></li>
+
+<li>2010
+<ul>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/01/">January (2)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/02/">February (1)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/03/">March (3)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/04/">April (3)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/05/">May (9)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/06/">June (14)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/07/">July (12)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/08/">August (13)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/09/">September (7)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/10/">October (9)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/11/">November (13)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2010/12/">December (12)</a></li>
+
+</ul></li>
+
+<li>2009
+<ul>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/01/">January (8)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/02/">February (8)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/03/">March (12)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/04/">April (10)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/05/">May (9)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/06/">June (3)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/07/">July (4)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/08/">August (3)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/09/">September (1)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/10/">October (2)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/11/">November (3)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2009/12/">December (3)</a></li>
+
+</ul></li>
+
+<li>2008
+<ul>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2008/11/">November (5)</a></li>
+
+<li><a href="http://people.skolelinux.org/pere/blog/archive/2008/12/">December (7)</a></li>
+
+</ul></li>
+
+</ul>
+
+
+
+<h2>Tags</h2>
+<ul>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/3d-printer">3d-printer (13)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/amiga">amiga (1)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/aros">aros (1)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/bitcoin">bitcoin (2)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/bootsystem">bootsystem (12)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/bsa">bsa (2)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/debian">debian (54)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/debian edu">debian edu (63)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/digistan">digistan (7)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/english">english (98)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/fiksgatami">fiksgatami (13)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/fildeling">fildeling (12)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/intervju">intervju (10)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/kart">kart (15)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/ldap">ldap (8)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/lenker">lenker (4)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/ltsp">ltsp (1)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/multimedia">multimedia (14)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/norsk">norsk (140)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/nuug">nuug (119)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/open311">open311 (2)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/opphavsrett">opphavsrett (24)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/personvern">personvern (46)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/reprap">reprap (11)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/rfid">rfid (2)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/robot">robot (4)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/rss">rss (1)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/sikkerhet">sikkerhet (23)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/sitesummary">sitesummary (3)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/standard">standard (24)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/stavekontroll">stavekontroll (1)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/stortinget">stortinget (2)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/surveillance">surveillance (9)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/valg">valg (6)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/video">video (22)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/vitenskap">vitenskap (1)</a></li>
+
+ <li><a href="http://people.skolelinux.org/pere/blog/tags/web">web (17)</a></li>
+
+</ul>
+
+
+ </div>
+ <p style="text-align: right">
+ Created by <a href="http://steve.org.uk/Software/chronicle">Chronicle v4.4</a>
+</p>
+
+ </body>
+</html>
projects / homepage.git / commitdiff