hobodave.com

home about
Subscribe to RSS Feed

EPEL primary.sqlite.bz2 404 Not Found errors10 Feb 2011

I recently had an epic battle with yum and the EPEL repository. I wasted enough time on it that I think it will be useful if others experience similar problems.

The Problem

I recently installed a new VM with [CentOS 5.5 x86_64]. Like usual I add the EPEL and IUS Community repositories so that I can have access to git 1.7, PHP 5.3, MySQL 5.1, and Python 2.6 among other things. Everything was fine for 2 days until I went to install a package today. A simple yum install php53u-gd resulted in a wall of 404 error messages when it tried to fetch some cryptic bzipped file:

http://nas1.itc.virginia.edu/fedora-epel/5/x86_64/repodata/bd1da...-primary.sqlite.bz2: 
 [Errno 14] HTTP Error 404: Not Found
Trying other mirror.
Error: failure: 
repodata/bd1dac3a3d6ad62385741a9d50273ec1b2bcbd76-primary.sqlite.bz2 from
epel: [Errno 256] No more mirrors to try.

I tried repeatedly to clean the yum metadata and cache, even yum clean all a few times. The problem persisted.

Troubleshooting

My first step was to actually look at one of the mirrors returning 404 and verify that the file truly wasn’t there. Sure enough, it wasn’t. I also noted that they had recently been updated on 09-Feb-2011.

centos mirror repodata

I checked several other mirrors that were 404’ing and saw the same thing. Where was my machine fetching this information from? I asked around on IRC (#epel on freenode) and “nirik” suggested I try URLGRABBER_DEBUG=1 yum update which gave me the following output:

INFO:urlgrabber:attempt 1/10: http://mirrors.servercentral.net/fedora/epel/5/x86_64/repodata/repomd.xml
INFO:urlgrabber:creating new connection to mirrors.servercentral.net (405445232)
INFO:urlgrabber:STATUS: 200, OK
INFO:urlgrabber:success

Navigating to the ServerCentral EPEL mirror showed me the problem.

servercentral repodata

The bd1dac3a3d6ad62385741a9d50273ec1b2bcbd76-primary.sqlite.bz2 was not here either, but what really caught my eye was that the repo hadn’t been updated since 27-Jan-2011.

YUM keeps a cache of the EPEL RPM database and assorted metadata in /var/cache/yum/epel. Checking this location on my system showed that I had the bd1dac3a3d6ad62385741a9d50273ec1b2bcbd76-primary.sqlite.bz2 file present. I also checked the contents of repomd.xml and found the root cause:

<data type="primary_db"> 
  <location href="repodata/bd1dac3a3d6ad62385741a9d50273ec1b2bcbd76-primary.sqlite.bz2"/> 
  <checksum type="sha">bd1dac3a3d6ad62385741a9d50273ec1b2bcbd76</checksum> 
  <timestamp>1296232850</timestamp> 
  <size>3646557</size> 
  <open-size>15332352</open-size> 
  <open-checksum type="sha">b75a93c694f8cbfacca64e4d145a03595775f785</open-checksum> 
  <database_version>10</database_version> 
</data>

Aha! Not only was the mirror out of date, but the repomd.xml was out of sync with the mirror itself.

The Solution

CentOS is configured to use the fastestmirror yum plugin by default. My machine had an affinity for the ServerCentral mirror because it was the fastest. To fix this I simply edit /etc/yum/pluginconf.d/fastestmirror.conf adding an exclude as shown:

[main]
enabled=1
verbose=0
socket_timeout=3
hostfilepath=/var/cache/yum/timedhosts.txt
maxhostfileage=10
maxthreads=15
exclude=servercentral.net

Follow this up with a simple yum clean dbcache metadata and you’re golden!

bundle-phu - minify, bundle, and compress your js/css in Zend Framework17 Jan 2010

I’ve used a few different CSS/JS bundlers, but none have ever fulfilled all that I needed. Specifically, I wanted one that could do all of the following:

Thus, I created bundle-phu. Bundle-phu is a set of Zend Framework view helpers that do all of the above.

Bundle-phu is inspired by, bundle-fu a Ruby on Rails equivalent.

Before

<script type="text/javascript" src="/js/jquery.js"></script>
<script type="text/javascript" src="/js/foo.js"></script>
<script type="text/javascript" src="/js/bar.js"></script>
<script type="text/javascript" src="/js/baz.js"></script>
<link media="screen" type="text/css" href="/css/jquery.css" />
<link media="screen" type="text/css" href="/css/foo.css" />
<link media="screen" type="text/css" href="/css/bar.css" />
<link media="screen" type="text/css" href="/css/baz.css" />

After

<script type="text/javascript" src="bundle_3f8ca8371a8203fcdd8a82.css?1234567890"></script>
<link type="text/css" src="bundle_3f8ca8371a8203fcdd8a82.css?1234567890"></script>

Highlights

Installation

  1. Place the BundlePhu directory somewhere in your include_path:

     your_project/
 |-- application
 |-- library
 |   `-- BundlePhu
 |-- public

  2. Add the BundlePhu view helpers to your view’s helper path, and configure the helpers:

<?php
class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
    protected function _initView()
    {
        $view = new Zend_View();
        $view->addHelperPath(
            PATH_PROJECT . '/library/BundlePhu/View/Helper',
            'BundlePhu_View_Helper'
        );

        $view->getHelper('BundleScript')
            ->setCacheDir(PATH_PROJECT . '/data/cache/js')
            ->setDocRoot(PATH_PROJECT . '/public')
            ->setUseMinify(true)
            ->setMinifyCommand('java -jar yuicompressor -o :filename')
            ->setUseGzip(true)
            ->setGzipLevel(9)
            ->setUrlPrefix('/javascripts');

        $view->getHelper('BundleLink')
            ->setCacheDir(PATH_PROJECT . '/data/cache/css')
            ->setDocRoot(PATH_PROJECT . '/public')
            ->setUrlPrefix('/stylesheets');

        $viewRenderer = Zend_Controller_Action_HelperBroker::getStaticHelper('viewRenderer');
        $viewRenderer->setView($view);
        return $view;
    }
}
  1. Ensure your CacheDir is writable by the user your web server runs as
  2. Using either an Alias (apache) or location/alias (nginx) map the UrlPrefix to CacheDir. You can also do this using a symlink from your /public directory. e.g. /public/javascripts -> /../data/cache/js

Usage

As both these helpers extend from the existing HeadScript and HeadLink helpers in Zend Framework, you can use them just as you do those.

<? $this->bundleScript()->offsetSetFile(00, $this->baseUrl('/js/jquery.js')) ?>
<? $this->bundleScript()->appendFile($this->baseUrl('/js/foo.js')) ?>

Simple Nagios probes in ruby10 Jan 2010

I needed to write a custom nagios probe a few weeks ago. I googled for existing solutions in Ruby, but surprisingly found none. A nagios probe can really be written in any language, it just has to return a single line of output and an exit code of 0 (OK), 1 (WARNING), 2 (CRITICAL), or 3 (UNKNOWN). I chose Ruby because of the syntactical simplicity, as well as the ease of bundling it as a gem using Gemcutter.

You can view the source here.

Installation

# gem install nagios-probe

Usage

Simply create a subclass of Nagios::Probe and define the following methods:

Example

class MyProbe < Nagios::Probe
  def check_crit
    true
  end

  def check_warn
    false
  end

  def crit_message
    "Things are bad"
  end

  def warn_message
    "Things aren't going well"
  end

  def ok_message
    "Nothing to see here"
  end
end

To use your probe you must wrap it in a begin/rescue block to catch any exceptions and accurately report the status to Nagios.

begin
  options = {} # constructor accepts a single optional param that is assigned to @opts
  probe = MyProbe.new(options)
  probe.run
rescue Exception => e
  puts "Unknown: " + e
  exit Nagios::UNKNOWN
end

puts probe.message
exit probe.retval

Copyright © 2010 David Abdemoulaie. All rights reserved. Hosted by GitHub and powered by Jekyll.