puppet-rspec debugging

While introducing rspec-puppet into a big and grown puppet codebase at Jimdo we needed to debug stuff and get more verbose output while writing the first tests. As the interwebs aren’t very chatty about the topic, here for all the distressed googlers:

Configure debug output and a console logger in your test (or helper or somewhere):

it "should do stuff" do
  Puppet::Util::Log.level = :debug
  Puppet::Util::Log.newdestination(:console)
  should ...
end

hth :)

devopsdays in Hamburg

the last weekend i attended to devopsdays in hamburg. first thanks to patrick debois and marcel wegermann for doing a really great job of conference organisation. and thanks to the sponsors making the location, food and beer possible ;)

DevOps is a relatively young movement of people that think developers, operations and also QA have to work together instead of
creating departments and isles. It’s about communication and automation of software delivery processes.

For example Jez Humble had a talk on “Continuous Delivery“ (the book is must read) which is the brother of Continuous Integration: If you integrate continuous why not deploy it continuously. Rule 1: DONE MEANS RELEASED

How does that fit with Scrum? Scrum says: DONE MEANS “PASSES ACCEPTANCE TESTS” and will be released “somewhen” (well, yes, this is my very personal problem with scrum ;) )

So there were talks about and even a real-world example introduction of Kanban which IMHO perfectly fits the continuous delivery process. Check out this nice Scrum vs. Kanban minibook. In kanban you can visualize your value chain - and in software development a value is commonly created at the time you deployed / released your changes so it’s being used by your customers / users - so follows RULE 2: “ready for blahblub” is waste.

So we want our changes deployed continuously and automatically, but we do not want to press the red button until we are sure that production will not be destroyed. so we need unit tests, functional tests and so on. this is what jez’s book is all about. i also heard about nice tools like Vagrant, which allows you to simulate production env on every developer’s laptop with VirtualBox and integrating it into your configration management. Or firing up some instances for integration / functional testing etc (very short and incomplete explanation, yeah i know)

Ah configuration management. Didn’t get any? How many servers to manage? Does not matter. RULE 3: USE CONFIG MANAGEMENT OR DIE (more or less rule zero). Use puppet or chef. those were the most mentioned tools at the conf.

As mentioned, Devops is also about communication and evangelism. There was some thinking of a devops manifesto like the agile one.

Just another rule: I had some small discussion with Jez about feature branches. Feature branches and cont. integration do not well fit together. Rather use branch by abstraction.

This are some just my very own impressions and learnings of this weekend, there was even much more stuff. Hopefully videos will come online in the near future :)

Thanks all people involved. I really appreciate the devops movement. Feels like coming from heaven at the right time.

rdiff-backup: the easiest way to backup (and restore)

Today I finally managed to set up an incremental backup for my workstation. What do we need? Well, nothing more than rdiff-backup, an opensource command line tool with all the powers you need.

it just backups. and your last backup is always available 1:1 at the destination (no strange storage formats etc., just dirs and files). Diffs and metadata are stored separately. So if you want a
backup that does its job, is plain, is easy to restore, has no unneccessary features and “just works” than rdiff-backup is the right tool for you.

Here a small bash script I write to accomplish the mission:

#!/bin/bash  
P=/home/soenke  
DESTINATION=mydestionationserver.local  
INCLUDE="  
$P/Documents  
$P/.gnupg  
"  

echo "$INCLUDE" | rdiff-backup --include-filelist-stdin --exclude $P/ $P/ $DESTINATION::/home/soenke/backup  

In my case “mydestionationserver.local” is a local mediacenter server running Ubuntu and a SSH server. INCLUDE has one backup src per line. P is the prefix dir, in my case my homedir. As you can
see, I’m using a whitelist of dirs/files to be backupped. If you want your full homedir get saved, just use:

rdiff-backup /path/to/src <server>::<destination-dir>

Just check the examples to learn more.

PHP: SortingIterator

I just had the need for an Iterator that can sort itself by a user defined callback.

My special use case is that the DirectoryIterator of PHP does not sort the file list so it’s pretty random. But my program logic relies on files being sorted by filename.

So here’s the little class:

class SortingIterator implements IteratorAggregate
{

        private $iterator = null;

        public function __construct(Traversable $iterator, $callback)
        {
                if (!is_callable($callback)) {
                        throw new InvalidArgumentException('Given callback is not callable!');
                }

                $array = iterator_to_array($iterator);
                usort($array, $callback);
                $this->iterator = new ArrayIterator($array);
        }


        public function getIterator()
        {
                return $this->iterator;
        }
}

Basically it uses the functionality of the function “iterator_to_array()” to convert the iterator into an array and then let the array getting sorted by usort() and the user-defined callback.

Then the sorted array is wrapped into an ArrayIterator so it can be used with all Iterator functions, decorators and whatever again.

Usage example:

function mysort($a, $b)
{
        return $a->getPathname() > $b->getPathname();
}

$it = new SortingIterator(new RecursiveIteratorIterator(new RecursiveDirectoryIterator('/home/soenke/muell')), 'mysort');

foreach ($it as $f) {
        echo $f->getPathname() . "\n";
}

Without sorting Iterator:

/home/soenke/tests/test/a.txt
/home/soenke/tests/test/b.txt
/home/soenke/tests/test/ä.txt
/home/soenke/tests/test/aaa.txt
/home/soenke/tests/test/abc.txt
/home/soenke/tests/test/az.txt

And with Sorting-Iterator:

soenke@turingmachine:~/tests$ php dirit.php
/home/soenke/tests/test/a.txt
/home/soenke/tests/test/aaa.txt
/home/soenke/tests/test/abc.txt
/home/soenke/tests/test/az.txt
/home/soenke/tests/test/b.txt
/home/soenke/tests/test/ä.txt