Procrastinate away!

I’ve just released version 0.2 of procrastinate, the library that is out to stop you from using threads ever again. Indeed, as has been observed elsewhere, threads in programming are evil and belong into limbo at least, if not downright hell.

Opinion to the side. While procrastination will not get a lot done, procrastinate will. You can use it to simply and directly distribute work to a few cores and then collect back the results, all in a few lines:

require 'procrastinate'
include Procrastinate

class Worker
  def fact(n)
    # Insert your factorial implementation here.
  end
end

scheduler = Scheduler.start(SpawnStrategy::Throttled.new(5))
worker = scheduler.create_proxy(Worker.new)

futures = 10.times.map do |i|
  [i, worker.fact(i)]
end

futures.each do |i, fact_i|
  puts "Factorial(#{i}) == #{fact_i.value}"
end

scheduler.shutdown

Get the code here. This uses at most 5 worker threads processes (Throttled.new(5)) to compute factorials behind the scenes and returns the results through an intricate system of pipes to the main process. All of this is hidden.

There are only two really important differences to calling fact(n) directly:

• Your methods return value must be able to pass through Marshal.dump and Marshal.load. No returning lambdas, at least not for now.
• You cannot share data with the rest of your code. Your methods will get executed in total isolation and any changes to the state of your objects made will just get lost.

And it is precisely that second point that I consider a feature, not a bug. This is real multiprocessing with no data sharing, making your functions as pure as fresh snow.

Status, When is this ready?

Procrastinate is where some of my sweat is invested right now; the library will probably evolve some more before I declare it done. It is today very useful for a number of things.

And just to drive this point home, I’ll show you another application of procrastination: System Administration! Here’s a tool I’ve been also hacking on lately: ndo. Just do a quick

gem install ndo

and you’re almost ready. For this to really work, you have to create a host set first. A host set is a file below ~/.ndo called anything you like. This file contains one line per host you want to execute a command on. Let’s say we want to talk to ‘barbie’ and ‘ken’ at the same time: I’ll call that set ~/.ndo/wonderland:

barbie
ken

With this, I can now see if either barbies or kens disk is full:

> ndo wonderland "df -h"
barbie {
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/sda1              74G  9.0G   61G  13% /
    none                   32G  532K   32G   1% /dev
    none                   32G     0   32G   0% /dev/shm
    none                   32G  224K   32G   1% /var/run
    none                   32G     0   32G   0% /var/lock
    none                   32G     0   32G   0% /lib/init/rw
    none                   74G  9.0G   61G  13% /var/lib/ureadahead/debugfs}

ken {
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/sda1              56G  7.1G   46G  14% /
    none                   32G  400K   32G   1% /dev
    none                   32G     0   32G   0% /dev/shm
    none                   32G  200K   32G   1% /var/run
    none                   32G     0   32G   0% /var/lock
    none                   32G     0   32G   0% /lib/init/rw
    none                   56G  7.1G   46G  14% /var/lib/ureadahead/debugfs
    /dev/sr0              3.9G  3.9G     0 100% /cdrom}

This will easily scale over hundreds of servers and give you a scriptable server infrastructure.

Summary

To drive this home: Start doing things with procrastinate now. Send me your ideas and sample applications. Together we’ll eliminate threading from the face of this earth.

And perhaps, execute ‘sudo rm -rf /’ on all of your servers today. Or not. Better not in fact.