Ok boys,

the last two nights I spent on an issue which seems soooooo weird to me, that I have to write about it. I don’t know if it’s because I’m, even after a year of Ruby development, still too much Java-minded and just don’t get the idea of Ruby’s class variables or if they are totally screwed up them self… Anyway, here is my story:

I tried to enhance the StupidBackground plugin a little. It should be possible to assign workers to a certain “runner”, so that it would be easy to have more than one background runner. Maybe one for your quick running tasks and one for the ones that take eons to finish.

Until tonight we stored the list of tasks a runner should run in a class variable of that runner. That worked out fairly good until I made the changes described above. After these changes the code runs somehow like this:

• Run “main runner”
• Check all Worker classes
• If a worker uses a different worker than the default one, start another worker and fork it away (via Daemonized)
• This forked away runner checks all workers again for the ones, using it
• After the fork the “main runner” continues with the worker checking and maybe forks away more runners, when needed.

That means there is a process which forks away more and more processes and there are class variables. To make a long story short: An awful idea! All the processes shared the class variables! Which led to a state where every runner ran every task. As I said before, it took me quite a while to debug this, but finally the guys from #ruby-de on irc.freenode.org bailed me out.

So if class variables are the problem, was is the solution? Instance variables! Pretty good idea, but what to do in a class context like:

1
2
3
4
5

  class Blah
    def self.foo
      #what to do here?
    end
  end

As I said before my mind is still polluted with Java so I forgot about an important fact at this point: Class is a full-fledged object. So there would be no problem with using instance variables in this context. Ok, you have to understand that those instance variables are not accessible as an instance variable in the instance context of this class but I think this is comprehensible.

But I’m still astonished that the following is possible:

1
2
3
4
5
6
7
8
9
10

  class Blah
    def self.foo
      @test_var ||= Array.new
      @test_var << @test_var.size
    end
  end
 
  p Blah.foo # [0]
  p Blah.foo # [0,1]
  p Blah.foo # [0,1,2]

This might be not new to you if you really know Ruby, but if you’re just slowly getting to it from some other language such concepts might be hard to remember when you need them. But I hope with this post I can jog your memory :) .

Kudos to the guys at #ruby-de for the nice and quick help!

Yours,

Thorben
FEtMab-Team

Bookmarks to:

Hi again,

the time has come to “release” our first Rails plugin! *wooooooooah*

It’s mission is really simple: Do things “in” a Rails application periodically, in the background. Ok, sounds like a perfect job for cronjobs? Yes, it is. But let me explain this:

We are just two guys, coding all day long. Server administration is not our favorite sport and we do not have a cool admin guy to keep things going. Due to that, we once had a cronjob running which we didn’t monitor as we should’ve. The end of the story is, that our job was not executed for 2 weeks until we recognized it.

So, cronjobs may be just perfect, but we really don’t want the extra pain of monitoring an extra tool and keep the crontables up to date with every deployment and so on. Our solution:

StupidBackground

Ohhh yes my dear, there are plenty of Ruby and/or Rails tools out there that do that background job thing very well: BackgrounDRb, BackgroundFu and much more cool tools. But while some of these tools just are too much for our needs (BackgrounDRb), the other tools, which are simple enough, do not cover exactly what we wanted to do (BackgroundFu): Periodically call some methods which calculates things and then stores them to the database.

So that’s when StupidBackground comes into play.

It’s sooo stupid, that even 5 year olds should immediately know how to use it. But just in case your mind is somehow different to that of a 5 year old, I’ll try to explain it for you anyway:

1. Install it

script/plugin install \ http://svn.fetmab.net/svn/plugins/stupid_background/tags/0.1.1

2. Code your worker

Just create a new class in “lib/workers” which inherits from “StupidBackground::Worker” like this:

1
2

  class FiveYearOldFighter < StupidBackground::Worker
  end

Now just add some methods to this. When these methods are called by StupidBackground they are in a complete Rails environment created by “script/runner“. So feel free to use any models or whatever classes you like to use within your application. Like this:

1
2
3
4
5
6
7
8
9

  class FiveYearOldFighter < StupidBackground::Worker
    def fight(quantity, victory_chant)
      puts victory_chant if FiveYearOld.find(
          :all, :limit => quantity, 
          :order => "badass_rank DESC").all? {|five_year_old|
              five_year_old.fight
          }
    end
  end

Last open question: How can we configure how often we would like to fight 5 year olds? It’s easy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

  class FiveYearOldFighter < StupidBackground::Worker
 
    #THE NEXT TWO LINES ARE THE MAGIC!
    call(:fight).every(1.hour).with(20, "What a fight!")
    call(:fight).every(10.minutes).with(5, "Eaaasy one!")
 
    def fight(quantity, victory_chant)
      puts victory_chant if FiveYearOld.find(
          :all, :limit => quantity, 
          :order => "badass_rank DESC").all? {|five_year_old|
              five_year_old.fight
          }
    end
  end

To stay trained we now fight five 5 year olds every ten minutes and every hour we’re in the mood to fight 20 of them at the same time! Cool.

3. Start the background magic

Just one last thing to do:

rake background:start

And if you fought enough 5 year olds:

rake background:stop

Oh, and if you’ve forgotten if you’re right now fighting some 5 year olds try this:

rake background:status

That’s it. Pretty simple, but it is enough for our needs. Maybe this plugin is of some use for you.

Behind the scenes

StupidBackground uses the Daemonize library from Travis Whitton, tiny but great! This fact also means, that StupidBackground will not work on Windows machines.

The background behavior is realized via a stupid script/runner call, it’s not high sophisticated but again: it fits our needs.

The execution of the tasks happens in an non sophisticated way, too: StupidBackground just determines how long it is until the next task needs to be run, then sleeps this periods and then runs the job.

So if you have one job which should run in 5 seconds and another job which should run in 6 seconds, the first job runs in 5 seconds and when it’s finished the second job runs. So the intervals which you specify in a worker are not accurate enough for brain surgery… You have to see them as rough intervals which works, you guessed it already: great for us.

Any worker is just instantiated once! So if you would like to share informations between two jobs in the same worker you easily can. But be aware of unattended side effects, when one method saves information in an instance variable and another job reads it, even if it should not! The simplest way to get around this is to outsource any logic from your worker and then write two workers which both use the outsourced code.

If you would like to overwrite the initializer of the worker ensure that your initialize method takes one argument and passes this to the super initializer:

1
2
3
4

  def initialize(method)
    #your stuff
    super(method)
  end

Somehow like this.

Summary

• Periodically, task execution in a Rails application, in the background.
• No gems needed.
• No extra YAML or any other configuration. Anything is configured within your workers.
• One instance per worker, NOT per job.
• No threading if two jobs should run at the same time! Just waiting for one to finish, then starting the other.

Ok, I hope this is of some help for some of you. Feel free to add comments.

Yours,

Thorben
FEtMab-Team

Bookmarks to:

Just a few days ago the confreaks guys made the Ruby Hoedown keynote What makes code beautiful? available through their site. Not all the other talks are available as well!

A great service and most likely there are some very interesting sessions. Sadly until now I had no time to watch anything but the keynote mentioned above. But I like this kind of philosophical conversation about code conventions, so I think at least this session if definitely worth seeing.

Yours,
Thorben
FEtMab-Team

Bookmarks to: