Table of Contents

An introduction to "init scripts"

I still remember how I used to fear that the server my first production Rails app on would need a reboot. Every time that thing booted there would be something that didn't come back up after booting. So I adopted the fine art of copying stuff from the Internet and putting it into /etc/init.d/ on my server; an art referred to by my friend Christian as "scraping chewing gum off the street and putting it in your mouth".

One of the problems of this approach to "systems administration" is that a lot of the stuff you find out there is really crappy. In fact, a lot of the "init scripts" shipping with most Linux distributions is really crappy. And do you know why most of those scripts are so bad? Because writing good classical init scripts is really hard. The good news is there's no reason to write those things anymore, but more about that later. First of all, let's have a look at how "init scripts" work.

What's with the quotes around "init scripts"?

You may have noticed that I'm putting quotes around the term "init scripts". There is no single thing called init scripts; what most people refer to when using the term is the System V init convention of using runlevels, and having processes started and stopped as the system enters each runlevel.

This article covers this specific variant of "init scripts". Let's get to it.

System V initialization

The notion of runlevels is simple, and very old. There's no standard for what each runlevel represents, except most distributions seem to agree on:

  • Runlevel 0 is halt
  • Runlevel 1 is single-user mode
  • Runlevel 6 is reboot

A System V computer will have a default runlevel, specified in /etc/inittab. So one of the first things your kernel does during boot is to start a very special process - init - which in turn starts all other processes you need to get your work done.

It's important to realize that there isn't a single program called "init" out there, there are several variants. In fact, the Linux kernel lets you specify a full path to whichever init you prefer to boot with, just like ls lets you pass parameters like a to display hidden files.

Assuming your /etc/inittab looks like this:

id:2:initdefault

This means you computer by default will use runlevel 2. This means that init will (indirectly, usually by running /etc/init.d/rc 2) look up all (executable) files in /etc/rc2.d/ with a name beginning with the letter S (for start), sort them by name and execute every one of them with a single argument: start. This means that S1foo will start before S2bar, or even S1gee. Apart from sorting by name, there's no communication between these processes. If you need process A to be started before process B you need to make sure it ends up before B when sorting by name.

Apart from having processes started when a runlevel is reached, you can specify that processes should be killed when entering the runlevel. The same rules apply, except you prefix your scripts with the letter "K" instead of "S", and they are invoked with the single argument stop instead of start.

You wouldn't normally put your scripts into /etc/rc2.d/, but rather symlink them there from another location. So if you have a script named "webserver" that you want to have running in runlevel 2 and stopped as the server reboots, you would normally write a script that can be called with the argument start or stop, put it into /etc/init.d/ (the convention), and create symlinks for it like so:

ln -s /etc/init.d/webserver /etc/rc2.d/S10webserver
ln -s /etc/init.d/webserver /etc/rc6.d/K10webserver

So far this is pretty basic, right? It gets worse. init's launcing of these processes is pretty much fire-and-forget. You need to parse the arguments; start and stop are mandatory, but you'll earn absolutely no respect in the community unless you at least try to respond to restart. You need to daemonize; double-fork, reopen standard in/out etc. - if you can't deal with this you shouldn't be writing "init scripts". You'll need PID files and deal with situations where a PID file exists, but the process doesn't. And you'll need to keep your program running, possibly restarting it after a crash.

No explicit dependencies

Furthermore, the dependencies between services/damons are implicit, and requires you to juggle around with the numbering of your files. One thing I've been running into quite often is that daemons requiring a MySQL connection to be available would be attempted started before the MySQL server is running.

Control flow duplication

Now would be a good time to have a look inside /etc/init.d/ on your computer. Every one of the files in there will at least have this:

# See how we were called.
case "$1" in
  start)
      do_start()
      ;;
  stop)
      do_stop()
      ;;
esac

Apart from the hideous bash syntax, any programmer should be uncomfortable with the level of duplication in these files. The good news is that there is absolutely no reason why you should be wasting your time on this stuff, as no serious operating system uses System V's init any more. Not even RedHat Enterprise Linux.

Upstart

Upstart replaced sysvinit in Ubuntu 6.10, aka Edgy Eft. That's right: Ubuntu hasn't shipped with sysvinit since 2006. And when all the crap you've been dropping into /etc/init.d/ on your Ubuntu servers still "works" after all these years, it's because Upstart is backwards-compatible with sysvinit.

Go ahead, have a look at the file /etc/init/rc.conf on an Ubuntu machine. These ~20 lines of code are what runs your /etc/init.d/rc script, which in turn starts your "init scripts".

Congratulations, you've just read an Upstart-based "init script". Notice how it looks more like configuration than scripting? Let's have a look at /etc/init/hostname.conf from a basic Ubuntu machine:

description     "set system hostname"

start on startup

task
exec hostname -b -F /etc/hostname

There's a description, statements on when it should start, the task keyword, and an exec statement. There's no command line parsing, no pid files, no daemonization, no sourcing of other shell scripts, there isn't even any mention of runlevels - just the word startup. It looks like what it is: configuration of a command that will be invoked with one of three arguments:

  • start
  • stop
  • restart

Upstart "init scripts" aren't shell scripts, so they cannot be run like /etc/init.d/ scripts. To manually start/stop/restart Upstart-based tasks, you use the initctl program with one several parameters:

initctl start webserver
initctl stop webserver
initctl restart webserver

or even

start webserver
stop webserver
restart webserver

To learn more about writing Upstart recipes, read man 5 init on an Upstart-based system. I guarantee you that after 30 minutes you'll be writing "init scripts" of better quality than anything you ever dropped into /etc/init.d/ on some poor computer.

Upstart is evented

Instead of relying on runlevels, Upstart focuses on events that happen on your system, and use these events in your start on and stop on definitions. You're free to add your own events, and emit them:

initctl emit i-just-farted

Any service that has a start on i-just-farted definition will start once you do this. Your operating system will emit its own events, probably more useful than this once, check their documentation for this.

Upstart doesn't make you daemonize

One of my favorite features of Upstart is that it doesn't require me to daemonize; rather it tries to keep my service running for me (if I ask it to). No double-forking, no PID files and no external process babysitting. Simply create a program with an event loop, and add it to your Upstart service description like this

description     "My service"

start on filesystem or runlevel [2345]
stop on runlevel [!2345]

respawn
exec /usr/bin/myservice

The last line contains the entire command to be run. If the contents of /usr/bin/myservice looks like this:

#!/usr/bin/env ruby

while true do
end

Upstart would be happy to run it for you.

Upstart keeps it running

Not only will Upstart run the process for you; if you look at this line:

1: respawn

If for some reason a program started by Upstart crashes, and you add the respawn instruction in your recipe, Upstart will re-launch it when it crashes. If you're on an Ubuntu machine right now, try it out.

Create the file /usr/bin/myservice and make it look like this

1: while true do
2: end

Make it executable

sudo chmod 0755 /usr/bin/myservice

Then create the file /etc/init/myservice.conf and make it look like this:

description     "My service"

start on filesystem or runlevel [2345]
stop on runlevel [!2345]

respawn
exec /usr/bin/myservice

Now, let Upstart know you created this service:

sudo initctl reload-configuration

And tell Upstart to run it:

sudo start myservice

Upstart will tell you what it just did:

myservice start/running, process 12386

Make a note of the PID reported by Upstart, and kill it:

sudo kill 12386

Then try running a ps -ef |grep myservice and you'll see a new process. Upstart did that for you.

What are you waiting for?

Upstart is in RedHat Enterprise Linux since version 6, and Ubuntu since 6.10. That all your servers are most likely running Upstart. Now would be a good time to have a look at what you've put in your /etc/init.d. man 5 init will give a usable introduction to how to write Upstart recipes.

I'll be writing a followup article to this article covering systemd, which is what Upstart should have been.

comments powered by Disqus