I still remember how I used to fear that the server my first
production Rails app on would need a reboot. Every time that thing booted
there would be something that didn't come back up after booting. So
I adopted the fine art of copying stuff from the Internet and
putting it into /etc/init.d/
on my server; an art referred to by
my friend Christian as "scraping chewing gum off the street and
putting it in your mouth".
One of the problems of this approach to "systems administration" is that a lot of the stuff you find out there is really crappy. In fact, a lot of the "init scripts" shipping with most Linux distributions is really crappy. And do you know why most of those scripts are so bad? Because writing good classical init scripts is really hard. The good news is there's no reason to write those things anymore, but more about that later. First of all, let's have a look at how "init scripts" work.
You may have noticed that I'm putting quotes around the term "init scripts". There is no single thing called init scripts; what most people refer to when using the term is the System V init convention of using runlevels, and having processes started and stopped as the system enters each runlevel.
This article covers this specific variant of "init scripts". Let's get to it.
The notion of runlevels is simple, and very old. There's no standard for what each runlevel represents, except most distributions seem to agree on:
A System V computer will have a default runlevel, specified in
/etc/inittab
. So one of the first things your kernel does
during boot is to start a very special process - init - which in
turn starts all other processes you need to get your work
done.
It's important to realize that there isn't a single program called
"init" out there, there are several variants. In fact, the Linux
kernel lets you specify a full path to whichever init you prefer to
boot with, just like ls
lets you pass parameters like a
to
display hidden files.
Assuming your /etc/inittab
looks like this:
id:2:initdefault
This means you computer by default will use runlevel 2. This means
that init will (indirectly, usually by running /etc/init.d/rc 2
)
look up all (executable) files in /etc/rc2.d/
with a name
beginning with the letter S (for start), sort them by name and
execute every one of them with a single argument: start
. This
means that S1foo
will start before S2bar
, or even
S1gee
. Apart from sorting by name, there's no communication
between these processes. If you need process A to be started before
process B you need to make sure it ends up before B when sorting by
name.
Apart from having processes started when a runlevel is reached, you
can specify that processes should be killed when entering the
runlevel. The same rules apply, except you prefix your scripts with
the letter "K" instead of "S", and they are invoked with the single
argument stop
instead of start
.
You wouldn't normally put your scripts into /etc/rc2.d/
, but
rather symlink them there from another location. So if you have a
script named "webserver" that you want to have running in runlevel
2 and stopped as the server reboots, you would normally write a
script that can be called with the argument start
or stop
, put
it into /etc/init.d/
(the convention), and create symlinks for it
like so:
ln -s /etc/init.d/webserver /etc/rc2.d/S10webserver ln -s /etc/init.d/webserver /etc/rc6.d/K10webserver
So far this is pretty basic, right? It gets worse. init
's
launcing of these processes is pretty much fire-and-forget. You
need to parse the arguments; start
and stop
are mandatory, but
you'll earn absolutely no respect in the community unless you at
least try to respond to restart
. You need to daemonize;
double-fork, reopen standard in/out etc. - if you can't deal with
this you shouldn't be writing "init scripts". You'll need PID files
and deal with situations where a PID file exists, but the process
doesn't. And you'll need to keep your program running, possibly
restarting it after a crash.
Furthermore, the dependencies between services/damons are implicit, and requires you to juggle around with the numbering of your files. One thing I've been running into quite often is that daemons requiring a MySQL connection to be available would be attempted started before the MySQL server is running.
Now would be a good time to have a look inside /etc/init.d/
on
your computer. Every one of the files in there will at least have
this:
# See how we were called. case "$1" in start) do_start() ;; stop) do_stop() ;; esac
Apart from the hideous bash syntax, any programmer should be uncomfortable with the level of duplication in these files. The good news is that there is absolutely no reason why you should be wasting your time on this stuff, as no serious operating system uses System V's init any more. Not even RedHat Enterprise Linux.
Upstart replaced sysvinit in Ubuntu 6.10, aka Edgy Eft. That's
right: Ubuntu hasn't shipped with sysvinit since 2006. And when
all the crap you've been dropping into /etc/init.d/
on your
Ubuntu servers still "works" after all these years, it's because
Upstart is backwards-compatible with sysvinit.
Go ahead, have a look at the file /etc/init/rc.conf
on an Ubuntu
machine. These ~20 lines of code are what runs your /etc/init.d/rc
script, which in turn starts your "init scripts".
Congratulations, you've just read an Upstart-based "init
script". Notice how it looks more like configuration than
scripting? Let's have a look at /etc/init/hostname.conf
from a
basic Ubuntu machine:
description "set system hostname" start on startup task exec hostname -b -F /etc/hostname
There's a description, statements on when it should start, the
task
keyword, and an exec
statement. There's no command line
parsing, no pid files, no daemonization, no sourcing of other shell
scripts, there isn't even any mention of runlevels - just the word
startup
. It looks like what it is: configuration of a command
that will be invoked with one of three arguments:
Upstart "init scripts" aren't shell scripts, so they cannot be run
like /etc/init.d/
scripts. To manually start/stop/restart
Upstart-based tasks, you use the initctl
program with one several
parameters:
initctl start webserver initctl stop webserver initctl restart webserver
or even
start webserver stop webserver restart webserver
To learn more about writing Upstart recipes, read man 5 init
on
an Upstart-based system. I guarantee you that after 30 minutes
you'll be writing "init scripts" of better quality than anything
you ever dropped into /etc/init.d/
on some poor computer.
Instead of relying on runlevels, Upstart focuses on events that
happen on your system, and use these events in your start on
and
stop on
definitions. You're free to add your own events, and
emit them:
initctl emit i-just-farted
Any service that has a start on i-just-farted
definition will
start once you do this. Your operating system will emit its own
events, probably more useful than this once, check their
documentation for this.
One of my favorite features of Upstart is that it doesn't require me to daemonize; rather it tries to keep my service running for me (if I ask it to). No double-forking, no PID files and no external process babysitting. Simply create a program with an event loop, and add it to your Upstart service description like this
description "My service"
start on filesystem or runlevel [2345]
stop on runlevel [!2345]
respawn
exec /usr/bin/myservice
The last line contains the entire command to be run. If the
contents of /usr/bin/myservice
looks like this:
#!/usr/bin/env ruby while true do end
Upstart would be happy to run it for you.
Not only will Upstart run the process for you; if you look at this line:
1: respawn
If for some reason a program started by Upstart crashes, and you
add the respawn
instruction in your recipe, Upstart will
re-launch it when it crashes. If you're on an Ubuntu machine right
now, try it out.
Create the file /usr/bin/myservice
and make it look like this
1: while true do 2: end
Make it executable
sudo chmod 0755 /usr/bin/myservice
Then create the file /etc/init/myservice.conf
and make it look
like this:
description "My service"
start on filesystem or runlevel [2345]
stop on runlevel [!2345]
respawn
exec /usr/bin/myservice
Now, let Upstart know you created this service:
sudo initctl reload-configuration
And tell Upstart to run it:
sudo start myservice
Upstart will tell you what it just did:
myservice start/running, process 12386
Make a note of the PID reported by Upstart, and kill it:
sudo kill 12386
Then try running a ps -ef |grep myservice
and you'll see a new
process. Upstart did that for you.
Upstart is in RedHat Enterprise Linux since version 6, and Ubuntu
since 6.10. That all your servers are most likely running
Upstart. Now would be a good time to have a look at what you've put
in your /etc/init.d
. man 5 init
will give a usable introduction
to how to write Upstart recipes.
I'll be writing a followup article to this article covering systemd, which is what Upstart should have been.
comments powered by Disqus