WolfMUD: Journal for Saturday 16th March, 2019

Up to Main Index Up to Journal for March, 2019

JOURNAL FOR SATURDAY 16TH MARCH, 2019
______________________________________________________________________________

SUBJECT: A whirlwind tutorial for WolfMUD + Docker
DATE: Sat 16 Mar 20:38:54 GMT 2019

Life is still being rather shitty to me at the moment, so I thought I’d give
it a poke in the eye by doing something I enjoy — writing. To that end, I’ve
put together this — long overdue — tutorial, with additional notes and a few
insights.

I’ve said many times before that WolfMUD, and MUDs in general, are a great
tool for learning. It encompasses so many different areas of programming from
networking and protocols to concurrency, file handling, text parsing, text
formatting, logging, data structures and algorithms just to name a few — all
while having some fun. Now we are going to use WolfMUD to learn a little about
Docker :)

Comments, corrections and suggestions gratefully received:

diddymus@wolfmud.org

# # #

INTRODUCTION

I get a lot of emails asking a lot of questions about running a WolfMUD server
using Docker. I’m not going to start providing Docker images though, well I
have no plans to provide them, maybe if enough people ask for them I might.
This tutorial is not the usual “pull down an Alpine Linux image, layer on a Go
image, set up a development environment and build everything into a final
image”. There are plenty of other tutorials around the internet that cover
that. This tutorial, very verbosely and with plenty of examples, takes the
WolfMUD server, stuffs it into an image and runs everything in a container.
Very simple, basic, straight forward stuff — a gentler introduction to Docker
you might say.

What is Docker? Docker is operating system level virtualisation. It is like
running a lightweight virtual machine, but it uses the host’s operating system
kernel. It is not hardware virtualisation, like Qemu or VirtualBox, so you
can’t use Docker to run Windows on a Linux machine for example.

Now I’m not a Docker guru, there may be better ways of doing things. Where
there are multiple ways of doing something I’ve tried to explain the options
and my rational for my choices. I have no way of knowing what will work best
for other people.

By default the WolfMUD binaries I build and provide for downloading are built
with CGO_ENABLED=0. This means that the server needs no supporting libraries
to run and uses the pure Go network resolver. This makes WolfMUD an ideal
candidate to use for learning and experimenting with Docker as there are no
external dependencies to worry about :)

In this tutorial lines in the examples starting with ‘$’ show commands you can
type. The ‘$’ itself represents a command prompt on the command line and
should not be typed. Values in angle brackets, such as ‘<user>’ should be
replaced with a suitable value, in this case you would replace the ‘<user>’
with your actual user name.

All examples were tested on an Intel system running Debian (testing/buster).
Packages used were docker.io version 18.09.1+dfsg1-5+b10 and busybox-static
version 1:1.30.1-2. The version of WolfMUD used is v0.0.13. For Raspberry Pi
users there are some additional notes towards the end of this tutorial. If you
are adventurous enough to be using a different architecture or operating
system that supports Docker this tutorial should still be useful as a guide,
although you may need to adapt things a little to suit your particular setup.

# # #

PREPARATION

I’m going to assume you have downloaded one of the binary versions of WolfMUD
and have unpacked it. In which case the directory and file structure should
look something like this:

For use in Docker we will use a minimal WolfMUD installation that will include
only the following directories and files:

First of all you will need to have Docker installed. On a Debian system this
is ridiculously easy, you just have to install the docker.io package:

$ sudo apt-get install docker.io

Unless you want to keep using sudo for everything Docker related you may want
to add yourself to the docker user group. I suggest reviewing the Docker
security documentation[1] first to understand the implications of this. The
gist of the advice is to only give trusted users the ability to control
Docker. With that said, to add yourself to the Docker group:

$ sudo usermod -aG docker <user>

You will then need to log out and log back in again for the changes to take
effect.

That is all of the preliminaries out of the way:

√ Unpack a WolfMUD binary download
√ install docker: apt-get install docker.io
√ add user to docker group: sudo usermod -aG docker <user>
√ sign out and back in again for group change to take effect

Now we need to edit the WolfMUD configuration file. The default configuration
file the WolfMUD server uses only listens on the local interface. When running
in a Docker container listening on the local interface will not work, only the
container can access its own local interface. We need to be able to forward
network ports on the host to the container, to do that we need to use an
interface visible to the host. Edit the file data/config.wrj and delete the
value of ‘127.0.0.1’ from the entry for ‘Server.Host’ so that the entry looks
like:

// Server configuration
Server.Host:
Server.Port: 4001
Server.IdleTimeout: 1h

This will cause the WolfMUD server to listen on all available network
interfaces. Save the changes to the configuration file.

Next we need to create a new file called ‘Dockerfile’. This file needs to be
in the same directory as the WolfMUD ‘server’ executable. The Dockerfile
should contain these lines:

FROM scratch
EXPOSE 4001/tcp
COPY server /
CMD ["/server"]

This file is used as the blueprint or recipe when creating Docker images. An
image is like installation media. From an image we can create and run one or
more containers — which may run on one or more machines. In this tutorial we
will be creating our own base image, an image that does on depend on other
images. Normally you would take an existing image, for example BusyBox, and
extend it by adding your own files and configuration.

What does our docker file do? The ‘FROM scratch’ says that this image is based
on the ‘scratch’ image. This is a special built in empty image. If we instead
used ‘FROM busybox’ Docker would download the busybox image automatically, if
we did not already have it, from the Docker repository and use that as a
starting point for our image. The ‘EXPOSE 4001/tcp’ says we are going to use
port 4001 for tcp networking. The next line ‘COPY server /’ copies the file
‘server’ from our build context (current directory) to the root of the image’s
filesystem[2]. The final line ‘CMD ["/server"]’ specifies the command to run
when a container built from this image is started. Remember to save this file
as ‘Dockerfile’.

The WolfMUD server is going to need some data files, the configuration file
and the zone files. We could COPY the files into our Docker image using our
Dockerfile the same way we copied the server executable. The server is also
going to need somewhere to store player files. If we add files directly to the
container any changes made will be lost if the container is deleted and
recreated. Instead we will use a data Volume to persist our files and any
changes to them. Volumes exists independently of the containers and can be
used by multiple containers at the same time.

Create a new, empty volume called ‘wmdata’ using the following command:

$ docker volume create wmdata
wmdata
$

You can see a list of what volumes you have using:

$ docker volume ls
DRIVER VOLUME NAME
local wmdata
$

If you make a mistake, or want to start over, you can delete a volume using:

$ docker volume rm <volume name>

Next we need to build our Docker image using our Dockerfile. Change to the
directory containing the Dockerfile, in Docker-speak the current directory is
known as the build context. To build the image run:

$ docker build --tag wm:v0.0.13 .
Sending build context to Docker daemon 7.107MB
Step 1/5 : FROM scratch
--->
Step 2/5 : EXPOSE 4001/tcp
---> Running in eda9ae48404d
Removing intermediate container eda9ae48404d
---> b2c06bc413fb
Step 3/5 : COPY /server /
---> e615c487513c
Step 4/5 : COPY busybox /
---> 82d90ddf2371
Step 5/5 : CMD ["/server"]
---> Running in dcd22087331e
Removing intermediate container dcd22087331e
---> d8055c1c1a20
Successfully built d8055c1c1a20
Successfully tagged wolfmud:v0.0.13
$

For brevity, so that the examples fit within the site’s 80 column width, I’ve
called the image ‘wm’. The version of WolfMUD I have used is v0.0.13.
Including the version number makes it easy to use multiple versions of an
image. For example you could repeat this tutorial for WolfMUD v0.0.14 and run
different versions of the server in different containers. We can list our
current images using:

$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
wm v0.0.13 d8055c1c1a20 43 seconds ago 7.04MB
$

If you make a mistake, or rebuild the image, you can remove unwanted images
using:

$ docker image rm <image>

The ‘<image>’ can be specified as either the image id, which in the above
example is ‘d8055c1c1a20’, or ‘wm:v0.0.13’. From the example ‘d8055c1c1a20’ is
the image id that was automatically assigned by Docker and will be unique for
each image created. If you want to use the image id you need to use the one
shown on your system and not the example one shown here.

Next, we want to create a container from our new image image:

$ docker create -v wmdata:/data -p4001:4001 --name WolfMUD wm:v0.0.13
7e317c60a7e4d5a4a7e0234e4d8e51ad3938c3095ae1ac63983cd87ee7df2b16
$

This will create a container named ‘WolfMUD’ and mount our wmdata volume under
the /data directory in the resulting container. Network connections for port
4001 on the host will be forwarded to the WolfMUD server on port 4001 in the
container. We can list our containers using:

$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7e317c60a7e4 wm:v0.0.13 "/server" 3 seconds ago Created WolfMUD
$

When created the container also has a unique id assigned to it by Docker. In
this case ‘7e317c60a7e4’. This allows us to refer to the container either by
the container id or by the name ‘WolfMUD’ which we assigned to the container.

Initially the data directory the container will be using, from the wmdata
volume we created, will be empty. How do we populate the wmdata volume and get
our configuration and zone files into the container? We can use Docker’s ‘cp’
command to copy files to/from the host and the Docker container:

$ docker cp data WolfMUD:/

This will copy the local data directory into the container WolfMUD at the
container’s filesystem root ‘/’. Finally we get to start the container and our
WolfMUD server:

$ docker start WolfMUD
WolfMUD
$

To check the status of the container and make sure it is running we can use
the ‘ps’ command again:

$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7e317c60a7e4 wm:v0.0.13 "/server" 2 min ago Up 7 sec :4001->4001/tcp WolfMUD
$

With the container started you should be able to connect to the WolfMUD server
via Telnet in the usual way:

$ telnet -e~ 127.0.0.1 4001
Telnet escape character is '~'.
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '~'.

World
Of
Living
Fantasy

Welcome to WolfMUD!

Enter your account ID or just press enter to create a new account, enter
QUIT to leave the server:
>

If you wish to stop the container, and check it has stopped:

$ docker stop WolfMUD
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7e317c60a7e4 wm:v0.0.13 "/server" 1 hour ago Exited(2) 6 sec ago WolfMUD
$

Old container instances can be removed using:

$ docker rm <container id>

You can also remove all stopped containers using:

$ docker container prune
docker container prune
WARNING! This will remove all stopped containers.
Are you sure you want to continue? [y/N] y
Deleted Containers:
7e317c60a7e4d5a4a7e0234e4d8e51ad3938c3095ae1ac63983cd87ee7df2b16

Total reclaimed space: 0B
$

Once the container is running you can connect to the instance to see the live
WolfMUD server output using:

$ docker attach --sig-proxy=false WolfMUD
2019/03/12 18:16:51 U[ 2Mb +0b ] O[ 4466 +14] T[ 222 +0] G[ 26 +0] P 0/0
2019/03/12 18:17:01 U[ 2Mb +0b ] O[ 4468 +2] T[ 222 +0] G[ 26 +0] P 0/0
2019/03/12 18:17:11 U[ 2Mb -144b ] O[ 4468 +0] T[ 222 +0] G[ 26 +0] P 0/0
:
:
:

Ctrl-c will end the previous command. To see the complete log you can use:

$ docker container logs WolfMUD
2019/03/14 19:46:40.383974 config.go:113: Server started, logging using UTC…
2019/03/14 19:46:40.384049 config.go:288: Found configuration file: /data/c…
2019/03/14 19:46:40.384058 config.go:131: Loading: /data/config.wrj
2019/03/14 19:46:40.384280 config.go:204: Data Path: /data
2019/03/14 19:46:40.384391 config.go:207: Set permissions on player account…
2019/03/14 19:46:40.384402 config.go:214: IP connection quotas are disabled…
2019/03/14 19:46:40 Switching to short log format.
2019/03/14 19:46:40 Allocated pool for 2560 buffers
2019/03/14 19:46:40 U[ 751kb +751kb] O[ 334 +334] T[ …
:
:
:

You can see automatically updating container statistics using:

$ docker stats
CONTAINER ID NAME CPU % MEM USAGE/LIMIT MEM % NET I/O BLOCK I/O PI
7e317c60a7e4 WolfMUD 4.02% 20.45MiB/7.258GiB 0.28% 2.42MB/4.6MB 0B/30.7kB 12

Ctrl-c exits back to the command line.

# # #

ADDING BUSYBOX

When testing containers I like to drop in a static BusyBox so that I can do
things like listing the player files:

$ docker exec WolfMUD /busybox ls -l /data/players
total 4
-rw-rw---- 1 0 0 365 Mar 12 17:33 90d9988c2b7014d622a62681e5643674.wrj
$

Or checking the configuration file being used:

$ docker exec WolfMUD /busybox cat /data/config.wrj
// Copyright 2019 Andrew 'Diddymus' Rolfe. All rights reserved.
//
// Use of this file is governed by the license in the LICENSE file included
// with the source code.
//
// config.wrj - Main WolfMUD server configuration file. For details of the
// options and their settings see docs/configuration-file.txt.
//
// Server configuration
Server.Host:
Server.Port: 4001
Server.IdleTimeout: 1h
Server.MaxPlayers: 20480
Server.LogClient: true
:
:
:
$

You can even run an interactive shell for poking around:

$ docker exec -it WolfMUD /busybox sh

BusyBox v1.30.1 (Debian 1:1.30.1-2) built-in shell (ash)
Enter 'help' for a list of built-in commands.

/ # ls
busybox data dev etc proc server sys
/ # cd data/
/data # ls
config.wrj players zones
/data # ls -l
total 12
-rw-r--r-- 1 0 0 1273 Mar 12 16:56 config.wrj
drwxr-xr-x 2 0 0 4096 Mar 12 16:53 players
drwxr-xr-x 2 0 0 4096 Mar 1 16:01 zones
/data #

NOTE: You could pull the BusyBox image from the Docker repository and use that
as a base image to build a WolfMUD image on. Would you learn as much? Do you
trust that any images you pull down are actually what they say they are and
have been built how they say they have been built? Hrm, just a thought…

To add BusyBox, first install the busybox-static package:

$ sudo apt-get install busybox-static

You may already have the non-static version installed called ‘busybox’. It
should be safe to replace ‘busybox’ with ‘busybox-static’.

Alternatively, if you are more adventurous, you can grab it directly from the
BusyBox website[3]. The static busybox executable needs to be copied into your
Docker build context, the same directory where your Dockerfile is located:

$ cp -a `which busybox` .

By using a static version of BusyBox we don’t need to worry about having to
add any dependencies BusyBox needs into the container as well. This helps to
keep things very simple.

Now we need to add BusyBox to the container. We could just use ‘cp’ to copy
the BusyBox executable into the container:

$ docker cp busybox WolfMUD:/
$ docker exec -it WolfMUD /busybox ls -l
total 6888
-rwxr-xr-x 1 1000 1000 1945856 Mar 2 08:11 busybox
drwxr-xr-x 4 1000 1000 4096 Mar 14 12:55 data
drwxr-xr-x 5 0 0 340 Mar 14 12:55 dev
drwxr-xr-x 2 0 0 4096 Mar 14 12:55 etc
dr-xr-xr-x 169 0 0 0 Mar 14 12:55 proc
-rwxr-xr-x 1 0 0 5092848 Mar 14 11:52 server
dr-xr-xr-x 13 0 0 0 Mar 14 12:55 sys

We could then make the change permanent using Docker’s ‘commit’ command. This
would use our current container and create a new image for us:

$ docker commit WolfMUD
sha256:0a55f9b12f12e9d99e8dd6b259ff58dd0017f348bca6a7e63c4e6d8995541fac
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
<none> <none> 0a55f9b12f12 7 seconds ago 7.04MB
wm v0.0.13 2217bf35e043 2 minutes ago 7.03MB

However, that would make our image and container non-reproducible — which is
frowned upon in Docker circles. Using ‘cp’ and not committing might be okay
just for testing or debugging, where you don’t want to keep BusyBox in your
image or container.

Instead we are going to edit our Dockerfile, rebuild the image and create a
new container. The new image and the container will then be reproducible from
the Dockerfile.

Edit the Dockerfile and insert a new ‘COPY’ line under ‘COPY server /’. This
will copy busybox from your build context into the image when we rebuild it:

FROM scratch
EXPOSE 4001/tcp
COPY server /
COPY busybox /
CMD ["/server"]

Save the changes to the Dockerfile. Then rebuild the image:

$ docker build --tag wm:v0.0.13 .
Sending build context to Docker daemon 7.107MB
Step 1/5 : FROM scratch
--->
Step 2/5 : EXPOSE 4001/tcp
:
:
:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
wm v0.0.13 2217bf35e043 2 minutes ago 7.03MB
<none> <none> 7e317c60a7e4 2 hours ago 7.03MB
$

Make sure the old container is stopped:

$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7e317c60a7e4 wm:v0.0.13 "/server" 2 mins ago Up 7 sec :4001->4001/tcp WolfMUD
$ docker stop WolfMUD
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORT NAMES
7e317c60a7e4 wm:v0.0.13 "/server" 1 hour ago Exited(2) 6 sec ago WolfMUD
$

Next we will remove the old WolfMUD container:

$ docker rm WolfMUD
WolfMUD
$

Create a new container based on our new image with BusyBox added:

$ docker create -v wmdata:/data -p4001:4001 --name WolfMUD wm:v0.0.13
9347c47a25c62d67b26d32fd8d5f71ba27c31234135d27dbb1dd4bc5ea8999bb
$

Finally we start the new container, then check that it is actually running:

$ docker start WolfMUD
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9347c47a25c6 wm:v0.0.13 "/server" 2 mins ago Up 7 secs :4001->4001/tcp WolfMUD
$

We should now be able to use BusyBox to poke around:

$ docker exec -it WolfMUD /busybox ls -l
total 6888
-rwxr-xr-x 1 0 0 1945856 Mar 13 12:48 busybox
drwxr-xr-x 4 1000 1000 4096 Mar 13 19:57 data
drwxr-xr-x 5 0 0 340 Mar 13 19:57 dev
drwxr-xr-x 2 0 0 4096 Mar 13 19:23 etc
dr-xr-xr-x 176 0 0 0 Mar 13 19:57 proc
-rwxr-xr-x 1 0 0 5092780 Mar 12 16:56 server
dr-xr-xr-x 13 0 0 0 Mar 13 19:57 sys
$

# # #

BACKUPS! BACKUPS! AND THRICE, BACKUPS!

All of our data is stored in the wmdata volume. This gets mounted inside the
container under the /data directory. How do we backup our data? We can use
BusyBox and good old tar for that:

$ docker run --rm --volumes-from WolfMUD -v $(pwd):/backup \
wm:v0.0.13 /busybox tar zcvf /backup/wmdata.tgz /data/
tar: removing leading '/' from member names
data/
data/zones/
data/zones/zinara_caves.wrj
data/zones/zinara.wrj
data/zones/zinara_south.wrj
data/zones/quiet.wrj
data/players/
data/players/.wrj
data/players/.gitignore
data/players/90d9988c2b7014d622a62681e5643674.wrj
data/config.wrj
$ ls -l data.tgz
-rw-r--r-- 1 root root 15835 Mar 13 20:08 wmdata.tgz
$

To restore our data we can do the same again, but this time extract the files
from the backup archive:

$ docker run --rm --volumes-from WolfMUD -v $(pwd):/backup \
wm:v0.0.13 /busybox tar zxvf /backup/wmdata.tgz
data/
data/zones/
data/zones/zinara_caves.wrj
data/zones/zinara.wrj
data/zones/zinara_south.wrj
data/zones/quiet.wrj
data/players/
data/players/.wrj
data/players/.gitignore
data/players/90d9988c2b7014d622a62681e5643674.wrj
data/config.wrj
$

This may seem a rather clunky way of doing backups, but it’s actually the
recommended Docker[4] way of doing it :( Why is there no ‘docker volume
something’ where something is save, restore, export or import? *shrug*

# # #

DOCKER ON RASPBERRY PI

This tutorial has also been tested on a Raspberry Pi 3 running Rasbian, which
makes an ideal little platform for experimenting with Docker. Just make sure
the version of WolfMUD is for ARM7 and not Intel. Packages used were docker.io
18.06.1+dfsg1-2+rpi1 and busybox-static 1:1.30.1-2. The version of WolfMUD
used is v0.0.13. In addition I had a few issues initially and had to:

- add ‘cgroup_enable=memory swapaccount=1’ to /boot/cmdline.txt
- run ‘update-alternatives --config iptables’ switch to -legacy from -nft
- run ‘update-alternatives --config ip6tables’ switch to -legacy from -nft
- reboot

I have not tried this on a Raspberry Pi Zero W yet. “What? Why would you do
that?” I hear you cry. Why indeed. The answer lies in Docker swarms. You can
set up a swarm, a collection of machines, and automatically deploy, run and
manage containers across the swarm.

The Raspberry Pi Foundation has a nice blog post[5] about Docker on Raspberry
Pi and some helpful links to further information.

There would be a few issues with WolfMUD, as it is, on a swarm. You would be
able to login the same character on each instance of WolfMUD and the saved
version of the player would be the last one logged out. The instances would
have no knowledge of each other — they would all be separate. However, if they
all used the same Docker volume for data you could login to one server, grab
an item in your inventory, logout, log into a different instance and drop the
item on a totally different server - kinda neat :)

This reminds me of an unreleased feature in the Java version of WolfMUD called
‘shadow link’. I think I might have to bring it back at some point. Shadow
link basically let you run different zones[6] on different server instances
and you could transparently walk from zone to zone across servers without even
knowing it. You could even throw items, cast spells and fire projectiles
across server boundaries as if it was all running on a single, normal WolfMUD
server. Its intended use was to share the burden of running a MUD by splitting
it up across multiple servers, each server run and maintained by a different
sysop.

# # #

FINAL THOUGHTS

Docker is a very interesting piece of software. Especially when you start
running services (multiple container instances) and swarms (multiple container
instances across multiple machines).

I hope you have found this tutorial helpful and interesting. Maybe you had no
intention of using Docker, but the tutorial has piqued your interest and
curiosity enough to try it out.

One thing I find quite alarming is the regularity with which the Docker daemon
crashes and takes out all of the running containers when restarted. Usually
the error is along the lines of:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x55f56c55c6b8]

goroutine 379 [running]:
github.com/armon/go-radix.recursiveWalk(0x0, 0xc000c9da60, 0xc000c9d900)
/build/docker.io-kA0j42/docker.io-18.09.1+dfsg1/.gopath/src
/github.com/armon/go-radix/radix.go:519 +0x28

This seems to be a known problem and has been reported:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=918375

Debian has not packaged the latest version of Docket yet, they have v18.09.1
and the latest is currently 18.09.3, so the problem may have already been
fixed. Maybe I should stop being lazy and grab the latest version and build it
myself…

--
Diddymus

[1] Docker security: https://docs.docker.com/engine/security/security/

[2] There is a feature you can use in a Dockerfile where ‘ADD’ will
automatically unpack archives such as WolfMUD-linux-amd64.tgz for us.
We could us this feature and then run something like BusyBox’s sed to
change the configuration file. I’ll leave that as an exercise for the
reader for now…

[3] BusyBox website: https://www.busybox.net

[4] Docker backup, restore or migrate volumes:
https://docs.docker.com/storage/#backup-restore-or-migrate-data-volumes

[5] Raspberry Pi Foundation Blog “Docker comes to Raspberry Pi”:
https://www.raspberrypi.org/blog/docker-comes-to-raspberry-pi/

[6] You could also load, for example, 5000 locations and have them
automatically spread across 5 server instances. That didn’t work out so
well as some instances would be very busy and others quiet. A better
idea would be to balance players across instances and ‘move’ the
locations to them. Gah! Now I’m designing shadow link v2 in my head ;)

Up to Main Index Up to Journal for March, 2019