Up to Main Index                             Up to Journal for March, 2016

                      JOURNAL FOR SUNDAY 6TH MARCH, 2016
______________________________________________________________________________

SUBJECT: Jars of records and WolfMUD data files
   DATE: Sun  6 Mar 15:22:22 GMT 2016

Today is Mother's Day in the UK. If you can't get to see your mum think about
picking up the phone and have a chat for a while. I have :)

Last Saturday I had a quiet day programming on WolfMUD and wrote a new Record
Jar handler from scratch. I could have reused the old code, but I tend to find
rewriting usually produces better code than the previous version. In the case
of the Record Jar the code this time is much simpler, handles more corner
cases and is a vast improvement generally.

For those not familiar with the Record Jar format, it is a plain text file
format used by WolfMUD for all of its data files. For WolfMUD the files end in
.wrj for WolfMUD Record Jar. The format is based on RFC5322 - Internet Message
Format[1] and the Record Jar format described by Eric Raymond in "The Art of
Unix Programming", chapter 5[2].

Essentially it is a plain text file with a colon separating fields and data.
There may also be a block of free text following a blank line:


  //
  //	Zone: Zinara
  //
        Ref: L1
       Name: Fireplace
    Aliases: TAVERN FIREPLACE
      Exits: E→L3 SE→L4 S→L2
  Inventory: N1

  You are in the corner of a common room in the Dragon's Breath tavern. There
  is a fire burning away merrily in an ornate fireplace giving comfort to
  weary travellers. Shadows flicker around the room, changing light to
  darkness and back again. To the south the common room extends and east the
  common room leads to the tavern entrance.
  %%
      Ref: L2
     Name: Common Room
  Aliases: TAVERN COMMON
    Exits: N→L1 NE→L3 E→L4

  You are in a small, cosy common room in the Dragon's Breath tavern. Looking
  around you see a few chairs and tables for patrons. To the east there is a
  bar and to the north you can see a merry fireplace burning away.
  %%


These are the first two locations in WolfMUD. Comments start with '// '. Then
we have field names - which contain no whitespace - followed by a colon. In
the example above "Ref", "Name", "Aliases", "Exits" and "Inventory" are field
names. The text after the colon is the data for each field. Data may continue
over multiple lines. Then we have a blank line which separates the fields from
the free text block. In WoldMUS this is usually used for descriptions. Lastly
there are the record separators consisting of two percent signs '%%'. The file
would be the jar and in this case it has two records representing two
locations.

Some of the differences to the format in WolfMUD include:


  - Field names, data and free text can contain Unicode
  - Field names are case insensitive
  - Whitespace handling is more lenient and can precede comments and fields
  - The line endings can be CRLF or LF
  - Comments start with "// "


What are Some of the improvements in the rewrite? In the free text block
leading whitespace and blank lines will be preserved. If a record consists of
only a free text block a blank line is not required. Cleaner and simpler code.

Why did I pick this format? From a users point of view it is simply a text
file. If you can type, you can create your own worlds. You can use any editor
that handles plain text files. You can use tools like spell checkers, grammar
checkers, grep, sed and awk. You can easily use them with a version control
system. After many, many years plain text is still the universal format.

So far I have the main Record Jar handler written which parses a .wrj file. The
returned data is a simple slice of maps. Records are a map[string][]byte. This
makes iterating a Record Jar very simple:


  f, _ = os.Open("zinara.jar")
  jar := recordjar.Read(f)
  for _, record := range(jar) {
    for field, data := range record {
      // Interesting stuff goes here
    }
  }


There is a decoder to simplify converting the []byte data into different
types, but you do not have to use it:


  flag := recordjar.Decode.Boolean(data)
  duration := recordjar.Decode.Duration(data)
  greeting := recordjar.Decode.Bytes(data)


The last exampe of Bytes may seem odd, we pass in a []byte and get a []byte?
We actually get a copy returned - so the original file data is not pinned in
memory by a slice over a small part of it.

So far the server configuration has been modified to be read from a Record
Jar. The first zone, the Town of Zinara, can almost be loaded. I've written a
lot of the Attribute unmarshalers required. As soon as that work is completed
I'll be pushing out the changes.

In the meantime I've fixed a few issues which I'm about to push out to the
public dev branch now:


  - Output of short strings caused layout issues on Windows
  - The messages for the EXAMINE command didn;t read quite right
  - Regression when just hitting enter, a blank line had been reintroduced
  - Attribute interface and code should be in their own files


--
Diddymus

  [1] https://www.rfc-editor.org/rfc/rfc5322.txt

  [2] http://www.catb.org/esr/writings/taoup/html/ch05s02.html


  Up to Main Index                             Up to Journal for March, 2016