Title: Journal for Wednesday 27th March, 2019
Abstract: A WolfMUD Wednesday update!
    Tags: record-jar, recordjar, static site generator

                    JOURNAL FOR WEDNESDAY 27TH MARCH, 2019
______________________________________________________________________________

SUBJECT: A WolfMUD Wednesday update!
   DATE: Wed 27 Mar 22:54:49 GMT 2019

If you have not read it already, please take a moment to read my previous
entry file://22.html I don’t mind waiting……… all done? Great! :)

Bear with me, this entry is about WolfMUD. Need to provide some context first…

Recently I have found myself needing a little project to work on. Something
different for a while. With the impending demise of the server this website
and WolfMUD’s Git repository are hosted on, my thoughts drifted in that
direction. In particular it focused on the custom static site generator I
wrote and use.

The generator uses the same record jar format WolfMUD uses and all of the
pages are just plain text. You can see an example by looking at the original
text file for this — file://27.txt — or any other page.

Just go to any page on this site and change the .html to .txt in the URL if
you are curious. Yup, I’ll wait… don’t forget to come back…

The site generator and record jar format it uses pre-dates WolfMUD. It was
originally written for another site I had called “Justifying Text”. I idly
thought to myself “I really need to update the generator to use WolfMUD’s
recordjar package”. Over the past week I’ve actually rewritten the generator.
It’s almost done, but not quite. As well as bringing it up to date I’ve made
it more disk I/O friendly for SSDs and Flash storage — just in case I have to
host everything myself, using one of my Raspberry Pi.

To my horror, I found some issues with WolfMUD’s recordjar package! :P ;)

First of all, fields within a free text section were being identified as
actual fields and not as part of the free text. That’s a big no-no and
something I was surprised I had let slip through.

Another surprise was that record separators ‘%%’ within a free text section
end the current free text section. This was only a small surprise as I haven’t
implemented a way to escape the record separator in any way. A really should
have listened to the wise Eric Raymond[1]:


  “In this case, it’s important that you have a well-defined way to
   escape the record delimiter so it can appear in text;”


So why didn’t I listen? I wanted the text to be as close to the actual text
used as possible. I didn’t want to see ‘\%%’ for example in my text files when
it should be ‘%%’. For now I’ve implemented a compromise. An actual record
separator after a free text section must not be indented. If it is indented it
will be seen as part of the text in the free text section.

For some reason I thought it was a good idea to sort string lists. Can’t
remember why I thought it a good idea at the time, but I was wrong. Unlike
other lists, which have keys, the only ordering is the original order of the
string list. If the ordering has some human meaning, but you then sort the
list, you are stuffed.

The last item on my “what have I done wrong” list involves line feeds. Those
handy little, invisible characters at the end of a line that cause a new line
to be started.

                                       Here is one of the little buggers! --->

When writing WolfMUD’s recordjar package I tried to be clever. Never bodes
well does it? I think I’m starting to see a pattern here. I though it would be
a good idea to only preserve meaningful white-space. This works well. Until
WolfMUD reformats all outgoing text using the text.Fold function that is, then
the extra effort is wasted. More importantly, should reading a jar file be
monkeying around with your carefully crafted text in the first place? I have
come to realise it should not.

In working on the static site generator I have achieved three things. I have a
better generator that is nearly finished. I have improved WolfMUD’s recordjar
package. Last of all, I have made the recordjar package more useful outside of
WolfMUD.

The fixes for fields in the free text section, record separators in the free
text section and preserving line feeds in the free text section, plus a few
other nits are on the public dev branch. Get them while you can!

In case you are wondering this site currently has 113 indexes and 374 pages.
To create all of the HTML, and gzip versions of the HTML pages, from 618 plain
text files take about 1 second :)

--
Diddymus

  [1] The Art of Unix Programming; Data File Metaformats; Record-Jar Format:
      http://www.catb.org/esr/writings/taoup/html/ch05s02.html