Up to Main Index Up to Journal for June, 2021
JOURNAL FOR SATURDAY 26TH JUNE, 2021
______________________________________________________________________________
SUBJECT: I think I just vindicated my WolfMini experiment
DATE: Sun 27 Jun 04:00:28 BST 2021
Last time I said I was probably going to make the WolfMini code available via
the main WolfMUD Git repository. However, that has not happened yet :( I
wanted to make a few more changes before going public with the new code and
then things went a little south…
After adding the multiplayer code I then integrated my new mailbox delivery
system. Things looked good, performance was good, everything worked. I then
unleashed the botrunner on the server. No matter what I tried I always had a
bottle-neck around 32,000 bots where performance dropped off sharply and
responses could take ages. Stupid bots :(
It’s taken me the best part of a week of late nights to debug and test — a
full test can take over 30 minutes to run. Was it worth it? Was the experiment
going to be a big flop?
Well, while writing this, the server has 64,000 bots and myself logged in.
Response times are sub-second :D
The server process is using about 110%-120% CPU across four cores (so about
25%-30% per core) on my i5. Memory usage reported by the top command is 1.5Gb.
The only tweak I made was to the networking to access all the ports — I’ve even
stayed with the stock three starting locations.
Oh, and the desktop is running the server AND the botrunner…
The maximum number of ports per IP address is limited to 65,535 — with the
lower 1024 ports reserved that’s 64,511 available. So, if I want to try with
more than 64,000 bots I’ll need multiple listeners on multiple IP addresses.
(See my addendum at the end for more on this!)
At the moment the server is not folding and formatting any outputted text.
Even so, this all looks very promising.
I wonder how many bots my Raspberry Pi, especially the Zero W, will be able to
handle now? Hrm… one moment dear reader, please wait… using TCP/IP over USB
the Pi Zero W can handle 1921 bots using about 40Mb RAM and 92%-97% CPU, and
keep sub-second response times :)
ADDENDUM:
Having mentioned the need to use multiple IP addresses I thought “why not just
add another local interface?”. I added 127.0.0.2 and did a few more tests.
I’ve just had 128,000 bots and myself logged in to the server :)
I had to run two botrunners, one using 127.0.0.1 the other 127.0.0.2, so that
enough ports were available. I also ran the botrunners with “nice -n19” to
stop them starving the server process of CPU. In the botrunner I increased the
number of actions per bot run — otherwise bots were finishing and logging in
again before they were all loaded. I also reduced the frequency of commands to
one every 2-3 seconds otherwise the combined botrunners used so much CPU the
server couldn’t run properly. At 128,000 bots that’s still an average of
51,200 commands a second being processed. Server came in at 3Gb RAM fully
loaded with bots. Response times… still sub-second ;)
I implemented a simple #MBOX command to show me the number of active players.
An active player always has one mailbox so this works. This is me, on top of a
giant toadstool while 128,000 bot are running around…
[On top of a giant toadstool]
You are standing on top of one of giant toadstools. Around you you can see a
vast cavern, it's floor covered by a forest of these toadstools. The one you
are on has a hole cut into the cap allowing access down via some stairs cut
into the stalk.
You see Player15157 here.
You see Player17794 here.
You see a glass ball here.
You see exits: Down
>
Player17794 says: HI!
>
You see Player17794 leave.
>#mbox
Mailboxes: 128001
>
You see Player17794 enter.
>
Here is a snapshot from the top command after all of the bots have connected
and are running around:
top - 22:53:08 up 7 days, 2:48, 7 users, load average: 4.59, 4.65, 6.03
Tasks: 143 total, 1 running, 141 sleeping, 0 stopped, 1 zombie
%Cpu0: 8.4 us, 22.4 sy, 29.4 ni, 35.0 id, 0.0 wa, 0.0 hi, 4.7 si, 0.0 st
%Cpu1: 7.0 us, 21.1 sy, 24.1 ni, 43.2 id, 0.0 wa, 0.0 hi, 4.5 si, 0.0 st
%Cpu2: 7.4 us, 23.3 sy, 26.0 ni, 39.1 id, 0.0 wa, 0.0 hi, 4.2 si, 0.0 st
%Cpu3: 6.9 us, 18.7 sy, 27.6 ni, 42.4 id, 0.0 wa, 0.0 hi, 4.4 si, 0.0 st
MiB Mem : 7412.0 total, 1551.0 free, 5293.8 used, 567.1 buff/cache
MiB Swap: 2304.0 total, 2274.2 free, 29.8 used. 1686.8 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9959 diddymus 20 5348.8m 3.0g 3.4m S 121.3 41.7 47:28.24 game
10063 diddymus 39 19 3587.9m 591.1m 3.1m S 82.7 8.0 37:15.81 botrunner
10070 diddymus 39 19 3539.9m 591.3m 3.1m S 82.1 8.0 37:12.71 botrunner
I seem to have reached the limits of my desktop PC for running both the server
and botrunners. If I upgraded my old 10/100 switch, off-loaded the botrunners
to another machine or two I might get the desktop running, oh I don’t know,
maybe quarter to half a million bots? I think a small fleet of Raspberry Pi 4
botrunners would do nicely :D
--
Diddymus
Up to Main Index Up to Journal for June, 2021