Up to Main Index                          Up to Journal for February, 2019

                   JOURNAL FOR THURSDAY 28TH FEBRUARY, 2019
______________________________________________________________________________

SUBJECT: One-liners and ring buffers
   DATE: Thu 28 Feb 22:42:23 GMT 2019

This week has been, hrm… not full of joy and wonders :( I have managed to
squeeze in a little time for WolfMUD, and have just enough time left for one
last journal entry this month.

As I mentioned in my last post[1], I’ve been trying to fight off this nasty
one-liner:


  seq 1 1000000 | xargs -n1 bash -c 'netcat -z 172.16.1.1 4001'


Actually that is a poor example, it’s spinning up a bash shell each time it
executes netcat. That’s an artifact from where I was testing different sleep
delays using 'netcat -z 172.16.1.1 4001;sleep 1;' and varying the sleep time.

A better, or nastier — depending on how you look at it, example would be:


  seq 1000000 | xargs -P8 -n1 -i@ netcat -z 172.16.1.1 4001


How is the fight going? I can fend off the one-liner across ethernet with the
CPU[2] running at about 30% with most of that being kernel processes and
software interrupts:


  top - 22:25:41 up 7 days,  6:36,  5 users,  load average: 0.15, 0.17, 0.23
  Tasks: 157 total,   1 running, 156 sleeping,   0 stopped,   0 zombie
  %Cpu(s): 2.1 us,  4.3 sy, 0.0 ni, 89.8 id, 0.3 wa, 0.0 hi,  3.5 si,  0.0 st
  MiB Mem :  7432.3 total,  1070.7 free,  1950.7 used,  4410.8 buff/cache
  MiB Swap:  2304.0 total,  2304.0 free,     0.0 used.  5099.7 avail Mem

    PID USER      PR NI    VIRT   RES   SHR SWAP S %CPU  %MEM    TIME+ COMMAND
  16889 diddymus  20  0  104.7m  7.8m  2.8m 0.0m S 30.6   0.1  0:42.43 server


This is using Go 1.12 that was just released. To put that into some sort of
perspective, the server is accepting and rejecting 3,000 connection attempts
per second. Not too bad, but also not terrific — I’m just using one machine to
throw traffic at the server. Most of the CPU time is spent in syscalls that
handle accepting and closing the network connections — and I don’t see how I
can optimise that.

I still say using iptables is a much better idea…

As part of my solution I’ve implemented a ring buffer for recording connection
attempts. The implementation is a fixed size (uses an array) power of 2 ring
buffer with free running indexes and bit-masking instead of modulo arithmetic.

All of the methods will inline and it’s turned out to be quite fast :)

Although I don’t need all of the functionality, the ring buffer is complete.
It has copious comments and tests so that other people can reuse it if they
want to. The tests will need fixing up if the array size is changed. Changing
the ring buffer to use a slice instead of an array is also trivial, but
performance will suffer. You can’t say I haven’t warned you now :P

I’m still working on improving the actual limiter. All of the testing,
benchmarking, tweaking performance, commenting the code and updating the
configuration documentation is taking some time. However, the ring buffer,
with tests, has just been pushed out to the public dev branch for those
interested in taking a look.

I also want to see how my Raspberry Pi3 running the server fares against the
i7 running the one-liner, but I haven’t had time — yet! ;)

--
Diddymus

  [1] Journal for Saturday 23rd February: 23.html

  [2] Client: i7-860 @ 2.80GHz, Server: i5-2400 @ 3.10GHz


  Up to Main Index                          Up to Journal for February, 2019