System logs

I’ve been making a logging module, along the lines of !SysLog and the built-in Select module.

To give it a try, you can download from here.
Open the zip and double-click on the SysLog module. Type

*SysLog TestLog 0 Test message

in a task window, and TestLog will appear in your RAM disc (the location is hard-wired at the moment).

Written so far are

*SysLog <log name> <level> <message...>

Logs given message.

*Syslog <log name> <level>

Sets the detail level for the given log. A higher number means more messages are recorded.

*Syslog_Options <filename>

Loads log options from the given file.

The options file can set detail level and also divert to another computer¹. Only use divert within your private home network, as such message are transmitted in plain text.

*SysLog_Status <log name>

Display the current options for the given log

*SysLog_Status *

Display options for all known logs (including default options).

and SWI’s

SysLog_LogMessage
SysLog_LogUnstamped
SysLog_LogData
SysLog_LogFormatted
SysLog_FlushLog

Apps/modules I know of that send to SysLog:

AntiSpam
CryptRandom
Reporter (use *ReportOpt SysLog)
SysLogD
SysLogDev
UnixLib (syslog() function)
Vncserv ("*Set vncserv_debug 1" before starting)
WimpLog
WinEd

Some applications may say that SysLog has not been seen, in which case “*Set <SysLog$Dir> RAM:$” should keep them happy.

Please try any applications you have that work with SysLog, or use some of the commands above, and let me know how it is.

¹ When sending logs to another computer, they are sent by UDP to port 514. Messages are in RFC 5424 ‘IETF’ format. They can be received on Linux using syslog-ng or similar.

I’ve been making a logging module, along the lines of !SysLog and the built-in Select module.

Looking OK.
A re-implementation of the old Doggysoft(Jon Ribbens) / Tom Hughes v020 SysLog done in C? Minus a couple of SWI as you say.
It identifies itself as 0.20 as well.

First test abusing the old SysLog “Test” works nicely (once the SysLog_IRQMode call is REM’d out)

It identifies itself as 0.20 as well.

The module name/version I chose so that existing applications would be ok with it.

(once the SysLog_IRQMode call is REM’d out)

So, about that. I put off all my flushing (writing to files, network) till callbacks; effectively you’re always running with IRQMode on. I guess this could mean some messages get lost if you had lots of uninterrupted SVC mode logging or were trying to debug a SVC mode crash (sounds like a case for Reporter or DADebug). I would like to hear about any other code that uses IRQMode or the other more advanced SWI’s. I saw EtherUSB can call SysLog, but I don’t know how I persuade it to do that.

First test abusing the old SysLog “Test” works nicely (once the SysLog_IRQMode call is REM’d out)

Abusing the Test that comes with !SysLog (100000 messages) when logging over UDP annoys RPCEmu, saying “Too many open files” (but not when logging to a file??). I don’t know where the problem lies on that one.

I saw EtherUSB can call SysLog, but I don’t know how I persuade it to do that.

You only get syslog output from etherusb with the debug version of the module. The swis used are in etherusb.c.utils

Abusing the Test that comes with !SysLog (100000 messages) when logging over UDP annoys RPCEmu, saying “Too many open files” (but not when logging to a file??). I don’t know where the problem lies on that one.

Now there’s an interesting question, or multiples.

I wonder is that filling the buffer space on the RO stack or on the RPCEmu network stack? (guessing you’ve got the NAT setup)

Yes NAT. After doing the 100000 messages, on the RISC OS side there are no open files, and three open internet sockets (not opened by me).

On the Linux side, RPCEmu has open 999 UDP sockets. I don’t know if that’s deliberate or not.

Each flush of the log buffer does

socket()
sendto()
close()

When sendto() is called on a UDP socket that does not have any local port set, a local port number is assigned.

Reuse the socket, rather than create a new one each time.

This results the same local port number being used, allowing and requiring the NAT implementation in RPCEmu to reuse sockets. This will help with any NAT and stateful firewall.

Time for some Sunday afternoon SysLog.

The latest download is here

As well as fixing the socket problem upthread, this adds

*SysLog_Flush <logname>

which immediately flushes the named log, and

*SysLog_Flush

which immediately flushes all logs.

The logs are now written (if local) to <System$LogDir>; an obeyfile is included in the test archive to set the log directory.

The periodic flush task is started automatically when entering the desktop. This means that you can start SysLog in your boot sequence, by copying into !System.500.Modules.Test and adding

X RMLoad System:Modules.Test.SysLog

to the end of Choices.Boot.PreDesktop

If you have a Linux computer with syslog-ng, you can try out the remote logging by adding the following to /etc/syslog-ng/syslog-ng.conf:

source s_net {
  syslog(
    ip("0.0.0.0")
    transport("udp")
    port(514)
  );
};

destination d_riscos { file("/home/joebloggs/riscos.log"); };
log { source(s_net); destination(d_riscos); };

and of course put a

* Remote:192.168.0.1 0

line in your syslog client options file (replacing 192.168.0.1 with the address or name of your server). Then you can for example do *ReportOpt SysLog and see all of the Reporter output appear on your Linux computer.

and of course put a

* Remote:192.168.0.1 0

line in your syslog client options file (replacing 192.168.0.1 with the address or name of your server). Then you can for example do *ReportOpt SysLog and see all of the Reporter output appear on your Linux computer.

I don’t have a setup to test, but would that work with
* Remote:MySyslogSrv 0
where there was: either a DNS server entry for MySyslogSrv, or an entry in the client machine hosts file?

Yes it does :-)

Yes it does :-)

My kind of person – I have an intense dislike of embedded IPs even if they sit in a config file¹. The IPs of everything I get near are whatever the DHCP server hands out and the method of access is by name.
I think the entry might need to be MySyslogSrv.home(or appropriate domain) in some cases. FQDN can be an issue.²

Oops, my NAS is reporting disk errors in part of the RAID. Stuff to do.

¹ I have a long-running battle with medical equipment suppliers (a couple in particular) where they persist in doing 20th Century stuff.
One: DICOM does not require a manually configured fixed IP, DICOM requires a known relationship between the AETitle of the sending modality and the receiving server. DNS lookup at both ends works very nicely.

I am dealing with a nasty application crash where the whole machine gets thoroughly stiffed, and SysLog likes like it would be very useful. I have managed to set up syslog-ng on a Linux machine and after a bit of puzzling things out I’ve got it working across my network.

To hook it into my application’s existing debugging code it would probably be best to use the SWIs rather than the star command. Can you point me to documentation for SysLog_LogMessage and SysLog_Flush please?

Don’t worry. I have located a version of !SysLog on our Risc PC, complete with documentation and C veneers. That should be enough to get me started.

Just as a note- Andy S of ‘Paint update’ – modified RPCEmu to output ‘I think’ Function names to a file.

For future reference, here’s what I’ve implemented:

A log id is a zero-terminated string, maxmimum 10 characters.

There are no return values, and no errors are returned.

SysLog_LogMessage &4C880

On entry
   R0 -> log id
   R1 -> zero-terminated message
   R2 =  detail level (0 - 255)

SysLog_FlushLog &4C882

Flush a log immediately

On entry
   R0 -> log id, or 0 to flush all logs

SysLog_LogUnstamped &4C884

On entry
   R0 -> log id
   R1 -> zero-terminated message
   R2 =  detail level (0 - 255)

If logging to a file, then only the message itself is logged, without timestamp etc.

SysLog_LogData &4C88A

On entry
   R0 -> log id
   R1 -> detail level (0 - 255)
   R2 -> data
   R3 =  amount to read, in bytes
   R4 =  start offset, in bytes

SysLog_LogFormatted &4C88B

On entry
   R0 -> log id
   R1 -> zero-terminated format string
   R2 =  detail level (0 - 255)
   R3 -> optional parameter block (0 otherwise)
         +0 argument 0
         +4 argument 1
         +8 argument 2
        +12 argument 3
        +16 argument 8
        +20 argument 9
        +24 argument 10
        +28 argument 12
        +32 argument 14
   R4 = argument 4
   R5 = argument 5
   R6 = argument 6
   R7 = argument 7

If the parameter block is not given in R3, then the corresponding arguments will be taken as 0.
Format string:

A % sign can be followed by either '%' in which case a single '%' is output, or a formatter.

A formatter is 

  <argument number><specifier>

where argument number is one of 0 - 12 or 14, leading zeros accepted, and specifier is

  a | c | d | e | f | p<specifier> | s | x | z

The value of the given argument will be interpreted using the specifier as follows:

 a - (address) IP v4 address
 c - (character) ASCII character
 d - (decimal) Signed integer, as decimal
 e - (error) Unix error as a descriptive string
 f - (file) Full path for an open file handle
 p - (pointer) Address of value to be interpreted using following specifier 
               "(null)" if 0, following specifier ignored
 s - (string) Address of Control-character-terminated string
 x - (hexadecimal) Unsigned integer, as hexadecimal
 z - (zero-string) Address of zero-terminated string

Just as a note- Andy S of ‘Paint update’ – modified RPCEmu to output ‘I think’ Function names to a file.

I didn’t get as far as retrieving function names, though that would be very useful. It just logs the address of the function and the contents of the registers. It’s painfully slow because it only works with the interpreted ARM emulation.