MidiMon 1.00, a USB MIDI app

I store my program sources on a linux computer. I have a samba share enabled on linux so I can access my programs from RISCOS using lanmanfs – like a network drive. I program in the normal way on RISCOS putting h and c files in their own folder on the share. The git stuff I do on linux using linux git or on windows via ssh. So on riscos you compile over the share. The main problem is you can’t run an application over lanmanfs because of the ‘!’ so I make the makefile copy it over to RamFS and run it from there.

Filetypes are not an issue – Lanman takes care of it if you are creating files on riscos otherwise you soon figure out how to filetype when creating a file on the linux end. You can make changes from other OS and use VNC to run/compile the program. There is a problem with Lanmanfs and timestamps that affects amu. I have a fix for that if you want it.

which means 0,0,0,0 is a valid midi command.

Where does it say that?

CIN=0 is reserved, which means it shouldn’t appear “in the wild”, which means what you’re going to encounter will be <non-zero>, <byte>, <byte>, <byte>.
Furthermore, I think that with the specific exception of SysEx data (which will have a 4-7 initial byte), 0,0,0 is not a valid MIDI command. It is self-synchronising (as original serial data) in that the control byte has the top bit set (which is why NoteOn is &9x) and the data bytes do not have the top bit set (which is why it’s all 0-127).
Which means, ignoring what the CIN byte may be, three nulls is just not a valid MIDI sequence.

Now if normal packets could be packed into a usb packet you need to know how many there are or what value is used to signify the end.

One would assume nulls, as is specified elsewhere in the spec. Unfortunately the spec goes into a lot of detail in explaining the data format of the four bytes, but not how that may be arranged in a 64 byte payload.

We know neither therefore we can only read one with confidence.

Which, as can be demonstrated by the two dumps from my keyboard, can lead to data loss.

The one where we walk the raw buffer is about twice as long, and while I’ve not pored over it, the values do seem to make sense and the numbers run in sequence (&B8, then &B9, etc for setting up the channels one after the other).

So, clearly my keyboard is sending multiple MIDI events in one USB packet.

If we assume we can use sysex data (ends) to determine when to stop reading midi packets in the usb packet then you have to have specified that that usb packet will have the sys end or the usb packet is full of sysex data (continues) in which case the sysex data (ends) will appear in another packet.

?

The only thing we can safely assume is that SysEx will remain SysEx until the end-of-SysEx packet is received. It could be followed by other MIDI events. SysEx is just like any other MIDI data packet. The only reason that it’s handled slightly differently in USB is because there’s no assumed length. You know NoteOn takes two parameters, while ControlChange takes one, and Pitchwheel takes two, etc. But SysEx is “until the end marker”. I guess they could have simply had “continue until &F7 and null pad it”, but by doing it the way it’s done makes it easier for the receiver to parse.

I just ran the SimpleTest a couple of times and…

1     3      &D, 64, 247    10261037           SysEx data (ends)
1     3      &F0, 67, 16    10261037  System : SysEx start
1     3      &4C, 8, 1      10261037           SysEx data (continues)
1     3      &D, 64, 247    10261037           SysEx data (ends)
1     3      &F0, 67, 16    10261037  System : SysEx start
1     3      &4C, 8, 2      10261037           SysEx data (continues)
1     3      &D, 64, 247    10261037           SysEx data (ends)
1     3      &B0, 7, 110    10261037  Control Change
1     3      &B1, 7, 50     10261037  Control Change

My keyboard has 1 jack so I get 1 midi packet per usb packet

That would make sense, but in the real world…

Please feel free to look at https://heyrick.eu/blog/files/keyboardinit2.txt and tell me where the reading is going astray and duping stuff, because if it is, I can’t see it. It looks to me like there’s actually multiple MIDI events in each USB packet.

because on the other module no bits set is ‘reset everything’

It should be doing that. There’s some code to pick out the various options for what buffers need cleared, and at the end there’s this:

      if (options == 0)
      {
         // Reset to initial state (sort of)
         midi_clock = 0;
         timestamp_type = TIMESTAMP_FASTCLOCK; // fast clock by default
         mididevice[port].bufferhead = 0;
         mididevice[port].buffertail = 0;
         mididevice[port].txptr = 0;
         mididevice[port].txbuffer[0] = 0;
         mididevice[port].lasterror = 0;
         mididevice[port].suppress_clock = FALSE;
         mididevice[port].rx_activesense = FALSE;
         mididevice[port].tx_activesense = FALSE;
         realtime_pass   = FALSE;              // don't put realtime into rx buffer by default
         realtime_ignore = FALSE;              // act upon realtime events by default
         clock_stopped   = FALSE;              // beat clock enabled by default
         mididevice[port].internalhead = 0;
         mididevice[port].internaltail = 0;
      }

The only thing is doesn’t do is clear the transmit buffer. Which it should. I’ve just added that.

Grrr… Just read the Acorn docs. It says that special mode bits (30 and 31) can only be cleared with a full module reset. So I’ll have to back out that change.

Teach me to fiddle with something without Ring the Fing M to make sure it’s a bug and not supposed to be like that. Comment added to make it clearer for the next time. ;)

The proof is in the pudding.

This was recorded by the “colin1midi” release from this morning. When I lined up my fingers on eight keys and pressed them as quickly as possible, then released them as quickly as possible…

0     3      &90, 65, 66    0         Note on  (channel 1)
0     3      &90, 60, 60    0         Note on  (channel 1)
0     3      &90, 62, 64    0         Note on  (channel 1)
0     3      &90, 64, 66    0         Note on  (channel 1)
0     3      &90, 67, 64    0         Note on  (channel 1)
0     3      &90, 69, 66    0         Note on  (channel 1)
0     3      &90, 62, 0     0         Note off (channel 1)
0     3      &90, 72, 0     0         Note off (channel 1)
0     3      &90, 65, 0     0         Note off (channel 1)
0     3      &90, 69, 0     0         Note off (channel 1)
0     3      &90, 67, 0     0         Note off (channel 1)

Eight keys pressed. Six note on events, five note off events.

Repeated with the current code that walks the buffer, eight of each.

So, clearly, when things happen at the exact same time, they will be sent in the same USB frame. So buffer walking is going to be necessary to avoid the risk of data loss.

(o’course, if the MIDI spec actually clarified this…)

Well, didn’t get anything done with SimpleSeq today, but the MIDI module is improving, so that’s all good.
Now time to go brew some linguine and enjoy a bit of Netflix.

Thanks, everybody, for your input and help. We’re getting there!

@colin: I also keep RISC OS stuff on a Linux machine, but I use NFS (!Sunfish) which solves some of the problems you’ve found with SMB, as long as you can get it setup with the flaky front end.

Thanks David I’ll look into that.

I agree with your solution Rick just not your reasoning. the 0 command is reserved for future expansion so could appear. There is also no reason to expect that after all the midipacket has been read that the usb packet padding bytes will be 0s – it’s not specified. It’s specified that the 4byte midi packet is 0 padded but nothing about the usb packet. I suppose it’s expecting the device to send unpadded data in the transfer. Note the padding must be coming from the device as riscos would pad to 64 bytes.

@Lauren

Tried your program with Ricks latest module.

Without the midi module loaded your program crashes so you need to rmensure his module in the !run file and die more elegantly if you don’t have it.

One keyboard which is just midi out the monitor monitored the keypresses. The test menu option crashed the program.
The other keyboard is midi in and out. It didn’t show any keypresses – Ricks test program did. But the on screen piano did work and I could change instruments and reset the keyboard.

Are there any ‘C’ music progs that need updating to 32bit – that can be downloaded?

the 0 command is reserved for future expansion so could appear.

Unlikely – they’ve released USB MIDI 2.0 and it’s not directly compatible. It’s bi-directional and switches a number of the seven bit values to sixteen bit values, so a MIDI 2.0 device will support MIDI 2.0 in addition to MIDI 1.0 (which is the default until negotiated otherwise).

So, I don’t envisage much happening with 1.0 now they have 2.0.

There is also no reason to expect that after all the midipacket has been read that the usb packet padding bytes will be 0s

That’s a fair point, so let’s look shall we?

In module_veneer_ticker_handler() is the following:

#ifdef RXDEBUG
         if (size != 0)
            debug("%02X %02X %02X %02X\n",ptr[0],ptr[1],ptr[2],ptr[3]);
#endif

I replaced this with the following:

#ifdef RXDEBUG
         if (size > 0)
         {
            // debug("%02X %02X %02X %02X\n",ptr[0],ptr[1],ptr[2],ptr[3]);
            int  linethrow = 0;
            char dline[80] = "";      // these are actually at the top of the
            char dpart[8] = "";       // function, but here for this example
            int  step = 0;

            for ( step = 0; step < 64; step++ )
            {
               // Dump the entire 64 bytes
               sprintf(dpart, "%02X ", ptr[step]);
               strcat(dline, dpart);
               linethrow++;

               if ( linethrow > 15 )
               {
                  debug("%s\n", dline);
                  dline[0] = '\0';
                  linethrow = 0;
               }
            }
         }
#endif

(the weirdness with sprintf/strcat is due to differences in how Reporter works compared to DADebug. Let’s just say that outputting a byte at a time went comically wrong due to the automatic prefix)

This allowed me to dump and look at the entire 64 bytes transferred each time, and here’s what turned up:

MIDI: module_veneer_ticker_handler (1277 - 1003786): header size 8, data length 64, tick 1126
MIDI: module_veneer_ticker_handler (1297 - 1003786): 0F FE 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003786): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003786): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003786): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: midi_real_time_message (1395 - 1003786): Deal with Real Time message &FE on port 0.

MIDI: module_veneer_ticker_handler (1277 - 1003801): header size 8, data length 64, tick 1266
MIDI: module_veneer_ticker_handler (1297 - 1003801): 04 F0 7E 7F 07 09 01 F7 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003801): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003801): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003801): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: midi_rxinsert (1363 - 1003801): returning packet { 03, F0, 7E, 7F }
MIDI: midi_rxinsert (1363 - 1003801): returning packet { 03, 09, 01, F7 }

MIDI: module_veneer_ticker_handler (1277 - 1003840): header size 8, data length 64, tick 1665
MIDI: module_veneer_ticker_handler (1297 - 1003840): 04 F0 43 10 04 27 30 00 04 00 08 00 06 00 F7 00 
MIDI: module_veneer_ticker_handler (1297 - 1003840): 04 F0 43 10 04 4C 02 01 04 00 01 10 05 F7 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003840): 04 F0 43 10 04 4C 02 01 04 20 42 11 05 F7 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003840): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: midi_rxinsert (1363 - 1003840): returning packet { 03, F0, 43, 10 }
[...etc many times...]
MIDI: module_veneer_ticker_handler (1277 - 1003846): header size 8, data length 64, tick 1726
MIDI: module_veneer_ticker_handler (1297 - 1003846): 0F FE 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003846): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003846): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003846): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: midi_real_time (1466 - 1003846): Real Time message received

MIDI: module_veneer_ticker_handler (1277 - 1003852): header size 8, data length 64, tick 1766
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B8 07 4F 0B B8 01 00 0B B8 0A 40 0B B8 0B 7F 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B8 5B 28 0B B8 5D 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: midi_rxinsert (1363 - 1003852): returning packet { 03, B8, 07, 4F }
[...etc...]

MIDI: module_veneer_ticker_handler (1277 - 1003852): header size 8, data length 64, tick 1768
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B8 47 40 0B B8 48 40 0B B8 49 40 0B B8 4A 40 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B8 00 7F 0B B8 20 00 0C C8 00 00 0B B8 65 00 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B8 64 00 0B B8 06 02 0B B8 65 00 0B B8 64 01 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B8 06 40 0B B8 65 00 0B B8 64 02 0B B8 06 40 
[...]
MIDI: module_veneer_ticker_handler (1277 - 1003852): header size 8, data length 64, tick 1770
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0E E8 00 40 0B B9 07 4F 0B B9 01 00 0B B9 0A 40 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B9 0B 7F 0B B9 5B 14 0B B9 5D 00 0B B9 47 40 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B9 48 40 0B B9 49 40 0B B9 4A 40 0B B9 00 7F 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B9 20 00 0C C9 70 00 0B B9 65 00 0B B9 64 00 
[...]
MIDI: module_veneer_ticker_handler (1277 - 1003852): header size 8, data length 64, tick 1771
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B9 06 02 0B B9 65 00 0B B9 64 01 0B B9 06 40 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B B9 65 00 0B B9 64 02 0B B9 06 40 0E E9 00 40 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BA 07 4D 0B BA 01 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
[...]
MIDI: module_veneer_ticker_handler (1277 - 1003852): header size 8, data length 64, tick 1774
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BA 0A 40 0B BA 0B 7F 0B BA 5B 00 0B BA 5D 14 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BA 47 40 0B BA 48 40 0B BA 49 40 0B BA 4A 40 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BA 00 00 0B BA 20 70 0C CA 21 00 0B BA 65 00 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BA 64 00 0B BA 06 02 0B BA 65 00 0B BA 64 01 
[...]
MIDI: module_veneer_ticker_handler (1277 - 1003852): header size 8, data length 64, tick 1775
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BA 06 40 0B BA 65 00 0B BA 64 02 0B BA 06 40 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0E EA 00 40 0B BB 07 34 0B BB 01 00 0B BB 0A 26 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BB 0B 7F 0B BB 5B 1E 0B BB 5D 1E 0B BB 47 40 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BB 48 40 0B BB 49 40 0B BB 4A 40 0B BB 00 00 
[...]
MIDI: module_veneer_ticker_handler (1277 - 1003852): header size 8, data length 64, tick 1776
MIDI: module_veneer_ticker_handler (1297 - 1003852): 0B BB 20 75 0C CB 1B 00 0B BB 65 00 0B BB 64 00 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
MIDI: module_veneer_ticker_handler (1297 - 1003852): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Can somebody else please try the same thing to verify that the unused data is in fact zeroed and it’s not just my keyboard? (though I’d have expected stuff like Rhapsody and SimpleTest to have gone a bit gonzo by now if this wasn’t happening)

Note the padding must be coming from the device as riscos would pad to 64 bytes.

The data length is 64, so maybe RISC OS is padding it to a packet size?

Without the midi module loaded your program crashes so you need to rmensure his module in the !run file and die more elegantly if you don’t have it.

Yup – either check for MIDI_Init, or call the SWI and check if V is set on return.

The simple tester does this:

SYS &604E2, 0 TO ; f%
IF (f% AND 1) THEN PRINT "MIDI initialise error - is module loaded?" : END

That’s calling “XMIDI_Init”, but in BASIC we can’t use the SWI name here as the name lookup will fail if the module isn’t present, so I call it by X-form SWI number and see if there’s an error back.

I think the 0.10d – the last version I’ve seen I’m losing track – is as good a solution as any. You are reading all the 0 commands treating them as valid and doing nothing with them because they are reserved and do nothing at the moment.

The alternative is to short circuit and break out of the loop when you first encounter a 0 which is treating it as invalid.

The difference is that if it is valid a different midi packet could appear after it.

There is no perfect solution. Just deal with the problem if someone complains :-)

My keyboard is padded with 0

The data length is 64, so maybe RISC OS is padding it to a packet size?

Mine is padded to 38 bytes with a 64 byte max USB packet size.

Love to know how they came to the value 38 when the midi frames are 4 bytes wide.

Just deal with the problem if someone complains :-)

Works for me. :)

Love to know how they came to the value 38 when the midi frames are 4 bytes wide.

Parity bits?

Nah, it’s easy. It’s 64 bytes. They’re just counting in base 19 and starting from the second entry because that’s array index [1].

What does ‘Endpoint’ mean – is it a fancy word for ‘File’.

What does ‘Endpoint’ mean – is it a fancy word for ‘File’.

Try Google – it’s quite helpful for things like this.

Must be me – after looking at Duck Duck go I’m just as wise :-(

@Colin Ferris
I used this website:
https://www.beyondlogic.org/usbnutshell/usb5.shtml#EndpointDescriptors

@Hi John et le Petit Four.

@MIDI Users
Very interesting the discussion on long packets like SysEx, my keyboard does not generate them and I cannot test them.

Rhapsody tests with the latest version of the MIDI module are good. The arrival of the data is well taken into account and ceals in a stable manner, nice.
So I was able to work on the capture made by Rhapsody.
Rhapsody (1998) was looking for version 2.00 of the module. I found in my belongings the MIDI Interface User guide for Archimedes. (I can scan it, to make it available…)

In this manual FastClock does not exist. so Rhapsody is expecting a microbeat timestamp… that’s why the notes don’t have the right duration at all.

I did the conversions from Fastclock (ms) to microbeat.
Test version of !RunImage to download, save the previous one :-)
https://jeanmichelb.riscos.fr/Download/Rhap4_runimageTest.zip

But there are still synchronization problems. The notes are offset from the tempo given by the metronome.
You have to anticipate “a little” :-) not easy…

About the MIDI module (Rick) I don’t really know where we are for the versions and the changes made.
I wanted to rebuild a Rhapsody version that would use the microbeat timestamp for testing, which was supposed to be the case with older versions of the module, but which never returned a timestamp in RXcommand.
I continue to adapt the Fastclock, if I can replace the microbeat version in Rhapsody.

What does ‘Endpoint’ mean

Simplest way is to think of it as another word for “port”, not unlike the ports used by internet servers (http port 80, POP3 port 110, etc).

So you have a device, and that device offers a number of ports. Unlike network ports, USB ones only work in one sense so we need two ports to work with a MIDI keyboard. One to send data to the keyboard, and the other to receive data from it.

my keyboard does not generate them and I cannot test them.

Try putting your arm on top of as many keys as will fit, then attempt to press them all at the same time. That’s what I did to test before I realised that my keyboard has an option to spew out it’s startup settings on demand (and really exercise the buffers!).

Rhapsody is expecting a microbeat timestamp… that’s why the notes don’t have the right duration at all.

Well, yeah, that would explain it. ;)

But there are still synchronization problems.

That’s okay, things can be fiddled with and lined up – but first the note durations. Are they now more what you’re expecting?

I can’t test your updated version at the moment, I’m in Bain-de-Bretagne getting my little VSPs ¹ insides fiddled with, and hopefully the ABS sorted out at long last. Kind of imagine this will be in the ballpark of €400…

Then I’ll walk around the Leclerc ² (Having run the fridge down so I can defrost it), and since it’s the start of my holiday, I might also visit Picard ³.

that would use the microbeat timestamp for testing

There isn’t one. At the moment the implementation of beat timing only responds to external timing events. So if your keyboard doesn’t blabla as much as mine, won’t work.

However, if you can convert from milliseconds to Rhapsody’s desired timing, it’ll be more accurate. The conversion isn’t that hard, just a bit fiddly in the maths.

¹ Voiture Sans Permis – my toy car.

² A bit like Tesco.

³ A bit like Iceland (the shop).

@Colin Ferris
Or a simpler version.

endpoints are what you ‘open’ when transfering data to/from a usb device. A device may have many endpoints – some IN some OUT. The device tells you the type of data it expects at the endpoints in its configuration descriptors. You can think of a descriptor as a C structure it’s just a way of accessing the configuration information on the device.

That’s the simpler version? 😂

An “endpoint” is a “port”.

Job done.

:-) depends what a port is

Has anyone tried a loopback to see what minimum latency they can get.

Is there such a thing as loopback on USB MIDI?

You don’t want to go via USB-Serial as so many of those things have terrible latency (even worse than would be expected restricting to 31Kbps sequential).

Well you know what I mean – I’m a newbee at all this.

My keyboard plays midi from a computer. It it also sends midi to a computer. It has voices that can be played from the computer that I can’t play from the keyboard so my newbee thinking is press key on keyboard, event goes to computer event goes back to keyboard sounds like loopback to me.

Hmm – one sea port usually
has lots of quays – coal,oil,timber and liners etc.

sounds like loopback to me.

Fair enough, but are your ears calibrated for the sorts of timings involved?

https://en.wikipedia.org/wiki/Precedence_effect

I didn’t envisage having sound from keyboard and midi I can silence the keyboard. It’s the delay between action and sound I’m thinking about.

Basically the minimum delay to play a midi synthesizer – if that’s the right term.

2 A bit like Tesco

but posher.

3 A bit like Iceland (the shop)

but posher.

Just my opinion based on more recent UK experience than Rick.

Watching this thread with interest; not getting kbd out of loft yet, but have located USB/DIN lead ready!