MidiMon 1.00, a USB MIDI app

Is it possible that the midi device is returning transfers with 0 bytes

I don’t think so. My experience of USB device microcontrollers is that this is handled by the hardware. If there are no bytes in the IN endpoint to send, a NAK is returned to the host in response to the IN Token request. When data is finally available it will just stay in the endpoint until the next IN Token is received.

What does the usb_wakeuprx actual do ? I would have thought it needs to trigger sending IN Tokens at regular intervals until a packet is received. If not then it is up to us to send usb_wakeuprx calls at regular intervals until we get data.

I think this will happen automatically for Interrupt endpoints governed by the bInterval field in the endpoint descriptor but this field is ignored for Bulk endpoints, so the USB spec says.

I got the first part a bit wrong, the micro will respond with NAKS until told to send data in the endpoint by the device software. So it is possible to send zero length packets. It’s just that I’ve never done that.
I know that my keyboard doesn’t send empty packets as I debugged it at packet level on the PC when I was first looking into this. I wouldn’t know about Ricks.

I don’t think so. My experience of USB device microcontrollers is that this is handled by the hardware. If there are no bytes in the IN endpoint to send, a NAK is returned to the host in response to the IN Token request. When data is finally available it will just stay in the endpoint until the next IN Token is received.

Yes that is the normal way but as I said I have seen a device where this was configurable and I think defaulted to not waiting for data.

What does the usb_wakeuprx actual do ?

It’s equivalent to request_transfer(min(r3, sizeof(bufman buffer free space)). If a transfer is in progress – which it is in UpCallRXDataPresent – it queues the transfer for when the current transfer is complete. Only 1 request_transfer is queued. so call it 10 times and only the last is queued.

The process starts on line 3029 in this file, where it sets str→delayed_read and str→nextcount and calls start_read() at line 2510. In start_read() if usb is busy it returns otherwise the transfer is started. When the current usbtransfer completes start_read is called again and the transfer queued by wakeuprx is run.

I wouldn’t know about Ricks.

You mean Jean-Michel? My keyboard is fine, because it never shuts up.

Only 1 request_transfer is queued.

So there’s no harm in dropping it in there too see if it improves things.

Anybody know why MIDI, dealing with a series of four byte packets, decided to use the bulk method rather than the interrupt one as would be expected by serial data?

Anybody know why MIDI, dealing with a series of four byte packets, decided to use the bulk method rather than the interrupt one as would be expected by serial data?

I think it’s meant to carry large amounts of data as well as just notes etc. The MIDI spec defines the DownLoadable Sound format (DLS files) which can be big.

Ok so I plugged in a keyboard run the midi module which showed the keyboard detected. Run simpletest in a taskwindow and nothing. Compiled a debug version of the module with the RXDEBUG flag set and a couple of lines appeared then nothing. I was getting buffer overflow on all keypresses – the keypresses were being received. Tried increasing the callback buffer size but had a few crashes and still no output through the simpletest though I wasn’t getting overflow anymore. Have a feeling the buffer wasn’t emptying. I’ll investigate further.

run the midi module which showed the keyboard detected

Which MIDI module? Mine or his?

with the RXDEBUG flag set

Sounds like one of my options, but good to clarify.

yours

Okay, cool.
I’ll hold off doing anything until you’ve had a rummage. It’s a bit hard for me to trace faults when there aren’t any, but if you have some hardware that isn’t playing, it’ll be much faster/easier for you to poke around and see what’s up.

Hi all MIDI users
This morning I managed to obtain data from the keyboard, note + timestamp and even tested real-time input with Rhapsody, this is the first time!

But…now it doesn’t work anymore:-(
I created a debug version of the module and in fact the upcall does not take place. I have to reset the keyboard several times.
USB buffer not filling up?

Colin, thanks for your help and I hope you can clarify the issue.
Rick’s module will only be better.
The ARMX6 may react differently than a Pi4?

@Rick I tested the Simpletest program with Peter’s module and I still have the right information.

Here’s a question to ponder. When you start a score in Rhapsody, there’s nothing. And unlike Maestro there’s no way to specify a tempo for the score (that I have found) except to manually add a tempo marker to the score.

I spent a lot of time trying to get my keyboard to work, and when it works the result in Rhapsody is not good except for the pitches of the notes… I hope to find it.

I created a debug version of the module and in fact the upcall does not take place

Did you add -DRXDEBUG to CDFLAGS in the makefile.

But…now it doesn’t work anymore:-

I too am experiencing intermittent problems. Initially I wasn’t getting ticker events and the buffer was overflowing simpletest did nothing, now the ticker event is working and I’m getting data aborts when I press a key while the ticker_handler is processing the buffer – still nothing appears in simpletest. The odd thing is the change happened with Ricks compiled module too. I’ve not seen a problem with usb receiving data.

I don’t know anything about the midi standard so perhaps Rick you can explain the output below.

The upcall receives 38 bytes per transfer and you process it 4 bytes at a time in the ticker handler. Looking at the data on the ‘returning packet’ lines, could the data be decoded on a variable length basis – I’m not sure if the ‘padding’ bytes in each 4 byte command are valid packets – if you see what I mean. In any case 4 doesn’t go in to 38 so something doesn’t look right.

MIDI: module_veneer_upcall_handler (1535 - 16018): Upcall: 38, 8, 8
MIDI: module_veneer_ticker_handler (1261 - 16019): used_space = 46
MIDI: module_veneer_ticker_handler (1269 - 16019): header size 8, data length 38, tick 37903
MIDI: module_veneer_ticker_handler (1275 - 16019): 09 90 30 01
MIDI: midi_rxinsert (1295 - 16019): Message count is &03 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 03, 90, 30, 01 }     4
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1365 - 16019): Written data to empty buffer, generating MIDI NewByte event.
MIDI: midi_rxinsert (1295 - 16019): Message count is &01 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 01, 94, 00, 00 }     6   2
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1295 - 16019): Message count is &03 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 03, 7F, 20, FA }     10
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1295 - 16019): Message count is &02 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 02, 03, 03, FC }     13  1
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1295 - 16019): Message count is &03 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 03, 7F, 20, FA }     17
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1295 - 16019): Message count is &03 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 03, 01, 97, 20 }     21
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1295 - 16019): Message count is &01 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 01, 00, 00, 00 }     23  2
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1295 - 16019): Message count is &03 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 03, 92, 00, 00 }     27 
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1295 - 16019): Message count is &03 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 03, 00, 97, 20 }     31
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
MIDI: midi_rxinsert (1295 - 16019): Message count is &02 to port 00
MIDI: midi_rxinsert (1352 - 16019): returning packet { 02, 03, 03, FC }     34 1
MIDI: midi_timestamp (1497 - 16019): Applying timestamp to command:
MIDI: midi_timestamp (1505 - 16019): MIDI beat clock (0).
18:57:42.47 ** Error **
  Error  : &80000002
  Message: Internal error: abort on data transfer at &209724C4

Port  Bytes  Data         Time     Annotation
----  -----  -----------  -------  --------------------------------------
0     3      &90, 59, 17  0  Note on  (channel 1)
0     3      &80, 59, 46  0  Note off (channel 1)

Don’t know what happened in the printout earlier but I did a dump in the incoming packets and they were just 4 bytes followed by 34 0’s I’ll have to look for the header.tick time tommorrow. I changed the code to treat the incoming packet as a byte stream, as opposed to a word stream that ends with a 0 first byte, before I discovered this. I just need to generate a code that uses less than 4 bytes to see if that is correct.

I don’t know anything about the midi standard

MIDI sends packets that are four bites long. There’s a prefix byte (specific to USB) that provides additional data as, IIRC, USB can support multiple “cables” (because it’s the 21st century and we can do better than being limited to 16 channels per device).
This is then followed by one to three bytes of MIDI data, in the same format as always, null padded for the remaining bytes.
MIDI data is always 32 bits in length.

Page 16 of https://www.usb.org/sites/default/files/midi10.pdf explains what to expect.

And yes, you’ve discovered the problem with the padding. ;)

I’ll have to look for the header.tick time tommorrow

To get the timing working, there’s a variable at the top that is set to something like CLOCKTICK_MIDIBEAT, change it to the one that means the fast clock (probably …_FASTCLOCK, but check in the header) and it’ll default to the millisecond ticker. I’ve done that in the one I posted here for JMB.
Sorry I can’t be more specific. It’s Saturday, first day of my summer holiday, I’ve only just got up, haven’t yet inserted any tea… ;)

But what I am sure about is MIDI being 4 bytes / 32 bits. Refer to the spec linked above for details.

The MIDI spec defines the DownLoadable Sound format (DLS files) which can be big.

Are these actually sent over MIDI? If so, they ought to be broken into chunks 64-128 bytes in length and then sent byte by byte as SysEx. Which means a ton of four byte USB packets. ;)

There appears to be an option “Association Control” that can be used for high speed data transfer, but as far as I can determine from the brief description, it appears to hijack the endpoint (effectively replacing the MIDI stream). I’m not sure if it is undoable either as the docs seem to imply that the endpoint is terminated at the end (so reconnect in order to get MIDI back?). At any rate, that’s not MIDI… and to be honest, it probably should have been something like MTP working alongside streaming MIDI rather than sort of on top of it, maybe. The docs aren’t clear (how does one even know the remote device supports this?) because whatever it is, it isn’t MIDI. ;)

Follow-up – Yamaha keyboards can often support a program called “Musicsoft Downloader” to send data to the keyboard and read it back.

It uses painful amounts of SysEx…

https://sandsoftwaresound.net/hidden-arranger-smarts-file-system-sysex/

Something I’ve noticed while I was working:

It receives data from my keyboard, and displays the manufacturer name (in place of the product name). So far so good

In USB MIDI 0.10, it seems to always return the manufacturer name even if the product name is available; for instance, here is the output from *MidiUSBInfo for my two devices:

*midiusbinfo
MIDI USB information:
  Timestamp type is MIDI clock
  MIDI Clock  is 0 (Song Position = 0)
  Current TX channel is 0 (real channel 1 on real port 0).
  MIDI device (#0) is USB13, VID 1C75, PID 0601,
  Manufacturer Arturia
  Product      Arturia MicroFreak
  using IN file handle 254  Endpoint 1  Size 64,
  and  OUT file handle 253  Endpoint 2  Size 64.
  number of cables 1.
  MIDI device (#1) is USB15, VID 1A86, PID 752D,
  Manufacturer 
  Product      USB2.0-MIDI
  using IN file handle 252  Endpoint 2  Size 32,
  and  OUT file handle 251  Endpoint 2  Size 32.
  number of cables 1.

Using 0.08, the second device there previously would show “USB2.0-MIDI” whereas now it’s the blank string from Manufacturer. Similarly, the MicroFreak always just shows as “Arturia”. This does make me realize though that I need to put in a fall-back to use the device number if the manufacturer is also blank.

I need to get caught up on this thread but similarly, I haven’t had any luck so far with multiple devices yet; if there are any device changes, so far both devices just stop working with both the test program and !MidiMon until the module is restarted. But I haven’t looked into it much more than that yet.

Using 0.08, the second device there previously would show “USB2.0-MIDI” whereas now it’s the blank string from Manufacturer.

There’s been some private discussion about this as some devices don’t bother to set up the information correctly. It’s not unique to MIDI, mind you…

I would say, as that call is specifically providing product and manufacturer IDs, that if the product name is blank, we fix it up by constructing a fake name, so software using this information can get something sensible (we shouldn’t leave the product blank, that’s just kicking the can down the road – remember that this stuff is exposed via an API call to read the device info).

Note: If the product name is the manufacturer ID (I’ve seen this with cheap keyboards), then we’ll do nothing, just fake up a product name if necessary. It’s not up to us to try to make sense of the info, just to make sure that there’s something useful in the product field as that’s the most likely to appear on screen.

It’s not ideal, but it’s not our job to fix other people’s half-arsed implementations.

Right, although maybe I’m not understanding but in this case it’s not that the product name is blank, it’s that it’s always returning the manufacturer ID no matter what…

Also, I’m poking at trying to get monitor input working with MidiSupport but I can’t seem to clear the buffers or get any kind of events for incoming MIDI… it looks like MidiSMon is using some MidiSupport specific SWIs instead of using the Acorn ones directly? Not sure. it’s late here so I’ll probably have to leave it tonight.

@Rick

Sound font or sample files DLS,SF2 etc. are binary. To send via midi protocol (7bit data) would be tedious to say the least (I’d say stupid). There’s probably a number of methods from a number of manufacturers but in answer to your question,
“Why Bulk and not Interrupt”, basically, I don’t know, the requirements seem identical with a UART link. Hey ho, just another one of life’s little mysteries.

@Lauren

MidiSMon does indeed talk directly to Midi Support and is displayed as a driver in the routing window of MidiMan.
Just make sure you have the drivers loaded and routed using MidiMan.
The main thing that can mess things up is if you have MidiSupport and Ricks USBMidi running at the same time. When it starts, MidiSupport should replace any MIDI module by its own MIDI module but it’s easy to then load another MIDI module afterwards, and of course they all report as MIDI !!

Ok so the 0 time appears to be correct in the default mode. In case I’ve broken something would anyone like to try colin1midi. Just double click on ‘runsimpletest’ the module will load and program run.

Right done that, seems to work fine.

Simple MIDI test
================


Play something, ESC to quit.


Port  Bytes  Data         Time     Annotation
----  -----  -----------  -------  --------------------------------------
0     3      &90, 60, 98  0  Note on  (channel 1)
0     3      &90, 60, 0   0  Note off (channel 1)
0     3      &90, 62, 106 0  Note on  (channel 1)
0     3      &90, 62, 0   0  Note off (channel 1)
0     3      &90, 64, 109 0  Note on  (channel 1)
0     3      &90, 64, 0   0  Note off (channel 1)
0     3      &90, 65, 109 0  Note on  (channel 1)
0     3      &90, 65, 0   0  Note off (channel 1)
0     3      &90, 67, 106 0  Note on  (channel 1)
0     3      &90, 67, 0   0  Note off (channel 1)
0     3      &90, 69, 106 0  Note on  (channel 1)
0     3      &90, 69, 0   0  Note off (channel 1)
0     3      &90, 71, 103 0  Note on  (channel 1)
0     3      &90, 71, 0   0  Note off (channel 1)
0     3      &90, 72, 110 0  Note on  (channel 1)
0     3      &90, 72, 0   0  Note off (channel 1)
Escape

*midiusbinfo
MIDI USB information:
  Timestamp type is MIDI clock
  MIDI Clock  is 0 (Song Position = 0)
  Current TX channel is 0 (real channel 1 on real port 0).

  MIDI device (#0) is USB7, VID 0A4D, PID 129D,
  Manufacturer Keystation Mini 32
  Product      Keystation Mini 32
  using IN file handle 252  Endpoint 1  Size 64,
  and  OUT file handle 251  Endpoint 2  Size 64.
  number of cables 1.
*

Bonjour,
@Colin
After resetting my Keystation keyboard, simpletest works, but there is no timestamp, fastclock is running.
With Rhapsody steptime capture works, it does not request a timestamp.

One positive thing is that now RxCommand returns the note, but the timestamp (FastClock value) is not added.

I am continuing the tests to see which version of the module (Rick) will be the most effective with my keyboard.

For information about the MidiSupport philosophy
https://web.archive.org/web/20160102211118/http://www.somascape.org/riscos/midi/ms.html

@Peter: Nope, have been doing a clean boot between switching between MIDI modules. Still no luck on input, though. MIDI_Init doesn’t empty any of the buffers, no events ever fire, and Rx_Command doesn’t seem to fetch anything… it’s probably something on my end I’m sure but dunno what yet. Output works fine though!

@Peter
It’s good that we have the same keyboard,the comparisons will only be better.
What result with usbmididebug?
What surprises me is that MidiClock remains at 0 while the fastclock is running.

 SYS “MIDI_FastClock”, -1 TO, a%

 PRINT a%

For SysEx, I think ESP had already done the job (lost, gr!!!) I tested their SysExy program with the ESP version of MIDISupport and it works with MIDI files that have SysEx data, the KeyStation keyboard does not generate any to my knowledge.

On the other hand, Rick’s keyboard generates a lot…

@Rick Bonne vacances.

@Lauren
What keyboard do you use?

With the Keystation keyboard, I had to do a full reset of the keyboard itself:

To send a Full Reset message:

1. Make sure your computer is running.

2. Switch on the Keystation Mini 32 MK3 by plugging in the USB Cable from your computer while simultaneously holding down the Octave “+” and “-” buttons.