Let's get started with a Pandora port

So soon 8-) are you kidding? (snigger). I hope so. It’s running at the moment – I’ll see what issues remain and let you know :-).

Not for the faint of heart:

http://members.home.nl/janrinze/riscos/ro515.jpg

RO 5.15 running on the Beagleboard.

here it also shows there is a 255ms timer on the first USB hub.. might be the reason of the slow USB performance of the beagleboard.

Jan Rinze.

I haven’t had time to fully investigate that timer yet, but I have a strong feeling the OS ignores it. It’s a value that’s invented by the EHCI hub driver, specifying the update rate of the fully software-implemented interrupt endpoint that’s used to communicate between the hub driver and the main USB driver. And according to the USB 2.0 spec, a value of 255 is meant to be invalid – for high speed devices the number should be interpreted as 2^N microframes, and the range of N is (IIRC) 1-16.

I wonder if people have similar problems on an Iyonix. the issue is that SCSIFS does max 2.5 files per second. That means for example 1kb files are read at a pace of 2.5 files per second. this is quite annoying and makes the USB disk very slow.. If the problem exists for Iyonix users it can be assumed it lays in the common code, if not it may be in the HAL or the specific EHCI driver.

best regards,

Jan Rinze.

I think the Iyonix has exactly the same speed problem as the beagleboard. I tried running that modified version of RISCOSmark you sent me on my Iyonix, using a USB drive attached to one of the EHCI ports. Although I can’t remember what the exact results were I’m fairly certain they showed the same trend as on the beagleboard, i.e. small files transfer a lot slower than large ones.

I haven’t tried the OHCI ports yet. Since they’re USB 1.1 they certainly won’t be any faster for large files, but if they’re faster for transferring lots of small files then it would suggest that the problem is in the EHCI driver.

A brief update on the USB OTG driver: I haven’t had much time to work on it the past couple of weeks due to it being crunch time at work (guess where I am now? :P), but the driver is almost at the state where it provides operational peripheral-mode functionality. If you connect the board to a computer it will go through all the right motions to read the device details and assign it an address & configuration, but for some reason it doesn’t yet show up as an attached device. Hopefully I’ll be able to find the time to take a look at this tonight.

Once the basic peripheral mode functionality is done I should be able to add support for host negotiation protocol, then write the proper host-mode driver. I gave up on writing a driver which only supports host mode after I found that although I could force the controller into host mode using the test registers, it wasn’t reporting any interrupts for device insertion/removal – so it’s probably much better to go with the proper route of implementing peripheral mode & host negotiation protocol.

Jan de Boer (!A310 author) came by and brought his Iyonix. We tested things and somewhat to my surprise it yielded identical results. Not just roughly the same, no … identical!! Which means that the stack SCSIFS > SCSIUSB > DeviceFS > EHCIDriver has been timing locked. So if this is fixed both the Iyonix and the Beagleboard will benefit.

Jeffrey: With OTG on the beagleboard you need to make sure to have the latest u-boot installed. I would be delighted to get your code to compile and betatest :-)

Any chance to get in contact with Ben Avison?

Looking at the SCSIUSB there is a lot of potential misses in the SCSIUSB TickerV routine. GetOffTickerV(device) will remove all references to ‘device’ from the SCSIUSB tickerV linked list. But GetOnTickerV(device) does not check if there is already one placed on the SCSIUSB tickerV linked list. So potentially if the device was added twice it would generate a loop in the linked list.. also when one has finished it will remove both and potentially stop the SCSIUSB TickerV. so the second request never will get issued..

could be that this is desired behavior though.

I’m here. Six years on I can’t remember the details of SCSISoftUSB, but I’ll have a quick look at the source.

GetOffTickerV doesn’t remove all references to device, it exits when it finds the first one – the implication being that each my_usb_device_t block is only ever expected to be present on the list once. The linked lists are walked with interrupts off so they obviously don’t want to be any more complex than absolutely necessary – and since they’re static functions you know they don’t have to support any uses outside this C file.

Also I note that GetOnTickerV is only called from glue_DoCommand, which has a semaphore (device->command_active) preventing concurrent execution of more than one command for the same my_usb_device_t at the same time. In SCSI terms, it only supports a queue of depth 1. This means that GetOnTickerV should not need to check whether device is already on the list.

Thanks Ben,

I am still trying to find why both the Iyonix and the Beagleboard show that file access to the USB mass devices is timing locked. The max transfer rate is about 3MB/sec for huge files and the speed for very small files (1024bytes up to 16k) is 4 files per second. The latter is the most annoying. specially because RISC OS often has many small files.

So very likely the problem is a timer somewhere that none considered to have impact. This seems to be some ‘call every’ so that file reads are started on every 250 ms interval..

I really cannot lay my finger on it but there is a 128ms base in the EHCI driver, a 10 ms base in SCSIUSB , and some others that i could not track down on their settings. So each module is polling the next and this results in a weird cascade of timing values. For what its worth, raw reading from the EHCI can be done at more than 10 MB/sec from the same USB disk under RO5.

Sounds like there’s potentially a huge performance improvement if you can figure out the problem.

When SCSISoftUSB was first written, we didn’t have EHCIDriver yet, so it wasn’t possible to test the system at such high data rates. I would not be surprised if the problem is caused by timers being too slow or buffers too small to cope with USB2 data rates.

I’ve now made some worthwhile progress on the USB OTG driver. After getting DADebug working (not an easy task, since there still seemed to be instances where the board stopped responding to serial input) I managed to get some useful debug output, which confirmed my suspicions that all I was doing wrong was sending an incorrect device descriptor. So the end result is that peripheral mode now works (although at the moment the USB driver provides no real functionality as a peripheral). In the future I’m thinking that the board could appear as a hub when connected to another computer; that way multiple soft-device drivers could register themselves with the driver to provide additional functionality similar to the features provided by the Linux USB gadget drivers (network interface, mass storage device, serial port, etc.) I’m not sure if this is the same technique as used by the Simtec USB stack, or even by the Linux USB stack, but it seems like a sensible way to do things to me.

One thing that I’m thinking could be particularly useful would be the ability to debug the OS over the USB connection – have a simple peripheral-mode driver that implements a debugger protocol, allowing the memory & registers to be read & written, breakpoints to be placed, etc. It could also act as an additional interface for DebugLib output.

I also think I’ve worked out why I haven’t been making much progress with getting host mode working: I need to configure the TWL/TPS chip to enable the USB VBUS! If I boot into Linux and then reboot into RISC OS then VBUS is enabled and the USB controller is in host mode, and will trigger insert & remove interrupts when devices are connected. But if I cold-boot into RISC OS then VBUS seems to be off, and the controller will only work in peripheral mode.

The TWL/TPS (exact chip depends on beagleboard revision) is the same chip used to provide the RTC, audio, power control, etc. So I’ll probably have a go at hooking it up to the IIC module so that the chip is ready for use when we want to start using the other features (It’s either that or I add code to the USB driver to program the chip directly via the ULPI port)

Before I do this I’ll check my current code into CVS. Am I right in thinking that the MUSBDriver module should be under the Castle license, and placed in the ‘mixed’ folder in CVS? (since it relies on some of the NetBSD headers & code). I also need to register the module name – I’m guessing that the information here is still correct, and relevant to code being donated to ROOL? Is there anything special I should enter? (e.g. ROOL as the company name)

the same chip used to provide the RTC, audio, power control, etc. So I’ll probably have a go at hooking it up to the IIC module

The IIC module is just a front to SWI OS_IICOp these days (don’t get confused by browsing the trunk version – all current ROM builds use the HAL branch of the IIC module). OS_IICOp itself is a layer on top of the HAL IIC calls, so what you want is to implement IIC in the HAL. It’s not a good idea to have things like RTC or audio control having duplicate IIC drivers in them or calling out to the USB driver to do it for them!

Am I right in thinking that the MUSBDriver module should be under the Castle license, and placed in the ‘mixed’ folder in CVS?

You’ve said that it uses NetBSD code, but not whether it uses Tematic’s/Castle’s modifications and additions to the NetBSD codebase, which is significant.

It’s not the intention of the Castle licence to grab rights over your own original work, see clause 1.2.3:

If any part of any Derivative 
Work distributed by You can reasonably be considered an independent and 
separate work then this Licence shall not apply to any such part where You 
distribute such part as an independent and separate work

So you could licence individual source files which are all your own work under any compatible licence (the obvious alternative here would be to use the BSD licence so your improvements can be fed back to the NetBSD guys if you so wished). But as soon as there’s any dependency on Castle-licensed code then the module as a whole can only be distributed under the Castle licence.

Of course, if you can’t be bothered putting in the investigation work to see whether your code would build with vanilla code straight from the NetBSD project, then adopting the Castle licence is always a safe option.

As to where to put such a module in CVS - castle/ is intended for pure Castle-licensed code, and bsd/ is intended for pure BSD-licensed code (there is very little of that). mixed/ is a dumping ground for code to which more than one licence applies, and also for miscellaneous single-use licences (Desk, JPEG and so on). We’ve placed BSD-derived Castle code in mixed/ as an acknowledgement that there are subtle extra requirements on BSD code (maintaining the BSD licence header and in some cases mentioning the authors in “advertising materials” – something which we are possibly not doing as often as we maybe should).

I also need to register the module name – I’m guessing that the information here is still correct, and relevant to code being donated to ROOL?

Yes, it’s still up-to-date, and all old allocation addresses still redirect to us. Saying the allocation is for ROOL doesn’t really add much – the contact details are mainly there as a point of reference for any future queries regarding the allocation.

(maintaining the BSD licence header and in some cases mentioning the authors in “advertising materials” – something which we are possibly not doing as often as we maybe should).

The “advertising clause” is no longer relevant, for sources copyrighted by Berkeley at least:

ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change

I think the BSD folks have been working towards removing advertising clauses from other parties… this issue makes the BSD license incompatible with GPL and causes pain for anyone who has to market software (otherwise your adverts have to contain hundreds of lines of small print).

You’ve said that it uses NetBSD code, but not whether it uses Tematic’s/Castle’s modifications and additions to the NetBSD codebase, which is significant.

Good point!

• Snippets of NetBSD-licensed code from the USB drivers (e.g. the software side of the hub implementation)
• Code snippets and files that are Castle-licensed (e.g. the skeleton for the module frontend, assembler call veneers & functions as used in the EHCI/OHCI drivers)
• My own code

The code follows roughly the same structure as the other USB drivers – all the bits which interface with RISC OS are in Castle-licensed files, and all the bits which implement the USB protocol and talk to the hardware are in NetBSD licensed files. No code from a Castle-licensed file has been placed into a NetBSD-licensed file, or vice-versa. But in most cases the files would have been modified by me.

So as I understand it, the files containing code from Castle-licensed files will have to come with the Castle license, and the files contianing code from NetBSD-licensed files will have to come with the NetBSD license. Files which only contain my own code will likely be NetBSD licensed as well, since 99% of the time they’ll be relating to the USB & hardware driver rather than an interface to RISC OS.

Sounds like very solid reasoning by Jeffrey. This also makes it more clear for others how to go about to add BSD licensed sources to RISC OS 5. Technically using a separate dir like it is done in the USB drivers makes sense. It will help programmers to keep track of whar is allowed/possible and how to distinguish between the two. (even though licenses are part of the files anyway..)

Another pressing issue is how to keep NetBSD sources up to date with changes from the NetBSD source tree. It would be interesting and wise to have a method to get updates from the NetBSD tree in the form of patches. A well known tool ‘patch’ can apply those patches to the files. This way we can ensure that bug fixes and new features are going upstream in the RISC OS sources. These can be automated up to the point that patches fail. applying patches at regular intervals will make life easier and logfiles of the patches can be used to check if something went wrong.

I just discovered that UpCallV is called a huge number of times when using SCSISoftUSB. It may be that this is causing the delays on read/write. If these are handled only at centi second intervals then it would explain the slow speed of umass devices. UpCallV is used by SCSISoftUSB, DeviceFS , Buffers and FileSwitch. Since RAMFS does not seem to use either the Buffers,SCSISoftUSB or DeviceFS it seems that the combination of these many UpCallV calls may be causing the long delays when doing file access to umass devices.

I may be completely wrong here but i hope to find a cause for the slow umass file access.

‘patch’ is very sophisticated for sure, but please be more careful with your reasoning before starting on a run away paragraph. patch doesn’t even come close to being able to do all the things you claim, and it will rightfully reject failures on minor differences.

In many cases, well managed source control – CVS/SVN makes this much easier (and things like git allow distributed development), and differences can often be merged with relative ease and in a controlled manner. But this is also often not true as well, and even in the best case still requires manual intervention. You can see examples of this with the GCCSDK autobuilder, where patches regularly have to be updated (although the case there is reversed).

With a 4 year old port, there’s an excellent chance you’ll need to redo the port from scratch. Alternatively, you might be able to get away with checking out the sources from the time of the RISC OS snapshot, putting the RISC OS version over the top, then CVS updating.

I am not sure what you refer to that I claim that patch might be able to do.. patch is pretty simple and i don’t expect it to do anything else but applying patch files on source code and report if that succeeded or not.

CVS updating with the RISC OS snapshot might be tricky but worth a try.. 4 years of not keeping maintenance is indeed quite a challenge to catch up. Specially when some things get refactored so the non-NetBSD code might not work anymore.

I am not sure what you refer to that I claim that patch might be able to do

It’s pretty hard to read your post in other than the way I interpreted it. You glossed over a good deal of detail (presumably because you weren’t aware), and then just repeated exactly something I said in your defense, which doesn’t seem entirely honest.

This stuff is complicated – not beyond comprehension, but it must be done properly in a well thought out way, or it simply won’t work, or we’ll be left with hacks that cause more problem in future. Throwing out methodologies and tools as buzzwords doesn’t do anyone any favours.

I’m not trying to be harsh or anything, nor do I want to dispel your enthusiasm, but please post (a little) less, and be more careful in what you do say. If you don’t know, then ask – don’t assume or gloss over. That kind of thing has proven over and over to be harmful to RISC OS in the past.

If you want to try the whole CVS update thing, then by all means do so. I think this might give you a better appreciation for the issues involved.

The “advertising clause” is no longer relevant, for sources copyrighted by Berkeley at least

That’s good to know – I hadn’t heard about that. A little further investigation reveals that the NetBSD foundation moved all their code to the 2-clause BSD licence last year (it’s not clear whether that’s retroactive though) and Open BeOS’s FAQ says it doesn’t use the advertising clause. Still, we should really do a careful audit of all the BSD-derived code – a particularly thankless task.

I just discovered that UpCallV is called a huge number of times when using SCSISoftUSB. It may be that this is causing the delays on read/write. If these are handled only at centi second intervals then it would explain the slow speed of umass devices

Well, the UpCalls are a fundamental part of the flow control as far as I remember, so you need a certain number of calls. There aren’t usually many routines registered on UpCallV (if you run DebugTools, *Vectors will list them) and they are normally very fast to execute. That said, they probably shouldn’t need to be called excessively often.

UpCalls are always handled synchronously so your suggestion “these are handled only at centi second intervals” doesn’t make any sense. If an UpCall handler were to rely on callbacks to do their work then there might be an issue, but that just wouldn’t work at all for filing system code because filing system operations (load, save, copy etc) all execute in SVC mode and callbacks are suspended for the duration.

After a bit of a tidy up, the code is now in CVS. Just don’t expect too much from it; it’s still only the software portion of the root hub implementation (for host mode), and a simple peripheral-mode driver.

Next step is still likely to be getting I2C working. After a quick look through the source a few days ago I have a feeling I’ll be able to reuse a fair amount of the Tungsten HAL I2C driver, so hopefully I’ll have something working in the next few days.

I have started a patch set against the original NetBSD sources. After cleaning it up I will take the time to remove unnecessary changes and make them use #defines in a special riscosdefs.h file. That will allow the sources to be as close to the originals as possible. Next step will be doing incremental updates from CVS and fix where patching does not work anymore.

I can understand that this is not what most people here understand or endorse but for me it seems the only way to get a well supported USB stack. On top of that new features will emerge during this endeavor. Most importantly I will test the intermediate results.

Don’t wait up for results.. this will take some time.

No, that’s great. And moving RISC OS specific stuff elsewhere is a good idea. There’s another important piece from NetBSD – the internet stack, and a very similar approach will be needed for that, to be updated to the latest. I know that the ROL version has already had this kind of update, and is therefore somewhat newer than the RO5 one.

A brief progress update:

I2C is almost working. The code works, but it only works reliably if the debug output is enabled. With the debug code disabled, it has a habit of either completely stiffing the machine or getting stuck in a loop somewhere. It looks like there’s some timing-sensitive thing somewhere, as either enabling a small bit of debug output in the right place or using ‘TRACE ON’ before running my BASIC test app seems to cause the code to start working.

I think it’s about time I tried adding some more low-level debugging code. I don’t yet feel up to the task of writing a USB-based debugger, so instead I’ll have a go at configuring the beagleboard’s USER button to cause an FIQ when it’s pressed. That way, as long as FIQs remain enabled, I should be able to hit the button and get a register dump over the serial port, which is about as much as I can hope for at the moment.