A day with a RiscOS machine

Sorry, late to this discussion, but had to comment on this:

it took an inordinate amount of time for ChangeFSI to display even modest sized JPEGs. I must try some other packages in case it is just a ChangeFSI issue

Why on earth are you using ChangeFSI as an image viewer? It’s for image processing, hence it is quite slow.

but it is quite stark when the JPEG you want to look at opens in a couple of seconds with XP’s Quickview but takes around a minute each under RISC OS.

Use SwiftJPEG or one of the other viewers that use the built in OS routines. They display 8Mpix photos in under a second.

“Use SwiftJPEG or one of the other viewers that use the built in OS routines. They display 8Mpix photos in under a second.”
That prompts me to ask why ChangeFSI doesn’t use OS routines. Would it then not work on all versions of RISC OS, though apps like SwiftJpeg and PrivateEye must have to deal with that problem also.

That prompts me to ask why ChangeFSI doesn’t use OS routines.

Because, as Bryan already pointed out:

It’s for image processing, hence it is quite slow.

Which means that you use ChangeFSI for a high quality image conversion, and a quick viewer for exactly that.

ChangeFSI

It uses the boustrophedon transform with Floyd–Steinberg dithering, ¹ which achieves good results but is fairly demanding.

¹ How to find the closest colour, 30 Jan 92

“Which means that you use ChangeFSI for a high quality image conversion, and a quick viewer for exactly that.”
OK as long as the displayed image is needed at the higher quality, I guess, otherwise a quick and dirty display with the OS routines until a decision to save/convert has been made would be friendlier.
I used to hide !ChangeFSI to stop it from claiming the RunType, but I have hidden the lines setting the run types in the ChangeFSI Boot and Run this time round to see if that has the desired effect.
Other image programs use ChangeFSI so it does need to be seen, It resides in the Utilities folder so that should be fine.

Which means that you use ChangeFSI for a high quality image conversion

As long as you you’re happy for “high quality” to mean “getting the gamma correction completely wrong” (or have a linearly-calibrated monitor… which you don’t, if you’re using ChangeFSI for “high quality” anything).

ChangeFSI can’t even convert images to 256 colours correctly.

ChangeFSI can’t even convert images to 256 colours correctly

Ticket #236 seems to be the nearest relevant open ticket, but doesn’t seem to cover that. Should this failing be logged separately?

No, it’s my usual rant about gamma which few understand and almost no one bothers to get right.

Ask ChangeFSI to dither grey &7F7F7F (or &808080) into two colours (for example) and it’ll give you a checkboard of black and white, which is totally incorrect by a very large margin unless you have a linearly-calibrated monitor. The same problem affects all colour calculations made by anything Wilson wrote.

All my RISC OS machines have had linearly calibrated output since the hardware permitted as it’s the only way to get ChangeFSI, ColourTrans, FontManager etc to get the right result.

No, it’s my usual rant about gamma which few understand and almost no one bothers to get right.

Sources are available… You could fix this? :-)

No SPi is not dasy chained.

It can be, at least according to the diagram about half way down the wiki page. Though, if you have a 1:1 arrangement (1 master, 1 slave) it ought to be possible to hot-plug, except there is no “device connected” detection other than either rigging up a micro switch somewhere, or repeatedly polling the device to see if it replies.

SPi does have its limitations, though not as big as the problems of USB.

Speed being a big one. It looks like SPI to an SD card can run at 0-25MHz. Since native 4 wire SD interfaces can run up to 50MHz (maybe more, depends on the hardware) and has a higher throughput due to more data lines and USB can keep up with it (USB2 can read data from hard disc faster than a Class 10 card can write it), I think it is safe to say that SPI is not going to be the quick protocol here.
Now, don’t get me wrong, it’s going to be okay for talking to slower devices like keyboards. However the sort of demands I make of USB (live file system, copying video files hundreds of megabytes long, streaming analogue video into the computer in a digital form so my computer is a tv…), I really don’t think SPI would cut it.
Add to that, the lack of discovery protocol built in to SPI. It is just a shift register at each end, so how do you know what is connected, what it’s capabilities are, and what it requires? USB defines this, SPI does not.

Remember that the Cards are required to have a fairly advanced wear leveling system. And having looked up the specs that I could locate it seems that most current generation cards have greater than 50000 cycles, that is not shabby when combined with wear good leveling you would have to write more than half of the card 50000 times in 20 years (reasonable maximal time for a primary storage device).

Never that simple. Computers work with sectors, which are typically somewhere between 512 and 4096 bytes. Flash memory works with blocks, which are typically around 128K in size. Writing to different logical sectors may cause the same SD block to be moved. Note, also, that certain blocks (free space map, root directory, log files, etc) may be written extremely regularly, which may well trigger the entire block to be moved. All that for a mere few bytes written.

I found this: Adding spare erase blocks, say 3 per wear-leveling unit, those cards end up having around 37 erase blocks per wear-leveling unit. Given this scenario and a lifetime of 10,000 write cycles, you would end up with 370,000 write cycles on a particular wear-leveling unit, not counting additional stress caused by copying around used blocks as part of the actual wear leveling if you’re not simply writing to the same sector every time. Not exactly very much…
On my sd-card, I get roughly 900 writes per day for /var/log with a total filesystem size of around 5MB. Being pessimistic (i.e. assuming most writes end up in the same 4MB unit), this might result in a total lifetime of the card of 411 days. The underlying math is, of course, much more complex and depends on the distribution of writes but I assume the 411 days are not too far off in my use case. [http://plugcomputer.org/plugforum/index.php?topic=1074.0]

However, the problems of using SD cards as main storage are as follows:

1. Due to wear levelling, there are dangers of data loss. A smart controller will want to move blocks when the card is quiescent, perhaps maintaining a list of blocks to rearrange. It seems many base their calculations of available free space on the FAT. The implication being – what happens for a non-FAT filing system? (it seems journalling file systems trash SD cards quite quickly).
2. Due to the above, is there a way to tell the card when computer is going down? To move one sector, you will need to read a lot of others, combine them, and write a bunch back. Wear levelling doesn’t perform journalling, so how does the card know if the computer is quiescent, or being turned off?¹ And how long until you notice what data has been trashed?
3. As far as I am aware, there is no mechanism to report device health. Wear levelling will mask problems until the card fails.

¹ I know RISC OS does not correctly signal dismounts to USB devices. All of my USB devices with lights, the light goes off when I tell the Windows computer to eject the device. My hard discs go a step further and park and spin down. I have not connected a hard disc to RISC OS, but given that USB devices keep their light on, I don’t imagine the drive would shutdown.

I have, in my time, suffered three hard disc failures and three flash media failures.
Hard discs:

1. Started to fail more as it warmed up, so over the course of a week, powered up from cold and copied the data off. Some stuff lost, but only a handful of files.
2. I dropped the handle of a screwdriver directly onto the top of the drive. Instant death. The assorted wildlife learned how to say rude things in several different languages.
3. Sticky spindle. Got it running with some deft wrist flicks. Enough to copy off the data.

Flash now:

1. Flash in my MP3 player. One day the player stopped playing music. Looking in the status, installed memory is 0 megabytes.
2. MicroSD card in phone failed. Root directory and map and boot sectors trashed. Windows chkdsk could not fix. doschk on my PVR didn’t want to touch it (couldn’t ID the media type). Eventually found a program that was able to extract most of the data by looking through the raw data for folder signatures. Recovered most files, but some had really weird names; and files over about 64Mb were damaged. Used an SD formatter to reformat the device. It failed again a few days later, so I’m guessing it is now “faulty”.
3. I have a USB flash key that the computer won’t recognise. It goes in, the LED blinks endlessly, and Windows tells me that I need to insert a disc…

While it is possible to have sudden catastrophic hard disc failures (and why backing up is always important), I think that for age related issues, a hard disc should provide warnings of impending failure. A flash device, on the other hand, is likely to one day just cease working. Not a little bit, not damaged sectors, but entirely. The SD in my Pi is going okay, as are most people’s (else we’d hear a lot about this), but what is the anticipated service life? And what happens when that period expires?

That meens that you would have to replace most of the contents of your boot driive more than 6 times per day just to exceed the limit in 20 years.

See above. My eeePC runs on two SSDs, and I would love to think that they will run for a long time (no thanks to &@#%$ Avast!), but I know that if simple calculations were accurate, the service life would be quoted as 10-20 years, not ~3.

And now that computers have at least a few hundred MB of RAM there is no good reason for disk swapped memory.

# Let's Do The Time Warp Again...

These dinky little SoCs have a few hundred Mb on board. Big computers have obscene amounts for obscene wastage. I couldn’t get Firefox (with many non-active tabs) running at the same time as Thunderbird on my eeePC (due to SSD there is no swap so when memory runs out, it runs out). Got a 2Gb SoDIMM for €8 at a boot sale. Now it works…
Oh, and it's just a jump to the left...

Remember also that most things access SD Cards over SPi

No, they don’t.
Micro controllers and bit-banging will use SPI. Proper SD interfacing devices will use the SD four wire method.

though they are solid state so in some respects more reliable than an HDD which is a mechanical device.

I wouldn’t say more reliable, I’d say less fragile. I can drop a screwdriver onto an SD card and it likely won’t be the end of the world. I think the differences between flash and spiny-discs will be defined by their fault behaviour. I don’t think that there is really enough empirical data here yet – though looking online it appears that SanDisk 32Gb Class 10 cards have a worryingly high failure rate (and, as concerned me, defined by the device it is inserted in no longer recognising the card). I don’t know if these reports are for legitimate retail-packed SanDisk cards, or knock-offs with a SanDisk label…

IIC are SPI completely incapable of operating at the speeds and distances of USB, firstly crosstalk in the cable would destroy signal, USB avoids this using differential transceivers and twisted pair cable. Then they assume the round-trip time is less than one clock cycle, IIC requires a sender wait after every bit for a potential stop signal from the recipient, and SPI does not provide means for a master to recognise the start of a message from a slave. For USB 2 there could be an entire byte in the cable.

And there are more non proper implementations than there are proper, just look at the MCU based implementations of SD by SPI alone, it greatly outstrips the DONE PROPER market. So you said “No they don’t, Oh YES they do.”

That there are more bad (car) drivers than good does not really justify being bad. I would imagine that proper, licenced (as a consumer device would need to be to legitimately use the logo) SD interfaces far outstrip those home-brew and budget kit using SPI. Let’s start with digital cameras, add in mobile phones, then personal media players, and laptops/netbooks, and throw in multi-card readers for good measure.

To be honest, I think the primary use of SPI in home brew kit is because the proper method is proprietary and that carries boring complications. The secondary use of SPI in home brew kit is because it is simpler to implement, and you don’t really need to have epic throughput if you are storing data to be crunched by a 10MHz MCU.

Though a Class 6 SD card is still fast enough to keep up with raw HD video in real time.

Raw HD? Really? Given that RGB 4:4:4 at 1080P runs to around 150MiB/sec – I want one of those cards! Though, you might find hard discs better, as a 32GiB card will only hold around three minutes of video.

…you probably meant compressed HD…

And if for some reason 900Mb/s (aproximate limit before runing into big design isues with the AC signal)

Sorry, maths is not my strong point. What does that equate to in frequency? My rough “guesstimate” puts that at around 90MHz. What sort of cabling arrangement would you use at those speeds?

Why use USB instead of one of the better suited Serial busses out there?

More complete. The SPI bus is… basically a pair of shift registers. It isn’t that different to the stuff we were going thirty years ago with the 6522VIA.

USB offers…

• a clearly defined interface, both electrically and physically (even the colour of the wires is standardised)
• inexpensive (it’s only four wires to the attached device)
• insertion detection (a pull-up resistor, that also defines device speed)
• specifications for the permissible output current per port (that most SoC devices completely ignore, the spec says up to 500mA per port)
• a defined specification for device identification, a computer can probe what is connected and read various characteristics
• a defined specification for the control/endpoint stuff
• a set of defined protocols for things like mass storage devices and certain types of audio devices. This is why many USB flash storage devices will “just work”, and why my MIDI module doesn’t care if I am using my Yamaha keyboard with built in MIDI or the older keyboard with a MIDI dongle – they both look alike at the USB side.

SPI offers…?

Allow me to quote Wiki to make the point clear: “The SPI bus is a de facto standard. However, the lack of a formal standard is reflected in a wide variety of protocol options. Different word sizes are common. Every device defines its own protocol, including whether or not it supports commands at all. Some devices are transmit-only; others are receive-only. Chip selects are sometimes active-high rather than active-low. Some protocols send the least significant bit first.”

You can understand, I trust, that a de-facto standard that is actually a hodge-podge of sort-of-similar things is great for people working with MCUs that want to hook some bits together simply, but in a commercial sense it is utterly dead. Nobody wants to buy a “thingy” and wonder “will this even work with my computer”? At least with USB, driver issues aside, devices should plug in and work.
Example – I bought a USB DVD writer for a handful of euros in a boot sale a couple of weeks ago. I plugged it into my eeePC and I started up SMPlayer and then watched “Ted”. Didn’t need any specific drivers, the device identified as a DVD writer, piped out the MPEG data from the DVD, and it was all recognised and set up from the stuff built into Windows XP. It might (power issues aside) even work (at least as far as a CD/DVD drive) under RISC OS? Not going to try, it has a y-lead so it can suck power from two USB ports…

Anyway, SPI is largely “undefined”, and is dictated mostly by the device you want to talk to. I would imagine, looking at a few pages of info on SPI, that I could probably hook up a BBC Micro’s User Point to talk SPI through the VIAs Shift Register. Throw in some handshaking and we’d be good to go, right?

No, it’s my usual rant about gamma which few understand and almost no one bothers to get right.

Sources are available… You could fix this? :-)

Does anyone know whether the ROL forked version of ChangeFSI has addressed this? (Does it include ColourTrans changes?)

Does anyone know whether the ROL forked version of ChangeFSI has addressed this? (Does it include ColourTrans changes?)

It doesn’t.

Gamma doesn’t just affect colour matching (when changing from mode to mode), but all calculations involving colour components, including scaling and antialiasing.

Another ChangeFSI example would be downscaling a black and white checkerboard by 50% – it’ll give you &7F7F7F (or &808080), which again is badly wrong unless you have a linear output.

Sources are available… You could fix this?

Fixing ChangeFSI doesn’t fix ColourTrans.

Fixing ColourTrans doesn’t fix FontManager.

Fixing FontManager doesn’t fix BlendTable.

Fixing BlendTable doesn’t fix PhotoDesk, Artworks, ImageMaster, etc

The reason I advocate a linearly calibrated monitor is that there’s too much broken to fix, and having such a calibration means the expense of doing it correctly is avoided anyway. Win-win.†

The only down-side is dealing with all the icons that other people design on their 2.2 monitors. :-)

† I should point out that ideally one wouldn’t use non-gamma corrected RGB with only 8bits per component. Even 16bits is a bit grotty. I’ve never found banding to be a problem, but it theoretically can be if you’re using numerically linear as opposed to perceptively linear.