Benchmarks

Have you tried testing RO5 on the VRPC – to see how it compares with RO4?

Have you tried testing RO5 on the VRPC

I didn’t think anyone had got RO5 running on VRPC, or have I forgotten something?

Have you tried testing RO5 on the VRPC – to see how it compares with RO4?

No. I just wanted to update the VRPC benchmarks to the lastest hardware a i5-6600 at 3.5GHz with a WinMark of 7.9 for storage and 7.6 for processor and 7.0 for graphics.

Thanks for the update. Any chance of completing the Pi3 ‘real world’ tests?

A message for Chris.

“Patch for RISC OS 5.24: This archive applies two changes (to an internal bus speed, and SATA PHY calibration) to the RISC OS 5.24 ROM release which users can apply. The changes are already in ROMs dated 12-Sep-2018 or later. "

And on CSAA: “We discovered a difference between what the datasheet and the chip think are in some clock registers, so have been able to speed up an internal bus by 100%. The processor core still runs at 1500MHz, but when data is moved around the bus that now runs twice as fast – particularly useful in large screen modes when there are lots of pixels on the move.”

https://shop.elesar.co.uk/index.php?route=product/product&product_id=51

It would be cool to have an update benchmark on http://www.svrsig.org/HowFast.htm to integrate this patch.
(and RISC OS 5.24 tests too)

And on CSAA: “We discovered a difference between what the datasheet and the chip think are in some clock registers, so have been able to speed up an internal bus by 100%. The processor core still runs at 1500MHz, but when data is moved around the bus that now runs twice as fast – particularly useful in large screen modes when there are lots of pixels on the move.”

It would be cool to have an update benchmark on http://www.svrsig.org/HowFast.htm to integrate this patch.

Chris’ table is based on RISCOSmark which runs each test serially. While this free update gifts Titanium with a handy 100% bus speedup (peak throughput was 4GB/s now 8GB/s) it’ll only likely be reflected as a smallish uptick (~15%) in the graphics acceleration in 16M colour modes – none of the other RISCOSmark tests were saturating that particular bus.

none of the other RISCOSmark tests were saturating that particular bus

Graphic tests are done in 1920×1080C16M, so the results could perhaps be a bit better now :)

I have updated the benchmarks on http://www.svrsig.org/HowFast.htm to include the Raspberry Pi model 4 (4G).

I have updated the benchmarks… to include the Raspberry Pi model 4 (4G).

Am I alone in feeling that the Pi4 results are more than a little disappointing? Of course it’s early days still in the OS port, but from the graphics aspect particularly (rectangle copy, artworks apple rendering) the 4 is only 25% or so faster than the 3+, and blown into the weeds by the Titanium. I have no user knowledge of the Pi4, and the ‘real-world’ experience may be quite different; though again the benchmarks suggest otherwise.

Of course it’s early days still in the OS port, but from the graphics aspect particularly (rectangle copy, artworks apple rendering) the 4 is only 25% or so faster than the 3+, and blown into the weeds by the Titanium.

For rectangle copy/fill the Pi uses a generic DMA controller for hardware acceleration. It gets the job done, but it’s probably not tuned for maximum performance like the GC320 2D accelerator that’s used in the Titanium.

Am I alone in feeling that the Pi4 results are more than a little disappointing?

I don’t use pi’s much but found little difference for my usage between the pi4 and my B+. VNC may be slightly snappier using a usb ethernet adapter in the pi4 but was a little surprised that the usb dwcdriver showed virtually no improvement – not that anyone really wants to use the dwcdriver. USB audio still struggles at high res and generally usb (dwcdriver) suffers from the same problems that are there in the B+. Really looking forward to the XHCI driver and some decent USB performance which is the main reason to get it.

Am I alone in feeling that the Pi4 results are more than a little disappointing?

You are alone :-)

The currently available benchmarks show (at least) two things (to me):

First, the CPU provides a welcomed speed up, as well as the RAM. That a board at a price point of roughly 10% of a Titanium board can be so fast is a good thing.

Second, there are certain areas that have a lot of potential for further speed up: the graphics driver, and the Ethernet and USB3 area. Whether a USB3 (that’s the XHCI driver Colin refers to) connected harddrive will rival the speed of the S-ATA stuff on the Titanium will be interesting to find out. Same for the internal Gigabit Ethernet.

Based on past experience, the graphics driver acceleration will make little real world difference, but the Ethernet and/or mass storage speed will make a world of a difference (see real-world comparison of Titanium vs. ARMX6 – despite a big difference in CPU speeds, the ARMX6 does not feel slow at all).

You are alone :-)

Just call me Greta (not Thunberg, the other one)! But thanks for the insights, and I agree entirely with your point about relative prices.
EDIT: on the USB3 vs. S-ATA point, just looked up the headline speeds, which are 5 Gb/sec for the former and 1.5 to 6 Gb/sec for the latter; so should be close.

The important thing with the RPi 4 is to use some sort of cooling so the CPU is never throttled. Also update to the latest onboard firmware as this has various fixes to help reduce throttling with no cooling.

Results below are for actively cooled Rpi4 @2147MHz. Average CPU temperature is 42 C.

RISCOSmark 2.04 (30-Dec-2015) by Richard Spencer 2003
Comparison with RiscPC SA 202MHz running RISC OS 4.02 800x600,256
(HD benchmarks are in kilobytes/sec)

MOS Utilities                   5.27 (12 Feb 2020)
FileSwitch                      2.87 (12 Oct 2019)
C Library                       6.03 (27 Jul 2019)
USBDriver                       1.29 (19 May 2018)
FileCore                        3.75 (06 Jul 2017)
Shared Sound                    1.20 (18 Jun 2016)
SCSIFS                          1.35 (20 Jul 2018)
LanManFS                        2.62 (04 Jun 2019)
ShareFS                         3.59 (11 Sep 2016) Access+
SharedUnixLibrary               1.15 (10 Feb 2019) (C) UnixLib Developers, 2001-2019
HAL-based system (BCM2835 VDU device) tested at Mon,24 Feb 2020.20:01:14
Filing system: SCSI:HardDisc4.$
Graphics Resoloution: 1920x1080, 16M colours

Test                                                   Benchmark
Processor - Looped instructions (cache)         5291841             2975%
Memory - Multiple register transfer               82001            50617%
Rectangle Copy - Graphics acceleration test        1340              553%
Icon Plotting - 16 colour sprite with mask        55127             2756%
Draw Path - Stroke narrow line                    13949              894%
Draw Fill - Plot filled shape                     10671              731%
Draw Render - Render draw file                     5637             1725%

SSD - USB 2.0
HD Read - Block load 8MB file                     32768             1098%
HD Write - Block save 8MB file                    35617             1171%
FS Read - Byte stream file in                       497              240%
FS Write - Byte stream file out                     504              262%

RAM Disc

HD Read - Block load 8MB file                    421766            14143%
HD Write - Block save 8MB file                  1403184            46142%
FS Read - Byte stream file in                      4902             2368%
FS Write - Byte stream file out                    5204             2710%

SD Card UHS-1 A1

HD Read - Block load 8MB file                     23405              784%
HD Write - Block save 8MB file                    19819              651%
FS Read - Byte stream file in                       999              482%
FS Write - Byte stream file out                     372              193%

Comparison with RiscPC SA 202MHz running RISC OS 4.02 800×600,256

Shouldn’t this maybe to be updated these days to a modern baseline like Pi1 or Pi2? Because it’s kind of insane to come up with speeds that are fifty thousand times faster [edit: no, I suck at maths] than the RiscPC; especially given that the RiscPC’s capacity for data transfer is pretty much a joke when compared to anything released in this epoch.

…speeds that are fifty thousand times faster than the RiscPC;

It doesn’t trouble me, personally, it’s just a metric. Comparing the memory speed above with say, a Titanium @ 20,870%, would it make a difference if the comparison was 5350% vs. 2206%, which would be the result of re-baselining on the Pi Zero? Actually, talking of the RiscPC, one of the strange things about the perception of speed, looking back, is how subjective it is: I remember upgrading my RiscPC from ARM710 to StrongArm and being utterly astonished at the latter’s blinding speed! I’ve had a similar sensation with every major step forward in hardware, and yet most of these machines moved at a snail’s pace compared with modern kit.
(PS. is it not actually c.480 times faster [50617% divided by 105%]?)

Note to self, if one sucks at maths, one shouldn’t write posts with maths in them when one is tired, or one will end up looking a right ass.

Yes, it’s a percentage, not a multiplier, so… 506x faster (why is the baseline not 100%?).

would it make a difference if the comparison was 5350% vs. 2206%, which would be the result of re-baselining on the Pi Zero?

It enables a simpler comparison between the two, certainly.

Or to put this into context, consider your car. Can probably manage around 120 without bits falling off, if it really needed to. My car? Downhill with a tail wind it might manage something around 35 or so.
If our baseline was a generic Audi (Arya singing optional!), then my car would probably manage a few tens of percent. Your car? Depends upon the car, but likely in the ballpark of 100% (either side). Therefore one can tell pretty quickly the difference between the two.
Now, if our baseline was the Shuttle on reentry (a mere 17,500mph give or take a few), then how would the cars fare? How many zeros after the decimal point for mine….and yours?

It’s back to front (the baseline being vastly quicker) than the speed tests here (baseline being vastly slower) but it is used to illustrate the point that the more divergent the baseline from that which is being tested, the less useful the percentages become. After all, if we’re comparing with a machine that’s nearly a quarter century old, then most things are going to start to show ridiculous numbers like the fifty thousand odd; which makes comparing two modern machines based upon an older machine harder than it has to be.

is how subjective it is: I remember upgrading my RiscPC from ARM710 to StrongArm and being utterly astonished at the latter’s blinding speed!

Yes. A3000 to A5000, the processor went from 8MHz to 25MHz and the memory went from 2MiB to 4MiB and the harddisc went from 40MiB to 1GiB (SCSI, two partitions). This was followed by a RiscPC that was a doubling of processor speed (to 40MHz ARM7) plus architectural differences, plus the big benefit of 2MiB VRAM so the video didn’t impact the use of the system (what was it, MODE 31 where the screen blanked out every time you accessed the floppy on the older machines?).
Then, the biggest jump of all. To the Beagle. A 1GHz processor. Oddly enough, moving over to the original Pi it didn’t fall drastically slower.

By drastically slower, I mean like the time I went to Spain and took my A3000 and left the A5000 back in the UK; and couldn’t believe how ridiculously tragically terminally slow it was. In the days when it was my only machine, I used it, I developed on it, I even used Ovation for DTP on a floppy-only machine. But going back to it from a more capable machine, I was like “what the hell?”.

I would imagine the 26 bit era machines would drive me scatty now I’m used to RISC OS on a Pi2; which might chug along at 400MHz but can crank it to 900MHz when necessary. And a big screen with full colour… The BBC Micro has retro charm. I think the A5000 would just annoy (let’s not even think about the A3000)…
Oh, god, it’s 3.10 so it’ll be VDU text everywhere. I can’t believe I spent years looking at that.

Subjectiveness works both ways. :-)

took my A3000 and left the A5000 back in the UK; and couldn’t believe how ridiculously tragically terminally slow it was

Ah yes, the A5000, undoubtedly the Rolls Royce of early Acorn machines! I was ignorant of the differences between the various S/H Archimedes around in the early nineties (and indeed, of most matters relating to computing); an A410/1 was available from AJS Computers for 400 quid(!) and I went for that, subsequently upgrading the ARM2 to ARM3 courtesy of Watford Electronics and adding more RAM, but that didn’t help the slow bus, of course. I think that machine’s H/D was all of 49 MB!! Still, it hooked me on RISC OS, gave way to a RISC PC in ‘97 (I’ve never been an ‘early adopter’), and I still keep the Faith*.
*EDIT: now with a Pi3 and RPCEmu on a Windows laptop when away from home. Full marks to the latter’s developers for 0.9.2 BTW; the wireless networking works flawlessly).

I have updated the benchmarks on my web site – these now allow a selection of machines to be compared. Individual machines can be added to or removed from the comparison. Also the disc speed tests can be selectively included or excluded. This makes the table easier to view as you can fit onto the screen just the information you want.

The method of selection uses javascript and DOM and so Netsurf struggles (OK it can’t do it). Clicking on the ‘add’ or ‘remove’ button calls a javascript function and Netsurf silently borks. Hiding an item also makes the ‘hide’ button for it invisible so that you can see which items are selected.

.vir {display: table-cell;}

function hide(classname) {
  var x=document.getElementsByClassName(classname);
  var i;
  for (i = 0; i < x.length; i++) {
    x[i].style.display = "none";
  }
  document.getElementById(classname).style.visibility = "hidden";
}
...
<input type="button" id="vir" width="40" value="VRPC" onclick="hide('vir')">

Full marks to the latter’s developers for 0.9.2 BTW; the wireless networking works flawlessly).

I know you say you aren’t “an early adopter” but 0.9.3 (with some bug fixes) has been out since May 7th.¹
As to the networking, the NAT solution is very stable and even works flawlessly on a wireless interface

¹ Update of the ROOL wiki page in progress. This one will include the fast/easy setup for NAT option networking.

Rick, you forgot the jump from 40MHz ARM7 to 200MHz (later 233MHz) StrongARM (about 7x). That was the 2nd largest jump in performance of Acorn systems, with the BBC Master to the Archimedes 310 obviously being the biggest (at least 25x).

but 0.9.3 (with some bug fixes) has been out since May 7th.

I upgraded to it on May 9th :-). And you’re right, networking is flawless on a wireless interface (I’m typing and sending this on 0.9.3/5.27 running on a Win7 laptop).

That was the 2nd largest jump in performance of Acorn systems

I still vividly recall the astonishing difference between ARM7 and S/ARM when I upgraded my Risc PC – operations on the latter seemed /instantaneous/ in comparison to the former.

I upgraded to it on May 9th :-)

Teasing. I didn’t upgrade until late July, but that was down to a vision problem¹ rather than any caution on running the new version.

I wonder what comes after v0.9.9?

¹ Rather difficult to do anything on a computer that you can’t see other than a brighter blur than the immediate surroundings.

I wonder what comes after v0.9.9?

0.9.10. Or perhaps 2020? Or 365? You never know nowadays.

0.9.10. Or perhaps 2020? Or 365? You never know nowadays.

X, or X followed by another seemingly random letter.