Getting modules ready for 64-bit RISC OS
Cameron Cawley (3514) 156 posts |
SharedCLibrary already has routines using this for 64-bit integer operations, and I wouldn’t be surprised if the compiler is able to inline all of these except for variable shifts, multiplication, division and floating point conversion: https://gitlab.riscosopen.org/RiscOS/Sources/Lib/RISC_OSLib/-/blob/master/s/longlong
I can think of quite a few projects that use assembler for performance reasons, including some with AArch64 assembler: https://sourceware.org/git/?p=newlib-cygwin.git;a=tree;f=newlib/libc/machine/aarch64;h=5ed9f3ad68824c567d09a2f8f2f665efce7867e8;hb=HEAD There are many projects that use intrinsics for similar reasons, but those aren’t available for RISC OS, so that isn’t as useful for present day discussions. The main thing with assembler IMHO is (1) to use it selectively, (2) to keep each function sufficiently isolated so that it can be easily substituted at either compile time or runtime. In particular, assember is something that’s mainly used in libraries – for example, half of the libraries I mentioned above are for audio and video codecs, so in a hypothetical multimedia stack, the codec interface would be a mix of C and assembler, the middle layers would be written in pure C, and player applications or other clients (e.g. web browsers or video games) would use whatever the author feels like using. As a result, assembler would only be used in a single place where it can easily be substituted, and authors of client applications wouldn’t have to worry about it. It’s also worth noting that I haven’t figured out a reliable method for using `LDM` and `STM` from C without using assembler, so that’s at least one reason I can think of why using assembler would result in faster code compared to plain C. |
Rick Murray (539) 13806 posts |
Certainly. And as you noted, mostly video codecs, where speed is critical. But assembler isn’t a solution to speeding up general purpose code that could just as easily be written in C or, for that matter, BASIC/Python/whatever. Especially as, in assembler, you’ll be going a lot more slowly as all the mundane crap (balancing the stack, working out register assignments, blah blah blah) are your responsibility. We invented compilers to deal with all of that for us, and to allow for code that is mostly12 readable. 1 I know about Perl 2 And calling pointers to functions in C, or just C’s pointers in general… |
Dave Higton (1515) 3497 posts |
For heaven’s sake, you supporters of assembly language. Please get real. Maintainability is far more important than that last bit of speed. Maintainability (specifically, lack of it) is the very reason why we’re struggling and arguing about whether a move to 64-bit ARM is even possible. It only has to be good enough. While the struggle for speed had some merit in the days of 8MHz processors, it counts for far less these days – CPUs are about two and a half orders of magnitude faster now. Aim for maintainability. |
tymaja (278) 172 posts |
This explains, to an extent, why I do still want to use assembly for certain things (and even when I am porting ARM32 BASIC to ARM64) – to learn ARM64, but also for performance reasons – and also so I don’t have to use assembly when I don’t need to. I tend to prototype code in BBC BASIC, then rewrite speed critical bits in assembly!. A bonus of my little BASIC project is learning to program in C after all these years!. Another reason is that the kind of things I am interested in, such as 3D graphics (and fast 3D graphics algorithms etc) makes me gravitate towards assembler. Apart from BASIC32→BASIC64, if I do work on RISC OS source code stuff, I would work in C, but would use assembler too … but only for code that benefits from a speedup (with fallback to C code). |
Rick Murray (539) 13806 posts |
I’ll just leave this here…
|
tymaja (278) 172 posts |
Maintainability is far more important than that last bit of speed. Maintainability (specifically, lack of it) is the very reason why we’re struggling and arguing about whether a move to 64-bit ARM is even possible.” I agree 100%; I think RISC OS 64, if it appears, will be written in C – but I wouldn’t be averse to parts of the HAL (that interface to hardware) are in assembler, for speed. But that would really be a few % of the HAL code in total, as even there, most code is not executed enough to need hand optimisation. I will keep working on my personal hobby project of ARM64 assembler BBC BASIC. What I learn there will be used for putting 64-bit ints into RISC OS 32 BBC BASIC (and is helping me be a better BASIC programmer – one example being that I did’t realise the >>> operator existed till a few days ago, until I came across it in the assembly! |
Mr Rooster (1610) 19 posts |
Remember as well that for general purpose systems code C code is in many cases better for speed. You can re-compile C code as compiler optimisations improve or underlying instruction sets change. With hand optimised assembly you need to re-optimise for each new generation. An OS will always have a small amount of assembly for the low level stuff, but that should be kept at the absolute minimum. The NT kernel was designed for the AT&T Hobbit, ported to x86, MIPS, Alpha (both 32 and 64 bit), Itanium and x64. Had it been largely assembler that would never have happened.
It has to be code that benefits from a speed up because it’s used often too. That’s why you find it in video/audio codecs. You can make an arbitrary bit of code 10000 times faster, but if that code is run in a loop with user interaction in the mix you’ll have all your improvements swallowed up by the idling waiting for the meat sack. It’s very humbling to spend an age optimising a piece of code to then discover that in the overall scheme of things it makes no difference as the amount of time spent in that code has gone from negligible to 1000X less than negligible. |
Clive Semmens (2335) 3276 posts |
This, ABSOBLOODYLUTELY!! |
Rick Murray (539) 13806 posts |
The Ur-Example that I tend to point to is Linux. That, coupled with Busybox, has been ported to all sorts of devices and architectures. Internet cameras, broadband routers, smart toasters, printers. I do not believe the year of Linux on the desktop will ever happen. Because by the time the behemoth (you know who) fades away, it won’t be because Linux is better, it’ll be because people’s use cases move away from such machines. They’re already in transition, at work very few people have a traditional PC, it’s all laptops that go with the person. Running the behemoth, of course, because that’s where the software is. But, really, if the software was more cloud based (as it is starting to be) they could in theory ditch all of that for something like a Chromebook. But those people obsessing over Linux on the desktop have missed one vital thing. That’s not the only market for a competent OS. In fact, it’s not even the major one. It’s just the one that people see because it is right in front of their eyeballs. Just like ARM has quietly been kicking that one-trick pony called Intel 1 and their various slightly broken incarnations of the x86 architecture 2, Linux has been making huge inroads elsewhere. Android? That’s a bastardised version of Linux. Almost half the world’s population are Android users (according to a quick Google search). Major win for both Linux and ARM there (moreso for ARM because Apple…). Embedded devices tend to come in two varieties. The basic not-terribly-smart ones like toasters and breadmakers that use microcontrollers and are likely to be based upon oddities such as the SAM8 MCU, something sort-of based upon a 6502 or Z80, or all sorts of chips that have an 8051 core. You’ll find a shocking amount of 8051 stuff around. The smarter ones use a microprocessor (think “like a Pi”). These will be ARM, MIPS, 68K… just to name a few. And many of these devices, particularly if they need to try to be long-term reliable, will be running Linux. It’s simpler than rolling a custom OS and doesn’t have the licencing issues of a commercial proprietary OS, which makes it a good fit for embedded devices. And what makes that possible? Not being written in assembler! Linux can, with a little bit of glue necessarily written in assembler, be ported to a new system so long as the system supports the things that Linux needs. And for lesser machines? Clever people have produced answers. Linux ELKS, for instance, can run on an 8086 box.
Assembler – Just Say NO. 1 Yes, I’m being mean. 2 But this one is justified. |
Clive Semmens (2335) 3276 posts |
The only people writing assembler nowadays should be the folk who design the compilers. |
jan de boer (472) 78 posts |
Can you trust us to be grown-up enough to be able to decide when to use assembler and when not? Thank you. |
Clive Semmens (2335) 3276 posts |
Of course. If you have the relevant expertise, fine. If not, we can still trust you, and on your own head be it.
For sure, for the bulk of it – but unlike almost anyone else, you will have to do some work in assembly language to tailor your compiler to the instruction set of the target hardware. |
Rick Murray (539) 13806 posts |
There have been people around these parts extolling the virtues of assembler, mostly based upon preconceptions that are quite out of date (like the “the compiler can’t make code as good as a programmer would”).
If we except programs ported from other systems (such as WebKit → Iris), what large new program has been written recently?
Today’s practice elsewhere. I’m talking about RISC OS itself. As in how will it be translated to ARM64, should such a thing ever happen? You know, it has been suggested in the past the idea of automating the translation of ARM32 to ARM64 using an LLM.
If course not. But this is RISC OS. It’s not like we have much choice.
That’s not the fault of the language, that’s a fault of not providing source code. It was… less common to do so in the old days. This also applies to utilities/applications/modules written in assembler. Some can be rescued by using Armalyser and patching up the bad bits. Others (I’m looking at you, Impression) are devious to the point of being an uphill struggle worthy of standing alongside the Epic of Gilgamesh and that is with the source code! As for BASIC, I’ve seen some code that’s been BasCompressed to within an inch of its life, and honestly, I gave up rather than wading through that mess. Yes, C renders the code almost unintelligible, but this isn’t the fault of C, it’s the fact that the source has disappeared into the giant bit-bucket of time. Otherwise, one might be able to take that program from 1988 and recompile it with the latest DDE and whatever libraries it used, and have a working executable. I’ve done exactly that with some of my old code. But, again, it requires the source to exist. If it’s lost it’s lost.
It’s only a problem if you’re new to how C works (or the specifics of the DDE’s Shared MakeFiles 1). When you include a library’s header
Or using a language where that sort of thing is provided by default.
Honestly, any people who are “getting things done” will not be talking about it on an internet forum, nor will they care what a bunch of observers think/feel/say. But there needs to be some visible pushback against the use of assembler lest those who stumble upon this think we’re all still rocking like it’s 1987.
As Clive points out above, while the compiler itself will be written in a higher level language (again, future proofing and portability), there’s no escaping the fact that a deep understanding of assembler will be required because of what the compiler is actually doing. 1 I still eschew the Shared Makefiles because I want to know exactly what is going on with any given project, not have to wade through things (or ask here) in order to do simple things like include another library, or change the default flags to stop stripping out function names, or to define flags like the one to turn on the debugging code, etc. It’s probably not hard, but I know how my setup works and it works for me… |
jan de boer (472) 78 posts |
Familiarity with the instruction set of the target processor is required but the bit handling routines to construct executable code do not necessarily require the use of assembler language in the source code of the compiler that is written. Maybe i understand you incorrectly. |
Clive Semmens (2335) 3276 posts |
Indeed, although it would take a strange mind indeed to think of the instructions in terms of bit patterns rather than something a little more human-friendly – such as oh, maybe assembly language?
In the 1990s, I did use it, because some of the things I wanted to do were horribly slow in BASIC, and I knew that my code would only ever be used on RISCOS machines, which in those days were all ARM26 machines. I can imagine* using some assembler language on some machine these days for programs for personal use, but I can’t imagine writing anything I’d publish for general consumption that way. * I have a good imagination. I don’t think I’m likely to do this at all. |
jan de boer (472) 78 posts |
It’s totally dependent upon what you write. C still has more problems with only 16 registers than the 32 ones in Arm64, so as long as Risc Os is not yet 64-bit, assembler at specifically chosen locations can be faster than C. On other processors and other compilers the balance will be different.
then none has been written with a lot of assembler in it too. So what’s the point in warning against assembler in programs if there are none written. I do not advocate using assembler a lot, only where it can be useful and as little as possible. If an LLM can be used for translation, there is even less reason not to use 32-bit assembler. Reap the benefits now and let the LLM do the worrying later.
Of course, nonetheless, those C programs are gone. I also revived some crunched Basic programs, it’s a lot of work – first replace SYS/SWI numbers with names, then rename variables into more meaningful ones, try to find out if assemblercode can be rescued, which MOVs PC can be replaced by MOV because a CMP follows after the BL, etc. But i succeeded a few times, e.g. BubbleImpact. C executables without source i do not even want to have a look at. So whatever is to blame, the effect is the same, many C programs are gone while Basic with some assembler can be rescued sometimes.
Why? Don’t be afraid of what other people think. They don’t care at all, most probably. And retro is in vogue nowadays.
This contradicts what has been said before. How can one gain a deep understanding of assembler if one has not practiced it by writing a lot of code in it? For finding errors and profiling, it is also useful to be able to ‘read’ assembler. Discouraging learning / using assembler does not seem the right thing to me. Rick, i know and deeply respect your expertise in ARM assembly, don’t question that. It’s an expertise I wish more people have, they should not be discouraged.
everything I write is primarily for personal use. I very often encounter things that are horribly slow on a Pi5 in Basic, and as they need very little code i transfer them to assembler. 120X Faster, problem solved. Less effort than rewriting the whole program in C. As for C, now and then i want to compile something (e.g. Arculator) for Raspberry Pi OS and discover that the required libraries are unavailable, Windows- only or out-of-date. Compiling it for Risc OS is totally out of question. I guess that the other way round, writing for Risc OS and hoping that that can be converted for other OS’es, is equally impossible. I gave up wanting world fame, I want the results now, not after many months of battling with C or waiting for a better Basic compiler. I guess we do not differ too much in what we want for Risc OS. I really do not advocate returning to assembler-only large programs, but a little sprinkling of assembler here and there is not bad. Not bad enough imho to warrant SHOUTING criticisms against its use, as evidenced previously in this thread. |
Rick Murray (539) 13806 posts |
That is true, however on the whole a compiler can easily outperform a human. How much depends upon the compiler. Once upon a time they used to make some rather unpleasant code, but that’s not true these days. Now your compiler can rearrange the code to better fit the scheduling demands of a specified processor – something I would imagine most people don’t pay that much attention to unless it is extremely speed critical code.
It’s a fair comment when referring to RISC OS migrating to 64 bit (or some odd 42 bit DEC whatever) that the continued use of assembler is not a winning idea.
Suggestion: Copy some code (proper code, as found in the OS source) and ask ChatGPT to translate it into your language of choice.
Which means the correct solution to this problem is to write a 64 bit A310 emulator and run RISC OS on that. 😋
Seems to me that the hard part of compilation is not the assembler output…
Is this not partly because of the rudimentary nature of RISC OS that when something goes pop you’re staring at the low level gibberish? (and, as is all too often the case “anonymous function” after “anonymous function”).
I should point out that if/when RISC OS goes 64 bit, it’s an expertise that will help a little bit but not a lot. Like what’s going on won’t be completely alien, but it’ll be a strange new world…
That’s not C’s fault.
Depends on what you want done. The reason that I wrote my raycaster in BASIC was because, when compiled using ABC and VFP, even though ABC is not as good a compiler as Norcroft, with the ability to use VFP natively, it utterly wiped the floor with anything Norcroft (and FPE) could manage. And, no, I didn’t contemplate using assembler. 😉
Mortals, you defy the Compiler! I sentence you to travel among unknown vectors; until you find the Interrupt Service Handler, your bodies will stay as lifeless as Power Off mode. (ba ba ba bada baba ba-dum….) |
Clive Semmens (2335) 3276 posts |
Retro in vogue? Not with me it ain’t. I use RISCOS because I know how to make it jump through hoops, not because of nostalgia. I use the Mac for the apps it comes with, and a (very) few shareware apps, but I don’t write programs for it and don’t know how to make it jump through hoops. I’m an old dog, still good enough at my old tricks, but with little inclination to learn new ones. |
Chris Mahoney (1684) 2165 posts |
Of course, then you need to update RISC OS to run on that 64-bit A310! (Yes, I know that’s not what you meant) 😛
Recently I could see two different ways of tackling a problem, and I wondered which would be more efficient. I ended up writing both… and they both compiled to exactly the same binary. Modern compilers tend to work from a “what does this code accomplish?” perspective rather than compiling line by line. In this case, both blocks of code accomplished the same thing, and therefore compiled to the same thing. It still surprised me though! |
Simon Willcocks (1499) 509 posts |
It’s a bit annoying when your home spun memmove function, written to avoid having to link in a library, gets automatically replaced by a call to memmove! |
jan de boer (472) 78 posts |
have a look at www.stardot.org.uk/forums. New demos and games are being written for BBC-B and Electron, ULA replacements in FPGAs, floppy-disc emulators, Econet, new motherboards. It’s thriving. On my local computer club only two, including myself, are doing something with Risc OS, the others enjoy tinkering with BBC’s and Masters and Electrons.
Rick, i used Risc OS because i enjoyed using it, because i get things done, if necessary even by writing something, instead of having to battle against the OS (Windows). The reason I reacted was being annoyed by this:
you Rick did not write this, I know, but you cite this and argue time and again against use of assembler. What i said, i know full well the advantages (getting things done if there are constraints) and disadvantages (poor maintainability and slow development), because i have some experience. I write things because i enjoy it, and if i want to include assembler it’s my choice. I do not earn money with it, i do not guarantee anything, so i have no obligations towards you whatsoever. Be happy that someone else on this forum is exploring Arm64 and is making good progress, don’t bicker at anyone who dares to question your evangelism that assembler must be rooted out at all cost (that’s how it lands on me). Try to respect how other people look at things, sometimes. I try to remain polite, but you obviously deem it worthwile that i should be punished by making a fool out of me:
I understand from your own words that the Norcroft compiler cannot compile to VFP, so i suspect that replacing critical real-type calculations by VFP assembler could help, but you seem to find this mad and preposterous. The other day i found that in BASIC, when you get back a dynamic area with an address just under &80000000, and do e.g. block1%=DAaddress%+&100000, then BASIC crashes. Solution is A%=DAaddress%,B%=&100000,block1%=USR with [OPT pass:.add add 0,0,1, cmp0,#0,mov 15,14.:]: But this is a mad, bad idea, i understand. Shouting at people who do not totally agree with you and ridiculing them and misrepresenting what they say (i would argue for small amounts of assembler if necessary and unavoidable, and not at all that everything should be written in it) is not the correct way to talk to each other, i’m convinced. Life is too short to even continue to respond to this. So, congratulations with your opinions. I’m out. |
Colin Ferris (399) 1809 posts |
Strange – that Rick was the go-to for 32bit ARM assembler :-/ |
Clive Semmens (2335) 3276 posts |
I don’t doubt it. All I wrote was, Not with me it ain’t. I’m well aware that it is with others. I hope they’re having fun. I had no intention of ridiculing anyone. Even when I wrote ABSOBLOODYLUTELY, that’s mostly me being glad to see the back of it* – with a side serving of “please don’t use it other than where absolutely necessary in anything for general distribution.” * I loved ARM26 and wrote a load of stuff using it; I literally wrote the book on ARM32 – but never used ARM32 in anger, just wrote (a lot) of very simple test programs to check that I understood what it was doing in order to document it correctly. Documenting ARM64 would have been my next job, but I had a look at it (still in the early stages of development), and thought, “stuff that – that’s a compiler target” so I retired instead. |
Rick Murray (539) 13806 posts |
There was a time when it was necessary and when people actually wrote things in assembler. But, then, there was a time when we habitually counted cycles and that sort of thing was printed in the instruction set datasheets. NOP is &EA and it takes 2 cycles. ;)
Yes, and people are still writing stuff for the PDPs. And…? Or have we just been written off as “retro”?
Fair enough. But I should point out that, if/when the 64 bit switch takes place, exactly none of that assembler will work. So the general idea should be to, wherever possible, reduce dependencies on assembler code. If you know what you’re doing and if there are specific use cases (or you’re just doing something for yourself), then by all means use assembler. That comment therefore wasn’t aimed in your direction. However, take note. Impression still isn’t 32 bit native. OvationPro is, and what’s more, there’s a Windows version.
That was actually responding to your comment regarding SHOUTING. As to what the quote “Mortals” etc is about, I take it you never watched Ulysses 31 as a child. Oh well… Either way, lighten up. It wasn’t meant as an insult.
I shouldn’t have to resort to patching in FP code because the compiler can’t handle it by itself. Plus, FPA and VFP are not directly interchangeable (word order differences). I probably ought to see if SoftFP can help…
Hehe, that’s
🤷 Didn’t I say right at the start that some assembler is always necessary – that low level glue code and very time-sensitive stuff. It just shouldn’t be turning up in everyday things any more. But, if you want to misrepresent what I say to try to score points on an argument we weren’t even having….. Anyway, break time is nearly over so need to bring this to a close. Hope to see you around soon and, just so you know, the main ASSEMBLER BAD (in shouty capitals) is aimed at those who would want to port the OS to AArch64 and repeat the mistakes of the past. |
Rick Murray (539) 13806 posts |
I’d just like to take a moment to reiterate Jan’s point about lost C programs. Yesterday evening I successfully converted an old 26 bit program (a fairly simple one) into a 32 bit program. The basic process was:
Eventually, a simple little command line program built decades ago using Acorn C release 4 was running naively on my Pi. Today, I wanted to step through the process again, in order to document the process and “you need to do this and this because this”. Bloody hell. In the good moments, CLib throws oddball things like telling me the size of the stubs don’t match (yet, somehow, that exact same code works in the donor). In the not so good moments, it’s an instant total machine freeze. It’s been a while since I’ve felt the need to spit out a sentence that includes swearing in five languages 2. Given how incredibly fragile this is, it’s really more trouble than it is worth and I can’t help but feel that a program of any degree of complexity would be “a bloody nightmare”. And, yes, it would probably help quite a lot if I pulled the stubs apart line by line to understand exactly what is going on. But, honestly, life is too short and I was only looking to see if it is possible. Answer? Yes, it can be done. But with a huge pile of caveats. Maybe it’s a simple thing I’m getting wrong and something like Pyromaniac would be like “no, dummy, you don’t pass that to CLib!”. I don’t know. I just know I need a nice calming bowl of Weetabix 3 and maybe a dose of DragonForce 4 to relax to. 1 ARMalyser doesn’t seem to understand CLib Stubs type 5, but since an old program won’t either, it can be yanked out. 2 For semantic reasons, I’m counting Québécois and French as two different languages. 3 It’s quarter past eleven. I was up by half five. You don’t think I’m going to attempt to “cook” anything more complicated than “chuck in bowl, pour milk over” do you? I’m not a teenager any more so I’m barely conscious by this point. But, then, that’s me on any Monday morning… 4 This sort of thing is often cheesy (bonus points: anything ever created by Rhapsody (of Fire)) but at least it’s more upbeat than the usual stuff I listen to. ;) |