Compiling BASIC - again!

Recent discussion has led me back to this.

I never used RiscBASIC, although it seems to be the compiler that compiled nearly all ARM BASIC. However, it is lost to us in 32-bit-land, isn’t it?

I used ABC a long while ago and now again to check things out. It has many differences from ARM BASIC that the user has to take into account and it is clear that I do not like it much. ;-)

With modern, very fast machines it seems to me that compiling code is unlikely to give much benefit, but I am willing to be persuaded otherwise.

To that end, can anyone provide me with a segement of code, say a few kilobytes, that is significantly faster when compiled with ABC?

My view you run into the fundamental shortcomings of BASIC long before you start having to worry about performance on modern machines, so compiling is quite an ineffectual sticking plaster.

For better performance and language features we’ve got Norcroft C and GNU g++ (I’m not even considering CFront to be usable C++).

What we really need for all those too scared of a compiled language is to get Python3 to the level of performance comparable to BASIC, and with a standardised Wimp and Toolbox modules.

@ David

What we really need for all those too scared of a compiled language is to get Python3 to the level of performance comparable to BASIC, and with a standardised Wimp and Toolbox modules.

While I agree with your logic, I am not 100% sure it will ever be possible to bring interpreted Python3 to the same performance of BBC BASIC (and for different reasons). But it’s worth a try sure.

Also don’t forget Lua, I have been playing with it and it seems very much usable on RISC OS as well as having a compiler too to squeeze some more perfs out of a program as well as supporting modular expansion via C/C++ written modules.

But yeah as a general rule Python is more popular, so it could be a good alternative to BASIC. In the end it is the defacto “new BASIC” on Linux, macOS and Windows.

What we really need for all those too scared of a compiled language is to get Python3 to the level of performance comparable to BASIC, and with a standardised Wimp and Toolbox modules.

There is the start of a Toolbox library for Python. If life and my actual job didn’t keep getting in the way … however they do pay me.

I don’t know about performance compared to BBC BASIC (compiled or otherwise). I guess the key thing is more it has to be “fast enough” on modern hardware.

Arguably It’s better to have a program that’s a bit resource heavy than one that doesn’t exist because it’s too much effort to write it.

I don’t think Python could ever be as fast as BASIC, but it there is still some room for improvement. Comparing the single tasking runs of some noddy programs on the same Pi 3B under Raspbian and RISC OS I get:-

Language   Program      Linux     RISC OS  Difference
--------   ------------ -------   -------  ----------
Python 2   Countdown    304.813   466.07      65%
Python 2   GridShade1    93.203   171.43      54%
Python 2   GridShade2    18.678    31.97      58%
Python 2   GridShade3    10.672    18.24      58%
                                              
Python 3   Countdown    374.797   625.62      60%
Python 3   GridShade1    96.873   156.53      62%
Python 3   GridShade2    23.839    45.62      52%
Python 3   GridShade3    13.311    22.76      58%

I’d love to see how fast these would go under BASIC, but without any support for dynamic memory allocation or structures, they would be a hideous mess of peeking and poking, and I draw the line at that.

As for Lua, I ended up with 3 different versions with no clear idea of which was the latest, couldn’t really get any of them to do anything useful, so deleted the lot.

I’m not sure if any of those tests use FP, but I tried to build a VFP version but it just blew up when I tried to do any floating point. I believe others have made it work, so I need to look into it again.

I’d love to see how fast these would go under BASIC, but without any support for dynamic memory allocation or structures, they would be a hideous mess of peeking and poking, and I draw the line at that.

A bit off my original question, but dynamic memory allocation is really very straightforward with a short BASIC library. Mine has just one function. I have been using it for decades. ;-)

Structures are a bit trickier, but depending on how flexible you want them can be done at the program level without all that indirection. That is not to say that having them built in would not be very much easier.

Edit: If they are noddy programs could you send one to me. I have translated some Python in the past.

Chris: no FP in any of those, RISC OS programmers avoid FP like the plague.

Steve: sticking them on GitHub is the plan. There are C++, C#, Python (2, 3, pypy2 and pypy3 compatible) and Perl versions. I had to make seperate RISC OS Python and Perl variants with the multitasking ripped out, but I may find the equivelent of #ifdef’ing it out at some point.

has many differences from ARM BASIC that the user has to take into account

I find the main thing (which is why I rarely use it) is not so much the weirdness of LOCAL, but the fact that it is stuck firmly in the RISC OS 3.10 era of BASIC. If you’re used to anything later, ABC will choke.

that is significantly faster when compiled with ABC?

Your average desktop program is unlikely to see a great boost, but any program that does a lot of number crunching will.
So, I have a program that reads a JPEG (800×1135) and scales it up to 1600×2270, and then performs a vertical filter and a horizontal filter on the scaled up image. It works by converting the JPEG into a 32bpp sprite and using direct data manipulation.
It was part of the playing around with Manga to see if I could make a resizer that was less crap than the OS JPEG plotter and a little faster than ChangeFSI… but I think my maths just isn’t up to making a decent edge enhancement filter.

Anyway. Pure BASIC: Raw scaling took until 18.78s. Vfilter took until 98.31s, Hfilter took until 186.06s which, three minutes is how long it took BASIC to process that image.

Compiled ABC… stiffed the machine. God I hate this shonky default memory handling that craps itself when hit with a ~20MiB DIM. This is v4.20 by the way.

Compiled ABC, rebuilt with the “{NEWHEAP}” option. Raw scaling took until 6.14s. Vfilter took until 21.59s. Hfilter took until 36.57s.

[we’ll skip over the fact that ABC output absolute rubbish instead of a sprite; I can’t be assed to figure out why, it’s the difference in processing speed I am interested in here, it will have done the same amount of processing…….just wrongly. ;-) ]

So, three minutes versus a little over half a minute. For a program that essentially reads and writes words, storing and manipulating them, a few bazillion times.

You pay quite a hefty overhead for the interpreter interpreting the same stuff over and over.

I don’t have a comparison with C, but if I remember, it was about 16 seconds. That’s because Norcroft can look at the code at block level and optimise, including using rather more registers than ABC ever would. ABC is, itself, quite heavy on loading and saving registers as it only seems to want to use R0 and R1 when and where possible (and sometimes R2).

compiling code is unlikely to give much benefit

Personally, I’d just keep BASIC as is. There’s something to be said for the fact that the software that suffered the least with the multudinous changes in the ARM is those things written in BASIC. Hell, a silly little thing I wrote on my A3000 under RISC OS 2 works quite happily on my Pi, thirty one years later.
Not one single compiled program runs natively on my Pi. Needs to be recompiled. Which isn’t hard, but does mean you need sources. And the amount of software ‘lost’ to us (RiscBasic, for instance?) demonstrates that without sources or an elastoplast like Aemulor, there’s no hope.

So, ABC… Maybe if it’s commercial software with secrets inside. Maybe if your program does a lot of number crunching and you don’t know assembler.

Note, by the way, that it assembles reals using FPA instructions.

RISC OS programmers avoid FP like the plague.

We’d like to avoid FP since most stuff in RISC OS land likes to use a software emulation of a long-dead FP chip in preference to anything that might exist in the actual silicon…

…but it’s hard to actually avoid FP. Use any real variable in ABC, it compiles it as FPA (as I mentioned above). Use float or double in C and Norcroft will use FPA instructions.

Not one single compiled program runs natively on my Pi. Needs to be recompiled. Which isn’t hard, but does mean you need sources.

It doesn’t have to be recompiled, I’ve converted a few C applications and modules to 32 bit using ARMalyser, as it is possible to make the code shuffle up to accept additional instructions, and then tack on a new bit of stubs code. I don’t like doing it though as it very tricky to find bugs.

Getting stuff to run on ARMv7 and later is more of an issue than simple 32 bit compatibility, as it’s far more difficult to use static analysis to find suspect non aligned code. Flag preserving instructions are easy to spot, and 75% of the time can just be changed to non flag preserving with no issues. For alignment it is far easier to let the compiler work out how to do it correctly.

Your average desktop program is unlikely to see a great boost, but any program that does a lot of number crunching will.

I see that, but I think druck would agree that you need to choose your language. OK for testing an algorithm, but not in practice.

So, I have a program that reads a JPEG (800×1135) and scales it up to 1600×2270

Could you let me have just that bit to look at? It is the only answer to my original request.

but I think druck would agree that you need to choose your language

I did. I’m not writing this in French! ;-p

OK for testing an algorithm, but not in practice.

The thing is, BASIC (and several other well known interpreted languages) are popular not because of speed but because of simplicity.
If we agree to gloss over the horror that is dealing with Wimp blocks in BASIC, it is otherwise a really straightforward and easy language.
C, in comparison is scary. A single misplaced semicolon and the compiler has a nervous breakdown. Some of the error messages read like gibberish. Accidentally overshooting an array can cause all sorts of weird mischief, and the compiler is totally fine with you accessing element -123456 of an array that doesn’t exist. The language often seems like it is trolling you for the sake of it. Did you notice there’s no such thing as a string? It’s a sequence of bytes held in memory that is terminated by a zero byte ¹, technically referenced using something like *(str+0) but that’s a mouthful so we all use the array-like syntax str[0]. Oh, and entire books have been written on the subject of pointers alone.

Yeah. BASIC’s simpler. ;-)

Could you let me have just that bit to look at?

Mail sent.

¹ And that’s only convention. If you want to write your own string handling routines, you can define a string as anything you like, including a leading length byte and an &0D terminator if you want.

C, in comparison is scary. A single misplaced semicolon and the compiler has a nervous breakdown.

I don’t get this. If I miss a semicolon, the compiler tells me that I’ve missed a semicolon, and gives me a line number to check. My current editor underlines it in red as I’m typing, too. I can’t compile the code until I’ve fixed the mistake. That’s kind-of reassuring; if the code compiles, I’ve only made semantic mistakes – that’s plenty to be going on with.

In contrast, BASIC won’t tell me that I typed list instead of list%¹. If I’m lucky, the interpreter will report the problem before I release the software, but that depends on the offending line ending up being executed and actually being syntactically wrong. If it was something like

DEF PROCfoo
LOCAL bar

REM Do local stuff with bar%

ENDPROC

then I can confirm, from bitter experience, that it can take weeks to track down.

technically referenced using something like *(str+0)

Um, you mean *str, don’t you? Although, usually, I don’t find the need to poke strings directly: I just pass them to string-handling library functions.

the array-like syntax str[0]

Pointers and arrays are the same thing, you know. :-)

There are upsides. Doing

len = strlen(str);
for (i = 0; i < len; i++)
  str[i] = toupper(str[i]);

is a lot clearer than

FOR i% = 1 TO LEN(str$)
  MID$(str$, i%, 1) = FNtoupper(MID$(str$, i%, 1))
NEXT i%

(and don’t mention that there isn’t a standard FNtoupper() for us to use). (Or that you’d probably implement it such that str$ = FNtoupper(str$).)

Oh, and entire books have been written on the subject of pointers alone.

It’s a good job that BASIC doesn’t have pointers… oh, bother.

numbers% = 10
DIM block% (numbers% * 4) - 1

FOR i% = 0 TO numbers% - 1
  block%!(i% * 4) = i%
NEXT i%

or, perhaps

int i, numbers = 10;
int *block = calloc(numbers, sizeof(int));

for (i = 0; i < numbers; i++)
  *(block + i) = i;

Look, not a *4 in sight…

Yeah. BASIC’s simpler. ;-)

Maybe. ;-)

¹ Tokenize will make a go of doing so, though. It’s not 100%, but the “variable used and not assigned”/“assigned and not used” warnings are quite helpful…

Python is a bit better than BASIC as it detects certain faults in the entire program on starting, but like BASIC some errors are only found by the interpreter when execution hits that line. PyLint is useful for finding even more issues before running.

As someone has now worked out the BNF of BASIC in What would AArch64 BBC BASIC look like? perhapse someone can come up with a BBlint :)

Mail sent.

Not received. ;-(

steve at kappa dot me dot uk

perhaps someone can come up with a BBlint :)

You are very cruel to suggest it. My brain has not recovered from the previous effort. ;-)

I think a full linter is out of the question – think EVAL, DATA and FN returns – but there are partial solutions as Steve has with his Tokenize and you can get quite a lot of information from a Crunchie Analysis file. From times past there are a couple of profilers, so there can clearly be something. I am told RiscBASIC does the job. The question is: would anyone be bothered to use it?

Resent. I know your email, you contacted me recently. ;-)

If you still don’t see anything, check your spam box. They’re both from my heyrick.eu address.

I’ll reply to other Steve’s message this evening.

If I miss a semicolon, the compiler tells me that I’ve missed a semicolon, and gives me a line number to check.

And rather than saying “it’s borked, I’ll let the user fix this”, it ploughs on getting further and further removed from reality until a hundred error messages later you’d wonder if it has given up on your program and is trying to parse Tolstoy in his native language…

and gives me a line number to check

Try missing a ‘}’ and see where the compiler loses it. I find it’s usually the start of the next function where it’s like “the hell? that’s not supposed to be there!”.

I can’t compile the code until I’ve fixed the mistake.

Clearly not something that runs on RISC OS. ;-)

Kind of wish the Arduino IDE did that, given how much for freaking EVER it takes.

BASIC won’t tell me that I typed list instead of list%

True, and depending on how you’ve done it, it might create and use an entirely new variable. Fun to debug.

usually, I don’t find the need to poke strings directly:

All the bloody time, given that the ranged string functions (strncpy etc) cannot be trusted to return valid (null terminated) strings if they’re too long. Instead of copying “n-1” bytes and sticking a null at the end, it’ll just copy “n” bytes and hand you that to deal with even though it’s technically not a string.

Um, you mean *str, don’t you?

I meant the alternative to str⁰, str¹, str², etc.

Pointers and arrays are the same thing, you know. :-)

I know, but it can be a bit mindscrewy to somebody who’s used to “an address” being this thing, and an array being that thing.
Then walk in the fun of structs and you can transmogrify your pointer into something completely different. A fairly standard thing to do with data structures passed by address in SWI calls (either as a module or as a client).

is a lot clearer than

I grew up with BASIC, and now use C, so both are equally clear.

MID$(str$, i%, 1) = FNtoupper(MID$(str$, i%, 1))

Yeah, I’d have an FNupper that returns a string, however…

That line assumes a linear 1:1 relationship between bytes and characters. Okay, we all do it, myself included. I guess it’ll be a while yet before we see normal versions of RISC OS using something more capable than Latin1.

Look, not a *4 in sight…

Hehe, so long as you’re using a 32 bit system and are dealing with sizes and addresses less than 2GiB.

then I can confirm, from bitter experience, that it can take weeks to track down.

Oh, I can imagine. With BASIC, I have a very simple rule – do NOT recycle names. Things defined as LOCAL don’t have the same name as anything global, or – as much as possible – the same name as any other LOCAL definitions. I guess I’ve been bitten by that one too at some time.

The question is: would anyone be bothered to use it?

Something I find ABC useful for is chucking a program at it simply to see if it compiles, in order to get a quick syntax check. But I’m finding it increasingly less useful given it’s ancient level of BASIC support.

C, in comparison is scary. A single misplaced semicolon and the compiler has a nervous breakdown. Some of the error messages read like gibberish.

Probably can be set up by serious users, but I tend to use defaults in editors/compilers. I find the random use of whitespace, (tabs and spaces) leads to a few issues such as making patches.
But also if the indenting position of a block of code was taken seriously, wouldn’t just that block be scanned for errors, instead of looking for or using a closing component (; bracket) anywhere up to the end of the file?

Ronald, if you like that sort of thing, Python is waiting for you with open arms.