Crunchie for crunching BASIC

One can do quite a bit with StrongED’s Search and Replace.

Hang on. All in good time. Tomorrow probably. ;-)

The “problematic line” is the third in the crunched part, beginning with ELSE…

Sorry, but Textile and pling do not mix. Could you make that a code block.

That’s why I won’t be using Crunchie. I am very happy to see it available for RISC OS 5, because I used to use StrongBS back in the day, but these days I don’t bother squishing stuff written in BASIC. It’s just too much of a PITA later on.

+1
Big pain in ass.
Like: Oh look I spotted a problem with a program and the source is sort of present but so buggered about that it would easier for a total noob to rewrite from scratch than for a half proficient coder to try to fix.
e.g. Not going to name names, but someone out there updated a program that generates a module and the source was updated to be “32bit” except the switch to SVC mode was pointing at mode number 3 (26 bit) which makes the program useless on anything without a 26bit mode.
Not sure how easy that would be to fix if the source BASIC was crunched.

It’s been so long since I’ve used BASIC’s assembler that I don’t remember the default OPT settings. 0 TO 3 STEP 3? Or was it 0-2?

Wasn’t there something, possibly ancient history¹, that required higher numbers than single digits to kick in support for newer instructions like MSR and MRS.

¹ Yup, history. MRS etc are fine in the raw BASIC assembler. There will be some antique change in CVS…

To have an option to use Throwback like the rest of the DDE tools?

I am flattered that you should think of it in those terms, but yes, an element of throwback could be useful. My preference is to do it through *ReportError T.

a munged program is best considered to be “read only”.

That was a good summary of the problems of attempting to edit crunched code.

Maybe what we need is something that added the extra assembler stuff to RISC OS 5’s BASIC? ;-)

Of course. I have developed a fairly comprehensive set of macros, building on some I found on your site a good while ago. ;-)

You can’t concatenate multi-line IF statements like Raik’s code seems to imply

Of course. Although it is a bit garbled by Textile, the crunched version does obey the rules, I think.

Is the download changed?

I have fixed all of my offending GOTOs and a 154K BASIC program now seems to work …
… but it is tripping up over some CHR$(13)s

or, is there a work around?

but it is tripping up over some CHR$(13)s – or, is there a work around?

Standby for an update. This is a bug I have fixed. See my earlier post.

If you urgently need a work around I think CHR$(+13) would defeat the current crunch logic.

I think CHR$(+13) would defeat the current crunch logic.

That causes an error at cruch time, but cr%=13: CHR$(cr%) is OK.

I now have a 154K BASIC program running. The full test will start at 7am tomorrow morning, but it looks good so far.

Many thanks.

In terms of benefits that this could bring, replacing SWI names with numbers can yield some pretty significant benefits. I recall on an A3000 (OK, several orders of magnitude slower) I was using a few SYS calls by name to plot a (20×20 tile?) map on the screen. You could see it drawing the tiles – it was unusable. Changing to numbers (thus removing all the look-ups) made the map plotting seemingly instant.

Of course, a programmer will naturally optimise such loops as best the can without the tool, but being able to use longer variable names, and full-name SYS calls makes the code far more readable. A cruncher can then optimise for speed before release.

I recall Andrew Pullan had a custom tool for Webster, which (if memory serves) pulled together various smaller libraries (which were more extensively annotated for ease of development) then crunch and merge for speed. This made development far more practical.

I recall Andrew Pullan had a custom tool for Webster, which (if memory serves) pulled together various smaller libraries (which were more extensively annotated for ease of development) then crunch and merge for speed.

There’s also Tokenize which in-lines libraries, converts SWI names to numbers, and applies a CRUNCH to the code (plus checking stuff like variable and FN/PROC names, if you wish).

This made development far more practical.

Tokenize’s main purpose was to allow BASIC programs to be held in a text-based revision control system (eg. SVN or Git) with the benefits of diff and merge, which falls very firmly into the same category. It doesn’t have to be used for that, however.

There is a native RISC OS version on the site above, alongside the Linux one.

I have fixed all of my offending GOTOs – I now have a 154K BASIC program running.

Good. Could you say how you fixed the GOTOs?

I have uploaded version 0.72, which has a fix for the CHR13 bug and also allows lowercase assembly opcodes.

This can be downloaded from the bottom of my Downloads page , where any other updates will appear from time to time without further announcement, unless there is a major change.

It has been encouraging to see the hunger with which Crunchie has been greeted, both here and privately. It clearly fits a need, despite some who see crunching as the work of the Devil. ;-)

Clearly there is quite a lot to do still and most of the problems seem to be with assembler. I was not going to include this at first, but when I did I may have been over-ambitious. A simpler approach that does not attempt to crunch data directives and register names might be better and I will revisit this.

Remember, please, that Crunchie is not a replacement for StrongBS. ;-)

replacing SWI names with numbers can yield some pretty significant benefits

Indeed it can. Although this is done by Crunchie, there are still some old applications written in BASIc that can do only this.

being able to use longer variable names, and full-name SYS calls makes the code far more readable.

For me, that is the primary reason for crunching.

pulled together various smaller libraries […] then crunch and merge for speed. This made development far more practical.

At the moment Crunchie can crunch a list of libraries to the same names as the main program, but it still expects those libraries to be loaded with LIBRARY ¹. That is the way I work, but in discussion with Martin and others it is clear that there is a desire for merging and I will eventually implement that.

¹ Judicious use of a path vath variable can ensure that a program can use crunched libaries if available, or fall back to the source.

Good. Could you say how you fixed the GOTOs?

Using a mixture of:-
– multi-line IF THEN ELSE ENDIF
– WHILE UNTIL
– ENDPROC
– leaving in the ones that crunchied OK

Unfortunately some of the logic does not work now. :(
So, a fix to !Crunchie handling of GOTOs would be nice. Some of my code was written over 30 years ago. I have reverted to the 30 year old code blocks.

Good. Could you say how you fixed the GOTOs?

Having thought about the meaning of your question, I have now fixed the GOTOS rather than by trying to re-code to remove them.

Each

GOTO linenumber

line_number%=linenumber: GOTO line_number%

All except one, that is, to stop !Edit discarding all of my line numbers. The code uses blocks of line numbers for each PROC and FN.

My 154K BASIC GPS Tracking server is now running with 24 live GPS trackers talking to it (with TCP/IP HTTP) every 30 seconds.

And the server is sending 30 second live updates to !RiscOSM for my car. The location of the rest of the trackers can be seen on a Google map. The Google map web page being produced by the same crunchied server.

!Crunchie is magic :)

GOTO linenumber

This bug is a result of not accounting for the way BASIC codes line numbers in these cases, using the token &8D and 3 bytes. These bytes are always non-top-bit-set and Crunchie was seeing them as variables and crunching them, hence the weird decoded line numbers. A fixed version is now available. I hope it works. ;-)

A fixed version is now available. I hope it works. ;-)

GOTO linenumber and CHR$(13) now both Crunch OK for me with version 0.73 (03 Jan 2019)

Thank you.

Crunch OK for me with version 0.73 (03 Jan 2019)

I agree. I not know why, but my “problem childs” are working.

Thanks a lot.

I think some moderate crunching might be a trade-off between programmer friendliness (can use SWI names) and efficiency (can use SWI numbers), however I think enough time has passed that most of us who are left are aware of the…issues…involved in dealing with heavily crunched code.

Plus, as any one of the AmCog games amply demonstrates, it isn’t exactly as if crunching BASIC is an absolute necessity for speed.

Remember, please, that Crunchie is not a replacement for StrongBS. ;-)

Yet. :-)

I haven’t had a chance to look at Crunchie “live” yet (due to being ill since Christmas eve), so this may already be present… It sounds like a version with a nice options window to choose what degree of crunching is required would be nice.

For example, whatever SWI numbers rather than names doesn’t make the code too hard to edit later, but yields performance and minor space savings. Other compression options that allow the code to be re-edited later (ie. doesn’t break Edit/StrongEd basic<→text system) would be useful, so that it is possible to crunch code without making it effectively “broken” for re-editing.

It seems like Crunchie may already offer all this, but if it doesn’t, this kind of control (so that you can choose between gentle, medium or heavy crunching) would seem the obvious way to go.

As SWI names appear popular. What about “de-crunchie” which turns SWI numbers to text. It could even be a load option for something like StrongED.

You could also think about about splitting multi statement lines. Renaming variables/procedures where sensible to say “global_a_isused5times” so when looking at the source you have a clue about the importance.

The strongBS release notes gives a good list of things that are possible to compress in BBC basic.

it isn’t exactly as if crunching BASIC is an absolute necessity for speed.

Of course, but I have reports that even this flawed version of Crunchie has increased the speed of some programs to a useful extent. The gains are marginal and crunching should not be a compensation for poorly constructed algorithms. If you want faster BASIC then for many operations the specialist keywords of Basalt provide it – in spades. If only Basalt itself was not a problem at the moment. ;-(

Remember, please, that Crunchie is not a replacement for StrongBS. ;-)
Yet. :-)

Never. The methods employed a quite different.

It sounds like a version with a nice options window to choose what degree of crunching is required would be nice.

I have a ‘nice’ desktop front-end for Crunchie, but it relies on Basalt, so I will not release it for a while.

Other compression options that allow the code to be re-edited later […] would be useful

The first thing Crunchie does is to use BASIC’s CRUNCH %1111. To quote the manual: “CRUNCH %10 may make a program uneditable”. So no joy there.

this kind of control (so that you can choose between gentle, medium or heavy crunching) would seem the obvious way to go.

If you want SWI name replacement there are applications from way back that probably still do that. On the other hand I question the notion of partly crunching a program and then editing it. Why would you do that?

If you want SWI name replacement there are applications from way back that probably still do that.

Which is indeed so – !SpeedySWI APDL b172 does run under RO5 in RPCEmu, not tested on ARMv7+

On the other hand I question the notion of partly crunching a program and then editing it. Why would you do that?

To quote Andrew Rawnsley earlier:

For example, whatever SWI numbers rather than names doesn’t make the code too hard to edit later, but yields performance and minor space savings.

i.e. Speed benefits (Faster machine than in the days of the author, so I added three zeros to the number of iterations) 26.62 seconds vs. 12.24 seconds, small memory benefits without leaving the program cryptic and uneditable.

but I have reports that even this flawed version of Crunchie has increased the speed of some programs to a useful extent.

Add another report to the list. A very useful speed increase.

Speed benefits, […] small memory benefits without leaving the program cryptic and uneditable.

If there are plenty of SWIs, then SWI conversion plus the setting of Basic$Crunch to ensure CRUNCH %1111, will give much of the benefits, but have you tried editing a program with only numebers after SYS? ;-)

Once upon a time I experimented with including a routine in a program that did the SWI conversion at runtime, but that was before Basalt, which does it anyway.