Crunchie for crunching BASIC

Crunchie is a crunching utility for BASIC files written in BASIC.

It can be downloaded from:

As far as I know, there is currently no reliable BASIC cruncher that runs on RISC OS 5.

About a year ago I started to play around with ideas for crunching, prompted by the possibility of including this in Basalt. As time went by, this just grew from an amusement into to a practical program.

Recently it has been put through its paces by Martin Avison to crunch the Organizer application, which is large and complex, with many libraries.

With several bugs and corner cases ironed out I think the New Year is a time to release a beta version for further testing.

Crunchie is an on-going project, so comments are welcome, as well as bug reports. There are several features waiting to be included.

Crunchie is a single-pass program and does:

• delete comments, blank lines, empty statements and spaces
• shorten variable, procedure and function names
• convert SWI names after SYS/SWI to numbers
• convert ASC with literal strings to numbers
• convert CHR$ to a literal string
• convert numeric constant variables to literal values
• convert binary literal values to hexadecimal
• minimise statements, such as FOR NEXT IF THEN
• remove debugging statements beginning with VDU:
• concatenate lines
• remove unwanted routines at the end of the program
• crunch multiple files to the same variable values

Crunchie is intended to crunch a program that runs and has been tested, although there are a number of controls to cater for particular requirements, such as included assembler.

If selected, Crunchie reports progress in a Reporter window, from which a file can be saved and then analysed. This will be improved.

In addition to straight crunching, Crunchie can be set to translate a program that includes features that would prevent it running directly, into a running program.

If selected, translation does:

• event constants to literal values, such as @Wimp_ScrollRequest to 10
• user constants to literal values, such as @Window_Cancel to 1
• COMAL: identifies defined PROC and FN names and inserts the keyword
• program tags for GOTO GOSUB RESTORE
• ON OFF to TRUE FALSE
• RETURN value to =value
• END with CASE WHILE IF PROC to the keyword
• ENDCASE to WHEN and ENDIF to ELSE to exit structures early
• Multiple ELSEIF in IF ENDIF given the correct number of ENDIFs

Some of these are experimental.

Excellent, Steve! Will download and see how it goes. My BASIC programs probably won’t stress it much though – they’re all stand-alone, with no libraries, and none of my recent (still in use) programs include any assembler even. All being well I’ll crunch !Run in all my publicly offered apps – but leave the uncrunched version alongside for people to inspect or edit to their heart’s content.

All being well I’ll crunch !Run in all my publicly offered apps – but leave the uncrunched version alongside for people to inspect or edit to their heart’s content.

Excellent spirit, making code properly visible with comments for the eddificashun of lesser mortals.

With current processor speeds I do wonder how much benefit such crunching produces though.

With current processor speeds I do wonder how much benefit such crunching produces though.

Indeed. That’s why I don’t write assembler any more, too. But I’m very willing to give Steve’s Crunchie a whirl, as much for interest as anything else.

Processor speeds and memory sizes both.

I wonder if the name of the PROC’s – FN’s could be left in – ie the line before the PROC/FN

Just in case the full source is lost!!

REM PROC_Org_name
DEFPROCS
DO something
ENDPROC

Personally, I’d rather not. If the full source is lost, you’re on a hiding to nothing trying to make sense of a program of any significance with just the PROC and FN names. Just make sure you don’t lose the source!

I do wonder how much benefit such crunching produces though.

Some speed, but not a huge amount. The memory savings are probably not worth anything. It is the principle of making small gains more than anything.

Just in case the full source is lost!!

That is something that makes many crunched 90s programs a lost cause, but we would not leave out the source for public programs now, would we? ;-)

Private programs benefit from the obsurity, even if it just prevents customers fiddling with the code and creating problems for the maintainer to fix.

with comments for the eddificashun of lesser mortals

Sad to say, I’m not the best commenter ever 😞 nor am I the prettiest coder ever. Eddification is liable to be as much about how crap you can be and get away with it as anything else.

Quote from the manual:-

By default Crunchie removes empty lines, which BASIC’s CRUNCH does not do, because it could remove lines that are targets for GOTO, GOSUB or RESTORE. To disable this, include the switch -emptyoff or -e

It may be better to set the defaults the same as BASIC’s CRUNCH. This would give a higher confidence that programs which run with BASIC’s CRUNCH would run after being crunchied.

All of my programs with GOTOs fail after being cruchied. The targets for the GOTOs are not empty lines, although some have REM COMEFROM comments and some are ENDPROCs

Using -emptyoff, still produces incorrect output for some GOTOs with output like GOTO484372.INPUTS with the following line joined even with -joinoff

Thanks a lot. The tool is what I’m searching for, but I’m to stupid to get a usable result. The produced code will not work. A lot of errors (e.g. missing "" or “no such mnemonic” and other). Have play around with any of the arguments, but no success. I not use GOTO, GOSUB etc. in my code.
The normally basic code is working and if I try in TaskWindow something like…

*RAM (Basicfile is placed in RAMDisc)
*basic -load basicfile
…
>LIST (to visit basicfile)
>CRUNCH %11111 (or other options)
>LIST (if I like)
>SAVE “crnchfile” (save the crunched file)
>QUIT

… the crunchfile is working like the basicfile.

What are comparable settigs in Crunchie? What should I try?

Being in written in BASIC it will likely survive a long time to come.

Joe Taylor wanted to incorporate a BASIC compacter within AppBasic, especially one that loads in libraries and cuts out orphan FN/PROCs that are not in the dependency graph. He had some problems over this as the author of StrongBS, his first choice, had left the RISC OS scene and could not at that time be contacted (I believe he returned later). There were also Shrink and BASCompress. But StrongBS and BASCompress did not work on the Iyonix. Some of the issues over updating were because the compression programs written in BASIC had themselves been compressed, which also caused problems in RISC OS 5. Joe thought he could achieve his purpose with the ABC compiler, but that did not work with the A9.
At one time, as a temporary stopgap, he used a BASIC library linker that I wrote for him in Lua!

The moral, as I see it, is do not try to hide your source code. It is a sure path to obsolescence. It is ironic that one of Joe’s ambitions with AppBasic was to have it all in BASIC so that it would work without change on future versions of RISC OS.

Should the crunchied program be:
– viewable with !Edit?
– edit and saved with !Edit?

All of my programs with GOTOs fail after being cruchied.

Certainly none of the thousands of lines of code I helped Steve to test with included any GOTOs, so that may have been overlooked.

The produced code will not work. A lot of errors (e.g. missing "" or “no such mnemonic” and other).

I never saw Missing "" – what line number was that at?
No such mnemonic sounds as if the code may include assembler, and you did not specify the -assemble switch.

one of Joe’s ambitions with AppBasic was to have it all in BASIC so that it would work without change on future versions of RISC OS.

Unfortunately, there are some vital core parts that are in compiled Basic, and the source is not available. So if/when something affects those programs they cannot be recompiled.

– edit and saved with !Edit?

As the Manual says, “Crunched programs may be impossible to edit”

All of my programs with GOTOs fail after being cruchied.

GOTOs? ;-)

Using -emptyoff, still produces incorrect output for some GOTOs with output like GOTO484372.

Without the source I cannot comment, but do all your IF statements have a THEN before the GOTO? Crunchie needs THEN to follow the syntax – it is not really very clever, but it will remove the THEN for you.

A lot of errors (e.g. missing "" or “no such mnemonic”

The assembler is a minefield for Crunchie. It was never included at the beginning. As it stands it requires opcodes in uppercase, because that is how I do it. However, I have now allowed lowercase, but not mixed case. It does not do any syntax checking.

I have just fixed a bug with CHR$ that could result in Missing ". It allowed CHR$&0D, a CR, to be inserted as a literal string, but BASIC always interprets a CR as the end of a line, even when in a string. This leaves a hanging endless literal string.

More problems with the source, at least the line of code, are welcome.

Should the crunchied program be:
– viewable with !Edit?

Possibly. It could help to identify problems, but it is largely unreadable.

– edit and saved with !Edit?

No. There is no point. Edit the source.

It is ironic that one of Joe’s ambitions with AppBasic was to have it all in BASIC so that it would work without change on future versions of RISC OS.

I echo the other comments you made about unusable crunchers from the past, but that point strikes a particular chord.

I am caught in a quandry. Basalt does not work reliably on the latest versions of RO5, although it does on my mini.m. All my source uses the ExtededBASICAssembler module, which only runs on 26-bit systems and requires the BASIC 1.20 module to be loaded. I have recently swapped everything else onto the mini.m and I am left with the orphan code on VRPC. It is possible to use Aemulor, but it limits some things I like and it is still not very convenient. I have started on converting the source, but it is huge. There is lesson here about assembler versus high-level languages. ;-(

I have started on converting the source, but it is huge.

One can do quite a bit with StrongED’s Search and Replace. When the contexts play an important role I like to try a scripted solution to large scale conversions. It usually takes me multiple attempts before I get it right, but even so, for large sources it is quicker and less error prone than wading through and correcting by hand.

GOTOs? ;-)

Yeah, Yeah! I am working on it.

but do all your IF statements have a THEN before the GOTO?

Yes, but the ones that fail have lots of other code in between the THEN and the GOTO

EDIT: some hours later

Changing the code to

IF condition THEN

bla: bla: bla

GOTO 12345

ENDIF

Neither does

IF Condition THEN bla: bla: bla:

IF condition THEN GOTO 12345

Yes, but the ones that fail have lots of other code in between the THEN and the GOTO

I will have another look at that. It was not one of my priorities. Maybe it is something to do with the line number coding.

One can do quite a bit with StrongED’s Search and Replace.

A lot of that, but over multiple files (hundreds) it is difficult to undo and the question is how often to backup. I have it assembled, but there are inevitable new bugs.

A lot of that, but over multiple files (hundreds) it is difficult to undo and the question is how often to backup.

Revision (version) control? That gives you as many backups as you want, without any of the hassle of taking copies yourself.

The assembler is a minefield for Crunchie.

I not use assembler. I am happy if I get something to work in basic. ;-)

I have just fixed a bug with CHR$ that could result in Missing ".

Is the download changed? Looks like the same I have.

More problems with the source, at least the line of code, are welcome.

Have search the line with the “mnemonic error”. The “problematic line” is the third in the crunched part, beginning with ELSE…

WHEN “file_”
SYS"Wimp_SpriteOp",&28,,type$CHR$(0) TO ,,,w%,h%,,mode%
SYS"OS_ReadModeVariable",mode%,4 TO ,,xs%
SYS"OS_ReadModeVariable",mode%,5 TO ,,ys%
h%=h%<<ys%
w%=w%<<xs%
IF text$="" THEN
temp_blk%!8=y%-h%
temp_blk%!12=x%w%
temp_blk%!20=&1700613A
temp_blk%!28=1
temp_blk%!32=LENtype$
$text_space%=type$
ELSE
temp_blk%!8=y%-h%-(10<<ys%)
sl%=(8<<xs%)*LENtext$
IF sl%>w% w%=sl%
temp_blk%!12=x%w%
temp_blk%!20=&1700612B
DIM name_space% (2LENtype$)
$name_space%="s"+type$
temp_blk%!28=name_space%
ENDIF

WHEN"file_":SYS&400E9,&28,,P2$CHR$(0)TO,,,w%,h%,,W2%:SYS&35,W2%,4TO,,X2%:SYS&35,W2%,5TO,,Y2%:h%=h%<<Y2%:w%=w%<<X2%:IFo1$=""THEN
o0%!8=y%-h%:o0%!12=x%w%:o0%!20=&1700613A:o0%!28=1:o0%!32=LENP2$:$T2%=P2$
ELSE:o0%!8=y%-h%-(10<<Y2%):Q2%=(8<<X2%)*LENo1$:IFQ2%>w%w%=Q2%
o0%!12=x%w%:o0%!20=&1700612B:DIMU2% (2LENP2$):$U2%="s"+P2$:o0%!28=U2%
ENDIF

If selected, Crunchie reports progress in a Reporter window, from which a file can be saved and then analysed. This will be improved.

To have an option to use Throwback like the rest of the DDE tools?

You may have issues with using Edit to deal with crunched code. Maybe StrongEd as well. This is because these editors convert the BASIC to text for editing, and from text back to BASIC when saving. Some crunched sequences may trick the conversion into using the wrong tokens and…broken program. Some crunched tokenised sequences may also expand to be longer than the line limit when expressed as text and…broken program.
You can sometimes edit such programs using Zap (which deals with tokenised BASIC directly), but in general a munged program is best considered to be “read only”.

The moral, as I see it, is do not try to hide your source code. It is a sure path to obsolescence.

That’s why I won’t be using Crunchie. I am very happy to see it available for RISC OS 5, because I used to use StrongBS back in the day, but these days I don’t bother squishing stuff written in BASIC. It’s just too much of a PITA later on. Especially if the squished version is the one you have and the source is MIA.

All my source uses the ExtededBASICAssembler module

Maybe what we need is something that added the extra assembler stuff to RISC OS 5’s BASIC? ;-)

I know what you mean though – ExtBasAsm was lovely. When it came time to ditch it, I went one of two ways – I either rewrote the code in C, or I ported the code to ObjAsm. It’s been so long since I’ve used BASIC’s assembler that I don’t remember the default OPT settings. 0 TO 3 STEP 3? Or was it 0-2?

The assembler is a minefield for Crunchie. It was never included at the beginning.

Remember this?

Very clever. Very insane. Pain in the backside to deal with.

Maybe it is something to do with the line number coding.

It is only 5 figure line numbers which are failing for me. (but not all of those)