Stronger curiosity Or File Corruption?

I have had other oddities caused by the fact the SE tokenises keywords in comments.

I think that is what the BASIC routine MATCH does with comments after ‘;’. It does not after ‘REM’ comments. However, SE will still do the syntax colouring, which can be confusing. ;-)

Another gotcha is putting a ‘:’ in a ‘;’ comment, which the assembler takes as the start of another instruction.

I think that is what the BASIC routine MATCH does

Yup. ; and \ are not REM, and their effects are different. As you say, the assembler comments are tokenised and have a terminator, REM is not and does not.

THEN 3 Frames

Uggh! If only there had been 65280 frames.

Just for nemo, Basalt makes RESTORE NEXT a synonym for RESTORE+1, which may be a neater syntax. ;-)

And one byte shorter than RESTORE+TRUE!

Well blow me. Never would of spotted that Martin. I assumed as it was a comment it would of been ignored. Thats 2 things today I have learned.

Thanks very much :-)

Greg

I assumed as it was a comment it would of been ignored.

As others have said, any text after REM is ignored, but BASIC’s tokeniser, being line based, has no concept of the built-in assembler and so will try and tokenise anything on the lines between [ and ] – subject to the usual rules, of course.

This leads to some interesting things, not just the tokenisation of bits of comments after ;. Mnemonics like AND are tokenised in full, while ORR gets treated as one might naively expect. ISTR that the assembler bit of BASIC has additional code to handle the tokenised forms of anything that the tokeniser might have treated as a tokenisable keyword.

It was finding stuff like this whilst carrying out real world testing which made writing Tokenize such an in interesting and educational exercise. :-)

StrongED/Zap treat it differently
ie

LDR R0,[R1,#0] ;Then help THEN 3

Pity StrongED/Zap can’t change reg ‘0’ into ‘R0’.

Help check mistakes – with ‘#0’ being mistaken with ‘0’.

Zap bug.

Zap bug.

Chase Rick, chase.

See Rick chase the bug. :)

Create a Basic file in Zap, then open a new view and put it in byte mode.

Type a semicolon and then Alt-140, and wonder at the sight of Zap putting a token after the semicolon, but expanding it to display it, resulting in the cursor being in the wrong place.

Place the cursor after the semicolon and before the token, and press Alt-140 again, and wonder at Zap turning the original token into untokenised text while placing the new token before it. Such fun.

Zap bug.

Are you sure? Zap has bugs, but this one’s BASIC…

*BASIC
AU.
  10DIM C% 8
  20P%=C%
  30[ OPT 2
  40  LDR R0,[R1,#0] ; Then help THEN 3
  50]
>SAVE "$._tokerr"

Shift-dragging that to Zap shows:

...Þ C% 8....P% = C%....[ OPT 2..(' LDR R0,[R1,#0] ; Then help … ™TC@..2.].ÿ

Note what follows “Then help” is the tokenised THEN followed by a line number.

If the code was originally written in Zap, it would not have tokenised the stuff in the comments. StrongEd does, and colours it as keyword and line number.

So it’s only a Zap bug if you consider a bug to be doing something logical.

and wonder at Zap

Okay, that’s weird. But given as BASIC itself tokenises tokens in assembler comments, I’m not going to give this oddity any more consideration than the time it takes to write this sentence. ;-)

Such fun.

And there I was thinking “such fun” was gently tugging on the cat-handle while said cat was occupied doing something.

[health warning: do not do this to a cat that doesn’t consider you to be “an acceptable provider of snacks and strokes”]

Are you sure?

I presume that’s a rhetorical question. Have you actually tried doing what I suggested?

Create a Basic file in Zap, then open a new view and put it in byte mode.

The bug I am referring to is that while Basic tokenises assembler comments, as I said here:

the assembler comments are tokenised and have a terminator, REM is not and does not.

…when you type the same thing into Zap it does not tokenise. This is the bug.

Basic tokenises assembler comments, and does not tokenise REM. Full stop. If Zap’s Basic mode does something different, then it is wrong.

Well blow me. Never would of spotted that Martin. I assumed as it was a comment it would of been ignored.

As both Steves have explained BASIC’s tokeniser, which StrongED uses, is line based and it has no idea whether a line is assembly or not.

Another problem with assembler comments is using top-bit-set characters, next time the file is loaded they will be converted to keywords.

Anyway, thanks to Martin the problem has been found. I’ve changed the error message so that it includes the BASIC line number of the first line with a line number reference. That should make it easier the find out why line numbers aren’t being stripped.

That should make it easier the find out why line numbers aren’t being stripped.

Considerably! Thanks Fred. I had planned to suggest it … but you beat me to it!

Basic tokenises assembler comments, and does not tokenise REM. Full stop. If Zap’s Basic mode does something different, then it is wrong.

Unless BASIC itself is wrong?
What possible use is it to tokenise a comment in code that isn’t even BASIC?

And, no, hysterical raisins is not an excuse.

Unless BASIC itself is wrong?

BASIC is, by definition, perfect – you should know that, Rick.

Yes, I’m wearing a reflective metallic hat and asbestos underwear…

Just recently someone sent me an invoice with their bank sort-code and account number. When I tried to pay, the bank’s software complained that they were inconsistent. I telephoned the sender of the invoice; sure enough there was was a digit wrong, a 1 had become a 5. When I see an error that occurs because a single ASCII code has gained or lost a power of 2, i.e. a single bit, I suspect that a cosmic ray has zapped that bit. The word bit-rot comes to mind. But are there any statistics on bit-rot? Am I just perpetuating an urban myth?