Partition Manager

There is no real garbage collection in BASIC, AFAIK – and I don’t think there ever has been.
String allocation can lead to wastage if there are lots of strings which change size in certain ways, but I suspect it was done like that so that allocation is very fast. That is why I asked for an example, and if it was all types of variable.

If by ‘leak’ you mean that they use more memory each time the PROC/FN is called, then can you give an example?

Not really, beyond the fact that after creating/deleting a few partitions BASIC has exhausted its memory.

Strings could be part of the problem as it uses quite a few of them, I’ll try reworking any that adjust the string length. Memory block DIM’s are almost certainly an issue as I need local versions of a lot of them – I’ll have to write my own Heap handler to deal with those.

And just to compound matters, I may have found a buffer overrun in HForm, as I’ve had to extend one of its block DIM’s to prevent random crashes whilst formatting a partition. I actually have 3 issues with HForm which I need to investigate and potentially report as bugs.

I spent most of yesterday comparing identical drives partitioned in Windows and PM, which highlighted a few +1 issues. Now that I know the GPT is correct, I’m certain the reason drives aren’t showing is not partition related. I’ll probably implement the FAT formatter as that’s trivial compared to FileCore – I only spotted late yesterday that FAT32 partitions appear as drives with PartMan loaded.

Is there a formatter/partition utility available for PartMan? My next step is to compare a partition that PartMan recognises, with one I’ve initialised.

Strings could be part of the problem as it uses quite a few of them, I’ll try reworking any that adjust the string length. Memory block DIM’s are almost certainly an issue as I need local versions of a lot of them – I’ll have to write my own Heap handler to deal with those.

If you’re repeatedly using memory block DIMs that is probably your problem, since this will leak memory, while using LOCAL variables will not.

You don’t need to write your own heap handler, you can use DIM LOCAL as long as you RMEnsure a version of BASIC that supports it (1.34 and later). Don’t forget that the latest version of BASIC can easily be softloaded on machines running older versions of RISC OS (as long as it’s done before any BASIC Wimp programs start).

Memory block DIM’s are almost certainly an issue

If they are in PROCs which are called repeatedly, they will certainly leak memory. But as James says, any BASIC from v1.34 onwards (19 years ago!) makes DIM LOCAL so simple, and the later versions are easy to use on any version of RISC OS.

If you have not discovered the Reporter command *ReportMem $ it displays the storage used by variables, and the free string space blocks – for example

Memory: Prog=566 Vars=512 Free=10,912 Stack=552 Undefined=0 Slot=16384 
$Size  Blocks    Free
  136       1     136
  236       1     236
Total       2     372

you can use DIM LOCAL as long as you RMEnsure a version of BASIC that supports it (1.34 and later)

Quite, that was my preferred option 3 above. I need to modify the code and add END to the debug output so I can see where the leaks are occurring.

Actually, BASIC 1.39 or later is safer, to avoid a possible clash with v1.37 in RO6.20? Safer still are very recent ones, which are standalone and do not rely on BASICTrans, so are easily softloaded.

What am I doing wrong, I get the error Items can only be made local in a function or procedure with the following sequence (with BASIC 1.81 under RO3.71):

PROCtest
END

DEF PROCtest
LOCAL A%()

DIM A%(1)
DIM B% LOCAL 64
ENDPROC

The LOCAL A%() appears to break DIM LOCAL.

The LOCAL A%() appears to break DIM LOCAL.

The order of creation of LOCAL objects matters. DIMming local arrays must come last, so change the order and all should be fine.

Edit: This is not clear in any BASIC manual, I think, but is laid out in my StrongHelp manual

Thanks, I’m working on reordering now…another thing I’ve noticed is DIM LOCAL causes stack corruption and BASIC aborts when you declare the variable again.

Adding a preceding LOCAL < var > appears to resolve the issue:

LOCAL block%
DIM block% LOCAL 64

Strings could be part of the problem as it uses quite a few of them, I’ll try reworking any that adjust the string length.

It would be very strange for normal use of BASIC string variables, especially local ones, to exhaust memory in this way. An artificial use of strings can do it, as Martin says, but you would have to be quite perverse to do it; ;-)

Memory block DIM’s are almost certainly an issue as I need local versions of a lot of them

Claiming memory for local use has always been tricky, but DIM LOCAL is certainly a great help. It may not be essential though, but to suggest alternatives I would have to have look at the code – aah! there are a lot of DIMs inside DEFs. More to follow. ;-)

I’ll have to write my own Heap handler to deal with those.

I can offer you a pretty straightforward single BASIC routine to manage memory above HIMEM. If you want to search it out you will find it in the ‘Source’ directory within my Strings library

I discovered a while back that, although DIM block% LOCAL 64 creates the block locally, the variable block% is NOT local – in most cases you’d want to declare block% as a local variable before the DIM LOCAL. In fact I can’t imagine why you’d ever want to have it as a global variable, so it surprises me that DIM LOCAL behaves as it does – it seems to me that it ought to make the pointer a local variable as well as the DIM’d block.

Adding a preceding LOCAL < var > appears to resolve the issue:

Oh yes! The < var > holds the pointer to the DIMmed block. If it is not made LOCAL it will be global and still pointing to an address somewhere around the stack that is no longer valid.

You don’t need to write your own heap handler, you can use DIM LOCAL as long as you RMEnsure a version of BASIC that supports it. […] Don’t forget that the latest version of BASIC can easily be softloaded on machines running older versions of RISC OS

That looks like a recipe for disaster. True, a single copy of BASIC can be softloaded on a machine with BASIC in ROM, but it cannot be done more than once. Using RMEnsure in this program would do that, but what if the user or another application has already RMEnsured an earlier version number?

The proper way is to offer the user of the older machine the opportunity to install the new version of BASIC if they are to run your program. How this installation would be done has been raised for discussion before, and I suspect it would be something like installing regularly updated versions of SCL and FPE in the old days.

In fact I can’t imagine why you’d ever want to have it as a global variable, so it surprises me that DIM LOCAL behaves as it does – it seems to me that it ought to make the pointer a local variable as well as the DIM’d block.

That looks very persuasive, but consider, say:

FOR ptr%=block% TO block%+size%:<do it>:NEXT ptr%

block% might be local, but unless you explicitly make it so, ptr% is not and will be left invalid on exit. Best to make all the necessary variables local and not rely on a side effect. ;-)

That looks like a recipe for disaster. True, a single copy of BASIC can be softloaded on a machine with BASIC in ROM, but it cannot be done more than once. Using RMEnsure in this program would do that, but what if the user or another application has already RMEnsured an earlier version number?

The proper way is to offer the user of the older machine the opportunity to install the new version of BASIC if they are to run your program. How this installation would be done has been raised for discussion before, and I suspect it would be something like installing regularly updated versions of SCL and FPE in the old days.

Sorry, I was being a bit unclear when said “RMEnsure a version of BASIC that supports it”, what I meant was RMEnsure that you have a sufficiently new version of BASIC and refuse to run if not. As you say, attempting to RMLoad a new version would be a bad idea – that wasn’t what I was suggesting. Taking into account Martin’s note about RISC OS 6, something like the following in !Run would do:

RMEnsure BASIC 1.39 Error This application requires BASIC 1.39 or later (installing 1.75 or later recommended)

Then include a note in the documentation about how to download a new BASIC from this site and load it by adding an Obey file to Choices:Boot.PreDesk

The reason for recommending 1.75 or later is that works on everything back to RISC OS 3.1 – the latest version is 1.81 but the changes post 1.75 are fairly minor.

Sorry, “run in 1MB” and “support ARMv2”?

We really should not be worrying about such old machines. They will never run a new version of Filecore or support larger discs, so their existing copies of HFORM are adequate, and there is no benefit to using a newer version.

The oldest machine which can sensibly use a newer Filecore and benefit from a new HFORM, is a Risc PC soft loading RISC OS 5. So at least 8MB and an ARM610.

True, a single copy of BASIC can be softloaded on a machine with BASIC in ROM, but it cannot be done more than once

Which is why I stated “run the code in its own BASIC environment on older machines”. I’ll make a bespoke version of BASIC to avoid a conflict with whatever is currently running. Bailing out early telling folk “update your BASIC” is not an option as it’s very likely an older machine will only have one drive – which you’re trying to initialise.

We really should not be worrying about such old machines. They will never run a new version of Filecore or support larger discs, so their existing copies of HFORM are adequate, and there is no benefit to using a newer version.

If you read the OP, you’ll see RO3.x is one of my main targets for this. That includes RO 3.0, with all it’s restrictions and RO 3.20, where I plan to backport long filenames and partition support.

Has it been established how many users would benefit from an effort to bring modern features to the Archimedes class machines which could run 3.2x?

I suspect it wont be many, and that benefit would be very small.

Having used that class of machine to soft load most of RISC OS 3.x while beat testing for Acorn, it’s really not a great experience. Newer code designed for more recent machines is larger and slower. There isn’t any possibility of creating a new masked ROM, so a good chunk of the available RAM is needed, ruling out the 1MB and 2MB machines, restricting it to the A440, A4 and A5000. Even once it’s loaded the ancient IDE interfaces on those machines (motherboard or Podules) isn’t going to allow any larger discs, negating most of the benefits of the newer Filecore, and it’s pretty pointless partitioning tiny discs.

You could set up an emulator to give 16MB on on an Archimedes class machine, and soft load just about all of RISC OS 5 features, but FileCore isn’t much use on an emulator, and why not just emulate something more recent?

So in essence is it really worth the additional development time to make Patitional Manager and new HFORM work a the tiny number of old machines, which wont gain much if any benefit, and if still in use are likely to want to retain 100% compatibility with RISC OS 3.10?

There isn’t any possibility of creating a new masked ROM, so a good chunk of the available RAM is needed, ruling out the 1MB and 2MB machines, restricting it to the A440, A4 and A5000.

Ah, you are far behind reality in this case.

A Stardot member did a design for a flash-rom based ROM adapter, for every old machine, with 4 MB capacity provided to RISC OS (with a larger flash capacity of course, and a boot screen allowing you to switch between different versions). Hence the project “RISC OS 3.2x”, so that you can actually save some RAM by putting everything inside ROM – the IP stack, the nested WIMP, the updated SharedCLib, assorted NIC drivers, Toolbox modules, and the updated FileCore.

It is a big project, and I think the hardware is not yet 100% reliable at the moment. And it turned out to be complicated to cater for all machines – 4-ROM, 2-ROM, different spacings between chips…

There were enough problems trying to use a flash carrier in the Risc PC for Select development, so I don’t think making even more aged machines less reliable, and likely to brick, a very tempting proposition.

If it works reliably, Flash ROM is a much better option, because you can switch between RISC OS versions easily. I don’t think there are many users from the retro scene that would like to have RISC OS 3.2x exclusively, because it is likely to break existing software – and existing software is the sole reason to run classic computers. Switching between Arthur, RISC OS 2, RISC OS 3.1x and RISC OS 3.2x easily (i.e. without physically changing EPROMs/PROMs/ROMs) is very attractive.

Whether the solution now created will be reliable – we’ll see. Testing is ongoing.

Other systems had such switchable OS ROM constructs for a long time now, so I don’t see a reason why it shouldn’t be possible in principle.

There were enough problems trying to use a flash carrier in the Risc PC for Select development, so I don’t think making even more aged machines less reliable, and likely to brick, a very tempting proposition.

I had the original RISC OS 4.00 in Flash on those carrier boards. Worked without problems, but I heard of others having problems. The Risc PC was always a very timing-sensitive machine. C32, StrongARM, RAM, PC card, podules – every change could make it unreliable. We’ll have to see what happens with the Flash ROM. I don’t think the old machines are as sensitive – remember the RISC OS 3.1 update for the first generation Archimedes machines, those were sitting on carrier boards, too.

I’m afraid this is turning something which can provide some very useful future functionality in to yet another lets make it work on ancient machines that few people are using, and even fewer people would want to upgrade. Concentrate on developing for what people are using now, and let someone else, if they so wish, port it to RISC OS three point whatever.

Concentrate on developing for what people are using now, and let someone else, if they so wish, port it to RISC OS three point whatever.

Isn’t it Jon – who’s writing the Partition Manager – that is targetting 3.20 more than 5.x?

I believe I’ve now resolved the memory leaks, thanks to all the advice from people above. PartMan is happily picking up and adding drives for partitions as I create them. FileCore formatting is still not working, so I think I’m going to drop the HForm code route and spend a few months figuring out and documenting FileCore, to the point I can write my own formatter. I’ve also started documenting SCSI on the Wiki, which is going to take a few weeks.

I believe PartMan only supports SCSI, so I’ve yet to test SDFS partitioning. I need to add FAT formatting so I can swap drives to a PC to verify that. ADFS only supports full disc FileCore, so that should just work once I have the FileCore formatter implemented.

As mentioned previously I also need to figure out how to handle errors, in particular handling failures when writing the primary partition table. My initial thought is to present a dialog informing that all data is lost…drive faulty. At this point I can only abort the partition table write and either quit, or risk it crashing when it tries to read it back. The disk will unfortunately be in an indeterminate state. Provided a backup was taken of the partition table (yet to be implemented) you could attempt to write it back, but your chances are slim and a corrupt partition table will cause problems. I’ve crashed PartMan with a corrupt GPT for example.

Isn’t it Jon – who’s writing the Partition Manager – that is targetting 3.20 more than 5.x?

It was me the last time I looked – every spare minute since Dec 30th so far. I don’t personally use RO5 other than to check things run on it and I’ve clearly stated what is in/out of scope for this project in the OP, so I’m not sure why some people are now questioning how I spend my unpaid time. There is a paid bounty that covers partition management, I would direct people to that if they want a RO5 only tool.

If people have constructive input for RO3.20, can I direct them to the thread on StarDot.

For discussion of the project to replace Acorn ROM’s with Flash, please visit this StarDot thread.

For discussion of the project to replace Acorn ROM’s with Flash

I cannot resist noting how similar this is to modifying my classroom of BBC Bs to take 64k EPROMs and static RAM sideways memory; flash was not really in the picture then.

This was not one of those plug-in boards from Watford etc, but modification of the circuit board. Surprisingly, the board was actually designed to allow this.

Sorry for the diversion.

I believe I’ve now resolved the memory leaks,

Did you do this without DIM LOCAL? My swift look at your code suggested that it should not be necessary, if you set up global re-useable buffers.