RAMFS size limit

This is a hard-coded limit set at ROM compile time and appears to be independent of the dynamic area size limit set generally within !Boot.

At present the RAMFS maximum size is set to 128Mbytes and I would like to suggest it should be set to 500Mbytes – many machines have 1Gbyte or 2Gbyte of RAM and so a limit of 500Mbytes for RAM disc seems not unreasonable.

I know that development of sparse dynamic areas is being pursued separately but this limit could be adjusted now.

Not gonna happen. On a machine with 2GB of RAM there’s barely enough logical address space left as it is (due to the free pool being a regular DA).

However, the more noise people make about wanting bigger RAM discs, the higher priority I’ll consider my DA/memory management improvements, increasing the chances that I’ll actually do some work on it (since my last update in December I haven’t had a chance to do any work on it!)

So machines with more RAM have less to spare? Help! Why?

Anyway it has happened on the Panda – OSupdate 8 has set the limit to 500Mbytes (RISC OS 5.21 14-Feb 2015). Using RAMFS (and you need over 400Mbytes) cuts down the ROM compile time on the Panda from 14min to 4min54sec and the ROM unpack (UnTarBZ2) time from 348s to 41s. So it is quite worthwhile.

So machines with more RAM have less to spare?

Yes.

Help! Why?

Due to the free pool being a regular DA.

• 2GB free pool
• 512MB application slot
• 512MB RAMFS
• 256MB RMA
• 96MB misc (ROM, page tables, kernel workspace, etc.)
• 64MB screen (32MB doubly-mapped)
• 32MB PCI heap
• 32MB font cache
• 4MB + 4MB*numdrives per FileCore filesystem (e.g. SCSIFS uses 36MB due to hot-plugging support, ADFS with one floppy + one HD uses 12MB)
• 16MB system sprites
• 16MB wimp sprite pool
• 16MB toolbox DA
• 16MB PipeFS
• 1MB RTSupport

That’s about 3665MB, depending on filesystem setup. Take away some more space for memory mapped IO (HAL-specific, could be anything from 1MB to hundreds of MB) and you’re not left with much remaining for third-party apps. Use the machine for a while and you might find that the remaining space has fragmented and apps start refusing to load.

If we can stop the free pool & RAMFS using ridiculous amounts of logical address space then that will open the door to ‘unlimited’ RAM disc sizes (only limit being as much free RAM as you have), and could also reasonably allow us to up the application slot limit to 1GB (or perhaps make it CMOS configurable up to about 3GB)

What about using !Memphis – as a temp situation?

What about using !Memphis – as a temp situation?

Works, been using it for some while. You need to use the Wayback machine to find a copy.
riscos.info points to a dead link

Edit: Correction, you need to go to http://www.paymentlabs.com/riscos/tutorials/memphis

Memphis, from what I remember was faster than ram disc as well. I haven’t benchmarked that, but I’m fairly confident that is true.
Which reminds me I should try the save/load scrap option again, it maybe stable after the cache updates on my arminix

Err – you could perhaps change the ‘message’ file – to point to your Web site :-)

Due to the free pool being a regular DA.

I think I now understand. Would it be possible on a 2Gbyte machine to have an option that, on start up, reduced the free pool to 1.5Gbyte and fixed the RAMFS size at 0.5Gbyte? Until sparse memory is sorted out of course.

A thought occurs to me. What do you do in the case of the IGEPv5 which has 4Gbytes of RAM?

RISC OS does only use 2032 MiB on the IGEPv5 (Full Edition) with 4 GiB RAM.
There is a reserve of 16 MiB at the end to avoid overflow in address calculations.

If we fixed some of the bugs in the OS and got rid of the free pool DA then I think we’d be able to push that up to 3GB (IIRC the OMAP5 has its main IO region in the first GB of the physical address space, and everything above that is dedicated to RAM?)

To get the full 4GB we’d have to update the OS to support the large physical address extension, and update lots of code/APIs so that they treat physical addresses as 64bit instead of 32bit.

However, the more noise people make about wanting bigger RAM discs, the higher priority I’ll consider my DA/memory management improvements, increasing the chances that I’ll actually do some work on it (since my last update in December I haven’t had a chance to do any work on it!)

Are any OS changes needed for this or is it entirely a UnixLib problem? (Constant DA growth, even though NS isn’t actually using loads of memory.)

Are any OS changes needed for this or is it entirely a UnixLib problem?

Mostly a UnixLib problem. Sparse DAs have existed since RISC OS 3.8 (although prior to RISC OS 5.16 there was a bug which meant that errors during grow/shrink operations would sometimes be lost). So on RISC OS 5.16+ it should be relatively easy to change UnixLib to use sparse DAs and have it release unused pages from the middle of heaps.

If you have problems with fragmentation (especially if you’re only dealing with large resources like images, HTML files, etc.) then it would probably be worth looking into using a fragmentation-free sliding heap for storing your cached resources. Since that will use the standard OS_ChangeDynamicArea grow/shrink interface it will obviously work on anything RO 3.5+.

If you want to cache “unlimited” amounts of stuff in RAM without using loads of logical address space then you’ll have to wait for my DA improvements. You’ll be able to create a small DA in logical space which acts as a window into a larger set of physical memory pages, with the ability to grow/shrink the physical pool and map/unmap pages into logical space at will. Of course this would also require your code to be much more flexible with how it handles the resources, but it will give you complete flexibility. About the only thing which it won’t be able to fix is fragmentation of the address space which DAs are allocated in (although I can’t see a reason why we couldn’t add support for defragging DAs in the future – as long as the owner of the DA can cope with the DA being moved, of course!)

The problem with NetSurf is that the core assumes a flat, sparse, well-managed VM. Changing it to a sliding heap or other indirect model would be huge effort and damage performance on other platforms. ISTR that RISC OS has a “private” bit for DAs, such that they are swapped out with the process, meaning address space isn’t needlessly wasted globally, but this was never actually implemented. Is this still the case?

ISTR that RISC OS has a “private” bit for DAs, such that they are swapped out with the process, meaning address space isn’t needlessly wasted globally, but this was never actually implemented. Is this still the case?

Correct.

Of course the problem with that approach is working out where the address space should come from – do we reserve xxMB of address space for private DAs? Do we allocate them from the regular DA address space? (trying to map them ontop of existing private DAs where possible) Or do we take it out of a task’s application slot?

A better approach might be to just add support for sparse application slots, so that programs have full control over what pages they use. I don’t think there’s too much work needed in order to make the OS support it. We could even fix system() so that it doesn’t restrict the amount of memory available to the child process!

Yes, I was going to say that given WIMP slots can be so much larger under RISC OS 5, it’s probably worth fixing them rather than DAs.