ShareFS for Linux?

EDIT: I’m astonished that there are enough file extensions defined to make running RISC OS applications from Linux anything approaching reliable. If you’re not using ,hex notation, what’s the extension for a sprite file? An Obey file? A module? An Absolute file? A BASIC file? A template file? The mind boggles.

Sprites are “.ff9” suffixed, picked up by OvationPro and DPScan.
My BASIC detokeniser recognises “.basic” (because VB has already claimed .bas).
The rest? Doesn’t matter, obey/module/template/etc are about as useful on non-RISC OS as .EXEs are to RISC OS, so the most logical thing is <name>.<hex-suffix> and not worry too much about what the file is. ;-)

[I prefer hex extension to the ,xxx notation for moving files around as it will survive an 8.3 filesystem whereas comma-suffixes won’t; although to be frank, my preferred method is to dump the files into a zip using SparkFS and then nothing gets trashed so long as the file is similarly unpacked with SparkFS on a RISC OS machine…]

Martin: I think your home directory has permissions of 0700. I recreated your issue of sharing out your home directory with those permissions, and have fixed that.

I have fixed the issue where shares don’t disappear from the Discs window when access.py shuts down.

As for newly created files not appearing, RISC OS chaches directory listings and will resolutely refuse to re-read the listing unless the server end sends a message that the directory has changed (I think… I need to get wireshark to confirm that), or if RISC OS writes a file to the server, or if RISC OS remounts the disc. So, the temporary work around is to remount your disk if you write something to your Linux disk after you have mounted the share on RISC OS.

As for newly created files not appearing, RISC OS chaches directory listings and will resolutely refuse to re-read the listing unless the server end sends a message that the directory has changed

I suspect it’s more complicated than that, what with already-existing files also not appearing and all.

It’ll have to wait until this evening or tomorrow for me to test.

Martin asked:

If you’re not using ,hex notation, what’s the extension for a sprite file? An Obey file? A module? An Absolute file? A BASIC file? A template file? The mind boggles.

This is identical to the HostFS problem, which ‘foreign’ ShareFS implementations must also implement. Unfortunately it’s not a simple problem (though it may at first appear so) and cannot be solved one filename at a time (which is the usual mistake).

The answer to your question is that the Host (HostFS, or this script) will map certain RISC OS filetypes to file extensions, and the rest must be encoded as hex postfixes. Many file extensions do make sense of course, but there’s not usually any value in applying extensions to RO-specific files (but there are exceptions).

I have seen .spr for Sprites and .oby for Obey files – there are utilities on other OSes that can translate Sprite files to other image formats, and an Obey file can be reasonably edited in some other text editor, so having file extensions is useful for those. BASIC is often .bas (eg BBC Basic for Windows).

However, for Modules, Absolutes and Templates there’s no value in having extensions so the ,xxx postfix is used instead.

It should be pointed out that the reverse mapping will depend on Host OS – on Windows a new TextFile should have a .txt extension but, as you point out, on Linux it may be better to use no extension at all. This mapping is part of the Host implementation, and ought to be editable by the user (as it is with HostFS for example).

However, as has been demonstrated, the Host cannot decide on extension/postfix removal on a per-filename basis. If a directory contains File.txt and File.jpg (which is quite common, sadly) then they cannot both be reported as “File” to RISC OS. It may be that “File” and “File/jpg” would be best, but that has to be maintained even if the Host deletes “File.txt”, which implies a persistent RO/Host filename map.

I’ll leave all the implications of this one-to-many mapping problem to the reader, but it is fair to say that there hasn’t been a perfect implementation yet… but it is one that should be solved. For consistency, I wonder if it could be internalised in RO itself (rather than relying on the various DOSFS, HostFS, SambaFS implementations inventing their own).

One final thought… with the increased prevalence of journalled filing systems, it would seem that many Hosts could now store RO filetype metadata separately from the filename. I suspect that nevertheless, transparent extension/postfix handling is more useful.

.spr for Sprites and .oby for Obey files

And apparently both .drw and .aff for Draw.

Using a hex code as a file extension is a Very Bad Idea, which is why a different separator is used when the hex is appended – usually comma, though I seem to recall seeing semicolon somewhere.

For the record, .aff is Firefox’s dictionary file extension, and I suspect there are rather more of them than holidaying Draw files.

it would seem that many Hosts could now store RO filetype metadata separately from the filename

Which is precisely what MacFS did, all those years ago. Bravo, Monsieur Tytgat!

James, your first edit today (d9898e2 Refactor code that specifies which interface to use. ) broke something, that version and later I don’t see any shared drives.

Wouter, I changed the command line argument parsing code to use getopt(). You have to type “./access.py -i ” now. I should have mentioned that somewhere, sorry!

I realise that I’m resurrecting an old thread, but I was delighted to see that ShareFS for Linux was even possible!

However… although the share pops up on the BBxM, clicking on its icon just produces a couple of hourglasses and “The machine containing the shared disc does not respond”. I had to alter the Hostaddr variable (to “192.168.16.47”), and I thought it better to specify -i eth1 because eth0 on the motherboard exists but is broken (but leaving it unspecified produces identical symptoms). There is no diagnostic info in the terminal window.

What do I do next?

Can that – I didn’t realise that my Linux box has a firewall active. A first glance suggests that it works now I’ve added some rules.

I seem to remember from the other thread that it requires ifconfig to be installed but that isn’t always the default now on new Linux distros (Debian for example has switched to “ip” instead). Not sure if thats any use…?

Looks like it now uses /sbin/ip so you should be OK on Debian.

https://bitbucket.org/dboddie/accesspluspython/commits/0248adae16a317d17b7d8954e0630a94c65039f5

Nice. I didn’t have the latest source in my working directory when I looked for ifconfig but it seems that James fixed that issue last year. I’d forgotten about that. James is pretty much the only person developing this software these days given that I don’t really have a need to run it.

If I ever get round to networking RPCEmu on my Linux system then it’s possible that I might play with file sharing between host and emulator, though HostFS already serves that purpose very well.

I’ve had a play (I used the James Woodcock fork). Anything that is dragged into the directory viewer at the RISC OS end appears instantly at the Linux end. Deletions likewise. However, things that are dragged in at the Linux end don’t appear at the RISC OS end immediately, even after Refresh at the RISC OS end – although I see it has turned up later. It seems that they only appear and disappear at 30 second intervals, which would equate with the period at which routine broadcasts are sent out.

David Boddie, is your version now more up to date than James Woodcock’s?

It seems that they only appear and disappear at 30 second intervals

That wouldn’t particularly surprise me. I don’t think the Linux end watches the filesystem for changes, so it wouldn’t send notifications to the RISC OS end.

I’m currently on holiday, so can’t look at the code easily.

David has pulled all my changes, so it shouldn’t matter which repo you use.

OK, thanks, James.

Having nothing better to do, I’ve just been searching for any Python code that could watch a folder for file creation/deletion/etc events. I’m out of my depth. I can find inotify and watchdog API libraries (and warnings not to use pyinotify), but there seem to be some difficulties. One is that they seem to be blocking calls (I may be wrong). Another is that you’d either have to do a recursive search for events, or the Linux end would have to know what folder(s) the RISC OS end is watching; is that even possible?

I don’t remember much about the way it all works. The 30 second broadcast sounds familiar. It may be that RISC OS peers keep track of which directories or even filing systems other peers have open and either send broadcasts more regularly, or perhaps just when the local system accesses them.

I’m guessing that you would use inotify to get information about file/directory changes, so that should be simple enough. It would also be fairly straightforward to keep a cache of the directories accessed by other peers, though it would be more efficient if entries could be removed from the cache as peers stop watching those directories. Does anyone know if there is a message to indicate that a peer is no longer interested in updates?

All this has set me thinking about how ShareFS (or anything like it) could work. Its main feature is that it’s one-to-many.

When a change is made to a shared folder or drive, presumably there are two ways to get remote directory viewers updated:

1) The server broadcasts every change. The clients each decide whether they need to act on the information.

2) Each client declares an interest in each viewer that they have open (and rescind the interest when they close it). The server has to maintain a list of interests, but only has to send relevant changes to interested parties.

It would also be fairly straightforward to keep a cache of the directories accessed by other peers, though it would be more efficient if entries could be removed from the cache as peers stop watching those directories. Does anyone know if there is a message to indicate that a peer is no longer interested in updates?

I just tried an experiment. BBxMRAM is shared. On a Raspberry Pi, I opened and closed a viewer on BBxMRAM twice while running Wiresalmon on the BBxM. The only Access/ShareFS/Freeway messages were the 30 second updates. Which is a pity.

I just tried an experiment. BBxMRAM is shared. On a Raspberry Pi, I opened and closed a viewer on BBxMRAM twice while running Wiresalmon on the BBxM. The only Access/ShareFS/Freeway messages were the 30 second updates. Which is a pity.

Yes, I think I would expect that – if the client hasn’t been told that the server has changed, then it won’t re-read the directory listing. It will just used the cached listing.

What you might want to try is to open the viewer on the Raspberry Pi, then, on the BBxMRAM, create a file in the shared directory to see if the BBxMRAM sends a unicast or broadcast message to tell clients that it has been updated.

To keep things simple, start working on the root directory of the share, then see how subdirectories behave.

I have had chance to look at the code. The fix to make filesystem changes appear instantly on the client is fairly simple.

I’ll tidy up what I’ve done and check it in at some point soon.

Anyone know where this has gone? Neither the link on this page or the last is working for me.

Bitbucket deleted all their Mercurial repositories so it migrated to GitLab. Look at this repository for the latest changes.