ShareFS for Linux?

This is more in the ‘cloud cuckoo land’ class of pipe dream than the ‘nice to have’ one, and I know that networking RISC OS machines to Linux ones is technically possible, given the right protocols, faffing about with connections, trying to find client software that works, untwizzling the subtle incompatibilities, eye of newt and toe of frog, etc. I’ve never got any of them to work, myself, hence why I’m still using USB sticks and even remote FTP sites to transfer files between two computers sitting right next to each other and both connected to the same network.

Say what you like about ShareFS, but at least it just works. Many’s the time I’ve wished I could just type $ share ~ and have my home directory pop up on the ARMini or Iyonix, or access the contents of a RISC OS drive over the wireless on my laptop.

I don’t think the ShareFS protocol is even in the public domain, is it? I know the sources are still secret, but is there anything which would allow somebody to write a clone?

A collaborative reverse engineering effort?

David Boddie’s AccessPlusPython works for me, although not quite as easy as typing “share ~”. Get it from here. You’ll want the latest code from mercurial.

No joy, although the documentation is so sparse I could easily believe I was doing something wrong. Maybe the router doesn’t consider its wired and wireless networks to be part of the same ‘local network’? At any rate, it told me it had created a share, but it wasn’t one which any of the RISC OS machines could see.

It’s probably something to do with the broadcast address it uses to advertise shares. I think the code assumes that your netmask is 255.255.255.0. If it’s any different, you’ll need to modify the code.

If you let me know what your Linux box’s ip address and netmask are, I can tell you what to do.

You’ll also need to open UDP ports 32770, 32771 and 49171 on your firewall.

Oh, dear, this is exactly the kind of user-unfriendly nastiness which I was referring to earlier. I hate Linux networking…

I don’t know what a netmask is, let alone what mine is, but a likely string in the ifconfig output for wlan0 is “Mask:255.255.224.0”. I’m not on my local network ATM so I can’t report my IP address or test anything.

I have uploaded a new patch onto my fork of the code here

Download the code from the link on that screen, then run “./access.py wlan0”. The new code should work out all it needs to know. Fingers crossed!

I don’t know what a netmask is, let alone what mine is, but a likely string in the ifconfig output for wlan0 is “Mask:255.255.224.0”. I’m not on my local network ATM so I can’t report my IP address or test anything.

A net mask is run of bits that are set to let the system know that it can assume that those bits in the first portion of your network address can be assumed to be identical to others you wish to communicate with, all other addresses are considered foreign to the local network and are sent to the gateway address.
255.255.224.0 indicates that the first 19 bits are set to be local network address identifier. 255.255.255.0 would be a 24 bit mask and would allow for 255 local addresses. Since one is your gateway address and one is the broadcast address you are left with 253 for systems. For a home network, most default to 255.255.255.0 although most homes would fine operating with 255.255.255.240 (12 hosts plus gateway and broadcast) see CIDR references for a few decent tutorials.

A 19 bit mask implies 8090 IP’s, which is a very large broadcast domain and indicates that the local network is less than optimally configured. My colleague across the room would suggest that the network manager was “barking”…

Can I check that I’ve got the right end of the stick here, in that access.py runs on a Linux system, and allows a RISC OS machine to access shares defined in the .access file on that linux machine?

If so, I’m having some problems… Firstly, if I try to start it specifying the interface, I get this:

neil@raspberrypi ~ $ ./access.py eth0
br0:1: error fetching interface information: Device not found
Traceback (most recent call last):
  File "./access.py", line 244, in <module>
    PrintShareName = print_share_name(Hostaddr)
NameError: name 'Hostaddr' is not defined

neil@raspberrypi ~ $ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr b8:27:eb:ab:14:98
<snip>

If I start it without specifying the interface, I get this:

neil@raspberrypi ~ $ ./access.py
Starting...
Started sharing

Which looks promising, but when I try to logon to Access+ from the ShareFS icon, I get a message that nothing is being shared. Should I be using the same user ID and password that is used to start the access.py script on the unix machine? I’ve checked the permissions there, and everything checks out. For reference I’m using Raspbian linux on another R-Pi as the server, which is basically Debian compiled with armhf.

Thanks,

My mistake, sorry Neil. I left a hardcoded string in my access.py. I have fixed that in the latest version.

You shouldn’t need to log onto Access+, the discs are shared as normal ShareFS shares.

BTW, if you are planning on sharing out your home directory, then I’d be inclined to share it out read-only. You can share /tmp as read-write. I haven’t seen AccessPlusPython lead to data loss, but I would be inclined to err on the side of caution.

That works perfectly, thanks :)

Very nice. Just one showstopper for me: it doesn’t support translation of the character encoding (most Linux distros use UTF-8 these days).

Blimey, it almost works! Almost.

This time I got as far as getting a new shared disc to pop up in the ARMini’s discs window. I double-clicked on it, and it was added to the icon bar.

Then I clicked on the icon bar, whereupon the Linux terminal promptly spewed out the following:

Exception in thread Listener:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "./access.py", line 5971, in listen
    self.read_share_socket()
  File "./access.py", line 5046, in read_share_socket
    self._read_share_socket(s, data, address)
  File "./access.py", line 5719, in _read_share_socket
    infolist, trailer, path, handle = share.catalogue_path(ros_path)
  File "./access.py", line 2463, in catalogue_path
    path, rest = self.descend_path(path, check_mode = self.read_mask)
  File "./access.py", line 1990, in descend_path
    next_path = os.path.join(path, names[0])
TypeError: 'NoneType' object is unsubscriptable

Meanwhile on the ARMini, the infamous red ‘countdown hourglass’ started ticking, eventually followed by “The machine containing the shared disc does not respond”.

Here’s the output of ifconfig:

wlan0     Link encap:Ethernet  HWaddr 8C:A9:82:6B:9B:A2  
          inet addr:192.168.1.5  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::8ea9:82ff:fe6b:9ba2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2080 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1830 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1324990 (1.2 MiB)  TX bytes:272868 (266.4 KiB)

Note the Mask is back to 255.255.255.0. I was on a university network before!

Finally, just in case it’s relevant, this is the .access file I’m using:

Laptop       /home/Martin           0666    30.0        truncate        0xfff

I got the impression that it crashes when it encounters a file with a “difficult” name.

Frank: yes, it’s fairly old code. I’m sure UTF-8 translation can be added. All my filenames are plain old ASCII, so I haven’t had a problem with UTF-8.

Martin: That’s odd. I wonder if, as Wouter suggested, it is falling over a difficult name. As a test, it may be worth changing .access to point to /tmp instead of /home/Martin to see if a less exciting directory works better. We should be able to use that as a starting point for debugging. I’ll take a look at the backtrace you posted as well.

I don’t think there are any difficult names in my home directory, although looking at what it made of /tmp, we may have differing definitions of ‘difficult’.

Oh yes, changing it to /tmp worked, as in it didn’t crash and it displayed a directory which more or less matches up with the contents I can see locally. Unfortunately most of the files seem to be missing – there are 50 items in /tmp, but only 7 were successfully shared!

At least part of the problem seems to be that it attempts to strip file extensions, which means any filename with a full stop in it anywhere loses the bit afterwards, and /tmp contains a large number of directories called “virtual-Martin.xxxxx” where xxxxx is the only differing part, which may be confusing it. I would strongly advise that you delete the extension-removing code altogether.

The unreadable root-owned files are also missing. That doesn’t account for everything, though.

Now I’m going to attempt to read and write something…

The Linux filer window updates in real time! I wasn’t expecting it to support that.

Again, about the extensions: it really should be the user’s responsibility to add and delete them as desired, to avoid unpredictable behaviour. They’re not even compulsory for plain text on Linux. I copied a text file named ‘Help’, and it was added to the RISC OS filer window as ‘Help’, but the Linux one as ‘Help.txt’.

No other problems observed just yet. Would I be pushing my luck to attempt to stream an MP3?

EDIT: Deletion seems to work fine too.

EDIT2: Terminating the share with Ctrl+C doesn’t seem to remove it from Resources:$.Discs, although that may be a hangover from all the failures I had earlier (I haven’t rebooted the ARMini yet). Wait, scratch that – some time after I finished writing that paragraph, the share did actually disappear. All the same, shares on RISC OS machines always disappear fairly promptly, and the Linux share certainly has no trouble appearing.

This time, for the first time, I got an extra line of shutdown output:

^CShutting down:
Terminating the listening thread
Terminating thread for share: Laptop
Terminating the polling thread
Closing files
Finished
Terminating the listening thread
Terminating thread for share: Laptop
Terminating the polling thread
Closing files
Exception ValueError: 'I/O operation on closed file' in <bound method Peer.__del__ of <__main__.Peer instance at 0x1a64b90>> ignored

I have thus far been unable to transfer any files in the opposite direction, from Linux to RISC OS, due to the aforementioned problems with RISC OS apparently not seeing the whole directory. There may be a special rule for what I have to call files to get RISC OS to see them, but I haven’t worked out what it is yet!

That’s slightly good news, then.

The extension deleting thing will explain why not many files in /tmp are shown to RISC OS. I think that the code was originally written to use MimeMap, which would explain the extension stuff you are seeing. Somehow you have to encode the RISC OS filetype on the Linux host, either by adding a Unix style extension, or a RISC OS style ,xxx filetype extension. I’d imagine if you asked two people their opinion on this, you will get three opinions.

Streaming an MP3 should work… I’ve copied some fairly large files around, so it would be worth a try.

The shares are broadcast once every 30 seconds. If RISC OS doesn’t get a share broadcast after some time, it assumes the host has died and removes it from the list. I’d imagine RISC OS sends a “Share X is shutting down” message that access.py may not be sending correctly. I’ll need to check that.

If you had a unique filename + extension file in /tmp, that should transfer from Linux to RISC OS. Perhaps it was a bit silly suggesting pointing the share to /tmp… Could you not even open the Help file you created?

Files which I copied from RISC OS to the Linux share displayed and could be opened on both systems. Files copied locally to /tmp from elsewhere on Linux were not so lucky.

And no, it’s not just extensions which account for the absence, there are a large number of missing directories with no full stops in their name at all. (Oddly, the one directory which was visible – “jna” – also had no full stop.)

As for preserving filetypes, anybody attempting this should know that they’re transferring data between two incompatible filing systems, so they make allowances for it. Type data doesn’t need to be preserved, since the point is to transfer files to Linux, not via Linux to something else running RISC OS – ShareFS already does that! Automated filename extension fiddling causes way more problems than it solves.

Like email, if it’s important, you can always zip it.

EDIT: Put another way, do you think the Linux attributes of ‘user’ and ‘group’ ownership should be bodged into files transferred to RISC OS?

EDIT2: For the avoidance of doubt, I tried renaming the file I was attempting to transfer to RISC OS to have an extension. Still didn’t appear.

I think that if the user running access.py doesn’t have read access to a file (or directory), then it won’t be displayed in RISC OS. But if you copied them from elsewhere on Linux, then you should have read access, so that’s a bit odd…

You could try changing “truncate” in your .access file to “suffix”. I don’t think that will necessarily stop it adding a .txt suffix, but it will stop access.py stripping extensions from files when sending directory listings to RISC OS.

I think if I copied a sprite file onto ShareFS, I’d expect to be able to double click the sprite file to load it into !Paint. This can only happen if the filetype info is saved on the server. Likewise, I keep various apps on my ShareFS server so that RPCEmu on various machines can access them. So, they need their filetype info saved. I’m sure an option can be added so that RISC OS filetype info is lost when copying to a share, but that’s not how it was orignally written.

Changing “truncate” to “suffix” fixed nothing, although, of the seven files which do appear, five of them now have extensions which they didn’t before. The sixth one never had an extension (but had always appeared), and the seventh one is the “jna” directory.

And I’ve checked some of the directories which aren’t appearing, and I’m the owner – as I was of the file I was attempting to transfer.

Just tried saving a brand new file called “TestShare.txt” into /tmp, and that also failed to appear.

I strongly suspect that the directory indexing is silently failing on a certain file – perhaps related to the bug I saw in my home directory earlier? – and so only the entries it has already read are being reported.

EDIT: I’m astonished that there are enough file extensions defined to make running RISC OS applications from Linux anything approaching reliable. If you’re not using ,hex notation, what’s the extension for a sprite file? An Obey file? A module? An Absolute file? A BASIC file? A template file? The mind boggles.

Sounds like there quite possibly is a bug in the directory listing on your machine. I’ll have to look at some debug stuff tomorrow.