ShareFS protocol or opensource module
Phil Pemberton (7989) 65 posts |
I’m playing around with David Boddie and Mark Woodcock’s AccessPlusPython server software, with the intent of fixing some of the bugs (notably the one that’s clobbering filetypes when I try to compile code across an Access share). I’m curious if it’s a protocol issue. I noticed that the ShareFS module is only present in the Git repository as a binary blob. With that said, Freeway is there, so network discovery is at least documented in some fashion. Is there any documentation available on the ShareFS protocol, or any plans to open-source the ShareFS module with the rest of RISC OS? Cheers, |
dgr (375) 16 posts |
I can’t help with the documentation for the ShareFS protocol but I was also poking around with AccessPlusPython the other day after a hankering to get my Pi booting via a network !Boot. I had AccessPlusPython running using Python 3 under macOS. I had to fix a few things to get it working correctly. IIRC, it really needs to use SO_REUSEADDR before binding any of the UDP sockets and the broadcast code needed some attention (e.g. rather than figuring out the broadcast address, just use |
Andrew Rawnsley (492) 1443 posts |
We (RISC OS Dev) have gained permission from all the original guys for the release of ShareFS etc, so if you get in touch with ROOL via email, they should be able to send you a copy of the source. Alternatively, get in touch with us at ROD, but it is better to have a proper submission chain via ROOL, in case you wish to feed back changes. |
Phil Pemberton (7989) 65 posts |
@dgr: I’ve already converted it from a Mercurial repository to a Github one (https://github.com/philpem/AccessPlusPython) — I’d be happy to incorporate your changes into that :) My email is philpem at gmail :) @Andrew: Cheers – will drop an email to ROOL shortly. |
Dave Higton (1515) 3497 posts |
Good news! Thanks, Andrew. |
James Byrne (3371) 29 posts |
For what it’s worth, I’ll mention that I did do some work on ShareFS getting on for 15 years ago (with the intention of making it work over TCP) that was never finished because a change of job meant I no longer had the time to spend on it. However, somewhere on an old hard disc I do have a version where the source code has been considerably cleaned up to make it more comprehensible, and I also have a notebook with some notes on how the protocol works. When the ShareFS code makes it into GitLab publicly I’ll try and motivate myself to dig all this out from the attic and make it available, because while I suspect I’m unlikely to find the time to be able to finish what I was doing I think it could be beneficial to anyone else who wants to work on it. |
Jeffrey Lee (213) 6048 posts |
Excellent!
Even better! |
John WILLIAMS (8368) 493 posts |
= less incomprehensible? |
Dave Higton (1515) 3497 posts |
Funny you should say that. It occurred to me a long time ago that a new version of ShareFS that worked over TCP might be a good thing – specifically on condition that it lost none of its ability to run with existing old versions of ShareFS, so no compatibility would be lost. The first question to ask is: would it be a good thing? We all seem to have got used to setting ShareFSWindow to 1. Well, that’s an easy default to change, but only the tiniest deal. I can imagine that it would be nice to lose the double hourglass, during which nothing else works, when communication is lost. But what else, if anything, is there to gain? |
Jeffrey Lee (213) 6048 posts |
TCP could be a stepping stone to SSL/TLS (although I’d imagine there’d still be many more stones on that path) |
Dave Higton (1515) 3497 posts |
That might make it possible to use ShareFS between two sites without needing to set up a VPN first, although the broadcast traffic might present an interesting problem. |
Colin (478) 2433 posts |
While having the too high a setting grinds everything to a halt I find a too low setting slows transfers down. At one stage I was using ShareFSWindow 8 but I can’t remember the combination of machines I was using for that to work. Switching to tcp has to be a good thing for maximizing performance. |
David Boddie (1934) 222 posts |
I’ve created https://gitlab.com/dboddie/accesspluspython from the original Mercurial repository on Bitbucket. I may well replace the original with a git repository once Atlassian delete it. |
Phil Pemberton (7989) 65 posts |
Thanks for that David, I’ll look at archiving my repository and linking it to yours. No sense in stepping on each others’ feet. I have seen the patch you left but haven’t had a chance to test it yet. Too much time spent playing with Econet! |
David J. Ruck (33) 1629 posts |
AccessPlusPython seems to work quite well on my Linux Pi4, but is very slow even under pypy. I was hoping it might help me track down the issue of ShareFS dieing after a few tens of thousands of files, when copying from an ARMx6 mini.m to an Iyonix. |
Colin (478) 2433 posts |
Does it die if you fetch from the iyonix instead of sending from the mini.m? Are you running a large wimpslot? If so does changing it back to 640k fix the issue? Basically anything you can do to slow down the fast machine when sending from fast machine to slow machine is a good thing and sometimes sharefswindow 1 isn’t enough. If it is a speed difference issue I would expect fetching from the slow machine to work as then it controls the data rate. |
Alan Adams (2486) 1147 posts |
I seem to recall from a discussion a few years ago, that when shareFS pushes a file, the remote end first creates the file, then fills it with zeros before accepting and writing the received data. This delay can cause timeouts for the sender. Does your sender always die when it reaches a significantly large file? |
David J. Ruck (33) 1629 posts |
Thanks for the tips. I normally only get issues when synchronising almost the entire disc between two machines, the changes are mostly small lots files, and I’ve had issues with both push and pull. I’ll try to set up a reproducible failure, and experiment with sharefs windows and wimp slots etc. |
Colin (478) 2433 posts |
Having just tried my iyonix for the first time in ages it is painful when used over VNC compared to my armx6 no wonder you have problems transferring lots of data. |
Colin (478) 2433 posts |
As the first program I saw after dusting off my Iyonix was Discknght I thought I’d do a test so I set |
Colin (478) 2433 posts |
I changed the flow control setting on my 8 port ethernet switch and that changed to 4MB/s |
André Timmermans (100) 655 posts |
The problem is probably that ShareFS probably creates and sets the filesize before copying the data to ensure that it has enough space on the target. You have two possibilities, creating the file and altering its size with SWI OS_Args,3 or creating the file directly with the correct size using OS_File,7. With OS_Args,3 the extended size is automatically filled with zeroes, with OS_File,7 the behaviour is left to the underlying FS. I had the same issue with Samba when I still had my RISC PC and that the connection timed out when attempting to transfer large files to the RISC PC. I had to recompile the provided sources with OS_Args,3 commented out to get it working. |
Chris (2061) 72 posts |
Does this still work? I have set this up on my DietPi NAS and nothing is picked up by any of my machines. Is there some weirdness to the config .access that i am missing? |
Paolo Fabio Zaino (28) 1854 posts |
You could try to run WireSalmon on your RISC OS machines to see if they are receiving the broadcasting from the SahreFS on Linux. http://www.cp15.org/networking/ I do not know if there are newer releases somewhere else, sorry. |
Timo Hartong (2813) 204 posts |
I saw Andrew has the code could I have it ?. |