HostFS Character Encoding on Ubuntu 2.0

If I save a file to HostFS with RPCEmu 0.9.2 running RiscOS 5.19 containing spaces, I think I’m getting Non-breaking spaces or hard spaces in my filename. This is translated by Ubuntu 2.0 as an invalid encoding and looks like this “Does�This�Filename�Contain�Hard�Breaking�Spaces (invalid encoding)”. I’ve tried different versions of RiscOS and all the latin n alphabets on the RiscOS side to no avail.

I wonder is there a configuration setting for rpcemu which would stop filenames spaces from RO been translated into non-breaking spaces?

Thanks for all the hard work that’s gone into RPCEmu.

Andrew.

The translation from spaces into non-breaking-spaces happens on the RISC OS side of things. RISC OS cannot handle true spaces in filenames, so auto-translation of user input was a sensible cop-out strategy.

But of course in theory HostFS could re-map those. There is no configuration for such a mapping that I’m aware of – last time I looked, the translation was hard-coded.

Out of interest, what application?

Style Guide Save as dialogue boxes should have A~ validation string to reject spaces.

RISC OS cannot handle true spaces in filenames

My Unix support colleagues would say “spaces are command parameter delimiters” or some such.
They would also “helpfully” point a script at any identified directory where Windows users had placed items with spaces in the name and the script would rename the files by introducing an underline character instead…

Sunfish, LanmanFS and Lanman98 map between hard and soft spaces for remote filing systems.

But don’t use spaces (or top bit set characters) in filenames is the best advice if you want them to be used on multiple platforms.

Please, somebody write a comprehensive document about which mappings currently exist in the various FSes…I think the PRMs at least documented DOSFS’ approach on mapping the DOS to RISC OS world wrt reserved characters.

I didn’t realise that RO was introducing the non-breaking spaces. All this help has saved me quite a bit of time. Usually I blame RO as the last thing to look at during diagnosis.

!Edit on RO5.19 was used to create the filename with text character gaps. I do remember some discussion about RO allowing spaces in filenames on the Internet when RO supported spaces in filenames and found the fact that some apps wouldn’t let me type spaces in the save dialogue box to be confusing when the filer would let you introduce spaces into a filename. I remember thinking wow this feels like a modern OS and quite happily held down alt and typed 32 into the save dialogue box.

With regards to writing a script to replace spaces with underscores I was thinking along those lines and will write something which also checks for long filenames and paths too. I’ve seen these spaces even in an DVD for RO which showed up as differences in my BASH scripts when I was creating an ISO and checking from Linux to RO. I always wondered why I could never archive that DVD properly. I’ve noticed this kind of thing on SunFish, LanManFS, so this information helps me solve a lot of issues.

I will put some blutac under myspacekey. Seriouslythoughthatisaverysensiblesuggestion.

Thanks,forallyourhelp.HopefullythedelimiteronLinuxcan’tbechanged,withinascript,fromaspacetoacomma.

Well I didn’t expect that it would have been the Window Manager writeable icon processing that was substituting a hard space for space when spaces were explicitly excluded with A~ validation. When was that bodge introduced? [Edit: Answer: Window Manager 4.87 (24 Jan 2003)]

And of course if *Alphabet UTF8 is in play, you get a UTF-8 encoded hard space. That plays well with underlying filesystems’ filename translation…not.

If you subsequently switch to *Alphabet UTF8, all the spaces translated to hard spaces prior to that, either by the Window Manager or filesystems, disappear. Is there not yet a bounty to fix the Font Manager to output something sensible when invalid UTF-8 sequences are encountered? Like a UTF8-Latin1 ‘encoding’?

that was substituting a hard space for space when spaces were explicitly excluded with A~ validation.

Hard spaces are required in order to create a valid filename. That it works when “A~ ” validation is in use is probably a bug/oversight.

When was that bodge introduced?

The whole damn thing is a bodge.

In the Filer, copy a file as “what the fsck”.

Then go to the command line.

*.
Dir. SDFS::RISCOSPi.$.Public.Scratchpad Option 02 (Run) 
CSD  SDFS::RISCOSPi.$.Public.Scratchpad
Lib. SDFS:"Unset"
URD  SDFS:"Unset"
file          WR/WR   what the fsck WR/WR   what the fsck WR/WR 
*Type what the fsck
Parameters not recognised
*Type "what the fsck"
File name 'what the fsck' not recognised
*Type "what the fsck"
This is a file!
*

The final *Type operation only worked because I held down Alt while pressing Space to make hard spaces. The user should not need to know to do this.

Oh, and the one on the right? That’s what happens (as Stuart notes) when the system is doing the same blind-idiot translation in UTF-8 mode. You get… a UTF-8 hard space, and no damned idea how to access the file (it’s “what<alt-194><alt-160>the….” the user really shouldn’t need to know that!).

Is there not yet a bounty to fix the Font Manager to output something sensible when invalid UTF-8 sequences are encountered?

That would make such a difference. It would no longer be a case of either UTF-8 or Latin1, it would become a case of “you can put the system in UTF-8 mode and it won’t break everything that dares to use a character not available on the BBC Micro”.

The change to the Wimp to force translation of spaces to hard spaces when A~ validation is used to reject space is a deliberate change in the RISC OS 5 tree. It doesn’t do it if hard space is explicitly rejected as well, or when space is not being accepted because it’s not in the accept validation, e.g. number fields. Checked RO 4.39, that doesn’t do the space → hard space bodge.

Wonder if this behaviour is in any of the user-facing documentation? As Rick says, user just thinks they’ve got spaces.

Checked RO 4.39, that doesn’t do the space → hard space bodge.

Are you sure? The “map spaces to hard spaces” thing was in RISC OS 4.mumble, as far as I know: it was controversial enough at the time, IIRC. I’ve just saved a file called “Test Text” from Zap in 4.37, and that arrived in HostFS with a hard space (character 160) in it.

You can rename files / create directories on my RO 4.39 (on RPCEmu) with space→hard space substitution.

Tried Edit and PipeDream Save as dialogue boxes, they still reject space on this 4.39.

Edit: And WinEd r101 won’t let me insert spaces in Save as either on this 4.39.

Edit: SYS"OS_File",0,“hostfs:$.Temp.foo bar”,&8000,&8080 saves “foo” without any space.