<, > and illegal characters in filenames

In RISC OS it is possible to have and create filenames which contain < and >.

Many programs read filenames using OS_GBPB and pass them back to FileSwitch SWIs without GSTrans escaping them. This will cause an action on a file the programmer didn’t intend.

For example if you try to delete a directory named “<94>”, which OS_GSTrans will decode to “^”, using the filer part of the parent directory will be deleted instead. To create such a directory using RISC OS ask to create a directory called “<60>94>”.

Such maliciously named objects can, for example, be distributed in zip files read using SparkFS or stored on a file server. Any relative pathname can be used, and it effects copying as well as deleting so can be used to install malware into !Boot. A malicious remote filing system could use this to read some system variables.

Some RISC OS filing systems will read other illegal filenames (eg “^” in Sunfish and “T.T” in DOSFS) so GSTrans encoding is not always necessary.

It appears the simplest fix is to add “<” to the list of illegal characters and for FileSwitch to hide objects with illegal names by changing the wildcard match routine and OS_GBPB 8. Unfortunately this could cause friendly files to disappear, perhaps Filecore be should changed to map “<” to some other character on both read and write, but this could cause duplicate Filenames.

RPCEmu HostFS will also read filenames containing newlines which makes directory listings into nonsense, but I think it is otherwise harmless.

“<60>94>”

You bad, bad man.

It appears the simplest fix

Strictly, as FileSwitch insists on GSTransing the filenames it is given, it ought to GSTrans encode filenames it hands out IFF the result of GSTrans on the leaf results in a different leaf (canonicalise too, obvs). This at least could be imposed by an API-fixing module, a new FileSwitch is not necessary.

Also bear in mind that FileCore-based FSes do not have quite the same filename rules as mere FileSwitch FSes, so copying files from FS to FS is not possible in general anyway.

In fact, it is possible to have FSes which are not even FileSwitch based, and they can do whatever they like with filenames.

Another HostFS observation – some versions map $ and ^ in Win to < and > in RO.

You bad, bad man.

Look on the bright side. It’s taken thirty years to get here… ;-)

as FileSwitch insists on GSTransing the filenames it is given

Why does it GSTrans? A filename shouldn’t really have a ‘<’ character for anything more than introducing a system variable (Something$Dir). Is GSTrans how that is actually implemented?

It’s taken thirty years

Breaking news: RISC OS found to be insecure. Who knew?!

Why does it GSTrans

Because of the Beeb (did the Atom have it? I don’t know). Because it was impossible to type some characters from the keyboard. You could then type any character from |@ to |!|? with only a certain degree of mental arithmetic.

The <…> syntax was added in Arthur I think (or RO2) and sysvars are undeniably marvellous. However, the FS API has always been asymmetrical.

The calls that are responsible are:

• OS_Args 7
• OS_FSControl 37, 52 & 54
• OS_GBPB 6-12

It ought to be straightforward to knock up a module that fixes the symmetry of these calls. I need to do a bit more investigation to make sure there aren’t any non-GSTrans APIs. If there are, all bets are off.

The non-GSTrans version of the attack will work against *Copy etc with a destination filing system which accept ^ to refer to the parent directory. This can not be solved by wrapping the Fileswitch API.

For the solution of GSTrans escaping to work the right number of GSTrans passes need to be used, and one would need to consider the possibility extra passes such as filing systems that call the FileSwitch API, use of the do command, and handling of system variables containing paths. It will also prevent user visible use of a bare < in a pathname.

The non-GSTrans version of the attack will work against *Copy etc with a destination filing system which accept ^ to refer to the parent directory.

I’ve clearly not had enough coffee to parse that sentence. ^ is a perfectly sensible element in a pathname to Copy. What do you mean?

use of the DO command

GSTrans can be a pain when setting sysvars (and macros) and I have to confess to occasional arbitrary insertion of ‘|’ chrs in a flailing panic while trying to formulate the right syntax for an Alias$@RunType.

The non-GSTrans version of the attack

Given we have a system call that happily grants total access to any code that requests it (not to mention modules have this by default), the use of words such as “attack” are a bit disengenious. Want to trash the root block to make the filesystem vanish? There’s a SWI for that. Far simpler ways to cause trouble than words filenames. So it’s probably better to consider this a bit of overzealous API design than a big security flaw.

Because of the Beeb

Given the many restrictions on filenames in the Beeb era, why is this being applied to filenames at all?

It ought to be straightforward to knock up a module that fixes

Hmm, a module to fix a module. Given that I can’t imagine anything seriously uses GSTrans in filenames, would it not be an idea to “just fix it” in one of the beta versions, and see what (if anything) breaks?
You know, the ZPP method…

Most RISC OS software relies on GSTrans on filenames to refer to system variables, disabling it is not an option.

Of course we need to fix the filer’s habit of executing everything in sight. Making SWI calls requires running a program, and how else is Mallory able to execute a program on Alice’s computer?

disabling it is not an option

Stupid question: and to fix it?

bq. The non-GSTrans version of the attack will work against *Copy etc with a destination filing system which accept ^ to refer to the parent directory.

I’ve clearly not had enough coffee to parse that sentence. ^ is a perfectly sensible element in a pathname to Copy. What do you mean?

^ is valid in the command line, but not in the filesystem.

Suppose we have a floppy disc containing a directory “Pictures” containing a file called literally “^.!Boot.Choices.Tasks.Malware”

1. The user wants to copy Pictures to her hard disc so she executes: *Copy :0.$.Pictures :4.$.Pictures R
2. FileSwitch asks FileCore to list :0.$.Pictures
3. FileCore returns the name “^.!Boot.Choices.Tasks.Malware”
4. Fileswitch ask Filecore to load the path “:0.$.Pictures.^.!Boot.Choices.Tasks.Malware”
5. FileCore loads “:0.$.!Boot.Choices.Tasks.Malware”
6. Fileswitch ask Filecore to save to the path “:4.$.Pictures.^.!Boot.Choices.Tasks.Malware”
7. FileCore saves to “:4.$.!Boot.Choices.Tasks.Malware”
8. Which is executed next time the computer boots.

If FileCore didn’t treat ^ as a reference to the parent directory it would write harmlessly to “:4.$.Pictures.^.!Boot.Choices.Tasks.Malware”. If copying with FilerAction FileSwitch would convert “:4.$.Pictures.^.!Boot.Choices.Tasks.Malware” to “:4.$.!Boot.Choices.Tasks.Malware” and the attack would work with any destination filing system.

^ is valid in the command line, but not in the filesystem.

Where, I wonder, does the hat get converted?

I suspect (but can’t be bothered to get out of bed to check the sources) the command line passes the filename as given and it is FileSwitch that canonicalises the path (handling the ^). Everything that you have described is by design. The problem is not having '^' be the parent (akin to “..” in DOS/Linux), the problem is that it seems to be that it appears from your description as if ‘^’ can be a valid file name, which is just utterly illogical.

BTW, please drop the “malware” references. You’re looking at filename oddities, I could utterly trash a machine with about five lines of BASIC. So let’s be realistic here… ;-)
Oh, and while we’re being realistic, your user would probably drag and drop her files. Much simpler.

Suppose we have a floppy disc containing a directory “Pictures” containing a file called literally “^.!Boot.Choices.Tasks.Malware”

And how would such a file be created? “^” and “.” aren’t valid characters for filenames.

Your original example of <94> works (as in, it results in much hilarity).

Strictly, as FileSwitch insists on GSTransing the filenames it is given, it ought to GSTrans encode filenames it hands out IFF the result of GSTrans on the leaf results in a different leaf (canonicalise too, obvs).

Yes, having FileSwitch escape filenames that are returned to users seems like a reasonable fix.

It’ll be a bit tricky to fix for the OS_GBPB directory enumeration calls – if the low-level FS fills the user’s buffer then there won’t be any space left for FileSwitch to insert the escape characters. So in that case it would probably have to work out how many escaped filenames it can actually hold, and then re-call the low level FS (with the original continuation value) requesting that exact number of entries, in order to get a continuation value that it can return to the caller.

I need to do a bit more investigation to make sure there aren’t any non-GSTrans APIs. If there are, all bets are off.

The PRMs pointed me towards OS_FSControl 53. A search of some ROM sources suggests that it isn’t used very often, so I don’t think there’ll be much breakage if FileSwitch’s behaviour is changed.

Demostration: http://timbaldwin.fastmail.co.uk/fs_copy_demo.zip

And how would such a file be created? “^” and “.” aren’t valid characters for filenames.

By altering the filesystem with an editor for example, then give the SD card to the victim. I modified a StrongHelp filesystem in Zap and included it in the above zip file. And you can create “^” in Linux then access the directory using Sunfish.

I could utterly trash a machine with about five lines of BASIC. So let’s be realistic here… ;-)

As far as I know a BASIC program is powerless on RISC OS unless it is executed, if you know otherwise please explain. My hypothetical user doesn’t trust you and therefore won’t run your program. Of course we would need to fix the other security flaws:

• The filer auto-boots applications (also a nuisance).
• Double-clicking a file in the filer runs it according to it’s current type, not the type shown.
• Many programs load files using OS_File 255 etc, but file may have grown since it’s size was checked, or it could be a different file with the same name.
• A malicious party creates or deletes files causing filer icons to move causing the use to click a file they did not intend.

And how would such a file be created? “^” and “.” aren’t valid characters for filenames.

By altering the filesystem with an editor for example, then give the SD card to the victim. I modified a StrongHelp filesystem in Zap and included it in the above zip file. And you can create “^” in Linux then access the directory using Sunfish.

In this case you could argue that it’s the lower-level filesystem’s fault for not checking their filenames correctly – but for practical reasons it probably would make sense for FileSwitch to be the one doing the checks (especially since FileSwitch is the one that’s responsible for the rules in the first place)

I do at times wonder why file names still have constraints on the characters allowed in the naming of files. Perhaps a reason to expand the number of characters used in a filename is that RISC OS handles files differently to other operating system; in terms of their naming conventions.

Should we not consider expanding the use of the character set used in filenames versus placing limitations on what can and can’t be used?

And how would such a file be created? “^” and “.” aren’t valid characters for filenames.

In ResourceFS anything is possible, really anything ;-)

Of course, only certain names will be recognised, but you can certaining include ‘^’ and ‘^.!Boot.Choices.Tasks.Malware’ appears as a directory.

In this respect, ResourceFS is for amusment only.

Rick asked:

Given the many restrictions on filenames in the Beeb era, why is this being applied to filenames at all?

Because: compatibility; it’s useful; sysvars

Timothy said:

^ is valid in the command line, but not in the filesystem.

Strictly, ^ is valid as an API input. It may also be valid within a particular media format. It is NOT valid as an API output.

Suppose we have a floppy disc containing a directory “Pictures” containing a file called literally “^.!Boot.Choices.Tasks.Malware”

Then whatever is sitting between FileSwitch and the disc image has made a serious and unforgivable error. Such a construction MUST be transformed.

Rick mused, perhaps rhetorically:

Where, I wonder, does the hat get converted?

By FileSwitch in one of a surprising number of code paths responsible for filename processing.

Jeffrey worried:

It’ll be a bit tricky to fix for the OS_GBPB directory enumeration calls

No, the GBPBV claimant tail-calls multiple times asking for one entry at a time. IF an entry is returned that must be expanded and there is no room, the /previous/ R3/R4 are returned.

OS_FSControl 53

Fortunately that aberration does not return a filename, so its deep wrongness can be ignored.

Timothy exclaimed, tinnily:

Double-clicking a file in the filer runs it according to its current type, not the type shown.

Well, OS_File was supposed to have a Run As Type reason code, which currently falls through to plain old Run (and Run As can be retrofitted using only 6 instructions, I happen to know!), but the prospect of the Filer ignoring changes in type because it hasn’t noticed the update is probably worse.

Many programs load files using OS_File 255 etc, but file may have grown since its size was checked, or it could be a different file with the same name.

And further down that road lies never leaving the house because a window might be open in another room. GBPB is available for those aware of buffer overruns.

A malicious party creates or deletes files causing filer icons to move causing the use to click a file they did not intend.

TBH the fact that all RO applications honour icon clicks a fraction of a millisecond after the window has just opened is a much greater problem.

Steve enjoyed:

In ResourceFS anything is possible, really anything ;-)

Yup, and as I said, non-FileSwitch FSes (just like the old days) really can do anything. So, yes, RO could have DOS-format filenames if we wanted, or URLs. And if RO had a filename manipulation API built in from the start it could profitably have switched to a more popular format. However, since apps have to fiddle with filenames and paths, we’re stuck with at least a semblance of 1982 syntax forever… though I do wonder how many roll-your-own functions handle -this- construction these days.

Timothy:

As far as I know a BASIC program is powerless on RISC OS unless it is executed,

True, but it’s easy to get something executed. Just make it so that the user wants to run the program. Perhaps a little slideshow application (keeping with your images theme)? I would imagine, for the majority of users, it would be easier to trick them into running a tainted application than it would be to wait until they try to copy from here to there using the command line…

The filer auto-boots applications (also a nuisance).

The filer auto-boots applications (very bloody useful, so when I double-click a file, it doesn’t say “An application that loads a file of this type has not been found by the Filer. Open a directory display containing the required application and try again.”, leaving you to go find the application in order to run a file.
Yes, it can be abused. So can most of RISC OS.

Double-clicking a file in the filer runs it according to it’s current type, not the type shown.

What do you mean by this? The current type is the type shown…

…unless you mean if I take a text file and call it “index/html”, it will open in Zap and not the browser? Is that it?

Many programs load files using OS_File 255 etc, but file may have grown since it’s size was checked, or it could be a different file with the same name.

That’s possible, but “reasonably” unlikely given the usual method is to look at the length of a file, allocate that much memory, then load it, all as subsequent operations.

However, the method I prefer (instead of blind loading) is to look at the size of the file, allocate that much, open the file, OS_GBPB that much data, then close the file. It’s more work, sure (all two SWI calls), but it gets around the glaring omission that OS_File 255 does not include any way of specifying a buffer size when loading at a specific address.

A malicious party creates or deletes files causing filer icons to move causing the use to click a file they did not intend.

<yawn> A malicious party trashes your $ directory for the evulz. A malicious party scans for and deletes every file typed &FF8. A malicious party reads your SMTP server details from !POPStar or whatever and quietly posts one spam every fifteen seconds from your account.
A malicious party … is being held tonight at half eight, BYOB.

A really evil malicious party opens up every single DrawFile on your machine and rotates the third object by 117 degress. Now that is nasty. :-P

That said, if something is able to modify files on your machine in order to move their positions in anticipation of a click in order to trick the user into clicking something unintended…. sorry….. I call bull. If there’s something on your machine capable of doing all of that, you’ve already been pwned.

Now to nemo for something a little less DailyMail:

I like the quote descriptions by the way. Yes, I mused rhetorically. :-)

it’s useful; sysvars

So if GSTrans is responsible for converting <Meh$Dir> and also could theoretically create a file called “<22><12>”, then this is surely one to file under the law of unintended consequences?

Such a construction MUST be transformed.

Indeed. Reserved characters are reserved characters. End of.

The only place one could “forgive” a failure to transform is when said character is not possible in a legal sense (such as Timothy’s idea of hacking the disc). I wonder what would happen if I hacked a directory to contain a file called “$”? Sorry, though, you’re a decade too late for me to try. I’d give it a whirl on an old floppy disc, but I’m not going to mess with the filesystem of a USB key in order to test a small theory…

The questions are therefore: is it possible to (via the legal API) give a file a name like that (something using a reserved character) and if not, should the filesystem check for such a thing?

OS_FSControl 53

Fortunately that aberration […] so its deep wrongness […]

Whoo. [fx: loads StrongHelp]

WTactualF? Seriously… WTactualF? Why do we even have a call to set one of the base directories (CSD, URD, etc) to some random who-the-hell-knows without making any attempt to verify it is a valid path? What special kind of Fail is this?

>SYS "OS_FSControl", 53, "#HOLD#A#CHICKEN:IN;THE.AIR/", 0,"SDFS"
>*.

Filing system SDFS: does not support special fields
>

The source doesn’t give any indications either. Anybody know why such a weird call even exists?

So, yes, RO could have DOS-format filenames if we wanted, or URLs.

Hmm… URLFS. That could be useful! No more writing HTTP code when something could just let me “open” a URL “file” and read from it. :-)

it could profitably have switched to a more popular format.

Such as? Oh, and I should point out that I find the POSIX “everything hangs off /” to be an utter abomination.

I quite like how RISC OS deals with its filesystems. Everything is distinct by device (a little like A:, C:, D: in Windows, only the prefix is a tad more meaningful than a single letter), with the applied magic that is system variables to mean that you can have all the flexibility of treating everything as a generic “files here, but you don’t need to worry where” with each filesystem itself being logically self contained for when you need to go into the Filer.

though I do wonder how many roll-your-own functions handle -this- construction these days.

How many guesses to I get? ;-)

And a quick one for Andrew:

Should we not consider expanding the use of the character set used in filenames versus placing limitations on what can and can’t be used?

What should happen if I create a file called “<Obey$Dir>”? The idea of blocking stuff in filenames is so that certain characters ($, ., &, #, etc) have a specific and clearly defined meaning with no ambiguities.

Timothy…

Your server has a simple little script to dump the contents of a file. Essentially it is called “viewer.php” and it is:

echo shell_exec('cat '.$GET['file']);

and you call it with a URL such as: …viewer.php?file=COPYING.TXT

So far so good.

But, wait, what if we were to try this: …viewer.php?file=COPYING.TXT;rm -r /

:-)

Everything can have issues…

What should happen if I create a file called “<Obey$Dir>”? The idea of blocking stuff in filenames is so that certain characters ($, ., &, #, etc) have a specific and clearly defined meaning with no ambiguities.

I agree. But I would expect that if you created a file with that name, it should do nothing, unless it had an Obey file type. That’s the way I would expect it to be implemented… The file type being the mechanism by which a program knows how to interpret a particular file.

… a user perspective ;-)

Rick said:

So if GSTrans is responsible for converting <Meh$Dir> and also could theoretically create a file called “<22><12>”, then this is surely one to file under the law of unintended consequences?

And that is what Timothy has pointed out, and what my mostly finished module fixes. That file would be returned as |<22>|<12> which is perfectly fine thank you.

what would happen if I hacked a directory to contain a file called “$”?

In an ideal world, the FS responsible for the media format would transform that filename into a legal one – exactly as HostFS very nearly does. Whatever escape format it used need only make sense to it though, it would not be GSTrans format.

For example, it could use the broken bar ¦ – so an actual 0xA6 is indicated by ¦¦, and then all kinds of other awkward characters can be encoded:

¦@-¦[   Ctrl codes 0-27
¦-      Ctrl code 28
¦]      Ctrl code 29
¦~      Ctrl code 30
¦_      Ctrl code 31
<160>   Space
¦'      "
¦=      #
¦§      $
¦Ø      %
¦6      ^
¦+      &
¦¤      *
/       .
¦/      /
¦;      :
¦Ð      Del
¦<160>  <160>

Unfortunately translating filing systems such as LanManFS, MacFS, HostFS etc do not usually bother with a 1:1 mapping, working on the principle that many odd cases are very unlikely.

It’s easy to forget that not long ago, filename syntax and hierarchical behaviour was entirely at the whim of the FS, eg DFS “directories”.

But I would expect that if you created a file with that name, it should do nothing, unless it had an Obey file type.

I was referring to the idea that such names are interpreted by the system as being special values, so if such a file actually existed (or <Edit$Dir> for instance), how would one even refer to it? Perhaps with a | prefix? But now it’s starting to get messy…

That file would be returned as |<22>|<12> which is perfectly fine thank you.

Except, it’s not.

This happens every time I mess with GSTrans – just when you think it’s safe to get back in the water.

*Set a <60>65>
*Set b |<65>
*If "<a>"="<b>" Then Echo Same!
Same!

*If "<60>65>"="|<65>" Then Echo Same! Else Echo Idiot!
Idiot!

*Rename a <60>97>
*Rename b |<98>
File name '|b' not recognised

I wished I’d checked that before writing the code. OK, time to replace all those “|<” with “<60>”.

This is because FileSwitch, though it uses GSTrans, does not use all its translation options. Not that any of that is documented in the API of course (what’s new?).

That said, if something is able to modify files on your machine in order to move their positions in anticipation of a click in order to trick the user into clicking something unintended…. sorry….. I call bull. If there’s something on your machine capable of doing all of that, you’ve already been pwned.

Again you seem to be treating this as an all or nothing matter. Ability to modify some files does NOT imply the ability modify any file, there are many file server programs for RISC OS and a hypothetical client the propagates changed file notifications could also be vector for this attack. It does however require good timing.

What do you mean by this? The current type is the type shown…

Not if it changed outside of the RISC OS instance in question since the directory was listed. This would require a network or hypervisor FS. Locally having two files the same name has similar effect.