Interesting code execution inconsistent behavior

While testing various aspects of executing code on RISC OS, I observed some inconsistencies in the code execution process it self, here are few examples:

When someone tries to run code that will call Wimp_Initialise from a TaskWindow:

*!RunImage 
(for a multitasking BASIC App fails with WindowManager 
 in use error when it tries to initialize the task)

*File_Run !RunImage 
(everything works fine and the multitask app is 
 executed in the desktop correctly)

*Run !RunImage 
(same issue as first case, execution fails with 
 error WindowManager in use error)

Also, it appears there are other differences when executing something using Run, I need to look at the RO Source code, but in certain tests showed some inconcistency when executing:

*Run !Foo 
(where Foo is an App directory that will execute
 a multitasking app using !Run)

vs

*Filer_Run !Foo 
(Foo is same as above, but in this case everything 
 always works fine)

This is not necessarly a bug (althought if in modern OS world it would definitely be considered a bug), but I think it was worth mentioning it somewhere.

Maybe it would be useful to add the info on the Wiki?

Not a bug nor inconsistent.

Filer_Run will cause the run action to be performed on the given file by the Filer, whereas the other two will try to run the application directly within the TaskWindow environment, and as you noticed, Wimp tasks don’t work like that.

Think of Filer_Run as a command that means “pretend I double-clicked on this file”.

Sure Rick,
but, from a user point of view the behavior is inconsistent. The issue is most likely caused by TaskWindow itself.

If you try to write a piece of code that checks if the desktop is active and so tries to execute “Wimp_Initialise” that will not end up well. But one can detect this and avoid quitting and run a multi-tasking code within the TaskWIndow and for example open Windows on the desktop.

Now, if you do that, the pseudo Multi-task apps gets executed in a 3rd type of environment, so basically we have 3:

1) Pure CLI
2) Regular Desktop
3) TaskWindow “Limbo”

When you are in a TAskWIndow Limbo, which it’s very easy to enter, just write a desktop app that does not call Wimp_Initialise your code will be executed within the TaskWindow pageing, which is well… :D

However, you can open windows and draw icons and, with some hacks, even control them.

The other not available call is Wimp_Poll (this one makes sense given that the TASKwindow is a PMT hack on top of a CMT WIMP). Wimp_Poll will fail because it’s being executed in a PMT environment that however doesn’t support a `nice` like call, so kinda half backed.

In other words, it’s definitely inconsistent, I am sorry to disagree on that, in some code tests I was able to do quite a bit of interesting things.

However everything can be solved in multiple ways, one is the traditional “developer is responsible fo reverything”, so an App could Detect it’s being executed in a TAskWindow context and so, instead of initialising normally, it should just execute itself again via “Filer_Run” (or start itself as a task) and then the parent task just end.

Tested that and works really well.

from a user point of view the behavior is inconsistent.

Only if you don’t understand what the commands are actually doing.

Filer_Run is, as the clue in the name suggests, asking the Filer to run something. As such, it cannot be used outside of the desktop (see note ¹ below).
One very important thing to note is that the currently defined “Run action” will be applied to the file. That is to say, JPEGs and text files (etc) may have different behaviour on your system to mine depending on what apps are running and what has claimed those file types.

Run, Exec, BASIC -quit, etc etc are the underlying commands to actually do stuff, for certain types of file. ²

The two methods are related, but they are not the same thing.

If you try to write a piece of code that checks if the desktop is active and so tries to execute “Wimp_Initialise” that will not end up well.

Applications, that is to say anything liable to call Wimp_Initialise, are supposed to be self-contained and not be running in TaskWindows.
In the unlikely event that there’s a legitimate reason to run a potentially multitasking app in a TaskWindow as well, there’s a handy call that’s basically “Am I in a TaskWindow?”.

However, you can open windows and draw icons and, with some hacks, even control them.

Basically, the TaskWindow is a Wimp app. It’s a Wimp app with many tentacles to do what it does, but it’s a Wimp app. You’re running inside a Wimp app.

Which is why Initialise fails, Poll blows up, and you can (hairily!) create windows and icons and such.
This is perhaps TaskWindow’s fault for not blocking the host from being able to call Wimp functions, but that would mean vetting every SWI call and that would take time.

That you can sort-of maybe do some Wimp stuff in it is “undefined behaviour”.
Literally, the first line in the PRM describing TaskWindow is:

The TaskWindow module is intended to allow programs which do not call SWI Wimp_Poll to be pre-emptively scheduled in the RISC OS desktop.

:-)

As for this part…

However, you can open windows and draw icons and, with some hacks, even control them.

…that noise that you hear is the sound of a Gerph screaming.

In other words, it’s definitely inconsistent, I am sorry to disagree on that,

We’ll have to disagree then, because as far as I’m concerned, wondering why you can’t run a multitasking program in a TaskWindow is about the same as wondering why you can’t hook into callbacks in BASIC. ³

The TaskWindow Is Not Intended For Multitasking Apps.

in some code tests I was able to do quite a bit of interesting things.

Oh, I’ll bet. RISC OS doesn’t have a habit of saying “nope”. It would rather try to be a good little Brownie, have a go, and then sit there bawling its eyes out when it all comes crashing down.

so an App could Detect it’s being executed in a TAskWindow context and so, instead of initialising normally, it should just execute itself again via “Filer_Run”

Or be like everybody else and just crash. Because (say it with me) The TaskWindow Is Not Intended For Multitasking Apps.

That people don’t check for this is not an omission on anybody’s part. It’s simply because (say it with me again) The TaskWindow Is Not Intended For Multitasking Apps.
Anybody who tries will have to expect things to screw up. Just like if you try fiddling with OS_UpdateMEMC or poking random values into the memory tables or closing all open files and opening different files with the same handles….. RISC OS won’t prevent or protect you from any of that (well, maybe UpdateMEMC does nothing these days? not looked) as it’s expected that if you’re fiddling around like that, you’re happy to deal with the consequences.
Trying to run an app in a TaskWindow is the same thing. It’ll bork. It has been that way since RISC OS 3.

¹ You can certainly call Filer_Run from the command line, I find it is often useful to press F12 and Filer_Run something that the system knows (such as <ChangeFSI$Dir>.!Run) rather than bother to find it in the filing system.
However, the effects don’t happen until you return to the desktop world and Filer processes the command(s).

² It isn’t possible to completely escape file actions. If you tried to *Run a JPEG, you may well end up with a C backtrace. This is because a JPEG is not a known type that the system can handle, so it looks to see how to handle it. My backtrace is basically the system (in this case, I think it’s FileSwitch) starting up PrivateEye to handle the JPEG, and that subsequently crashing out because one can’t do that in the command line environment.

³ Unless you’re Steve, that is.

For anyone who’s actually testing their software thoroughly, this is how to avoid issues with this particular use case (using a BASIC example for brevity):

SYS "TaskWindow_TaskInfo" TO TWCheck%
IF TWCheck%=0 THEN
  SYS "Wimp_Initialise",300,&4B534154,"Your Task name"
ELSE
  SYS"OS_GetEnv" TO xarg$
  IF xarg$<>"" THEN
    IF LEFT$(xarg$,6)="BASIC " THEN xarg$=MID$(xarg$,7)
    IF LEFT$(xarg$,6)="-quit " OR LEFT$(xarg$,6)="-load " THEN xarg$=MID$(xarg$,7)
    IF LEFT$(xarg$,1)=CHR$34 THEN xarg$=MID$(xarg$,2,LENxarg$-3)
    OSCLI("Filer_Run "+xarg$)
  ENDIF
  END
ENDIF

REM rest of your code...

This is also useful if one wants to make code that can run both in single AND multitasking.Obviously in such case, don’t try to run itself again via filer_Run.

I wouldn’t use TaskWindow_TaskInfo to check whether you’re within the desktop already. TaskWindow shouldn’t be considered special. Call Wimp_Initialise and if you get an error take appropriate action.

Problems with the reinvocation you propose:

• You may generate more errors: Filer may not even be present.

• Long command lines (over 256 characters) will cause breakage: The suggested way of invoking the application from BASIC will fail in the case where the CLI is longer than about 245 characters, because the parameter will be too long for BASIC strings, so you’ll have truncated the text.

• Long command lines (using DDEUtils) may not work: The command line that was set by DDEUtils to be retrieved as the long command line might not be in force by the time that your Filer_Run is executed.

• The command line arguments will not work: The Filer_Run command will pass through the filename of the BASIC file as supplied to Filer_Run, together with any parameters. This will fail because Filer_Run should not be used for this. You should use TaskManager_StartTask in such cases – not least because the Filer might not be running on the system. TaskManager_StartTask is only present on RISC OS 4 and later. You could use Wimp_StartTask (or *WimpTask) but that’s not guaranteed to work within a TaskWindow prior to Select 2.

• The file executed may not be the one you expected: When a prefix is in force in the TaskWindow the execution of the command outside of the TaskWindow will not take account the name of the file being run by the invoked command outside of the TaskWindow. Similarly, if a temporary filesystem is in force at the time of the execution, it will not be present by the time that the Filer_Run (or whatever) is executed, so would also execute a different command. You might try to canonicalise the filename before you re-invoke yourself, but you probably also want to do the same with any parameters that are passed as well. And, of course, that may blow your command line length limits, so you might need to switch to the long command line extensions.

• Filer_Run may never actually run the file: Filer_Run will honour any shift key held by the user at the time, to open a directory instead of running an application, or load into a text editor. As such you could use the Filer_Run -noshift (or whatever the option was) that was introduced in RISC OS 4, but obviously it’s not present elsewhere. Or you could wait until shift is released or notify the user. Or you could use the TaskManager_StartTask instead.

• Your wimpslot will not be as expected: By issuing Filer_Run, you’re requesting that the program that’s run be started with the Next slot, not the currently in place slot size, which would have been carefully set up by the !Run file. As such, you may git more or less memory than you are expecting – and might not even start because of that. With some sizes of application and implementations, this may result in the machine hanging fatally. Select 3 (I think) onwards, will attempt to size your slot based on the size of the program being run, but this is a last ditch effort to try to do what you asked and not to cause the machine to hang fatally ¹.

• Depending on what the desktop is doing, your environment variables may not be set as expected: If you were to have been invoked with <Obey$Dir>.!RunImage, the Obey$Dir variable may no longer be set to what you expect by the time that Filer comes to perform the run.

All in all, you’re better off just telling the user that multitasking applications cannot be run within a taskwindow, because dealing with all that just isn’t reasonable (or, in some cases, possible). Reporting a useful error message telling the user what to do, rather than crashing with a side effect, is a much better course of action than failing to handle these cases.

¹ Actually it’s not – it’s a means by which you can ensure that SEAs work properly, but it has the effect of making execution of many programs at least a bit safer.

I wouldn’t use TaskWindow_TaskInfo to check whether you’re within the desktop already. TaskWindow shouldn’t be considered special. Call Wimp_Initialise and if you get an error take appropriate action.

Sure, but that will require a bit more work, btw the BASIC example was just to express a concept (aka used as psudo code), not an actual complete implementation (I have mentioned: to be short, maybe only Oxford’s English is allowed here?).

You may generate more errors: Filer may not even be present.

This is true, but one can replace Filer_Run with Wimp_StartTask.

Long command lines (over 256 characters) will cause breakage:

The problem you are referring to applies only to BBC BASIC (again I wasn’t specifically suggesting a definitive implementation that would have taken more space). In C it would not break anything, in Assembly the same, in Prolog and Forth same, c++ same.

For the specifics of BBC BASIC, one can use a block to store a longer string, so if the need is to have a complete implementation instead of usign BASIC as a pseudo code then that could be a way to do it.

The command line arguments will not work: The Filer_Run command will pass through the filename of the BASIC file as supplied to Filer_Run, together with any parameters.

You can retrieve an app parameters-list if that’s what you refering to and pass them along.

The file executed may not be the one you expected:

Not the case when you are calling an executable directly, unless you’re using a macro or an alias, but that (unless the original developer set it) is a responsability of who creates the alias/macro etc.

Filer_Run may never actually run the file:

true, replace it with Wimp_StartTask ;)

Your wimpslot will not be as expected:

Play a bit more with string manipoulation and, instead of calling yourself back, call your !Run.

Depending on what the desktop is doing, your environment variables may not be set as expected:

While this is true as a general statement, in most cases you’ll get the path resolved and NOT the <Obey$Dir>. form, while, if for whatever reasons, <Obey$Dir> is not defined and has been used, then obviously you won’t reach the executable either, so this last point seems irrelevant to me.

All in all, you’re better off just telling the user that multitasking applications cannot be run within a taskwindow

That is another good approach and you can decide what to do, or even leave it as the traditional form and let the Wimp give that error message “WindowManager in use” etc. which may be crytic for some user, but oh well.

You may generate more errors: Filer may not even be present.

This is true, but one can replace Filer_Run with Wimp_StartTask.

Exept you cannot prior to Select 3 because it’s unsafe to use in a TaskWindow.

The command line arguments will not work: The Filer_Run command will pass through the filename of the BASIC file as supplied to Filer_Run, together with any parameters.

You can retrieve an app parameters-list if that’s what you refering to and pass them along.

Pass them along how? As I stated Filer_Run cannot be given parameters to pass to the application.

The file executed may not be the one you expected:

Not the case when you are calling an executable directly, unless you’re using a macro or an alias, but that (unless the original developer set it) is a responsability of who creates the alias/macro etc.

You’re talking about a command on the command line being run in a taskwindow. So if a user typed in /!MyApp you’ll get the filename !MyApp passed through, and that would be expanded to Run (not canonicalised), and in your !Run file. Because of Prefixing, that might not be valid when you are outside the taskwindow. Similarly, if the user typed RAM:!MyApp then your application being run is on the temporary file system RAMFS, which won’t be in force when the Filer_Run came to happen.

Filer_Run may never actually run the file:

true, replace it with Wimp_StartTask ;)

Which, as I stated, cannot be used reliably within a TaskWindow.

Your wimpslot will not be as expected:

Play a bit more with string manipoulation and, instead of calling yourself back, call your !Run.

That might be more reliable, but it’s nothing like what you suggested.

Depending on what the desktop is doing, your environment variables may not be set as expected:

bw. While this is true as a general statement, in most cases you’ll get the path resolved and NOT the <Obey$Dir>. form, while, if for whatever reasons, <Obey$Dir> is not defined and has been used, then obviously you won’t reach the executable either, so this last point seems irrelevant to me.

You’re assuming that the path is resolved by something. It’s not. If the user typed /!MyApp arguments then FileSwitch will try to run !MyApp.!Run with the arguments supplied. If that file is an Obey file that contains Run <Obey$Dir>.!RunImage %*0 then your environment will contain <Obey$Dir>.!RunImage arguments. Nothing will have expanded the <Obey$Dir>.

(actaully, in Pyromaniac I do perform canonicalisation at execution time, but I consider that to be a bug of laziness rather than a feature, as it goes against what RISC OS Classic will do – despite the fact that I think it’s a reasonable thing)

All in all, you’re better off just telling the user that multitasking applications cannot be run within a taskwindow

That is another good approach and you can decide what to do, or even leave it as the traditional form and let the Wimp give that error message “WindowManager in use” etc. which may be crytic for some user, but oh well.

It’s always better to turn an error into something that tells the user what you couldn’t do, not what went wrong, in my opinion. But given the inability to solve all the above problems, I believe it’s better to report that it cannot be done in this case, rather than try to deal with the multitude of cases that cannot work, or are very difficult to handle. Expending such effort might help with the tiny number of people who do that, and would be the right thing to do if you could handle it right, but it’s so hard to do properly that if you’re handling it at all, an error is the best solution.

🍿

@ Charles

My friend, are you taking this as some sort of intellectual challenge? If so I’ll stop replying, I am not interested in such type of discussions.

The reason why I have this feeling, is because of comments like this one:

You’re assuming that the path is resolved by something. It’s not. If the user typed /!MyApp arguments then FileSwitch will try to run Run with the arguments supplied. If that file is an Obey file that contains Run <Obey$Dir>.!RunImage %*0 then your environment will contain <Obey$Dir>.!RunImage arguments. Nothing will have expanded the <Obey$Dir>.

Now, this is easy to solve, in your !Run use this:

If "<Obey$Dir>" <> "" Then Set MyApp$Dir <Obey$Dir>
If "<MyApp$Dir>" = "" Then Error You are trying to execute this App in an ambiguous way, blah blah

...

<MyApp$Dir>.!RunImage %*0

What happens in this case can be one of the followings:

Scenario 1
- The Filer has previously seen !MyApp and has MyApp$Dir correctly set
- We run /!MyApp from the TaskWindow, the !Run will NOT erase MyApp$Dir and everything will be ok

Scenario 2
- The Filer has not yet seen !MyApp and so MyApp$Dir is not set at all
- We run /!MyApp from a TaskWindow, <Obey$Dir> has not been set, so the 2nd check is true and the user gets whatever useful error message you want (I just typed an example, don’t go pedantic on it, imagine the text says whatever you like lol)

Scenario 3
- The user has multiple copies of !MyApp and click on one that has not previously seen by the Filer
- MyApp@Dir gets updated correctly and everything works fine

I hope you do not want me to list all possible scenarios, right?

In general, given the frail nature of RISC OS, one should have that check anyway in a !Run file because things may go wrong also because the system has been put in an unstable state.

It’s always better to turn an error into something that tells the user what you couldn’t do, not what went wrong

That was exactly my original point, the error returned is ambiguos. So one can either return a better error message or try to work around it.

Not sure why we are having this endless discussion except you’re trying to debug all possible issues on a forum?

Exept you cannot prior to Select 3 because it’s unsafe to use in a TaskWindow.

My understanding is that quite a lot of things are unsafe in a TaskWindow anyway, but again two options: return a meaningful non-ambiguous error message (for consistency) or try your luck and see if you can manage to make it work even in this case.

Anyway thanks for the details…

Paolo, I’m not sure if you know who Charles is. Let’s just say that his knowledge across all of RISC OS, from top to bottom, exceeds that of the rest of us here by some orders of magnitude.

@ Dave

Paolo, I’m not sure if you know who Charles is

Dave, I do very well, not sure why you’re mentioning this.

The point here is: We should test before making assumptions.

Chalers, allegedly, doesn’t use RISC OS 5 (or so he mentioned multiple times, not sure if it is still the case), however here is an example of tests outcome:

Charles said “you can’t pass parameters through Filer_Run”, this is true only for OLD releases of RISC OS, while it DOESN’T seems to be true for RO 5.

RO 5 seems to support the use of "" to wrap a command and its parameters for Filer_Open, while older releases of RO will return a “Bad String” error if you try that (how do I know? I test everything, literally everything).

So, the following works fine in RO 5.28:

Filer_Run "!MyApp test"

And my app can retrieve test.

While for older RO one can always use PipeFS to store parameters, but this would be a workaround.

Also, before posting my comments, I indeed tested my findings on everything included RO 3.10, and in all honesty I have a perfectly working Launchpad that can be executed from a TaskWindow on all RO releases.

There are obviously advantages on making apps executable from a TaskWindow, as an example one could create an editor that we can run from a local directory and it opens a file into a wimp based editor if the wimp is active, something like:

*MyEdit @.test

But again, just my 0.5c, nothing more.

“you can’t pass parameters through Filer_Run”,this is true only for OLD releases of RISC OS, while it DOESN’T seems to be true for RO 5

Documentation is here

in which we have:

Notes

This command can only be used within the desktop environment, or from within a Desktop boot file.

@ Steve

Here is the source:

https://gitlab.riscosopen.org/rool/Filer/-/blob/master/s/Commands#L320

Read it.

Filer_Run seems to be using ReadArgs macro, to read all passed parameters.

Given that on RO 5 ReadArgs seems to have been improved and can correctly determine that a string between double quotes is a single parameter, as a side effect of this, now Filer_Run seems to be able to accept a file path + (that file’s) parameters when they are specified between double quotes. If this is the case, then the documentation needs to be updated btw.

I need to double check this, but I am not home and this is all I can do using a phone.

[edit]
edited a bit my English, ‘cause when typing from a phone it’s even worst than usual! Sorry for that :(
[/edit]

@ Steve

Here is the source:

I don’t need to read it.
I’m not arguing, merely point out that the documentation makes a statement, if the document doesn’t reflect the actual behaviour then the document needs alteration.
Or, if the actual behaviour is not as desired, then the code needs alteration to match the documented behaviour.

I do recall discussion about alterations/improvements to ReadArgs – maybe the current behaviour is an unintended consequence, maybe it was intended, and the documentation update was missed.

{quote}

@ Charles

My friend, are you taking this as some sort of intellectual challenge? If so I’ll stop replying, I am not interested in such type of discussions.

{/quote}

I’m sorry that you feel that way. You were describing some situations that you encountered unexpected (to you) behaviour and mechanisms to avoid them. I was pointing out that these situations generate unexpected (to most people) behaviour and that there is much more care needed to handle such cases than was suggested.

When you are talking about unexpected behaviour in special circumstances, I don’t think it’s unreasonable to discuss the other unexpected behaviour that is caused by the solution.

{quote}

The reason why I have this feeling, is because of comments like this one:

Now, this is easy to solve, in your !Run use this:

If "<Obey$Dir>" <> "" Then Set MyApp$Dir <Obey$Dir>
If "<MyApp$Dir>" = "" Then Error You are trying to execute this App in an ambiguous way, blah blah

...

<MyApp$Dir>.!RunImage %*0

{/quote}

WTF?! But… but… but… that’s not the way I understood it.

I’m clearly wrong as your example demonstrates. FileSwitch must be expanding that when it runs it – which I didn’t think happened. I thought it retained the originally specified name, which was the reason I asserted that it was not getting expanded. In RISC OS Pyromaniac, I keep the expanded version and pass it on to the environment, but I thought that was my laziness and that I was doing it wrong. It looks like I’m right in how RISC OS Pyromaniac implements it due to lazinesss, and wrong in my understanding of the execution.

Sorry about that. It’s not me being awkward, but my having a misunderstanding about the manner that particular operation worked. Oops.

That was exactly my original point, the error returned is ambiguos. So one can either return a better error message or try to work around it.

Not sure why we are having this endless discussion except you’re trying to debug all possible issues on a forum?

Sorry, you stated that “this is how to avoid issues with this particular use case”… which generated its own set of issues (which, I’m sorry, I was wrong about one of them). If you’re giving a solution to a problem and that solution causes other problems – particularly if the problem you’re working around is an odd case – it’s worth being aware of those problems. Or to look at it another way, I was responding with “this is the set of issues you will encounter if you apply this solution”.

Charles said “you can’t pass parameters through Filer_Run”, this is true only for OLD releases of RISC OS, while it DOESN’T seems to be true for RO 5.

I don’t see how it can be true. In order to support Filer_Run the command must be passed first to DataOpen, and the DataOpen cannot be passed parameters. When the DataOpen bounces it could have remembered the additional parameters passed to the command through the yourref/myref and reintroduce them into the execution of the alias’d command, BUT then that would have produced a disparity in behaviour between when you executed a file with parameters and there is an application present to handle the DataOpen after the Filer_Run (no parameters passed) and the action when there is no application present, and the alias could be used. This could have been addressed in RISC OS 5, but I don’t see a mechanism by which you might do that which wouldn’t itself introduce the inconsistency for applications that do not support it (or potential crashes).

So, the following works fine in RO 5.28: Filer_Run "!MyApp test" And my app can retrieve test.

I believe that would be more of a bug, because of the inconsistent behaviour I describe above, than a feature. It would be unwise to rely on the inconsistent behaviour.

@ Charles

I’m sorry that you feel that way.

All good, by reading your last post I realised that I had wrong assumptions. I was 1000% sure that you knew that “trick”, so your previous comment got me confused and I had no idea why you were posting that. So, my fault too, sorry.

I believe that would be more of a bug

I agree, anyway I switched to use PipeFS to store parameters passed from the first run and that seems to work fine from RO 3.10 onwards. I’ll run more tests to ensure everything is stable and the OS doesn’t get in a funky state.