Running programs

There are many ways programs can be started from other programs in RISC OS. Do I understand these correctly?
(I’m updating this posting in response to any corrections, so it’s all in one place.)

Wimp_StartTask

• saves the current application and starts a new one.
• resumes the caller (“returns”) when the new application either exits or calls Wimp_Poll the first time

Filer_Run

• Queues a filename to be sent with Message_DataOpen, if that doesn’t and should that bounce
• If that fails, execute with Wimp_StartTask
• may be asked to examine the state of the shift key or not

OS_CLI – module star command or utility

• Runs the command and returns to the caller, unless it in turn calls…

OS_CLI – application or Relocatable Module Run (start)

• Replaces the current application with the new one, never returns

*WimpTask

• Uses Wimp_StartTask, exits via OS_Exit if run from outside a Wimp Task (whatever that means)
• Not safe to use from BASIC (ditto)

*StartDesktopTask

• This *Command starts a new task next time the desktop environment is entered
• Unlike Filer_Run, this takes a whole command line, with parameters, not just a file name

*Run

• Calls OS_CLI (with the same restrictions)

OS_FSControl 2

• Kills your application unless OS_UpCall 256 is claimed
• Issues Service_NewApplication to inform modules. If the service is claimed, the error is sent to your parent’s error handler.
• Unless the exit handler is below memory limit then all handlers below memory limit are reset to their defaults (OS_ReadDefaultHandler).
• The currently active object pointer is set; the current time and command tail are set (OS_WriteEnv).

OS_FSControl 4

• Files with load and exec addresses will be loaded and executed (are there still any?)
• Files of type &FF8 (Absolute) will be loaded and executed at &8000.
• Utilities (type &FFC) will be loaded into RMA and run.
• All other filetypes will be searched for in Alias$@RunType_xxx, where xxx is the filetype. i.e &FA8.
• It is possible that this call will not exit depending on the execution of the file.

*ShellCLI_Task

• Depricated

*ShellCLI

• Uses secret sauce: Shell_Create, Shell_Destroy

When do you use this instead of Filer_Run, or vise-versa?

The wiki doesn’t help, but I have a feeling it might be something to do with passing parameters…?

Oh, and you forgot the basic *Run command. ;)

Thanks!

For non wimp application I use a taskobey file with:
dir <obey$dir>
!RunImage
The command line application opens in a Taskwindow,
With command filer_run !RunImage, no taskwindow.

Thanks for the Running program list.

I think that’s the same as `*Run !RunImage` in the appropriate directory.

It’s not safe from another task setting the current working directory between the two lines. I just put a “fx 129, 255, 127” between the two lines and changed the working directory from a filer window and it ran the !RunImage in a different directory. (Five minutes later, I should have RTFM’d the timeout rather than just choosing the maximum! “fx 129, 255, 0” is about 2.5 seconds, long enough to check.)

I also wonder where the output from a Filer_Run program is directed to. If it’s run from a command line (not a WimpTask), it opens a window that blocks the Wimp until closed.

I also wonder where the output from a Filer_Run program is directed to

Like anything else started with Wimp_StartTask, it’s hoped that it doesn’t output anything before Wimp_Poll.

opens a window

That’s a generous description! Clears an area of screen and sets a text window is more like it.

It’s not safe from another task setting the current working directory between the two lines.

Even little ol’ me notes that there is a reason for doing a

set MyAppDir <Obey$Dir>

followed by

Run MyAppDir.!RunImage

I even managed to recall the bit about MyAppDir being fine and not needing to be MyApp$Dir although everyone’s habits are so ingrained most instances are in the MyApp$Dir format

OS_CLI – application or Relocatable Module Run (start)

* Replaces the current application with the new one, never returns

Technically this will return, when the application exits. It’ll return via the error or exit environment handler. So if the application/module is well-behaved (i.e. it makes sure to invoke the error/exit handler that was in place when it was started), then the caller of OS_CLI can set up special error/exit handlers to catch when the child exits and resume execution. This is how system() works – it moves the application’s heap to the top of the wimpslot, uses OS_ChangeEnvironment 0 to set a soft cap on how much memory the child can use (to stop it overwriting the parent – although this also has the downside in that it stops the child from being able to grow its wimpslot), and then sets up error/exit handlers which will undo all that and resume execution of the parent when/if the child exits.

Also you’re missing OS_FSControl 2/4 from your list – these are the calls that do the core work of starting a new application.

Here’s a flowchart I put together while working on the multicore work, which you might find useful. It’s a bit out of date (no AbortTrap) but it should cover most of the key areas.

https://www.phlamethrower.co.uk/misc2/ProcessModel.pdf

Thanks for the flowchart. Looks pretty simple. Wibble. :)

I have to admit to having largely ignored exiting programs up until now, it was more important to get them running in the first place. I’ll have to look into the exit handler thing in more detail.

Has anyone ever done a flowchart of the Wimp?

• Application starts as the only process, mapped in at &8000
• Calls Wimp_Initialise to mark it as multitasking
• ???
• Profit!

Has anyone ever done a flowchart of the Wimp?

Do you mean what happens when you click a mouse button or hit a key, and then where does the user action event flow in the code to produce the effect on screen etc?

Edit: Interesting question that could be useful to anyone writing a C equivalent of the Filer/Wimp/Switcher¹ and how that presents to the User Apps

¹ Lots of interlinks between them and more

Has anyone ever done a flowchart of the Wimp?

The problem is, the Wimp is many things so any flowchart will be extraordinarily hairy.

Wimp → Automatic behaviours (redraw, autoscroll, open submenu, etc)
Wimp → Underlying actions (responding to events for key presses, mouse movement, etc)
Wimp → Memory/application shuffling (for BlockCopy, it gets hairy)
Wimp → Task startup (POV of application)
Wimp → Task startup (POV low level from within another task)
Wimp → Task shutdown
Wimp → Polling (task switching (poll entry))
Wimp → Polling (providing events to the user (poll exit))
Wimp → Polling (deciding which task is ‘next’ (null events/PollIdle/etc))
Wimp → Message passing
Wimp → Creating a window
Wimp → Drawing a window (including validating its position/flags)
Wimp → The window stack
Wimp → Creating an icon
Wimp → Drawing an icon
Wimp → Creating a menu
Wimp → Decoding menus
Wimp → Caret handling
Wimp → Working out what needs redrawn (don’t forget, could be a window with others on top)
Wimp → Displaying sprites (including working out how to draw in current mode)
Wimp → Colour handling
Wimp → The filter mechanism (that few people use!)
Wimp → Handling the desktop font and actions with said font
Wimp → Handling errors, including the less common parts of ReportError like “return immediately”.

And that’s off the top of my head. I’m sure there’s plenty of stuff I’ve missed.

I thought I’d ask if Jeffrey had an easy way to make a black and white version suitable for sending to a laser printer, but I tried anyway (from my phone) and… it’s damn near unreadable because there’s so much.

https://heyrick.eu/blog/images/20221230!processmodel.jpeg
(taken with the macro camera, which is why it looks a bit odd)

I think I’m going to have to toss the file to the PC and see if my old copy of Acrobat can print this across multiple pages, or something.

Edit: Done. At 36%, the text is small but readable and it spreads the content across three pages (landscape). If I were to print at 100%, it would spread across 24 pages. Nerdiest wallpaper ever?

What I am trying to implement is a reduction in the number (hopefully to zero) of used Service Calls, Up Calls, Handlers, etc. in favour of a more streamlined approach (with emulation of those functions for older code).

The Wimp shouldn’t be the lord of all multi-tasking, but the cooperative model is nice and practical (there is only one person sitting at the computer). Threads being able to set the poll word and trigger the Wimp to look at them will keep the responsiveness while also allowing programs to run multiple non-Wimp threads to update windows and stream audio.

@ Simon

Another bit of info is using the "" to wrap a filename.

It is supposed to allow the use of spaces in a file name or directory name, so, for instance:

*StartDesktopTask "@.!Runimage params"

Means a file name that should be called @.!Runimage\x20params, while:

*StartDesktopTask @.!RunImage params

Will run @.!RunImage and will pass it params as parameter

Not sure if this may help you, but the use of "" can be confusing.

You meant to type StartDesktopTask in your second example, Paolo.

Threads being able to set the poll word and trigger the Wimp to look at them will keep the responsiveness while also allowing programs to run multiple non-Wimp threads to update windows and stream audio.

Thanks Ronald, fixed the typo! Cheers :)

Oh, ShellCLI_Task has a horrible interface! Fortunately “This command is now deprecated, and must not be used.”

https://www.riscosopen.org/wiki/documentation/show/*ShellCLI_Task

This list is… odd… because many of these aren’t ways of starting programs and you’ve not really defined what a ‘program’ is… but most of the things you cite are jut the ways that users pass call on to other parts of the system – they largely just decompose to one set of calls OS_FSControl 4, and OS_FSControl 2.

4 is used for the vast bulk of the times that you’ll actually start something.

2 is used by 4 (IIRC) to indicate that a new application is starting, and by any client that wishes to start an application without running a new file (eg entering a module, or *Go).

It isn’t OS_CLI that is starting a module but OS_Module 2.

If you define a program as being something that has a new program environment, then largely this decomposes to things that pass through OS_FSControl 2. That means:

• OS_FSControl 4 to run programs.
• *Go
• OS_Module 2 to start a program.
• (anything else that emulates these calls… which isn’t that much – my experimental filesystem cache module performs these operations to cache executed programs)

So all the other SWI calls that you mention are pretty much irrelevant to the process of starting a program. They all decompose to those those 4 (and really they should be encouraged to not use the latter mechanism, as that freedom makes things awkward for any future development).

Filer_Run – Queues a file name for execution with Wimp_StartTask when the Filer’s Wimp_Poll returns may be asked to examine the state of the shift key or not.

Incorrect. This operation queues the filename to be sent with Message_DataOpen, and should that bounce, you get the Wimp_StartTask.

*Run – Calls OS_CLI (with the same restrictions)

Incorrect. It calls OS_FSControl 4.

OS_FSControl 4 … (sequence of operations)

This is decomposed further in RISC OS Select and RISC OS Pyromaniac to make it simpler to replace these operations, and thus support extensions.

Instead of directly performing the execution, OS_FSControl is largely a dispatcher.

• Directories are redirected to .!Run (ie we continue as if the !Run file has been found).
• Files with type &FF8 get dispatched to OS_FSControl 60 (FSControl_RunAbsolute)
• Files with type &FFC get dispatched to OS_FSControl 62 (FSControl_RunUtility)
• Files with load and exec addresses get dispatched to OS_FSControl 61 (FSControl_RunUntyped)
• Other filetypes get dispatched to OS_CLI.

In RISC OS Select:

• The handling of OS_FSControl 60 is performed by the AIF module.
• The handling of OS_FSControl 62 is performed by the TransientUtility module.
• The handling of OS_FSControl 61 is handled as a fallback inside FileSwitch.

These changes were made in Select to make it easier to modify the mechanism by which processes are started, to allow easier hooking of the system should you wish to execute unsupported code on the system (eg running 26bit code on a 32bit only system), and to allow the handling of untyped files to be entirely removed in the future more easily. The implementation of both AIF and TransientUtility is almost entire in C – moving the code to the vector, out of Fileswitch meant that it could be implemented and maintained separately, and didn’t involve having to restart FileSwitch, which is significantly messy.

The implementation for `riscos.filesystem.Filesystem`s `run` method (which is directly called by the OS_FSControl 4 vector handler) in RISC OS Pyromaniac is thus:

    def run(self, ropath, args):
        if self.debug_fsrun:
            print("Run file %r %r" % (ropath, args))

        direntry = self.findobject(ropath, self.lookup_path_prefix('Run'))
        if direntry.objtype == 0:
            raise FSFileNotFoundError(self.ro, "File '%s' not found" % (ropath,))
        if direntry.objtype & 2:
            # This is a directory, so we need to check for !Run
            ropath_run = '%s.!Run' % (ropath,)
            direntry_run = self.findobject(ropath_run, self.lookup_path_prefix('Run'))
            if direntry_run.objtype != 1:
                raise FSFileNotFoundError(self.ro, "Object '%s' is a directory" % (ropath,))

            # The file we will run is the !Run file
            ropath = ropath_run
            direntry = direntry_run

        filetype = direntry.filetype()
        loadaddr = direntry.loadaddr
        execaddr = direntry.execaddr

        if filetype == filetypes.FileType_Application:
            # Absolute
            try:
                self.ro.kernel.api.os_fscontrol(highfsi.FSControl_RunAbsolute, direntry.fullpath, args, 0)
                return True
            except RISCOSError as exc:
                if exc.errnum != errors.ErrorNumber_BadFSControlReason:
                    raise

            # There is no handler for an absolute file... so we fall back to just running it like
            # it's untyped, and fall through to the untyped handler.
            filetype = None
            loadaddr = 0x8000
            execaddr = 0x8000

        if filetype is None:
            # Untyped file
            try:
                self.ro.kernel.api.os_fscontrol(highfsi.FSControl_RunUntyped, direntry.fullpath, args, 0,
                                                loadaddr, execaddr)
                return True
            except RISCOSError as exc:
                if exc.errnum != errors.ErrorNumber_BadFSControlReason:
                    raise
                return False

        if filetype == filetypes.FileType_Utility:
            # Utility
            self.ro.kernel.api.os_fscontrol(highfsi.FSControl_RunUtility, direntry.fullpath, args, 0)
            return True

        # Typed file which isn't handled internally
        varname = 'Alias$@RunType_%03x' % (filetype,)
        load_action = self.ro.kernel.api.os_readvarval_string(varname)
        if not load_action:
            raise FSUnknownRunActionError(self.ro, "No Run action specified for file '%s' of type &%03x"
                                                        % (ropath, filetype))
        else:
            self.ro.kernel.api.os_cli('@RunType_%03x %s %s' % (filetype, direntry.fullpath, args))
            return True

See https://gerph.org/riscos/ramble/filesystems.html for a small segment on FileSwitch file execution restructuring, and https://www.riscos.com/support/developers/riscos6/programmer/codeformats.html which gives more information on the manner in which code is executed in RISC OS Select and the extensions to make it easier to replace segments of the system and introduce debugging information.

Well, not surprising because it’s clear as mud what a program is and whether it will return to the program that starts it or replace it. That’s why I wrote this!

This, for example, is very confusing to me:

OS_FSControl 4 to run programs.
OS_Module 2 to start a program.

What’s the difference between running and starting a program?

Incorrect. [*Run] calls OS_FSControl 4.

Ah, *Run only works on files, not commands, so

*Run cat

says “File ‘cat’ not found”.

So, I’m guessing the sequence is:

1. OS_WriteEnv to set the command line
2. OS_FSControl 4 to call…
3. OS_FSControl 2 to clear the way and then
4. load and run the file?

Bearing in mind the existing application may refuse to be replaced by claiming UpCall 256.

OS_CLI, on the other hand, tries to run module commands before defaulting to OS_FSControl 4.

Blast, the command name is passed to OS_FSControl 2 in R3

Well, not surprising because it’s clear as mud what a program is and whether it will return to the program that starts it or replace it. That’s why I wrote this!

This is why I keep on saying repeatedly that RISC OS needs a process model before you can do any sort of work on how programs operate. It doesn’t have one right now and none of the things that we’ve discussed here are actually required for you to write RISC OS programs. Tying these things down and abstracting parts of the system so that it would be possible to make the system more flexible in the future was the primary goal of my work on RISC OS Select.

This is why I described a program in terms of things that set up new environments, as that covers the 3 primary cases of running an absolute, entering a module, and *Go.

As for ‘whether it will return to the program that starts it or replace it’, once you have passed the call to OS_FSControl 2, you always return using OS_Exit or OS_GenerateError (or implicitly by returning an error from a SWI or generating an exception which calls OS_GenerateError). This is why you must always have environment handlers for Exit and Error present, and have either installed or reset the abort handlers to something that will call the error. But the point is that you always return to that environment. Whether the environment is the environment of the program that started you or the program that started it, is down to the service and upcalls issued as part of OS_FSControl 2.

OS_FSControl 4 to run programs.

OS_Module 2 to start a program.

What’s the difference between running and starting a program?

A typo… I meant ‘start a module’ for the OS_Module entry point. OS_Module 2 is enter module.

So, I’m guessing the sequence is:

No.

OS_FSControl 4 is used to say “I want to run this file”. You make that call, and you may never return.

As part of the implementation of OS_FSControl 4 (obviously on the vector), a call will be made to OS_FSControl 2 IF a new application is being started (which it might not be, because the file might not be found, it might be typed to execute a RunType, or whatever). It’s only when you go down the OS_FSControl 60 path (RunAbsolute) (or its equivalent in the older RISC OS 5 source) that it starts to work out the operations for running an absolute file and calls OS_FSControl 2.

The purpose of OS_FSControl 2 is to do the necessary operations to say ’I’m about to take away your application… speak now or you will never be able to hold your peace because I’ll have overwritten you’, and sets up the environment for you – you don’t do that environment setup if you call OS_FSControl 2. Clients are meant to remove their environment handlers on receiving these calls and restore the environment back to a from which they can be returned to cleanly, which is usually their caller’s handlers, but might be the high memory handlers installed as part of the ‘move up memory’ trick that is used in place of process handling in C.

For reference, the implementation of OS_FSControl 2 in RISC OS Pyromaniac:

@handlers.osfscontrol.register(highfsi.FSControl_StartApplication)
def OS_FSControl_02(ro, reason, regs):
    """
    OS_FSControl &2 - Informs RISC OS and the current application that a new application is starting
    =>  R0 = 2
        R1 = pointer to command tail to set
        R2 = currently active object pointer to write
        R3 = pointer to command name to set

    <=  Registers preserved - may not return
    """
    tail = ro.memory[regs[1]].string_ctrl
    cao = regs[2]
    command = ro.memory[regs[3]].string_ctrl

    if ro.kernel.filesystem.debug_osfscontrol:
        print("StartApplication: command=%r, tail=%r, CAO=&%08x" % (command, tail, cao))

    # Issue UpCall 256 (new application starting)
    #   If claimed, return 'Unable to start application'
    rout = ro.kernel.call_upcall(osupcall.UpCall_NewApplication, preserve=True, rout=[0])
    if rout[0] == osupcall.UpCall_Claimed:
        raise RISCOSSyntheticError(ro, "Unable to start application (UpCall was claimed)")

    # Issue ServiceCall &2A (NewApplication)
    #   If claimed, we must issue OS_GenerateError 'Unable to start application'
    rout = ro.kernel.call_service(services.Service_NewApplication, preserve=True, rout=[1])
    if rout[1] == services.Service_Serviced:
        # Claimed, so error.
        # FIXME: We cannot actually use OS_GenerateError because that'll just raise the
        #        exception here, which will be caught by the OS_FSControl dispatch and turned
        #        into a returned error. We need a better way to explicitly call the error handler.
        raise RISCOSSyntheticError(ro, "Unable to start application (Service was claimed)")

    # Reset environment handlers
    # If ExitHandler >= MemoryLimit:
    #   Reset all handlers to their default values
    memorylimit, _, _ = ro.kernel.api.os_changeenvironment(envnumbers.MemoryLimit)
    exithandler, _, _ = ro.kernel.api.os_changeenvironment(envnumbers.ExitHandler)
    if exithandler <= memorylimit:
        if ro.kernel.filesystem.debug_osfscontrol:
            print("Resetting the environment handlers to defaults")
        for envnumber in (envnumbers.UndefinedHandler,
                          envnumbers.PrefetchAbortHandler,
                          envnumbers.DataAbortHandler,
                          envnumbers.AddressExceptionHandler,
                          envnumbers.OtherExceptionHandler,
                          envnumbers.ErrorHandler,
                          envnumbers.CallBackHandler,
                          envnumbers.BreakPointHandler,
                          envnumbers.EscapeHandler,
                          envnumbers.EventHandler,
                          envnumbers.ExitHandler,
                          envnumbers.UnusedSWIHandler,
                          envnumbers.UpCallHandler):
            handler, handlerws, handlerbuffer = ro.kernel.api.os_readdefaulthandler(envnumber)
            ro.kernel.api.os_changeenvironment(envnumber, handler, handlerws, handlerbuffer)

    # Set the CAO
    ro.kernel.api.os_changeenvironment(envnumbers.CAOPointer, cao)

    # Set the environment string and time
    quin_time = ro.kernel.api.os_word_quin_time()
    ro.kernel.api.os_writeenv(' '.join([command, tail]), quin_time)

    # FIXME: Set temporary FS to current FS

The implementation in RISC OS Pyromaniac of OS_FSControl 60 which actually runs the application (which is similar to the behaviour of OS_FSControl 4 in the older RISC OS 5 source) is:

    def run_absolute(self, ropath, args, length):
        # FIXME: Check what the AIF module does
        if self.debug_fsrun:
            print("Run Absolute file %r %r, length %i" % (ropath, args, length))

        needs_loading = (length == 0)
        if length == 0:
            direntry = self.findobject(ropath, self.lookup_path_prefix('Run'))
            if direntry.objtype != 1:
                self.ro.kernel.api.os_file_make_error(ropath, direntry.objtype)
            length = direntry.size
            ropath = direntry.fullpath

        # Load the file
        if needs_loading:
            self.load(direntry.fullpath, None, 0x8000)

        # Issue OS_FSControl FSControl_StartApplication r1=tail, r2=CAO, r3=module title
        # *that* is the thing that actually sets up the OS_WriteEnv call, and issues
        # upcalls, services, and manipulates the error handlers.
        self.ro.kernel.api.os_fscontrol(highfsi.FSControl_StartApplication,
                                        args,
                                        0x8000,  # FIXME: Check the CAO
                                        ropath)

        # FIXME: Service UKCompression

        # Start the application, with a flattened SVC stack.
        raise RISCOSApplicationEntry(regs={15: 0x8000},
                                     mode=self.ro.regs.MODE_USR)

As you can see, there are two FIXME marks – one for checking what the AIF module does, which is actually pretty well documented in the codeformats link that I provided, and the Service_UKCompression, which I don’t handle yet, which is also documented pretty well in there and the SA documentation.

You can also see that NONE of the checks that are mentioned in the codeformats document are performed here. They should be, but I’ve been lazy and implemented what ancient RISC OS does, not the safer RISC OS Select implementation. Hence the FIXME.

The exception RISCOSApplicationEntry is a call out to the upper parts of the system to execute the code – it flattens SVC stacks (and other actions for such behaviour) and then calls the execution address requested. It’s implemented as an exception here because we need to flatten the SVC stack and consequently we also need to flatten the Python stack as well, which you can do by raising an exception so that we catch it at the point of a flat stack.

OS_CLI, on the other hand, tries to run module commands before defaulting to OS_FSControl 4.

You say that as if OS_CLI is an alternative to OS_FSControl 4… which it isn’t. OS_CLI has its own sequence of operations which it performs which include a lot of different things. Module commands are not ‘programs’ in the sense you’re talking about here. They’re just transfers of control that don’t involve any environment. UNLESS they call OS_Module 2 in order to enter a module, but that’s quite irrelevant to OS_CLI and just part of the follow on operations.

OS_CLI does not itself start any program. It just calls other calls that do the work. In almost all cases, all the program starts boil down to OS_FSControl 4 or OS_Module 2, both of which call OS_FSControl 2. OS_CLI does a lot of things, but actually running programs isn’t one of them – it’s just a dispatcher.

Blast, the command name is passed to OS_FSControl 2 in R3

Of course it does – it’s docuemnted explicitly as doing that, so that it can set up the environment.

Thanks for all that, Charles. It will take me a while to digest.

I have a process model, and it works in that it does what I want. Having to chop and change it to match the obscure (to me) aspects of the messages sent around by RISC OS is what’s giving me headaches.