hello

aaaaa

I don’t have a lot of experience with android, but the whole java thing look to me like fake multi tasking, […] Android doesn’t really handle windowed multi tasked apps, an application is made of different ‘activity’ and each activity have it’s own interface and is more or less run in full screen exclusive mode with only asynchronous task handling, […]

This depends upon how “real” you want your multitasking. I find Android is a lot better at performing multiple operations than iOS. For example, my first phone (Moto DEFY), I could choose to open half a dozen web pages in the browser, then switch to read email, while listening to an MP3. By the time I read the email, I could switch back to the browser and the pages should have loaded. This is no longer possible on my Sony phones, leave the browser for anything (even change MP3), going back will trigger a refresh. Most bl**dy annoying behaviour.
Yes, apart from “widgets”, pretty much every app runs full screen. This isn’t because Android is bad or can’t multitask, it’s because there is an entirely different paradigm at work on mobile devices. I don’t think I have ever seen a “window” in the traditional sense on either Android nor iOS.
As for iOS… System tasks (App Store, Email, etc) can multitask, but it is really quite surprising how many things are unable to multitask. I can download things using Dolphin, and I can upload videos directly to the iPad using the server built into VLC. However if data is being transferred and I switch out to another task……

It seems the core system it use would be able to do what i want to do, but if i understand well the principle, you can have only one loop per wimp application,

Sort of. Generally there is only one polling loop per application, although for special cases I write special loops with their own handler code. An example here is a multitasking error box so the application can be ‘frozen’ (it will “bing” and push the error message to the front) while polling continues so everything else works.

and the os manage each application as a separated task in the multi tasking system, but a wimp application cannot start a new multi tasked wimp application wihtin itself ?

It can start a new Wimp application, but if you are thinking of threads, then no. RISC OS has a concept of applications, it doesn’t have a concept of threads.

The idea would be to have the server application run as a wimp task, which would mostly be listening for request on a socket and process them, if it find the request target a cgi file, at this point it should be able to create a new task with its own Wimp_initialize and own polling loop.

If you want to repeatedly check to see if there has been any activity, this should work. If you want to use any events to be notified upon activity, this absolutely will not work. The difference is that you cannot pass addresses to things within your application space to OS routines that call you (ClaimEvent, Callback, CallAfter, etc). This is because the addresses in your application space will only be valid while your application is paged in, and at all other times, those addresses will point at something else entirely.

The poll would not be called within those ‘sub task’, but as it’s a non windowed pure server task, it shouldn’t receive much event on the poll, unless there are event linked to file system operation, but then each ‘sub task’ would be run independently from the main server wimp application.

I have written a telnet server and it works more or less like this:

• Main server loop is a module application. This is because there is a module providing centralised services for what the server is attempting to do. It listens on the nominated port for connections.
• When a connection is detected, it is quickly filtered (if the IP address is on the ban list) or the server is disabled for maintenance – in which cases the server will send a brief message and disconnect.
• If the connection is still wanted, the server will start up a task to handle the connection, passing a “port number” and the socket handle to the task being started.
• The server then schedules a three second callback.
• At this point, it is expected that the task to handle this connection will start and take up control of the socket. If this is successful, the task registers itself with the server module and in doing so will cancel the callback.
• If the callback fires (wasn’t cancelled in time), the server will send a message to the user then disconnect them and finally log an error.
• The server module is given the task handle of tasks that start and register themselves, so it can also detect task crashes and tidy up (take over the socket, send an apology, disconnect, log an error). The task must “unregister” prior to quitting normally. This also has the benefit that if the task is in a crash state, it doesn’t need to worry too much about the current connection, the server module will tidy up.

All the way through this, the server is capable of responding to incoming requests and spawning new tasks to handle them. I tried a while back with the server on a Pi and connections from my PC (via WiFi LAN) and I think I had around 14 connections running concurrently before I got bored. ;-)

Each would have their own buffer setup by the main server task, and the main server task could still process new requests while the sub task is doing what it has to do.

The problem there is in having the buffer and the task accessing it being separate. If you are implementing the server in a module, it might be doable. If you are implementing the server as a regular task, it is not. Sure, you can do it using the Wimp_TransferBlock (IIRC) SWI, but how that works is basically to page out part of your task, page in part of the task with the data required, copy stuff around, then page in/out as necessary to restore sanity. It would be better to just maintain your own block.

The main server task would just need to know when the sub task has ended and get it’s output buffer back, or the sub task could send the result directly on the socket by itself if socket sharing between two wimp task is possible.

If you send a “register” Wimp_Message to your server, you can extract the task handle from that. Then just keep an eye on what tasks are closing down. If the handle matches…

runtime stack has to be set up. An assembler ‘veneer’ is the
only way to do that.

Can’t GCC do this? With the DDE, you can pass a stub (created by CMHG) to OS_CallEvery which will set up the environment correctly to call the handler of your choice.

Note that you are limited in what you can do (probably no file operations, FileCore is not re-entrant) so your CallEvery handler will need to schedule a Callback which is when it is safe to do stuff. But you will need an interlock to not schedule if one is currently pending (things get upset otherwise).

Am i correct if is understand that for example, if i would set such a timer on a wimp application, but that the wimp application would be ‘busy’ and not calling the polling function, eg running a php cgi, the timer callback would still be called ?

Sort of. What you say is correct, but if the address passed to CallEvery / Callback is within the application that will be polling, you must remove all traces of this prior to executing the poll. Think of what would happen if the event fires and the OS branches to the address given which is some random place in some random application…

That said, as Steve points out, if any of your tasks are not polling, then yes, the timed events will still fire but everything will appear to have ground to a halt. Multitasking will stop. No if, but, or maybe.

This is why I said above that RISC OS is not Linux. You cannot think of the Linux paradigm of writing some code and letting the OS decide how best to execute it. Applications for RISC OS that are to work in a multitasking world need to be written with that in mind from the ground up. You will come astray if you think of linear application code and then try to bolt on some multitasking behaviour later.

Also, be careful of David’s suggestion:

The best for your Wimp application would be to launch CLI tasks in task windows. So they’ll use PMT.

Yes, this is correct. The TaskWindow provides a pre-empted environment to run non-Wimp tasks within the Wimp (most of us use it as a quick and easy command line, but it also holds true for compilers and assemblers!); however it submits you to the same restrictions as a normal polled app on the use of addresses passed to OS routines that will call you back at unknown times. Namely, if you have a CallEvery running and the TaskWindow module polls (thus you’ll get paged out), BANG!

aaaaa

I have read also that a wimp application can be made as a module ?

Yes. In two ways. The common way is a module that starts itself as a standard Wimp task. Runs at &8000.

The more complicated way (and not directly possible with DDE) is to run as a module task. The difference here is that it runs in module space, not &8000, so when it is paged out, the addresses are still valid.

would respond to sys calls asynchronously ?

How do you mean? I don’t recall the exact SWI that I use in my task, Socket_Select I think. Anyway, given a handle reference, it will inform me if there is data pending. So I don’t need events, I don’t need to fart around with CallAfter, I just do this:

do
  poll the wimp
  if (event)
    deal with it

  check the socket
  if (data)
    deal with it
until bored

I’ll keep looking into this, but i’m sure there is a way to do what i want to do in a better way than spawning a new application instance each time.

Because of the nature of what I am doing, I need to spawn a new task to uniquely handle each connection. This is not necessary, it makes a lot of the internals of my code easier – the internal state of the task relates to the user that is connected without worrying about anything else. It is possible to handle multiple connections in a server in a single task. I suggest that you look at the source code of WebJames, the RISC OS http server. That might provide some instruction on how all these parts fit together.
WebJames can also run a(n ancient) version of php, it might be interesting for you to see how this is achieved.

The thing is the first application need to have a way to communicate with the others,

Wimp_SendMessage from the sender, to the Message event of the receiver’s polling loop. You can even have it with registered-must-be-signed-for delivery if you like. ;-)

Or is there a framework one level below wimp that i could use to spawn task programatically ?

The Wimp’s task management interacts with the kernel’s paging and memory handling code. Best not interfere with that.

RISC OS (which is not Linux…) doesn’t really have stuff like “framework”. There is a basic eighties style single tasking single user operating system. On top of that sits a clever task switcher and windowing system. There is no in-between. This isn’t Linux where you have loads of layers between the bare metal and the UI the user sees.

and then sever would do it’s own task switching inside of it’s pool of running cgi task.

For this plan, you have three possibilities, in decreasing order of baldness-inducing:

1. Write your own multitasking system. Non-trivial. ;-)
2. Write something akin to what TaskWindow does. I suggest you read through http://gerph.org/riscos/ and realise that TaskWindow is….hairy (to put it mildly).
3. Rethink your server plan so that it can be written as a RISC OS application.

and either also a socket to send the buffer back directly on the socket that would have been initialized by the main application, or either it would need be able to communicate back with the first app to send it’s output when it’s done.

Does it? Can you not just hand over the socket handle and let the subtask talk directly to the socket for the duration of its work?

The way i see it if that would be possible with this function, there would be a single wimp app with 3 main entry point :

What about:

do
  poll the wimp
  if (event)
    deal with it

  check server socket
  if (action pending)
    deal with it

  check cgi list
  if (cgi is active)
    for (counter = 1 to cgi task count)
      if (cgi is in use)
        execute something for cgi #counter
    next
  endif

until bored

Your complication is in wanting to run PHP → is there a PHP available that fits in with the RISC OS way of doing things?

I only have a dim memory of the RISC OS PHP (last used it over a decade ago) but I think it was a single-tasking application that ran as a single tasking program. I’m not sure how the data from PHP got to the server – you’ll need to look at WebJames, it might even have linked into the TaskWindow I/O like how Edit does…

aaaaa

Write your own multitasking system. Non-trivial. ;-)

Time to make some announcement I planned…

Wimp2 implement CMP multitasking via time limit on task. It’s more complex than this, but basically: 1/ set a timer 2/ when interrupted, check if you were running application code, from an application that does not crash in Wimp2 mode 3/ if OK, simulate a Wimp_Poll call to go to next task.

It’s what Wimp2 did, but with the need to implement a microtimer and to redirect some system calls (with a lot of problems). With microtimers now in HAL and source code of RISC OS available, it should be much much more easy to (re)write Wimp2 today, since it will be only an ‘interrupt and switch’ module (to be more precise “set new timer if next task is PMT compliant (Ok, it’s not really PMT… much more a time limit on CMT), and switch”). Unfortunately, you can’t rely on Wimp2 for this part, since this software is a GPL one. A new, BSD, version would be better.

Nota: with timers in HAL, it would be easy to change timer delay on the fly, to make a system more or less responsive (less for calculation, more for servers, 60 fps for games or multimedia). Or even not to fix timers (CMT mode). The author of Wimp2 confirmed me that it’s more or less the way Wimp2 worked, and a way to recreate it.

Bounty of 500 euros from my eBay account to the one that will make it again.
Just contact me via RISCOS.fr.

Nota: I have also other bounties to announce, with the same amount (or less) money. Port of RPCEmu, image file support for FilecoreFS (IMHO, should be here: https://www.riscosopen.org/bounty/polls/10) , HardFP support in FPEmulator (https://www.riscosopen.org/bounty/polls/6 ?), and a few others.

aaaaa

@huebby: I hope you do not mind me saying this. I am finding your prose very hard to read. The sentences tend to be too long for my eyes to follow. How about a few more full stops? The secret of simple English is to use parataxis. Avoid phrasal conjunctions and relative clauses. ;)

.. unless with some kind of preemptive behavior.

Not so. The answer is to use coroutines, with plenty of yield points in the cgi code. After all, the wimp and applications see each other as mutual coroutines. See section 9.4, Non-Preemptive Multithreading, of the third edition of PiL (Programming in Lua, by Roberto Ierusalimschy, ISBN 978-85-903798-5-0).

aaaaa

[I wrote this last night and got sidetracked, never clicked on Save Reply… duh!]

the problem with the program you posted is that the ‘execute something for cgi’ cannot be really ‘interrupted’

That is why I said: Your complication is in wanting to run PHP → is there a PHP available that fits in with the RISC OS way of doing things?

If we were writing a script processor, we could poll every X lines of input, or after X centiseconds had elapsed. But taking something written for another system and another methodology… that’s more complicated.

But with your program , while the cgi is executing, the server cannot do anything else, and the task of the cgi cannot be really broke down easily into chunks, unless with some kind of preemptive behavior.

Not quite. The cgi should do a little bit and then return, doing a bit more on the next call. You can understand why PHP could be ‘difficult’.

but the problem with the cgi thing is that it’s assimilated to a socket in the sense it’s accessed as a file resource,

…on Linux, perhaps. Does our Internet stack even do that?

David:

Time to make some announcement I planned…

I see I’m not the only one thinking about this possibility.

It’s what Wimp2 did, but with the need to implement a microtimer and to redirect some system calls (with a lot of problems). With microtimers now in HAL and source code of RISC OS available, it should be much much more easy to (re)write Wimp2 today, since it will be only an ‘interrupt and switch’ module (to be more precise “set new timer if next task is PMT compliant (Ok, it’s not really PMT… much more a time limit on CMT), and switch”). Unfortunately, you can’t rely on Wimp2 for this part, since this software is a GPL one. A new, BSD, version would be better.

We don’t need Wimp2 at all. What we have in mind is the basic premise of what Wimp2 was trying to achieve. Now that the source of RISC OS is now available, the cleanest method would be to fold this functionality into the Wimp itself. Consider, if you like, two new SWIs. Being called something like Wimp_EnterPMT and Wimp_ExitPMT, this ought to suggest to you that what we are saying here is… What follows is a piece of code that does not multitask, ever to you to make it work. The Wimp will then let it run for a specific amount of time (say, 5cs) before yanking control away from it and switching the task out. This can also be a potentially ‘cleaner’ implementation as the Wimp itself can make a distinction between switching tasks and polling. The difference being that polling is an event-based mechanism; while the task switching could run with the task more or less ‘frozen’, only switched in and out until the PMT code has completed.

The EnterPMT call should take an options word for setting various parameters (like “this is a low priority job” and “this needs FP/VFP/Neon registers preserved” and “this runs in SVC mode”, etc). Preferably flags and not the yucky version numbers and magic words rubbish that has blighted the API since Arthur…
I would suggest that the timeslice is a “suggestion” and is left as rough grained as “low priority”, “default”, “high priority”. It should then be left to the Wimp to decide what this should actually mean.

Envisaged problems:

• It will be co-operating with CMT tasks. A task stop (error box, etc) will block the system. This is to be expected, we’re asking the Wimp to PMT us, not everything.
• There is a possibility of zombie windows. For example, if a task is blocked within PMT code and another task closes a foreground window which would require the block task’s window to be redrawn. What to do?
  Depending on complexity, it might be possible to register a routine within the task that the Wimp can call to deal with such events (this will pre-suppose C or assembler, it won’t work with BASIC); otherwise perhaps the best option is to get the Wimp to clear the window to background colour by itself, and mark the window as needing a complete redraw (when polling resumes).
  This is not, I should add, a problem specific to us. I’ve seen zombies on Windows – including full screen windows that you can neither move nor close. It may be that we cannot allow windows to auto-close (unless we flag it and auto-open at end of PMT code?).
• You will notice that my suggestion generally runs the task is time slices but does not fake a polling loop (akin to how Wimp2 worked). This removes a lot of the problems regarding how to handle poll events. If we can assume the program is incapable of responding to input (else it could just poll like normal) then it is a waste of effort to do this. It will need to be considered what to do with events that happen to the task. My personal preference is to record the state of all windows; then allow windows to be moved (closed, maybe?) and be redrawn to background automatically while in the PMT code. Anything else will cause a beep and no action will be taken. When PMT is finished, windows are restored to their original size/position and redraws are fired for each window that needs this. The idea here is to have the least amount of work to do whilst offering the least impact to the rest of the system. If you look at Wimp2, you’ll see a fair number of issues revolved around dealing with a polling loop (which is how Wimp2 worked) while the application was not actively polling. We, if using the Wimp itself, can come up with an alternative which is a non-poll task switch that switches out on a timer. No polling necessary. But only the Wimp can do this.
• It may be that some events may be missed altogether (such as Wimp Message). Tant pis, as they say over here. I am not sure that it is actually possible to multitask a non-multitasking piece of code without either running into the possibility of missing events, or blocking the system for each message that our task might be interested in. It’s just a loss we’re going to have to accept, unless anybody has some real bright ideas. [the Wimp should, itself, claim Message_PreQuit in such a case…]

Mostly just thinking out aloud…

aaaaa

aaaaa

It’s why most of the modern os and language like Android or As3 more give out framework that are used to develop application,

;-) You’re sort of comparing apples and oranges here.

Android 5.0.2 (24 days ago) → evolution of Android (2008) → built upon Linux (1991ish but that incarnation is too basic for something like Android) → ideas derived from commercial Unix and MINIX but code is related to neither.

On the other hand:

RISC OS 5.20 (2014) → evolution of RISC OS (1989) → derived from Arthur (1987) → which has a lot in common with the BBC Master MOS (1986) → derived from the BBC MOS (1981).

Seriously. If you have some free time and nothing to read, go grab yourself a copy of the BBC Advanced User Guide (PDF here) and browse it. Look at, for example, Vectors (p253) and also Events (p287). Seem familiar? :-)

We can but wonder at what might have been; however RISC OS, for all of its simplicity, is what we have here and now, and indeed if you compare Android starting up on any mobile phone (takes about a minute at its fastest on my phones) with RISC OS starting on a slower less capable device such as a Pi (takes maybe fifteen seconds from power up to desktop (beta versions are slower thanks to the list of module inits)), you can see that something simpler and less complicated wins in the speed stakes.
Android is, certainly, a hell of a lot more capable and has frameworks for everything under the sun (doesn’t Android 4.something have an API to read the movement sensor to determine if the user is running, walking, cycling, in a car…?); however the price you pay for this is an increasingly severe lag in the UI. My Xperia U was fairly slick when I bought it, and with a dual core 1GHz processor, it isn’t a slouch. Thanks to dozens of Google Play updates (which I think actually patch other parts of Android), switching tasks can be counted in seconds, sometimes double digits. The touch UI can lag behind my fingerwork, and some of the more intensive apps (Amazon, anybody?) can take 45s-1min to load. I’ll be getting a new phone soon (contract change). I’m sure it’ll be hot poop until Android has evolved enough to make it crawl. :-/

So there’s your choice. A nice feature laden system that will struggle on anything “last year”, or a simple system with a basic multitasking environment (the core OS doesn’t) that runs fast enough to make you gasp.

aaaaa

The Wimp will then let it run for a specific amount of time (say, 5cs) before yanking control away from it and switching the task out. This can also be a potentially ‘cleaner’ implementation as the Wimp itself can make a distinction between switching tasks and polling.

If it’s possible to apply this on old code (with or without success), why not…

The Wimp2 approach was simple: interrupt applications (that’s always possible), see if we are in a safe part of execution, then switch. If not working, blacklist the application. And of course a framework to be sure that an application is “time limit” compliant. Catch me if you can :)

Nota: this is not PMT. Much more CMT with interruptions. So it’s not compatible with SMP. Anyway I prefer AMP with a simple interface (connect to other cores via CLI, send data, send commands, get data). The same interface could be used for clusters. CMT and SMP in RISC OS could be better, but if old software cannot work any more, it’s in fact a new OS.

If we were writing a script processor, we could poll every X lines of input, or after X centiseconds had elapsed. But taking something written for another system and another methodology… that’s more complicated.

My opcode engine has a very simple execution loop that could be used for this. Self modifying opcodes are also an idea to share CPU time between scripts. Of course, it’s not PHP any more :)

Code available on request.

i just take the asynchrousnous java class of the android sdk as an example of how this mechanism can be implemented, but it’s not very complicated in itself.

Not complicated?
Hmmm, first implement Java on RISC OS then we can check what works easily.

Note: This isn’t dissing the ideas, it is just pointing out how often in looking at what can be done you hit the “well things are simple in Java, but we don’t have Java” or “my PC browser is just fine on this or that site that uses Flash on the front page, but RISC OS browsers won’t display anything because there isn’t a flash enabled browser”.

aaaaa

David:

The Wimp2 approach was simple:

The Wimp2 approach looks simple, but is in fact horrendously complicated.

Yes, you can always interrupt an application – and that is what Wimp2 did – but in order to task switch outside of the Wimp, you effectively need to call Wimp_Poll. Which is what Wimp2 did.
Now consider how you are going to handle the wimp polling behaviour of a program that is not capable of polling. Suddenly it gets very complicated.

This is why I suggest that this really needs to be done by the Wimp itself. It is the one place in the entire scheme where the task switching can be performed without a polling loop behaviour. In essence, the Wimp will know that the task is unable to receive events so it will just let it run for a length of time before switching it out. An external module cannot do this without making a hell of a lot of assumptions (and we all know that “assume” makes an ASS out of U and ME (sorry, lame, really lame…)).

but if old software cannot work any more,

Indeed, this is the important point; and why my proposal is for elective task switching (namely, the app “asks”). It needs to work alongside and co-exist with existing software.

Self modifying opcodes

Ah. That old chestnut. Otherwise known as “how to mess up the pipeline and the cache in one easy lesson”.
Self-modifying was a cool hack in the ’80s. Now it is just a bit icky.

h0bby1:

well i made a system like that in asm for intel, it’s preemptive, and can have this kind of behavior, and the whole iso is about 1.2Mo and it contain plenty of other stuff.

Sounds like you wrote the crux of the multitasker from the ground up.

Here, you either have to co-exist with the current Wimp; or you will need to write your own. Note that yours will need to be compatible with the Wimp (quirks and all) or people will be like “Oh no, this doesn’t run XYZZY!”.

Having said that, I think you might be better to downsize your plans a tad, see how you get on with something a little less radical than reinventing the task switching methodology.

Yes, you can always interrupt an application – and that is what Wimp2 did – but in order to task switch outside of the Wimp, you effectively need to call Wimp_Poll. Which is what Wimp2 did.
Now consider how you are going to handle the wimp polling behaviour of a program that is not capable of polling. Suddenly it gets very complicated.

I know. But most people don’t want CMT. They just want a more fluid UI.
For this, Wimp2 was limited, but a good first step.

In a way, we need the two solutions.
When the timer interrupts the task:
1/ the ability to throw a Wimp_Poll for classic or old applications that does not crash with this solution.
2/ the ability to switch tasks that does not use Wimp_Poll (needs to change their code).

My idea is to get first part 1, ie more speed to switch between some Wimp tasks. A ‘better than nothing solution’ or a CMT accelerator, if you want :)

Without the timer & vectors part, source code of Wimp2 is not very big. Problem: it’s GPL, so no way to integrate this CMT accelerator in the ROM. There is no other solution than to write another one from scratch.

For me, plan is clear. I just want something that can switch Wimp task by simulating a Wimp_Poll. Some applications will work, some will not (example, the Filer).

To ease the work of the accelerator, some applications could say ’I’m compliant’ when starting. Simply by using a Wimp2_Run command in their !Run file. A list of compatible software could also be loaded in memory with the module. I think the 2 solutions are good and could coexist.

Then, if there is a way to make the second part, that’s OK for me too, but later.

The problem is that everyone wants something powerful, complex and perfect. So we are waiting. A short term solution would be cool too (example, the Midi module: not perfect, but good enough to use Midi and make new applications).

“Self modifying opcodes”
Ah. That old chestnut. Otherwise known as “how to mess up the pipeline and the cache in one easy lesson”.
Self-modifying was a cool hack in the ’80s. Now it is just a bit icky.

You don’t get the point: self modifying opcodes in my opcode engine. I talk of an interpreter engine. Here, all is possible. Of course, a lot of people will say: self modifying code is not good for developers. Not so true, if it becomes a facility (to implement threading or debugging for example). The same with reverse code (ability to play the code in the other direction): precious for debugging.

A long time ago, processors became too complex for these techniques. So we throw them from compilers. But why did we forget them on interpreters?

aaaaa

aaaaa