ExtdAttrib

Some months ago at my job I detected the tortoise utility at a collegue. Tortoise is a plugin for the windows explorer to set special icons for files which have been checked out with subversion indicating their state (like updated).

I thought about a similar tool for RISC OS but I did not find a possibility to enhance the Filer in a required way.

Meanwhile the source of the Filer is open. However now I thought about whether it would be a good attempt to modify only the filer in a required way or whether there might be better solutions.

I came to the conclusion that an extended attribute handling module will solve the things better.

So I wrote a still uncomplete specification to explain the concept which I called ExtdAttrib.

For I don’t know how to provide the html file containing the specification at this forum I put it on my homepage.

It would be glad if someone can have a look on it and can tell me whether such a concept would make sense in general and if it makes sense what should/could be improved.

In short: ExtdAttrib allows fetching and modification of attributes for a single file even in multitasking mode. Beneath the usual attributes like Read/Write it shall provide icon and icon overlay data and can be extended arbitrarily by attribute handlers which might store their attributes in an arbitrary way which might include additonal files or inside ceratin file formats inside the file itself.

Of course this scheme can be used standalone but it would be nice if it could be used by the Filer e.g. to allow filtering of files by a certain attribute contents.

Regards

Thomas Milius

With the help of the ROOL-team and a lot of trial and error with Netsurf and Oregano2 I have managed it to put the specification in the WIKI. The link to my web site is therefore now obsolete.

Interesting. Hopefully in the future, RISC OS will support filing systems that are able to store arbitrary metadata with files. Would this scheme fit in with any such developments?

Making the memory caller allocated may require calling twice, once to find buffer size. This could be very expensive for some operations. Would it be better for the callee to allocate, and provide a function to free the memory.

If fetching attributes is multitasking, how are race conditions handled, such as attrs changing in the middle of reading them?

What restrictions are there on the calling program, and on the programs providing the attributes? For example, if the svn libraries were used to provide attributes, they would need to be called as a new application. If the calling application was in a critical routine at the time, e.g. a wimp redraw loop, it may not be able to cope with a new application starting.

Why do interrupts need to be disabled when writing the pollword? Having the module poking into the application’s workspace at an arbitrary time later seems a bit dangerous, what happens if the calling program has exited/died and the memory no longer exists?

How does this cope if more than one handler can provide attributes for a particular file?

Alex

Many thanks for posting the important questions.

Interesting. Hopefully in the future, RISC OS will support filing systems that are able to store arbitrary metadata with files. Would this scheme fit in with any such developments?

Are there any such developments in the moment? I assume that for such filing systems two SWIs for setting/fetching such attributes should be add to File-Switch.

These SWIs could be integrated into ExtdAttrib. So ExtdAttrib would handle a more filesystem independent general level whilst the additional SWIs will handle the direct access to data.

I assume that such SWIs will requires tags for identifying the attributes. It would make sense to ensure that the same tags are used. However because there might be different standards defined outside of RISC OS on net filing systems translations might be required.

Until such a general concept is available the according filing systems have to provide a handler directly inside ExtdAttrib.

Making the memory caller allocated may require calling twice, once to find buffer size. This could be very expensive for some operations. Would it be better for the callee to allocate, and provide a function to free the memory.

I expect that the handler will cache the results so that a second call immediately afterwards will obtain the result from the cache. I assume that in most cases the upper bound of memory can be determined. Eg. inside a Filer a first request for the first file can determine the amount of memory for a certain attribute scheme add e.g. 10% security and all other requests for further files can be performed using the size. If there is one case in which the required size is bigger it can be refetched and all fetches afterwards can use the bigger size.

Independently of this I want to implement ExtdAttrib so that it uses its own dynamic area, so of course a internal allocation could be implemented. And I think the fetch call can be easily changed in such a way that it can offer handling of both ways similar to the handling of the pollword.

If fetching attributes is multitasking, how are race conditions handled, such as attrs changing in the middle of reading them?

This is difficult. I can ensure inside ExtdAttrib that read/modification requests are queued in multitasking or rejected in non mulitasking because another multitasking request is running. On the other side I could change the handler interface in such a way that it provides a pointer to the list of required attributes and I can provide a flag for the handler whether this is the first handler call for the request independently of the attribute at all. So the handler has the chance to collect all attributes in one go and cache them which ensures a certain integrity also.

I don’t see any acceptable possibility to ensure excluding of race activities in case that ExtdAttrib is bypassed. In net filing systems there can be always parallel activities by other users. I don’t see a proper solution to handle this. However the handler can collect requests at modification. Perhaps I should provide a similar concept for fetching too (see above).

What restrictions are there on the calling program, and on the programs providing the attributes? For example, if the svn libraries were used to provide attributes, they would need to be called as a new application. If the calling application was in a critical routine at the time, e.g. a wimp redraw loop, it may not be able to cope with a new application starting.

Ok this is a problem but I am not seeing a chance to solve this due to OS limitations. Perhaps anyone has a good idea.

I am demanding that handlers must be able to perform their operation in multitasking and non multitasking way. When a program is fetching attributes inside a critical routine it can (not must!) request the attributes in a non multitasking way and the handler has to cope with this. This complicates the handler of course but this feature is necessary to grant usage under non Wimp situations. In the example described above the application has the possibility to collect and store the data on its side before performing the redraw.

Note that the interface has been designed in such a way that the control is always on the side of the caller. An application should arrange the attributes in such a way that all attributes which can be obtained quickly (non multitasking) shall be performed first. At first multitasking attribute the fetch will return to the application. This can continue now displaying the attributes already provided and for the others it can display a dummy and swap out. Then the handler with the pending request will be started, fetch the required information and swap out. Then the application will do a requery when new information is available and redisplay the result. The result is designed in such a way that the information can be extended continuously.

Why do interrupts need to be disabled when writing the pollword? aving the module poking into the applications workspace at an arbitrary time later seems a bit dangerous, what happens if the calling program has exited/died and the memory no longer exists?

Ok a simple read and modify ARM operation like for semaphores will do the job also and quicker.

Two situations:

1. The calling program provides the pollword. Then it should cancel all pending queries. If it crashes without doing so indeed there is a danger. I provided this way because a program may perform other activities which are also multitasking and are also requiring a pollword. For an application can cope only with one pollword at time it can tell ExtdAttrib the pollword to use.

2. ExtdAttrib provides the pollword. Then the pollword is a part of the handle. In case that the program crashes the handle will stay open (which might cause some locks at handlers in case of collected modifications). The pollword itself is located inside a solid maintained memory.

How does this cope if more than one handler can provide attributes for a particular file?

I have updated the description on the WIKI in accordance to my latest version which explains my actual handler concept (I am working offline on the completion). It is really not perfect but perhaps someone is having better ideas.

Thomas

I have put a new version of the specification on the WIKI. It is still not finished. Especally the part regarding calling the handlers will change.

I started the implementation of the module a while ago. However I am very busy in the moment so I am making only slow progress. I think that around 50 percent of the programming is done now. I am exspecting the first working version end of September/beginning of October.

Thomas

We’re looking forward to it. :)