h2. RISC OS Filing System Update This page describes plans for changes to update the RISC OS filing system stack. h3. Goals * Support large files (64 bit offsets). * Support foreign format discs (e.g. FAT32 USB drives) larger than 2GB. * Support a modern disc format to use as a native filesystem, that will handle large discs more efficiently. * Provide a standard method of partitioning discs. * Provide support for inserting additional block driver layers to implement things like RAID. * Implement file and directory cacheing. * Provide an interface for storing arbitrary metadata with files on filing systems that support it. * Add support for symlinks. * Simplify interfaces (for new code) wherever possible. * Retain backwards compatibility for userspace applications. * Retain backwards compatibility for old filing systems. * Provide a unified formatter which can write any supported formats to any hardware. * Provide a standard way of creating filer iconbar icons. * Some support for nonblocking/async IO? h3. Plan The following is an initial plan on how the changes can be split into smaller steps that can be implemented and tested individually. Some of the phases could be done in a different order, or in parallel. h4. Phase 1 * Design new userspace APIs * Trivially implement them, mapping to the old ones. * Provide a patch to do the same for older OSes. * Ideally work with ROL so the same can be added to RO6. h4. Phase 2 * Design new interface between fileswitch and filesystems. * Implement fileswitch changes (possibly as a separate module, so it can be in C). * Non-disc filing systems (e.g. Sunfish) could then be ported to the new interface. h4. Phase 3 * Design new block driver interface (common to discs and image filing systems). * Port some filing systems (e.g. Filecore, DOSFS) to the new interface, and test as an imagefs. h4. Phase 4 * Port or rewrite some hardware drivers to implement the new block interface. * Can now test with a normal ADFS/Filecore hard disc using the new stack. h4. Phase 5 * Implement disc formatter. * Implement disc partitioning. * Implement file cacheing. h4. Phase 6 * Port a new filing system (e.g. ext3, ZFS) to RISC OS, using the new interfaces. h3. Userspace API changes h4. General concepts h5. Filetypes Load and Exec addresses are only supported on old filing systems, new ones can only set the datestamp and the filetype separately. Therefore the new SWIs do not provide any method of reading or writing the load/exec addresses. Existing SWIs may give an error if used on a new filing system and the load/exec addresses written do not represent a filetype and datestamp. All new interfaces use 32bits for the filetype, and it is expected that new filesystems will store the filetype as 32bits where possible. h5. Symbolic links New filing systems may support symbolic links. Symlinks in any part of a pathname other than the leaf will always be dereferenced by fileswitch (and an error will be returned if they are broken links). New calls have a flag to indicate if the symlink should be dereferenced or not when it is the leafname. Old calls that do not have this flag will always dereference (with the exception of OS_File 6, which will always delete the link and not the target). In new calls that return an object type (OS_File 26 and OS_GBPB 13) a value of 4 is used to signify a symlink. This value will only be returned if symlinks are not being dereferenced by the call, or if the leaf is a broken symlink). h5. Extended attributes New filing systems may support extended attributes, to allow arbitrary metadata to be stored with any file or directory. An extended attribute consists of a name (null terminated, can contain any characters except null), and associated data (which can be any length, and may contain binary data). To avoid clashes with attribute names, they are organised into namespaces. A name should consist of its namespace, followed by a colon, followed by the attribute name. Applications can use a unique namespace for any data they store, to avoid overwriting data from another application. Should namespaces be allocated by the RISC OS allocations service? Two namespaces are currently reserved; riscos for any use by the OS, and user for any use the end user wishes. Any attribute name without a namespace (i.e. without any colon in the name) is reserved for the end user. Filing systems may use extended attributes to store information in if needed, and this may be exposed to the user. For example the filetype may be stored as an extended attribute named riscos:filetype. h5. File Locking OS_Find gains an extra flag on file opening to allow files to be opened by more than one process at once. OS_Args gains calls to lock byte ranges of files. h5. Changes to SWIs pre.. Unless otherwise stated, all new calls behave the same as the ones they are replacing. OS_Args Use of OS_Args 0 with r1 != 0 is deprecated. OS_Args 10 should be used instead. Use of OS_Args 1 is deprecated. OS_Args 11 should be used instead. Use of OS_Args 2 is deprecated. OS_Args 12 should be used instead. Use of OS_Args 3 is deprecated. OS_Args 13 should be used instead. Use of OS_Args 4 is deprecated. Filing systems should be able to sort this out by themselves. Use of OS_Args 6 is deprecated. Filing systems should be able to sort this out by themselves. OS_Args 10 Read 64bit file pointer => r0 = 10 r1 = file handle (must not be 0) r0 = 11 r1 = file handle r2 = lower 32bits of file ptr r3 = upper 32bits of file ptr OS_Args 12 Read 64bit file extent => r0 = 12 r1 = file handle r0 = 13 r1 = file handle r2 = lower 32bits of file extent r3 = upper 32bits of file extent OS_Args 14 Lock file => r0 = 14 r1 = file handle r2 = flags r3 = start lo r4 = start hi r5 = length lo r6 = length hi r0 = 26 r1 = filename r2 = flags r0 = 28 r1 = filename r2 = flags r0 = 29 r1 = filename r2 = flags r3 = attr name (null terminated) r4 = data r5 = data length Flags are: bit 0 don't resolve symlinks Will create if doesn't exist, or overwrite is does. OS_File 30 Read extended attribute => r0 = 30 r1 = filename r2 = flags r3 = attr name r4 = data buffer r5 = data length buffer length if overflowed) Flags are: bit 0 don't resolve symlinks OS_File 31 Delete extended attribute => r0 = 31 r1 = filename r2 = flags r3 = attr name r0 = 13 r1 = directory name (not wildcarded) r2 = ptr to buffer r3 = num objs to read r4 = start r5 = buffer length r6 = flags C flag corrupted Buffer 0 file type 4 file length lo 8 file length hi 12 attributes 16 object type 20 time/date 25 name (0 terminated) Flags are: bit 0 don't resolve symlinks Should we just require the user to call GBPB 9 and then OS_File to get info? With cacheing it wouldn't be such a penalty as now. OS_GBPB 14 write bytes to given 64bit pointer => r0 = 14 r1 = file handle r2 = pointer to buffer r3 = bytes to write r4 = file position lo r5 = file position hi r6 = flags? r0 = 14 r1 = file handle r2 = pointer to buffer r3 = bytes to write r4 = file position lo r5 = file position hi r6 = flags p.