Territory_ConvertTimeFormats

Probably immaterial really. You’d only need to set up the field once, not a million times…
…and if part of the bitfield does need to change in the loop, calculate the part that doesn’t outside of the loop and merge in the changes.

Don’t forget, BASIC also supports this notation: %1000001000000011

Yes, but there needs to be a conversion from local time to UTC (or vice versa) for the R4 difference to be applied (as stated in the source). So, R3 = &80301 or R3 = &80103 should work.

I’ve been mulling it over whilst making a paella  for lunch and concluded that both would be needed.

You have to do a conversion to make the subtraction or addition part work, so, to make the calculation sensible, you have to convert back to how you started to get the time right. This you’d do with or without bit 19 0f R3 set, but with zero in R4 or  timezone within territory.

As I’m not sure what value to put in that last if I wanted to, I’d be tempted to use it with bit 19 still set and zero in R4.

Does that make sense?

But you’re wanting the ordinals to compose a yyyymmdd filename? So just convert from 5-byte to that, with appropriate offset in R4.

I don’t think it converts to string…?

You’d really need to do it in three steps:

• Use ConvertTimeFormats to change the five byte to local time (see note). For UK, there’s a flag to get it to work out whether or not to apply DST adjustment automatically.
• Use ConvertTimeFormats to apply the desired offset (bit 19, offset in R4).
• Use ConvertDateAndTime to make a date string out of it.

Note: the documentation says OS_Word returns a five byte UTC value, not local time, however throwing the five byte read directly at OS_ConvertStandardDateAndTime gave the correct time. Does that SWI automatically translate, or was OS_Word returning local or something?
My system should be set to UTC with an offset of +2h (but I’m up in the garden in a hammock so…).

It seems that this call might be missing something. You can let it work out whether or not to adjust for DST, but it seems that one must supply the appropriate timezone in R4. This is a bit nonsense if you’re not using the default timezone.
(I’m using a custom build of the UK module with the following time zones: GMT, BST, CET, CEST; that’s how I keep discovering ways of seeing how half-assed the timezone handling is ;) )

Might be less trouble to get the current offset using ReadCurrentTimeZone and bung it in manually? One could bump the value by any other desired offset at the same time and thus extinctify two avians with one haematite.

So did someone actually report that you had to change timezone in order for the code to do the additional offset arithmetic? Brain-dead if true, and needs a BIG note in the documentation.

The following, according to the posts made this afternoon, should work. It doesn’t

DIM  localtime% 5
DIM result% 128, time2% 5
?localtime%=3
SYS "OS_Word",14,localtime%
rsize%=128
format$="%CE%YR%MN%DY:%24:%MI:%SE"
SYS "Territory_ConvertTimeFormats",-1,localtime%,time2%,&80103
*REPORTDUMP (localtime%) (5)
*REPORTDUMP (time2%) (5)
SYS "Territory_ConvertDateAndTime",-1,time2%,result%,rsize%,format$ TO ,rsize%
?rsize%=13
*REPORT ($result%)

The reportdumps show that the two 5-byte buffers contain identical values, even though it was told to converty from GMT to UTC. It fails if asked to convert the other way too. Does it need R0 to be something other than -1 for anything to happen?

[Edit] The above works if I set bit 18 instead of bit 16. The documentation for bit 18 says "Bit 18: Calculate bit 16 automatically ".
WTF does that mean?

And this fails with “unsupported subreason”. ???

DIM  localtime% 5
DIM result% 128, time2% 5
?localtime%=3
SYS "OS_Word",14,localtime%
rsize%=128
format$="%CE%YR%MN%DY:%24:%MI:%SE"
offset%=3*24*60*60*100: REM 3 DAYS
flags%=3 + (1<<8) + (1<<18) + (1<<19)
SYS "Territory_ConvertTimeFormats",-1,localtime%,time2%,flags%,offset%
*REPORTDUMP (localtime%) (5)
*REPORTDUMP (time2%) (5)
SYS "Territory_ConvertDateAndTime",-1,time2%,result%,rsize%,format$ TO ,rsize%
?rsize%=13
*REPORT ($result%)

Without the (1<<19) and the offset it correctly adds 1 hour to the current time.

And with the <1<<19) but without ther (1<<18) it correctly adds 3 days and 1 hour. So bits 18 and 19 seem to be mutually exclusive.

and I can now confirm that if the two timezones are the same, it doesn’t do any arithmetic, merely copies the buffer from input to output unchanged.

Cmon it’s not that hard to subtract two 5 byte time values! Especially when the subtrahend is a small positive value ;-)

For one who was initially sceptical about the whole idea, or even its possibility, you seem to have developed rather an interest – to the extent that it may be you who is further usefully adding to the Wiki page that Rick initially updated.

I look forward to reading about your further discoveries as I take my RISC OS machine off-line for the great ceiling reskim and just follow proceedings for a day or two.

An exciting voyage of discovery, whereas my original problem could easily be solved by temporarily using a US time zone where it isn’t even teatime yet whilst creating my date-based filename on turning-in. But that would be cheating.

All rather more fun than I imagined!

merely copies the buffer from input to output unchanged.

If by “timezone” you mean local or UTC for both input and output, that’s entirely as expected, the wiki documentation says as much.

So bits 18 and 19 seem to be mutually exclusive.

Makes sense. On one hand you’re saying “work out if there’s a DST offset and apply it” and on the other you’re saying “apply this offset”.

Bit 18: Calculate bit 16 automatically ". WTF does that mean?

Exactly what it says. Bit 16 means to convert as if daylight savings was in use. Always.
Bit 18 makes it a bit more useful by meaning “figure out if daylight savings would apply to the given date and if it would, do so”.

So did someone actually report that you had to change timezone in order for the code

I think you’re maybe getting a little mixed up here.
I mentioned that if your system timezone was not the first one in the list (given that R4 gives the timezone when it’s not centiseconds), then you appear to need to specify it manually. Which, yeah, is a bit dumb.

This wouldn’t ordinarily affect people with simple territories (UK, France, Germany…) as they’ll all only have the one timezone.
However I want my system to be “British” but understanding European time, so I made my own module with the CET/CEST timezones added. And every so often run into a problem regarding the lousy implementation of such. A few years back I submitted a fix for CLib using the wrong SWI to obtain the offset, meaning that other timezones simply couldn’t ever have worked for any C program until… when was it, 2015 or so? It’s quite clear that RISC OS never made much impact in the US or it might have been noticed. Either that or everybody just used UK settings and bodged the offset manually. ;)

So going back to the OP, the requirement was to take the local time, subtract a few hours and convert it into a formatted date/time string. The reason for trying to use the Territory_ConvertTimeFormats function is that it seems to be the only system-supplied function capable of doing the subtraction.

It becomes apparent from the experiments, if not from the rather limited documentation, that the subtraction only occurs as a side effect of a conversion between different timezones, which introduces an additional unwanted offset.

So we’re left with the need to do an unsigned 5-byte subtraction in BASIC, which is possible, but not exactly straightforward. After the subtraction there’s a simple Territory function to do the formatting.

There are a number of ways to address the subtraction in BASIC, all a little clumsy. The one I prefer is to work byte by byte from LSB to LSB+3, using integers subtraction, and checking for a borrow (bit 8 non-zero). The borrow is then included in the next byte up. Finally subtract the borrow from the LSB. This for subtracting an integer (4-byte) value from the buffer holding a 5-byte value. A similar approach works for 5-byte – 5-byte.

DEF PROCsubtimes(result%,input%,offset%)
LOCAL I%,in%,O%,R%,C%
  FOR I%=0TO3
    in%=input%?I%
    O%=( offset% >> (I%*8) ) AND &FF
    PROCsubbytes(R%,in%,O%,C%)
    result%?I%=R%
  NEXT
  result%?4 = (input%?4)
  IF C%>0 THEN result%?4 = (result%?4) - 1
ENDPROC
DEF PROCsubbytes(RETURN R%,in%,O%,RETURN C%)
  R%=in%-O%-C%
  IF R%<0 THEN C%=1 ELSE C%=0
  R%=R%AND&FF
ENDPROC

Who’da thought something so simple would be so complicated? ;)

The method I posted earlier using Ordinals works for adjusting centiseconds, seconds, minutes, hours and (a small number of) days. It does not work for months or greater.

But as Stuart has pointed out, adding or subtracting a time offset is simple:

REM Read current time
DIMtime%4:?time%=3:SYS"OS_Word",14,time%

REM An offset of -3 hours in cs
O%=-3*60*60*100

REM Add to the time
O%+=?time%:?time%=O%:time%!1+=O%>>8

REM Display
SYS"Territory_ConvertStandardDateAndTime",-1,time%,&8300,256TOA$:PRINTA$

RISC OS is in dire need of a Calendar manipulation API. There was the DateUtils module that used to be distributed with !Allocate, but that was rubbish – even so, I have made a new 32b version of it. But it’s poo as is. There’s an opportunity to add some useful functionality though.

The method I posted earlier using Ordinals works for adjusting centiseconds, seconds, minutes, hours and (a small number of) days. It does not work for months or greater.

But does even hours work in the case where subtracting a number of hours also changes the day, and potentially month and year as well.

e.g. 1-Jan-2000:01:00:00 – 3 hours, which SHOULD result in 31-Dec-1999:22:00:00. Which is exactly the case the OP was wanting to use it for.

REM Add to the time
O%=?time%:?time%=O%:time%!1=O%>>8

That works in this case because the offset is small enough that the top byte of the integer is zero, so there’s no possibility of a carry into the 5th byte of the time.

RISC OS is in dire need of a Calendar manipulation API.

Agreed. This discussion has shown up a number of deficiencies in the Territory module for doing this. e.g. The maximum year for ordinals conversion is very different (by 200 years) from the maximum year for dateandtime conversions. The documentation of the format function is minimal, confusing, and in some aspects incomplete/wrong.

DateAndTime produces a year in the 1900’s if the top two bytes of the 5-byte time are ffff, It works for ffef, feff and efff.

But does even hours work in the case where subtracting a number of hours also changes the day, and potentially month and year as well.

Yes, it does. But big proviso this is observed behaviour not defined functionality, so I thought I’d do it manually in centiseconds.

That works in this case

Which is the case I was trying to help with. Those few instructions work for offsets of ±248 days, which seems appropriate for an attempt to change a few hours.

The documentation…in some aspects incomplete/wrong.

This is true of all the documentation, sadly.

ffff

The maximum is &FFFFA81BF which is 06:57:57.75 03-Jun-2248.

So what API would we like to see? I’d like to resurrect DateUtils, which already has _WeekNo, _AddToDate, _TodaysDate and _CurrentTime SWIs though not because they’re terribly useful.

Add/subtract ordinals? “DateUtils_Add”,timeptr,cs,se,mi,hr,dy,mo,yr? A Convert- style adder whose inputs and output can all be different formats (eg this TimeDate plus these Ordinals output as a DateString)?

Distressingly, I have written a module that does various timedate manipulations, but I can’t find it right now.

That works in this case because the offset is small enough that the top byte of the integer is zero, so there’s no possibility of a carry into the 5th byte of the time.

You’ve misread the code, and misquoted it.
Look closely and you’ll see that it works.

Rick wrote:

Who’da thought something so simple would be so complicated? ;)

Times and dates are an absolute nightmare, it’s only with Python and its libraries I’ve had confidence to go anywhere near this sort of thing. With the RISC OS APIs, it’s a non starter.

can’t find it

D’oh. It’s my own version of DateUtils, which I’d misplaced:

The last version of DateUtils I’m aware of is dated 1993! This total rewrite contains bugfixed implementations of the existing SWIs (e.g. AddToDate can actually cope with more than 28 days!) but that’s for backwards compatibility, they’re not great APIs.

The new SWIs do datetime conversions to various packed and integer representations, including _Now which returns number of days and number of centiseconds separately, which is often convenient. But it’d need the reverse functions too.

It also contains the *RMDate command, which I’d totally forgotten about.

Times and dates are an absolute nightmare, it’s only with Python and its libraries I’ve had confidence to go anywhere near this sort of thing.

It took the Java guys up until Java 8 to implement something sensible. Nice to see that reinvent-the-wheel is alive and kicking everywhere.

Anyone wanting to implement something sensible for RISC OS could do worse than looking at Joda-Time for inspiration.

Thanks Steffen. The Chronologies are a good way of getting away from the Territory mess (and the fact that most of us only have the one). The Period is essentially Ordinals, DateTime is effectively 5-byte time but in ms. Much of the formatting and field access can already be done by Territory calls (not that they wouldn’t benefit from a nicer wrapper – eg DateUtils_WeekNo can already be got via %WK).

So adding a set of Ordinals to a datetime clearly would be a useful function. The “display this date in Ethiopic form” is perhaps less important but would be nice.

One of the best bits though is the date string parsing, which has always been massively restricted in RO. I had to go to quite some lengths in DateUtils to grok Module dates, as they’re not defined to be machine readable so could conceivably be in any language (it recognises 89 different month names because of that).

It becomes apparent from the experiments, if not from the rather limited documentation, that the subtraction only occurs as a side effect of a conversion between different timezones, which introduces an additional unwanted offset.

Uh… no… Not really.

We are “leveraging” a system call to convert between one timezone and another because, well, because the time handling is crap and the system uses a five byte value that doesn’t exactly lend itself to simple operations like “add five hours”.

The call normally translates between UTC (what OS_Word hands us) and whatever the local timezone is, however by supplying our own offset, we can make this SWI instead add or subtract arbitrary amounts of time, however we might need to consider also mucking around with the local timezone so that it isn’t in UTC if that’s not our own timezone.

It’s difficult, but let’s explain.

OS_Word → gives us a five byte value IN UTC.

OS_ConvertStandardDateAndTime → converts the UTC five byte value into a string representation in the defined timezone.

Territory_ConverTimeFormats → can either: 1, read in UTC or local time and output local time or UTC (in any combination); or 2, read in “a time value” and output “a time value” with a custom offset applied. In the second case we’re essentially tricking the conversion by saying one is UTC and the other is local. It doesn’t check, it doesn’t care, it just does what is asked of it.

So, it’s easy enough to read the time using OS_Word and to paste in some offset or other using Territory_ConverTimeFormats, but it’s worth remembering that it’s an offset from UTC.
What you do next determines whether or not any additional work is needed. If you want to convert to a string using Territory, you’ll probably need to look at how to make it local time. You can either do this by converting it again, or by simply telling the string SWI that it’s UTC and you want it in local…
…alternatively, OS_ConvertDateAndTime will just assume the input is UTC (as that’s what OS_Word would have given) and would output in local time.

If you provide the right flags to Territory_ConverTimeFormats, the applied time offset will be what you tell it and it will not matter one bit what the system timezone is or whether or not DST in in force.

Here is proof. First, some code.

REM >_dsttest

DIM fivebyte% 5
DIM buf% 32

PRINT "TZ+0 NO DST test"'"================"
*Configure NoDST
*Configure TineZone +0
fivebyte%?0 = 3
SYS "OS_Word", 14, fivebyte%
SYS "Territory_ConvertStandardDateAndTime", -1, fivebyte%, buf%, 32+(3<<30),,0 TO ,t%
t%?0 = 13
PRINT "Raw system time = "+$buf%
PRINT "Five byte time  = ";
PROCprintfive

SYS "Territory_ConvertTimeFormats", -1, fivebyte%, fivebyte%, 3 + (1<<8) + (1 << 19), (100*60*60*12)
PRINT "Added 12 hours  = ";
PROCprintfive


PRINT '"TZ+1 DST test"'"============="
*Configure DST
*Configure TimeZone +1
fivebyte%?0 = 3
SYS "OS_Word", 14, fivebyte%
SYS "Territory_ConvertStandardDateAndTime", -1, fivebyte%, buf%, 32+(3<<30),,0 TO ,t%
t%?0 = 13
PRINT "Raw system time = "+$buf%
PRINT "Five byte time  = ";
PROCprintfive

SYS "Territory_ConvertTimeFormats", -1, fivebyte%, fivebyte%, 3 + (1<<8) + (1 << 19), (100*60*60*12)
PRINT "Added 12 hours  = ";
PROCprintfive

END

:

DEFPROCprintfive
  FOR l% = 0 TO 4
    PRINT ;fivebyte%?l%;
    IF l% < 4 THEN PRINT ", ";
  NEXT
  PRINT
ENDPROC

When run, this is what is output:

*_dsttest
TZ+0 NO DST test
================
Raw system time = 16:27:49 17-Apr-2023
Five byte time  = 47, 138, 154, 150, 90
Added 12 hours  = 47, 117, 220, 150, 90

TZ+1 DST test
=============
Raw system time = 16:27:49 17-Apr-2023
Five byte time  = 50, 138, 154, 150, 90
Added 12 hours  = 50, 117, 220, 150, 90
*

You can clearly see that in both cases, the five byte values are identical ¹ regardless of whether we’re in DST or not, or in TimeZone +0 or +1. There should have been a two hour discrepancy if unintended things were going on. But since we’ve told the SWI to only apply our offset, then it has done that, to make a five byte time that is twelve hours later (than UTC, not local time).

The reason I’m using Territory’s ConvertStandardDateAndTime instead of the OS one is because I can override the automatic conversion to print out the raw (UTC) system time.
If you used the OS’s version, it will have fiddled the output to be local time. If that is what you observed, it’s OS_Convert(Standard)DateAndTime doing that, not the Territory module. You can clearly see the five byte outputs are the same and not two hours different.

¹ The first byte will be different, it’s ticking centiseconds here; it’s the rest we look at…

One of the best bits though is the date string parsing,

Something sorely missing from RISC OS is the simple ability to output and parse various standard and commonplace date strings that can be found on-line. Like those used in cookies ¹, 8601 (old and new) ², and various RFC formats (for usenet headers, email headers, and so on) ³.

And probably some others I’ve forgotten.

¹ Monday, 17-Apr-23 18:55:00 CET (also as RFC850; the difficulty here is in understanding the timezone acronym)

² 2023-04-17 18:55:00 (old, space between)
2023-04-17T18:55:00 (new, ‘T’ between)

³ Mon, 17 Apr 2023 18:55:00 +0100 (RFC822/2822/RSS)
Mon, 17 Apr 23 18:55:00 +0100 (RFC1036)
2023-04-17T18:54:12.123456+0100 (RFC3339)

Unlike the other fields, the timezone %TZ is entirely at the whim of the Territory, which is awkward. In my notes I have proposed %MZ=month-as-number(0-11), %H2=hour-as-24(01-24) and %H1=hour-as-12(00-11), together with %IN, %IO, %I3, %IC and %IY for various Islamic calendar fields. One could imagine similar for other Chronologies.

But numeric fields aside, the usual question persists: Whither the Territories.

And I don’t know whether Muh, Saf, RaA, RaT, JuA, JuT, Raj, Shb, Ram, Sha, DaQ and DaH would make sense as 3-chr months for %I3 in the absence of Unicode. And Turkish month names are even more awkward.

Might I suggest we don’t need to be too concerned about Arabic dates until such time as FontManager can make a passable attempt to rendering the language?