Further thinking on Internationalisation

As some may recall I have sometimes written about internationalisation support and why Territories are a chocolate teapot.

I have proposed using IETF language codes and a user-configurable language preference string to allow i18n to be done properly. To recap, Territories conflate too many unrelated things to be terribly useful. User interface language should not be dictated by timezone, for example.

One of the challenges is supporting regional language variations where much of the localised content is common, but some things have to change – eg tyre/tire, receção/recepção, dins/dans-le – and duplicating large resource files for the sake of a single word or cultural reference is problematic. It is also prone to rot – it’s easy to add a new token to a Messages file or icon to a Sprite file and forget to propagate that to all those other copies.

So I’m trying a new approach here:

These are three “English” resource directories for an application. Instead of Territory/Country numbers they’re identified by IETF Language Code. A system-wide Language Preference String is checked for (and if not present, an appropriate one for the configured Territory is used), and its contents checked against the available localisations in preference order, with (optional) simplification. So “fr-CA,en-CA” would match fr-CA if available, then fr, then resort to Canadian English, and finally The Queen’s.

But rather than duplicating all that common content, I’m using SoftLink files to duplicate other translations – the en-CA dir contains links to the en files, for example (this is mostly superfluous and just for this example, but see later). Now this is how SoftLink files have worked where supported – they just contain the target path (and in this scenario they’re always relative paths). However, I’ve extended the syntax as illustrated by the en-US example.

When the SoftLink contains more than one path, the targeted files are merged. The en-US “Messages2” file contains only a single line – the one that mentions “color”. The “Messages” SoftLink starts with that file, but then merges in the rest of the en Messages. The result is that the US-localised line will be used from the en-US file, and all the rest comes from the shared en.

The system can merge Messages, Sprites and Template files. CC’s ABI files do not contain localised text so probably don’t need merging, but I haven’t handled Resource files yet (because I don’t use the Toolbox).

I’m tentatively using Sys$Language as the Language Preference String, but I’ve requested an allocation and we’ll see what happens. Incidentally, whereas en-GB-WLS would simplify (if not present) to en-GB and then to en, putting a language code in brackets prevents simplification, enabling a Canadian auto-mechanic to choose this:

*Set Sys$Language [en-CA],[en-US],en

Which would try really hard to ensure they get “tire” and “sidewalk” (as it does not simplify en-CA to en before trying en-US).

are a chocolate teapot.

You and me both. That I have to build my own ROM image with a customised UK territory in order to have English, CET/C’EST, and € as a currency is…a deplorable state of affairs really. I’ll stop here as I’m sure everybody is “oh God, don’t get him started on that”. :-)

Territories are a chocolate teapot.

Possibly in orbit between the orbits of Earth and Mars?

One of the most hilarious things about Territory is

One million… whatever it is you use where you are

Possibly in orbit

I think it was Russell who said “Most people would rather die than think, and many do”.

Sounds credible, certainly.

As for the million, I’m in Cambodia.

seems a good idea to me, perhaps another option to be currency and thousands seperater?

Possibly in orbit between the orbits of Earth and Mars?

No, that one is God’s teapot. Because clearly God is British and can’t be without a spot of tea.

“Most people would rather die than think, and many do”.

I’m sure Brexit will help them achieve this goal.

Because clearly God is British Yorkshire and can’t be without a spot of tea.

Absolutely.

Kevin thought

perhaps another option to be currency and thousands separator

My first reaction was to write:

Territory already provides that and that’s perfectly sensible – a million (scalar) is a million is a million, regardless of grouping and decimal character. But as Dr Evil was trying to point out, a million dollars is not a million Euros is not a million riel. The currency aspect of Territory is a bit of “CIA Worldbook” thinking, and as relevant to internationalisation as which side of the road a country drives on.

But I suppose you may mean that a user could wish to override the grouping that would be expected in their ‘territory’ and choose some other form? I’m all for choice, so I take that on board. That’s something else to consider for Territory mark II, which I will probably have to come to.

There are many bits of the Territory API that are deeply silly, such as every Territory requiring the English name of every other Territory in its Messages, which is not only geometrically unscalable and pragmatically impossible but also totally nonsensical – if you want to change to Turkish it’s probably because you can read Turkish, so “Türkçe” (Turkce) may be a better name than “Turkish” regardless of what Territory you’re currently in (subject to Alphabet/Script/Keyboard etc – GUI FTW).

That’s how you do language selection

That’s how you do language selection

Aside from saying “choose a region” and then listing things that aren’t regions ;)

There are many bits of the Territory API that are deeply silly, such as every Territory requiring the English name of every other Territory in its Messages, which is not only geometrically unscalable and pragmatically impossible but also totally nonsensical

I totally agree, and this ground has been over several times.

I once produced a “Region” system for modifying soft-loaded territory modules by patching, but it was limited in what it could do and was thoroughly unprofessional. ;-)

Then I did a proof-of-concept “Universal Territory” module which interacts with the Territory Manager as normal, but can be changed on the fly to use shared resources of languages, currencies, timezones and all the other things that a user might actually want to specify. Some of what you suggest here is reminiscent of that, and territory/region lists in the local language was there.

I do not propose to do more with it, but put forward the idea as a way to proceed without breaking the current Territory API.

There are many bits of the Territory API that are deeply silly, such as every Territory requiring the English name of every other Territory in its Messages, which is not only geometrically unscalable and pragmatically impossible but also totally nonsensical – if you want to change to Turkish it’s probably because you can read Turkish, so “Türkçe” (Turkce) may be a better name than “Turkish” regardless of what Territory you’re currently in (subject to Alphabet/Script/Keyboard etc – GUI FTW).

That brings to mind one item that really bugs me – if you on a system operating in a language/character set other than your native one the drop down showing the alternatives should always display your native one in your native way. (Many instances of doctors changing the character set for a printer to Chinese or whatever so then changing back is a case of learning that language or reset to defaults (which tend to be English USA (near enough to a real language to work with)

Totally agree with your example drop down.

Would any of you believe this is work? Testing response times on random secure sites :)

and was thoroughly unprofessional. ;-)

You were limited in what it could do due to how it did it.
What you turned out had more flexibility than any territory stuff Acorn ever made…

without breaking the current Territory API.

I strongly disagree. The current Territory setup is broken in so many ways. Something better really needs to be created (and please shoot whoever had Toolbox Resource languages be given numbers instead of language codes). It should be created with no respect to the previous borked Territory mess.
And for Territory itself? A shim that translates the better API into something that looks and feels like the old API.

I mean, we’re looking at several things that are only tenuously linked:

• Default practice for a country (currency code, how numbers are formatted)
• Language (which is definitely not tied to a country)
• Timezone
• Miscellaneous (international dialing code, how phone numbers are shown, etc)

The primary here is very obviously language. All of the rest should be freely configurable. So rather than UK, there’s “en-gb” (or maybe just “en”?).

And, yes, a choice of language should obviously be displayed in that language, not what it looks like in English.

Steve said

I did a proof-of-concept “Universal Territory” module

Did you write that up or publish it?

On the subject of module date strings, I’ve been further down the “date” rabbit hole and now know more about Chinese dates and historical Vietnamese Eras than I wanted to, and I regret that.

I also regret that Territory’s ConvertTimeStringToOrdinals thinks Flooberry hoohah, 000008 novichok 02019doobry doobry is a valid date, but not (08 Nov 2019).

Oh yes. A looong time ago there was an Acorn module called DateUtils. It wasn’t very good. I’ve resurrected it to host some additional functionality. Has anyone ever seen a more recent version than 1.02?

My favourite language selection anti-pattern is not found in Territory, but on many modern websites – you can select your language, and they are somehow sorted, and it is alphabetically, but not by the name that is shown, but by their english name…i.e. you find “Deutschland” between other languages starting with “G” in english.

I’m using SoftLink files to duplicate other translations

“SoftLink” as in Martin Würthner’s SoftLinks? Did that ever get 32bitted?

“SoftLink” as in Martin Würthner’s SoftLinks? Did that ever get 32bitted?

Maybe, maybe not

There’s probably a storage area many leagues down with lots of treasure…

I’m using SoftLink files

To be specific, &FDC – as used by NFS to indicate a broken Unix symlink. I am not relying on anyone else’s code or imagefs behaviour. It just has to be a filetype that isn’t FFF, FF9, FEC or FAE really. I now have merging of Resource files too (uggh, horrid format). You raise a good point though… I should use a filetype that definitely hasn’t got filing system potentially messing with it.

Oh it recurses too of course, up to 8 levels deep.

you find “Deutschland” between other languages starting with “G” in english

LOL. Notional prize for anyone who can work out why I put the months in this order (excerpt):

		=	"GEN",1		; 4 IT
		=	"JUN",6
		=	"JÚN",6		; 16 IS
		=	"AGO",8		; 4 IT, 5 ES, 8 PO
		=	"ELO",8		; 12 FI
		=	"SEP",9
		=	"MAR",0		; ***NOTE*** =3 unless 12:FI when =11

Did you write that up or publish it? (a proof-of-concept “Universal Territory” module)

No. It was far too far from a finished product, and I stopped before tackling all the time SWIs, but the timezone treatement was much more flexible than at present.

It did show that a single client module, I called Region, could reproduce what individual territory modules do. There was an aim to share data rather than have multiple copies of it, for instance the substitution tables.

It would have required Language Manager and Alphabet Manager modules to be useable in general.

Yeah, I have an Alphabet module that does all the stuff that Territory pretends is territory-specific. Incidentally, I did notice in passing that all the A4com territory modules used the same uppercase/lowercase/etc tables regardless of Alphabet… so it doesn’t look like anyone ever cared about them. Regardless, I’ve done the whole lot.

I’ve put in an allocation for the “Language” module already.

used the same uppercase/lowercase/etc tables regardless of Alphabet

Erk! How does that work ¹, given that Latin2 is all full of Slavic stuff?

¹ Rhetorical question – it’s about as broken as all the rest of Acorn’s attempts at coping with internationalisation – I mean this was just dumb brokenness.

How does that work

You get the Latin1 tables no matter what. It’s only France, Turkey and Esperanto (I think) that were affected, as everything else used Latin1 anyway. Never seen a Territory module using Latin2.

Latin9 gets its Šš & Žž affected, and the Ÿÿ. Latin3 though was badly affected – Ħħ, Ĥĥ, İı, Şş, Ğğ, Ĵĵ and Żż are all ignored. How did all those Esperanto users cope? Ho ve!

To the best of my knowledge, Latin2, Latin4 and Latin5 have never been used for a Territory. And Arabic, Cyrillic, Hebrew and Greek seem to be entirely theoretical after the BBC Master. Anyone have a Territory for one of them?

How did all those Esperanto users cope?

They probably both decided to use English :)

To the best of my knowledge, Latin2, Latin4 and Latin5 have never been used for a Territory. And Arabic, Cyrillic, Hebrew and Greek seem to be entirely theoretical after the BBC Master. Anyone have a Territory for one of them?

I know this isn’t one of the listed ones, but I’ve seen a Maori territory (number 37) which presumably used Latin7 since that seems to be the only one with the required characters.

Correction: Latin6 also contains the characters.