Resolver 0.73 is broken

Resolver 0.73 is broken. There are two symptoms of failure: one is that entries in the Hosts file are correctly looked up, but must be matched case sensitively; the other is that lookups in the Hosts file can fail completely. Note that, if something is being looked up that is in the Hosts file and also in an external resolver (e.g. one’s router), it will be looked up case insensitively in the external resolver, and may mislead one into thinking that Resolver 0.73 is working.

See the thread in Community Support, “Is Resolver 0.73 broken?”.

Resolver 0.73 is broken.

Resolver 0.73 change log

I may have misunderstood, but it looks like the real code is a binary blob, for which we can’t see the sources. © Ant Ltd. They’ve been updated, hence the header going from 0.72 to 0.73.

I may have misunderstood, but it looks like the real code is a binary blob, for which we can’t see the sources.

Well, that code from Ant has been broken in a variety of ways for over 20 years, one more peculiarity is just a small bit of decoration on the icing on the cake.

Yes, but it has just gone from working to not working. Pretty clearly.

The important questions are about who will fix it, and when. All the open source parts of RISC OS are available for diagnosis, fixing and review by anyone. Not the binary blobs, though. So who kicks off a fix? Do we need to notify ROOL so that they can? We can’t rely on them reading here AFAIAA.

The important questions are about who will fix it, and when.

Hopefully the same person who broke it in the first place, as they should know what they did and so can undo it!

Another important question being why was this apparent change not mentioned in the change log? Because neither “Drop COMPAT_INET4 support” nor “built with cc 5.89” should have changed the behaviour (unless something is very wrong).

Because neither “Drop COMPAT_INET4 support” nor “built with cc 5.89” should have changed the behaviour (unless something is very wrong).

Unless the previous behaviour is INET4 compatibility and anything now sending queries to Resolver is expected to deal with mixed case response to a mixed case query.

Which was the buggy version of the compiler?

expected to deal with mixed case response to a mixed case query.

Case shouldn’t matter.
https://datatracker.ietf.org/doc/html/rfc4343

The important questions are about who will fix it, and when

Looking at the bug tracker I can’t see any entry for this issue so I am assuming that none has been made but it is expected that magically discussing it here should get it fixed?

Out of interest has a test been made with a totally clean install of both the ROM/Firmware elements as well as the latest hard disc images just to further help narrow anything down that may or may not be interacting with Resolver 0.73?

Normally I used to be in the habit of testing the new ROM’s and Hard Disc images each day they became live but have fallen out of that recent and on this one may not have seen it anyway as i am testing the ROD stack as well.

Case shouldn’t matter.
https://datatracker.ietf.org/doc/html/rfc4343

My reading of that is slightly different – I think it says a name in all uppercase must match an uppercase or lowercase version, and similarly an all lowercase name must match either all uppercase or all lowercase versions.

I don’t see anything that allows mixed case names to be matched against either all uppercase or all lowercase.

The case-insensitive comparison is done on a per-byte (octet) basis.

And that’s immaterial to whether Resolver appears to be using Hosts at all, even when the same case name is given as that in the file.

My reading of that is slightly different

Section 3:

I’ve highlighted the pertinent parts. It’s not saying a string that IS ALL UPPERCASE or is all lowercase must match, it’s saying a value within the string as reading it character by character must match.
Therefore A = a, B = b…
…so AB = ab = Ab = aB.

In other words, case insensitive (at least for regular ASCII).

But, yeah, there appear to be “other issues” too.

In other words, case insensitive (at least for regular ASCII).

Indeed. However A-umlaut might be expected to match a-umlaut, and maybe also A or a, which from the above would not happen.

When I first encountered DNS (1980’s?) I seem to recall the character set was limited to uppercase and lowercase ASCII, numbers and one of minus or underscore – I can’t remember which. Things change…

Not as clear-cut as “it’s all case-insensitive”

As Alan notes:

When I first encountered DNS (1980’s?) I seem to recall the character set was limited to uppercase and lowercase ASCII, numbers and one of minus or underscore – I can’t remember which.

and then the document Rick quoted a part of, but omitted other bits.

2. Case Insensitivity of DNS Labels

DNS was specified in the era of [ASCII]. DNS names were expected to
look like most host names or Internet email address right halves (the
part after the at-sign, “@”) or to be numeric, as in the in-addr.arpa
part of the DNS name space. For example,

foo.example.net.
aol.com.
www.gnu.ai.mit.edu.
or 69.2.0.192.in-addr.arpa.

Case-varied alternatives to the above [RFC3092] would be DNS names
like

Foo.ExamplE.net.
AOL.COM.
WWW.gnu.AI.mit.EDU.
or 69.2.0.192.in-ADDR.ARPA.

However, the individual octets of which DNS names consist are not
limited to valid ASCII character codes. They are 8-bit bytes, and
all values are allowed. Many applications, however, interpret them
as ASCII characters.

and for extra fun:

4. Case on Input and Output

While ASCII label comparisons are case insensitive, [STD13] says case
MUST be preserved on output and preserved when convenient on input
However, this means less than it would appear, since the preservation
of case on output is NOT required when output is optimized by the use
of indirect labels, as explained below.

Which looks rather like the Resolver is supposed to preserve case, but what other bits do with that preserved case string in comparisons is up to the other bits – i.e. as I said earlier the update in the Resolver is highlighting a case sensitivity problem elsewhere.

Looking at the bug tracker I can’t see any entry for this issue so I am assuming that none has been made but it is expected that magically discussing it here should get it fixed?

I’ve filed ticket 576.

I’ve filed ticket 576

Hopefully that will get even more attention.

Also given this statement:

i.e. as I said earlier the update in the Resolver is highlighting a case sensitivity problem elsewhere.

Would it not be prudent as I said earlier to just do a new sdcard build using the latest ROM and Hard Disc image and see if the issue is still there?

It looks just broken to me.

Doing a strings on the ResolverSA shows that it’s got strings that are garbage in place of the correct hosts file:

eg:

ResolverConfig
Resources:$.Resources.Resolver.Messages
BadSWI
%.*s.%.*s
/etc/hosts
Inet$HostName
/etc/hosts
Inet$SearchDomains
Inet$LocalDomain

If I compare this to the Resolver module I have lying around (an old Select one, but it’ll be approximately the same…

gethostname
Inet$HostName
makehost
_gethtent
_gethtbyname
InetDBase:hosts
_inetaddr
_inetraddr
_gethtbyaddr
_getfwbyname
_getfwbyaddr
freeway_init
freeway_final
_getnbbyname
getanswer
res_query
res_search
res_querydomain
%.*s.%.*s
res_launch
hostclear
cacheclear
cachemerge
cachecheckpurge
cachefind
InetDBase:hosts
cacheadd
cacheinit
cachefini

However, we can be clearer with this and actually link the module and use it. Using the build script:

#!/bin/bash
set -e
riscos-kitten Resources.UK.Messages Resources.UK.CmdHelp > resjoined
riscos-resgen -apcs 3/32/nonreent/fp/swst Resources aof/resources resjoined Resources.Resolver.Messages
riscos-link -verbose -rmf aof/ResolverSA aof/resources C:o.stubsG -o Resolver,ffa

And then if I turn some FS debug on and load the module, issuing a ResolverConfig and printing the SHA of the repo I was testing…

charles@laputa ~/external/ResolverBlob (master)> pyrodev --common --command 'rmload Resolver' --command gos --boot-debug osfile,osfind
Read catalogue info with path
  Filename = 'Resolver'
  Path = None
  DirEntry = <DirectoryEntryNative('Resolver'/'Resolver,ffa', 1, &fffffa5a, &52c27f0f, 20300, &33)>
Load file
  Filename = 'Resolver'
  Path = '@.'
Read catalogue info with path
  Filename = '/etc/hosts'
  Path = None
  DirEntry = <DirectoryEntry('/etc/hosts', 0, &00000000, &00000000, 0, &0)>
Read catalogue info with path
  Filename = '/etc/hosts'
  Path = None
  DirEntry = <DirectoryEntry('/etc/hosts', 0, &00000000, &00000000, 0, &0)>
Supervisor

*help resolver
==> Help on keyword 'Resolver' (Module)
Module is: Resolver        0.73 (16 May 2022) (Acorn) © ANT Ltd, 1997

Commands provided:
  ResolverConfig
*resolverconfig
Read catalogue info with path
  Filename = '/etc/hosts'
  Path = None
  DirEntry = <DirectoryEntry('/etc/hosts', 0, &00000000, &00000000, 0, &0)>
*git rev-parse HEAD
1c33b5a299d5dba129bbcc18e118f3ab42428dfb

I don’t think I really need to go any further.

The module as is presently on that branch is just stuffed.

I don’t quite see how you’re getting to the point of finding case insensitiveness being an issue as it shouldn’t even be able to read the hosts file, but that’s where I’d start looking for what the problem might be.

I may be a broken record, but… automated testing?

I may be a broken record, but… automated testing?

The sound of common sense again is heard :-)

I may be a broken record, but… automated testing?

The sound of common sense again is heard :-)

FTFY.

FTFY.

Any testing is better than none but you have to have a plan or a set of criteria on which you are going to test and the expected outcomes you have if you want to do it correctly.

Are you testing new features?
are you testing existing capabilities to see if the changes have affected any thing? etc

Just getting something out and then thinking because you have no feedback it is Ok isn’t the best option.

As I said I used to test the new daily ROM’s and hard disc components as they came out but have got out of that habit for one reason or another and perhaps that has replicated a bit across all regular users.

Compounding this specific issue is that I, like I guess many users, just do DHCP and I also have the ROD stack I am testing/have tested.

Anyway hopefully now it is highlighted thanks to Dave then this can be fixed.

To be clear, my own tests that check the RISC OS Pyromaniac resolver’s Hosts file implementation checked whether the addresses could be looked up in the configured file, whether the entries worked for named hosts and for aliased hosts, and that comments were ignored, plus a few of the different ways that the SWI calls can be used.

I did not have any tests for the behaviour being correct when different character cases were used, nor for checking whether the InetDbase:Hosts file was being used properly. It does have tests for these now. And those tests did show up that the matching didn’t work properly if the case differed. I now have 25% more tests because I missed those tests in the testing that I have got, and I won’t break that again.

Compounding this specific issue is that I, like I guess many users, just do DHCP

1. Makes sense because there are very few occasions where there is no DHCP source, and where it exists you can rely on it to produce repeated results
2. Downside – the alternate may not get tested and when it doesn’t work the logical assumption is that the user didn’t configure things correctly¹ because the alternate (DHCP) is producing working setups

¹ I have literally lost count of the occasions where suppliers did not follow the instruction to “set to DHCP” and were surprised when what they did failed to work because they missed a critical aspect or mistyped a value or two. Or their system didn’t properly register in DNS and parts of our config relied on the FQDN rather than a fixed IP.