Enforced HTTPS - coming to a browser near you

I will let this linked page do the explaining, but in short:

Major browser developers are starting the push to eliminate mixed HTTP / HTTPS use in a page, which would see certain sub-elements of a page not loading. NB. Gravatar use in these pages is already using HTTPS, so people who like that are still OK.

Also noted in the reference page is the HTTPS First
“Next up, we are going to default all addresses from the URL bar to prefer HTTPS, with a fallback to HTTP if the site does not load securely. This feature is already available in Firefox Nightly.”

Recent checks showed some RO related websites using HTTP but sitting on an incompatibly configured HTTPS host, this forthcoming HTTPS First policy would break access to those sites.

I’ve seen work problems relating to this clamp down on mixed use (HTTP in HTTPS pages) when using MS Edge or Chrome. Fortunately, we have an experienced team so no showstoppers so far, but even there the “solution” at present is tweaking to force legacy mode use. Not a long term strategy.

So, if you have web pages, and you want them to be accessible to MS Edge/Chrome/Firefox users, it’s time to review them and look at configuring them appropriately.

Two of the affected websites are www.riscos.com and www.virtualacorn.co.uk (HTTPS fails with a certificate error; HTTP access still works) and I understand Aaron is unwell so may not be in a position to correct the site certificates.

Both sites have been excluded from the Wayback Machine, so if any of you have been meaning to consult any of the documentation on those websites (for example, there are HTML versions of books that may be difficult to find elsewhere) I suggest you do so while you still have the chance.

www.virtualacorn.co.uk (HTTPS fails with a certificate error; HTTP access still works)

To be frank, that has two failings, the second is that the certificate has expired.

The first is that even before it expired it would have prompted an error on the first visit since the certificate is a “self-signed” one, i.e. it’s actually generated on the server and is of a type that should never be used on the open net.

If that was corrected, with something like a Let’s Encrypt certificate, it will work for the moment, but if you check through there is a mix of https and http link content so it’s a good example of one case that will likely fail once the browser developers tighten things.

if you check through there is a mix of https and http link content

Ah. I should take a look at my website for this issue, I presume.

I guess it is coming to the time when this (http://heyrick.eu/blog/index.php?diary=20240611 – note the obvious box at the top) will no longer be optional.

https://www.riscos.com illustrates what can happen if a site is set up with just HTTP – nothing earthshaking about that at the time many of these were done – and then time moves on and we have a HTTPS First push.

Error code: SSL_ERROR_BAD_CERT_DOMAIN
Looks like this one is maybe¹ sitting on a secure-secure.co.uk server with a hostname of the format web123.secure-secure.co.uk
and where the (free, no cost) wildcard certificate matches that: *.secure-secure.co.uk
but the CNAME DNS record is only useful to send connections to the right IP, and it can’t address the root problem that the certificate is for *.secure-secure.co.uk not *.riscos.com

Again, using a Lets Encrypt certificate would fix that, and not break the bank.

¹ Saying maybe as the Reverse lookup of the IP gives web17.extendcp.co.uk

My own website’s pages are all https; hostinger does that for me, and all my links to my own site are internal. But I link to other sites via http from many of my pages. Do I need to check that the linked sites are already https and change my links to suit, or will people’s browsers automatically try https anyway?

Again, using a Lets Encrypt certificate would fix that, and not break the bank.

Some hosting companies like to milk their customers and don’t allow installation of Lets Encrypt for SSL.

Ah. I should take a look at my website for this issue, I presume.

That was the intention of highlighting this. I think you’re OK Clive, minor tidy up, e.g the link to the Green Party manifesto pages is HTTP (and they divert to HTTPS) same for East Cambs Green party and wikipedia.

I’d really like it if no RO related sites became inaccessible due to avoidable incompatibilities with major browsers, hence the reminder/heads up.

Slightly ironically, the links Stuart singled out also bring some assistance as this page on VirtualAcorn has a link to a link checker utility XENUlink that will trawl your website and examine the links.

Some hosting companies like to milk their customers and don’t allow installation of Lets Encrypt for SSL.

Why do I feel Druck stirring from slumber to mutter about justice for “Squirrel abusers” or some such?

I think you’re OK Clive, minor tidy up, e.g the…

Crikey – was that you who crawled ~1700 pages of my site at great speed around 10:00 this morning, Steve? Many thanks, anyway!

Edit…

And then, about 12:00, all the same ones (apart from one…) and perhaps a handful that didn’t get crawled 1st time? There’s actually a lot that whoever it was hasn’t crawled… :)

Crikey – was that you who crawled ~1700 pages of my site at great speed around 10:00 this morning, Steve?

XENU says there are 3304 URLs in there :)
There are quite a few external links (134) that launch http, to a list of 68 hosts.
Of those 68 they, mostly, can be converted to HTTPS and you’re done.

Then there are the ones that are plain broken or need a little work:

infocenter.arm.com	     redirects	<a href="https://developer.arm.com/documentation/">https://developer.arm.com/documentation/</a>
policy.greenparty.org.uk	redirects	<a href="https://greenparty.org.uk/about/our-manifesto/">https://greenparty.org.uk/about/our-manifesto/</a>
replay.web.archive.org	redirects	<a href="https://web.archive.org/">https://web.archive.org/</a>
teawithstrangers.org	Cert error	Error code: SEC_ERROR_EXPIRED_CERTIFICATE 
the-stable.lancs.ac.uk	new site	<a href="https://www.lancaster.ac.uk/lec/">https://www.lancaster.ac.uk/lec/</a> 
tumble.conskeptical.net	broken link	
<a href="http://www.aidansemmens.co.uk">www.aidansemmens.co.uk</a>	http only	
<a href="http://www.bedandbreakfast.am">www.bedandbreakfast.am</a>	broken link	
<a href="http://www.esru.strath.ac.uk">www.esru.strath.ac.uk</a>	Cert error	Error code: SEC_ERROR_EXPIRED_CERTIFICATE 
<a href="http://www.highview-power.com">www.highview-power.com</a>	redirects	<a href="https://highviewpower.com/">https://highviewpower.com/</a>
<a href="http://www.islandconnections.eu">www.islandconnections.eu</a>	parked	parked domain
<a href="http://www.madhusree.com">www.madhusree.com</a>	http only	
<a href="http://www.nda.gov.uk">www.nda.gov.uk</a>	http	use: <a href="https://www.gov.uk/government/organisations/nuclear-decommissioning-authority">https://www.gov.uk/government/organisations/nuclear-decommissioning-authority</a>
<a href="http://www.ocrwm.doe.gov">www.ocrwm.doe.gov</a>	broken link	
<a href="http://www.wolfbane.com">www.wolfbane.com</a>	http only	
<a href="http://www.xin-publishing.uk">www.xin-publishing.uk</a>	http	does redirect to: <a href="https://xin-publishing.uk/">https://xin-publishing.uk/</a>
 

That is brilliant, Steve – you’re a hero! I wouldn’t have had a clue what to do, other than work my way through the whole thing checking every link & editing as necessary…

Over the years, I’ve had a handful of emails mentioning broken links, that I’ve fixed or removed as appropriate – but I know from the logs that lots of people either don’t try them anyway, or don’t bother to report them.

I’d do a search in the files for the http:// instances and convert to https://

You have more work on the ones in the Textile buggered list of 16 above. If they are friends/relatives you can nudge them to update, others – maybe delete or make a note that time and tide…

Anyway, I thought that having dug a hole, I might as well put some effort into pointing to the easy route out.

I’d do a search in the files for the http:// instances and convert to https://

Yup, that much I can manage! It’s knowing which ones that would break that would have been a problem for me.

I might as well put some effort into pointing to the easy route out.

You’ve certainly done that! Many, many thanks. As I say, you’re a hero.

What did you use to do this? I ought to cast an eye over the dung pile that passes for my site…

What did you use to do this?

From earlier:

Slightly ironically, the links Stuart singled out also bring some assistance as this page on VirtualAcorn has a link to a link checker utility XENUlink that will trawl your website and examine the links.

Needs a PC.

Then just do a find in the window that opens looking for http:// or export as txt, drop into a spreadsheet app, sort by host, remove duplicate hosts, check the response of all hosts to being called with a browser (Firefox) set to HTTPS only, recheck any that fail with a more forgiving setting on the same browser.

Normal fayre for an IT support person from my viewpoint¹. I probably should dig around for a few better tools.

¹ Which might not be that normal if you consider the reactions of my younger cow-orkers to set the normal standard and how close I match (or not)

Needs a PC.

Mac won’t do?

Mac won’t do?

There’s mention of it working under a particular utility in the web page I referenced.

Rick:
330 http references in a set of 5074 links, many broken Eurovision links – bonus ;)

hhtp://www.crtvg.es/
error code: 404 (not found), linked from page(s):
https://heyrick.eu/ricksworld/digibox/older/chlist.html

Works on https (or http (rather than hhtp) :)

I’ve found a W3C link checker https://validator.w3.org/checklink , but it’s kind of slow as rather than listing links, it tries to follow and validate them. It’d take all weekend to parse my blog, never mind anything else, so I stopped it.

330 http references in a set of 5074 links

I guess there’s some work to do. Are those links on the /blog part? That’s my main concentration these days.

Speaking of which, does it even sanely handle the /blog part? I’ve found some things consider it to be a single page (/blog/index.php), as they don’t understand that the parameter (?diary=YYYYMMDD) changes the content. I guess this is why some things (like wikis) use quiet redirects, so the user might be /blog/articles/YYYYMMDD which is a fake address that is changed to the internal reference; so it works and has a unique URI for each page.

many broken Eurovision links

On-site or off? If off, NMFP. ;)

bonus ;)

Now, now, we all have dumb things we enjoy.

As for the link checking, I can’t help but think that this is the sort of thing that I ought to be able to do quickly with a bit of PHP. Just iterate through anything that is .html or .php in the site (recursive search), then load each document in turn and search for the href element in a and img tags. If it begins “http:” then list it (ignore everything else so we aren’t bogged down with internal links and other guff). This won’t validate the links exist, but should point out which ones may need to be upgraded.
I’ll have a rummage this weekend if I’m not out taming the vegetation. Was supposed to be doing that now, but rain…

Now, now, we all have dumb things we enjoy.

:)

Was supposed to be doing that now, but rain…

Cats – out. One hides under the patio table, another comes back later: dry, nicely brushed and not immediately requiring food.

On-site or off? If off, NMFP. ;)

https://heyrick.eu/eurovision/2005/scorecard2005.html#Israel not found
https://heyrick.eu/eurovision/2005/scorecard2005.html#Moldova not found
etc

As for the link checking, I can’t help but think that this is the sort of thing that I ought to be able to do quickly with a bit of PHP. Just iterate through anything that is .html or .php in the site (recursive search), then load each document in turn and search for the href element in a and img tags. If it begins “http:” then list it (ignore everything else so we aren’t bogged down with internal links and other guff).

If you’re doing something like that, you could likely have it look for the mixed content items (as detailed in the notification page) in the various pages

There’s mention of it working under a particular utility in the web page I referenced.

8~) Since you’d very kindly done it for me already, I hadn’t bothered to follow the link – have now 8~) & might find it useful in the future. Haven’t yet checked whether the utility works on the M1 Mac (my main machine) – but I do also have an ancient Intel Macbook Pro (hand-me-down from our son) that I use when away from the desk (the Command Centre, according to the offspring…)

https://heyrick.eu/eurovision/2005/scorecard2005.html#Moldova not found

Well, now, that’s because some twat write crappy links like this:

<a name="#Portugal"></a>

I ought to find the idiot that did that code and give them a piece of my mind…