Who likes manga?

And now a whinge…

This worked fine:

   sp = getservbyname("http", "tcp");
   server.sin_port = sp->s_port;

While this produced utter rubbish:

   sp = getservbyname("https", "tcp");
   server.sin_port = sp->s_port;

Why? Is https not a known protocol?

It took me quite a long time of tinkering with the SSL code to see why it was sitting forever pending connection until I tried looking to see what port the thing was connecting to, and discovering the rubbish that was being set by getservbyname(). Grrrr….

I “fixed” it by hardwiring in the usual port:

   server.sin_port = 47873; // Port 443 in network order

And now a whinge…

While this produced utter rubbish:

   sp = getservbyname("https", "tcp");
   server.sin_port = sp->s_port;

Why? Is https not a known protocol?

No, https is not a known protocol, the list of protocols is in !Internet.files.Services, this is basically as the risc os tcp ip stack is 20 plus years old.

You are not checking the return value of getservbyname(), it returns NULL on error, so sp→s_port is a zero page dereference.

I “fixed” it by hardwiring in the usual port:

   server.sin_port = 47873; // Port 443 in network order

You can use the library function to convert order for you

   server.sin_port = htons(443);

You may have this if you have NetFetch or the like (look in !System.500.Modules.Network for “Secure32”)

Usual place for Secure32 is !Hermes.Resources so a copy may need to be moved to !System.

Edit: Hermes has both Secure32 & SecureG. Any preference…?

Nice work Rick!

I am still very much incapacitated (ended up in hospital again last week) s no email ATM. Sorry to anyone awaiting replies. Am needing assistance for (literally) everything ATM.

Feel free to include the module with !Manga if you wish.

You probably only need SecureG (1.04 I think).

The other versions in NF exist for various reasons. For example, WebsterXL preferred fetching with 1.03 for some reason so that program defaults to the older 32bit version. Not an issue today though! Also, 1.01 was included because OS3.70 machines had issues with 1.04/StubsG(universal) version. I forget what/why. But RO 3.70 was broken in various ways so is fairly safe to disregard.

You are not checking the return value of getservbyname(),

Sorry. I should hang by head in shame, but I’m still blown away that https is unknown. I mean, I know that RISC OS internet stuff is old, but jeez……

htons(443);

Thanks. I knew there was a macro to do that, but didn’t feel like wading through loads of header files to find it. I’ll change the code. ;-)

Feel free to include the module with !Manga if you wish.

Thanks. That makes things simpler.

Version 0.19 is now available, with a few small enhancements on the fetching side of things…

Beta track auto-update, or pop over to my private server for the archive → http://heyrick.ddns.net/manga/
And, yes, the updates that used to be around 68K are now going to be around 435K because the SSL module is enormous.

Here’s what’s new:

0.19  2018/03/06

      Changed code to use htons() for setting the port number. Thanks to
      Peter Howkins for this suggestion.
      
      Altered SSL fetcher method to use a time-based delay instead of just
      polling the wimp a few hundred times (told you v0.18 was a quick hack!).
      The thing is, without any delay, the data transfers are corrupted (and
      may abort early). With a delay, they are okay. The default delay is now
      3cs in between each chunk, and the chunks can be up to 4KiB to account
      for the delay.
      As the appropriate delay *may* depend upon processor speed (2cs+ works
      fine on my Pi2, 1cs messes up, so I've settled on 3cs to give some
      margin); there is a system variable "Manga$NCFetch" which you can set:
        *Set Manga$NCFetch 5
      to give a 5cs delay. Hopefully 3cs will work okay for everybody...
      If you find your manga index truncated (missing some/most entries) or
      your images not updating or updating with rubbish, then this problem is
      biting you. Add some more delay! [and delete "mrindex" in the cache]
      On my Pi2 - a 3cs delay gives me ~60K/sec
                  a 2cs delay gives me ~85K/sec
                  a 1cs delay gives me ~177K/sec and sometimes corrupts
                  a 0cs delay gives me ~250K/sec and always fails
      Given it takes several seconds for ChangeFSI to scale the images, you
      might prefer 2-3cs and good data? I know I do...
      
      Since the downloader no longer has a slidey bar for manga fetches (due
      to the lack of a "Length:" header - is this the fault of CloudFlare?),
      it would report how much has been fetched; which is typically 330K or
      so for the index, 11K-20K for chapter info, and 60K-160K (ish) for the
      images.
      The thing is, a rapidly changing number is pretty useless as anything
      other than an indication that something is happening. It might as well
      have been WOPR trying to crack the nuclear launch codes...
      So this has been changed to report the size in bytes (up to 4K), and
      then Kbytes beyond. It uses the OS routine, so Kbytes here has a small
      'k' for some obscure reason (given 'M' and 'G' are both capitals).
      
      Now performs an extra check to ensure the SSL module is present.
      
      Altered !Run to look for "SecureG" in !System, reverting to the one
      within !Manga if that doesn't load. [older machines without the
      BootCommands module will always have the one loaded from !Manga].
      
      Manga is now supplied with the SecureSockets module by kind permission
      of Andrew Rawnsley (R-Comp). This makes it a lot easier for you to get
      Manga working after the HTTPS change. Just start it up and hit the
      ground running. :-)

Please let me know if you experience any problems.

If not, I’ll consider it good in a couple of days and push it to !Store.

This version built at 10pm. Probably would have been a couple of hours earlier… but I was… testing with “Koe no Katachi”. Yes, testing. That’s it. That’s what it was. I wasn’t reading manga instead of coding, the reading was totally part of a rigorous development verification procedure¹. :-)

Manga link → Koe no Katachi

¹ There, 20 syllables of bull to confound and astound so you fail to realise I was, indeed, reading manga instead of coding…but, then, that’s what the program is for, right? Right?

I’ve pushed out v0.20. SSL seems more stable and doesn’t mess up, so I’ve disabled the throttling. Now fetches stuff at ~250K/sec and gets it all. ;-)

Updates from the regular place…

Home server is off for the moment – going out and risk of thunderstorms. It’ll be back on this evening, weather permitting…

Much faster.

Just a stupid idea: a prefetcher. I mean, a way to automatically/silently put the next page in cache. And so the slide bar would only appear if you read too fast :)

I’ve just updated from 0.08 to 0.20 and now I’m getting an AODT immediately on launch (before it appears on the icon bar). Any suggestions on things to try?

I’m using a homebuilt 7-Mar-2018 ROM on a Pi 3.

Any suggestions on things to try?

Is SecureSockets Pi3 safe?!? It is…..old. Can you use Debugger to narrow down the location of the error?

Try setting:

set debugger$dumpoptions -file annotated
set debugger$annotatedfile $.whatever

Then make Manga crash, then look at the $.whatever file.

I dropped SecureG onto PatchSWP but it didn’t find anything wrong with it.

The debug log is here, but I have no idea how to read it :(
Hopefully it’s obvious to you; don’t waste too much time on my account.

Weird. The annotated dump doesn’t appear to give a disassembly around the exception point. Is this an oversight? It could be something as simple as an errant instruction that newer processors fault?

What I can tell you is that it isn’t fixable:

R15 = 207478d0 = Module area +7478d0 = SHA1_Update +24c R14_svc = 00000000 is system workspace

It’s in SecureSockets, in the SHA1_Update function.

As a last ditch, do you have access to the “Secure32” incarnation of SecureSockets?

[hmm, this implies Hermes etc might not work on a Pi3 either?]

Sorry; I didn’t see your reply until now.

I don’t have Secure32, but I do have some other things to try when I have some time. I transplanted the SD card into my Pi 1 and had the same problem. My next step will be to do a fresh install of RC15 and then copy my current ROM, current Boot etc. across until I figure out what’s making it fail.

OK, testing done.

Fresh install of RC15 on Pi 1: Works.
ROM updated to latest nightly: Crashes.

No other changes.

Fresh install of RC15 on Pi 1: Works.
ROM updated to latest nightly: Crashes.

That’s close to 12 months of changes, have you tried any ROM between those?

No other changes.

Latest “HardDisc” install too?

I’ve just updated from 0.08 to 0.20 and now I’m getting an AODT immediately on launch (before it appears on the icon bar).

Manga 0.20 starts up without error and goes on to display Manga pages on the RPi3B+ running OS5.23 (18-Mar-18), including BCMVideo 0.49. It was a clean install, there was no previous version. SecureSockets 1.04 is on the case

That’s close to 12 months of changes, have you tried any ROM between those?

No; I don’t have any others available. [Edit: Actually, I have one from October but I don’t have time to try it right now; I’ll update this post in due course.]

Manga 0.20 starts up without error and goes on to display Manga pages on the RPi3B+ running OS5.23 (18-Mar-18), including BCMVideo 0.49. It was a clean install, there was no previous version.

Now that’s bizarre! Mine was also with BCMVideo 0.49.

Now that’s bizarre!

Some more bizarre.

The test has been repeated on the RPi3 running the bcm048 custom ROM and that was all good too.

bizarre

ZeroPain module?

Pain.

Time: Tue Mar 20 08:14:18 2018
Location: Offset 0005a240 in module SecureSockets
Current Wimp task: Manga
Last app to start: Filer_Action

R0  = 000001e0 R1  = 00000001 R2  = 20350710 R3  = 0000000a
R4  = 00000000 R5  = 0000000f R6  = 00000014 R7  = 20350710
R8  = 00004000 R9  = 00000000 R10 = fa20021c R11 = fa207afc
R12 = 00000000 R13 = fa207ad0 R14 = 00000000 R15 = 203ec99c
DFAR = 00000000  Mode SVC32  Flags nZCv if   PSR = 60000113

203ec954 : e5d41000 : LDRB    R1,[R4,#0]
203ec958 : e1800401 : ORR     R0,R0,R1,LSL #8
203ec95c : e7820103 : STR     R0,[R2,R3,LSL #2]
203ec960 : ea000054 : B       &203ECAB8
203ec964 : e1a0512e : MOV     R5,R14,LSR #2
203ec968 : e20ee003 : AND     R14,R14,#3
203ec96c : e35c0000 : CMP     R12,#0
203ec970 : 17920103 : LDRNE   R0,[R2,R3,LSL #2]
203ec974 : 0a000006 : BEQ     &203EC994
203ec978 : e35c0001 : CMP     R12,#1
203ec97c : 0a000007 : BEQ     &203EC9A0
203ec980 : e35c0002 : CMP     R12,#2
203ec984 : 0a000007 : BEQ     &203EC9A8
203ec988 : e35c0003 : CMP     R12,#3
203ec98c : 1a000009 : BNE     &203EC9B8
203ec990 : ea000006 : B       &203EC9B0
203ec994 * e5d40000 * LDRB    R0,[R4,#0]
203ec998 : e1a04001 : MOV     R4,R1
203ec99c : e1a00c00 : MOV     R0,R0,LSL #24
203ec9a0 : e4d41001 : LDRB    R1,[R4],#1
203ec9a4 : e1800801 : ORR     R0,R0,R1,LSL #16
203ec9a8 : e4d41001 : LDRB    R1,[R4],#1
203ec9ac : e1800401 : ORR     R0,R0,R1,LSL #8
203ec9b0 : e4d41001 : LDRB    R1,[R4],#1
203ec9b4 : e1810000 : ORR     R0,R1,R0
203ec9b8 : e2831001 : ADD     R1,R3,#1
203ec9bc : e7820103 : STR     R0,[R2,R3,LSL #2]
203ec9c0 : e1510005 : CMP     R1,R5
203ec9c4 : 2a00000d : BCS     &203ECA00
203ec9c8 : e5d40000 : LDRB    R0,[R4,#0]
203ec9cc : e2843001 : ADD     R3,R4,#1
203ec9d0 : e1a00c00 : MOV     R0,R0,LSL #24

Ah, it’s just pain…

This is mentioned in the help file:

BTW, if you are using a high vector version of RISC OS, SecureSockets creates
zero pain issues. [ https://www.riscosopen.org/forum/forums/5/topics/3955 ]

As far as I’m aware, nobody has yet built the module from source using DDE28 or GCC…

I completely forgot about ZeroPain, but isn’t it supposed to drop a log in $? It certainly used to…

Edit: Or does RC15 not include the module anymore? Admittedly I haven’t checked.

But it sounds like that could be the problem! I’ll give it a go after work. Thanks for the pointers :)

Edit 2: Oh right, RC15 was low-vector so it wouldn’t have included the module. That would certainly explain it!

I can’t say for certain because I <cough> built myself a low vector version </cough>, but I think the options now are to:

• Bomb spectacularly if the application so much as mentions the word zero
• Whinge endlessly
• Just shut up and fake it

I made myself low vector builds because I didn’t want endless logs (or bomb outs) for applications I can’t fix, and just got in the habit of doing that. My next build will probably be ZPP if there’s a “shut up and fake it” option…

This isn’t to mean that I think all is well. I’m quite aware that pain relates to program faults (in the above example, LDRB R0,[R4,#0], R4 is zero so we could be looking at the use of an uninitialised pointer…), however knowing that something needs to be fixed and actually fixing it are two very different things. I’m off the opinion that if something (with a flaw) more or less worked for the last decade, it can carry on working now… And thanks but don’t spam my flash media with endless logs. They’re of no use if I’m not trying to fix something! ;-)