Printing

Matthew: the problem is the line that we have to provide in R3. It has to begin with “ipp://” or “ipps://”, even though the protocol is HTTP. Yet if I do that, the error is “No fetcher service”.

Other than that, your method is perfect :-)

The protocol in R3 tells it which fetcher to use. If you use http:// instead of ipp:// then it’ll use the HTTP fetcher and will hopefully work. I don’t have a way to actually test/confirm this myself though :)

The protocol in R3 tells it which fetcher to use. If you use http:// instead of ipp:// then it’ll use the HTTP fetcher and will hopefully work.

Correct, except you’ll also need to have “:631” to give the port, as in my outline above. I’ll try and knock up some BASIC to prove it later this week.

Getting the URL_Fetcher module to respond to ipp: and ipps: would be useful if we were expecting a web browser to need to follow links, but what you’re needing to do is to tell the printing system to send stuff to an IPP printer. The user might still type ipp:// in the configuration box (if they have to type anything, rather than find the printer automatically) but the printing system could translate that into http or https fetches behind the scene, couldn’t it?

If you give the URL as http, it will access the printer’s home page.

You need http protocol to an ipp URL.

While testing ipptool all these worked without having to specify the port number,

https://<printer ip>/ipp/print
http://<printer ip>/ipp/print
ipp://<printer ip>/ipp/print

I get a tls error when using ipps. It tries to connect to port 631

ipps://<printer ip>/ipp/print

but using ipps://<printer ip>:443/ipp/print worked.

If you give the URL as http, it will access the printer’s home page.

What is “it” in this situation?

What is “it” in this situation?

The URL module’s URL_GetURL SWI. Accessing the printer’s web pages tells me unequivocally that its URL for IPP purposes begins with either ipp:// or ipps:// but the URL I have to give to URL_GetURL has to begin with http:// or https:// If I use an ipp(s) URL, the call fails immediately with “No fetcher service”.

I must backtrack on what I said earlier. I’ve been banging my head against the IPP brick wall some more today, and have finally got some communication with my son’s printer using the URL module and an https URL. However, all the requests I have tried return a status of 0×0400, which is “Bad request”. Unfortunately, as is normal with these things, it gives me no clue as to what’s bad about it. I started with Rick’s request. The printer’s natural language attribute is “en” rather than “en-us”, but correcting that makes no difference. Nor does putting the correct IP address in the printer-uri tag.

What would be nice is to find a magic command that tells me everything that I can ask it – perhaps “requested-attributes” was it, but it doesn’t work for me and this printer.

Oh well, I’ll probably add a few more bruises to my head again tomorrow.

Right, have a go with the BASIC program I’ve just uploaded. The zip file also includes a modified AcornHTTP in plain and debugging versions (you will probably need a RAM disc for the debugging one as I think I compiled it for use without SysLog).

The program is based on Rick’s FindIPP, but is less clever, because the host is fixed at 192.168.1.66 which on my network is our Xerox Phaser 6600. Should be easy for you to alter.

As you will see, I am using an http URL to connect to the printer, using URL_GetURL. I have copied all of Rick’s stuff for building the application/ipp body for the request. I’ve no understanding of it!

Why the need for the modified AcornHTTP module? Well, to start with I was getting 406 Not Acceptable back from the printer, along with some HTML. The headers generated by AcornHTTP are not exactly the same as those produced by Rick’s code, so I went through to see which ones mattered. The differences were (with Rick’s version on the right):

“HTTP/1.0” versus “HTTP/1.1” — didn’t matter
“Host: 192.168.1.66:631” versus “Host: 192.168.1.66” — didn’t matter
“Connection: close” versus nothing — didn’t matter
“Accept: /” versus nothing — didn’t matter
“Accept-Encoding: deflate, gzip” versus nothing — this was the significant one.

After reading up about this header, I modified AcornHTTP to pass

“Accept-Encoding: deflate, gzip, identity”

instead, and that gave me a response. The docs say that “identity” is always assumed, but perhaps it is not for IPP printers! It shouldn’t do any harm to add it, so I will create another merge request soon.

You may find your printer is fussy in other ways, though. If there is a document stipulating exactly how IPP clients should talk to IPP printers, it might give guidance on the HTTP headers that should be used.

I tried with https but the program hung. I’ve no idea whether my printer supports it anyhow.

Note that in my code I am using “http://” in the URL passed in R3 to URL_GetURL, but the printer-uri tag/value that I pass in the binary payload starts “ipp://” just like Rick’s does.

I don’t think the initial http or https makes any difference to the payload, it’s just used by the URL module to know how to direct the rest of the request.

Note that in my code I am using “http://” in the URL passed in R3 to URL_GetURL, but the printer-uri tag/value that I pass in the binary payload starts “ipp://” just like Rick’s does.

Me too. Sorry for not making that clear in my earlier posts.

I don’t have a problem communicating by http or indeed https with the printer, except for one simple aspect: the printer insists on https, and if the communication is by http, the http response is “426 Upgrade Required”. I’m one small step ahead of you in that https works for me.

The problem I’m seeing is at the level of IPP. The responses begin with 0×0101, which is the version number; the next 2 bytes are the status code, which should be 0×0000 for a successful transaction. All the responses I’ve received have a status code of 0×0400, which means “Bad Request”. So my immediate problem will be solved when I can generate a request that the printer likes.

Let’s see if I can reproduce a response here (the uncommented 4-char hex values are simply the lengths of the following strings):

[0101] ; Version 1.01
[0400] ; Status: Bad Request
[00000001] ; Request number that this response relates to
[47] ; Get attribute, character set (IIRC)
[0012] attributes-charset
[0005] utf-8
[48] ; Get attribute, language
[001B] attributes-natural-language
[0002] en
[03] ; End of request 

Are you sending something similar to my request?
https://www.riscosopen.org/forum/forums/5/topics/14646?page=5#posts-130521

I’d be inclined to try removing from the &47 (inclusive) to the &44 (exclusive) so you’re only sending the bare minimum, and add things back one by one until it falls over.

Aha! I found a sample Wireshark capture on the Internet. The body is as we expect EXCEPT that there is an extra byte, value 0×01, between the operation-id and the first tag. So I binary edited it in to my binary body file that I’ve been sending, and blow and lehold, I have a much longer response!

Now the problem is that I don’t know what that extra byte is for or what it means. But I’ll find out somehow.

The response status is not actually 0×0000, it’s 0×0001, which is sort of a near miss.

More stuff that I’ve read along the way is that the third attribute MUST be the printer URI; and there should be a fourth attribute, the requester name. I have become Fred Scuttle.

… and now I realise that I must have failed to copy the byte when I was constructing my code, because it’s there in Rick’s example.

is that the third attribute MUST be the printer URI; and there should be a fourth attribute, the requester name.

You can tell this was written by a committee, can’t you?

Only they would insist upon providing the printer’s URI to a command intended to determine the printer’s actual URI… and that a name is provided, which ought to make stuff all difference to the printer.

Glad you’re making progress! I had misunderstood your earlier post when you said you were getting Bad Request and status 0×0400. I was thinking you were talking about an HTTP 400 Bad Request, so I fully hoped that my tweak to the AcornHTTP module would set you on your way when actually you had got further than I had realised.

Looks like your son’s printer copes with

Accept-Encoding: deflate, gzip

Accept-Encoding: deflate, gzip, identity

I take it, therefore, that you are happily using the URL Fetcher module for your IPP and no further changes are needed there?

By the way, AcornHTTP 1.08 is now in GitLab so should be in the nightly builds. It’s mainly bug fixes for cookie handling.

You may be interested to know that AcornHTTP does declare HTTP/1.1 to the server for GET requests, but HTTP/1.0 for all other requests. I will have to find out what the differences really are and whether there is any good reason for this. It doesn’t explain why the API I was using (which used GET) gave me 2MB of uncompressed XML as a response. It must just do that!

gave me 2MB of uncompressed XML as a response

At what point did you examine this? Because if its the final output handed to the client, then this is normal. The use of gzip to compress data is supposed to be transparent and only happening to data “in transit”.

I examined it using the debugging build of AcornHTTP, so I could see what the server was actually sending back, before AcornHTTP had processed it.

Matthew, I’ve a question for you. I’m using the URL module for the IPP transactions. When requesting the printer attributes, there is no way AFAIK to know in advance how big the payload will be. It’s easier if the complete payload goes into a buffer. The question is how to know that the buffer is big enough.

What I’ve found, in my case where of course the URL module is calling the HTTP module, is that the first URL_ReadData that returns with R5 > 0, also returns with R4 = 0, i.e. at that stage no bytes have been read into the buffer. This makes it dead easy to do a realloc() if necessary.

The question is: is that guaranteed to be the case? i.e. that, first time R5 > 0, no bytes have been put into the buffer.

Edited to add: it’s a binary body, so there’s a Content-Length header.

Hello Dave! I’ve been puzzling over the sources for part of the afternoon and I do not fully understand what is going on. It’s interesting that R4=0 the first time that R5>0 is returned. I am not sure why that is, but I don’t think it is the kind of thing you can rely on, as there is nothing in the fetcher specifications to say they should work like that.

I tried testing against my printer, and it does not send a Content-Length header, so R5 stays at -1 throughout, even though the response is binary.

I think, therefore, that you will need to allocate a reasonably sized buffer initially, and use realloc to grow it if you need to. The only way to avoid this is to process what you receive as you go.

For example, when I am receiving XML I pass the buffer to the parser, and the parser tells me how much of it was digested. If there is a bit left over (because a tag was incomplete, for example), I move it up to the front of the buffer, and I pass an offset into the buffer to URL_ReadData next time, so that I fill up what remains of the buffer.

The other thing I noticed when testing my TestIPP BASIC script just now was that the AcornHTTP module seems to have swallowed the whole header, so that all my BASIC program received was the binary body of the response! I am sure this is not supposed to happen, and I am baffled as to what is going on. I think I may have uncovered another bug in the AcornHTTP module.

the AcornHTTP module seems to have swallowed the whole header

This is normal. Most clients are interested in the requested content, not the preamble.

R2 to URL_GetURL is normally 1 to GET a request.
If you want headers as well, then it’s 1 + (2 << 8) to set the appropriate flags.

Note that this isn’t documented under GetURL as it’s an AcornHTTP option (so it’s "method dependent).

I tried testing against my printer, and it does not send a Content-Length header, so R5 stays at -1 throughout, even though the response is binary.

Interesting. I wonder how the receiving end is supposed to tell the difference between end of transmission and a pause?

The other thing I noticed when testing my TestIPP BASIC script just now was that the AcornHTTP module seems to have swallowed the whole header, so that all my BASIC program received was the binary body of the response! I am sure this is not supposed to happen

Funny you should say that. I thought the header should be returned too. I share your surprise, if not your bafflement.

There’s another thing that’s starting to concern me. AFAICS any extra header lines, and the body, need to be contiguous in a buffer. This is OK when just sending small stuff, but it might become rather uncomfortable when using it to send a printout if the printout is big and full of fine detail, as one might get when trying to print out a raw image from a camera, in high resolution, to a large page. Wouldn’t it be nice if we could separate the header and body, and particularly if we could send the body from a file instead of from RAM? We could give a filename or a file handle. That would then mean that the dumper module could write a temporary file to disc, thus removing any practical size limit.

Wouldn’t it be nice if we could separate the header and body,

Yes. I’m surprised this wasn’t how it was from the outset.

and particularly if we could send the body from a file instead of from RAM?

Enormously, especially for the reason that you describe. There’s no need to allocate a massive wodge of memory if it’s possible to just load in 16K chunks at a time, say, and send those according to the uplink speed.

I’ve never actually POSTed a large amount of data. Is it possible to loop for status in a manner akin to reading data? If so, it would be quite useful to know it’s 25% done, 30% done, blah blah.

If anybody is going to be fiddling with the URL fetcher, then I have a relatively simple wishlist item to add:

In the case of a redirection (301, 302…) it would be highly useful to extract the Location pointer from the header and send it as the content body.
Because the process at the moment is:

• Try to fetch something
• Get a 30X returned
• Repeat the exact same fetch, with headers to be returned
• Parse the headers to find the new location
• Fetch from the new location

If URL helped out here, the middle three steps (and an entire fetch) could be eliminated. It’s kind of useful, especially given there are a fair few sites that use a redirect to push plain http to https.

I suppose an alternative is to do the POST in the same way that we do ReadData, i.e. there’s a buffer of a size that’s specified in the call, and it’s filled by the caller as many times as necessary until the total length has been fulfilled. Or is this just the chunked transfer that has been talked about previously?