Printing system upgrade

The work I’ve done on printing via IPP has shown that the existing printing system is not suitable. Let’s look at how it works today:

• The Printer Definition File is fixed for a given printer model, and specifies the resolutions and colours that the printer supports.

• The Printer Manager allows selection of printer, resolution, paper size, and the pipe to the printer (USB, parallel, file, etc.)

• The pipe is unidirectional, “fire and forget” other than flow control, which is dealt with below the level of the Printer Manager.

• There is no way to get any information about the printer from the Printer Definition File to the dumper module.

How it needs to work for IPP:

• The information about the printer’s capabilities, including resolution, paper size, colour etc. and, most importantly, its URL, are obtained from the printer via multicast DNS.

• The Printer Manager allows selection of printer, resolution and paper size.

• The data are written to a file, then sent to the chosen printer via HTTP.

The Printer Manager currently has no idea of any printer’s URL, nor can it send via HTTP; this is entirely different from just shoving the data into a pipe.

Is anyone interested in collaborating on a new architecture for the printing system? We could discuss it here, and write up specifications on wiki pages.

mDNS only provides a subset of information. It’s best considered an identification of the printer and the correct URL, to then permit the printer to be properly interrogated once it has been selected.

At any rate, as Dave mentions, IPP is fundamentally incompatible with how !Printers works.

• There are n printers, any number of which could be active at any given time. Including none.
  If you notice on Android, the list of printers dynamically updates and periodically refreshes itself.
• User selects a printer, which then leads to a dialogue about copies, colour or mono, etc. Same advanced options may include quality (euphemism for resolution), paper type, etc etc. This information is specific to the chosen printer.
• User then printers, magic happens.
  Usually. Not always. ;)

It’s a very fluid bi-directional protocol. What they have in common is that the underlying transport (mDNS/IPP) and the raster encoding are the same. Which means that the printer driver really only needs to know how to cope with mono and colour to support pretty much everything (it’s a little more complex with AirPrint as there are numerous types of URF for god knows what reason; but modern devices will do IPP Everywhere too, so URF is mostly an anachronism).

It’s also a protocol where there may be zero to a dozen printers available, with potentially different ones each time around. So the existing Printers behaviour of plonking lots of icons down is a bit of a non-starter.

a new architecture for the printing system?

Do you remember the RISC OS 2 printing? It didn’t have a unified Printers. Instead it was an app PrinterDM, PrinterLJ, PrinterPS…

I wonder if it might be worth looking into how that worked to see if it could present some ideas?

What occurs to me is the inescapable requirement to use the driver front end to choose an IPP printer before asking the application to print, because the entire system is predicated on the idea of a printer already having been selected. It’s not possible, I don’t think, to start off a print job and then choose what printer to send it to.

What occurs to me is the inescapable requirement to use the driver front end to choose an IPP printer before asking the application to print, because the entire system is predicated on the idea of a printer already having been selected. It’s not possible, I don’t think, to start off a print job and then choose what printer to send it to.

On all other operating systems I have used you select the printer during the process of starting off the print job, but with RISC OS the applications assume the printer has been chosen, and you just get a Print button that starts the process straight away. That makes it hard to introduce a discovery stage without rewriting all the applications!

I suppose one approach would be if the Printer Manager periodically discovered the printers via mDNS. How often would that need to be done, in order to pick up any printers which the user had only just turned on, without introducing an annoying delay before you can select the printer on the iconbar?

What occurs to me is the inescapable requirement to use the driver front end to choose an IPP printer before asking the application to print, because the entire system is predicated on the idea of a printer already having been selected. It’s not possible, I don’t think, to start off a print job and then choose what printer to send it to.

Different printers support different resolutions, colours and paper sizes, and the raster data are created to that particular specification, so it would be nonsensical to start a print job without knowing what printer it was to be sent to.

What happens if a printer goes away between its selection on the iconbar and somebody pressing Print? Does another printer get selected, or the job thrown away, or an error message?

I agree the fact that Print dialogues don’t call out to the driver is awkward, because often the contents of the Print dialogue depend on driver settings (don’t ask about colour if the printer is mono, etc). Having all of that modal in Printers is awkward because you might want different apps to have different settings (colour for Paint, mono for PDF, or whatever)

Another issue with the current setup is it’s not really designed for having lots of printers. Somebody on a network could have dozens of printers show up via mDNS or IPP. Some of which are useful, some of which the user would never choose (eg physically in another building or owned by somebody else) and some are niche that you’d only want to use in rare circumstances (the fax machine, the A0 roll printer or the laser cutter). Simply creating them all with iconbar icons wouldn’t work, you’d need some way to curate ‘your’ printers. If you did dynamic icon creation the iconbar would continually change as printers on the network got turned on and off.

The work I’ve done on printing via IPP has shown that the existing printing system is not suitable […] The Printer Manager currently has no idea of any printer’s URL, nor can it send via HTTP; this is entirely different from just shoving the data into a pipe.

I think all of the bits you need exist already, or have precedents with other printers supported. After all network printing isn’t new – even Econet can do it.

Here’s what it sounds like may be needed:

• Ability to discover a printer (in the background?) – USB printers are already handled this way, by RemotePrinterSupport (for no particular reason other than it’s a module so could see Service_USB), which causes a PDF picker dialogue to pop up. That’s not so bad for USB because that’s local to the computer and if a printer appears it’s probably because I just plugged the wire in. The USB PDF picker extends if several printers are discovered, so if <many> network printers are found this would effectively queue them rather than result in <many> consecutive pop ups. There was also some work to do parallel printer autodiscovery for various STB’s but that never really went anywhere due to its limited support and then USB came along, but the sources are hanging around if you dig a bit
• PDFs are fixed for a given printer model – not sure why that wouldn’t be the case here, I buy an IPP capable printer, its properties are fixed at the time of manufacture. You could get really fancy and generate a PDF on the fly (like we do MDF for monitors via EDID), but that sounds like polish since I’m only likely to have a couple of printers at most and each lasts many years.
• Connection is to a URL or something – that’s OK, LanManFS, NFS, Econet already handle this. The more recent network-y printing (LanManFS, RemotePrintFS, JetPrintFS) all work by presenting a file-like interface and inviting a magic string to go in the File writable in the connections dialogue which is one way. Alternatively adding a PrinterType$N is an option, though even those tend to just resolve to files in DeviceFS now.
• Not a direct pipe – spool it to a file, do whatever transfer packaging you want on the file. Printer Manager already knows how to spool jobs to files locally and send later. If the pipe moves or changes name that’s easily dealt with via a system variable, so IPP$TargetPrinter can be set as often as needed by the chatter on the network, and the user just enters the variable name (or, if we’re not using the File writable, Printer Manager can know to use whatever).
• Printers that vanish – again, this is already catered for. It’s a ratchet system so printers you use which you’ve selected a PDF for are saved in Choices and appear whenever Printer Manager starts. They can however be greyed out, including dynamically – for example, a shared printer when the other ShareFS computer is being rebooted.

There is no way to get any information about the printer from the Printer Definition File to the dumper module

That’s not the dumper’s concern. PDriver worries about those things.

After all network printing isn’t new – even Econet can do it.

True, but what all of those have in common is that it’s a one way dialogue. You’re supposed to know what sort of printer is connected, the network is just the interface that you throw data into.

I buy an IPP capable printer, its properties are fixed at the time of manufacture.

That’s true, but you’re missing two things.

The first is that for some printers, the paper size is dynamic. My HP3630, for example, recognises the difference between A4, A5, and 15×10. Not only that, but behaviour changes. A4 is plain paper, 15×10 is photo paper. This is set up (once) in the configuration of the printer itself, then afterwards is handled automatically by the printer. Part of the IPP reporting is what sort of paper is inserted.

The second thing is that the entire point of IPP Everywhere, following on from AirPrint, is that it shouldn’t matter to you what the printer is. As long as it is visible, you should be able to choose it, print to it, and get some sort of sensible output.
Mopria takes this a step further by supporting things like scanners as well.

My new tablet doesn’t have any specific drivers (I tried, it went horribly wrong, but then Android 11 seems to have broken things…again). What it does have is a default built in printer driver.

When I choose to print something, say a Google Docs document, the print dialogue will open. It will be blank for a second or two as printers are searched. Then it’ll pop up with the details of the last printer I used, if it is found again. If I’m happy with that, I can fiddle with some settings below (landscape? page range? don’t use colour?). If not, I pick a different printer and fiddle with the settings.
Then I tap on the printer icon (that actually looks more like a blocky space invader) and it prints.

Next time, same process. Nothing of the previous session is remembered (orientation, page count, etc).

Under Windows (XP, as that’s what I use), it’s almost the same, as the print dialogue itself allows a way to choose a printer and set it’s properties at the point of printing. The only difference is that printers must be installed and known to the system, but this is understandable as XP far predates any of this stuff.

As you can see, IPP is quite different from the current model of choosing a printer, with the options predefined, then printing to it.

Edit: a thought – does the RISC OS driver even support such things as choosing between quality levels (like draft and best) without physically altering the printer’s settings?

Not a direct pipe – spool it to a file,

Spool what to file? The paper dimensions, resolution, and bit depth in the raster image may be dependent upon the chosen printer. For instance, a mono laser isn’t going to want a 24 bit bitmap.

Printer Manager already knows how to spool jobs to files locally and send later.

Yes, because Printer Manager is hardwired with knowledge of what printer is expected to be connected and what data format it requires. If “Sally” is a LaserJet, it receives an LJ dump regardless of whether it’s hooked to 0.254, parallel, or RemotePrinterFS.

You could sort of fake this with the current setup by installing PDFs for known printers and greying out the ones that don’t respond to the inquiry, which may be how things would have to proceed… however that’s kind of breaking the intended flexibility of IPP for the limitations of our printing system.

That’s not the dumper’s concern.

Dumper needs to be able to dynamically alter it’s output depending upon not only the chosen printer, but the current configuration of said printer.
Remember, above I said that my inkjet responds to paper sizes. All I need to do is take out the A4 pages and drop in a photo sized paper, and it will change. IPP printing changes accordingly. So under RISC OS, the dumper (and the rest of the system) would also need to be aware of this change.

True, but what all of those have in common is that it’s a one way dialogue. You’re supposed to know what sort of printer is connected, the network is just the interface that you throw data into.

Surely that’s only at the printing stage, though? Just as Printers currently finds USB printers and makes them available, is there any reason why it couldn’t look for IPP printers and add them to the iconbar (or make them available in a list which can be selected from to add them to the iconbar), independently of the applications which are trying to print?

RISC OS applications do not expect to be able to choose a printer; they expect to be told that this is the selected printer, and here’s what it can do. You’re not going to change that without having yet another major extinction event for those apps which aren’t being actively maintained.

What we need to do is to find a way to make IPP work within the confines of RISC OS. Given that both Linux and Windows will find IPP (I assume) printers and list them as being available in the printers configuration – and even allow you to select them as being ‘default’ – how is that different to Printers finding IPP printers in the background and offering them up for inclusion on the iconbar? What stops Printers having a pre-emptive negotiation with a new IPP printer that it sees, to find out what it can do and then go on to make that information available as it would for the same information gleaned from a PDF in the more traditional manner?

without having yet another major extinction event

It wouldn’t be an extinction event. Unlike architectural differences, this one would be optional and as such, if it didn’t work with existing software, nobody would use it regardless of how many bells and whistles it offered.

What we need to do is to find a way to make IPP work within the confines of RISC OS.

Indeed. What needs done is finding a way of making a modern printing method work with a thirty year old concept of how printers behaved.

Given that both Linux and Windows will find IPP

I’m only familiar with Android (and lightly, iOS 7). My Windows is far too old to be used as any sort of measure.
So, somebody with modern stuff will need to fill us in on how IPP works for them.

What stops Printers having a pre-emptive negotiation with a new IPP printer that it sees

It’s certainly a possibility, hence this discussion.

Having thought about it a little while watching GITS, what comes to mind is that in any case Printers ideally needs modifications in order to allow an easy (temporary) selection of parameters such as colour or mono, draft/normal/high quality, etc.
Having to “Duplicate” an entire printer definition just to have a slightly different configuration…is rather dumb.

Doing this could easily tie in with the IPP options, and would present something closer to how things are dealt with on other platforms.

Dave – correct me if I’m wrong, but I don’t think IPP has any option to select a paper type (glossy, photo, plain…). Certainly on my inkjet, there is no way of doing this, the preconfigured paper sizes are set up with that fixed within the printer. Which is quite a pain in the arse if I want to do an A4 photo print, need to log into the printer, make the change, then print.
Sadly, the Epson didn’t last long enough for me to play around with anything other than the basics.

What stops Printers having a pre-emptive negotiation with a new IPP printer that it sees

This probably isn’t a big deal for a home user, but in other situations there could be numerous printers, some of which aren’t too be used. Some of which may require some form of authentication in order to use.

So your idea of listing the printers before adding them to the iconbar (but, to make it properly useful, auto-add the printers the user has previously selected) is a good one. Certainly it would keep the iconbar tidy.

Another idea, is that if a printer the user has chosen is not available, perhaps show it on the iconbar greyed out so it’s clear that it wasn’t located. It can “come alive” as and when.

For this we would need the icon to have three states – active, inactive, and not found. However as mentioned earlier, ShareFS could present an idea of how to do this.

What happens if a printer goes away between its selection on the iconbar and somebody pressing Print?

It has always been possible for this to happen. Just switch a printer off. I’m not sure how important it is to specify the handling of this error case.

Another issue with the current setup is it’s not really designed for having lots of printers. Somebody on a network could have dozens of printers show up via mDNS or IPP. Some of which are useful, some of which the user would never choose (eg physically in another building or owned by somebody else) and some are niche that you’d only want to use in rare circumstances (the fax machine, the A0 roll printer or the laser cutter). Simply creating them all with iconbar icons wouldn’t work, you’d need some way to curate ‘your’ printers. If you did dynamic icon creation the iconbar would continually change as printers on the network got turned on and off.

Agreed. The best way I can see is to have a print finder app that offers to create a definition file for each printer it finds; the user then decides which ones to let it create.

PDFs are fixed for a given printer model – not sure why that wouldn’t be the case here, I buy an IPP capable printer, its properties are fixed at the time of manufacture.

Surely you aren’t suggesting that the RISC OS community curate an enormous collection of definition files? In any case, there is still one vital piece of information missing: the printer’s address.

Also, I wouldn’t trust any printer manufacturer to maintain the same relative path throughout a production run, or through firmware updates.

There’s another thing: the relative path for a printer is different if it’s being accessed directly or via a print server.

The only thing that makes sense is to read the information from the printer (or server). I favour doing this by means of a printer finder app.

Connection is to a URL or something – that’s OK, LanManFS, NFS, Econet already handle this. The more recent network-y printing (LanManFS, RemotePrintFS, JetPrintFS) all work by presenting a file-like interface and inviting a magic string to go in the File writable in the connections dialogue

The problem is that the dumper, which is currently the entity that sends the data to the printer, never sees the URL; it is merely presented with an open handle. This is no good whatsoever when the data have to be sent via HTTP.

without having yet another major extinction event

The discussion has reminded me of one possible extinction sub-event, if you like. Currently a Printout file can be dragged to a printer icon on the icon bar. We don’t want to lose that ability. Fortunately PWG-Raster and URF files can be identified from their first few bytes, although it gets a bit harder if the destination-specific header has aleady been prepended.

What stops Printers having a pre-emptive negotiation with a new IPP printer that it sees, to find out what it can do and then go on to make that information available as it would for the same information gleaned from a PDF in the more traditional manner?

Something akin to this was implemented for USB printers by Castle in RISC OS 5. Plugging a new USB printer offered the opportunity to install it, if it was recognised. Of course, a lot were not!

A few weeks ago I looked at the Printer Manager’s code. It’s a very large lump of BASIC (the line number increment has to be less than 10) with few comments. I managed to add another choice of printer type and animate the choice pretty much correctly, but taking it to the next step and dealing with that different printer type looked daunting.

Let’s look at where we are, or could be without a great deal of extra work.

It doesn’t look too difficult to write an app that can interrogate for printers (via mDNS) and offer to create Printer Definition files for each one found. It’s possible that PDFs might be able to contain more info (e.g. URL) than they do at present, without upsetting the Printer Manager, though this remains to be tested. Having conventional PDFs means that a particular printer can be selected from multiple candidates as the target for a print job.

There’s a problem in what to do with URLs etc. in that the Printer Manager can’t understand them, and information from the PDFs is not passed to the printer dumper, which is the entity that is currently expected to send the data to the printer, given only a file handle that’s open for write. So I can’t yet see a way to get the printer data stream to the printer with the present architecture. The next best way that I can see is to print to file and drag that file to a new printer transport app on the icon bar. Even then, that app has to know the URL of the chosen printer, and I can’t see a way with the present architecture. Maybe the new transport app could interrogate the printer manager via a Wimp message and be given the filename of the selected PDF? Then it could parse the PDF itself for the info it wants.

Does any of that suggest a better idea to anyone?

given only a file handle that’s open for write. So I can’t yet see a way to get the printer data stream to the printer with the present architecture.

This is horrible… but not so much thinking outside the box as refusing to acknowledge that there is a box.

>DIM p% 128
>f% = OPENOUT("null:$.192.168.1.14/ipp/print/")
>SYS "OS_Args", 7, f%, p%,,,128 TO ,,,,,sz%
>p%?(128-sz%) = 13
>PRINT $p%
null:$.192.168.1.14/ipp/print/
>CLOSE#f%
>

;)

You can’t use “:123” notation to specify port 123, FileSwitch will choke on trying to parse the name. But no reason a different valid character couldn’t be used, so long as the module knows how to reconstitute it.
This is a valid filename:

f% = OPENOUT("null:$.https;//192.168.1.14;631/ipp/print/")

Something, anything, else would probably be better. But for out-of-band data transfer within the constraints of the current system, it’s theoretically workable.

Update:

Strangely enough, this works too…!

f% = OPENOUT("null:$.https;//192.168.1.14;631/ipp/print/#xres=300#yres=300#width=123456#height=123456")

You can’t use “:123” notation to specify port 123

In the PC world http://server.sub.domain 123 is correct for some instances.

OK, Rick, but where does the source info for that come from, in respnse to a user clicking on one of his Printer Definition Files on the icon bar?

Tell Printers that it’s too print to file, so that ought to be the file handle that is opened and passed along. Or have I missed something?

Annoyingly, that part is Printers specific, not PDF specific. It’s held in the main config file.

It’s far from perfect (not even remotely so) but within the constraints of the existing system, I’m not sure there’s any reliable way to do this. After all, if there are two printers set up using the same PDF, but one is high resolution and one uses draft resolution, looking at the PDF alone won’t help.

And we’re back to what I was saying earlier in that the way Printers works is annoyingly not quite right for this sort of thing, and it’s perhaps why other operating systems make a distinction between choosing a printer and choosing what settings are used, whereas on RISC OS doing such a thing apparently requires two different “printers” to be installed.

Again, the old way made sense with the hardware of the day (a dot matrix didn’t really have resolutions as such, and most people used laser at the default resolution of either 150 or 300dpi), but with the options available now, it’s just a badly designed headache. Things like quality, colour or not, and paper size (and bin, if applicable) should be on the fly adjustable.

But, alas, a lack of developer time is always an issue.

As an alternative idea, which might be simpler. Just do what PrintPDF does. Output the bitmap to file. When it has been created, a helper program can pick up on that (in the same way PrintPDF does) to fix up the IPP headers, retrieve the PDF (print def) which should have the URL in it somewhere, and then dump the data to the printer.

It’s not as nice and direct as printing straight to the printer, but on the other hand it’s probably a hell of a lot less hassle to implement.

After all, if there are two printers set up using the same PDF, but one is high resolution and one uses draft resolution, looking at the PDF alone won’t help.

It doesn’t matter. The resolution etc. are dealt with by the printer manager, and result in the dumper being told to dump at the chosen resolution, colour depth, paper size, etc.

Things like quality, colour or not, and paper size (and bin, if applicable) should be on the fly adjustable.

And they still are.

The bit that’s missing is to get destination information from a PDF and into the transport app.

I might have to look again at the Printer Manager code and see if I can get it to parse one or two new items out of a PDF, and respond to requests via Wimp messages to supply those items to the requester.

What is the recommended way to notice that a file has been closed?

What is the recommended way to notice that a file has been closed?

PrintPDF just tries to open the file on null events

osbool convert_queue_ps_file(char *filename)
{
  os_error		*error;
  os_fw			file;

  /* Try and open the file, to see if it is already open.  If we fail for any reason, return. */
   */

  error = xosfind_openupw(osfind_NO_PATH, filename, NULL, &file);

  if (error != NULL || file == 0)
    return FALSE;

  osfind_closew(file);

  // The file was closed when we tried to open it...

It’s not ideal, as there’s a race condition after the file is closed, but it seems to work OK in practice.

ETA It actually checks first to see if the file exists and has non-zero length, before checking to see when it closes.

It actually checks first to see if the file exists and has non-zero length, before checking to see when it closes

I recall a setup with a DOS based application, on a Windows machine, transferring into an AIX SAMBA share to be processed and returned.
As I recall the file complete status was taken from a file size comparison loop and timestamp comparison – basically waiting for an unchanging file size and timestamp.

PrintPDF just tries to open the file on null events

Yeah. All my lovely delusions about spotting when that file was opened, noting the file handle, and tracking when it is closed…fall over in a heap when I remember that you just can’t do that sort of thing in an application.

So, the deceptively simple way it is then. ;)

noting the file handle, and tracking when it is closed

which will fail if something else opens a different file at the wrong moment, and gets the same file handle.

You can keep trying to open it for writing, which will fail if something else has it open for reading or for writing.