How do I give the user a list of 10,000 items?

Problem – RISC OS has no “combobox”.
It can be ‘faked’ by using an icon with the little menu button beside it, that opens a pop-up menu.

Problem – it is slow as hell on a rainy Tuesday.
Given an example text of “Additional menu item #” (where # is the number), DeskLib can use Menu_Extend() to create a menu containing 10,000 entries in 33 seconds (standard RPi). The first 30% flies, the last 30% crawls. The code does a load of realloc() calls, so there is room to optimise (maybe grouping items together and adding them in batches of 25?).

The problem is the Wimp. Click to open this menu. The desktop completely freezes (even Alt-Break doesn’t work) as we’re in the Wimp and the Wimp is taking a mere one minute and thirty four seconds to open the menu. To its credit, it does manage it – but that sort of speed is unacceptable.

Is there any other method whereby a list containing thousands of items can be created and easily selected by the user? At this time I don’t need 10,000 items. But I do need 3,796 (growing slowly).

Presenting me with a menu with 4000 items on it is a waste of time – more than one screen full and I get bored.

It is a very specific application where there are, as I stated, currently 3,796 items to choose from.
I’m going to say no more at this time (no code written yet), except that that’s the number of items available.

Would the toolbox treeview do the job or building a menu tree rather than a single menu?

Ah, you mean to cheat and do:

A -> xxx
     xxx
     xxx
B -> xxx
     xxx
     xxx

?

Mmmm, that might work. I’ll have to knock up a cradle to test that.

I wouldn’t know – what you are doing is top secret :-)

You can create a textfile or html file listing tens of thousands of items within seconds. Does it have to be a popup menu? Open an html file and capture URI requests. See the !Launch example in the RiscLua distro.

I’m a bit surprised. My database program (not officially released and written in Basic) could create alphabetically sorted menus with tens of thousands entries in seconds, just by means of the plain Wimp_CreateMenu and that sort of things – on the RiscPC.

My problem however was that the longer those menus, the more memory they needed. Therefore, I rewrote that routine by Font_painting all the entries. Now it doesn’t need any memory at all and is still fast.

You can show any Window as an menu. So create an Window without auto draw and then just paint the visible items yourself using either Wimp_PlotIcon or Wimp_TextOp. You would then need to detect mouse clicks to figure out what was clicked upon, but I don’t think it would be too complicated. This way the Wimp doesn’t need to create any menu items at all which should speed it up dramatically. Obviously it’s much more work for you as a developer, but the end user experience should be the same as using any other menu.

Just had an amusing/twisted thought.
You could always multitask the menu create and have a filer copy style window and have a “faster” button.
[This comment was for humour, not a serious suggestion]

Colin:

I wouldn’t know – what you are doing is top secret :-)

Not secret, I just want to say too much in case the project doesn’t ever get finished. ;-) I have a fair few projects that I started and… kinda got bored somewhere along the way.

I have created a menu with an alphabetical parent and all the stuff nicely organised. The parser can extract the data from source (all 3,796 items) extremely quickly. This, once parsed, is saved as an 820KiB predigested block of data, which means subsequent loads fly along. As each menu is a lot smaller (I think the largest is about 320 items or so), it builds and displays almost instantly. So, yes, this is a good way to do it. And from a user point of view, more friendly than a huge list (actually, the huge list was a dumb idea, but it’s how an Android app I use does it and even though it irks me every time, I didn’t engage the brain cells (actually, I just kind of wanted to know if the Wimp was up to 10,000 indirected menu items!)).

The testbed code works. Now I have the easy-peasy part – actually writing the damn thing! <sigh>

You can create a textfile or html file listing tens of thousands of items within seconds. Does it have to be a popup menu? Open an html file and capture URI requests.

So the idea is to parse the data, make an HTML file, and use another app (a browser) to handle the choice for us? Uh – doesn’t that sound to you like a workaround for a failing of the UI? Your app shouldn’t need to use another app to do something simple.

the more memory they needed.

Uh-hu… The test code sucks up 1212KiB just to hold the object data (about 800KiB) and the menus (the rest). Still, I have 208844KiB free, so I need to breathe and let it go and remember how many (enormous) resources VB needed just to display a messagebox saying “Hi!”. ;-)

You could always multitask the menu create and have a filer copy style window and have a “faster” button.

My app will need to fetch data from the internet. Perhaps just for you, I’ll add a “Faster” button in the fetch status window. :-D

just to display a messagebox saying “Hi!”. ;-)

For what it’s worth, RISC OS needs 79 bytes to do the same thing:

SYS "Wimp_ReportError", "    Hi!"+CHR$(0), %001110010001, "Rick says..."+CHR$(0)

Slightly smaller, yet, written in assembler:

  ADR   R0, hi
  LDR   R1, flags
  ADR   R2, ricksays
  MOV   R3, #0
  MOV   R4, #0
  MOV   R5, #0
  SWI   "Wimp_ReportError"
  MOV   PC, R14
.hi
  EQUS  "    Hi!"+CHR$(0)
.flags
  EQUD  %001110010001
.ricksays
  EQUS  "Rick says..."+CHR$(0)

The source is larger, but the result is a mere 60 bytes. Nerr-nerr VB! ;-)

LIBRARY "Test"
PROCtest

Will do any program you like in a few bytes.

Size of the code is meaningless.

@Rick

Just to satisfy myself I knocked up a quick Toolbox app with a StringSet gadget. Using VRPPC, it created a menu from 4000 comma-separated strings in a couple of seconds, and opened the menu with just a small perceptible delay. Even allowing for a fast PC I think this compares favourably with your initial figures.

@ Colin:

Size of the code is meaningless.

The point was comparing how much “baggage” VB needed for doing a simple task. Unfortunately even when compiling as native code, there is no mechanism for including the library routines actually used in the executable. No, you still need all of the baggage.

@ Steve: Just for interests sake, how long was the text? “Additional menu item #” needs to be indirected.

Hmm…

I created a menu with 4000 items (numbers). It took 17 seconds for the Wimp to open. I wonder what DeskLib is doing?

   base_menu = Menu_New("List", "1");
   for (recurse = 2; recurse <= 4000; recurse++)
   {
      sprintf(workspace, "%d", recurse);
      base_menu = Menu_Extend(base_menu, workspace);
      // incredible abuse of realloc() going on here! ;-)
   }

Creating a 4000 entry string by making the entries as a 20K string and creating the menu in one go, it created the menu instantly but still required 17 seconds to show it.

Mmm… DeskLib creates a standard menu block. Putting trace code into the program, the delay is happening inside Wimp_CreateMenu … which is a short assembler wrapper for the SWI of the same name.

The toolbox (eventually) just calls Wimp_CreateMenu too. It might be interesting to try your program on a Pi, see if it takes time too.

RedSquirrel – RiscPC 700, RISC OS 3.70: 47 seconds (multitasking), 43 seconds (full screen).

RPCEmu – v0.8.9 recompiler, RISC OS 5.19 (14-Mar-13): 7 seconds (multitasking, 89MIPS), 6 seconds (full screen).

Real Pi – 256MiB model B at 700MHz: 17 seconds.

My PC is an Intel Atom clocked at ~1.6GHz.

Just for interests sake, how long was the text?

Probably somewhat less than you need. I just created number strings from the index in the loop creating the data.

It might be interesting to try your program on a Pi, see if it takes time too.

Quite bit worse. About 6 seconds to fill the StringSet gadget on initialising and about 13 seconds to diplay the menu. Given the short strings, these figures would seem to be comparable to yours. My PC is really quite fast with VARPC. ;-)

If your menu data is not dyamically changed, has anyone suggested pre-creating the menu structure and loading it into memory when the program starts? Or is that what you have done with the submenu method?

Is there an ‘elegant’ API way to disable the behaviour of menus automatically right-aligning stuff that it thinks are key shortcuts? At the moment I replace spaces with hard spaces before pasting the strings into the menu block.

As a side note, the Wimp miscalculates the position of right-aligned shortcuts when the menu has a scroll bar. I’ve put in a bug report; though it is only “Minor” as a menu that provides shortcuts and is large enough to overflow the screen height is something of a pathological case.¹
It isn’t something that I need, I just noticed it due to the Wimp messing with things that were NOT intended to be shortcuts (did anybody think of the side effects of adding Red, Green, and Blue as checked-for words? :-) ).

¹ Maybe not so pathological, long menus (wasn’t Impression famous for those?) could more easily overflow TV style video output if people are using CVBS or s-video instead of digital).