Help me understand Cloverleaf

As an example, the recent issue with WinEd and SaveAs dialogue boxes took me about a week to track down. Were I working in my usual “paid for” environment, it would have taken as long as was needed to turn on “throw all exceptions” in the debugger, run the code through the problem, and then look at the state of the software when it broke. On RISC OS, I ended up reading the sources for the Window Manager, in assembler and with extremely poor comments, in order to try to find things that I could do to narrow the problem down, then doing the same with the DeskLib library. Finally, I added lots of debug output to Reporter, before trawling through it line by line.

Ah, glad it’s not just me then. I’ve been having many similar experiences developing Paint, admittedly spending less time trying to understand the assembler (almost a lost cause given my limited experience of it) but quite a bit reading the C sources.

@Andy & @Steve Fryatt

Well then is on the agenda to get better debugging tools. Or what is what is missing?

Does this tool going in the right direction?
https://fuentesp.github.io/UCDebug/

Could a JTAG debugger board, with the right tools, help?

Well then is on the agenda to get better debugging tools. Or what is what is missing?

My lack of experience hasn’t helped. I only acquired DDT when I started the Paint bounty and I don’t find it at all intuitive to use. I see that it can single step source code (I’m yet to figure out where it reads the path to the source files) and display variables but compared to a modern IDE where they’d all be tabulated, visible constantly, with writeable text boxes to edit their values, on DDT having to click through dialogue boxes and trawl through lines of text just to see them is very clunky. It doesn’t really multitask and on the version I’m using in RPCEmu mouse clicks don’t even seem to line up with the vertical scroll bar. Never mind the fact that I can’t seem to run DDT a second time after quitting it without rebooting!

I don’t know if I’m doing something wrong, but very often when an exception occurs Context doesn’t have stack frames that can get me all the way back to the code that caused the problem.

The end result is I’ve had more luck and faster progress with logging to Paint’s own trace file to debug problems and only fire up DDT when I really need to as it’s quite painful.

Does this tool going in the right direction?

I don’t know yet. The !RunImage is in ELF format which RPCEmu can’t recognise. I vaguely remember reading a way to convert ELFs to Absolute on here recently, but I forgot what it was.

!UCDebug works with ELF-format executables, and needs them to be linked to address 0×18000. To obtain a !UCDebug-compatible executable from an assembly code, run the following lines:

as -o file.o file.s
ld -Ttext=18088 exec file.o

It doesn’t sound like this is making things more intuitive or faster to navigate. Perhaps with further development it will though.

I only acquired !DDT […] and I don’t find it at all intuitive to use.

It isn’t.

I see that it can single step source code

For C programs, you need to use the ^C (continue run) until it hits the main() function and stops itself… or you’ll be single-stepping through the entire C runtime setup and all over ROM.

Unfortunately, this is also true of all of the stubs branch points into CLib. Really, DDT ought to be smart enough to recognise this and auto-skip this stuff, so you don’t end up wandering around ROM or the RMA.
But, then, DDT was totally cool with an uninitialised pointer accessing zero page (my system is low vector so it wasn’t faulted, but surely a Debugger should flag it up?).

(I’m yet to figure out where it reads the path to the source files)

It might be an assumed thing? If my !RunImage is to be debugged, and there’s a ‘c’ subdirectory with the source within, it just works.

display variables […] visible constantly

So very much this.
To the point where one “runs a line of code” and then has to go hunting down the differences.

It doesn’t really multitask

It doesn’t multitask at all. Because of how RISC OS works, what you see when you see the DDT UI is a completely fake “desktop”. The rest of the system is suspended.

and on the version I’m using in RPCEmu mouse clicks don’t even seem to line up with the vertical scroll bar.

Wait until you have the fun of using it with something that runs on a Pi and does direct VDU output. Suddenly all of the text will become half height, and if you’re really lucky it’ll go completely batshit crazy and paste bits of window furniture all over the place, and output the entire screenful of text messages on top of each other at the top left of the screen.

Never mind the fact that I can’t seem to run !DDT a second time after quitting it without rebooting!

Well, when it gets its panties in a twist, rebooting is usually the only option.

The end result is I’ve had more luck and faster progress with logging to Paint’s own trace file to debug problems and only fire up !DDT when I really need to as it’s quite painful.

Ditto. I make heavy use of DADebug. A #define DEBUG 1 switches the debug() function in or out of the source. debug() behaves like printf(). And it’s a simple matter to drop messages to debug() around things that are problematic (or, in some cases, just embed them into the code with their own #define for tracing flow through entire functions – you’ll see this in the EBook source). Then, pop open a TaskWindow, enter *DADPrint and there’s the entire record of what happened.
It’s a hell of a lot quicker/simpler than using DDT. And it doesn’t poop itself at the drop of a hat either.

The !RunImage is in ELF format which RPCEmu can’t recognise.

SharedLibs needs to have been seen to set up the ELF RunType.

I vaguely remember reading a way to convert ELFs to Absolute on here recently, but I forgot what it was.

elf2aif in GCC.

I’m yet to figure out where it reads the path to the source files

In Options→source tree

I like to use reporter and debuglib – well my own similar thing I used before debuglib became available:

In the makefile for a module put:

ifneq ($(filter debug%,${MAKECMDGOALS}),)
CFLAGS   += -DDEBUGLIB
LIBS     +=  ${NET5LIBS}
endif

For an app where Mkdebug makes a DDT file, don’t use mkdebug or change it to use the normal compilation with DEBUG=TRUE in the amu command line and add this to the makefile:

ifdef DEBUG
CFLAGS   += -DDEBUGLIB
LIBS     +=  ${NET5LIBS} ${DEBUGLIBS}
endif

create debug.h

#ifndef DEBUG_H
#define DEBUG_H

#include "debuglib/debuglib.h"


#ifdef DEBUGLIB

#define debugf(...)\
do \
{ \
    dprintf(("", "%s (%d): ",  __func__, __LINE__)); \
    dprintf(("",__VA_ARGS__)); \
} while(0)

#else

#define debugf(...) do {} while (0)

#endif


#endif

And first thing in main or module_init

  debug_initialise(Module_Title, "RAM:$.log", "");
  debug_set_device(REPORTER_OUTPUT);
//  debug_set_device(DADEBUG_OUTPUT);
//  debug_set_device(FILE_OUTPUT);

I’ve included the option to switch output to a file/DaDebug for demonstration purposes you can redirect to different places see the documentation for Debuglib in the sources.

debugf is used like printf and will disappear when the DEBUGLIB flag isn’t defined.

Then after running reporter you will see the debug output in real time in the reporter window:

debugf("myoutput=%d\n", myoutput);

becomes:

 Module_Title:__func__ (__LINE__): myoutput=3

so if you just want to see if an option is taken just add debugf("\n") and the line number will tell you if that point is reached.

If time is important you can add the monotonic time in the debugf macro. Works with either modules or apps

Well then is on the agenda to get better debugging tools. Or what is what is missing?

Sorry, but what does that question even mean?

@Andy S:
Raik Fischer suggested DeskDebug fom
Screenshot DeskDebug
www.weissniklaus.ch

But seems currently not available. I tried to contact him now. Let’s see..

@Steve Fryatt
What I ment is: what functions are missing in current available debugger(s).
Or it is ‘just’ the user interface that is low quality.

@Rick Murray

NEVER! How many window managers are there for Linux now?

There’s always one more needed …

I would like to echo the debugger comment.

I use Visual Studio with C# and Chrome with JavaScript, and both environments are miles ahead. I would like to be able to pause and single-step lines of code, inspecting variables, and even altering their contents and/or the code as I go.

I found developing USBJoystick meant printf() statements to Reporter and being careful to avoid a complete machine lock-up.

And intellisense would be nice…

What I ment is: what functions are missing in current available debugger(s).

I’ve never managed to get DDT to work successfully, so I can’t comment on that.

Or it is ‘just’ the user interface that is low quality.

As Richard has said, “what other platforms do”. A debugger that allows the code to be run, stopped, stepped through, have exceptions caught, variables examined, and so on. For now, I’d settle for the simple stuff that developers coming to RISC OS from other platforms will expect.

I’m yet to figure out where it reads the path to the source files

Sounds like I am about to reveal a “secret” only the ancient Acorn C/C++ Jedis are aware of! Joking :)

To make DDT find your source code on the fly just place your executable in the directory that contains also your .c subdirectory (where the C source files are), basically DDT uses Run$Path to search for the source, but the debug info only allow it to find for example myfile.c which obviously on RISC OS is c.myfile hence the executable your are debugging must be placed just in the directory above where your .c.myfile etc are.

You can also provide the source path after it loaded the executable use Options → Source Tree and basically provide the whole absolute path till the .c directory (excluded), However it generally works better when you copy the executable just out of the .c directory and load it in DDT from there.

BEFORE you try the above make sure you have compiled your code with the cc option -g and the link option -Debug otherwise DDT won’t be able to combine the source with the binary.

To set a breakpoint to YOUR C main function use menu Breakpoint → type main in the input field and then click on “At Address”, yes this is confusing because you can also see a button that says “At Procedure” and another that says “At symbol”, but main is an address and do not get confused with “__main” or “_main” these two are part of the C standard and if you break points at these two you’ll watch the whole CLib initialisation (so if you have time to waste is actually a good exercise to figure out how the RISC OS SharedCLib works).

At this point just execute “Continue” (^C) and DDT will jump starting at the beginning of your C code, if you did all the above right DDT will automagically display your C source where your main function is.

Now you can start playing with DDT features, for example you can start using Display → Symbols etc.

Now if you want to step in at Source level from the menu select Single Step and then Step by source statement, then OK and voila’ you’re debugging at C source level without any trick.

In the little spare time I have I am writing an article on my blog on how to use DDT because the original documentation is not… let’s says… very user-friendly (sorry ROOL!), I’ll try to focus on this task and try to finish it, if it’s useful for people.

Hope this helps!

What I ment is: what functions are missing in current available debugger(s).

I’ve never managed to get DDT to work successfully, so I can’t comment on that.

Just my 0.5c, to learn how to use DDT I had to almost reverse-engineer it back in the days, it was very badly documented and so was the Norcroft compiler, so it took a long time to figure out how it works. However once found out how it actually works it’s very easy to use and has all the features someone would want from a debugger.

What DDT is missing for example is when an application start using console input then DDT disappear from the screen until it can get back control and this is an issue caused by RISC OS cooperative multitasking and not specifically by DDT. So certain missing features are not really on the debugger per se, they are caused mostly by the environment if I can use this term.

Maybe the easiest way to get a competent debugger would be the emulator route. For most things, a pure-interpreter would be fast enough (after all, debuggers on other OSes also slow down execution rather a lot), it is probably the only way to allow debugging of modules without ripping RISC OS completely apart, and it would allow a lot of control over the execution of code and inspection of state.

I understand that DDT is also some kind of emulator, but its approach seems to be more fragile when trying to interwork with the OS. A completely separated sandbox emulator would make things a lot easier.

@ Steffen Huber

Maybe the easiest way to get a competent debugger would be the emulator route.

Emulators offer a fantastic way to debug OS level yes, I use QEMU for example a lot to debug the Linux Kernel’s changes and patches I have to do sometimes at work. However they also have limitations: They not always represent the bare-metal correctly, they not always represent the multi-core environment exactly has it should be. But I wouldn’t worry too much about these limitations initially.

I understand that DDT is also some kind of emulator

As far as I remember it actually is a regular module which uses clever tricks with the RISC OS cooperative multi-tasking, it basically freezes RISC OS multi-task when you step in or get to a breakpoint and unfreeze it when it has to proceed forward. So it’s not as someone said “a single task” it’s actually an “hybrid single/multi-task” module or a better definition would be step-motion module. So not really an emulator in the strict sense or the term.

The reason why DDT is good to debug Applications and not the Kernel stuff is the above explanation, it plays with the WIMP basically to obtain a debugger-like process if you want. Some people may argue with this (I am sure I know a couple that will comment quickly on that lol), but the reality is that at the time of RISC OS 2 with machines that could have max 4MB RAM which included video RAM and module area etc… to build a proper multi-tasking debugger that could also hold Registers’s values while other apps were running, in an environment as fragile as RISC OS was (prone to crashes etc.) the DDT approach was probably one of the best possible if not the best.

Nowadays with machines that have 4GB RAM for like 50 quid I agree that DDT is not longer needed as much as it was on the Archimedes. I used it a lot back in the days and IIRC DDT is the only debugger on RISC OS that allows me to debug not just C and ASM, it actually can debug Pascal, C++ and I remember debugging on it also ABC compiled BASIC programs, so it’s not as bad as been described IMHO.

so it’s not as bad as been described IMHO.

It seems to me to be less stable than it was in the RISC OS 3.10 days (but, then, machines were a lot simpler and less variable than these days).

It is also a pain to use – you have to ask it to list the contents of registers and specific variables. It doesn’t seem capable of holding open a window listing “variables in scope” and updating that as the program is stepped through. I’ve also noticed it, a number of times, completely missing when the value of a register changes. Possibly I’m not doing it right? I don’t know. What I do know is that while it can do something that nothing else does, I tend to use it as a last resort, not as my go-to debugger of choice.

So it’s not as someone said “a single task”

I think the point we are making is that it does not co-exist within the Wimp any more than allowing a multitasking program to run in the Wimp environment. When the debugger is active, the rest of the system is suspended. No multitasking, nothing.
For lots of fun, stick it into trace mode (where it will report each line executed) and tell it to ^Continue. You’ll see everything go nuts as the debugger is entered, reports something executed, then goes back to the Wimp, ping-pong until epileptic fit.
The debugger does not multitask. The client does, when the debugger UI isn’t active.
That’s the point we’re making. ;-)

and not the Kernel stuff is the above explanation,

A better one: it’s just a module running in module space with module level privilege. Hence, having module level code debug and trace module level code would be…interesting. As in “kaboom!”.

It seems to me to be less stable than it was in the RISC OS 3.10 days (but, then, machines were a lot simpler and less variable than these days).

It’s definitely less useful than how it was on RO 2 and 3, that’s for sure. In terms of stability I haven’t experienced so many problems as reported here, so I really cannot say. To me it works sufficiently fine, but obviously other people may have had nightmares from it… I have heard similar stories about GDB, which is way more powerful that DDT could possibly ever be, so everything is possible in this world :)

That’s the point we’re making. ;-)

Which is a point of view not the universal truth as for everything we are saying, so (like mine) it’s just a point :)

A better one: it’s just a module running in module space with module level privilege. Hence, having module level code debug and trace module level code would be…interesting. As in “kaboom!”.

Again, it seems we are just talking past each other, DDT was not meant for kernel or drivers development, it was a relatively simple and clever idea to debug Applications, it was better than nothing.

Anyway, sure we want a better debugger, ok I’m in, let’s make it.

I try to avoid using DDT wherever possible as it has gone through phases where it would lock the system up more frequently than my buggy application did. 95% of Fireworkz debugging I can do under Visual Studio, anything else is tracked down using Reporter. [Edit: and if that doesn’t work because of lock ups (maybe every other year), I can output stuff under a hacked RPCEmu, via a private SWI, to the Linux console]

Having steered clear of gcc/gdb on RISC OS, how good/bad is the gdb experience here? Did anyone ever get remote debug working so you can develop on GCCSDK/Linux and debug a RISC OS application, even if that’s down a serial line?

Also for who is experiencing problems with DDT on RPCEmu few possible useful advices (given that no versions were provided):

- Try to download the latest RPCEmu (latest at the time of this is 0.9.3)
- Make sure you are using a stable RO 5 release like for example 5.24
- I usually use video resolution like 1920 × 1080 at 30Hz 32T Colours or 16M Colours
- Make sure you have installed latest ROOL DDE (at the time of writing this is DDE29c)
- Make sure you have installed DDE following the installation instructions as well as used !SysMerge script in the AcornC/C++.Developer directory (and if you update DDE make sure you run that script again)
- Make sure you have followed the instructions I gave previously on using DDT and the compiler
- If your application requires a lot of memory make sure it has it and has been allocated for it

This usually makes it stable enough for me, obviously RISC OS is not bullet proof when it comes down to stability and applications doing odd things. So if this is the case then there isn’t much any debugger can do right now.

Hope this helps to make it more stable, to me it works fine on RPCEmu (I am using RO 5.24 and StrongARM config with 64MB RAM and HostFS).

Some demo/examples on using DDT would be nice.

@ Colin Ferris

Cool, Adding my experience in an article, will publish it ASAP and I can do an “hazardous” thing (just for the love of the RO community :) ) will try to make a video tutorial of what I know about DDT, ROOL/Jeffrey if you guys wants to review it before publication to fix all my mistakes please feel free to let me know and I’ll make sure you review the material before publication, thx

Stuart wrote…

…using Reporter. [Edit: and if that doesn’t work because of lock ups…

Reporter can cater for some lock-ups, especially if they are reasonably repeatable. The *ReportOpt Log option causes anything reported to be immediately flushed to disc. so it is visible after a restart. It does slow processing slightly, but it can speed up debugging! But not a total solution, I agree.