EU Cyber Resilience Act and RISC OS

I think we are all aware of the likelihood of any of that happening. A far more realistic solution is to add the following text:-

NOT FOR SALE OR USE IN THE EU

The EU Cyber Security Regulation Act (CRA) primarily impacts software

The text says: The proposed Regulation will apply to all products with digital elements whose intended and reasonably foreseeable use includes a direct or indirect logical or physical data connection to a device or network.

As such, would this exempt software and/or devices that are not intended to connect to any sort of remote service?

I mean, in it’s more painfully reductio ad adsurdum interpretation, my SimpleSeq software would be counted under the CRA because it uses my MIDI module to facilitate a direct physical connection to a MIDI piano…!
Yes, it’s stupid. But it satisfies the description. Actual likelihood of RISC OS being pwned from a MIDI compliant piano using MIDI events? Uhhh… let me think about that. <walks off shaking head>

Open Source Free RISC OS and Applications: These remain unaffected. The EU CRA does not impact free open-source RISC OS and associated apps.

Could be wrong – this is part of why the likes of Debian are upset. Quote: a commercial activity might be characterized not only by charging a price for a product, but also by charging a price for technical support services, by providing a software platform through which the manufacturer monetises other services, or by the use of personal data for reasons other than exclusively improving the security, compatibility, or interoperability of the software.

This means that the bounty scheme should be okay as it is for improving the software (though the administration bounty isn’t), however here at ROOL various things are sold (such as the DDE), which risks annulling the “open source” exemption.

Keep in mind that this level of effort will require time and will present bugs that need addressing, so it won’t be completed “overnight”

Yup, but remember one must release a product with no known vulnerabilities and make a formal disclosure within 24 hours; penalties are set by member states, but are in the region of 10-15 million (or ~2% of annual global turnover, whichever is higher).

‘Wish List’ for Enhanced Security and Compliance:

Won’t happen. That’s an upheaval that is not far off “rewrite the OS for 64 bit”. And, note, isolation and permission models may well break plenty of existing APIs simply because RISC OS was never designed with security in mind.
After all, look how much stuff hands out pointers to the module (kernel) area to user mode applications. I suspect the reason that filing systems use a Dynamic Area for the drive maps is to section them away from the RMA and make them inaccessible in user mode; to minimise potential corruption.
Look also at how many parts of the OS are designed to be easy to hook into – how many things sit on Vectors and Service Calls.

The problem is the OS design.

We need a way to define and enforce what apps can and cannot do

The OS provides the following: OS_… AddToVector, ChangeEnvironment, Claim, ClaiProcessorVector, Control, EnterOS, InstallKeyHandler, IntOff/On, plus the memory SWIs for messing with memory in some way; plus SWI, Service, and event handlers in modules are called in a privileged mode (even though I’d bet 99% of that code does not need privs for what it does).

The problem is the OS. Not so much a hurdle as a chasm to rival the Valles Marineris.

A far more realistic solution is

I’ve just written a bit about this on my blog having read through some (not all, it’s a massive pile of verbiage) of the text of the CRA and various third party sites (that are either linked with the EU and cheerlead about some sunlit uplands; or are not linked with the EU and think it is just bad).

The conclusion I have come to, given the ambiguities and that fact that lone programmers / small teams are treated exactly the same as massive multinationals is, sadly, that I would log into the heyrick server and systematically identify files that contain programs and delete them.
All of them. Gone.
And no further software releases, updates, fixes…

Yes, it’s the “nuke it from space” option, but I do this for free and for fun. Legal advice isn’t an option, paying fines isn’t an option. The only viable option is a blunt instrument, deletion.

Multiply me by plenty of other EU based lone/small-team programmers who may well arrive at the same conclusion, and ask yourself where the next generation of coders is likely to come from. Probably not Europe.

I’d add a facepalm emoji, but I’m writing this with Netsurf…

Yes, it’s the “nuke it from space” option

Cue obligatory xkcd – https://xkcd.com/2347/ . Just wait for JoeThePlumber to remove that bit of plumbing from the interwebs…

NOT FOR SALE OR USE IN THE EU

While I’d (a tad reluctantly) do this for my website (and the SourceForge / GitHub repos) there’s also the issue of other distribution mechanisms such as PackMan and PlingStore.

This means that the bounty scheme should be okay as it is for improving the software (though the administration bounty isn’t), however here at ROOL various things are sold (such as the DDE), which risks annulling the “open source” exemption.

Two different products. OS and application.

Word != Windows¹

The conclusion I have come to, given the ambiguities and that fact that lone programmers / small teams are treated exactly the same as massive multinationals is, sadly, that I would log into the heyrick server and systematically identify files that contain programs and delete them.

Where you are neither seeking to monetise the program, nor the after-not-sale-service that seems a bit drastic.

¹ The tendency of some users to refer to one by the name of the other is simply an example of N.I

I’d add a facepalm emoji, but I’m writing this with Netsurf

That, I might say, is a side-benefit :)

Two different products.

True, but one is free and one isn’t and… “by providing a software platform through which the manufacturer monetises other services”.

It is vague enough that ROOL selling a mug might make them be considered “commercial”. Note that it is talking about the “manufacturer” here, not the product.

that seems a bit drastic.

Indeed. But if I fall into the scope of the CRA, not only must I comply with it, there’s the Product Liability Directive (PLD) waiting in the wings. In this case, you can no longer disclaim liability by saying “it’s free, use it at your own risk” as you, as the programmer, will be able to be held liable.

Again, with no exemptions for small/sole projects and barely any consideration for open source that underpins a lot of stuff in use right now (the number of things around here that are based upon Linux and Busybox…).

@ Rick

Could be wrong – this is part of why the likes of Debian are upset.

make peace with your opinions ahah ;) I have mentioned this and you’ve tried to correct me, I have removed it and you have tried to correct me again, that is one step from the “forum useless opinionated posting”.

In a more serious reply, yes there are potential issues all over the place, but the regulation still need final approvals. So, for the edge cases there might still be last minute changes. The last status I am aware of is:
“The agreement reached is now subject to formal approval by both the European Parliament and the Council. Once adopted, the Cyber Resilience Act will enter into …”

However, for the like of folks selling products, services and/or support for software based on RISC OS, that is going to be most likely either the end of the line on the EU market OR the adoption of their complex process and part nothing bad happens, because RISC OS can’t be fixed.

For the free stuff, if used free and/or to build free tools and no paid support added, that should be fine (at least for the time being).

This means that the bounty scheme should be okay as it is for improving the software (though the administration bounty isn’t), however here at ROOL various things are sold (such as the DDE), which risks annulling the “open source” exemption.

The bounty could be seen as funding and so fall in that category.

In general, if any money transaction is involved directly or indirectly, then the interested parties should seek for legal advices when the regulation is fully definitive and passed (there should be a certain amount of buffer time before it becomes official). We are not lawyers (unless some of you are).

The conclusion I have come to, given the ambiguities and that fact that lone programmers / small teams are treated exactly the same as massive multinationals is, sadly, that I would log into the heyrick server and systematically identify files that contain programs and delete them.

You are obviously free to do whatever you think it’s good for you.

In my case, for now, no changes on the horizon. The repository on GitHub has plenty of process to report and determine vulnerabilities and I am moving toward isolated execution. This, as far as I can see on the regulation is not a must for code that does NOT involves economical transactions of any sort and form. So plenty of tolerance.

But as pointed out wisely by Steve, your approach seems a bit drastic. However, I concur that the regulation seems heavily influenced by lobbyists and made for large companies to specifically impact small business and Open Source derivative work (which in many cases is used by many Open Source contributors to pay their bills, given a lot of Open Source users are just free loaders or behave as such) and for that I am not happy at all with the way this has been implemented.

@ Steve

Two different products. OS and application.

Word != Windows1

Yes and No. In the sense that:

a) The regulation, at the moment at least, covers every single software element, included drivers, libraries, services. So, you are correct in your definition, but no, that definition does not apply to the specific “wording” used on the regulation.
b) Even is Word != Windows, it Word allows privilege escalation vulnerabilities on Windows, then that’s fully covered and considered a problem that needs fix. On RISC OS, everything can have access to everything, hence the OS IS the problem and the Application or Library or driver IS the problem. Does that makes sense? Anything that becomes an attack vector will need fixing, which on RISC OS most likely will be an hard fix in most cases. Sure you can fix a buffer overflow, but on RISC OS, to gain control, you don’t actually need a buffer overflow.

Can BBC BASIC EVAL function evaluate a string that inject code? Yes, here is a quick example written in 3 seconds, represent every single program that may contain a function that runs some cli command:

REM Demo BBC BASIC code injection vulnerability
:
ON ERROR ON ERROR OFF:REPORT:PRINT" at line ";ERL:END
:
xarg$=""
SYS"OS_GetEnv" TO xarg$
x$=""
IF LEN(xarg$)>0 THEN
  IF LEFT$(xarg$,11)="BASIC -quit" OR LEFT$(xarg$,11)="BASIC -load" THEN xarg$=MID$(xarg$,13)
  IF LEFT$(xarg$,1)=CHR$(34) THEN REPEAT:xarg$=MID$(xarg$,2):UNTIL LEFT$(xarg$,1)=CHR$(34):xarg$=MID$(xarg$,3)
  PRINT xarg$
  x$=EVAL(xarg$)
ENDIF
:
PRINT x$

END

DEF FNCallSys(cmd$)
  OSCLI(cmd$)
=""

Now save it, call it test and then run:

*test FNCallSys("cat")

And obviously you’ll be able to run a system command. This is already soo bad for the EU CRA that ROOL will be forced to fix EVAL in BBC BASIC which it’s going to be extremely hard to fix for all use cases.

I used cat to be nice lol, but the same on BBC BASIC on Linux will have to respect the Permissions Model (described in my previous post). There is obviously more, you could call function that runs a SWI or whatever.

Given there is no Permissions Model (and not reinforced) you can even use this to modify (and so inject malware) on other people’s programs… I mean, Druck is right, this is something that can’t be handled on RISC OS. Certainly not in a short amount of time.

make peace with your opinions ahah ;)

You’re a lawyer? ;) <— note smiley!
Right now opinions are all there is until the final version is published.

That two people can read the same thing and arrive at different outcomes…might mean the document is just badly written.

I would like your opinion on my comment regarding the software that connects to a MIDI piano – it seems to me that the proposal is so broad that such a thing would be within scope, even though it’s patently ridiculous.

The bounty could be seen as funding and so fall in that category.

I called out the admin bounty as the rest are for improving the OS, and “other than exclusively improving the security, compatibility, or interoperability of the software” would imply exonerating the bounties from being “commercial”. Tea cups, on the other hand… 😪

You are obviously free to do whatever you think it’s good for you.

I don’t think it’s good. But between this and the forthcoming liability thing, I think it’s time to say that I’ve had a good thirty five years (and so has RISC OS). But just as the cessation of 32 bit is writing on the wall for the OS, this is writing on the wall for my software releases. I’ll probably still tinker around with things (like my sequencer), but I just won’t be releasing stuff as of when the CRA, in it’s current incarnation, is passed.
It needs exemptions for non-commercial software, hobbyist programmers, and the sorts of categories that will end up getting shafted by the requirements of this directive. Given that this reeks of lobbying money at work, I won’t hold my breath.

Make no mistake, deletion is not an option that I want to do. Even if I drop dead tomorrow, I’d like my stuff to linger around in case it’s useful. But my interpretation of this makes keeping it around (even if I’m not selling anything) simply non-viable.
And since I live in the EU, don’t think a “not for use in the EU” message is going to get me far…

I think it’s pretty clear that the only way this will be fixed is for those unfortunate enough to still be under the yoke of the second Holy Roman Empire (sorry watched The Omen again last night) to get their politicians to do something about it before it’s too late for the EU software industry.

That two people can read the same thing and arrive at different outcomes…might mean the document is just badly written.

Absolutely true.

I would like your opinion on my comment regarding the software that connects to a MIDI piano – it seems to me that the proposal is so broad that such a thing would be within scope, even though it’s patently ridiculous.

Actual likelihood of RISC OS being pwned from a MIDI compliant piano using MIDI events? Uhhh… let me think about that.

Indeed there are MIDI messages that are strings. Your driver is written in C (I haven’t looked at it in a while, other things kept me busy), so ensure your code is not subject to both buffers overflow and (most importantly for RISC OS) stack overflows that can be triggered using C strings. Literally Strings in C are No’Mans Land and C (included C99) does have dangerous functions, thes includes functions like strncpy (note the N).

So yes, it’s in scope AFAIU too. The thing is, MIDI isn’t something that is reachable from outside (where outside means another computer on the net or the internet), so I am not sure how easy to exploit it would be (certainly a MIDI player is more useful in this context)

I’ll give it a check tomorrow if you want. But yes in theory it is possible to hack a MIDI driver through malformed MIDI messages (it depends how it’s written). What matters in this discussion is that RISC OS requires no privileges escalation, so a simple stack overflow can bring an attacker to gain complete control of the system.

It is however possible to write perfectly safe C code, it’s just not intuitive and, certainly not fun.

The problem is the OS design.

Absolutely, I have mentioned this to exhaustion, it has (at its core) the same design Acorn MOS had, it wasn’t meant to be a professional OS, it was meant to be a port of Acorn MOS and the team had to do it quickly too. I mean, they made a miracle for the time, but now it ain’t anything we should use for any serious application.

Which leads to the thoughts I am having at the moment: What about toys? I think there might be a way around this mess by classifying it as a toy or something that is NOT meant for general and professional use or something like this. Again, I am not a lawyer, but I don’t think EU CRA covers toys as well, I am guessing thought.

theory it is possible to hack a MIDI driver through malformed MIDI messages

From a piano? ;)

to do something about it before it’s too late for the EU software industry.

Option B: Popcorn!

Permission Model: RISC OS’s lack of a robust permission model is a significant hurdle. We need a way to define and enforce what apps can and cannot do, including their access to files, networking and writing capabilities. While I’ve incorporated this into my VM, again the VM it’s most likely not a solution for older applications.

One of my favourite things about RISC OS is the way the user is involved in moving data around. I want programs to know what I’m willing to tell them, not give them carte blanche to scan every image or know every contact on my device. I know in practice that’s what happens, but it can be changed.

From a piano? ;)

From another MIDI device, but yes, in the case of your MIDI driver (as I have mentioned) it’s not something that is convenient to use as an attack vector. Normally folks get hacked via Files, network and USB devices. But in the case of your USBMIDI it’s easy, create a MIDI Device via an Arduino:

https://docs.arduino.cc/tutorials/generic/midi-device

And use it to stack overflow your driver. However no one will do this, because RISC OS doesn’t even has a login to bypass, so, as I said, you’re fine. But yes, if a Windows MIDI driver is found to have a vulnerability the above hack will work.

not give them carte blanche to scan every image or know every contact on my device. I know in practice that’s what happens, but it can be changed.

Yes it “could” be done, but it will require a lot of changes and work.

I did some experimentation and came to the conclusion that writing my own virtualized environment it’s easier and quicker, and keeps full compatibility with the past if needed. As a matter of fact, it’s already working and adds hybrid preemptive multi-tasking too, which is nice. All of this in few hundreds KB of executable, so not bad and the VM builds for Linux, Windows, macOS and BSD as well, so byte code is zero-effort portable for now and the interpreter is more than twice as fast as the BBC BASIC interpreter. Much faster than Python too.

I could have used RiscLua or Python, but they would have needed a lot of changes to implement processes and a security model, so I went for my own design instead.

You can try to modify RO, just keep in mind that the API has to be left untouched for old software, so that will give you some puzzles to solve and the assumption is that an App can access kernel space and vice-versa, so you’ll need to do some serious “byte-jokey” work to make everybody happy. Add Process understanding to the Kernel and move it away from the WIMP, add a security model for Modules too (they are not strictly been used only to extend the OS, so you have plenty of Apps using modules as some sort of DLLs basically), Threads maybe (Jeffrey has already done a lot of work for threads). But, as you said, it should be feasible for someone willing to put his head down and refactor a lot of portions of the OS.

Everyone seems to be focused on the software Side of this proposed new legislation but correct me if I am wrong this seems to be a way to shutdown all forms of making money from the internet that the government of the day does not like, so if you are a content provider that goes against the agreed narrative they will use the legislation to shut you down. It’s the end of freedom of speech, to go on YouTube or rumble to say what you like and make money. George must be turning in his grave, just one more nail in the coffin of free speech as well as the end your sovereignty.
Just a thought for you mull over as your security cameras are watching and recording. Feel free to comment while you are allowed to as no money has changed hand in the making of this drivel.

This is just about Agile, feel free to skip.

Is this a weakness of agile in that the goalposts can keep moving, or just a manager that failed to pin the customer down on exactly what they wanted and fully document that prior to writing any code?

The whole point of agile is that you intentionally don’t try to “pin the customer down on exactly what they wanted and fully document that prior to writing any code”; you start with a sketch of what they (think they) want and improve on it as you’re going along.

In our particular case it was a weak communication of the Product/sales department with the customer. I always had the impression they were making those requirements up without consulting the actual customer. So it would have gone bad regardless the process, however Agile did help to correct the trajectory even in such an unfortunate case.

In that case it wasn’t a problem with Agile, it was that you weren’t doing it at all! There should never be a department between the team and the customer; they’re supposed to talk to each other as they’re going along. AIUI, anyway.

I think we are all aware of the likelihood of any of that happening. A far more realistic solution is to add the following text:-NOT FOR SALE OR USE IN THE EU

I can already see the new slogan (or “claim” as the marketing droids say nowadays in Germany): “RISC OS – The Operating System that was saved by Brexit”.

“RISC OS – The Operating System that was saved by Brexit”.

More likely, “RISC OS – The Operating System that was PRESERVED by Brexit”.
That’s “preserved” as in an insect embedded in amber – you can see what it was, but it no longer functions, or shows any sign of life.

Pessimism aside, coders can always throw code up and leave people to do what they will with it.

I don’t like throwing anything up, makes bits hurt. 🤮

“RISC OS – The Operating System that was saved by Brexit”.

Maybe we should apply for UK Government funding: I believe they’re looking for positive things to have come out of Brexit at the moment… Who knows, they might pay us to 64-bit it…

Another “traditional” element of Agile methodologies are patterns, from design patterns to service patterns and architecture patterns.

Patterns, no matter if design patterns or architecture patterns or testing patterns or UI patterns (every Style Guide is a description of resuable UI patterns) or whatever are completely orthogonal to “Agile”. They are used everywhere in engineering, and in Software Engineering since the first subfunction call was implemented (i.e. the late 50s probably).

Of course the term “patterns” was probably not in common use before the publication of the famous GoF book, but this was firmly aimed at object-oriented design and reusability and predates the “Agile” movement by nearly a decade.

But maybe you meant exactly that with “traditional”, which would prove my point that Agile is really just a freshly mixed bag of methods that were in use in practice for a long time.

Those are all re-usable elements, which become fundamental, when someone realize that the true “knowledge” in Agile is knowing that Requisites WILL change.

Nearly every development process ever seriously defined in Software Engineering deals with “changing requirements”. You have read the original Royce work on the waterfall model (without actually naming it “waterfall” IIRC)? Or Boehm’s spiral model?

Testing has been an important part…

True, but it used to be a separate department, not the developers writing the test before the code.

In many circumstances, QA is still a seperate department (and there are good reasons for it). Whether developers write unit tests and/or integration tests and/or end-to-end tests and if they do it before or after writing the code is largely irrelevant in the grand scheme of things. TDD as a practice is usually a very short-cycled thing and I have never seen it used successfully for anything other than small-scale unit tests. I tried it as an experiment for UI code and found it very inefficient.

the Agile/Scrum/XP guys pretended that this is a new idea while they were just reinventing the wheel one more time

Ermmm please let’s avoid these unfounded and personal opinions.

Unfounded? Please substantiate this rather hilarious accusation. I have vast experience with people from the Agile fanclub, some of them were even paid for their questionable advice on “Agile” transformation.

Nearly all of what Agile/Scrum/XP brought to the table was already known and used before, sometimes (like unit testing, without calling it “unit testing”, because “unit” is a rather bad word for what Kent Beck really wanted to describe because it is too overloaded – not that anyone came up with a better word for it!) these things were taught already in IBM advanced programming courses in the 70s for COBOL and PL/I.

I was there at various developer conferences during the Agile hype, and I can assure you that countless Agile evangelists tried to paint “existing” development techniques in the worst colours and erected their strawmen e.g. about the classic Waterfall process and what were “typical” methods used in traditional Software Engineering. Agile to the rescue. A decade before Agile, it was the same pattern but with the OO hype.

To be clear, no one in the modern Software Engineering claims they invented software testing or want to take credit for ALL the software testing techniques invented over the years.

This is correct, and yet it was you who wrote “And Testing is another very important aspect of modern Agile methodologies” which triggered me to write my clarification. Which you omitted in your quote for reasons unknown.

Here is a brief history of Software Testing (and WHEN it became an important part of..): https://www.geeksforgeeks.org/history-of-software-testing/

Altough this ignores large parts of commercial software development happening mostly in the “big iron” part of IT – which is not surprising, since the academic circles usually have little overlap with that section (I studied for 6 years and got to know all kind of esoteric systems like VMS and MASPAR and Cray and Paragon and Tandem NonStop, but didn’t hear a word about IBM S/360 and successors), which is a shame because it would have considerably reduced some wheel-reinvention – it largely agrees with my point. Although I find the “era” designations a bit arbitrary.

Can you provide evidence of where Agile methodologies authors claimed that testing was a new and original idea? Because this is where Steffen’s incorrect assumption is.

It was not my claim. My sentence was just a clarification to your rather misleading claim about testing.

Maybe you just misunderstood which kind of people and what kind of statements I meant when I said “the Agile/Scrum/XP guys pretended”. Obviously, this DOES NOT mean that someone literally said or wrote “Hey, we invented testing”. Instead, they pretended that the old methods did something in a very different way and that their new way was somehow superior. But they both described the old way wrong and failed to provide reasoning why their specific new way was better. And typically recommended a “one size fits all” approach for vastly different settings. Typical example: Scrum Sprints and how the Scrum Team should be sized and structured.

Before Agile methodologies both these two activities were done in specific (and late) phases of a project and generally not by the developers.

Very few “Agile” setups let the developers do testing/QA on all the levels, mainly because developers usually often don’t have the necessary testing/QA skill, and one golden rule of software development is that it is a bad idea to test your own code. Especially true in highly regulated areas like finance and insurance, where there is extra QA by the customer after the product is delivered. And very early iterative development processes did testing early, and with the same mixture of developers and testers/QA/customers doing the verification. This was a standard approach in the mid/late 90s when I started my professional software development career.

Even in classic project planning, the testing activities started when the specification was ready, so basically at the same time or even before the first line of code was written.

One basic rule of Software Engineering since is inception was “find errors as early as possible, because the later you find them, the more costly it is to fix them”. Which is a large part of what the “Agile” approach tries to tackle. Like many approaches before.

which would prove my point that Agile is really just a freshly mixed bag of methods that were in use in practice for a long time.

As far as I can tell your initial point was accusing the Agile movement of stealing credit for testing techniques, which I provided evidence they have never claimed that, so are you doing another “Steffenism”?

or whatever are completely orthogonal to “Agile”

I see, you literally haven’t got a clue of what I said…

Everyone is entitled to believe whatever they wish, you shared your opinion, I provided evidence of complete absence of the claim you suggested the Agile movement did (bad boys, bad). Now if you have proof of your allegations, please share links I’m very happy to read. Otherwise you’re wasting your time. I haven’t even finished reading your comment, ’cause there were no links to documents where these so “bad” Agile boys claimed the stuff you seem to believe.

Agile put emphasis on iterative cycles, collaboration and reusability of code, group working to ensure good communication across the team. You don’t like it? Then don’t do it mate ;)

I leave you with a diagram, they help a lot. This is a diagram of one of the most used traditional methodologies (pre-Agile):

If you can’t understand this either, oh well I tried…

Indeed there are MIDI messages that are strings.

Where?

The closest one gets to data is SysEx and that has clearly defined start and stop parameters (and it’s easier with USB MIDI because in serial SysEx and realtime messages can interrupt other messages).

That being said, the MIDI protocol is pretty tightly defined (high bit set means control of some sort, high bit unset means it’s a days byte) and if anything goes astray then simply start tossing data until you get to something you’re expecting.
Again, it’s easier with USB as it is all based around four byte packets that are pretty much the 1-3 bytes of traditional MIDI with an extra byte to specify additional things in USB (like the use of “cables” so we’re not just limited to 16 channels any more).

The whole point of agile is that you intentionally don’t try to “pin the customer down on exactly what they wanted and fully document that prior to writing any code”;

I, briefly, many moons ago, worked with a badly specified project where the customer didn’t know what they wanted and kept changing things.

To my mind, it’s a terrible way to engineer anything. You wouldn’t construct a building without knowing how many floors it will have and what it’ll be made out of; and you wouldn’t design a car without knowing if the engine is in the front or the back, the drive is in the front or the back, or…

Likewise, surely if you’re going to create a software product for a customer, the design and iterative stage should happen long before coding begins. Sure, allow a little bit of leeway, but there shouldn’t really be any surprises. Coding shouldn’t begin until the problem has been analysed, broken down into steps, the formulæ tested, the customer given mock-ups, etc etc.

It probably shouldn’t be a surprise that the project was a catastrophe. ;)

I’m as guilty as the next man at writing a program starting with an idea in my head and refining it as the code is written, but if it’s something primarily for me then that’s fine. If it was for somebody else, as a paid creation, I’d want the program to be fully designed (and the design approved) before a single line of code was written.

Another example. About fifteen years ago I offered to help a local expat make a simple website to exhibit his photographs. I discussed what he had in mind, made some drawings, turned them into simple mockups, and so on.
After he said okay, I went away, and put together a program (written in C so it would run under my RISC OS and his Windows machine) that would scan a directory of prepared images and cross-reference them with a simple text file giving descriptions, and generate the HTML for a web page. In this way, all be needed to do was shrink a photo in his photo editor and add an entry to the description file to update the page with the new photo.
I had already thought ahead and implemented the creation of category sub pages which was useful when he realised that fifty images on one page would be painful using the dialup of the day. But although it got done, he eventually “fired” me and tried to stiff me on payment because he only know what he didn’t want, but was completely clueless to what he did want. That black background was the wrong shade of black. This text should be here, no move it back, no it looked better when… and so on, for damn near everything.
Even though there was a plan, it was followed up by an actual HTML mock-up (written in Zap), and the entire thing was “approved”…
…either he secretly wanted it done for free or he really had no idea.
In the end I got paid half. So I left him the created website and the program to make it. I doubted he’d have been able to use it without instructions, and the instructions hadn’t yet been written as we were still trying to tie down what it was he actually wanted (and modifying the program accordingly).
I had TurboC++ and the source on a floppy disc (as I used my RiscPC with copro at home) and as soon as I left for the last time I asked mom to stop at a nearby house, where I tucked disc into their wheelie bin. It would have been collected later that day. I wanted nothing more to do with the project, or him.

If Agile works for you, great. It just seems far too much like “make it up as we go” for me.
And, no, I never volunteered to make a website for anybody afterwards, even though a few offers came my way. It just wasn’t worth the hassle.

Bonus reading: https://heyrick.eu/blog/index.php?diary=20141102

I sympathise, but there are ways to negotiate contracts that avoid that sort of problem, ranging from time and materials to fixed price, based on initial estimates of user stories, where the customer has to drop as many estimated hours’ of stories as they subsequently demand in changes.

Where?

Some examples of text fields:

Marker
0xFF 0x06 length text

Cue Point
0xFF 0x07 length text

Program Name
0xFF 0x08 length text

Device Name
0xFF 0x09 length text

There are more IIRC.

I, briefly, many moons ago, worked with a badly specified project where the customer didn’t know what they wanted and kept changing things.

You’d be surprised of how common this can be, but it’s not because customers are “bad word”. It’s normal that people (especially in large conglomerates) don’t have a full knowledge of how things truly work (this is normally called “Data Flows”, just for the record), and so when someone has to design an entire process it can take some serious effort (this is why previous methodologies had issues). Agile can have issues too (nothing is perfect in this world), but it’s designed to have such process explored while building something at the same time. Now this actually can help a lot a customer to figure out what they really need (again remember want and need in this dimension can mean completely different things).

In the case I described for example, Simon is correct, we did NOT use an Agile approach, it was just management insisting it was agile, but it was just a bunch of folks trying to hold onto their relevance in a project to the point it started causing problems. This is something that can happen of many reasons, like lack of competence (of course, this is alway possible), but also lack of experience or fear (especially in companies where people can get laid off very quickly) as well as distrust on the engineering team, hidden goals and a lot more.

Likewise, surely if you’re going to create a software product for a customer, the design and iterative stage should happen long before coding begins.

Not necessarily, for instance, if a customer needs a byte code VM for a custom language (yes this can be a thing in high secured environments, because if someone doesn’t know how some code actually is encoded and executed, can’t hack it very easily right?), so you already know a lot of things needs to be put in place in a certain way and design can start, because reusable means also that portion of your stuff will be used in multiple projects.

For instance the definition of your byte code in that example will be a consequence of a set of requirements, yes, so late start, but the memory management won’t so you can start that earlier.

In the case of a music application, the decision of adopting MIDI files is going to be pretty much a standard, so you’ll already put in place libraries that handles this for you and in the language you decide to use, this regardless of the ongoing process to decide how to store the music notes.

In a web application based on micro service architecture is the same, the secure authentication between components is something that you already know how it has to be built.

While in the case above, you may start with a customer tables that is denormalized and then normalize it later when you have a better idea of what the user wants to do with that data. There are also different analysis processes like starting from the UI, so you’ll derive the relationships between entities based on how the customer wants to interact with the UI and organize the input.

So, as soon as you have a mockup of a customer input UI, you can immediately start designing the table that will store that data. But again these are just examples there are more approaches possible and, again, Agile methodologies are about to make your team agility to adapt to changes better, so the assumption is always that things WILL change.

Now this is a good approach even for projects that need maintenance, because what was good in the 80s (I don’t know, think of RISC OS ;) ), it’s not sufficient anymore in this day and age, so the requirements have changed, right? ;)

HTH