DeepSeek and BBC BASIC

It’s unfortunate that DeepSeek is usually too busy, but it understands BBC BASIC way better than ChatGPT. It wrote both a prime generator and even a Mandelbrot generator, both of which ran immediately without any editing on my part. ChatGPT’s Mandelbrot generator, on the other hand, gets many keywords wrong, mangles syntax, doesn’t distinguish between integers and floating points, forgets ENDIFs, etc. And even when the programme runs, after much editing, it produces something in black and white that is anything but the Mandelbrot fractal.

However, so far DeepSeek hasn’t yet managed to write a working assembly version of the fractal generator. But that is probably due to my clumsy instruction, as I certainly can’t write one myself.

I downloaded DeepSeek app to my Android phone.
It was very insistent that I sign up and pay several pounds a month. I was able to bypass the crud and make a couple of queries which it answered satisfactorily, but the third time it said I had exhausted my allocation of freebies.
So I uninstalled it.

@Paul how are you using it?

I’m surprised. I use DeepSeek both on my Android phone (as an app) and on my Windows computer. It didn’t insist at all to pay whatever money, and there seems no restrictions to the number of chats whatsoever, apart from the server being very busy. After all, doesn’t it advertise itself by being completely free?

Thanks Paul- I had downloaded a scam.
I now have the genuine article as far as I can tell.
It has beaten Gemini in its first test.
I asked DeepSeek if days of the week in Spain started with capital letters.
It correctly said no they did not.
Gemini on the other hand wrongly insisted they did.
Copilot agreed with DeepSeek.

but it understands BBC BASIC way better than ChatGPT

I can concur. My standard test is to ask for a simple raycaster, as it’s a pretty short program that will test its knowledge of how well it really knows BBC BASIC.

The result? I had to change MODE 12 to MODE 27, fudge in <<1 to the plot positions, and add a bunch of WAIT commands so it didn’t flicker to hell and back. These I can put down to differences between an Archimedes (that it probably would have worked fine on) and a modern Pi.
The thing it got wrong? The INKEY values for the cursor keys were gonzo. I looked up the correct codes in StrongHelp (sadly I can’t thank Steve :( for taking the time to list them all so I didn’t have to do the mental arithmetic from the internal codes).

Then? It worked. Dead simple, no frills whatsoever, and used a quirky small-hop method to make the maths stupidly simple, but it worked.
I have never had a program of any complexity (like, more than 10 lines) that wasn’t somehow broken on ChatGPT and/or threw in some random bits of other dialects of BASIC, or just lost the plot and started to make stuff up. In fact, the first time I gave that challenge to ChatGPT, the result was so broken that I gave up rather than attempt to work out what the hell the program thought it was doing.

This AI knows me, it knows my site. It… thought I had cancer and chemo. When I pointed out its error, it just froze for ages (hey, did it go back and reread my entire blog? ;) ) and then gave an update that was still not quite right but at least this time correctly identified who had cancer. I wonder if it’ll get it wrong next time?

I had downloaded a scam.

What email address did you provide? Did you give a password? Have you used that address/combo anywhere else?

I maintain a selection of “burner addresses”, one of which I gave to DeepSix… DeepStar… DarkStar… yes, the sentient bomb, I think I’ll just call it DarkStar from now on.

I now have the genuine article as far as I can tell.

Look at the app info and see who supplies it. When something new pops up, so do the scammers, unfortunately.

I asked DeepSeek if days of the week in Spain started with capital letters.
It correctly said no they did not.

I think it’s a romance language thing. It’s the same in French (and Italian, and Portuguese).
We English speakers probably got the habit from the Germanic influence. Germans like capitalising things. ;)

had downloaded a scam.
bq.. What email address did you provide? Did you give a password? Have you used that address/combo anywhere else?

I logged in via my Google account so I did not have to key in a password. I trust that Google would not be sharing it!

It was more of an elephant trap than a scam. Almost every clickable area of the screen led to paying for an account.

I maintain a selection of “burner addresses”, one of which I gave to DeepSix… DeepStar… DarkStar… yes, the sentient bomb, I think I’ll just call it DarkStar from now on

Aw, it’s so much more fun to use an address that goes through a cloud based mail filter and let that deal with the ramped spam by submitting the source address to multiple DNS blacklists :)
Or do you have that feature on your burner addresses?

Don’t fool yourself about what might have been going on in the background while the app was running on your phone.

Also, logging into an app via Google may be sharing some of your info with the app.
Go to https://myaccount.google.com/, find “Security” and choose it, then go down and ensure that there’s nothing unexpected under “Third-party apps with account access”. If there is, Manage → Remove (and note what it actually had access to – because revoking access to whatever it had means Google will no longer allow any access, the scammers however will carry on using what they got (your email address?)).

As I said, I used GMail/Yahoo! to set up some fake burner addresses that are valid but can be ditched if they become troublesome.
Some sites think my name is Oré Washinda. 😂

PS: No, my addresses don’t blacklist. They’re just an isolation so I don’t give out an address I actually use to all and sundry. I learned that lesson hard with my heyrick.co.uk mailbox (that’s the reason there isn’t one).

Quick note for DeepSeek users:

While I am still working to improve the knowledge GraphDB for DeepSeek, be careful using it, there was a discovery of exposed users data yesterday, so I’d recommend caution with it.

[edit]
More info about the recent security issue with DeepSeek for who has an account there: https://www.bleepingcomputer.com/news/security/deepseek-exposes-database-with-over-1-million-chat-records/
[/edit]

To ensure general use safety, I am opening up the BBC BASIC work on ChatGPT for who’s interested in giving it a test, but I am still working on the ToolBox knowledge and the WIMP programming, so both DeepSeek and the ChatGPT will have problems there, sorry. It takes a lot of time to generate code examples for these models.

As for who prefer DeepSeek, I am also working on a bundle of the RAG + DeepSeek R1 to run locally (for safety), but that requires HW that is above a Pi4, so not sure if it’s of interest here.

As of right now, it’s safer to use the same RAG on ChatGPT, but again: these are alpha quality, so if people don’t send me feedback to improve the RAG, progresses will be slow. I am also planning to fuel LLAMA 3.3 JFYI.

What do I need the most? The question you’ve asked, litterally, copy and past it on an email and possibly the wrong answer, so I can check where it started to go wrong (these are predictive models after all, aka transformers). That’s it. Helping me improving the knowledge GraphDB is 2 copy and past away. No, I am not building this for my self. As a matter of fact I am also publishing the data in the form of automated services, so the community will not lose anything.

Side (relatively) good news, also Google AI started to absorb RO data, so now googling for RISC OS SWI can have results in Google. But there I can’t control the quality or even fix issues. I have now started an automated service that publish SWI summaries on social networks for these people to collect data from.

As for who prefer DeepSeek, I am also working on a bundle of the RAG + DeepSeek R1 to run locally (for safety), but that requires HW that is above a Pi4, so not sure if it’s of interest here.

The only DeepSeek-R1 model released is the 671b model, which is going to need 1TB to run. I think you’re referring to the Qwen/Llama distilled models.

I’ve run the 1.5b/7b Qwen distilled models on my 5 year old laptop, which although not massively quick, did work – its CoT is quite long on these models though and it was constantly backtracking to try different routes. I switched to the Qwen distilled 32b model running on a 4090 and that was really quick, with fairly accurate results.

I skim read the white paper last year and was interested in trying a cold start with curated training from examples I produced programmatically. I’m not sure if/when I’ll have time to do that, but I reckon it could probably be turned into a useful tool to produce ARM or convert ARM to C/C++.

In the meantime, I’ve used it a few times to produce code for particular problems I’ve been working on. The smaller models weren’t great (1.5b / 7b), but the 32b model produced usable results.

I noticed with the smaller models, if asked for BBC BASIC it would start of with that, but usually ended up with a QBASIC solution. Likewise if asked for ARM3 the solution would either be x86 or ARM7. With the 32b model, it obviously faired better and when asked to covert an ARM7 solution to ARM3 it did a fairly decent job of it…certainly good enough for reference.

The only DeepSeek-R1 model released is the 671B model, which requires 1TB of memory to run. I think you’re referring to the Qwen/Llama distilled models.

I have developed a platform that can use all of them, as well as different model types, but it’s a commercial product. And yes, it is also capable of running R1. As for the hardware, I was being slightly sarcastic in this context. In the real world, all DeepSeek models are quite well-optimized compared to, for example, Meta’s LLAMA 3.×. So, a 64-core AMD system with a couple of well-suited NVIDIA GPUs is enough to run the model decently.

Regarding distilled models, there is also V3. However, as you’ve noted, working with distilled models can be tricky. (In all honesty, given the number of magic words, the “,,,” syntax for SYS, and other peculiarities of RISC OS, working with any AI can often be tricky!)

curated training from examples I produced programmatically

That is what I have been working on for the last few years. The biggest problem you’ll encounter is the “leaking” effect generated by statistically similar formats. You mentioned QBASIC, but in my experience, the biggest troublemaker is BBC BASIC for Windows. I noticed this while working on the BBC BASIC RAG I made available in preview last week. DeepSeek models (all variants) seem to respond better than ChatGPT when it comes to single-task BBC BASIC programs. This is probably because they contain less BBC BASIC for Windows data in their training sets.

Last night, I managed to generate the first correct WIMP program using ChatGPT (again, in my RAG, while standard ChatGPT is nowhere near writing a BBC BASIC program that makes any sense). However, in further tests, it started hallucinating mistakes again.

My DS-R1 results for WIMP and ToolBox usage are at similar levels, so no major breakthroughs so far.

I noticed with the smaller models, if asked for BBC BASIC, it would start off with that but usually end up with a QBASIC solution. Likewise, if asked for ARM3, the solution would either be x86 or ARM7.

I call this “statistical leaking”. It’s an issue caused by the nature of predictive models. BTW, this is exactly why they are NOT reasoning, and there is no AGI—that’s all BS to milk investors.
The problem is that each step in generating a response is determined by statistical significance. When programming languages share a lot of statistically significant similarities, the model can unpredictably switch between them. This isn’t necessarily hallucination; rather, the statistical weight of certain tokens influences the response. For example, the token PRINT is more statistically significant in datasets containing QBASIC samples than in those containing BBC BASIC (for obvious reasons, right? More examples!). Assembly is the pinnacle of this issue: mnemonics like MOV are so common across different ISAs that the neural network struggles to stay on track.

There are ways to improve precision, but they require modifying the foundational libraries used by the inference engine. To experiment and conduct R&D on this, I had to build an entire ecosystem from scratch:

https://paolozaino.wordpress.com/portfolio/zlnn-zed-liquid-neural-network-accelerated-lnn-in-rust-for-performance-safety-and-improved-precision/

So, it’s not a simple problem to solve when using backends you can’t control.

HTH