Updated port of SQLite
|
There was some discussion last month around SQLite and getting it updated. I’ve taken the old port of SQLite 3.3.6 created by The Really Small Software Company and have updated it to the latest version of SQLite (3.42). It’s built with the DDE and should work back to RISC OS 3.1. The port is available for download here or via PackMan. Source is here (you may wish to read the BuildNotes file). Version 3.42 (formerly known as beta 8 – if you already have b8 then you don’t need to redownload this one)
Changed in beta 7
Changed in beta 6
Changed in beta 5
Changed in beta 4
Changed in beta 3
Changed in beta 2
Changed in beta 1
Fixed in alpha 13
Fixed in alpha 12
Changed in alpha 11
Changed in alpha 10
Changed in alpha 9
Fixed in alpha 8
Changed in alpha 7
Changed in alpha 6
Fixed in alpha 5
Fixed in alpha 4
Changed in alpha 3
Fixed in alpha 2
Notes
Please report any bugs in this thread. There are certain to be some I haven’t found! Thanks to Really Small for the original port; I probably wouldn’t have got this far without that as a baseline. |
|
IIRC the main issues we had with latest sqlite in Iris (it uses it for various local storage things) related to re-enterancy and handling of multiple instances. I guess for non-ported apps (generally DDE’s strength) it wouldn’t matter, but is there anything here that might help with such things? |
|
It’s a pretty basic port at the moment, so doesn’t have much in the way of handling RISC OSisms. The SQLite docs actually advise against having multiple instances so I haven’t done anything special to handle them. Presumably Iris is using UnixLib and the “Unix version” of SQLite. Functionally it’s probably fairly similar to this new one. |
|
Alpha 2 is now available, complete with source code. The first post has been updated with details. |
|
very nice job @ Chris! :) |
|
As a little interim update, here is an up-to-date version of the command-line shell. This is shell 3.35.5, as opposed to the older version included with alpha 2. The underlying SQLite library is the same as alpha 2. Next on the agenda for the main library itself is to get the automated tests working. I’ve run some of them manually and none of them have failed, but I haven’t yet automated them. There are a lot, so automation is a necessary step! |
|
Alpha 3 is now available (see first post). This is based on SQLite 3.36, and released on the same day as the non-RISC OS version :) |
|
You’re on fire. Nice work. |
|
I haven’t had much free time over the past couple of months, but today I finally got this up and running. Unfortunately there are some failed tests, so this library shall retain the “alpha” tag for now. I’m not sure when I’ll have time to fix up the issues, but will post any updates in this thread. |
|
I fixed1 an issue and a number of the failed tests now pass. Unfortunately it didn’t fix all of them, so I’ll have to delve deeper at some point. The sorts of things that are failing are edge cases with SQL that nobody in their right mind would actually write:
Nevertheless, I want to get all the tests working for the confidence that everything behaves correctly. 1 It’s really a workaround rather than a fix; I had to disable CSE optimisation to get it to work. I’m not sure whether it’s something non-standard in the code doing it, or whether it’s an issue with the compiler. I simply noticed that it worked fine2 in DDT but not when running a ‘real’ build. 2 And by “fine” I mean “the file’s so large that DDT starts truncating line numbers once you pass line 99999 and then crashes on quit”… |
|
In my experience, that is often something using a value that it expects to have a default “unset” value of null, which isn’t the case in the real build. For example, if you have: int x; int y; Then at runtime, until I always declare as: int x = 0; int y = 0; even if three lines down I set it to be something different. Always begin with a known state, or relying upon any sort of state that you can’t guarantee will eventually give you headaches. |
|
FTFY: When built for debug, they will be zeroed. On platforms where you can run debug builds without a debugger, this catches out a lot of people when they finally make a release build. At least with Norcroft, having to run under DDT ensures no one ever builds for debug if they don’t need to. |
|
The problematic code is (or rather, appears to be) part of SQLite itself and not my ‘interface’ code, so I don’t want to make any changes to it. The last thing I want to do is introduce strange new bugs :) |
|
I finally stopped procrastinating and had a look into this. First of all, here’s the actual failing SQL test case:
This is supposed to return nothing, but instead returns -544.4. The underlying issue is that SQLite calls Even more interestingly, if I separate the code off into its own file (nantest.c) then it works as expected! Can anyone shed any light on this? I don’t really want to modify the core sqlite3.c file if I can help it, so I’d love to understand what’s going on here. |
|
The reason the smaller/faster version doesn’t work is because the order of words in an FPA I’m also not convinced when you say
as -280366 looks pretty number-y, and not at all nan-y to me. |
|
You’re quite right! Post updated to match reality :) Thanks for the explanation. |
|
Upon further investigation, I’ve had to add a new entry to the Known Issues list.
Hopefully this will be something I can fix, but it probably won’t be any time soon. I expect to release Alpha 4 shortly, which makes use of |
|
Alpha 4 is now available. This is a milestone release, as SQL Logic Test now passes in its entirety. I forgot to update the BuildNotes text file in the source archive: sqlite3-in is documented as being an untouched copy from SQLite 3.36. This is not correct, as it contains the isnan() fix mentioned above. This fix should be included in SQLite 3.37 so a hand-edited file should no longer be required in the future. |
|
It ended up being trivial. So, hot on the heels of alpha 4 comes alpha 5. Details in the first post. Please note that this version will not correctly read databases created with alpha 4. If you need to retain existing data then you should open your database with alpha 4, |
|
While some simple actions (creating a DB, adding data, reading it back) are successful under 3.1, some of the test scripts are failing under ArcEm when using HostFS. I don’t yet know whether this is a SQLite issue, or an ArcEm issue. I also tried using ADFS under ArcEm but this crashed. The SQLite tests all pass under OS 3.6, but at this stage I’m not ‘certifying’ the port on earlier OS versions. I tried running under Arculator on Windows 10 but couldn’t figure out how to get the mouse working. There don’t seem to be any settings for it and Google didn’t return anything useful! I can see the mouse pointer, but it moves in steps of ~64 pixels and therefore I can’t click anything on the icon bar as the pointer just skips right past it. Has anyone else run into this? |
|
I can now confirm that this is an ArcEm/HostFS issue. I’ve only run a limited number of tests under 3.1 but it’s now returning correct results when using ADFS :) |
|
Nice! BTW Chris, would it be ok if I upload both the HTTPLib and the SQLite ports on the RISC OS Community on GitHub? I am working on create a cross platform build (that runs on RISC OS too) to build code that also pulls down dependencies. Given that I am using your HTTPLib and the SQLite port for a couple of projects it would be nice to have the source there (mostly handy). Thanks for this effort BTW, really cool to see SQLite working on RISC OS 3.1 as well! |
|
That’s fine, but please put a note on there making it clear that that’s not their official home. One of these days I need to finish off HTTPLib… |
|
thanks and of course I’ll make sure it’s displayed that those are only mirror repositories, thanks a lot! :) |
|
Alpha 6 of SQLite for RISC OS is now available from the usual place, based on SQLite 3.37. There are no RISC OS-specific changes. If you’re building from source, please note that the “amalgamate” utility has been updated, and that the new version is required when using 3.37. It’s also compatible with 3.36. |