Python 3.8 - alpha release

Surely if Python requires CryptRandom (and any other modules) they should be RMEnsured in the !Run file?

Surely if Python requires CryptRandom (and any other modules) they should be RMEnsured in the !Run file?

Yes, although !Run just launches it in a taskwindow. You can also run a python script by double-clicking on it. I am not sure about RMEnsuring things in the !Boot file, but that’s where it sets up the run action.

If the !Boot file is changed to:
set Alias$@RunType_a73 Run <Python3$Dir>.!Run %*0

And the !Run file is changed to:
add any RMensures at the beginning
TaskWindow "<Python3$Dir>.python3 %0" -name "Python" -ctrl -quit -display -wimpslot 16M

Does that achieve what is required?

If the Alias$python3 is required to just run a script without starting a TaskWindow, then perhaps another obey file could be used which also included the RMEnsures.

I may be barking up the wrong tree, knowing nothing of Python!

If the !Boot file is changed to […]

That was going to be my suggestion. I was trying to think of a way around the conditional use of -display, but thinking about it, I suspect it might not matter (it could probably be left out of both).

One further observation: !Boot should not be unconditionally setting any system variables. !Run can do this; !Boot must only set them if they are currently unset.

+1 to Steve’s post immediately above – that’s quite important.

Did we come to any decisions on whether Python3 will have different filetypes to Python2? The only counter-argument would be if Python 3 was going to use the same filetype and claim precedence over Python 2. However, IIRC Stefen made a fairly convincing case for separate filetypes due to version incompatibilities.

Simpler to just RMEnsure those modules on which it depends as the first thing in (RISC OS) Python’s main() if the executable is to be used directly to execute Python code. Or better still (specifically for CryptRandom), in UnixLib when /dev/urandom is accessed for the first time?

!Boot must only set them if they are currently unset.

Unless, of course, you are the only “owner” of a file/namespace. Which isn’t the case here. ;-)

+1 to Steve’s post immediately above

+2. It’s rather annoying how many things claim text, sprite, JPEG, etc simply by virtue of booting in alphabetical order. I find SwiftJPEG is quite picky about the JPEGs it will load, while PrivateEye isn’t… But S comes after P and I’m too lazy to muck around editing obey files…

However, IIRC Stefen made a fairly convincing case for separate filetypes due to version incompatibilities.

Yeah. So nice that the rest of the world gives a …. about backwards compatibility, eh?

How hard would it be to convert Python 2 code to Python 3 code?

Yes, that should work. I was thinking of if you wanted to run python without a taskwindow, but in that case you will have to do the RMensure stuff yourself.

Python 3 has its own type (since alpha 2 or 3) distinct from python 2, so arguably it does “own” the types.

How hard would it be to convert Python 2 code to Python 3 code?

Potentially difficult (or at least time consuming). That’s why Python 2 repeatedly got its end-of-life date pushed out until it finally expired at the start of this year.

Simpler to just RMEnsure those modules on which it depends as the first thing in (RISC OS) Python’s main() if the executable is to be used directly to execute Python code. Or better still (specifically for CryptRandom), in UnixLib when /dev/urandom is accessed for the first time?

I have a feeling that UnixLib already RMensures SharedUnixLib internally. The one that doesn’t seem to be getting caught is CryptRand.

How hard would it be to convert Python 2 code to Python 3 code?

Looking online, it seems as if Python 3 really ought to be able to load (for backwards compatibility) Python 2, because many sources will tell you that the print statement is now a print function (needs brackets) and that they fixed integer division to output a non-integer result when appropriate (ie 5/2 is 2.5, not 2). Plus strings are Unicode, not that I’ve can imagine that making much difference in RISC OS.
So it looks easy, right?

Well, they didn’t mention all the other things. Dig deeper and you’ll find https://www.digitalocean.com/community/tutorials/how-to-port-python-2-code-to-python-3 which shows a larger number of fundamental differences.

So, in theory it would be possible for Python3 to have some sort of compatibility loader, but in reality you just gotta convert it, which is less onerous than it sounds given that 2to3 exists.

So, in theory it would be possible for Python3 to have some sort of compatibility loader, but in reality you just gotta convert it, which is less onerous than it sounds given that 2to3 exists.

I was thinking of an automatic way of converting python 2 to python 3, and it appears that someone has already thought of it and got something working.

Is there anyone who’d be able to test my GPIO library? It’s a rough port of the RPi.GPIO library and implements most of the features from that. You’d need to have kit that uses GPIO and probably be familiar-ish with using the RPi.GPIO library so you can tell me what I’ve done wrong!

This is my first foray into writing Python code, so it’s probably very rough around the edges and I’m sure the “real programmers” on here would turn their noses up at it, but hopefully, for now, it’s better than nothing and will suffice until someone comes along and does it “properly”.

Drop me an email to a.m.conroy at owlart dot co dot uk

I’ve come across an issue which is not a problem with the port itself but with the Python standard library.

The “unittest” package works when invoked by running a tests script:

if __name__ == '__main__':
    unittest.main()

But it doesn’t when used in discovery mode: python3 -m unittest discover -s lib-python.mything.tests -t lib-python -p test*

It makes a lot of assumptions that file extensions exist and that they will be “.py”.

Obviously this isn’t an impediment on finalising the port and getting it out of the door but it would be nice if it were improved afterwards. Having looked into the unittest package I feel a little trepidatious suggesting it as that code looks pretty old and warty.

Yes, there is a LOT of stuff that assumes that python files are names something.py, or more generally extensions exist and are separated with a dot. It’s causing me a lot of headache with pip too.

It’s not helped by code doing stuff like name = os.path.join(“foo”,“bar”,“baz.txt”) – which on RISC OS will result in “foo.bar.baz.txt” instead of “foo.bar.baz/txt” (or even “foo.bar.baz” (type text). I did have a stab at making it try to be “clever” but there lots of corner cases.

Alpha 5 now available from https://github.com/c-jo/cpython/releases/tag/riscos-3.8.0-a5

How hard would it be to convert Python 2 code to Python 3 code?

It’s pretty straight forward for the majority of code, unless you like writing very tricky Python. There is a tool on Linux called 2to3 which does most of the donkey work, such as putting brackets around all the print statements. The bits it doesn’t do is convert some things which no longer return lists (e.g. map(f,a) needs to be list(map(f,a)) ) and that IO calls return arrays of bytes rather than strings so file.read() needs to be file.read().decode(‘utf8’) and file.write(string) needs to be file.write(string.encode(‘utf8’)).

Finally some progress on pip. I’ve managed to install (and uninstall) a simple package..

python3 -m pip install py_descriptive_statistics
Collecting py_descriptive_statistics
Using cached py_descriptive_statistics-0.2.tar.gz (1.8 kB)
Installing collected packages: py-descriptive-statistics
Running setup.py install for py-descriptive-statistics … [?25ldone
[?25hSuccessfully installed py-descriptive-statistics-0.2

python3
Python 3.8.0a5 (default, May 1 2020, 15:13:46)
[GCC 4.7.4] on riscos
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import py_descriptive_statistics
>>>

python3 -m pip uninstall py_descriptive_statistics
Found existing installation: py-descriptive-statistics 0.2
Uninstalling py-descriptive-statistics-0.2:
Would remove:
sdfs::chipmunk.$.work.cpython-pip.lib.site-packages.py_descriptive_statistics-0/2-py3/8/egg-info
sdfs::chipmunk.$.work.cpython-pip.lib.site-packages.py_descriptive_statistics.*
Proceed (y/n)? y
Successfully uninstalled py-descriptive-statistics-0.2

Still much to be done, but a step in the right direction.

Ooh. That is amazing news! You are doing a great job.

Alpha 6 now out: https://github.com/c-jo/cpython/releases/download/riscos-3.8.0-a6/Python38a6.zip

The new big feature here is pip. I am sure it will break in all sorts of ways, but that’s why it says ‘alpha’..

I’m trying this from a pretty fresh install due to accidentally snapping my MicroSD card in half, so my install may be missing some things.

So far it seems to need zlib1g and libssl to get pip up and running.
I can’t seem to get past it falling over on the !Scrap directory though.

Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory:
 'SDFS::HardDisc0.$.!Boot.Resources.!Scrap.ScrapDirs.ScrapDir.pip-install-eusvdt2q
.octoprint.octoprint.static.img.tentacle-20x20-light@2x/png'

No. I had no intent on using it as an OctoPrint server. I was just trying random packages that I could think of.

e: broke the output up into a few lines because code doesn’t wrap.

It’s not helped by code doing stuff like name = os.path.join(“foo”,“bar”,“baz.txt”)

I am not acquainted with the source code for Python, so I may be talking through my hat. In Lua’s source code the directory separator character is given by a line #define LUA_DIRSEP ... which makes life easier. But in fact there is only one place where Lua needs to know about filesystem syntax. The library-loading function require takes as its argument the leaf-name of a library to load. To find where the library is it uses #define LUA_PATH ... and #define LUA_CPATH ... for defining lists of templates. A ? sign in them stands for the leaf name. If these are undefined it looks up similarly named environment variables. In this way true portability is maintained (in its proper sense, not just portability between versions of Unix).

I have just downloaded the Python 3.8.3 sources to have a look. The sheer size (84 Mbytes) makes my head hurt – Lua is 2 Mbytes. I take my hat off to you Chris. It is fairly clear that anything beyond Posix or Windows did not exist for Guido van R. No porting of Python to tumble-dryers or food-mixers, then :). I guess it must be a heavy task just to filter out what cannot ever be ported to RISC OS, and what can with a lot of effort.

There are directory and extension separator characters in Python, I think Chris was commenting that they weren’t used appropriately here as there was an assumption that the extension separator is always a “.” (it is on every other system except classic Mac OS which didn’t have them.)