<Global/HalEntries.h> missing

I’ve been trying to build the settime module but it fails because the unixlibzm library cannot be found. This turns out to be because the making of unixlib fails while building ROOL.BuildEnv – or any other environment for that matter – so it isn’t exported.

It fails because c.gtimeofday can’t open <global />. A search of my sources reveal the only HalEntries file is in Sources.HAL.HALTester.hdr.

Is there something missing in the sources – I downloaded and merged mixed,gpl,castle,cddl and bsd from the CVS repository.

It sounds like you’ve grabbed everything from CVS manually – in which case you won’t have the right branch of the kernel. It’s only the HAL and Cortex branches which have the HALEntries header (if the header’s there, it should have been exported to the ‘Export’ folder during the export headers phase).

To avoid any other potential problems I’d suggest using the ‘checkout’ Perl script in order to grab the right versions of everything. Except I’m not sure which products file is most appropriate for building the settime module (and was that a typo for nettime? I’ve never even heard of settime!)

If you’re interested in using the ‘checkout’ script then there’s some info on it on this, this, and this wiki page.

Sorry, I did mean nettime.

Apparently I can’t use the scripts because I can only download from my Iyonix and the wiki page says that I need linux or cygwin to use the scripts.

I have tried tarballs and had no luck with them so I thought using cvs and downloading everything should do the trick – apparently not. I’ve read pages and pages of wiki – some contradictory and still no luck. Anyway I’ll plod on.

I won’t pretend I understand what you’re talking about but it if has anything to do with getting NetTime to run on the BB - try this:

ftp://www.a4com.dyndns.org/pub/Beagle/nettime.zip

Apparently I can’t use the scripts because I can only download from my Iyonix and the wiki page says that I need linux or cygwin to use the scripts.

Yes, I’m fairly certain that people have reported problems with trying to get the scripts to work on RISC OS. Perhaps I should have a quick look into it and see if it’s easy to fix.

I have tried tarballs and had no luck with them so I thought using cvs and downloading everything should do the trick – apparently not. I’ve read pages and pages of wiki – some contradictory and still no luck. Anyway I’ll plod on.

Can you remember which pages were the contradictory ones? It would be nice to be able to fix them!

wiki/documentation/pages/How+to+build+RISC+OS

This page implies that after extracting, the tarball will produce a directory which is the build directory. This isn’t the case in the tarballs I tried, the tarballs are a directory higher and the build directory is a merging of all of the RiscOS directories that these top level directories contain.

I’ve since found out that some of the directories extracted may be in the wrong place due to a tar problem – but but thats another story.

contents/documents/cvs-repository-guide

In the overview the list of directories in /cvsroot is different from the list of directories when browsing the cvs repository, This makes you wonder if you only need the directories listed on the page.

contents/downloads/cvs_access

Implies that after logging on simply doing

checkout some/file/or/directory

will get you the latest sources yet you say it doesn’t – I’m afraid I don’t know enough about cvs to know what I’m doing :). But if it doesn’t then the page doesn’t explain enough.

I’ve been at this for 2 days. Maybe I was just lucky in picking nettime to compile otherwise I may not have known that there is something wrong with what I’ve downloaded – Srcedit and Paint compile OK. The whole process is very discouraging.

Encouraged by Jeffrey Lee to abandon CVS on Riscos to get the Sources I went back to my original plan of downloading the TungstenDev tarball. I finally managed to get the rom compiled so I thought I’d pass on what I had to do – it may help someone.

For Reference I’m using Netsurf to download the files.

From http://www.riscosopen.org/content/downloads/other-zipfiles I downloaded a version on 2011-01-08 04:06:56

Download UnTarBZ2

This downloads a file called UnTarBZ2. Set the type to Utility and run it – this creates !UnTarBZ2. Run this.

Download TungstenDev

This downloads a file called src-iyo-dev.

1) Rename the file src-iyo-dev/tar/bz2

2) drag it to the UnTarBZ2 icon on the iconbar – the UnTarBZ2 control panel pops up.

3) Create a destination directory where you want the extracted files to go.

4) Drag this directory to the Dest field of the UnTarBZ2 control panel.

5) Click run.

Now you should only have a TungstenDev directory inside your destination directory but you don’t because of a problem with the Tar archive. To recreate how the archive should be:

1) Move castle into TungstenDev

2) Move mixed into TungstenDev

3) Move Autobuild into TungstenDev.castle.RiscOS.Utilities

4) Move RiscOS into TungstenDev.castle

5) Move Utilities into castle.RiscOS

6) rename TungstenDev.castle.RiscOS.Sources.Networking.AUN.AUNMsgs.Resources.UK.Resources.ShareFS.!Sprites,ff9000640 TungstenDev.castle.RiscOS.Sources.Networking.AUN.AUNMsgs.Resources.UK.Resources.ShareFS.!Sprites and set the filetype to sprite.

7) rename TungstenDev.castle.RiscOS.Sources.Video.Render.Fonts.Manager.Utils.!FontEd.Sources.Test.FontFile,ff6000640 TungstenDev.castle.RiscOS.Sources.Video.Render.Fonts.Manager.Utils.!FontEd.Sources.Test.FontFile and set the filetype to font

8) rename TungstenDev.castle.RiscOS.Utilities.Autobuild.ABRelease.Resources.Tungsten.soft.!SoftLoad.!Run,feb000640 TungstenDev.castle.RiscOS.Utilities.Autobuild.ABRelease.Resources.Tungsten.soft.!SoftLoad.!Run and set the type to obey.

You now have the files as they should have been extracted.

You need to create the build environment from this as follows. Into the destination folder you created, which should now only contain the TungstenDev directory, drag

TungstenDev.bsd.RiscOS

TungstenDev.castle.RiscOS

TungstenDev.gpl.RiscOS

TungstenDev.mixed.RiscOS

You now have a working directory called RiscOS containing all the files in the right place for building.

To set up the build environment and compile the RiscOS rom:

1) Open your AcornC/C++ directory so that the RiscOS build system knows where to find your Norcroft compiler.

2) Run RiscOS.Library.InstallTools. Your compilers and the necessary tools should then be copied into the RiscOS build system.

3) Run RiscOs.Apps.!Builder.

4) Click on it’s icon on the iconbar. A ‘Register build directories’ window should pop up where you are asked to ‘Drag a build directory into this window’. If you don’t get this window click on ‘Register build tree…’ on !Builders iconbar menu. Drag the ‘RiscOS’ directory into this window and click Add and Save.

5) Click on the !Builder Icon and this time a window opens where you need to select the ‘Build directory’ (the one you just registered) and ‘Environment’.

To build the complete ROM chose TOOL.Tungsten as the Environment and tick the following Build options:

List Clean Clean all Export Headers Export Libraries Export Resources Make ROM Install ROM Join ROM

6) Click on Build and about 30 minutes later a rom image appears in RiscOS.Images

Check that there has been no errors during the build as it is possible to get a rom image with compilation errors. So in the output from the compilation:

1) Skip the cleanup phase (search for ‘export’) errors removing files don’t matter.

2) Search for ‘amu: *’ all the remaining errors except 1 should be caused by removing files. The only exception is the creation of CLIB:swis.h which is ok.

If you get more errors than that something went wrong in the process.

For further information on what environment and options to choose read:

http://www.riscosopen.org/wiki/documentation/pages/How+to+build+RISC+OS http://www.riscosopen.org/wiki/documentation/pages/Builder

I have fixed the problem with tar. Can I send you a copy to try it?

I’d love a copy but don’t particularly want to broadcast my email address on here. Can you put it up on a web page?

You can find a copy of “UnTarBZ2c.zip” at drobe

Thanks, I tried it out and I found the following problems.

It needs to fix the problem when the filename is exactly 100 chars long when ‘000640 ’ is added to the end of a filename. This occurs because you don’t get a 0×0 at the end of the filename.

With this fix it would have untared TungstenDev perfectly.

Im a bit troubled by your solution to recreating the original paths from previous filenames eg. If the filenames come in the following sequence

1. TungstenDev.castle.RiscOS…
2. RiscOS…
3. TungstenDev.mixed.RiscOS…

where … is the rest of the path/filename

then you moves RiscOS into TungstenDev.castle but it may need to go in TungstenDev, TungstenDev.mixed,be on it’s own or be further down the tree

For me the only solution is to change the archive format tar, with its 100 char limit, is clearly not up to the job.

When I found that the full filename is not in the tar file I wondered how other systems (Windows) still managed to get it right. The only way is to guess the full path. I can think of a thousand ways my method could go wrong, but it doesn’t.
In your example it doesn’t go wrong because line 2 is only created if all the other directories ’...’ are empty, but that change is very slim.
I will fix the exactly 100 chars bug and upload it after the weekend.

The only way is to guess the full path.

Actually there are several different tar archive sub-formats which have been created over the years, to work around issues with the original format such as the 100 character filename limit.

So if you find which sub-format the archive was created as (I think GNU tar will tell you this) then all you need to do is look up the implementation details of that format (or at least the bits for fixing the truncated filenames) and copy/reimplement that.

There are different tar sub-formats but even more headaches. New formats could not be read by old programs etc. The only fix is a new tar version or an other archive program like pax.

Colin, your problem will not happen because all sub directories are created first so the jump from point 2 to point 3 will never happen.

I have uploaded a new of “UnTarBZ2d.zip” at drobe
You can see what the new versions changed if you search for **. For OmapDev 137 entries are changed. This is including 15 names that had to be limited to 100 chars.

I tried the program and it seemed to work ok but the resulting sources wouldn’t compile because of the error:

modgen: Unable to read length of ‘Resources.UK.Resources.ShareFS.!Sprites’

It turned out that the file was decompressed as

TungstenDev.castle.RiscOS.Sources.Networking.AUN.AUNMsgs.Resources.UK.Resources.ShareFS.!Sprites,ff9

with a file type of text.

As the filetype is taken from the load and exec data in the tar header it would appear that the tar file was created that way.

I think that there is something wrong in the software that creates the tarball from the cvs source.

It may be significant that this was the only file with a 100 char filename that wasn’t a text file and so the only one that would have the filetype appended in the cvs repository. I’m guessing the filenames are changed and filetypes are added by another program after the tar file of the cvs checkout has been created. The file in question would have been seen as …!Sprites,ff9000640. As it has an unrecognisable file type the leaf name is left alone and the load and exec data given a text filetype.

Anyone from ROOL like to comment?

I’m not from ROOL, but…

The files are stored in CVS with the ,xxx extension. When the tar files are created the ,xxx extenions are left intact. When it works properly, the version of tar that gets used in !UnTarBZ2 should detect the ,xxx extenisons and convert them back to filetypes. So if you’re using Peter’s “fixed” version of UnTarBZ2 then it looks like there’s still a bug left for him to fix.

I have to look into it but I don’t think the filetype is taken from the load/exec data. It is concatenated to the filename. This way the filetype can be maintained if the files are extracted on an other filing system (like windows). I think when the filename+filetype == 100 chars, it goes wrong because the last four characters are not ’,xxx’ any more. Fixing the size should happen earlier in the program.

Colin, your were right, Jeffrey to. The filetype is taken from the filename as a ,xxx extension and store in the loadaddress. This happens a bit earlier in the program. I didn’t noticed it because most of the files are text files, the default file. In TungstenDev there were about 30 files where the filename had to be truncated. Only three were not text files.

New of “UnTarBZ2e.zip” at drobe

That extracts fine and the sources that were extracted compiled ok.

I looked through the source file and there’s still a problem though.

The filetypes should only stripped from the filenames if the -T flag is set in the command line. If you search the tar sources for CommaFileTypes (flag set with -T) you’ll see that it is only used in 2 places, one while reading a tar file (c.tapio) and the other while writing a tar file (c.replace). It shows that the filenames are only changed when CommaFileTypes is true.

I’m not asking you to fix it but there is also a problem creating tar files with the program. Where CommaFileTypes appears in c.replace the program can create a filename 103 characters long as the filesize check is done before adding the extension. The same is true of the -X flag (AddFileTypeExtension variable in the program)

Personally I think that the program rool is using to create the tar file is broken – it shouldn’t be able to create a name >= 100 chars. In which case your changes wouldn’t be necessary.

I am glad it’s working now.
As far as I know there are no names >100 chars in the tar. I think it is because the files stored in the CSV all have filetype ,xxx behind them so the program isn’t adding them. The problem is with names == 100 chars. They cause trouble because the terminator is missing. I think the tar program should handle it better. What I did was just fix it the moment the files are written to disk but the whole tar program should be checked for trying to access characters after the end of the buffer.

My changes are needed for fixing the path+filenames >100 but the program should handle names equal 100 itself.

The tar file wasn’t created with the tar program in untarbz2 as that one limits the filename to 99 chars (unless extensions are added) and doesn’t store files that have longer filenames.

There should never be a filename >=100 chars there should always be a terminating 0. You can’t program to handle long filename bugs in other programs universally, only in this specific case where it is assumed that it is only a trailing 0 that is missing. You don’t know if the filename written wasn’t longer – the mode field was probably written after the name field, overwriting the end of the name. I note SparkFS overwrites character 100 with a 0×0 before reading the filename which is probably the only sensible universal solution.

As I say the bug is in the program used to create the tar file in the first place.

• The release of UnTarBZ2 is planned to be accompanied by the creation tool
• The issues with UnTarBZ2 are not forgotten

The release of UnTarBZ2 is planned to be accompanied by the creation tool

The creation tool being talked about there is CreateSEC, not the tool used to create the tar files.

The tool used to create the tar files is most likely just a bog-standard version of GNU tar (and bzip2 for the compression).

As I say the bug is in the program used to create the tar file in the first place.

Highly unlikely.

Problem is, tar is a very old program. If I read the definition correct, the filename field is 100 chars long. Not used bytes are filled with zero’s. That is handy for C programs because they use the zero as string terminator but the name can also be exactly 100 characters and then there will be no handy terminator. The terminator is not part of the filename.
But until an other archive program is used, the patched version seems to work.