The Elephant in the Browser

Making NetSurf use a mainstream browser engine Madness, and completely against the goals for the project. We’re aiming to make a lightweight web browser.

Note: NetSurf has had a complete JavaScript interpreter (SpiderMonkey) for the whole 3.x release series

Isn’t SpiderMonkey the engine that drives JavaScript in Firefox…?

Remaining bits are to rewrite the layout code to support layout changes driven by DOM events, and to hook the JavaScript interpreter up with the DOM and a few other bits of the browser. (A lot of work.)

Would it be worth putting together a bullet list of requirements for the update to the DOM so that any new developers can look at them? I know there are doubtless other issues and things to develop but if we can get the JavaScript/DOM working then it’ll be a big step for the credibility of NetSurf (on any platform).

Then again, banking on RISC OS is a really, really bad idea.

Why? What is it about RISC OS that is inherently insecure? What makes RISC OS worse than, say XP (which we’ve identified a lot of people are still using)? If NetSurf could be made to support all of the required security protocols – what would make running NetSurf on RISC OS less secure than running NetSurf on Linux/Windows/etc.?

I know plenty of individuals who haven’t upgraded from XP.

That could be me.

Current estimated usage is W7 at 55%, XP at 19%, W8/8.1 at 14% and Vista at 2%.

The upgrade from XP to 7 was not a straightforward one, even though I saw its value, and I did not do it. When support for XP was withdrawn I thought I would have to do something and bought a W8 machine. Even after upgrading to 8.1 and lots of tweeks, it is not an experience I enjoy. Then I cottoned on to the POS registry change and XP has been happily updating itself since. Even though Security Essentials says it does not work, it continues to download virus information and do regular scans. The machines, 5 and 9 years old, work just as I want them to and I am missing nothing, which is probably how many users view RISC OS 4.02. I do have PIs and an ArMX6. ;-)

Rick Murray:

Essentially, I was shooting down this as being unrealistic compared with reality: They’re written for multi-core processors, running at GHz speeds with GBs of RAM

Yes, but you’re not comparing like-for-like. Try running a desktop Webkit-based browser (or Firefox, which is similar in terms of efficiency) on an original Pi and see how well it goes. Phones are a totally different environment. Other apps are usually suspended with only the foreground app actually executing any code, for example. (Of course, it will be able to use both cores, which it wouldn’t on RISC OS, but its closer to a fair comparison.)

Over the weekend I was working on reducing the amount of memory NetSurf uses to store computed styles, and laying the groundwork for improving the CSS selection performance. But obviously that won’t be of much interest to RISC OS users, who all have staggeringly powerful hardware at their disposal. :)

David Gee:

On RISC OS, you have to implement algorithms that break nicely into short chunks so that you can yield back to the OS while also maintaining state between bursts of activity. On other platforms, you can just get on with writing the algorithm in a clear, maintainable way and let the OS worry about keeping the rest of the desktop alive.

Alas, that isn’t really the case

For NetSurf development is has been exactly the case. For example, it was only due to RISC OS users reports of the desktop becoming unresponsive that John-Mark did this . It ran fine before, on the other platforms we run on.

Steffen Huber:

The port was done in a comparatively short timeframe, so I cannot believe that “herculean” is the right word. Any numbers? Maybe in comparison to the amount of effort that went into NetSurf up to now?

I thought it took quite a long time, with Chocky working full time on it. I don’t remember exactly how long it took him. He used to hang out in the NetSurf IRC channel when he was doing that work. Anyway, if it was a quick job, surely someone would have updated the port by now?

NetSurf has taken loads of effort, no doubt about it. And because none of the core developers can work on it full time, progress is slow.

Jess Hampshire:

Is the Epiphany Browser relevant?

It uses webkit and gtk. Statically linking to both of them would make it absolutely vast!

Glen Walker:

Isn’t SpiderMonkey the engine that drives JavaScript in Firefox…?

Yes, although we’re using a very old version of it, and its not useful for very much because we don’t have any of the DOM bindings done. As I said, the engine is unimportant. Its the binding between the JS engine and everything JS code can affect which is were all the work for us will be. Also, as I said before, we’ve been evaluating Duktape for the JS engine. Its more in keeping with NetSurf’s goals but untested in a browser.

Would it be worth putting together a bullet list of requirements for the update to the DOM so that any new developers can look at them? I know there are doubtless other issues and things to develop but if we can get the JavaScript/DOM working then it’ll be a big step for the credibility of NetSurf (on any platform).

Its not updates to the DOM but the bindings between JS and the DOM. The best person to ask about this is Vincent who has done the work in this area. Please ask on the developer mailing list.

Apps are usually feature poor compared to the website. Good luck getting an app API from a bank, and doesn’t youtube change how it works rather often?

Weboob provides a lot of code. For API, most of the time they are self documented/declared. Web API is a very interesting world :)

Getting an API from the banks would be the simple part, getting it authorised would be the interesting/impossible part.

Of course, I talk of public API. For banks, W3C works on a standard.

The only reason you can use an Android phone like that is because there’s a business model for businesses to develop single-service apps for Android.

Not exactly. It’s because there are API. Some examples:
http://www.publicapis.com/

For banks, Weboob provides code for almost every french bank. So, yes, it’s difficult, but not impossible.

Look here:
http://weboob.org/modules

PS: of course, you can also simulate a connection. API are just facilities.

Ideas of use:

Social networks and blogs > mapped to Pluto
YouTube > mapped to multimedia player
Banks > mapped to a personal account manager
News > mapped to a RSS / news reader
Etc.

Can’t give other examples, as I do not use other websites daily :)

Of course, a lot of people go on many websites. But they don’t really need RISC OS (or Linux, or Windows, or OS X). They just want a web browser.

I know plenty of individuals who haven’t upgraded from XP.

That could be me.

Current estimated usage is W7 at 55%, XP at 19%, W8/8.1 at 14% and Vista at 2%.

My experience of windows is the opposite. I’ve only had 1 pc – a laptop – it had xp on it when I bought it new, it has windows 8.1 on it now – I couldn’t wait to get shot of previous operating systems. For me 8.1 is the best version of windows I’ve used by far and has finally made using windows bearable.

Except the APIs don’t expose every functionality of every website (when an API is offered), and applications coded to the APIs (unless it’s a one-site API) don’t expose every functionality of the API.

Websites move faster than the APIs do, because the APIs are lower priority for the companies. And, the APIs aren’t necessarily public if they even exist. Once you’re coding to private APIs, you might as well be coding to a screen-scrape of the website (which is how existing YouTube playback tools on RISC OS work today, actually).

Also, for the YouTube case, the UI for YouTube is fundamentally very different from the UI for a standalone media player that’s playing from your filesystem. Again, might I remind everyone that the Windows Phone version of the YouTube app is literally just the YouTube mobile website in an app? APIs are the direction that everyone’s moving AWAY from, and where you can use a native app that uses an API, it’s almost always released by the company that made the service you’re connecting to. While I’m sure my credit union (a form of member-owned bank in the US) didn’t develop their Android app themselves (for that matter, they’ve outsourced their online banking system entirely), I didn’t download a generic “mobile banking” app and tell it to connect to my credit union, I downloaded my credit union’s mobile banking app. And, there’s things I can’t do on the mobile app, that I can on the desktop website.

As far as security, RISC OS has the fundamental problem that everything runs as root. While RISC OS has no services running out of the box (actually, I should try portscanning my RPi to verify that), and therefore a most likely non-existent external attack surface, that doesn’t matter nowadays, as most people are behind a NAT. And, yes, RISC OS is an absolutely tiny target as far as market share, but it’s got some doozies of bad security architecture, and I doubt RISC OS development is at all security-focused. Some of these are shared with XP, some of these are worse than XP.

Everything runs as root. Let’s say that an exploit is found in NetSurf, and is exploited to, say, insert some malware into the RISC OS ROM. (Or, really, attacking !Boot would be enough.) Granted, XP has the same problem out of the box, but can RISC OS reasonably be run as a “normal user”? XP can.

Parts of kernel memory can’t be protected for compatibility reasons. Not that this actually matters due to everything running as root, but this enables deeper attacks on the OS even if real permissions are implemented.

There’s a complete lack of a security culture – “it works” is the goal, not “it’s kept working safely”. That’s not likely to change, simply because RISC OS is too small of a project to afford people working full-time on keeping it secure.

Update distribution is a mishmash of methods that are confusing and not kept up to date. For that matter, I just checked ROOL’s raspberrypi-testing repository, and found that the latest ROM in the repo is RC12, and the latest nightly is even older! (Actually, I might be willing to help with packaging.) And, is the !Boot structure packaged at all?

Also, another argument against running RISC OS for banking today, although not a fundamental issue, is that the RISC OS browser that’ll work with the most banking sites is Firefox 2.0.0.20. That browser likely has some serious security vulnerabilities that may even be able to be exploited without a single line of RISC OS-specific code – exploits meant to attack both x86 Windows and PPC OS X users, for instance, will hit RISC OS just as hard.

Glen Walker:

Why? What is it about RISC OS that is inherently insecure? What makes RISC OS worse than, say XP (which we’ve identified a lot of people are still using)?

It would not take much to hide some code in memory that responds to keypresses, picks up on URL fetch requests, and then pushes that data to a remote server. Such a thing can run as a chunk of code that is neither an app nor a module but is linked in to the necessary events. Or, if it was an app, it could be an app that can very easily be made to disappear from TaskManager.
Plus, it could notice when an application is being loaded, and embed itself into the executable. Essentially, a “virus”. Yes. But even better, poking around in other module’s/app’s workspace is a doddle. RISC OS provides no attempt to restrict or sanitise such accesses.

This is assuming that I can get you to run a ‘tainted’ program in the first place. Maybe my MIDI module infected your machine? Well, it didn’t (feel free to check) but how many people check what is being installed/upgraded? We RISC OS users just don’t expect this sort of thing, so if it does ever happen…

However, this isn’t the end of the story. NetSurf probably has some sort of buffer overrun vulnerability lurking in it where the right trigger conditions can cause some “arbitrary” (non-browser) code to be executed. I don’t say this with knowledge, I say this with respect to the size of NetSurf’s source code and the fact that this exact thing is now turning up all the time in supposedly “secure” Linux and MacOS. So let’s set the right conditions to cause code to be executed, and then let’s plug some extra things into !Boot. Your operating system will neither prevent this nor notify you that it just happened. Yet, from that point on, some extra stuff would be loaded every time the machine boots. Your private browsing history and keypresses? Now not so private.

That isn’t to say that XP (etc) doesn’t have a billion weaknesses itself. However the difference is that XP has at least half a dozen active anti-virus kits available for it. While these are not absolute security, they should make it a little bit harder for a responsible user to be badly affected.

This is an academic argument, but not one to take lightly. Not so long ago non-Windows users were gloating over how crap was Windows attempt at so-called security and how good their systems were. Well, the times they are a-changin’ and Windows has improved a lot since the idiotic decision in XP to make users Admin by default; and now that Linux/iOS/MacOS have reached a critical mass, they are finding themselves the target of attack and – guess what – they’re suffering the same problems that Windows suffered. Okay, granted, Linux is inherently more secure than XP ever was, however when a vuln can result in a privilege escalation that can give the affected code root level access to the machine, it doesn’t matter what the reason or excuse, you’ve just been pwned, the end.

If NetSurf could be made to support all of the required security protocols – what would make running NetSurf on RISC OS less secure than running NetSurf on Linux/Windows/etc.?

Because of all of the above. If the attack vector is the computer itself, it doesn’t really matter what protocols were in use.

I’ll give you a recent example. The nice bloke providing my personal private email address (that’s the one I’ve shared with about a dozen people) nearly had a nervous breakdown the other week. A load of spam was being relayed out from a German IP. Instantly the mail service was blacklisted. Bad times. He threw every resource he could think of at the problem, and it didn’t fix it. I asked him to go back over the logs of the initial attack and they showed something rather interesting. There was no attack. The German IP logged in as this user (not me, I should add), with the user name and the password, and then proceeded to peddle crap to the world. Turns out that the person’s computer had been compromised, so unknown third parties already knew the email login details (and who knows what other login/password combinations). No amount of intelligence on the server was going to fix this. That login/password combination had to go. That was the only option.

In short – when it comes to security, think of Anne Robinson’s infamous “You are the weakest link, goodbye.” and you’ll see that there are plenty.

Michael:

For example, it was only due to RISC OS users reports of the desktop becoming unresponsive that John-Mark did this . It ran fine before, on the other platforms we run on.

Why it would be good if the Wimp itself could support some form of pre-emptive behaviour. I do not think it would be too hard to add this (when the task is entered in the regular cycle of polling, the Wimp will note the state and set a ticker, and when the ticker expires the Wimp will task-switch away). The Wimp polls like usual, the task thinks it is running continuously.

Okay, it isn’t a complete PMT solution, but how likely is it that PMT will be implemented for RISC OS in the near future? It’s been a quarter century and…. ;-)

Yes, but you’re not comparing like-for-like. Try running a desktop Webkit-based browser (or Firefox, which is similar in terms of efficiency)

Ah, here you are moving the goalposts. It isn’t memory or processor speed, it is multitasking.

Phones are a totally different environment. Other apps are usually suspended with only the foreground app actually executing any code, for example.

You’re thinking of iOS! ;-) While Android typically only tends to run one thing at a time (due to things assuming they can go full screen), most Android devices are quite capable of multitasking with an MP3 player running, ES Downloader running, Avast scanner running, and the browser being the active task (but the others still there and status on the notification panel). Also, periodically, mail is checked and stuff like that.
Perhaps not so different to RISC OS using properly written apps that PollIdle and don’t respond to Null Events (so only get attention when necessary). Just because an app is loaded, maybe even on-screen, doesn’t mean that RISC OS is actually running it until it needs to.

I can’t compare desktop Firefox with a desktop webkit browser as I don’t let Chrome anywhere near my machine. On Android, however, Firefox is exceedingly slow.

David:

For banks, Weboob provides code for almost every french bank. So, yes, it’s difficult, but not impossible.

Is this bank-sanctioned code, or determined by decoding the website? The reason I ask is that CMB provides a useful facility called “Virtualis” (essentially a “fake” Mastercard good for a specific amount one-off payment; I don’t use anything else with eBay). This is provided by the app (iOS and Android) and is available as a Flash thingy for Windows or the CMB website. As such, the API for this is not open so I don’t think you’d find it easily. If at all.
I would like to think that an app¹ would be a little more sophisticated than to fire commands at a web server and scrape the document returned…

From Weboob’s CMB module:

        data = self.browser.open("https://www.cmb.fr/domiweb/prive/particulier/releve/0-releve.act").content
        parser = etree.HTMLParser()
        tree = etree.parse(StringIO(data), parser)

¹ A notable exception being Bouygues’ app which is something like 8MiB of app just to display the mobile version of their website… huh?

A load of spam was being relayed out from a German IP

Noted a blast of stuff from a .de address to an old pipex account used by Christine. Didn’t touch any of the other accounts.

That isn’t to say that XP (etc) doesn’t have a billion weaknesses itself. However the difference is that XP has at least half a dozen active anti-virus kits available for it. While these are not absolute security, they should make it a little bit harder for a responsible user to be badly affected.

My brother used to work as a programmer for one of the major commercial anti-virus companies and when I told him I was installing Windows XP on an old computer he said “under no circumstances connect it to the Internet”. I haven’t used XP since…

Will probably only be using RISC OS offline for the most part.

There’s a complete lack of a security culture – “it works” is the goal, not “it’s kept working safely”. That’s not likely to change, simply because RISC OS is too small of a project to afford people working full-time on keeping it secure. […]

I’m OK with all of this. And I would add that to port WebKit or Firefox to RISC OS is IMPOSSIBLE. That’s why I suggest a good ‘document only’ web browser + some apps based on web services + virtualization of Linux/whatever for other needs. IMHO, it will be much more simpler to have a working hypervisor for the Pi2 or IGEPv5 than to port a web browser. Then you’ll just need a minimal Linux OS with Firefox.

For complex browsing, I simply use a PC. My problem is that Avalanche hangs a lot and is not usable on modern computers. Don’t really need for more than a working VNC client :)

Is this bank-sanctioned code, or determined by decoding the website?

A bit of the two?

Will probably only be using RISC OS offline for the most part.

With a firewall and a web browser without scripting (at all), the cracker that will manage to penetrate your RISC OS system will be very very strong :)

IMHO, it’s almost impossible as surface attack is (almost) null in both directions.

I’d like to note that lynx of all browsers has had remote code execution vulnerabilities: http://www.cvedetails.com/vulnerability-list/vendor_id-5836/product_id-9869/Lynx-Lynx.html

Don’t really need for more than a working VNC client :)

You may find that the RISC OS RDP client gives better results and is more reliable. That’s what I use to get remote access to an Ubuntu box.

I can’t resist the comments about security. The main issue is that 99% of people don’t understand what security is and have most people tend to assume SSL means something is secure, whereas it does nothing to secure the endpoint.
One of the nice things about RISC OS is it’s simplicity, assuming you don’t have sharing enabled on your drives – I’m not sure what benefit a firewall would bring to RISC OS security. There isn’t anything running normally to firewall as far as I’m aware.

Rick Murray:

Yes, but you’re not comparing like-for-like. Try running a desktop Webkit-based browser (or Firefox, which is similar in terms of efficiency)

Ah, here you are moving the goalposts. It isn’t memory or processor speed, it is multitasking.

No. While you’re discussing getting one of these browser engines running on RISC OS systems and making comparisons to how they perform on mobile devices as “evidence” for anything, it is all of these things, as well as the others mentioned in this thread by myself and others. (Such as only one usable CPU core, availability of hardware acceleration, and other limitations of RISC OS, for example).

As for choosing a minimum RISC OS system to support, we went with RiscPC with RO4. Judging from bug reports and talking to users at shows, the RiscPC is still the most used.

aaaaa

A couple of points.

Firstly, while systems with ‘user’ as well as ‘admin’ access are more secure program running under your ID can still delete all of your data. Assuming you keep backups (how many do, on RISC OS? Given the need to—in some cases—look at the hard disks of defunct computers for important files, I wonder!) your own data may well be more valuable. To that extent the security offered by user accounts is a bit illusory.

Secondly, Android will kill off processes not related to the foreground app if needed to maintain responsiveness. Properly written apps will transparently restart themselves if needed after they have been killed.

Assuming you keep backups (how many do, on RISC OS?

It isn’t the only reason for having a 6TB raid array a central storage but it does sit high on the list.

To be honest with the cost of typical “have your own cloud storage” boxes these days¹ the real questions why would you not get a NAS?

¹ Cheaper than a typical smartphone (iPhone / Android)

You may find that the RISC OS RDP client gives better results and is more reliable. That’s what I use to get remote access to an Ubuntu box.

I use it too. But we can’t say to future users that the solution is to switch from VNC to RDP :)

Here’s an interesting take on Javascript:

http://tantek.com/2015/069/t1/js-dr-javascript-required-dead

Just a reminder that sites should never rely totally on JS. So, as far as RISC OS sites are concerned, we’re practically future proof and accessible from web archives.

Just a reminder that sites should never rely totally on JS.

We don’t really have the choice with HTML5. Many parts of it require JavaScript.

we’re practically future proof and accessible from web archives

If those archives store the data in an accessible manner!

We are at the start of the Digital Dark Ages…and perhaps some of us have already experienced this? There are many formats of files from many different programs and quite often its impossible to recover the data. Not only that, the actual file system used to store the file may be different… How many different Word Processors or Word Processing Software have there been? How many of those files are still able to be opened today?

There is some effort underway to create “snapshots” of whole computer systems as Virtual Machines BBC News Article but then you have the problem of how do you store that virtual machine? What format disk do you use? What about tape? Optical vs. magnetic? Flash memory? Something else?

I think it is a laudable goal to try and create data that is easily retrievable (and I always try to write documents using a plain text or at worst XML/LaTeX format) but I also think it is impossible to preserve everything for the future. JavaScript (and HTML5 as a result) is very useful and widely used so instead of choosing to abandon it based on how hard it would be to archive it; far better would be to create some mechanism whereby the useful data can be archived from a JavaScript rich website.

Glen, you are right, I was being disingenuous with my comments regarding archiving.

David, I’m not sure about your comment regarding HTML5 requiring Javascript. Are you referring to browsers implementations of form validation and drag and drop?

Is it not intrinsic to the specification that older browsers can still render ‘new’ pages, with or without Javascript?

I don’t believe the specification says anything about functionality without JavaScript. It’s certainly considered to be good accessible design to gracefully degrade functionality as browsers get older, but eventually you’ll get to a point where there’s just not enough functionality to use the site. And, many modern sites are written such that the JavaScript parts are actually necessary.

Besides, since when are pages actually coded to the W3C specifications?

David, I’m not sure about your comment regarding HTML5 requiring Javascript. Are you referring to browsers implementations of form validation and drag and drop?

WebRTC, Canvas, etc. Most of these new technologies need JavaScript. Basically, the whole thing we call HTML5 is more ‘HTML5+CSS3+JS’, and there is not a lot of solutions to use one without the others.

Anyway, the good question for Netsurf is ‘complete solution’ VS ‘partial solution’. A few entry points for JS<>HTML interaction will permit to make JS enhanced webpages that work under Netsurf. Perhaps it’s possible today, but there is no example of what can be done with the JS engine of NetSurf.