Spam

I should be finding some time to work on this tomorrow (28th) NZ time, though the cookie issue is thorny.

I constructed the ROOL site from a bunch of at the time best-of-breed (well, as far as I could tell) independent Rails applications. The source code is all in our public Subversion repository. They were modified to tie into a shared template system and the shared single sign-on system, as at the time Rails was new and raw and there were few if any proper SSO solutions – these days there are plenty of more modern alternatives I’d use instead of Hub (e.g. OAuth 2, facilitating sign in with Facebook/GitHub/etc.).

The lightest touch changes to the independent applications meant basically checking for a Hub cookie and if found, creating a shadow user in the local database for that application and creating a new session recognised by that specific application which is otherwise thereafter independent of Hub. I missed the very obvious issue that deleting a Hub account for good reason – e.g. spam account – would thus invalidate only the Hub account sign-in, but not any of the independently visited applications.

So, my first task several years later is to find an equally light touch solution to fix this by adding in a bit of code which always checks the Hub cookie as well as whatever ad hoc session system the individual application supports. Order of focus will be Wiki – because that’s the hardest to repair, so although it’s not often attacked the damage is really difficult to fix when it is – then Forum – because that gets attacked most often – then bug tracker. Canvass (the bounty system) is tied directly into Hub so needs no modifications; the CMS, blogging engine and rather neglected photo gallery engine have only admin user access only anyway.

All the while, I’ve got to avoid the very strong desire to upgrade the whole thing to at least Rails 4, ideally 5, since we’re waaaay out of date but since almost every application is now unsupported it’d be a mammoth task. Not sure I want to tackle that on my holiday after such a busy year… Downtime needed. I’ll try to restrict myself to the MVP to help protect us against spam threats.

Should I have time after the above, next on the list would be first-post-moderation for new users in the forum. Also, Steve’s had a request for Canvass to hide completed bounties from the front page for years which I’ve never yet tackled, because each time I come to work on the ROOL site, the Ruby environment and my local operating system have changed so much that I have to restart from scratch establishing a local development environment with a working early Rails and Ruby set of versions plus Hub.

Order of focus will be Wiki – because that’s the hardest to repair, so although it’s not often attacked the damage is really difficult to fix when it is

I’ve repaired a few individual pages by using the rollback, but I can see how that is pretty limited.

All things considered data loss is probably the worst so losing the forums is preferable. Both should be temporary – assuming a system backup

All the while, I’ve got to avoid the very strong desire to upgrade the whole thing to at least Rails 4, ideally 5, since we’re waaaay out of date but since almost every application is now unsupported it’d be a mammoth task.

The question is whether transferring data and user accounts to a new build Rails 5 setup is easier than upgrading through the revisions (it’s probably odds on a single step is a non-starter)

Quiet round here at the moment anyway so I’m sure everyone can live without any access at all for a while.

these days there are plenty of more modern alternatives I’d use instead of Hub (e.g. OAuth 2, facilitating sign in with Facebook/GitHub/etc.).

Ugh.
What I really like about ROOL is there’s no Facebook integration of any kind.

I’m somewhat sick of all this “Like” nonsense following people around the internet. Thank god for aggressive blocking on the PC and Android. Though the iPad that I’m using at the moment doesn’t offer this, so I don’t tend to visit potentially annoying sites using it.

because each time I come to work on the ROOL site, the Ruby environment and my local operating system have changed so much that I have to restart from scratch establishing a local development environment with a working early Rails and Ruby set of versions plus Hub.

I think that says all that needs to be said about backwards compatibility. :-/

I’ve repaired a few individual pages by using the rollback, but I can see how that is pretty limited.

Generally speaking, Wikis are not designed for data deletion. In fact I had to jump through hoops to erase spam on ARMwiki. It seems everything is supposed to be preserved, so data is restored by reverting, and everything gets dumped into the history. Hmmm…

All things considered data loss is probably the worst so losing the forums is preferable. Both should be temporary – assuming a tested system backup

There, fixed that for you.

Quiet round here at the moment anyway so I’m sure everyone can live without any access at all for a while.

Wah! No! I have no life so I check the forums at least twenty times every hour. Even when I’m asleep. :-)

More seriously – a potentially preferable situation during any possible migration might be to simply kill Hub for a while. This means nobody can log in, which should have the overall effect of making the entire site read only. So documentation and stuff is still available, but updates are not.
I suggest this because there’s a lot of stuff in the Wiki that doesn’t exist elsewhere. Not to mention retaining access to the source code, image updates, etc for however long the job takes.

Just please don’t replace everything with an animated road cone and a message about the site being under maintenance. :-)

The question is whether transferring data and user accounts to a new build Rails 5 setup is easier than upgrading through the revisions (it’s probably odds on a single step is a non-starter)

It’s difficult. I’m being forced to update Hub 2 to 3 as I just can’t get rational behaviour under Rails 2 on this install now, things have moved on too far; it requires extensive changes as the API modifications were large, right down to a new application boot mechanism, a new router API and a new mailer API plus extensive deprecations in things like the basic database access API and various Rails-related constants. It’s grunt work mostly, not much thought needed, but time consuming. I’m in the middle of it, writing this to have a break from it for a moment. Rails 3 up to 4 is easier, with 4 being a going concern. 5 is probably “too new”, with some dubious decisions on some of the things they’ve added that I reckon will need a good 12-18 months to shake down for security holes.

Hub’s small and even though I’ve forgotten most of it, at least it’s my code originally. Test coverage is abysmal – I wasn’t really into TDD back then, just dev test, I’ve got wiser with age – so it’ll need a good by-hand shakedown. Other applications are bigger and not my code so the potential for mistakes is much larger.

What I really like about ROOL is there’s no Facebook integration of any kind.

For SSO it’d just be a way to get a new account, nothing more. I could extend the Hub application with it. We get endless problems with people who try to sign up but end up with the confirmation message in junk mail and don’t realise it; also, people get account fatigue. Yet another site with yet another password. So using something like OAuth 2 reduces the barriers. Someone sees us on Facebook, goes to our site and can get straight in with a click or two and no other hassle. Not on my immediate roadmap though, many more pressing issues and close to zero time since emigration.

I think that says all that needs to be said about backwards compatibility. :-/

Rails was developed around the time when the industry completely lost its mind and decided that developers had nothing better to do than write, rewrite and re-rewrite their applications endlessly to the whims of ivory tower open source projects with a strong sense of how gracious and magnanimous they were to offer you the code at all and we should all just be grateful and shut up! So each version of Rails is a nightmare of removed or changed APIs, sometimes with things simply renamed and nothing else, plus incompatible changes to the language itself (Ruby 1.8→1.9→2.0 all produced breaking changes, though it’s settled down a little in the 2.x release cycle) and all the libraries (“gems”) do the same. One ends up locking to old versions because there are not enough hours in the day.

I’ve largely abandoned Rails development as a result. All my apps are out of date and any dev time I might have to add features or fix bugs is removed by basic maintenance.

The rest of the industry has been following, because it’s an easy lazy path – don’t write good APIs, just write crap and break it in the next release – even Apple did it in an iOS major version change. No prior deprecation, just a complete API break that was unannounced. It was one of a few indicators that signalled the start of their sharp decline in quality.

(Footnote: I really need to get this entire site bilingual for Textile or Github Flavoured Markdown. The latter “won” by a mile – Textile is highly anachronistic now, even in Rails land).

The rest of the industry has been following

My favourite is Google with AngularJS 2. They managed to break a significant amount of APIs between a Release Candidate and the final release.

Not to mention that of course AngularJS 1 is completely different to AngularJS 2. With no upgrade path at all.

Well six hours of effort and I can’t even get Hub working on Rails 3; it’s just endless errors. Rails “documents” itself mostly via blogs, which is of course insane, and the majority of Rails 2→3 upgrade information is long gone in dead links. I’m left fishing around in the Google cache and so-forth. One of the roadblocks was “request.request_uri” being renamed to “request.url”, which not only requires a sweep of the Hub code, but also the supporting Hub gem. Trouble is, the Hub SSO gem is shared across all our applications, so now I’ve had to do a major version bump for an internal API change just so that I can have some code locked to the old version for Rails 1 or 2, and some code locked to the new version on Rails 3.

It’s now up to the point of just entering an infinite redirection loop (how is that even possible with just a Rails version update?!) and I’m supposed to be on holiday to get rid of stress after a very intense year, so at this point I’m going to give up. The frustration level has reached the red line! Maybe I’ll get back to this some other day this holiday.

Rick:

All things considered data loss is probably the worst so losing the forums is preferable. Both should be temporary – assuming a tested system backup

There, fixed that for you.

Hmm, unless you’ve restored and run the backup on the same or matching hardware it ain’t tested. Can be done regularly I suppose, but how much were you wanting to pay for the testing work?

Wah! No! I have no life so I check the forums at least twenty times every hour. Even when I’m asleep. :-)

Automated scan for changes in the page, pop up a notification with sound :)

Andrew:

Well six hours of effort and I can’t even get Hub working on Rails 3; it’s just endless errors. Rails “documents” itself mostly via blogs, which is of course insane, and the majority of Rails 2→3 upgrade information is long gone in dead links.

That was the kind of thing I was suspecting might come up, hence mentioning the possibility of exporting the current data and importing to a modern build.

Hmm, unless you’ve restored and run the backup on the same or matching hardware it ain’t tested.

True, but you’d be surprised how often I’ve come across backups that have actually failed (completely) and nobody knows until somebody needs to restore from said backup.
Like the company that hacked together some batch files to back up what would have been about 3MB of client data onto a single floppy disc. They did at least rotate discs so they could go back a couple of days, but since they didn’t like lots of rubbish appearing on the screen (it was “untidy”), commands were set to direct all output to NUL: and the disc was ejected when it finished whirring.
Had they seen screen output, they’d have known it had been choking and writing only part of the data for years.
Then their harddisc failed. And they asked me (via a person with a propensity of volunteering me to fix difficult problems) why their backups didn’t work.

It takes a certain tact to tell a company – All your client data? It’s gone.
(especially when there is so much stupidity involved – silencing output for tidiness, never bothering to check the backups were good…)

Automated scan for changes in the page, pop up a notification with sound :)

Isn’t that sort of the point of RSS? ;-)

Well six hours of effort and I can’t even get Hub working on Rails 3; it’s just endless errors. Rails “documents” itself mostly via blogs, which is of course insane, and the majority of Rails 2→3 upgrade information is long gone in dead links.

Note to self – stick to php, never look at Rails.

Okay, php has changed too, but the only things I’ve noticed in my use (since ~1998ish) have been the disabling of some deprecated features, and URL variables are no longer auto-generated but need to be read from the _GET array.

That was the kind of thing I was suspecting might come up, hence mentioning the possibility of exporting the current data and importing to a modern build.

Which leads us to the obvious question – are we sure the export data would be anything like what the importer can deal with?
[for great examples just look at different versions of Excel when dealing with practically any format that isn’t its own, or different versions of Word when dealing with pretty much anything including its own format]

Which leads us to the obvious question – are we sure the export data would be anything like what the importer can deal with?

Where I work we call it a concept that needs testing, after testing it’s called a plan.
In this instance it seems in place upgrades are a recipe for frustration or worse.

hence mentioning the possibility of exporting the current data and importing to a modern build.

I don’t think you quite understand. The Rails 2 to 3 upgrade process involves rebuilding or at least modifying your app and migrating data as necessary. Usually, migrations aren’t required. If I were to just rewrite Hub from scratch, it’d be a much larger task and take far longer than I have available.

When it comes to things like the Forum engine or blog engine, major version updates (where there have been any) often do involve complex export and import processes, but they’re very delicate. Not to be taken lightly. All these options are available, but for me, step 1 is to establish a basic Hub-aware dev environment through which I can replicate the forum “stays logged in after user is deleted” scenario and from there patch the forum to fix it.

I can do this in the test environment on Arachsys but from this geography the latency is too high for that to be really practical. So I develop locally, then export to the test environment on the real server as proof of concept, then finally deploy to live.

(Oh and some progress has been made but it’s wading through treacle – every step of the way there’s a new error of some kind).

I should be finding some time to work on this tomorrow (28th) NZ time
(Oh and some progress has been made but it’s wading through treacle – every step of the way there’s a new error of some kind).

Treacle or quicksilver, we’re very grateful that rool spend their holiday time making the site/risc os better

All your client data? It’s gone.
(especially when there is so much stupidity involved – silencing output for tidiness, never bothering to check the backups were good

Know that feeling as I was called out to help a colleague who couldn’t get a system back up after a power failure and also subsequent power backup failure!

Tried all 3 sets of backup discs and all failed to load and then found the immortal words in the site log “Backup failed again with disc check error , I must find time to look at this one day”

Andrew: is there progress to report?

New spam – not in forum. Tucked in news pages user ID 3167

I’ve deleted the account, but I can’t see any way to delete his posting.

I’d suggest that mailing Andrew wouldn’t go amiss. Both for the cleanup now and for a possible route for you for the future.

Looks like the “school of spam” found this site. The 317x member ;-)

Most of the spams have a free phone number. Maybe we ought to program our modems to continually call it?

What’s a “modem”?

Maybe we ought to program our modems to continually call it?

Desisting only when we want to make a call ourselves? All very well if you’re expecting no incoming calls (apart from cold callers, who can go to hell anyway).

Most of the spams have a free phone number. Maybe we ought to program our modems to continually call it?

Won’t work from France, Rick!

What’s a “modem”?

Very droll, Chris!

When I get my Pace modem back in the UK to use with the CallerID program I’d like to program it to answer calls from a list of rogue numbers and then hang-up.

Might waste someone’s time or money!

I always felt that Spam is a delicious — if perhaps nutritiously void — meat (best served fried from road-side burger vans) and doesn’t deserve these awful connotations linking it with such despicable behaviour.

Hurrah for The Real Spam!

All very well if you’re expecting no incoming calls (apart from cold callers, who can go to hell anyway).

Well I have to say I can take or leave the calls from the “Microsoft Support Centre” (either India or Bradford by the sound of things)
I did get an offer of sexual activity but it was a bloke (I think) and I don’t swing that way. They are interesting to play with but if you’re trying to concentrate the wittering in your ear is rather distracting.

Spam season again – user id’s 3194 and 3195 the subject lines are indication enough, can’t bothered incrementing the views index.