Web sites with many Rails applications
Preface
This document describes the installation process for the entire set of applications used on the RISC OS Open Limited (ROOL) web site, such that you may create a copy of the ROOL site in functional terms.
Please note that while the underlying structure of the web site is governed by the licences of the individual applications from which it is built, “RISC OS Open Limited” is a registered company. You must not create a web site that may mislead visitors into believing they are looking at something produced by or directly affiliated with RISC OS Open Limited, without prior written permission.
What does it do?
The ROOL site ties together a collection of Rails applications with a single sign-on mechanism and shared template-based common styles, all within one domain. At the time of writing it provides static asset content management with Radiant, a Typo-based blog (news) engine, a forum based on Beast, a bug tracker and SVN browser system using Collaboa, CVS browsing under Rails via a wrapper application called RCVSWeb and a Wiki engine driven by I2 and a photo gallery that uses the Gullery application. Full details and links to application home pages can be found in the ROOL site applications list.
Read-only access to the majority of the site is available to anyone. Visitors can contribute to many sections of the site by logging into the single sign-on engine, giving them access to all applications. Accounts are created with minimal fuss and only store e-mail address (used as a log on ID), real name and password. They provide an effective way to combat spam on top of the mechanisms already provided by the individual applications. Different permission levels in accounts provide different levels of access allowing more sophisticated interfaces to be presented to trusted users.
The administrative interfaces of those applications that have them are logged into separately by site admins. Such interfaces are not generally included in the single sign-on mechanism, to help minimise the scope of application alterations. It’s assumed that site admins don’t mind logging on a few times for different interfaces, whereas day to day users would get very annoyed if they had to do that within what is, after all, presented to the visitor as a single coherent web site.
This is a big document, but you’re doing a big thing by installing all of these applications. Steps are described in quite some detail; some people may feel patronised at times, but I’d rather that than feel confused because of a lack of information! Alternatives are presented and there are plenty of examples. By patiently working through things one step at a time you’ll arrive at a large, highly interactive and well integrated web site pretty quickly.
Prerequisites and nomenclature
It is assumed that you have shell access or an equivalent to a Unix-like operating system with user accounts enabled, in order to perform actions such as:
- Create, move/rename and delete directories
- Create, copy, move/rename and delete files
- Create, copy, move/rename and delete symbolic links
- Download .tar.bz2 archives and unpack them
The location of the home directory in which the web site is to be installed will henceforth be referred to as either just ~ where this may be used (the standard shortcut for ‘home’), or in full form as /home/username/ where a complete path is needed. Adapt this for the location relevant to your specific system.
Where this document mentions a ‘Unix path’ it refers to the location of some resource in the Unix filesystem, typically based around your home directory – e.g. /home/username/www/foo.html.
Where this document mentions a ‘Web path’ it refers to the path component of the URL you would use to view the resouce in a web browser given your prevailing web server configuration, e.g. /foo.html.
Initial downloads
Download the following tarballs from:
http://www.riscosopen.org/content/downloads/web-site-tarballs…and place them in a temporary folder called ~/temp.
This gives you the Radiant content management system and the Hub single sign-on application with supporting library. That’s enough to verify the basic integrity of the overall site structure.
Basic site directory structure
Set up the following structure using mkdir or your favourite equivalent tool:
mkdir ~/live mkdir ~/live/www mkdir ~/live/log mkdir ~/live/rails mkdir ~/live/rails/fcgi_sockets mkdir ~/live/perl mkdir ~/live/python mkdir ~/devel mkdir ~/devel/www mkdir ~/devel/log mkdir ~/devel/rails mkdir ~/devel/rails/fcgi_sockets mkdir ~/devel/perl mkdir ~/devel/python mkdir ~/shared mkdir ~/shared/rails mkdir ~/shared/rails/images mkdir ~/shared/rails/layouts
- The
livetree is where the working guts of your site will go. - The
develtree mirrors it and allows you to test changes to the site without interfering with the live copy. When happy that changes work you can copy the changes from development to live trees and restart the live services.
Supporting both live and devel trees requires that the live web services are able to work on the usual ports (typically 80 for HTTP and 443 for HTTPS) while the development services work on some other ports of your chosing. If your hosting arrangements don’t allow this, you may have to make do with just the live tree and ignore the parts of this document describing the development services, although this does make subsequent site changes tricky – you have no option but to experiment on your live site. In such circumstances, choices include:
- Put up with the restriction! You can, at least, use the site statistics engine to determine when your site is at its least busy and endeavour to only schedule maintenance work during such times.
- Use another shell account entirely and clone your site between the two accounts, using one of them as the development tree; this may not be desirable if you have to pay extra for such a facility.
- If you have virtual hosting available from a single user account, you could switch live versus development configuration based on request host using a single central server configuration file.
The web server will be pointed at www as the document root and told to write log files in log. Individual applications live in the directory named after their primary language of implementation – in most cases this means rails. The fcgi_sockets directories inside rails are a place to put Unix domain socket files. When using FastCGI (see later) you will probably want to connect to the FCGI processes using Unix domain sockets, which require a filesystem representation. The fcgi_sockets directories give you a place to collect all of them.
Location of applications
From the perspective of a visitor to your web site, applications that make up the site live in different directories. The first directory in the path of any URL will indicate the section of the site being visited, for example:
- http://www.example.com/content/… => content management system
- http://www.example.com/forum/… => forum software
Start thinking now about the names you want to use for these directories. It’s up to you – choose any names you like to house each application. Based on my own experiences I recommend some rules:
- Don’t use the name of the application. If you change to a new application of a different name with similar features in future, management of the site can become very confusing. However:
- Don’t be tempted to hide the way your site works by using obscure names, such as single or multiple digit numbers. There’s nothing wrong with visitors seeing from the URL in their web browser that they’re looking at your blog, photo galleries or whatever and using sensible names makes it easier to understand the site yourself, as the maintainer. Following on from this:
- Do use a name which indicates what the application does (e.g. forum/forums, blog/news, photos/gallery/galleries).
Each Rails application lives inside a directory within the live tree, along with its counterpart copy living in the devel tree. Only the application’s public directory is of direct interest to the Web server. This is why the directory structure listed above places the www tree outside the rails, perl etc. trees; the web server document root does not directly include any part of any application. Instead, symbolic links inside the www tree point back to applcation public folders. I’ll come back to this later, since we need to configure some servers first.
Server configuration
Database server
No special settings should be needed for your database server. I use a PostgreSQL installation running on a Unix domain socket inside directory ~/postgres, which also holds all the other database files, including postgresql.conf.
Web server
Rather than repeat what’s already written elsewhere, break away from this article for a minute and check out HowToInstallApplicationsInSubdirectories on the Rails wiki. If you want to read it in full then fair enough – it’s useful background reading and more of how the ROOL site works will be familiar afterwards. To cut to the chase, though, skip down to the “Web server configuration” part. Read through that section, stopping when you reach the next one, “Problematic applications”.
I’ll assume you follow the configuration file fragment approach described by the article. You’ll notice how application names appear in the configuration, just as described above, in the main configuration file to set up parameters identifying the application in use. Here’s where you match whatever names you chose for the symbolic links in the www directory to whatever names you chose for the directories inside the rails tree, where your applications’ contents live. The regular expression matches a URL and so in turn the www directory symbolic link names, while var.appname is set to the directory name. As mentioned earlier, it’s much simpler if you use the same names for both, but as you can see, you don’t have to.
An example of the main configuration file at ~/devel/lighttpd.conf is given below, with placeholders for information that you’ll need to supply based on your specific site configuration:
# Basic configuration
# Remove comments on one or more of the following
# lines for debugging.
#debug.log-request-header = "enable"
#debug.log-response-header = "enable"
#debug.log-file-not-found = "enable"
#debug.log-request-handling = "enable"
server.bind = "Your-IP-Address"
server.port = Your-HTTP-Port
server.modules = ( "mod_auth", "mod_accesslog", "mod_cgi", "mod_rewrite", "mod_alias", "mod_fastcgi", "mod_redirect" )
server.document-root = "/home/username/devel/www"
server.errorlog = "/home/username/devel/log/errorlog"
accesslog.filename = "/home/username/devel/log/accesslog"
server.max-keep-alive-requests = 0
# Turn on SSL - this is the section that might need the most
# modification to fit your certificate arrangements
$SERVER["socket"] == "Your-IP-Address:Your-HTTPS-Port" {
ssl.engine = "enable"
ssl.pemfile = "Path-To-pemfile.pem"
ssl.ca-file = "Path-To-bundle.ca-bundle"
server.name = "Domain-For-SSL"
}
# Global site information
dir-listing.activate = "disable"
cgi.assign = ( ".cgi" => "/bin/perl" )
index-file.names = ( "index.html", "home.html", "dispatch.fcgi", "index.php" )
mimetype.assign = ( ".png" => "image/png",
".jpg" => "image/jpeg",
".jpeg" => "image/jpeg",
".gif" => "image/gif",
".mp3" => "audio/mpeg",
".ogg" => "application/ogg",
".pdf" => "application/pdf",
".css" => "text/css",
".xml" => "text/xml",
".html" => "text/html",
".htm" => "text/html",
".cgi" => "text/html",
".rb" => "text/x-ruby",
".yml" => "text/x-yaml",
".yaml" => "text/x-yaml",
".txt" => "text/plain" )
# Redirect to the CMS home page if necessary.
url.redirect = ( "^/$" => "/content/" )
# Standard CGI Perl is used by default. Set up FastCGI mappings for the
# various Rails applications that make up the site.
$HTTP["url"] =~ "^/hub(/.*)*" {
var.appname = "hub"
include "lighttpd-rails.conf"
}
else $HTTP["url"] =~ "^/content(/.*)*" {
var.appname = "content"
include "lighttpd-rails.conf"
}
# ...Etc, for other applications.
The Rails Wiki article also mentions setting up FastCGI parameters and specifying environment variables with the bin-environment key in the fastcgi.server hash entry in the configuration file. You can use the configuration file fragment example in the article more or less exactly as-is, but for the whole of the ROOL site you’ll need more environment variables than shown in the example:
RAILS_ENV– set this todevelopment,testorproduction, as per your requirements. Often, you usedevelopmentfor thedeveltree of applications andproductionfor thelivetree. Once the site is live, you can copy databases to take snapshots of live data for use in the development tree. A utility script to do this for PostgreSQL is listed towards the end of this article.RAILS_ASSET_ID– if you want to stop Rails adding query strings –?12345678– to the end of static asset URLs, set the asset ID to an empty string (""). If you don’t care, omit this variable.RAILS_RELATIVE_URL_ROOT– set this to'"/" + appname'as described in the Rails Wiki article.SERVER_DOCUMENT_ROOT– set this to the location ofwww. In the live tree, then, this will be/home/username/live/wwwor, for the development tree,/home/username/devel/www(replacing/home/usernamewith the place where you’ve been storing everything so far, obviously).HUB_CONNECTION_URI,HUB_PATH_PREFIXandHUB_RANDOM_FILE– see below.GEM_PATH– if you use more than one Gem repository (see HowToUseMultipleGemRepositories), specify the appropriate path to all of the repositories here.
The values for the Hub variables mentioned above depend upon your Hub installation. In brief:
- Given that you’ve already set up the
rails/fcgi_socketsdirectories,HUB_CONNECTION_URIwill usually be set todrbunix:/home/username/live/rails/fcgi_sockets/hub_drb, changing/home/usernameas required and swappinglivefordevelin the development tree version of the configuration file fragment. The leafname “hub_drb” is arbitrary and up to you. HUB_PATH_PREFIXis set to the Web path for the Hub application, which is usually/hubbut may be different if you didn’t use the namehubfor the symbolic link to the Hub applicationpublicdirectory in~/live/wwwand/or~/devel/www.HUB_RANDOM_FILEis a Unix pathname pointing to a file of between 1K and 16K in side of data completely private to your server, ideally pseudo-random in content. A highly compressed archive with its header removed might qualify, or better, use something like the.rndfile output by the PuTTY tools.
For full details on these variables and why they’re set up as they are, please see the documentation at the Hub home page.
Given all of the above, a typical configuration file fragment in ~/devel/lighttpd-rails.conf would look like this, with placeholders for information you need to supply to match your particular server arrangements:
# Inclusion fragment for LightTPD configuration files. Sets up
# environment for FCGI Rails support. Set "var.appname" first.
server.error-handler-404 = "/" + appname + "/dispatch.fcgi"
fastcgi.server = ( ".fcgi" => ( "localhost" =>
( "min-procs" => 2,
"max-procs" => 2,
"socket" => "/home/username/devel/rails/fcgi_sockets/" + appname + "_fcgi",
"bin-path" => "/home/username/devel/rails/" + appname + "/public/dispatch.fcgi",
"bin-environment" => ( "RAILS_ENV" => "production",
"RAILS_ASSET_ID" => "",
"RAILS_RELATIVE_URL_ROOT" => "/" + appname,
"SERVER_DOCUMENT_ROOT" => "/home/username/devel/www",
"HUB_CONNECTION_URI" => "drbunix:/home/username/devel/rails/fcgi_sockets/hub_drb",
"HUB_PATH_PREFIX" => "/hub",
"HUB_RANDOM_FILE" => "/home/username/.rnd",
"GEM_PATH" => "/home/username/devel/rails/gems:/lib/ruby/gems/1.8" )
)))
Once all that’s done, you would launch LightTPD with a command such as the one shown below, though don’t do it just yet:
lighttpd -f /home/username/devel/lighttpd.conf ...or... lighttpd -f /home/username/live/lighttpd.conf
Hub DRb server
The Hub application requires a Distributed Ruby server, which is included as part of the Hub gem. This is described in the Hub documentation in detail, but don’t worry about it for now – we’ll come back to this later, once Radiant and the Hub application are installed.
Application installation
Unpacking the applications
I’ll assume you have a working LightTPD web server and PostgreSQL database server already. If not, you must first install those, or equivalents with which you are more comfortable, then carry on with the steps described below.
Unpack the tarballs downloaded earlier:
cd ~/temp tar -zxvf radiant.tar.bz2 tar -zxvf hubssolib.tar.bz2 tar -zxvf hub.tar.bz2
The flags, in order shown, cause tar to decompress the archive first, extract its contents, operate in verbose mode and take the next parameter as an input filename. Some tar binaries require a slightly different syntax, e.g. tar -z -x -v -f <filename>. Some don’t handle bzip2 compression at all and you’ll need to use the bzip2 binary first. Again, the exact syntax depends upon the exact variant you use. Common examples are:
bunzip2 radiant.tar.bz2 > radiant.tar ...or... bzip2 -d radiant.tar.bz2 > radiant.tar ...then... tar -xvf radiant.tar
Note the lack of the ‘z’ switch to tar, since the archives are already decompressed.
Now copy the decompressed applications to the devel tree – we’ll start work there, since live is only meant to be set up once we’ve verified correct operation in the development (test) tree. The target directory name (e.g. content in ~/devel/rails/content) is one of the two places where you get to choose the name under which you want the application to be stored, as mentioned in the previous section.
cp -r rails/radiant ~/devel/rails/content cp -r rails/hub ~/devel/rails/hub
We’ll come back to the hubssolib archive later.
Creating symlinks
Make symbolic links in ~/devel/www to each of applications’ public folders. For example:
ln -s ../rails/content/public ~/devel/www/content ln -s ../rails/hub/public ~/devel/www/hub
The use of ../ means that the links in the www part of the tree point in relative, not absolute terms, to the public directories. If the links are copied (staying as links), they will work equally well in the live tree.
Configuring applications in general
In general for any applications – not just Radiant and Hub – you’ll need to do the following:
- Make sure that the
dispatch.cgi,dispatch.fcgianddispatch.rbscripts in the applicationpublicdirectories mention anenvorrubybinary available on your system (use ‘@which env’ or @’which ruby’ to report the binary location if you’re unsure). - Check
config/environment.rband make sure there are no hard-coded path references for resources which may be in a different location on your setup, amending any such paths as necessary. There shouldn’t be any such changes needed for Radiant or Hub. - Ensure
config/database.ymlcontains your database’s connection details. - Set up the database tables for this application (see below).
The database tables require a bit of work on two fronts:
- Each application is different; some just use migrations, some have special scripts.
- With migrations, you may need to set up the environment variables that
environment.rbuses.
You may wish to set RAILS_ENV to development, production or test in order to set up the database tables for a particular execution environment – this is a standard Rails concept so if unsure please consult Rails documentation for more information. You’ll need to create those databases before you can fill them in; for example:
createdb -h /home/username/postgres -E=UTF8 radiant
With the variables set up, change into the root directory of each application (e.g. ~/devel/rails/content) and issue whatever command is necessary for that application. Each application is different so be sure to consult its README, INSTALL or similar documentation to find out how it should be installed, paying attention to the information related to database setup in particular. For Hub just use the Rake migration command:
rake db:migrate
For Radiant 0.5, use its database creation script. You then need to run migrations too, in order to pick up items related to ROOL extensions.
ruby script/setup_database production rake db:migrate
If you get any errors about unexpected nil values or similar, check the application’s config/environment.rb as it’s most likely that you need to set up another environment variable.
Configuring Hub specifically
Apart from the external environment variables already described, Hub has some internal settings you’ll need to modify. Edit its config/environment.rb and scroll to the bottom of the file. Here you will find variables such as EMAIL_ADMIN, EMAIL_PREFIX and EMAIL_HOST which configure various aspects of Hub. Each has a comment explaining the variable beforehand so it should be easy enough to set up.
Installing the Hub gem
Now we need to install the Hub library gem. It is supplied in source form so you’ll need to build it. Change back to the temp directory where you unpacked the tar files and into the gem sources:
cd ~/temp/rails/gems/hubssolib
Build the gem:
gem build hubssolib.gemspec
A .gem file should be written. Now install this using gem install <filename>.gem. if you don’t have write access to the main gem repository you will need to create a local one. See this Rails Wiki article for details. Confirm that the gem is installed using a command such as:
gem query -n hubssolib
You should see the gem listed in the command’s output. Ensure GEM_PATH is set if using multiple repositories.
Shared and static content
Copying over
We’re almost ready to give the site a try. There are two remaining tasks – copy over shared content and copy static data into the www tree.
First, change into your temp directory and download a couple of archives:
Unpack the archives in the same way as before. The shared resources will extract into a rails tree within your temp directory while the static resources extract into a www tree. Copy the shared resources for live and development sites with a command such as this:
cp -r rails/shared/* ~/shared/rails/
Copy the static web resources into the live and/or development trees – for example:
cp -r www/static/* ~/devel/www/
Links in applications
At the time of writing every Rails application except for Radiant uses shared template resources. Radiant doesn’t need to do this because it’s the source of the shared template files – they’re held in its database and exported to the filesystem when modified (this is described later). Radiant fetches the information from there directly. To make life simple, the other applications refer to the shared resources as if they were local files. We use more symbolic links to share the single exported copy of each template file between the applications. In each case you will find a shared directory inside app/views, so for example:
cd ~/devel/rails/hub/app/views/shared ln -s ~/shared/rails/layouts/fixed_* .
Using the wildcard in the ln command means that all three shared template files are linked in at once.
Starting up
The Hub DRb server
Before running the web server, we need to get the DRb server going. This is pretty easy. The server Ruby script is included in the Hub application sources under the name server/hub_sso_server.rb – for example, in ~/devel/rails/hub/server/hub_sso_server.rb. Start the DRb server using something like this script – it’s best to create a special script for this and save it somewhere meaningful:
HUB_CONNECTION_URI="drbunix:/home/username/devel/rails/fcgi_sockets/hub_drb" HUB_RANDOM_FILE="/home/username/.rnd" GEM_PATH="/home/username/devel/rails/gems:/lib/ruby/gems/1.8" export HUB_CONNECTION_URI HUB_RANDOM_FILE GEM_PATH rm -f /home/username/devel/rails/fcgi_sockets/hub_drb ruby /home/username/devel/rails/hub/server/hub_sso_server.rb &
The script above works for the bash shell but may need adapting for others. Consult your shell documentation for details. Note that the HUB_CONNECTION_URI, HUB_RANDOM_FILE and GEM_PATH settings must be exactly the same as those given in the lighttpd-rails.conf configuration file fragment. You should be able to run the server now; confirm that it’s running with a command such as:
ps -Af | grep hub_sso_server
This will only work if your server’s installation of ps supports the “f” flag to give the full command used to launch each process. If it does so but there’s more than one Hub server listed, someone else is running Hub too :-) – check for the one which is running from your Home directory. If you want a more elegant approach and your system supports the pgrep command, use one of:
pgrep -u username -l -f ruby ...or... pgrep -u username -l -f "ruby /home/username/devel/rails/hub/server/hub_sso_server.rb"
Obviously, replace username with your actual user name. The -f flag specifies a pattern match for the search, so in the first case you’ll get a list of all ruby processes running under your username and in the second case you’ll get an exact match for the command used to launch the DRb server. Based on this, we can use pkill to send SIGHUP to the process and have it exit relatively cleanly:
pkill -HUP -u username -f "ruby /home/username/devel/rails/hub/server/hub_sso_server.rb"
Once you’ve got it working you can put a command like this into a script file to sit alongside your DRb server start script.
For more information on Hub, please consult the Hub home page.
The Web server, at last!
Before starting, make sure:
- You’ve property configured Radiant and Hub, including setting up their database tables
- Your database server is running
- The Hub DRb server is running
Now start your web server, using – for LightTPD at least – a command similar to the one given earlier. Wait a while to check that no errors are reported at the console; if they are, you’ll have to check the error details carefully to try and diagnose the problem. Running the Rails applications in development mode rather than production may result in more information being placed in each application’s log file and you can enable various useful web server logging features through its configuration file as described earlier. Hopefully, though, everything will work – you should be able to visit your site:
- http://www.yourdomain.com/robots.txt to check that basic static content can be served.
- http://www.yourdomain.com/images/risc_os_open/banner.png to check that static images can be served too.
- http://www.yourdomain.com/content/ for Radiant (change
contentif you used a different name for the symbolic link to Radiant’spublicdirectory). The very first time you fetch this, if you’ve no pages defined in the database, Radiant should redirect you to the admin interface. At the time of writing, Radiant’s default admin user name isadminwith passwordradiant. You should be able to log in, change these defaults to something private and set up a Home page (don’t forget to change the “Status” pull-down menu underneath the text editing region to “Published”), then revisit the/contentURL and view that page. See the Radiant Home page for more information. - https://www.yourdomain.com/hub/ for Hub single sign-on. Now you can try to set up an account to see if Hub is working. The first user account that gets created is automatically an administrator. After that, accounts only have normal privileges. If you get a 404 error, first try logging out of your shell, logging back in, stopping the DRb and web servers, then restarting them while still in your home directory (issue the command
cd ~to be sure). - https://www.yourdomain.com/content/admin for Radiant’s administrative interface, to set up more content – again, see the Radiant Home page for more information.
With luck you’ll get everything working with few or no glitches. If you get stuck, feel free to “e-mail me” at ahodgkin@riscosopen.org. Watch out for stray FCGI or web server processes that have been left kicking around by accident as these can be very confusing (to help out here, see the scripts in the next section). If you use more than one Gem repository, make very sure that you’re accessing the correct repositories and don’t have any GEM_PATH environment variables kicking around that might have found their way into a process environment. I spent a long time trying to figure out a really strange Hub redirection which turned out to be the DRb server finding a very old Hub gem in a repository I didn’t even remember I still had :-)
Useful management scripts
I recommend you create a collection of scripts in ~/devel/services with corresponding versions in ~/live/services. Server restarts become much easier when you can just type something like ~/devel/services/restart to get everything done in one go, without touching your live services at all!
In all examples, change the paths, where they appear, to suit your installation. Don’t forget to update the #! /bin/bash to point to wherever your copy of bash is, or adapt the scripts for a different shell if you use one.
DRb server
Starting
As described earlier, one possible script, which we’ll call drb_start, is listed below:
#! /bin/bash # # Start the Hub DRb server. HUB_CONNECTION_URI="drbunix:/home/username/devel/rails/fcgi_sockets/hub_drb" HUB_RANDOM_FILE="/home/username/.rnd" GEM_PATH="/home/username/devel/rails/gems:/lib/ruby/gems/1.8" export HUB_CONNECTION_URI HUB_RANDOM_FILE GEM_PATH rm -f /home/username/devel/rails/fcgi_sockets/hub_drb ruby /home/username/devel/rails/hub/server/hub_sso_server.rb &
While the DRb server shouldn’t be interested in any other external environment variables, future versions might add a feature that could be disturbed by stray variables set on the command line. An alternative implementation for the above uses the env command and if you have that command available, I recommend you use this second version of the script:
#! /bin/bash # # Start the Hub DRb server. URI="drbunix:/home/username/devel/rails/fcgi_sockets/hub_drb" RND="/home/adh/.rnd" GEM="/home/username/devel/rails/gems:/lib/ruby/gems/1.8" rm -f /home/username/devel/rails/fcgi_sockets/hub_drb env -i PATH=$PATH HUB_CONNECTION_URI=$URI HUB_RANDOM_FILE=$RND GEM_PATH=$GEM ruby /home/username/devel/rails/hub/server/hub_sso_server.rb &
The env command is told to ignore any previous environment, then we import the three Hub variables into its new environment before running the server, all through that last rather long line.
Stopping
Script drb_stop:
#! /bin/bash # # Kill the Hub DRb server. pkill -HUP -u username -f "ruby /home/username/devel/rails/hub/server/hub_sso_server.rb"
Don’t forget to replace username in the -u username switch with your actual user name.
Listing processes
Script drb_list:
#! /bin/bash # # List the process details for the Hub DRb server. pgrep -l -u username -f "ruby /home/username/devel/rails/hub/server/hub_sso_server.rb"
Don’t forget to replace username in the -u username switch with your actual user name.
Web server
Starting
As with the DRb server there are two ways to start the Web server, using a web_start script. The simple way is as follows:
#! /bin/bash # # Start the web server, assuming it is already stopped. lighttpd -f /home/username/devel/lighttpd.conf
Unfortunately environment variables can leak to the FCGI processes and cause real problems. The most common is RAILS_RELATIVE_URL_ROOT which may have been set on the command line for testing purposes. To ensure a pristine environment for the web server, use env again:
#! /bin/bash # # Start the web server, assuming it is already stopped. env -i PATH=$PATH lighttpd -f /home/username/devel/lighttpd-devel.conf
Stopping
Script web_stop:
#! /bin/bash # # Shut down the Web server and all FCGI processes. # First, politely kill the web server. It should bring down Rails # FCGI processes with it. pkill -INT -f /home/username/devel/lighttpd.conf # Wait a bit, then politely try to kill off any remaining Rails # related processes. sleep 2 pkill -INT -f /home/username/devel/rails # After waiting a bit longer, if there's anything left, force it # to exit immediately. sleep 1 pkill -9 -f /home/username/devel/lighttpd.conf pkill -9 -f /home/username/devel/rails
Listing processes
Script web_list, which lists LightTPD and FCGI processes:
#! /bin/bash # # List the process details for the web server. pgrep -l -u username -f "/home/username/devel/lighttpd.conf" pgrep -l -u username -f "ruby /home/username/devel"
Everything
Tie things together with scripts all_start, all_stop and restart.
Starting
First the simple all_start script:
#! /bin/bash # # Start all servers, assuming currently stopped. /home/username/devel/services/drb_start /home/username/devel/services/web_start
Stopping
Next, all_stop, which is also very simple:
#! /bin/bash # # Stop all servers. /home/username/devel/services/drb_stop /home/username/devel/services/web_stop
Restarting
Finally, the restart script. It isn’t called all_restart because you can use command line filename completion more easily, since (unless you create others) it’s the only command in ~/devel/services which starts with the letter ‘r’. Of course, you’re quite free to choose your own preferred names for any of the scripts, or ignore them and use your own methods.
#! /bin/bash # # Restart all servers. echo Stopping... /home/username/devel/services/all_stop sleep 2 echo Starting... /home/username/devel/services/all_start echo Done.
Customisation
Before adding in other applications, it’s time to customise the Hub and Radiant installations already present.
Customising Radiant
This is where you start to set up your site for real. At the moment the site is using borrowed header, footer and sidebar snippets stored in the shared tree. These should actually be exported from Radiant automatically. This is done with the Radiant administrative interface, all you need is a web browser and up-and-running site. If you share database contents between the live and development services then you only need to do this once, else you’ll need to copy over the information through whichever method you feel most comfortable. The copydb script given later may prove useful.
- Set up a layout describing the overall structure of Radiant-based pages on your site. This should include one or more snippets; as it stands the ROOL-originated applications expect to use three, for static header, footer and sidebar components, so you’d need to modify each individual application layout if you want to move away from this model.
- Set up (at least) three snippets for the aforementioned header, footer and sidebar components. The exact content will depend in part on the way in which your layout includes them. I’ll give examples below that match the ROOL applications’ expectations.
- Build the Home page.
Setting up the layout
Visit https://www.yourdomain.com/content/ on your new site. If you’ve not set up any Radiant content yet then you’ll be redirected to the login page for the administrative interface. Otherwise, add “admin” to the end of the URL to go there manually. The default user name is admin with password radiant. I recommend you work over an HTTPS connection the whole time to keep further login information changes private. Go to the “users” section (link at the top right) and change the admin user name and password to something that only you know. I won’t go into more details about Radiant administration since the Radiant web site already contains all the informtion you need.
Go the ‘Layouts’ tab and use the ‘New layout’ button. Give the layout whatever name you like. In the main text area, construct an HTML page that forms the bare bones of your site. Include room for a header, footer and sidebar. To be compatible with the expectations of the ROOL-originated application layouts as they currently stand:
- The header snippet must be called
fixed_headerand be included immediately after thebodytag. - The sidebar snippet must be called
fixed_sidebarand will be a piece of free-floating HTML that may be included as part of a more old fashioned (but widely browser compatible) table sidebar column or CSS-based column layout. It doesn’t even have to be a true sidebar really, this is just another piece of content that’s repeated from page to page to help link together the applications. - The footer snippet must be called
fixed_footerand appear after all other body tags have closed, just before the closing/bodytag.
Here’s a bare-bones example including in-page comments:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="de" lang="de" dir="ltr">
<head>
<title>My web site: <r:title /></title>
<meta http-equiv="Content-type" content="text/html; charset=iso-8859-1" />
<link type="text/css" rel="stylesheet" href="/css/global.css" />
</head>
<body>
<r:snippet name="fixed_header" />
<!-- Contents -->
<p />
Some preamble, e.g. a table header, set of DIVs, etc.
<p />
<r:content />
<p />
More preamble, to lead into sidebar layout
<r:snippet name="fixed_sidebar" />
<r:content part="sidebar" />
<p />
The second tag above includes any custom sidebar components directed from
Radiant itself, which will only appear in Radiant-generated pages. There's
nothing to stop you adding in other snippets here too, for use with Radiant
content.
<p />
Finally, the footer.
<r:snippet name="fixed_footer" />
</body>
</html>
Note that this example accesses the stylesheet /css/global.css. Other applications like Hub use /css/risc_os_open.css currently, which was created when you unpacked the downloaded ROOL site static resources archive and copied its contents into the www tree. Eventually you’ll need to modify each application layout to suit your new model and include the new CSS file, which you have to create. Of course, you can use whatever headers and external resources you like in practice – this is just an example. Although fragments of the layout are shared (the header, footer, sidebar and – if you want one – common stylesheet), it is not possible to export the overal Radiant layout to other applications. They all have very different structures and requirements which means they need to be edited by hand.
The small ‘More’ link underneath the layout’s name leads to a content type field which by default is blank. If you use HTML or XHTML (processed as HTML under tag soup rules as is dangerous, but nonetheless customary and unavoidable for Rails applications) then you can leave the field blank.
Use the ‘Create layout’ button to keep the changes, then move on to snippets.
The shared header, sidebar and footer snippets
The ROOL version of Radiant is extended to support exporting snippets to the filesystem. We use this to allow the CMS to export template components to other applications so that a single update in the CMS results in changes reflected across all applications immediately (though you may need to flush per-application caches).
Go to the ‘Snippets’ tab. You need to create a minimum of the three snippets already mentioned, along with any others you find handy. First, the header. Use the ‘New Snippet’ button to create each one. If you get a Rails application (500) error, check that you remembered to run the rake db:migrate migration as well as the standard Radiant database setup described earlier. If not, shut down the server, run the migration and restart everything.
The header
Name the first snippet fixed_header and use the ‘More’ link underneath to open the ‘Export to file’ and ‘Command to run’ options. Set the path for the exported header file in the first of these, e.g. /home/username/shared/rails/layouts/fixed_header.rhtml. Leave the command field blank for now. In the main text area, in content terms, you can write anything you like. In structure terms, the markup language must be XHTML, compatible with Radiant and work in Rails when used as a .rhtml component. Given these restrictions, the filter drop-down menu underneath the body needs to be left at ‘none’. Very simple example:
<h1 align="center">A new web site</h1> <p align="center"><i>Navigation options might to go here</i></p>
The ‘Command’ field which we left blank earlier can be useful if you want to force application cache flushes when the snippet is altered, since some applications will not reflect the shared snippet change immediately because of caching. You may even choose to run a server “clean start” script which shuts everything down, wipes the caches and restarts everything – being aware that any visitors looking at your site at that particular moment may be annoyed by it being shut down briefly, which logs them out of Hub! I’ll leave that up to you for now.
TODO: At some point, revisit this once I’ve come up with a good way to clear caches myself. Right now my own solution is not satisfactory and the command I run is actually a script with the contents commented out.
The sidebar
The next snippet needs to be called fixed_sidebar. Again, use the ‘More’ link underneath the name to access the extra options and set up the export path, e.g. /home/username/shared/rails/layouts/fixed_sidebar.rhtml. The same content restrictions apply, so leave the ‘Filter’ menu set to ‘none’. An example:
<b>Common sidebar content</b>
The footer
The footer can be treated in the same way, but we can do something a bit more interesting to illustrate a way of attaining a limited mixture of Radiant and ERb markup within the shared content. Set it up in the same way as the others, under name fixed_footer. If that’s good enough for you, leave it there. Otherwise, set the ‘Filter’ menu to ‘ERb’. The ROOL version of Radiant includes extensions for limited embedded Ruby support inside Radiant pages in a similar manner to the way that ActionView presents an ERb environment to rhtml content. Indeed, the code to produce a useful environment inside the ERb execution context was produced by examining the ActionView implementation.
I did this work for several reasons. One was to be able to produce the “Validated” links in the static footer. Since the footer is a piece of content held in a shared location, how can it change its link to W3C validator pages so that the W3C validator knows which URL it is supposed to be validating? By using ERb, we can construct the link on-the-fly for applications including our exported, static copies of the footer in the shared .rhtml files. By extending Radiant with ERb support, Radiant can parse those same ERb instructions too.
<a href="http://validator.w3.org/check?uri=<%= url_encode(@request.protocol + @request.host + @request.request_uri) %>">Valid XHTML 1.0</a>
Since this access the request object as a variable and since Rails is deprecating that access mechanism, the above code will have to change in some future version of Rails. I’ll cross that bridge when I come to it; the example shown above works fine in Rails v1.2.×.
Another reason for doing this was to be able to include the “Powered by…” and theme information that, if you’re sharp-eyed, you will have seen at the bottom of each ROOL page. These change from application to application, despite the footer being apparently a piece of static shared content. They do so using partials called _powered.rhtml and _theme.rhtml in each application’s app/views/shared directory. You can see examples (if you haven’t already noticed them from the time you made the symlinks to the header, footer and sidebar files) in Hub:
ls ~/devel/rails/hub/app/views/shared
Radiant doesn’t use partials like this, though, it uses snippets in its database. So I needed a way to use the partials when executing outside of Radiant, or snippets inside it. The key is that the ERb environment inside Radiant does not include Rails’ render method, because it’s not part of ActionView. We can write ERb which checks for this and uses either ERb methods or Radiant tags to include the relevant content. This works because ERb content in the snippet is parsed before being passed on to Radiant’s tag parser. So for example, the ‘Powered by’ stuff is included with something like this:
<% if methods.member?("render") %><%= render :partial => "shared/powered" %><% else %><r:snippet name="powered" /><% end %>
You’ll need to set up a Radiant snippet called powered to go with this, or could just omit the ‘else’ part of the statement if you don’t want a Radiant equivalent at all.
I wanted commenting in the output source to check for errors, so you’ll notice that the Hub partial files include a closing HTML comment before the text and an opening one after. This only works if appropriate opening and closing comments appear in the shared footer, before the ERb code given above. If _powered.rhtml contains this:
-->Powered by Hub<!--
…Then using the following:
<!-- Powered start <% if methods.member?("render") %><%= render :partial => "shared/powered" %><% else %><r:snippet name="powered" /><% end %> Powered end -->
…Will produce this output:
<!-- Powered start -->Powered by Hub<!-- Powered end -->
The same thing is done for the theme comment, with associated partial _theme.rhtml in all applications except Radiant and an associated Radiant snippet called theme. In all, an example footer might look like this:
<p align="center">
Site design © Me...<br />
<!-- Theme start <% if methods.member?("render") %><%= render :partial => "shared/theme" %><% else %><r:snippet name="theme" /><% end %> Theme end --><br />
<!-- Powered start <% if methods.member?("render") %><%= render :partial => "shared/powered" %><% else %><r:snippet name="powered" /><% end %> Powered end -->
</p>
<p align="center">
<a href="http://validator.w3.org/check?uri=<%= url_encode(@request.protocol + @request.host + @request.request_uri) %>">Valid XHTML 1.0</a><br />
<a href="http://jigsaw.w3.org/css-validator/validator?uri=<%= url_encode(@request.protocol + @request.host + @request.request_uri) %>">Valid CSS</a>
</p>
You have to make sure that your pages really are valid HTML and CSS if you’re going to use the last two links! At least they make it easy for you to check, since you just follow the link on the page to perform the validation.
The Home page
You can test your header, footer and sidebar changes immediately in Hub. If you’ve got everything right you’ll have been exporting your shared files the moment you created them, overwriting the ‘borrowed’ ROOL files. Hub will show these changes immediately and give you an idea of how much Hub’s default layout, which (unless you have edited it) is still unchanged, works with the above snippets. Bear in mind that the styles for Hub are still coming from risc_os_open.css since that’s what its default layout uses.
If you try and view Radiant content, though, you will still redirect you to the administrative interface because you’ve not set up a Home page yet. We’ll do that now to complete the Radiant customisation process. This is easy; go the ‘Pages’ tab, use the ‘New Homepage’ button and set things up however you like. You can use the full range of behaviours. Don’t forget to set the ‘Status’ menu to ‘Published’ if you want the page to actually be visible. Set the ‘Layout’ menu to whatever you named for your standard default layout. Pages that are children of the Home page can be left to inherit this, but the Home page itself must specify the layout directly. For more information on the various options, consult the Radiant documentation on the Radiant home page.
ROOL’s Radiant includes another extension which in newer Radiant versions exists in other forms – an RSS feed summary generator. Radiant has a more generic parser available but the ROOL extension is a quick and easy way to get feed summaries. Set the ‘Behaviour’ menu to ‘News’ and use the following form of Radiant tag:
<r:news feed="http://www.riscosopen.org/news/xml/rss20/feed.xml" headlines="3" dates="1" https="1" escaped="1" />
You give the Radiant tag the number of headlines you want to list, whether or not to include dates, whether or not to fetch over HTTPS and indicate what you want to do about URL escaping. The last two parameters are a bit fiddly, so read the description in the file at ~/devel/rails/content/app/behaviors/news_behavior.rb, underneath all the comments at the start of the file. Other examples:
Body content! <r:news feed="http://www.riscosopen.org/news/xml/rss20/feed.xml" headlines="3" dates="1" https="1" escaped="1" /> <r:news feed="http://www.theregister.co.uk/headlines.rss" /> <r:news feed="http://rss.slashdot.org/Slashdot/slashdot" headines="6" /> End of body content.
Currently, the ‘News’ behaviour’s news tag cannot handle Atom feeds.
You’ll notice that Radiant always creates ‘body’ and ‘extended’ tabs above the main text area. These are page parts. You can usually delete the ‘extended’ one (click on the tab to select it, then use the red circular ‘-’ icon on the right). If you add a part (use the circular green ‘+’ icon on the right) called ‘sidebar’, the provided you kept the relevant line from the example layout given earlier, you can add more sidebar content on a per-page basis within that page’s sidebar part. As a reminder, the magic line in the layout was:
<r:content part="sidebar" />
It’s probably a lot more obvious now what this is actually doing and from this one example you should be able to see how quite complex systems can be built up inside Radiant alone, even before going near any of the other site applications.
All the examples above aren’t going to win any awards for aesthetics – but then, the look of the site is up to you :-)
Customising Hub
All you need to do with Hub is modify its default layout to suit your purposes, in ~/devel/rails/hub/app/views/layouts/default.rhtml (common leafnames for any application are default.rhtml or application.rhtml, though some applications choose something a bit more esoteric). Customising the application layout is something that needs to be done for each application you include in the site. Hub is a simple example and there are strong parallels between the layout you set for Hub and the layout you already set up in Radiant, so by getting these two things configured correctly, you will have done the lion’s share of site configuration.
The aim at the end of this stage is to have Radiant and Hub presenting static content and the Hub log-in mechanism under the integrated look and feel that you want for your entire site – images, colours, layout, the lot. If you get all that done as comprehensively as you can at this stage, it means minimum changes later when there are more applications to configure. You can deal with each application as you add it, setting it up completely in one go, rather than having to repeatedly make incremental changes to each application as you change your mind on a design feature. There’s usually no avoiding some amount of that kind of work, but it can be minimised.
As your site grows, evolves and perhaps occasionally changes its design, you’ll need to go back through the applications from time to time to make design changes. The more common features you can draw out into shared components, the easier this will be. You don’t have to limit yourself to the header, sidebar and footer mechanism from the ROOL site either; that’s just a starting point, giving an example of one way of dividing up the content.
The other applications
Now we get to add in the rest of the site’s applications, one by one. To start off with we’ll just do the simple applications with no new external tools dependencies.
- TODO – coming when I get time. The most important information is already here. Most applications are obvious – same steps as for Radiant – but some require Python and Perl tools which have extra configuration settings. The AWStats statistics analysis interface is handled separately and requires a dedicated configuration section in
lighttpd.conf. - TODO – advanced topics like using
var.pathprefixin LightTPD configuration to avoiddevelandlivecopies of the config file differing in content except for times when thedevelcopy is being experimentally modified;env.HOMEto avoid/home/usernameeverywhere; appendingvar.PIDto FCGI socket names to allowSIGINTto gracefully restart Lighty assuming a sufficiently new version of the server. - TODO – replace SERVER_DOCUMENT_ROOT with Lighty native variable if present? Replace definition with server.document-root rather than constructing it on the fly?
Automatic startup
If your service provider’s machine(s) are restarted, your server processes will get taken down. You need to make sure they’re restarted when the machine on which they run gets rebooted. Different service providers work in different ways; you need to have some way to, on reboot, run your database server, the DRb server and the web server in that order.
Adding new applications later
Hopefully, you’ve got your site running. What if you want to add new applications later, or upgrade or replace existing components?
- Consult HowToInstallApplicationsInSubdirectories at the Rails wiki.
- Look at the existing applications and see how they’re configured. For example, check out initial versions of applications from the ROOL Subversion repository, then check use
diffto see how the initial versions differ from current versions (or usesvn diffdirectly). - If you need to add new gems for your new applications, the HowToUseMultipleGemRepositories article on the Rails wiki may be useful.
Good luck!
Useful utilities
I find the following script, saved under the name copydb, a handy shortcut for copying databases with PostgreSQL:
# Copy a PSQL database. Assumes ROOL host. Syntax: # # ./copydb <old_db_name> <new_db_name> pg_dump -h /home/rool/postgres $1 > ~/temp/$1.dump createdb -h /home/rool/postgres -E=UTF8 $2 cat ~/temp/$1.dump | psql -h /home/rool/postgres $2 rm ~/temp/$1.dump
This assumes that a scrap directory called ~/temp exists. Obviously, you’ll need to replace the -h /home/rool/postgres details with your own database server connection details and may want to modify the flags given to createdb. I’m sure there are plenty of other ways to do this too.
Contacting me
If you want to get in touch about this article, you can contact me at ahodgkin@riscosopen.org. I can’t promise to offer full-on support to step you through each part of installation or solve every problem you might encounter because my free time is too limited, but I’ll do my best to answer queries. Feel free to send me suggested changes for this article if you find something that’s unclear, omitted or just plain wrong, using the same e-mail address.
Social
and
ROOL Store
Buy RISC OS Open merchandise here, including SD cards for Raspberry Pi and more.