Changesets can be listed by changeset number.
The Git repository is here.
- Revision:
- 373
- Log:
Initial import of Radiant 0.9.1, which is now packaged as a gem. This is an
import of the tagged 0.9.1 source checked out from GitHub, which isn't quite
the same as the gem distribution - but it doesn't seem to be available in an
archived form and the installed gem already has modifications, so this is
the closest I can get.
- Author:
- rool
- Date:
- Mon Mar 21 13:40:05 +0000 2011
- Size:
- 7587 Bytes
| 1 | Storage |
| 2 | ======= |
| 3 | |
| 4 | __Rack::Cache__ runs within each of your backend application processes and does not |
| 5 | rely on a single intermediary process like most types of proxy cache |
| 6 | implementations. Because of this, the storage subsystem has implications on not |
| 7 | only where cache data is stored but whether the cache is properly distributed |
| 8 | between multiple backend processes. It is highly recommended that you read and |
| 9 | understand the following before choosing a storage implementation. |
| 10 | |
| 11 | Storage Areas |
| 12 | ------------- |
| 13 | |
| 14 | __Rack::Cache__ stores cache entries in two separate configurable storage |
| 15 | areas: a _MetaStore_ and an _EntityStore_. |
| 16 | |
| 17 | The _MetaStore_ keeps high level information about each cache entry, including |
| 18 | the request/response headers and other status information. When a request is |
| 19 | received, the core caching logic uses this meta information to determine whether |
| 20 | a fresh cache entry exists that can satisfy the request. |
| 21 | |
| 22 | The _EntityStore_ is where the actual response body content is stored. When a |
| 23 | response is entered into the cache, a SHA1 digest of the response body content |
| 24 | is calculated and used as a key. The entries stored in the MetaStore reference |
| 25 | their response bodies using this SHA1 key. |
| 26 | |
| 27 | Separating request/response meta-data from response content has a few important |
| 28 | advantages: |
| 29 | |
| 30 | * Different storage types can be used for meta and entity storage. For |
| 31 | example, it may be desirable to use memcached to store meta information |
| 32 | while using the filesystem for entity storage. |
| 33 | |
| 34 | * Cache entry meta-data may be retrieved quickly without also retrieving |
| 35 | response bodies. This avoids significant overhead when the cache misses |
| 36 | or only requires validation. |
| 37 | |
| 38 | * Multiple different responses may include the same exact response body. In |
| 39 | these cases, the actual body content is stored once and referenced from |
| 40 | each of the meta store entries. |
| 41 | |
| 42 | You should consider how the meta and entity stores differ when choosing a storage |
| 43 | implementation. The MetaStore does not require nearly as much memory as the |
| 44 | EntityStore and is accessed much more frequently. The EntityStore can grow quite |
| 45 | large and raw performance is less of a concern. Using a memory based storage |
| 46 | implementation (`heap` or `memcached`) for the MetaStore is strongly advised, |
| 47 | while a disk based storage implementation (`file`) is often satisfactory for |
| 48 | the EntityStore and uses much less memory. |
| 49 | |
| 50 | Storage Configuration |
| 51 | --------------------- |
| 52 | |
| 53 | The MetaStore and EntityStore used for a particular request is determined by |
| 54 | inspecting the `rack-cache.metastore` and `rack-cache.entitystore` Rack env |
| 55 | variables. The value of these variables is a URI that identifies the storage |
| 56 | type and location (URI formats are documented in the following section). |
| 57 | |
| 58 | The `heap:/` storage is assumed if either storage type is not explicitly |
| 59 | provided. This storage type has significant drawbacks for most types of |
| 60 | deployments so explicit configuration is advised. |
| 61 | |
| 62 | The default metastore and entitystore values can be specified when the |
| 63 | __Rack::Cache__ object is added to the Rack middleware pipeline as follows: |
| 64 | |
| 65 | use Rack::Cache do |
| 66 | set :metastore, 'file:/var/cache/rack/meta' |
| 67 | set :entitystore, 'file:/var/cache/rack/body' |
| 68 | end |
| 69 | |
| 70 | Alternatively, the `rack-cache.metastore` and `rack-cache.entitystore` |
| 71 | variables may be set in the Rack environment by an upstream component. |
| 72 | |
| 73 | Storage Implementations |
| 74 | ----------------------- |
| 75 | |
| 76 | __Rack::Cache__ includes meta and entity storage implementations backed by local |
| 77 | process memory ("heap storage"), the file system ("disk storage"), and |
| 78 | memcached. This section includes information on configuring __Rack::Cache__ to |
| 79 | use a specific storage implementation as well as pros and cons of each. |
| 80 | |
| 81 | ### Heap Storage |
| 82 | |
| 83 | Uses local process memory to store cached entries. |
| 84 | |
| 85 | set :metastore, 'heap:/' |
| 86 | set :entitystore, 'heap:/' |
| 87 | |
| 88 | The heap storage backend is simple, fast, and mostly useless. All cache |
| 89 | information is stored in each backend application's local process memory (using |
| 90 | a normal Hash, in fact), which means that data cached under one backend is |
| 91 | invisible to all other backends. This leads to low cache hit rates and excessive |
| 92 | memory use, the magnitude of which is a function of the number of backends in |
| 93 | use. Further, the heap storage provides no mechanism for purging unused entries |
| 94 | so memory use is guaranteed to exceed that available, given enough time and |
| 95 | utilization. |
| 96 | |
| 97 | Use of heap storage is recommended only for testing purposes or for very |
| 98 | simple/single-backend deployment scenarios where the number of resources served |
| 99 | is small and well understood. |
| 100 | |
| 101 | ### Disk Storage |
| 102 | |
| 103 | Stores cached entries on the filesystem. |
| 104 | |
| 105 | set :metastore, 'file:/var/cache/rack/meta' |
| 106 | set :entitystore, 'file:/var/cache/rack/body' |
| 107 | |
| 108 | The URI may specify an absolute, relative, or home-rooted path: |
| 109 | |
| 110 | * `file:/storage/path` - absolute path to storage directory. |
| 111 | * `file:storage/path` - relative path to storage directory, rooted at the |
| 112 | process's current working directory (`Dir.pwd`). |
| 113 | * `file:~user/storage/path` - path to storage directory, rooted at the |
| 114 | specified user's home directory. |
| 115 | * `file:~/storage/path` - path to storage directory, rooted at the current |
| 116 | user's home directory. |
| 117 | |
| 118 | File system storage is simple, requires no special daemons or libraries, has a |
| 119 | tiny memory footprint, and allows multiple backends to share a single cache; it |
| 120 | is one of the slower storage implementations, however. Its use is recommended in |
| 121 | cases where memory is limited or in environments where more complex storage |
| 122 | backends (i.e., memcached) are not available. In many cases, it may be |
| 123 | acceptable (and even optimal) to use file system storage for the entitystore and |
| 124 | a more performant storage implementation (i.e. memcached) for the metastore. |
| 125 | |
| 126 | __NOTE:__ When both the metastore and entitystore are configured to use file |
| 127 | system storage, they should be set to different paths to prevent any chance of |
| 128 | collision. |
| 129 | |
| 130 | ### Memcached Storage |
| 131 | |
| 132 | Stores cached entries in a remote [memcached](http://www.danga.com/memcached/) |
| 133 | instance. |
| 134 | |
| 135 | set :metastore, 'memcached://localhost:11211/meta' |
| 136 | set :entitystore, 'memcached://localhost:11211/body' |
| 137 | |
| 138 | The URI must specify the host and port of a remote memcached daemon. The path |
| 139 | portion is an optional (but recommended) namespace that is prepended to each |
| 140 | cache key. |
| 141 | |
| 142 | The memcached storage backend requires [Evan Weaver's memcached client library][e]. |
| 143 | This is a [fast][f] client implementation built on the SWIG/[libmemcached][l] C |
| 144 | library. The library may be installed from Gem as follows: |
| 145 | |
| 146 | sudo gem install memcached --no-rdoc --no-ri |
| 147 | |
| 148 | Memcached storage is reasonably fast and allows multiple backends to share a |
| 149 | single cache. It is also the only storage implementation that allows the cache |
| 150 | to reside somewhere other than the local machine. The memcached daemon stores |
| 151 | all data in local process memory so using it for the entitystore can result in |
| 152 | heavy memory usage. It is by far the best option for the metastore in |
| 153 | deployments with multiple backend application processes since it allows the |
| 154 | cache to be properly distributed and provides fast access to the |
| 155 | meta-information required to perform cache logic. Memcached is considerably more |
| 156 | complex than the other storage implementations, requiring a separate daemon |
| 157 | process and extra libraries. Still, its use is recommended in all cases where |
| 158 | you can get away with it. |
| 159 | |
| 160 | [e]: http://blog.evanweaver.com/files/doc/fauna/memcached/files/README.html |
| 161 | [f]: http://blog.evanweaver.com/articles/2008/01/21/b-the-fastest-u-can-b-memcached/ |
| 162 | [l]: http://tangent.org/552/libmemcached.html |
