About the design and implementation of Catz
Written in 2011 so may contain outdated information.
The key attributes
- The service is stateless: the HTTP GET request's URL identifies the resource requested. There are no cookies nor sessions.
- The public access is dynamic but read only. All data is loaded offline by a loader sumsystem.
- There is one simple straightforward layout and graphical design that brings you straight to the heart of the service. There is no fancy intro pages or other things to waste my or visitors' time with.
- The single layout is designed to be usable without scrollbars from portrait mobile devices to landscape desktop screens. Its fanciness is sacrificed.
- This is a dual-language system: English and Finnish. Everything is available in both languages and you can switch languages on all pages. There will be no more languages in the future.
- Navigation is provided via short text hyperlinks instead of icons.
- The minimum amount of nonsense is provided.
Technologies used
Here are some of the technologies chosen during the project. My target was to choose minimal elegant lightweight technologies.
| The technology chosen | Why | |
|---|---|---|
| Core language | Perl | I had been using Perl for a long time but my skills with it were limited. I chose Perl because the language is highly effective and this project was a good opportunity to learn how to master the language in an modern way. |
| Framework | Mojolicious MVC Framework | I was aware of Mason and mod_perl but now I thought that it was a time to give MVC a go. [FrameworkComparison I tested several Perl MVC frameworks] and chose Mojolicious. |
| Database | SQLite | [DatabaseComparison I considered both Postgres and SQLite] for the project but at the end SQLite was a natural choice since it offered all I needed in a minimal package. |
| Templating engine | Mojoicious' built'in ep | Is fast and feature rich and included in Mojolicious. Uses Perl as control langauge so there is no need to learn a separate langauge. |
| Caching | Memcached | Very reliable and tested caching solution to throw in key-value -pairs and to get them evicted on a LRU rule when the memory gets filled. |
| JavaScript library | JQuery | Widely used industry standard |
| Production OS | Debian | It has all this system needs |
Implementation
The system is based on Model-View-Controller paradigm: Controllers drive the system. Models provide data from database. Templates contain the HTML code. So the layout (templates). Both object oriented and procedural programming styles are used. There are plenty of procedural modules to provide needed functionality and utilites for the system. CPAN modules are used if they were found handy.
Basically all visible text strings are separated from the templates and provided to the templates via hash reference $t. There are a few pages that have plenty of written text and for those pages the text is in the template.
Data management
Models access database via DBI and SQL. There is no ORM or other fancy things. This is mainly because I can write SQL inside out and because the runtime access to database is read only.
Data is loaded offline from tagged text files to SQLite database. The database is versioned by making each version a database file. Database files are named YYYYMMDDHHMISS.db so the version tag is stated already in the filename. New data version (new file) is detected automatically by the production system and the production switches to use it immediately invalidating cached data. Also rolling backwards is possible.
DBI access to database data is array-based and so positional. My tests indicate that access by column name that requires construction of hashes generates a significant overhead with large result sets.
Caching
Caching has been taken into extremes. A simpler caching mechanisms could do just fine. Database version is a triggering factor with the caching: all cache data is considered valid until the database version changes forward. At that point all cached data is considered invalid.
Server side caching
On the server side cache gets rendered pages, Model responses and SQL statement resultsets. Memcached with big memory allocation eats all these via Cache::Memcached::Fast and starts to evict old entries based on the default Memcached LRU behavior. The caching key always includes the database version so there is no need to set expiry time. With an new database version all keys change and stale content is left unused.
Client side caching
Nginx servers photos and some other static assets. They get 7 days of client side cache lifetime and after that if-modified-since should come into play.
Other resources are served by Hypnotoad/Mojolicious. I have found it easiest to set a fixed one hour lifetime for all content served by the backend. Both expires and max-age headers are set according to this rule. The application sends the database version number (YYYYMMHHDDHH24MISS) formatted to last-modified header making it possible for the clients to verify the cached content by sending if-modified-since.
Testing
The system has a comprehensive automated functionaliy test suite based on Test::More and Test::Mojo. Currently there is no other levels of automated testing. Only the application functionality gets tested automatically.