skip to main content

TAMU Webmaster's Blog

Information and insight from the A&M Webmasters


Serving Up TAMUtimes

In a blog post two weeks ago, Erick introduced our latest website, TAMUtimes. While that post is a great introduction to the idea of TAMUtimes it doesn’t shed much light on the technical hurdles of serving the site and emails to the audience. In this post, I’ll discuss the technologies that allow us to handle the traffic generated from bulk emails and subsequent page views.

The TAMUtimes application server

As he mentioned the application is running on WordPress. What he didn’t mention is it is running as part of a WordPress 3.0 multisite installation along with two other blogs we maintain. The application itself is served by Apache and has a MySQL service running behind it. As you can probably imagine, this is an incredibly slow configuration.

Under normal operations, with the help of a simple caching plugin, this configuration would limp by and we’d be able to expand it if demand for any of the sites increased. But we didn’t have such a luxury on a site whose main traffic would be generated by a bulk email that would eventually be sent to over 300,000 individuals relatively simultaneously. We needed something to handle the load.

Enter a reverse-proxy cache

You might have already guessed, but this site is a perfect candidate for a reverse proxy. After some discussion, we chose to give Varnish a try. I experimented with Varnish on some personal projects, so I already had an idea of the impact it could have sitting in front of Apache serving up WordPress projects, so it wasn’t a difficult decision.

I ran some benchmarks using the Apache Bench tool (ab2) and found the Varnish server increased the mean number of requests per second we could handle by roughly 6,500%. In the benchmarks, the server was able to complete 959 requests in 200 seconds without Varnish, whereas it completed 50,000 requests in 158.8 seconds with the proxy cache in place.

One step further

At the same time, we noticed that other emails being sent to large distribution lists also caused some lag on some of our servers (because of the images included in the email.) Modern practices dictate that we should move our images to a separate domain name (or subdomain in our case) so that’s what we did. We stood up a server running Nginx to handle static assets such as image, stylesheets, JavaScript source, etc. with far-future expiration dates and a few other tweaks. We moved the TAMUtimes images (as well as many other images) to the static assets server, and haven’t looked back.

The final result

As a result, when the email goes out for TAMUtimes, we don’t see a spike on the Apache server—whereas in my benchmarks it was customary to approach the “time to swap” line if not to cross it and not look back. We haven’t yet seen a spike in usage on the assets server, and I don’t foresee one anytime soon.

The moral of the story is to always benchmark your applications and your systems serving them up well in advance of deployment. If we hadn’t, several of our systems would have fallen flat on their proverbial faces.


Friday, September 16th, 2011 Miscellaneous, Systems No Comments

Introducing TAMUtimes

I’m sorry it’s been so long since our last post. We were given a new project several months ago that, for various reasons, we have not been able to disclose or discuss. Now that the veil is lifted, I can reveal the background of the TAMUtimes project.

TAMUtimes grew out of the need for updating the old TAMU News website and the Aggie Hotline email. It combines all of our marketing, communication, and PR efforts under one umbrella. It will effectively take the place of the old news vehicles and replace them with a centralized service.

The central mandate for the new service was that it should place Marketing & Communications in the role of aggregating content from around campus rather than being the source of publication. That allows communicators from within the colleges, who are closer to the actual story, to have a greater hand in getting our word out. It also adds a layer of efficiency since staff who had been tasked with duplicating effort could be reassigned to more productive duties.

The TAMUtimes project is a marriage between the traditional online website and subscription-based email. Using the Maestro platform from CIS we create and send out an HTML email twice a week with featured news stories from around campus. The features from this email are automatically added to the website as the feature block there, keeping the email and the website in synch.

In keeping with the mandate to aggregate content, most of the information on the site is pulled in through various RSS feeds. Only a small amount of content, mostly internal university news, is actually entered into the system by our staff.

The website itself is based on WordPress, though heavily customized. To be honest, if I had to start over on the project, I probably would have gone a different route, but we just didn’t have the expertise yet in our content management system to allow the level of custom programming that we needed in the time allowed. The end result is a bit of a Frankenstein’s monster behind the scenes, but hopefully with enough cosmetic surgery that the outward appearance isn’t so frightful.

As we move forward we will soon be integrating elements of TAMUtimes into the university website and other sites we maintain. This will allow us to integrate one more major university component into our ongoing concept of creating a unified web presence out of our individual previously-disconnected web silos.

Tags: ,

Friday, September 2nd, 2011 News 4 Comments