skip to main content

TAMU Webmaster's Blog

Information and insight from the A&M Webmasters


Upgrade to CentOS 7

In my post about Ansible at the beginning of the year, I mentioned that we made the decision to migrate all of our virtual servers to CentOS 7 from SuSe Enterprise. The process is coming along steadily and we now have all of our HTML production and test servers, as well as our application test server, moved over to CentOS 7.3.

Other than a few differences, the change has been very smooth and easy. The major issues have not been related to changing distros, but some of the packages. The move from Apache 2.2 to 2.4 was probably the biggest just due to the syntax changes. Most of those are fairly minor though just to get the sites up and running again under the new version.

The main reason for the move was that we wanted to align ourselves more with what TAMUIT was moving toward. They have been migrating a lot of things over to CentOS as well, and we decided that now was a good time. A lesser reason was that when it comes to help, especially online, there is just so much more out there for Redhat/Fedora/CentOS than SuSe.  Most of the time this wasn’t a real issue, but every once in a while we ran into things that where much harder to troubleshoot in SuSe than if we had been using a Redhat or Ubuntu based distribution.

I will miss zypper though. I still think that overall it is a much better package manager than most any other one I’ve used. Yum is not bad now that I’m getting more used to it, but zypper just had so much built into it that makes things just a little easier to do and find.

Overall we are happy with the change and look forward to continuing with CentOS for the foreseeable future.

Tuesday, September 26th, 2017 Systems No Comments

WordPress CAS Authentication Issues

So apparently several of us around campus simultaneously experienced a similar issue recently with the CAS authentication plugin many of us use. The basic issue was that we were getting “Application Not Authorized” when trying to log into an HTTPS protected WordPress site using the CAS Maestro plugin. Our friend Donald St. Martin over in Engineering wrote up this great walk through of the problem and how to fix it.

HTTPS and the CAS Maestro WordPress plugin


Tuesday, January 31st, 2017 CMS, Systems, Uncategorized, Web Security No Comments


Solr is an open source enterprise search platform, written in Java, from the Apache Lucene project. Its major features include full-text search, hit highlighting, faceted search, real-time indexing, dynamic clustering, database integration, NoSQL features and rich document (e.g., Word, PDF) handling. Providing distributed search and index replication, Solr is designed for scalability and fault tolerance.

Databases and Solr have complementary strengths and weaknesses. SQL supports very simple wildcard-based text search with some simple normalization like matching upper case to lower case. The problem is that these are full table scans. In Solr all searchable words are stored in an “inverse index”, which searches orders of magnitude faster.

Solr exposes industry standard HTTP REST-like APIs with both XML and JSON support, and will integrate with any system or programming language supporting these standards. For ease of use there are also client libraries available for Java, C#, PHP, Python, Ruby and most other popular programming languages.





Wednesday, November 16th, 2016 Site Architecture, Systems, Web Content No Comments

WPEngine Update

So we have been using WPEngine for several months now and been able to go through a major site launch.  Sometimes making a move like this can take a while to fully evaluate, so I thought I would just give another update after having used the service for a while.

Honestly, we’ve been very happy with the experience.

WPEngine’s staging feature has been an invaluable tool in getting sites prepped and tested before being easily pushed into production. Another massively helpful use of this is in testing plugin and theme updates before applying them. Simply copy your site to the staging area, apply the updates, and then verify the results. This gives you a nearly sure-fire way of knowing if an update will break your site or not.

Also, we have found their tech support to be great. It is easy to get a hold of someone knowledgeable, and their employees are very empowered to do a lot without having to go through an escalation process. The few times we have had an issue, they were able to quickly find and fix them with one phone call. When we launched Lead By Example, we called to let them know we had an important site coming online. They went out of their way to check out everything for us from their end to make sure it all went smoothly.

They also have a pretty good status page at You can subscribe to the service and get email alerts for any issues that might affect you. Thankfully it is usually pretty quiet, but it is nice to have the additional notifications.

That’s about it for now. As always, let us know if you have any questions. We’d be happy to talk further with anyone about WPEngine or anything else on your mind.


Tuesday, October 11th, 2016 CMS, Systems No Comments

We’ve Moved!

Recently, we completed our move to WPengine to host all of our WordPress sites, including this blog.  It involved some untangling of multi-site installs, as well as the normal bumps that come with moving to a new way of doing things, but overall the process was fairly smooth. We feel like this move will help us in the future, not only with keeping everything updated and current, but also speeding up our process of getting out new sites as the need arises.

I love running my own machines, but in this instance it just made too much sense for us to move to a hosting service. WPengine was very helpful on the few occasions when we needed it and were very knowledgeable. There are also a number of features that will be very useful in the future that were not easily done on our end.  In all, we have been happy with the results so far and are looking forward to being a little more flexible and agile in the future.

There have been a number of groups around campus that have moved to WPengine. If you are considering a move to a hosted service, or have other questions, feel free to reach out to us. We’d be happy to discuss it with you.

Thursday, May 19th, 2016 CMS, Systems No Comments

“Today” Cache Troubleshooting (Part 1)

As I’ve mentioned before we’ve been using Varnish to improve our performance on WordPress sites for a while now. Though we are on our third iteration of the cache server, there always seems to be a few kinks to work out when we launch something new.  The “Today” news site is no exception.

Our first release on Today was not as smooth as we wanted. The performance was way below what we were expecting, and below what we saw on Tamutimes. We discovered that something was circumventing the cache server and directly accessing the backend web and database servers. Upon further inspection it was discovered that the culprit was some redirects from the old Tamutimes site. After fixing those, things were much smoother.

My point in sharing this is that, I thought I would take this post and the next going over two of the built-in Varnish tools for monitoring that helped us identify what was happening.


Varnishhist is a neat tool that basically just shows a graphic of what is going on with your cache server. It is invoked just by running the following on the command line:



And here is an example of what our cache server looks like at a normal time:

Screenshot from 2015-02-02 16:03:11


So reading this is fairly simple. It really is just showing the cache “hits” and “misses” and the time it took to serve the request. The “|” symbols are the hits and “#” represents a miss, or a call to the backend server. Along the bottom is the time to serve the request. Basically the more to the left the hits are the better. 1e0 is equal to one second, so 1e-1 is equal to 1/10th and 1e-6 is equal to 0.000001 or around one microsecond. Anything in the -4 to -5 range is pretty good. 

Also from this graph you can roughly see that almost all of the requests are hitting the cache. None of these requests are even making it to the Apache server. This is massively helpful with WordPress. You can see how much slower the misses are when they have to go all the way through Apache and likely make a database call and return that information.

When we launched the Today site, we were seeing almost the exact opposite of this graphic. The peak was all on the misses side at around 1/10th to 1 second. We knew something was wrong immediately. While Varnishhist doesn’t show a lot of detail, it can be very helpful to have running to give you a very quick look at what is going on.

Next post I will go over the other main tool – varnishstat.

Thursday, February 5th, 2015 Systems No Comments


We have been using Varnish to cache our WordPress sites (such as this one and Tamutimes) for several years now. We feel that it has worked really well for our needs, adding a bit more speed to our sites to help with higher traffic days.

Varnish is highly configurable and is transparent to its backend so it doesn’t require any code modification on your web servers. The way we have configured our installation is to use RAM as the cache for our static content which can serve out requests very quickly.  In addition, it saves on requests on the backend so those requests that do get sent to it get served out a bit quicker as well. All this makes for a snappier experience on the users’ end.

Another one of the great things about this setup is that we can also cache multiple sites on different machines. This means we can just put the necessary CNAMES on the varnish server and then point them to different machines as needed. The only thing required is a restart of the Varnish service to make the change. This helps a huge amount when we are migrating sites to new machines.

Overall we’ve been very happy with Varnish. For WordPress especially it has given us a little extra peace of mind that it can help out when there will be more traffic. I really encourage anyone to look into it if you run your own web servers and see if it could help you out.

Varnish Website

Varnish on Wikipedia

Monday, September 15th, 2014 Systems No Comments

In The Beginning…

We feel like we’ve come a long way since we started re-vamping the Marketing & Communications computing environment, so I thought I would make my first “real” post just an outline of where we came from…

At times we have lamented not having a picture of what my office looked like when I first started working with Marketing & Communications in 2006. To say it was “less than ideal” is putting it kind of mildly. From an office crammed full of servers with only a window unit for cooling, to our current setup has been quite a big change. Having that picture would have been a nice reminder.

When I came on, everything was located in Bizzell Hall in one office. Obviously from a power and cooling standpoint this was not the saftest environment. No redundant power. No central AC. No AC backup. Not good. After some discussion with our administration we were able to get approval to move our mail and main file servers to the Teague machine room. This was a huge step. Fortunately our backup procedure was pretty good so we were much better off after that initial change.

Slowly over the next year or two we were able to aquire new hardware and transition all of our web services machines to Teague. After adding some dedicated database (mySQL and MSSQL) servers, we had most all of our core equipment and production servers in a much more stable and conrolled environment. It was around this time (2008ish) that we started looking at virtualizing machines. We previously had most every service separated out on its own hardware. Starting with stand alone VMWare ESX machines, we were able to start consolidating machines and services into a more manageable virtual machine environment. This allowed much better management features and better efficiency with our hardware.

In 2011 we started the process of moving toward our current setup by beginning to transition to the new CIS Cloud services. This gave us access to the full blown VMware suite and feature set which previously had been out of our price range. This is mostly where we stand now, being able to let CIS manage the backend system letting us concentrate on the server managment side.

Tuesday, August 5th, 2014 Systems No Comments

Hello World!

Since this is the first time I am posting to the Webmaster blog, I thought I would go ahead and just make this a general introduction.

“Hi, I’m Charlie.”

I’m the system administrator as well as general “IT guy” in Marketing & Communications. Simply put, my job is to make sure all the sites and applications we develop are up, running and accessible. I started here in 2006 as a part of CIS’s Customer Applications group. Over the next three years or so, I slowly moved from half-time to full time at Marcomm. Eventually in 2010 they decided just to hire me directly as a regular employee and I have been here since.

My main duties consist of taking care of all our web and application servers which are currently housed with the CIS Infrastructure group VM service. We do have a few machines on our own hardware remaining in the Teague machine room which we hope to have moved by the end of the summer. In my next few posts I hope to outline some of our server hardware and services history and outline some of the decisions we made along the way. Later on I hope to talk about ways we are trying to improve the way we do things and keep up with all the interesting new technologies out there.

Looking forward to it!

Thursday, July 10th, 2014 Systems No Comments

Serving Up TAMUtimes

In a blog post two weeks ago, Erick introduced our latest website, TAMUtimes. While that post is a great introduction to the idea of TAMUtimes it doesn’t shed much light on the technical hurdles of serving the site and emails to the audience. In this post, I’ll discuss the technologies that allow us to handle the traffic generated from bulk emails and subsequent page views.

The TAMUtimes application server

As he mentioned the application is running on WordPress. What he didn’t mention is it is running as part of a WordPress 3.0 multisite installation along with two other blogs we maintain. The application itself is served by Apache and has a MySQL service running behind it. As you can probably imagine, this is an incredibly slow configuration.

Under normal operations, with the help of a simple caching plugin, this configuration would limp by and we’d be able to expand it if demand for any of the sites increased. But we didn’t have such a luxury on a site whose main traffic would be generated by a bulk email that would eventually be sent to over 300,000 individuals relatively simultaneously. We needed something to handle the load.

Enter a reverse-proxy cache

You might have already guessed, but this site is a perfect candidate for a reverse proxy. After some discussion, we chose to give Varnish a try. I experimented with Varnish on some personal projects, so I already had an idea of the impact it could have sitting in front of Apache serving up WordPress projects, so it wasn’t a difficult decision.

I ran some benchmarks using the Apache Bench tool (ab2) and found the Varnish server increased the mean number of requests per second we could handle by roughly 6,500%. In the benchmarks, the server was able to complete 959 requests in 200 seconds without Varnish, whereas it completed 50,000 requests in 158.8 seconds with the proxy cache in place.

One step further

At the same time, we noticed that other emails being sent to large distribution lists also caused some lag on some of our servers (because of the images included in the email.) Modern practices dictate that we should move our images to a separate domain name (or subdomain in our case) so that’s what we did. We stood up a server running Nginx to handle static assets such as image, stylesheets, JavaScript source, etc. with far-future expiration dates and a few other tweaks. We moved the TAMUtimes images (as well as many other images) to the static assets server, and haven’t looked back.

The final result

As a result, when the email goes out for TAMUtimes, we don’t see a spike on the Apache server—whereas in my benchmarks it was customary to approach the “time to swap” line if not to cross it and not look back. We haven’t yet seen a spike in usage on the assets server, and I don’t foresee one anytime soon.

The moral of the story is to always benchmark your applications and your systems serving them up well in advance of deployment. If we hadn’t, several of our systems would have fallen flat on their proverbial faces.


Friday, September 16th, 2011 Miscellaneous, Systems No Comments