Hello lazy web, I’m looking for some advice on caching proxy servers and thought you might have some good pointers!
We currently front our site with 3 caching proxy servers to offload static content from the web servers and we haven’t found the right solution yet. We’ve tried a number of solutions including Oops and Apache with mod_cache (which we’re currently running) but they are either missing a critical feature, have buggy caching behavior, or crash under heavy load. The main features I’m looking for are:
1. Caches static content and pages and respects the expires header
2. Can gzip content on the fly (this saves us a bundle in bandwidth costs)
3. Can easily clear the cache when we move a new release live
I’d prefer an open-source product but if we end up needing to go with a commercial solution as a last resort I’m open to that as well, thanks for your suggestions or help!
Did you try Squid?
Cheers,
Matthew
Yes, when we tried Squid it didn’t seem to have gzip content compression support which is a must have for us. The commercial solutions like F5, Cisco, and Zeus have it and we have it working with Apache and Mod_proxy but we’re finding that Mod_cache is buggy and occasionally crashes under heavy load.
Can you do the gzipping in the web application, not at the proxy? Compression consumes some CPU, so the proxy seems not the right place. If contents are to be cached it would be no big deal.
I also posted this question to Linked in:
http://www.linkedin.com/answers/technology/information-technology/computer-networking/TCH_ITS_CNW/47885-8167?goback=%2Eahp
It turns out that they have now added gzip compression to squid in the HEAD branch. Additionally our old sysadmin who works at MySQL now recommended we try Lighttpd with mod_cache and mod_compression so we’ll be doing some testing with both of those solutions.
Ignacio, we’ve found that putting the gzip compression on the proxies balances out the CPU load better, otherwise the proxies just sit there with a 0.01 load avg doing IO.
Yep, that is expected. Now, if by gzipping you mean that the file is being compressed _before_ caching, that is fine, else you are just wasting CPU cycles at your proxies.
I would bet that apache would most probably _not_ gzip before caching, but squid will probably do. Just a guess.
How about Varnish, http://varnish.projects.linpro.no ?
It’s been a year and a half since I wrote this blog post and Greatschools has been using ZXTM (which is commercial) and have extremely happy with it. It’s an amazing product with incredible in-depth monitoring and scripting:
http://www.zeus.com/products/zxtm/
Over the past year and half the situation hasn’t changed much on gzip/deflate situation with squid so I still don’t think there’s a great open-source site accelerator offering but I hope that changes!