ShoutMeLoud – Shouters Who Inspire

Superlinks
≡ Menu
≡ Menu

The Biggest Downside Of WPEngine Managed Hosting : Overage Charge

bluehost
The Biggest Downside Of WPEngine Managed Hosting : Overage Charge

When I shifted to WpEngine hosting few months back,  I shared the complete guide on moving and also added why I’m loving their hosting. If you have missed all these details, here are few things which you should know:

Before moving to WpEngine, I was hosted on a private VPS and one of the freelancer networking guy was managing my server. The performance was good but there was few issues related to downtime and site slowness. More over, one biggest problem with freelancer or less known hosting company is the accountability. So I decided to spend $250/month on WPEngine hosting, and moved ShoutMeLoud to their server. Without any second thought, WPEngine delivers what they promised, and so far I’m in love with the quality of hosting and website performance. Though $250/month is indeed a high-price, and any knowledgable guy would recommend to go for a VPS or dedicated server. In my case, I decided to stick to WPEngine, as I don’t want to see even a second downtime for my main blog, and when quality is everything, price is the least of concerns.

So What’s the problem with WPEngine?

Two of the biggest complain you will hear about WPEngine is Pricing and customer support. In past 2 months, They have significantly improved on their customer support by adding live-chat, but the pricing issue remain the same. After using WPEngine for more than 2 months, I encountered one of the biggest pricing issue with WPEngine. This post is about the problem and the possible solution to the problem.

WPEngine Overcharged

First of all, WPEngine pricing is based on the visits and the way they count the visits is different from what you see on your Google analytics or WordPress.com stats program. According to ShoutMeLoud Google analytic stats, I get close to 325,000 visits a month, and I opted for WPEngine Business plan which allows up to 400,000 visits a month, and is priced at $250.

Last time when I logged into my WPEngine dashboard, I was surprised to see an extra $188 bill for overage visits, and for this month, I was about to cross the threshold of 400,000 visits. I was quite surprised as, my WordPress.com stats and Google analytics was showing lower than 400,000 visits. Before I raise my concern to their team, I again had a look at their help page related to visits, and I realised I missed understanding few important points. Here is what their visits help page has to say:

  1. When a human being first arrives on the site and loads the page, staying there for 31 seconds, that’s a visit.
  2. If that same human then clicks a link and sees another page, that’s not anew visit; that’s part of the same visit.
  3. If that same human doesn’t have cookies or javascript enabled, still all that should count as one visit.
  4. If that same human loads the site with a different browsers, that’s still not anew visit; that’s part of the same visit.
  5. If that same human bookmarks the site, then 11 days later comes back to the site, that is a new visit.
  6. When a robot loads the site (like a Google or Bing search bot), that’s a visit, but if one robot scans 100 pages quickly, that’s one visit.  (You might disagree that a robot is a “visit,” but consider that from a hosting perspective, we still had to process and serve all those pages just like it was a human being, so from a cost or scaling perspective, bots count the same as humans.)
  7. If a robot scans 20,000 pages over the course of a month, that’s not just one visit.  It shouldn’t be 20,000 visits, but neither should it be 1. Something in the range of 100-1,000 visits is acceptable.
  8. There are additional cases too where the “right thing to do” is less clear. For example, take the case of a “quick bounce.” Suppose a human clicks a link to the site, then before the site has a chance to load the human clicks “back.” Does that count as a visit? Our servers still had to render and attempt to return the page, so in that sense “yes.”  But a human didn’t see the site and Google Analytics isn’t going to see that hit, so in that sense “no.” Because we need the notion of a “visit” to correspond to “the amount of computing resources required to serve traffic,” we round off in favor of saying “yes.”

Two important points that I missed reading on the same page was:

  • We reset our notion of “unique IP address” every day.
  • Robots have the same few IP addresses, so they will be consolidated within one day, but will count again the next day.

Though, everything was well-written but I missed reading the point about resetting the unique I.P. Address every day, and even bot visits are counted as new visits after every 24 hours. ShoutMeLoud is one active blog, and not only Google bots, but there are virtually many Good + bad bots which crawl ShoutMeLoud daily, and I realised that added a lot to my number of visits according to WPEngine.

First of all, this notion of counting visits from WPEngine is not only flawed, but it definitely adds a big hole in the pocked of any existing WPEngine users.  As I was considering to move away from WPEngine hosting after knowing how they are counting visits, and certainly this is not a customer-friendly model of pricing, I wanted to see if I could do something to make my WPEngine stats closer to my Google analytic stats. I asked them few questions, which will help any existing WPEngine user or anyone who is planning to purchase hosting from WPEngine:

Q) Image hotlinking: So if an image is hot linked somewhere, and it’s access by a person on site x (which hot linked my images), will that be counted as a visit?

Ans.)Yes, that counts as a visit. Hotlinking can be blocked, but CloudFlare is a simpler solution, with advantages because it works on the DNS level.

Q) If an image which is hosted on my server and is opened anywhere will that be counted as visits?

Ans.) Right, counts as a visit.

Q) CDN: If both the above answers are yes, and if I serve my images using CDN powered by WPEngine will that be counted as visits?

Ans.) Any request that hits CDN instead of the server does not count as a visit. So CDN is a helpful tool as well. But if these hotlinks pre-exist the enabling of CDN, it’s a tougher issue to address. Again, because CloudFlare is at the DNS level, it’s a simple, comprehensive solution.

Q) I see that WpEngine counts bad bots as visits too, do you have a solution for blocking these bad bots/spam/scrapers bots?

Ans.) CloudFlare allows you settings to fight abusive bots. But your bot traffic looks healthy and legit; although you did have over 5k visits from the bot for the popular Chinese search engine “Baidu.”

Q) Why does Google Analytics show a much lower visitor number?

Ans.) The number of visitors in Google Analytics is always going to be considerably lower. If you type the question into a Google search you will see many people out there with various hosts are experiencing the same thing. The reason being is GA tries very hard to provide information only on human visitors that matter for Ad Traffic and will not include any visitors that have JavaScript disabled on their browsers.

A unique IP is counted as unique in an entire month, while our system looks at unique IPs on a per day basis. They also will not include Google bots or various other bots in their visitors count. GA is used for marketing/advertising purposes and that is why it works so hard to only count what they believe to be human visitors. Our server stats monitoring tracks all unique visitors because from a server impact standpoint, a bot or a human is still pulling data down from the server and using CPU resources in the process.

Q) Is there a problem or something wrong with the GA implementation?

Ans.) Absolutely not. They have a specific purpose, to try and show you how many human visitors have come to your site. Our own stats has a specific purpose to show from a server impact perspective how much data is being transferred over our servers

Q) Also wondering if you could look into my above question related to WordPress.com stats program? Is it also flawed? or they work different than yours?

Ans.)  I wouldn’t say those stats are flawed. It’s just a tool that’s designed more for advertising purposes, for example to measure “reach”. A good way to get stats that accurately reflect the traffic that actually hits the server is to see the stats (like “Daily Usage CSV”) available in your User Portal.

WpEngine cons

How to lower down WpEngine Bills?

First of all, despite of liking everything about WpEngine, I hate the way they are over-charging their customers. I always considered WPEngine to be a customer-centric company, and they have implemented many great features like Website security, free DNS, Website backup, Staging area, and many more to give important features to their customers. But, sadly they have not implemented any measures to block the bad bots from accessing your website. None the less, they are charging their customers for such visits, which doesn’t even matter.

WPEngine team, if you are listening to this, I would love to see you implementing such features in your future iteration.

Anyways, to sum everything in simple words:

  • WPEngine charges for bad bots visits.
  • They reset their visit counters every 24 hours for human and for bots.
  • They will charge you for hot linked images or images visits from Google image search.
  • They don’t have any server level implementation for blocking bad bots visits

Now, after going through all this; I had a choice to move away from WPEngine, or keep paying the price. But, there is no fun leaving something which is good, and without making an extra effort. So first, I tried finding few .htaccess code to block bad bots visits. Alas! I couldn’t find any updated list of bad bots, so .htaccess was not a viable option. For now, I have implemented Cloudflare CDN, and will see how WpEngine stats shows the difference. If it all works out, I will be happy to stick with them, else there is no thought on saying Goodbye to WPEngine.  So, if you are an existing WPEngine customer, I highly recommend you to implement cloud flare CDN which is free (Also suggested by WPEngine staff), and it will help you to lower down your hosting bill.

Is WPEngine a good hosting company or bad?

There is a very thin line between deciding anything as bad or good. My part of job is to share my experience, and in terms of hosting quality WpEngine is par excellence, but in terms of pricing, I don’t find it reasonable due to over-pricing. According to my opinion, any busy site will usually face the issue of high-bill on WPEngine hosting, as busy websites usually gets more bot visits, more hot linked images due to copy-paste posts. Undoubtedly, it might not be a very cost-effective solution for a busy website, where as a small WordPress website will not face such issue.

It’s important for any existing WPEngine users to know how they are charging, and count visits. More over, you should definitely implement measures like Cloudflare CDN or any custom solution to block bots visits to keep your limited hosting bill in check.  If you are an existing WPEngine customer, I would love to hear your review and opinion regarding their pricing plans and if you have faced any issue with their over-charging, I would love to hear your story.

  • Author Bio

  • Latest Post

Article by Harsh Agrawal

Harsh has written 1094 articles.

If you like This post, you can follow ShoutMeLoud on Twitter. Subscribe to ShoutMeLoud feed via RSS or EMAIL to receive instant updates.


{ 7 comments… add one }

  • Shan

    I wish I had found this post before I wasted a few months and loads of dollars on overage fees at WPengine.
    It’s nearly a year later and nothing has changed in the way that they address this issue or warn customers about it.
    Their support staff is dismissive and their suggestions on how to combat the situation are not remotely helpful.
    The most recent idea from a ‘senior’ support staffer was to use googles webmaster tools to restrict how much google bots crawl the site. The problem is, you can’t actually do this if CDN is enabled, which they recommend you use. The worst.

    Reply
  • Lawrence Tam

    so I too was a bit shocked with the overage and counting bots, I came from the HostGator Dedicated hosting and never had to worry about that.

    The bigger thing was
    - speed
    - uptime

    I have yet to install cloudflare or CDN as I’m using their smallest package but compared to what I was spending with Hostgator dedicated it’s still a very good deal.

    I had a discussion with WPengine about having ONE point of failure, if you use cloudflare and if that service were to go down. They seem to recommend it and it’s something I will want to implement within 30 days (been testing speeds WITHOUT CDN to get a good picture of speed as some users say their site got slower with cloudflare)

    I have some 4 different speed tests going from hostgator dedicated linux -> WPengine’s smallest package without CDN or cloudflare.

    so far pretty happy but like you stated… want to reduce bot traffic.

    Reply
  • Jared

    Thanks for the article. A month with Wp-Engine and while the performance is great and support, the overcharge definitely hurts. I’ll try and implement some of the suggestions you offered here.

    Reply
  • Sumit Chhikara

    Cloudflare is a nice cdn. You just need to change the nameservers given by cloudflare. I used free version of cludflare and it blocked lots of spam bots. Paid version can offer you far better service.

    Reply
  • OKay Marketing

    Thanks Harsh! I too was wondering about this… I had CloudFlare implementation on my list of things to do at WPEngine, but now I am going to bump that up to top priority.

    Thanks for the heads up.

    Reply
  • Rajeesh Nair

    Of course I can’t afford to pay $250/month as of now.. But its always a good thing to know which are the best options to go for? Either VPS or WPEngine or Dedicated Server, when the traffic rises and bandwidth will become a thing to be concerned..

    Let us know how it went after the changes and implementing CDN on your blog.

    Reply
    • alan

      Bro please believe me,wpengine is a piece of crap.
      They are acting like apple,by just charging a vps available for 50$ at 250$ .and they say they are giving awesome support.
      Hostgator guys are well known wp experts and they kinda know all possible wp problems and they fix them also,with huge respect towards us.I really don’t get what people want as support from hosting company,,which hostgator , bluehost and other companies cannot give.are all these rich wpengine paying guys talking to that awesome support 20 hours a day..

      Reply

Leave a Comment