Web server outages and DDOS attacks

Forums › IT Pro and developers › Web server outages and DDOS attacks - how common are they?

1 | 2 | 3 | 4

spearsniper

133 posts

Master Geek

#1465724 8-Jan-2016 11:06

marpada:

With Amazon, what about hardware problems, which you obviously can't fix yourself? Is there no support whatsoever? Or do they charge for a ticket. I have seen some companies now charging for support tickets.

By default AWS accounts come with no support (you can only add tickets to raise some default limits, like the number of Elastic IP Addresses). If you need help from AWS you can try creating a forum threads and pray for some AWS engineers to pick up. There's a Developer support plan for a fixed monthly fee (around $50?) and the Business plan that costs 10% of your monthly bill. Even if you spend thousand of dollars support is just ok and scoped to only AWS infrastructure or services. They're never going to ssh to your instances for troubleshooting.

Hardware problems on AWS are some of the easiest to fix, even with no hardware experience.

It goes like this... log onto the AWS console, select your instance, press stop. Once it has stopped, press start.

Your server will migrate to a new host, and be as good as new.

You can automate this with cloudwatch alarms, so you do not even need to login - it will just fix itself.

I suspect marpada has not actually used AWS support, otherwise he would have a different view of it. They are a very customer centric company, and will help out with whatever they can provided it is running on their platform.

marpada

476 posts

Ultimate Geek

#1465742 8-Jan-2016 11:23

Hardware problems on AWS are some of the easiest to fix, even with no hardware experience. It goes like this... log onto the AWS console, select your instance, press stop. Once it has stopped, press start. Your server will migrate to a new host, and be as good as new. You can automate this with cloudwatch alarms, so you do not even need to login - it will just fix itself. I suspect marpada has not actually used AWS support, otherwise he would have a different view of it. They are a very customer centric company, and will help out with whatever they can provided it is running on their platform.

Actually I'm fairly experienced with AWS

OP mentioned that his current support manages the server. Don't expect that from AWS.

ArcticSilver

729 posts

Ultimate Geek

#1465766 8-Jan-2016 11:51

I have spoken to the OP over the phone (and hopefully been of some help).

The key point here is that she really needs a managed service. Any unmanaged hosting has a lot of associated risk, without the proper knowledge this can lead to lots of potential problems and inevitably bill shock when some one is needed to fix it.

On the phone I have recommended to her a managed option in NZ, one with a CDN (giving the best of both worlds). These options come in at around the same price as her current hosting.

This gives both great support and a local NZ number to call.

That being said, overseas hosts are still worth investigating. It has been a long time since I hosted overseas and am not up to play with the best international hosts for large virtual servers or dedicated servers. I'm sure Robin could benefit from some recommendations for a provider in the US that:

Must have dedicated resources (either a VPS or Dedicated Server)
Has good peering and performance.
Run's their servers off SSD's (virtual or dedicated)
Is a managed service
Has good knowledgeable support
Has at least 2 dedicated, resoanbly moden Xeon CPU's and 6GB or more of RAM.

The likes of Hostgator, Bluehost, Dreamhost and shared hosting in general are not appropriate for her requirements.

Cloudflare has a lot of merit, however this will not solve the underlying issue if it is host related, at most it will only make the issue less apparent.

spearsniper

133 posts

Master Geek

#1465777 8-Jan-2016 11:59

marpada:
OP mentioned that his current support manages the server. Don't expect that from AWS.

Correct - AWS is not a managed service provider. That is why they have a partner network for these kinds of task.

Their paid support is some of the best I have ever dealt with (both Business and Enterprise support). They make the majority of the NZ Hosting providers look like they need to go back to school.
If your server gets DDoS'd, AWS have teams of people there to help 24/7, and can do a lot in the background without you even knowing.

Architecting for HA in AWS is super simple, and needn't cost the earth. They have load balancers, a CDN network, WAF service, and autoscaling, that will give you the best shot at keeping your site up regardless of the attack. As a lot of this is managed by AWS, you don't need to patch it, manage the hardware, connectivity etc.

Watch the end of this clip. You will see a real world DDoS attack, and how to mitigate it.
https://www.youtube.com/watch?v=OT2y3DzMEmQ

RobinmNZ

24 posts

Geek

#1465782 8-Jan-2016 12:09

An update - the latest 18 minute outage (of the entire server) a couple of hours ago was caused by:

"What we found was multiple IPs from two different services, both with just a handful of connections each, which is suspicious. One of them is called Psychz and is known for shell accounts and abuse. I blocked part of their range, but I need to find the other half of the listing. The other set was from some hosting organization in Latin America somewhere. They're blocked entirely. What we also found was over 3000 connections from the server itself to itself:
3062 63.247.137.160"

I responded that the only change I had made was adding Cloudflare to two sites (not the big one) but haven't had a response about that yet.

jarledb

Webhead
3257 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#1465910 8-Jan-2016 13:51

I host some big WordPress sites with WPengine (which in turn uses Linode). Linode has had some major problems with DDOS lately. You can read more about it here: http://status.linode.com/incidents/mmdbljlglnfd - WPengine ended up moving their customers from Linode to another VPS provider.

Jarle Dahl Bergersen | Referral Links: Want $50 off when you join Octopus Energy? Use this referral code
Are you happy with what you get from Geekzone? Please consider supporting us by making a donation or subscribing.

MurrayM

2456 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#1465913 8-Jan-2016 14:02

jarledb: I host some big WordPress sites with WPengine (which in turn uses Linode). Linode has had some major problems with DDOS lately. You can read more about it here: http://status.linode.com/incidents/mmdbljlglnfd - WPengine ended up moving their customers from Linode to another VPS provider.

Yup. I'm a Munzee player and the munzee.com website is hosted by Linode. Their Atlanta data center was under constant attack from Christmas until early new year, causing the Munzee website to go totally offline between the 1st and 3rd of January.

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

jnimmo

1097 posts

Uber Geek

#1466370 9-Jan-2016 10:32

Have recommended earlier in the thread the guys at rtCamp to provide managed hosting, they know Wordpress inside out and would fit all of the requirements above, I think they use a few different VPS providers (AWS, Linode or DigitalOcean) depending on client requirements https://easyengine.io/services/managed-hostingThe important message being it is a managed service they back with SLAs, and should cost less than current provider, and run beautifully.

darylblake

1162 posts

Uber Geek

Trusted

#1466425 9-Jan-2016 10:58

If you are looking at AWS you may want to consider using an ELB and multiple c3/c4 instances rather than a m3/m4 type instance. Since you are running web servers I have found that the ram is not really made use of enough but the CPU is what gets chewed up faster. This is more so the case under nginx with node or a php-fpm module rather than apache, which needs a bit more memory. You get a bit more CPU grunt and the instance is a bit cheaper, but you do get less ram.. But if you don't need it then why pay for it.

Also CDNs like cloudflare and cloudfront actually help prevent attacks and have tools to defer attacks. Wordpress tends to get attacked a lot due to its popularity, so keep them up to date.
Having static content served out of things like amazon S3 via cloudfront or cloudflare also means less hits on your server, and delivers the content to other countries faster in most cases. So when you are looking at the access/error logs to find out what the attackers are actually doing, you can work your way back to blocking IP ranges and fixing code etc from there.

When you are a bit more competent with AWS you can set up cloudformation to horizontally scale your web servers, this actually works out a hell of a lot cheaper if you use spot instances, and a reserved RDS instance for your database.

RobinmNZ

24 posts

Geek

#1501462 29-Feb-2016 13:13

Hi all

Here is an update as to how all this turned out.

Arctic Silver was a HUGE help to us and also helped us migrate to a new NZ based server. For our big, main site, we chose a CDN with Site Host. That's been running for nearly a week now - and guess what, no outages or other problems.

We learned during this process that the dedicated server we had in the US was using a desktop grade hard drive. The jury is still out on the causes of our regular outages but the suspicion is hardware failure or a memory issue that the old server folks either did not know how to address or chose not to.

So our main (large) site, which gets way more offshore visitors than NZ, is now on the CDN. Our other sites have moved to a Site Host based shared server, and again, all is going well.

1 | 2 | 3 | 4