Welcome!

By registering with us, you'll be able to discuss, share and private message with other members of our community.

SignUp Now!
adv ex on 5 january 2024
adv ex on 22 February 2024
adv ex on 22 February 2024
banner Expire 26 April 2024
Rescator cvv and dump shop
banner expire at 13 May

Yale lodge shop
UniCvv
banner Expire 1 April  2021

How to make online services resilient to DDoS attacks

RedX

TRUSTED VENDOR
Staff member
Joined
Nov 26, 2020
Messages
598
The global coronacrisis has increased consumer interest in e-Commerce, distance learning, online communications, and entertainment resources. As a result, against the background of a steady increase in the number of DDoS attacks, there was a sharp surge in Them. Owners of services and resources are forced to double their attention to protecting them from the effects of intruders and bots "charged" with them.

So how can we ensure the stability and availability of online services in the face of the threat of new massive cyber attacks-paid for by competitors or initiated by the attackers themselves, for example, for the purpose of blackmail?

It should be noted right away that there is no single, absolutely universal approach to solving this problem: the actions that it departments responsible for maintaining and securing Internet resources need to take significantly depend on what exactly needs to be protected. Thus, improving the resilience of sites and web applications will require some measures, various Internet services and online games based on TCP and UDP-others, and networks-others.

Websites and web applications
If your goal is to secure your sites and web applications, the first thing to consider when building their protection against DDoS attacks is whether you have access to the server. If you have hosted sites and applications on a server that you can fully control, you should not only take care of enabling external DDoS protection, but also properly prepare the server itself: optimize the network stack of the operating system so that the server can withstand high loads. To protect against DDoS, it is very important to ensure high server performance, including processing incoming requests to it over the network. Otherwise, even in the absence of an attack, there is a risk of the so-called Habra effect, when the appearance on the Internet of a publication that quickly became very popular leads to a sharp increase in the load on the Internet resources mentioned in it.

The standard settings of the operating system's network stack and the Apache or Nginx web server running in productive operation mode usually have significant limitations that must be addressed. In particular, you need to pay attention to the parameters that determine the performance of the Nginx server and the Linux network stack. You should also pay attention to the optimization of your DBMS - MySQL or any other one that you choose-they should work quickly.

If your site uses popular CMS engines, such as Joomla!, WordPress, or Drupal, be sure to use the publicly available recommendations for configuring their performance. It is necessary to ensure that the site has high performance in standard mode this will increase its chances of coping with a DDoS attack: when it starts, the "fast" site will be easier to protect.

If you have decided to deploy an application or website on an external platform and entrust protection against DDoS attacks to its owner, then you should at least clarify whether they can protect your Internet resource from attacks at the application level-the seventh level of the OSI (L7) model. In any case, you can connect an external security service, but you need to configure it so that the IP address of the real server is not visible to the attacker either through mail headers, open ports, or other services.

Here's a practical example: an online store can't accept orders after the attack starts, even though the protection service was enabled. The reason for the denial of service is that the previous address of the server on which the web service is deployed is still accessible via the Internet, which was used by the attacker. After the hosting provider "closes" this address for external access via HTTP ports, the denial of service is temporarily stopped. However, later new DDoS attacks again disable the e-Commerce service, because the attacker started attacking via port 22 (SSH), which is still open. Then the hosting provider moves the service to another server, but after a while the denial of service starts again. It is likely that an experienced attacker extracted the address of the new server from the email headers…

This example shows how important it is to ensure that IP addresses are inaccessible to everyone except the DDoS defender. Well, to check whether the server addresses are visible from the outside, for example, the Shodan service will help. Also, it is not superfluous to view the history of previous IP addresses, for example, here.

Internet services and online games based on TCP and UDP
In order to ensure the stability of services that interact with users via TCP and UDP, you must first optimize the network stack of the operating system. First, you should make sure that the network card interrupts are distributed across different processor cores. In most modern systems, this allocation is made automatically, however, additional care should not be taken.

Network card, interrupts, queues…
On the other hand, most modern servers have multiple processor cores. Since the OS treats each of them as a separate processor, we can evenly distribute the load from interrupts between them. There are two ways to do this.

  • The first and recommended option is to use hardware queues. Modern network cards have multiple interrupt queues, usually from 4 to 16. For some reason, they are often disabled by default on Linux. We need to enable them, and then distribute them evenly across the processors.
  • The second method is called RPS (Receive Packet Steering). This is a relatively new kernel mechanism that automatically distributes the load between all cores, regardless of whether there are multiple hardware queues on the network card or not. Use this method only if you have more cores than hardware queues (by the way, consider disabling SMT / HyperThreading — this will be very useful during an attack).

The steps you need to take:

1) Install the latest recommended version of the Linux kernel for your distribution-it will most likely include the latest version of your network card driver. In some cases, you should build it separately for your kernel according to the instructions.

2) Distribute interrupt loads evenly between cores. First, you need to determine which interrupt numbers the network card uses (in our case, eth0).

Code:
# cat /proc/interrupts | grep eth0

After running this command, you will be able to see all the interrupt numbers that your NIC uses, as well as their actual distribution across cores. Write down these numbers.

Now we need to change the SMP affinity to assign a different core to each interrupt (interrupt 1 > cpu 1, interrupt 2 > > cpu 2, etc.).:

Code:
# echo <affinity> > /proc/irq/xx/smp_affinity

For example, for a 4-core processor:

Code:
# echo 1 > /proc/irq/83/smp_affinity # 1= "0001" i.e. 1st core
# echo 2 > /proc/irq/84/smp_affinity # 2= "0010" i.e. 2nd core
# echo 4 > /proc/irq/85/smp_affinity # 4= "0100" i.e. 3rd core
# echo 8 > /proc/irq/86/smp_affinity # 1= "1000" i.e. 4th core

If there are more cores, then we use hexadecimal numbers.

Note: in modern Linux distributions, this distribution is often performed automatically by the irqbalance service, but in some cases it performs suboptimal balancing. If you decide to distribute interrupts across cores yourself, don't forget to turn off irqbalance.

3) Optional: enable RPS (this may not be necessary, see above)

Code:
# echo f > /sys/class/net/eth0/queues/rx-0/rps_cpus

Select the appropriate device (if it is not eth0) and all receive queues that it supports (rx-0..n).

Code:
# echo f > /sys/class/net/eth0/queues/rx-0/rps_cpus

3) Optional: enable RPS (this may not be necessary, see above)

Note: in modern Linux distributions, this distribution is often performed automatically by the irqbalance service, but in some cases it performs suboptimal balancing. If you decide to distribute interrupts across cores yourself, don't forget to turn off irqbalance.

If there are more cores, then we use hexadecimal numbers.

Code:
# echo 1 > /proc/irq/83/smp_affinity # 1= "0001" i.e. 1st core
# echo 2 > /proc/irq/84/smp_affinity # 2= "0010" i.e. 2nd core
# echo 4 > /proc/irq/85/smp_affinity # 4= "0100" i.e. 3rd core
# echo 8 > /proc/irq/86/smp_affinity # 1= "1000" i.e. 4th core

For example, for a 4-core processor:

Code:
# echo <affinity> > /proc/irq/xx/smp_affinity

Now we need to change the SMP affinity to assign a different core to each interrupt (interrupt 1 > cpu 1, interrupt 2 > > cpu 2, etc.).:

After running this command, you will be able to see all the interrupt numbers that your NIC uses, as well as their actual distribution across cores. Write down these numbers.

Code:
# cat /proc/interrupts | grep eth0

2) Distribute interrupt loads evenly between cores. First, you need to determine which interrupt numbers the network card uses (in our case, eth0).

1) Install the latest recommended version of the Linux kernel for your distribution-it will most likely include the latest version of your network card driver. In some cases, you should build it separately for your kernel according to the instructions.

The steps you need to take:

The first and recommended option is to use hardware queues. Modern network cards have multiple interrupt queues, usually from 4 to 16. For some reason, they are often disabled by default on Linux. We need to enable them, and then distribute them evenly across the processors. The second method is called RPS (Receive Packet Steering). This is a relatively new kernel mechanism that automatically distributes the load between all cores, regardless of whether there are multiple hardware queues on the network card or not. Use this method only if you have more cores than hardware queues (by the way, consider disabling SMT / HyperThreading — this will be very useful during an attack).

On the other hand, most modern servers have multiple processor cores. Since the OS treats each of them as a separate processor, we can evenly distribute the load from interrupts between them. There are two ways to do this.

The first thing you need to work on is the network card driver. When a frame arrives at it, it must initiate a system interrupt that "asks" the processor to suspend the current task and process the incoming portion of traffic. However, if each frame caused an immediate interrupt and "distracted" the CPU from current tasks, a noticeable performance decrease could be observed even on the simplest network operations, such as file transfer via FTP. Therefore, these interrupts are queued, which accumulates on the network card and is processed by the processor at a time. This usually happens 250-1000 times per second, and the less often the lower the CPU load, but the higher the delay.

You also need to evaluate the interdependence of the components of the protected application and study what happens if, for example, the DBMS or one of the services becomes unavailable. Your goal is to ensure that the attack does not lead to simultaneous failure of all the application components that users interact with. Recommendations for improving the availability of each service can be found on the official project websites.

In addition, you need to make sure that the protected service has at least several entry points. Then, if, for example, an attack leads to the unavailability of one of the IP addresses, then you can quickly replace it in the DNS table with another address. Alternatively, if some of the IP addresses that are provided to your clients are attacked, you can provide them with other addresses, first making sure that your clients really want to access them for example, tokens can be used to authenticate them.

It is important to design and configure the service in such a way that unauthorized users will not be able to find out the real IP addresses that are used to access the service by authorized users.

And again an example from practice: one of our clients has implemented a kind of "hierarchy" of users in the game service: after the player reaches level 20, he is given a new IP address. This protection option avoids the situation when an attacker registers, gets access to the game, and almost immediately starts an attack on the game server.

Services based on the TCP Protocol have a certain advantage in protecting against DDoS attacks, since the Protocol itself is better adapted to repel attacks. Much more effort will be required to secure servers running on the UDP Protocol. This Protocol does not involve connections, and if the server is subjected to a targeted attack that mimics game packets, the traffic will not be filtered - unless you inform your DDoS defender in advance about the details of the server architecture and functioning, think through ways to repel atypical attacks together, and test their effectiveness on several test attacks.

Networks
Ensuring network resilience is perhaps the most difficult case in terms of providing effective protection against DDoS attacks. On the one hand, this is due to the fact that you often have to protect not only your Internet resources, but also those that belong to your customers who have placed their it assets inside your network-and among them, we note, there can be any services. On the other hand, the large number of IP addresses that you most likely have encourages attackers to use them, for example, to organize relatively weak attacks that are carried out simultaneously on many addresses-this impact will significantly slow down the entire infrastructure.

The first thing to take care of is that the edge router has sufficient performance to cope with the increased load that may occur after the start of a DDoS attack. And, of course, you should not install cheap routers designed for use in the home or small office, as well as old low-power routers at the network border most likely, they will become the first victims of a DDoS attack on the network. You should specify the bandwidth specified by the manufacturer, evaluate the current load, and, if possible, conduct a series of stress tests using tools available on the Internet, such as hping3 - it is available in almost any Linux distribution.

Second - you need to make sure that your IP addresses cannot be determined by tracing (traceroute), and those that can are protected by ACLs. The fact is that an attacker can trace to the DDoS defender and find out the butt IP address, which, as a rule, is not protected, and launch an attack on it. Therefore, on the one hand, you need to secure addresses using ACLs (your defender will help you with this), and, on the other hand, you need to hide addresses from tracing both from outside and from inside the network (this is necessary so that addresses cannot be detected by insiders).

hping3
hping3 is a powerful utility that can simulate many types of attacks with different parameters. Here is an example of how to check your server for vulnerability to a small SYN attack:

Code:
# hping3 -S <--fast|--faster|--flood> -p 80 xx.xx.xx.xx

80 (HTTP) in this example is the destination port. Pay attention to the other parameters (hping3 --help) to understand how attacks can differ from each other.

I recommend that you study the documentation for the utility that is available by command man hping3and try to generate packages of various protocols yourself to see how this will affect the load. Since the utility can only use one CPU core, you can run multiple instances of hping3 on the same computer to increase the power of a test attack.

Use this utility carefully and only for testing your own resources! Unfortunately, most of the Internet is vulnerable to such simple operations, even if the attacker uses a much smaller channel than the attacked one.

Use this utility carefully and only for testing your own resources! Unfortunately, most of the Internet is vulnerable to such simple operations, even if the attacker uses a much smaller channel than the attacked one.

I recommend that you study the documentation for the utility that is available by command man hping3and try to generate packages of various protocols yourself to see how this will affect the load. Since the utility can only use one CPU core, you can run multiple instances of hping3 on the same computer to increase the power of a test attack.

80 (HTTP) in this example is the destination port. Pay attention to the other parameters (hping3 --help) to understand how attacks can differ from each other.

Code:
# hping3 -S <--fast|--faster|--flood> -p 80 xx.xx.xx.xx

hping3 is a powerful utility that can simulate many types of attacks with different parameters. Here is an example of how to check your server for vulnerability to a small SYN attack:

An example of a utility that is often used for performing primitive attacks is hping3. You can use it to stress test your server before attackers do it for you.

General recommendations
In any case, you should not rely entirely on protection against DDoS attacks. You need to think through attack response scenarios in advance so that you clearly understand what to do and how to do it if the measures taken to protect against DDoS are insufficient.

In addition, you need to think about how you will enable DDoS protection and how long It will take. In particular, you need to check the timing of the timeout (TTL) for DNS records: if, for example, the timeout is two days, then when switching to a new DNS record, your site will not be available for two days. And, of course, when protecting critical resources, you need to take care of DDoS protection in advance.

For dynamic attack detection and automatic activation of protection, we strongly recommend using a sensor-this measure will provide more extensive traffic management capabilities, save on enabling DDoS protection, reduce latency in non-attack mode, and most importantly-significantly increase the speed of response to the start of an attack by automatically triggering the sensor instead of enabling protection manually. Therefore, before entering into a contract with an anti-DDoS service provider, it is useful to clarify whether it provides a DDoS sensor.
 
Top Bottom