FlowSpec vs. RTBH: How LiveShield Minimizes Side Effects During Mitigation

RTBH (Remotely Triggered Black Hole) blocks all traffic directed to a given IP address, including legitimate traffic. This is its fundamental difference from FlowSpec, which filters only packets matching a specific attack signature. LiveShield minimizes this side effect by prioritizing FlowSpec and triggering RTBH only as an emergency mechanism, when precise filtering is no longer enough, rather than as the first and only reaction to every attack.

This distinction has a direct effect on service quality during an attack: badly configured mitigation can do more harm to a customer than the attack itself, by cutting off access to the very service it was meant to protect.

FlowSpec lets you define a filtering rule based on specific traffic characteristics, such as destination address, protocol, source or destination port, TCP flags or packet size, and install it directly in the hardware forwarding path of the edge router. The router drops only packets matching that signature, and all the remaining traffic to that address passes through undisturbed.

RTBH works completely differently: it announces the entire route to an IP address (most often a /32) as a black hole, so the router drops everything directed to it: attack traffic, but also legitimate traffic going to the same address. There is no selection at the packet level, only a complete cut-off of the target.

A simple rule follows from this: wherever FlowSpec can be applied, the rest of the customer's traffic does not suffer. RTBH is the last-resort solution, used when FlowSpec is no longer enough, not a default first step.

Filtering rules (FlowSpec) and blackholing rules (RTBH) are checked independently of each other, based on separately configured thresholds for the same prefix. If the filtering rule's threshold is exceeded first, the system triggers FlowSpec. At the same time, in parallel, the blackholing threshold is checked, and as long as traffic does not exceed it, RTBH is not triggered.

In practice this means that if blackholing thresholds are set higher than filtering thresholds, which is the recommended configuration, FlowSpec takes over mitigation first, and RTBH switches on only when, despite FlowSpec filtering, traffic keeps growing and approaches the limit at which link saturation is threatened. A typical configuration pattern looks like this: local FlowSpec filtering up to a critical amount of traffic, and only above that level RTBH as protection against saturating the uplink to the outside world.

The recommended practice when configuring the blackholing threshold is to leave only IP-address-level control in the profile (without breaking it down by protocol) and to set the threshold close to the maximum amount of traffic the given uplink can actually accept, rather than close to zero. This way RTBH genuinely protects against port saturation instead of triggering on every small attack that FlowSpec would have handled anyway.

The system evaluates filtering rules the same way a router evaluates routes in routing: a more specific prefix has higher priority than a general one. As a result, for a single critical address (e.g. a service server) within a broader subnet, you can configure a separate /32 rule with different thresholds than the rule covering the whole subnet, and the more specific rule is applied first.

This makes it possible, for example, to set aggressive, low thresholds for TCP-SYN traffic directed to ordinary customers (because by default they should not receive many such packets), while for servers, which by nature receive a lot of SYN traffic from many clients at once, higher thresholds or a different filtering mechanism are left in place, so as not to cut off traffic that is normal for them.

This mechanism has one clear limitation: it works well when we want to single out an individual address within an attack on the whole subnet. However, it is not simple to exclude a specific address from mitigation if the attack covers the whole subnet and a broad FlowSpec or RTBH rule is already acting at the subnet level. There is no way to announce an even more specific route that would take one address out from under that rule.

Collateral damage is not limited to the RTBH mechanism itself: an incorrectly chosen, too-low filtering threshold can cut off legitimate traffic even though the system technically acted according to its configuration. A typical example from practice: an aggressive threshold for IP fragmentation (IPFRAG), set to protect against attacks using fragmented packets, may start cutting off legitimate IPsec tunnel traffic. The ESP protocol naturally fragments with larger packets, so with a low threshold and a rule blocking all fragments to a given address, the customer's entire VPN tunnel stops working.

This shows why threshold tuning is a process, not a one-off action: a configuration that looks safe on paper may in practice block a specific, legitimate traffic pattern peculiar to a given network. That is why LiveShield records packet dumps for triggered rules, which makes it possible to check afterwards exactly what traffic was cut off and whether the rule worked as intended.

Minimizing collateral damage in LiveShield rests on three mechanisms working together: the priority of FlowSpec over RTBH (RTBH as an emergency safeguard, not the first line of defense), the priority of more specific prefixes over general ones (the ability to set different thresholds for critical addresses within a broader subnet), and deliberate tuning of filtering thresholds so that the filtering rules themselves do not become a source of harm. None of these mechanisms works fully automatically from day one. They require thresholds to be configured to the specific network and later observation, but it is precisely this configurability that allows the side effects of mitigation to be limited to a minimum.

Do RTBH and FlowSpec work in LiveShield simultaneously, or one after the other? Filtering rules (FlowSpec) and blackholing rules (RTBH) are checked independently and in parallel, based on separate thresholds. If the blackholing threshold is set higher than the filtering threshold, in practice FlowSpec reacts first, and RTBH switches on only when traffic keeps growing despite filtering.

Can a specific IP address be excluded from mitigation during an attack on the whole subnet? Not in a simple way. Under normal conditions a separate, more specific rule with different thresholds can be configured for a single address, but when the attack covers the whole subnet and a broad FlowSpec or RTBH rule is already acting on that subnet, it is not possible to announce an even more specific route excluding one address from it.

How do you set the blackholing threshold so that it does not trigger on every small attack? The recommended practice is to leave only IP-address-level control in the blackholing profile and to set the threshold close to the maximum amount of traffic the uplink can actually accept.