A firewall decides whether a connection is allowed. It does not usually ask whether the allowed connection is carrying something malicious. That second question is the job of intrusion detection and prevention systems. The two names differ by one letter, but the difference in where they sit and what happens when they make a mistake shapes how every security team deploys them.

Position decides behaviour

An intrusion detection system (IDS) receives a copy of traffic, typically from a switch’s mirror (SPAN) port or a network tap. It analyses that copy and raises alerts. Because it is not in the path, it cannot stop anything directly, and if it crashes or falls behind, traffic continues unaffected.

An intrusion prevention system (IPS) sits inline. Every packet passes through it, and it can drop packets, reset connections or block a source in real time. That power comes with two risks. If the IPS fails, traffic may stop unless it is designed to “fail open” with a bypass. And if it wrongly identifies legitimate traffic as an attack, it blocks a real business transaction.

IDSIPS
PlacementOut of band, on a copy of trafficInline, in the traffic path
ActionAlerts for humans or other systemsBlocks automatically, and alerts
Effect of a false positiveAn analyst wastes timeA customer or business process is blocked
Effect of a device failureLoss of visibilityPossible outage unless bypass is configured
Latency addedNoneSome, usually small

Today, a separate IPS appliance is less common. Most next-generation firewalls include IPS functions that can be switched on per rule, and cloud providers offer managed equivalents. The concepts remain the same.

How they decide something is bad

Signature-based detection compares traffic against patterns describing known attacks: a specific byte sequence in an exploit, a malformed protocol field, a known malware command format. Open-source engines such as Snort and Suricata use rule languages for this, and commercial vendors maintain their own signature feeds. Signatures are precise and explainable; an alert names exactly which rule matched. Their weakness is that they only catch what someone has already described. A new exploit, or a known one slightly modified, can pass unnoticed.

Anomaly-based detection learns what normal looks like, for instance typical traffic volumes, which hosts talk to which, and the usual sizes of DNS responses, and flags deviations. It can spot previously unseen behaviour such as a print server suddenly sending gigabytes to the internet. Its weakness is that “unusual” is not the same as “malicious”. A quarter-end batch job, a new application rollout or a backup run all look anomalous.

Protocol analysis sits between the two. The engine understands how protocols such as HTTP, DNS or SMB should behave and flags violations, such as an HTTP header far longer than any real browser sends. Most real products combine all three.

The arithmetic of false positives

The reason tuning dominates IDS work is a statistical effect called the base rate fallacy. Consider an illustrative network that generates 10 million sessions a day, of which 100 are genuinely malicious. Suppose a detection rule catches 99 percent of attacks and has a false positive rate of only 0.1 percent.

It will correctly flag 99 malicious sessions. It will also wrongly flag 0.1 percent of the roughly 10 million benign ones, which is about 10,000 sessions. Of roughly 10,100 alerts, fewer than 1 percent are real. A rule that sounds excellent on paper produces a queue in which analysts must wade through about a hundred false alarms for each true one.

This is why mature teams care so much about precision, the share of alerts that are real, and why an IPS is usually configured to block only on high-confidence signatures while everything else stays in alert-only mode.

Tuning without blinding yourself

Tuning means adjusting rules so that alerts match your environment. Sensible practices include:

  • Disabling signatures for software you do not run. An exploit for a web server platform you do not own cannot succeed, so alerts about it only add noise, although some teams keep them as low-priority reconnaissance indicators.
  • Writing narrow exceptions rather than disabling a rule entirely. If a vulnerability scanner triggers alerts, exclude the scanner’s address for that rule, not the rule for everyone.
  • Moving new signatures through a period in alert-only mode before allowing them to block.
  • Recording every tuning decision with a reason and a review date, so exceptions do not become permanent blind spots.

The danger in tuning is over-correction. A team drowning in alerts may disable whole categories, and the next real attack in that category passes silently. A noisy rule is a problem to be fixed, not just switched off.

Encryption and placement limits

A network IDS reading TLS traffic sees handshakes and metadata but not content. Signatures that depend on payload bytes stop working unless traffic is decrypted first, usually at a TLS inspection proxy or a load balancer that terminates TLS, with the sensor placed behind it. Sensors also see only traffic that passes their tap point. A sensor at the internet edge misses attacks between internal hosts, which is precisely where lateral movement happens.

These limits explain why host-based detection has grown in importance. A host-based IDS or EDR agent watches the operating system itself, sees decrypted data, and knows which process generated a connection. Network and host sensors complement each other: the network view is hard for an attacker on one machine to tamper with, while the host view sees the detail the network cannot.

Written by the RP Strativa Team. Session counts and detection rates in the worked example are illustrative.

Leave a Reply

Trending