Here is a number that should bother you: 11 days.
That is the global median dwell time for 2024, according to Mandiant's M-Trends 2025 report. The time between when an attacker first touches your environment and when you know about it. Eleven days.
Now tell me your packet retention window.
If you are like most organizations, it is 48 to 72 hours. Maybe 7 days if you have invested in storage. Which means that when your IDS fires on day 11, the packet evidence from day 1 is gone. The initial reconnaissance. The credential probing. The first lateral movement. All of it. Overwritten by a clock that had no idea what it was deleting.
This is the rolling buffer problem. And the industry has been solving the wrong part of it for twenty years.
The Rolling Buffer Is Not a Storage Problem
Every organization with packet capture infrastructure has a rolling buffer. It fills, the oldest data is purged, and the cycle repeats. Most teams have spent years optimizing within this constraint: tuning retention windows, adding compression, selectively deploying capture to the highest-value segments.
More storage gives you a longer clock. Better compression gives you a longer clock at lower cost. Sampling gives you a statistical picture but sacrifices forensic integrity. NetFlow preserves connection metadata but loses the content.
None of them changes the mechanism by which rollover decisions are made. The root cause remains untouched: rollover is time-driven rather than knowledge-driven.
A 10 Gbps link running at 50% average utilization generates roughly 54 TB of raw data per day. A 72-hour retention window requires approximately 162 TB of dedicated storage before redundancy or compression. A 30-day window would require 1.6 PB. Most organizations stop well before that number and accept the retention window that fits their storage budget.
The retention window most organizations operate under is not the retention window they want. It is the retention window they can afford. These are different constraints. Only one of them is worth solving.
The Clock Does Not Know What It Is Deleting
The rolling buffer has a design assumption embedded in it that is rarely made explicit: that old traffic is less valuable than new traffic.
This assumption is wrong in exactly the cases that matter most.
Consider the lifecycle of a typical enterprise compromise. An attacker conducts initial reconnaissance: scanning, DNS enumeration, credential probing. This activity looks like noise. It generates no alerts. It falls below the threshold of any behavioral rule tuned to avoid alert fatigue. After a period of dormancy, the attacker returns and establishes a foothold. This is when the first alert fires.
By the time the alert fires, the reconnaissance traffic that establishes scope, timing, and method is already outside the retention window. The SOC team has a detection event and no context for it.
Those packets existed. They were captured. They were deleted by a clock that had no way to know they would eventually become evidence.
The data you need most in an investigation is disproportionately old data. An attacker's initial access, reconnaissance, and lateral movement precede detection by days to weeks. Time-based rollover deletes exactly the window that matters most, before you know it matters.
The Architectural Requirement: Intelligence Before Rollover
A different framing of the problem leads to a different class of solution. If the issue is that rollover decisions are made without knowledge of packet significance, the solution is to build that knowledge before the rollover decision is made.
This requires the capture infrastructure to analyze traffic continuously as packets arrive and produce a persistent, queryable record of what it found — protocol states, application-layer behavior, IDS correlations, anomaly signals, and behavioral context — organized in a structure that can answer the question "is this window worth keeping" in milliseconds.
Intelligent retention is not about keeping more data. It is about keeping the right data.
If that record exists at rollover time, the decision changes fundamentally. Instead of purging everything older than N hours, the system asks: which pending windows contain anything flagged by an IDS rule? Which contain traffic to or from a watched host? Which contain TLS anomalies, DNS tunneling signals, or unusual port distributions?
The windows that answer yes to any of those questions are retained. The windows that answer no — routine web browsing, backup traffic, NTP synchronization, broadcast noise — are released.
The same storage budget that supports a 72-hour rolling buffer can support a year or more of flagged and high-value traffic retention, if the retention decision is made by an index rather than a clock.
The Index That Makes It Possible
Vantage's capture engine, Sentinel, builds this analytical record continuously
during capture. The structure is called the Query Analytics Tree — QAT — written
to a companion .index file alongside the PCAP capture as packets arrive.
The QAT is a hierarchical index that covers the full capture at multiple levels of resolution simultaneously. Within each window, it accumulates keys from every analyzer running against the capture: TCP flow states, HTTP summaries, DNS queries and anomalies, TLS handshake outcomes, GTP tunnel events, IDS rule matches from Suricata and Zeek, and user-defined retention rules.
The storage overhead of the full metadata stack is constant at approximately 0.59% of the underlying capture size, at any scale.
At 1 PB of capture data: Packet data 1,000 TB QAT + metadata 6 TB (0.59%) The 6 TB survives the packets indefinitely. Overhead does not grow with capture complexity.
The practical implication: the metadata that enables intelligent retention costs less than 1% of the storage it governs. Retaining the index permanently while applying aggressive packet-level retention policies is not a storage trade-off. It is essentially free.
What Tiered Retention Looks Like in Practice
Vantage's retention policies are expressed in the same Vantage Query language used to investigate captures. There is no separate retention configuration interface, no vendor-specific policy language to learn.
Tier 1 Days 0–7 Full capture
All packets, full metadata. Baseline.
Tier 2 Days 7–30 Selective retention
IDS-flagged windows, anomaly signals, watched hosts.
Typically 5–10% of original packet volume.
Tier 3 Days 30–365 Investigation retention
Active investigations and legal holds only.
Typically 1–2% of original volume.
Tier 4 Year 1+ Permanent analytical record
Metadata only. No packets.
0.59% of original volume. Full query capability.
A Tier 2 retention policy in Vantage Query:
RETAIN PACKETS WHERE suricata.alert IS PRESENT OR zeek.alert IS PRESENT OR tls.anomaly IS PRESENT OR dns.anomaly IN (TUNNEL_SIGNAL, FAST_FLUX, NXDOMAIN_STORM) OR ip.addr MATCHES watchlist.high_value_hosts FOR 30 DAYS
The same language you use to investigate an incident is the language you use to define what gets kept. No context switching. No separate tool.
What Remains After the Packets Are Gone
The permanent analytical record is a fully queryable metadata set with no underlying packets. For every window it covers — including windows whose packets were released at Tier 2 or Tier 3 — it answers:
- ✓ Which flows existed — IPs, ports, protocol, duration, byte and packet counts
- ✓ What protocols were active — HTTP URIs, DNS queries, TLS handshake outcomes
- ✓ What anomalies were detected — IDS matches, TLS downgrades, DNS tunneling signals
- ✓ What the traffic looked like statistically — flow distribution, retransmit rates, failure patterns
- ✗ Packet payload content for windows whose packets were released — deliberate trade-off
An investigation that opens two years after an incident can still establish that a specific IP was communicating with a specific destination on a specific date, that a particular IDS rule fired, and that the DNS query pattern was consistent with a tunneling signal.
This is not the same as having the packets. But it is considerably more than having nothing. And it is what most investigations actually need to establish timeline and scope.
What This Changes for Your Team
The investigation that was previously impossible
With a tiered policy tuned to flag IDS alerts, TLS anomalies, and high-value host traffic, an organization running continuous capture on a 10 Gbps link can retain flagged traffic for a year or more on the same storage budget that previously supported a 72-hour rolling buffer. Because 5 to 10 percent of a year's traffic is less than three days of all traffic.
When the alert fires on day 16, the initial reconnaissance traffic from day 1 is still there. Not because storage budgets expanded. Because the traffic from day 1 was flagged by the QAT during capture and written into the Tier 2 retention set.
The compliance window without the compliance storage cost
Many organizations face regulatory requirements to retain network evidence for 90 days, 180 days, or a year. Meeting these requirements with traditional full-packet retention is a storage infrastructure project. With intelligent retention, the compliance requirement applies only to traffic the policy defines as compliance-relevant — a small fraction of total capture volume.
No rip and replace
Organizations already have Wireshark, Suricata, Zeek, SIEMs, and existing capture infrastructure. Vantage does not replace any of it. Sentinel captures alongside existing tools and writes metadata that Suricata and Zeek analysis populates automatically. Quarry provides a virtual filesystem layer that existing tools read directly without modification.
The Shift Worth Making
The rolling buffer is not going away. Storage economics will never make full-packet retention cheap enough to keep everything forever. But the decision about what to keep does not have to be made by a clock.
It can be made by an index that was built as packets arrived, that knows what every analyzer found in every window, and that can evaluate a retention policy in milliseconds at rollover time.
The clock does not know what it is deleting. Your retention policy should.
See Intelligent Retention in Action
We'll walk through tiered retention configuration with real capture data, against your actual retention requirements and storage budget.
Request a Demo