VPC Flow Logs: What It Is and When to Use It

Definition

VPC Flow Logs is an Amazon Web Services (AWS) feature that captures metadata about the IP traffic going to and from network interfaces within your Amazon Virtual Private Cloud (VPC). It provides a detailed log of all accepted and rejected network connections, helping developers and architects monitor network connectivity, troubleshoot security rules, and perform security analysis without impacting network performance.

How It Works

VPC Flow Logs operate by non-intrusively capturing network flow information from the underlying AWS networking infrastructure. The data collection process happens outside of the instance's data path, ensuring there is no impact on network throughput or latency.

Architecture & Data Flow:

1. Enablement: You can enable Flow Logs at three different scopes: for an entire VPC, a specific subnet, or an individual Elastic Network Interface (ENI). Enabling it at a higher level, like a VPC, automatically monitors all ENIs within that scope.
2. Capture & Aggregation: Once enabled, the service captures metadata for IP traffic flows. A "flow" is a 5-tuple that uniquely identifies a connection: source IP, destination IP, source port, destination port, and protocol. This data is collected over a configurable period called an aggregation interval (typically 1 or 10 minutes).
3. Publication: After the aggregation interval, the collected records are formatted and published to a specified destination. You can choose one of three AWS services as the destination for your logs:
   • Amazon CloudWatch Logs: A common choice for real-time monitoring, alarming, and interactive analysis using CloudWatch Logs Insights.
   • Amazon S3 (Simple Storage Service): Ideal for long-term archival, compliance, and large-scale analysis using tools like Amazon Athena.
   • Amazon Kinesis Data Firehose: Used for streaming the log data to other destinations in near real-time, such as Amazon OpenSearch Service for advanced analytics and visualization or to third-party tools.

Each log record contains fields like the source/destination IPs and ports, protocol number, packets and bytes transferred, the start and end of the aggregation window, and an action (ACCEPT or REJECT). You can use a default log format or define a custom format to include additional metadata, such as instance-id, tcp-flags, or pkt-srcaddr for traffic going through NAT Gateways and Transit Gateways.

Key Features and Limits

• Flexible Scoping: Can be enabled on a VPC, subnet, or ENI level.
• Destination Options: Supports publishing logs to Amazon CloudWatch Logs, Amazon S3, and Amazon Kinesis Data Firehose.
• Traffic Filtering: You can choose to capture all traffic, only accepted traffic, or only rejected traffic, which helps reduce data volume and cost.
• Customizable Log Formats: In addition to the default format, you can specify which metadata fields to include in the log records for richer analysis.
• No Performance Impact: The logging mechanism is detached from the network path and does not affect instance performance, network throughput, or latency.
• File Format Options: When publishing to S3, you can choose plain text or Apache Parquet format, which is compressed and optimized for analytics queries, reducing storage and query costs.
• Service Limitations:
  • VPC Flow Logs do not capture all traffic. Excluded traffic includes requests to the Amazon DNS server, Windows license activation traffic, instance metadata traffic (169.254.169.254), DHCP traffic, and traffic to the default VPC router.
  • You cannot enable flow logs for a peered VPC unless it is in the same AWS account.
  • There is a limit of 250 flow log subscriptions per resource per account.

Common Use Cases

1. Security Threat Detection: Analyzing flow logs helps identify anomalous traffic patterns, such as port scanning, attempts to connect to known malicious IP addresses, or data exfiltration. Services like Amazon GuardDuty use flow log data as a primary source for threat detection.
2. Troubleshooting Network Connectivity: When an application cannot connect to another service, flow logs can quickly diagnose the problem. By filtering for REJECT records, you can determine if a Security Group or Network Access Control List (NACL) rule is blocking the traffic.
3. Compliance and Auditing: Many compliance frameworks (like PCI-DSS) require logging and monitoring of all network traffic. VPC Flow Logs provide a durable, agentless audit trail of every connection attempt into and out of your VPC.
4. Network Cost and Performance Optimization: By analyzing fields like bytes and packets, you can identify the "top talkers" in your network. This insight is useful for optimizing network architecture, planning capacity, and attributing data transfer costs.

Pricing Model

The VPC Flow Logs feature itself is free; however, you pay for the data ingestion and storage at your chosen destination.

• Vended Logs Pricing: AWS categorizes the data published by Flow Logs as "Vended Logs." This data is subject to ingestion charges that vary by destination.
• Amazon CloudWatch Logs: You are charged for data ingestion and storage. Pricing is tiered, so the cost per GB decreases as volume increases.
• Amazon S3: You are charged for the Vended Logs data ingestion fee plus standard S3 storage (per GB-month) and data access (per request) charges.
• Amazon Kinesis Data Firehose: You pay the Vended Logs ingestion fee plus the Kinesis Data Firehose ingestion charges.

Always consult the official AWS Pricing Calculator and the Amazon CloudWatch pricing page for the latest details, as costs can be significant for high-traffic environments.

Pros and Cons

Pros:

• Agentless and Non-Intrusive: Provides network visibility without needing to install or manage agents on instances, and with zero performance impact.
• Comprehensive Visibility: Captures metadata for all IP traffic across a VPC, subnet, or ENI, offering a complete picture of network communications.
• Natively Integrated: Seamlessly integrates with other AWS services like S3, CloudWatch, Athena, and Kinesis for storage, analysis, and visualization.
• Enhances Security: Acts as a foundational data source for security monitoring tools like Amazon GuardDuty and third-party Security Information and Event Management (SIEM) systems.

Cons:

• Not Real-Time Packet Capture: Flow logs provide metadata summaries over an aggregation interval, not the actual packet payloads. For deep packet inspection, a different tool is needed.
• Potential for High Data Volume and Cost: In a busy VPC, flow logs can generate terabytes of data, leading to significant ingestion and storage costs if not managed carefully (e.g., by filtering traffic).
• Analysis Requires Tooling: Raw flow logs are simple text lines. Deriving meaningful insights requires setting up a query engine (like Amazon Athena), a visualization tool (like OpenSearch), or using CloudWatch Logs Insights.
• Logging Gaps: Does not capture certain types of AWS-managed traffic, which can leave blind spots in specific troubleshooting scenarios.

Comparison with Alternatives

VPC Flow Logs vs. VPC Traffic Mirroring:

• Purpose: Flow Logs capture metadata (who talked to whom, when, how much). Traffic Mirroring captures the actual packet content (the data payload).
• Use Case: Use Flow Logs for broad, continuous monitoring, security analysis, and troubleshooting access rules. Use Traffic Mirroring for deep, surgical inspection, such as forensic analysis, intrusion detection that requires payload analysis, or debugging complex application protocols.
• Cost & Performance: Flow Logs are generally more cost-effective. Traffic Mirroring can be more expensive and consumes instance bandwidth, as it duplicates and forwards every packet.

VPC Flow Logs vs. Amazon GuardDuty:

• Relationship: These are complementary services, not alternatives. GuardDuty is an intelligent threat detection service that uses VPC Flow Logs (along with DNS logs and CloudTrail logs) as a primary data source.
• Function: Flow Logs provide the raw data. GuardDuty analyzes this data using machine learning and threat intelligence to automatically identify malicious activity like port scanning, malware, and command-and-control communication, then generates actionable security findings.

Exam Relevance

VPC Flow Logs are a fundamental networking and security topic on several AWS certification exams:

• AWS Certified Solutions Architect - Associate (SAA-C03): Expect questions on choosing the right destination (S3 vs. CloudWatch) based on a use case (e.g., long-term archival vs. real-time alarming) and using logs to troubleshoot Security Group or NACL issues.
• AWS Certified Security - Specialty (SCS-C02): Deep knowledge is required. Questions may focus on analyzing log data to identify security incidents, integrating logs with SIEMs, and using custom formats for forensic analysis.
• AWS Certified Advanced Networking - Specialty (ANS-C01): Questions often involve complex troubleshooting scenarios, automating the enablement of flow logs across an organization, and analyzing traffic patterns in hybrid or multi-VPC architectures.

Examinees should know how to create a flow log, the differences between destinations, how to interpret log fields, and common use cases for security and troubleshooting.

Frequently Asked Questions

Q: Do VPC Flow Logs impact my network performance?

A: No. VPC Flow Log data is collected outside of the path of your network traffic. Enabling, modifying, or disabling flow logs has no impact on the network throughput or latency of your resources.

Q: What is the difference between 'accepted' and 'rejected' traffic in the logs?

A: 'Accepted' traffic is a flow that was allowed by the relevant Security Groups and Network ACLs. 'Rejected' traffic is a flow that was explicitly denied by a Security Group or Network ACL. Analyzing rejected traffic is a primary method for troubleshooting connectivity issues and identifying potential security threats like port scans.

Q: Why don't I see traffic to the Amazon DNS server or for instance metadata in my flow logs?

A: VPC Flow Logs intentionally exclude certain types of traffic that are managed by AWS to reduce log clutter and cost. This includes traffic to the Amazon-provided DNS server, instance metadata service (169.254.169.254), Windows license activation, and DHCP. If you use your own DNS server, traffic to it will be logged.

---

This article reflects AWS features and pricing as of 2026. AWS services evolve rapidly — always verify against the official AWS documentation before making production decisions.