Log Forwarder Setup Guide

The Log Forwarder feature in VergeCloud allows users to stream different types of logs to external systems like Kafka, S3, and Syslog. It provides visibility into HTTP requests, security events, DNS activity, and internal platform operations.

Use Cases

• Forward HTTP access logs to S3 for long-term storage and compliance.
• Stream WAF or DDoS logs to a Kafka topic for real-time threat monitoring.
• Send DNS request data to your Syslog aggregator for visibility and analysis.
  

Step 1: Select Log Types

Select one or more types of logs to forward. Supported log types:

• HTTP Requests
• WAF Events
• DNS Requests
• Errors
• Event Logs
  

Note: The selected log type cannot be changed later in edit mode.

Step 2: Filter Logs

Configure filters to limit which logs are forwarded. Each log type has its own set of fields available for filtering. If no filter is applied, all logs of that type are forwarded.

Available Fields

• Method – HTTP method (GET, POST, etc.)
• Scheme – HTTP or HTTPS
• Domain Name – Host header
• Referer – Referring URL
• IP Address – Visitor IP
• User Agent – Browser/device
• Country – GeoIP-based location
• AS Number – ISP or organization
• Content Type – MIME type
• Response Status Code – e.g., 200, 404
• Server Port – Port receiving request
• Bytes Sent – Outbound data
• Bytes Received – Inbound data
• Upstream Time – Origin server latency
• Cache Status – HIT, MISS, BYPASS
• Request ID – Unique ID per request
• URI – Request path
• Query String – Parameters
• JA3 Fingerprint – TLS client fingerprint

• Timestamp – Time of event
• Remote Address – Client IP
• Domain Name – Host header
• Data – Full security event log

• Timestamp – Time of request
• UUID – Unique log ID
• Record – Subdomain queried
• Type – Record type (A, AAAA, etc.)
• IP – Client IP
• Country – GeoIP location
• AS Number – ISP/Network
• Response Status Code – DNS response
• Process Time – Latency

• Client IP – Source of request
• Upstream Protocol – Protocol to origin
• Upstream URI – Requested resource
• Upstream Port – Port used
• Upstream IP – Origin server
• Domain Name – Host header
• HTTP Version – 1.1, 2, or 3
• Request Method – GET, POST, etc.
• Request URI – Failing endpoint
• Real Timestamp – Time of error
• Error Message – Error details
• PoP Site – Edge server ID
• Request ID – Unique identifier

• Domain Name
• HTTP Request Host Header
• JA3 Fingerprint
• Timestamp
• Method
• Scheme
• IP Address
• Country
• Response Status Code
• Server IP
• Server Port
• URI
• Query String
• Firewall Log
• Proxy Log
• DNS Log
• DDoS Challenge Log
• Rate Limit Log
• WAF Log
  

Step 3: Select Destination

You can forward logs to one of the following destinations:

• Amazon S3: Requires Bucket Name, Region, Access Key ID, and Secret Access Key.
• Kafka: Requires Broker Address, Topic Name, and optional credentials for secure clusters.
• Syslog: Requires Syslog server address and port.
  

Note: Support for Datadog and Loggly is planned in future updates.

Best Practices & Considerations

• Use filters to limit log noise and control cost/storage.
• Secure your destinations with proper access policies.
• Do not expose S3 secrets or Kafka credentials in shared environments.
• Use logs in your observability stack (e.g., Grafana, ELK, or Splunk).
• Always test your log forwarding in staging before production.