Enhancing Events with Geolocation Data in Logstash

Posted on

If you are using Logstash and need to enrich event data with geolocation information based on IP addresses, the following filter configuration can help. This setup checks if the source IP is an external IP and applies geolocation enrichment. For internal IPs, geolocation is skipped to optimize processing.

Recommended Logstash Filter for Geolocation Enrichment

if [source][ip] and [source][ip] =~ /d+.d+.d+.d+/ {
    cidr {
        add_tag => [ "source_internal" ]
        address => [ "%{[source][ip]}" ]
        network => [ "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16" ]
    }
    if "source_internal" not in [tags] {
        geoip {
            source => "[source][ip]"
            target => "[source][geo]"
            tag_on_failure => [ ]
        }
    } else {
        mutate { remove_tag => [ "source_internal" ] }
    }
}

Explanation

  1. Initial Check for IP Address
    The filter first verifies whether the [source][ip] field exists and matches the format of an IPv4 address.
  2. Internal Network Check with CIDR
    • The cidr filter checks if the IP belongs to private/internal network ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16).
    • If the IP matches any of these ranges, the tag source_internal is added to the event.
  3. Geolocation Enrichment
    • If the source_internal tag is not present, the geoip filter is applied to enrich the event with geolocation information.
    • The enriched data is stored under the [source][geo] field.
  4. Tag Cleanup
    • If the source_internal tag was added, it is removed after processing to keep the event clean and avoid unnecessary metadata.

Benefits

  • Efficient Processing: Skipping geolocation enrichment for internal IPs reduces resource consumption and improves Logstash performance.
  • Accurate Enrichment: External IPs are enriched with reliable geolocation data, enhancing the value of logged events.
  • Scalable Design: The configuration can easily be expanded by modifying the network ranges or adding more conditions.

Example Use Case

Assume you are processing logs from multiple network sources. Internal traffic from your organization’s private networks does not require geolocation enrichment, while external traffic needs to be tagged with location data for analysis and monitoring purposes. This filter simplifies that workflow and optimizes data processing.

By implementing this Logstash configuration, you ensure efficient and targeted geolocation enrichment for your Elasticsearch logs.

The post Enhancing Events with Geolocation Data in Logstash appeared first on SOC Prime.

Related Posts