Network Telemetry & Log Collection Pipeline

This open-source project provides a ready-to-deploy data collection pipeline that aggregates network telemetry and security logs from ALE switches and access points into an ELK stack (Elasticsearch, Logstash, Kibana) for real-time analysis and visualization.

The solution collects three categories of network data: DPI telemetry generated by application enforcement policies configured via OmniVista, RADIUS accounting events forwarded by switches or APs using AAA profiles, and firewall traffic logs received via syslog. A Filebeat agent handles log collection at the source, while Logstash parses and enriches each data stream before indexing it into Elasticsearch under dedicated daily indices keeping RADIUS, DPI, and firewall data cleanly separated for efficient querying and dashboard building in Kibana.

A lightweight Flask server receives configuration files pushed from network devices, and a companion shell script (push_config.sh) automates the upload process from the switch side. Data synchronization is fully automated via a cron job running every 10 minutes, ensuring the ELK stack remains continuously updated without manual intervention. FreeRADIUS is integrated to correctly parse and process RADIUS accounting records received from the network infrastructure.

This connector project enables network operators to gain unified visibility into application usage, user authentication activity, and firewall traffic, leveraging the native DPI and AAA capabilities of OmniSwitch and OmniAccess Stellar devices as data sources for a modern observability stack.