SIEM & Threat Intel Pipeline

Overview

This project automates the operational lifecycle of handling raw system authentication logs. By parsing incoming authentication streams across heterogeneous systems, extracting malicious entities, and programmatically evaluating their reputational context, the pipeline scales triage capabilities and outputs ready-to-share threat feeds without manual engineering overhead.

Technical Deep Dive & Key Features

• 1. Automated Log Ingestion & Normalization

  Leveraged the Kibana API to programmatically query and pull raw auth event telemetry. Designed custom ingestion logic to extract and normalize authentication strings hailing from disparate systems—specifically focusing on Linux SSH, Windows SSH, and Windows RDP structural logs. Normalized data is safely structured and stored in an internal PostgreSQL relational database.

• 2. Threat Intel Enrichment

  Integrated the VirusTotal API to dynamically query the reputation of isolated source IP addresses. The pipeline enriches raw telemetry datasets on-the-fly with actionable external parameters like malicious detection counts, registered autonomous system numbers (ASNs), and historical system behavior context.

• 3. Risk Ranking & Prioritization

  Constructed a custom algorithmic scoring architecture to separate noisy ambient scanning traffic from highly targeted attacks. Attacker infrastructure is prioritized, weighted, and ranked based on total attack volume, historical reputation metrics, targeted services, and repeat offender timelines.

• 4. STIX-Compliant Structured Export

  Normalized threat findings are mapped directly into valid STIX 2.x (Structured Threat Information Expression) schemas. Indicators of Compromise (IoCs) are packaged as structured JSON objects alongside clean Markdown summaries, ready for automation integration into TIP/SIEM systems or public repository publication via GitHub.

Core Tech Stack