Role Purpose
The Data Platform Engineer (Cybersecurity) is responsible for designing, building, and maintaining scalable, real-time data platforms that support security operations, threat intelligence, and cyber forensics.
This role focuses on developing high-throughput data pipelines, integrating with SIEM/SOAR systems, and creating data lake architectures that enable advanced analytics, incident investigations, and audit readiness. The position plays a critical role in ensuring that security data is reliable, structured, and actionable for SOC and risk teams.
Key Responsibilities
1. Real-Time Security Data Pipelines
- Design and implement real-time data ingestion pipelines for security events
- Integrate pipelines with:
- SIEM platforms (e.g., Splunk, Sentinel, QRadar)
- SOAR tools and security monitoring systems
- Ensure high availability, scalability, and low-latency data processing
- Handle large-scale event streaming and log ingestion from diverse sources
2. Threat Intelligence Correlation Engine
- Develop and maintain threat intelligence correlation engines to support SOC operations
- Integrate external and internal threat feeds to:
- Enrich security events
- Improve detection accuracy
- Enable real-time correlation and contextualization of threats
- Work closely with SOC teams to refine detection logic and use cases
3. Cyber Data Lake Architecture
- Design and manage a centralized cyber data lake for:
- Security logs
- Incident data
- Forensic and audit records
- Ensure the platform supports:
- Scalable storage and retrieval
- Data retention and lifecycle policies
- Compliance with regulatory and audit requirements
- Optimize data structures for:
- Investigations
- Reporting and analytics
- Machine learning use cases
4. Data Engineering & Platform Optimization
- Build robust ETL/ELT processes for structured and unstructured security data
- Ensure data quality, consistency, and governance across pipelines
- Optimize performance and cost efficiency of cloud-based data platforms
- Implement monitoring, logging, and alerting for data pipeline health
5. Security Analytics Enablement
- Enable advanced analytics capabilities for:
- Threat detection
- Incident response
- Behavioral analytics
- Support integration with:
- BI tools (e.g., Power BI)
- Machine learning models
- Provide datasets and structures optimized for SOC reporting and dashboards
6. Compliance, Audit & Governance
- Ensure data platforms meet:
- Internal security policies
- Regulatory and audit requirements (banking environment)
- Maintain proper data lineage, traceability, and audit trails
- Support audit requests and forensic investigations with reliable data access
Qualifications & Experience
Education
- Bachelor’s or Master’s degree in:
- Data Engineering
- Computer Science
- Information Systems or related field
Experience
- 8–10 years of experience in:
- Data engineering / big data platforms
- Cloud-based data architecture (Azure / AWS)
- Hands-on experience working with:
- High-volume, real-time data pipelines
- Security or operational data systems
- Experience in cybersecurity or SOC environments (highly preferred)
- Exposure to regulated industries (banking/financial services) is an advantage
Technical Skills
- Strong expertise in:
- Azure Data Services (Data Factory, Synapse, Event Hub, etc.)
- Databricks (mandatory experience preferred)
- Experience with:
- Streaming technologies (Kafka, Spark Streaming, or equivalent)
- SIEM/SOAR integrations
- Proficiency in:
- Knowledge of:
- Data lake architectures (Delta Lake, Lakehouse models)
- Security data schemas and log formats
- Familiarity with:
- Cloud platforms (Azure, AWS)
- Data governance and security best practices
Certifications (Mandatory / Preferred)
- Microsoft Certified: Azure Data Engineer Associate
- Databricks Certified Data Engineer Professional
Soft Skills
- Strong analytical and problem-solving capabilities
- Ability to manage large-scale data environments
- Effective collaboration with cybersecurity and SOC teams
- Strong documentation and communication skills
Key Competencies
- Real-Time Data Engineering
- Cybersecurity Data Platforms
- Threat Intelligence Integration
- Data Lake Architecture
- Cloud Data Engineering
- Security Analytics Enablement
Ideal Candidate Profile
- Experienced data engineer with strong cloud and streaming expertise
- Proven ability to handle high-volume, real-time security data
- Familiar with SOC operations and cybersecurity data use cases
- Capable of designing platforms that support analytics, investigations, and compliance
- Strong balance of engineering depth and operational reliability
