Data Engineer � Observability & Insights Platform

Posted on July 7, 2025

Apply Now

Job Description

  • Job Title: Data Engineer � Observability & Insights Platform
  • Location: Remote
  • Employment Type: Contract
  • Experience: 5+
  • Role Overview:
  • We are building an advanced Observability Platform designed to empower our product and platform teams with real-time insights into the availability, reliability, and performance of our digital products. As we enhance our platform's capabilities, we are embarking on a strategic journey to integrate predictive analytics, machine learning, and advanced event management for deeper business and operational insights.
  • This role is ideal for a Data Engineer who thrives at the intersection of engineering, analytics, and observability, and who is passionate about transforming raw observability data into valuable business intelligence.
  • Key Responsibilities:
  • 1. Observability Signal Correlation
  • Integrate and analyze signals from logs, metrics, and traces using Grafana and Prometheus
  • Enrich observability data by correlating it with business context to enable meaningful insights
  • 2. Data Enrichment & Pipeline Development
  • Build and maintain data pipelines to enhance technical signals with business metadata
  • Leverage OpenTelemetry (OTel) for observability instrumentation across systems
  • 3. Machine Learning Integration
  • Design, build, and deploy ML models for anomaly detection, forecasting, and incident noise reduction
  • Continuously improve ML solutions to increase relevance and business value of incident signals
  • 4. Disruption Prediction & Risk Mitigation
  • Identify trends and patterns that can predict business disruptions and support preemptive actions
  • 5. Action Enablement
  • Make observability insights actionable for business stakeholders through accessible dashboards and tools
  • Support both automated and manual decision-making processes
  • 6. Cross-Functional Collaboration
  • Work closely with IT, DevOps, and Business teams to ensure alignment between technical implementations and business objectives
  • 7. Continuous Improvement
  • Monitor and optimize data pipelines for accuracy, reliability, and performance
  • Required Skills & Qualifications:
  • Proven experience as a Data Engineer or in a similar role focused on observability and analytics
  • Strong proficiency in SQL and Python
  • Experience working on Google Cloud Platform (GCP)
  • Expertise in BigQuery for business intelligence and analytics
  • Hands-on knowledge of Grafana, Prometheus, and Splunk as monitoring/observability tools
  • Familiarity with OpenTelemetry (OTel) for observability instrumentation
  • Experience with big data technologies such as Apache Spark, Kafka, and Airflow
  • Machine Learning & Analytical Expertise:
  • Practical experience applying ML techniques to observability data for anomaly detection and forecasting
  • Ability to reduce noise in incident alerts and deliver more relevant and high-value insights
  • Strong analytical mindset to interpret complex datasets and identify actionable trends

Required Skills

strong proficiency in sql and python experience working on google cloud platform (gcp) expertise in bigquery for business intelligence and analytics hands-on knowledge of grafana prometheus and splunk as monitoring/observability tools familiarity with opentelemetry (otel) for observability instrumentation experience with big data technologies such as apache spark kafka and airflow machine learning & analytical expertise: