
Job Description
Pivotal is seeking a Data Engineer to build and operate the flight data warehouse — the system of record for telemetry off every aircraft in the fleet. Flight logs land continuously from customer and test aircraft; you will turn them into curated datasets, automated fleet health alerts, and dashboards that the firmware, GNC, battery, flight operations, and MRO teams rely on to make airworthiness and design decisions. This is a small team building production infrastructure, so you will own pipelines end to end: the data model, the services that run on it, the infrastructure as code that deploys them, and the monitoring that tells us when they break.
Responsibilities:
Pipeline development: Build and maintain the ingest and transformation pipelines that land aircraft telemetry from S3 into the flight data warehouse (AWS Glue, Athena), including schema evolution as firmware logging changes.
Data modeling: Design and maintain curated marts — flight runs, component flight hours, vehicle modes, battery flight features — in SQL, with clear grain, documented lineage, and idempotent incremental refresh.
Fleet alerting services: Develop and operate the Python services that evaluate flight-by-flight health checks and deliver alerts to engineering and customer-facing channels via SNS and Slack.
Data quality: Own the automated quality gates — mart freshness, source-to-mart reconciliation, and watermark integrity — and block deploys that would publish incorrect fleet data.
Analytics enablement: Build the dashboards and query interfaces engineering teams use to monitor the fleet, and convert repeated one-off analyses into supported datasets.
Collaboration: Work directly with firmware, GNC, battery, systems engineering, flight operations, and MRO to define what gets logged, what gets alerted on, and what “good” looks like for each signal.
Qualifications:
Bachelor’s degree in Computer Science, Computer/Electrical Engineering, or a related technical discipline
3+ years of professional data engineering experience building and operating production pipelines
Strong SQL, including window functions and query tuning on a distributed, columnar engine (Athena/Trino/Presto, Spark, Snowflake, or BigQuery)
Proficient in Python for production services — not just notebooks: packaging, dependency management, unit tests, and code review
Hands-on experience with the AWS data stack: S3, Glue, Athena, Lambda, SNS, CloudWatch, and IAM
Dimensional modeling and warehouse design, including partitioning strategy for large tables
Excellent analytical and written communication skills, with the ability to work across multidisciplinary teams
Preferred Qualifications:
Attributes aligned with Core Values:
Optimize Your Resume for This Job
Get a match score and see exactly which keywords you're missing
Job Details
- Category
- Software
- Employment Type
- Contract
- Location
- Palo Alto, CA
- Posted
About Pivotal Future
Positive actions and investments for nature have been held back by scarcity of real data, inaccurate models, and the absence of analysis tools to easily generate meaningful insights. That’s where we come in. We provide real and actionable nature intelligence to reveal risks, build resilience and drive competitive advantage. Proud recipient of the 2024 World Economic Forum "Top Innovator" for building the most promising solutions that drive progress on the United Nations Sustainable Development Goals (SDGs).
More Roles at Pivotal Future





Similar Software Roles



Found this role interesting?