Data Engineer

Tasks

• Build Apache Spark data enrichment pipelines
• Build and scale data pipelines
• Collaborate with product, engineering, and data science teams
• Create and manage AWS Glue catalogs
• Define and enforce data contracts and schemas
• Design and maintain data architectures on AWS
• Execute data governance with lineage and schema design
• Implement data quality validation checks
• Implement observability and debugging mechanisms
• Monitor pipeline accuracy and completeness
• Optimize pipeline query and storage performance
• Orchestrate workflows with Apache Airflow
• Query data using Amazon Athena
• Troubleshoot data delivery and device tracking issues

Perks/Benefits

• Flexible location
• Full remote flexibility
• Hybrid options

Skills/Tech-stack

AWS Glue | AWS S3 | Amazon Athena | Amazon Web Services | Apache Airflow | Apache Spark | Attribution | Data Contracts | Data Governance | Data Monitoring | Data Quality | ELT | ETL | GRPC | Java | Kafka | PySpark | Python | Query Optimization | REST | SQL | Streaming | Web Services