Senior Data Engineer
Tasks
- Build ingestion pipelines
- Deploy ETL resources with CloudFormation
- Design ETL pipelines
- Develop semantic layer
- Implement automated ETL reporting
- Implement data quality checks
- Integrate ETL common library into jobs
- Maintain dataset documentation suite
- Operationalize derived data products
- Parse and ingest XML filings
- Participate in agile delivery with Jira and GitHub
- Perform security scans
- Prevent duplicate data loads
- Store curated data in Apache Iceberg
- Write Python code with tests
Perks/Benefits
Skills/Tech-stack
AWS Glue | AWS Lambda | Amazon Athena | Amazon EMR | Amazon MWAA | Amazon Redshift | Amazon S3 | Amazon SNS | Amazon SQS | Apache Hive | Apache Iceberg | Apache Spark | Athena | Avro | CI/CD | CloudFormation | Data Lake | Data Quality | Deequ | ETL | EventBridge | GitHub | ORC | Oracle | Parquet | PostgreSQL | PySpark | Python | Step Functions | Trino | XML
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R13d ago
-
App Maint Engineer BackEnd SQL/ETL Sr MXN 276K-300KAzure SQL | Data integration | Database Modeling | Deadlock Resolution | ETLHybrid work | L3 supportMid-level Full TimeMéxico R3h ago
-
Contratação de Pessoa Jurídica para Desenvolvimento de Sistema de Coleta de Dados - Vaga Nº 107/2026 BRL 20K-31KAccess Control | Android | Audio capture | Audit Logging | AuthenticationSenior-level ContractHíbrido (Manaus, AM, Brasil) R5h ago
-
Backup administration | Backup architecture | Backup monitoring | Business Continuity | CommvaultRemote workSenior-level ContractRemote R6h ago
-
Azure | Azure Cloud | Azure Data | Azure Data Factory | Azure DatabricksRemote workMid-level ContractRemote R6h ago
-
Senior Data Engineer – Data Quality & Observability |1789 MXN 911K-1300KAlerting | Amazon Aurora | Amazon Aurora PostgreSQL | Amazon Redshift | Aurora PostgreSQLCross-functional collaboration | Fully remote | Opportunity for ownershipSenior-level Full TimeChihuahua, Mexico R7h ago
-
Data Engineer (Snowflake & DBT) INR 1000K-2000KAgile | Airbyte | CI/CD | Change Data Capture | DBTRemote workMid-level Full TimeRemote R8h ago
-
Associate Distinguished Engineer (Data Governance Framework, Data Catalog Implementation) INR 1300K-2000KBusiness glossary | Compliance | Data Architecture | Data Cataloging | Data ClassificationAwareness sessions | Continuous learning opportunities | Training programsMid-level Full TimeRemote, India R10h ago
-
Senior AI Engineer (m/f/d) EUR 36K-45K.NET | AI Search | AIOps | API Integration | AngularChildcare support | Coursera access | Experience days | Flexible hours | Free specialist access for health wellbeingSenior-level Full TimeGranada, AN, Spain R10h ago
-
GCP, Airflow/CONSULTANT SPECIALIST INR 1500K-2500KAccess Control | Apache Airflow | Apache NiFi | Audit Logging | BigQueryFlexible working | Opportunities for professional developmentSenior-level Full TimePune, Maharashtra, India R11h ago
-
AWS SageMaker | Airflow | Amazon EC2 | Amazon EKS | Computer VisionContinuous learning and development | Flexible schedule | Fully remote | Mentorship opportunitiesMid-level Full TimeIndia R15h ago
-
Anthropic | Benchmarking | CI/CD | Docker | FastAPIAutonomy | Fully remote | PJ contract structureSenior-level Full TimeBrazil R17h ago
-
API documentation | Airflow | Amazon Web Services | Apache Beam | Apache SparkBirthday day off | Dental insurance | Educational support programs | Fitness and wellness partnerships | Fully remote workMid-level Full TimeBrazil R17h ago
-
Senior Machine Learning Engineer AUD 148K-210KAccess Control | Agent systems | CI/CD | Capabilities Based Access Control | Error RecoveryHealth and wellness benefits | Hybrid work schedule | Remote work flexibilitySenior-level Full TimeAustralia - Melbourne - Hybrid R21h ago
-
Staff Machine Learning Engineer AUD 140K-185KAWS | Amazon ECS | Amazon ElastiCache | Amazon S3 | Audit LoggingSenior-level Full TimeAustralia - Melbourne - Hybrid R21h ago
-
Senior Machine Learning Engineer AUD 148K-210KACID | Distributed Systems | Durable Execution | Event Sourcing | Fault ToleranceHybrid work | Remote work flexibilitySenior-level Full TimeAustralia - Melbourne - Hybrid R21h ago
-
API Development | Autogen | C# | C++ | Cloud PlatformsLong-term engagement | Paid Holidays | Remote work | Training and professional growthMid-level Full TimeIndia Gate, New Delhi, India - … R21h ago
-
Software Engineer AI/ML USD 112K-150KA/B | A/B Testing | AWS | Anomaly Detection | Automated testingDental insurance | Employee assistance program | Health and wellness programs | Health insurance | Retirement benefitsMid-level Full TimeEvendale, United States R21h ago
-
Azure MLOps | Azure Machine Learning | Azure Synapse | Data Deduplication | Data GovernanceSenior-level ContractRemote R21h ago
-
Principal Data Engineer INR 3740K-5876KAirflow | Amazon Athena | Amazon EMR | Amazon Redshift | Apache FlinkHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R21h ago
-
Senior Data Engineer INR 3715K-5449KAirflow | Batch Processing | Bitbucket | CICD | Data LakesHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R21h ago
-
Senior Data Engineer INR 3715K-5449KAirflow | Batch Processing | Bitbucket | CICD | Data LakeHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R21h ago
-
Databricks Data Engineer - Bengaluru INR 1500K-2500KAWS Glue | Alation | Amazon Athena | Amazon EMR | Amazon S3Career growth opportunities | Flexible schedule | Work from anywhereMid-level Full TimeIndia-Bangalore-Office-Kalyani Vista R21h ago
-
Azure Bicep | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data FactoryCareer development | Inclusive culture | Technical trainingSenior-level Full TimeIND-Hyderabad-Hybrid, India R21h ago
-
Sr Data Engineer USD 91K-163KAI/ML | Azure DevOps | CI/CD | Code review | Data Compliance401k contribution | Career development opportunities | Comprehensive benefits | Incentive and recognition programsSenior-level Full TimeEden Prairie, Minnesota, UNITED STATES; REMOTE R21h ago