Senior Data Engineer
USA (Westport, CT), United States
USD 130K-150K Senior-level Full Time
Tasks
- Build and maintain lakehouse medallion architecture
- Contribute to knowledge sharing
- Design and implement batch data pipelines
- Design and implement streaming data pipelines
- Develop and optimize PySpark transformations
- Develop and optimize SQL transformations
- Enforce engineering best practices
- Evolve logical data models
- Evolve physical data models
- Implement data monitoring
- Implement data quality checks
- Integrate semi structured data sources
- Integrate structured data sources
- Integrate unstructured data sources
- Mentor junior data engineers
- Perform code reviews
- Troubleshoot data incidents
- Troubleshoot performance bottlenecks
- Troubleshoot pipeline failures
- Write infrastructure as code
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | Amazon Web Services | Apache Airflow | Apache Flink | Apache Iceberg | Apache Kafka | Apache Spark | Azure Data | Azure Data Factory | CI/CD | DBT | Dagster | Data Factory | Delta Lake | Docker | GCP Dataflow | Git | Great Expectations | Helm | Infrastructure as Code | Kubernetes | Prefect | PySpark | Python | SQL | Terraform | Web Services | “as-code”
Education
Roles
Regions
Countries
States
Cities
Related jobs
-
Senior-level Full TimeRemote, Canada; Computer/Software; Canada, Canada; Remote, … R2h ago
-
ANSI SQL | Amazon Redshift | Analytical modeling | Cloud Data | Cloud Data PlatformsCareer growth | Onsite work | Possible contract extension | Successful Completion Bonus | Support from consultantsMid-level Full TimeJuno Beach, FL; Juno Beach, FL; …2h ago
-
AWS | Bitbucket | Confluence | Data Analysis | Eclipse401k employer contribution | Direct contribution to 401k vesting | Educational assistance | Federal Holidays | Healthcare benefitsSenior-level Full TimeAnnapolis Junction, Maryland; Annapolis Junction, Maryland; …3h ago
-
Software Engineer 1 - (Python, UI/UX, LLM) USD 169K-207KAWS | Argo CD | CI/CD | Containerization | Docker401k match | Educational assistance | Federal Holidays | Healthcare coverage | PTOMid-level Full TimeAnnapolis Junction, Maryland3h ago
-
Senior Applied AI Engineer 3 - (Python, LLMs, RAG) USD 241K-295KAI Agents | APIs | Backend Development | Data Pipelines | Docker401k matching | Education assistance | Healthcare coverage | Paid time offSenior-level Full TimeAnnapolis Junction, Maryland3h ago
-
Cloud Software Engineer 3 - (Cloud, Hadoop, Java) USD 228K-278KAnsible | Apache Kafka | Apache Lucene | Apache Solr | Avro401k matching | Educational assistance | Federal Holidays | Healthcare coverage | Paid time offSenior-level Full TimeFort Meade, Maryland R3h ago
-
Software Engineer III, AI/ML, Platforms and Devices USD 147K-210KAndroid | C++ | Data Processing | Debugging | Language ProcessingSenior-level Full TimeSeattle, WA, USA8h ago
-
AI Strategy Lead USD 192K-278KAgentic AI | Artificial Intelligence | As-a-Service | Business Intelligence | Data AnalysisSenior-level Full TimeChicago, IL, USA; Sunnyvale, CA, USA8h ago
-
Agents | Artificial Intelligence | Automated Evaluation | C++ | Code reviewSenior-level Full TimeSunnyvale, CA, USA8h ago
-
API Development | Artificial Intelligence | C# | C++ | Computer VisionMid-level Full TimeSunnyvale, CA, USA; Atlanta, GA, USA8h ago
-
Staff ML Engineer, Search Ads Shopping Relevance Models USD 207K-300KC++ | CI/CD | Data Analysis | Data Processing | Deep learningSenior-level Full TimeMountain View, CA, USA8h ago
-
Software Engineer, ML Accelerators, DeepMind USD 174K-252KC++ | Hardware Accelerators | JAX | Kernel development | Machine LearningMid-level Full TimeMountain View, CA, USA; San Francisco, …8h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Flexible management approach | Professional growth opportunities | Remote or distributed workMid-level Full TimePalo Alto, CA, USA12h ago
-
Bash | Cloud infrastructure | Data Ingestion | Data Ingestion Pipelines | Data ProcessingAsynchronous culture | Career growth | Impactful mission | Remote-friendly cultureMid-level Full TimeLos Angeles, CA, USA12h ago
-
Bash | Cloud platform | Data Processing | Docker | GCPAsynchronous culture | Friendly work environment | Remote workMid-level Full TimeBellevue, WA, USA13h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Bonus | Equity | Remote-friendlyMid-level Full TimeDenver, CO, USA13h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Friendly laid-back atmosphere | Opportunity for career growthMid-level Full TimeAnaheim, CA, USA13h ago
-
Bash | Data Ingestion | Data Ingestion Pipelines | Data Processing | DockerAsynchronous culture | Flexible management approach | Mission driven product impact | Supportive team environmentMid-level Full TimeProvo, UT, USA13h ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous culture | Competitive benefits | Flexible management | Remote workMid-level Full TimeFort Worth, TX, USA13h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Career growth | Remote workMid-level Full TimeHouston, TX, USA13h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerMid-level Full TimeCollege Park, MD, USA13h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerMid-level Full TimeSaint Paul, MN, USA13h ago
-
Bash | Cloud platform | Data Processing | Data Processing Pipelines | DockerAsynchronous culture | Competitive benefits | Flexible management | Remote-friendly | Supportive teamMid-level Full TimeDurham, NC, USA13h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous work culture | Competitive bonus | Equity compensation | Flexible management approach | Remote-friendly workMid-level Full TimeCary, NC, USA13h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Flexible management | Friendly work environment | Portfolio review support | Remote-friendlyMid-level Full TimeCleveland, OH, USA13h ago