Senior Data Engineer ID75059
Tasks
- Build AWS and on premise infrastructure with Terraform
- Build ingestion workflows from third party APIs
- Consult stakeholders to gather requirements
- Design and build data warehouses and lakehouses
- Design and build scalable data lakes
- Develop data ingestion and processing at scale
- Implement Python ETL and ELT pipelines
- Manage and optimize Parquet Avro and ORC storage
- Optimize data pipelines for performance
- Orchestrate data workflows with Airflow
- Translate business goals into data architecture roadmaps
- Troubleshoot data latency and consistency issues
Perks/Benefits
- Education budget
- Exciting projects
- Fitness budget
- Flexible schedule
- Mentorship
- Office options
- Personalized growth roadmaps
- Professional growth
- Remote work options
- Team activity budget
- Techtalks
Skills/Tech-stack
API Integration | AWS | Advanced SQL | Advanced SQL optimization | Airflow | Apache Spark | Avro | Big Data | Columnar Storage | Data Lake | Data Warehouse | DuckDB | ELT | ETL | Elasticsearch | Jupyter | Jupyter Notebook | Lakehouse | Machine Learning | Medallion Architecture | NoSQL | ORC | OpenSearch | Pandas | Parquet | Polars | PySpark | Python | SQL | SQL Optimization | SageMaker | Snowflake | Spark | Terraform
Education
Roles
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R7d ago
-
AWS | Bash | CI/CD | Collibra | Data Governance100 percent onsite | Onsite support in McLean VASenior-level ContractMc Lean, United States4h ago
-
Analytics & Reporting (SAP & Alteryx) USD 90K-150KAlteryx | Alteryx Designer | Alteryx Server | Data Analysis | Data BlendingMid-level Full TimeScott Air Force Base, United States4h ago
-
A/B | A/B Testing | Apache Airflow | Apache Kafka | Apache SparkSenior-level ContractSanta Monica, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Advanced SQL | Advanced SQL optimization | AirflowEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeFort Lauderdale, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Apache Airflow | Apache Spark | Avro | Columnar StorageFlexible schedule | Mentorship | Personalized growth roadmaps | Remote work options | TechtalksSenior-level Full TimeRichmond, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Apache Airflow | Apache Spark | AvroEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionSenior-level Full TimeTexas City, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | AWS SageMaker | Apache Airflow | Apache Spark | AvroFlextime | Mentorship | Personalized growth roadmaps | Professional growth | Remote work optionsSenior-level Full TimeBoca Raton, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | AWS SageMaker | Airflow | Apache SparkEducation budget | Exciting projects | Fitness budget | Flexible schedule with remote and office options | FlextimeSenior-level Full TimeBaltimore, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Amazon S3 | Apache Airflow | Apache Spark | AvroFlexible schedule | Mentorship | Personalized growth roadmaps | Remote and office options | TechtalksSenior-level Full TimeIrving, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPIs | AWS | Airflow | Apache Spark | AvroEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeJersey City, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Airflow | Avro | Big DataExciting projects | Flexible schedule | Flextime | Mentorship | Office optionsSenior-level Full TimeOrlando, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Apache Airflow | Apache Scala | Apache SparkExciting projects | Flexible schedule | Global collaboration | Mentorship | Modern solutionsSenior-level Full TimeHouston, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Advanced SQL | Advanced SQL optimization | AirflowEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeTallahassee, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Airflow | Apache Spark | AvroEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionSenior-level Full TimeBlacksburg, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPIs | AWS | Airflow | Apache Spark | AvroEducation budget | Exciting projects | Fitness budget | Flexible schedule | MentorshipSenior-level Full TimeDallas, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPI Integration | AWS | Apache Airflow | Apache Spark | AvroEducation budget | Fitness budget | Flextime | Mentorship | Office optionsSenior-level Full TimeWest Palm Beach, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Apache Airflow | Apache Spark | Avro | Cloud platformFlexible schedule | Mentorship | Personalized growth roadmaps | Remote and office options | TechtalksSenior-level Full TimeAustin, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Avro | Big Data | Columnar StorageFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Professional growthSenior-level Full TimeLos Angeles, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | AWS SageMaker | Apache Airflow | Apache Spark | AvroEducation budget | Fitness budget | Flextime | Mentorship | Office work optionSenior-level Full TimePort Charlotte, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Amazon Web Services | Apache Airflow | Apache Spark | AvroFlexible schedule | Mentorship | Office option | Personalized growth roadmaps | Professional growthSenior-level Full TimeAtlanta, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Apache Airflow | Apache Spark | Avro | BigQueryEducation budget | Fitness budget | Flexible schedule | Mentorship | Office work optionsSenior-level Full TimeJacksonville, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Apache Avro | Apache Jupyter | Apache ParquetFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote optionsSenior-level Full TimeBoston, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAPIs | AWS | Airflow | Apache Kafka | Apache SparkEducation budget | Fitness budget | Flexible schedule | Mentorship | Professional growthSenior-level Full TimeChicago, United States5h ago
-
Senior Data Engineer ID75059 USD 156K-190KAWS | Airflow | Apache Spark | Avro | Columnar StorageExciting projects | Flexible schedule | Flextime | Mentorship | Personalized growth roadmapsSenior-level Full TimeNew York, United States5h ago