Data Engineer
Tasks
- Architect data integration pipelines
- Build data storage warehouses data lakes and data lakehouses
- Create and optimize Apache Spark jobs and clusters
- Design data models and schemas for analytics
- Develop Python and PySpark data pipelines
- Develop and maintain ELT/ETL processes
- Implement automated testing and performance testing
- Implement data orchestration workflows
- Mentor junior engineers
- Migrate Azure Synapse to Azure Data Lake
Perks/Benefits
Skills/Tech-stack
ARM | Agile | Airflow | Apache Spark | Azure | Azure Artifacts | Azure Blob | Azure Blob Storage | Azure Boards | Azure Data | Azure Data Factory | Azure Data Lake | Azure Databricks | Azure DevOps | Azure Pipelines | Azure SQL | Azure SQL Database | Azure Stream | Azure Stream Analytics | Azure Synapse | Azure Synapse Analytics | Blob Storage | CI/CD | DBT | Data Factory | Data Governance | Data Lake | Data Modeling | Data Quality | Data Warehousing | Databricks | Databricks Workflows | Delta Lake | Delta Sharing | ELT | ETL | GitHub | GitHub Actions | Infrastructure as Code | Jenkins | Jira | Kafka | NumPy | Pandas | PySpark | Python | SDLC | SQL | SQL Database | Scala | Spark SQL | Stream Analytics | Synapse Analytics | Terraform | Unity Catalog | “as-code”
Education
Roles
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Applied AI Engineer - Bay Area USD 211K-263KArtificial Intelligence | C plus plus | C# | Embeddings | Feature Engineering401k | Comprehensive health and wellness benefits | Learning and development opportunities | Unlimited time offMid-level Full TimeHQ (San Francisco)26d ago
-
Data Engineer I USD 92K-123KAPI Integration | Data Governance | Data Modeling | Data Quality | Data cloud401k match | Career advancement opportunities | Employee resource groups | Flexible PTO | Flexible work environmentMid-level Full TimeMorrisville, NC, US, 275604h ago
-
Member of Technical Staff (AI Software Engineer, Agents) USD 220K-405KAI Evaluation | Agent architecture | Browser technologies | Chrome DevTools | Chrome DevTools ProtocolSenior-level Full TimeSan Francisco6h ago
-
Sr Technical Solutions Engineering USD 130K-178KAWS | Automated Patch Deployment | Azure | Bash | CloudFormation24x7 on-call support | Secure facility accessSenior-level Full TimeMcLean, Virginia10h ago
-
Staff Technical Solution Engineering USD 153K-210KAir-gapped | Air-gapped networks | Automation | Bash | Cloud infrastructure24x7 on call coverage flexibility | Benefits package | Secure facility onsite workSenior-level Full TimeMcLean, Virginia10h ago
-
98: Technical Staff Quantum Science and Engineering USD 145K-220KANSYS HFSS | Analog circuit | Analog circuit design | COMSOL | Circuit design401k matching | Health, dental, and vision insurance | MIT-funded pension | Mentorship programs | Paid leaveSenior-level Full TimeLexington, MA, US11h ago
-
Mid-level Full TimeCosta Mesa, California, United States11h ago
-
AWS | Apache Spark | Big Data | Cloud platform | Data EngineeringSenior-level Full TimeMcLean, Virginia; Washington, D.C.12h ago
-
AWS | Apache Spark | Azure | Cloud platform | Data EngineeringConference and meetup promotion | Equity eligibility | Performance bonus eligibility | Travel opportunitiesSenior-level Full TimeMaryland; Virginia; Washington, D.C.12h ago
-
Analytics Engineer USD 147K-225KApache Airflow | BigQuery | DBT | Databricks | Python401k | Comprehensive benefits | Equity | Flexible time offSenior-level Full TimeUS Remote, San Francisco, CA; New … R12h ago
-
Staff Data & Machine Learning Engineer USD 118K-136KDBT | Data Architecture | Data Governance | Data Quality | Data Streaming401k match | Dental insurance | Family planning resources | Flexible vacation | Fully remoteSenior-level Full TimeRemote - USA R12h ago
-
Staff Software Engineer, AI Native Web Platform USD 198K-272KCI/CD | CMS | Design Systems | JavaScript | LLMSenior-level Full TimeMountain View, California12h ago
-
Senior AI Engineer, Real-World Data USD 125K-175KAI orchestration | AWS | AWS Fargate | AWS Lambda | Agile deliverySenior-level Full TimeUS Remote R12h ago
-
Senior-level Full TimeNew York, NY13h ago
-
Senior Data Engineer USD 153K-214KAWS Security | Amazon Web Services | Data Ingestion | Data Pipelines | Data Transformation401k match | Dental insurance | Long-term disability | Medical insurance | Paid HolidaysSenior-level Full TimeWashington, District of Columbia, United States13h ago
-
Staff Data Platform Engineer USD 210K-240KAuditing | Azure Event | Azure Event Hubs | Batch Processing | CI/CDHealth plan subsidies | Paid global offsites | Remote-first work culture | WFH office reimbursementSenior-level Full TimeRemote - US R13h ago
-
GTM AI Engineer USD 192K-238KAPI Integration | Claude | Data Enrichment | Data Pipelines | Error HandlingAdoption leave | Commuter benefits | Dental insurance | Disability insurance | ESPPMid-level Full TimeSan Mateo, CA, United States13h ago
-
Software Engineer USD 120K-145KC++ | Consensus | Cypher | Distributed Systems | DockerHybrid work arrangement | Remote workMid-level Full TimeMilpitas, CA14h ago
-
Sr AI Engineer USD 84K-105KC# | Data Preprocessing | Digital Signal | Digital Signal Processing | Edge AICommuter benefits | Dental insurance | Employee resource groups | Flexible spending account | Life insuranceSenior-level Full TimeColumbia, MARYLAND, United States14h ago
-
LA Kings - Sr. Data Engineer USD 111K-135KAlerting | Apache Spark | Batch Processing | Cost Optimization | Data Governance401k match | Health savings account | Life insurance | Paid Holidays | Parental leaveSenior-level Full TimeEl Segundo, CA14h ago
-
A/B | A/B Testing | B testing | C++ | Cloud Computing401k employer match | Family planning support | Flexible vacation | Gender-affirming care | Healthcare benefitsSenior-level Full TimeRemote - United States R14h ago
-
Senior Analytics Engineer USD 147K-190KAgile | Airflow | CI/CD | DBT | Data GovernanceCasual environment | Company retreats | Learning from leaders | Wellness programSenior-level Full TimeNew York, New York14h ago
-
Data Analytics Engineer USD 95K-115KCursor | DBT Core | GitHub Copilot | SQL | Salesforce401k matching | Health insurance | Learning budget | Life insurance | Paid vacationMid-level Full TimeBoulder, Colorado, United States14h ago
-
Data Analytics Integrator USD 92K-158KData Engineering | Data Modeling | Data Pipelines | Data Quality | Data Reconciliation401k | Dental insurance | Discretionary PTO | Health insurance | Professional development opportunitiesMid-level Full TimeAtlanta, GA14h ago
-
Senior Backend Engineer- AI Agents (Remote) USD 180K-240KA/B | A/B Testing | API Design | AWS | AzureFlexible vacation policy | Health insurance coverage | Team offsites and in person collaboration | Work with distributed teamSenior-level Full TimeUnited States R15h ago