Data Engineer
Tasks
- Architect data integration pipelines
- Build data storage warehouses data lakes and data lakehouses
- Create and optimize Apache Spark jobs and clusters
- Design data models and schemas for analytics
- Develop Python and PySpark data pipelines
- Develop and maintain ELT/ETL processes
- Implement automated testing and performance testing
- Implement data orchestration workflows
- Mentor junior engineers
- Migrate Azure Synapse to Azure Data Lake
Perks/Benefits
Skills/Tech-stack
ARM | Agile | Airflow | Apache Spark | Azure | Azure Artifacts | Azure Blob | Azure Blob Storage | Azure Boards | Azure Data | Azure Data Factory | Azure Data Lake | Azure Databricks | Azure DevOps | Azure Pipelines | Azure SQL | Azure SQL Database | Azure Stream | Azure Stream Analytics | Azure Synapse | Azure Synapse Analytics | Blob Storage | CI/CD | DBT | Data Factory | Data Governance | Data Lake | Data Modeling | Data Quality | Data Warehousing | Databricks | Databricks Workflows | Delta Lake | Delta Sharing | ELT | ETL | GitHub | GitHub Actions | Infrastructure as Code | Jenkins | Jira | Kafka | NumPy | Pandas | PySpark | Python | SDLC | SQL | SQL Database | Scala | Spark SQL | Stream Analytics | Synapse Analytics | Terraform | Unity Catalog | “as-code”
Education
Roles
Regions
Countries
States
Cities
Related jobs
-
API | AWS Glue | Amazon Redshift | Amazon Web Services | Apache AirflowFully remote | Mentorship opportunities | On-call supportSenior-level Full TimeOrlando, FL, United States R13h ago
-
Senior Data Engineer USD 140K-160KAirflow | Apache Spark | Auto Loader | CI/CD | Change Data Capture401k | Paid leave | Tuition reimbursementSenior-level Full TimeNorth Hollywood, CA, United States13h ago
-
Mid-level Full TimeMiami Lakes, FL, United States15h ago
-
Hive | Microstrategy | MySQL | Oracle | Python401k | Dental insurance | Disability insurance | Firm paid holidays | Flexible spending accountEntry-level Full TimeWashington, DC, United States15h ago
-
Agile | Calculus | Classification | Linear Algebra | Machine LearningSenior-level Full TimeMc Lean, VA, United States17h ago
-
Agile | Amazon Web Services | As-a-Service | Calculus | ClassificationMid-level Full TimeTysons, VA, United States17h ago
-
Algorithms | Data Structures | Java | JavaScript | LLM toolsSenior-level Full TimeMc Lean, VA, United States17h ago
-
AWS | Autogen | Automated testing | Azure | CI/CDOnsite workSenior-level ContractDallas, United States19h ago
-
Databricks Engineer USD 97K-162KAPI Integration | Agile | Data Governance | Data Modeling | Data QualityMid-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …20h ago
-
Data Scientist, Innovation Hub USD 108K-108KAudit Logging | Azure | CI/CD | Federated identity | Integration TestingCareer growth | Healthcare | On-site childcare | PTO | Paid HolidaysMid-level Full TimeTexas-Dallas-5323 Harry Hines Blvd20h ago
-
Databricks Engineer USD 107K-179KAPI Integration | Agile | Cloud Computing | Data Governance | Data ModelingTravel 10 to 15 percentMid-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …20h ago
-
Senior Data Engineer USD 133K-158KAKS | Azure Data | Azure Data Factory | Azure Data Lake | Azure DatabricksSenior-level Full TimeCincinnati, Ohio, United States20h ago
-
AI Program Lead USD 159K-265KAI Governance | Agent-Based Artificial Intelligence | Agent-based | Artificial Intelligence | Business IntelligenceMentorship | Professional development | Travel up to 25 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States20h ago
-
Associate BI Analytics Engineer USD 98K-163KDAX | Data Modeling | Data Warehousing | Databricks | ELTMid-level Full TimeUSA-VA-Herndon20h ago
-
Senior Software Engineer - Data & Analytics Federation USD 160K-240KApache Arrow | Asynchronous frameworks | C++ | Distributed Systems | Linux401k match | Dental insurance | Life insurance | Medical insurance | Paid HolidaysSenior-level Full TimePrinceton20h ago
-
Data Engineer, Employee Technology Support USD 110K-130KCause analysis | Data Governance | Data Modeling | Data Monitoring | Data Quality401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeNew York20h ago
-
AI orchestration | Alerting | Caching | Distributed Systems | DockerDental insurance | Medical insurance | Paid time off | Savings plan options | Vision insuranceSenior-level Full TimeSan Francisco, CA, United States R20h ago
-
A/B | A/B Testing | APIs | Anthropic | B testingDental insurance | Medical insurance | PTO | Remote | Savings plan optionsMid-level Full TimeSan Francisco, CA, United States R20h ago
-
Alerting | Batching | Caching | Distributed Systems | DockerDental insurance | Medical insurance | Paid time off | Remote work | Savings planMid-level Full TimeSan Francisco, CA, United States R20h ago
-
Security Engineer III - AI/ML USD 170K-215KAPI Development | AWS | Anomaly Detection | Azure | BERTSenior-level Full TimePlano, TX, United States21h ago
-
Agent Orchestration | Bias Mitigation | Data Structures | Data Structures and Algorithms | Graph theorySenior-level Full TimeSunnyvale, CA | Bellevue, WA | …21h ago
-
Staff Software Engineer, Search Local Ads Data Platform USD 207K-301KArtificial Intelligence | C++ | CSS | Data Storage | Data StructuresSenior-level Full TimeMountain View, CA, USA21h ago
-
Senior Software Engineer, AI/ML, Core USD 174K-253KC++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA21h ago
-
Software Engineer III, AI/ML, Gmail Ads Quality USD 147K-211KC++ | Data Processing | Data Storage | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA21h ago
-
Bring-up | Data Analysis | Digital Signal | Digital Signal Processing | Domain analysisSenior-level Full TimeAustin, TX1d ago