Senior Data Engineer Extraction Enhancement
Tasks
- Automate deployments testing and release processes
- Automate monitoring and alerting
- Collaborate with stakeholders on process improvements and automation
- Develop and optimize data pipelines with Azure services
- Implement CI/CD pipelines with Azure DevOps
- Implement data ingestion transformation and integration workflows
- Improve pipeline performance and scalability
- Maintain runbooks and SOPs
- Manage incident response and perform root cause analysis
- Manage version control with git
- Monitor system performance and ensure high availability
- Participate in on call rotations and critical issue handling
- Perform data analysis and debugging
- Process large scale data with PySpark and Spark
- Provide level 2 level 3 production support for data platforms
- Support release cycles and change management
- Write and optimize complex SQL queries
Perks/Benefits
- 401k plan
- Dental insurance
- Disability insurance
- Health insurance
- Life insurance
- PTO
- Paid Holidays
- Parental leave
- Vision insurance
Skills/Tech-stack
Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Lake Storage | Databricks | Git | Infrastructure as Code | PySpark | SQL | Synapse Analytics | Terraform | “as-code”
Education
Roles
Related jobs
-
Senior Software Engineer, Knowledge Graph USD 196K-230KApache Flink | Apache Spark | Data Ingestion | Data Processing | DatabasesEmployee travel credits | Remote eligibleSenior-level Full TimeUnited States8h ago
-
Specialist Solutions Architect - AI/ML USD 180K-247KAI guardrails | Amazon Web Services | Apache Spark | Artificial Intelligence | Cloud ComputingMentorship | Remote work | Technical training | Travel up to 30 percentSenior-level Full TimeUnited States9h ago
-
Lead Data Engineer USD 115K-130KAgile | Apache Airflow | Azure Data | Azure Data Factory | Cloud Orchestration401k | Dental insurance | Medical insurance | Paid leave | Tuition reimbursementSenior-level Full TimeUniversal City, CALIFORNIA, United States9h ago
-
Ansible | ArgoCD | CI/CD | Chef | Configuration ManagementSenior-level Full TimeNew York, NY, United States11h ago
-
Senior-level Full TimeCosta Mesa, California, United States15h ago
-
API Gateway | AWS Glue | Access Control | Apache Airflow | Apache AtlasSenior-level Full TimeFAIRFAX, VA, United States15h ago
-
Senior Enterprise Data Operations Support Engineer (Tier-3 Expert Engineering Support) USD 125K-153KAWS GovCloud | Apache Airflow | Apache Kafka | Apache Spark | Azure GovernmentSenior-level Full TimeFAIRFAX, VA, United States15h ago
-
AI Developer - Model Creation & Full Stack USD 150K-175KAWS | Angular | Azure | CI/CD | D3.jsRemote work | USPS Public Trust Clearance eligibleMid-level Full TimeWork from home, VA, United States R15h ago
-
Enterprise Data Engineering & Operations Lead SME USD 107K-190KAWS GovCloud | Access Control | Apache Kafka | Apache Spark | Azure GovernmentSenior-level Full TimeFAIRFAX, VA, United States15h ago
-
Data/ML Scientist SME USD 105K-150KAWS GovCloud | Anomaly Detection | Apache Spark | Bayesian Causal Inference | Bayesian MethodsMid-level Full TimeFAIRFAX, VA, United States15h ago
-
AI Data Engineer USD 165K-225KAudit Columns | CI/CD | Cortex Analyst | Cortex Complete | DBT401k company match | Flexible paid time off | Learning and development | Medical benefits | Paid parental leaveSenior-level Full TimePhiladelphia, PA, United States15h ago
-
AWS | Agile | Azure | Cloud platform | DB2401k match | Disability insurance | Life insurance | Medical, dental, and vision insurance | Paid bench timeSenior-level Full TimeCincinnati, Ohio, United States16h ago
-
Lead AI Engineer USD 180K-280KCI/CD | ColBERT | Docker | Faiss | Fine Tuning401k match | Bonus | Childcare benefits | Dental insurance | Disability insuranceSenior-level Full TimeQuincy, MA, United States16h ago
-
AI Developer USD 140K-180KAPI Integration | Agile | Authentication | Azure | Azure FunctionsPaid Holidays | Paid time off | Tuition reimbursementEntry-level Full TimeDallas, Texas, United States16h ago
-
API Integration | AWS | AWS Glue | Batch Processing | Code reviewSenior-level Full TimeIndianapolis, IN, United States R17h ago
-
Senior-level Full TimeUnited States17h ago
-
Senior-level Full TimeUnited States17h ago
-
Product Engineer USD 120K-200KAWS | FFmpeg | Next.js | Node.js | PostgreSQL401k access | Flexible PTO | Health, dental, vision coverage | Paid mealsEntry-level Full TimeSan Francisco, CA, US18h ago
-
ADA | API Development | Application Programming | Application Programming Interface | C#Active secret clearanceSenior-level Full TimeAtlanta, Georgia19h ago
-
Senior Google Cloud Data Engineer USD 170K-240KAlerting | Anomaly Detection | Apache Beam | Apache PubSub | Batch Data ProcessingCareer advancement | Dental insurance | Health savings account | Hybrid work options | Medical insuranceSenior-level Full TimeNew York City20h ago
-
API Testing | Bash | C# | CI/CD | JUnitEntry-level Full TimePennsylvania, Exton20h ago
-
Supply Chain Data Engineer Ii USD 94K-118KDBT | Data Governance | Data Modeling | Data Pipelines | Data Quality401k | Disability insurance | Employee stock purchase plan | Health insurance | Life insuranceMid-level Full TimeWayne, PA, US, 190871d ago
-
AI/ML Engineer 2 USD 101K-165KAI Agents | API Development | AWS | Azure | CI/CDDisability insurance | Family leave | Flexible spending accounts | Life and AD D Insurance | Medical/Dental/Vision insuranceSenior-level Full TimePhiladelphia, PA, US, 191031d ago
-
Staff AI/ML Engineer USD 240K-270KAWS | Agentic Workflows | Cloud platform | Data Curation | Deep learning401k | Commuter benefits | Dog-friendly office | Equity | FSA benefitsSenior-level Full TimeSan Francisco, CA1d ago
-
Software Engineer - Perception USD 115K-180K3D Reconstruction | AWS | Azure | C++ | CI/CD401k retirement plan company match | Dental coverage | Flexible PTO | Medical coverage | Paid HolidaysSenior-level Full TimeSan Francisco, US1d ago