Senior Data Engineer Extraction Enhancement
Tasks
- Automate deployments testing and release processes
- Automate monitoring and alerting
- Collaborate with stakeholders on process improvements and automation
- Develop and optimize data pipelines with Azure services
- Implement CI/CD pipelines with Azure DevOps
- Implement data ingestion transformation and integration workflows
- Improve pipeline performance and scalability
- Maintain runbooks and SOPs
- Manage incident response and perform root cause analysis
- Manage version control with git
- Monitor system performance and ensure high availability
- Participate in on call rotations and critical issue handling
- Perform data analysis and debugging
- Process large scale data with PySpark and Spark
- Provide level 2 level 3 production support for data platforms
- Support release cycles and change management
- Write and optimize complex SQL queries
Perks/Benefits
- 401k plan
- Dental insurance
- Disability insurance
- Health insurance
- Life insurance
- PTO
- Paid Holidays
- Parental leave
- Vision insurance
Skills/Tech-stack
Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Lake Storage | Databricks | Git | Infrastructure as Code | PySpark | SQL | Synapse Analytics | Terraform | “as-code”
Education
Roles
Related jobs
-
Senior Software Engineer, Knowledge Graph USD 196K-230KApache Flink | Apache Spark | Data Ingestion | Data Processing | DatabasesEmployee travel credits | Remote eligibleSenior-level Full TimeUnited States9h ago
-
Specialist Solutions Architect - AI/ML USD 180K-247KAI guardrails | Amazon Web Services | Apache Spark | Artificial Intelligence | Cloud ComputingMentorship | Remote work | Technical training | Travel up to 30 percentSenior-level Full TimeUnited States9h ago
-
Lead Data Engineer USD 115K-130KAgile | Apache Airflow | Azure Data | Azure Data Factory | Cloud Orchestration401k | Dental insurance | Medical insurance | Paid leave | Tuition reimbursementSenior-level Full TimeUniversal City, CALIFORNIA, United States10h ago
-
Ansible | ArgoCD | CI/CD | Chef | Configuration ManagementSenior-level Full TimeNew York, NY, United States12h ago
-
Senior-level Full TimeCosta Mesa, California, United States15h ago
-
API Gateway | AWS Glue | Access Control | Apache Airflow | Apache AtlasSenior-level Full TimeFAIRFAX, VA, United States16h ago
-
Senior Enterprise Data Operations Support Engineer (Tier-3 Expert Engineering Support) USD 125K-153KAWS GovCloud | Apache Airflow | Apache Kafka | Apache Spark | Azure GovernmentSenior-level Full TimeFAIRFAX, VA, United States16h ago
-
AI Developer - Model Creation & Full Stack USD 150K-175KAWS | Angular | Azure | CI/CD | D3.jsRemote work | USPS Public Trust Clearance eligibleMid-level Full TimeWork from home, VA, United States R16h ago
-
Enterprise Data Engineering & Operations Lead SME USD 107K-190KAWS GovCloud | Access Control | Apache Kafka | Apache Spark | Azure GovernmentSenior-level Full TimeFAIRFAX, VA, United States16h ago
-
Data/ML Scientist SME USD 105K-150KAWS GovCloud | Anomaly Detection | Apache Spark | Bayesian Causal Inference | Bayesian MethodsMid-level Full TimeFAIRFAX, VA, United States16h ago
-
AI Data Engineer USD 165K-225KAudit Columns | CI/CD | Cortex Analyst | Cortex Complete | DBT401k company match | Flexible paid time off | Learning and development | Medical benefits | Paid parental leaveSenior-level Full TimePhiladelphia, PA, United States16h ago
-
AWS | Agile | Azure | Cloud platform | DB2401k match | Disability insurance | Life insurance | Medical, dental, and vision insurance | Paid bench timeSenior-level Full TimeCincinnati, Ohio, United States17h ago
-
Lead AI Engineer USD 180K-280KCI/CD | ColBERT | Docker | Faiss | Fine Tuning401k match | Bonus | Childcare benefits | Dental insurance | Disability insuranceSenior-level Full TimeQuincy, MA, United States17h ago
-
AI Developer USD 140K-180KAPI Integration | Agile | Authentication | Azure | Azure FunctionsPaid Holidays | Paid time off | Tuition reimbursementEntry-level Full TimeDallas, Texas, United States17h ago
-
API Integration | AWS | AWS Glue | Batch Processing | Code reviewSenior-level Full TimeIndianapolis, IN, United States R18h ago
-
Senior-level Full TimeUnited States18h ago
-
Senior-level Full TimeUnited States18h ago
-
AI Research Engineer USD 133K-183KAnalytics | Client requirements | Data Modeling | Data Visualization | Data integrationMid-level Full TimeUSA - Atlanta, GA; USA - …18h ago
-
AI/ML Software Engineer - Recent Graduate USD 83K-83KAPIs | AWS | Algorithms | Angular | Azure401k matching | Dental insurance | Flex PTO | Health insurance | Life insuranceEntry-level Full TimeCharlotte, NC18h ago
-
Product Engineer USD 120K-200KAWS | FFmpeg | Next.js | Node.js | PostgreSQL401k access | Flexible PTO | Health, dental, vision coverage | Paid mealsEntry-level Full TimeSan Francisco, CA, US18h ago
-
Associate AI/ML Engineer USD 125K-200KAgile | Apache Spark | Data Pipelines | DevOps | Git401k match | Education & training benefits | Healthcare dental vision coverage | Paid Holidays | Paid time offMid-level Full TimeTacoma19h ago
-
ADA | API Development | Application Programming | Application Programming Interface | C#Active secret clearanceSenior-level Full TimeAtlanta, Georgia20h ago
-
Senior Google Cloud Data Engineer USD 170K-240KAlerting | Anomaly Detection | Apache Beam | Apache PubSub | Batch Data ProcessingCareer advancement | Dental insurance | Health savings account | Hybrid work options | Medical insuranceSenior-level Full TimeNew York City20h ago
-
API Testing | Bash | C# | CI/CD | JUnitEntry-level Full TimePennsylvania, Exton20h ago
-
Site AI Engineer - USVI USD 130K-179KAPI Integration | AWS | AWS Bedrock | AWS Lambda | Airflow401k match | Backup daycare days | Commuter benefits | Dental insurance | Disability insuranceMid-level Full TimeCharlotte Amalie, Saint Thomas, U.S. Virgin …1d ago