1000000567.Data Engineer Lead.Info Tech Services
Tasks
- Automate recovery version control and recurring workflows
- Build anomaly detection metadata validation and data lineage tracking
- Collaborate with data architects on schemas and domain models
- Conduct architectural reviews and select data engineering tools
- Coordinate integrations event publishing and system changes
- Coordinate project planning and workload balancing
- Coordinate with cybersecurity and IT operations on patching
- Define engineering goals OKRs and performance metrics
- Define engineering standards
- Design data pipelines
- Develop and deliver scalable data systems
- Drive automation for deployments and recurring tasks
- Ensure pipeline reliability performance and security
- Establish data governance access control and privacy compliance
- Guide containerization and infrastructure as code
- Implement CI/CD and observability pipelines
- Implement data quality validation and QA practices
- Implement platform observability metrics logging alerting
- Implement real time streaming and event based architectures
- Lead data engineering function
- Manage cost and resource optimization for cloud data services
- Manage onboarding and training for new team members
- Mentor and lead data engineering staff
- Monitor log and manage incidents for data operations
- Optimize batch processing and analytics query performance
- Orchestrate data workflows across cloud and on prem
- Oversee data pipeline APIs and workflows
- Perform root cause analysis for recurring data issues
- Promote test driven development and code review
- Support compliance audits and data control implementation
Perks/Benefits
- N/A
Skills/Tech-stack
AI Automation | AWS Glue | AWS Kinesis | Access Control | Alerting | Apache Airflow | Apache Flink | Apache Spark | Automated incident management | Azure Data | Azure Data Factory | CI/CD | Cause analysis | Cloud Architecture | Cloud Computing | Cloud Pub/Sub | Code review | Containerization | Data Factory | Data Governance | Data Lineage | Data Quality | Data Security | Data Streaming | Data Validation | Databricks | Event Driven | Event-driven architecture | Google Cloud | Google Cloud Pub/Sub | Google Dataflow | Hadoop | Hybrid Cloud | IaC | Incident Management | Infrastructure as Code | Kafka | Logging | MLOps | Machine Learning | Monitoring | Observability | Privacy Compliance | Pub/Sub | Python | Real Time | Real-Time Data Streaming | Real-time Data | Root Cause Analysis | Root cause | SQL | Scala | Snowflake | Test-Driven | Test-Driven Development | Version control | Workflow Orchestration | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
API | Axon | Customer360 | Data Governance | Data ManagementSenior-level ContractAustin, United States3h ago
-
Data Engineer USD 62K-62KAzure Data | Azure Data Factory | DBT | Data Factory | Data Modeling401k matching | Dental insurance | Disability insurance | Flexible spending account | Internal promotion opportunitiesEntry-level Full TimeKS, Leawood4h ago
-
Product Manager - Quantitative Data Solutions USD 140K-295KAgile | Alternative data | Capital Markets | Data Engineering | Data Science401k match | Dental insurance | Health insurance | Incentive compensation | Life insuranceMid-level Full TimeNew York4h ago
-
A/B | A/B Testing | B testing | Data Analysis | Data ModelingSenior-level Full TimeSan Jose, California, United States4h ago
-
Computer Vision | Language Processing | Machine Learning | Natural Language | Natural Language ProcessingSenior-level Full TimeSan Jose, California, United States4h ago
-
Vice President, Gen AI Product Management USD 550K-550KArtificial Intelligence | Generative AI | Language Models | Large Language Models | Machine LearningExecutive-level Full TimeSunnyvale, CA, USA5h ago
-
Product Manager, YouTube GenAI Creation Experiences USD 138K-198KData Analysis | Generative AI | Go-to-market | Machine Learning | Product ManagementMid-level Full TimeSan Bruno, CA, USA5h ago
-
Analytics Technologies | Cloud Architecture | Coaching | Cross-functional | Cross-functional leadershipSenior-level Full TimeAtlanta, GA, USA; Boulder, CO, USA5h ago
-
Executive Director, Data Governance Lead USD 120K-173KCompliance | Data Governance | Data Lineage | Data Management | Data PrivacyFlexible hybrid work model | Health and life insurance | Paid time off | Pension/retirement benefits | Personal/Family Care leavesSenior-level Full TimeJersey City, NJ, United States R6h ago
-
Data Engineers - Senior #IN1265 USD 108K-133KAccess Control | Agile | Cloud Platforms | Data Governance | Data IntegritySenior-level Full TimeColumbus, IN, United States6h ago
-
Data Engineer Principals #IN1214 USD 127K-155KAgile | Apache HBase | Apache Hive | Apache Kafka | Apache SparkSenior-level Full TimeColumbus, IN, United States6h ago
-
Staff Data Engineer USD 114K-171KCloud Platforms | Data Modeling | Data Pipelines | Data Warehousing | Data integrationDental insurance | Health care | Paid time off | Retirement plan | Sick leaveSenior-level Full TimeResidence Based, Residence Based, US8h ago
-
Associate Manager, Amazon Insights and Analytics USD 115K-173KAmazon Marketing Cloud | Amazon Vendor Central | Amazon marketing | CPG analytics | Data ModelingDental insurance | Health insurance | Paid time off | Retirement plan | Sick leaveMid-level Full TimeResidence Based, Residence Based, US8h ago
-
Trading Associate-Leverage USD 85K-110KBloomberg | Compliance reporting | ETF Portfolio Management | Excel | Investment complianceEntry-level Full TimeMilwaukee, WI; Oak Brook, IL; New …9h ago
-
AI Program Manager USD 140K-170KAPIs | Analytics | Artificial Intelligence | Automation | Change ManagementDisability benefits | Equity awards | Health insurance | Life insurance | Paid time offMid-level Full TimeNew York, New York12h ago
-
Staff Partner Engineer, Azure USD 150K-206KAI Platform | AI Platform Integration | API Integration | Apache Spark | Architecture DiagramsAnnual performance bonus | EquitySenior-level Full TimeNew York City, New York; San …13h ago
-
Technical Program Manager, Robotics Data Acquisition USD 257K-300KCapacity Planning | Data Pipelines | Data Quality | Hardware Integration | Performance AnalysisMid-level Full TimeSan Francisco13h ago
-
Fabric - Sr Data Engineer USD 112K-185KAPI-based integration | Access Control | Active Directory | Azure | Azure Active DirectorySenior-level Full TimeUnited States14h ago
-
GCP Data Engineer USD 68K-168KBigQuery | CI/CD | Cloud Storage | Cloud platform | ComposerCompany-Paid Holidays | Employee assistance program | Hybrid work | Life and disability insurance | Medical, dental, and vision coverageMid-level Full TimeNashville, TN, US16h ago
-
GCP Data Engineer USD 53K-122KAirflow | Apache Beam | BigQuery | CI/CD | Cloud StorageCompany-Paid Holidays | Employee assistance program | Life and disability insurance | Medical, dental, and vision coverage | Paid time offMid-level Full TimeNashville, TN, US16h ago
-
GCP Associate Data Engineer USD 43K-105KAI machine learning | Airflow | Apache Beam | BigQuery | CI/CD401k retirement savings | Company holidays | Disability insurance | Employee assistance program | Life insuranceMid-level Full TimeBridgewater, NJ, US16h ago
-
Data Engineer - Production Support USD 74K-99KADLS | Azure | Azure Data | Azure Data Factory | Azure Databricks401k match | Employee assistance program | Employee stock purchase plan | Health and wellness allowance | Hybrid work environmentMid-level Full TimeIrving - 6011 Connection, United States16h ago
-
Data Engineer, Integrations USD 140K-177KAWS | Airflow | Azure | Boomi | CCPA401k match | Dental insurance | Life insurance | Medical insurance | Tuition reimbursementMid-level Full TimeWaltham Office, United States16h ago
-
Airflow | Apache Spark | CI/CD | Data Lakehouse | Data ModelingMid-level ContractATLANTA, GA16h ago
-
Data Engineer USD 90K-126KAgile | Azure Data | Azure Data Factory | Change Data Capture | Crystal Reports401k matching | Loan forgiveness | Tuition reimbursementMid-level Full TimeChicago, IL, United States17h ago