1000000567.Data Engineer Lead.Info Tech Services
Tasks
- Automate recovery version control and recurring workflows
- Build anomaly detection metadata validation and data lineage tracking
- Collaborate with data architects on schemas and domain models
- Conduct architectural reviews and select data engineering tools
- Coordinate integrations event publishing and system changes
- Coordinate project planning and workload balancing
- Coordinate with cybersecurity and IT operations on patching
- Define engineering goals OKRs and performance metrics
- Define engineering standards
- Design data pipelines
- Develop and deliver scalable data systems
- Drive automation for deployments and recurring tasks
- Ensure pipeline reliability performance and security
- Establish data governance access control and privacy compliance
- Guide containerization and infrastructure as code
- Implement CI/CD and observability pipelines
- Implement data quality validation and QA practices
- Implement platform observability metrics logging alerting
- Implement real time streaming and event based architectures
- Lead data engineering function
- Manage cost and resource optimization for cloud data services
- Manage onboarding and training for new team members
- Mentor and lead data engineering staff
- Monitor log and manage incidents for data operations
- Optimize batch processing and analytics query performance
- Orchestrate data workflows across cloud and on prem
- Oversee data pipeline APIs and workflows
- Perform root cause analysis for recurring data issues
- Promote test driven development and code review
- Support compliance audits and data control implementation
Perks/Benefits
- N/A
Skills/Tech-stack
AI Automation | AWS Glue | AWS Kinesis | Access Control | Alerting | Apache Airflow | Apache Flink | Apache Spark | Automated incident management | Azure Data | Azure Data Factory | CI/CD | Cause analysis | Cloud Architecture | Cloud Computing | Cloud Pub/Sub | Code review | Containerization | Data Factory | Data Governance | Data Lineage | Data Quality | Data Security | Data Streaming | Data Validation | Databricks | Event Driven | Event-driven architecture | Google Cloud | Google Cloud Pub/Sub | Google Dataflow | Hadoop | Hybrid Cloud | IaC | Incident Management | Infrastructure as Code | Kafka | Logging | MLOps | Machine Learning | Monitoring | Observability | Privacy Compliance | Pub/Sub | Python | Real Time | Real-Time Data Streaming | Real-time Data | Root Cause Analysis | Root cause | SQL | Scala | Snowflake | Test-Driven | Test-Driven Development | Version control | Workflow Orchestration | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Generative AI | Language Models | Large Language Models | Machine Learning | Program ManagementSenior-level Full TimeSeattle, Washington, USA6h ago
-
Software Engineering Manager, LLM Training USD 170K-277KCUDA | Containerization | Context Parallelism | Data I/O | Data parallelismEntry-level Full TimeMountain View, CA, United States8h ago
-
Data Engineering Analyst Lead/Scientist - Remote USD 90K-150KAWS | Data Compliance | Data Engineering | Data Privacy | PythonSenior-level Full TimeUnited States, UNITED STATES, United States R10h ago
-
Director, Data Governance & Observability USD 165K-190KAPI Design | Alerting | Automated testing | CI/CD | DashboardsExecutive-level Full TimeNew York, NEW YORK, United States10h ago
-
Senior-level Full TimeCincinnati, OH, United States11h ago
-
Bill of Materials | Change Management | Data Cleansing | Data Governance | Data Migration401k | Collaborative team environment | Cross training | Dental insurance | Medical insuranceMid-level Full TimeWichita, KS, United States11h ago
-
Data Science Manager, AI Products USD 230K-280KAWS Bedrock | AWS Bedrock AgentCore | Agentic AI | Agents SDK | Bedrock AgentCore401k match | Employee assistance program | Flexible paid time off | Flexible spending accounts | Health insurance coverageSenior-level Full TimeNew York, NC R12h ago
-
Senior Data Engineer USD 111K-124KAccess Control | Agile | Azure | CI/CD | Data Governance401k contributions | Education assistance | Life and disability coverage | Medical, dental, and vision coverage | Paid sabbaticalSenior-level Full TimeAtlanta, Georgia or Gainesville, FL13h ago
-
Manager, Marketplace Partner Analytics USD 110K-115KCapacity Planning | Data analytics | Demand Planning | Ecommerce | Excel401k retirement program | Dental insurance | Disability insurance | Employee assistance program | Employee discountMid-level Full TimeJericho, NY, United States13h ago
-
Senior-level Full TimeBoston, Massachusetts, United States13h ago
-
Access Control | Access control systems | Camera systems | Commissioning | Control SystemsBenefits package | High visibility role | Long-term growth opportunityMid-level Full TimeSan Jose, CA, United States13h ago
-
Data Analytics & Engineering Opportunities USD 65K-105KHive | Microstrategy | MySQL | Oracle | Python401k with firm profit share | Dental insurance | Disability insurance | Firm paid holidays | Flexible spending accountEntry-level Full TimeWashington, DC, United States13h ago
-
Mid-Level Data Engineer USD 90K-98KAPI Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageRemote workMid-level Full TimeWork from home, VA, United States R14h ago
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R14h ago
-
BigQuery | Curriculum Development | Data Governance | Data Lakehouse | Data ModelingAsynchronous hiring process | Part-time hoursSenior-level Part TimeBoston, US17h ago
-
Manager, Data Quality Engineering USD 140K-160KAutomated testing | Azure Data | Azure Data Factory | Azure Data Lake | CI/CD401k matching | Adoption Assistance | Childcare tuition discounts | Company bonus | Employee stock purchase discountMid-level Full TimeAnn Arbor, MI, United States18h ago
-
Classification Algorithms | Data Analysis | Deep learning | Language Models | Language ProcessingSenior-level Full TimeSan Jose, California, United States18h ago
-
Senior Product Manager, Generative AI, Google Cloud USD 192K-278KCloud platform | Generative AI | Google Cloud | Google Cloud Platform | Language ModelsSenior-level Full TimeNew York, NY, USA19h ago
-
Associate Director, Integrated Analytics - Asundexian USD 143K-214KAI | Claims data | Claims data analysis | Competitive Analysis | Data AnalysisDental insurance | Health insurance | Paid time off | Relocation assistance | Retirement planMid-level Full TimeWhippany, New Jersey, US22h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeDiversity and Inclusion Commitment | Flexible asynchronous culture | Laid-back atmosphere | Portfolio and LinkedIn supportMid-level Full TimeBellevue, WA, USA1d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Flexible management | Remote-friendly, distributed teamMid-level Full TimeDenver, CO, USA1d ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudMid-level Full TimePalo Alto, CA, USA1d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Friendly work environment | Portfolio and LinkedIn submission flexibility | Remote-friendly environmentMid-level Full TimeCupertino, CA, USA1d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Bonus | Competitive salary | Equity | Laid-back atmosphereMid-level Full TimeSilver Spring, MD, USA1d ago
-
Bash | Data Processing | Docker | GCP | LinuxAsynchronous culture | Remote-friendly workMid-level Full TimePittsburgh, PA, USA1d ago