Lead Data Engineer #3194
Tasks
- Define reference architecture for manufacturing data
- Design and implement Palantir Foundry pipelines
- Develop incremental near real time data pipelines
- Establish pipeline orchestration best practices
- Implement data lineage and versioning in Foundry
- Mentor data engineers and enable onboarding
- Model manufacturing ontology for plants assets and orders
- Optimize compute and control pipeline cost
- Review pipelines ontology models and performance workflows
- Set up Role Based Access Control
- Translate manufacturing processes into analytical data models
- Validate data quality and enable auditability
Perks/Benefits
- Flexible working hours
- Global collaboration
- Learning and development programs
- Opportunity for advanced tools and technologies
- Performance rewards and recognition
Skills/Tech-stack
Access Control | Code Repository | Compute Optimization | Data Governance | Data Lineage | Data Quality | Data Quality Validation | Data pipeline | Event Based | Event-based data | Ontology Modeling | Palantir Foundry | Pipeline Orchestration | Python | Quality validation | Role-Based Access | Role-Based Access Control | SQL | Series data | Time Series | Time Series Data | Transforms
Education
N/A
Roles
Data Engineer | Data Platform | Data Platform Engineer | Engineer | Lead | Platform Engineer | Technical Lead
Related jobs
-
Staff Software Engineer - Database Engineering INR 2771K-4064KAmazon Aurora | Ansible | Cassandra | Citus | Configuration ManagementSenior-level Full TimeBangalore, India R4h ago
-
Snowflake & DBT Expert INR 1500K-2500KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | CI/CDSenior-level Full TimeNoida, UTTAR PRADESH, India4h ago
-
APIs | Agent workflows | C++ | Caching | CancellationSenior-level Full TimeBengaluru, India5h ago
-
Senior-level Full TimeBangalore5h ago
-
Lead - Data Scientist INR 4000K-5449KCalculus | Constraint Optimization | Cross-validation | Database Schema | Database Schema DesignSenior-level Full TimeBengaluru, India5h ago
-
GCP Data Engineer Lead INR 2000K-2500KApache Airflow | BigQuery | CI/CD | Cloud Composer | Cloud FunctionsSenior-level Full TimeHyderabad, //TS, India5h ago
-
Access Control | Apache Airflow | Auditability | Azure | Change Data CaptureCollaborative environment | Documentation and architecture diagram creation support | Flexible contract engagement | Fully remote | Knowledge sharing cultureSenior-level Contract Full TimeIndia R7h ago
-
Alerting | Batch Processing | Big Table | Data Modeling | Data ObservabilityFully remote | Inclusion and collaboration culture | Professional growth opportunities | Well-being support | Work-life balance initiativesMid-level Full TimeIndia R7h ago
-
AI Engineer/Scientist INR 1800K-3000KAWS | Agentic Workflows | C++ | Data Structures | Data Structures and AlgorithmsHybrid work schedule | Mentorship | Onsite days 3 days per weekMid-level Full TimeBengaluru R8h ago
-
Senior Engineer - Platform & Data Infrastructure INR 3000K-5000KAWS IoT | AWS IoT Core | ClickHouse | ECS | GoSenior-level Full TimeHyderbad11h ago
-
Sr Technical Lead-Data Engg INR 1500K-2500KAzure | Crystal Reports | DAX | Data Governance | Microsoft FabricSenior-level Full TimeINDIA - BENGALURU - BIRLASOFT OFFICE, …11h ago
-
Lead Database Developer-PostgresSQL INR 1500K-2500KAWS | AWS RDS | Azure | Backup and Recovery | Capacity PlanningSenior-level Full TimeR271- Noida, India11h ago
-
Lead Analyst, Data Operations INR 3000K-4000KAWS | Code review | Dashboard Development | Data Governance | Data PipelinesCollaborative culture | Leadership Training Program | Learning and development programs | Multicultural experienceSenior-level Full TimeIndia R14h ago
-
Lead Analyst, Data Operations INR 3000K-4000KAWS | LookML | Looker | Metabase | Power BICollaborative multicultural environment | Leadership training | Learning and development programsSenior-level Full TimeIndia R14h ago
-
Apache Spark | Avro | Azure Data | Azure Data Factory | Azure Data LakeRemote work optionMid-level Full TimeDallas / Philadelphia, USA; Bangalore, India; …15h ago
-
AWS | Amazon Aurora | Amazon RDS | Amazon S3 | CI/CDMid-level Full TimeDahisar West, Maharashtra18h ago
-
Data Platform Operations Engineer (Databricks, DBT, Snowflake, AWS, Aurora) - Mumbai, MH - AS A INR 1300K-2000KAmazon Aurora | Amazon S3 | Amazon Web Services | Backup and Recovery | CDCMid-level Full TimeMumbai, MH18h ago
-
Apache Spark | Data Pipelines | Data Validation | Databricks | Feature EngineeringMid-level Full TimeMumbai, MH18h ago
-
Data Platform Engineer GBP 76K-100KAWS EC2 | AWS Glue | AWS IAM | AWS Secrets | AWS Secrets ManagerCertification study budget | Course and conference budget | Funded certification paths | Hybrid/Remote flexibility | On-call rotationSenior-level Full TimeAshford, England, United Kingdom; Bengaluru, Karnataka, …18h ago
-
Lead MySQL Database Engineer INR 2500K-3100KAWS Aurora | AWS RDS | Amazon Web Services | Ansible | Backup and RecoveryCareer development | In-office work culture | Non-salary benefitsSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Kafka Administrator INR 3000K-4200KAWS | AWS Glue | AWS IAM | AWS Secrets | AWS Secrets ManagerBudget for courses and technical conferences | Funded certification paths | Hybrid/Remote flexibility | In-office work culture | Mentorship and career progressionSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Lead Software Engineer - Machine Learning INR 5000K-6612KCross-validation | Data Pipelines | Distributed Systems | Low Latency | Low-Latency SystemsSenior-level Full TimeHyderabad, India21h ago
-
Software Engineer III - Python INR 3000K-5000KAlerting | CI/CD | Docker | Infrastructure as Code | KubernetesSenior-level Full TimeBengaluru, Karnataka, India21h ago
-
Senior-level Full TimeBANGALORE, Karnataka, Inde21h ago
-
Agentic AI | Agile | Artificial Intelligence | Docker | Generative AISenior-level ContractShaikpet, India22h ago