Lead Data Engineer #3194
Tasks
- Define reference architecture for manufacturing data
- Design and implement Palantir Foundry pipelines
- Develop incremental near real time data pipelines
- Establish pipeline orchestration best practices
- Implement data lineage and versioning in Foundry
- Mentor data engineers and enable onboarding
- Model manufacturing ontology for plants assets and orders
- Optimize compute and control pipeline cost
- Review pipelines ontology models and performance workflows
- Set up Role Based Access Control
- Translate manufacturing processes into analytical data models
- Validate data quality and enable auditability
Perks/Benefits
- Flexible working hours
- Global collaboration
- Learning and development programs
- Opportunity for advanced tools and technologies
- Performance rewards and recognition
Skills/Tech-stack
Access Control | Code Repository | Compute Optimization | Data Governance | Data Lineage | Data Quality | Data Quality Validation | Data pipeline | Event Based | Event-based data | Ontology Modeling | Palantir Foundry | Pipeline Orchestration | Python | Quality validation | Role-Based Access | Role-Based Access Control | SQL | Series data | Time Series | Time Series Data | Transforms
Education
N/A
Roles
Data Engineer | Data Platform | Data Platform Engineer | Engineer | Lead | Platform Engineer | Technical Lead
Related jobs
-
Engineering Lead,Microservices,Java Springboot /Sr. Associate Director, Technology Management INR 1500K-2400KApache Airflow | Apache Beam | Apache Flink | BigQuery | Cloud StorageMid-level Full TimePune, Maharashtra, India R11h ago
-
Sr. Cloud Engineer - Databricks/AI focus ( 5-9 years) Bangalore INR 2400K-3000KAKS | Bash | BigQuery | CI/CD | Cloud SecuritySenior-level Full TimeBengaluru, KA, India13h ago
-
Account Analytics and Impact Lead INR 2000K-5000KArtificial Intelligence | CRM | ChatGPT | Claude | Data ModelingSenior-level Full TimeHaryana, Gurugram, India14h ago
-
Vice President - AI/ML Technical Developer INR 4000K-5199KAgentic Frameworks | Alerting | Automated testing | Cause analysis | Code reviewExecutive-level Full TimeBengaluru, Karnataka, India14h ago
-
ARM Microcontroller | Arc | C# | Clock | DMAEntry-level Full TimeBengaluru, KA, India14h ago
-
Data Engineer (Python ETL) INR 800K-2000KETL | Oracle Database | PL/SQL | Performance Tuning | PythonMid-level Full TimeIN-MH-Pune14h ago
-
Mid-level Full TimeIN-MH-Pune14h ago
-
API Design | AWS | Apache Spark | Azure | Big DataSenior-level Full TimeBangalore - Karnataka - India Pune …15h ago
-
Senior Analytics Engineer INR 3000K-4000KAWS | Cloud platform | Confluent Cloud | DBT | Data IngestionAgile team collaboration | Knowledge sharing culture | Mentorship opportunitiesSenior-level Full TimeBangalore - Karnataka - India Pune …15h ago
-
Snowflake Data Engineer- Consultant INR 1500K-2000KAWS Glue | AWS Lambda | AWS S3 | Access policies | AirflowCorporate social responsibility programs | Discounts on products and services | Maternity leave | Networking opportunities | Paternity leaveMid-level Full TimeBangalore, Karnataka, India15h ago
-
Principal Engineer, Technical Architect INR 2000K-5000K.NET | API first | API-first design | Access Control | Agent OrchestrationSenior-level Full TimeRemote, India R16h ago
-
Principal Engineer, Technical Architect INR 2000K-5000K.NET | API Design | Access Control | Artificial Intelligence | Batch inferenceSenior-level Full TimeRemote, India R16h ago
-
Data Scientist - Senior INR 3600K-5000KAgile | Anomaly Detection | CI/CD | Classification | Computer VisionSenior-level Full TimePune, Maharashtra, India16h ago
-
Applied AI/ML Engineer INR 1800K-3000KAgent workflows | Agentic Systems | Auditability | Drift Detection | Error HandlingMid-level Full TimeBengaluru, Karnataka, India16h ago
-
ML Compiler Engineer, TPU Performance Optimizations INR 2500K-5000KC++ | CUDA | Compiler optimization | Computer Architecture | DebuggingSenior-level Full TimeBengaluru, Karnataka, India16h ago
-
Senior-level Full TimeIndia16h ago
-
AI Data Foundation Engineer INR 3000K-5000KApache Airflow | Apache Beam | Apache Kafka | Apache Spark | CI/CDSenior-level Full TimeIndia17h ago
-
Data Engineer –GCP BigQuery INR 3000K-4000KAdobe Experience | Adobe Experience Platform | BigQuery | Cloud Data | Cloud Data ArchitectureSenior-level Full TimeIndia17h ago
-
Associate Director (AI Hub - GDC) INR 1500K-3300KAWS | Apache Kafka | CI/CD | Cassandra | Cloud platformMid-level Full TimeHyderabad, Telangana, India18h ago
-
Sr IT Analyst INR 1500K-2500KETL | Informatica IDMC | Informatica PowerCenter | Linux | Python24 7 shift model | On-call supportSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India18h ago
-
Senior-level Full Timebengaluru, India18h ago
-
Senior-level Full Timebengaluru, India18h ago
-
Senior-level Full Timebengaluru, India18h ago
-
Sr.Data Engineering INR 2000K-2500KAzure Data | Azure Data Factory | Azure Databricks | Azure Event | Azure Event HubsSenior-level Full Timebengaluru, India18h ago