Big Data / PySpark Engineering Lead - Vice President
PLOT NO-1, S.NO. 77, India
INR 2000K-3000K (estimate) Senior-level Full Time
Tasks
- Benchmark performance against legacy systems
- Build CI CD pipelines for data jobs
- Collaborate with product managers on data availability
- Conduct code reviews and mentorship
- Conduct risk assessment and compliance support
- Design scalable data processing pipelines
- Develop data models and schema designs
- Implement data parity testing
- Implement data validation and encryption
- Integrate Big Data tools and frameworks
- Migrate legacy systems to data lakehouse
- Monitor system health and troubleshoot bottlenecks
- Optimize SQL queries
- Orchestrate phased migration cutover
- Perform schema evolution to Parquet Avro and Iceberg
- Re engineer ETL and ELT jobs with Spark
- Translate business requirements into technical specifications
- Tune distributed computing clusters
- Write high-performance Python code
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Flink | Apache Hive | Apache Kafka | Apache Spark | Avro | Bitbucket | CD | CI | CI/CD | CSV | Collibra | Data Lakehouse | Data Lineage | Data Validation | ELT | ETL | Encryption | Git | HBase | Hadoop | Iceberg | Impala | Informatica | JSON | MongoDB | Parquet | Performance Benchmarking | Presto | PySpark | Python | SQL | Schema evolution | Shell Scripting | Spark SQL | Starburst | Trino | Unix | YARN
Education
N/A
Related jobs
-
Airflow | Azure Data | Azure Data Factory | Azure DevOps | Azure FunctionsEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India10h ago
-
Senior Data Engineer- AWS INR 2000K-2500KAWS Glue | AWS Lambda | Amazon Redshift | Amazon Web Services | Apache AirflowSenior-level Full TimeBangalore North, India14h ago
-
Java, GCP, ReactJS/Consultant Specialist INR 2200K-3200KAnsible | Apache Airflow | Apache Atlas | BigQuery | ClaudeFlexible working | Professional developmentMid-level Full TimeHyderabad, Telangana, India R15h ago
-
Senior-level Full TimeIN-KA-Bangalore15h ago
-
Java, GCP, ReactJS/Consultant Specialist INR 2200K-3200KAnsible | Apache Airflow | Apache Atlas | BigQuery | ClaudeFlexible working | Professional developmentMid-level Full TimeHyderabad, Telangana, India R15h ago
-
Consultant - Cloud AI Developer INR 1400K-2500KAWS | Application Modernization | Application rationalization | Azure | Azure OpenAIMid-level Full TimeBangalore, Karnataka, India16h ago
-
Engineer - Embedded Software INR 2400K-4000KAgile | Assembly | Bare Metal | Black-Duck | C#Global collaborationSenior-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101316h ago
-
AI Outcome Customer Engineer, FDE, Google Cloud INR 3799K-6000KAI Model Deployment | AI model | APIs | Access Management | Cloud infrastructureSenior-level Full TimeMumbai, Maharashtra, India; Bengaluru, Karnataka, India17h ago
-
AI Customer Engineer, Developer AI, Google Cloud INR 2500K-5000KAI Agents | API Gateway | Audit Logging | Cloud platform | Distributed tracingConference participation | Travel opportunitiesSenior-level Full TimeBengaluru, Karnataka, India17h ago
-
Mid-level Full TimePune, Maharashtra, India17h ago
-
DTICI_SalesProductServices_DataEngineer_T9 INR 1500K-2500KAWS | Agile | Apache Spark | Azure | Azure DataMid-level Full TimeBengaluru, Karnataka, India17h ago
-
DTICI_DnA_LeadDataScientist_T7 INR 3000K-4800KAWS | Azure | Data Analysis | Exploratory Data Analysis | MLOpsChallenging problems | Collaborative environment | Cross-functional exposure | Professional growthSenior-level Full TimeBengaluru, Karnataka, India17h ago
-
Entry-level Full TimePune, Maharashtra, India17h ago
-
AI Full stack Engineer INR 2000K-5000KAPI Development | Agentic AI | BigQuery | Cloud SQL | Cloud platformSenior-level Full TimeIndia17h ago
-
Associate Process Manager INR 500K-1500KAzure SQL | Azure Synapse | BI Data modeling | Compliance | DAXMid-level Full TimeMumbai, Maharashtra, India17h ago
-
Associate Process Manager INR 1200K-2000KAlteryx | Data Integrator | Data Integrity | Data Pipelines | Data QualityMid-level Full TimeMumbai, Maharashtra, India18h ago
-
Agentic AI | Automation Anywhere | Azure OpenAI | Databases | Generative AIEntry-level Full Timebangalore, India18h ago
-
Lead Software Engineer - Data Platforms, REST APIs, SQL INR 3000K-5000KAWS | Agile | Authentication | Automated testing | CI/CDSenior-level Full TimeHyderabad, Telangana, India18h ago
-
Senior-level Full TimeBengaluru, Karnataka, India19h ago
-
QA Engineer INR 940K-1600KData Quality | Data Reconciliation | Data Validation | Data Warehousing | ETL TestingCareer development | Flexible work hours | Inclusive work culture | Insurance | Learning opportunitiesMid-level Full TimePune, India R19h ago
-
QA (ETL Test Engineer) INR 1000K-2000KAgile Scrum | Azure DevOps | Batch Processing | Data Modeling | Data ProcessingMid-level Full TimeNoida, Uttar Pradesh, India19h ago
-
Senior Analyst - AI/ML INR 2500K-5000KAWS | Artificial Intelligence | Automl | Azure | Cloud platformMentorship | TrainingSenior-level Full TimePune, Maharashtra, IND, 41101420h ago
-
Vice President - Risk Analytics INR 3300K-4000KAsset classification | Credit Risk | Data Governance | Data Quality | Data StorageAnnual health checkup | Dental benefits | Gender neutral policy | Group term life insurance | Inclusive workplace cultureExecutive-level Full TimeGurugram, India20h ago
-
Principal Software Engineer INR 1500K-3000KApache Spark | C# | C++ | Continuous integration | Distributed SystemsSenior-level Full TimeIndia, Karnataka, Bangalore21h ago
-
Mid-level Full TimeBengaluru, Karnataka, India21h ago