PySpark Data Engineer | Big Data & Analytics
Bengaluru - GTP, India
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Automate data ingestion and validation
- Build ML pipelines
- Build reusable automation scripts
- Collaborate with stakeholders
- Deploy ML models
- Develop and optimize data pipelines
- Develop streaming and batch workflows
- Monitor pipeline performance
- Perform feature engineering
- Transform and clean data
- Troubleshoot pipeline issues
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | Airflow | Amazon S3 | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Synapse | CI/CD | Cassandra | Data Factory | Data Lake | Docker | Git | GitHub Actions | Hadoop HDFS | Hive | Jenkins | Jupyter | Kubernetes | MongoDB | MySQL | NumPy | Pandas | PyCharm | PySpark | Python | SQL Server | Scikit-learn | Spark DataFrames | Spark MLlib | Spark SQL | VS Code
Education
Roles
Data Engineer | Data Scientist | Engineer | PySpark Data Engineer | Scientist
Related jobs
-
Data Scientist INR 1000K-2000KA/B | A/B Testing | AWS | B testing | CI/CDAgile team collaboration | Remote communication supportMid-level Full TimePune, India7h ago
-
Associate Engineer - Embedded Software INR 1200K-3000KAgile | Assembly | Bare Metal | Black-Duck | C plus plusMid-level Full TimeHadapsar, Pune, Maharashtra, IND, 4110139h ago
-
Senior-level Full TimeKochi, India10h ago
-
DevOps Engineer (AI Platform) INR 2000K-4800KAccess Control | Application Insights | Azure DevOps | Bicep | Cloud NetworkingSenior-level Full TimePune, Maharashtra, India10h ago
-
ML Developer (with Computer Vision Expertise)- Eurasia INR 2500K-5000KAWS | Adversarial Networks | Algorithms | Azure | Convolutional Neural NetworksSenior-level Full TimePune City, India11h ago
-
Software Developer (AIML) INR 1800K-2700KAWS | Azure | CI/CD | Computer Vision | Convolutional Neural NetworksFlexible working hours | Growth and learning | Referral bonus | Team building events | Training and booksMid-level Full TimePune, India11h ago
-
Mid-level Full TimeKarnataka, Bengaluru, India12h ago
-
Data Scientist INR 3000K-4000KAI Assistants | Amazon Athena | Amazon Redshift | Data Lake | Data ModelingEntry-level Full TimeGurugram, Haryana, India12h ago
-
Analyst-Data Science INR 2500K-4064KActive Learning | Apache Hive | Apache Spark | Bayesian Models | Decision TreesCareer development | Holistic wellbeing support | Learning opportunitiesNone Full TimeGurugram, HR, India12h ago
-
Senior-level Full TimePune, Maharashtra, India13h ago
-
Data Scientist INR 3000K-4000KAmazon Athena | Amazon Redshift | Artificial Intelligence | Data Modeling | Data WarehousingEntry-level Full TimeGurugram, Haryana, India13h ago
-
AI and Automation Engineer INR 2000K-4800KAI Security | BigQuery | Chain-of-Thought | Claude API | Cloud FunctionsAnnual leave | Casual leave | Public holidays | Sick leaveSenior-level Full TimeGurugram, India13h ago
-
Lead - Data Scientist INR 2500K-4720KAnomaly Detection | Cloud Computing | Data Governance | Deep learning | DevSecOpsCareer development opportunities | Inclusive workplace | Mentoring and team developmentSenior-level Full TimeNoida,Uttar Pradesh,India R13h ago
-
Sr. Data Scientist / Data Scientist INR 2000K-4500KAgent workflows | Azure | CI/CD | Computer Vision | Data AnalysisSenior-level Full TimePune, Maharashtra, India14h ago
-
Sr.Data Engineering INR 1500K-2500KAzure | Azure Data | Azure Data Factory | Azure Databricks | Azure Key VaultSenior-level Full Timebengaluru, India14h ago
-
Associate Consultant - Data Engineer(Python/Pyspark) INR 1500K-2200KAgile | Azure | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeBangalore, Karnataka, India14h ago
-
Bi Data Analyst (6-9 YRS)- CHN/BLR INR 1500K-3000KClassification | Data Modeling | Data Visualization | Decision Trees | Deep learningSenior-level Full TimeIndia15h ago
-
Engineering Lead - Data & Analytics INR 1600K-3000KAWS | Agentic Analytics | Agile | Artificial Intelligence | Automated DeploymentSenior-level Full TimePune, MAHĀRĀSHTRA, India15h ago
-
Java Fullstack Engineer with GenAI & React INR 1500K-2500KAgile Development | Azure | Design Patterns | Docker | Extreme programmingEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India16h ago
-
Data Scientist INR 1800K-3000KApache Spark | Artificial neural networks | Chunking | Data Structures | Deep learningMid-level Full TimeKarnataka, India17h ago
-
Senior-level Full TimeMumbai, Maharashtra, India17h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India17h ago
-
Data Scientist 2-Digital Banking Kotak 811-Regional Sales INR 1400K-2000KAlternate data | Credit Risk | Data Analysis | Data Quality | Data scoringMid-level Full TimeBangalore, Karnataka, India17h ago
-
Embedded Hardware Development INR 2500K-4460KCAN-FD | Codebeamer | DSO | EMMC | Engineering Test ManagementSenior-level Full Timebengaluru, India17h ago
-
Software Engineer INR 1500K-2100KAWS | Agile | Bash | CI/CD | CloudWatchFamily wellbeing benefits | Health and wellbeing benefits | Hybrid work | Paid time off | Work-life balance benefitsMid-level Full TimeHyderabad, India R18h ago