PySpark Data Engineer | Big Data & Analytics
Bengaluru - GTP, India
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Automate data ingestion and validation
- Build ML pipelines
- Build reusable automation scripts
- Collaborate with stakeholders
- Deploy ML models
- Develop and optimize data pipelines
- Develop streaming and batch workflows
- Monitor pipeline performance
- Perform feature engineering
- Transform and clean data
- Troubleshoot pipeline issues
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | Airflow | Amazon S3 | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Synapse | CI/CD | Cassandra | Data Factory | Data Lake | Docker | Git | GitHub Actions | Hadoop HDFS | Hive | Jenkins | Jupyter | Kubernetes | MongoDB | MySQL | NumPy | Pandas | PyCharm | PySpark | Python | SQL Server | Scikit-learn | Spark DataFrames | Spark MLlib | Spark SQL | VS Code
Education
Roles
Data Engineer | Data Scientist | Engineer | PySpark Data Engineer | Scientist
Related jobs
-
Senior-level Full TimeKochi, India9h ago
-
DevOps Engineer (AI Platform) INR 2000K-4800KAccess Control | Application Insights | Azure DevOps | Bicep | Cloud NetworkingSenior-level Full TimePune, Maharashtra, India10h ago
-
ML Developer (with Computer Vision Expertise)- Eurasia INR 2500K-5000KAWS | Adversarial Networks | Algorithms | Azure | Convolutional Neural NetworksSenior-level Full TimePune City, India10h ago
-
Software Developer (AIML) INR 1800K-2700KAWS | Azure | CI/CD | Computer Vision | Convolutional Neural NetworksFlexible working hours | Growth and learning | Referral bonus | Team building events | Training and booksMid-level Full TimePune, India10h ago
-
Mid-level Full TimeKarnataka, Bengaluru, India11h ago
-
Data Scientist INR 3000K-4000KAI Assistants | Amazon Athena | Amazon Redshift | Data Lake | Data ModelingEntry-level Full TimeGurugram, Haryana, India11h ago
-
Analyst-Data Science INR 2500K-4064KActive Learning | Apache Hive | Apache Spark | Bayesian Models | Decision TreesCareer development | Holistic wellbeing support | Learning opportunitiesNone Full TimeGurugram, HR, India11h ago
-
Senior-level Full TimePune, Maharashtra, India12h ago
-
Data Scientist INR 3000K-4000KAmazon Athena | Amazon Redshift | Artificial Intelligence | Data Modeling | Data WarehousingEntry-level Full TimeGurugram, Haryana, India12h ago
-
AI and Automation Engineer INR 2000K-4800KAI Security | BigQuery | Chain-of-Thought | Claude API | Cloud FunctionsAnnual leave | Casual leave | Public holidays | Sick leaveSenior-level Full TimeGurugram, India13h ago
-
Sr. Data Scientist / Data Scientist INR 2000K-4500KAgent workflows | Azure | CI/CD | Computer Vision | Data AnalysisSenior-level Full TimePune, Maharashtra, India13h ago
-
Sr.Data Engineering INR 1500K-2500KAzure | Azure Data | Azure Data Factory | Azure Databricks | Azure Key VaultSenior-level Full Timebengaluru, India13h ago
-
Associate Consultant - Data Engineer(Python/Pyspark) INR 1500K-2200KAgile | Azure | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeBangalore, Karnataka, India14h ago
-
Bi Data Analyst (6-9 YRS)- CHN/BLR INR 1500K-3000KClassification | Data Modeling | Data Visualization | Decision Trees | Deep learningSenior-level Full TimeIndia14h ago
-
Engineering Lead - Data & Analytics INR 1600K-3000KAWS | Agentic Analytics | Agile | Artificial Intelligence | Automated DeploymentSenior-level Full TimePune, MAHĀRĀSHTRA, India14h ago
-
Java Fullstack Engineer with GenAI & React INR 1500K-2500KAgile Development | Azure | Design Patterns | Docker | Extreme programmingEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India15h ago
-
Data Scientist INR 1800K-3000KApache Spark | Artificial neural networks | Chunking | Data Structures | Deep learningMid-level Full TimeKarnataka, India16h ago
-
Senior-level Full TimeMumbai, Maharashtra, India16h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India16h ago
-
Data Scientist 2-Digital Banking Kotak 811-Regional Sales INR 1400K-2000KAlternate data | Credit Risk | Data Analysis | Data Quality | Data scoringMid-level Full TimeBangalore, Karnataka, India16h ago
-
Embedded Hardware Development INR 2500K-4460KCAN-FD | Codebeamer | DSO | EMMC | Engineering Test ManagementSenior-level Full Timebengaluru, India16h ago
-
Software Engineer INR 1500K-2100KAWS | Agile | Bash | CI/CD | CloudWatchFamily wellbeing benefits | Health and wellbeing benefits | Hybrid work | Paid time off | Work-life balance benefitsMid-level Full TimeHyderabad, India R17h ago
-
CDP,Snowflake,SQL,Java, API, AWS - (6-9 Yrs) - CHN INR 2000K-3000KAPI Integration | AWS | Audience Management | Customer Data | Customer Data PlatformSenior-level Full TimeIndia17h ago
-
Senior Data Scientist INR 3000K-5000KAWS | Cloud platform | Data sets | Deep learning | Google CloudSenior-level Full TimeBangalore17h ago
-
Lead Assistant Manager INR 1500K-3000KAWS | Agile | Angular | Application Monitoring | Artificial IntelligenceMentoring | Team collaboration | Technical leadershipSenior-level Full TimePune, Maharashtra, India17h ago