PySpark Developer / Senior Data Engineer
Seri Lingampally, India
A INR 1500K-2700K (estimate) Senior-level Contract
Tasks
- Build and maintain data pipelines using PySpark and Apache Spark
- Design scalable data pipelines
- Develop ETL workflows for batch and near realtime processing
- Implement data validation monitoring and logging frameworks
- Integrate data from databases APIs files and cloud storage
- Mentor junior engineers
- Optimize Spark jobs for performance reliability and cost
- Participate in design reviews and code reviews
- Support deployment scheduling and orchestration of data pipelines
- Transform large structured and unstructured datasets
- Troubleshoot job failures data quality issues and performance bottlenecks
- Write reusable modular PySpark code
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Caching | Data Quality | Data Validation | DataFrame | ELT | ETL | Git | Joins | Linux | Logging | Monitoring | Partitioning | PySpark | Python | RDD | SQL | Shuffling | Window Functions
Education
N/A
Roles
Data Engineer | Developer | Engineer | PySpark Developer | Senior Data Engineer
Related jobs
-
ML Developer (with Computer Vision) INR 3000K-5000KAWS | Algorithms | Azure | Computer Vision | Convolutional Neural NetworksMid-level Full TimePune City, India3h ago
-
Senior Agentic AI Engineer ID80140 INR 2000K-5000KAnthropic API | CI/CD | Cloud Security | GenAI | Infrastructure as CodeAnnual learning budget | Flexible hours | Internal tech talks | Mentorship | Remote workSenior-level Full TimeJaipur, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic API | Artifact Repositories | Authentication | Automated testing | CI/CDAnnual learning budget | Flexible hours | Mentorship | People focused support | Remote workSenior-level Full TimePune City, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | CI/CD | DNS | DockerAnnual learning budget | Flexible hours | Learning budget | Mentorship | People focused supportSenior-level Full TimeKolkata, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | CI/CD | DNS | FirewallsFlexible hours | Learning budget | Mentorship | Remote work | Well-being programsSenior-level Full TimeMumbai, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | CI/CD | CMDB | DNS | FirewallsAnnual learning budget | Flexible hours | Internal tech talks | Mentorship | People focused supportSenior-level Full TimeHyderabad, India3h ago
-
Senior AI Engineer ID80139 JPY 8600K-9700KAnthropic | Automated testing | CD pipelines | CI/CD | CI/CD pipelinesFlexible hours | Learning budget | Mentorship | Remote work | Support programsSenior-level Full TimeKochi, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | Automation | CI/CD | DNSAnnual performance reviews | Collaborative culture | Flexible hours | Internal tech talks | Learning budgetSenior-level Full TimeGurgaon Kty., India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | CI/CD | DNS | Git | GitHubFlexible hours | Internal tech talks | Learning budget | Mentorship | People focused supportSenior-level Full TimeBangalore North, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automation | CI/CD | DNS | Data ProcessingAnnual performance reviews | Collaborative culture | Flexible hours | Internal TechTalks | Learning budgetSenior-level Full TimeNoida, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | CI/CD | DNS | FirewallsAnnual performance reviews | Flexible hours | Learning budget | Mentorship | Remote workSenior-level Full TimeNew Delhi, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | CI/CD | DNS | FirewallsFlexible hours | Learning budget | Mentorship | Performance reviews | Remote workSenior-level Full TimeIndore, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnomaly Detection | Anthropic | Automated testing | CI/CD | DNSFlexible hours | Learning budget | Mentorship | Performance reviews | Remote workSenior-level Full TimeJaipur, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | CI/CD | DNS | FirewallAnnual learning budget | Flexible hours | Mentorship | Remote work | Well-being programsSenior-level Full TimeTrivandrum, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | CI/CD | DNS | FirewallsFlexible hours | Learning budget | Mentorship | Remote work | Well-being programsSenior-level Full TimeChennai, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | CI/CD | DNS | Firewalls | GitAnnual performance reviews | Flexible hours | Learning budget | Mentorship | Remote workSenior-level Full TimeCoimbatore South, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | CI/CD | DNS | GitFlexible hours | Learning budget | Mentorship | Performance reviews | Remote workSenior-level Full TimeJalalabad, India3h ago
-
Senior AI Engineer ID80139 INR 2000K-4500KAnthropic | Automated testing | CI/CD | DNS | FirewallsAnnual learning budget | Flexible hours | Mentorship | Remote work | Well-being programsSenior-level Full TimeAhmedabad, India3h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | Azure | CI/CD | Cloud Security | Cloud platformAnnual learning budget | Collaborative culture | Flexible hours | Internal tech talks | MentorshipSenior-level Full TimeJaipur, India3h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAmazon Web Services | Anthropic | Azure | CI/CD | Cloud SecurityAnnual learning budget | Collaborative culture | Flexible hours | Internal tech talks | MentorshipSenior-level Full TimeBengaluru, India3h ago
-
Senior-level Full TimeChennai, India4h ago
-
Data Engineer - Solution Expert INR 1500K-2700KBigQuery | CI/CD | Cloud Security | Cloud platform | Data ModelingSenior-level Full TimePune, MH, India6h ago
-
Generative AI Application Developers INR 1800K-4500KC++ | Computer Vision | Debugging | Generative AI | JavaSenior-level Full TimePune, Maharashtra, India6h ago
-
Generative AI Application Developers INR 1500K-2500KAI integration | Deep learning | Generative AI | Language Processing | Machine LearningMid-level Full TimePune, Maharashtra, India7h ago
-
Senior-level Full TimeMumbai, India7h ago