Senior Data Engineer (Python, PySpark, Kafka)
Tasks
- Apply dimensional modeling
- Build PySpark data transformation jobs
- Develop Python data pipelines
- Develop real-time streaming pipelines
- Implement Kafka Connect
- Implement Kafka streams
- Manage code with Git
- Manage relational databases
- Optimize data warehouse architecture
- Work with Hadoop HDFS
- Work with Hadoop Hive
- Work with Hadoop YARN
- Write SQL queries
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Kafka | Data Warehousing | Dimensional Modeling | Git | GitHub | HDFS | Hadoop | Hive | Kafka Connect | Kafka Streams | PostgreSQL | PySpark | Python | SQL | SQL Server | YARN
Education
Roles
Related jobs
-
Full Stack AI Engineer - 3162 (Remote) INR 2000K-4500K.NET | AI Foundry | Agile | Azure AI | Azure AI FoundryFlexible working hours | Learning and development programs | Remote work | Rewards and recognition | Work-life balanceSenior-level Full TimeChennai, India R4h ago
-
Mid-level Full TimeIN-TN-Chennai5h ago
-
Senior-level Full TimeIN-HR-Gurgaon5h ago
-
AI Developer with Move Framework INR 1800K-2800KAgentic AI | Artificial Intelligence | Chatbot | MCP | MoveworksMid-level Full TimeIN-KA-Bangalore5h ago
-
Senior QA Engineer - GenAI, VoiceAI, Telephony, and IVR testing INR 1800K-3000KAPI Testing | ASR | Automation frameworks | CCaaS | Call Flow TestingSenior-level Full TimeNoida, Uttar Pradesh, India6h ago
-
Mid-level Full TimeBengaluru, Karnataka, India6h ago
-
Elite Software Engineering Pipeline (All Levels) INR 2475K-3500KAWS | Amazon Neptune | Angular | Autogen | Database MigrationHybrid work | Inclusive culture | Structured mentorshipSenior-level Full TimePune, MH, India6h ago
-
3-5 Years Experience in DevOps(MLOps), ML-Python INR 2500K-4225KAngular | DevOps | MLOps | Machine Learning | PythonSenior-level Full Timecoimbatore, India6h ago
-
Senior-level Full TimeBangalore, Karnataka, India7h ago
-
Group Manager - Data Engineering INR 1500K-2500KAgile | Apache Airflow | Apache Kafka | BigQuery | CI/CDMid-level Full TimeGurgaon, HR, India7h ago
-
Senior AI/ML Engineer I INR 3000K-4000KAzure | Data Pipelines | Distributed Systems | Docker | Fine TuningCertification programs | Education reimbursement program | Employee assistance program | Flexible work options | Health coverageSenior-level Full TimeBangalore, Karnataka, India7h ago
-
DAX | Data Documentation | Data Governance | Data Quality | Data WarehousingAsync first collaboration | Flexible overlap schedule | Remote workSenior-level Full TimePune, Maharashtra, India8h ago
-
Entry-level Full TimeIndia8h ago
-
Mid-level Full TimePune, ASIA, India9h ago
-
Consultant - Python Developer with Gen AI using GCP INR 2000K-3500KAI Agents | API Development | Cloud platform | Containerization | Gemini EnterpriseWork from client office 5 days per weekSenior-level Full TimeBangalore, Karnataka, India9h ago
-
Senior-level Full TimeCoimbatore, India10h ago
-
Senior Engineer, AI, Quality Assurance INR 1800K-3000KAgile | Artificial Intelligence | CI/CD | Data Quality | Drift DetectionSenior-level Full TimeChennai, IN10h ago
-
Senior-level Full TimeChennai, IN10h ago
-
Analytics Solution- Associate INR 1640K-2461KAgile | Change Data Capture | Continuous Deployment | Continuous integration | Data CaptureSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Mid-level Full TimeHyderabad, TS, India10h ago
-
Senior Engineer, Industrial Engineering INR 2400K-3500KCalculated Fields | DAX | Data Modeling | Data Warehousing | ETLSenior-level Full TimeBengaluru, KA, India10h ago
-
Software Engineer - Spark, scala, Python, SQL, Databricks INR 1500K-2100KAirflow | Apache Spark | Automated testing | Azure Data | Azure Data LakeEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India10h ago
-
AWS Glue | AWS IAM | AWS Lambda | AWS S3 | Amazon EC2Mid-level Full TimeHyderabad, Telangana, India11h ago
-
Senior-level Full TimeHyderabad, Telangana, India11h ago
-
Data Driven Development for Data/AI/CV based applications INR 2500K-3500KAdversarial Networks | AutoEncoder | Azure Machine Learning | Convolutional Neural Networks | Deep learningAgile work environment | Strong communication cultureSenior-level Full Timebangalore, India11h ago