IN_Senior Associate_ Pyspark Developer _Data & Analytics _Advisory _Kolkjata
INR 2500K-4000K (estimate) Senior-level Full Time
Tasks
- Build data pipelines
- Create ETL pipelines with observability
- Design data models for analytics and reporting
- Develop Spark jobs batch and streaming
- Implement Kafka ingestion and NiFi flows
- Implement data quality checks
- Perform unit and integration testing
- Schedule and orchestrate data jobs
- Tune SQL and Spark performance
- Write technical design documents
Perks/Benefits
Skills/Tech-stack
AWS Glue | Agile | Amazon EMR | Amazon S3 | Apache Airflow | Apache Atlas | Apache NiFi | Apache Ranger | Apache Sentry | Apache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure HDInsight | Azure Synapse | Bucketing | CI/CD | Cloud Dataproc | Data Lake Storage | Data Modeling | Data Quality | Data Warehousing | Data pipeline | Databricks | Delta Lake | Dimensional Modeling | Docker | ETL Testing | Git | GitLab CI | Google BigQuery | Google Cloud | Google Cloud Dataproc | Grafana | HDFS | Hadoop | Hive | Impala | Integration Testing | Jenkins | Kafka | Kerberos | Kubernetes | Linux | Oozie | Partitioning | Prometheus | PySpark | SQL | Scala | Shell Scripting | Tez | Unit Testing | YARN
Education
Bachelor of Engineering | Master of Engineering | Master of Technology
Roles
Related jobs
-
WFS Analytics 168231 INR 1500K-2400KAgile | Architectural patterns | Design Patterns | Java | JavaScriptMid-level Full TimeHyderabad, IND-TG, India4h ago
-
Mid-level Full TimeChennai, India7h ago
-
Senior-level Full TimeChennai, India7h ago
-
Mid-level Full TimeChennai, India7h ago
-
Lead Consultant, ETL Data Engineer and Python INR 2520K-4000KData Governance | Data Modeling | Data Quality | Data Warehousing | Data integrationSenior-level Full TimeIndia-Bangalore8h ago
-
Senior-level Full TimeIN-TN-Chennai8h ago
-
Gen AI INR 2500K-4512KAPI Development | Chroma | Data Engineering | Faiss | JavaFull-time employmentSenior-level Full TimeIN-TN-Chennai8h ago
-
Senior-level Full TimeIN-AP-Hyderabad8h ago
-
Data Engineer INR 1200K-2040KAWS CDK | AWS Glue | AWS Glue Data Catalog | AWS Lambda | Amazon AthenaSenior-level Full TimeIN-TN-Chennai8h ago
-
ETL Developer INR 800K-1500KBusiness Intelligence | Data Transformation | Data integration | ETL | PySparkSenior-level Full TimeIN-TN-Chennai8h ago
-
Core Data Platform - Software Engineering-Associate-Bengaluru INR 2500K-3500KAWS | AWS Lambda | Agile | CloudFormation | Data ModelingGlobal networks | Mindfulness programs | Personal finance offerings | Training and development | Wellness programsSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Mid-level Full TimeGurgaon, Haryana, India9h ago
-
Alteryx & Analytics Developer - Consultant INR 900K-1200KAlteryx Designer | DAX | Data Governance | Data Modeling | Data QualityMid-level Full TimeBangalore, Karnataka, India9h ago
-
Software Engineer (Scala and Spark) INR 2516K-3356KAgile | Apache Spark | Azure | CI/CD | Cause analysisEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India10h ago
-
Senior Software Engineer, Cloud Bigtable INR 2000K-3500KBigtable | C++ | CI/CD | Cassandra | Column-storesSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Software Engineer, Cloud Infrastructure Engineering INR 1000K-1500KAPIs | Capacity Planning | Compute Technologies | Data Structures | Data Structures and AlgorithmsMid-level Full TimeBengaluru, Karnataka, India10h ago
-
Senior-level Full TimeMumbai, Maharashtra, India11h ago
-
Azure Data & Fabric Developer -Deputy Manager - RNA INR 1500K-2000KApache Spark | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryMid-level Full TimeMumbai, MH, India11h ago
-
Mid-level Full TimeNoida, Uttar Pradesh, India11h ago
-
Mid-level Full TimeGurgaon, HR, India11h ago
-
Assistant Manager - Data Engineering INR 1500K-2000KAPIs | Angular | Apache Airflow | BigQuery | CSSMid-level Full TimeGurgaon, HR, India11h ago
-
DAX | Dashboard Development | Data Documentation | Data Modeling | Data VisualizationMid-level Full TimeGurgaon, Haryana, India11h ago
-
Senior Engineer – AI / Cloud INR 1800K-3500KAPI Integration | AWS ALB | AWS CLI | AWS EC2 | AWS ECSSenior-level Full TimePune, Maharashtra, India11h ago
-
Quant Analytics Associate Senior INR 500K-1039KAgile | Alteryx | Data Mapping | Data Modeling | Data ReconciliationMid-level Full TimeMumbai, Maharashtra, India12h ago
-
AI Engineer INR 2000K-4500KAWS Bedrock | Agent workflows | Alerting | Amazon SageMaker | Automated testingAccreditations | Annual excellence awards | Bike to work | Career development program | Educational assistanceSenior-level Full TimeMumbai, MH, India12h ago