Big Data Engineer | PySpark, Hadoop Ecosystem, Cloud Integration & Data Migration
Pune - Hinjewadi (Ascendas), India
INR 3000K-4200K (estimate) Senior-level Full Time
Tasks
- Apply data governance security and compliance
- Automate workflows and improve reliability
- Design scalable data pipelines
- Develop ETL workflows
- Implement data ingestion and transformation
- Maintain data lineage and documentation
- Optimize PySpark jobs
- Perform data validation and reconciliation
- Tune query and pipeline performance
Perks/Benefits
- N/A
Skills/Tech-stack
ANSI SQL | AWS | Access Control | Apache Airflow | Apache Oozie | Avro | Azure | CSV | Data masking | Data orchestration | ELT | ETL | Encryption | GCP | GDPR | HDFS | HIPAA | Hadoop | Hive | HiveQL | Impala | JSON | MapReduce | ORC | Parquet | Pig | PySpark | Python | SQL Server | Scala | Shell | Spark SQL | Sqoop
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Software Engineering Professional INR 1500K-2500KAI orchestration | AIOps | APIs | AWS | Agent FrameworksMid-level Full TimeBengaluru, IN, 5601032h ago
-
Software Engineering Professional INR 1500K-2500KAI orchestration | AIOps | API Integration | AWS | Agent FrameworksMid-level Full TimeBengaluru, IN, 5601032h ago
-
Software Engineer - Associate INR 1500K-3000KAWS | AWS Lambda | Amazon ECR | Amazon ECS | Amazon S3Development opportunities | Reward & recognition | Volunteering opportunities | Wellness programs | Work-life balanceMid-level Full TimeHyderabad, India4h ago
-
Staff Engineer (Python/AWS) INR 2500K-3465KAPI Gateway | AWS | AWS CDK | Aurora MySQL | Aurora PostgresDEI focus | Development opportunities | Reward and recognition | Volunteering | WellnessSenior-level Full TimeHyderabad, India4h ago
-
Senior-level Full TimeBengaluru, KA, India6h ago
-
Senior Data Engineer INR 3000K-4000KAWS | Apache Spark | CI/CD | Data Modeling | Data QualityBike to work | Educational assistance | Employee assistance program | Employee discount | Fitness classesSenior-level Full TimePune, MH, India6h ago
-
Senior Software Engineer - SAST INR 2475K-3465KCompiler programming | Integration Testing | Java | Kotlin | Parser generatorSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
Staff Software Engineer (Python) INR 2000K-4000KAPIs | AWS | Authentication | Batch Processing | Cause analysisCollaborative team environment | Leadership development | Training programsSenior-level Full TimeMumbai, MH, India6h ago
-
Software Engineer II INR 2000K-2525KAWS | CI/CD | Data Structures | Docker | EC2Professional development | Training coursesMid-level Full TimeMumbai, India7h ago
-
Generative AI Developer INR 1400K-1700KCloud Functions | Cloud Storage | Cloud platform | Compute Engine | GeminiDigital learning platform access | Elder care | Flexible work | Health checks | Insurance with top-upsMid-level Full TimeBangalore, IN9h ago
-
AI Engineer INR 2000K-4000KAWS Bedrock | AWS Bedrock Agents | AWS Bedrock Guardrails | AWS VPC | Agile methodologiesSenior-level Full TimeIndia Office11h ago
-
Tech Lead AI Engineer INR 1800K-3000KAI Evaluation | AWS Bedrock | AWS Lambda | AWS SageMaker | Agile methodologySenior-level Full TimeIndia Office11h ago
-
Sr Software Engineer – AI Ops INR 2000K-3440KAPI Gateway | AWS Lambda | Amazon API | Amazon API Gateway | Amazon CloudWatchExtended hours flexibilitySenior-level Full TimeIndia Office11h ago
-
Data Engineer INR 2000K-3000KApache Kafka | Apache Spark | Cloud Computing | Data Modeling | Data WarehousingFlexible working | Hybrid workingSenior-level Full TimeHyderabad Knowledge Park Tower 2, India11h ago
-
Cloud data engineer INR 2800K-4000KAPI Integration | AWS Glue | AWS Lambda | AWS Step Functions | Amazon RedshiftSenior-level Full TimePune/Mumbai, MH, IN11h ago
-
Principal, Disaster Recovery Engineer INR 3000K-4200KAWS | Air-gapped | Air-gapped environments | Azure | BashSenior-level Full TimePune, India11h ago
-
Staff Data Engineer INR 2400K-3380KAWS Cloud | AWS Glue | Agile | Amazon Athena | Amazon CloudWatchSenior-level Full TimeBangalore - Manyata Tech Park Road, …11h ago
-
ETL Pentaho Developer, AS INR 2200K-4000KData Marts | Data Modeling | Data Quality | Data Warehousing | Data integrationAccident insurance | Childcare reimbursement | Employee assistance program | Flexible leave policy | Gender neutral parental leaveEntry-level Full TimeBangalore, Velankani Tech Park, India11h ago
-
App Engineer (AI & Automation) INR 2000K-3465KAI Assisted Development | AI-Assisted Development Tools | AI-powered automation | API Integration | Claude CodeMid-level Full TimeIndia16h ago
-
HR Tech Data Engineer INR 3000K-4000KAPI Integration | AWS | Access Control | Active Directory | Anthropic APICollaboration culture | Flexible learning opportunities | Inclusive work environment | Training and supportSenior-level Full TimeIndia (Hyderabad)18h ago
-
Data Engineer INR 1500K-2525KAPI Integration | Apache Airflow | Azure Data | Azure Data Factory | Azure SynapseMid-level Full TimeIndia - Hybrid R20h ago
-
Mid-level Full TimeHyderabad, India21h ago
-
Sr. Software Engineer - Data INR 2520K-3380KAWS | Batch Processing | Data Modeling | Distributed Systems | DockerERG events | Equipment and Support | Health coverage | Holidays | LinkedIn Learning accessSenior-level Full TimeBengaluru, Karnataka, India21h ago
-
Sr. Staff Engineer INR 2800K-4000KAI Safety | API Gateway | APIs | AWS | Apache SparkEnhanced medical benefits | Family well-being benefits | Paid time off | Work-life balanceSenior-level Full TimeHyderabad, India22h ago
-
AI Ecosystem & Intelligence Platform INR 2500K-4600KAPI Integration | AWS | Azure | Cloud Platforms | Cloud platformSenior-level Full Timebangalore, India22h ago