Engineer - PySpark/SQL
Bengaluru, Maruthi Onyx - TESCO TSA, India
INR 1500K-2500K (estimate) Mid-level Full Time
Tasks
- Automate workflows orchestration and scripting
- Build and maintain data pipelines
- Collaborate with data scientists to build machine learning models
- Design and implement data lake and lakehouse
- Develop data processing and analysis algorithms
- Develop dimensional normalized and curated data models
- Ensure security governance and regulatory compliance
- Implement data lineage access management and auditability
- Implement data quality validation monitoring and alerting
- Optimize batch and real-time data pipelines
- Set up CI CD pipelines and coding standards
- Tune partitioning indexing and query performance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Glue | Access Management | Agile | Alerting | Amazon Athena | Amazon IAM | Amazon S3 | Apache Spark | CI/CD | Data Lineage | Data Modeling | Data Quality | Databricks | DevOps | Dimensional Modeling | Indexing | Jira | Lakehouse | Monitoring | Orchestration | Partitioning | PySpark | Python | Query Tuning | SQL | Scrum | Unix Shell
Education
N/A
Roles
Related jobs
-
Senior-level Full TimeHyderabad, India8h ago
-
Data Engineer, Cloud Supply Chain Data Engineering INR 1500K-2200KBigQuery | Data Governance | Data Monitoring | Data Pipelines | Data QualityCollaborative team environment | Mentoring opportunitiesMid-level Full TimeBengaluru, Karnataka, India8h ago
-
Senior Staff Software Engineer, Geo INR 2500K-3465KAnomaly Detection | C++ | Continuous Learning | Data Governance | Data InfrastructureSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Application Engineer, General Software Development INR 1500K-3500KAI integration | Agentic integration | C++ | Cloud platform | Data exchangeMid-level Full TimeHyderabad, Telangana, India9h ago
-
Senior Staff Software Engineer INR 3417K-4800K.Net Core | AWS | Azure | CI/CD | Cloud NativeEnhanced medical benefits | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India9h ago
-
Mid-level Full TimePune, Maharashtra, India9h ago
-
Principal Data Science - Associate Director- Bangalore INR 1500K-2700KAI Governance | AI System Design | Advanced Analytics | Apache Spark | Artificial IntelligenceMid-level Full TimeBangalore, KA, India9h ago
-
Data Engineer INR 3000K-4000KData Ingestion | Data Pipelines | Data Quality | Data Transformation | Data ValidationSenior-level Full TimePune, Maharashtra, India9h ago
-
Senior AI/ML Engineer I INR 2771K-4500KAI vector search | Apache Spark | Autogen | Azure DevOps | CI/CDCareer advancement programs | Certification programs | Continuous learning opportunities | Dental coverage | Education reimbursement programSenior-level Full TimeBangalore, Karnataka, India10h ago
-
Mid-level Full TimeGurugram, Haryana, India10h ago
-
AWS | AWS Lambda | Amazon EMR | Amazon S3 | Apache AirflowEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time off | Wellness plansSenior-level Full TimePune, MH, India10h ago
-
DE&A - AIML - Data Science - Machine Learning INR 2500K-5000KAI orchestration | AWS | Agentic AI | Azure | CI/CDSenior-level Full TimeIndia10h ago
-
AWS | Agentic Workflows | Azure | CI/CD | Cloud platformCharity matching | Corporate volunteering | Dental insurance | Medical insurance | Paid time offMid-level Full TimeBengaluru, KA, IN11h ago
-
Engineering-L2-Bengaluru-Associate-Software Engineering INR 1500K-2500KApache Spark | Avro | CI/CD | Cause analysis | ClusteringMid-level Full TimeBengaluru, Karnataka, India11h ago
-
Engineering-L2-Bengaluru-Analyst-Software Engineering INR 1500K-2500KApache Spark | CI/CD | Cause analysis | Clustering | Data ModellingEntry-level Full TimeBengaluru, Karnataka, India11h ago
-
Senior Java Backend Engineer with GenAI skills INR 2500K-3465KAgile | Cloud Computing | Design Patterns | Docker | Extreme programmingEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India12h ago
-
Data Engineer I INR 1500K-2000KAWS | Apache Spark | Automated testing | Azure | Azure DataHybrid work | Inclusive work environment | Professional developmentMid-level Full TimeBangalore, Karnataka, IN12h ago
-
Java Fullstack Engineer with GenAI & React INR 1500K-2500KAgile | Design Patterns | Docker | Extreme programming | GitHub CopilotEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India12h ago
-
Data Engineer I INR 1500K-2000KApache Spark | Automated testing | Azure Data | Azure Data Factory | Azure DatabricksCareer development | Learning opportunitiesMid-level Full TimeBangalore, Karnataka, IN12h ago
-
Engineering-L2-Bengaluru-Associate-Software Engineering INR 1800K-3500KApache Spark | C++ | CI/CD | Continuous Delivery | Data CurationExposure to finance domain | Fast-paced environment | Global team collaborationMid-level Full TimeBengaluru, Karnataka, India12h ago
-
Cloud Data Engineer - AI/ML INR 4000K-5000KAmazon Web Services | Apache Spark | Batch Processing | Cassandra | Cloud platformSenior-level Full TimeBengaluru; Hyderabad12h ago
-
Cloud Data Engineer - AI/ML INR 4000K-5000KAmazon Web Services | Azure | Cassandra | Cloud platform | Dimensional ModelingSenior-level Full TimeBengaluru; Hyderabad12h ago
-
Data Analyst/ETL Engineer INR 1500K-2500KAWS Glue | BI tools | Data Flows | Data Transformations | ETLMid-level Full TimeChennai, India13h ago
-
Senior AI Engineer INR 2500K-5000KAWS | Azure | BM25 | CI/CD | ContainerizationFlexible work environment | Remote work options | Self-care supportSenior-level Full TimeHyderabad, India13h ago
-
Senior Software Engineer INR 1500K-3000KAPI | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data ExplorerSenior-level Full TimeIndia, Telangana, Hyderabad; India, Karnataka, Bangalore; …14h ago