Specialist, GSF DnA Data Engineer
IND - Telangana - Hyderabad (HITEC City), India
INR 1500K-2465K (estimate) Mid-level Full Time
Tasks
- Apply unit and integration testing and automated data tests
- Build dimensional models for analytics
- Create technical documentation and runbooks
- Define data SLAs and SLOs
- Design batch and streaming data pipelines
- Develop and maintain ETL ETL transformations
- Develop scalable processing with PySpark and Spark
- Implement CI/CD quality gates
- Implement data quality controls and validation
- Manage metadata lineage and data catalog practices
- Optimize ETL ETL jobs for performance cost and reliability
- Orchestrate and monitor data pipelines
- Package and deploy workloads with Docker
- Provision and manage AWS resources with Infrastructure as Code
- Set up GitHub version control and CI CD
- Support operational handoffs and production support
- Write and tune advanced SQL
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Glue | AWS Lambda | AWS Step Functions | Amazon CloudWatch | Amazon EMR | Amazon S3 | Anomaly Detection | Automated testing | CI/CD | Data Catalog | Data Governance | Data Lake | Data Lineage | Data Quality | Data Warehousing | Data profiling | Databricks | Databricks Workflows | Delta Lake | Delta Lake ACID | Dimensional Modeling | Docker | ELT | ETL | Encryption | GitHub | IAM | Infrastructure as Code | Integration Testing | Lakehouse | Least Privilege | Metadata | Observability | Orchestration | PySpark | Python | Reconciliation | SCD Type-2 | SQL | Schema evolution | Slowly Changing Dimensions | Snowflake Schema | Spark Caching | Spark Partitioning | Spark Shuffle | Star Schema | Step Functions | Terraform | Type 2) | Unit Testing | “as-code”
Education
Roles
Related jobs
-
AWS | Agentic AI | Apache Spark | Databricks | Generative AICollaborative global engineering culture | Continuous learning | Flexible work environment | Opportunity to work on cutting edge GenAI | Remote-friendly work environmentSenior-level Full TimeIndia3h ago
-
.NET | APIs | AWS | Alerting | AzureAdoption leave | Employee assistance program | Health care benefits | Maternity leave | Mental wellness programsSenior-level Full TimeIndia5h ago
-
AI Data Engineer INR 2000K-3300KAWS | Agent memory | Audio Processing | Batch Processing | Cloud ComputingEquityMid-level Full TimeNew Delhi7h ago
-
Data Engineer, Analytics Engineering, VP INR 1500K-6000KAWS Data | AWS Data Catalog | AWS Glue | AWS Glue Catalog | AWS Glue Data GovernanceExecutive-level Full TimeBengaluru, India9h ago
-
Senior Machine Learning Engineer, Model Training & Evaluation INR 2500K-4500KBenchmarking | Checkpointing | DeepSpeed | Distributed Training | Experiment trackingAccidental insurance | Flexible hours | Hybrid work | Life insurance | Medical insuranceSenior-level Full TimeBangalore, India (Hybrid) R10h ago
-
Sr. Delivery Solutions Architect INR 3000K-3900KData Architecture | Data and AI | Distributed Systems | Executive Communication | Program ManagementHybrid work | Travel 30 percentSenior-level Full TimeBengaluru, India10h ago
-
AWS Glue | AWS Lambda | Amazon QuickSight | Amazon S3 | DBTSenior-level Full TimeIndia19h ago
-
API Testing | Agile | Airflow | Azure | BitbucketEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai/Pune, India19h ago
-
Gen AI Engineer INR 2000K-4800KAI Governance | API Integration | AWS Bedrock | Azure OpenAI | Azure OpenAI ServiceSenior-level Full TimeIN-TN-Chennai20h ago
-
Senior-level Full TimeIN-TN-Chennai20h ago
-
Dot Net FSD AWS INR 1400K-1800K.Net Core | API Gateway | AWS CDK | AWS CodePipeline | AWS Well ArchitectedMid-level Full TimeIN-KA-Bangalore20h ago
-
Python Pyspark -Support Role INR 1500K-2000KApache Airflow | BMC | Informatica | PySpark | PythonHybrid work model | Night shift workEntry-level Full TimeIN-TN-Chennai R20h ago
-
Software Engineer INR 850K-1500KAPI Development | Artificial Intelligence | Automation | Azure | DebuggingMid-level Full TimeIN-KA-Bangalore20h ago
-
Software Engineer - Data engineer INR 2800K-4000KAWS | Azure | Data Governance | Data Lake | Data ModelingAccess to cutting-edge technologies | Collaborative work environment | Professional growth | Training and developmentSenior-level Full TimeBangalore - Karnataka, India - Kalyani …20h ago
-
(#1556) Sr. Dotnet Engineer INR 1500K-3500KAPIs | AWS | Audit Logging | Authentication | AuthorizationSenior-level Full TimeAndhra Pradesh, Visakhapatnam, India20h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India21h ago
-
APIs | AWS | Access Control | Apache Spark | AzureCollaborative engineering culture | Flexible working setup | Hands-on innovation | Learning and technical growth | Remote-friendly work environmentSenior-level Full TimeIndia21h ago
-
Associate Software Engineer INR 1500K-1910KAlerting | Angular | Azure | Azure Kubernetes | Azure Kubernetes ServiceEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India21h ago
-
Mid-level Full TimeGurugram, India21h ago
-
AI Engineer – Retail/CPG Domain INR 1500K-2500KAWS | Azure | Cloud platform | Data Governance | Data ModelingMid-level Full TimeGurugram, Haryana, India21h ago
-
Gen AI Engineer INR 300K-390KAPI Development | Amazon Web Services | Azure | Cloud platform | EmbeddingsEntry-level Full TimeIndia21h ago
-
Software Engineer III - Python, Spark, AWS INR 1800K-3000KAWS EMR | Amazon CloudWatch | Amazon Redshift | Amazon Web Services | Apache HadoopSenior-level Full TimeBengaluru, Karnataka, India22h ago
-
Data Science – (Gen AI Developer) - Analyst INR 2000K-2600KAutogen | CI/CD | Data Analysis | Data Visualization | ETLEntry-level Full TimeBengaluru, Karnataka, India22h ago
-
Software Test Engineer- Must have- Python, Pyspark, SQL, Good to have-playwright or selenium, databricks or snowflake INR 1632K-2156KAPI Testing | Agile | Azure | Bitbucket | CI/CDEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimePune, MH, India22h ago
-
DTICI_Engineering_Data_Analysis_Engineer_T9 INR 2000K-2600KApache Spark | Classification | Data Analysis | Data Wrangling | Deep learningSenior-level Full TimeBengaluru, Karnataka, India22h ago