Senior Data Engineer
Tasks
- Architect batch and real time data pipelines
- Build ingestion frameworks for data quality lineage schema evolution
- Create technical specifications and reusable components
- Design highly available fault tolerant AWS architectures
- Develop data modeling frameworks for analytics
- Enforce data engineering best practices with CI CD testing orchestration logging observability
- Implement security automation for HIPAA and SOC 2 compliance
- Mentor junior engineers through technical reviews and training
- Optimize performance and compute costs for Databricks and Snowflake
- Produce technical documentation and runbooks
Perks/Benefits
Skills/Tech-stack
AWS Lambda | AWS S3 | AWS SQS | Airflow | Auditing | CI/CD | Caching | Cluster configuration | DBT | Data Lineage | Data Modeling | Data Quality | Data integration | Databricks | Databricks Workflows | Delta Lake | Distributed Computing | ELT | ETL | Encryption | Event Driven | Event-driven architecture | HIPAA | Indexing | Infrastructure as Code | Kafka | Logging | Observability | Orchestration | PHI handling | Partitioning | Performance Profiling | PySpark | Python | RBAC | SOC 2 | SQL | Schema evolution | Snowflake | Snowflake Schema | SparkSQL | Star Schema | Streaming | Terraform | Tokenization | “as-code”
Education
Roles
Related jobs
-
Solutions Architect INR 2400K-3500KADLS | AWS | Airflow | Azure | CassandraBootcamp training | Broadband reimbursement | Continuous learning opportunities | Medical insurance | Paid certificationsSenior-level Full TimeIndia - Remote R3h ago
-
Sr. SW Engineer, Machine Learning INR 2475K-3465KA/B | A/B Testing | Agent systems | B testing | CI/CDFinancial wellness support | Hybrid work | Mental health support | Time offSenior-level Full TimeBengaluru, India3h ago
-
A/B | A/B Testing | AutoGPT | B testing | Dimensionality ReductionExecutive-level Full TimeMumbai Goregaon, India11h ago
-
Machine Learning Engineer IV INR 4443K-7998KAPIs | Amazon Web Services | Automated testing | Cloud Native | Cloud platformBonuses | Disability insurance | Employee resource groups | Inclusive culture | Life insuranceSenior-level Full TimeRemote, India R14h ago
-
Senior-level Full TimePune City, India18h ago
-
Mid-level Full TimePune, Maharashtra, India18h ago
-
AM - Cloud Data Engineer INR 1500K-2500KAWS N/A | Apache Airflow | Apache Kafka | Apache Spark | Azure DataClient-facing experience | Consulting experience | Growth mindset cultureSenior-level Full TimeBangalore, Karnataka, India18h ago
-
AI Engineer – Reliability & Warranty Analytics INR 1800K-3000KBacktesting | BigQuery | CI/CD | Calibration | Cloud RunMid-level Full TimeChennai, Tamil Nadu, India19h ago
-
Associate III Business Intelligence-SUPPORT SERVICES-Data & Analytics - In House Engineering INR 800K-1800KBig Data | Big data concepts | C++ | Data Visualization | Data conceptsMid-level Full TimeBangalore, Karnataka, India20h ago
-
AI Engineer – Retail/CPG Domain INR 1800K-2500KAWS | Azure | Data Governance | Data Modeling | Data SecurityMid-level Full TimeGurugram, Haryana, India20h ago
-
Consultant - Python Developer with Gen AI and AI Foundry INR 2000K-4000KAWS | Autogen | Azure | Azure OpenAI | Azure OpenAI ServiceSenior-level Full TimeBangalore, Karnataka, India21h ago
-
Customer Implementation Engineer ( Python | AI/ML) INR 2500K-4500KAWS | Azure | CI/CD | Cause analysis | Cloud platformFlexible-hybrid work | Health insurance | Paid parental leave | Paid time offMid-level Full TimeBengaluru21h ago
-
Associate Principal Engineer (India Office) INR 1500K-2000KADLS | Azure Data | Azure Data Factory | Azure DevOps | CDCMid-level Full TimeBangalore, Karnataka, India22h ago
-
Senior-level Full TimePune, Maharashtra, India22h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India22h ago
-
Applied AI ML Lead INR 2500K-4800KAI Agents | AWS Lambda | AWS S3 | AWS Step Functions | Agentic AISenior-level Full TimeHyderabad, Telangana, India22h ago
-
Software Engineer (Backend Developer) INR 1500K-3500KAWS | Active Directory | Agile | Algorithms | Data StructuresMid-level Full TimeBangalore, India Office (BANGALORE) R22h ago
-
Agentic AI | Agile | DAX | Data Governance | Data ModelingSenior-level Full TimeBengaluru, Karnataka, India22h ago
-
Lead Software Engineer - Java/Python, LLM, AWS INR 3000K-5000KA/B | A/B Testing | AWS | Agile | Amazon AuroraSenior-level Full TimeBengaluru, Karnataka, India23h ago
-
Platform Engineering Professional - Data Engineer INR 2200K-3500KAWS | Amazon Kinesis | Apache NiFi | Apache Spark | CI/CDMid-level Full TimeBengaluru, IN, 56010323h ago
-
AWS Data Engineer INR 1500K-6000KAWS | Automation | Cloud Architecture | Data Governance | Data IngestionExecutive-level Full TimeHyderabad, Telangana, India23h ago
-
Mid-level Full TimePune, Maharashtra, India23h ago
-
Senior-level Full TimeNoida, Uttar Pradesh, India1d ago
-
Consultant - AI Engineer INR 2000K-4500KAutogen | CI/CD | Cloud infrastructure | CrewAI | Data PipelinesSenior-level Full TimeBangalore, Karnataka, India1d ago
-
Generative AI Engineer INR 2000K-4500KAmazon Web Services | Autogen | CI/CD | Cloud platform | CrewAISenior-level Full TimeHyderabad, Telangana, India1d ago