Senior Data Engineer
Tasks
- Architect batch and real time data pipelines
- Build ingestion frameworks for data quality lineage schema evolution
- Create technical specifications and reusable components
- Design highly available fault tolerant AWS architectures
- Develop data modeling frameworks for analytics
- Enforce data engineering best practices with CI CD testing orchestration logging observability
- Implement security automation for HIPAA and SOC 2 compliance
- Mentor junior engineers through technical reviews and training
- Optimize performance and compute costs for Databricks and Snowflake
- Produce technical documentation and runbooks
Perks/Benefits
Skills/Tech-stack
AWS Lambda | AWS S3 | AWS SQS | Airflow | Auditing | CI/CD | Caching | Cluster configuration | DBT | Data Lineage | Data Modeling | Data Quality | Data integration | Databricks | Databricks Workflows | Delta Lake | Distributed Computing | ELT | ETL | Encryption | Event Driven | Event-driven architecture | HIPAA | Indexing | Infrastructure as Code | Kafka | Logging | Observability | Orchestration | PHI handling | Partitioning | Performance Profiling | PySpark | Python | RBAC | SOC 2 | SQL | Schema evolution | Snowflake | Snowflake Schema | SparkSQL | Star Schema | Streaming | Terraform | Tokenization | “as-code”
Education
Roles
Related jobs
-
Senior AI Engineer INR 3000K-5000KAPI deployment | Adversarial Testing | Auto Retraining | Azure ML | CI/CDSenior-level Full TimeBangalore3h ago
-
Azure Data | Azure Data Factory | CI/CD | Data Factory | Data ReconciliationSenior-level Full TimeBangalore, India3h ago
-
Senior Talend Developer / Lead Data Engineer INR 1500K-2700KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsSenior-level Full Timebengaluru, India4h ago
-
Solutions Architect INR 2400K-3500KADLS | AWS | Airflow | Azure | CassandraBootcamp training | Broadband reimbursement | Continuous learning opportunities | Medical insurance | Paid certificationsSenior-level Full TimeIndia - Remote R5h ago
-
Sr. SW Engineer, Machine Learning INR 2475K-3465KA/B | A/B Testing | Agent systems | B testing | CI/CDFinancial wellness support | Hybrid work | Mental health support | Time offSenior-level Full TimeBengaluru, India5h ago
-
Sr GIS Data Engineer -Python l AI ML l SQL INR 1800K-2500KArcGIS | Batch Processing | Data Pipelines | Data Quality | FMESenior-level Full TimeMumbai, MH, India5h ago
-
Mid-level Full TimeIT Centre Bengaluru GDTC, India9h ago
-
Senior Data Engineer INR 1500K-2500KAccess Control | Azure | Azure Data | Azure Data Factory | Azure SQLSenior-level Full TimeHyderabad, IN, 500 0089h ago
-
A/B | A/B Testing | AutoGPT | B testing | Dimensionality ReductionExecutive-level Full TimeMumbai Goregaon, India13h ago
-
Machine Learning Engineer IV INR 4443K-7998KAPIs | Amazon Web Services | Automated testing | Cloud Native | Cloud platformBonuses | Disability insurance | Employee resource groups | Inclusive culture | Life insuranceSenior-level Full TimeRemote, India R15h ago
-
Mid-level Full TimeGurugram19h ago
-
Senior-level Full TimePune City, India19h ago
-
Senior AI Engineer INR 3000K-4500KAPI Development | Anthropic Claude | Bias Mitigation | BigQuery | Cloud ComputingContinued development opportunities | Discretionary bonus scheme | Enhanced maternity leave | Enhanced paternity leave | Flexible workingSenior-level Full TimeKarnataka20h ago
-
Mid-level Full TimePune, Maharashtra, India20h ago
-
AM - Cloud Data Engineer INR 1500K-2500KAWS N/A | Apache Airflow | Apache Kafka | Apache Spark | Azure DataClient-facing experience | Consulting experience | Growth mindset cultureSenior-level Full TimeBangalore, Karnataka, India20h ago
-
AI Engineer – Reliability & Warranty Analytics INR 1800K-3000KBacktesting | BigQuery | CI/CD | Calibration | Cloud RunMid-level Full TimeChennai, Tamil Nadu, India20h ago
-
Associate III Business Intelligence-SUPPORT SERVICES-Data & Analytics - In House Engineering INR 800K-1800KBig Data | Big data concepts | C++ | Data Visualization | Data conceptsMid-level Full TimeBangalore, Karnataka, India21h ago
-
AI Engineer – Retail/CPG Domain INR 1800K-2500KAWS | Azure | Data Governance | Data Modeling | Data SecurityMid-level Full TimeGurugram, Haryana, India22h ago
-
Senior-level Full TimeIndia, Bengaluru (Hybrid) R22h ago
-
Consultant - Python Developer with Gen AI and AI Foundry INR 2000K-4000KAWS | Autogen | Azure | Azure OpenAI | Azure OpenAI ServiceSenior-level Full TimeBangalore, Karnataka, India23h ago
-
Customer Implementation Engineer ( Python | AI/ML) INR 2500K-4500KAWS | Azure | CI/CD | Cause analysis | Cloud platformFlexible-hybrid work | Health insurance | Paid parental leave | Paid time offMid-level Full TimeBengaluru23h ago
-
Associate Principal Engineer (India Office) INR 1500K-2000KADLS | Azure Data | Azure Data Factory | Azure DevOps | CDCMid-level Full TimeBangalore, Karnataka, India23h ago
-
Senior-level Full TimePune, Maharashtra, India23h ago
-
Data Engineer (Snowflake | DBT) INR 2475K-3465KAirflow | CI/CD | DBT | Data Quality | Data Testing401k match | Catered lunches | Collaborative office environment | Flexible vacation | HSA contributionsSenior-level Full TimeIndia | Pune23h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India23h ago