IN_Manager_Data Engineer_Application Technology_Advisory_Kolkata
Tasks
- Automate Databricks workspace deployment
- Build ETL/ELT pipelines
- Configure CI/CD pipelines
- Create unit tests and enforce code quality
- Design bronze silver gold architecture
- Develop reusable Python libraries
- Implement data quality checks and validation
- Implement deployment promotion workflows
- Implement network security with private link and VNet injection
- Implement schema evolution and partitioning
- Manage Unity Catalog lineage and ACLs
- Manage secrets and key vault integrations
- Optimize Spark transformations
- Package Python code into wheels
- Set up audit logging and run tracking
- Tune performance for large datasets
Perks/Benefits
Skills/Tech-stack
Asset bundles | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure Event | Azure Event Hubs | Azure Synapse | Azure Synapse Analytics | Bicep | CI/CD | Cosmos DB | Data Factory | Data Lake Storage Gen2 | Databricks | Databricks (asset) | Databricks Asset Bundles | Databricks CLI | Databricks Delta | Databricks Delta Live Tables | Databricks Repos | Delta Lake | Delta Live Tables | Event Hubs | Git | GitHub Actions | Infrastructure as Code | Kafka | PySpark | Python | Synapse Analytics | Terraform | Unity Catalog | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology | Master of Science
Roles
Related jobs
-
Senior-level Full TimeIN-TN-Chennai4h ago
-
AWS Data Engineer INR 1500K-2040KAWS CDK | AWS Step Functions | Amazon Glue | Amazon RDS | Amazon RedshiftSenior-level Full TimeIN-TN-Chennai4h ago
-
AWS Data Engineer INR 1500K-2040KAWS CDK | AWS Step Functions | Amazon Glue | Amazon RDS | Amazon RedshiftSenior-level Full TimeIN-TN-Chennai4h ago
-
Senior-level Full TimeIN-AP-Hyderabad4h ago
-
Technical Delivery Manager INR 1317K-2280KAgile | Data Management | Data Warehousing | Data masking | Data subsettingMid-level Full TimeIN-KA-Bangalore4h ago
-
Assistant Vice President - AI Runtime Security INR 1500K-6000KAI Governance | AI RMF | Access Control | Amazon CloudWatch | Anomaly DetectionExecutive-level Full TimeIndia5h ago
-
Lead ML Engineer INR 2000K-3800KAgile | Autogen | Automl | Azure | CI/CDEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeVadodara, TN, India6h ago
-
Forward Deployed Engineer, GenAI, Google Cloud INR 2500K-5000KAlgorithms | Apache Beam | Apache Hadoop | Apache Spark | C++Embedded customer development | Technical feedback to product roadmap | White glove deploymentSenior-level Full TimeBengaluru, Karnataka, India; Gurugram, Haryana, India6h ago
-
Software Engineer III, AI/ML, Google Cloud INR 2500K-4725KAudio generation | Data Processing | Generative AI | Image Generation | Language ModelsSenior-level Full TimeHyderabad, Telangana, India6h ago
-
Forward Deployed Engineer, GenAI, Google Cloud INR 1800K-2500KAPI Integration | Agent systems | CrewAI | Data Pipelines | Deep learningMid-level Full TimeBengaluru, Karnataka, India; Mumbai, Maharashtra, India6h ago
-
Software Engineer III, Google Cloud Bigtable INR 1500K-3000KAgile | Algorithms | Automated testing | C++ | CI/CDSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
Senior Software Engineer, Applied AI, Vertex AI Search INR 2500K-4500KC++ | Data Structures | Data Structures and Algorithms | Debugging | Distributed SystemsSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
Mid-level Full TimeHyderabad, TS, IN; Bengaluru, KA, IN7h ago
-
Junior GenAI Developer INR 600K-900KAWS | Amazon Bedrock | Anthropic API | Chroma | EmbeddingsCertification programs | Health insurance | Language courses | Mentorship | Professional developmentEntry-level Full TimeAll Cities, India R7h ago
-
Junior GenAI Developer INR 600K-900KAWS | Amazon Bedrock | Amazon OpenSearch | Chroma | EmbeddingsCertification programs | Health insurance | Language courses | Mentorship | Professional developmentEntry-level Full TimeAll Cities, India R7h ago
-
Junior GenAI Developer INR 600K-900KAWS | Agentic Workflows | Chroma | Embeddings | FaissCompany events | Health insurance | Internship opportunities | Language courses | MentorshipEntry-level Full TimeAll Cities, India R7h ago
-
Senior-level Full Timehosur road bangalore, India7h ago
-
Senior-level Full Timehosur road bangalore, India7h ago
-
AI Engineer - Acon - BLR INR 2500K-4500KAI Search | AI Studio | Agentic Workflows | Application Security | Arize PhoenixSenior-level Full TimeIndia7h ago
-
AI Engineer INR 1500K-2400KAgentic AI | Agile | Apache Spark | Artificial Intelligence | AutomationAccidental insurance | Life insurance | Meal arrangements | Medical insurance | Professional development opportunitiesMid-level Full TimeGurugram, HR, IN8h ago
-
Data Engineer with AI developer INR 2000K-2800KAPIs | AWS SageMaker | Azure Data | Azure Data Factory | Azure Machine LearningSenior-level Full Timebangalore, India9h ago
-
Member of Technical Staff INR 2500K-4500KAutomated testing | Batching | CI/CD | Caching | Cloud infrastructureSenior-level Full TimeBengaluru, KA, IN9h ago
-
Gen Ai Lead - Architect INR 3334K-4800KAgent systems | BigQuery | Cloud Functions | Cloud Pub/Sub | Cloud RunSenior-level Full TimeIndia9h ago
-
Senior - Data and App Modernization INR 2500K-3500KCI/CD | Data Cleansing | Data Denormalization | Data Modeling | Data NormalizationSenior-level Full TimeGurgaon, Haryana, India9h ago
-
Data Engineer INR 1500K-2000KAWS | Ab Initio | Apache Flink | Apache Spark | Apache Spark StreamingMid-level Full TimeBengaluru, Karnataka, India10h ago