Data engineer with data bricks
Tasks
- Build and manage data lake and Delta Lake
- Design and build data pipelines
- Develop end-to-end machine learning pipelines
- Enable ETL for extraction transformation and loading
- Implement CI/CD pipelines
- Ingest and transform large complex datasets
- Integrate and orchestrate data from multiple sources
- Manage Databricks clusters and cloud infrastructure
- Optimize and tune spark jobs
- Provide technical leadership
- Support data governance and stewardship
- Support testing and deployment of data services
- Track and deploy models with MLflow
Perks/Benefits
- N/A
Skills/Tech-stack
API | AWS | Apache Spark | Azure | Bash | CD | CI | CI/CD | Cron | DB2 | Databricks | Delta Lake | DevOps | Distributed Systems | DynamoDB | ELT | ETL | GCP | JSON | Java | Kanban | MLflow | NoSQL | Python | R | SAFe | SQL | SSIS | SSRS | Scrum | T-SQL | XML
Education
N/A
Roles
Related jobs
-
Software Engineer III, Spanner SQL, Data Types, Google Cloud INR 2000K-3000KAPI Design | C++ | Database Internals | Database Performance Optimization | Database performanceSenior-level Full TimeBengaluru, Karnataka, India2h ago
-
AI/ML Infrastructure Software Engineer INR 2500K-4800KC++ | Hyperparameter Optimization | JAX | Keras | ML ExplainabilitySenior-level Full TimeBengaluru, Karnataka, India2h ago
-
AI and Data Engineer, Google Cloud INR 1500K-2500KAccess Management | Apache Hive | Apache Pig | Apache Spark | Data ComplianceMid-level Full TimeBengaluru, Karnataka, India2h ago
-
Staff Software Engineer, Data Indexing, Web Apps INR 2000K-3200KArtificial Intelligence | C++ | Data Pipelines | Data Storage | Distributed SystemsSenior-level Full TimeBengaluru, Karnataka, India2h ago
-
Ai/Ml Testing-Ai, Ml, Deep Learning, Data Mining, Analytics INR 1000K-2100KAPI Testing | Accuracy testing | Data Validation | ETL | EmbeddingsMid-level Full TimeChennai, Tamil Nadu, India3h ago
-
AES - DE -JAVA - Generative AI Application Developers INR 2500K-4000KAWS | Agile | Azure | CI/CD | Data EngineeringSenior-level Full TimeBangalore, Karnataka, India3h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India3h ago
-
Mid-level Full TimeIndia4h ago
-
Lead Assistant Manager INR 2125K-3500KAI Foundry | Access Control | Active Directory | Application Insights | Azure Active DirectorySenior-level Full TimeGurugram, Haryana, India4h ago
-
QA Engineer - GenAI, VoiceAI, Telephony, and IVR testing INR 1500K-2500KAPI Testing | ASR | Automation frameworks | CCaaS | Call RoutingMid-level Full TimeNoida, Uttar Pradesh, India6h ago
-
Senior Analyst, Business Analytics INR 2520K-3380KApache Airflow | Apache Spark | Cloud platform | Data Analysis | Data CleansingRelocation assistanceSenior-level Full TimeMumbai, MH, IN6h ago
-
Sr Data Security Platform Engineer INR 1500K-3500KAPI | Azure | Azure Functions | Azure Logic | Azure Logic AppsSenior-level Full TimeIndia7h ago
-
Senior Specialist_Python/AI Engineer INR 2475K-4500KAWS | AWS CodeBuild | AWS CodePipeline | AWS Lambda | AWS X-RaySenior-level Full TimeBangalore, Karnataka, IN, 5601007h ago
-
Staff Software Engineer (L6) - Partner Ecosystem INR 3000K-4000KGo | Java | Open Source | Python | SQLSenior-level Full TimeBengaluru, India8h ago
-
Sr. Machine Learning Engineer INR 2500K-3800KAWS | Amazon Bedrock | Classification | Data Exploration | DatabricksERG events | Equipment provided | Health coverage | Holidays | Learning and developmentSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India9h ago
-
Software Engineer III - Data Engineer INR 2000K-3900KAWS Lambda | AWS Secrets | AWS Secrets Manager | Agile Scrum | AirflowSenior-level Full TimeMumbai, Maharashtra, India9h ago
-
Mid-level Full TimePune, IN14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Machine Learning Engineer 4 INR 2000K-3500KAWS | Alerting | Artificial Intelligence | Automation | AutoscalingSenior-level Full TimeNoida, India R14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Senior Data Migration Engineer INR 2800K-4000KApache Airflow | Bulk Insert | CI/CD | Cx Oracle | DBTSenior-level Full TimeBangalore, KA, IN14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago