Databricks Platform Engineer
Tasks
- Automate deployments with GitHub Actions
- Build dbt transformation pipelines
- Configure cross region replication and failover
- Deploy and optimize Databricks compute for data science workloads
- Design Databricks workspaces and clusters on AWS
- Enable compliance monitoring
- Ensure data lineage and secure data sharing
- Establish CI CD workflows for releases
- Implement cluster policies and autoscaling
- Implement data governance with Unity Catalog
- Implement high availability and disaster recovery
- Maintain MLflow integrations for model lifecycle tracking and deployment
- Manage Fivetran connectors
- Monitor cost and usage
- Monitor pipeline health and data freshness
- Orchestrate data ingestion and transformation across Databricks S3 and Snowflake
- Provision infrastructure with Terraform
- Set up job orchestration for notebooks and models
Perks/Benefits
- N/A
Skills/Tech-stack
AWS ECS | AWS EKS | AWS IAM | Amazon CloudWatch | Amazon S3 | Amazon VPC | Amazon Web Services | Autoscaling | CI/CD | Cross Region | Cross-region replication | DBT | Databricks | Datadog | Disaster Recovery | Disaster Recovery Planning | Fivetran | GitHub Actions | High Availability | MLflow | Monitoring | PySpark | Python | Recovery Planning | Terraform | Unity Catalog | Web Services
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Regions
Countries
States
Related jobs
-
AWS PrivateLink | Cortex AI | DBT | Data masking | DatabricksMid-level Full TimeColombia22h ago
-
AI Agents | Cloud Computing | Data Visualization | Data Wrangling | Decision TreesCareer development | Hybrid work schedule | MentorshipMid-level Full TimeBogota, COLOMBIA, Colombia1d ago
-
Senior-level Full TimeLatin America#LATAM, Remote, Colombia (Hybrid) R1d ago
-
AWS | Amazon Redshift | Azure | Databricks | MongoDBAdditional days off | Birthday day off | Flexible vacation | Great Place to Work ranking | Internet connectivity creditSenior-level Full TimeArgentina R1d ago
-
AWS RDS | AWS Security | Amazon Web Services | Apache Spark | AutomationEquipment and office stipend | Flexible PTO | Laptop and tools | Learning and development stipend | Paid exams and certificationsSenior-level Full TimeARGENTINA R1d ago
-
Airflow | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCareer development | Company-provided equipment | English lessons | Flexible working options | Learning opportunitiesSenior-level Full TimeBogotá, Bogota, Colombia R2d ago
-
Amazon Web Services | Apache Spark | Application Profiling | Application debugging | C#Education budget | Fitness budget | Flexible schedule | Mentorship | Office work optionSenior-level Full TimeCartagena, Colombia2d ago
-
Amazon Web Services | Apache Spark | Application Profiling | C# | C++Education budget | Fitness budget | Flextime | Mentorship | Office optionsSenior-level Full TimePereira, Colombia2d ago
-
Mid-level Full TimeMedellín, Medellín, Antioquia, Colombia, Antioquia, Colombia2d ago
-
Amazon Redshift | Analytics engineering | Churn analysis | DBT | Data ArchitectureChildcare support | Fully remote work | Healthcare coverage | Home office equipment allowance | Learning and development budgetSenior-level Full TimeColombia R2d ago
-
Amazon Redshift | Analytics engineering | Business Intelligence | DBT | Data ModelingFully remote | Healthcare coverage | Home office and equipment allowance | Learning and development budget | Paid parental leaveSenior-level Full TimeArgentina R2d ago
-
API Development | AWS SageMaker | Azure Machine Learning | CI/CD | Computer VisionContinuous learning opportunities | International team collaboration | Remote work optionsSenior-level Full TimeMedellin, Colombia2d ago
-
Senior-level Full TimeMedellin, Colombia2d ago
-
HUB - ETL / Data Engineer COP 54000K-74400KAWS | Apache Spark | Azure | Cloud platform | Data integrationEntry-level Full TimeBogotá D.C., Bogotá D.C., CO, 1101113d ago
-
Senior-level Full TimeBogotá, CO, 1102313d ago
-
Airflow | Algorithm Optimization | ETL | Feature Engineering | Feature SelectionSenior-level Full TimeCOL-Bogotá, Colombia3d ago
-
APIs | Azure | Azure AI | Cloud Computing | Data PipelinesClient project exposure | Personal developmentSenior-level Full TimeBogotá D.C., Bogotá D.C., CO, 1101113d ago
-
API Gateway | API Integration | AWS Lambda | AWS X-Ray | Agile methodologyLife insurance | Maternity leave | Meal voucher | Mobile services subsidy | Paid vacationsSenior-level Full TimeColombia4d ago
-
Azure DevOps | CI/CD | Data Engineering | Data Pipelines | Data ScienceSenior-level Contract Full TimeArgentina4d ago
-
Amazon Web Services | Apache Airflow | Apache Spark | Data Lakes | Data PipelinesEvery other Friday off | Flexible work schedule | Health bonus | Local holidays | Paid sick daysEntry-level InternshipColombia R4d ago
-
Data Engineer (FinOps) ID56348 COP 54000K-74400KAWS | Apache Airflow | CD pipelines | CI/CD | CI/CD pipelinesEducation budget | Fitness budget | Flextime | Mentorship | Office optionsMid-level Full TimeCali, Colombia4d ago
-
Data Engineer (FinOps) ID56348 COP 54000K-74400KAWS | Apache Airflow | CI/CD | DBT | DagsterEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsMid-level Full TimeCartagena, Colombia4d ago
-
Data Engineer (FinOps) ID56348 COP 54000K-74400KAWS | Airflow | CI/CD | DBT | DagsterEducation budget | Fitness budget | Flextime | Mentorship | Office workMid-level Full TimeManizales, Colombia4d ago
-
Data Engineer (FinOps) ID56348 COP 54000K-74400KAWS | Apache Airflow | CI/CD | DBT | DagsterEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsMid-level Full TimeVillavicencio, Colombia4d ago
-
Data Engineer (FinOps) ID56348 COP 54000K-74400KAWS | Apache Airflow | CI/CD | Cloud platform | DBTEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsMid-level Full TimeUsaquen, Colombia4d ago