Databricks Platform Engineer
Tasks
- Automate deployments with GitHub Actions
- Build dbt transformation pipelines
- Configure cross region replication and failover
- Deploy and optimize Databricks compute for data science workloads
- Design Databricks workspaces and clusters on AWS
- Enable compliance monitoring
- Ensure data lineage and secure data sharing
- Establish CI CD workflows for releases
- Implement cluster policies and autoscaling
- Implement data governance with Unity Catalog
- Implement high availability and disaster recovery
- Maintain MLflow integrations for model lifecycle tracking and deployment
- Manage Fivetran connectors
- Monitor cost and usage
- Monitor pipeline health and data freshness
- Orchestrate data ingestion and transformation across Databricks S3 and Snowflake
- Provision infrastructure with Terraform
- Set up job orchestration for notebooks and models
Perks/Benefits
- N/A
Skills/Tech-stack
AWS ECS | AWS EKS | AWS IAM | Amazon CloudWatch | Amazon S3 | Amazon VPC | Amazon Web Services | Autoscaling | CI/CD | Cross Region | Cross-region replication | DBT | Databricks | Datadog | Disaster Recovery | Disaster Recovery Planning | Fivetran | GitHub Actions | High Availability | MLflow | Monitoring | PySpark | Python | Recovery Planning | Terraform | Unity Catalog | Web Services
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Regions
Countries
States
Related jobs
-
Data Engineer ID56348 COP 54000K-74400KAWS | AWS Glue | Airflow | Amazon Athena | Amazon AuroraFlexible schedule | Mentorship | Office option | Personalized growth roadmaps | Remote work optionMid-level Full TimeBucaramanga, Colombia10h ago
-
Data Engineer ID56348 COP 54000K-74400KAWS Glue | Airflow | Amazon Athena | Amazon Aurora | CI/CDEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsMid-level Full TimeCali, Colombia10h ago
-
Data Engineer ID56348 COP 54000K-74400KAWS Glue | Airflow | Amazon Athena | Amazon Aurora | CD pipelinesEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsMid-level Full TimeVillavicencio, Colombia10h ago
-
Data Engineer ID56348 COP 54000K-74400KAWS Glue | Amazon Athena | Amazon Aurora | Apache Airflow | CI/CDEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsMid-level Full TimeBarranquilla, Colombia10h ago
-
Data Engineer ID56348 COP 54000K-74400KAWS Glue | Amazon Athena | Amazon Aurora | Apache Airflow | CI/CDFlexible schedule | Mentorship | Personalized growth roadmaps | Remote work option | TechtalksMid-level Full TimeManizales, Colombia10h ago
-
Data Engineer ID56348 COP 54000K-74400KAWS Glue | Amazon Athena | Amazon Aurora | Apache Airflow | CI/CDFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Remote optionsMid-level Full TimeCartagena, Colombia10h ago
-
Data Engineer ID56348 COP 54000K-74400KAWS Glue | Airflow | Amazon Athena | Amazon Aurora | CI/CDEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsMid-level Full TimeMedellín, Colombia10h ago
-
Data Engineer ID56348 COP 54000K-74400KAWS Glue | Airflow | Amazon Athena | Amazon Aurora | CI/CDEducation budget | Fitness budget | Flextime | Mentorship | Office work optionsMid-level Full TimeBogota, Colombia10h ago
-
Data Engineer ID56348 COP 54000K-74400KAWS Glue | Amazon Athena | Amazon Aurora | Apache Airflow | CI/CDFlexible schedule | Mentorship | Remote and office options | TechtalksMid-level Full TimePereira, Colombia10h ago
-
Amazon ECS | Amazon EMR | Amazon Redshift | Apache Airflow | Apache SparkDistributed team collaboration | Remote workMid-level ContractArgentina (Remote); Brazil (Remote); Colombia (Remote); … R1d ago
-
Data & Analytics Manager LATAM COP 54000K-72000KAmazon Web Services | Apache Spark | Azure | Cloud Computing | Cloud platformMid-level Full TimeBogotá, Bogotá, Colombia1d ago
-
AWS | AWS Lambda | Amazon SageMaker | CI/CD | CloudFormationAWS certification sponsorship | Continuous learning support | Medical insurance | Paid public holidays | Paid sick leaveSenior-level Full TimeMedellín, Antioquia R1d ago
-
Amazon Web Services | CI/CD | Data Pipelines | Distributed Systems | GitAWS certification sponsorship | Continuous learning support | Fully remote | Medical insurance budget | Paid public holidaysSenior-level Full TimeMedellín, Antioquia R1d ago
-
Data and Analytics Manager - Collections COP 42000K-42000KAzure | Data Lakes | Data Warehousing | Database performance | Database performance tuningCareer development | Inclusive work environmentMid-level Full TimeBogota, DC, CO2d ago
-
Contract: Lead Analytics Engineer - Data Team MXN 1040K-1300KAWS | Airflow | Continuous integration | DBT | DagsterSenior-level Full TimeRemote - Argentina; Brazil; Mexico; Colombia; … R2d ago
-
Mid-Level BI / Analytics Engineer COP 54000K-72000KApache Airflow | Azure Data | Azure Data Factory | Azure Fabric | CI/CDCareer growth opportunities | Comprehensive benefits | Hybrid work model | In-office option | Inclusive cultureMid-level Full TimeArgentina; Colombia2d ago
-
Apache Airflow | Azure Data | Azure Data Factory | CI/CD | DBTComprehensive benefits | Flexible work model | Growth opportunities | Hybrid work model | In-office optionSenior-level Full TimeArgentina; Colombia2d ago
-
Mid-Level Data Engineer COP 54000K-74400KAzure Data | Azure Data Factory | Azure DevOps | Azure Functions | CI/CDComprehensive benefits | Growth opportunities | Hybrid work model | Inclusive culture | Leadership visibilityMid-level Full TimeArgentina; Colombia2d ago
-
Azure Data | Azure Data Factory | Azure DevOps | Azure Functions | CI/CDCompetitive base | Comprehensive benefits | Flexible work model | Growth opportunities | Hybrid remote or in officeSenior-level Full TimeArgentina; Colombia2d ago
-
Apache Airflow | Confidence scoring | DBT | Dagster | Data LineageIndefinite-term labor contract | Life insurance | Occasional overtime | Prepaid medicine | Work from homeMid-level Full TimeNationwide, Colombia2d ago
-
APIs | Airbyte | Airflow | DBT | Data ModelingEquipment reimbursement | Fully remote | Home office setup budget | Language learning sessions | Paid sick daysSenior-level Full TimeColombia R2d ago
-
Senior-level Full TimeColombia3d ago
-
Access Control | Agile | Azure Data | Azure Data Factory | Azure DatabricksHybrid workSenior-level Full TimeColombia3d ago
-
Senior-level Full TimeColombia3d ago
-
Senior-level Full TimeColombia3d ago