Lead Spark Data Engineer
Tasks
- Build Query Validator
- Design scalable data solutions on Spark and Flink
- Develop Spark Adapter for metrics translation
- Enforce data governance, security, and quality practices
- Ensure parity between Spark and Flink implementations
- Implement IoT Query Language grammar
- Implement relationships logic using Graph/Ontology
- Maintain documentation
- Manage Azure and Databricks resources
- Monitor and optimize workloads
- Participate in agile activities
- Transform and prepare data with SQL, Python, Java
Perks/Benefits
- N/A
Skills/Tech-stack
ANTLR | Agile methodologies | Apache Flink | Apache Spark | Artifact management | Azure DevOps | Azure ecosystem | CI/CD | Data Governance | Data Modeling | Data Quality | Data Security | Data Warehousing | Databricks | Delta Lake | GitHub | Graph/Ontology | Java | Python | SDLC | SQL
Education
Roles
Regions
Countries
States
Related jobs
-
AWS RDS | AWS Security | Amazon Web Services | Apache Spark | AutomationEquipment and office stipend | Flexible PTO | Laptop and tools | Learning and development stipend | Paid exams and certificationsSenior-level Full TimeARGENTINA R22h ago
-
Airflow | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCareer development | Company-provided equipment | English lessons | Flexible working options | Learning opportunitiesSenior-level Full TimeBogotá, Bogota, Colombia R1d ago
-
Amazon Redshift | Analytics engineering | Churn analysis | DBT | Data ArchitectureChildcare support | Fully remote work | Healthcare coverage | Home office equipment allowance | Learning and development budgetSenior-level Full TimeColombia R1d ago
-
Amazon Redshift | Analytics engineering | Business Intelligence | DBT | Data ModelingFully remote | Healthcare coverage | Home office and equipment allowance | Learning and development budget | Paid parental leaveSenior-level Full TimeArgentina R1d ago
-
Atlassian | Confluence | Data Mapping | Data Migration | GitFloating holidays | Good working environment | Remote work | Vacation daysSenior-level Full TimeColombia - Remote R1d ago
-
Amazon Kinesis | Amazon Web Services | Apache Airflow | Apache Kafka | Apache SparkAnnual team trip | Birthday off | English lessons | Extra vacation week | Monthly benefits creditsSenior-level Full TimeArgentina R2d ago
-
Amazon Web Services | Apache Airflow | Apache Spark | Data Lakes | Data PipelinesEvery other Friday off | Flexible work schedule | Health bonus | Local holidays | Paid sick daysEntry-level InternshipColombia R3d ago
-
AWS Bedrock | AWS Glue | Airflow | Amazon Kinesis | Amazon OpenSearch100% remote work | Annual stipend for Learning and Development | Equipment and office stipend | Flexible PTO | Generous holidaysSenior-level Full TimeARGENTINA R3d ago
-
Platform Database Engineer COP 48000K-60000KAWS | AWS Lambda | Access Control | Amazon CloudWatch | Amazon EC2On-call rotation | Remote workMid-level Full TimeColombia - Remote R7d ago
-
Embeddings | GPT-4 | Human-in-the-loop | Langchain | LanggraphFloating holidays | Remote work | Vacation daysSenior-level Full TimeColombia - Remote R8d ago
-
Data Engineer – Azure Cloud & Security COP 54000K-74400KApplication Security | Application Security Group | Azure Data | Azure Data Factory | Azure DevOpsComprehensive benefits | Flexible work model | Hybrid work option | Inclusive culture | Leadership visibilityMid-level Full TimeColombia; Argentina R13d ago
-
Entry-level Full TimeBogota, Colombia R14d ago
-
Senior-level Full TimeBogota, Colombia (Remote Friendly) R15d ago
-
API | Azure Data | Azure Data Factory | Azure DevOps | Azure FunctionsMid-level Full TimeBogota, CO R16d ago
-
Sr. Data Engineer (Snowflake/dbt) USD 152K-204KAccess Control | Clustering | Compute Optimization | DBT | Data GovernanceFully remoteSenior-level Full TimeRemote (Mexico); Remote (Uruguay); Remote (Chile); … R18d ago
-
AWS CDK | AWS CloudFormation | AWS Glue | AWS Lambda | AWS Step FunctionsPaid time off | Remote work | Work autonomySenior-level Full TimeBogota R21d ago
-
Airflow | Apache Hadoop | Apache Hive | Apache Spark | Data ModelingPaid time off | Remote work | Work autonomySenior-level Full TimeBogota R22d ago
-
APIs | Azure | Azure Functions | Azure Redis | Azure Redis CacheRemote workSenior-level Full TimeRemote but local to Bogotá, Colombia R22d ago
-
AWS Glue | AWS QuickSight | Agile | Amazon Athena | Amazon S3Paid time off | Remote work | Work autonomy | Work-life balanceSenior-level Full TimeBogota R23d ago
-
AI workflows | API Integration | Azure | Cloud Platforms | Cloud Platforms (AWSCollaboration with experienced engineers | Exposure to modern AI practices | Work on production AI systemsSenior-level Full TimeBogota, Colombia (Remote Friendly) R28d ago
-
AI Solutions Engineer USD 59KAI | API Design | Cloud infrastructure | Data Engineering | DevOpsCommunity engagement | Egoless collaboration | Impactful work | Mutual trust | Professional growthSenior-level Full TimeRemote (Colombia) R29d ago
-
AWS Lambda | AWS SageMaker | CI/CD | CNN | Deep learningAWS certification sponsorship | Learning Support | Medical insurance | Paid sick leave | Paid vacationSenior-level Full TimeMedellín, Antioquia R30d ago
-
AWS | DBT | DMS | Data Lake | Data ModelingFlexible work hours | Remote work supportSenior-level Full TimeColombia, Remote R1mo ago
-
AI Voice Technologies | AI voice | Cloud services | Deep learning | DockerContinuing education | Flexible hours | Health coverage | Paid time off | Remote workSenior-level Full TimeMedellin, Antioquia, Colombia R1mo ago
-
AWS | Azure | Deep learning | Docker | Google CloudFlexible hours | Health coverage | Paid time off | Remote work | Technology stipendSenior-level Full TimePalmira, Valle del Cauca, Colombia R1mo ago