Lead Spark Data Engineer
Tasks
- Architect data solutions on Spark and Flink
- Build Query Validator
- Develop Spark Adapter
- Enforce data governance and security practices
- Ensure logic parity between Spark and Flink
- Implement IoT Query Language grammar
- Implement relationships logic
- Maintain documentation
- Manage Azure and Databricks resources
- Monitor and optimize workloads
- Participate in agile activities
- Transform and prepare data with SQL, Python, Java
Perks/Benefits
- N/A
Skills/Tech-stack
ANTLR v4 | Abstract syntax trees | Apache Spark | Apache Spark internals | Artifact management | Azure AD | Azure DevOps | CI/CD | Compliance | Custom DSLs | Data Design | Data Modeling | Data Warehousing | Databricks | Delta Lake | Encryption | GitHub | Graph/Ontology | Graph/Ontology traversal | Java | Parsers | Python | SQL | Spark internals
Education
N/A
Roles
Related jobs
-
Mid-level Full TimeGuadalajara Mexico R19h ago
-
Software Engineer MXN 342K-420KAWS | Alerting | Cause analysis | Data Modeling | Database Performance OptimizationMid-level Full TimeGuadalajara Mexico R19h ago
-
Mid-level Full TimeGuadalajara Mexico R19h ago
-
Mid-level Full TimeGuadalajara Mexico R19h ago
-
Senior Data Platform Engineer (Python/AWS) - Finance MXN 1040K-1410KAWS CDK | AWS CloudFormation | AWS Glue | AWS Lambda | AWS Step FunctionsPaid time off | Remote work | Work autonomySenior-level Full TimeMexico City R19h ago
-
Senior Software Engineer MXN 1040K-1300KAWS | Alerting | Cause analysis | Data Ingestion | Data ModelingSenior-level Full TimeGuadalajara, Mexico R19h ago
-
Senior Software Engineer MXN 1040K-1300KAWS | Alerting | Data Modeling | Distributed Systems | JavaSenior-level Full TimeGuadalajara, Mexico R19h ago
-
Airflow | Apache Hive | Apache Spark | Data Modeling | ETLPaid time off | Remote work | Work autonomySenior-level Full TimeMexico City R1d ago
-
FBS AIOps Engineer MXN 721K-1001KCopilot Studio | Dynatrace | ITIL Foundation | Python | ServiceNowFlexible work arrangements | Inclusive work culture | Paid time off | Private health insurance | Training and developmentSenior-level Full TimeMexico - Remote R2d ago
-
Data Engineer / Integrations Specialist - Contract MXN 1040K-1300KAPI Integration | Async Programming | Authentication | Data Quality | Data ReconciliationCollaborative team culture | Cross-functional impact | Fast-paced environment | Growth opportunities | Remote workSenior-level Full TimeMexico, Remote R2d ago
-
Mid-level Full TimeRemote, Mexico R2d ago
-
Senior-level Full TimeRemote, Mexico R2d ago
-
Senior Data Engineer - Technology MXN 1040K-1300KAWS Glue | AWS S3 | Agile | Amazon Athena | Amazon QuickSightPaid time off | Remote work | Work autonomy | Work with top American companiesSenior-level Full TimeMexico City R2d ago
-
FBS Sr Power BI Engineer (Database Integration Experience) MXN 686K-1001KBI Desktop | DAX | Data Modeling | Data Reconciliation | Data integrationCompetitive salary | Flexible work arrangements | Health insurance | Paid time off | Performance-based bonusesSenior-level Full TimeMexico - Remote R3d ago
-
Bash | C# | C++ | Docker | GitFlexible schedule | Fully remote | Project-based collaborationSenior-level Full TimeMexico - Remote R3d ago
-
la26.006 - Data Engineering Lead MXN 660K-720KApache Spark | Azure | Data Modeling | Data Warehousing | DatabricksCareer development opportunities | Collaborative work environment | Continuous learning | Flexible working arrangementsSenior-level Full TimeMexico City, Mexico City, Mexico - … R3d ago
-
Data Analysis | Data Ingestion | Data cleaning | Deployment Workflows | Exploratory Data AnalysisFlexible schedule | Part-time freelance | Remote workMid-level FreelanceMexico - Remote R3d ago
-
Sr Data Scientist /Machine Learning Engineer MXN 1040K-1300KAzure | Data Pipelines | Data Visualization | Experimental Design | Feature EngineeringRemote workSenior-level Full TimeRemote, Guadalajara, Mexico R3d ago
-
AI Engineer I (Remote) USD 30K-39KCI/CD | Cloud AWS | Distributed Systems | Docker | Full StackCareer growth opportunities | Creative tech environment | Remote workEntry-level Full TimeMexico, Remote R4d ago
-
Computational Algorithms | Machine Learning | Numerical Methods | Problem Solving | PythonFlexible schedule | Professional development opportunities | Remote workMid-level FreelanceMexico - Remote R4d ago
-
Computational Algorithms | NumPy | Pandas | Problem Solving | PythonFlexible schedule | Gain AI project experience | Remote workMid-level FreelanceMexico - Remote R4d ago
-
Apache Spark | Cloud Computing | Data Modeling | ETL | GitCareer growth opportunities | Professional development | Remote work | Team collaborationMid-level Full TimeMonterrey, Nuevo León, México R7d ago
-
Infrastructure Software Engineer, Telemetry MXN 720K-900KC# | C++ | Databases | Debugging | Distributed SystemsEntry-level Full TimeRemote - Mexico R7d ago
-
Machine Learning Engineer - Remote MXN 750K-1062KAPIs | Azure | CI/CD | CPU | CloudFlexible work arrangements | Global team collaboration | Innovative work environment | Opportunity for learning and developmentSenior-level Full TimeGuadalajara, MX R8d ago
-
Machine Learning/AI Engineer (LATAM Remote) MXN 1040K-1300KBackend Development | Cloud Platforms | Computer Vision | Data Processing | Database optimizationCareer growth opportunities | Collaborative environment | Innovative projects | Remote workSenior-level Full TimeRemote, Guadalajara, Mexico R9d ago