Senior Big Data Engineer - SAP Concur Complete
São Leopoldo, BR, 93022-718
BRL 304K-396K (estimate) Senior-level Full Time
Tasks
- Build data pipeline infrastructure
- Build infrastructure as code
- Build lakehouse and data warehouse architectures
- Collaborate on canonical data models
- Design data pipelines
- Develop ETL/ELT workflows
- Ensure data quality and observability
- Implement CI CD for data code
- Implement real-time streaming pipelines
- Maintain Delta Lake and Apache Iceberg tables
- Optimize pipeline performance and costs
- Process batch and streaming data
- Set up data validation and lineage
Perks/Benefits
Skills/Tech-stack
ACID | AWS CDK | AWS EMR | AWS Glue | AWS Kinesis | AWS Lambda | AWS RDS | AWS S3 | Amazon Redshift | Apache Airflow | Apache Atlas | Apache Flink | Apache Iceberg | Apache Kafka | Apache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Synapse | Azure Synapse Analytics | Cloud Storage | CloudWatch | DBT | Data Lake Storage | Data Modeling | Databricks | Datadog | Dataframes | Delta Lake | Docker | DynamoDB | ELT | ETL | GitHub Actions | Google BigQuery | Google Cloud | Google Cloud Storage | Google Dataproc | Google Pub/Sub | Grafana | Great Expectations | HDFS | Hive | Idempotency | Kafka Streams | Kubernetes | Lakehouse | Monte Carlo | NoSQL | OpenLineage | OpenTelemetry | Pub/Sub | PySpark | Python | SQL | Schema evolution | Snowflake | Soda | Spark SQL | Structured Streaming | Synapse Analytics | Terraform | YARN
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
AWS | Airflow | Amazon Redshift | Azure | BigQueryFlexible working hours | Remote workSenior-level Full TimeBrazil - Remote R15h ago
-
Data Engineer BRL 18K-18KAWS | Apache Spark | Azure | CI/CD | DatabricksDental insurance | Employee assistance | Flexible schedule | Gympass | Health insuranceSenior-level Full TimeBrasil18h ago
-
Staff Machine Learning Operations Engineer - Brazil BRL 230K-265KAWS | Alerting | Apache Airflow | Azure | CI/CDFlexible work hours | Home office setup support | Remote environment | Self-managed PTO | Work in your own time zoneSenior-level Full TimeBrazil R23h ago
-
Apache Airflow | Apache Spark | Data Lakes | Data Warehouses | ELTAccess to internal learning platform | Discounts on educational programs | Meal or food allowance | Medical and dental assistance | Mentoring and professional development programsSenior-level Full TimeBrazil1d ago
-
Engenheiro de Dados SR BRL 27K-40KBigQuery | CI/CD | Cloud Data | Cloud Data Pipelines | Cloud platformCareer development | Collaborative work environment | Employee well-being support | Training and workshops | Work-life balance supportSenior-level Full TimeSão Paulo, State of São Paulo, …1d ago
-
Airflow | Apache Spark | BigQuery | CI/CD | Cloud ComposerContinuous learning | Flexible schedule | Knowledge sharing | Professional development | Remote workMid-level Full TimeBrazil R1d ago
-
Apache Flink | Apache Kafka | Apache Spark | CI/CD | Data ModelingBirthday day off | Childcare support | Dental insurance | Educational partnerships | Extended parental leaveSenior-level Full TimeBrazil1d ago
-
Agile | Apache Hadoop | Apache Hive | Apache Spark | Azure BlobHybrid workSenior-level Full TimeAlphaville - Barueri, BR, 06.454-000 R1d ago
-
AWS | AWS Glue | AWS Lambda | AWS Step Functions | Amazon AthenaChildcare assistance | Dental insurance | Gym access | Health insurance | Language learning supportMid-level Full TimeBrazil1d ago
-
Apache Airflow | CI/CD | Cloud platform | DBT | Data LakehouseDental insurance | Employee discounts | Extended parental leave | Gym and wellness partnerships | Health insuranceSenior-level Full TimeBrazil1d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeSão Paulo, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeRio de Janeiro, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAirflow | Amazon Redshift | Apache Hive | Apache Spark | DBTFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Remote optionsSenior-level Full TimeRecife, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeCampinas, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAgile | Amazon Redshift | Apache Airflow | Apache Hive | Apache KafkaFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeBrasília, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAirflow | Amazon Redshift | Apache Spark | DBT | Data ArchitectureFlexible schedule | Mentorship | Personalized growth roadmaps | Professional growth | Remote and office optionsSenior-level Full TimeCuritiba, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeFlorianópolis, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeSalvador, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkEducation budget | Fitness budget | Flextime | Mentorship | Office work optionSenior-level Full TimeBelo Horizonte, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAirflow | Amazon Redshift | Apache Spark | DBT | Data ArchitectureFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeSão Bernardo do Campo, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkEducation budget | Fitness budget | Flextime | Mentorship | Office optionsSenior-level Full TimePorto Alegre, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAgile | Amazon Redshift | Apache Airflow | Apache Hive | Apache KafkaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeSorocaba, Brazil3d ago
-
AI Engineer – Agent Platform - VAGA EXCLUSIVA PARA PCD BRL 154K-198KAPI Integration | AWS | Automation | Chatbots | Cloud PlatformsFlexible medical benefits | Life insurance | Retirement options | Volunteer programsMid-level Full TimeSAO PAULO, SP, Brazil3d ago
-
CI/CD | Data Engineering | Data Governance | Data Modeling | Data ObservabilityBirthday day off | Childcare assistance | Culture leisure partnerships | Dental plan | Employee discountsSenior-level Full TimeBrazil4d ago
-
DevOps Architect BRL 240K-275KAmazon Linux | Ansible | Azure | Bash | CI/CDExchange program | Health insurance | Life insurance | OPD | Paid time offSenior-level Full TimeBrazil - Remote R4d ago