Senior Data Engineer
Tasks
- Automate data quality checks
- Automate deployments with CI/CD
- Build and maintain streaming data architecture
- Build real-time stream processing applications
- Build streaming ETL pipelines
- Configure Kinesis Analytics applications
- Configure and manage Apache Kafka clusters
- Create and optimize streaming data topologies
- Create real time aggregations and windowing
- Design Amazon Kinesis Data Firehose solutions
- Design Amazon Kinesis data streams
- Design fault tolerant streaming systems
- Develop real-time data pipelines
- Implement Kafka Connect pipelines
- Implement Kafka security with SSL TLS SASL and ACLs
- Implement complex event processing patterns
- Implement data lineage and monitoring
- Implement data serialization and schema evolution
- Implement disaster recovery and high availability
- Implement event-driven architectures
- Implement infrastructure-as-code
- Integrate streaming systems with data lakes and warehouses
- Manage Kafka Schema Registry
- Monitor streaming system health and performance
- Optimize Kinesis shard management
- Provide technical leadership and mentorship
Perks/Benefits
- 401k match
- Life insurance
- Long-term disability
- Medical, dental, and vision insurance
- Paid Holidays
- Paid time off
- Short-term disability
Skills/Tech-stack
ACL | AWS Lambda | Access Control | Agile | Amazon Kinesis | Amazon MSK | Apache Flink | Apache Kafka | Apache Spark | Apache Spark Streaming | Avro | CI/CD | CloudFormation | CloudWatch | Confluent Schema Registry | Data Firehose | Data Lineage | Docker | Encryption | GitHub | Grafana | Infrastructure as Code | JSON | Java | Kafka Connect | Kafka Schema Registry | Kafka Streams | Kinesis Analytics | Kinesis Data Firehose | Kubernetes | NoSQL | Prometheus | Protobuf | Protocol Buffers | Python | SASL | SQL | SSL/TLS | Scala | Schema Registry | Spark Streaming | Terraform | “as-code”
Education
Roles
Related jobs
-
Featured Feat. Software Engineer USD 40K-150K.NET | APIs | AWS | Angular | AzureContractor hours flexibility | Remote workMid-level ContractRemote R3d ago
-
Featured Feat. Senior Software Engineer USD 80K-170KAlgorithms | C++ | Data Structures | Go | JavaContractor flexibility | Remote workSenior-level ContractRemote R3d ago
-
Agent Orchestration | Airflow | Argo Workflows | Artifact versioning | Autonomous workflowsRemote work flexibilitySenior-level Full TimeRemote - United States R7h ago
-
Senior Databricks Engineer USD 180K-247KAWS | Autoscaling | Azure | CI/CD | CachingVisa sponsorshipSenior-level Full TimeCanada R7h ago
-
Senior-level Full TimeCanada R8h ago
-
Senior Data Engineer USD 160K-180KAPI | AWS | Airflow | Change Data Capture | DBT401k | Equipment provided | Equity compensation | Flexible PTO | Home office stipendSenior-level Full TimeRemote R10h ago
-
AI & Data Engineering Apprentice USD 44K-63KAWS | Agile | Azure | Cloud Computing | Data EngineeringCareer coaching | Continuous learning culture | Flexible work arrangements | MentorshipEntry-level ApprenticeshipRemote R11h ago
-
A/B | A/B Testing | AWS | Airflow | Amazon Redshift401k matching | Employee assistance program | Flexible time off | Flexible work arrangement | Paid HolidaysMid-level Full TimeRemote, US R12h ago
-
Associate AI Engineer USD 144K-180K.NET | APIs | ASPNet | AWS | Azure401k matching | Dental insurance | Hybrid work model | Medical insurance | Paid time offMid-level Full TimeIrving, TX R13h ago
-
Data Engineer-Secret Clearance Required USD 100K-127KAWS | AWS Glue | AWS Redshift | Azure | Azure Data401k match | Bereavement leave | Disability insurance | Employee assistance program | Employee discount programSenior-level Full TimeRemote - Nationwide, United States R14h ago
-
Data Engineer - Costa Rica USD 100K-120KAWS | Azure | Cloud Computing | Data Lake | Data WarehouseDental insurance | Disability insurance | Employee assistance program | Employee discount program | Health insuranceMid-level Full TimeCosta Rica R14h ago
-
Machine Learning Engineer, Data Mining USD 144K-192KActive Learning | Batch inference | CI/CD | Data Augmentation | Data Curation401k match | Dental insurance | Health savings account | Life insurance | Medical insuranceSenior-level Full TimePittsburgh, Pennsylvania, United States; Remote U.S. R16h ago
-
Machine Learning Engineer, Data Mining USD 144K-192KActive Learning | Batch inference | CI/CD | Data Augmentation | Data Drift401k match | Dental insurance | Health insurance | Health savings account | Life insuranceSenior-level Full TimeBoston, Massachusetts, United States; Remote U.S. R16h ago
-
Analista MLOps / Ingenier@ MLOps EUR 35K-40KAPIs | AWS | Airflow | Azure | Azure FunctionsCareer development | Continuous training | Flexible schedule | Remote work | Social benefitsMid-level Full TimeMadrid, Spain R17h ago
-
Data Engineer USD 115K-162KAWS | Apache Spark | Azure | Cloud platform | Data VisualizationRemote workMid-level Full TimeRemote R17h ago
-
AI Developer USD 140K-170KAPI Development | Algorithms | Amazon Web Services | Azure | CI/CDAdvanced English communication support | Remote positionMid-level Full TimeRemote R17h ago
-
Agile | Artificial Intelligence | Data Architecture | Data Engineering | Data GovernanceHybrid work environment | Mentorship | Training and development | Work from home daysEntry-level Full TimeMidrand, GP, South Africa R17h ago
-
Bash | Data Pipelines | Distributed Systems | Docker | GCPAccess to cutting-edge technologies | Autonomy | Distributed first work environment | Fully remoteMid-level Full TimeSaudi Arabia R18h ago
-
Bash | Cloud platform | Data Ingestion | Data Pipelines | Distributed SystemsAccess to cutting-edge AI technologies | Autonomy | Bonus opportunities | Collaborative engineering culture | Distributed first work environmentSenior-level Full TimeBelgium R18h ago
-
Bash | Data Pipelines | Distributed Systems | Docker | GCPAccess to cutting-edge technologies | Autonomy | Bonus opportunities | Collaborative culture | Distributed first work environmentSenior-level Full TimeUnited Arab Emirates R18h ago
-
Bash | Cloud platform | Data Pipelines | Distributed Systems | DockerAutonomy | Bonus opportunities | Collaborative culture | Distributed first work environment | Equity opportunitiesSenior-level Full TimeTurkey R18h ago
-
Bash | Cloud platform | Data Pipelines | Distributed Systems | DockerAutonomy | Bonus opportunities | Collaborative culture | Distributed first work environment | Equity opportunitiesSenior-level Full TimeAustralia R18h ago
-
Bash | Cloud platform | Cost Optimization | Data Pipelines | Distributed SystemsAccess to cutting-edge technology | Autonomy | Collaborative culture | Distributed first | Fully remoteSenior-level Full TimeSouth Africa R18h ago
-
Bash | Data pipeline | Distributed Systems | Docker | GCPAutonomy | Collaborative culture | Distributed first work environment | Equity opportunities | Fully remoteMid-level Full TimeMexico R18h ago
-
Bash | Data Pipelines | Distributed Systems | Docker | GCPAccess to cutting edge AI and data engineering technologies | Collaborative engineering culture | Distributed first work environment | Fully remote | High ownership autonomyMid-level Full TimeRomania R18h ago