Senior Data Engineer
Tasks
- Automate data quality checks
- Automate deployments with CI/CD
- Build and maintain streaming data architecture
- Build real-time stream processing applications
- Build streaming ETL pipelines
- Configure Kinesis Analytics applications
- Configure and manage Apache Kafka clusters
- Create and optimize streaming data topologies
- Create real time aggregations and windowing
- Design Amazon Kinesis Data Firehose solutions
- Design Amazon Kinesis data streams
- Design fault tolerant streaming systems
- Develop real-time data pipelines
- Implement Kafka Connect pipelines
- Implement Kafka security with SSL TLS SASL and ACLs
- Implement complex event processing patterns
- Implement data lineage and monitoring
- Implement data serialization and schema evolution
- Implement disaster recovery and high availability
- Implement event-driven architectures
- Implement infrastructure-as-code
- Integrate streaming systems with data lakes and warehouses
- Manage Kafka Schema Registry
- Monitor streaming system health and performance
- Optimize Kinesis shard management
- Provide technical leadership and mentorship
Perks/Benefits
- 401k match
- Life insurance
- Long-term disability
- Medical, dental, and vision insurance
- Paid Holidays
- Paid time off
- Short-term disability
Skills/Tech-stack
ACL | AWS Lambda | Access Control | Agile | Amazon Kinesis | Amazon MSK | Apache Flink | Apache Kafka | Apache Spark | Apache Spark Streaming | Avro | CI/CD | CloudFormation | CloudWatch | Confluent Schema Registry | Data Firehose | Data Lineage | Docker | Encryption | GitHub | Grafana | Infrastructure as Code | JSON | Java | Kafka Connect | Kafka Schema Registry | Kafka Streams | Kinesis Analytics | Kinesis Data Firehose | Kubernetes | NoSQL | Prometheus | Protobuf | Protocol Buffers | Python | SASL | SQL | SSL/TLS | Scala | Schema Registry | Spark Streaming | Terraform | “as-code”
Education
Roles
Related jobs
-
Featured Feat. Software Engineer USD 40K-150K.NET | APIs | AWS | Angular | AzureContractor hours flexibility | Remote workMid-level ContractRemote R3d ago
-
Featured Feat. Senior Software Engineer USD 80K-170KAlgorithms | C++ | Data Structures | Go | JavaContractor flexibility | Remote workSenior-level ContractRemote R3d ago
-
Data Scientist Senior (H/F) EUR 65K-65KAPI | AWS | Agile | Bash | CI/CDBest Workplaces for Women | Cooptation bonus | Great Place to Work certification | Inclusive hiring process training for managers | Internal eventsSenior-level Full TimeNeuilly-sur-Seine, IDF, France R2h ago
-
HPC Test Data Analysis Technology Development Engineer JPY 3600K-5268KAdaptive testing | Anomaly Detection | Data Analysis | Defect prediction | DockerFlexible working environment | Hybrid work | Inclusive workplace | Remote work 2 days per weekEntry-level Full TimeKodaira, Japan R3h ago
-
Autonomy | C++ | CPU GPU | CPU GPU Debugging | Critical Systems401k | Health insurance | Paid Company Holidays | Paid time off | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R8h ago
-
Autonomy | C++ | Data Ingestion | Data Ingestion Pipelines | Deployment401k | Health insurance | Paid Holidays | Paid time off | Phone stipendMid-level Full TimeSan Carlos - Hybrid R9h ago
-
Agent Orchestration | Airflow | Argo Workflows | Artifact versioning | Autonomous workflowsRemote work flexibilitySenior-level Full TimeRemote - United States R9h ago
-
Senior Databricks Engineer USD 180K-247KAWS | Autoscaling | Azure | CI/CD | CachingVisa sponsorshipSenior-level Full TimeCanada R9h ago
-
Senior-level Full TimeCanada R9h ago
-
Sr. AI Engineer (Applied AI & ML Systems) USD 132K-165KAgentic AI | Context engineering | Continuous Improvement | Data Engineering | Data PipelinesE learning license | Hackathons | Healthcare benefits | Home office setup allowance | Identity theft protectionSenior-level Full TimeUnited States R10h ago
-
Senior Data Engineer USD 160K-180KAPI | AWS | Airflow | Change Data Capture | DBT401k | Equipment provided | Equity compensation | Flexible PTO | Home office stipendSenior-level Full TimeRemote R11h ago
-
AI & Data Engineering Apprentice USD 44K-63KAWS | Agile | Azure | Cloud Computing | Data EngineeringCareer coaching | Continuous learning culture | Flexible work arrangements | MentorshipEntry-level ApprenticeshipRemote R13h ago
-
A/B | A/B Testing | AWS | Airflow | Amazon Redshift401k matching | Employee assistance program | Flexible time off | Flexible work arrangement | Paid HolidaysMid-level Full TimeRemote, US R13h ago
-
Associate AI Engineer USD 144K-180K.NET | APIs | ASPNet | AWS | Azure401k matching | Dental insurance | Hybrid work model | Medical insurance | Paid time offMid-level Full TimeIrving, TX R15h ago
-
Data Engineer-Secret Clearance Required USD 100K-127KAWS | AWS Glue | AWS Redshift | Azure | Azure Data401k match | Bereavement leave | Disability insurance | Employee assistance program | Employee discount programSenior-level Full TimeRemote - Nationwide, United States R16h ago
-
Data Engineer - Costa Rica USD 100K-120KAWS | Azure | Cloud Computing | Data Lake | Data WarehouseDental insurance | Disability insurance | Employee assistance program | Employee discount program | Health insuranceMid-level Full TimeCosta Rica R16h ago
-
Senior Lead AI/ML and Full Stack Engineer - R01567102 USD 150K-237KAccess Control | Agent systems | Azure | CMDB | Context ManagementHybrid work environmentSenior-level Full TimeMiami, Florida, US R16h ago
-
Senior Healthcare Data Engineer USD 104K-199KConditional Aggregation | Data Modeling | Data Quality | Data Reconciliation | Data Validation401k matching | Employee assistance program | Family building benefits | Flexible spending accounts | HolidaysSenior-level Full TimeSeattle, Washington, United States R16h ago
-
Software / Machine Learning Engineer GBP 40K-50KAWS | Computer Vision | Deep learning | Generative AI | Hugging FaceFlexible work hours | Gym discounts | Learning resources | Mental health support | MentorshipMid-level Full TimeLondon R18h ago
-
Software / Machine Learning Engineer EUR 42K-57KAWS | Computer Vision | Data analytics | Deep learning | Generative AIGym discounts | Hybrid work | Learning resources | Mental health support | MentorshipMid-level Full TimeMadrid R18h ago
-
Software / Machine Learning Engineer EUR 42K-57KComputer Vision | Data analytics | Deep learning | Generative AI | Hugging FaceFlexible work schedule | Gym discounts | Learning resources | Mental health support | MentorshipMid-level Full TimeMilan R18h ago
-
Machine Learning Engineer, Data Mining USD 144K-192KActive Learning | Batch inference | CI/CD | Data Augmentation | Data Curation401k match | Dental insurance | Health savings account | Life insurance | Medical insuranceSenior-level Full TimePittsburgh, Pennsylvania, United States; Remote U.S. R18h ago
-
Machine Learning Engineer, Data Mining USD 144K-192KActive Learning | Batch inference | CI/CD | Data Augmentation | Data Drift401k match | Dental insurance | Health insurance | Health savings account | Life insuranceSenior-level Full TimeBoston, Massachusetts, United States; Remote U.S. R18h ago
-
Analista MLOps / Ingenier@ MLOps EUR 35K-40KAPIs | AWS | Airflow | Azure | Azure FunctionsCareer development | Continuous training | Flexible schedule | Remote work | Social benefitsMid-level Full TimeMadrid, Spain R19h ago
-
Data Engineer USD 115K-162KAWS | Apache Spark | Azure | Cloud platform | Data VisualizationRemote workMid-level Full TimeRemote R19h ago