Senior Infrastructure Kafka Engineer
Tasks
- Administer Kafka clusters
- Automate infrastructure provisioning with Terraform
- Build real time data pipelines using Kafka producers
- Build real time data pipelines using Spark Streaming
- Build streaming consumers using Kafka Streams
- Configure Kafka brokers and topics
- Deploy Kafka in production and disaster recovery
- Design scalable Kafka infrastructure
- Ensure compliance with security policies
- Implement Disaster Recovery
- Implement monitoring, logging, and alerting
- Integrate systems using Kafka Connect
- Lead incident response and escalation
- Maintain documentation and best practices
- Maintain runbooks and playbooks
- Monitor and troubleshoot Kafka incidents
- Partner to design secure event driven architectures
- Perform partitioning and performance tuning
- Run health checks and capacity planning
Perks/Benefits
Skills/Tech-stack
AWS | Alerting | Apache Kafka | Bash | Confluent Kafka | Containerization | Datadog | Disaster Recovery | Docker | Event Driven | Event-driven architecture | GCP | Git | Grafana | Kafka Connect | Kafka Producers | Kafka Streams | Kubernetes | Linux | Logging | Microsoft Azure | MongoDB | Monitoring | MySQL | Networking | NoSQL | Observability | Oracle | Perl | PostgreSQL | Powershell | Python | RabbitMQ | SQL | SQL Server | SVN | Spark Streaming | Splunk | Terraform | Windows
Education
Related jobs
-
Finance Analytics Engineer USD 76K-95KAI | Acceptance Testing | Dashboarding | Data Modeling | Data Pipelines401k matching | Bereavement leave | Employee assistance program | Employee discount program | Health and dental insuranceSenior-level Full TimeRemote - Nationwide, United States R2h ago
-
Infrastructure Engineer - Storage USD 100K-120KAnsible | Azure | Azure Blob | Azure Blob Storage | Azure Files401k plan | Bereavement | Disability insurance | Employee assistance program | Employee discount programMid-level Full TimeSt. Louis, MO, United States2h ago
-
Senior-level Full TimeHerndon, VA4h ago
-
Senior Platform AI Engineer USD 119K-180KAPI Design | Asynchronous programming | Authentication | Concurrency | Distributed SystemsSenior-level Full TimeCenter, Center District, IL5h ago
-
Senior-level Full TimeCenter, Center District, IL5h ago
-
Senior-level ContractJersey City, United States6h ago
-
AWS Lambda | Amazon DynamoDB | Amazon Kinesis | Amazon SNS | Amazon SQSHybrid workSenior-level ContractSeattle, United States6h ago
-
Database Engineer USD 107K-158KAgile | Amazon EMR | Amazon Web Services | Data Modeling | Database performance401k retirement plan with employer matching | Company paid memberships | Flexible spending account | Flexible work schedule | Health, dental, vision, and life insuranceMid-level Full TimeHerndon, Virginia6h ago
-
Lead Software Engineer - Java/Python - Learn AI / LLM USD 175K-215KAgile | Amazon Web Services | Application Resiliency | Artificial Intelligence | CI/CDBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeNew York, NY, United States7h ago
-
Quant Analytics [Multiple Positions Available] USD 150K-185KAWS Redshift | CTE | Data Aggregation | Data Enrichment | Data TransformationBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site wellness centersSenior-level Full TimePlano, TX, United States7h ago
-
Benchmarking | CUDA | Communication optimization | Data parallelism | Deep learningMid-level Full TimeSeattle, Washington, United States7h ago
-
Machine Learning Engineer USD 130K-194KAI machine learning | AWS AI | AWS AI Machine Learning | Amazon DynamoDB | Amazon EC2Professional development | Work from homeMid-level Full TimeRemote, NY, US R7h ago
-
Software Engineer III - Data, AWS, ETL, Java/Python, USD 173K-185KAPIs | AWS | Agile methodologies | Apache Airflow | Apache FlinkBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimePlano, TX, United States7h ago
-
Algorithms Engineer USD 72K-120KARIMA | Anomaly Detection | Causal Inference | Causal forests | Change point detectionEntry-level Full TimeCenter, Center District, IL7h ago
-
Data parallelism | Deep learning | Distributed Training | GPU Acceleration | Model BenchmarkingMid-level Full TimeSan Jose, California, United States8h ago
-
Partner Engineer, Generative AI USD 173K-247KAWS | Agent Orchestration | Azure | Bias Mitigation | C plus plusSenior-level Full TimeMenlo Park, CA8h ago
-
AI Research Scientist, SysML - FAIR USD 143K-208KArtificial Intelligence | C# | C++ | Co-design | Compiler designMid-level Full TimeMenlo Park, CA | Boston, MA …8h ago
-
Data Engineer, Analytics (Technical Leadership) USD 175K-242KDashboards | Data Architecture | Data Governance | Data Marts | Data ModelingSenior-level Full TimeMenlo Park, CA | New York, …8h ago
-
AI Research Engineer, FAIR Chemistry USD 141K-208KApplied Mathematics | Artificial Intelligence | Computational statistics | Data Science | Density Functional TheorySenior-level Full TimeSan Francisco, CA8h ago
-
IP Validation Engineer - Machine Learning Accelerators USD 142K-203KAHB | APB | AXI | Android | C#Cross-functional collaboration | On device AI work | Prototype and silicon developmentMid-level Full TimeSunnyvale, CA | Burlingame, CA8h ago
-
Mid-level Full TimeMenlo Park, CA8h ago
-
Research Engineer - Perception and Machine Learning USD 177K-251KC++ | Computer Vision | Data Pipelines | Knowledge Distillation | Language ModelsSenior-level Full TimeRedmond, WA | Menlo Park, CA …8h ago
-
Research Engineer - Computer Vision and Robotics USD 141K-208K3D Reconstruction | C plus plus | Computational imaging | Computer Vision | Data AnalysisMid-level Full TimeRedmond, WA8h ago
-
Data Engineer, PAR USD 173K-242KAgent Orchestration | C# | C++ | Data Architecture | Data GovernanceCareer growth | Mentorship | Skill developmentSenior-level Full TimeMenlo Park, CA8h ago
-
Data Scientist, Analytics (Technical Leadership) USD 160K-190KAI Workflow Optimization | AI workflow | Agent Orchestration | Bias Mitigation | Causal InferenceCareer development | World class analytics communitySenior-level Full TimeRemote, US | Bellevue, WA | … R8h ago