Real-Time Data Engineer
Tasks
- Architect Kafka clusters
- Build CI/CD pipelines
- Build Kafka Connect pipelines
- Configure Kafka topic and ACL changes
- Deploy Kafka clusters
- Design HA DR for Kafka
- Design partitioning replication topic strategy
- Implement Cluster Linking
- Implement GitOps patterns
- Implement Kafka security ACLs
- Implement Kafka security RBAC
- Implement Kafka security SASL
- Implement Kafka security mTLS
- Implement MirrorMaker 2
- Implement observability monitoring
- Integrate Datadog monitoring
- Integrate Kafka with corporate IdPs
- Lead incident response
- Maintain technical documentation
- Mentor engineers through code review
- Onboard application teams to Kafka
- Operate Kafka Connect
- Operate Kafka Streams
- Operate Kafka clusters
- Operate Schema Registry
- Operate ksqlDB
- Optimize Kafka capacity
- Optimize Kafka storage tiering
- Run post incident reviews
- Set up Grafana dashboards
- Set up Prometheus metrics
- Set up multi region active active
- Use Confluent Control Center
Perks/Benefits
- N/A
Skills/Tech-stack
ACLs | AWS MSK | Ansible | Apache Kafka | Azure Event | Azure Event Hubs | Bash | CI/CD | Confluent Control Center | Confluent Operator | Confluent Platform | Control center | Data Governance | Data Lineage | Datadog | Event Hubs | Flink | GitOps | Go | Grafana | KSQL | KSQLDB | Kafka Connect | Kafka Streams | Kubernetes | MTLS | MirrorMaker 2 | Prometheus | Python | RBAC | SASL | Schema Registry | Spark Streaming | Strimzi | Terraform
Education
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R11d ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R8h ago
-
Senior Data Engineer - Finance USD 172K-203KARR | AWS | Apache Iceberg | Apache Spark | CRMDental insurance | Employee discounts | Flexible wellness time off | Health insurance | Hybrid workSenior-level Full TimeHybrid - San Francisco, California R13h ago
-
Sr. Data Engineer (HYBRID) USD 145K-160KAWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon DynamoDBComplimentary club membership | Hybrid work schedule | Personal training | Pilates | Shop discountsSenior-level Full TimeNew York, NY, United States R13h ago
-
Staff Software Engineer, Data Products USD 171K-224KAWS | AWS Lambda | Amazon Athena | Amazon Redshift | Amazon S3401k retirement savings plan | Flexible time off | Generous parental leave | Health, dental, and vision insurance | Lifestyle spending account mental health support solutionsSenior-level Full TimeRemote, USA R14h ago
-
Senior Data Engineer - Agentic AI Engineering USD 138K-173KAWS | Access Control | Airflow | Azure | Batch dataRemote-friendlySenior-level Full TimeUnited States of America R15h ago
-
Senior Machine Learning Engineer, Safety USD 216K-303KData Preprocessing | Deep learning | Distributed Training | Embeddings | Hugging FaceSenior-level Full TimeRemote - United States R15h ago
-
Algorithms | Concurrency | Data Structures | ESQL | ElasticsearchFlexible schedule | Generous vacation days | Health coverage | Parental leave | Volunteer timeSenior-level Full TimeUnited States R16h ago
-
Principal Data Engineer USD 129K-180KAdobe Analytics | Adobe XD | Amazon Web Services | Bitbucket | CI/CDRelocation Support N/A | Remote work permittedSenior-level Full TimeUnited States R1d ago
-
AI/ML Software Engineer USD 95K-140KAWS | Agent systems | Agentic Workflows | Autogen | AzureDisability insurance | Life insurance | Paid parental leave | Paid time off | Retirement plan matchSenior-level Full TimeEvendale, United States R1d ago
-
Senior Legal AI Engineer USD 100K-185KAI Agents | Citation Evaluation | Data Protection | Enterprise Security | Ethical AIEmployee assistance program | Fitness reimbursement | Flexible work schedule | Headspace app access | Mental health daysSenior-level Full TimeUnited States of America, Eagan, Minnesota R1d ago
-
Conversational AI Engineer USD 100K-150KAgent systems | Agentic Workflows | Chunking | Cost Optimization | EmbeddingsSenior-level Full TimeUnited States - Remote R1d ago
-
Edge AI Engineer USD 100K-150KC++ | Core ML | DSP | Embedded Systems | Energy optimizationCareer growth | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Applied AI Engineer USD 100K-150KAblation Studies | Computer Vision | Data Quality | Data quality monitoring | Deep learningSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
AI Interaction Designer USD 100K-150KAgent systems | Agentic Workflows | Chunking | Deterministic systems | EmbeddingsCareer growth potential | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Agentic Systems | Computer Vision | Data ValidationSenior-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Agentic Systems | Computer Vision | Data QualitySenior-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HiveCareer growth | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Big Data Application Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HudiRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Senior Data Engineer – Hadoop USD 100K-150KAWS EMR | Apache Airflow | Apache Atlas | Apache Flink | Apache HudiRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Senior Data Engineer USD 100K-150KAWS | Airflow | Apache Kafka | Azure | Azure DevOps100 percent remote work | Career growth | Direct W2 employmentSenior-level Full TimeUnited States - Remote R1d ago