AI Data Engineer - Security (Kafka Experience)
Tasks
- Apply data governance and RBAC
- Build AWS data lake zones
- Build ELT models in Snowflake
- Design event streaming platforms
- Develop idempotent DAGs
- Encrypt and lifecycle S3 data
- Implement CDC with Snowpipe
- Implement IAM and Secrets Manager
- Ingest data from GraphQL APIs
- Ingest data from REST APIs
- Land data in S3 as Parquet
- Manage Glue Data Catalog
- Manage Kafka schema and performance
- Normalize and enrich data in Python
- Optimize Kafka consumer groups
- Optimize Snowflake warehouse performance
- Orchestrate pipelines with AirFlow
Perks/Benefits
Skills/Tech-stack
API Key | AWS Glue | AWS IAM | AWS Secrets | AWS Secrets Manager | Airflow | Amazon S3 | Apache Parquet | CDC | CI/CD | Data Governance | DynamicFrames | ELT | Git-based CI/CD | GraphQL | Kafka | OAuth2 | PySpark | Python | RBAC | REST API | Secrets Manager | Snowflake | Snowpipe
Education
N/A
Roles
Related jobs
-
Data Engineer - Costa Rica USD 91K-111KAWS | AWS Lambda | Azure | Data Governance | Data LakeBereavement leave | Dental insurance | Disability insurance | Employee assistance program | Employee discount programMid-level Full TimeCosta Rica R9h ago
-
Director - Microsoft Cloud & AI Solution Architecture (Communications, Media & Technology) USD 225K-240KAKS | Ansible | Application development | Azure DevOps | Azure Firewall401k match | Bereavement leave | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeLos Angeles, CA, United States R9h ago
-
Python and .Net Developer USD 100K-131K.NET | BigQuery | Cloud platform | Django | DockerCareer growth | Flexible work environment | Performance feedback | Remote jobMid-level Full TimeRemote R12h ago
-
Python AI Tech Lead USD 130K-175KAPI Development | AWS | Anthropic | Apache Spark | AzureFully remote | Long-term B2B contract | Multinational Multilingual Environment | Private healthcare | Stable International ProjectsSenior-level Full TimeRemote R12h ago
-
Machine Learning Engineer USD 130K-194KAI machine learning | AWS AI | AWS AI Machine Learning | Amazon DynamoDB | Amazon EC2Professional development | Work from homeMid-level Full TimeRemote, NY, US R14h ago
-
Machine Learning Engineer CHF 106K-150KAWS SageMaker | CI/CD | Data Pipelines | Docker | Information RetrievalCharity donation matching | Complimentary coffee and refreshments | Flexible working hours | Health and fitness subsidy | Healthy snacksMid-level Full TimeZürich, ZH, Switzerland R15h ago
-
Software Specialist (TL) - AI/ML - Monetisation GBP 35K-40KC++ | Causal Inference | Data Analysis | Data Modeling | Data PipelinesSenior-level Full TimeRemote, UK | London, UK R15h ago
-
Data Scientist, Analytics (Technical Leadership) USD 160K-190KAI Workflow Optimization | AI workflow | Agent Orchestration | Bias Mitigation | Causal InferenceCareer development | World class analytics communitySenior-level Full TimeRemote, US | Bellevue, WA | … R15h ago
-
Data Engineer (Azure Databricks) EUR 40K-50KApache Spark | Azure Data | Azure Data Factory | Azure Databricks | CI/CDCertification programs | Health insurance | Internal mobility | Mentorship | Professional developmentMid-level Full TimeMadrid, MD, Spain R16h ago
-
Data Engineer - Aeroline - Aix-en-Provence EUR 45K-54KApache Spark | ELT | ETL | JavaScript | Palantir FoundryEmployee representative council | Health insurance | Meal vouchers | Profit sharing | Referral bonusesSenior-level Full TimeAix-en-Provence, Provence-Alpes-Côte d'Azur, France R18h ago
-
Amplitude | Analytics engineering | Business Intelligence | Data Governance | Data ModelingDirect impact on product decisions | Hybrid work setup | Innovation-focused team | Office access | Work in a fast-growing companyMid-level Full TimeStockholm, Sweden R20h ago
-
Adaptive Control | Apache Spark | Azure DevOps | C++ | CI/CDFlexible hybrid work model | Flexible vacation policy | Paid Holidays | Paid time off | Personal and family careSenior-level Full TimeNew York, NY, US, 10001-8604 R1d ago
-
Adaptive Control | Agile | Audio Processing | Azure DevOps | C++Flexible vacation policy | Hybrid work model | Paid Holidays | Travel opportunitiesMid-level Full TimeNew York, NY, US, 10001-8604 R1d ago
-
AWS | Benchmarking | Code Quality | Code review | Data Pipelines401k matching | Dining | Flexible spending accounts | Healthcare | Onsite gymSenior-level Full TimeRemote - US R1d ago
-
GenAI Engineer - EY GDS Spain - Hybrid EUR 49K-66KAPI Development | AWS | Agentic AI | Autogen | AzureCoaching | Continuous learning | Inclusive cultureMid-level Full TimeMalaga, ES, 29590 R1d ago
-
AWS CloudFormation | AWS Glue | Amazon Kinesis | Amazon Redshift | Apache AirflowRemote workMid-level Full TimePakistan - Remote R1d ago
-
AI Team Lead PHP 150K-250KAutonomous Agents | Azure | Azure Machine Learning | BERT | CI/CDPaid time off | Performance bonus | Training and developmentSenior-level Full TimeEgypt - Remote R1d ago
-
Data Engineer (Python/Hybrid/Athens) EUR 38K-60KAPIs | Data Modeling | Data Pipeline Monitoring | Data Transformation | Data pipelineCourse support | Free coffee | Hybrid work | Life insurance | Lunch allowanceEntry-level Full TimeAthens, Attica, Greece R1d ago
-
AI-Native Engineer (Full-Stack / Agentic AI Engineer) PLN 246K-378KAWS | Agent Orchestration | Airflow | Automated testing | CrewAIAutonomy | Direct stakeholder access | Paid software licenses | Premium gear | Remote-firstMid-level Full TimeWarsaw, Masovian Voivodeship, Poland - Remote R1d ago
-
AI Engineer EUR 62K-69KContext engineering | Experimentation | LLMOps | Language Models | Large Language ModelsFlat hierarchies | Flexible work from home | International work environment | Private health insurance | Remote work optionMid-level Full TimeSpain - Remote R1d ago
-
Senior Software Engineer INR 3200K-4600KApache Airflow | CI/CD | DBT | Data Governance | Data WarehousingSenior-level Full TimeBangalore - Carina, India R1d ago
-
Data Cloud Engineer INR 1019K-1500KAPI Integration | Data Cleansing | Data Governance | Data Modeling | Data cloudHybrid workMid-level Full TimeRemote (IND), India R1d ago
-
Software Engineer III (AI/ML) USD 113K-169KAPI Design | AWS | CrewAI | Docker | DynamoDBCompany sponsored retirement plan | Flexible work environment | Health and wellness benefits | Hybrid work options | In person orientation processSenior-level Full TimeRemote (USA), United States R1d ago
-
Sr. Data Engineer - Cloud Security (Hybrid, ISR) ILS 380K-473KAWS | Apache Iceberg | Apache Spark | Cassandra | DockerCompetitive vacation and holidays | Employee networks | Paid adoption leave | Paid parental leave | Professional development opportunitiesSenior-level Full TimeTel Aviv (Sky Tower), Israel R1d ago
-
GenAI Architect - Agentic USD 156K-202KAPI Gateway | AWS | AWS Bedrock | AWS Lambda | AWS Step FunctionsLearning and growth | Remote workSenior-level Full TimeUSA - Remote, United States R1d ago