AI Data Engineer - Security (Kafka Experience)
Tasks
- Apply data governance and RBAC
- Build AWS data lake zones
- Build ELT models in Snowflake
- Design event streaming platforms
- Develop idempotent DAGs
- Encrypt and lifecycle S3 data
- Implement CDC with Snowpipe
- Implement IAM and Secrets Manager
- Ingest data from GraphQL APIs
- Ingest data from REST APIs
- Land data in S3 as Parquet
- Manage Glue Data Catalog
- Manage Kafka schema and performance
- Normalize and enrich data in Python
- Optimize Kafka consumer groups
- Optimize Snowflake warehouse performance
- Orchestrate pipelines with AirFlow
Perks/Benefits
Skills/Tech-stack
API Key | AWS Glue | AWS IAM | AWS Secrets | AWS Secrets Manager | Airflow | Amazon S3 | Apache Parquet | CDC | CI/CD | Data Governance | DynamicFrames | ELT | Git-based CI/CD | GraphQL | Kafka | OAuth2 | PySpark | Python | RBAC | REST API | Secrets Manager | Snowflake | Snowpipe
Education
N/A
Roles
Related jobs
-
Featured Feat. AI Engineer (MTS) USD 160K-300KAPI Development | AWS | Amazon Web Services | Deep learning | FastAPIMentoring | Open source contributions | Remote workMid-levelRemote R12d ago
-
Featured Feat. Data Engineer USD 80K-150KData Monitoring | Data Quality | Data Validation | ELT | ETLRemote workEntry-levelRemote R12d ago
-
Applied AI Engineer USD 150K-200KAPIs | Agent systems | Chunking | Cloud infrastructure | Data Pipelines401k | Employee assistance program | Flexible PTO | Flexible work schedule | Health benefitsSenior-level Full TimeRemote R8h ago
-
Staff Data Engineer CAD 180K-195KAmazon Redshift | BigQuery | CDC | DBT | Data GovernanceCompassion leave | Local meetups | Paid sick and safe time | Pension contribution | Remote-firstSenior-level Full TimeCalgary, Alberta, Canada R9h ago
-
Mid-level Full Time北京 R12h ago
-
API Development | AWS | Artificial Intelligence | Caching | Data PipelinesAWS Partnership training | Advanced engineering resources | Early access to cloud capabilitiesSenior-level Full TimeBogota, Colombia (Remote Friendly) R12h ago
-
Senior AI Engineer GBP 75K-75KAWS | Agent systems | Artificial Intelligence | Azure | CI/CDAnnual bonus | Discounted gym membership | Electric vehicle leasing | Experience days | Hybrid workSenior-level Full TimeLondon, United Kingdom R14h ago
-
Mid-Level Data Engineer USD 90K-98KAPI Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageRemote workMid-level Full TimeWork from home, VA, United States R14h ago
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R14h ago
-
Applied AI Analytics Lead 14 Months FTC GBP 70K-90KAzure Foundry | Copilot Studio | DBT | Data Modeling | Microsoft FabricHybrid work modelSenior-level ContractLondon, England, United Kingdom R15h ago
-
AWS | AWS EC2 | AWS Kinesis | AWS Lambda | AWS S3Flexible fully remote work | Home office setup allowance | Learning and development budget | Medical dental life disability coverage | Paid parental leaveMid-level Full TimeCanada R16h ago
-
Senior AI Engineer | Sage Home Loans USD 150K-220KAgent Orchestration | Automated Regression | Automated regression testing | Cost Optimization | DPO401k match | Disability insurance | Employee assistance program | Flexible paid time off | Flexible spending accountsSenior-level Full TimeCharlotte, NC R16h ago
-
AWS Glue | AWS Lambda | Airflow | Amazon S3 | AzureRemote workSenior-level Full TimeRemote R16h ago
-
Data Engineer (MS) (Remote) INR 2040K-3380KCI/CD | Data Transformation | Data Validation | Date normalization | ETLMentorship opportunities | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India R16h ago
-
AI / LLM Integration Engineer (Remote) INR 1800K-2800KAPI Integration | Anthropic SDK | Async APIs | Benchmarking | Confidence scoring100 percent remote | Collaborative culture | Inclusive team culture | Professional growthMid-level Full TimeMaharashtra, Pune, India R16h ago
-
Data Engineer - Transport - Nantes EUR 47K-55KAnsible | Apache Airflow | Apache Hadoop | Apache Hive | Apache KafkaCooptation bonus | Employee profit-sharing | Employee representative council | Health insurance | Meal vouchersSenior-level Full TimeNantes, Pays de la Loire, France R16h ago
-
ML / LLM Engineer (Remote) INR 2500K-3000KAmazon Web Services | Azure | Classification | Feature Engineering | Language ModelsRemote workMid-level Full TimeMaharashtra, Pune, India R16h ago
-
Cloud Data Engineer - Lead Consultant EUR 49K-70KCloud platform | Google Cloud | Google Cloud Platform | Python | SQLContinuous training and development | Hybrid work | Remote work 4 weeks per year | Team-building activities | Work with leading technology companiesSenior-level Full TimeAmsterdam, NH, Netherlands R16h ago
-
Evergreen - Mathematics for Machine Learning USD 80K-300KAutodiff | JAX | Linear Algebra | Machine Learning | Matrix OperationsPart-time workMid-level Full TimeRemote, Remote, BR R17h ago
-
C++ | Cloud platform | Data Pipelines | ETL | Google CloudCDI | Career growth opportunities | Flexible work environment | Telework 1 day per weekSenior-level Full TimeCastelnaudary, France R18h ago
-
AI Solution Architect GBP 75K-100KAI Services | API Development | API Gateway | AWS | AWS BedrockBusiness travel | Free eye tests | Glasses Support | Hybrid work schedule | Incentivized certifications and accreditationsSenior-level Full TimeLondon, Birmingham, Manchester, Newcastle upon Tyne, … R18h ago
-
AI Engineer EUR 61K-79KAWS Agent | AWS Agent Core | AWS Bedrock | AWS SageMaker | Agent coreCareer development program | Mentoring program | Remote work | Training budget | WFH allowanceMid-level Full TimeEuropean Union R20h ago
-
Apache Spark | Data Modeling | Data Quality | Data Visualization | Data WarehousingAccessibility support | Career growth | Fully remote | Inclusive workplace | Work-life balanceSenior-level Full TimeIndia R20h ago
-
Maitrise Ouvrage/Support Booking & Risk H/F EUR 21K-21KOffice 365 | Python | SQLAgile team environment | Career coaching | HR support | Modern campus services | Remote work opportunityEntry-level Full TimeEurope, France, Ile-de-France, 92 - Hauts-De-Seine R20h ago
-
Data Engineer Databricks (H/F) EUR 47K-55KAmazon Web Services | Apache Spark | Azure | Azure DevOps | CI/CDCareer development | Flexible remote work | Meal tickets | Paid time off | RTT daysSenior-level Full TimeSAINT OUEN, France R22h ago