AI Data Engineer - Security (Kafka Experience)
Tasks
- Apply data governance and RBAC
- Build AWS data lake zones
- Build ELT models in Snowflake
- Design event streaming platforms
- Develop idempotent DAGs
- Encrypt and lifecycle S3 data
- Implement CDC with Snowpipe
- Implement IAM and Secrets Manager
- Ingest data from GraphQL APIs
- Ingest data from REST APIs
- Land data in S3 as Parquet
- Manage Glue Data Catalog
- Manage Kafka schema and performance
- Normalize and enrich data in Python
- Optimize Kafka consumer groups
- Optimize Snowflake warehouse performance
- Orchestrate pipelines with AirFlow
Perks/Benefits
Skills/Tech-stack
API Key | AWS Glue | AWS IAM | AWS Secrets | AWS Secrets Manager | Airflow | Amazon S3 | Apache Parquet | CDC | CI/CD | Data Governance | DynamicFrames | ELT | Git-based CI/CD | GraphQL | Kafka | OAuth2 | PySpark | Python | RBAC | REST API | Secrets Manager | Snowflake | Snowpipe
Education
N/A
Roles
Related jobs
-
Featured Feat. AI Engineer (MTS) USD 160K-300KAPI Development | AWS | Amazon Web Services | Deep learning | FastAPIMentoring | Open source contributions | Remote workMid-levelRemote R12d ago
-
Featured Feat. Data Engineer USD 80K-150KData Monitoring | Data Quality | Data Validation | ELT | ETLRemote workEntry-levelRemote R12d ago
-
Mid-level Full Time北京 R6h ago
-
Mid-Level Data Engineer USD 90K-98KAPI Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageRemote workMid-level Full TimeWork from home, VA, United States R9h ago
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R9h ago
-
AWS Glue | AWS Lambda | Airflow | Amazon S3 | AzureRemote workSenior-level Full TimeRemote R10h ago
-
Data Engineer (MS) (Remote) INR 2040K-3380KCI/CD | Data Transformation | Data Validation | Date normalization | ETLMentorship opportunities | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India R10h ago
-
AI / LLM Integration Engineer (Remote) INR 1800K-2800KAPI Integration | Anthropic SDK | Async APIs | Benchmarking | Confidence scoring100 percent remote | Collaborative culture | Inclusive team culture | Professional growthMid-level Full TimeMaharashtra, Pune, India R11h ago
-
ML / LLM Engineer (Remote) INR 2500K-3000KAmazon Web Services | Azure | Classification | Feature Engineering | Language ModelsRemote workMid-level Full TimeMaharashtra, Pune, India R11h ago
-
Evergreen - Mathematics for Machine Learning USD 80K-300KAutodiff | JAX | Linear Algebra | Machine Learning | Matrix OperationsPart-time workMid-level Full TimeRemote, Remote, BR R11h ago
-
C++ | Cloud platform | Data Pipelines | ETL | Google CloudCDI | Career growth opportunities | Flexible work environment | Telework 1 day per weekSenior-level Full TimeCastelnaudary, France R12h ago
-
AI Solution Architect GBP 75K-100KAI Services | API Development | API Gateway | AWS | AWS BedrockBusiness travel | Free eye tests | Glasses Support | Hybrid work schedule | Incentivized certifications and accreditationsSenior-level Full TimeLondon, Birmingham, Manchester, Newcastle upon Tyne, … R13h ago
-
Maitrise Ouvrage/Support Booking & Risk H/F EUR 21K-21KOffice 365 | Python | SQLAgile team environment | Career coaching | HR support | Modern campus services | Remote work opportunityEntry-level Full TimeEurope, France, Ile-de-France, 92 - Hauts-De-Seine R15h ago
-
Data Engineer Databricks (H/F) EUR 47K-55KAmazon Web Services | Apache Spark | Azure | Azure DevOps | CI/CDCareer development | Flexible remote work | Meal tickets | Paid time off | RTT daysSenior-level Full TimeSAINT OUEN, France R16h ago
-
Analytics Engineer EUR 48K-78KAmazon Redshift | Cloud platform | DBT | Data Modeling | Data TestingAnnual blood tests | Company apartments | Extra paid vacation days | Flexible schedule | Insurance and wellness programsMid-level Full TimeKaunas, Lithuania R16h ago
-
Senior Databricks EUR 46K-55KAWS | Apache Spark | Azure | Azure DevOps | Batch ProcessingCareer coaching | Conference speaking opportunities | Flexible telework | Meal tickets | Paid time offSenior-level Full TimeSAINT OUEN, France R17h ago
-
Anthropic Claude | Async Programming | ChatGPT | Claude Code | CodexDirect user impact | Flexible work schedule | Occasional international travel | Work from home optionMid-level Full TimeSlovenia / Remote R18h ago
-
Data Operations Engineer INR 2040K-3380KApache Airflow | Data Pipelines | Data Refresh | Data Warehousing | Data pipelineEmergency incident response support | On-call rotationSenior-level Full TimeBengaluru, INDIA, India R18h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100% remote work | Career growth opportunities | Flexible work environmentMid-level Full TimeEstonia R20h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100 percent remote work | Autonomous work environment | Career growth | Flexible work environment | International team cultureMid-level Full TimeHungary R20h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | ForecastingCareer growth | Flexible work environment | Remote workMid-level Full TimeFinland R20h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScript100% remote work | Career growth opportunities | Flexible work environmentMid-level Full TimeCzechia R20h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100% remote work | Career growth opportunities | Flexible work environment | International team cultureMid-level Full TimeNorway R20h ago
-
API Integration | Anomaly Detection | Data Modeling | Docker | Machine Learning100 percent remote work | Autonomy | Career growth | Flexible work environment | International team cultureMid-level Full TimeLuxembourg R20h ago
-
API Integration | Anomaly Detection | Data Modeling | Data Pipelines | DockerCareer growth | Flexible schedule | International team culture | Remote workMid-level Full TimeCroatia R20h ago