Data Platform Engineer
Tasks
- Automate pipeline orchestration
- Build ETL/ELT workflows
- Deliver curated reliable documented datasets
- Design data models and storage layouts
- Design end to end big data pipelines on Hadoop
- Develop monitoring alerting and logging
- Develop streaming data pipelines
- Document data architectures schemas and runbooks
- Evaluate and adopt new big data and cloud technologies
- Implement data governance lineage quality controls
- Lead performance reviews and architecture audits
- Mentor junior engineers
- Optimize Spark and MapReduce jobs
Perks/Benefits
- N/A
Skills/Tech-stack
AWS EMR | Apache Airflow | Apache Atlas | Apache Flink | Apache Hive | Apache Hudi | Apache Kafka | Apache Oozie | Apache Pig | Apache Spark | Apache Sqoop | Azure HDInsight | BigQuery | CI/CD | Collibra | Databricks | Delta Lake | Distributed Systems | HBase | HDFS | Hadoop | Iceberg | Infrastructure as Code | Java | Kubernetes | MapReduce | ORC | Parquet | Python | SQL | Scala | Shell | Snowflake | Spark Streaming | Trino | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R7d ago
-
Senior-level Full TimeSunnyvale, CA | Remote, US | … R9h ago
-
Software Engineer, Machine Learning USD 213K-251KAlgorithms | Data Structures | Graph theory | Information Retrieval | Language ProcessingSenior-level Full TimeSunnyvale, CA | Los Angeles, CA … R9h ago
-
Senior-level Full TimeRemote, US | Bellevue, WA | … R9h ago
-
Staff Data Scientist - Digital Intelligence USD 191K-230KAdversarial Modeling | Anomaly Detection | Apache Spark | Data Processing | Distributed dataSenior-level Full TimeHybrid - US R15h ago
-
Director of Data Platform Engineering USD 180K-220KAccess Control | Amazon Redshift | Apache Airflow | Apache Flink | Apache KafkaComprehensive health coverage | Remote-first workplace | Unlimited PTOExecutive-level Full TimeRemote - United States R16h ago
-
Data Scientist ll - Digital Intelligence USD 140K-170KDatabricks | Experiment analysis | Feature Engineering | Machine Learning | Model EvaluationMid-level Full TimeHybrid - US R17h ago
-
Senior Data Engineer, Platform USD 119K-149KAWS DMS | Access Control | Airflow | Amazon Web Services | CI/CDSenior-level Full TimeRemote - US, United States R20h ago
-
Edge AI Engineer USD 100K-155KC++ | Core ML | Cross Platform Inference | Cross-platform | Edge inferenceSenior-level Full TimeUnited States - Remote R20h ago
-
Full Stack AI Engineer USD 140K-170KAWS | Agent workflows | Anthropic | Azure | CI/CD401k matching | Birthday off | Employee assistance program | Medical/Dental/Vision insurance | Paid parental leaveSenior-level Full TimeRemote South Carolina, Remote South Carolina, … R20h ago
-
Senior-level Full TimeUnited States - Remote R20h ago
-
AI Research Engineer USD 100K-150KAccelerator hardware | Agentic Systems | Computer Vision | Data Quality | Data labelingCareer growth | Remote workSenior-level Full TimeUnited States - Remote R20h ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HudiRemote workSenior-level Full TimeUnited States - Remote R20h ago
-
Hadoop Big Data Developer USD 100K-150KAirflow | Apache Flink | Apache Hive | Apache Pig | Apache SparkCareer growth | Remote workSenior-level Full TimeUnited States - Remote R20h ago
-
Senior-level Full TimeUnited States - Remote R20h ago
-
Backend AI-Forward Data Engineer USD 100K-125KAWS | Airflow | Anomaly Detection | Automated testing | Azure DataDental insurance | Disability insurance | Health insurance | Life insurance | Paid HolidaysSenior-level Full TimeRemote South Carolina, Remote South Carolina, … R20h ago
-
Senior Data Engineer USD 100K-150KAWS Glue | Agile | Airflow | Amazon Redshift | Apache SparkCareer growth potential | Full-time benefits | Remote workSenior-level Full TimeUnited States - Remote R20h ago
-
Batch Processing | Confluence | Data Pipelines | Data Quality | Data ValidationHybrid work schedule | Remote work considerationSenior-level Full TimeOhio - Columbus, Three Nationwide Plaza, … R20h ago
-
Data Platform Architect USD 100K-175KAccess Management | Alation | Amazon Redshift | Apache Flink | Apache HudiRemote workSenior-level Full TimeUnited States - Remote R20h ago
-
LLM Engineer USD 100K-150KAdapter-Tuning | DPO | Dataset curation | DeepSpeed ZeRO | Efficient AttentionCareer growth | Remote workMid-level Full TimeUnited States - Remote R20h ago
-
Model Optimization Engineer USD 100K-150KBenchmarking | C++ | CUDA | Continuous batching | DeepSpeedSenior-level Full TimeUnited States - Remote R20h ago
-
Prompt Engineer USD 100K-150KAgentic Workflows | Chunking | Cost Optimization | Embeddings | EvaluationCareer growth | H1B transfer support | Remote workMid-level Full TimeUnited States - Remote R20h ago
-
Robotics Software Engineer USD 100K-150KBehavior Trees | C++ | Concurrent Systems | Control Algorithms | Critical softwareCareer growth | Code review mentorship | Remote workMid-level Full TimeUnited States - Remote R20h ago
-
AI Operations Engineer USD 100K-150KApache Beam | Apache Spark | CI/CD | Code review | Data LineageCareer growth | Remote workSenior-level Full TimeUnited States - Remote R20h ago
-
ML Platform Engineer USD 100K-160KAPI Gateway | Abuse detection | Automated rollback | Autoscaling | C++Senior-level Full TimeJersey City (India Square), NJ R20h ago