Big Data Application Developer
Tasks
- Automate pipeline workflows and dependency management
- Build streaming data pipelines
- Design big data pipelines on Hadoop
- Design data models and storage layouts
- Develop ETL and ELT workflows
- Document data architectures and runbooks
- Evaluate and adopt new big data and cloud technologies
- Implement data governance and quality controls
- Integrate data platforms with analytics systems
- Lead performance reviews and architecture audits
- Mentor engineers and support engineering standards
- Optimize Spark and MapReduce jobs
- Orchestrate pipelines with workflow engines
- Set up monitoring, alerting, and logging
Perks/Benefits
- N/A
Skills/Tech-stack
AWS EMR | Airflow | Apache Flink | Apache Hive | Apache Hudi | Apache Iceberg | Apache Pig | Apache Spark | Apache Sqoop | Azure HDInsight | BigQuery | CI/CD | Data Governance | Data Lineage | Data Quality | Databricks | Delta Lake | Distributed Systems | HBase | HDFS | Hadoop | Infrastructure as Code | Java | Kafka | Kubernetes | Logging | MapReduce | Monitoring | NoSQL | ORC | Oozie | Parquet | Python | SQL | Scala | Shell Scripting | Snowflake | Spark Streaming | Trino | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R6d ago
-
Principal Software Engineer USD 105K-159KAWS | Agile | Azure | Bitbucket | CI/CDFlexible hybrid work model | Health and life insurance | Paid time off | Pension/retirement benefits | Personal/Family Care leavesSenior-level Full TimeTampa, FL, United States R13h ago
-
Principal AI Engineering Architect USD 224K-308KAWS | Data Processing | Docker | Google Cloud | High AvailabilityAdoption and surrogacy reimbursement | Cancer Care Program | Employee Assistance Program (EAP) | Fertility coverage | Flexible time offSenior-level Full TimeUnited States - Remote R21h ago
-
Staff Data Analytics Engineer USD 159K-220KApache Airflow | Apache Spark | BigQuery | DBT | Data ContractsFlexible working hours | Remote work flexibilitySenior-level Full TimeAmericas - Remote, US - Remote, … R1d ago
-
AI workflows | APIs | Artificial Intelligence | Data Infrastructure | Distributed SystemsDirect collaboration with executive leadership | High impact ownership | Rapid Career Growth Opportunities | Remote-first environment | Visa sponsorshipMid-level Full TimeSan Francisco, California, CA, United States R1d ago
-
Analytics Engineer USD 86K-165KAPI | Agile | Data Cleansing | Data Modeling | Data orchestration401k match | Dental insurance | Flexible work schedules | Life insurance | Medical insuranceSenior-level Full TimeUS-FL-REMOTE, United States R1d ago
-
Data Scientist Lead Consultant - Remote USD 90K-153KAWS | Azure | Machine Learning | Predictive Modeling | PythonConnectivity reimbursement | Equipment provided | Remote workSenior-level Full TimeUSA - NJ (Remote), United States R1d ago
-
Principal Data Engineering - Hybrid USD 112K-193KAPI Design | Apache Spark | Artifactory | Azure | Azure Databricks401k contribution | Career development opportunities | Comprehensive benefits package | Hybrid work schedule | Incentive and recognition programsSenior-level Full TimePrimary location: Eden Prairie, Minnesota R1d ago
-
Technical Director - ML & Algorithms USD 161K-225KAnomaly Detection | Apache Flink | Apache Spark | Auditability | Classification401k matching | Certificate reimbursement | Disability coverage | Health and wellness coverage | Leadership trainingExecutive-level Full TimeUnited States, Remote, United States R1d ago
-
Senior Data Engineer (Exadata) USD 91K-163KAnsible | Automation | Backup infrastructure | Capacity Planning | Cloud Service401k contribution | Career development | Comprehensive benefits | Equity stock purchase | Incentive and recognition programsSenior-level Full TimeEden Prairie, Minnesota, UNITED STATES; REMOTE R1d ago
-
Conversational AI Engineer USD 100K-150KAgent systems | Agentic Workflows | Embeddings | Evaluation | IndexingMid-level Full TimeUnited States - Remote R1d ago
-
Mid-level Full TimeUnited States - Remote R1d ago
-
Applied AI Engineer USD 100K-150KAccelerator hardware | Agentic Systems | Computer Vision | Data Quality | Data quality monitoringRemote workMid-level Full TimeUnited States - Remote R1d ago
-
Algorithmic Trading Developer USD 100K-150KAudit trails | Backtesting | C++ | Cloud Native | ConcurrencyCareer growth | H1B transfer support | Long term projects | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
ML Data Engineer USD 100K-150KApache Beam | CI/CD | Data Deduplication | Data Lineage | Data ModelingBenefits | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Robot Application Developer USD 100K-150KBehavior Trees | C++ | Concurrent Systems | Control Systems | Critical softwareCareer growth | Health benefits | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
Robotics Software Engineer USD 100K-150KBehavior Trees | C++ | Concurrent programming | Control Systems | Embedded SystemsMid-level Full TimeUnited States - Remote R1d ago
-
Senior AI Solutions Engineer USD 164K-215KAI Observability | Agentic Systems | Airflow | Amazon S3 | Amazon Web Services401k plan | Annual Caltrain Pass | Fastrak reimbursement | Fitness reimbursement | Health, dental, vision insuranceSenior-level Full TimeUS - Remote R1d ago
-
Senior Data Engineer USD 147K-190KAWS Lambda | Agile | Airflow | Amazon Redshift | Amazon SNS401k matching | Casual work environment | Dental insurance | Fitness reimbursement | Flexible paid time offSenior-level Full TimeRemote - US R1d ago
-
Data Visualization Engineer USD 90K-112KArcGIS | Cartography | D3 | Data Visualization | GDALCommuter benefits | Employee assistance program | Health savings account | Home office reimbursement | LinkedIn Learning accessMid-level Full TimeSan Francisco, CA R1d ago
-
Lead Data Engineer USD 160K-193KAmazon Redshift | CI/CD | DBT | ELT | ETL401k match | Adoption benefits | Employee resource groups | Flexible time off | Health insuranceSenior-level Full TimeDeerfield, IL, United States R1d ago
-
Senior Storage Engineer USD 145K-170KCapacity Planning | Cause analysis | Cloud Platforms | Data center | Distributed file systems401k match | Dental insurance | Employee assistance program | Health insurance | Health savings accountSenior-level Full TimeRemote, Texas R1d ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-200KAPI Testing | Cloud Computing | Cypher | Data Processing | Data QualityProfessional growth | Remote work | Travel opportunitiesSenior-level Full TimeReston, VA, United States R1d ago