Big Data Engineer
Tasks
- Analyze data for analytics and reporting
- Build analytics platforms
- Design data processing pipelines
- Document engineering solutions
- Ensure reliable performant cost effective production data platforms
- Ingest structured and unstructured data
- Operate Hadoop based data systems
- Support machine learning data workloads
- Transform and process large datasets
- Troubleshoot and debug data pipelines
Perks/Benefits
Skills/Tech-stack
AWS EMR | Apache Airflow | Apache Atlas | Apache Flink | Apache HBase | Apache Hadoop | Apache Hive | Apache Hudi | Apache Iceberg | Apache Kafka | Apache Oozie | Apache Spark | Apache Sqoop | Azure HDInsight | CI/CD | Collibra | Databricks | Delta Lake | Distributed Systems | Fault Tolerance | HDFS | Infrastructure as Code | Java | Kubernetes | NoSQL | Partitioning | Python | Replication | SQL | Scala | Shell | Spark Streaming | Trino | “as-code”
Education
Roles
Related jobs
-
AWS | Azure | CI/CD | Containers | DNS401k match | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSan Diego, CALIFORNIA, United States R1d ago
-
AI Engineer USD 180K-225KAWS | Agentic Workflows | Airflow | BigQuery | CI/CD401k match | Dental and vision coverage | Employee Assistance Program (EAP) | Employee discounts | HSA contributionsSenior-level Full TimeRemote, US R1d ago
-
AI Engineer USD 140K-182KCaching | Embeddings | Error Recovery | Evaluation | Fallback LogicCollaborate with small fast moving team | Competitive equity | Direct ownership | Full-time employment | Hybrid workMid-level Full TimeNew York, New York, United States R1d ago
-
Senior Data Engineer II USD 145K-207KAirflow | Alerting | BigQuery | CI/CD | CloudMeaningful greenfield work | Onsite visits up to four times per year | Remote work within United States | Work-life balanceSenior-level Full TimeUnited States - Remote R1d ago
-
Senior Machine Learning Engineer I USD 150K-241KAWS | Azure | Batch Processing | Data Pipelines | Data labelingDental insurance | Emotional wellness support | Employee resource groups | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R1d ago
-
Principal Machine Learning Engineer USD 240K-420KAgent systems | Architecture | C plus plus | Compliance | Deep learning401k plan | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSanta Clara, California, United States R1d ago
-
Agent architecture | Benchmarking | Biomedicine | Context engineering | Data integrationFlexible working hours | Generous vacation | Hybrid work policy | Optional equity donation matching | Parental leaveMid-level Full TimeSan Francisco, CA | New York … R1d ago
-
Site Reliability Engineer - Ceph Storage USD 128K-192KAnsible | Block devices | CRUSH | Cause analysis | CephEmployee stock purchase plan | Generous time off | Healthcare benefits | Parental leave | Remote work optionsSenior-level Full TimeUnited States R1d ago
-
Senior Data Engineer USD 140K-170KAPIs | Azure DevOps | DBT | Data Architecture | Data GovernanceDiscretionary bonus | Hybrid work schedule | Training and developmentSenior-level Full TimeNew York, New York, United States R1d ago
-
Staff Software Engineer, Storage Services USD 177K-364KAI | C plus plus | Capacity Planning | Consensus | Distributed SQLSenior-level Full TimeNew York, NY, US; Remote, US R1d ago
-
Staff Software Engineer, Data Warehouse Foundation USD 177K-364KApache Airflow | Apache Flink | Apache Spark | Apache Superset | Data PipelinesSenior-level Full TimePalo Alto, CA, US; Remote, US R1d ago
-
Senior Data Engineer USD 109K-164KAI Services | Azure AI | Azure AI Services | Azure Data | Azure Data Factory401k retirement plan | Employee assistance program | Health spending account | Life insurance | Long-term disabilitySenior-level Full TimeAustin, Texas, USA R1d ago
-
Senior Data Engineer- People Analytics USD 180K-207KAccess Control | Airflow | Amazon Redshift | CI/CD | Cloud platformSenior-level Full TimeUnited States R1d ago
-
Senior Machine Learning Engineer USD 130K-160KAWS CDK | AWS Fargate | AWS IAM | AWS Lambda | AWS SAMEmployer match option | Flexible spending accounts | Health insurance | Long-term disability | Paid leaveSenior-level Full TimeRemote, United States R1d ago
-
Senior Data Visualization Engineer USD 114K-142KAdobe Illustrator | Adobe Photoshop | ArcGIS | Cartography | D3Employee assistance program | Home office reimbursement | Paid parental leave | Tuition reimbursement | Wellness programSenior-level Full TimeSan Francisco, CA R1d ago
-
API Integration | AWS | Agentic Workflows | Azure | CrewAIEmployee recognition | Employee referral bonus | Flexible work locations | Mobility among contracts | Paid time offSenior-level Full TimeAnnapolis Junction, Maryland R1d ago
-
Staff Software Engineer, Applied AI USD 200K-260KDataset curation | FastAPI | Golden datasets | Langchain | Langgraph401k retirement plan | Annual learning and development stipend | Employer HSA contribution | Equity incentive plan | Flexible vacation policySenior-level Full TimeU.S. Remote R1d ago
-
Senior Solution Engineer, Retail & Consumer Goods USD 165K-216KCloud Computing | Data Lake | Data Warehouse | Data fabric | Data meshRemote work | Travel opportunitySenior-level Full TimeUS, Remote R1d ago
-
Lead Data Engineer - Tax Product Development USD 140K-210KAgile | Apache Airflow | Apache Spark | Azure Cosmos | Azure Cosmos DBSenior-level Full TimeUnited States R1d ago
-
Principal Data Engineer USD 200K-280KAI Assisted Development | AWS | Airflow | Amazon Redshift | AzureSenior-level Full TimeDenver, CO or Long Beach, CA R1d ago
-
Senior Data Engineer USD 155K-180KAWS | Apache Hadoop | Apache Kafka | Apache Spark | Artificial IntelligenceHealth insurance | Opportunities for career growth | Opportunities to lead teams | Paid Holidays | Paid time offSenior-level Full TimeUnited States R1d ago
-
Data Engineer USD 152K-223KAWS | Apache Hadoop | Apache Kafka | Apache Spark | AzureHealth insurance | Opportunity for career growth | Paid Holidays | Paid time off | Training and developmentMid-level Full TimeUnited States R1d ago
-
Embedded Software Development Engineer (Senior Level) USD 120K-160KC# | C++ | Critical software | DO-178 | Embedded SystemsCritical illness insurance | Employee assistance program | Health insurance | Legal assistance | Mental health benefitsSenior-level Full TimeEast Hartford, CT, United States R1d ago
-
Embedded Software Verification Engineer (Mid level) USD 99K-134KC# | C++ | Case Development | Cause analysis | DO-178CBereavement leave | Critical illness insurance | Employee assistance program | Legal assistance | Medical, dental, and vision insuranceMid-level Full TimeEast Hartford, CT, United States R1d ago
-
C# | C++ | Case Development | Cause analysis | DO-178CCritical illness insurance | Dental insurance | Employee assistance program | Health insurance | Legal assistanceEntry-level Full TimeEast Hartford, CT, United States R1d ago