Data Engineer
Tasks
- Architect build and maintain data pipelines and ETL
- Build time series features using Spark SQL and PySpark
- Collaborate with engineering data science and product teams to deliver data driven insights
- Design data models using Medallion architecture for forecasting
- Implement data governance and security practices
- Optimize Spark jobs tune partitions manage broadcast joins mitigate data skew
- Partner with data scientists to provide clean datasets for model training and deployment
- Translate messy data sources into scalable well modeled structures
Perks/Benefits
Skills/Tech-stack
Apache Airflow | Apache Spark | Data Governance | Data Modeling | Data Security | Data Skew | Databricks | Delta Lake | ETL | Medallion Architecture | Partitioning | PySpark | Python | SQL | Series forecasting | Time Series | Time Series Forecasting
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Data Engineer II USD 77K-115KData Modeling | Data Quality | Data Warehouse | Dimensional Modeling | ETLMid-level Full TimeSioux Falls, SD, United States2h ago
-
Senior Embedded Software Engineer USD 145K-182KAMD Xilinx | ARM Cortex | C# | Digital Signal | Digital Signal Processing401k match | 9 80 Workweek | Dental insurance | Employee assistance program | HSA/FSASenior-level Full TimeCamarillo, CA, United States3h ago
-
Finance Analytics Engineer USD 76K-95KAcceptance Testing | Data Modeling | Data Pipelines | Data Quality | Data Transformations401k matching | Bereavement leave | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeRemote - Nationwide, United States R3h ago
-
Data Engineer-Secret Clearance Required USD 100K-150KAWS | AWS Glue | Amazon Redshift | Azure | Azure Data401k matching | Bereavement | Employee assistance program | Health, dental, vision coverage | HolidaysSenior-level Full TimeRemote - Nationwide, United States R3h ago
-
API Gateway | Azure | BI Cloud Connector | BI Cloud Connector Console | BI Publisher401k matching | Bereavement | Disability insurance | Employee assistance program | Employee discount programSenior-level Full TimeRemote - Nationwide, United States R3h ago
-
A/B Testing | AI Studio | Adobe Analytics | Airflow | Amazon SageMaker401k | Dental insurance | Medical insurance | Paid Holidays | Paid parental leaveSenior-level Full TimeDallas, Frisco6h ago
-
API Integrations | AWS Glue | AWS Lambda | Amazon Redshift | Amazon S3Mid-level ContractLos Angeles, United States6h ago
-
Lead Data Expert USD 97K-163KData Architecture | Data Automation | Data Modeling | Data Quality | Data VisualizationOnsite 30 to 50 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States7h ago
-
CRM | Data Mining | Deep learning | Email outreach | Knowledge graphsMid-level Full TimeSan Jose, California, United States8h ago
-
Senior Software Engineer, AI/ML, Google Public Sector USD 174K-252KAlgorithms | C++ | Cloud Object Storage | Data Structures | Distributed ComputingSenior-level Full TimeReston, VA, USA9h ago
-
Staff Software Engineer, Intelligent Database Management USD 207K-300KAI | API Design | AlloyDB | Audit Logging | BigtableSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA9h ago
-
Forward Deployed Engineer I, GenAI, Google Cloud USD 102K-145KAPI Development | Agent Framework | Agent systems | Cloud Computing | CrewAISenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …9h ago
-
C++ | Data Mining | Data Processing | Deep learning | Few-Shot LearningSenior-level Full TimeMountain View, CA, USA9h ago
-
Software Engineer III, AI/ML, Google Ads USD 147K-211KC++ | Data Processing | Debugging | Information Retrieval | Language ProcessingSenior-level Full TimeMountain View, CA, USA9h ago
-
Senior Data Engineer USD 140K-165KAI/ML | Apache Airflow | CRM Integration | Cloud Data | Cloud Data PlatformsSenior-level Full TimeNY, United States11h ago
-
Senior Staff Data Engineer USD 123K-185KAPI | Automated testing | Cloud platform | Data Analysis | Data GovernanceDental insurance | Health care | Paid time off | Retirement plan | Sick leaveSenior-level Full TimeResidence Based, Residence Based, US12h ago
-
ML Engineer USD 190K-320KCost Optimization | Data Versioning | Dataset Operations | Dataset curation | Eval Frameworks401k matching | Dental insurance | Employee assistance program | Health insurance | Stock optionsSenior-level Full TimeSan Francisco14h ago
-
Senior-level Full TimeOklahoma City16h ago
-
Data Engineer USD 105K-115KBig Data | Cloud Computing | Data Modeling | Data Pipelines | Data StorageActive secret clearanceMid-level Full TimeSan Diego, CA, US16h ago
-
Principal Data Engineer USD 200K-240KAWS | Agentic Workflows | Anomaly Detection | Batch pipelines | CCPA401k plan | Commuter benefits | Flexible vacation | Life insurance | Long-term disabilitySenior-level Full TimeBoulder, Colorado or New York City, … R16h ago
-
Agile Development | CI/CD | Data Pipelines | Docker | GitHubAbility to obtain TS SCI with CI Polygraph | Active TS clearanceSenior-level Full TimeHuntsville, AL18h ago
-
Staff Machine Learning Engineer USD 159K-309KAWS | Airflow | Apache Spark | BigQuery | Cloud platform401k plan with company match | Commuter benefits | Disability coverage | Electric Car Charging Station | Employee assistance programSenior-level Full TimeMountain View, USA19h ago
-
Staff Machine Learning Engineer USD 152K-261KAWS | Airflow | Apache Spark | BigQuery | Cloud platform401k plan with company match | Commuter benefits | Disability insurance | Electric Car Charging Station | Employee assistance programSenior-level Full TimeMountain View, USA19h ago
-
Senior AI Solutions Engineer, Enterprise Knowledge Work USD 260K-325KAgentic Systems | Dspy | Evaluation | LLM orchestration | LanggraphCollaborative culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States; …19h ago
-
Senior AI Solutions Engineer, Software Engineering USD 260K-325KAgentic Software | Agentic Software Engineering | Agentic Workflows | Benchmarking | Code generationCollaborative culture | Five days per week | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States; …19h ago