Senior Data Engineer ID75059
Blacksburg, United States
USD 156K-190K (estimate) Senior-level Full Time
Tasks
- Build AWS and on prem infrastructure with Terraform
- Build scalable data lakes data warehouses and data lakehouses
- Consult with stakeholders and leadership to gather requirements
- Design and implement Python data engineering solutions
- Develop ingestion workflows from third-party APIs
- Implement robust ETL ELT processes at scale
- Manage and optimize file formats and columnar storage
- Optimize data pipelines and troubleshoot latency consistency issues
- Orchestrate data pipelines with Airflow
- Translate business goals into data architecture roadmaps
Perks/Benefits
Skills/Tech-stack
API ingestion | AWS | Airflow | Apache Spark | Avro | Big Data | Columnar Storage | Data Ingestion | Data pipeline | Data pipeline orchestration | DuckDB | ELT | ETL | Lakehouse | Medallion Architecture | NoSQL | ORC | Pandas | Parquet | Pipeline Orchestration | Polars | PySpark | Python | SQL | Snowflake | Spark SQL | Terraform
Education
Roles
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R11d ago
-
Data Engineer USD 114K-190KApache Airflow | Big Data | Data profiling | EJB | ETLHealth insurance | Holiday pay | Learning and development | Life insurance | Long-term disabilitySenior-level Full TimeUSA-VA-Chantilly5h ago
-
Senior Digital Analytics Engineer - Customer Experience USD 114K-200KA/B | A/B Testing | Adobe Analytics | Adobe Customer Journey | Adobe Customer Journey AnalyticsCareer development | Health and well-being benefits | Inclusive team | Remote work allowedSenior-level Full TimeCleveland, OH, United States5h ago
-
Apache Spark | Automated testing | CI/CD | Data Governance | Data Modeling401k match | Dental insurance | Life insurance | Medical insurance | Paid HolidaysSenior-level Full TimeNew York5h ago
-
Databricks Engineer USD 107K-179KAPI Integration | Agile | Data Governance | Data Modeling | Data QualityDiscretionary incentive program | Mentorship | Professional development | Travel opportunitiesMid-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …5h ago
-
Research Engineer, ML H-W/S-W Codesign USD 173K-249KC plus plus | Computer Vision | Deep learning | Edge AI | Hardware ArchitectureEntry-level Full TimeSunnyvale, CA6h ago
-
Senior ML Engineer, AI-Driven SDLC Quality USD 174K-253KAgentic Systems | Dataset development | Distributed Computing | Evaluation Frameworks | Generative AISenior-level Full TimeSan Jose, CA, USA6h ago
-
Senior Software Engineer, Google Cloud Storage USD 174K-253KC plus plus | C# | Cloud Computing | Distributed Systems | Error detectionSenior-level Full TimeNew York, NY, USA6h ago
-
Staff Software Engineer, Agentic Data and Evals USD 207K-301KArtificial Intelligence | C++ | CI/CD | CSS | Data StorageSenior-level Full TimeSunnyvale, CA, USA6h ago
-
Staff Software Engineer, Data USD 117K-171KAPIs | AWS | CI/CD | Data Engineering | Data ObservabilityBonus/incentive programs | Dental | Health care | Paid time off | RetirementSenior-level Full TimeCreve Coeur, Missouri, US9h ago
-
Senior / Principal Infrastructure Engineer - ML Platform USD 278K-345KAWS | CI/CD | Cloud infrastructure | Cloud platform | Cluster managementEquity compensation | Onsite schedule Tuesday Wednesday ThursdaySenior-level Full TimeSan Mateo, CA, United States R12h ago
-
Senior-level Full TimeCosta Mesa, California, United States15h ago
-
Robotics Software Engineer, Sensor Integration USD 191K-253KAgent systems | C++ | Computer Vision | Distributed Systems | Embedded SystemsEmployee assistance programs | Health benefits | Recovery Benefits | Travel opportunitiesMid-level Full TimeCosta Mesa, California, United States15h ago
-
Software Engineer - Business Analytics USD 100K-125KAPI Integration | Cloud Computing | Data Modeling | Data Visualization | Database401k matching | Equity package | Global Entry | Health and wellness stipend | Health, dental, vision insuranceEntry-level Full TimeSan Diego16h ago
-
Entry-level Full TimeAlcoa, TN, US17h ago
-
Senior Software Engineer - Backend & Machine Learning USD 180K-250KAWS | Amazon Web Services | Argo | Blue-Green Deployment | Blue/greenSenior-level Full TimeNew York, New York, United States17h ago
-
Mid-level Full TimeCarmel, Indiana- Hybrid OR St. Louis, … R17h ago
-
Mid-level Full Time9633 McLean VA Non-specific Customer Site, …17h ago
-
Senior AI Infrastructure Engineer, LLM/AI Platforms USD 140K-215KAWS | Airflow | Amazon SageMaker | Ansible | BatchingCompetitive vacation and holidays | Comprehensive wellness programs | Employee networks | Paid adoption leave | Paid parental leaveSenior-level Full TimeUSA VA Remote, United States R17h ago
-
Senior-level Full TimeChicago IL, United States17h ago
-
Senior-level Full TimeUS-WA-Bellevue18h ago
-
Senior-level Full TimeMansfield, MA (Remote) R18h ago
-
Staff Software Engineer, Applied AI USD 200K-240KDataset curation | Evaluation | FastAPI | LLM | Langchain401k | Co-working space reimbursement | Equity incentive | Flexible vacation policy | HSA contributionSenior-level Full TimeU.S. (Remote) R19h ago
-
Senior-level Full TimeUnited States20h ago
-
Mid-level Full TimeMountain View, California, United States20h ago