Data Engineer
Tasks
- Assess data storage processing and transmission for privacy compliance
- Convert and optimize legacy SQL and Spark patterns
- Ensure data protection measures across information lifecycle
- Evolve table schema and enforce constraints
- Identify assess and report security risks
- Manage temporal data modeling state changes
- Migrate extraction logic to Lakehouse
- Migrate job scheduling from legacy frameworks
- Optimize data partitioning and clustering
- Reconcile migrated data using data validation frameworks
- Serve as technical liaison for data handoff and sign off
- Transfer datasets and ensure data integrity
- Troubleshoot SQL and basic scripts
Perks/Benefits
- N/A
Skills/Tech-stack
ANSI SQL | Apache Iceberg | Apache Spark | Avro | CI/CD | Data Clustering | Data Modeling | Data Partitioning | FTP | HDFS | Hadoop | Hive | JSON | Java | Kafka | Kubernetes | Parquet | Python | SCD Type-2 | SDLC | SQL | Schema evolution | Snowflake | Sybase IQ | Temporal Data | Temporal Data Modeling | Type 2)
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Senior Software Engineer - Production Control Plane USD 200K-287KAmazon Web Services | Azure | C++ | CI/CD | Cloud platformSenior-level Full TimeUS-CA-Menlo Park8h ago
-
AI Search | AWS | Azure | Azure AI | Azure AI Search401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States10h ago
-
API | Angular | C# | C++ | Celery401k retirement plan | Company shuttles | Dental insurance | Employee stock purchase plan | Life insuranceSenior-level Full TimeRedmond, WA12h ago
-
Data Modeling | Deep learning | Ecommerce | Fine Tuning | Human-in-the-loopEquity | Health benefits | Hybrid work flexibility | Remote work up to 4 weeks per yearSenior-level Full TimeSan Francisco, CA R12h ago
-
Deployment Engineer - Commercial Site Team USD 120K-180KConfluence | Industrial Automation | Jira | Linux | Project ManagementCustomer site work experience | Travel opportunitiesMid-level Full TimeReno, NV12h ago
-
Senior Data Engineer USD 115K-145KAWS S3 | Airflow | Azure Blob | Azure Blob Storage | Blob Storage401k | Dental insurance | Discounts | Medical insurance | Paid leaveSenior-level Full TimeNew York, NEW YORK, United States R12h ago
-
Robotics ML Expert, AI USD 60K-60KControl Systems | Dm_control | Domain Randomization | Gymnasium Robotics | JAXAsync collaboration | Flexible schedule | Remote workSenior-level Full TimeMiami R12h ago
-
BI Data Engineer (Remote WA, OR, ID & CA) USD 88K-116KAzure Synapse | C# | DAX | Data Governance | Data Modeling401k match | Health insurance | Paid Holidays | Paid vacation | Sick timeMid-level Full TimeBellevue, Washington R12h ago
-
Artificial Intelligence | Confluence | Dashboards | Electronic Hardware | Electronic Hardware ValidationBenefits package | Flexible schedule | MentorshipEntry-level ContractFoster City, CA12h ago
-
Senior-level Full TimeUS-CA-Menlo Park13h ago
-
Machine Learning Engineer USD 185K-260KAdvertising Auctions | Airflow | BigQuery | Convolutional Neural Networks | Data Pipelines401k employer match | Caregiving support programs | Comprehensive healthcare benefits | Family planning support | Flexible vacationMid-level Full TimeRemote - United States R13h ago
-
Senior Data Engineer USD 119K-161KAnalysis Services Tabular | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeDenver, CO, United States R14h ago
-
Senior Data Engineer USD 119K-161KAnalysis Services Tabular | Azure Analysis | Azure Analysis Services | Azure Analysis Services Tabular | Azure DataMentorship opportunities | Remote workSenior-level Full TimeKansas City, MO, United States R14h ago
-
Data Engineer III/Senior (AI and Gen AI) USD 130K-150KAgentic Workflows | Azure | Azure Data | Azure Data Services | Bicep401k | Dental insurance | Employee stock purchase plan | Health insurance | Life insuranceSenior-level Full TimeIrving, TX, United States R14h ago
-
Data Engineer III/Senior (AI and Gen AI) USD 130K-150KAzure | Bicep | CI/CD | Docker | Kubernetes401k retirement plan | Dental insurance | Employee stock purchase plan | Health insurance | Life insuranceSenior-level Full TimeEdison, NJ, United States R14h ago
-
Data Engineer III/Senior (AI and Gen AI) USD 130K-150KApache Kafka | Azure | Azure Data | Azure Data Factory | Azure Databricks401k | Dental insurance | Employee stock purchase plan | Life insurance | Long-term disability insuranceSenior-level Full TimeEllenwood, GA, United States R14h ago
-
Data Engineer III/Senior (AI and Gen AI) USD 130K-150KAzure | CI/CD | Compliance | Data Lineage | Data Quality401k retirement plan | Dental insurance | Employee stock purchase plan | Health insurance | Life insuranceSenior-level Full TimeRomulus, MI, United States R14h ago
-
Principal MLOps Engineer USD 150K-200KAWS | Azure | CI/CD | Docker | Git401k match | Annual tech and gadget budget | Education and training benefits | Healthcare dental vision insurance | Paid HolidaysSenior-level Full TimeRemote, US; DMV; McLean, VA; Boston, … R14h ago
-
Principal MLOps Engineer USD 150K-200KAWS | Azure | CI/CD | Container hardening | Docker401k match | Annual tech budget | Education & training benefits | Flexible work options | Health, dental, vision coverageSenior-level Full TimeRemote, US; DMV; McLean, VA; Boston, … R14h ago
-
Principal MLOps Engineer USD 150K-200KAWS | Azure | CI/CD | Docker | Git401k match | Annual snacks budget | Education and training benefits | Fully covered healthcare dental and vision | Paid HolidaysSenior-level Full TimeRemote, US; DMV; McLean, VA; Boston, … R14h ago
-
Senior-level Full TimeArlington, VA15h ago
-
IT Intern- Data Engineer Meridian or Chubbuck USD 36K-50KAgile | Azure Data | Azure Data Factory | Azure Data Lakehouse | Azure DatabricksEntry-level InternshipMeridian, Idaho, United States; Chubbuck, Idaho, …15h ago
-
Mid-level Full TimePleasanton, CA, United States15h ago
-
Generative AI Consultant USD 94K-114KAI Services | AWS | Azure | CI/CD | Chain-of-Thought401k matching | Cell phone stipend | College loan repayment plan | Dental insurance | Flexible spending accountMid-level Full TimeNew York, NY, United States16h ago
-
C++ Embedded Simulation Engineer USD 120K-160KBash | C plus plus | CAN | Computer Vision | Data logging401k | Dental insurance | FSA | Hybrid work schedule | Medical insuranceSenior-level Full TimeMountain View, California, United States16h ago