Data Engineer
Lincoln - Nelnet Center, United States
R
USD 100K Mid-level Full Time
Tasks
- Apply classification tags at Bronze for promotion to Silver
- Apply required tags for cost attribution
- Automate reconciliation tie out freshness and schema validation as CI CD checks
- Build and maintain CDC and batch ingestion pipelines to Snowflake Bronze
- Build conformed Silver models from business rules and grain
- Collaborate on Docker containerization and Snowflake provisioning
- Contribute to CI CD pipeline templates
- Design and maintain raw snapshot tables with minimal transformation
- Detect upstream schema drift and document source schemas
- Implement data quality checks freshness and schema validation at ingestion boundary
- Implement historical tracking and prevent silent overwrites
- Implement idempotent load patterns with deduplication
- Implement incremental logic and performance tuning
- Manage source adapters for relational document API and file sources
- Monitor Dagster orchestration scheduling retries and failure alerting
- Monitor Snowflake write limits and tune ingestion patterns
- Participate in incident response for failed loads and latency cascades
- Perform Bronze to Silver reconciliation tie outs
- Support HVR to Redpanda CDC migration and validate integrity
- Track hard deletes in Snowflake
Perks/Benefits
- 401k
- Dental insurance
- Employee assistance program
- Employee stock purchase program
- FSA
- HSA
- Hybrid work
- Life insurance
- Long-term disability
- Medical insurance
- Short-term disability
- Student loan repayment
- Tuition reimbursement
- Vision insurance
- Wellness program
Skills/Tech-stack
CDC | Dagster | Data Modeling | Data Quality | Data Reconciliation | Deduplication | Docker | Historical Tracking | Idempotent Processing | Incremental loading | Kafka | MongoDB | Oracle | PostgreSQL | Python | Redpanda | SQL | SQL Server | Schema drift | Snowflake | Snowflake Streams | Snowflake Tasks | Snowpipe
Education
N/A
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R24d ago
-
Data Engineer USD 91K-114KANSI SQL | API | Amazon Redshift | Apigee | Azure401k with company match | Bereavement leave | Employee assistance program | Employee discount program | Family planning benefitsMid-level Full TimeRemote - Nationwide, United States R1d ago
-
Senior Data Engineer USD 111K-139KAPI Integration | Amazon Redshift | Apigee | Azure | BI Publisher401k match | Bereavement leave | Dental insurance | Employee assistance program | Employee discountsSenior-level Full TimeRemote - Nationwide, United States R1d ago
-
AI & Machine Learning Vice President USD 120K-220KAWS | Alteryx | Anomaly Detection | Azure | C#401k | Dental insurance | Disability insurance | Health insurance | Hybrid work modelExecutive-level Full TimeUnited States R1d ago
-
Staff Machine Learning Engineer, Applied Research USD 189K-389KC++ | Graph Neural Networks | Information Retrieval | Java | Language ProcessingSenior-level Full TimeSan Francisco, CA, US; Remote, US R1d ago
-
Director, Data Analyst/BI Engineer USD 138K-175KAWS | Access Control | Aggregated Event Measurement | Airflow | BigQuery401k match | Healthcare Dental Vision | PTO | Paid Holidays | Parental leaveExecutive-level Full TimeUnited States R1d ago
-
Sr Machine Learning Engineer I USD 150K-241KAutomated retraining | Bayesian Filter | CI/CD | Cloud Computing | Data Association401k match | Dental insurance | Emotional & mental wellness support | Employee resource groups | Fitness programsSenior-level Full TimeSeattle, Washington, United States R1d ago
-
Data Engineer USD 160K-200KAWS | Apache Spark | Azure | CI/CD | DBT401k match | Continuing education support | Employee assistance program | Flexible spending accounts | Generous paid time offSenior-level Full TimeAuburn Hills, MI, US R1d ago
-
Senior Data Engineer II, AI Native USD 148K-216KAmazon Web Services | Apache Airflow | Apache Spark | Azure | BigQuery401k matching | Dental insurance | Disability insurance | Employee assistance program | Equipment reimbursementSenior-level Full TimeRemote, USA R1d ago
-
API Design | AWS | Apache Airflow | Apache Flink | Apache KafkaDental insurance | Employee discounts | Employee equity | Health insurance | Pet insuranceSenior-level Full TimeRemote - United States R1d ago
-
Senior Data Engineer USD 115K-175KAWS Fargate | AWS Glue | AWS IAM | AWS Lambda | AWS MWAACharitable giving program | Company-Paid Holidays | Dental insurance | Flexible PTO | Life insuranceSenior-level Full TimeMassachusetts - Boston R2d ago
-
Autonomy | C++ | DDS | Data Ingestion | Edge Computing401k | Healthcare coverage | Paid Holidays | Paid time off | Phone stipendMid-level Full TimeSan Carlos - Hybrid R2d ago
-
Senior Analytics Engineer USD 148K-218KAutomation | BI tools | CCPA | CI/CD | DBTDental insurance | Equity | Medical insurance | Remote work | Vision insuranceSenior-level Full TimeRemote, USA ; Remote, Canada R2d ago
-
A/B | A/B Testing | API Development | AWS Bedrock | Access Control401k plan | Healthcare premium coverage | Paid Company Holidays | Paid time off | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R2d ago
-
Autonomy | C++ | CPU GPU | CPU GPU Debugging | DDS401k | Healthcare | Paid Holidays | Paid time off | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R2d ago
-
Amazon Web Services | Apache Airflow | Apache Flink | Apache Hive | Apache HudiContinuing education | Dental insurance | FSA | HSA | Health insuranceSenior-level Full TimeRochester, MN, United States R2d ago
-
AI Engineer II USD 125K-145KAPI Gateway | AWS | AWS Bedrock | AWS Lambda | Amazon APICareer advancement | Employee recognition | LinkedIn Learning Platform | Open PTO | Paid HolidaysMid-level Full TimeRemote US R2d ago
-
Apache Airflow | Apache Hudi | Apache Iceberg | Apache Kafka | Apache Kafka StreamsContinuing education | Dental insurance | Flexible spending account | Health savings account | Medical insuranceSenior-level Full TimeRochester, MN, United States R2d ago
-
Cleared Hybrid Data Engineer (5418) USD 103K-171KAPI strategy | AWS | AWS Glue | AWS Redshift | Access ControlHealth insurance | Paid leave | RetirementSenior-level Full TimeHanover, MD R2d ago
-
APIs | AWS | Agentic AI | Azure | Claude401k | Dental insurance | Fully remote US based | Generous PTO | HolidaysSenior-level Full TimeRemote USA, United States R2d ago
-
Principal Data Engineer USD 190K-210KAmazon Kinesis | Containerization | Helm | Infrastructure as Code | JuliaEquity | In person office hubs | PTO | Remote workSenior-level Full TimeBoston, MA / Remote R2d ago
-
Principal Software Engineer, Data USD 190K-210KAmazon Kinesis | Containerization | Data Engineering | Event Processing | HelmEquity | PTO | Remote workSenior-level Full TimeBoston, MA / Remote R2d ago
-
AWS | Agile | Apache Hive | Apache Spark | Cloudera Data PlatformRemote workSenior-level ContractLincoln, United States R2d ago
-
Middle Data Engineer (Azure Databricks) USD 160K-223KApache Spark | Azure Data | Azure Data Factory | Azure Databricks | Azure DevOpsCertification programs | Health insurance | Mentorship | Professional development opportunities | Relocation programMid-level Full TimeMadrid, MD, Spain R2d ago
-
Senior AI Engineer USD 155K-175KAWS | Agile | Azure | Computer Vision | GitCareer development | High responsibility | Remote workSenior-level Full TimeJersey City, New Jersey, United States R2d ago