Senior Data Engineer
Tasks
- Build batch and streaming data lake architectures
- Collaborate with business users on reporting requirements
- Design develop maintain ETL pipelines on Databricks
- Develop and maintain Databricks pipelines and datasets
- Document development work and procedures
- Implement data cleansing procedures
- Implement data quality checks and validations
- Integrate Databricks with data sources and systems
- Maintain SSRS reports dashboards and visualizations
- Maintain legacy SSIS packages
- Migrate legacy systems to Databricks
- Optimize Databricks jobs for performance and cost
- Optimize data flow performance and minimize data latency
- Participate in data warehousing design and implementation
- Troubleshoot Databricks pipeline errors and performance issues
- Troubleshoot SSIS package errors and performance issues
- Troubleshoot SSRS report issues
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Azure | Azure Data | Azure Data Factory | Batch Processing | Data Cleansing | Data Factory | Data Lake | Data Quality | Data Warehousing | Databricks | Delta Lake | ETL | Functions | Python | SQL | SQL Server | SSIS | SSRS | Stored Procedures | Streaming Data | T-SQL
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Science in Computer Science | Bachelor of Science in Information Systems
Roles
Related jobs
-
Senior Artificial Intelligence/Machine Learning Engineer USD 125K-225KAPI Development | Context engineering | Guardrails | LLM Evaluation | Language ModelsBasic life insurance | Company-paid certifications | Healthcare | Language courses | Long-term disability insuranceSenior-level Full TimeUnited States8h ago
-
Senior Artificial Intelligence/Machine Learning Engineer USD 135K-227KAPI Development | Context engineering | Guardrails | Language Models | Large Language ModelsCompany-paid certifications | Conferences | Healthcare | Language courses | Life insuranceSenior-level Full TimeUnited States8h ago
-
Abstention Logic | Benchmarking | Claim Decomposition | Confidence scoring | Contradiction Detection401k | Medical | Paid sick leaveSenior-level Full TimeSouth San Francisco, United States9h ago
-
Data Engineer USD 109K-160KAccess Control | Agile | Amazon Web Services | Apache Airflow | Azure401k savings plan | Company holidays | Employee assistance program | Healthcare coverage | Paid time offMid-level Full TimeWoburn, MA, United States9h ago
-
Postgres Database Engineer USD 155K-232KAnsible | Automation frameworks | Backups and restores | Bash | CloudNativePGSenior-level Full TimeAustin, TX, United States9h ago
-
Principal Engineer - Supply Chain AI Solutions USD 125K-183KAWS | Agentic Workflows | Airflow | Authentication | AuthorizationSenior-level Full TimeNovi - Michigan, USA - Cabot …9h ago
-
Principal Engineer - Partner with Customers AI Solutions USD 125K-183KAWS | AWS Bedrock | Anthropic | Azure | Azure OpenAISenior-level Full TimeNovi - Michigan, USA - Cabot …9h ago
-
Principal Engineer - HR AI Solutions USD 125K-183KAWS | AWS Bedrock | Anthropic | Azure | Azure OpenAIBackground investigation and drug screen required | Employee discounts | Hybrid work | Training opportunitiesSenior-level Full TimeNovi - Michigan, USA - Cabot …9h ago
-
Principal Engineer - Finance AI Solutions USD 125K-183KAWS | Access Control | Airflow | Auditability | AzureEmployee discounts | Training opportunities | Wellness benefitsSenior-level Full TimeNovi - Michigan, USA - Cabot …9h ago
-
Audio ML Engineer (Research) USD 134K-196KAudio signal processing | Bias checking | Cloud deployment | DSP | Data AugmentationDisability coverage | Employee discounts | Health, dental, and vision plans | Life insurance | Paid sick leaveMid-level Full TimeNorthridge - California, USA9h ago
-
Embedded Software Engineer USD 97K-143KAgile | Agile Software Development | Agile software | Bash | Bug trackingEmployee discounts | Paid sick leave | Paid vacation and holidays | Training opportunities | Tuition reimbursementMid-level Full TimeCarlsbad – California, USA9h ago
-
Machine Learning Engineer USD 100K-120KAWS | Apache Spark | Classification | Data Quality | Data Quality ChecksHybrid work model | Onsite 1 to 2 days per week | Opportunities for growthMid-level Full TimeOakbrook Terrace, IL, US R10h ago
-
ClickHouse | Data Ingestion | Data Modeling | Data integration | ETLEntry-level Full TimeSan Jose, California, United States10h ago
-
Research Scientist / Software Engineer, LLM/Agent Platform - TikTok - Content Ecology AI Innovation & Platform USD 160K-230KAccelerator optimization | Deep learning | DeepSpeed | Distributed Systems | Evaluation FrameworksSenior-level Full TimeSan Jose, California, United States10h ago
-
Big Data | Data Modeling | Data Pipelines | Data Warehousing | PythonEntry-level Full TimeSan Jose, California, United States10h ago
-
C++ | Data Processing | Debugging | Entity Extraction | Fine TuningSenior-level Full TimeMountain View, CA, USA11h ago
-
Practice Customer Engineer, Data Analytics, Google Cloud USD 152K-221KAlooma | Apache Spark | Architecture Design | Batch Processing | Big DataSenior-level Full TimeMountain View, CA, USA; Boulder, CO, …11h ago
-
Dev and AI Infra Software Engineer, Releases, XR USD 207K-300KAI Software Engineering | AI software | Automation frameworks | Bug detection | Build AutomationSenior-level Full TimeSan Jose, CA, USA11h ago
-
Staff ML Engineer, Search Ads Auto-Bidding USD 207K-300KC++ | Deep learning | Experiment design | Keras | Language ModelsSenior-level Full TimeNew York, NY, USA11h ago
-
Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | CI/CDCollaborative team environment | Learning & Mentorship | On-the-job training | Onsite work opportunityEntry-level Full TimeIrvine, CA, US13h ago
-
Data Engineering Summer Intern (May-Aug '27) USD 52K-96KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageEntry-level InternshipIrvine, CA, US13h ago
-
AI Integration Engineer USD 130K-160KAI Search | AI Services | API Keys | AWS | AWS Bedrock401k match | Dental insurance | Flexible spending accounts | Health insurance | Paid HolidaysSenior-level Full TimeWashington, DC, US14h ago
-
Cyber Software Engineer Staff (Embedded) USD 120K-224KBlue Team | C++ | CISSP | Code Analysis | Cybersecurity401k match | Education assistance | Employee assistance program | Flexible spending accounts | Flexible work schedulesSenior-level Full TimeOrlando, FL, US17h ago
-
Senior Machine Learning Engineer, Payments USD 191K-223KAirflow | Anomaly Detection | Data Processing | Deep learning | Fine TuningSenior-level Full TimeRemote-USA R20h ago
-
AI Engineer (Early Career) USD 50K-120KAI Agents | JavaScript | Language Models | Large Language Models | Machine LearningEntry-level Full TimeSan Francisco Bay Area21h ago