Senior Data Engineer - Databricks
Tasks
- Assess ETL workflows BI tools and reports
- Author and optimize SQL queries
- Build batch and streaming data pipelines
- Craft database and data warehouse solutions
- Design and build data pipelines and data products
- Develop search and retrieval solutions
- Evaluate feasibility identify risks and recommend solutions
- Lead data environment migration
- Operate in regulated cloud environments with security and governance
- Optimize distributed processing performance
- Support Agile software development
- Support integration of machine learning workflows
- Translate business needs into data solutions
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Apache Spark Structured Streaming | Data Governance | Data Pipelines | Data Warehousing | Databricks | Delta Lake | Distributed Processing | GenAI | Lakehouse | MLflow | Machine Learning | Public Sector | Public Sector Security | Python | Query Optimization | SQL | Spark Structured Streaming | Structured Streaming
Education
Roles
Related jobs
-
Acceptance Testing | Business Objects | Cause analysis | Data Visualization | Data WarehousingOn-site work | Security clearance supportSenior-level Full TimeReston, VA; Information Technology5h ago
-
Amazon Redshift | Data Modeling | Data Quality | Data Warehouse | Data Warehouse ArchitectureOnsite work at client site | Paid on time | Potential project extensions | Support from consultantsMid-level Full TimeJuno Beach, FL; Juno Beach, FL; …5h ago
-
Lead Software Engineer / Data Analyst - (MUST be eligible for a TS/SCI security clearance) - 1822 A USD 115K-185K.NET | AWS | Angular | Automated testing | CI/CDEvery other Friday off on 9 80 schedule | Onsite work | Relocation assistanceSenior-level Full TimePatrick Space Force Base, FL6h ago
-
API Design | BigQuery | CI/CD | Chunking | Cloud BuildHybrid work | Local preferred | Onsite 4 days per weekMid-level Full TimeDearborn, United States R8h ago
-
Agent Orchestration | Agentic AI | Airflow | Autogen | BentoMLSenior-level Full TimeCharlotte, United States9h ago
-
Agile | Azure Databricks | Azure Logic | Azure Logic Apps | Azure SynapseEntry-level Full Time InternshipPella, IA, United States9h ago
-
PROJECT - Lead Analytics and Insights Engineer II USD 137K-229KAcceptance Testing | Agile | Artificial Intelligence | Azure DevOps | ConfluenceSenior-level Full TimeRosslyn, Virginia, United States9h ago
-
AI Foundry | Agent systems | Agentic AI | Amazon Web Services | Artificial IntelligenceMid-level Full TimeMechanicsburg, Pennsylvania, United States9h ago
-
Intelligent Automation Manager | TS/SCI USD 137K-229KAWS Bedrock | AWS Textract | Agile | Amazon Web Services | Data ArchitectureSenior-level Full TimeRosslyn, Virginia, United States9h ago
-
Senior/Staff LLM Application Engineer - Data Application USD 108K-190KAgentic Systems | Benchmarking | Data Querying | Data analytics | Fine TuningSenior-level Full TimeSan Jose, California, United States10h ago
-
Action Conditioned World Models | C plus plus | Computer Vision | Deep learning | Dexterous ManipulationOpen science contributionsEntry-level Full TimeMenlo Park, CA10h ago
-
AI workflows | Anomaly Detection | Data Architecture | Data Lakehouse | Data StorageSenior-level Full TimeMenlo Park, CA10h ago
-
Research Engineer, Frontier AI, Incubation, DeepMind USD 174K-252KAlgorithm Design | C++ | Compute Optimization | Data Structures | Distributed SystemsEntry-level Full TimeMountain View, CA, USA11h ago
-
C++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA; Pittsburgh, PA, …11h ago
-
Artificial Intelligence | C++ | CSS | Data Storage | Distributed ComputingHealth insurance | Paid time off | Parental leave | Retirement plansSenior-level Full TimeSunnyvale, CA, USA11h ago
-
Deep learning | Generative AI | JAX | Machine Learning | Model DeploymentSenior-level Full TimeMountain View, CA, USA11h ago
-
Senior Data Engineer, gTech Risk USD 156K-226KApache Flume | Apache Spark | Data Modeling | Data Pipelines | Data VisualizationSenior-level Full TimeChicago, IL, USA11h ago
-
Staff Software Engineer, Data Protection USD 207K-300KAccess Control | Big Data | Big data analytics | C# | C++Senior-level Full TimeNew York, NY, USA11h ago
-
Software Engineer, AI/ML, Computer Vision, XR USD 147K-210K3D Reconstruction | Android | Auto-exposure | C++ | Computer VisionMid-level Full TimeSan Jose, CA, USA11h ago
-
Senior Software Engineer, Data Engineering, Cloud AI USD 174K-252KAccess Control | Alerting | Artificial Intelligence | Batch Processing | Cloud StorageSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA11h ago
-
Data Engineer USD 220K-226KC++ | Data Exploration | Data Modeling | Data Visualization | Data pipelineHybrid scheduleMid-level Full TimeMountain View, CA, USA R11h ago
-
Software Engineer, Generative AI, Shopping on Gemini USD 147K-210KBackend Development | C++ | Computer Vision | Data Preparation | Data ProcessingMid-level Full TimePittsburgh, PA, USA; Mountain View, CA, …11h ago
-
Senior Software Engineer USD 193K-253KC++ | Cause analysis | Debugging | Machine Learning | Model DeploymentHybrid scheduleSenior-level Full TimeNew York, NY, USA R11h ago
-
AI Solution Security Engineer USD 147K-210KAccess Control | C++ | Cloud Security | Data Flow | Data Flow SecurityMid-level Full TimeChicago, IL, USA11h ago
-
Software Engineer USD 149K-175KAlgorithms | C# | C++ | Cause analysis | Code ReviewsBenefits | Hybrid scheduleEntry-level Full TimeMountain View, CA, USA R11h ago