Senior Data Engineer

Required Experience and Skills:

We are seeking a highly skilled and motivated Data Engineer with expertise in SQL development, CI/CD, and custom development for database and ETL solutions. This role is ideal for candidates with hands-on experience in DevOps practices, Flyway-based database versioning, and data engineering within Oil & Gas operations (upstream, midstream, and downstream).

Minimum 9+ years experience

• TSQL
• SQL Server
• SSIS
• Oil and Gas Industry experience

 Nice to have:

• Database Administration
• Python

Key Responsibilities:

Database Architecture & Design

• Design scalable and efficient SQL database architectures tailored to upstream, midstream, and downstream oil and gas operations.
• Develop and maintain logical and physical data models to support data warehousing and business reporting needs.

SQL Development & Optimization

• Write complex T-SQL code including stored procedures, views, functions, and triggers.
• Optimize large-volume queries for production logs, telemetry, and field operations to improve performance and reliability.

Data Integration & ETL Pipelines

• Build and manage robust ETL/ELT pipelines for structured and unstructured data from sources such as SCADA, historians, ERP systems (e.g., SAP), CMMS, drilling reports, and IoT devices.
• Ensure data validation, integrity, and transformation align with business and regulatory standards.

CI/CD & DevOps Integration

• Develop and manage Flyway-based SQL migration scripts to version control database changes.
• Integrate database deployment automation into CI/CD pipelines using tools like Azure DevOps or GitHub Actions.
• Ensure consistent deployment processes across development, test, staging, and production environments.

Custom Development & Platform Support

• Leverage Visual Studio and .NET tools to build supporting utilities or integrate with existing applications.
• Provide performant SQL backends for reporting platforms such as Power BI, Spotfire, or Tableau.

Collaboration & Support

• Collaborate with data analysts, architects, and business stakeholders to translate operational requirements into scalable data solutions.
• Support application teams and business users during feature rollouts, deployments, and upgrades.

Documentation & Best Practices

• Document database structures, processes, and technical design decisions.
• Enforce and advocate for SQL development best practices, code reviews, and maintainability standards.

Required Qualifications:

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
• 8+ years of experience in SQL development and database architecture.
• Strong command of Microsoft SQL Server, SSIS, and T-SQL.
• Experience in Oil & Gas domains (e.g., well lifecycle, production volumes, pipeline operations).
• Hands-on experience with Flyway for database versioning and change management.
• Proficiency in Visual Studio and custom software development tools.
• Familiarity with data modeling tools (ER/Studio, Erwin) and version control systems (Git).

Preferred Qualifications:

• Experience with cloud data platforms (Azure SQL, AWS RDS, Snowflake).
• Understanding of WITSML standards and Oil & Gas-specific data structures.
• Exposure to Python, PowerShell, or other scripting languages for data automation.
• Experience with data governance, MDM, and data quality frameworks.

Soft Skills:

• Analytical mindset with strong problem-solving capabilities for data-intensive operations.
• Excellent communication skills, with the ability to explain technical solutions to business users.
• Team-oriented and collaborative, working efficiently with cross-functional groups.
• Detail-oriented with a focus on data accuracy, validation, and compliance.
• Self-driven and accountable, capable of managing workload independently in fast-paced environments.
• Flexible and adaptable, able to pick up new tools and domains quickly as project needs evolve.
• Proactive thinker who anticipates potential issues and proposes effective solutions.
• Strong organizational and time management skills with the ability to juggle multiple priorities effectively.

Datavail is a leading provider of data management, application development, analytics, and cloud services, with more than 1,000 professionals helping clients build and manage applications and data via a world-class tech-enabled delivery platform and software solutions across all leading technologies. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes, and is an AWS Advanced Tier Consulting Partner, a Microsoft Solutions Partner for Data & AI and Digital & App Innovation (Azure), an Oracle Partner, and a MySQL Partner.