Big Data & Cloud Infrastructure Engineer

Description

We are looking for an experienced Big Data & AWS Cloud Platforms Specialist to support, manage, and optimize our data processing infrastructure built on Databricks and AWS.

This role is critical to maintaining the performance, security, and reliability of our data platforms and ensuring seamless operation of analytics and data engineering workloads.

You will also play a key role in CI/CD, release management, platform governance, and cloud infrastructure support.

Main Accountabilities

• Support, monitor, and troubleshoot Production/QA/DEV environments and workloads.
• Ensure high platform availability and resolve operational issues quickly
• Administer, maintain, and tune AWS infrastructure, Databricks and Spark clusters for performance and cost efficiency
• Manage workspace configuration, compute resources, and user access
• Conduct platform health checks, monitor logs, and perform root cause analysis
• Participate in the SDLC by reviewing code, testing changes, and maintaining documentation
• Maintain CI/CD pipelines to support continuous integration and deployment of data workflows
• Coordinate release management, ensuring smooth and secure deployments across environments
• Support AWS networking and cloud infrastructure, including VPCs, subnets, route tables, and security groups
• Work with infrastructure teams to ensure platform scalability and connectivity across systems
• Implement and manage access controls using IAM, Unity Catalog, and other governance tools
• Ensure compliance with organizational security policies and regulatory standards
• Monitor data access logs and perform regular security reviews

Requirements

Education, Skills and Experience

• BSc or MSc degree in Computer Science, Data Engineering, or a related field
• At least 2 years of experience in big data operations, platform support, or and cloud data infrastructure
• Proficient with AWS , Databricks, Delta Lake, and Apache Spark in production environments
• Strong skills in Python, and SQL
• Experience with CI/CD, release management, and modern SDLC practices
• Solid understanding of AWS networking and cloud infrastructure fundamentals
• Experience with platform-level monitoring, troubleshooting, and performance tuning
  

Preferred Skills

• Databricks certifications (e.g., Data Engineer Associate/Professional)
• Experience with Airflow, or other orchestration and IaC tools
• Knowledge of Unity Catalog or other data governance platforms
• Exposure to regulated industries and enterprise security compliance

Competencies

• Excellent interpersonal skills with ability to communicate effectively at all levels
• Strong analytical and problem-solving capabilities
• Results-oriented mindset
• Effective time management with ability to multi-task and prioritize work
• Consistent application of software development practices and methodologies