Solution Data Engineer Sr.

Tasks

• Administer Databricks environment
• Develop data processing logic with Python SQL and PySpark
• Handle metadata management with AWS Glue
• Implement data governance with Unity Catalog
• Ingest and transform Adobe marketing data
• Maintain Snowflake architecture and performance
• Maintain data operations in BigQuery
• Manage Delta Lake tables
• Manage code lifecycle with Git
• Manage data warehousing in Amazon Redshift
• Manage serverless querying with Athena
• Optimize data pipelines
• Own end to end data pipeline maintenance and reliability
• Schedule and monitor data pipelines with Airflow

Perks/Benefits

• Health benefits
• Remote work

Skills/Tech-stack

AWS | AWS Glue | Airflow | Amazon Athena | Amazon Redshift | Cloud platform | Databricks | Delta Lake | Git | Google BigQuery | Google Cloud | Google Cloud Platform | PySpark | Python | SQL | Snowflake | Unity Catalog