Solution Data Engineer Sr.
Tasks
- Administer Databricks environment
- Develop data processing logic with Python SQL and PySpark
- Handle metadata management with AWS Glue
- Implement data governance with Unity Catalog
- Ingest and transform Adobe marketing data
- Maintain Snowflake architecture and performance
- Maintain data operations in BigQuery
- Manage Delta Lake tables
- Manage code lifecycle with Git
- Manage data warehousing in Amazon Redshift
- Manage serverless querying with Athena
- Optimize data pipelines
- Own end to end data pipeline maintenance and reliability
- Schedule and monitor data pipelines with Airflow
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | Airflow | Amazon Athena | Amazon Redshift | Cloud platform | Databricks | Delta Lake | Git | Google BigQuery | Google Cloud | Google Cloud Platform | PySpark | Python | SQL | Snowflake | Unity Catalog
Education
N/A
Related jobs
- No jobs found.