Senior Data Engineer (1043) - DataSF - Office of City Administrator
San Francisco, CA, United States
USD 145K-159K (estimate) Entry-level Full Time
Tasks
- Administer Snowflake data warehouse
- Build and maintain data infrastructure
- Collaborate with data scientists analysts and engineers
- Create data models and SQL optimization
- Design data pipelines
- Develop Infrastructure as Code with Terraform
- Ensure data quality and availability
- Implement access control and security policies
- Ingest and structure batch and streaming data
- Manage data monitoring and troubleshooting
- Perform performance tuning and cost optimization
- Use dbt for data transformation
- Write tests and maintain documentation
Perks/Benefits
Skills/Tech-stack
Airflow | Azure Data | Azure Data Factory | Batch Processing | BigQuery | DBT | Dagster | Data Factory | Data Governance | Data Modeling | Data Privacy | Data Security | Databricks | ELT | Infrastructure as Code | Kafka | Kinesis | Pandas | PySpark | Python | SQL | Snowflake | Snowpipe | Streaming Data | Terraform | “as-code”
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R4d ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Bonus | Equity | Friendly atmosphere | Remote-friendly environmentMid-level Full TimeSeattle, WA, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Portfolio and LinkedIn application support | Remote-friendlyMid-level Full TimeChicago, IL, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous work culture | Flexible, hands-off management | Opportunity to build accessible learning products | Remote or distributed teamMid-level Full TimeSan Francisco, CA, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | GCPAsynchronous culture | Bonus opportunities | Flexible management approach | Remote workMid-level Full TimeDallas, TX, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Bonus | Competitive compensation | Equity | Remote-friendlyMid-level Full TimeWashington, DC, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Friendly work environmentMid-level Full TimeRedmond, WA, USA5h ago
-
Bash | Data Processing | Docker | GCP | LinuxAsynchronous culture | Bonus | Equity | Friendly work environment | Remote/distributed workMid-level Full TimeScottsdale, AZ, USA5h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous work culture | Career growth | Collaborative team | Remote-friendlyMid-level Full TimePlano, TX, USA5h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeMid-level Full TimeSan Antonio, TX, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Laid-back atmosphere | Remote-friendly, distributed teamMid-level Full TimeSan Mateo, CA, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous work culture | Hands off management approach | Portfolio and LinkedIn sharing during application | Remote distributed settingMid-level Full TimeSacramento, CA, USA5h ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous culture | Friendly laid-back atmosphere | Opportunity to make impactMid-level Full TimeCulver City, CA, USA5h ago
-
Artificial Intelligence | Bash | Cloud platform | Data Ingestion | Data Ingestion PipelinesAsynchronous work culture | Bonus | Equity | Friendly work environment | Remote or distributed workMid-level Full TimePasadena, CA, USA5h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Hand off management | Remote-friendly, distributed teamMid-level Full TimeIndianapolis, IN, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | GCPAsynchronous culture | Bonus | Equity | Flexible, hands-off management | Remote-friendly environmentMid-level Full TimeNaperville, IL, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Flexible remote setting | Friendly work environmentMid-level Full TimePrinceton, NJ, USA5h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous work culture | Friendly work environmentMid-level Full TimeProvidence, RI, USA5h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerFlexible asynchronous culture | Friendly work environment | Impactful product | Remote/distributed workMid-level Full TimeBaltimore, MD, USA5h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Bonus | Equity | Flexible management approach | Remote-friendly, distributed teamMid-level Full TimeBend, OR, USA5h ago
-
Bash | Cloud infrastructure | Cloud platform | Data Ingestion | Data ProcessingAsynchronous culture | Friendly work environment | Opportunities for high impact on productMid-level Full TimeTucson, AZ, USA5h ago
-
Bash | Data Processing | Data pipeline | Docker | GCPAsynchronous culture | Career growth | Friendly work environment | Remote workMid-level Full TimeRiverside, CA, USA5h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeMid-level Full TimeBaton Rouge, LA, USA5h ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous work culture | Friendly laid-back atmosphere | Opportunity for impact | Remote friendly distributed settingMid-level Full TimeGainesville, FL, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | GCPAsynchronous culture | Entrepreneurial team | Flexible work environment | Remote or distributed work | Supportive management approachMid-level Full TimeTulsa, OK, USA5h ago