Senior Data Engineer (1043) - DataSF - Office of City Administrator
San Francisco, CA, United States
USD 145K-159K (estimate) Entry-level Full Time
Tasks
- Administer Snowflake data warehouse
- Build and maintain data infrastructure
- Collaborate with data scientists analysts and engineers
- Create data models and SQL optimization
- Design data pipelines
- Develop Infrastructure as Code with Terraform
- Ensure data quality and availability
- Implement access control and security policies
- Ingest and structure batch and streaming data
- Manage data monitoring and troubleshooting
- Perform performance tuning and cost optimization
- Use dbt for data transformation
- Write tests and maintain documentation
Perks/Benefits
Skills/Tech-stack
Airflow | Azure Data | Azure Data Factory | Batch Processing | BigQuery | DBT | Dagster | Data Factory | Data Governance | Data Modeling | Data Privacy | Data Security | Databricks | ELT | Infrastructure as Code | Kafka | Kinesis | Pandas | PySpark | Python | SQL | Snowflake | Snowpipe | Streaming Data | Terraform | “as-code”
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R4d ago
-
Apache Iceberg | CI/CD | Compaction | Cost Management | DBTEmployee Stock Purchase Plan (ESPP) | Flexible spending wallets | Health care coverage | Paid time off | Remote-first work environmentSenior-level Full TimeRemote US R5h ago
-
Data Engineer USD 40K-52KData Quality | Data quality monitoring | OCaml | Python | Quality monitoringGroup discussions | Mentorship | Project ownershipEntry-level Full TimeNew York, New York, United States8h ago
-
Batch Processing | Bias Mitigation | C++ | Data Processing | Experimentation401k employer match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R8h ago
-
AWS | Apache Iceberg | Application refactoring | Architectural Reviews | Cloud ComputingFlexible work culture | Mentorship | Work-life balanceSenior-level Full TimeNew York, New York, USA9h ago
-
Principal Engineer - Data Platform USD 221K-387KAccess Control | Apache Spark | Audit Logging | CDC | CompactionSenior-level Full TimeSanta Clara, California, United States R9h ago
-
Machine Learning Engineer, Relevance and Personalization USD 166K-195KA/B | A/B Testing | Apache Airflow | Apache Kafka | Apache SparkMid-level Full TimeUnited States10h ago
-
Forward Deployed AI Engineer USD 90K-110KAPI Integration | AWS | Access Management | Agentic AI | Automated testingEntry-level Full TimeChicago, IL, United States12h ago
-
Senior Software Engineer - Computational Platform USD 154K-200KAPI Development | AWS | Cloud Networking | Data Modeling | Data Pipelines401k plan | Commuter support | Company-provided laptop | Flexible paid time off | HolidaysSenior-level Full TimeRemote; Watertown, Massachusetts, United States R12h ago
-
Intern, AI Engineer (3 Months, Full-Time) USD 50K-50KAPI Integration | Agentic AI | Chunking | Code Automation | FastAPI401k with company match | Career advancement | Employee assistance program | Flexible time off | Health savings accountEntry-level Full Time InternshipScottsdale, Onsite12h ago
-
Senior Data Engineer USD 120K-126K3NF | API Design | API Management | Apache Airflow | Auditing401k match | Charitable Gift Matching | Community giving day | Dental insurance | Discounted personal insuranceSenior-level Full TimeWestern States or Midwest12h ago
-
Senior-level Full TimeDeerfield, IL, United States R12h ago
-
Data Engineer USD 115K-135KAPIs | Anomaly Detection | CI/CD | Cause analysis | Change Data Capture401k matching | Dental insurance | Health insurance | Paid Maternity Leave | Paid time offSenior-level Full TimeChicago, Illinois, United States14h ago
-
Artificial Intelligence Machine Learning USD 99K-200KC# | Data Engineering | Data Governance | Data Modeling | Data PipelinesDental insurance | Employee referral bonus | Health insurance | Identity theft protection | Legal protectionMid-level Full TimeAnnapolis Junction, Maryland14h ago
-
Embedded Software Engineer– Technical Lead USD 150K-175KARM | Agile | Build systems | C++ | CI/CD401k match | Health insurance | Paid time off | Relocation not availableSenior-level Full TimeWaltham, MA, United States15h ago
-
Senior Data Engineer - Databricks USD 180K-247KAWS | Access Control | Apache Spark | Azure | CI/CD401k match | Career development program | Corporate Benefit Program | Educational resources | Employee Referral Bonus ProgramSenior-level Full TimeDenver, CO R15h ago
-
AWS | AWS Config | Checkov | CloudTrail | CloudWatchDomestic travel up to 100 percent | On-call support | Remote workMid-level Full TimeRemote - USA R15h ago
-
Analytics Engineer II USD 76K-86KCI/CD | DBT | Data Architecture | Data Modeling | Data Quality401k match | Dental insurance | Disability insurance | EAP | Life insuranceMid-level Full TimeNorth Bethesda, MD16h ago
-
Senior-level Full TimeOrlando, FL, United States16h ago
-
AWS | Agile | Apache Spark | Azure | Big DataMid-level Full TimeUnited States16h ago
-
Data Engineer I USD 87K-137KAPIs | Clay | Data Governance | Data Modeling | Data Quality401k match | Conference attendance support | Employee resource groups | Flexible PTO | Flexible work scheduleMid-level Full TimeMorrisville, United States16h ago
-
Global Total Cost and TVM Data Analytics Engineer USD 128K-166KAlteryx | Artificial Intelligence | Benchmarking | Dashboard Development | Data ModelingChild care subsidy | Dental insurance | Employee resource groups | Flexible family care | Medical insuranceSenior-level Full TimeAllen Park, MI, United States16h ago
-
Generative AI Engineer USD 138K-213KAWS | Azure | Caching | Cloud platform | Databricks401k matching | Generous paid time off | Health, dental, vision benefits | Work from home equipment providedSenior-level Full TimeNew York, NY, United States17h ago
-
Data Visualization & Integration Specialist USD 115K-150KAPI Integration | Bill of Materials | Data Governance | ETL | ExcelHybrid work scheduleMid-level Full TimeMarietta, GA, United States18h ago
-
Azure Data | Azure Data Factory | Azure DevOps | Azure Synapse | Azure Synapse Analytics401k plan | Company holidays | Floating holidays | Paid time off | Remote workSenior-level Full TimeUnited States R18h ago