GCP Data Engineer
Chicago, IL; Chicago, Il; Chicago, IL; Chicago, IL; Chicago, IL; Chicago, IL
R
A USD 123K-178K (estimate) Mid-level Full Time
Tasks
- Adopt CI/CD and DevOps best practices
- Build data models for Data Lakes and Warehouses
- Collaborate with analytics and machine learning teams
- Design batch data pipelines
- Design streaming data pipelines
- Develop SQL for transformation and validation
- Ensure fault tolerant data pipelines
- Implement data quality checks and anomaly detection
- Implement medallion bronze silver gold architecture
- Implement secure data access with IAM
- Ingest data using Pub Sub Dataflow and Cloud Storage
- Migrate legacy ETL to dbt
- Optimize BigQuery schemas and partitioning
- Optimize pipelines for performance and scalability
- Process data with Python and Apache Beam
- Set up monitoring logging and alerting
- Transform data with BigQuery dbt and Python
- Use Git for version control and automated deployments
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Beam | Apache Spark | BigQuery | CI/CD | Cloud Build | Cloud Dataflow | Cloud Storage | Cloud platform | DBT | DevOps | Git | GitHub Actions | Google Cloud | Google Cloud Platform | IAM | Jenkins | Pub/Sub | Python | SQL | Talend
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R26d ago
-
Senior Data Engineer (Remote) USD 108K-150KData Architecture | Data Lake | Data Modeling | Data Quality | Data VisualizationRemote workSenior-level Full TimeUnited States R2h ago
-
Amazon EKS | Amazon EMR | Amazon Redshift | Amazon S3 | Amazon Web ServicesComprehensive insurance | Diversity and inclusion | Flexible work arrangements | Hybrid remote workMid-level Full TimeReston, VA R2h ago
-
Amazon Redshift | Amazon Web Services | Apache Airflow | CI/CD | CTEComprehensive insurance | Diversity and inclusion | Flexible work arrangementsMid-level Full TimeReston, VA; Reston, VA R2h ago
-
Big Data | ETL | Elasticsearch | JSON | Jupyter401k match | PTO | Paid medical | Training supportSenior-level Full TimeSterling, VA R2h ago
-
Big Data | Continuous Monitoring | Cybersecurity | Data Translation | Data extraction401k match | Full paid medical | Paid time off | Training supportSenior-level Full TimeSterling, VA R2h ago
-
Big Data | EML | ETL | Elasticsearch | JSON401k match | Full paid medical | Paid time off | Training fundingSenior-level Full TimeSterling, VA; Annapolis Junction, MD R2h ago
-
Agile | Amazon S3 | Amazon SQS | Apache NiFi | Bash401k match | Fully paid medical | PTO | Training supportSenior-level Full TimeSan Antonio, TX R2h ago
-
Agile | Amazon S3 | Amazon SQS | Apache NiFi | Bash401k match | Fully paid medical | Hybrid work | Onsite clearance support | Paid time offSenior-level Full TimeColumbia, MD R2h ago
-
Data Governance | Data Modeling | Docker | ETL | JSON401k match | Fully paid medical | Paid time off | Training supportSenior-level Full TimeColumbia, MD R2h ago
-
Data Governance | Data Modeling | Data Quality | Data schemas | Docker401k match | Fully paid medical | Paid time off | Training supportSenior-level Full TimeSan Antonio, TX R2h ago
-
Continuous Monitoring | Data Translation | Data labeling | ETL | Elasticsearch401k match | Fully paid medical | Paid time off | Training supportSenior-level Full TimeSan Antonio, TX R2h ago
-
Data Engineer USD 110K-144KAPI Development | Azure | Data Security | Distributed Computing | DockerMid-level Full TimeChicago, Ill; Remote, Remote; Remote, Remote; … R3h ago
-
Technical Solutions Architect II -- Data Engineer USD 150K-180KAmazon Redshift | Apache Airflow | Apache Kafka | Azure Data | Azure Data Factory401k matching | Dental insurance | Employee assistance program | Health insurance | Medical leaveMid-level Full TimeRemote - Nationwide, United States R4h ago
-
Mid-level Full TimeChicago, IL; Chicago, IL; Chicago, IL R4h ago
-
Apache Spark | Distributed Systems | Documentation | Hadoop | Java401k | Dental insurance | Disability insurance | Flexible vacation policy | Free coffeeSenior-level Full TimeBoston, MA; Boston, MA; Boston, MA; … R4h ago
-
Apache Hadoop | Apache Spark | Data Mining | Distributed Computing | Java401k | Commuter benefits | Dental insurance | Disability insurance | Employee wellness benefitsSenior-level Full TimeBoston, MA; Boston, MA; Cambridge, MA; … R4h ago
-
Amazon Redshift | Apache Airflow | Claims data | Cloud Computing | Common Data ModelCollaborative team | Remote workSenior-level Full TimeBoston, MA; Boston, MA; Wellesley, MA; … R4h ago
-
Access Control | Azure | Azure Backup | Azure CLI | Azure DataMentorship | Remote workSenior-level ContractSaint Paul, United States R6h ago
-
Edge Computing AI Engineer USD 100K-105KC++ | DSP | Edge inference | Embedded Systems | Federated LearningCareer growth | Remote workMid-level Full TimeUnited States - Remote R20h ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HudiCareer growth | Direct W2 employment | Remote workSenior-level Full TimeUnited States - Remote R20h ago
-
Machine Learning Data Engineer USD 80K-100KApache Beam | CI/CD | Data Lineage | Data Modeling | Data QualityDirect W2 employment | Remote workSenior-level Full TimeUnited States - Remote R20h ago
-
Data Platform Engineer USD 100K-150KAutoscaling | C++ | Capacity Planning | Distributed Systems | FinOpsRemote workSenior-level Full TimeUnited States - Remote R20h ago
-
Robot Application Developer USD 100K-150KC++ | Computer Vision | Control Systems | Gazebo | IMUCareer growth | Remote workSenior-level Full TimeUnited States - Remote R20h ago
-
Kafka Platform Engineer USD 130K-180KAWS MSK | Access Control | Access Control Lists | Access Control Lists ACLs | AnsibleSenior-level Full TimeUnited States - Remote R20h ago