Data Engineer Self Service Analytics and Real Time Data Platforms
Tasks
- Build AI ready data products for machine learning and GenAI
- Build near real time streaming data pipelines
- Create semantic layers and metrics frameworks
- Design governed data access patterns for AI and conversational analytics
- Design scalable ETL/ELT pipelines
- Design scalable analytics and AI optimized data models
- Develop curated data products
- Enable governed self service business data models
- Ensure data quality governance metadata and discoverability
- Implement data monitoring and observability
- Implement real-time dashboards
- Integrate data pipelines with ML platforms
- Optimize complex SQL queries
Perks/Benefits
- 401k
- Dental insurance
- Disability benefits
- Life insurance
- Medical insurance
- Paid time off
- Tuition assistance
- Vision insurance
Skills/Tech-stack
Airflow | Apache Kafka | Cloud Pub/Sub | Cloud platform | Data Governance | Data Modeling | Data Pipelines | Data Quality | Data Warehouse | Databricks | Distributed Systems | ELT | ETL | Embeddings | Event Driven | Event-driven architecture | Feature Engineering | GCP | GenAI | Google Cloud | Google Cloud Platform | Google Cloud Pub/Sub | Kafka Streams | Lakehouse | Machine Learning | Metadata Management | Observability | Pub/Sub | Python | SQL | Spark | Streaming Data | Streaming data pipelines | Superset | Vector Databases | Vertex AI
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R26d ago
-
ActiveMQ | Agile | Amazon Web Services | Apache Beam | Apache CassandraMid-level Full TimeSan Mateo, CA; Irving, TX; Sunnyvale, …1h ago
-
Senior-level Full TimeSan Bruno, CA; San Bruo, CA; …1h ago
-
Mid-level Full TimeSunnyvale, CA; San Bruno, CA; Dallas …1h ago
-
API | Amazon Web Services | Amazon Web Services (AWS) | Apache HDFS | Apache SparkRemote workMid-level Full TimeRemote, US; Sunnyvale, CA; Dallas, TX R1h ago
-
Adobe Analytics | Airflow | Apache Hive | Apache Spark | Data LineageSenior-level Full TimeSan Bruo, CA; San Bruno, CA; …1h ago
-
Mid-level Full TimeAtlanta, GA; San Bruno, CA; Dallas …1h ago
-
Mid-level Full TimeSunnyvale, CA; San Bruno, CA; Dallas …1h ago
-
Mid-level Full TimeSunnyvale, CA; San Bruno, CA; Dallas …1h ago
-
Mid-level Full TimeSunnyvale, CA; San Bruno, CA; Dallas …1h ago
-
Mid-level Full TimeSunnyvale, Ca; San Bruno, CA; Dallas …1h ago
-
Apache Spark | Avro | Azure Data | Azure Data Factory | Azure Data LakeRemote work optionMid-level Full TimeDallas / Philadelphia, USA; Bangalore, India; …1h ago
-
AWS | Amazon S3 | Angular | Anomaly Detection | Apache AirflowMid-level Full TimeIrving, TX; Sunnyvale, CA; Sunnyvale, CA; …1h ago
-
Apache Airflow | Apache Drill | Apache Hive | Apache Spark | ElasticsearchMid-level Full TimeCarlsbad, CA; San Bruno, CA; Dallas …1h ago
-
Mid-level Full TimeSan Bruno, CA; Sunnyvale, CA; Sunnyvale, …1h ago
-
Mid-level Full TimeSunnyvale, CA; San Bruno, CA; Dallas …1h ago
-
Mid-level Full TimeSunnyvale, CA; San Bruno, CA; Dallas …1h ago
-
Mid-level Full TimeSunnyvale, CA; San Bruno, CA; Dallas …1h ago
-
Apache Hadoop | Apache Hive | Apache Pig | Apache Spark | Business ObjectsMid-level Full TimeDallas, TX; San Bruno, CA; Dallas …1h ago
-
Mid-level Full TimeSan Bruno, CA; San Bruno, CA; …1h ago
-
Azure | Cloud platform | Confluence | Continuous Deployment | Continuous integrationMid-level Full TimeDallas, TX; San Bruno, CA; Dallas …1h ago
-
Apache Airflow | Apache Hive | Apache Spark | Backend Development | BigQueryMid-level Full TimeSunnyvale, CA; San Mateo, CA; San …1h ago
-
Cloud platform | Data Quality | Google Cloud | Google Cloud Platform | HadoopEntry-level Full TimeSan Bruno, CA; Sunnyvale, CA; San …1h ago
-
Apache Airflow | Apache Drill | Apache Hadoop | Apache Hive | Apache ImpalaMid-level Full TimeSunnyvale, CA; San Bruno, CA; Dallas …1h ago
-
Apache Airflow | Apache Spark | BigQuery | Cloud Dataflow | Data QualityEntry-level Full TimeSunnyvale, CA; Sunnyvale, CA; San Mateo, …1h ago