Data Architect
Tasks
- Add monitoring and observability to pipelines
- Assess data pipelines and data flows
- Build data infrastructure with Infrastructure as Code
- Create documentation for integrations and systems
- Design and build production data pipelines
- Design dimensional models and data warehouses
- Develop ETL and ELT processes
- Develop reusable pipeline components
- Establish data standards and naming conventions
- Evaluate data architecture against best practices
- Handle semi structured data transformations
- Implement CI/CD pipelines and automated testing
- Implement Slowly Changing Dimensions
- Implement backup recovery and access controls
- Implement data quality checks and validation rules
- Implement error handling and retry logic
- Integrate data from APIs and operational systems
- Optimize pipeline performance and cost
- Perform database performance tuning
- Refactor legacy pipelines for reliability
- Troubleshoot pipeline failures and data discrepancies
Perks/Benefits
Skills/Tech-stack
API Integration | API Keys | API Management | Amazon Kinesis | Amazon Web Services | Apache Airflow | Apache Spark | CI/CD | CMDB | Capacity Planning | Cloud platform | CloudFormation | Configuration Management | Configuration Management Database | DBT | Data Mart | Data Modeling | Data Observability | Data Quality | Data Vault | Data Warehouse | Data integration | Database performance | Database performance tuning | Dimensional Modeling | Docker | ELT | ETL | Error Handling | Git | Google Cloud | Google Cloud Platform | Great Expectations | ITOM | ITSM | Indexing | Infrastructure as Code | JSON | Kafka | Kubernetes | Logging | Microsoft Azure | Monitoring | Monte Carlo | OAuth | Pagination | Partitioning | Performance Tuning | Prefect | PySpark | Python | Rate Limiting | Retry logic | Reverse ETL | SQL | ServiceNow API | Single Sign-On | Slowly Changing Dimensions | Soda | Terraform | Testing | Web Services | XML | “as-code”
Education
N/A
Roles
Related jobs
-
Amazon Web Services | Amazon Web Services (AWS) | Apache Spark | Azure | Data GovernanceCompetitive compensation | Flexible remote work | Travel as neededSenior-level Full TimeUnited States3h ago
-
Lead Data Engineer – Azure Databricks USD 150K-180KARIMA | Azure Databricks | Data Processing | Data analytics | Machine LearningOnsite workSenior-level Full TimeTampa, United States5h ago
-
HR Analytics & Technology Engineer USD 110K-204KAlteryx | Artificial Intelligence | Data Governance | Data Modeling | Data Quality401k matching | Generous time off | Onsite fitness center | Tuition reimbursementSenior-level Full TimeUS-Rhode Island-Providence5h ago
-
Cause analysis | Cloud Computing | Data Enrichment | Data Monitoring | Data Processing401k match | Dental insurance | Health insurance | Life insurance | Paid HolidaysSenior-level Full TimeNew York5h ago
-
Deloitte Technology - US Associate GenAI Engineer USD 80K-100K.Net Core | AWS | Agile | Angular | AzureMid-level Full TimeHermitage, Tennessee, United States6h ago
-
Senior-level Full TimeDallas, Texas, United States6h ago
-
Infra Engineer- US USD 120K-150KAWS | Amazon IAM | Azure | Cloud Cost Optimization | Cloud SecurityMid-level Full TimeNew York City, NY, US6h ago
-
As-a-Service | C++ | Cloud Functions | Cloud platform | Compute TechnologiesSenior-level Full TimeSunnyvale, CA, USA7h ago
-
Research Engineer, Gemini Latent Thinking, DeepMind USD 207K-300KAblation Study | Algorithm Development | Deep learning | Experiment design | Language ModelsSenior-level Full TimeCambridge, MA, USA; Mountain View, CA, …7h ago
-
Software Engineer III, Infrastructure, Cloud Storage USD 147K-211KC++ | Cloud Storage | Compute Technologies | Continuous integration | Data StorageSenior-level Full TimeSeattle, WA, USA7h ago
-
Manufacturing Test Engineer, Data Center USD 120K-172KBSCAN | Bash | Bring-up | Calibration | Correlation studyMid-level Full TimeAtlanta, GA, USA7h ago
-
Software Engineer, Compiling, Quantum AI USD 147K-211KC++ | Classical coding theory | Coding theory | Compiler architecture | CompilersMid-level Full TimeMountain View, CA, USA; Los Angeles, …7h ago
-
Research Scientist, Robotics, Embodied AI, DeepMind USD 147K-211KDeep learning | Language Models | Machine Learning | Python | Reinforcement LearningSenior-level Full TimeMountain View, CA, USA7h ago
-
Software Engineer, Decoding, Quantum AI USD 147K-211KC++ | Concurrency | Device Driver Development | Device driver | Driver DevelopmentMid-level Full TimeLos Angeles, CA, USA; Goleta, CA, …7h ago
-
C# | C++ | Co-design | Compiler technology | Computer ArchitectureSenior-level Full TimeSunnyvale, CA, USA7h ago
-
Kinetic Machine Learning Engineer USD 90K-123KClustering | Data Quality | Data quality profiling | Drift monitoring | Ensemble learning401k | Dental insurance | Educational assistance | Flexible savings account | Health savings accountMid-level Full TimeUnited States8h ago
-
AWS S3 | Almabase | Amazon Athena | Azure Data | Azure Data LakeMid-level Full TimeRemote within Texas, TX, US R10h ago
-
AI Data Platform Lead USD 164K-229KAWS | Airflow | Audit Logging | DBT | Data GovernanceFloating holidays | Wellness daySenior-level Full TimeUnited States12h ago
-
ALB | ALB/NLB | AWS | AWS CDK | AWS CloudSenior-level Full TimeDallas, Texas, United States15h ago
-
Statistics & Python Expert - Freelance AI Trainer USD 146K-146KCombinatorics | Graph theory | MATLAB | NumPy | Number theoryFlexible hours | Freelance opportunities | Project based workSenior-level FreelanceNew York, New York, United States … R18h ago
-
Statistics & Python Expert - Freelance AI Trainer USD 146K-146KCombinatorics | Graph theory | Mathematics | NumPy | Number theoryFreelance opportunity | Part-time project-based workSenior-level FreelanceFlorida, United States - Remote R18h ago
-
Statistics & Python Expert - Freelance AI Trainer USD 146K-146KC# | Combinatorics | Graph theory | MATLAB | NumPyFlexible hours | Part-time opportunities | Project based workSenior-level FreelanceTexas, United States - Remote R18h ago
-
Statistics & Python Expert - Freelance AI Trainer USD 146K-146KC# | MATLAB | NumPy | Pandas | PythonPart-time availability | Project based workSenior-level FreelanceMichigan, United States - Remote R18h ago
-
API Integration | Agent systems | Asynchronous processing | Chunking | Cost OptimizationCompetitive salary based on experience | High-impact role | Opportunity to scale AI systems | Strong ownershipMid-level Full TimeAustin, Texas, United States - Remote R18h ago
-
AI Full Stack Engineer - KS001 USD 160K-225KAds API | Agent systems | Anthropic Claude | Cost monitoring | EmbeddingsHigh-impact role | Strong ownershipMid-level Full TimeAustin, Texas, United States - Remote R18h ago