Technical Lead / Solution Architect
USD 180K-247K (estimate) Senior-level Full Time
Tasks
- Collaborate with data scientists product managers business analysts
- Configure cluster settings
- Create and maintain architecture diagrams and runbooks
- Create automated testing strategies
- Define coding standards
- Define end-to-end data architecture
- Design data ingestion storage processing serving
- Design migration from on premise to Azure Databricks
- Implement CI/CD pipelines
- Implement data governance security compliance
- Integrate with DevOps for monitoring and reliability
- Lead technical delivery and implementation
- Mentor engineers in best practices
- Optimize Apache Spark jobs
- Provide hands on guidance for data pipelines
- Select Azure services and open source tools
- Translate business requirements into technical specifications
- Tune Delta Lake performance
Perks/Benefits
- N/A
Skills/Tech-stack
Advanced SQL | Agile | Apache Kafka | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure Key Vault | Azure Monitor | Azure Stream | Azure Stream Analytics | Azure Synapse | Azure Synapse Analytics | Bicep | CI/CD | Data Factory | Data Lake Storage Gen2 | Databricks | Delta Lake | Docker | Event Driven | Event Hubs | Event-driven architecture | GitHub Actions | Infrastructure as Code | IoT Hub | Kappa Architecture | Key Vault | Kubernetes | Lambda Architecture | MLOps | MLflow | Medallion Architecture | Pandas | Parquet | PySpark | Python | Scala | Scrum | Stream Analytics | Synapse Analytics | Terraform | Unity Catalog | “as-code”
Education
N/A
Related jobs
-
Architect Data Quality - Morning - Fulltime USD 114K-182KAnomaly Detection | BCBS 239 | Cause analysis | Cloudera | DAMA-DMBOKSenior-level Full TimeLahore, Karachi, Islamabad4d ago
-
Artificial Intelligence | Automated testing | CI/CD | Cloud Computing | Code reviewSenior-level Full TimeLahore R8d ago
-
Ingeniero de datos Bilingüe ( Databricks- AWS) USD 180K-247KAPI | AWS CloudFormation | AWS Glue | AWS IAM | AWS LambdaCoaching | Continuous improvement culture | Learning ecosystem | MentorshipSenior-level Full TimeLatambar, Pakistan21d ago
-
APIs | Agile | Application Programming | Application Programming Interfaces | AutomationSenior-level Full TimeLahore, Pakistan1mo ago
-
Agentic AI Solution Architect INR 2000K-5000KAWS IAM | AWS Lambda | Access Control | Agent Orchestration | Agent guardrailsOnsite workSenior-level Full TimeLahore, Pakistan R1mo ago
-
Associate Team Lead- Data Engineer INR 1500K-3000KAWS Glue | AWS Redshift | Amazon Web Services | Apache Airflow | AzureSenior-level Full TimeLahore, Punjab, Pakistan1mo ago
-
Senior Data & AI Architect USD 165K-221KAI RMF | Access Control | Airflow | Amazon SageMaker | Apache SparkSenior-level Full TimeKarachi/ Lahore/ Islamabad1mo ago
-
AI/ML | Azure Data | Azure Data Factory | Azure Data Lake | Azure SynapseSenior-level Full TimeKarachi, Pakistan1mo ago
-
DevOps Architect INR 2500K-4000KAmazon Linux | Ansible | Azure | Bash | CI/CDExchange program | Health insurance | Life insurance | OPD | Paid time offSenior-level Full TimeLahore, Punjab, Pakistan - Remote R1mo ago