Sr. Databricks Azure Data Engineer
Tasks
- Build automated Parity Test Suite for data validation
- Design data solutions on Spark and Flink
- Develop Python SDK for Jupyter Notebooks
- Document solutions and workflows
- Enforce data governance, security, and quality standards
- Implement data quality rules
- Maintain CI/CD pipelines
- Manage Azure and Databricks resources
- Monitor and optimize data pipelines
- Participate in agile activities
- Transform and prepare data using SQL, Python, Java
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Apache Spark internals | Azure DevOps | Azure Security | Azure ecosystem | CI/CD | Data Governance | Data Modeling | Data Quality | Data Warehousing | Databricks | Dataframes | Deequ | Delta Lake | GitHub | Great Expectations | JUnit | PySpark | Pytest | Python | Spark internals
Education
N/A
Roles
Related jobs
-
Access Control | Airflow | Amazon Redshift | BigQuery | CI/CDRemote work flexibilitySenior-level Full TimePakistan - Remote R15h ago
-
Attribution tracking | Automation | Business Suite | Close CRM | Conversion APIRemote workEntry-level Full TimePakistan - Remote R2d ago
-
Backend AI & Data Pipeline Engineer USD 70K-98KAWS Fargate | AWS Lambda | AWS Step Functions | Amazon Bedrock | Amazon ECSFlexible working hours | Fully remote | Hands-on engineering experience | Long-term progression | Opportunity to build real product impactEntry-level Full Time InternshipIslamabad, Islamabad Capital Territory, Pakistan R5d ago
-
AWS Bedrock | AWS EC2 | AWS ECS | Amazon S3 | Anomaly DetectionRemote work | VPN AccessSenior-level Full TimeLahore, Pakistan R8d ago
-
AI Engineer USD 130K-195KAsynchronous programming | Background tasks | Batching | Caching | Content FilteringAnnual company retreat | Exposure in AI | Fitness reimbursements | Global team experience | Health insuranceMid-level Full TimePakistan - Remote R9d ago
-
Data Engineer USD 135K-165KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure SynapseHealth insurance | Paid time off | Training and development | Work from homeSenior-level Full TimePakistan - Remote R10d ago
-
AWS | Azure | Bias Mitigation | CI/CD | EmbeddingsRemote workSenior-level Full TimeLahore, Pakistan R15d ago
-
Mid-level Full TimePakistan - Remote R15d ago
-
Applied AI Engineer (Automation) USD 150K-220KAPIs | AWS | Amazon Bedrock | Asynchronous programming | AzureFull-time | Remote workSenior-level ContractPakistan - Remote R16d ago
-
Senior Data Engineer INR 2000K-2000KACID transactions | Airflow | Apache Spark | Automated testing | BackfillingFull-time employment | Hybrid workSenior-level Full TimePakistan - Remote R22d ago
-
Senior Computer Vision Engineer USD 165K-240KC++ | Deep learning | Image Processing | Keras | Machine LearningHealth insurance | OPD | Paid time off | Training and developmentSenior-level Full TimePakistan - Remote R26d ago
-
AI tools | APIs | Automation tools | Data Pipelines | IntegrationsFast-moving environment | Global compensation | Mentorship | Ownership | Real business systemsEntry-level Full TimeLahore, Pakistan R1mo ago
-
Infrastructure Engineer (Data) EUR 40K-66KAnsible | Bash | Cassandra | Kafka | KubernetesGrowth opportunities | Learning opportunities | Remote workMid-level Full TimePakistan; India; Slovakia; Columbia; Brazil; Argentina; … R1mo ago
-
DevOps Engineer USD 125K-185KAnsible | ArgoCD | Bash | Cloud Computing | DevOpsFull-time | Remote workSenior-level ContractPakistan - Remote R1mo ago
-
.Net 8.0 | AI systems | API Integrations | ASP.Net Core | C#Fast-paced environment | Ownership of technical systems | Remote work | Startup environment | Team leadership opportunitySenior-level Full TimePakistan - Remote R1mo ago
-
.Net 8.0 | AI Pipelines | API Integrations | ASP.Net Core | C#Senior-level Full TimePakistan - Remote R1mo ago