Lead Software Engineer - Python, Databricks, AWS
GLASGOW, LANARKSHIRE, United Kingdom
GBP 72K-100K (estimate) Senior-level Full Time
Tasks
- Automate remediation of recurring issues
- Build data models for reporting
- Collaborate with business stakeholders on data strategies
- Design data pipelines
- Develop ETL/ELT workflows
- Develop in CI CD SDLC with Agile
- Document data flows and transformation rules
- Ensure data quality, security, and lineage
- Implement monitoring and alerting
- Manage production support and SRE for data pipelines
- Mentor engineers and ensure delivery standards
- Migrate legacy systems to cloud data warehouses
- Perform code reviews and debugging
- Transform and process large-scale data
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Step Functions | Agile | Airflow | Apache Spark | Avro | CI/CD | CSV | Data Lineage | Data Modeling | Data Quality | Data Security | Databricks | Delta Lake | ECS | ELT | ETL | JSON | Jenkins | Lambda | Parquet | PySpark | Python | S3 | SNS | SQL | SQS | SonarQube | Spinnaker | Step Functions
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Lead Software Engineer | Software Engineer
Related jobs
-
API | Accessibility | Alternative data | Backtesting | CloudSenior-level Full TimeLondon9h ago
-
Forward Deployed Engineer III, GCC (French, German) GBP 100K-163KAPIs | Computer Vision | CrewAI | Data Processing | GenAISenior-level Full TimeLondon, UK10h ago
-
Data Platform Engineer GBP 55K-75KApache Flink | Apache Kafka | Automation | BigQuery | Cloud SpannerHybrid work from homeEntry-level Full TimeStoke-on-Trent, England, United Kingdom R11h ago
-
Analytics Engineer GBP 65K-75KAWS | Airflow | Amazon QuickSight | Amazon Redshift | CI/CDCompany events | Employee resource groups | Extended parental leave | Health coverage | Learning pathsSenior-level Full TimeEdinburgh, United Kingdom12h ago
-
Solution Engineer - Data & AI GBP 61K-105KAzure Cosmos | Azure Cosmos DB | Azure Data | Azure Data Factory | Azure DatabaseSenior-level Full TimeUnited Kingdom, London, London12h ago
-
Data Science & AI Specialist GBP 28K-28KAWS | Advanced Analytics | Artificial Intelligence | Cloud platform | Data PipelinesDiscounted products | Gym membership | Holiday purchase scheme | Paid carer’s leave | Pension schemeMid-level Full TimeIpswich, GB, IP5 3RE12h ago
-
Data Engineering | Deep learning | Distributed Training | GPU | MLOpsEmployee discount | Employee sample sales | Flexible benefits allowance | Paid annual leave | Personalized learningSenior-level Full TimeLondon, England, United Kingdom13h ago
-
Senior Data Engineer - Data Science Platform GBP 60K-70KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageDiscretionary bonus scheme | Employee discount | Employee sample sales | Flexible benefits allowance | Paid annual leaveSenior-level Full TimeLondon, England, United Kingdom13h ago
-
Data Engineer - Core Engineering GBP 58K-71KAzure Data | Azure Data Factory | Azure DevOps | CI/CD | Data FactoryAnnual leave | Bank holidays | Employer pension matching | Hybrid work | Private medical insuranceEntry-level Full TimeEdinburgh, GB13h ago
-
Entry-level Full TimeGlasgow Campus, United Kingdom21h ago
-
AI Engineer, Enablement (London) GBP 110K-150KAWS | Agent architecture | Azure | Docker | Evaluation401k plan | Dental insurance | Flexible vacation | Meals on in office days | Medical insuranceMid-level Full TimeLondon23h ago
-
AI Research Engineer USD 200K-400KAgent systems | Constrained decoding | Data Curation | Evaluation Harness | Hugging FaceMid-level Full TimeNew York City23h ago
-
Staff Software Engineer, Inference GBP 116K-155KAutoscaling | BF16 | Batching | Benchmarking | C++401k matching | Casual work environment | Catered meals | Childcare support | Commuter benefitsSenior-level Full TimeLondon, England1d ago
-
C++ Algorithms Engineer GBP 50K-61KC++ | CI/CD | Configuration Management | Containerization | GPU integrationCycle and tech scheme | Employee assistance programme | Flexible working | Life insurance | Pension schemeSenior-level Full TimeManchester, England, United Kingdom1d ago
-
Embedded Software Engineer GBP 45K-60KAUTOSAR | Agile | C# | C++ | Embedded SystemsCycle and tech scheme | Employee assistance programme | Flexible working | Life insurance | Pension schemeMid-level Full TimeManchester, England, United Kingdom1d ago
-
Senior Full Stack AI Engineer GBP 92K-173KAI Agents | AWS | CI/CD | Distributed Systems | EmbeddingsCareer growth | Remote work | Travel 30 percentSenior-level Full TimeEurope, GB R1d ago
-
Embedded Software Engineer GBP 56K-61KAgile | C# | C++ | Debugging | GitCycle & Tech Scheme | Employee assistance programme | Flexible working | Life insurance | Pension schemeMid-level Full TimePlymouth, England, United Kingdom1d ago
-
Senior ML Engineer (AI Research/ Portability) GBP 190K-230KAPI Design | Agent systems | Authentication | CI/CD | Code reviewCareer growth | Collaborative culture | Flexibility | International environment | Learning opportunitiesSenior-level Full TimeAmsterdam, Netherlands; Remote - Europe; United … R1d ago
-
Senior ML Engineer (AI Research, Physical AI) GBP 92K-173KComputer Vision | Control | Deep learning | Distributed Computing | Distributed TrainingCareer growth and learning opportunities | Collaborative culture | Flexibility | International environment | OwnershipSenior-level Full TimeAmsterdam, Netherlands; Remote - Europe; United … R1d ago
-
Head of Data Platforms - (15 month FTC) GBP 65K-70KAWS | Cloud Computing | Continuous Improvement | DBT | Data EngineeringCoaching and development programs | Flexible working | Generous holidays | International working days | Learning allowanceExecutive-level Contract TemporaryLondon1d ago
-
Compute Infrastructure Platforms Lead Data Engineer GBP 84K-109KAccess Control | Data Modeling | Data Pipelines | Data archiving | Database BackupDiversity inclusion and respect cultureSenior-level Full TimeLONDON, United Kingdom1d ago
-
Senior Data Engineer I - Scalable Growth GBP 85K-115KAirflow | Batch pipelines | Big Data | Braze | Braze APISenior-level Full TimeLondon, United Kingdom1d ago
-
Software Engineer II - Databricks GBP 50K-68KAWS | Airflow | Amazon Kinesis | Amazon S3 | Apache SparkMid-level Full TimeBOURNEMOUTH, DORSET, United Kingdom1d ago
-
Backup and Recovery | C++ | Capacity Management | Compaction | DebuggingMentorship | On-call rotation | Open Source contributionSenior-level Full TimeLondon1d ago
-
Lead Data Scientist GBP 70K-90KAutomation | CI/CD | Feature Engineering | Hyperparameter Tuning | Infrastructure provisioningSenior-level Full TimeLondon, United Kingdom1d ago