Data Engineer - Capital Markets
Tasks
- Automate Databricks deployments across environments
- Build data pipelines
- Build data views for reporting and AI modeling
- Configure AWS data cataloging and storage
- Design data lakehouse architecture
- Develop and orchestrate data workflows
- Implement data governance and security
- Implement data lineage and auditing
- Implement real time reporting
- Integrate Databricks with AWS services
- Manage compute costs
- Optimize Delta Lake storage
- Set fine grained access controls
- Tune Spark clusters
Perks/Benefits
Skills/Tech-stack
AWS CodePipeline | AWS Glue | AWS IAM | Amazon MSK | Amazon S3 | Amazon Web Services | Apache Kafka | Apache Spark | Asset bundles | CI/CD | Data Lineage | Databricks | Databricks (asset) | Databricks Asset Bundles | Databricks Jobs | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | GitHub Actions | GitLab CI | GitLab CI/CD | PySpark | SQL | Streaming | Terraform | Unity Catalog | Web Services | Z-ordering
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Agile | Angular | Azure DevOps | CSS | Cloud DataHybrid RemoteMid-level Full TimeTampa R3h ago
-
AWS Glue | Amazon RDS | Amazon Redshift | Amazon S3 | Artificial Intelligence401k matching | Certification programs | Disability insurance | Employee resource groups | Fertility coverageEntry-level Full TimeRichmond, VA, United States R6h ago
-
Senior Full Stack & AI Engineer USD 100K-135K.NET | AI Foundry | Agile | Angular | Azure AIContinuous learning | Flexible schedule | Hybrid work modelSenior-level Full TimeMadrid, MD, Spain R8h ago
-
Senior MLOps Engineer USD 165K-250KAI Pipelines | AI endpoints | API Development | Audit Logs | AutoscalingContinuous learning opportunities | Flexible working arrangements | Health equity mission impact | Remote or hybrid optionsSenior-level Full TimeUnited States - Remote R1d ago
-
Data Engineer (2258) USD 119K-150KAWS | Azure | Data Quality | Data Security | Data Visualization401k | Dependent care | Discounted tickets | Entertainment Perks Programs | Flexible work schedulesMid-level Full TimeNew York, New York, United States R1d ago
-
Senior Quantum Software Engineer, Compiler [410] USD 125K-187KC# | C++ | CI/CD | CUDA-Q | CirqBenefits | Mentorship | Team collaborationSenior-level Full TimeHybrid (New Haven, CT, US) R1d ago
-
AWS | Azure | CI/CD | Deep learning | Evaluation methodology401k | Equity options | Health, dental, and vision | Relocation package | Sign-on bonusSenior-level Full TimeSan Francisco R1d ago
-
Cloud and Storage Engineer USD 100K-166KAmazon Web Services | Auditing | Backup and Recovery | Capacity Planning | Data DomainComprehensive benefits packageSenior-level Full TimeMiami, FL R1d ago
-
Cloud and Storage Engineer USD 100K-166KAmazon Web Services | As-a-Service | Auditing | Backup and Recovery | Capacity PlanningComprehensive benefits package | Professional growth opportunitiesSenior-level Full TimeAtlanta, GA R1d ago
-
AI Agents | API Development | Azure | CI/CD | LLMOpsCollaborative cross-functional team | Fully remoteMid-level Full TimeCanada, Ontario; Toronto, None; Toronto, Ontario; … R2d ago
-
API Gateway | AWS Lambda | AWS Step Functions | Amazon SQS | Apache Airflow401k matching | Employee stock options | Health benefits | Hybrid work onsite 2 to 3 days per week | Paid time offExecutive-level Full TimeNew York, United States R2d ago
-
Senior Machine Learning Engineer USD 160K-200KAWS | CI/CD | Computer Vision | Data Processing | DockerCommuter benefits | Dental insurance | Employee assistance program | Equity | Health insuranceSenior-level Full TimeArlington, VA R2d ago
-
Senior Computer Vision & Machine Learning Engineer USD 140K-206KAblation Studies | CI/CD | Data Pipelines | Docker | Error AnalysisAutonomy | Client project impact | Support for professional growthSenior-level Full TimeRemote, US R2d ago
-
Computer Vision & Machine Learning Engineer USD 150K-200KAblation Studies | CI/CD | Computer Vision | Data Pipelines | DockerEntry-level Full TimeRemote, US R2d ago
-
Staff Data Engineer USD 162K-224KAmazon Managed Workflows for Apache Airflow | Apache Airflow | Apache Druid | Apache Iceberg | CCPA401k employer match | Annual WIN bonus | Flexible vacation | Mental health resources | Paid HolidaysSenior-level Full TimeU.S. Remote R2d ago
-
Senior GTM AI Engineer USD 154K-213KAgent Orchestration | Agent Performance | Agent Performance Monitoring | Anthropic Claude | App Development401k match | Catered lunch | Company events | Desk setup stipend | Flexible spending accountsSenior-level Full TimeOakland, California, United States R2d ago
-
Software Engineer, Database Reliability & Performance USD 100K-168KAI tools | Automated testing | C++ | CI/CD | CircleCI401k | Annual professional development budget | Financial planning and legal advice | Flexible time off | Life insuranceMid-level Full TimeRemote US R2d ago
-
Staff Diagnostics and Embedded Cyber Security Engineer USD 150K-283KAUTOSAR | Automotive Ethernet | BLE | C# | C++Employee resource groups | Flexible family care days | Paid parental leave | Paid time off | Subsidized back-up child careSenior-level Full TimePalo Alto, CA, United States R2d ago
-
Senior Analytics Engineer USD 153K-206KAWS | Data Quality | ETL | Healthcare Claims | PySpark401k plan | Commuter benefits | Dental insurance | Disability insurance | Flexible spending accountSenior-level Full TimeRemote- US R2d ago
-
Senior Data Engineer USD 190K-210KAccess Control | Alerting | CI/CD | DBT | Data Governance401k | Flex PTO | Learning and development reimbursement | Medical, dental, vision plans | Paid parental leaveSenior-level Full TimeRemote (US) R2d ago
-
Senior AI Analytics Engineer USD 149K-212KAI Agents | Agent Frameworks | Automation | DBT | Python401k match | Employee assistance program | HSA contribution | Health insurance | Hybrid workSenior-level Full TimePlano,Texas,United States R2d ago
-
Senior Data Engineer (B2B) USD 105K-135KAutomated testing | BigQuery | DBT | Data Modeling | Data Quality401k matching | Annual class pass credits | Dental insurance | Extended holiday break | Flexible schedulesSenior-level Full TimeRemote, United States R2d ago
-
Senior Software Engineer, Data Platform USD 168K-297KAWS | Anomalo | Apache Airflow | Apache Iceberg | BuildkiteFlexible time off | Medical insurance | Modern family planning | Remote work | Retirement savings plansSenior-level Full TimeBay Area, CA, United States of … R2d ago
-
IAA - Principal Data Engineer USD 171K-256KAzure | Azure Synapse | Azure Synapse Analytics | CI/CD | Data GovernanceHybrid work schedule | Onsite expectation 2 days per weekSenior-level Full TimeChicago, IL (Hybrid) R2d ago
-
Senior Analytics Engineer USD 149K-195KAWS | Alerting | Azure | Cloud Data | Cloud Data PlatformsAnnual performance bonuses | Company equity grants | Hybrid work schedule | Unlimited vacationSenior-level Full TimeNew York, New York, United States R2d ago