Data Engineer - Capital Markets
Tasks
- Automate Databricks deployments across environments
- Build data pipelines
- Build data views for reporting and AI modeling
- Configure AWS data cataloging and storage
- Design data lakehouse architecture
- Develop and orchestrate data workflows
- Implement data governance and security
- Implement data lineage and auditing
- Implement real time reporting
- Integrate Databricks with AWS services
- Manage compute costs
- Optimize Delta Lake storage
- Set fine grained access controls
- Tune Spark clusters
Perks/Benefits
Skills/Tech-stack
AWS CodePipeline | AWS Glue | AWS IAM | Amazon MSK | Amazon S3 | Amazon Web Services | Apache Kafka | Apache Spark | Asset bundles | CI/CD | Data Lineage | Databricks | Databricks (asset) | Databricks Asset Bundles | Databricks Jobs | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | GitHub Actions | GitLab CI | GitLab CI/CD | PySpark | SQL | Streaming | Terraform | Unity Catalog | Web Services | Z-ordering
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Specialist Solutions Architect - Data Engineering & Warehousing (Digital Native Business) USD 180K-247KAWS | Anomaly Detection | Apache Kafka | Automated Ingestion | AzureConference presentations | Hackathons | Mentorship | Remote work | Training programsSenior-level Full TimeUnited States R9h ago
-
Senior Software Engineer (Data & AI Solutions) USD 125K-156KAWS | Apache Airflow | Apache Spark | BAM | Bioinformatics401k benefits | Commuter benefits | Employee referral program | Fertility care benefits | Free testingSenior-level Full TimeUS Remote R12h ago
-
Data & Analytics Engineer I or II (Time Limited) USD 93K-146KData Engineering | Data Modeling | Data analytics | Microsoft Fabric | Power BIEmployee assistance program | Flexible spending account | Floating holiday | Health savings account | Medical leaveMid-level Full TimeEverett, WA, US R13h ago
-
A/B | A/B Testing | API Integration | Airflow | Attribution ModelingCross-functional collaboration | Remote workSenior-level Full TimeUnited States R14h ago
-
Senior Embedded Engineer- Developer Experience USD 130K-179KBLE | Bluetooth Low Energy | C# | CI/CD | CMakeSenior-level Full TimeRemote, Seattle, Washington, United States R15h ago
-
Data Engineer USD 170K-209KAWS | Amazon Kinesis | CI/CD | Compliance Frameworks | Container OrchestrationSenior-level Full TimeUS - Remote; Canada - Remote R15h ago
-
Junior Data Engineer (New York, NY) USD 62K-83KAWS CDK | AWS CloudFormation | Apache Hive | Apache Spark | Continuous DeploymentHealth benefits | Paid time off | Professional development | Remote-first workplaceEntry-level Full TimeNew York City, NY metro area R15h ago
-
Junior Data Engineer (Philadelphia, PA) USD 62K-75KAWS CDK | AWS CloudFormation | Apache Hive | Apache Spark | Data pipeline403b match | Flexible work environment | Medical, vision, and dental coverage | Paid community service time | Paid parental leaveEntry-level Full TimePhiladelphia, PA metro area R15h ago
-
Data Engineer I USD 80K-110KAnalysis Services | Azure Data | Azure Data Factory | Azure DevOps | Azure MonitorHybrid work scheduleMid-level Full TimeRifle, Colorado, United States; Grand Junction, … R17h ago
-
Data Warehouse Software Engineer USD 70K-80KApache Airflow | Cloud Composer | Cloud platform | Git | Google BigQuery401k match | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeSomerville, Massachusetts, United States R17h ago
-
Senior Database Engineer USD 114K-195KAWS CloudFormation | AWS RDS | Access Control | Always On | Always On AvailabilitySenior-level Full TimeUS Remote R17h ago
-
Data Engineer USD 140K-348KDBT | Dagster | Data Architecture | Data Modeling | ETLCell phone reimbursement | Company recharge days | Flexible PTO | Health, dental and vision coverage | Home internet reimbursementMid-level Full TimeSan Francisco, CA • New York, … R17h ago
-
Senior, ML Engineer - 3D Reconstruction USD 177K-212KActive Learning | CI/CD | Cloud services | Deep learning | Docker401k match | AD and D insurance | Company holiday closures | Flexible schedule | Life insuranceSenior-level Full TimeAnn Arbor, MI, Remote - US R18h ago
-
Senior IT DataOps Business Intelligence Engineer USD 143K-185KAccess Control | Amazon Web Services | Anthropic | CI/CD | Census401k | Dental insurance | Education and learning stipend | Flexible vacation time | Health insuranceSenior-level Full TimeNew York City R18h ago
-
Senior Site Reliability Engineer - Storage USD 267K-356KAlertmanager | Amazon S3 | Ansible | ArgoCD | BuildkiteSenior-level Full TimeSan Francisco Office (Fremont St) R18h ago
-
Senior Machine Learning Engineer, Fraud Detection USD 185K-225KA/B | A/B Testing | API | AWS | AWS SageMaker401k match | Commuter benefit | Dental insurance | Flexible spending account | Flexible time offSenior-level Full TimeLos Angeles, California, United States; San … R18h ago
-
Staff Software Engineer - Experimentation USD 265K-331KAWS | CI/CD | Cloudflare | DBT | Data Pipelines13 paid holidays | 16-week parental leave | 401k | Carrot Fertility Membership | Carrot fertility reimbursementSenior-level Full TimeRemote R18h ago
-
Senior AI Engineer USD 120K-160KAWS | Benchmarking | Databricks | FastAPI | Flask401k Safe Harbor Plan | Bounty programs | Certification reimbursement | Free health insurance | Generous PTOSenior-level Full TimeSchenectady, New York, United States, Remote R18h ago
-
Senior Data Architect / Data Engineer USD 130K-180KCloud Data | Cloud Data Environments | Cloud platform | Data Cleanup | Data Migration100% remote | Background check | Flexible work hoursSenior-level Full TimeSchenectady, New York, United States, Remote R18h ago
-
Sr. Forward Deployed Engineer (FDE) - Public Sector USD 182K-250KAWS | Apache Spark | Azure | CI/CD | Cloud platformSenior-level Full TimeRemote - Washington D.C.; USCA R19h ago
-
Ai Engineer USD 150K-170KApache Airflow | Artificial Intelligence | Azure Data | Azure Data Factory | Azure DevOps401k | Employee assistance program | Health spending account | Life insurance | Long-term disabilitySenior-level ContractBaltimore, Maryland, USA R19h ago
-
AI Search | Application Insights | Application diagnostics | Azure AI | Azure AI Search401k | Dental insurance | Educational assistance | Health savings account | Life insuranceMid-level Full TimeNew York, United States R19h ago
-
Fullstack Software Engineer - Embedded Payments USD 95K-105KAPI Gateway | AWS | Apple Pay | Bitbucket Pipelines | CI/CDDiversity and inclusion programs | Extra Paid Volunteer Days | Flexible working arrangements | Free mental health support | Mentorship programsMid-level Full TimeKansas City, MO, United States R19h ago
-
Machine Learning Engineer USD 150K-200KAWS | Airflow | CI/CD | Cloud platform | DagsterDental | Healthcare | VisionMid-level Full TimeSan Francisco R20h ago
-
Access Control | Alerting | Amazon Athena | Amazon EMR | Amazon S3Employee discounts | Employee equity | Medical, dental, and vision coverage | Pet insurance | Unlimited PTOSenior-level Full TimeRemote - United States R20h ago