DataOps Engineer (AWS / Databricks)
Miami, United States
A USD 180K-247K (estimate) Senior-level Full Time
Tasks
- Build and maintain CI/CD pipelines
- Catalog data
- Define and implement DataOps strategy
- Define and operationalize data quality practices
- Define repository structure and organization
- Design PySpark based data pipelines
- Design and implement cloud data infrastructure
- Enable encryption
- Ensure data security controls
- Implement access management
- Implement data governance policies
- Implement deployment versioning and rollback strategies
- Lead architectural decisions for data stack
- Manage data lineage
- Migrate data from on-premise to cloud
- Monitor data pipelines
- Provide data observability
- Separate environments
- Set data ownership and data layers
- Set data validation and SLAs
- Standardize development workflows
Perks/Benefits
Skills/Tech-stack
AWS | Access Management | Amazon IAM | Amazon S3 | Amazon VPC | CI/CD | Data Engineering | Data Governance | Data Lineage | Data Quality | Data Security | Data Validation | Data orchestration | DataOps | Databricks | Encryption | Monitoring | Observability | PySpark | Repository Organization | Terraform
Education
Roles
Related jobs
-
Entry-level Full TimeCary, NC, United States2h ago
-
Apache Spark | Clustering | Data Ingestion | Data Ingestion Pipelines | Data ModelingSenior-level ContractLos Angeles, United States3h ago
-
Senior-level Full TimeAustin, TX, United States3h ago
-
Availability Engineering | Canary Deployment | Capacity Planning | Cause analysis | Change ManagementSenior-level Full TimeSeattle, Washington, United States4h ago
-
Automation | Canary Deployment | Capacity Planning | Cause analysis | Change ManagementSenior-level Full TimeSeattle, Washington, United States5h ago
-
Staff Software Engineer, NotebookLM, Generative AI, Labs USD 207K-300KAlerting | Computer Vision | Data Processing | Data Structures | Data structures algorithmsSenior-level Full TimeMountain View, CA, USA6h ago
-
Systems Development Engineer, Quantum AI USD 138K-198KAutomation | CI/CD | Cloud APIs | Cloud infrastructure | Configuration ManagementMid-level Full TimeGoleta, CA, USA6h ago
-
Senior-level Full TimeUnited States8h ago
-
AI/ML Engineer USD 117K-171KA/B | A/B Testing | API Development | AWS | B testingDental insurance | Health insurance | Paid time off | Retirement plan | Sick leaveSenior-level Full TimeResidence Based, Residence Based, US8h ago
-
Cloud Data Engineer USD 92K-113KAgile | Apache Airflow | Beam | BigQuery | Cloud ComposerOnsite workMid-level Full TimeAda, MI, United States, 4935510h ago
-
AWS | Azure | Cloud platform | Data Pipelines | Demand forecastingOnsite collaborationSenior-level Full TimeLewisville, TX, US13h ago
-
Senior Embedded Software Engineer USD 170K-210KC++ | Cause analysis | Configuration Management | Device identity | Diagnostics401k | FSA or HSA | Health, dental, vision insurance | Incentive stock options | Pre-tax commuter benefitsSenior-level Full TimeEmeryville, United States15h ago
-
Senior Machine Learning Engineer, Agentic USD 163K-245KArtificial Intelligence | Direct Preference Optimization | Evaluation | Fine Tuning | Human-in-the-loop401k matching | Catered meals | Employee events | Employer-paid disability insurance | Employer-paid life insuranceSenior-level Full TimeBellevue, WA; Menlo Park, CA15h ago
-
Senior ML Engineer, Fauna USD 184K-250KComputer Vision | Data Quality | Distributed Training | Edge Deployment | Evaluation PipelinesSenior-level Full TimeNew York, New York, USA17h ago
-
Forward Deployment Engineer - Gen AI USD 162K-224KAWS Bedrock | AWS SageMaker | Autogen | Azure OpenAI | ChromaCareer development opportunities | Individual responsibility | Travel to client sitesMid-level Full TimeNew York, New York, United States17h ago
-
Computer Vision | Experiment design | Experiment tracking | Generative AI | Image ProcessingSenior-level Full TimeAEP Headquarters, United States17h ago
-
Senior Software Engineer - Platform & MLOps USD 152K-230KAWS | Azure | CI/CD | Datadog | DockerDiscretionary incentive plan | Flexible work policy | Learning and development access | Medical benefitsSenior-level Full TimeSeattle, Washington, United States - Remote R17h ago
-
Data Engineer USD 84K-101KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCharity donation matching | Digital course library | Employee assistance program | Mindfulness program | Volunteer hours matching programMid-level Full TimeIndia Pune (Mississippi Building)17h ago
-
Lead Software Engineer (Python, LLM) - AskGartner USD 116K-170KAWS | AWS EMR | AWS Glue | AWS Kinesis | AWS QuickSight20 plus PTO days | 401k match | Collaborative culture | Employee assistance program | Employee stock purchase planSenior-level Full TimeIrving - 6011 Connection, United States17h ago
-
Senior Software Engineer, Knowledge Graph USD 196K-230KApache Flink | Apache Spark | Data Ingestion | Data Processing | DatabasesEmployee travel credits | Remote eligibleSenior-level Full TimeUnited States17h ago
-
Specialist Solutions Architect - AI/ML USD 180K-247KAI guardrails | Amazon Web Services | Apache Spark | Artificial Intelligence | Cloud ComputingMentorship | Remote work | Technical training | Travel up to 30 percentSenior-level Full TimeUnited States17h ago
-
Senior Data Engineer IS - Remote USD 122K-208KAI/ML | APIs | AWS Glue | Apache Airflow | Azure Data401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimePortland, OR, United States R18h ago
-
Principal AI Platform Engineer USD 167K-220KAgent Orchestration | Backend Development | Braintrust | Cost Optimization | Data PipelinesEquity | Flexible Token Limits | Health, dental, vision coverage | Unlimited paid time offSenior-level Full TimeSan Francisco, California R18h ago
-
Machine Learning Engineer USD 168K-198KAPIs | Amazon SageMaker | Anthropic API | Asynchronous processing | CI/CDAdoption reimbursement | Commuter stipend | Fertility reimbursement | Flexible PTO | Learning and development allowanceSenior-level Full TimeSan Francisco, California, United States19h ago
-
AWS MSK | Amazon Kinesis | Apache Flink | Apache Kafka | AutomationSenior-level Full TimeUSA - Atlanta, GA; USA - …19h ago