Data Platform Engineer
Remote South Carolina, Remote South Carolina, SC, United States
R
USD 125K-150K Mid-level Full Time
Tasks
- Administer Azure identity access and RBAC
- Administer Databricks workspaces
- Architect and enforce medallion layer standards
- Build self healing pipeline capabilities
- Design disaster recovery strategies
- Enforce Databricks cluster policies
- Implement CI/CD deployment pipelines
- Maintain Infrastructure as Code templates
- Maintain runbooks and technical documentation
- Manage Unity Catalog governance
- Manage environment parity across dev staging production
- Operate Azure data services stack
- Optimize Delta tables using Z ordering and partitioning
- Participate in on-call support
- Provide data pipeline platform observability
- Tune Spark jobs for performance and cost
Perks/Benefits
Skills/Tech-stack
ADLS Gen2 | Active Directory | Apache Spark | Azure Active Directory | Azure Cost Management | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure DevOps Pipelines | Azure Event | Azure Event Hubs | Azure Key Vault | Azure Monitor | CI/CD | Cost Management | Data Factory | Data Lake Storage Gen2 | Databricks | Databricks DBU | Delta Lake | Delta Live Tables | Delta Live) | DevOps Pipelines | Event Hubs | GitHub Enterprise | Infrastructure as Code | Key Vault | Log Analytics | Medallion Architecture | Optimize | Python | RBAC | SQL | Terraform | Unity Catalog | VACUUM | YAML | Z-ordering | “as-code”
Education
N/A
Roles
Data Platform | Data Platform Engineer | Engineer | Platform | Platform Engineer
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R26d ago
-
API | Amazon Web Services | Amazon Web Services (AWS) | Apache HDFS | Apache SparkRemote workMid-level Full TimeRemote, US; Sunnyvale, CA; Dallas, TX R1h ago
-
Senior Data Engineer (Remote) USD 108K-150KData Architecture | Data Lake | Data Modeling | Data Quality | Data VisualizationRemote workSenior-level Full TimeUnited States R3h ago
-
Amazon EKS | Amazon EMR | Amazon Redshift | Amazon S3 | Amazon Web ServicesComprehensive insurance | Diversity and inclusion | Flexible work arrangements | Hybrid remote workMid-level Full TimeReston, VA R3h ago
-
Amazon Redshift | Amazon Web Services | Apache Airflow | CI/CD | CTEComprehensive insurance | Diversity and inclusion | Flexible work arrangementsMid-level Full TimeReston, VA; Reston, VA R3h ago
-
Big Data | ETL | Elasticsearch | JSON | Jupyter401k match | PTO | Paid medical | Training supportSenior-level Full TimeSterling, VA R3h ago
-
Big Data | Continuous Monitoring | Cybersecurity | Data Translation | Data extraction401k match | Full paid medical | Paid time off | Training supportSenior-level Full TimeSterling, VA R3h ago
-
Big Data | EML | ETL | Elasticsearch | JSON401k match | Full paid medical | Paid time off | Training fundingSenior-level Full TimeSterling, VA; Annapolis Junction, MD R3h ago
-
AWS | AWS S3 | Amazon SQS | Apache Airflow | Apache Kafka401k match | Fully paid medical | Paid time off | Training supportSenior-level Full TimeColumbia, MD R3h ago
-
Algorithm tuning | Confusion matrix | Data Mining | Data Visualization | Feature Engineering401k contribution | Flexible schedule | Health insurance | Paid time offSenior-level Full TimeAnnapolis Junction, MD R3h ago
-
Agile | Amazon S3 | Amazon SQS | Apache NiFi | Bash401k match | Fully paid medical | PTO | Training supportSenior-level Full TimeSan Antonio, TX R3h ago
-
Agile | Amazon S3 | Amazon SQS | Apache NiFi | Bash401k match | Fully paid medical | Hybrid work | Onsite clearance support | Paid time offSenior-level Full TimeColumbia, MD R3h ago
-
Data Governance | Data Modeling | Docker | ETL | JSON401k match | Fully paid medical | Paid time off | Training supportSenior-level Full TimeColumbia, MD R3h ago
-
Data Governance | Data Modeling | Data Quality | Data schemas | Docker401k match | Fully paid medical | Paid time off | Training supportSenior-level Full TimeSan Antonio, TX R3h ago
-
Continuous Monitoring | Data Translation | Data labeling | ETL | Elasticsearch401k match | Fully paid medical | Paid time off | Training supportSenior-level Full TimeSan Antonio, TX R3h ago
-
Data Engineer USD 110K-144KAPI Development | Azure | Data Security | Distributed Computing | DockerMid-level Full TimeChicago, Ill; Remote, Remote; Remote, Remote; … R4h ago
-
Technical Solutions Architect II -- Data Engineer USD 150K-180KAmazon Redshift | Apache Airflow | Apache Kafka | Azure Data | Azure Data Factory401k matching | Dental insurance | Employee assistance program | Health insurance | Medical leaveMid-level Full TimeRemote - Nationwide, United States R5h ago
-
Mid-level Full TimeChicago, IL; Chicago, Il; Chicago, IL; … R5h ago
-
Mid-level Full TimeChicago, IL; Chicago, IL; Chicago, IL R5h ago
-
Apache Spark | Distributed Systems | Documentation | Hadoop | Java401k | Dental insurance | Disability insurance | Flexible vacation policy | Free coffeeSenior-level Full TimeBoston, MA; Boston, MA; Boston, MA; … R5h ago
-
Apache Hadoop | Apache Spark | Data Mining | Distributed Computing | Java401k | Commuter benefits | Dental insurance | Disability insurance | Employee wellness benefitsSenior-level Full TimeBoston, MA; Boston, MA; Cambridge, MA; … R5h ago
-
Amazon Redshift | Apache Airflow | Claims data | Cloud Computing | Common Data ModelCollaborative team | Remote workSenior-level Full TimeBoston, MA; Boston, MA; Wellesley, MA; … R5h ago
-
Access Control | Azure | Azure Backup | Azure CLI | Azure DataMentorship | Remote workSenior-level ContractSaint Paul, United States R7h ago
-
Edge Computing AI Engineer USD 100K-105KC++ | DSP | Edge inference | Embedded Systems | Federated LearningCareer growth | Remote workMid-level Full TimeUnited States - Remote R21h ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HudiCareer growth | Direct W2 employment | Remote workSenior-level Full TimeUnited States - Remote R21h ago