Data Engineer II - Enterprise Data & Analytics - Digital and Technology Partners - Remote

Data Engineer II - Enterprise Data & Analytics - Digital and Technology Partners - Remote

The Data Engineer II is responsible for designing, developing, and maintaining robust, scalable, and secure data architectures that facilitate efficient data collection, transformation, storage, analysis, and self-service access. Leveraging modern tools and methodologies, the Data Engineer II manages transformation logic, data models, and metric definitions, develops automated pipelines, and supports AI-driven workflows. By ensuring data reliability and quality, this role directly contributes to enhancing patient care delivery, clinical pathways, translational research, and overall clinical care. The Data Engineer II collaborates closely with business teams, data scientists, machine learning engineers, and backend developers to deliver impactful data solutions.

 

• Design, develop, and maintain scalable and reliable data pipelines using orchestration engines such as Airflow and Dataiku, ensuring seamless automation of data ingestion, transformation, and delivery processes.
• Deploy and maintain containerized applications and pipelines, employing technologies like Docker and Kubernetes to achieve resilient and maintainable data workflows.
• Develop, deploy, and operationalize AI workflows, including image processing, data categorization, and natural language processing (NLP) models, ensuring production-level reliability and performance.
• Implement DevOps best practices, including version control with Git, continuous integration and continuous deployment (CI/CD) pipelines, automated testing frameworks, and unit testing to facilitate rapid, reliable, and high-quality software deployments.
• Create and manage a scalable and maintainable data architecture, including designing reference tables and deploying AI-driven mechanisms to ensure reference tables remain accurate and current.
• Develop and maintain comprehensive data dictionaries, enforce data quality metrics, implement anomaly detection solutions, and establish atomic rollback processes to effectively manage and rectify data errors.
• Build robust data ingestion pipelines capable of handling diverse data sources, including AI-generated data, flat files, and RESTful APIs (both reading from and writing to endpoints).
• Collaborate closely with agile teams consisting of Application Developers, Database Developers, and Data Scientists, actively participating in sprint planning, stand-ups, and retrospectives.
• Create centralized documentation, diagrams, and metadata catalogs that clearly describe data solutions, facilitating knowledge sharing and ease of use.
• Design, implement, and manage data system monitoring, backups, and disaster recovery plans, safeguarding the integrity, availability, and security of data.
• Engage with stakeholders with a customer-focused approach, delivering solutions that align with scientific, research, and clinical objectives.
• Ensure adherence to industry best practices, HIPAA compliance, and institutional data governance policies and procedures.
• Maintain current knowledge of industry trends and emerging technologies, demonstrating flexibility and continuous learning to adapt and enhance skillsets relevant to data engineering.
• Develop standards and best practices documentation related to data management, architecture, and maintenance, and provide training and presentations to team members as required.
• Maintains strong desire to understand the “why” behind data.

Education Requirements                                                                                                                 

Bachelor’s degree in Computer Science or a related discipline; Advanced degree preferred.

Experience Requirements                                                                                                   

• 4+ years of relevant professional experience, preferably in data engineering, data pipeline development, and data science workflows, in a Linux environment.
• Strong knowledge of SQL and NoSQL databases, including Azure SQL, PostgreSQL/MySQL, and MongoDB or similar.
• Proficiency with at least two programming languages among Python, Scala, Java, or Go, with the flexibility to quickly learn additional languages.
• Hands-on experience working with RESTful APIs and services, preferably using Node.js, Django, or similar frameworks.
• Demonstrated expertise in containerization technologies such as Docker and Kubernetes.
• Solid understanding and practical experience with orchestration and automation tools like Airflow or Dataiku.
• Experience in developing, deploying, and maintaining AI workflows, including computer vision/image processing, NLP, and data categorization models.
• Proficiency in using DevOps tools and practices, including Git, CI/CD tools, automated unit testing, and GitHub Copilot or similar AI-assisted coding tools (e.g., ChatGPT).
• Experience building scalable data architectures with reference tables and automated AI-driven updating mechanisms.
• Familiarity with Hadoop, Spark, Kafka, and streaming data platforms and technologies.
• Knowledge of healthcare data standards such as HL7, and familiarity with integration tools like Mirth is a significant plus.
• Strong knowledge of Azure cloud data services, including Azure Databricks, Azure Fabric, Azure Data Factory, serverless computing, virtual machines, and cloud security.
• Strong knowledge of data visualization tools such as Power BI or Tableau.
• Strong knowledge of data structures, data formats, algorithms, and object-oriented design and practical experience with data serialization and storage formats such as Parquet, Avro, JSON, or ORC.
• Experience working within Agile methodologies and tools, specifically JIRA, is highly desirable.

Compensation Statement

The Mount Sinai Health System (MSHS) provides salary ranges that comply with the New York City Law on Salary Transparency in Job Advertisements. The salary range for the role is $90,000.00 - $135,285.00 Annually. Actual salaries depend on a variety of factors, including experience, education, and operational need. The salary range or contractual rate listed does not include bonuses/incentive, differential pay or other forms of compensation or benefits.

Non-Bargaining Unit, 223 - DTP Enterprise Data & Analytics - MSH, Mount Sinai Hospital

Strength through Unity and Inclusion

The Mount Sinai Health System is committed to fostering an environment where everyone can contribute to excellence. We share a common dedication to delivering outstanding patient care. When you join us, you become part of Mount Sinai’s unparalleled legacy of achievement, education, and innovation as we work together to transform healthcare. We encourage all team members to actively participate in creating a culture that ensures fair access to opportunities, promotes inclusive practices, and supports the success of every individual.

At Mount Sinai, our leaders are committed to fostering a workplace where all employees feel valued, respected, and empowered to grow. We strive to create an environment where collaboration, fairness, and continuous learning drive positive change, improving the well-being of our staff, patients, and organization. Our leaders are expected to challenge outdated practices, promote a culture of respect, and work toward meaningful improvements that enhance patient care and workplace experiences. We are dedicated to building a supportive and welcoming environment where everyone has the opportunity to thrive and advance professionally. Explore this opportunity and be part of the next chapter in our history.

About the Mount Sinai Health System:

Mount Sinai Health System is one of the largest academic medical systems in the New York metro area, with more than 48,000 employees working across eight hospitals, more than 400 outpatient practices, more than 300 labs, a school of nursing, and a leading school of medicine and graduate education. Mount Sinai advances health for all people, everywhere, by taking on the most complex health care challenges of our time — discovering and applying new scientific learning and knowledge; developing safer, more effective treatments; educating the next generation of medical leaders and innovators; and supporting local communities by delivering high-quality care to all who need it. Through the integration of its hospitals, labs, and schools, Mount Sinai offers comprehensive health care solutions from birth through geriatrics, leveraging innovative approaches such as artificial intelligence and informatics while keeping patients’ medical and emotional needs at the center of all treatment. The Health System includes more than 9,000 primary and specialty care physicians; 13 joint-venture outpatient surgery centers throughout the five boroughs of New York City, Westchester, Long Island, and Florida; and more than 30 affiliated community health centers. We are consistently ranked by U.S. News & World Report's Best Hospitals, receiving high "Honor Roll" status, and are highly ranked: No. 1 in Geriatrics, top 5 in Cardiology/Heart Surgery, and top 20 in Diabetes/Endocrinology, Gastroenterology/GI Surgery, Neurology/Neurosurgery, Orthopedics, Pulmonology/Lung Surgery, Rehabilitation, and Urology. New York Eye and Ear Infirmary of Mount Sinai is ranked No. 12 in Ophthalmology. U.S. News & World Report’s “Best Children’s Hospitals” ranks Mount Sinai Kravis Children's Hospital among the country’s best in several pediatric specialties. The Icahn School of Medicine at Mount Sinai is ranked No. 11 nationwide in National Institutes of Health funding and in the 99th percentile in research dollars per investigator according to the Association of American Medical Colleges. Newsweek’s “The World’s Best Smart Hospitals” ranks The Mount Sinai Hospital as No. 1 in New York and in the top five globally, and Mount Sinai Morningside in the top 20 globally.

Equal Opportunity Employer

The Mount Sinai Health System is an equal opportunity employer, complying with all applicable federal civil rights laws. We do not discriminate, exclude, or treat individuals differently based on race, color, national origin, age, religion, disability, sex, sexual orientation, gender, veteran status, or any other characteristic protected by law. We are deeply committed to fostering an environment where all faculty, staff, students, trainees, patients, visitors, and the communities we serve feel respected and supported. Our goal is to create a healthcare and learning institution that actively works to remove barriers, address challenges, and promote fairness in all aspects of our organization.