Senior Data Engineer
United States
Conexiom
Instantly turn unstructured trade documents into structured data with The Conexiom Platform.- As a Sr. Data Engineer, you will lead short- and long-term data strategy to deliver value incrementally while partnering with data scientists and engineers
- Build and maintain data pipeline architecture, optimize data flow, and collect data from cross-functional teams. This includes designing and implementing a new scalable data platform using modern data technologies
- Champion data validity and accuracy tests to ensure high-quality data
- Be one step ahead of data scientists and machine learning engineers by providing infrastructure and tools they can leverage to deliver ML solutions
- Work in a fast-paced Agile environment with a proven ML product already in the market, a strong team, and seniors to assist you in taking the next steps in your career.
Conexiom embraces diversity and equal opportunity. We are committed to building a team that represents a variety of backgrounds, perspectives, and skills. We are working to ensure that the profile of our staff reflects the profile of our customers and the communities they serve. For that reason, we seek resumes and expressions of interest from a broad and diverse talent pool. Strength comes from the inclusion of diverse perspectives and experiences.
KEY QUALIFICATIONS- Bachelor’s or Master’s degree in a quantitative field such as Physics, Engineering, Computer Science, or equivalent technical fields
- 4+ years’ experience in building a scalable data platform
- Advanced skills in SQL/ Python to script, automate, and integrate data pipelines
- Design, build, and maintain scalable batch and streaming data pipelines in Databricks to support analytics and machine learning needs.
- Play a key role in architectural decisions around data infrastructure and workflows.
- Write clean, well-tested Python code for processing large and complex datasets.
- Work with unstructured data and document databases like MongoDB.
- Build and manage CI/CD pipelines to streamline deployment and version control for data workflows.
- Set up automated monitoring and alerting to ensure pipelines run reliably and efficiently.
- Ensure data and code quality through robust unit testing, integration testing, and validation.
- Document workflows and processes to ensure they are easy to understand and maintain.
- Hands-on experience with Spark Structured Streaming
- Familiarity with vector search tools like Pinecone for working with embeddings.
- Terraform experience
- Azure experience (AWS or GCP okay too)
Impactful Work:
Play a pivotal role in transforming how businesses operate on a global scale.
Growth Opportunity:Be part of an ambitious company on a rapid growth trajectory, offering numerous opportunities for personal and professional development.
Remote Flexibility:Enjoy the flexibility of a fully remote position, allowing you to work from anywhere.
Inclusive Culture:Join a diverse team of innovative thinkers and doers, committed to fostering an inclusive environment where everyone can thrive.
Conexiom is proud to offer equal employment opportunities. If you have a disability or need that requires accommodation at any time during the recruitment process, please let us know. #LI-REMOTE* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Architecture AWS Azure CI/CD Computer Science Databricks Data pipelines Data strategy Engineering GCP Industrial Machine Learning MongoDB Physics Pinecone Pipelines Python Spark SQL Streaming Terraform Testing Unstructured data
Perks/benefits: Career development Startup environment
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.