Data Engineer

About the role:
As a Data Engineer, you will play an important role facilitating dataflows together with cybersecurity analysts focused on upstream data acquisition. You will take responsibility for understanding diverse data and file formats, and deliver tooling and automation to enable efficient analysis and ingestion of large quantities of sensitive information. You will be expected to implement best practices in data management, coding, testing, and deployment – ensuring scalability, performance, and security. Strong problem-solving skills, excellent communication, and the ability to adapt to new challenges in a fast-paced environment are highly useful for this role.
Key responsibilities:
• Analyze, parse and clean large datasets of structured and unstructured data• Collaborate closely with security analysts and lead developers to understand bottlenecks and to prioritize, correct and ingest cleaned data• Develop data ingestion frameworks to handle high-volume and high-variety data, including structured and unstructured data recovered from +2000 data sources across the deep, dark and open web• Implement data quality control measures to detect and handle data anomalies, duplicates, and inconsistencies• Work towards optimizing data pipeline performance for scalability, reliability, and latency• Troubleshoot data pipeline issues and data quality problems
Your skills, experience, and qualifications:
• Strong command of Linux CLI tooling and AWS CLI• Deep understanding of common data formats such csv, json, sql, excel files• Deep understanding of file encoding and related issues• Strong Python scripting skills to develop tooling• Strong problem-solving skills, with the ability to troubleshoot complex data pipeline issues• Excellent communication and collaboration skills, with the ability to work with data scientists, engineers, and senior stakeholders• Degree educated in Computer Science or relevant subject• Excellent spoken and written English • Positive, can-do approach to work, delivering on commitments• Creative approach to problem solving, can look at existing situations and problems in novel ways and propose innovative solutionsAbout us:
We’re a diverse group of visionary innovators who provide trading and workflow automation software, high-value analytics, and strategic consulting to corporations, central banks, financial institutions, and governments. More than 40% of the world’s largest companies use our solutions. Founded in 1999, we’ve achieved tremendous growth by bringing together some of the best and most successful financial technology companies in the world. 
Over 2,000 of the world’s leading corporations, including 50% of the Fortune 500 and 30% of the world’s central banks, trust ION solutions to manage their cash, in-house banking, commodity supply chain, trading and risk. 
Over 800 of the world’s leading banks and broker-dealers use our electronic trading platforms to operate the world’s financial market infrastructure.With 10,000 employees and offices in more than 40 cities around the globe, ION is a rapidly expanding and dynamic group. 
Within ION Analytics we help clients find advantage in a world of information overload and fragmented systems. Our business information solutions blend expert journalism and analytics with data to help banks, advisors, investors, law firms and corporations find opportunities in markets ranging from equities and fixed income to infrastructure and private equity. 
On our platform clients can break through silos to originate opportunities, find buyers, and use resources efficiently, and our advanced workflow automation tools cut through the noise to deliver personalized, targeted information that saves time and effort, and drives better decisions. 
ION is a rapidly expanding and dynamic group with 13,000 employees and offices in more than 40 cities around the globe, Our ever-expanding global footprint, cutting edge products, and over 40,000 customers worldwide provide an unparalleled career experience for those who share our vision.