Sr Software Engineer- Spark, Hive, SQL, AWS

About the role

Gartner is looking for a Data Engineer focusing on data engineering. This person will be a part of team developing and supporting Gartner ‘s client facing experience. This includes supporting all the data related operations within the team. This position will provide an opportunity to work with on several different technologies focusing on data modelling , data services and data science. This includes building and consuming web services, integrating search technologies, and building personalization , recommendation engines, data engineering best practices integration.

What You Will do

• Participate in architecture design and implementation of high-performance, scalable, and optimized data solutions. Prepare documentations and specifications

• Good Sql understanding with ability to create the data model from scratch. Help write and optimize in-application SQL statements. Ensure performance, security, and availability of databases.

• Handle common database procedures such as upgrade, backup, recovery, migration, etc.

• Design, build and automate the deployment of data pipelines and applications to support data scientists and researchers with their reporting and data requirements.

• Integrate data from a wide variety of sources, including on premise databases and external data sources with rest APIs and harvesting tools.

• Collaborate with internal business units and data science teams on business requirements, data access, processing/transformation and reporting needs and leverage existing and new tools to provide solutions.

• Work with team on managing AWS resources (EMR, ECS clusters, etc.) and continuously improve deployment process of our applications

• Promote the integration of new cloud technologies and continuously evaluate new tools that will improve the organization’s capabilities while leading to lower total cost of operation.

• Support automation efforts across the data analytics team utilizing Infrastructure as Code (IaC) using Terraform, Configuration Management, and Continuous Integration (CI) / Continuous Delivery (CD) tools such as Jenkins.

• Work with the team to implement data governance, access control and identify and reduce security risks.

What you will need

Need 5-7 years of experience in Solution, Design and Development of Cloud based data models, ETL Pipelines and infrastructure for reporting, analytics, and data science.

Must have

• Experience working with Spark, Hive, HDFS, MR, Apache Kafka/AWS Kinesis

• Experience working with both structured and unstructured data.

• Strong proficiency with SQL and its variation among popular databases

• Capable of troubleshooting common database issues.

• Experience with version control tools (Git, Subversion)

• Experience using automated build systems (CI/CD)

• Experience of Data Structures and algorithms

• Knowledge of different databases technologies (Relational, NoSQL, Graph, Document, Key-Value, Time Series, etc…). This should include building and managing scalable data models.

• Knowledge of Cloud based platforms (AWS) must have.

#LI-JJ3

Who are we? 

At Gartner, Inc. (NYSE:IT), we guide the leaders who shape the world.

Our mission relies on expert analysis and bold ideas to deliver actionable, objective insight, helping enterprise leaders and their teams succeed with their mission-critical priorities.

Since our founding in 1979, we’ve grown to more than 20,000 associates globally who support ~15,000 client enterprises in ~90 countries and territories. We do important, interesting and substantive work that matters. That’s why we hire associates with the intellectual curiosity, energy and drive to want to make a difference. The bar is unapologetically high. So is the impact you can have here.

What makes Gartner a great place to work? 

Our sustained success creates limitless opportunities for you to grow professionally and flourish personally. We have a vast, virtually untapped market potential ahead of us, providing you with an exciting trajectory long into the future. How far you go is driven by your passion and performance.

We hire remarkable people who collaborate and win as a team. Together, our singular, unifying goal is to deliver results for our clients.

Our teams are inclusive and composed of individuals from different geographies, cultures, religions, ethnicities, races, genders, sexual orientations, abilities and generations.

We invest in great leaders who bring out the best in you and the company, enabling us to multiply our impact and results. This is why, year after year, we are recognized worldwide as a great place to work.

What do we offer? 

Gartner offers world-class benefits, highly competitive compensation and disproportionate rewards for top performers. 

In our hybrid work environment, we provide the flexibility and support for you to thrive — working virtually when it's productive to do so and getting together with colleagues in a vibrant community that is purposeful, engaging and inspiring.

Ready to grow your career with Gartner? Join us.

The policy of Gartner is to provide equal employment opportunities to all applicants and employees without regard to race, color, creed, religion, sex, sexual orientation, gender identity, marital status, citizenship status, age, national origin, ancestry, disability, veteran status, or any other legally protected status and to seek to advance the principles of equal employment opportunity.

Gartner is committed to being an Equal Opportunity Employer and offers opportunities to all job seekers, including job seekers with disabilities. If you are a qualified individual with a disability or a disabled veteran, you may request a reasonable accommodation if you are unable or limited in your ability to use or access the Company’s career webpage as a result of your disability. You may request reasonable accommodations by calling Human Resources at +1 (203) 964-0096 or by sending an email to ApplicantAccommodations@gartner.com.

By submitting your information and application, you confirm that you have read and agree to the country or regional recruitment notice linked below applicable to your place of residence.

Gartner Applicant Privacy Link: https://jobs.gartner.com/applicant-privacy-policy

For efficient navigation through the application, please only use the back button within the application, not the back arrow within your browser.