Cloudera - Big Data Engineer

New Cairo, Egypt

Full Time Mid-level / Intermediate USD 31K - 72K * ^est.

MigrationIT

MigrationIT connects employers with IT professionals who are equipped to fulfill an employer’s distinct IT needs. The agency provides IT talents for several industries, including healthcare, banking, insurance, software development, retail,...

View all jobs at MigrationIT

Apply now Apply later

Posted 3 weeks ago

Operating supporting Cloudera data lake cluster
Operating & supporting Kafka cluster
Designing & building and operating Data Lake
Build and design the Bigdata Solutions from Scratch in terms of Data Ingestion Pipeline to CDP.
Data Preparation on CDP to Data Scientist teams for building ML Models on top of Ingested Data.
Implementation and design of Big Data jobs for ETL and Aggregation and automation of ETL Process.
Provide data and analysis needed to business intelligence and reporting team members.
Analyze Business requirements and design solution for Big Data and Analytics projects
Perform ETL Process from Data sources into Hadoop Cluster and monitor the success and failure of ETL process on daily basis.

Requirements

Bachelor’s degree in Computer Science, Engineering, or related field.
1-3 years of experience in the same role.
Proficient understanding of distributed computing principles.
Management of Hadoop cluster, with all included services.
Proficiency with Hadoop, Spark, HDFS.
Experience with NoSQL databases, such as HBase, Cassandra, MongoDB.
Experience with integration of data from multiple data sources.
Experience with building stream-processing systems, using solutions such as Storm or Spark-StreamingGood knowledge of Big Data querying tools, such as Pig, Hive, and Impala.
Knowledge of various ETL techniques and frameworks, such as Flume.
Experience with various messaging systems, such as Kafka.
Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O.
Experience with Cloudera distribution.