Specialist- Big Data Development and Operation ( Contract)

USD 94K-140K (estimate) Entry-level Contract Full Time Found 2d ago

Tasks

• Automate deployment and configuration
• Build reporting and summary tables with SQL
• Collaborate with analysts on metrics and logic
• Design data pipelines using Apache NiFi
• Develop and optimize scalable data ingestion processes
• Enforce data governance and security standards
• Manage big data platform components
• Transform datasets for machine learning
• Troubleshoot pipeline and infrastructure issues

Perks/Benefits

• N/A

Skills/Tech-stack

API Development | Airflow | Ansible | Apache NiFi | Bash | Big Data | Big Data Development | Data Development | Data Engineering | Data Governance | Data Lineage | Data analytics | Distributed Computing | ELK Stack | ETL Processes | Grafana | HDFS | Helm | Hive | Kafka | Logging | Microservices | Monitoring | Presto | Prometheus | SQL | Schema evolution | Spark | Trino | YARN