Python Data Engineer with AI/ML

Pune, MH, India

Full Time Mid-level / Intermediate USD 49K - 91K * ^est.

Expleo

Expleo is a trusted partner for end-to-end, integrated engineering, quality services and management consulting for digital transformation.

View all jobs at Expleo

Apply now Apply later

Posted 1 day ago

Overview

We are looking for a Python Data Engineer with expertise in real-time data monitoring, extraction, transformation, and visualization. The ideal candidate will have experience working with Oracle SQL databases, multithreading, and AI/ML techniques and should be proficient in deploying Python applications on IIS servers. The role involves developing a system to monitor live files and folders, extract data, transform it using various techniques, and display insights on a Plotly Dash-based dashboard.

Responsibilities

- Backend & Frontend Development: Build end-to-end solutions using Python for both backend and frontend functionalities.
- Data Extraction & Transformation: Implement data cleaning, regex, formatting, and data handling to process extracted information.
- Database Management: Insert and update records in an Oracle SQL database, ensuring data integrity and efficiency.
- Live File & Folder Monitoring: Develop Python scripts using Watchdog to monitor logs, detect new files/folders, and extract data in real time. Fetch live data from the database using multithreading for smooth real-time updates.
- Data Visualization: Develop an interactive dashboard using Plotly Dash or react for real-time data representation.
- Data Analytics & Pattern Finding: Perform exploratory data analysis (EDA) to identify trends, anomalies, and key insights.
- Cloud & AI/ML Integration: Leverage AI/ML techniques for data processing.
- Deployment & Maintenance: Deploy applications on an IIS server/Cloud and ensure system scalability and security.

Qualifications

BE/BTECH degree in Computer Science, EE, or related field.

Essential skills

Strong Python programming skills
Experience with Watchdog for real-time monitoring.
Expertise in Oracle SQL (data insertion, updates, query optimization).
Knowledge of AI/ML techniques and their practical applications.
Hands-on experience with Plotly Dash/React/Angular any UI framework for dashboard development.
Familiarity with IIS deployment and troubleshooting.
Good understanding of data cleaning, ETL pipelines, and real-time data streaming.
Strong debugging and problem-solving skills.
Prior experience working on real-time monitoring systems.