ML Ops Framework Setup : 2-Week Assessment

Objective: Discover and assess existing AI ML processes, create an MLOPs roadmap, and identify gaps in the current setup.

Key Challenges Addressed:

1. Data/model drift causes models to become ineffective over time
2. No centralized way to measure model performance
3. Poor adoption of models due to lack of model understanding
4. Disconnected deployment process

Outcome: An assessment of the existing MLOPS landscape for the following:

1. Assessment of model monitoring: a. Model drift - which includes Overall drift, drift trend, feature wise drift b. A centralized dashboard for persona-based insights for business users and data engineer. c. Tracking model execution and failures
2. Assessment of Model testing processes and recommendations on the same.
3. Assessment of existing model deployment practices and automation / standardization of CI/CD process.
4. Recommendations around Explainable AI that provides Global feature Importance, Local explanation for model explanations

Plan:

1. Week 1: a. Scope Identification: Business Use case b. Current Process discovery, identify challenges and gaps c. Understand tech-stack, design, and architecture
2. Week 2: a. Enable end customers in prioritizing top key components with respect to MLOps b. Share project plan/roadmap with respect to scope of the work c. Share recommendations.

MLOPs offering uses the following native Azure components for the proposed next steps/roadmap:

1. App Services: The monitoring web app and python backend code is hosted on azure Linux app services. Both the apps can be scaled automatically or manually on demand.
2. Microsoft Azure Data Factory: Used to fetch the status information of Data factory pipelines to track.
3. Databricks Workspace: MLFlow component of Databricks is used to fetch the data stored by notebook during execution.
4. Cosmos DB: With the flexibility of schema and changing nature of data, NoSQL helps accommodate requirements.