AIClubPro

Feature Engineering and KNN

​

This session builds upon the first. We will go over an important aspect of the ML pipeline, Feature Engineering. We will go over the transformations that are required before a dataset can be used by an AWS ML algorithm for building a model. We also examine the use case of Pricing Analysis and how a production AI based on Regression techniques can address this use case. We follow the lifecycle steps shown in the first session, but delve into more depth on how to perform appropriate feature engineering for different scenarios. We also introduce attendees to an ML algorithm called KNN and how it can be used for both regression and classification problems. In the code lab, attendees will perform feature engineering transforms, save the transformed data back in S3 and get started with configuring a KNN algorithm in AWS SageMaker.

​

• How to perform feature engineering as a part of the AI life cycle in production

• Overview of a business Pricing Problem and how to build an AI for it using Regression Algorithms

• Common approaches to Feature Engineering - One Hot Encoding and Missing Value handling

• Lab 2: Perform feature engineering on two datasets, Pricing and Churn. Build an AI in the cloud using AWS SageMaker (KNN)

Evaluating effectiveness of production AI and XGBoost

In this session, we will explore a new algorithm called XGBoost and configure its hyper-parameters in AWS SageMaker. We will also learn how to evaluate the performance of an AI algorithm and use this criterion to tune the hyper-parameters and pick the best model. In particular, we delve into evaluation metrics beyond accuracy and cover both ML metrics (such as Confusion Matrix) and production service metrics (such as latency and scale) that are important for production. As a case study, we will use the Pricing and Churn datasets that were introduced in the previous sessions

​​

• How to build and configure hyper-parameters for both regression and classification type of problems

• Understand the hyper-parameters exposed by AWS SageMaker and use KNN and XGBoost for performing training on the two types of problems: Classification and Regression

• How hyper-parameters affect solution quality and performance

• Lab 3: Train and evaluate several ML algorithms in the cloud using AWS SageMaker and AWS CloudWatch

AI deployment as a microservice in the Cloud

In this session, we will focus on deployment of the trained ML model into production as a microservice. This will involve several aspects, such as endpoint creation, IAM role creation and configuration of a Lambda and API Gateway. This session will focus on the application integration of production AI and how to test and evaluate this integration. In the code lab, attendees will build an end to end working AI and perform external evaluation of the AI to access its performance.

​

• How to deploy an AI in the cloud as a microservice

• Evaluate a production AI with both ML metrics (like Accuracy, Confusion Matrix, True/False Positive/Negative etc.) and code metrics (throughput, accuracy, etc.)

• How to test their AI service using CURL and Postman

• How to integrate their AI service into Python, Java or Javascript applications

• Lab 4: Build an end to end production pipeline in the cloud

Custom AI algorithms in the Cloud: Use Case - Sentiment Analysis

In this session, we will focus on creating custom algorithms and using them in the cloud. We dive deep into algorithms for Text Classification, particularly the Bag of Words approach, its operation, advantages and limitations. We will look into how AWS SageMaker allows the deployment of custom algorithms, such as text classification using bag of words. This will involve building a custom docker container and configuring an AWS client in the local environment to push the container into AWS ECR. This session will focus on building custom algorithms and deploying them as a microservice. In the code lab, you will build a custom algorithm (Sentiment analysis use case) and create a docker container that conforms to AWS SageMaker requirements for custom algorithms. You will build an end to end working AI for the custom algorithm

​

• How to build and evaluate production AIs that use Text Classification algorithms

• Overview of a business Sentiment Analysis problem and how to build an AI for it using Text Classification in either Binary or Multiclass form

• How to build custom AI algorithms

• How to build docker containers with the custom algorithms that conform to AWS requirements.

• How to push the docker container into AWS ECR so that they can be used by SageMaker.

• Lab 5: build an end to end production pipeline in the cloud using custom algorithms.

Skew in AI. How to Detect and Remove them in Production AI.

In this session, we will spend a part of the time to finish up the end to end lifecycle using the custom algorithm described in the previous session as it typically takes more than one session to cover. After that, we focus on the critical issue of data Skew in AI. and AI Bias. We cover how skew in data impacts the performance of AI and steps that can be taken to eliminate it. We will also cover the common ways that Bias can enter a production AI implementation and how to detect and remove Bias. As a case study, we use the publicly available Appointment planning and COMPAS dataset. In the code lab, attendees will build several AIs using these datasets. We will also introduce the overlay tool Pyxeda Navigator and provide resources for attendees to use it

​

• What is data Skew and how to overcome it.

• AI Bias and why should it be avoided?

• How can AI Bias enter? How can it be detected and removed?

• How to test for Bias and apply Feature Engineering to reduce it.

• Lab 6: Build several AIs and perform analysis to detect skew and bias

Mapping AI problems to Techniques and Drift in Production: Case Study - Making Recommendations

In the first 6 sessions, we have covered different types of use cases, how to map AI to each use case, and different aspects of production AI (hyper parameter tuning, REST API test and integration, etc.) In this session we combine these into a methodology for mapping problems to AI methods for numerical, categorical and free form text data. We cover in depth another powerful topic called Drift and its impact on real world production deployments. In the code lab, attendees will build a production recommendation system using publicly available data, illustrating the concepts covered so far

​

• A methodology for mapping use cases (with different data types) into AI algorithms and lifecycle steps

• The concept of drift in production and how it impacts a Business

• Techniques to detect drift in production

• Lab 7: Build and use a production AI for recommendations

Advanced topics in Production Cloud AI, Complete and showcase your project.​