Download PDF

FAIRLY Quick Start Guide

Step 1: Create an AI  (Watch tutorial - https://youtu.be/HhCRHkkQAWk)

1. From the top navigation bar, go to “Quick Start”.  
2. Select the first step “Create an AI”.
3. Click on the “Go to AI Center” button.
4. Click on the “New AI” button.
5. Enter a name for your AI, then click on the “Submit” button.

Note: Do not select any control bundles when you are creating an AI for the purpose of this training. (Controls are  mandatory requirements for an AI. If an AI has selected control bundles and it did not satisfy those requirements, you will not be able to create the report by design.)

Upload Datasets
1. From the top navigation bar, go to “AI Center” under “Compliance”. 
2. Click on the AI you have just created.
3. To upload a dataset, click on the “Datasets” panel and then click on the “New Dataset” button.
4. Enter a name for your dataset and upload a dataset in the .csv file format.

Create Model Record
1. From the top navigation bar, go to “AI Center” under “Compliance”. 
2. Select the AI you have just created.
3. To create a model record, click on the “Models” panel and then click on the “New Model” button.
4. Enter a name for your model record and select the model algorithm “Random Forest” for the purpose of this training.

Step 2: Analyze Risk

Data Validation (pre-model checks)
(Watch tutorial https://youtu.be/3Wd796iJfJk)

Prerequisites
1. Create an AI in step 1 first  if you don’t have an existing AI already.
2. Upload a configuration dataset as well as a dataset for validation using Upload Dataset instructions in step 1 also before proceeding.  

Instructions
1. From the top navigation bar, go to “Data Validation” under “Risk”.  
2. Select an AI using the first dropdown and a dataset using the second dropdown.
3. Click on the “Submit” button to view Data Validation results.

Data Drift (Watch Tutorial - https://youtu.be/PhLQv1JTIJs)

Prerequisites
1. Create an AI in step 1 first  if you don’t have an existing AI already.
2. Upload a reference dataset as well as a current dataset also before proceeding. 

Instructions
1. From the top navigation bar, go to “Data Drift” under “Risk”.
2. Select an AI using the first dropdown. 
3. Select your reference dataset and comparison dataset using the second row of dropdowns.
4. Click on the “Submit” button to view Data Drift results.

Bias Inspector (Watch tutorial - https://youtu.be/MRsW82VF4Rs)

Prerequisites
1. Create an AI in step 1 first  if you don’t have an existing AI already.
2. Have a  binary dataset with labeled protected features in .csv format.
3. If you are going to create a model result dataset analysis, need to create a model record in step 1 first.

Instructions
1. From the top navigation bar, go to "Bias Inspector" under "Risk". 

The Quick Bias Inspector will allow you to upload a csv file (binary data) of a model test set or a dataset without linking anything to an existing AI, dataset, or model. Please note that no one else in your organization will be able to view these results.

The AI-Based Bias Inspector will allow you to upload model test sets and datasets and relate them to your existing AI's, models, and datasets. Members in your organization and AI will also be able to view and modify this data.

2. For the purpose of this training, click on the “AI-Based Inspector” button:

To view previous results click on the “Dashboard” button.

To perform a new inspection select an AI and click on the “Next” button. Here you have two options:

Select Model Analysis to analyze a model result dataset: 
a. Select a model record in the dropdown and click on the “Next” button.
b. If you have not uploaded any model results select “Upload New Model Result Dataset” to upload a new model result dataset (in binary format).

Select Dataset Analysis to analyze a training dataset

3. Enter a name for your analysis..
4. Upload a binary dataset in .csv format for the analysis.
5. Select your target values and protected labels.
6. Click on the “Submit” button to upload.
7. You will be brought to the dashboard screen where you can view the results. To return to Increment a dataset navigate back to the AI Based Inspector.

To increment a dataset, select “Increment Dataset” after you have selected your AI in the AI-Based Bias Inspector:

a. Upload the file you wish to increment with. At this point you may also change the parameters of this dataset.
b. Click on the “Submit” button to upload this incrementation.
c. You will be brought to the dashboard screen where you can view your results. 

To increment a model result dataset, select “Increment Model Result Dataset” after you have selected your AI in the AI-Based Bias Inspector.

a. Upload the file you wish to increment with.  
b. Click on the “Submit” button to upload this incrementation.
c. You will be brought to the dashboard screen where you can view your results.

Explainability tool (Watch tutorial - https://youtu.be/W37ft_Sx-4M)
FOR USERS IN DATA SCIENCE ROLES

Prerequisites
1. Create an AI in step 1 first  if you don’t have an existing AI already.
2. Have a Python environment (e.g. Jupyter notebook).
3. Python libraries required to run the tutorial_notebook_neural_network.ipynb
pandas
keras
scikit-learn
numpy
shap
json
mlflow
datetime
abc
requests
urllib
shutil
yaml
lime
matplotlib
operator
base64
copy
io (edited)


Installation
1. Download client-tutorials zip file from https://www.fairly.ai/pov-tutorials.
2. Extract all files from the client-tutorials .zip file and save them to a [local directory].

Skip steps 3 to 6 if you are using Jupyter notebook from Anaconda. Otherwise, proceed to setup on your virtual environment:

3. Make sure that you are extracting the .zip file in your WSL file structure, not your default Windows file structure (ensure the file path begins with “\\wsl$\”)
4. Open a WSL terminal and navigate into asenion-client-pov directory (cd asenion-client-pov).
5. Create your virtual environment by running the following command in your terminal: python3 -m venv [environment name]. It is a good idea to give your virtual environment a simple name, such as .venv.
6. Activate your virtual environment using the command: source env/bin/activate.
7. Install all the python package dependencies by running: pip install -r requirements.txt.
This will install all the required packages for running asenion-client and any FAIRLY tutorial notebooks. If the command fails, you will have to install the packages manually, running the command pip install [library name].

Configuration 
1. Go to the FAIRLY app  and ensure you are logged into your account.
2. From the top navigation bar, go to “Report Center” under “Compliance”.  Click on the “New Report” button.
3. Click on the “Select AI” button. 
4. Click the “Download Yaml” link next to the AI you want to generate a report for.
5. Save the fairly.yaml file to the [local directory]/asenion-client-pov/ directory. (Please ensure that the file is still named “fairly.yaml”.)

Execution
1. Navigate to [local directory]/asenion-client-pov/notebooks/tutorial_notebook_neural_network.ipynb.
2. Run all cells. (If you received an SSL certification error. You will need to check with your system admin to install your organization's custom SSL certificate in your env. Please contact support@fairly.ai for assistance.)
3. Verify that the SHAP and LIME charts are generated successfully without any errors in the notebook. 

FOR EVERYONE

Prerequisites
1. A technical user already created an AI and executed an Explainability tutorial using steps above.

Instructions
1. From the top navigation bar, go to “Explainability Tool” under “Risk”.  
2. Select an AI using the first dropdown and a model using the second dropdown.
3. Click on the “Submit” button to view Explainability Tool results.

Step 3: Create an AI Report (Watch tutorial - https://youtu.be/4-JgdTevoU0

Prerequisite
1. Create an AI in step 1 first  if you don’t have an existing AI already.

Report Generation
1. From the top navigation bar, go to “Report Center” under “Risk”.  
2. Click on the “New Report” button to go to the Report Builder page.
3. Click on the “Select AI” button in step 1 of the Report Builder page.
4. Select the radio button next to the AI you wish to generate a report from. Click on the “Done” button.
5. Select the report type “AI Report” in step 2 of the Report Builder page. Click on the “Next” button.
6. On the Report Add-On page, you can only select the checkbox of the risk analysis you have completed for this AI.  For the purpose of this training, check all the checkboxes that apply to your AI. Click on the “Next” button.
7. Complete deliverable information and click on the “Next” button.
8. Fill out additional information for your risk analysis:

On the next few screens of the Report Builder, you will be prompted to fill out information about each add-on you have selected.
a. Each text box is required to be filled so if you have no comments or input for that field simply write “N/A” in the text box.
b. After you have filled out all boxes, you will see a Report Completion Status summary screen.  On this screen you will see if all the fields have had content entered correctly.  If there is no content and you see a red warning icon, please hit the back button and fill in the missing boxes.

9. After all sections are complete (mark with a blue shield icon), click on the “Submit” button to generate your report.
Once your report(s) has finished generating, you are able to download a complete AI report that includes all risk analysis as well as add-on reports that only include information about an individual risk analysis.  
10. Download all reports you would like to see.

Review report 

1. Open your downloaded reports in a PDF viewer.
2. You should be able to view all the information that was uploaded relating to the corresponding risk analysis you had run on the AI.



Questions, suggestions or found errors on this page? Please send your feedback to support@fairly.ai.