Merge pull request #776 from dianna-ai/overview_and_tutorials_change

Overview tutorial and other tutorial changes

Author: elboyran

README.md

@@ -109,116 +109,87 @@ If you get an error related to OpenMP when importing dianna, have a look at [thi
 You need:
 
 - your trained ONNX model ([convert my pytorch/tensorflow/keras/scikit-learn model to ONNX](https://github.com/dianna-ai/dianna#onnx-models))
-- 1 data item to be explained
+- a data item to be explained
 
  You get:
 
 - a relevance map overlayed over the data item
 
-In the library's documentation, the general usage is explained in [How to use DIANNA](https://dianna.readthedocs.io/en/latest/usage.html)
+### Template example for any data modality and explainer
 
-### Demo movie
-
-[![Watch the video on YouTube](https://img.youtube.com/vi/u9_c5DJewLU/default.jpg)](https://youtu.be/u9_c5DJewLU)
-
-### Text example:
+1. Provide your *trained model* and *data item* ( *text, image, time series or tabular* )
 
 ```python
-model_path = 'your_model.onnx'  # model trained on text
-text = 'The movie started great but the ending is boring and unoriginal.'
+model_path = 'your_model.onnx'  # model trained on your data modality
+data_item = <data_item> # data item for which the model's prediction needs to be explained 
 ```
 
-Which of your model's classes do you want an explanation for?
+2. If the task is classification: which are the *classes* your model has been trained for?
 
-```python
-labels = [positive_class, negative_class]
+```python 
+labels = [class_a, class_b]   # example of binary classification labels
 ```
-
-Run using the XAI method of your choice, for example LIME:
-
+*Which* of these classes do you want an explanation for?
 ```python
-explanation = dianna.explain_text(model_path, text, 'LIME')
-dianna.visualization.highlight_text(explanation[labels.index(positive_class)], text)
+explained_class_index = labels.index(<explained_class>)  # explained_class can be any of the labels
 ```
 
-![image](https://user-images.githubusercontent.com/6087314/155532504-6f90f032-cbb4-4e71-9b99-aa9c0de4e86a.png)
-
-### Image example:
+3. Run dianna with the *explainer* of your choice ( *'LIME', 'RISE' or 'KernalSHAP'*) and visualize the output:
 
 ```python
-model_path = 'your_model.onnx'  # model trained on images
-image = PIL.Image.open('your_image.jpeg')
+explanation = dianna.<explanation_function>(model_path, data_item, explainer)
+dianna.visualization.<visualization_function>(explanation[explained_class_index], data_item)
 ```
 
-Tell us what label refers to the channels, or colors, in the image.
+### Text and image usage examples
+Lets illustrate the template above with *textual* data. The data item of interest is a sentence being (a part of) a movie review and the model has been trained to classify reviews into positive and negative sentiment classes.
+We are intersted which words are contributing positively (red) and which - negatively (blue) towards the model's desicion to classify the review as positive and we would like to use the *LIME* explainer:
 
 ```python
-axis_labels = {0: 'channels'}
+model_path = 'your_text_model.onnx'
+# also define a model runner here (details in dedicated notebook)
+review = 'The movie started great but the ending is boring and unoriginal.' 
+labels = ["negative", "positive"] 
+explained_class_index = labels.index("positive")  
+explanation = dianna.explain_text(model_path, text, 'LIME')
+dianna.visualization.highlight_text(explanation[explained_class_index], model_runner.tokenizer.tokenize(review))
 ```
 
-Which of your model's classes do you want an explanation for?
-
-```python
-labels = [class_a, class_b]
-```
+![image](https://user-images.githubusercontent.com/6087314/155532504-6f90f032-cbb4-4e71-9b99-aa9c0de4e86a.png)
 
-Run using the XAI method of your choice, for example RISE:
+Here is another illustration on how to use dianna to explain which parts of a bee *image* contributied positively (red) or negativey (blue) towards a classifying the image as a *'bee'* using *RISE*. 
+The Imagenet model has been trained to distinguish between 1000 classes (specified in ```labels```).
+For images, which are data of higher dimention compared to text, there are also some specifics to consider:
 
 ```python
+model_path = 'your_image_model.onnx' 
+image = PIL.Image.open('your_bee_image.jpeg') 
+axis_labels = {2: 'channels'} 
+explained_class_index = labels.index('bee') 
 explanation = dianna.explain_image(model_path, image, 'RISE', axis_labels=axis_labels, labels=labels)
-dianna.visualization.plot_image(explanation[labels.index(class_a)], original_data=image)
+dianna.visualization.plot_image(explanation[explained_class_index], utils.img_to_array(image)/255., heatmap_cmap='bwr')
+plt.show()
 ```
+<img src="https://github.com/dianna-ai/dianna/assets/3244249/b03e4d4e-e3e8-4248-bf62-e3602b7f6d71" width="215" height="215">
 
-![image](https://user-images.githubusercontent.com/6087314/155557077-e2052094-d8ac-49d3-a840-0160256d53a6.png)
-
-### Time-series example:
-
+And why would Imagenet think the same image would be a *garden spider*?
 ```python
-model_path = 'your_model.onnx'  # model trained on images
-timeseries_instance = pd.read_csv('your_data_instance.csv').astype(float)
-
-num_features = len(timeseries_instance)  # The number of features to include in the explanation.
-num_samples = 500  # The number of samples to generate for the LIME explainer.
-```
-
-Which of your model's classes do you want an explanation for?
-
-```python
-class_names= [class_a, class_b] # String representation of the different classes of interest
-labels = np.argsort(class_names) # Numerical representation of the different classes of interest for the model
-```
-
-Run using the XAI method of your choice, for example LIME with the following additional arguments:
-
-```python
-explanation = dianna.explain_timeseries(model_path, timeseries_data=timeseries_instance , method='LIME', 
-					labels=labels, class_names=class_names, num_features=num_features,
-                                	num_samples=num_samples, distance_method='cosine')
-
-```
-
-For visualization of the heatmap please refer to the [tutorial](https://github.com/dianna-ai/dianna/blob/main/tutorials/explainers/LIME/lime_timeseries_coffee.ipynb)
-
-### Tabular example:
-
-```python
-model_path = 'your_model.onnx'  # model trained on tabular data
-tabular_instance = pd.read_csv('your_data_instance.csv')
+explained_class_index = labels.index('garden_spider') # interested in the image being classified as a garden spider
+explanation = dianna.explain_image(model_path, image, 'RISE', axis_labels=axis_labels, labels=labels)
+dianna.visualization.plot_image(explanation[explained_class_index], utils.img_to_array(image)/255., heatmap_cmap='bwr')
+plt.show()
 ```
 
-Run using the XAI method of your choice. Note that you need to specify the mode, either regression or classification. This case, for instance a regression task using KernelSHAP with the following additional arguments:
+<img src="https://github.com/dianna-ai/dianna/assets/3244249/e7623803-2369-40ad-b4ef-4a6ae4e902f1" width="215" height="215">
 
-```python
-explanation = dianna.explain_tabular(run_model, input_tabular=data_instance, method='kernelshap',
-                                     mode ='regression', training_data = X_train,
-                                     training_data_kmeans = 5, feature_names=input_features.columns)
-plot_tabular(explanation, X_test.columns, num_features=10)  # display 10 most salient features
-```
+### Overview tutorial
+There are **full working examples** on how to use the supported explainers and how to use dianna for **all supported data modalities** in our [overview tutorial](./tutorials/overview.ipynb).
 
-![image](https://github.com/dianna-ai/dianna/assets/25911757/ce0b76b8-f00c-468a-9732-c21704e289f6)
+#### Demo movie (update planned): 
+[![Watch the video on YouTube](https://img.youtube.com/vi/u9_c5DJewLU/default.jpg)](https://youtu.be/u9_c5DJewLU)
 
 ### IMPORTANT: Sensitivity to hyperparameters
-The XAI methods (explainers) are sensitive to the choice of their hyperparameters! In this [work](https://staff.fnwi.uva.nl/a.s.z.belloum/MSctheses/MScthesis_Willem_van_der_Spec.pdf), this sensitivity to hyperparameters is researched and useful conclusions are drawn.
+The explainers are sensitive to the choice of their hyperparameters! In this [work](https://staff.fnwi.uva.nl/a.s.z.belloum/MSctheses/MScthesis_Willem_van_der_Spec.pdf), this sensitivity to hyperparameters is researched and useful conclusions are drawn.
 The default hyperparameters used in DIANNA for each explainer as well as the values for our tutorial examples are given in the Tutorials [README](./tutorials/README.md#important-hyperparameters).
 
 ## Dashboard
@@ -252,7 +223,7 @@ DIANNA comes with simple datasets. Their main goal is to provide intuitive insig
 
 | Dataset                                                                                                                                                                                                                | Description                                                                                                                                                    | Examples                                                                                                                                 | Generation                                                                |
 | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------ |
-| Coffee dataset  <img width="25" alt="Coffe Logo" src="https://github.com/dianna-ai/dianna/assets/3244249/9ab50a0f-5da3-41d2-80e9-70d2c8769162"> | Food spectographs time series dataset for a two class problem to distinguish between Robusta and Arabica coffee beans.                                         | <img width="500" alt="example image" src="https://github.com/dianna-ai/dianna/assets/3244249/763002c5-40ad-48cc-9de0-ea43d7fa8a75)"> | [data source](https://github.com/QIBChemometrics/Benchtop-NMR-Coffee-Survey) |
+| [Coffee dataset](https://www.timeseriesclassification.com/description.php?Dataset=Coffee)  <img width="25" alt="Coffe Logo" src="https://github.com/dianna-ai/dianna/assets/3244249/9ab50a0f-5da3-41d2-80e9-70d2c8769162"> | Food spectographs time series dataset for a two class problem to distinguish between Robusta and Arabica coffee beans.                                         | <img width="500" alt="example image" src="https://github.com/dianna-ai/dianna/assets/3244249/763002c5-40ad-48cc-9de0-ea43d7fa8a75)"> | [data source](https://github.com/QIBChemometrics/Benchtop-NMR-Coffee-Survey) |
 | [Weather dataset](https://zenodo.org/record/7525955) <img width="25" alt="Weather Logo" src="https://github.com/dianna-ai/dianna/assets/3244249/3ff3d639-ed2f-4a38-b7ac-957c984bce9f">                                | The light version of the weather prediciton dataset, which contains daily observations (89 features) for 11 European locations through the years 2000 to 2010. | <img width="500" alt="example image" src="https://github.com/dianna-ai/dianna/assets/3244249/b0a505ac-8a6c-4e1c-b6ad-35e31e52f46d)"> | [data source](https://github.com/florian-huber/weather_prediction_dataset)   |
 
 ### Tabular 

dianna/visualization/text.py

@@ -6,7 +6,7 @@ def highlight_text(explanation,
                    input_tokens=None,
                    show_plot=True,
                    output_filename=None,
-                   colormap="RdBu",
+                   colormap="bwr",
                    alpha=1.0,
                    heatmap_range=(-1, 1)):
     """Highlights a given text based on values in a given explanation object.

docs/tutorials/0-overview.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../../tutorials/overview.ipynb"
+}

docs/tutorials/lime_images.nblink docs/tutorials/1-lime_images.nblink

@@ -1,7 +1,7 @@
 <img width="150" alt="Logo_ER10" src="https://user-images.githubusercontent.com/3244249/151994514-b584b984-a148-4ade-80ee-0f88b0aefa45.png">
 
 ## Tutorials
-This folder contains DIANNA tutorial notebooks. To install the dependencies for the tutorials, run
+This folder contains DIANNA tutorial notebooks. To install the dependencies for the tutorials, run (in the main dianna folder)
 ```
 pip install .[notebooks]
 ```
@@ -24,7 +24,7 @@ pip install .[notebooks]
 ||[Simple Geometric (circles and triangles)](https://doi.org/10.5281/zenodo.5012824)| Binary shape *classificaiton* |<img width="20" alt="SimpleGeometric Logo" src="https://user-images.githubusercontent.com/3244249/151539027-f2fc3fc0-282a-4993-9680-74ee28bcd360.png">|
 ||[Imagenet](https://image-net.org/download.php) |$1000$ classes natural images *classificaiton* | <img width="94" alt="ImageNet_autocrop" src="https://user-images.githubusercontent.com/3244249/152542090-fd78fde1-6dec-43b6-a7ae-eea964b8ae28.png">|
 |*Text*| [Stanford sentiment treebank](https://nlp.stanford.edu/sentiment/index.html) |Positive or negative movie reviews sentiment *classificaiton* | <img width="25" alt="nlp-logo_half_size" src="https://user-images.githubusercontent.com/3244249/152540890-c8e1e37d-f0cc-4f84-80a4-2c59176cbf4c.png">|
-|*Timeseries* | Coffee dataset  | Binary *classificaiton* of Robusta and Aribica coffee beans| <img width="25" alt="Coffe Logo" src="https://github.com/dianna-ai/dianna/assets/3244249/9ab50a0f-5da3-41d2-80e9-70d2c8769162">|
+|*Timeseries* | [Coffee dataset](https://www.timeseriesclassification.com/description.php?Dataset=Coffee)  | Binary *classificaiton* of Robusta and Aribica coffee beans| <img width="25" alt="Coffe Logo" src="https://github.com/dianna-ai/dianna/assets/3244249/9ab50a0f-5da3-41d2-80e9-70d2c8769162">|
 |           | [Weather dataset](https://zenodo.org/record/7525955) |Binary *classification* (summer/winter) of temperature time-series |<img width="25" alt="Weather Logo" src="https://github.com/dianna-ai/dianna/assets/3244249/3ff3d639-ed2f-4a38-b7ac-957c984bce9f">|
 |*Tabular*| [Penguin dataset](https://www.kaggle.com/code/parulpandey/penguin-dataset-the-new-iris)| $3$ penguin spicies (Adele, Chinstrap, Gentoo) *classificaiton*  | <img width="75" alt="Penguin Logo" src="https://github.com/dianna-ai/dianna/assets/3244249/c7716ad3-f992-4557-80d9-1d8178c7ed57"> | |
 |           | [Weather dataset](https://zenodo.org/record/7525955) | Next day sunshine hours prediction (*regression*) | <img width="25" alt="Weather Logo" src="https://github.com/dianna-ai/dianna/assets/3244249/3ff3d639-ed2f-4a38-b7ac-957c984bce9f">|