tutorials/33_Hybrid_Retrieval.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kTas9ZQ7lXP7"
   },
   "source": [
    "# Tutorial: Creating a Hybrid Retrieval Pipeline\n",
    "\n",
    "- **Level**: Intermediate\n",
    "- **Time to complete**: 15 minutes\n",
    "- **Components Used**: [`DocumentSplitter`](https://docs.haystack.deepset.ai/docs/documentsplitter), [`SentenceTransformersDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [`DocumentJoiner`](https://docs.haystack.deepset.ai/docs/documentjoiner), [`InMemoryDocumentStore`](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [`InMemoryBM25Retriever`](https://docs.haystack.deepset.ai/docs/inmemorybm25retriever), [`InMemoryEmbeddingRetriever`](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), and [`TransformersSimilarityRanker`](https://docs.haystack.deepset.ai/docs/transformerssimilarityranker)\n",
    "- **Prerequisites**: None\n",
    "- **Goal**: After completing this tutorial, you will have learned about creating a hybrid retrieval and when it's useful."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0hw_zoKolXQL"
   },
   "source": [
    "## Overview\n",
    "\n",
    "**Hybrid Retrieval** combines keyword-based and embedding-based retrieval techniques, leveraging the strengths of both approaches. In essence, dense embeddings excel in grasping the contextual nuances of the query, while keyword-based methods excel in matching keywords.\n",
    "\n",
    "There are many cases when a simple keyword-based approaches like BM25 performs better than a dense retrieval (for example in a specific domain like healthcare) because a dense model needs to be trained on data. For more details about Hybrid Retrieval, check out [Blog Post: Hybrid Document Retrieval](https://haystack.deepset.ai/blog/hybrid-retrieval)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ITs3WTT5lXQT"
   },
   "source": [
    "## Preparing the Colab Environment\n",
    "\n",
    "- [Enable GPU Runtime in Colab](https://docs.haystack.deepset.ai/docs/enabling-gpu-acceleration)\n",
    "- [Set logging level to INFO](https://docs.haystack.deepset.ai/docs/setting-the-log-level)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2g9fhjxDlXQb"
   },
   "source": [
    "## Installing Haystack\n",
    "\n",
    "Install Haystack and other required packages with `pip`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "L40ZxZW8lXQh"
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "pip install haystack-ai\n",
    "pip install \"datasets>=2.6.1\"\n",
    "pip install \"sentence-transformers>=3.0.0\"\n",
    "pip install accelerate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CJBcPNbBlXQq"
   },
   "source": [
    "### Enabling Telemetry\n",
    "\n",
    "Knowing you're using this tutorial helps us decide where to invest our efforts to build a better product but you can always opt out by commenting the following line. See [Telemetry](https://docs.haystack.deepset.ai/docs/telemetry) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "lUbTGVo4lXQv"
   },
   "outputs": [],
   "source": [
    "from haystack.telemetry import tutorial_running\n",
    "\n",
    "tutorial_running(33)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "usdANiAGlXQ9"
   },
   "source": [
    "## Initializing the DocumentStore\n",
    "\n",
    "You'll start creating your question answering system by initializing a DocumentStore. A DocumentStore stores the Documents that your system uses to find answers to your questions. In this tutorial, you'll be using the [`InMemoryDocumentStore`](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "cLbh-UtelXRL"
   },
   "outputs": [],
   "source": [
    "from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
    "\n",
    "document_store = InMemoryDocumentStore()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GZWBHcc8TKcv"
   },
   "source": [
    "> `InMemoryDocumentStore` is the simplest DocumentStore to get started with. It requires no external dependencies and it's a good option for smaller projects and debugging. But it doesn't scale up so well to larger Document collections, so it's not a good choice for production systems. To learn more about the different types of external databases that Haystack supports, see [DocumentStore Integrations](https://haystack.deepset.ai/integrations?type=Document+Store&version=2.0)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0rk8fdMzTb-I"
   },
   "source": [
    "## Fetching and Processing Documents\n",
    "\n",
    "As Documents, you will use the PubMed Abstracts. There are a lot of datasets from PubMed on Hugging Face Hub; you will use [anakin87/medrag-pubmed-chunk](https://huggingface.co/datasets/anakin87/medrag-pubmed-chunk) in this tutorial.\n",
    "\n",
    "Then, you will create Documents from the dataset with a simple for loop.\n",
    "Each data point in the PubMed dataset has 4 features:\n",
    "* *pmid*\n",
    "* *title*\n",
    "* *content*: the abstract\n",
    "* *contents*: abstract + title\n",
    "\n",
    "For searching, you will use the *contents* feature. The other features will be stored as metadata, and you will use them to have a **pretty print** of the search results or for [metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RvrG_QzirSsq"
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "from haystack import Document\n",
    "\n",
    "dataset = load_dataset(\"anakin87/medrag-pubmed-chunk\", split=\"train\")\n",
    "\n",
    "docs = []\n",
    "for doc in dataset:\n",
    "    docs.append(\n",
    "        Document(content=doc[\"contents\"], meta={\"title\": doc[\"title\"], \"abstract\": doc[\"content\"], \"pmid\": doc[\"id\"]})\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XPngNEs5q8Tw"
   },
   "source": [
    "## Indexing Documents with a Pipeline\n",
    "\n",
    "Create a pipeline to store the data in the document store with their embedding. For this pipeline, you need a [DocumentSplitter](https://docs.haystack.deepset.ai/docs/documentsplitter) to split documents into chunks of 512 words, [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder) to create document embeddings for dense retrieval and [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter) to write documents to the document store.\n",
    "\n",
    "As an embedding model, you will use [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) on Hugging Face. Feel free to test other models on Hugging Face or use another [Embedder](https://docs.haystack.deepset.ai/docs/embedders) to switch the model provider.\n",
    "\n",
    "> If this step takes too long for you, replace the embedding model with a smaller model such as `sentence-transformers/all-MiniLM-L6-v2` or `sentence-transformers/all-mpnet-base-v2`. Make sure that the `split_length` is updated according to your model's token limit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RrIN83JNCHhX"
   },
   "outputs": [],
   "source": [
    "from haystack.components.writers import DocumentWriter\n",
    "from haystack.components.embedders import SentenceTransformersDocumentEmbedder\n",
    "from haystack.components.preprocessors.document_splitter import DocumentSplitter\n",
    "from haystack import Pipeline\n",
    "from haystack.utils import ComponentDevice\n",
    "\n",
    "document_splitter = DocumentSplitter(split_by=\"word\", split_length=512, split_overlap=32)\n",
    "document_embedder = SentenceTransformersDocumentEmbedder(\n",
    "    model=\"BAAI/bge-small-en-v1.5\", device=ComponentDevice.from_str(\"cuda:0\")\n",
    ")\n",
    "document_writer = DocumentWriter(document_store)\n",
    "\n",
    "indexing_pipeline = Pipeline()\n",
    "indexing_pipeline.add_component(\"document_splitter\", document_splitter)\n",
    "indexing_pipeline.add_component(\"document_embedder\", document_embedder)\n",
    "indexing_pipeline.add_component(\"document_writer\", document_writer)\n",
    "\n",
    "indexing_pipeline.connect(\"document_splitter\", \"document_embedder\")\n",
    "indexing_pipeline.connect(\"document_embedder\", \"document_writer\")\n",
    "\n",
    "indexing_pipeline.run({\"document_splitter\": {\"documents\": docs}})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wSJlHxrhgQby"
   },
   "source": [
    "Documents are stored in `InMemoryDocumentStore` with their embeddings, now it's time for creating the hybrid retrieval pipeline ✅"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UgMgY-d9najg"
   },
   "source": [
    "## Creating a Pipeline for Hybrid Retrieval\n",
    "\n",
    "Hybrid retrieval refers to the combination of multiple retrieval methods to enhance overall performance. In the context of search systems, a hybrid retrieval pipeline executes both traditional keyword-based search and dense vector search, later ranking the results with a cross-encoder model. This combination allows the search system to leverage the strengths of different approaches, providing more accurate and diverse results.\n",
    "\n",
    "Here are the required steps for a hybrid retrieval pipeline:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ha8pNmnvqj4n"
   },
   "source": [
    "### 1) Initialize Retrievers and the Embedder\n",
    "\n",
    "Initialize a [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever) and [InMemoryBM25Retriever](https://docs.haystack.deepset.ai/docs/inmemorybm25retriever) to perform both dense and keyword-based retrieval. For dense retrieval, you also need a [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder) that computes the embedding of the search query by using the same embedding model `BAAI/bge-small-en-v1.5` that was used in the indexing pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "DVfQvnWYrMWr"
   },
   "outputs": [],
   "source": [
    "from haystack.components.retrievers.in_memory import InMemoryBM25Retriever, InMemoryEmbeddingRetriever\n",
    "from haystack.components.embedders import SentenceTransformersTextEmbedder\n",
    "\n",
    "text_embedder = SentenceTransformersTextEmbedder(\n",
    "    model=\"BAAI/bge-small-en-v1.5\", device=ComponentDevice.from_str(\"cuda:0\")\n",
    ")\n",
    "embedding_retriever = InMemoryEmbeddingRetriever(document_store)\n",
    "bm25_retriever = InMemoryBM25Retriever(document_store)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FC81c8RBrRFf"
   },
   "source": [
    "### 2) Join Retrieval Results\n",
    "\n",
    "Haystack offers several joining methods in [`DocumentJoiner`](https://docs.haystack.deepset.ai/docs/documentjoiner) to be used for different use cases such as `merge` and `reciprocal_rank_fusion`. In this example, you will use the default `concatenate` mode to join the documents coming from two Retrievers as the [Ranker](https://docs.haystack.deepset.ai/docs/rankers) will be the main component to rank the documents for relevancy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "GYso6_8BrhY8"
   },
   "outputs": [],
   "source": [
    "from haystack.components.joiners import DocumentJoiner\n",
    "\n",
    "document_joiner = DocumentJoiner()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r8_jHzmosbC_"
   },
   "source": [
    "### 3) Rank the Results\n",
    "\n",
    "Use the [TransformersSimilarityRanker](https://docs.haystack.deepset.ai/docs/transformerssimilarityranker) that scores the relevancy of all retrieved documents for the given search query by using a cross encoder model. In this example, you will use [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) model to rank the retrieved documents but you can replace this model with other cross-encoder models on Hugging Face."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "cN0woIxHs4Ng"
   },
   "outputs": [],
   "source": [
    "from haystack.components.rankers import TransformersSimilarityRanker\n",
    "\n",
    "ranker = TransformersSimilarityRanker(model=\"BAAI/bge-reranker-base\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y5jzzvUIstQ4"
   },
   "source": [
    "### 4) Create the Hybrid Retrieval Pipeline\n",
    "\n",
    "Add all initialized components to your pipeline and connect them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "y9sKO2Azjrsh"
   },
   "outputs": [],
   "source": [
    "from haystack import Pipeline\n",
    "\n",
    "hybrid_retrieval = Pipeline()\n",
    "hybrid_retrieval.add_component(\"text_embedder\", text_embedder)\n",
    "hybrid_retrieval.add_component(\"embedding_retriever\", embedding_retriever)\n",
    "hybrid_retrieval.add_component(\"bm25_retriever\", bm25_retriever)\n",
    "hybrid_retrieval.add_component(\"document_joiner\", document_joiner)\n",
    "hybrid_retrieval.add_component(\"ranker\", ranker)\n",
    "\n",
    "hybrid_retrieval.connect(\"text_embedder\", \"embedding_retriever\")\n",
    "hybrid_retrieval.connect(\"bm25_retriever\", \"document_joiner\")\n",
    "hybrid_retrieval.connect(\"embedding_retriever\", \"document_joiner\")\n",
    "hybrid_retrieval.connect(\"document_joiner\", \"ranker\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ii9x0gr9lXRT"
   },
   "source": [
    "### 5) Visualize the Pipeline (Optional)\n",
    "\n",
    "To understand how you formed a hybrid retrieval pipeline, use [draw()](https://docs.haystack.deepset.ai/docs/drawing-pipeline-graphs) method of the pipeline. If you're running this notebook on Google Colab, the generate file will be saved in \"Files\" section on the sidebar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "rXHbHru0lXRY"
   },
   "outputs": [],
   "source": [
    "hybrid_retrieval.draw(\"hybrid-retrieval.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eIYV19l16PKC"
   },
   "source": [
    "## Testing the Hybrid Retrieval\n",
    "\n",
    "Pass the query to `text_embedder`, `bm25_retriever` and `ranker` and run the retrieval pipeline:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "cf62bf3b3c6144629811874114dc527f",
      "0387b8e4546247f49f854f9729e6a3df",
      "d45dcf1e27a5401ca2c430cd6c322fdb",
      "864e853846b3406a9a8b743dbc1d96ff",
      "15f3697b990f40adb795dbcce3f626c1",
      "cf972cbfa1314f149b9860e199391170",
      "06c4d38f14674fd898aa85df17c0baa7",
      "a4d87d3d4a9a4dc4bccdc4a16be29096",
      "2398002062874506a558e59114359c54",
      "947922d2f75346cdbbb908d3888f832e",
      "c7a118b1f8a2491cacd5f76db668443e",
      "b395830b097f4862a6c8588ccd0fd91b",
      "4b9ea215e11844239e968960cd45cade",
      "097a7751c1b04aa3bf4a0a586a6dfa32",
      "9e5e169b22dd45068a2b03154b26ccf0",
      "827efd14f6f74c8ca0fb8df3a4359062",
      "2a7323e821724b129d01de39adb67c4f",
      "eeb568b15a1d465bab9f7a3843d611df",
      "c1626ca2ebf84556a692e76f16f3cab4",
      "5cd3d66281d247749d4c71daa7c6f49d",
      "107daf1c8f8f4203aff8058191ce9728",
      "31acc8ba63e240a6b4419ab4aa51c87a"
     ]
    },
    "id": "glS0-Xh3nLHY",
    "outputId": "eae6ba6e-136e-48a1-bfe3-3a97fcebca7c"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cf62bf3b3c6144629811874114dc527f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b395830b097f4862a6c8588ccd0fd91b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Ranking by BM25...:   0%|          | 0/37807 [00:00<?, ? docs/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query = \"apnea in infants\"\n",
    "\n",
    "result = hybrid_retrieval.run(\n",
    "    {\"text_embedder\": {\"text\": query}, \"bm25_retriever\": {\"query\": query}, \"ranker\": {\"query\": query}}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WvPv1cJ6gbBJ"
   },
   "source": [
    "### Pretty Print the Results\n",
    "Create a function to print a kind of *search page*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "raL_z_sByDoQ"
   },
   "outputs": [],
   "source": [
    "def pretty_print_results(prediction):\n",
    "    for doc in prediction[\"documents\"]:\n",
    "        print(doc.meta[\"title\"], \"\\t\", doc.score)\n",
    "        print(doc.meta[\"abstract\"])\n",
    "        print(\"\\n\", \"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mSUiizGNytwX",
    "outputId": "38e05986-1737-4c2b-bc08-7399299de37f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "State-of-the-Art HIV Management:An Update. \t 0.9928585290908813\n",
      "Within the past 3 years, dramatic changes have taken place in the standard of care for HIV patients. Despite improvements in care (with decreased mortality), the rate of new infections remains unchanged if not increased within most at-risk groups. This general overview is intended for the physician who, while not providing ongoing HIV care, desires an update on the major treatment issues. Current demographic trends, new methods available for testing, and the use of the viral load test for both staging and gauging response to the new combination antiretroviral treatment regimens are detailed. It is suggested that physicians consult with an experienced HIV clinician before starting a treatment regimen in the newly diagnosed patient.The primary HIV syndrome is reviewed in detail since this diagnosis is often missed and an opportunity for early intervention is lost. Physicians not providing ongoing HIV care must be comfortable making this diagnosis and doing an initial work-up. Focused prevention especially tailored to younger high-risk patients is reviewed. Treatment protocols (with an emphasis on new antiretrovirals), gauging success of treatment, and the management of treatment failures are reviewed in detail. Common antiretroviral drugs are listed with side effects, drug interactions, and average monthly costs. Care of pregnant patients and exposed healthcare workers is also briefly discussed. The need for more primary care-based prevention is also discussed.\n",
      "\n",
      " \n",
      "\n",
      "Management of tuberculosis in HIV-infected patients. \t 0.9715766310691833\n",
      "HIV-tuberculosis coinfection is currently one of the greatest health threats, affecting millions of people worldwide, with high morbidity and mortality. Treating both infections can be a challenge and requires some expertise due to multidirectional drug interactions, risk of overlapping side effects, high pill burden and risk of immune reconstitution inflammatory syndrome. This article reviews the general management of tuberculosis/HIV coinfection, focusing on the optimal time to start antiretroviral therapy and which treatments can be safely used. The randomized clinical trials designed to answer the question of when to start antiretroviral therapy (SAPIT, CAMELIA, STRIDE and TIME), published in the last two years, are described and discussed in detail. Summarizing these trials' conclusions, antiretroviral therapy should be started within two weeks of starting tuberculosis treatment if the patient has less than 50 CD4/mm3 and wait to the end of the induction phase (8-12 weeks after starting tuberculosis treatment) if higher CD4 cell counts exist. Treatment options for both tuberculosis and HIV, including the newer available drugs and those in clinical trials, are revised and recommendations for dose adjustments are made based on the latest available literature, with special attention to drug-drug interactions and the necessity of dose adjustments with some drug combinations.\n",
      "\n",
      " \n",
      "\n",
      "Why CCR5 is chosen as the target for stem cell gene therapy for HIV infection? \t 0.9700931310653687\n",
      "Since the first case reported in 1981, more than 60 million individuals have succumbed to HIV worldwide. Although great efforts have been put forth in developing therapeutic drugs and effective vaccines for the treatment and prevention of HIV infection, these efforts are not correlated to the reported case of cured HIV infection. The first case of cured HIV infection, from allogenic stem cell transplantation, may shed light on future prevention and therapy of HIV infection. The choice of gene target, however, must first be evaluated regarding stem cell based gene therapy for HIV infection. Out of the tens of genes that had shown anti-HIV infection potentials, CCR5 was described to be effective in stem cell based gene therapy in 2005. Here, we appreciate the clinical observations that directly led to the choice of CCR5, rather than other genes for stem cell gene therapy.\n",
      "\n",
      " \n",
      "\n",
      "Antiretroviral medication and HIV prevention: new steps forward and new questions. \t 0.9534240365028381\n",
      "During the past 2 years, several pivotal clinical trials have proven that the use of antiretrovirals by HIV-infected and at-risk uninfected persons can decrease the probability of HIV being transmitted sexually. The initial chemoprophylaxis studies evaluated tenofovir administered topically or orally (with or without emtricitabine). However, several questions remain. Some subsequent primary prevention studies did not replicate the results of the initial studies, raising questions about differences in the behaviors of participants in each study (in particular about medication adherence), as well as whether pharmacologic or local mucosal factors might explain the variable efficacy estimates. Other antiretrovirals and delivery systems are being evaluated to maximize the efficacy of primary chemoprophylactic approaches. At present, increasing access to antiretroviral treatment globally is a priority, because expanding access to medication that can prevent morbidity and mortality is itself an important public health goal and may reasonably be expected to decrease HIV incidence. However, for treatment as prevention to be maximally effective, increases in HIV testing, health care workers, and infrastructure are needed, in addition to medications and laboratory support for clinical monitoring. A combination of approaches is needed to most quickly decrease the current trends in HIV incidence, including early diagnosis and initiation of treatment for HIV-infected persons. These approaches can be coupled with appropriately tailored interventions for populations at greatest risk for infection (for example, men who have sex with men and sex workers), including male circumcision, behavioral interventions, and chemoprophylaxis. However, a substantial gap exists between current expenditures and unmet needs, which suggests that mobilization of political will is needed for this combination approach to be successful.\n",
      "\n",
      " \n",
      "\n",
      "Challenges of HIV treatment in resource-poor countries: a review. \t 0.9427611827850342\n",
      "The human immunodeficiency virus/acquired immunodeficiency syndrome pandemic have posed a significant public health challenge to the global community. Massive therapeutic interventions with antiretroviral drugs are being undertaken, yet problems and challenges exist. This review examines these problems and challenges as they affect the treatment of HIV infection in resource-poor countries such as Nigeria. The information was sourced from relevant literature using human immunodeficiency virus/acquired immunodeficiency syndrome journals, textbooks and Websites on human immunodeficiency virus/acquired immunodeficiency syndrome, highly active antiretroviral therapy, resource-poor countries as key words. Several studies have shown that the advent of highly active antiretroviral therapy in 1996 has significantly reduced morbidity and mortality among people living with HIV/AIDS (PLWHA). But in resource-poor countries, initiation and maintenance of highly active antiretroviral therapy has been associated with many challenges and problems such as: poor infrastructural base for the control programs; irregular or non availability of drugs; poor drug adherence; co-morbidities and opportunistic infections/malignancies; drug toxicities; drug/food and drug/drug interactions; laboratory monitoring of viral load; CD4 cell counts; full blood counts; electrolytes, kidney and liver functions. The review has shown that the solution to the pandemic lies in a multi-sectoral and holistic approach involving International and local agencies, and communities.\n",
      "\n",
      " \n",
      "\n",
      "Current progress and challenges in HIV gene therapy. \t 0.9385221600532532\n",
      "HIV-1 causes AIDS, a syndrome that affects millions of people globally. Existing HAART is efficient in slowing down disease progression but cannot eradicate the virus. Furthermore the severity of the side effects and the emergence of drug-resistant mutants call for better therapy. Gene therapy serves as an attractive alternative as it reconstitutes the immune system with HIV-resistant cells and could thereby provide a potential cure. The feasibility of this approach was first demonstrated with the 'Berlin patient', who was functionally cured from HIV/AIDS with undetectable HIV-1 viral load after transplantation of bone marrow harboring a naturally occurring CCR5 mutation that blocks viral entry. Here, we give an overview of the current status of HIV gene therapy and remaining challenges and obstacles.\n",
      "\n",
      " \n",
      "\n",
      "Newer drugs and earlier treatment: impact on lifetime cost of care for HIV-infected adults. \t 0.9286227822303772\n",
      "To determine the component costs of care to optimize treatment with limited resources. We used the Cost-Effectiveness of Preventing AIDS Complications Model of HIV disease and treatment to project life expectancy and both undiscounted and discounted lifetime costs (2010 €). We determined medical resource utilization among HIV-infected adults followed from 1998 to 2005 in northern France. Monthly HIV costs were stratified by CD4 cell count. Costs of CD4, HIV RNA and genotype tests and antiretroviral therapy (ART) were derived from published literature. Model inputs from national data included mean age 38 years, mean initial CD4 cell count 372 cells/μl, ART initiation at CD4 cell counts less than 350 cells/μl, and ART regimen costs ranging from €760 to 2570 per month. The model projected a mean undiscounted life expectancy of 26.5 years and a lifetime undiscounted cost of €535,000/patient (€320,700 discounted); 73% of costs were ART related. When patients presented to care with mean CD4 cell counts of 510 cells/μl and initiated ART at CD4 cell counts less than 500 cells/μl or HIV RNA more than 100,000 copies/ml, life expectancy was 27.4 years and costs increased 1-2%, to €546,700 (€324,500 discounted). When we assumed introducing generic drugs would result in a 50% decline in first-line ART costs, lifetime costs decreased 4-6%, to €514,200 (€302 ,800 discounted). As HIV disease is treated earlier with more efficacious drugs, survival and thus costs of care will continue to increase. The availability in high-income countries of widely used antiretroviral drugs in generic form could reduce these costs.\n",
      "\n",
      " \n",
      "\n",
      "Common clinical problems in children living with HIV/AIDS: systemic approach. \t 0.9138649106025696\n",
      "Clinical manifestations in children living with HIV/ AIDS differ from those in adults due to poorly developed immunity that allows greater dissemination throughout various organs. In developing countries, HIV-infected children have an increased frequency of malnutrition and common childhood infections such as ear infections, pneumonias, gastroenteritis and tuberculosis. The symptoms common to many treatable conditions, such as recurrent fever, diarrhea and generalized dermatitis, tend to be more persistent and severe and often do not respond as well to treatment. The use of Anti Retroviral Therapy (ART) has greatly increased the long term survival of perinatally infected children so that AIDS is becoming a manageable chronic illness. As the immunity is maintained, the incidence of infectious complications is declining while noninfectious complications of HIV are more frequently encountered. Regular clinical monitoring with immunological and virological monitoring and the introduction of genotypic and phenotypic resistance testing where resources are available have allowed for dramatically better clinical outcomes. However, these growing children are left facing the challenges of lifelong adherence with complex treatment regimens, compounded by complex psycho-social, mental and neuro-cognitive issues. These unique challenges must be recognized and understood in order to provide appropriate medical management.\n",
      "\n",
      " \n",
      "\n",
      "Barriers and facilitators of adherence to TB treatment in patients on concomitant TB and HIV treatment: a qualitative study. \t 0.9002022743225098\n",
      "Tuberculosis is a major public health problem in Ethiopia, and a high number of TB patients are co-infected with HIV. There is a need for more knowledge about factors influencing treatment adherence in co-infected patients on concomitant treatment. The aim of the present study is to explore patients' and health care professionals' views about barriers and facilitators to TB treatment adherence in TB/HIV co-infected patients on concomitant treatment for TB and HIV. Qualitative study using in-depth interviews with 15 TB/HIV co-infected patients and 9 health professionals and focus group discussions with 14 co-infected patients. We found that interplay of factors is involved in the decision making about medication intake. Factors that influenced adherence to TB treatment positively were beliefs in the curability of TB, beliefs in the severity of TB in the presence of HIV infection and support from families and health professionals. Barriers to treatment adherence were experiencing side effects, pill burden, economic constraints, lack of food, stigma with lack of disclosure, and lack of adequate communication with health professionals. Health professionals and policy makers should be aware of factors influencing TB treatment in TB/HIV co-infected patients on concomitant treatment for TB and HIV. Our results suggest that provision of food and minimal financial support might facilitate adherence. Counseling might also facilitate adherence, in particular for those who start ART in the early phases of TB treatment, and beliefs related to side-effects and pill burden should be addressed. Information to the public may reduce TB and HIV related stigma.\n",
      "\n",
      " \n",
      "\n",
      "Combination implementation for HIV prevention: moving from clinical trial evidence to population-level effects. \t 0.8731147646903992\n",
      "The promise of combination HIV prevention-the application of multiple HIV prevention interventions to maximise population-level effects-has never been greater. However, to succeed in achieving significant reductions in HIV incidence, an additional concept needs to be considered: combination implementation. Combination implementation for HIV prevention is the pragmatic, localised application of evidence-based strategies to enable high sustained uptake and quality of interventions for prevention of HIV. In this Review, we explore diverse implementation strategies including HIV testing and counselling models, task shifting, linkage to and retention in care, antiretroviral therapy support, behaviour change, demand creation, and structural interventions, and discusses how they could be used to complement HIV prevention efforts such as medical male circumcision and treatment as prevention. HIV prevention and treatment have arrived at a pivotal moment when combination efforts might result in substantial enough population-level effects to reverse the epidemic and drive towards elimination of HIV. Only through careful consideration of how to implement and operationalise HIV prevention interventions will the HIV community be able to move from clinical trial evidence to population-level effects.\n",
      "\n",
      " \n",
      "\n"
     ]
    }
   ],
   "source": [
    "pretty_print_results(result[\"ranker\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HZ5_NJDz52VE"
   },
   "source": [
    "## What's next\n",
    "\n",
    "🎉 Congratulations! You've create a hybrid retrieval pipeline!\n",
    "\n",
    "If you'd like to use this retrieval method in a RAG pipeline, check out [Tutorial: Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) to learn about the next steps.\n",
    "\n",
    "To stay up to date on the latest Haystack developments, you can [sign up for our newsletter](https://landing.deepset.ai/haystack-community-updates) or [join Haystack discord community](https://discord.gg/haystack).\n",
    "\n",
    "Thanks for reading!"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "0387b8e4546247f49f854f9729e6a3df": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cf972cbfa1314f149b9860e199391170",
      "placeholder": "​",
      "style": "IPY_MODEL_06c4d38f14674fd898aa85df17c0baa7",
      "value": "Batches: 100%"
     }
    },
    "06c4d38f14674fd898aa85df17c0baa7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "097a7751c1b04aa3bf4a0a586a6dfa32": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c1626ca2ebf84556a692e76f16f3cab4",
      "max": 37807,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5cd3d66281d247749d4c71daa7c6f49d",
      "value": 37807
     }
    },
    "107daf1c8f8f4203aff8058191ce9728": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "15f3697b990f40adb795dbcce3f626c1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2398002062874506a558e59114359c54": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2a7323e821724b129d01de39adb67c4f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "31acc8ba63e240a6b4419ab4aa51c87a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4b9ea215e11844239e968960cd45cade": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2a7323e821724b129d01de39adb67c4f",
      "placeholder": "​",
      "style": "IPY_MODEL_eeb568b15a1d465bab9f7a3843d611df",
      "value": "Ranking by BM25...: 100%"
     }
    },
    "5cd3d66281d247749d4c71daa7c6f49d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "827efd14f6f74c8ca0fb8df3a4359062": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "864e853846b3406a9a8b743dbc1d96ff": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_947922d2f75346cdbbb908d3888f832e",
      "placeholder": "​",
      "style": "IPY_MODEL_c7a118b1f8a2491cacd5f76db668443e",
      "value": " 1/1 [00:00&lt;00:00, 22.59it/s]"
     }
    },
    "947922d2f75346cdbbb908d3888f832e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9e5e169b22dd45068a2b03154b26ccf0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_107daf1c8f8f4203aff8058191ce9728",
      "placeholder": "​",
      "style": "IPY_MODEL_31acc8ba63e240a6b4419ab4aa51c87a",
      "value": " 37807/37807 [00:06&lt;00:00, 4599.09 docs/s]"
     }
    },
    "a4d87d3d4a9a4dc4bccdc4a16be29096": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b395830b097f4862a6c8588ccd0fd91b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4b9ea215e11844239e968960cd45cade",
       "IPY_MODEL_097a7751c1b04aa3bf4a0a586a6dfa32",
       "IPY_MODEL_9e5e169b22dd45068a2b03154b26ccf0"
      ],
      "layout": "IPY_MODEL_827efd14f6f74c8ca0fb8df3a4359062"
     }
    },
    "c1626ca2ebf84556a692e76f16f3cab4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c7a118b1f8a2491cacd5f76db668443e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cf62bf3b3c6144629811874114dc527f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_0387b8e4546247f49f854f9729e6a3df",
       "IPY_MODEL_d45dcf1e27a5401ca2c430cd6c322fdb",
       "IPY_MODEL_864e853846b3406a9a8b743dbc1d96ff"
      ],
      "layout": "IPY_MODEL_15f3697b990f40adb795dbcce3f626c1"
     }
    },
    "cf972cbfa1314f149b9860e199391170": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d45dcf1e27a5401ca2c430cd6c322fdb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a4d87d3d4a9a4dc4bccdc4a16be29096",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2398002062874506a558e59114359c54",
      "value": 1
     }
    },
    "eeb568b15a1d465bab9f7a3843d611df": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}