diff --git a/notebooks/spa/_toctree.yml b/notebooks/spa/_toctree.yml new file mode 100644 index 00000000..513a3b0d --- /dev/null +++ b/notebooks/spa/_toctree.yml @@ -0,0 +1,37 @@ +- title: Open-Source AI Cookbook + sections: + - local: index + title: Open-Source AI Cookbook + - local: issues_in_text_dataset + title: Detección de problemas en un conjunto de datos de texto con Cleanlab + - local: stable_diffusion_interpolation + title: Interpolación por difusión estable + - local: rag_with_hugging_face_gemma_mongodb + title: Creación de un sistema RAG con Gemma, MongoDB y modelos de código abierto + - local: tgi_messages_api_demo + title: Migración de OpenAI a Open LLM mediante la API de mensajes de TGI + - local: automatic_embedding_tei_inference_endpoints + title: Incrustación automática con TEI mediante puntos finales de inferencia + - local: faiss_with_hf_datasets_and_clip + title: Incrustación de datos multimodales para la búsqueda de similitudes + - local: fine_tuning_code_llm_on_single_gpu + title: Fine-tuning de un LLM de código personalizado en una sola GPU + - local: rag_zephyr_langchain + title: RAG simple utilizando Hugging Face Zephyr y LangChain + - local: rag_llamaindex_librarian + title: RAG "Bibliotecario" con LlamaIndex + - local: advanced_rag + title: RAG avanzado sobre la documentación de HuggingFace utilizando LangChain + - local: rag_evaluation + title: Evaluación RAG + - local: prompt_tuning_peft + title: Prompt tuning con PEFT + - local: labelling_feedback_setfit + title: Sugerencias para la anotación de datos con SetFit en la clasificación de textos con disparo cero + - local: pipeline_notus_instructions_preferences_legal + title: Crear un conjunto de datos de preferencias legales + - local: semantic_cache_chroma_vector_database + title: Implementación de la caché semántica para mejorar un sistema RAG. + - local: llm_judge + title: Utilización de LLM-as-a-judge para una evaluación automatizada y versátil + diff --git a/notebooks/spa/advanced_rag.ipynb b/notebooks/spa/advanced_rag.ipynb new file mode 100644 index 00000000..a87343ff --- /dev/null +++ b/notebooks/spa/advanced_rag.ipynb @@ -0,0 +1,1258 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "hUCaGdAj9-9F" + }, + "source": [ + "# Advanced RAG on HuggingFace documentation using LangChain\n", + "_Authored by: [Aymeric Roucher](https://huggingface.co/m-ric)_" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This notebook demonstrates how you can build an advanced RAG (Retrieval Augmented Generation) for answering a user's question about a specific knowledge base (here, the HuggingFace documentation), using LangChain.\n", + "\n", + "For an introduction to RAG, you can check [this other cookbook](rag_zephyr_langchain)!\n", + "\n", + "RAG systems are complex, with many moving parts: here a RAG diagram, where we noted in blue all possibilities for system enhancement:\n", + "\n", + "\n", + "\n", + "> 💡 As you can see, there are many steps to tune in this architecture: tuning the system properly will yield significant performance gains.\n", + "\n", + "In this notebook, we will take a look into many of these blue notes to see how to tune your RAG system and get the best performance.\n", + "\n", + "__Let's dig into the model building!__ First, we install the required model dependancies." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "NSX0p0rV9-9I" + }, + "outputs": [], + "source": [ + "!pip install -q torch transformers transformers accelerate bitsandbytes langchain sentence-transformers faiss-gpu openpyxl pacmap" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "8_Uyukt39-9J" + }, + "outputs": [], + "source": [ + "%reload_ext dotenv\n", + "%dotenv" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "eoujYMwW9-9J" + }, + "outputs": [], + "source": [ + "from tqdm.notebook import tqdm\n", + "import pandas as pd\n", + "from typing import Optional, List, Tuple\n", + "from datasets import Dataset\n", + "import matplotlib.pyplot as plt\n", + "\n", + "pd.set_option(\n", + " \"display.max_colwidth\", None\n", + ") # this will be helpful when visualizing retriever outputs" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Kr6rN10U9-9J" + }, + "source": [ + "### Load your knowledge base" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "qZLVIEVW9-9J" + }, + "outputs": [], + "source": [ + "import datasets\n", + "\n", + "ds = datasets.load_dataset(\"m-ric/huggingface_doc\", split=\"train\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "836Q7vF49-9K" + }, + "outputs": [], + "source": [ + "from langchain.docstore.document import Document as LangchainDocument\n", + "\n", + "RAW_KNOWLEDGE_BASE = [\n", + " LangchainDocument(page_content=doc[\"text\"], metadata={\"source\": doc[\"source\"]})\n", + " for doc in tqdm(ds)\n", + "]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0_LxjD5h9-9K" + }, + "source": [ + "# 1. Retriever - embeddings 🗂️\n", + "The __retriever acts like an internal search engine__: given the user query, it returns a few relevant snippets from your knowledge base.\n", + "\n", + "These snippets will then be fed to the Reader Model to help it generate its answer.\n", + "\n", + "So __our objective here is, given a user question, to find the most snippets from our knowledge base to answer that question.__\n", + "\n", + "This is a wide objective, it leaves open some questions. How many snippets should we retrieve? This parameter will be named `top_k`.\n", + "\n", + "How long should these snippets be? This is called the `chunk size`. There's no one-size-fits-all answers, but here are a few elements:\n", + "- 🔀 Your `chunk size` is allowed to vary from one snippet to the other.\n", + "- Since there will always be some noise in your retrieval, increasing the `top_k` increases the chance to get relevant elements in your retrieved snippets. 🎯 Shooting more arrows increases your probability to hit your target.\n", + "- Meanwhile, the summed length of your retrieved documents should not be too high: for instance, for most current models 16k tokens will probably drown your Reader model in information due to [Lost-in-the-middle phenomenon](https://huggingface.co/papers/2307.03172). 🎯 Give your reader model only the most relevant insights, not a huge pile of books!\n", + "\n", + "\n", + "> In this notebook, we use Langchain library since __it offers a huge variety of options for vector databases and allows us to keep document metadata throughout the processing__." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-uS6Mv8O9-9L" + }, + "source": [ + "### 1.1 Split the documents into chunks\n", + "\n", + "- In this part, __we split the documents from our knowledge base into smaller chunks__ which will be the snippets on which the reader LLM will base its answer.\n", + "- The goal is to prepare a collection of **semantically relevant snippets**. So their size should be adapted to precise ideas: too small will truncate ideas, too large will dilute them.\n", + "\n", + "💡 _Many options exist for text splitting: splitting on words, on sentence boundaries, recursive chunking that processes documents in a tree-like way to preserve structure information... To learn more about chunking, I recommend you read [this great notebook](https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/5_Levels_Of_Text_Splitting.ipynb) by Greg Kamradt._\n", + "\n", + "\n", + "- **Recursive chunking** breaks down the text into smaller parts step by step using a given list of separators sorted from the most important to the least important separator. If the first split doesn't give the right size or shape chunks, the method repeats itself on the new chunks using a different separator. For instance with the list of separators `[\"\\n\\n\", \"\\n\", \".\", \"\"]`:\n", + " - The method will first break down the document wherever there is a double line break `\"\\n\\n\"`.\n", + " - Resulting documents will be split again on simple line breaks `\"\\n\"`, then on sentence ends `\".\"`.\n", + " - And finally, if some chunks are still too big, they will be split whenever they overflow the maximum size.\n", + "\n", + "- With this method, the global structure is well preserved, at the expense of getting slight variations in chunk size.\n", + "\n", + "> [This space](https://huggingface.co/spaces/A-Roucher/chunk_visualizer) lets you visualize how different splitting options affect the chunks you get.\n", + "\n", + "🔬 Let's experiment a bit with chunk sizes, beginning with an arbitrary size, and see how splits work. We use Langchain's implementation of recursive chunking with `RecursiveCharacterTextSplitter`.\n", + "- Parameter `chunk_size` controls the length of individual chunks: this length is counted by default as the number of characters in the chunk.\n", + "- Parameter `chunk_overlap` lets adjacent chunks get a bit of overlap on each other. This reduces the probability that an idea could be cut in half by the split between two adjacent chunks. We ~arbitrarily set this to 1/10th of the chunk size, you could try different values!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "M4m6TwDJ9-9L" + }, + "outputs": [], + "source": [ + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "\n", + "# We use a hierarchical list of separators specifically tailored for splitting Markdown documents\n", + "# This list is taken from LangChain's MarkdownTextSplitter class.\n", + "MARKDOWN_SEPARATORS = [\n", + " \"\\n#{1,6} \",\n", + " \"```\\n\",\n", + " \"\\n\\\\*\\\\*\\\\*+\\n\",\n", + " \"\\n---+\\n\",\n", + " \"\\n___+\\n\",\n", + " \"\\n\\n\",\n", + " \"\\n\",\n", + " \" \",\n", + " \"\",\n", + "]\n", + "\n", + "text_splitter = RecursiveCharacterTextSplitter(\n", + " chunk_size=1000, # the maximum number of characters in a chunk: we selected this value arbitrarily\n", + " chunk_overlap=100, # the number of characters to overlap between chunks\n", + " add_start_index=True, # If `True`, includes chunk's start index in metadata\n", + " strip_whitespace=True, # If `True`, strips whitespace from the start and end of every document\n", + " separators=MARKDOWN_SEPARATORS,\n", + ")\n", + "\n", + "docs_processed = []\n", + "for doc in RAW_KNOWLEDGE_BASE:\n", + " docs_processed += text_splitter.split_documents([doc])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d5jJUMgb9-9M" + }, + "source": [ + "We also have to keep in mind that when embedding documents, we will use an embedding model that has accepts a certain maximum sequence length `max_seq_length`.\n", + "\n", + "So we should make sure that our chunk sizes are below this limit, because any longer chunk will be truncated before processing, thus losing relevancy." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "referenced_widgets": [ + "ae043feeb0914c879e2a9008b413d952" + ] + }, + "id": "B4hoki349-9M", + "outputId": "64f92a61-7839-476d-f456-7eefde04c20b" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Model's maximum sequence length: 512\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "ae043feeb0914c879e2a9008b413d952", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + " 0%| | 0/31085 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from sentence_transformers import SentenceTransformer\n", + "\n", + "# To get the value of the max sequence_length, we will query the underlying `SentenceTransformer` object used in the RecursiveCharacterTextSplitter.\n", + "print(\n", + " f\"Model's maximum sequence length: {SentenceTransformer('thenlper/gte-small').max_seq_length}\"\n", + ")\n", + "\n", + "from transformers import AutoTokenizer\n", + "\n", + "tokenizer = AutoTokenizer.from_pretrained(\"thenlper/gte-small\")\n", + "lengths = [len(tokenizer.encode(doc.page_content)) for doc in tqdm(docs_processed)]\n", + "\n", + "# Plot the distrubution of document lengths, counted as the number of tokens\n", + "fig = pd.Series(lengths).hist()\n", + "plt.title(\"Distribution of document lengths in the knowledge base (in count of tokens)\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "L3teXczl9-9M" + }, + "source": [ + "👀 As you can see, __the chunk lengths are not aligned with our limit of 512 tokens__, and some documents are above the limit, thus some part of them will be lost in truncation!\n", + " - So we should change the `RecursiveCharacterTextSplitter` class to count length in number of tokens instead of number of characters.\n", + " - Then we can choose a specific chunk size, here we would choose a lower threshold than 512:\n", + " - smaller documents could allow the split to focus more on specific ideas.\n", + " - But too small chunks would split sentences in half, thus losing meaning again: the proper tuning is a matter of balance." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "referenced_widgets": [ + "f900cf4ab3a94f45bfa7298f433566ed" + ] + }, + "id": "9hvIL2jO9-9M", + "outputId": "9baf219d-2954-4927-9681-e28572db90db" + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "f900cf4ab3a94f45bfa7298f433566ed", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + " 0%| | 0/17995 [00:00" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "from transformers import AutoTokenizer\n", + "\n", + "EMBEDDING_MODEL_NAME = \"thenlper/gte-small\"\n", + "\n", + "\n", + "def split_documents(\n", + " chunk_size: int,\n", + " knowledge_base: List[LangchainDocument],\n", + " tokenizer_name: Optional[str] = EMBEDDING_MODEL_NAME,\n", + ") -> List[LangchainDocument]:\n", + " \"\"\"\n", + " Split documents into chunks of maximum size `chunk_size` tokens and return a list of documents.\n", + " \"\"\"\n", + " text_splitter = RecursiveCharacterTextSplitter.from_huggingface_tokenizer(\n", + " AutoTokenizer.from_pretrained(tokenizer_name),\n", + " chunk_size=chunk_size,\n", + " chunk_overlap=int(chunk_size / 10),\n", + " add_start_index=True,\n", + " strip_whitespace=True,\n", + " separators=MARKDOWN_SEPARATORS,\n", + " )\n", + "\n", + " docs_processed = []\n", + " for doc in knowledge_base:\n", + " docs_processed += text_splitter.split_documents([doc])\n", + "\n", + " # Remove duplicates\n", + " unique_texts = {}\n", + " docs_processed_unique = []\n", + " for doc in docs_processed:\n", + " if doc.page_content not in unique_texts:\n", + " unique_texts[doc.page_content] = True\n", + " docs_processed_unique.append(doc)\n", + "\n", + " return docs_processed_unique\n", + "\n", + "\n", + "docs_processed = split_documents(\n", + " 512, # We choose a chunk size adapted to our model\n", + " RAW_KNOWLEDGE_BASE,\n", + " tokenizer_name=EMBEDDING_MODEL_NAME,\n", + ")\n", + "\n", + "# Let's visualize the chunk sizes we would have in tokens from a common model\n", + "from transformers import AutoTokenizer\n", + "\n", + "tokenizer = AutoTokenizer.from_pretrained(EMBEDDING_MODEL_NAME)\n", + "lengths = [len(tokenizer.encode(doc.page_content)) for doc in tqdm(docs_processed)]\n", + "fig = pd.Series(lengths).hist()\n", + "plt.title(\"Distribution of document lengths in the knowledge base (in count of tokens)\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Wc3riwX39-9M" + }, + "source": [ + "➡️ Now the chunk length distribution looks better!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "J1ho-UKM9-9M" + }, + "source": [ + "### 1.2 Building the vector database\n", + "\n", + "We want to compute the embeddings for all the chunks of our knowledge base: to learn more on sentence embeddings, we recommend reading [this guide](https://osanseviero.github.io/hackerllama/blog/posts/sentence_embeddings/).\n", + "\n", + "#### How does retrieval work ?\n", + "\n", + "Once the chunks are all embedded, we store them into a vector database. When the user types in a query, it gets embedded by the same model previously used, and a similarity search returns the closest documents from the vector database.\n", + "\n", + "The technical challenge is thus, given a query vector, to quickly find the nearest neighbours of this vector in the vector database. To do this, we need to choose two things: a distance, and a search algorithm to find the nearest neighbors quickly within a database of thousands of records.\n", + "\n", + "##### Nearest Neighbor search algorithm\n", + "\n", + "There are plentiful choices for the nearest neighbor search algorithm: we go with Facebook's [FAISS](https://github.com/facebookresearch/faiss), since FAISS is performant enough for most use cases, and it is well known thus widely implemented.\n", + "\n", + "##### Distances\n", + "\n", + "Regarding distances, you can find a good guide [here](https://osanseviero.github.io/hackerllama/blog/posts/sentence_embeddings/#distance-between-embeddings). In short:\n", + "\n", + "- **Cosine similarity** computes similarity between two vectors as the cosinus of their relative angle: it allows us to compare vector directions are regardless of their magnitude. Using it requires to normalize all vectors, to rescale them into unit norm.\n", + "- **Dot product** takes into account magnitude, with the sometimes undesirable effect that increasing a vector's length will make it more similar to all others.\n", + "- **Euclidean distance** is the distance between the ends of vectors.\n", + "\n", + "You can try [this small exercise](https://developers.google.com/machine-learning/clustering/similarity/check-your-understanding) to check your understanding of these concepts. But once vectors are normalized, [the choice of a specific distance does not matter much](https://platform.openai.com/docs/guides/embeddings/which-distance-function-should-i-use).\n", + "\n", + "Our particular model works well with cosine similarity, so choose this distance, and we set it up both in the Embedding model, and in the `distance_strategy` argument of our FAISS index. With cosine similarity, we have to normalize our embeddings.\n", + "\n", + "🚨👇 The cell below takes a few minutes to run on A10G!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "dalledM99-9M" + }, + "outputs": [], + "source": [ + "from langchain.vectorstores import FAISS\n", + "from langchain_community.embeddings import HuggingFaceEmbeddings\n", + "from langchain_community.vectorstores.utils import DistanceStrategy\n", + "\n", + "embedding_model = HuggingFaceEmbeddings(\n", + " model_name=EMBEDDING_MODEL_NAME,\n", + " multi_process=True,\n", + " model_kwargs={\"device\": \"cuda\"},\n", + " encode_kwargs={\"normalize_embeddings\": True}, # set True for cosine similarity\n", + ")\n", + "\n", + "KNOWLEDGE_VECTOR_DATABASE = FAISS.from_documents(\n", + " docs_processed, embedding_model, distance_strategy=DistanceStrategy.COSINE\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0zM-wfiJ9-9N" + }, + "source": [ + "👀 To visualize the search for the closest documents, let's project our embeddings from 384 dimensions down to 2 dimensions using PaCMAP.\n", + "\n", + "💡 _We chose PaCMAP rather than other techniques such as t-SNE or UMAP, since [it is efficient (preserves local and global structure), robust to initialization parameters and fast](https://www.nature.com/articles/s42003-022-03628-x#Abs1)._" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "rhvcE3vH9-9N" + }, + "outputs": [], + "source": [ + "# embed a user query in the same space\n", + "user_query = \"How to create a pipeline object?\"\n", + "query_vector = embedding_model.embed_query(user_query)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "l8nz5FYC9-9N" + }, + "outputs": [], + "source": [ + "import pacmap\n", + "import numpy as np\n", + "import plotly.express as px\n", + "\n", + "embedding_projector = pacmap.PaCMAP(\n", + " n_components=2, n_neighbors=None, MN_ratio=0.5, FP_ratio=2.0, random_state=1\n", + ")\n", + "\n", + "embeddings_2d = [\n", + " list(KNOWLEDGE_VECTOR_DATABASE.index.reconstruct_n(idx, 1)[0])\n", + " for idx in range(len(docs_processed))\n", + "] + [query_vector]\n", + "\n", + "# fit the data (The index of transformed data corresponds to the index of the original data)\n", + "documents_projected = embedding_projector.fit_transform(\n", + " np.array(embeddings_2d), init=\"pca\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "7Cl9Fw2A9-9N" + }, + "outputs": [], + "source": [ + "df = pd.DataFrame.from_dict(\n", + " [\n", + " {\n", + " \"x\": documents_projected[i, 0],\n", + " \"y\": documents_projected[i, 1],\n", + " \"source\": docs_processed[i].metadata[\"source\"].split(\"/\")[1],\n", + " \"extract\": docs_processed[i].page_content[:100] + \"...\",\n", + " \"symbol\": \"circle\",\n", + " \"size_col\": 4,\n", + " }\n", + " for i in range(len(docs_processed))\n", + " ]\n", + " + [\n", + " {\n", + " \"x\": documents_projected[-1, 0],\n", + " \"y\": documents_projected[-1, 1],\n", + " \"source\": \"User query\",\n", + " \"extract\": user_query,\n", + " \"size_col\": 100,\n", + " \"symbol\": \"star\",\n", + " }\n", + " ]\n", + ")\n", + "\n", + "# visualize the embedding\n", + "fig = px.scatter(\n", + " df,\n", + " x=\"x\",\n", + " y=\"y\",\n", + " color=\"source\",\n", + " hover_data=\"extract\",\n", + " size=\"size_col\",\n", + " symbol=\"symbol\",\n", + " color_discrete_map={\"User query\": \"black\"},\n", + " width=1000,\n", + " height=700,\n", + ")\n", + "fig.update_traces(\n", + " marker=dict(opacity=1, line=dict(width=0, color=\"DarkSlateGrey\")),\n", + " selector=dict(mode=\"markers\"),\n", + ")\n", + "fig.update_layout(\n", + " legend_title_text=\"Chunk source\",\n", + " title=\"2D Projection of Chunk Embeddings via PaCMAP\",\n", + ")\n", + "fig.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kWesCSGt9-9N" + }, + "source": [ + "\n", + "\n", + "\n", + "➡️ On the graph above, you can see a spatial representation of the kowledge base documents. As the vector embeddings represent the document's meaning, their closeness in meaning should be reflected in their embedding's closeness.\n", + "\n", + "The user query's embedding is also shown : we want to find the `k` document that have the closest meaning, thus we pick the `k` closest vectors.\n", + "\n", + "In the LangChain vector database implementation, this search operation is performed by the method `vector_database.similarity_search(query)`.\n", + "\n", + "Here is the result:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "VcjQzejH9-9N", + "outputId": "d5b817c2-1b0e-4e47-9658-4892a91e7c51" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Starting retrieval for user_query='How to create a pipeline object?'...\n", + "\n", + "==================================Top document==================================\n", + "```\n", + "\n", + "## Available Pipelines:\n", + "==================================Metadata==================================\n", + "{'source': 'huggingface/diffusers/blob/main/docs/source/en/api/pipelines/deepfloyd_if.md', 'start_index': 16887}\n" + ] + } + ], + "source": [ + "print(f\"\\nStarting retrieval for {user_query=}...\")\n", + "retrieved_docs = KNOWLEDGE_VECTOR_DATABASE.similarity_search(query=user_query, k=5)\n", + "print(\n", + " \"\\n==================================Top document==================================\"\n", + ")\n", + "print(retrieved_docs[0].page_content)\n", + "print(\"==================================Metadata==================================\")\n", + "print(retrieved_docs[0].metadata)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VjVqmDGh9-9N" + }, + "source": [ + "# 2. Reader - LLM 💬\n", + "\n", + "In this part, the __LLM Reader reads the retrieved context to formulate its answer.__\n", + "\n", + "There are actually substeps that can all be tuned:\n", + "1. The content of the retrieved documents is aggregated together into the \"context\", with many processing options like _prompt compression_.\n", + "2. The context and the user query are aggregated into a prompt then given to the LLM to generate its answer." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0xiXcG269-9N" + }, + "source": [ + "### 2.1. Reader model\n", + "\n", + "The choice of a reader model is important on a few aspects:\n", + "- the reader model's `max_seq_length` must accomodate our prompt, which includes the context output by the retriever call: the context consists in 5 documents of 512 tokens each, so we aim for a context length of 4k tokens at least.\n", + "- the reader model\n", + "\n", + "For this example, we chose [`HuggingFaceH4/zephyr-7b-beta`](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta), a small but powerful model.\n", + "\n", + "With many models being released every week, you may want to substitute this model to the latest and greatest. The best way to keep track of open source LLMs is to check the [Open-source LLM leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n", + "\n", + "To make inference faster, we will load the quantized version of the model:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "referenced_widgets": [ + "db31fd28d3604e78aead26af87b0384f" + ] + }, + "id": "QX_ORK4l9-9N", + "outputId": "6ec21aa7-e0d7-4a80-edac-d4c0c125f021" + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "db31fd28d3604e78aead26af87b0384f", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Loading checkpoint shards: 0%| | 0/8 [00:00\n", + "Using the information contained in the context, \n", + "give a comprehensive answer to the question.\n", + "Respond only to the question asked, response should be concise and relevant to the question.\n", + "Provide the number of the source document when relevant.\n", + "If the answer cannot be deduced from the context, do not give an answer.\n", + "<|user|>\n", + "Context:\n", + "{context}\n", + "---\n", + "Now here is the question you need to answer.\n", + "\n", + "Question: {question}\n", + "<|assistant|>\n" + ] + } + ], + "source": [ + "prompt_in_chat_format = [\n", + " {\n", + " \"role\": \"system\",\n", + " \"content\": \"\"\"Using the information contained in the context,\n", + "give a comprehensive answer to the question.\n", + "Respond only to the question asked, response should be concise and relevant to the question.\n", + "Provide the number of the source document when relevant.\n", + "If the answer cannot be deduced from the context, do not give an answer.\"\"\",\n", + " },\n", + " {\n", + " \"role\": \"user\",\n", + " \"content\": \"\"\"Context:\n", + "{context}\n", + "---\n", + "Now here is the question you need to answer.\n", + "\n", + "Question: {question}\"\"\",\n", + " },\n", + "]\n", + "RAG_PROMPT_TEMPLATE = tokenizer.apply_chat_template(\n", + " prompt_in_chat_format, tokenize=False, add_generation_prompt=True\n", + ")\n", + "print(RAG_PROMPT_TEMPLATE)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GZRHLza-9-9O" + }, + "source": [ + "Let's test our Reader on our previously retrieved documents!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "G4XprIih9-9O", + "outputId": "94c63d34-67ad-4f82-a3b4-2a32cecc8427" + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "To create a pipeline object, follow these steps:\n", + "\n", + "1. Define the inputs and outputs of your pipeline. These could be strings, dictionaries, or any other format that best suits your use case.\n", + "\n", + "2. Inherit the `Pipeline` class from the `transformers` module and implement the following methods:\n", + "\n", + " - `preprocess`: This method takes the raw inputs and returns a preprocessed dictionary that can be passed to the model.\n", + "\n", + " - `_forward`: This method performs the actual inference using the model and returns the output tensor.\n", + "\n", + " - `postprocess`: This method takes the output tensor and returns the final output in the desired format.\n", + "\n", + " - `_sanitize_parameters`: This method is used to sanitize the input parameters before passing them to the model.\n", + "\n", + "3. Load the necessary components, such as the model and scheduler, into the pipeline object.\n", + "\n", + "4. Instantiate the pipeline object and return it.\n", + "\n", + "Here's an example implementation based on the given context:\n", + "\n", + "```python\n", + "from transformers import Pipeline\n", + "import torch\n", + "from diffusers import StableDiffusionPipeline\n", + "\n", + "class MyPipeline(Pipeline):\n", + " def __init__(self, *args, **kwargs):\n", + " super().__init__(*args, **kwargs)\n", + " self.pipe = StableDiffusionPipeline.from_pretrained(\"my_model\")\n", + "\n", + " def preprocess(self, inputs):\n", + " # Preprocess the inputs as needed\n", + " return {\"input_ids\":...}\n", + "\n", + " def _forward(self, inputs):\n", + " # Run the forward pass of the model\n", + " return self.pipe(**inputs).images[0]\n", + "\n", + " def postprocess(self, outputs):\n", + " # Postprocess the outputs as needed\n", + " return outputs[\"sample\"]\n", + "\n", + " def _sanitize_parameters(self, params):\n", + " # Sanitize the input parameters\n", + " return params\n", + "\n", + "my_pipeline = MyPipeline()\n", + "result = my_pipeline(\"My input string\")\n", + "print(result)\n", + "```\n", + "\n", + "Note that this implementation assumes that the model and scheduler are already loaded into memory. If they need to be loaded dynamically, you can modify the `__init__` method accordingly.\n" + ] + } + ], + "source": [ + "retrieved_docs_text = [\n", + " doc.page_content for doc in retrieved_docs\n", + "] # we only need the text of the documents\n", + "context = \"\\nExtracted documents:\\n\"\n", + "context += \"\".join(\n", + " [f\"Document {str(i)}:::\\n\" + doc for i, doc in enumerate(retrieved_docs_text)]\n", + ")\n", + "\n", + "final_prompt = RAG_PROMPT_TEMPLATE.format(\n", + " question=\"How to create a pipeline object?\", context=context\n", + ")\n", + "\n", + "# Redact an answer\n", + "answer = READER_LLM(final_prompt)[0][\"generated_text\"]\n", + "print(answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rhRHZoww9-9O" + }, + "source": [ + "### 2.3. Reranking\n", + "\n", + "A good option for RAG is to retrieve more documents than you want in the end, then rerank the results with a more powerful retrieval model before keeping only the `top_k`.\n", + "\n", + "For this, [Colbertv2](https://arxiv.org/abs/2112.01488) is a great choice: instead of a bi-encoder like our classical embedding models, it is a cross-encoder that computes more fine-grained interactions between the query tokens and each document's tokens.\n", + "\n", + "It is easily usable thanks to [the RAGatouille library](https://github.com/bclavie/RAGatouille)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "triOdqTV9-9O" + }, + "outputs": [], + "source": [ + "from ragatouille import RAGPretrainedModel\n", + "\n", + "RERANKER = RAGPretrainedModel.from_pretrained(\"colbert-ir/colbertv2.0\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Minj2SV59-9O" + }, + "source": [ + "# 3. Assembling it all!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "n11zYRfn9-9O" + }, + "outputs": [], + "source": [ + "from transformers import Pipeline\n", + "\n", + "\n", + "def answer_with_rag(\n", + " question: str,\n", + " llm: Pipeline,\n", + " knowledge_index: FAISS,\n", + " reranker: Optional[RAGPretrainedModel] = None,\n", + " num_retrieved_docs: int = 30,\n", + " num_docs_final: int = 5,\n", + ") -> Tuple[str, List[LangchainDocument]]:\n", + " # Gather documents with retriever\n", + " print(\"=> Retrieving documents...\")\n", + " relevant_docs = knowledge_index.similarity_search(\n", + " query=question, k=num_retrieved_docs\n", + " )\n", + " relevant_docs = [doc.page_content for doc in relevant_docs] # keep only the text\n", + "\n", + " # Optionally rerank results\n", + " if reranker:\n", + " print(\"=> Reranking documents...\")\n", + " relevant_docs = reranker.rerank(question, relevant_docs, k=num_docs_final)\n", + " relevant_docs = [doc[\"content\"] for doc in relevant_docs]\n", + "\n", + " relevant_docs = relevant_docs[:num_docs_final]\n", + "\n", + " # Build the final prompt\n", + " context = \"\\nExtracted documents:\\n\"\n", + " context += \"\".join(\n", + " [f\"Document {str(i)}:::\\n\" + doc for i, doc in enumerate(relevant_docs)]\n", + " )\n", + "\n", + " final_prompt = RAG_PROMPT_TEMPLATE.format(question=question, context=context)\n", + "\n", + " # Redact an answer\n", + " print(\"=> Generating answer...\")\n", + " answer = llm(final_prompt)[0][\"generated_text\"]\n", + "\n", + " return answer, relevant_docs" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9nA4nwRQ9-9P" + }, + "source": [ + "Let's see how our RAG pipeline answers a user query." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "7ZTC1FtX9-9P", + "outputId": "22597be1-ab72-4f68-d577-0e12820463cf" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=> Retrieving documents...\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=> Reranking documents...\n", + "=> Generating answer...\n" + ] + } + ], + "source": [ + "question = \"how to create a pipeline object?\"\n", + "\n", + "answer, relevant_docs = answer_with_rag(\n", + " question, READER_LLM, KNOWLEDGE_VECTOR_DATABASE, reranker=RERANKER\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "SwW0oqhZ9-9P", + "outputId": "361f28ed-9cd5-40b8-f8c4-57e8e4a530d9" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "==================================Answer==================================\n", + "To create a pipeline object, follow these steps:\n", + "\n", + "1. Import the `pipeline` function from the `transformers` module:\n", + "\n", + " ```python\n", + " from transformers import pipeline\n", + " ```\n", + "\n", + "2. Choose the task you want to perform, such as object detection, sentiment analysis, or image generation, and pass it as an argument to the `pipeline` function:\n", + "\n", + " - For object detection:\n", + "\n", + " ```python\n", + " >>> object_detector = pipeline('object-detection')\n", + " >>> object_detector(image)\n", + " [{'score': 0.9982201457023621,\n", + " 'label':'remote',\n", + " 'box': {'xmin': 40, 'ymin': 70, 'xmax': 175, 'ymax': 117}},\n", + " ...]\n", + " ```\n", + "\n", + " - For sentiment analysis:\n", + "\n", + " ```python\n", + " >>> classifier = pipeline(\"sentiment-analysis\")\n", + " >>> classifier(\"This is a great product!\")\n", + " {'labels': ['POSITIVE'],'scores': tensor([0.9999], device='cpu', dtype=torch.float32)}\n", + " ```\n", + "\n", + " - For image generation:\n", + "\n", + " ```python\n", + " >>> image = pipeline(\n", + " ... \"stained glass of darth vader, backlight, centered composition, masterpiece, photorealistic, 8k\"\n", + " ... ).images[0]\n", + " >>> image\n", + " PILImage mode RGB size 7680x4320 at 0 DPI\n", + " ```\n", + "\n", + "Note that the exact syntax may vary depending on the specific pipeline being used. Refer to the documentation for more details on how to use each pipeline.\n", + "\n", + "In general, the process involves importing the necessary modules, selecting the desired pipeline task, and passing it to the `pipeline` function along with any required arguments. The resulting pipeline object can then be used to perform the selected task on input data.\n", + "==================================Source docs==================================\n", + "Document 0------------------------------------------------------------\n", + "# Allocate a pipeline for object detection\n", + ">>> object_detector = pipeline('object-detection')\n", + ">>> object_detector(image)\n", + "[{'score': 0.9982201457023621,\n", + " 'label': 'remote',\n", + " 'box': {'xmin': 40, 'ymin': 70, 'xmax': 175, 'ymax': 117}},\n", + " {'score': 0.9960021376609802,\n", + " 'label': 'remote',\n", + " 'box': {'xmin': 333, 'ymin': 72, 'xmax': 368, 'ymax': 187}},\n", + " {'score': 0.9954745173454285,\n", + " 'label': 'couch',\n", + " 'box': {'xmin': 0, 'ymin': 1, 'xmax': 639, 'ymax': 473}},\n", + " {'score': 0.9988006353378296,\n", + " 'label': 'cat',\n", + " 'box': {'xmin': 13, 'ymin': 52, 'xmax': 314, 'ymax': 470}},\n", + " {'score': 0.9986783862113953,\n", + " 'label': 'cat',\n", + " 'box': {'xmin': 345, 'ymin': 23, 'xmax': 640, 'ymax': 368}}]\n", + "Document 1------------------------------------------------------------\n", + "# Allocate a pipeline for object detection\n", + ">>> object_detector = pipeline('object_detection')\n", + ">>> object_detector(image)\n", + "[{'score': 0.9982201457023621,\n", + " 'label': 'remote',\n", + " 'box': {'xmin': 40, 'ymin': 70, 'xmax': 175, 'ymax': 117}},\n", + " {'score': 0.9960021376609802,\n", + " 'label': 'remote',\n", + " 'box': {'xmin': 333, 'ymin': 72, 'xmax': 368, 'ymax': 187}},\n", + " {'score': 0.9954745173454285,\n", + " 'label': 'couch',\n", + " 'box': {'xmin': 0, 'ymin': 1, 'xmax': 639, 'ymax': 473}},\n", + " {'score': 0.9988006353378296,\n", + " 'label': 'cat',\n", + " 'box': {'xmin': 13, 'ymin': 52, 'xmax': 314, 'ymax': 470}},\n", + " {'score': 0.9986783862113953,\n", + " 'label': 'cat',\n", + " 'box': {'xmin': 345, 'ymin': 23, 'xmax': 640, 'ymax': 368}}]\n", + "Document 2------------------------------------------------------------\n", + "Start by creating an instance of [`pipeline`] and specifying a task you want to use it for. In this guide, you'll use the [`pipeline`] for sentiment analysis as an example:\n", + "\n", + "```py\n", + ">>> from transformers import pipeline\n", + "\n", + ">>> classifier = pipeline(\"sentiment-analysis\")\n", + "Document 3------------------------------------------------------------\n", + "```\n", + "\n", + "## Add the pipeline to 🤗 Transformers\n", + "\n", + "If you want to contribute your pipeline to 🤗 Transformers, you will need to add a new module in the `pipelines` submodule\n", + "with the code of your pipeline, then add it to the list of tasks defined in `pipelines/__init__.py`.\n", + "\n", + "Then you will need to add tests. Create a new file `tests/test_pipelines_MY_PIPELINE.py` with examples of the other tests.\n", + "\n", + "The `run_pipeline_test` function will be very generic and run on small random models on every possible\n", + "architecture as defined by `model_mapping` and `tf_model_mapping`.\n", + "\n", + "This is very important to test future compatibility, meaning if someone adds a new model for\n", + "`XXXForQuestionAnswering` then the pipeline test will attempt to run on it. Because the models are random it's\n", + "impossible to check for actual values, that's why there is a helper `ANY` that will simply attempt to match the\n", + "output of the pipeline TYPE.\n", + "\n", + "You also *need* to implement 2 (ideally 4) tests.\n", + "\n", + "- `test_small_model_pt` : Define 1 small model for this pipeline (doesn't matter if the results don't make sense)\n", + " and test the pipeline outputs. The results should be the same as `test_small_model_tf`.\n", + "- `test_small_model_tf` : Define 1 small model for this pipeline (doesn't matter if the results don't make sense)\n", + " and test the pipeline outputs. The results should be the same as `test_small_model_pt`.\n", + "- `test_large_model_pt` (`optional`): Tests the pipeline on a real pipeline where the results are supposed to\n", + " make sense. These tests are slow and should be marked as such. Here the goal is to showcase the pipeline and to make\n", + " sure there is no drift in future releases.\n", + "- `test_large_model_tf` (`optional`): Tests the pipeline on a real pipeline where the results are supposed to\n", + " make sense. These tests are slow and should be marked as such. Here the goal is to showcase the pipeline and to make\n", + " sure there is no drift in future releases.\n", + "Document 4------------------------------------------------------------\n", + "```\n", + "\n", + "2. Pass a prompt to the pipeline to generate an image:\n", + "\n", + "```py\n", + "image = pipeline(\n", + "\t\"stained glass of darth vader, backlight, centered composition, masterpiece, photorealistic, 8k\"\n", + ").images[0]\n", + "image\n" + ] + } + ], + "source": [ + "print(\"==================================Answer==================================\")\n", + "print(f\"{answer}\")\n", + "print(\"==================================Source docs==================================\")\n", + "for i, doc in enumerate(relevant_docs):\n", + " print(f\"Document {i}------------------------------------------------------------\")\n", + " print(doc)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "w6iNo7lY9-9S" + }, + "source": [ + "✅ We now have a fully functional, performant RAG sytem. That's it for today! Congratulations for making it to the end 🥳\n", + "\n", + "\n", + "# To go further 🗺️\n", + "\n", + "This is not the end of the journey! You can try many steps to improve your RAG system. We recommend doing so in an iterative way: bring small changes to the system and see what improves performance.\n", + "\n", + "### Setting up an evaluation pipeline\n", + "\n", + "- 💬 \"You cannot improve the model performance that you do not measure\", said Gandhi... or at least Llama2 told me he said it. Anyway, you should absolutely start by measuring performance: this means building a small evaluation dataset, then monitor the performance of your RAG system on this evaluation dataset.\n", + "\n", + "### Improving the retriever\n", + "\n", + "🛠️ __You can use these options to tune the results:__\n", + "\n", + "- Tune the chunking method:\n", + " - Size of the chunks\n", + " - Method: split on different separators, use [semantic chunking](https://python.langchain.com/docs/modules/data_connection/document_transformers/semantic-chunker)...\n", + "- Change the embedding model\n", + "\n", + "👷‍♀️ __More could be considered:__\n", + "- Try another chunking method, like semantic chunking\n", + "- Change the index used (here, FAISS)\n", + "- Query expansion: reformulate the user query in slightly different ways to retrieve more documents.\n", + "\n", + "### Improving the reader\n", + "\n", + "🛠️ __Here you can try the following options to improve results:__\n", + "- Tune the prompt\n", + "- Switch reranking on/off\n", + "- Choose a more powerful reader model\n", + "\n", + "💡 __Many options could be considered here to further improve the results:__\n", + "- Compress the retrieved context to keep only the most relevant parts to answer the query.\n", + "- Extend the RAG system to make it more user-friendly:\n", + " - cite source\n", + " - make conversational" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "ml2", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.9" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/spa/automatic_embedding_tei_inference_endpoints.ipynb b/notebooks/spa/automatic_embedding_tei_inference_endpoints.ipynb new file mode 100644 index 00000000..6ec9a2c9 --- /dev/null +++ b/notebooks/spa/automatic_embedding_tei_inference_endpoints.ipynb @@ -0,0 +1,821 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5d9aca72-957a-4ee2-862f-e011b9cd3a62", + "metadata": {}, + "source": [ + "# How to use Inference Endpoints to Embed Documents\n", + "\n", + "_Authored by: [Derek Thomas](https://huggingface.co/derek-thomas)_\n", + "\n", + "## Goal\n", + "I have a dataset I want to embed for semantic search (or QA, or RAG), I want the easiest way to do embed this and put it in a new dataset.\n", + "\n", + "## Approach\n", + "I'm using a dataset from my favorite subreddit [r/bestofredditorupdates](https://www.reddit.com/r/bestofredditorupdates/). Because it has long entries, I will use the new [jinaai/jina-embeddings-v2-base-en](https://huggingface.co/jinaai/jina-embeddings-v2-base-en) since it has an 8k context length. I will deploy this using [Inference Endpoint](https://huggingface.co/inference-endpoints) to save time and money. To follow this tutorial, you will need to **have already added a payment method**. If you haven't, you can add one here in [billing](https://huggingface.co/docs/hub/billing#billing). To make it even easier, I'll make this fully API based.\n", + "\n", + "To make this MUCH faster I will use the [Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference) image. This has many benefits like:\n", + "- No model graph compilation step\n", + "- Small docker images and fast boot times. Get ready for true serverless!\n", + "- Token based dynamic batching\n", + "- Optimized transformers code for inference using Flash Attention, Candle and cuBLASLt\n", + "- Safetensors weight loading\n", + "- Production ready (distributed tracing with Open Telemetry, Prometheus metrics)\n", + "\n", + "![img](https://media.githubusercontent.com/media/huggingface/text-embeddings-inference/main/assets/bs1-tp.png)" + ] + }, + { + "cell_type": "markdown", + "id": "3c830114-dd88-45a9-81b9-78b0e3da7384", + "metadata": {}, + "source": [ + "## Requirements" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "35386f72-32cb-49fa-a108-3aa504e20429", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "!pip install -q aiohttp==3.8.3 datasets==2.14.6 pandas==1.5.3 requests==2.31.0 tqdm==4.66.1 huggingface-hub>=0.20" + ] + }, + { + "cell_type": "markdown", + "id": "b6f72042-173d-4a72-ade1-9304b43b528d", + "metadata": {}, + "source": [ + "## Imports" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "e2beecdd-d033-4736-bd45-6754ec53b4ac", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import asyncio\n", + "from getpass import getpass\n", + "import json\n", + "from pathlib import Path\n", + "import time\n", + "from typing import Optional\n", + "\n", + "from aiohttp import ClientSession, ClientTimeout\n", + "from datasets import load_dataset, Dataset, DatasetDict\n", + "from huggingface_hub import notebook_login, create_inference_endpoint, list_inference_endpoints, whoami\n", + "import numpy as np\n", + "import pandas as pd\n", + "import requests\n", + "from tqdm.auto import tqdm" + ] + }, + { + "cell_type": "markdown", + "id": "5eece903-64ce-435d-a2fd-096c0ff650bf", + "metadata": {}, + "source": [ + "## Config\n", + "`DATASET_IN` is where your text data is\n", + "`DATASET_OUT` is where your embeddings will be stored\n", + "\n", + "Note I used 5 for the `MAX_WORKERS` since `jina-embeddings-v2` are quite memory hungry. " + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "df2f79f0-9f28-46e6-9fc7-27e9537ff5be", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "DATASET_IN = 'derek-thomas/dataset-creator-reddit-bestofredditorupdates'\n", + "DATASET_OUT = \"processed-subset-bestofredditorupdates\"\n", + "ENDPOINT_NAME = \"boru-jina-embeddings-demo-ie\"\n", + "\n", + "MAX_WORKERS = 5 # This is for how many async workers you want. Choose based on the model and hardware \n", + "ROW_COUNT = 100 # Choose None to use all rows, Im using 100 just for a demo" + ] + }, + { + "cell_type": "markdown", + "id": "1e680f3d-4900-46cc-8b49-bb6ba3e27e2b", + "metadata": {}, + "source": [ + "Hugging Face offers a number of GPUs that you can choose from a number of GPUs that you can choose in Inference Endpoints. Here they are in table form:\n", + "\n", + "| GPU | instanceType | instanceSize | vRAM |\n", + "|---------------------|----------------|--------------|-------|\n", + "| 1x Nvidia Tesla T4 | g4dn.xlarge | small | 16GB |\n", + "| 4x Nvidia Tesla T4 | g4dn.12xlarge | large | 64GB |\n", + "| 1x Nvidia A10G | g5.2xlarge | medium | 24GB |\n", + "| 4x Nvidia A10G | g5.12xlarge | xxlarge | 96GB |\n", + "| 1x Nvidia A100* | p4de | xlarge | 80GB |\n", + "| 2x Nvidia A100* | p4de | 2xlarge | 160GB |\n", + "\n", + "\\*Note that for A100s you might get a note to email us to get access." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "3c2106c1-2e5a-443a-9ea8-a3cd0e9c5a94", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# GPU Choice\n", + "VENDOR=\"aws\"\n", + "REGION=\"us-east-1\"\n", + "INSTANCE_SIZE=\"medium\"\n", + "INSTANCE_TYPE=\"g5.2xlarge\"" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "0ca1140c-3fcc-4b99-9210-6da1505a27b7", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "ee80821056e147fa9cabf30f64dc85a8", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "VBox(children=(HTML(value='
`pd.DataFrame` -> `Dataset`" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "9bb993f8-d624-4192-9626-8e9ed9888a1b", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "df = pd.DataFrame(documents)\n", + "dd = DatasetDict({'train': Dataset.from_pandas(df)})" + ] + }, + { + "cell_type": "markdown", + "id": "129760c8-cae1-4b1e-8216-f5152df8c536", + "metadata": {}, + "source": [ + "I'm uploading it to the user's account by default (as opposed to uploading to an organization) but feel free to push to wherever you want by setting the user in the `repo_id` or in the config by setting `DATASET_OUT`" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "f48e7c55-d5b7-4ed6-8516-272ae38716b1", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "d3af2e864770481db5adc3968500b5d3", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Pushing dataset shards to the dataset hub: 0%| | 0/1 [00:00" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEgCAAAAAC/HnplAADVA0lEQVR4nJS9d9htWVEmXlUr7HDSl+93c+emu+kmKEGhAUEliNkZxcHBPI76M+uMYQZ1HB11dByzDphGxayoGEkCIggI0hE63/zl74Sd1lpV9fvjnO922+gjs5+n+97nnr3PWXvVqlpVb71VCwU+vguP/tKpcZ0UwbSZQW1KnX+sT7pPARQAgBKSGOjQq5g25ubJ3/cv/qDWziUHUOWmQ6+LR/RfeUwQEEBBAQEBP+Z+AUJQnd/08YzjaDjJQpdNS/bQWZOQmhIAAGrnAEQ1TTZkOoLoQAEQ9GiwAvOBHI1rOjwcqgkRM8NgILrF91+dv/93gWQQNOsc1eW03xbypM+fIJD5n2K1Q8cOUshiDk++/1+89lcVdTLqssReQen/WSCA+ISF8uTP6Z8O81+9UKLk0c2nOloUqUsLiqCNegcAE/LGAMBcFnr0ngQKqjAXCygwW1cz9QBEgR6fhaOB2I93QEdXRcGgV0ElsKKKgP/se83HAEAKmINGo1YQYNL/OH8nDDv1hyvjHAwgxo93Ic/f+2gMH6sgQAoC+LGC+tcuJQVk4VLbIrLrDjadRgeh8yVoa+3B8jT5SKyIiKrgrj6JigpqFFRVwV84FV2ZtodqPEh6wn3z6+MWyNEL5OawsLCzgoejptBy2kNF0IWKPuE+XazOxAzsrFc2lrx8vAoJHqEDAsw6sMDNxyvHox9XRVDUf0aOKKCA8PgK/ji/Ei1C9356EREohre94dWXu9mwa3rNeG86uPUpP/z6k1Ad9jpHBkFB24VGu4XpFBVhUX3gR15x+41NXCNJDRr6mIXx/6whqJg1xdCqQbpyDFAVFUA/xgTh4v+aYQxkYpGkc/Zwif/J5//ylXaPK/THJbCJxnzcpn5xox5Zjo95DnVuzwTw499AAEAF6VDfvPZJtvK+ufKBD9yxTzPb2kZUbbj/kd32Q82xtd44GUOquNgKEDpcKK2qqAIc3v3MYlKtnJ0OehmBNsWTfuf/WSDdbL1NDbz1OX2xb3xVlhtQBaSPsQ2oAKAKRhVHcN87/z0bBH7ni8qP83fq41M0NAKxvk32Y1fSv3IpzBXkYySic4VWBPwn5u1fuwTU4NJ4w1161MzOPfyu2V1hhcb1MWe4E4ptNVqOMGmQAVUBjVn4NbLYQ1QBEAkPHfyjUL9bXj+5PuxZ/2Rn6P/ZZBUGfu9zm62fflqvSX/zmc4kFQUkNE+6jxVUFTRyNKvprt/8ImsMTP707G0f5+8VoD0MD5+1s5Ws3Tl8ysc7zqN9/J8TxmJ8CKqKc+P+cUtEbesKAPnR8Yy0rlcuNyuZFymbInXJalPWrSk8FA0CKBIFmE+AXwwJUQENoZx76MxgDL3p/iOj4cp11z7Zhn+8JvTq1fi7fzhfyceUcHoIRapm08lkOquefF+IIXRd13FepGks1pxX5vjQx73Q6fUJ2sNvvaSX97C75+76430O5+ZhvjL+mV+bOxuqIiIi+nHvaUrtLF3YfeidlzPhotzck2PQAlSxnoXeKJwz+TAnG6u5i4eUee+ccy5GZlVEQwjKKQ1WbVcLxaFRPPjHd75HF9fV9/7YF0JEAEDuEiKmGaOEhKjcTXfOP3j/I5dhs4IxOou1W9qrY4wxhtBNqrZr2y5UdV3NJocHKQmQIWNRUmbec7qIqnY8XK7qpouskFhg/ksqLAosooCIgKgsiLMffUSL8f46//YBZPdUPZSuCQyAoogqmFRTQgSOjAgKjIgyl4OqIXVioDY0m8+mKqgCIiIwHbQWs+xbLzh7/8+blFgW6xeEdT6kuYIxs6hwil1TTbkF05j8vWcz4AJs45d3KvZ5lll7LON1Wp468EkLAyKA3IkoIBnrnLXGECGKRXX1R6amB76mYHqVK6q/rhOjiebqwvgXTVbnchAxznFjMzl/PiEaYww1F/92WsB2z9bZvf7y2oxAEUBgaiwCEPLilXjhWyb0+ezw3B2MwObipPGLZRoNMqt1R2vW4VwcgghAzlw5eW09qG4Gqfri7vlG7lwOqowgoJBAHQAgSMglRVRAUGBB0GQNoVSzT/nFF9BgqkM90hpQBRC1OPnWn6/L1t3Hove9r81hbukRQUVV28X4eB7JIOM8vAyUEgtTxsDUm+4Pey5ZE0mMc0IIx0cuEtHVlR6PYtMnrnwVCTg77BWinR5gk0vxc+ZFtxb5wfK/Gof4mNCbGKVX7F08hMwIx0hEtLJcmMPaGoXHXA3CSAggkFQJUIkXGja3GApZCwjtpZeDJ9GLSnN3XJUIuNHCy2K6OkBABEyEiIbNhYHN4NygiOeNhIP+B6AsIAS0ACZzZO2MEA1ytEmzzCAaATAGwCsnA8ZpL02X+y3IkbsHqqCisyXKnZn1AxHBBcmrRXwQEUQVaSGPxSwiBCRCREiQknJwwRkyuURKIGi8b1CFBWNPuujQXA20deFXHH2VzPcQ2O+ZlJCkLblIdmr8/h8fz2UYjuKRf1Eg6j2oOg8XHq57/TAbIQoikUiVy5LbbwfTAL2G0mIheCREQrQw31ZpAU4YBmtMfb0aMLS3UqDOBZDmI9aIhDj3mxER0M4BD748Sr6ZIM12E9xvHhovhSl3bF0HgMqiYAwBAgCDN6GJS6DgPKYWyZUl8dm7rnMV20MDc8tIIgoqGi7ac78UernM7l6mnZUrab4Q5tEiIaaFRMyRUqkoHoVV3LmY5YIBwSbWCMJIAkCI0BlJ/0wgeqQiQgCEBi6uLmfSRlSw4DnG9fGHXmZ6/Xh0/78oEOrYOZxODlJuuMmPTQEJQZmm4+PVaNJNB7FdylprABSI5sGI4Fw18UhXURufd7HWazUC0u6GCXP3HzWBWoDAKISgYBQVANUAIBLA5c0V4HSs293wdG+qTyOosWBoSCgphDgwJImNIwaHnY2ZJo7MEZ1W+23ddn/9ycNxz5AcCUQVAFTKbly/bb1/qBf/bMucOxERkZAQAwASPW5jZLHz0BEmJyASKhxYLKGOk2FmIGrkwqFxjlxmvKWo+vjzc1N85H0CggClbnuIETGSaW1TzrB79KObg+lK1fvXoRNKIc62dgrGzEFqMgCVlGSgNzzkZoltstvXqRfShSOgMvdeABEJkXH+RhY02GCMC4Rhf1PtYt91oBZBxS1MFh/tIQpkFGC7hFlx//WxurWgbmU9qwFNBswzZwGsQhICFkOJSYz3KCiswJEzKwWiXs86caP9bLHgQVURAejK6PzJwuwUTrffpzuDIi6gLxUFRFA3n//5I6LKuFBhJk7drPChKnKG6Qp535JE4YhsUu0VrCLJ0cSaxR5yNc5AVBGtZ0sTKr0AwIrHJmvypc0Tsyq/qln/okDa3M3O7+NyYFXhyIkMETptqsmFEsLxEbdXnglGBRERBKwCIi0czjmIpKCqruGY1yYBWqjGt4HXeaTEc0dGaIGHEh45XaCCIlc+eebgI893k6FP555SdIRkSVl6BiJzijahz5TQIHAXwaAjVDSQQ+rUOVLxQ+0Qj9xImiO8sMycVh9d9bPBcqGHYac333nRiSCiqiyEQQioKMqLvU3FKHDXb1ATe4o5iKgpPCIAYZgOKRlAG49mlhYacnVPQQREkOokGavoVNCe01WXy4GY2dK/LhB37kLXH4aZMc5JSDQAABWFdOLDn3SwdNcrsnb/IEEHQEQooiSCCIBWVVXnwIQCQKV9TdvYh+igmfX0yFjSPDwjApnHavNRgxcBEObtZ2Zu39/st2d25+1fGzfGACAsUFtktNZTJ0rSCdvMEYARY1lFhQm8MZpufO/Dt6UwGPcWZl1UAFW1Xj5/7OJt3eXiGgz5/q1r9UIzowga83gAM98ZSREIEQEQSQnUtsuOG6K01ouhNc5ba3MxQseHjapKOloA/CSTFRUIrPq6t2woag7VdvV3x7I+XXhRrv7CWvZkgegioEIWIJD0oGaWkpI32qmh1BmJYEkyfmRjUCXXGd4+oU4JQBUIgAhAEFVFQY1JjibLY6GsqcqJK7PWQ2UeWwYmnG8WCoBmITWkuYFDgMRk6szsNaeMjuISbJ+qtjePUW0Md5hJi7Z1GB0DKZPNpXUUE4LYFIEMGNUkBMJSW2MYe7qYmKSASAr9bhz9tPPxel93fkUNgSihJLKIoupUZCESXIT1oAhgrRTTg/0za4rceW/XatZcOQ9+YrpiVpctBMoaJ4g0V3RaiHSxFxkGna5cnq5ZBwm4LQxvPS/ZWb7cZiHLQQWA/hkNYSIk2bmUBc0oSpYUFUgNqqD1mJI2G5TNjmuIy8fsE+GJ+V+RLKqKWG7cVLRsSZNOKSEBVuJNAgQAvGpb55kJfUK+Q8BmYb8jCrGRerLhWiDQpAJI5KIxJEzsVAlVhAiRgJTJqCgSo0EAJNsNiPWqSQcz97bFdufVhFQEFEi0dDV3cPQeHSAaJOT5P6HMP0OhlKhj6IyzYqUxBtUSkVOXRBw4QlEkUlyAlov/ZP7lCCSaUfORNd9Fq5AyafdNnhpVxx05RZjjOB8TqTtQTAeHHSSGOJ12zCwiIgSKxiA6iGeJH81NGPeW9aqJXoBnCG2U2LZRkkQbR3BJc6v+cGCIwOzZPs3FsXAEEFGBDKEKABERgEFlkHDx9BpItrxZXhzZsTWo1qBK7DoFQhFFMoZARXDul6KgMSCKSoZQgdxsIAxPhEbm617oIkBqO6NJG7chqqqgc3urAECGCJQTLtz4BXoAaiTSVEwbYhSCGGJSAGEbbJuSTRbQgF59rSOxgKrOUyREiplsrZnIQNoqd7tovNYzUjaunaerVD9WQ5LD6kq7NmXRFFVQQVUlOQXggOJiHEG9pQ6viMaSn5QHwVFTUT8PtbCnw9ng+Kyv6nY3jKiardUhACLoEzSEHkddceHtI0YKFKwkAHlsgNsjR2pAVTQFMqoA5OY+EaKIsiIqqgIpsCoiA2i/dZ2aJ7yXKiIaTf5gNUqaDUV01lsJ80nDq5gJAYKKaHbVZC1SwQbEHpLzkDpAnwcEiyhCyXekiCYRMIESHH0TXt1HAQCSuBTBOi/gPXcBvZ245GC2D0QkwcxDh481WbWDOOtS1yF2mlsKgIioGJFAxFqdzRLUxRK4HcuGwpGKKKAiIMz8kENQHrTmwoX2bN9yBLd1PaAoPXa2L/MQ5XF/f+7nPw6vCxsHofDS9su2qNJOXy8fs4aSJiA0Ls7zofNwDZFkPmtAkBBJWBBFGWQtp0j+cdBu/jdVOLhyoxGTCYip1vts5nmCo7gJEyIiEihczfMgIAAhgexnWT+CVTRZ7sA6TJDEoYHgMlajSoswE+BojRHMEzAUwHCIFTE4YoeIWA1ROkwOiFmDdYCiHyuQHA+vVIYOPCUG0CAGDZI1gQyJ2hxTBWmyuh7N/pr3kvw/fRwtB81caH/mXWuf9bJDzCcChAfXEAvqR87YeR5C/0kKFQEB+ciWIyZo8PyzRsHlz+ptw0guPQWIkAlAVQ2CAggTKCIRMQKBAQHVORwlIIrMK2tISFcDYDAqoCpqzm2pY5sHk0x9iwG8qq3zoVmYe5PmKAGMcxNLTNjulh7EZpyEcyuEkrgTTgDjfGiEkI6Qk8cFuYhywLWck4wfKFWkE59BDNOTLkxpsCys6AMYWizQf3q52cWtaRNBuw4wNgEAkQiRdG7trTbrOs2HSfePlSY+ISU8X6kMBvfe/n2fv/9ffunOLZtPcyALs5MAYPi+dV5M/+OXKJIhBAAREQCx0ArVHz7h6gnfDnvLy7CzyUiEoJJCBGFBQwCgwjxP/hCCCqBymm/0gMAwSkY1mScODxQAKs9axI+AUZkeY0vzAAhVVQGJvDPztb3YQ65u0ZGo3c9dbJM1rDmoMHOKiSIqVEsjB4gIDIsIf6EkqKIASKgIkR1vOxWJCZBDaksFu9c/DgxKIQmA/jMaki4f9PBQRlWNgyyqjwgqyslFIQKOoW02bbK5pOm6SelqyH/kbvHK4Z//cffiL1txD42GOF4OzIarXjSIfLFI2ZMZBkxkUEWOllXrNQ3KauciL0EYSb3c19mQAZvIgIomYTLGWJkTOwQZCFSFwRAnMVZBgRAk2OBFxKaj9wJFJFApNpKsXPy7a5Gx7gfPR5vvQgwJFMgiytVk8GKPSw5D5QhFDFKxysAGyIDNZ5a8WSvm/j8bPKJWzOV45KVFazouCkJRQuFUCQm0bvgBu2LYMDPPjao9suUIqslf2TxgDWQ5aknaITF3w2h7hwoGCBQJ5MZVPd8LBrdvVzYtEBKoqJs6q/WJK6PdN7ztVc92TqRoCqI60ajNZydRCIzckgOC4gJYAgSADCB44nrtULwkIo9UxDbyMyVkV9arqoeEOabGJ7QS1Fl1nKBGUrCksfUIwpGIJIIxEk1AbBAVlxwX0TVuMbEW5msXw67t7RGCmK5b7gmpgvI8BCPlOfQjAEQIonoVmyPHRd0NZOZNA+7KMJ+yaVymnbTOBix6LXIhmusR9jXHe4EWGVWqS02z8YFtRomNobTfdeuaQhik4X7MWpf2T0Cbd1dNFoOwjyv1PVeaLoLWxhCCipBBMiYvEYkIQXvVYYhXRn1KtWnVOEIEVVWT+hIGe93/+bTHfvbTlk8dmiUpBp1xJm+tNJKAADdGuczXzdwvVABsJR/qRE/sZCMDRASCyGCk7wqKvXycGl4KIYKqigCisc4556ykGEJkiUnJGMJ5gmseayMgYQmS4HG3HI+cn5pATbum7BAGACwC83fDuYN75LTOqV2Pzw9BnO2BFM5ai+QRgLw3ygLAvJsdJSof/4InGnJVJCCCWSLgJMrJ5geWVOnQdLOk6gC5BlR6PFIHZRd6F1rjCA2EBfoCSKKSkBgJgUAn06R8caW1EpbYqyzcd2jXz/XWP/R7j3zSL50lqffOtI8em21fIxPnp2UonYmIcrbEkC8GSwsuW891HS1351dx3GQgfWWLbCjK1GeXrqXpSMy1YtEJgIgi8JyjABYZEJEEkEBVcf59c2SMyMiqQbbmqnE8GifvUmrlyiqmfAori8APEQWVCFEXkNoCGiXUI+gl4zDeuo6zCFZFrYiStcDW+gzS3jMJRf8J+UbnjucRokWkBsxe8sgKoMn2dpZLxyQn8wAQrdpUFSYZexX2FST1zSUXOwBVtYBEBkFJCZOyICgosA5ODiQ8E5BkzZmQsrnTqfWh/vjBj/xg1+/qQdWDvfLhN3zKc3eKlSl7oNt6LiXCmynRE4LYuSBbWutCWI7dLM8uT08NhRGJ05lCwl3P7N1z1sEZMESCsDAueDQ+IFSZ7/YKHkHmmVqcxxtrYJPFq5EhHsUVu4XhcOVmk0xtl+YKi4hAoHgUsD6+tpFksTmSpTSb+sRRLTJLGwmVgRUTare9OTcU8E/kAYq6gGGUCCzZXcotKJFy8leuzaFTPpurxUSK2qpt3eNeVkDfucuVbadBonhQBSJCMuozsqoAIiqcmLjZOZYrRguwmAVEyn/5C/RrB4d8GLvWWFO+5c233TjphxmEXpBnO/CE5ibh8moYu4C1fZkODrMCdQB/893f8IbLqKwOIF+3js4ZPXet6XpRY5hTjfRoQWNiAdAUDYII0NybWoDoiGRl2BEtiIRHgTMiom71M5/aZUd8OChYjSEEUKAnulSL6Hxh6gAAwIDKRC2HyKoqEtSQRCFvDRLjtf8SrWIO2IgqqTFmNzOoqioc424Rk8hhL4Y5OmCY4YleFgMI7xpvXOFataiKACoCwXomAZyDcHvnDxR3O2Vqpkz2yPJAvfq6Z03QDBuzQrPdd7zlzJedQYlLBwXbSq5VAZR2I7hcFq89j9ZVFcyoih/Z2vvQe/KXf/maB6tqOjrFLO4wh/FJU6OqJgOINGc5zbF9RUQFFVQANBRFEeY+LyIiSTEdoiZ3FUlAQCXV+pxV09oemjA+7hEMqM49qSdTghbQ2pFLGFj2LBhBFEBSNJYY0How4EL/uIousnT/1GQdecGCjAC7maoIsMSCtYzeKQxl7uewDa0jvWqy2CrgQ1NbKSUOIv35PCOmJvPJJktzF1DBOTcsMDmtjFg8gmmXvn58oWiWJ2rksbe88cyXPrfSild1pXUdRQ8sPuwVaPhxc4WoAmKipje/7a6N62//oo3cZr7KgmAcPKWPKtthdqmPSSwSRgQkkoWPj2DnUAgIzcHVOU1B5WgLMFmlCODS1Qmax2nN5WHIq9wy1dXTLOFiPP9EGIpzPRNVNUdebALddVoaogjGoJO5RWs1aLG3SCJc3XEWJk9RUVVVEdQkTbBv55CPKnX9EZB2SyvGqhCbzsisR49jWSlLCA9p0aBNarxhAYMG0AdVAFSapwahGNCMyh5w6SDvJOY6X+fTOsuWmpZseMMfvvQnTxftUp31JnGIri66aEAQZz7P+AijkEV+sN5831v+8eWvWV+vybZVcJgl0ejOsKntDI3TlAIIQCKDiCS0cGWUBRSBgAyKpLl7Kondwsh4jkhEPj0+yfPJb5YVQ2kTpbRuaB4mPCE00ifgv/McweITJZhaLA1AVEI1wIyEkitJgYNc9EilnqBfiycF55JhrBBpnuVv66wA7sbesTJYMGy4SYQkopJiTDYpfUiHY4xNBIg1k+tPETqAIbTS5QxN1MSHUK3m7lCCq9JmoylHBmfI2SyY1GUy+d+f/eAvfe2popXamtoUyYQeTc4Su66oV9iwmZdnwECMbWDk/uHrvjf+5BdeV4bUtXZ1kJIKmf7FymDvXmN3lo9bvYWBFfo96dRoAgMpCqasimrrzDhnENB4A6pknToMmrsWV8+bkMV5RtkYg2gMKuBul8X8ygbXpj1/E7GZr1hJstjT5xwTFWYWIFBmASIksRnuD5cOJjPpFZyctlD1oQYz85BmwyE4p0JGUyFpJSZJiAfYHweJUTQmcVwdDOtz/SYx5laMHRwX28TBWoHZlLK2QyxHiGjbDMQSNJ7NpQ4n00pSCElhp63tYX+2e7JusFxeGc4clx4GaKebtFNlxKyABkTIpuipXjYHfnrwu/c87Ts/pbiU2eAetwPqkiJAunQG6eoSmiW0K+Ydv3z4sm8ducbZLEuJKzuoQTVicUuC7rET2N650nXbmKdkx2BMlKwSAtQYXNUTTOU0b8H6XFKk3Gqo2iF4BKp1YmcZT1wExbm5modLeMEu8cVU2qRoHM1JQkfOLsKcGfD4MAkZ5vk0UtzZKV3oM0ZWW2BuZdZk1Nqsnhw8pc9zDFHzdhivLDWhv7d2/c7OIK8QkoJKV66kvXxvYJEVNIBS7kCQrEm4ovUgWmS1qbMeEtH4HW8D7P/DYNPysrUg6P1zOGVpUB9kJi8gREiXt0YnKBycoRsJR2sdKTkiFUOTbP1w/9iFTX3be1/xzaWEg3ozNsvVUb4NQV1KBNh95PkLVBcBwBrbTJY/5RnSwgBoWmUummGsgBSJNdxARi6ePPibJZ3cVfb6kWdgPbdJrCoaEM7Hg87FQVcIAyfmpRhqpbzoOk4ZpNHSbHti0mrguSFHO2f6yI56e8X0XBRyJREt3OKrbhUt/CxQFSDEq2x9pkfPPxUQNCkjJyiti7GXLEiqD651aSEQpBR06oZv2zP9Sxd1ihSrhBLLSWtGX22exSgJQCiawoIQNnIwWdtITQCoYc056wGb0P+Ml+Th4c2s36ICACHhYe4UOiiBUFIjdqVYKXxwg3T+mXqYLVuhokBBaPqDZjdfbTb/9I93vvJ2w82o7KoGV5qjgIMAOGOjJj7s5nS+ua1NdoCzDswQ6cKoHM2Wp9MAxkclm+Klp4nx1Xp5/kUpfPClMAZNfVMn72QmAkiEVdNqWVdwoSggsPOVggFgLSNXna2GrlstSWJztElaVBZgndVV0RSUiNH3Ca9ywuaDVbAgKjjPLj8RtYVI53ZN5xp1RiHWwUqmHurVrkJsTic9Egi0bdGYP/vxzYvtsi15v5/bwUZPZgUOjKbDEQIZIZOSBUCI49mpdNiB9KzJUBPZ2rkBB1MmftTzJDnfBcpN1y1nGBz5LLGmJK7cK3KI4NlWm3r+oB2K7feVLfZ31lauSPj717kveM5JzWvsSOx6+/DKQh5iBQ83AdnSlgkMugiDiWotVuNOsacnh3t/eR+87NaeZxBktmgufaLVtIdmb6XbDY+l2U43bnZ3OqudsZZUjUm0MxzuXzmjvRLagHYgwoJEcWY7m6WVauW2ykvXu1qfgaqAmi4QyOHINFkE10NdbNx45E4vODEAPN9bgI5gdJEtwNAD44zVDrqgAyoiazJ53S5fzUxi8DZmf/sjx55XUl91PMixw4K6K079+J6iRwRkVAhbiI40dD8rvWhKXfZrt15/BtCWzazIs9DYKl4zrRTRYj7MOBRcdys9Ly3kWTjsaMUiFA55xaxgPSQRLHJIVmF9/3Bz501v/8+3Z017bLfM90xRt70Tc+cMVFHcpbMEKDqeczHmGlKXvVhX2Up3Eu//tQ8/5aW924f7TZb7hpMFf1AA0HS1O7/Xf+yh3/XDumwP8drbVzM7LntOkoAZXVhd2rr/pjUF4K5u+92sZlvmW2wcedk5c80zemxTPJrQZFRAU/fo+kbcv2YQIYItVOcoMyAeVWmyzs2WzKOaea5bEYDilaEn4wFFidAFbkq/XzbG2Vb6V1Hl2ncqH/hf659LXvftMHGEBKJhqcfL7YXjZAkAOaoJGC1p2sClk4Xu3jNLD6f8aQ5s7QuIyVm53Ot1DUqeg8EENhdpqA1p2LYiNuvtuh5XYdC7shur9sSAooABxLS7NLLv/l/FD68cxuXDmXattcl7rf2CUy4U8ovOoG3bOhEsNERhlKbgReLhW95+5XlfcqpTmEKRycwYS2L2oHPUXd/L1y/L6NXXHKsVJrjW72qpfeGUWbL+hltZX131Cs5hiGINpCjybKmzRpu/ePbT12bWDtsj+DwgKKCk8cCH2PcuBKFMRXFOdrmaGNIFuXrhA8wppoDAZnp+VGaMoCImhQyQY3rkRsy0qkb9q3hQyBvf/sLWN8DWqD8spNRoLKpaaA+jgT7RHJ5HYiMChMWd1zeNv/UTNuNWFqMmm5NGzLU1FwdNwkwkz4HBZ5P+qhl1mBWSlAPScLLfG4pLaeXa/iw1mTIrEvI6nf+Vd3zml/dl2NsaphxBo8tmaaW7+oJC+4YUUxvYPO6o767j2OH73nbv2ie++hl7zRI24qSzWfRkg+zP+tBdLqMsDa/76CtJfRdXbNg2uQMNiRAwHOgOhFURa6BT8kGTCDlb65TUZlsffXYfTOP3y6txxoJfGcZKFIrAgAYWQdQCx9I5lRVUZE4HQkSexxAQXbOdeR8KY0UgdUrFrG0eWD7jqmlY7YUj1KU89OW5t39Dp8eySU9212ZJncQErhyYiYqbZ9Ssn+9WxoTplThou+G5E0up5sxaUrEIlKelmpMLvZoieyftSqjM2FmYBTAmA+rsmdqh23toxcpbPquc5WETBYm7pvy271FVwnakKsjkMDgzNWgRRNSlQQg++tS/nzynsrfdlgI2ZQ+88x320170ogCIbamB7DyHUEyW2+zCP/ShDvUt9y9df/mhEycJQ2dUyYLw5v7KWMtxacZ6wnXU70zNBXc5IYKKSCF9osRnZcXWNKx6ioSooA6j9mL/Q+fvNCBmoC6/fFPe5q4FRJyDlggAmk9X4kEJdXK1DLTzudYpx0TOXbrynFpXVKZZk0PW5THLLppjVTvYy0eDKI6xaLsylu3d3/h9x8SmWBAsB2ucgUzAROdHjyyRiJI4I3p4Ik+S9OSxETtD3bHKLo8cJqtAAKqYXH8PRju7uYCoRLHgAVDICZCoRMsKgphdXufD7eJwuUrCCcBaUU8CZl42Iws26XwvBEQQEomiqhAzRNTz62la2GB++q7Rlz0/xLwDXUCjR/5nwnB+WOW0NxzsTypuY2Od90EUBAknRd29+1MNba/YqQzsOFOviA7ndXwKngFIsQcuelRxQRc0eyBV5TqAMBkFGlxezyjBEXByxJppy4OQX+mvhp1HwoZcOjADzzGZct1fDNUyHmTkBp47ZxA4NAYzMzl79+2Gh9RJCuqLi83/KPNZwQqoigZRhFl0aFQkOTWoiIAGFVkpA+d8QiHbgWKIBi3PWQPatrZMbX78YkKweabRGhRFSgwExlhWtDYKBumXd41WK1WLgmqMBzN3HFFRAY8yZHoVIlLtFAGgzhVBjrWpcJO/+dPnff5T6r0y3yoWaZ0jgbAJPl5eFbUXlwaxhskEvVNhQ8qqxjJFN4UcemR6jJhH9hTJsyxQTlIjCNDfxaavge1RicackhLG1HfRYItVeXib06NaWYQjHlDwgYatu/BrWw9sE0/DgNkO144tDcudEWqHSzVPArcbzoBoN8hnJa5deo7xbVdSvZqfM9Off//3mAEwEigjOARWVUUrAjGXhApICLEBBSIwCKLJQ8qUkiZrAeaFqWOced+6cVFC4ADGIisYYzvVhEi2ZfQ2ctp5quPVY7VFLb0SgwABMykhoiLigm6zeFFSMIAtGgQ4zBMBjG27lP74wy/7rO4wXxdcjjoXyJHfGFznHRc90u2BI9uTyjgfQ1eSKqtgeVjS8wk6N9PMJfHCBpN3cR6HInKSaJL07hdjO9RFKA5AQCLAhyYn8A5A4WB9oRiPl+MiQBkys+/f84v3D/ub0vVN0WEG3b3T4HN7EO2JSgeemv0yZ4ta9Y2LttnvZTb1KdgQYPrLb/vRinmyNE+okRKKIRFxJkJbQjAkYA3UU0LSVGcQGTgg2QDWAVhEEETg3eFe58rmDz6JCyQil0TQWhSPiVUNIIIoqISBtscGLCjeihFMCCIKR1488ryFhBAs2JGoWqtR1P0iOoS+ySr5Ej0MJdZ1Zton0YhAypYxNgR6uIxxhP3ILYFTNkgirKCSbt5rTco0aUHRirKCJLdoYCFMQqorB82KjdYCLcJwEFUh3ueIYrw3VtKmqjDgAnw/ipuChVD8xZ9uBB7lPaOucbbxK6p7u3yuXFrOsjKS70xkEj4Y1UvTekw32CBLU146mJ35nj/9gS093aggIgFS60kElNnahOPjajwlVgXoHCAADyyrFVSLEoeWwc5dvFjPylCkSL1xL88KqyRK1iNHCwBgCD1AQGMVBwZH+eCAgjAaniMNhMqKoPMMGYKCCs0JbSrAlZACTkpB0L3+xB0/2DthZlLaSP0K9J8kdpyxSS9RZ/XyCTx/ou2mZH1UUkEFFG6X6jCBVDJ6Hyu2WWaFNLBRUUUUQktWdL2+uJ4AYT6MBZyLJmz1BYmjBTjE9YQIRyU2RwNobdLRYx+6aXU620/9FbsZjUfybbUycOPu8ttezsWkNGEIYlmbMtV5l8zJ2uv2wI1X6I/e+N1Vs6YDJ4poiLBDFEFQZuK4tankiJmTerYCaG0GCREUKEl9ApDtfAbrfTgc+sOw8sowtAqxSz0kA0CuQ2MAUdW6zhptH3iWHt68fOXYTCSSlzn5BZRV5gw2M89JzDMXoAAo0CmKQDuwRmQpjvhScbIG50kwzRa40VW42rQZdw/eIkwPvJLuvkGrJs9tJ2CEFRCVUOw0z1XqbIXGl8ONq0aIxS1mVUkAiGCJHrpZ3ALxAwAEISJsLq1ZwNg4TTu9Y8CEfDUdMOeXeNKpvOkj1w0HPZxiT8ePVCb21+SYbc1S7yyfy3I7aPfqUSSDsOSrpeHWshOfbZd8pbz0a18328iLSXMiiCIZUO9E1SJClLa+cLNRJVQFwQYEQdkgIAhZVdNkyYhVAALtJp43d3kVB9F1sZeTOARh8p4NgnBM4H1eUFNtg1kJ6CdFA6wwr7lA1HmqChBJRICIeJ6IAyJFISQGLqwRCaqQpQr73CabW+ZFiuuql1Uvcbry+Vmix/rw8J25QwpcF2VUUURSd0C5C9nU9mFWrJ5//3LfKyUpWFEUAFJyLAq+3AnYS2ATzOvSQIgI2oNCyAJYUV7bsI0+TguZ41aqsz7N7i6xY1t4iNPyRB8e3Jt125ku0UzLQTu1wy6NOEZGLX2WikfOFgMe9w9k48IHb5fNTmJ29vzSvGibQTmJs8Qsbb3XZYBkrKIczlhBQm0dIpHLo9rWJovWBsmgCVmSPdrg0JlEhRHEushB2iuT6zB2JjcC2mmCzJ5cg7ZKuUzJ+qwCleG2L7cHBiM1Dz6Go5PXS9GZSGxclze+rBlbdCy2KqqKWt/rknWoUAMVFkXFIM45UXPvt+nVuLV3HbnZxZvb7kzteLkEqp22WpqgPuXCtkgZAWJLn/AJbJMm8AiIwCLA2hRWOl6TItlpXxVRNRFSv4nDyWHb03JCKUvd5LRne9QYBhQFLCoYXjkMB28846xLde5iEUqSUxs2v4SzK7aOoweW/vHW1XIDsU80lhCWmA+flidByHep++9fX9RErp7lAZHFEat6F0KrmU1r95dlgVYBxLigrI4iOEFUclI2OmxyiFYBQZmFmcmSJQI0REqD7rC/Xt/1llct9bIYTNZp3Xjtu1R1PbbQ+A4ZLBLWo2mXE9QX3voBtwxygT/zi2W2cWWlHMPhyO2WaHIGUqNVgYXbseANhwSeIDSm6E2OIvpFNCLOH1b5apdttY0z68a2K5EMJfUi5GI7LzAzeASJeGUFAmmMQQAE7lEzBes27opUIQsuGImaRCJwtcpN3RvADA9fguyqYlHbCEqL6uBO/W9SMUjWgkQ1iIQOiM5gI6XIYJLCNSbe7oMattblHZmsl1lx0/r4Y/8Nocd2qUr73sYApU/JCEtSS5lP4aJpbakxooNUOI3GmWy2RhGNhJSFWhqbWwaClESt4SNeEhGihj7UW/nta855FhWclcaYSpNJIYcAyUUITiilNqN6+fDgD98x/LfPGVo596G3vegEsGxfM1le3TveY0XfgUJldk7rlj1Rq7BgL9/Jc8OxbRAAVR6vdh02rl9TnsWHjEHs1Th+erSZjWCI2WJURBTEed4CAQwzGEIpQUIE55xtKum7cPYd7FoHYhZ7uhUy7LiyeW/sOYDs3a4xD6XM+WYoc4aZoGSTN63nnTEeEoOxCpQZNFadOgS3+kixYkYbM4mWXeZdQxA0IpHdeOi9f/FVm7VoXa8ns98rUxuyYVRRIqRpBrMLm+UwSBRjQtdK0cul7iJ5MWVeTXOgnmmtBQJmIYcKYOb1PoQA6FFE3LBQCIkMBzG5s3VM/aFPJwIQaJ9YUbJceW3a/H/Hvve5h5KTf/bgCz//f5y9cKZ7+HQ1Hu6EHiO0CaGn4doBtBf7gYmgq1xXk888ThVQVOGoxJswDR7dLMAdXHcjXdubtQebKkABESBGsAHnBKWjdAvPawAxGDQswl5Cl9TJRkQ0YNLc80OlpBhdUzHj1CK4Bk+zRicKKiTzpk3KoJabd+w+NYsEZDAK2KTkrCIjcWeofXjn1l4YTBRZMFv2YCBVsYFsajj94u23VMtj27gwjaN2YkvbTZdSjEkRyiTN9NrQtQaBJfEo51ZcNhJm4FY7mZI3ntgiADN5UKJ5sQrMU55Zl5yJtQ7aRjNSWMqtsEJjhvnsmTbkya71NBltuCkf2Puuv6CGV/ZTfnl0wyO//oNZ8Yefe81++NXX/NtjSWJEMdRUs+G5k/n56aTJlntWl02SmOgqpeAoMByvh/Z9zyyndHcO8brqSlMtEbCiZcBY+4GZwyC0YEtgtJZE0WyX/czFlIJzbu9yfzk7SZy1CIyoikIxJU281+AsXVodUTDX9ITaIioICQCQKiorKT/6uiFiv0tsHXTJsBKBQudyFu/l4nV39Pf329wIEa1TZwC0IMCNx37hrS/KHsoe3C+bExcT+35P6pj3GAilrbquzsLFp3chWgytcXmnqiw8PWvIpVkr/bQxELYzi/PKcJNY5nW7CCAI1Il3XShWdtQDJkbLuUZqH2OAeINtUaTvIqCUhycv4y+88NLJ3h5sSLUxO1zVb/qCL//da7oMPv0bz/8AoExKBEJubly58B733o/eN9u48YaNl/b6TinLO0S42osPAVBYL94Jpty5kfQFvd6hGITICAyUoiGawx0LzANB0QAo4oqhBOShYy1WMuebY6xO4yKRByoEiPHAFmUvehGjdxigNlMAWDjs89KB6M/d89TEliQ5A8LzvmOqTB4YpDr1TA1l02uyXKgeBnHocZihHFy5cvxe7CIvTe95jpxozLYUfd8cLm2f38OltcF+DNvGOTGG22nmsyut9bY+uPjWk55S7Uy29JJb2mgKC4BIBuLi9YDmtC1SSa6k5nyPFIABOOY2FfHKsokAgqyRWIkkxZbufeRrnlqHdTyEgXhIYfOH9k62jsKpP/rC/b6TyZBE9bHp6EN/9DrnxK7gPe9p3njjc591TRZnvXnmYZ4nUcBy4jSt1GXY+uw6rs4M2Y4NR2oEkMqMIIninD1PSAg5igqAnOiaIESYx85sUGgJmToXaYGNIHgGrKelG6xY26nvrtFEps3iEWZy5Han+5YGqWoLlSRRAACVVcERGmPT4U1lxSM1HMuWpj1jIkDypIrXfn81uuwfqzfePPjMlTNbmc2cRnSTmMBgjGCrP0JL4yEIWKftyMfgoLe2dF2mLIVWKzeul8CVVQQkkegiI6qoQxQEJC20S5QbaILJHEHWdNr5obtmkLKZOLBAYKxvZNj4h6976ja7KZQmCaRR3Tw3n7nEsnzw1WMP2hSQoGjuHHzFpVPSEwHJh3n9rreffM4Ln7HZqKIKqJnz/DRB72K76lKS5/oW68y7NhlJbmZJXc8DqqAxMuesI4FPiZFAagav3IbSmypY7lwlFJamVhakheQFNVQW2qrKy7qpehwpn4yi4uONQBDQ731wMD3h1ChwCkJECsoKUYQUeNYbn2xatz9oc/Y87LkxSwQkS2G4fPmWrVsmhw9sPrVqB6Zr1FFbkTAAidTH0n7sSpWk3jquLFqTYQy33eKqaLPWIu0cRx2SKhqI4lVTMkVuAICMAUE03iOHzpReEmZSgCutJ6hcUPWM4wybprOObW+lzQNinqc6EbrGWIp5WG7LWp/y0UJmh1cOHDcv/MLv8+tVBioJjNS4uhb/+hvvy/yVRz+41GCWehOzIqnf9eEwbjqYHea6OyA3zZc1OvJ5UZaeUhsodwjG2iz3pCyd2syiahOYRU02q8ApWIp9qvud6LxcQsRMrdbx0lpmRU62WZM97E00rsMFDwtNhhFtaPx7+4OQM7rMABmM5YACegtSjFUOH7HDWfK+qN2ezMBIY/LDWz+p8anC0K5UvcG1x8efUJvORbKg5DNE5cRoVyOf62/UTthvqxQjwBlUM64ggBqNS5mZUhdFrFEFQ9IlykAjL4ClOeIIZEwUQYOqlBtG0x5mAogJtS1M3NCZ6UxrX/Hu//vqgAJzdjopAoB97ESFw+/+zq7fdLG0PX7gtX+4sTRhVgUFQ+1SOzmefvV9H94N8IWbp7Prp2cOD3TYkb0vaw+P292ZGM5YOzKOYxutMboIqFUAZVEQg4vGMmoXrQ0NCJGAGZzeWW9troKqCqSUADAlQhTHFrOuENCIc1I9AJB0DCKU/+OStmAtzns/UABpJAOw2d5Qq0ctgoqANQaBd6wDFtoYOFU7h++VoQVFMk5VIiqCBVYk69JUjvkyYherkxAoicRkOsgsoooGMllReIsWVRFRYjAZMMM8yYpEDABIzmoSMMRsHCTUZvyUHKTIkFHKnQ/CAEfDgL2v/J+vNsAAC6cGACBbuXxx7fDGbz72KZ/8xp/hg4H9+b857eT0AyMEiSnZYn+54P4D71kp8fDPuJk11Wd83ebx7VV/5ZY/fvqtUAVT2nFs4+UQS0aXEhJZoxaP+i2pggCBUUUCFqPzdpNGEgEj4rFLx4L1U3PUPy9FjE0VGGTFImPqiUK3qHIABIYoLkXz8K/Pat932ch5gylJn5KIU+aBrhycM88LVlLyaCM63jppoEK6ZiUTNQu7F7uxsBE2KsJAxGBAwJoQHmhGkWIWZPcOrf2YDQJN2SMrgSRHPs8dgFUA4QRGCJAAMSEokCFVAYMaFRBVFIhYKAS76WNbeu2WHn7d9g3lOd1d2hqMPkt53sdA5kA9APCFVevybxnu/c+v3j24QZp0sV1zW3EjEnJSAFcFHdNo1g2xN3N2eaX74Gv+8/OGs2Lp/M5ZNr2Lh4l3TIa7LonYPEYAa2hOIVzw5xUQwCSFo0WAgECBEARgZvajBY4GVASUrEldjCFGkU2vXQpOHKhhmldJaTAKwMGO37u0L5yyHZcXRe7NzJKzlkOcFeP9h572lKqQzijo1OZm7Dgl5bOlRjgivpumQbQSHdIcflZVIFCG+7OMA5PTQzTM2660ahIAqzhiEiKDAGoFQboAWSvyRE4+WhZjNXQOSJMiSkxoOrOZRKLhpO11P9WvrEzG8tExT5/aedF5C4VFPrTXmmxSXP7iw0/6qmsHh0sZQ1GlJeoQDFl0tj52JWxM41mWqQiJFY/Z13/H18v+qeG5WxHaaYB4mTWFDcscFXtkDKSkpMCMoPaoh+LcP5v3/bSAQvNmIynNZqPIqKAiKggZtWpdZrAZMSSSiJmoTYGMQVUBJWZ0cN+527iwrem6mbiilw0iEIfYHT+Aeqd/PDmniMJSIHTT2CTDcZOS2KPANj+0mfWckJBUQbIYFIFTwVdOZYUWbnmWBKGZrGQA5JlUlBRJQUUEySqiJAZjUxJCVQOoIKAGEEBTnJs4wmiYHNJgkryjFv1sWZvlND29FK6vlsc/NGwWTVdk0bSjXpq1Jzuvp8rnylPk0onpau3Xsiv9oAKKwtDIkrW2Cep8mEXrvQ0nv3/8H+0H3QOfRV1OWYHbqZtOVzWSco4AALIopEQUVATC+T8xLeh3CBiNQXQS8uJiUyZneV60oR2yaNIyh8NRnUFZVZS31kgERAQRnxBiTntvblvTA6v91IXUVqxoi34/81M22eHNy5giooHEBUsdU8iQ0yoDEC1cd7kw8MwSDBCQquSQGECZqp2nAoYiFrVDLLt0Glmx3zgCBGGZF4IRWARRBGEFJF10GJ7zxwiZ0VkVMADAOTBqs3+N1eQQzWi77KW8OOihr9v87yMSEYhIQlKjCHThuglsWTv29tboClim6fWP5idmIIqgEt3ecDAbuAmji4FXek04vLZt/+/d0oSvf0ZT+1mHputRd+XahH0OLqgiKpIighpURiUikQTzrlvz9anKSKKkEUfvoCSOmRCQAKWLiNNZRmmyEjPMh5V1jVfj56ANmda7RNN7Hr2+sFgBi/E9w12AGKpDFh2VbgdPZxYdkk8GA9qUAY5axZEQES3qfrqLazlHIKNzqt08cWcMbx0gRmVotwqKBuK6gLN0aAhIRWAefipaUkVD0iW0KoqoNM+iJUPKYpxNggYloYfAPLn4LJpOVoxEWxTMNRXT4WxJf7lNhgyKiDCRRQKcXnul3F0txssXH7vF8GjmP+2n6zNt1Rog4wxBNyy2i3xrpa1D9JAuUzqrdhl2337F9zE3UPR6NkGopk7VcseeASQJzbtpLHIvc2RBFcEwKKokiVZSchK5OJeDYKc0b4tddJqFul6J0LpADL0xGTaJCmEBMKipsFEP7vro7Z0h7OFMumiNtVqASmQdd9WjN1wLHJMaShFCXvgesDWOCjWLRJ2iQrXWJzLWsCgiaCMiitbLVKkUY1w6zGVqp10p3piwOy9UAwoGVAWAIkE9i8kQAllrULiS6FhUVUQAAzlNKlEPgmZDE24al+dXtOAWKuzyEMvd+77nVn2b8W/8+vDAuz287iWv7//+S//ggfdh2a4fHhRbL/nC74oYM/jUV4xDtjrqe5t8YMQsOpgOtuMg5s2Akl3L9qresb3e9ZtLnHx9z2dk4VEtD+tbZDA5GLYNpEazvADTzBBblq7puqZumhZITKZNBOWubQBj03Xc97uP3Suuri0pM6dUd8biQT1axsvF6Xx9epmGbV8YkFzR6+d2aSUjd/D6H7tVswyKdqqcQhTjvQGlYjB8ysZ917wkqBa9vulwMFzpheNnimFX+EeXk650okhExj36Z88I+QFCG5WMcRmM/+Ld7/+j3/j5e9+3fpwG2uvtfeiTYCPfhZRifdiOgdGSd0Hqw3zZObLzxoAqdt7sWUtUEvBZNy+TEFIlBDT9EDEmSm50WM+G0wKvnNxaGvPyfz135frX37htf/7vXvvIlzy4963DX779e970jdd+6Ze//cxKe2ryyDfc8WJTB8/GfPJf4zGVST4SzWEqHfg8PXCybW3qjYulY+eixYCYQJCTyy/eLHuFwpV6YsukLCQKHMU7AeoQNCVBFFYQdaoCSdUgkSgrkiB6x4nm9eSKSJg3IaTYaGR2wKxFLxGC7wWZ0xG2+m2+/Dd/eeNRX4ycGaxzpgEiRNUas2sNFq0KkHUomLRDdSaOayE6dItUNF/CvhWKqMJsCTTb2t8aDJbN8s4qpdBnmpY1QrduORNOVZdZZVUBRkIQAQsADEkSq6qCgc4KBwvWzBsPg2c1yOqqWBVNzZWvlvB7dRhH0Du9dM0NPzb+z2ap59qvHPzOe77ttX/7gzf9r/otH/yZ/W/7t1/6il9zdT37Nv9NT4UBG8XqVlyRAzxWNUUL2drU3XvMmM2DK9cNDmsXacuuVnXhH2DrwbOEj7zUhOW2zENddoAMDEISxGW1t8yWgs5bv9kEhgOjsqohImElYLDOSU1o5j2cyJAxiBlBKSE5YIF8g1EYPc67X2jPUXrwby8/TVSYFMkigTGEDEgEIgpLpztnMDEQkSSMmAoyLO1MnTlYnYOBLA8VI8OesyiMakEuhK/YJJfSfRdvdRzKWbl1pvBpgqpGjQYtbBRCRGM1t4hgSaWFIHrUfCWgKzBPIHNAgYDBIoPprB8xGZq4xJ+pO8WVU+987N47zN//6nItew//9dO+7s9+7Sfu/Pov/ew3/uU//Oof/crXfNVvvXat58KZu147eIs9uQd+5exotLIlPuUJsjjJDi+eXet4Kzu11La6GSmb7UsJgSNGryV09antnZG22x4NCqqTpKiKrh5noyTcWWGa1xKKxIaMY1ZBVWI1moAU7K7gvDUNEWHEjKRSZztENGjGq4lcDhQXheRlB+nS3UMwykKIRhSQCPQocuxvXVPWNgEgGmQWjYX2Seusl8ipAQABY5QfWykk5mKMzKGDmz8CMctm452mbxRnUS/cloPNZg3VaDQXA0oGwKBVg4BoUaS1wUAmzCKCEqNDAmZLwGjcEbfPeMamndCGCeMvgHMm4+fXvd/6gfcO63L7d//k9E829Vsm9/2w91c+9wf/y/QNN/3Czz5QP7rZgfzuL0+Ou33be8VXf3B3FE6n2Uaz58H43uc0Fy4ePrtPD8zC8cPKZOs4g6oe9yE0xjQPnVsrKmed7fkMFdUlYAVC8ARggNjO3ReVBNhFS1kMLCgyJ0trirJ0ocuSGjKWAAAtcHqIk6vK6By58XGu7FAqj/Oixwpj5s4XZWBWNIbSnPMLpCqIhFifTC4L4Mgisxqec7dZlnKF2AuoomhRt49xsCUgGQBDoBce+oLBuE6r942UMZssdzsudCvH74dsavMWqmAYraoFZGYGY1GRLVtgnVduMDcqlAwAoiFjxACAEeAU9ysdD5ab/mRXV6gBHj38W9+XLp/aXb/1zZ+83v6HvYf+YPzF+Gr76Nn/4P7DB77xkTMDgm7lu8rhYJtsO8ru3ig41psT9OFR23/Tpb/lL7jxPbuPLn36pf5jZmtTKuU2X/n5tcHmcRcm1x4bXCx9+5H1XoY4j2MZDbejHgcAcKoCIAJNBGKTWSNGVUQQQQQ1RTz9aJ0jkrGWVNSoqjy62pPD9a4gtuPjPjI4TiSgCpDzoHvr9MZxCXOOWwdk5my/OVx8hX3bg9YaFUgJrWOWFjBjWdHEWeNARZW7vU+IbJJNaA0igm5EiT1o2ntPITt0ww8VfYOI504MpkhhahIogShaEERDYFUNOSQIC+ghJ5MHRy5Kh0gayKuC1ZQZ3/hS19fy9tFeu71UXNjYK0++9Yvh/A07v/7ipd/7N/rvvvF/wJs+6Udf8AL61tP/5uDOay8e2xqYje4TkbHgA/yh3xlJGfPLu6/5nE+U/KNffSz7hluvfxVxE//HG8ypFTcbyEfCV1z7lTZh9fCjH/yBUfX3n4wP/8a/Wy8JbRdDAYouVcFKZVhdJvPKjQAE5HMNSpYZIBpMTMhor/nb1hmcN8QiBSaTbW+66uDpXUrstvp5MZ4aJVFhIHNhsjb5rf7ybq5ICMwRSQVEPCCpqhye6WxMnaqAASEVijaY5NqdYyo2WkBE0lBPNjV3KQtqLIoA/Pmj/V6Kq+bRO/OcmrL+yPWrzqbpuWfrcku4u2mskoqSUTTOAFgFY5xZ9DQUVcidTxzUhEieUuhZjWglEZpkzUGb7/VPNMPB5bRWZeZ7vvb7vmnjUbv39Mc2OvzJX9p71td+4zu+4P3mJXd91X+9Ze/Y1qB/MH7kBbum14Xy4F29zcOlCDkND86tbr3mDWflI9/3Tc88HPX37/jVa8Kw8YePnPivr/hOG2M57N34ApFeONGVZ8+uEVvTpcaCkmu7sO6S7TovogBoQI0jdSYFMgSqGAlFjbKljXGy1uCC9AuAFg6WxqlaZk7BcoHdgw8fykw4MDi7kXrDg5sunJoAAaiIBSJUkYAGQJjdTZKlRIRJnLNka9QMbUTcPk7oD5daIkSITbdke6YxysbMu2kXphLAeq9f9ni/v7X1wqwqQrRL05NR6WAzN6qoaJKyIjBbiym0vsobFuModkXTzXJ2gLYpAbmPASyGpEbtqIqbdzUD365M2qG70sMsvfY33/28Ak8/8rLzmZz5oT/8kU/e/c4b3/y0H19av71b3VmNMzn74eRjbIvZH799kLLhfgODpb/86Z+s166pf+Kr7nz9U3/9vlue86qv7e3MUM6/9Guef+mSWqtA6rGD6Q2IFYY+gPRnPXcAZaVeWrUSFTslspZgvBytrRASCbIqmMI1NWWD1NxSw2DHl4qIIAyuHk727lh+pCxdf1/bafPIn/9xe+JywYOaHF1Bg7y0lbbKA/LJ+RatN4SAaEIsewfnnzlYnYlRHlGdXCaN6R1M6uQFH7hZcOKF/MzkTXERwGmLwVGL+cF6O/nQZ0zWosbu9PpB8Ov8aLUWHAty5ytsdfopHBUUM3DD7SUAo7amgqvOVcZyCmgzkznyRkUg98TSoqEkCtAx+B4lzVKctII4XZapX7rjhpf+z+c23/Qds69SVvzMz6sK+b07f+LpN7/6xjDNOoz93uFsVHWj7fHr4rJItOXswvN+5dWnLm/9Uf/7Hpp+y2cNPmfpkf95enaiKv7+zzaurwZXGYwxu3ByxT/Q73uVxFk5IYZ2NEsxoRgFNSAsQcWGaK2zkHFSkBgHErMCmkrD1tKFm1YjAeIcvMMkytQMBjrxJuaj1/3eLdnBs+OxD19TxunTO5IuKQhP2tSEYIGreduU5Ea2wZM32HEoB9oxDKg+8D5MAKnLKdGpRVdtQ1Fpv0A0giDc48Ynbq+c3ggHZy67a2pvqPMHN+SAqWuCWhVkzQkVFIkcOUMAaHvbVR+9Oj2c+uVeampAYCISdkw0xDajTgxpNdLZbCk/QcgNIirbiRt24+JV/+c5+foPfsvWf7gOknRWDn7s5f/dvi5b73DsVi7i3/wEtnzioe2vHV/bq4s4deWNw1+42Z5945//3Rd1P7L86OvejZ/xy8enW/b+O2/o741XjsrCQfXcB1VwVJIiRfUrgzLO6kluNLJTiUc5EReilSgSrE9R8iGKxhDA1pvTzYsy6yMAzdsGEGOhOBnY2UNnszRNp2/iB7f39rPikOzaJZ85Z42Bw1EOPgZi44xU4ypVUlWt6T987PQwthBCQO4G+QSbnELMNeF1RgwlShaS0sOrlsiggBbTrhf4cnvlfSu47x44GZeYIe48U1KmBqwbOqYYc5X5iWoOc4uiZPc3ZHJ5M5qJXZHZJCtG3DS1y6xP2FbRxr52My5MwoMw2ijdNREdZCkxQUb1znqc7Byu6J6e/uXvHIUmrk/uXSrh8Je/I9mdpYOPXis3PyWH/t6HfvDcqtnKR6nM7M5/+DErbu0zvryW8jd/5/t//3Pf9iidrh+88fvqVepv7x4JRBCX3Wy3yoGtiSx82QE00DMmKSBIWnTZ6SVfqgKsVBXmLjalqMSQFaXa/F7LYo/Y1mqQjvkUB6n6yHAV+/z5n7MzgUupft9g6Z1Tvn84ysgQemqkaKtsiVuyZcZiIHQx8eWPuJX1a4/b2e5gXfaX1nYQdPcWEZWNeWui6IFR7jmTISoJmATWhnT35O//rOOT//V+l/suFefCcgTBsla24DBBPwq5eWlyQZjUWASejEJVECsgEW1RPNw7OJxNlJrdpDOEpjJ96lwd16YFvyplFEGAeo0K9v1vPus/rle9mz7v5lcND91SW8nvfgU0/+PZz7h4rD+2H3jtu+7rX8Fs58eqpVEcDnlayn7znT9w7Y+MR8d2h/4HD77wDfojP302e+z+177mhEycP3/1VD3k7Vey3bumbzokSFxOmjiEYtKYwB6SsSDCrLqvo65D4kvWaG3Lft1ob5S6KrpTX/Rosz6H/BaH59Sny5aW0AQyeZGn7ExnzpbTL862v+P/e/hdg4FRAO1MFdx06u6u9ne2L28dpmIwGpVW9pb7S7D94OVYHh8PV/SBpT532xmrQM5G2AELEPIjn1igKjHZyvQUmru+7bPU96eX//tXZiFTvW+tZxyDb2vkkMW2WE9sLAFZ0fmr2zLYS6cu+1134Qr7OKl2wuEBD4dZclTYm9bc6dRGX0I3qi/lfuUtLRGPQ0QDnZHV/K9++EPt4Vp74uSn2+0NPrzro7d98PPcY3/2XadTVb3pJ+/cfMO4KqO7+fu+sktQxNbg3mr/ge+8rnGQptXZhz7ZhYded9qapVOvXNlazwEfvcrHycaPncKSJw/kUzUdSoBpE+1gb9XGlBkxiggiALm28bDn+dHMzPbbvKcMyImRV+xt/64fex0IIKiIk/jYenYQe0T9vjPUD23XrqxioOnST7/pD67xlhMS5FCQjcE+nSCOL13Ym3z0/p2dLhvUPQxsVtcu4bm/X7rjmmXMull0kdqeVQTOFFW8mOqkAwATCdn61uxtDTy2s+xXz/Sg68Pk0ecT+NqY1h8rI3VN1idBMmo8m74CAljbXPmr+7reR9AWhXAxcqu3rK6tutSDPbjr5oEPzGggue64mZY33oUGwpQVYZr39if1X31aPN6dv2nrmE4KqItTD7x+8/js/ux554b66A/+yDOLE3sodnv0tF95xx9XcccNIefef/nJGws4HOVflj3watA/uFIcrz74Dae7zf08ra097SpByjVk4cH33LXcqREnM3z4jkl/NAs+RW/A7BpDCkjQDrd3jvU5P6yHJ0e745UMJgfRZ0t/fmr2wqXDsMhPgVCa3e1cF0uZGiutrTS3eb7bnpyNZj/93z5JCAwRgOMMwZQCQCY/9gyAEHnn3g9f/KXs0Xx92cISrzxz590PfK5EOx2J9dP1nNAmVYsMXBYjFDIuEJca0Fzyz5ixKbp33+xT3qpmm5QUrMb1gXW9aaSUKxEqGfZ9BRK19MPvv+nM8fyF1FsdcUW9aLgj4HjQd8tL6wYzNZablkrXX7VaraFyMkajk0b9/ru/v9hrb768GtJQLh+Hgy/7d1LwT33PwHbuo9fdfrYaq62Wju9sXvcJn/fg+99z4fx4xeRr//iFLS/ddeNfvP49M6h/4+aN2SRfcX5vQ0IYX7q6qWePfNLBYP85zzpljYVM9h/86283y92szVNnDJjKkLIA0rR4bPeG1TQAIEMIKcM2SNcc3P9bcIC+Kea1UyDE1cOMTAUfRE1VnXddFrpyvc7b9PKXgxAgAgh1mWECSgQAChC9a89e/7LtN58yh/UjjR0saX/9WXFvuRnWy7HI6vWC1ImAAVVxQy+A1hLG4awtdH/UmTPt7MFGWIeXbHG8KGpA6prBuIveAnYjnGM95ApWQrH87e638kGTNdSfUL+2gaPtUSp7uNE9k6BHqmzcENTkgZo4Q5bWH+YtW+hXfnCpmOAB1Mf2TlbHD9/9la+57iu21h+4g7t8Mgvl2HiGvIFi3AsnTr9o76FHHnrIDz/xDXffjrPXvqTNX/pJ8IH7bp3iLPmWfJQ6w6PqBEjQhaXpR//tzS6zCEjF6hrYfuz3djfCxHXHdKWeDaTN9ez5O5pio00MJFGtRGMydfmBHXAHpe2HvNvx5YE/LNL569zWGU0hywiSOpoOOwpbm/bvv2wtpb1j2pLvfE5VL8ZsutwZmyQzDB6YNk8Wt9S9Ha2v7NWP0ObpQoOeu+X4LMZQ2BAGgb1NVW//mEsgVFszmrFxzb23bMh+sfK+vRN5nG3sNDdLW5Dv0i1SzpYv2+loKA5ZXO5jFsiCkrWe07it7AbaJYZM+rkx1OHwLZvX+6JkMwNFElEriQrAjlKs1KrhYpzjyhf81otPaOsxayvXLJ36hOmlC6ebb1pv8un6H7yg6JwYhDkNQcT0b7mDKp+qP3nkFuh/2YeXuq/8x8lPnRo0xVPe8YevblakLt3etUcC4byjkNp94QRICC67cTm2g4PsGOM66qMrVePLJDa/8uGn+/xQnKqKHDXeBThxTF33yLlHHqEb7zizsXu3O3Z8sJRXlY0ZIPtcP/L+2Xl+2tNPXjvZ/r3/ne3ZpQeXh248oJntVT3Hy21+UGZw9cSc41fqgZyaLZ+YQqdFubbnjeoklUVsnHU2xQTUs03FaB3EFMH3087BvU+brWIz+atbuy5RcuNhj4ymBBbQI4VmXS2AcowpcwZAVSyAXZIBuyxoL9JS5ySih9BeyadkJHGuAEjCDgkIfAncztgwuQSA9UsfmtVU57jy5ldJGN/+nC+6HmZ/8urdnrjH0otHQVrxi4NKWIXVeKCef+rffXalD//8p3Zfcfav33Fj7E0Ozjz87a980RBMeODqUbke4HR2MLxpzRhDZAi0R1lay1ozWbEPPnWYOkcu70MOx87UvThoREEEbTCogMrFUy//zVuKW6958WNX3nlo+fIoTjbe+iW99pRz6jWO/db52265/4O/0b38lf9wf++B97zpA/WrvxuIOgegVY87F4em4f4RK/95/1cYc6AOMlQI7ZDHYTnTFqZ1icKhD9x5LU9GI8xRSHox0O7Z5682vv8P9376pqdc/NZNjkFUAaCDVpr2uG9w3vjGLApkLDgYVZYQTFRw+ZUVS8F41Jf0l5pkiUBFUQHBmdiKWWYIyBAzCb2Zacv/9PCP3b98x/Nu+r+fvpxPt74nYPzVn32++/EHe6feWOz1ZkvVUWP6Qq10xAFrc93PfW/RfcNXkXHtN//xX/xUfaY91bzxueNMUrz4gqsaUrXH4AG83SIhEZHtMJW4s+IQdf/eJW/Lfki5b8PGHWVr8szMGSgUDCga5c2/q7/8q2pXxDud1N5rct2H375dxuOe275Ll6//tGctm5fkcLn/Hb13/Pu/eMmbKT72U98d9vt+77n3Dvb8+u89B/fz3tXzUV78uv70WGvEzLn0YRR2YEgYqDUDia6cZT7pJBWejIJzPvR36/Qnj/6MWbs2/MHZdePQNunKrdJZANTEahrLcjrLUxLlFksVRUKyDNivk8YqB5Z0+f3HrxuGaZ7ZzXrWqUjpAAQWJ8dJCkFtuwIKGAoNxcqVk089oY/+0bu/qNvxWzd0XTb765//2bL6fFjd/Oi1WXX6wf6CuYNtRE1ZltlGD0dMrsnVxN/9wVuf+Ynvrz50+MzPuHPQcoft1aac5qDXTxepmPfFEOnAoNH3Pi+53uHqC87uzVpnqEvRSX9WerNfWMJFVSsYo3rxzpuzuk9NwUm91c4krtsN3HdRmwFCfS1stJonPv6Xf/UHtPLsta/5s7vfeRKKHsibP/SZ9yFffMsnQpZjOjr+4XqfpawbwyYQA7Cd2HFfDskNLoxrCLWDlGJebD+lswJgDEGlo+IlL6ZnDbP3Ni+2dS6hm05RhSEpkAJ1xsJQMSWTZZn3lgQB0UZHo2lO3QywsbZyVZMlwq4aBJdbbIkBUQANhUR5W28TTk6AOs15vFyHcjvL22c859zwT9/3FDMuJ+5XXvujz7h0vFuh8a31/uZWvwhzgWgvggZDNLYH731NDpojjV9795vH8TOfgTiV3kYA7aZNfjUwPDhJZI+H/uIsoC6n1FBT0jgGV+wdHNs9zHykQdY5GLWQ6VyPFy1POHb+kTtyliIBWoAQcy4Pk42rpDbIDJpTAh4hFrtvfPnT2p9865vufsBf+kygzsjm137nz77z8FXvrXu92pojDfGrabXF+4ph6+ZNoOOllTyh9McHYZAFYzVYh/eXAkRA0i3NMtz91Li+EyePrs7KFQ9ixo7RQUzgLKiyIYlBAmfksqzIjAKoWE/gAptqFLUjm91uS/WlbaARFLHaIRlMgsCJjbWoIFdWkmSdm4GvZqWZDWJVrHWdv3AK4KGf2/rVT3tkYJdan53P+4Nu0UMSACfsLUoMy+aBB16IycXJPd979jeag2PjvABd0rHNVa7omSOBNA+POj/uQ6tKRKrORFZ3e9FqGDgHo4d6NllLPiZj60C5WmQ0qAFBCVT05MWpLXhvBaUDsIMOzMGqPJgPHUZKOtmkGfQDRvcPP1CVv/35z976tb/8hGfBpRNc3fn6713DV9672a8RrnaZhTBi6eKF5ShISJrj3s71RYodRNszzdilXm41/MMtQ4sMKGnfe8x6FyysHrx1tLYy24DOToZsQBXROWzRo0m15MTahDY/hiqICpaJQYy1EDxiNxsaiFgnu1Tlvk49C9EbFFEIYEm0WNW0mwU2sTPLO9loamy0ujuCC3CqieU3f973Ll/emOzFk1v5sglbBo+8FMw6SZHIHri9/gZx+OU3rnzHy/RKJiavci9YYJMcDK45moD8cOR0m5YqQGNIlWNwXq/fomGTUrb/gb/7jBtbX4TZyEWKo1pQJao3i95wwv39+28vurDKRh1B0Ey6yyVVN/VDCEXZdTMvFkKf7wp3Zv/79ukyfN/XaVmd2F9aSv/7rz9Ef/bfP5W8h8dPeQtk9gcTl+fiVFgksK50Nbmuixcf/OCDlEYZuVPveX7oupaNxuLkIZ2qVqE4nD105/qsNzWNm5VoK0JrDEhtjDgJ9gCNAdsbxohRjYIFYNclBikwIy5a8B7ZUA15nJ2csZmVjjGDEHLqGal6HU37y83YYaicVAiccj289m76vJkMv+Hnrq+bIhZYTEurUfp86HWQamcZJ9YKqZjBwd9+Qq/NL/7V132GcueKDjobvUDstwO+vMRHm0h66MVdMTk+EyQQUXWQacLQmyl44WMP/9mjX7fRS6Z/4Dy62hhBmjdJUAQRMXjwt68BGtRlQAfgU+v1g3Y0dNpOLbbkKqBgi868/aYL9//03abzYQk5l2VkGP2bz9jZfwao0/kJSPMFQuNjMZ9scKGJzaAoLoWVhL7t2Xf99vLZLzrpt9cudb/2rv9y33vuKTdtnWDyoue/+cP58IbldwxWW59sZnce+IJ+IjLgmzwU9uI1GEOVfOvL0jc3dFhqgMwSgPFkORAmQGsczZvMxOhv+8AKtwbAaEwxsxIopbycveWrProy3PdsC50Gqtce3dv/7wfvSNnv/+IvLjXFbBF3VAOXDs0Gg9tvjmnV85PWY1WlW9zq9+yaF770/5SH05W8m3mYN482YkQOi6uHA9DsVFM88Fyji4q3+bGLAscv5Xff+03Dz6Zvf9Gr/v1DZ5qemVsBY4BUWZBIkpCl/PzbXhy0BIcAIki093cvSbc19TUPjdfzc9OVX5j95pVL1z178oNv+ZN1j7M+HO0/KqY8sQ5PumxzEzZFWqp8VkisZ4PokhqU/fKV168CAJyhm+AVu3/+9JccL7WmAq644Se+utfV977zus3y8pqOaXfFJcmiGDHGmmx9loa7l37t5vzGf3jwJ+L2ip/qAFurANbOieTMZIyd1x5gn+PekkfXkTOxSWgIVLQuptTcfLqKru7HrXy0fc+b7x1fLH7szLjX/fg337Dj9o4muJjqhplM9wrsDztt7nnw3vsOApfLL+dj586++8RXxE4H7UwHaWEUbLIsl0dXBRJ2lwzsroPReYGnzNkhuq/hJ677ln//wp+B//Zdn7uWBgnm9SK8OBacDCmAcY3jbcgY9voZQIc58IPD1UOojj2QUb5fZm8/V6fvetfwM374AxP59J2VJZmXAwEQaELn9UkC0UNGImuctGTJSnXoIxpR32kOFVnBvROT6vinLDuoXE9nxarTwkNYvm73TISB1bG/tJkl9QkhGaHJDGKo3nT375ypjfmSr3nOsU9/3qBJztqEaCAuCtVBmUXRGtS2sJxZjpkgpJb6yVgLXPf6M799Eg/82rSNd739/Ttbzzq+bm+xy+076BVblLJFGxkkhxUb3IwyLce/8ptTgHxgkOTNl46/8abhp567QZvicRI/AKYs8KXbrs7DbAxDzdbYLOqk5pJCYIvHb7rptZ9/229/w5/zcGxVFAwCakJVJCsgaAgF8HTRZbZZhU6KIjDSOdof2VPn8gN67J7z7XO/8V16zTuX92brcPHf/J9Xb/ij9r0GIKZIT24btX8wG/ltW/iuw6KwMJ70jFHxmSszoALqcoOH3fap6LgLpGmWWcSxGbTLFltzbOaKbOdphkHFJQJxI456OX3ut5+eju45+z/vv/OtxfZ6AYBW0HqK4gMoGqPRCFhC4UHYlt974Q2DxElnVekZDaANa9Mr5Q0zP8DdD/3JO+N119/w5pspvftdz656Tz3zHd+2zlm36BgWC90aLG9hVtHfv/69+WbhoAlZv/jIyyYX3nry/tt/dDSLUMRZdtScSYi63c2rAmm5wBQGYmHOTSNVBSDIJsvf8h3f88rf+rnx53zHB+9Yg1oFQAlQRZXQCANa5ZjLdfe9BD2OSyONRd45OWuGaWvVProT33Wnff5X7t71HwfR9cPhrf3tD973vdccaZogOisCTxbI3sBoVa+34rLCciq7ldw2Id/V0oKH1lc5wPp434EbKOASSGudS+Cqg7WBpIQDmWZqVSjbWUbBpt1bX1l69vVKcM3Bxun4iq4DSGCtQcw8KSRAa+2cuq/M3AxOhheeyZkZRIyznjQhxmPDx5p45rHw8JvunzzjJMdcd4sT/hfvv+7FB+3b/2ORT3JYNAJrsZd3WIQP/to7aERy6Imt76bm4m91D/zvbul3X136WZMeP5WABOrJ8av5kNQbTLNxPu8/RkikooqA0Qbzzq//4S879UM//FNvfM65E/MjyRQdCygDgYohYE7upjf85wA0YkHjtj/8kfe8d/UP94v3gOz3vzffviaBX47ru4NeGTfrr/+WP/iWOaMeDCsYMh9znOD4gCZL+UzIWgcxJV8fY+DOFit5B2AsZjEP01XF7tAWFBN6p1C2k37WP9n0a+bBrlqx3CmfW0miuOnT1vk1xcHljVPTxk37QwjBGmsVTZ4lBSJSJUJAEEDTYdx5xjT2RNVZnR+eq6brNZeHcf+RP/n9zWvuwBYKWrv01MlSeNa5H7vn9vEffUn0Ot9CgH1a2utOXvzDX3nwjGshmqTWYYrlh2n51Gp/7cFgVBCuWga1SabdxlWvph0CmBXvFqdYMyy6KNHoDx75w7v+4/e/+Jlf9ne/cs0zlInm7WeASBgWx76g7bLr28kgItd5Du3bf/yVy+mF+Z+sfNaF17/2D9rndBcnuyIbo90Tpz562+0f2tz/6kUD9PmZWwpPPLUVAAA+PKiD2V1JQ0RpgOx4lk8z7+QgsuYJ7JVN6bIl2PNuhKDWNWjqtsjz2t9o1GGEdDHPgCNC9/AzOaZYRxlde6sc5Mcn2rPQg5BcH2pLidV5FkCSFJwzoEpkKLX21sMMDABaTymwNQraaupu3H7BK5uXqQ1ARuLpDz1l33fH/+7XX2Nv/NEvISuLIzj84GHKBg8+94S9jZteHnwAT5H96mP3v+dH4Jq/me1e25eOV2uYixCp04YHVzWmLg5GdJM1rAAKDPMTVkS7125/6Ut/6O3f8rmv/m33mj++KQvWKACZpMZCVGJSZUAnMlr5+xcZ0nj/7/zlU1/+zY/xF7+8fc83bj3w7auf8Dvx8/72yjS++e/eAl+69Mbh8R+/9LsZzXv4IpOFq23mnnC98zqzzB+8bsTOaUiC05gFN3C8VJUeOOXHZj5TgjWAFI0HyJPtAUiXb/VnG40LJLtlgSn0QzzIMLGPm5PJuOnWQoDRYT8hWS+RSgviTQHM2ss5KGqL3mkgDz0zEaCGiZAFSPNJL8i0WYO/+f2X2SHEGK1Xjp6uLPUu7r79FaPdP2/B5w3MGZDF9lpw0/cOPEUorVJEFHQWrhx/z8+lG9q1R2rXBFeGoyOBABAfO3VM5rEEoF8reLLqkBeN0VA5T7Y4TF/82O/9/k++6I6VmXfP+uVZvrI72h4MKhIiQQOgzGgJVCk862/upAjnfvVeCn+y4ceD324nPyWTX9zik99/3cVzIb7+a15At2zJjn3t799zKysokIW2iBZb46N6IUhEnbGN8Rfvuaaue2tFeZA5EeAokg2yKtm8TF0BOUDvqO7TmAVsqwCYh+2eNMFa6u59riuvrLexzhoLZaybRpd6Xh04Hc2rn9GCWjRAxdgKtsHmtqtLUAHjslY0qbGgCCoAGENicusrfPiPz12rs2ABScHYJTldXMn+oX9c3oOfvbuWLXqoY7My8dvb/+mWmUElUkZVFRTOdt5cD1vlgSEm1Pl5CfMGv7x9zMnREc2Xp864Uag8zs+ratZCx10osnTjdcMfe/HL2v75089I5/vbQ3K9bmU7J0RVBUYHoGgy2M8vZCD26d+Nl5t3n9t56D1r0426Wh3ceu725d3dr1n9T299ZnG4lKajlP/bz3sNWwQAgRJEsYiN9Fpf5/sbkMXozZWDh24tQ9BTM+wYRNAlQlVOigENPNm+PeHic0VqMHI44F4OmQAkZYNURG/UOPtk99omCDafmpRxEGvIgAgbgzjvsUNqFAQQwRuIqCMnjbkJKPjaWATj3FY16y31/vYp2T3nPgfOXTv189bmWjZ8ePy/TCtjUBBErMzZO9bfBUN2re2YcX500zwIiq79yM2Q3Pz4FbjYNdkjG0Ue5uNECuPB8YOJXrntfX/1odt+6aYz7/z9R3/7BW95jqmCnYZ2owJQVVG2BCIExujSPeMcZnzo+8/4ZK5+4f0v3kuTFazSQ7Prd9d63/dfn6XTEenqYdEd+8Jf+WJGJCIIxoGic1qnnGjj0gmwdQ/Kd+Ky58lkPdrQWVLrK0MAisozRrHyMU7A0eXvXUW1jLirfRty9awWkQhQUi0fYxnBKgmUyYOxmqou66d5CpodgEHSZGVe92oIAslI7OQaPz12CZwxqIR6w217x1fKj9585a1fWNbvfOHVHtFTW45+/U/uAGEQJJlXbKJgtZEkkTT1flSEeYMCAABQ2z78UjjqawrtDbZ9x4nGEaACEC2DTxcPe/1rzl+462duxeB/+j/1f+NVX/9oL28HYWm2P48jAHR+VD1HomN2avd+NRymw2vc9Lq/9rvlxfTYVFePnT5R3nns9KeNj2FnQ5Zxuf1VpRVGBBGTXAgeyZnmIzdP5UTiDCQN/+gW06RpFZ3t2JnEMrOWDDgNQSBmT17nj1/6oTuEqMvsYZ5zUwo5cagcJSVtEdOTlcsaMqbfc5laY7rk5nldA5EJkUDY4pyvlUTA6GaGh1Ll5WCyAqj6/3P3ngGXXlXZ8Fprl7ud9vTp6ZlUSCNA6BBAivQiggjSBV4VRRR5pQkoIk2KiAiIihTpXQi9BEggDUhPps9TT73b3nut78d5npk4wzAhMwMf7/Vnyin3Ofc6e6+1V7kuLpjM1SfzcRODHz3CqbmP/N7s+G0Zom7y9betrxUIj2cJxhNButLJMG9WZtQFRFjl9REQsGbY2wRjriUAvuH+kbr6QoYxyyZg7Y2tNtLVk4XubTuzTNzZn3jlpvTCbz68u7hh4uYZCAKokEQLkwIJUZ0GfK/52htbE1k/Wtnxue2dXfGX09lTi4H72c/ohPD7U3lfZSUO68m6o5wXUAoRaKCM1sCXvuMn2XMer1c6eWTY/vCW+ytSesMo00CaOHCZRkqJYWgZXR1ohdtg51VbtXG1rZenUxDgwKVCES+gItVIfs4KAQA1MdLegTFBKg2IhL4Co2GVQByQEAU12DBNYEOW7poIqzqz0thwM87PJV/YMjtyO8/4+zeNV4gA9L/9V2ZmUaUESIpkBMiegNqjBBSCoUoLMch45xf0up5Xk57HCyzw7pMQdnaqqBAZ8/GKFLX58Ssfe/Hxb7nsP559+uwfRQ/9fP+vH3n+v9DCH59XDjMGRYSsmAEpoPEop5k/f5EFgNium77gJXkpf5OEUQsAIKiPnc+dn77oC3ftvPGMvqiGKITg0JDUGnJX2pf/+VPaH73+L4oJClr4jZtOVuDhQjXSgIGDcyGLERiFNsVIcPDWs4ZrlrzSgXBp+ayYEs+iODLKUwUYqWZ80NKiAAI8GQglgFKELvgQ2FcsAMxoxrSdHJCsibDhWWeRGxVZHST4gLbomNHMZHfrSSuNvjU33rjKfC+73YcWQg8y58a+QxAlBNBJqMjUuh97ECIIa9TvjKPF6c6qvA2w08dDMRoOR6ufE4c6JtBnPPNt1/rsLqf/6bM+3LT/7FqTD3nkd1o//ZeryqkQQmARCALsnVBXt9afckXfBAC2oYYQZes3JLVqgRvlDsNXj5Pi+ZdcvPm6312yprHy7a9dVynjSsaFPSNIsuyN69Zdt/3hagNbBrjiByeT7gnNETmQqmSEyhCHAFKvU7CvyPBzcG2WEJJ1/WIjhtgra8c+hBAUJfagjAARKMA2aqUVFwXECAioCI3RKDKWYhIOofZAmlKSFVnsr+vVUWyxykvv2/W2zt7/3pzrKg7thaXVAX7YdO2VG1q2uVLWntnXJSlNIKgqozDkuNOG4IHGFX8ERGu5nO0YC+ML8rZ+UTV1HBERIhFNQJfTEdzttX/630X2wHc+9LUv+NlmA51T4ayr77LjzVfnIGPRSReA68pLU3YXs8O6Cuw8SNAVmFE5clCNTBYbH24tI3rE6Y3yAaaIde+lz3nwiz+7gpbgTX/4+397GVB56tYnvv9L5y3ySJHwdyazHbdctn2+5xoC4hwYWwkHFqIiDQDhwNu6H7s7qbgAOUOnrlQwiryvyrKyRhkV64OWFqmAzkCSREo4bpnCGGvBQdpMoyQiZlAK0BhK6k49ckawJBWWFdqiRo9R2LRwyraFz92/M9t04YrpmU/YBKMQR5D/bJca6XSqWmmSKBWy5VGqgPJcecakBBqijAJpUmN5YSyqvVvBiVOVB6Bqs0nCisTAgcf0N0FJJbvWn/fSH//p1zh55H8sPPmrBXzp9Q9ev06fvvsfvx/itvbiEx1HJACwojPu3LzDWifGYwLKIakMwUZlUTtlHNLSs57ZaU2fMue626962J2ufe61YWSv+8Kbp827tmmn8c+fEo3mqKGc3/u+dNtlt5y18rNvf/U7P1js50kDSz+VadL97vY25Spypa8d+9KhDzUOEByXNUqRh+taS9BvS9i5QXu1S7uKdk23S52lPIpH7IoCMbii3GcYvZqb9Z5FAQKOeZ9QakRCYq+BxpUGKT2PojIRIDJAojVra/2NpySjr55xsm/pcrjk28u7tlZ5AsP24FuY2bIEz56VII6ccSwsIsKAwYsCYlyT3Qaok/ZyE1RNY8ns3XduBd+IyioGABKU4WgmWok773zogzZ87v3vOf+Rd37da1/4/c//1QlbCg/p+m1/+/Tz1rW4iHsgZAgRmkWZtOJtdx5FGhA8q5zF1AgOjAIAOGU0v7l+5gOePfnqADP/c6+nzv3zwnvvZuHy+Y98+CHzo5BVnOydg8pHuY3edN+HhFNhrj/sf3/hW19qndrsMG7gIiSqsI3MxH1oGAhOIlMvdVTl41DFeYoVpTDa2QqNSAq9c26iHiWaU1me2j1dm+607bqzTBGXTie2HDTWDAJCCEp7EEBCNdZURKQgqA0KOEYBlpAQJqxPzFiElGXM2YUAwc8u6wddexwlFUUZxhuuzXbazrbm7FJ946RXQVNNnjUTDj0AASnSoAEYveZV7a3VQLkyvS1WghKioOG6tmDlh2JwlUNjsuoDYfSjvQ8474LrvuvLHY2Xf+wJ8+aeP+uxqeeSm79yw8NPVh27gCKoxLvUMbe3bAOlQ2UMQNUKUT8m0MQVKxKnNnYjPu2b4EEvnFvfZ5TPnpnbYsqn2adPNKrfKt2MW56zVdkcXPmZqDZFLNN0ZnjK5Z+++ocn3ndw0rrc1ItzC9edElexHblEx+C9nQpgva2BGs5bxYuL+TmNvolm4nZryL2GalTUPyMardu1ZdhzE3P5rItjVwSzb1JJgyCwsoUGRli1DBGREiFARau7nASq0RebAyGSphBiF7S1StfLU+bk5vKuTKuGgon+9WeGMFsv4ddhtqgTnbNlwYDQF0QtSKRBgwSslbO3FfONltLqZPREnsgD/GyyirtlHSc8djG4d0r62G/9tVp622V3ecK6Xnt4vQzO+9lVtWsmg3xqZvunrnvMb432TFbiGUJVCJo6PnMBI6h9DAGoXpzKkAZNufnSa1dWFjYo7M9miyZxupqZ2fz7FPZsXErlwQ9eedYsA+hhwwPNuaWp2Q+8/Fq6+QRvgHuRT47b+OhhY++bbrn0bveiauLK6098w9vOK/F+D6n++7P5qWdsmWzdOrPzG+WDpl9SnP2w+JKr7WDFLW/zp607P56Yu3X711emb4ndqPfj4ssTmdQzeRqFQhJbjdLbGARFWzEiwDAemAdUpEGYZUyiAkgaKRCV1kVAhBLYCiuqKF2ZnNxjS8msDxAGinaeWy5pWy78I1nkuCwkAqCKZJQSkPCY/F441MZHQriWTERBP5oAtCBKAHDHSQpWMLWVXeUqS/0AW3rXCXu3POvZtljKBuK/vn5nutiwgbOi34ru9vXR5o1zCx0ILMJcGSKz4fvCYGKsHVnZ0I+udV/+UVzDnZ64eTT3mpdtAJjOORs0/cKsQDm5rEe2Ktq4a+MgdMraLnXC/MaFm9/0VlNuklrQpmMhjsbu9X/59I/cq8x8f/Inn3lWlMLVH//Y7PF/eFIDlCk2w9x9Pvqu7rlPOA/K+678aNOPdj/u7p2No/X9eio3e+H7X/rO9luyR5x/VmGXv/Cz7ffS62cS8Crav0IYkJUBBCTkoBERlVaEKMxAyGPRW6wp6FTZOCAp4IC7YwVciETNYZvr4UnzDsxU7JuLuVI1/uw9V5/sUx9KI0aEvGJKIiBBDIREiACxEJDsy+6W7XwoZQyMIMQ4bCidM2sIY7UVEd+G+XSwwlrV2CwrXu7+4R9MNoxk9agTVdIaPf1jb31RsyHjHY7IhQkw149StpBTBpCX/i2f4RMuuu9W6EdR1v/RVy7MnKEipl6YUHubkEhaGjDd9sZRoiUeqhZvP3Hb8u+868J+q49NqRwoRCg71Qyoj4FPnZvsHf+sdu3grPU3LJ29GaoIStnTavnffmJV28CcN+9/lyvDXaNRFtetwold7x768PnPyBtnCBqQPBs/84lUZaddOAPd9j6DAKBoI4A6Uh5BA6BCBAcACjHESAii9DwqnbmRBFBKROimk6dQ6RqGE0PDWbQzALqJzkL63ad53/7S27+3IXOFaLGeAkMQMDG5QNoWAEiCYjLSbnXLQgAQ3aVJARACVgFhhqBCD6YeszBB3C3jdrs9mO2HpizVKc1M733Zv968YTKJKx+nUsPui2982T8c55RSSEQBSq7t4t5TKnDVBOyIrvzgrfotWyuNuYsjKNK/ft5bz9G9Zrw8CcDFlOJe7GJnZDJPMyiSfqOO4cSb/vHd61baK0GnoziOISgoOk6wbjiNS1MK2pCyH06DvesogxECcKoDQQq+Oeg1Ug7cvIhVHUNgm4AGBURTz7t/o9KYm6pFj3hEDjU2AdfsMU4SCxAI6lhVQuNMEqMX1IYkDL0oCKgbrCHBkhmIWJCunZ5kikLVdmSk1w6hHq3koVSmn44++Z5bT8wc9qPYVEHCuOIjtScTaceehL23sRoTuMO4gBIVe9ItMTMoZi1EZ+hR1cpCAauUfb2W83jdDC21eAUm3aLPDN8bPnndRH+ixdqUTewsnnn9J59f23EzhM9GZZnx4ineoIGFT3988KRnXCDKGGciHlHavfOrX/2gpzbJT5acBtRexREEAJXbQGXiEheFfPCoP3/1ZX//oQ/X8cIMeA0EnBTWgGJT0nQYdHpp4abyfqPh0hFkUEm6NCkiMIy7WbMuDVV1AyrLZSJQRZSrKKdUtgKUKtXOgw0xKQben5zUogNqNlmeQomWcyMynjgJaVQhjawWAAqFCWZYyYjjpDfoUNCdInIgRHU8XQ5lSGBrleXcuWkob/i3jScHEtfyA+sjcanJCue0UhIUpH3th1t6Meh41PFiKgBCYHb+p6dHdUQelSpds7ZAC9MOQ8bCgiCJY6wmBWwhJnSDbg/r2dGFG69411w2mljqkJC3K3jJnhelVZLJfMuMtJKZ9TffnXzgL386vKfZxuC9EQ1AGkDDAy74z6c94nFWYmAhh0EFUhRQOzKxD3E1anzoXz95/Mp9LvjG0x72wE5tEQR2bQhYK4PgIxBqSwuaIDMAoCEFAQsyCRasZNAGMQAQRwBG0DKKFUhAkjG9bQQgTQAhYQMAal9FTI+JnW3mnSgjYmms6hYicQhBUi0CEryKfBxUmDbBaxQWTCofSqXMiJs2zinrFa1yAdbFN/3jDbvOxZ1ThKTAc5WZolS1YGQUIRtcbo6SqR03bRmhEEiQVRFaRKo1K2AtzsYU+l7XI4UavQgRrYoEoAAjjHmxrPcUn7Lp3j/5yOzejiubpBPSIXth9by775097TrPEysZ+saPv/yV1tMuvg+WBoHGoSQAADRd1Xjq8/b+ef/0i86asuMaGQtLZDRKOSze/dOtD3nqU+L51pAfct/kOx+on/DAbgtnhlnspQjJL6iAHBk0AiJj3C4cR9qxGpOvek6xDtq5tXQkEwLByhSEioBBIBt48i4JU5IvqxS6UQnrL6sLrz4WNvabmwYWgFB8mRGKF/K+CjZSxC2lzfzuZjvqaADUq3J5KEDDOEQQSEBQiTXku9qviXqND0djndTxUI7TwliERjr5AlzvWtAfBtEU56de/jZ4+N6qMhPFxGjd3/NfP+JNJ0SVRA0JOCZCHKMmUVE+91rz/e984cQ7bYw6q6rJoVhZ3L1t+9IzfudMX8GedZU1HPxFF37lo+k9BzqVpcy1TK33HeQOh0OnHX/+4xoRgclkYrQCAEYZa38WRmqFwaOMk8beY5CFhqci1Iwoje2YYYlOMCKUYign+B8uehOrjajIh6kcWIDAM2trPedFbjDWypPyK714i+5MI9dg1GrqGkm6pwgCAxlhb9drcAsTAVGtkhIKIiCLjJnvRBICm4xyMf6x31g+4/rZIMycZD6+78/e/KDZOh6tqNpf+enROzY5YIidYl6drRoj7xjRvsr0fe9762Wf35HHOmo000jllU82nP/4qdr2tIpplDn2jdzJXR58j7v9Qz7wHQ0s3ka30x6/NPQ4d6hMbSkETU4jExLpykilLeC4+UZhAAYpVG1HddEwELIiYEbKh6BMMbLrZOn71/+26Z9wwg9wIi6rnCF4QkKNWgliTUppozTs2dDPjweFxlTESoex/AQCSncaISAwBUJoOlUsbCY1bssay1ShIAoFDgxIwZU6sZKZ0R+Gr8briQ1CYAujhY7/56etzE7ckO54T/zsh+XgLAoEQUIQXFtw0Amxn282Vwyq4457DFQ5i4gEnkCtCUJVW8xGVTLoq2hkXKbj5c+9/sF/f2qz27FQ1an+Bff0CA0yVrdDQGA0IgEQWIhaOgQRvcqAjuTRiwiIDEPNLIHyYUg9RRDlK5K5m8zXu0/xreaeeGFzshcavZhQhIhS5XxA8kmmrTjkuRXVicTkHn3GBEHxqk4F9GcIvIbAQduKCbmbScBVmWdy4wCZWEQQFeZIWHiEQRL/SeOTPMFRxVQB6I2Lm/4pfeRe3vzihRc+eGqQeAAvEDtFILfpJqn7OLmBqyQKoULwPgUkDQDCwRGhiQYRLE/oJgBElPKgOVm/5rPPfcxjjwflberrg0tLPx9rFzxUTfHAxzUwEhFgu67YgKDXSliIZEWn4giRWVAQvQkI1mFVIEgIIBJ64IPtpmxI93be+kef6TY7t2y8ZEt7OSuW5oY0ZvCy4MVrkjjVFBxAunNzNqBm38agPVHA8WEdKeRTGnwEohwINjVAnooXI8C4KmyHq2NrAgF0MORQFTO9KfPo8uvDDcZT6p3mBafOeevMnfkld3lpaUMaqlSB8S7G/31ThtOwlHE5CZoA0UIQEQfMiiwAO58JwWzEoRdl1rkolWUaPGzuky/8MDcq0OH22uOXhh4LCAFOzdeKfBBUFpwHuKE7cxIVShuEgEhIGkgStoNakfiAJs2G5NG0d4f2cNv6331o48of3f+W5NbZH0ICkRkBaS0AwEDIRoONjdQM6vshDg23rNIohNqshXsoKC4jCeCJoOZoXU2+ijRoEhZmEEBBQUS3pirXY6JOVA4oX9j8kqe+eyUUOq2GBhWr1v3+6NRXfKIbMrewDln5KoMEmAFpfy/xZNdMEMReCMcyAuPGL1FcCygbla1aIiY/6bxhjVBOYrFywYmTj7/bM9sl92YPXUU/QoN4qpNRk8lUzbgSFTnvgWzw0XxSm3TkdPACASBxbO0uw82osHkU1SlD1iRcbsR2h5n9wivK3vr/6W0qE245Nhw5AWbPGCt0ZFUQDVJyY1ezTlEY8nTXSscTIVcw/tmjjmarqJUbM8pKAdZuNNveM+siRcFzXdHObXlYGibNvVVt1KLrcwlNK3rjSldPZd09N99pMl/ODDnPZe/mP/jDO7kGBDUrGIlKRQngWPhhDdIGWP3P23TxEAARAIBEYkBQlIw7oSACiWOZkH954etfGYUpryvUQHBQk8Iabu9WdZBBNChA76VlPWWQ51J7iqzlqQubdRVRow+kCZj3tFs+N7S3pRxyEy1nkUcrzZ4ss7vyBB197zNS5On2zAAicDDjhjcBVAqRpeCaRtRaOlFIHGXxzoWmIxG17wP7mrWgIaFamfocrneufFAmp1sh73b73aGPAq2boZVb27vvV207J2om4hhs65ZsbvNMdcMN13/89M241JbGSvOG0WOefu5qEe/gFsQjROsdj/z7v5QyDoLAovb1Ve3r4j/C99deeyKtoXZh6LVuDokQgg+SBT+p9uYNIgRBbMXbq612NFfz0KvBwHpNGiCH1m6refFluPSRn04MqhY4qBVzYBSWwBiA9FjxQ7nYaN71+G5feRtJfyVm5BoRZJWORIMKqJl1mUF/q46iU0/q1TeN4qxxXMtC1mi3KY829m789+NP4sm90MCy9rZ9lg+aZtbdbfu6/+qt7+hqdwsW/vOcVr9x5Pfm56Iy//EU/WIFoojQCx3ti+gyAkyh8qBjO6hLhzYi8c7HwGFHrqb7EConANOD5oyvtB/sWuidVAaa2eArNqa1UE+l3zvnt/Mvfnm6mj9+NNnTeaqQdEBHQXCsVglIFgnauJMuvPWbDdI6r3coIGY9zuUCeLXFgJAIkQ1UNvvuxr1PzCZv7WhCpdirVt237eG10xue0yn0Xhtq55Wqu404MVj70H4qv33Lyoa9U9fv+osLar0qlnD0QUvr//0Ff/y3dSJOtHb7trsDr3U4H3Oox3UDiqLBlEHoW8pIrShyeV7z/LLA7kFYXDZSOiDYanab528wYI3ddS5FdZH2YnAlytx2ufIfoxv/K6zr7oJBwzb6WjOiFxFEJEUoDKicj6osX7rP3dtfinWkKr66H5MQjSNRAWfOsQBYa3JRQVE9Azvq0qsJpUCCsJeCGr4XnbHkz16eUfOuiWRjqSGOIh3YKlc8aP6Lp22fvqX732e2ip4cI3sAri/SD/7hJx4fdFlmuH9s4XA+4/ZCl0gRdfP0B5fu7i7WbE6KszhKsnjrorQFd3/2WY2EgyIZwhbJrqV+Or1yc66S3RNzixYqHhUN2faCR2z/u5+enW/6iZKhzRjYCyhQChAUEYkgkvOtuuo3nnZy1bcJKzt7Qz9ZpdwZb1n9O2kRYAUMTJAs62su2kKD40ZKkbYY7CiLv/eZmd+Z7dfBrswEl7SYAkcjAUIOgJOd1z7l/YPP/Nan1psy4eyg/pqjhGBG0/B3/LrnzKSl4P4s7dGKujQYKnd85HLeuDB3uk3TJgEgEuFK5n2tNj9yqDAETaJ4JO1y0FquhqVLdcvI19sRr6eTl+a3PRgmr5mIhib2VKWxgAijFjU+dJICFEEValMMNl9gpgvWgdXGbb05knFEAwKgd28W9EYJK9Byy3mN+fkHxpaphcLg2DnjuxseZ5PEV63GMAzQGBQGmqmdQ7S23NEO7Sc+4z6vXS9l3DOAR+0W/W/Ee+dGMgk3fP7+myIIfh+xwNoPYM1Ch3PyB366fdledBrX3/fi0+srhhRIKDBojbWL0n7cWA4nLmc6dwFBQaeo2s3FToxzSTWIoeGt6ure9yfwQbPLjfnTt08tJcNY2VgZTaLGMkQcABUCBzH1yoQr2+JSlTScN3r74vEWDNWrn0QvnsYqgGYfeVv/9Fzup00pmt2MUIhQDNYw2SxzgVa9bKrEiSNL3lcOIsNcL2zqpY3tz3iVrW09PwuBAI+JRXpzK6aRR695ajaRAgMdGGUdKbQG0HAngLgvdeQ11JO9hFBNkbRD2PTT49cxtxABvF6Zoaafdsaum2/YELlI7Fluxn738U9Pp7rkp0RbifXAYtmGQEF7RsNlrSRijmS0Pq3j+gxdpZEsTYdast1zy4ktk0oi5SpWc6UKhgNbT1guiYx2r4RY2sikoWbiwFChZR/VkE9UECkIhVGBvPIFGGwsNt31//DNEtHTFAPBMfIhpj9RVSnMvPUFw993vemVtkMrDo0WZiBCp1ACaATvYMx8wSKIWGuNLEheIzMK25JsAHLaKR1QqrWj/77WOR8hp8ayKlJMTa1hqKP+tV/b+NjxuUiUUElYS1nNnnrrOVgVqrrplGvKqpXNtO2SmxFgkxZTkBilMIgW0oKklCDXSGwqpbisJ6c8dkYzjpwJi/24b2VAylUBbVUaAEKxtQaObqmj2rWqxLESLxBAaK3ReKBgIm9iudpCPCRho0MZsiXzt49ZmhtLqR7988ca0sWak2Fkz3zzY+98p+m9K5+7pjhtXZkeN2F5/qZbu35k4rQ5MdnMi9EwH45KmtqyaVJ7tySj3bvLRufEydnJloZQX3nFcvO0M6ZIGIJP0v25rFVgw4nRxFIkDd/3cQBfpvc6fcP8WINDrItqlQINl8zMt05vV7590q57dU29kv/kst+Kdw+mINhoZSPGkdUgoHJkZAkSUJzBOhqBh+BarPiU720MSNHkdaiaEifsPaAv1DAeS8e4yDPsBOipGdBBwfgcT4EZEEWiIfvGrceteR9oq7zgUDtTuv97xvMaNx53jFbGGrrTeTokM7Snv+KpT7nUDI+/6OS91/fLL7RIQnTcvU6GUHQXFnffUlCUTkZKbH/hG91SCNNmOkNEO2/ur+SgcXn9GXfqfu9D83Nz97h7k4ewv1FuFSqudKgQi6lhPdigb27ZsBhitaJWJ5iIVWFjpWIRJ9vbakmmdnQnknquvOoHF5nlpsFAeucJzRCnJBqYGRAEVCXGk69jKEJio6ky9nf7apE4i9n1N50ioa5JmDQR9ZLAejzAa26xibu1kxqvAMdneQyrg28LE53lvT+ZnF1dIVLYIOw4lp3/M3xD0jvJH6t4dxXXbJ5Z7tBKo995wjf+qqpdourygTEMzZhfnlZsayOAjKW3QbhvYwXAwYAgiHNa6jJojY0SjfOhXKbppPLRPqKC/XG0cUZ5b2av/ICzN++aoEGY2yDKErKray8ioa594PZMx598/ZKeUqfM/KQuq7pB85p2TzNQqBe7PFIJitG1tTayNmnltTXCwciQQZLZBOUeU0tSUoU7vlHl1lPczFTAyCxZLwgkFhgvP5HCtVswGkERgJRSyEppRaT0HN44mLxodhFFAJihqnRTF8PBF/7y9z/fVMfu/LGGk16wY3KlmOApZd9144481Tm09fxyI4qIjNEy1dS+dqIQODCzTGRQFXnhakE3zE1GcWt23VRmBa3N2hNTp54yq03DurX3J1xDgzCIQDj9DJL10xWRHwwLXSsiEAHSsSgokUvbUluLWxSVzU3zI5OqWhaTzi2EYBCpoMIjAnNQhAgUZfUIlZC2aW3dEBJF+tQLBoX3sq7X70+4NK37tTGhpL3CCgOCZoTLT2W17fQoEvQ+CLNIYAAAERjZZla37cS4jsPSAIBKsPfhl97J5t0t/jZFwWOC9K7/4iZVP5K8XEiePOkgw0GYnSxrrxT7OlRV6Vk4IClCUrp2YJM0S8sKTCPzpQGfj7xNITIwWPGRhnowCPv75/dXvjqRqxQ5p8+JzWg57qukJKMXJpA0gCK0VQSVVj4nnZ1406lxhJtu3DEZBZjduVRd06yCImqMaqlcBAUY9p4DKQODSQwRqMm9EKLZzKPBiy/ttYNuZisrC7vsVA6NGFRid3vR4DV5iPLdDwp+98UxxEIizKBWReqFxNbkI1MWdqygDOUwUnU2uOEhj8zTUacI2TE6f6wheu6Td83ZZCSxjmHL+57CpTQBgjLCPqA2nggAnB9TKCJY4ABE0Aq5ioDqymoNAEH3OG0g1MTG3jYC2WcQl5jSd3QN+ezCrPatMM9pvNCeGxKJSMBalRYqxFYoNJ72w5VJGCSd7acgDmDkcAcrr73S/V5zVCdSQhyEgZE0FpURg5yOarsum9NVnJ+TjNa7sHvjNx/z1nJ3K2257YtzW/+26wGAWTkFZZjAwXwERVRrRAlISApRgpCqQ6svI2XWJrVQJVCWP/nP7whIyNMAx+j8sc8gdOI3nyKDDlXeNZ/wzqeFFEIAbYRZR8C1Fg+IWgmtDuMBEUiQoBIIgokXZEEEbCis2Wpy3nJQa0yC+w1iOC7tsMkU99ISohWMMQxspccjo4DGQSvXw07fJ1JmF3z/YjFy5s9yl1Tne4jP+1qDuNFbf/VJFalyZTi7xFHXTwxl7j7Xr2wZqcVgW4vYu/gVEnadIQsnFoPF3qb73XL8G7ot35u5aTiV2N8OAtu21E5Vkbcn1HuNh0aImRkRGTEgigiLqCr1xAoVKECggnN23R9MOKCQjVMZx2TP2nfyLqcGECIBiFXxyP/83EOGjWDrvOGcHUVRnlBdtkbaNWStJXMsFgsKxg7bQrXrxk1bHalx45bEIHSbAax9BglgVJDaRwqRgxe1qgaKOC5F12CLKiXOFuW7rXjhpp1bt3Wa6y498/sn3Tr1ab9zRx1NXj957xTOmqQf8OaV2ixNnNLisHD/F/3RtsnF+Oa0ObjoIZtd0T8fzD2fdH79hxedPHvjpW99ak3r3QayqJ58v5desGW+kWA0L6GaXkozrhBri+Ne0n0/e/SVyih4QBQOmBZqd3LJZz+9dt+OrQcB4GzyR1e0TsiLqWHDFM98/z3ai1MuNErI4JtfXm6d8nsxL8yE2h2qC6Krr3nTU1/0ggcf6v33vY51YksI7LXCwDAWtbhNgY0pzl1DycJZ72hnjhf+4JILFxdGiblX+9y9W9a96pY0/nF8ynDHyid63Qedt/k06ydGRjubefveF7W++omLSIqvL345e9Rplqqt8tGTz7nhu187PZn5ypsf/oK4AbkMOt9ZfOcX3nr+MNs1uaDSsDA3QUEhACDd1lH/+zlnxXllKAihhABDuzD7LrVt6VhHV2vweu59tCt8f/PFjz5TwQObz379ulGjrEkN9N3uS/O3fMbO731hq+kP9foO3OUDA/OB4eMP8fh+Q2KiUYEOgIQKBIBIAiEAAgmi9gbQMtmBrYNKp976Z5ddd8YTH3nff98Lx39z4T/fMRxsNRPRbPTfr+g8438G/Yc8lRrpcBr3JvHxn/NP/95d7NX/PKMv7P3pa+45ZILj3kFnfWahM3Xl53TLKnaYYjxqvHTmub/7J9XGHdvmtO/OxIDCStamemGchbzXjKVEiS+FBBgwWzRXfOvz1DpW9Y8DYcsdy3sb/IC9n/jYifd9ZnG3zg8et7MRjSYgIyhU86StreUfJQN76Mn1nVln8vzN3UM9fJuVpSUQ6CBeK8WBQY1bBXE8qaDLoENVNaE/2gLo3WnveckFLzx5Zfn7E98dtbedTJFJ8p6KPn+irXb4c594/cv/Ke6n/f7mAhLX/8+/e/XvXzO9/aGf/8EJ9a3H9aL+XeQFcpqeXzr9uFeeS1iqCOKlqUo996JXPmpLvekzMUQrneC0FyP75qxFRFDOKEcBiMgEYQBUo5mv/9eLN19/SjjG9li7wYPoiq1z1KPpe169558u23qfB3/1IccXdSrogzWSJCVO3cXwQYwZ+xA2eigmk82Heny/O2H0HrxoEhaREEJgZl6dGxcIXJIMnEjT4mKvavnG+05507bJien+A/74ocM31GL3mnakPndiD+OLzas+X49CBu11o6Q3ima6L7nHy54xesElp+Nr7nrcchsa8yf+6dvvpb8t173jTL0EWsvATYmtqrM//KLLqiW+zyisTGpWY8cNIDyWImXmPaNoajKJsjjS4xneW74RPyM5pTzW5481NPnH0fJuXoDri6mzi6888RWf+mA3b0fLy0RKu8EoXl60HFHvUK8Pi3o3QJUd6vF9K4RAAVBwFkECgAIWP9ZwEGBBDspFXGpWmYJJBAdZ+ZwfP/ehz7r0uP/Gkx7Z9ulwzpdx+cyv/8255lunPeYzg0SBCsOJsg0rE9HonvbUa971gVe89nSBrNuBVvxwfZG7x+b5WXENAaQGLjadjoN/wkM+eBG7CBZOzAq0Y+4yEBAQZBbBiXJQNOJyuSUgwsLGfvc/DAz0sap/HIjyv3zLNgFnfjYBWUg6g/Qf/v2C+x2/PtNuvmMShLaSXBqdQ30aM720XnZtWJg+xOP7DYJaKeLgiICBtHgWEaQA47yUyxwIRzAwsagq8lAkcs6n3/qM4z+74ZptL1fR0lQZ6zI+9c7Fv3780d2/u+9d06DKeAJsoBZT4h9wHzYPP/O43BjpwDBZmixCZ1NvdpQtz+UWwdnpUYOHmD129iWvvfKe6eJiZAcuGkdZ495eERGQfqdTlEGlZYCxfrt+/P0HabM+1uePNdz4CjOS+IaZWxq1CaWZbYUtyz/6Lz7zzs/Y0rSFiyqFo2bt6vQQrx82O85MljOHbA9aq3RJlWzfDU57RksioEg8aGRGRBAB5TIcXv20Wj/rlVu6dXsUD5a7TqfRTOR6kLpuOyCHOA45JOtGNyVzFaNCslFVDmoThyJbhJaubrnqJCkKnUG5d1TE6XBomiVW35IdDTNter5Jo9OX9WPP38z5k/7gOG2HzVEkq62VzKQxeLDkPAFHUDgdQr/8z9dvGST6kP1RRwtB59qnO//m8tNCf2KY+Wi+HVyrljxVZILTcv022rzxPs/DgMSeLBN4DQCFRRfB8iSPmnXecWXTo7AFp/DnFzX3O3XFhhWj8RI4yPjJIjxmxUVgszgZ+WtPv+ynf1FADPnN649r+VENu4fUVv3agCMQg0VDsx10pyanvE5CdxhmhyWp4Plk06j26o3ty+JIqZVB76ytwx365HLvVDljHphOpomNoKhVYqo+tTTM5yknvYaPWcYksEoQAUh7FkYAGaSIeezTH13eLqNDM1scNThF1hbb5+/L19tlSQpqNGHgtUSGay866Z195x23XP+z7VtPOmUy1jKMS/INJkhgcRq6kzU1d052ds02PREEr/XPIa373wYR5SJWjCRhjX0QxzMyCICAVHcqV2+BO72uWa9rjtRoehq4Ittd1hv1kutQJeCx6iVpkQ9kJm45k8JKz20aSkq5jxrOGADJk3EfexhMgdfgcx2pKlrsqFEItuGJ6jheDKVdTiNkhCoNgIRjuVoUQO0DAiGDRo76UvFXj28PG8Ed6wUCiBjEnPeXT123sY4a0naxU1DFlTHBe1E0yVXrHvW1X/jI+jvd7fwT0wawBagjZ1emueqw6pmNC8ncfDNiQABBxp9/iL3NCqkiFAYWUqSEGJGQBMM4fGGcmKeWKfOJe0JtMGn0dK0xBklmEKQFuialUENL21BXkvkk4krShE2DIkG2husAvio9Kk2AKuuVStVRh2lXB6alaoCvtK7z1OlpqHPZ1d+yHHtbaQFGYRFAYCCNIqQQOCtdYip19ZXvriPxYI61A3GxVuLsBXf7xua8irw0VDeOTKMPhJG4UHZnp3qSnqyuLr7wyXOe+IA58rUaxrEdNfwo43LblVfmw9+723pBKDhRq93Kv8ggAhJpD8JMiogxwDjyD6QIhJkWV86+ajCJIkWCgL4tLihiH8UQguaKkL2PDI7qOk6cL+ZNsXegsM41hiAA15f94aBXQMyBQWu6adaG9kyq/LrtZmQnZ60xOm1F0h/sohUx/sHREidlnTEHGS9UFGaFxKAUsCoDOQvu66dcnHccmJ/z1Y4uEiljRUN+y4O/e0ZMrd06qiBFQAIGDXU15btFEodwj8Vk4toXTz/oYZtnpmq13HIZXH7LpT+Zu8t5x//l1+8+jPNm5AjCKrvxLzQIqXgFgzAEYUFhYCAIY2FGAA6Ny9J4z1VYcbPol8Plcjio0ZogSFw6yRxXhbauy1Uzcarys81gJpLQi8ixJp7NJtYdH1V4Qt1drrXihROzEdLeHfFek547Wey5ydXD+SXVHm1M1y8Zn99wgs5uOT4ICYqMR3bGFPBIigLQKDPDIFff+n8xBc14yNzR0YIWqiNKlH/WK5bm+nnk0pb3oQpxXYdIAFzIMh2E+rae33zczq9/Jp575sXtMOpc97yJ37/738cy/8a5J+tG+OoFGw2zGHeIY9O+KMsbZ67fpjV50QSoAkkQBWMmAQQo9Vde/+IHnVTNBQpQRZoJIDAgCaHUjBwIJDKVQG6N2HqUxhiY2QKKc4wl5YVbqdrbIz9w0F/k2FeGe8M6QKOeWMwzbUDi6emQofbtdZL/rP/gRljcuiIsON5IQQApgDLimBzyqO6/7eS3cOw15PpYr5HCaqjEg0pe+S+bVqKJJMNRZHMblJeIJDJVBqCHte3YGhIoGnUhyz9sPOxO5063qtfulsvveeZxd0l99JpHn8MCEtXjae+DsH+FiALrLZME0UqZVVOhFmFChbZ+SP2vD1RTI6TRlM81uVoImX0kBSTap8GDL+u+jnLPEBU6r2W0UupduZPhQk415g6WB43ZqO6bSehOpnXVzEZ9fVx7+lN2eEJi+rv95MKtuwYcbLc1lEc8yo46x89rwdWDoSAhAiOCBKF4r0uk/4O/knhQT0GVHmsfkjpfqRhGZs/DPzqZTNx8Pahi4/p+WieKAgN7FVVl2tK9UWmsq8u4aoWZnff+a/nspTt++3EnZys9vvQ1D36oZ4UelTv0OMLaX2ytpKmt94QhEJerNLBjpmoWTPdM/ja/4qU35jiaXynrypCdibwebPZeoFhYzqTf9x5jWQwTvrPkVFmwOmHdLXy23jLcc05bGatJKdUOSOzBtMYiDNCzkvzk+PttKQWYBTE4iI2r/LDj4iRnKZUWAa3WaIMMghclPJSO3Pyph51lpQEChzwYHzWMElKBMcLJnee/Adqj3fny7luv696oR1G3NbUylbbAh9j3fBu6Zjopd84O4kvql5G5+LHhVSfesufcTv6I9vsfGropLM7U2a71hzMIIKCyOOYaVcTAjIoQ9HjKQ/KTlxdP//rLovWh4qzBIycL8+Jb5nuO40aCPFPq2dnUhy0LWzojR4KBYzsT/SSZLtdV562HvjYELOgAiFCoHP/iJap59j6bpYxEGEgRs4pN8AJKIxCKBiFCYZDx3skcWBAxGlI//tlzpo61IdaQkHgWlL1T29ZNqLzdAQCoRsX1dP3esGfbYPvOOdNohjubgtu8O8RmNJq69c/BXbW3d9nlj7jhI3+31Gyd/KKleIF7Jh4ONxyy+33f3xABjQcmEGAREkFFwmNvKqLyAWx6kkzGmiswGIpObc1yjdWoJKuNNWEUUqjEyqbA0qpmNLvcR605CYWZU2WmFbILnHlWJCJGAAlFjA+0sel9FrxnRKrHpVrMnPdGS8AgoBWHVd4sUEEYUFE3XpKr5MJfTb4EACiI0oCwcdetBEVSk4BSkW50srtGFe/9i5eGm7926+KgzKY7tZr0Q+TGTn1x3cY92x/4uyef/Wj45kcveAJEO3Bm4jL7NXX/Q82573PqGACHN/SJFQpqZAIkQuHxZCpLcyUe7EjSlcRiVaGiuR3foTu3nbOFNyqoCKnyir3m0Bw0GlaqIu90aMd3HiLo0ASIkYADc+yD0hQ4ASAEYdKhqiFqAPvAAgLKKGGkwGAiCk54PPKzukAQhJkBYEXiq79d/Zs7UOfjWIFZEQirG096ylPvl7eriBkAhJUeJitT8qcb/8gHe/W33rfE0ydkRTSzSBb0P+VJTaYwX/yvx91tyr2fnhHesvQafMRHv33W3GFzWQysip07I1TAqDB4rcEzEcJ4jRRJOsxbBaKrMbK+zKcHEg/1xLxXiRY0wY2wY2sIaZ87etf6MqqFdTpUmBYhdkaYRQDEBSajOOhVjhNS7BueVQGoCJmV0cCClCtjSHwQQhBQqydxhAoJ2bvg7VL8iredccxP6Le5VRCcKPua+TcRjGKPSIQA/VZXNWotL/y72OPyRs+gFr704atk05kLX981uHHhDKDtX9mx8Yr/2ranf+opr3zk+b3UwLBRHeqHtM8gjsT4peszJvFAwLU1UgelYbUFz/uAVvI2Bs8AwVPUs80yUBBAAhCS3FsKhuJ+U3ZefWY7lUbKYUWxDWRr0N6Htaqe1hBYxuV6JIYMGMdjuz4Eo0QUgVeGBIkCAXtRClcrVSUpZFd72tXkT7w9RL+aKggAoHjRJB/6y4+eG6g0BgBEWEKMgsNq6iOPiMoYckg5aBx+/RvXXbb9n54DZVz6xjCD5bjODPbatS0g7gFPHnqCat/fiEHFQo4ljCWGGFARjgc3EXw/acPALmljyLFJBpzauk70khY2idSFUSKBonKYRdw6ZcI0vZt3nekRB0Oewrg3FwQVCWkcE9UBiBDQ0CgTCxAiiiIQJBLgGpEMkggjIYyL1AgGhAOjKlQxcZJdNr+qJRK0sCEYPOeDd+IiyxwjIoAGW0Z+ONHI1yvIOcEEPKDo33pYt7rqbsuT8a2b4h7waIqzhZkqHsWDZq+cAKjdodLz+1ZI0EHx8IetioAp0pAjIq4e1QERiqQum0XDYR20oeBTV9oQMucicDpx/UbVx7bKXbM/vYJJstzUwyhzLs9UHfthgo4ZFYoES4xGiRcWRAD03HJpWRkWZYh9pSCgwWCBBQFBh0BaA1QI42JycD4AYuE789G5k4dc+kcbzrKzuLxNnZiGUTNUDQAQQRT0GpwJN2xdlPInzkvMYcPp2dBYqZI8DVIZCw6D0l4vTYGUGJdVpn8ODf8BBmFVo+neNERDDFqhD6w0BU/CiASgao9EWg+FhbRWWAaSYNLcKHYOVeVFKwHkwEiKqBYTcaWS4Mhw7jUCEgKLckFZxYEEiCR4Ow6DhXh46V1hohuNl4iSICgBKBEWIaJKhBSh6FDUkldR1sFL7rf12NdBDgccNEOtXPb914NrnnpquiuGld1dSs498Z5MS5M91eyreNBZbhKJCFJ3ooixtk7qRImrUyZyRXN56uBxBEAYF3IFIYhySCSMWjsf0GhVgtXogxhmQUUkwIiE0lckXpQmQq2YQVMQBBHjRZDEm7Eenx1z8ZOUQRiUsBqTJxDJODUTADsW0dK4hg4oQIAwbixGkLEYFWIlAXRsY6yDn9Bw7PPuh4FoYG0Epv5o6xQoqKxjS/XexR9+8k2PfMTU7vWu34K6szATOJBCDhNV6dLcOtMog8LGiBuuau3cOFwrst82J4dAGlVALY4oaIXAQCwiwEEqNChulQyeUYgZFUJICESj1jWiQRCx7D0zsQkcFLEXFi5KhXZ8ZzETQUUoI0QBwP1Dk1qvsxiMIAKIBBIiCqCYBQkleEQISOhRUCtMamFMwf0KSlO/GM4CG8Bw63kNKiIVeQYBOxWfM2rsftI9nh8Chbd/eOLCxnFC5JZ3LOoNd7mv6gCrL7w3i92TLsqcMYsbd+4rse83CI8VEDCgQWAwhCAsPgAalDpoV9TEgBqAAAVCCIAkwQKIIIEO4/kIIvGMACTAEkAUMhhRukYY06AhQOAxCZkgkqwmqrTSU4ROjZn5ZZUSXsbaCMjMBMJjRuGAClUE0qwG6pjnsA4HVAIgQlfd39VMzBChKyFO88ZS84N/89Q/upjf8N73nZ068KwjAD/62Dtf8VUxjddf9oqpJH/ZFU9tM04sbdzfqrpvfDggSrF3t64wpQoiFRiAnQNUGoLzE3mtLIqY1SxlWTllFK6StgirwCiMWoHPHSlBJAqsjPYCEAQqpQiAUIvwWJFwXIkcdyMDE1Ya0BkZuxQJQgaCEAiQgnFRWYAQVO2sQquQrj1xs/4FUiq/IqAjDKQe/mGKAXyZsgBBCCyNxWn41p/8xaPu8qlNddUEcB6IcETNT7/+RQ989fzfRxkF9+KNt4SHPUBa+7pQ9hvEE3m3uB08xSqIjpwTlNrp1fNCXLO14rwlQhSW0gfUmhAAtUEOEhiYlQnKj5zWwkqBF2NCEFISfDCaAJAiZgYRBhARVIoEkVDEi+gxbxYQjUNbDWG1j3TclIWIIKidtxqNDthbfJAqjxm12+0EsrcsqvvYTxrljAZmJoXex06HAnHpSXd5+uxktGzExgDAoni+cfNrPv/U1xmLEEQ+v3np0huffed9OvK32bIUCmoQhUya2QURAYUUAqNSWIlS4seD8hw8B1QCgsQsgooZgBAA0aMIoAImECQIxgAQMBeKQERh7T0QhqBFgihCBiERYA5WlFCRsgTBcbMeChAQcWAkCQENslf7dFjr6NsXtY8VP8DthgCBi8JVJ415MZ3SIA6Nhj5MUQbVi957pl6pJvNYas8UqW1bfOPENz/id8soN6Fq179lu2fv/NG5bq2e878KbYKIQgRAUPO4+cfWSGOeKWXQe2URCcYVYRRApIbzaJSEEojQ41jlixRoCaAQAihgBkRFwKsNC0gE7IEDIKAX5HHYa2oMVGaro1HAY6fPiMhBoWYmpT0j0bh8SMyX7mkf60LhYcGkpI7Kn5ymQq0sMLBCK0FoqhpFZeg8+tE80M3FSR7Ls4ctK7O9eN1jhtHidCntxU4oTcdsjfprBtmfXHQawS/sqgJaBaB4xI2odlTXBsq0UUAZ4hg8W+FAWoIeVjrWREWqBwpGTQUQPBPZXjV3U7y+x6VAiCK2JGFcyRKlAaDyaA17qYCDji0idVszS71YCykQE4fCG+Mqb5hJK2AjIZBVHHDUGkzrRZaWcxm4EC+s/O1PP/BoUVwlAMe8hHsorEyuNFS38bD3d1IJ6siTz/u/h2H0tY3BC4IA52iqlUEwEx6IKPgMg+Jxrk/pCLkyqIgB4yCgbAgIzEJKc9PajcN+5bwxxhqXhSoIitdGEAlCpH1g8N7ERhgJepzxtmQdFqCMOF8QmMho5WtrlStGVQhVyQo5/eGm5XJUz1VFY3ZdfVrU+59vZO+57p1nnN6ntGbf+HU593RxenF68pK5RgQiAuFIz0X7tWdjJq6g9hhFWFe10VjWZcW21tZaKwMhAhMpBO+dZy8MzEBS2knsGZsNUEQ4OLbl7qYPejFEEQbv6hDEotROQyAjdQlVIRSqupmEIhB52hCqlX6jZCBxTsD5UA0GlSCXw9zJgiYVJVpGjavXN3dPfXXj+qXrO7tO2LX3/m+Zmj1t92vea+OaY7jdPLpHG2a6bOfJ+8/IFAgchY7vfVtWHZWRz/XNtfNMSquhMCgZDtbVAVUSgQQhAhBD45O2MAsDYrSSAHXCSCGKq2tfYb47G1I9X/hQCrk+AXMo8tiiRwuVmFBCAoWPdVWKMaEuy5OG19/LOCcgwpvJRgZFpiSwirN42UY20uJaWpfVXPf4lenlbPjS/hMfelKK3cbgZeGf9k6Col9b9FsP22Iu/T+fmiEJ47maI8P+sFeGJpRh2zjP7QN5hxH2V6bZeWXTBntWsRXvSOogHMSVNQOHbw4y5A6W4Kt8OBzVOU0M0kZr2XOU2kYmM1lmtUFZNOTRoFcJeZ1hpZXUgTTWEor1t379jPsFL4qELQIAMCv2IsKBtUZF7Koo6P4mi7rf/843nvV0J7av0kKyP5x7qYEiWZo80jtxB7EwO2xUT7vohW6crAtHmjrYb5DCaqh271GKhENVVYAkwYUl6PeKETcWWGyrgb5Prshd7bSvPGoJG5rrRou6MTEUMHEa63bVGtZNWwAnMWqq0Y77UxSCCCIFA060uEiBAAUnLuuHrG6OlCJF7A0AGA2hK2SsYp+jsImMVLHCzbfQyVf+16UXPeBR0IRRBvNJsxj8ziUrZJY3/bqWCPZ44oonfm7L6vk4HGk70v4C1U24vGvnVaOVqjeidjvarJJMY5TqWChxoZkGF7ThWgyNPbgSRkuSg65MNNLkPSit0NUKPAw3rkgSg6LKaAqiNZaOyTuKRYILLJAoAgBfc9Su+9aUKjKICgIBozGharNnAZBYvEdNQuw1x/yc15zVyIISZq5awDRsVA96wvPKdPumI7wRdxQ7NsP3XvyVfsMCgoj8Ip3i24V9Tv2Dl+tu3Q6naK0oaVqIK4ipxrhIQCeDbmUQgvcSgMaFCjTKeU9gDDiLXqM2RBJ8U4Umb7t1aoNTyisirQhRSZjIg3I5pT6oGFBHQWFg5IDZYpgu1OaBJQ/GYA5EsXWhb7RmRhxqMlqLx9HULeu++XdX+4lBwOWJgFHUa9LidB6/7+mdJ998yBGxY43Ni9OfeFQ1Pa6fCR2xM9u3Qq7c1KTerd/fVKzql+vKKY0AlNV6kBWhrbDOLVVBEQogYQgBiCgC0lYTMggaVZeNfmLqvf/2bDXdT8NgqocOEwtAbtzgxcxo0ItSgUmThFordqw0xde31nUj5UWIAMY66oJQt249rhpVzUZvtPiWwdsuUrjalr+KCqMfvuOuT4vKeIQp+HBgKuVYbWWSN2qXeQ2ufPDbzxk1AozL00caZu0zyEpbSe/6azZU3onSBI5BE4CKIFi2AfM0zjGMm8C9kBoLFhAKCypNCCigKFRlRw/D7PXre8SWPNQNQQUBlecxVXgIqCCwil0NShPUijgIUbrty9P36tjhSBkCADLBOwZFtt8esNL9KFxyyaN+D+bCuHC4/wv0LM7/VftVzbLJBYb0wC3jWBkEe4lB6FnQ7v3PIlrqHB173DbKIp//ZO9cKAo2kUJCrSWg9iVSxhaTUW/pJKQSFInjsSTrakDGqAiQIABJsFBRrpPl1jBSmgYmQhLv0QgHICIsBRWykPZBlALwsMp1YHY1p4eKk+AZSdiEEIAURfMmGVKSzL/i7s883vYjhQC3jfZXJufb0eBdN7/NDSelivMDa9XHyiAOjCepsir64nmZT4OBozMGvL9imCe1r1JldcxKK0QhBV5IR6bUTgcplrZNSxwaAGjHOgnCImIRCVEgaAgBCcwiTiW700hFqKVQsUMa9x6Ox2klAAlqxBq1IvZ+rOvBTGY6RkRLrmJrwFWAhkRCOQGLTc6u+NDf3BuqntIHfu32aDawfspVN5+Y9rNQ/4pGPwF8CpWKY1HyuQcPI1BwlMayb5sCoiKnQaQiYAmkHGslHLwyZZBQ0NRUa24pSrwPZNC71YY1DKCIABgVsqDCPElHrQ5EZTy0xeJ6JAmoFTIjIcB4ZBGQgFARIKIOiIQcIM9dDATLaQcDe45k9TzUyFv5xo98/BX3GEHDqH7jgG+usnJxOkkaL/j9e0nI3K+sYGWhNhHIIPn0JXWj0oNsLPpxxLjNBBWYEce+iDVyEDIEyhLDEEJsDOpCoqZADei9IjUWtUUkrJEQVgluiAhQ6QG5XhqsSPBaUIRJcQAY1zUMCYuwWJEgQHb19qJLN2BZef3O08+a0WQtBy+EWiU7O5//yOue++IhpwQjaR94w7tNv6lyyzPvSF89+N0t7Vu3HPk9uX03zkFU143WBz9zVd9y3wiM54CPmlMPAdSPFyYqNoY4COhAVoMgD/1MJKUKNYeUMYw1c1n8qm91gEQADAqCEAoWEcclNwdxTr6egIDiUVPAsbiVoOIACkMkPJaiFwEEEVugMom4b23eFNXO+zQE0Apk7wm7PvW8M5yiKiqqRjio6wdHWTd2zV4LFwbv7L8pPbBAcswa6QKqLjVv/N2Jc/4mxCsTfvViR+2kzlCpb+Wz5Wr1Q4iMAS9kdg62Rr7XKMGakgyWSokXVDUhAjObcQsiCAmjggBMuhS70qqpNqkDNFizVlgzCxFBUOLRIAsRcAhixtzvKD6DmlwdGfHK+tyyEHHtNnz8P78b6VFEVCkN7iCCAOQqCapUBTV2bbjq3U8758AveIQ36JDgGjjb+4bG71737t/+PdvNVi91pAfD/SvEJQvfmjIkIIA0rgKxAEVLMh0gW0xlLBalx62M4z4ERPQKA2gIQlpBcMFWklA1ng9iASRGDYFstWq4MUklAhEyA5Eftweh10pcQHTaoPegcov1+u3R3g+Z/3O3BdPhcMD540AIuTpzl3xq66M3Q54OrQ6gUcK+X+zRMgyPu2WQl6dXJuC7r/ywpZveuPySs2yRUlm29hVE7uj1btPkoG++qmFAAwIhrpExQlR51XMzIuNGEBQQAQBgGE++BwUBFYSgDQkg1ZXEOrCqVg0SUBupw9jFAPPaNqsRmYVo3PFA6AhDYKXSvFCJrspcqRk1/5UP/d4LWrfOpEXEB5w/DsQgU5XEPLr0k+v/SJMBZ/wIGmofBfjRMkggHNc067SA4XNfcidJ9nb++41//KTaN0cZ9NfKAEduEPRXzhsjdnWmEAFQIUiuXKIk5Wp1RQQZj4EGIho3UAEDAZMm9qxNUUmkBbXDsUE8Gi0uAJPSin1Y07kZfy/EIDAefADxDEYXzKiVhGqpI1PvvPpNx892TbYik/6A88eBYJFQS2JlcNk/f3BPK4UaLDhv115y1AyCBMAsg6ld0/LvNz93E+ydNHDtH535D3nkaZjuK8Xewfe/zdBn/3KAmFdXCBIzIDC7pLSJhzoatx6OtyoBECQkAPBjEU4hRPaCJE6UYlCAJCLjOj0AqREpjeKCXiXf51VuMgcIRIQCzIxa9SIDDCIewva/unxilO2BdSNurhyOEAsricHXUHeGcN3ll3WeOdcAr6FeCyOP2pYF4/kM2rEl/1FxznTfki26E7Z44O8+rdFr95pHeL3bGGTPNWmIWYMIKiLFLMIsdad0LA2NzCyy2pEz3igRidABIAggCygNHIiBMMh4+kMABUmACGtABAmsVg3iAZEUYhi/DdaAIILgmTHS5Ujvvenv33bxdBjoRl20obSH+ZJOE4gQQG3zlPde+U+P/K0pjqBec7JHrcWRBYhAnJOr/uaz0G/t2tANU3k6Cq+86c/PaMuBKgm/LG6zZd2807hIzKpT9wjCAlgkNSplepPMLAIYZNW7iwCRQsegkJGIRRup69U+qtUGREBnDIgEHwkDylgWWQAgAJBSiDWM9dVrJARmrpDFxly6a1/xf58Po5TzZkDqtw534GMFzhNK1G+Vkk/B8ON7HnjOKJ9ZI8Q9agYJjIqEFXzyB8+aQQ/NpbbuKmhW4fJ/OOHF02vXOWKDoL++rNyYoApRoNaKPSud1IUlGl19Vxn7Yw8sDGuKtIguoEZGpUQQg3N6XIjCNbvU1ghD8BZkfFIHAECAIEJKAazqpWMYa4kxWwpI9ejL//NvZ+6cY+tr7SCDlc5hvggH0ATgKpV4DBag/vrH+FH3iQ+i8D5CMAOROFe8s/qTaVhp5w2olBaUQjE97aSXrmWbj3yFlNfabrDklVEogcEqroM2UAdVJea6E0EAkNCPW0FlPNEl4gMaCqRzGg/3aGDPpHDNh3hNATSxQySUwKuxEnpmUgiySjoKfkyVzFFdYOxvufHT/3JG3nGk8nSY4XKcHI6gDJ0YcWiwtMSinJQt+OZHRme/8Ahv0IFwgIp8Wb31gXcNERbaVBF4Dblvsef4L/545givd5sS7s9K6xKvEIWBsKRxn2ilkR1YW4+pTyAgEoAAjacLwGtkUSAqAHINOhL2LESKw7gvVIMHDR5gHAwoEUEi9DieLzSkEAGx1OA9QAC7OFmvbH3B+5spBqQ70OgkMP5hFL4FV37j2gc+KK4iqAwxYslpUOINQKAAChgPW1DaJ9gybEOeYG2kO9lPqmc+7qINvkpI1h4vsiLpdgZ/8K7JXrvbgINTCrcT+7tORjfW2qcMq+evWikUIKoVCYtS5fjp45ZFEYmYAVAkGGIgCYaBghPFa0LCIojAohQEUBB4bViRxluTRmFlINS01oLKHhQy9TNz1dd2fr5INAfEO/BLG/fT+xiCI66b+P0PXfWg52X9lgxtBOLBFFEw42E6kdttEKpAaaidAbs8eeV/PsfOmpo07+vDqiGflFFj+c0/eLedybM7PEi0zyBVb4dHTrEUUIqYWSkJoFU91isVJ+NBcRyvFFGrM4JBkwCKV4AYPFNQmgARSwRC4UA0PtjzqqEZEYBZImAmQ1KtzokEEEZiF5V2x8dX3rWZCcLq9MIvCxEAKdDo4KwJxDz6l6tfuUWAoO/bObcFi9hZZnV7eBrXnsGaHXgVS1i+5Z+u/bNHIRQYj3syxo+rvjLDOF354nu+oJy5w6N2+1fI/CIHifWIRWvy3igKAbSpWVABM65uNeMtQYAFUBGCI2BU4mX8w0cmRYS0r2QLqyNRq3MgoJkFlCKC4NAoERRhIPKIQFyOmuXo7RN/d1yeyLiWewfS6QIgoEqIQUpJB9zuRf1Xb37A2aAUVFHtdWFjDEGUBuHDrcB9BqGgEaSrsvd85DHPNKMIWKk6JGuPV3EYtWBUTwz/ov3HcSx3tPtkv0FuGUqNRtcsSoP3KQEHMDp4D4pE9LgvDse3FcZ8ToTAEAIaCIGQWQT82HEo4CAgIIRIwCI83tshEhYgRQDBi9XAMJYNFgBBKYu094OPve/cWoMIqTtY8BMAQJTAqBH6rQKSrrvm0uV73kvXE10q5wbNEWkAJJRwuJ7gfee0AHqQpMvfe/nvPM1mvQlnACA4s8+gowYAOCO72/d+93lwh8ck9md7f1JTRaiDCCoIIQFgQFLovRABKBZe5ZcWWG15IQRR4oPS4FGBZ0RRIiyoFATPgAIGUEEQ8GtbFhEJe1YkAawhhyI8bvsNSOwG86/8+0exZQyogPkOZ0/zCCuOtAxbuxvN+bi1MvGDd9HT715yWtsiGWZMyIJ8WOe7dn9UWbWhlJ/829POUoiwbRM5n0DY//p+poZMZIx87u0f8u0jPqnjj4KpNauwmqFQzEgEQGN2vdWdSkCJsABCGOubCwGzUqGOlDhWJAaAAyCKMCCIaCENnpFX56VKpQiCC1pLAGPJ4djBsMIgEHz514/9s+1bYNhgIPCi7ugXY0JBgDJCWLDtwLpG/tn/bJ/cepf1kNV1I48UM6Icbstauz/dqZG54VNL+iUdT+TEBqkkw7BvSLIiQUXQy0Kevfgtd7zXeH/Ye6lEznoaT8MoCp7RkIR9A0wAAOPoaszyIYBECB4BFdZlprhmrcQjCAcBOx6DZmQy4JjWop/x0K0AavGiDHpCYBEokghcXVRv/VjsNY8ayEAS7mifE4JHZM/aQpdaPs9Uz5JF7+MiueGfR4tnn39hGjGrwzfjrt2fxW//293+WOctgDyCKsobyAQl7+upKONC2VKRKhJXv/A9d1ilYT9v7w99Giw7pTjomEtnMECkxAEwo9aM4gXCOMnIIiYIAqPS8cBQnQzapVckQJqrWidQBlCRYkZZPdXniAji2cWRIqxLQ8iokUH7YJiN95xysXD9Yyfa6I5B9/S+92NUfunWPV9tnXXOpIlsYakiTTUpAs9IQRR7q0WoUBASpqGKq93f/NKOU95QzcLudYe7Dqsqqh7+5/eEhGvhTJgkaORRqgA8k8Jl06g4ptrCykSh7IHfc/8KucwlbMURMGjFTo0LkoqrWrSS4LPIV2BNb+zYRaMyXNYSqwF5N21HARQKGWEfyEAIQpaYMaxahAVAOHCILBH5SrQC1MKiPWsJer4dieqO/vIbiQUGOOxW8sti7YuPyW+Eix9/7dKFDWetv1Oq5sy0AwyMSpFUlgAkBIi8Bqdr1wjf+tziiSffR9b3IqqjwwUBQYuj9+x6SVT4FgEwCoGvjPUu6Bik7k+qUKUwyvbOUL/lDorGbmMQnwSLYz5+ZlGCCphMKaiVOJcQCEJd0epepWoPqGy0Ar4BAwgxInpW48lZRiVMGlgIYJy296tOKGhFSOJro5GUhGACm+C1jXY3q1s/mr0fRBDhmDE0IAALEtQW/HL35p/8uNq92dw5nVkvpj3VVEEsV7F4C7A46q6E4puMmy48M/FaV2QEu+3DvL+wLuzy732k2Y9M7TjqdSIPWvJsVVIgWhm025zrOHDR6h7MhPe/ZwwBUBgQQ1CFVoG0YmI3Yp00fe0pJi5jDoxaq1K0lno4iF3oo2i92qcVQlCKHZKACDCiW3NACAKoqCYUZkBFsOqYEAFAhsPB9Pynf/heYAZ9FJOzB6IwmoADau9GdPKpd22Fbmtn9+rd3eW9QzAalmdB2lXPQjUv2eypnUdsmYgM1HGvNWoVRh/WV6PDiGdat5yMkXCKFOeLjZhDzCICRDiYmPDsdJwvHDcadbw/MDzer7ADAMK46rsBQ6QqiagCZTIQrmqt/chmrXmyRIQg3gdh0H1tKm9BO2bRmsTL2H97UAwCoGV8KAwy7hk2AMKCpAkCC61WAQWS0Rnbt2//wN38WNznmE3gRBIAkSCItUyQEXfMiXAnAoQwWti1MqH3LLUbjXXbJ1rrGkyVz6RgRmhiSw9tddi5eDE+YveIL24NFcXk8zKZUAIk357fWyjfXXGTm889Py59cdxI2fHh5X/htry9ND40gwgpiiwxAIuuvYqgKri/46Yd3VF1qklixT7Y3vKgwiR5pM/ilhmFSQ4MUteKUEShBCQQRlHAJALABBxAFAICekSFzKvNjMIiUk9dl//da+6uxyXSI27eOAj7S7ksIIGtF/IcGRaqqSKyINRqnwx5Cl4D02brIwCwRrwx4F2JMSQqx8OdwFmLV+7hzx91CqHCJI3RNd/+UTjz7Af6HBMtVVwZuuHTP03ve8JZtVm2B+0E+w+GV0hSEQRREFArECYtHnSepHq065b5L8aTx51x2vrIKqsBmLUr88HKyL9lZWL95tl0Y6yJmesqMkqQgD0ZzQ61X92xiJADkwIk8jVqIAmoUUC7oNlrUzb/bde7VUgZCPyq4OpRxJpBBkobBOAQIgwMTlmv6ljQA4CEwKQxCIFYVSuprbNsgIEAhNml9UFR0YEQEmcFX3bnx6EzXi8sfn7b3e47AdHQWnFiqBebAKbYcckXX3tc3eb+gT5pf28vyZjTAkFAacYhpVBiY/Puy773k2J67j2dlhIGGMs/Kaokbs4FxvOX4j3f+srSaQ9tZJoF0CtgtODH+5qwXW2KKJQCQCI3roBRrVYPJMwBmcXjzm3r0mDAi6JjqJbTHB9cySgEMQAOvSnjobRCQFIiAEo7pV0gQTJaoWdyRHWI0VBxWHsAjTIlUD/go4+ISlNe9++XPfePVG5wJSVB7StpO9QlJeuf/NTffu2dgjrIq6OsnrjhOxj7ZihcB5dMM/UDFYfQwIWvfKtzv3XrT80OfOHav2tbuyb88EvXuubWM2egjEyZW10Ha0PQwEY5MOQ91AoZiKhkZaXGSBwwJ5GrCSUIUmOw9/UveFL+ayeTOWLI+Aj8hBfdufmzf77hkU9CIhpr3v0vdJs3vuLEP5sYJqrmONRJGDVoFFr9fQZZvjHCMmNn/ELTTgwom3ft7Z+6acuZZ943GsTmoBLqvixnAk4sVDuWL/tOdfbZbXSh0fGFXomQUuuc8aIpMIQxbzs5FhDRtmRFENuaxyMmpKo9L/i3+7D8uvmvjhhjbkh5R/iTr3156+OLBozZVA9eCYV724/ePuuMAIUa493Tww46s2oQkfm9rvQpYjTa1HXluj2kvvft6NQL7x005a5Do0NyBI6H0yNYsdkV7/px5/5nrfNoqjwNgrDK6IMCCMELIiATBNAKajYUDBWKEASQPH/vfZ/aEPSvnQDrCLGaCYfdD1+3/t16efJQKz7HVKq373pV4iLpJrErmzDKRjEir+Zuty0NMU2IKr08V+EHP3vWU+4XuVJaXEuMw9oeUlas5BSl8JjCMKRR6H3qS+m9t1pVjXQLazA1y3hANTCiMPjYMGmpgA16DQWucjvW+LcX/mn8c05Kv2HYV9J95f3vktTZcOK2/3kblFq5qLziDeWN97jL8My5T2+z7cfc2TDhuPDHfEO/TK0LEhdmsPjG9S+564DjCKD0DSiF1CGF+XjcXYI0hAaUPmFz4/u/NXnfC0OlKURxFUAYSKEPpNEFdLFi0qFCNiSRrgIAESJ4fPInTlal/rWzyRwhxmddgPChb70DxzohCD9HbqpIVzLlo+dfdP8ZH5fddfX2XR9b/IAzqwYJfHNZJ+yTwu/9n1vn/nrTCK2WXhyPJGGfQF20DnjDtThIUCqvUEhJDbGCpTSB6z791fknzx2vV6JkGNOYhsmxNlyxJggBFQZgjRCZWgBIIYrmp31blO034Tcba73LRXjGW21DqXEd6GCD4A2bYl/G3SmWEBKsqWwsfPXR/SktAIAsQla3TCu65op/3fyCuwlUk/XItqoyA1CmdPagQZk1OAETAwTWIB6hnsS6PvVPH3fj/1y344IpYUQFwohAgOAdEtWekFEFAGQJngQASQTcJuXp6B8Ify1AgJiS3edyFY/bng7OxucnQ6+ZqEYlsS4KctBcmXmgnlrZt0MkgVOFN/7X5/7mtxDUcHJpiopmMhQUhDiGPDnEtW0gDCXGWJO1ACZoFQmv33Lhi/51x/0n4omSkIFQFBJ7p2nkIFGBsQRS2hgIMuZNdqOWQj38tbH6HHX0tizCqLlWsz/4azEJlXXiOZAWHRcS1RND1dGsK4nzxk3Ls42isfy1m/9jjihw5idFNQSytQr6QfbYdwUCoRRgrdajhAiIoP7n777qB487vZNPBqSk65OcjAuhjPSKgzytlKvF9DYOR41OosgYZ44vSTd+7Ty8R4rV1BxIo7Vt1Dx0kJKAtCUCUQpAA0gMCUgGgN1An5qId074ePkfV/7qYa0y04HNkY8T40KWdrtf+7tT3Y5N0bDD3rY7rQ4N8qo6YY4jZ9VUEanMqeWFnP1oZNbFT65yddge3t8YlO+r/rjQv7R6nG4vTj6pNPE2s+OSpRc+rRtFEGqSIxfZ6jZwZXD8oz655/l3LiAPKEryEHUH8/c4RRbKlNuj1KlomNiJDI2GEN2SLHcm/P8z9hgT5Zpfmh1Ih+mVJN07M7Xw/JMuOc5RhShIGPyRGiQFNzExiD/wL++5x1N6g4kFTeJMI5GfVQytOdOPPMzERQGlSYAZTTOejQB+/dThRwsclc0y+uU3GiyU0waX3Zsvuk9rIWotJ5oQEQ5JFn97sTQNN2WdfAJ++PItT1fAziFSovOo1ImpCylmfUKVtfX4mILdPacdn0Psj70i4a8GDG9e9zgyv/ROQ1ZpWw3/58lbLm4N0yYYo5C9P/KKXbLEm+YiXdcnvNf/VdZII5tNd6ztJFnbsM7Wf3aREhPx3lGIYhUEN6ES+TUywx1tqJooKvY1rdxuaCrYuhe7927mbkOX0hwryR/5zpFJqQcYG7Dmzf/4rXNbykRxYCWl1kM70Vt+0Enlis/qTqkt1iHI/A33gaSGX5X6xLGHVM1fOKV6CGgw+JC7vYNq4U7wcS0iqFD4iNk1e+3K2tzAti1u+Bc3P+ov7sJcBUWuNTIJj9xU3B+Ndl/7Md+U5qkXnL3OTo/ilYnY3+E2/v+/QWxvukyrX9oVa9A/esoTqCZQQiAGxiNnR55zbYkVSAQ2V9FEccKbXv77d4/KrCzaebPqrl/olO7Gz375uDNffW/58VXbPvqWU+//KLfSEUe/6ams/UtitHs6/PJRL2B38JBL037zDnfaHQ7Bx1wly1/764c+asPIJN0klCVNdq599yUPeP7JbnpIMTi/+ytfDpffMmiIU7/pUdbafeTBC/9+Sv18RehfBN3ovt3UsfAx64NCBgiTjzr9WY2LN69Ml1E9MT/pb/1P/4G75m2os4rA48knPv6KV/fLZnD/rwRZ4GpuHlLF/hdAq6njoLrN+PBRB9Y6dj7a/MYXpo+fuG591dxJxSUfyL4SQgO6oROXkMkoJPd6akv3k0bvwKzybxr2rYgVa2RVpvuXAQ0a20APj3rn5j54RYKG6/Tk171fwsY07m4YvOGfn/TvPLRmt52qHYV+nbX03oxTcv+PZHsBwG5fLwF/6agXtIYt3WYJxyzLikqVoqzqTp734u1b/EpG337zKR/fHPmJPszhctMnNuROyUaVS7MoDlUp/o2DXHs2CMgv70OSfH4LHcNeDw1AHrXoxfYTH/XYe0Xl1y75YsRmiRqpru0kmIrjJgAsbrWjPFKHrN3/pmDtTg6u+SMSCL/wuT8PGurjR/VEXcSHf+4dg1PWAlRJI4/++KUnrXvfW8qhmLxpXdEEdpFEXINFaJkfXhTq5Dc+7F1Db9txFMD90ikoLW1JUzGHDZjv8ACKBgEAK3lS3vNl7zrnvz/ECoFAdEMMWAFBCyCg2juKJMBvfJTF2nHEKNvWmW4LDO3jPhEBABYt4zHkQznLX1nXDaIL2YUTb3rrA4kBooMO5EEWzKBRHruWxV8Z0FZVny4/h1KC/XvWqPRApA2ieOfkkDvBr8wgzmgMG++dXZzkLoirD3wc/X8Xzcq6X9XnOVZACeh1DD88r9LA+1tPs8SSBFczECEdOpj8lRlElESj4v4vD31uWjEHrRA2+U/LaPQb70MUBxAdXT59Igs4s2/FhxB8AG1dYNTq0Fng25Bg/mIcqeW8GrShn/UmnYES4oPbYoY7XvwRoF/w2/nNAHoXFca96lEXKq9Hmd9nEQUAIuhRWOlQHyqI+tWtEFICGiYEQx2QD9qyQuNkd3n8a5e2PWKINxjRNdvv7iXAbahaipLFV0VutJIgh06i/uqcOjSgUko8gM4iOiiaQr902q397CBD/aYhiGLQ378IfAW3re5kMYnoLOt7MoR4SIP8yvZsU0XoE+hOghMV5KDRsABzW3/82F/Vpzl20IJ11bnqhXnmDGi/j7qjN1qcX3EmOmV2SknAQ4a9GkdWNAF4EiIfDIlwPRrk08Jk4tisTEp3Au74AXpfy6kVSEEmZDzSefD0abd58V/yoHm0aF33pcHX/kRC4CBWYMwfDGNN35/DhL3WWnWHEAwbczWcjqwYgCi3ptcprv3GD0yUTk+EwRdceO5ZGWNwlupoacqZUEUQrBsz9IEeZUXsEbtNxX1s+x/eXBV5URT1bjJaEcDjp4+fdoU94gri4bA0nR838dUH5McqY0DAQEQQAEEYRIOrBRCPeoWytlX00Sfs+z1FOH/FV26dPu3NVoHVkmejS1/6ynvmVoOPwspUpT2m4AMYgODBINa9JOu1Adzwa9/ArXc5D8JY3rRURgGEAPaK//jRE551h7tQ9q2Qw30Pn3V3vOQjstYiebRXCAGzyNq2LuLHXFNHfYV4nWfulX+97wCCC/PHmaiKqggkkPjKSFp++B3v3Dy9PMml0TVRXdoMuA7WQD1EWW5Dry34vX/HP7gAHI9ZW2Xfp8KhNK+99Tx9uIH5IwX2MTbPuviJR5uJeh+DAyoFIGMDjKdQQZgPOjIfqUGYBq382nP2ZXlHWpJABflIBVYE3vZVsvLBT/7H7HISYTdOBioWxctthd5hRLgwXYbMz78tPPOUymmOgBAgMEdjGivsdoqkKjrHXGu2jpbb/qeve++RsnoeiH3c9qs8atEqaSoAj0duD3g+HaFBME/rH194G6o5wGGMyikaj5wz6H4L4A8ueszkoDloDhsrUToaNdJKaQieRU/tnHKjz37seecLRCOdOkZEQANjciaRFBxFyMfcILqYXJ48vbjqgmP0/lbGCuO8esMrpXRgf9TnfTkK9r8upH0rvAsdq0cJEcj48j5vzU+55//Zk/bs2uU3neIzKGgah4nUgMqg7s3p+HfitzciqHlSBioF4MBCa5uWYSNewR3Wkd+3lx7meYrrDtsHfOFoGeTAX3iNajxsjgjCzAkAh2PQmRdMv75V9i+Qji8JUl/H4xuqAKSeDeGs4//wx43Tz/r40octK9u/9mbceNIMeUfYb3YXvvjMCAZR5GtjpFJ6vwdBgFBmMIr0HV7Ct9cgIlBHvn7Ad1b/faRblhzwJ4rPl3bP94kUinM+ciUn67bMrD/gdUe6ZYFXX/zBXxf7xjeWO6IGTb+PzrOMc4oHqfriM766IfP+mydPtgf//R/N3zFXXjt5/3vNMK6lV4+WzgbWwaAHCxzEIlQR1xEHWykC+IUsP8yEXLsbzl9p6DtAs4qVVmUwVnBMvOBUCMhi+3tuvu6qvdMnXdCanWkq7/eF1avz5P3ecPHHP6rWbT1x66xFQuQAGoODuLSUm30f43D3BaWO2NtqOPW9d/+z6OEhh2T3Xx4AAN73qR0PevKZMMxq5e27v/we9Adc8EgNMmyWZQeWOsM4AvEBbpyFRlKSBRAn+hcwZrAggfevem2JZXt56pf9hVYQ93UahkkArZhBAJduHa7csrOcuODCjSS5Gt/cfe+7Ok9OBWQwWi4+uKs3fec7H98JSgIYA155Rcx4e++LN8IBLED3nVsfbgfp7WU79f/afKIuYkC/NAfz/3TGvijjaDm3HJIwyCKmsHTrrdt3LNYunmnd6b4NpcAz6UMX/VlQgfCD/vDRlKeHrqgdCr2O12Fl2pkSYiglKS957082nXL6+XcBQq6FIpDAgLj2vmvz5CNNaKgaTIO78Rs/rGeOP+98KyXp0ppQI0a3Nwz3iiqlqqioX/KqSfGHmgA8+HXeKk8yyMDUdTK8I40qh/1gAOLD9i9ePXdcMn3clqgXfvbh+dP/TxwBIx6yc2xNWO8bL/tKlRbwS6dqsBtFAVEtTcEoxcs+/7GZxz1mQlQVDAkTgh+LMBzkW6oE6tLGsChJA0Yr9U9+GH7rblJlUNvaRzXdXoEWrDHEMEjh03uf4ZS93bmmMqnqGEIM/WalXdI/6gbBPhkx13/+psfcy+cTWI8aJrj41n97QXIYgwgLIkL119N/whGv/LL66HvXrWDHU5Uw7Pn8Z4fHP+kujdpKHpMEsRBG8bhJdV/GYW2evADUSpgj8I61Jr/7e5fyhffYRD2KDMpgzRccbgvCCpT2IQq/99rjh426PpwP2fe6Xa2G93Ft3dK6oIYDPHBJHmkYOGqMst6rfvzcxwuCODQAPWj7EFUavKhDb1kowgAI4fK/ed3ZvfiwRGEHvb5rEkTB+vIvXL/1IadJo7ChmICe1VB7naIfcw3LWu5qbZ4cnVjwNYASMAqcKi1d8/mfhPc5soN89japl8PBm8I3qy/d+nyvoDxsY9Xa/e411DC0cCWNnLMhgqNukJVJqHXlWlAHZcHXOkDCQMttZA/m0E59HPQgIucvuOAPsl+aQGDoJ3ZsuvT5T3zAhhlwRkFQTLDk5lCEAJxLYY32aN/1xqIDQsjB6FJrEI8KS4mgxui6715+wZ3n5nbNrT7/cAbxpkhKiL9y0+80C5Nnh+X7WrvfXheu5UYdKJLd6wu7iAcq0Bzx9O1yGvdskqcsBAzK6xrAx1QSgiI45BAkj9l8cdhwO5/1ztn4l+0HCq4f/vzmD826WLEHEFWDUQSlYSLvtTkwO7pmkPEgufdxKTFUGJVIioAF1Lb+5254znm3d4XU0cqEYPnKF4U53LOOq8Nlrfd9nIFKwSObPK45YXVIHyIE7EWrUlOR8fI09BCbRcJUWYRalEtKiH2V1baKfIg8jlSWp5U50mit28LB1/7tX9t55DAqf8GSKikCBhg1hg0us4WZW9/xpQf+4fF3WJXgQHhWmq/9+o8efuJJsfOkKJja+j2b1uTDD+IPk6KxPFl/7YFHeN1DGqReaziYv2rlvOOClG2oy4xq/4OzJ0oVwW5OPn9Ty808HjnFOigbqrTfoCOehcPKNYo/nfyzzvzsHi0zh4xWvAGvoTfasGtDXTa6ky+84gHPbkjrkIwTvzRqbw30eldfMzr/Xp1QNHyvo6qo2xzTqB7kC5emw+LcKLdH2r1/6CjLi1JQlh/4xsNO/ODU33RbRay8vvkEeHn0EgWw9Ir7vTt9eKa3/0FkhXNM59eV8SjSR/wLDd0W5r1nfxx5fpOGfuNQK66ImWhFJnNb1W368Qfuer+Jogkr6miR16A4RiLoTez8+sq55yW1zY2ULUfw8ycHcUU3V9TrXnvE1z2UQZADUsiLF/9jx6sXXvS73c4Ir/ngP2C5/MxP9ybh6sd+5/EfmlDDZvjTsx/V1tVlP32yMizqDss0rKGKYWHG/3hqfVz2Z7tRcqgtyxuo0VS9WS9m25t++sRHGYqkC9NHK1lYawXiGaIicpd8oXW/e1ZNEOw3x2IFB1tEgLvTn1z6gyP2wYcySBBBkhA+94m//ewVj3xwkXz8Ja97MAznED7xkf8YvHbne9V9XtHbXT3whGjwlPRf4k/8zxsbdXR4X3Z7PlDQvfYP/va0l7Ty7NApIVxOkq7JuC6eE79+zlkYNKHbosOTs99OOBrTKnknBgLZb3zkzIdPZODWRM0O+jzOlK9+4p2O+Ad56OCBtEKK0sdFT+q+5n559KO/fMejGw3jy/oRppt/8Z20uPyaL9Ubi7QXfWbdQ4prpgg9gz/iDAwP+mZXZ+H0d531rJ/qvFEc8nmVB2tg9P2XP+1fZjAvQ2NhV1b3D0ze3mFY4roui9xHqTGx9vd+U/my/77Jm3FK8uBglB28/YTTe0fcCnvoLQvG0i396y59ZgLgX7b9DdF2c0a/4XF56nUrr0rp3l8xcP0Wz/EIPvPdb7zzPBs4KQ65xdxe1FHP2r0baju//Y2nviRenDrE8/IMikS+9dkfP/9hoSRMS6MAIIyOlg/xiApA2BnFgRSMQmv0T5c//l4tXPXqBzyfe99b+P0iucO6IWs4pEHG9bXAOv+/rxez1P7+t7b15ga/e5961xw9fc+7T5LBX73OwcQNk7YxSv3Sn/17X2XDhq+PNMoRGqpde/Osr/XEl7defMgwWrhOF//zqjP+D4iuQrprg/dMER4xJchtryCCqAMrYhZTB0iu+pf5f1sNew+8Dq788/MauPzLZnwOwmFzWewe/vJ7wfAzv+OFQs2k3vl7E/d89h8sTvOjXnrrtmvT9MVZ7EL81fseYWFnmNVaV4D/9bOf1Bs3x3Vhd3f9Tc855bwkn+h2RplToWw6XUelb/RNUjHGo99+xLN/ZTy/Qr7McM8VVz3b+ET3s7wZhulgspdCvbxnfuDyPq2bzqbWNWCsThcBgAiLT6Ef29v/OzmsQYbNS173/tmy7gRU4IHIm3w0s2edX/mTrXD2SVtDHQ+T9D+edMTtM2YxW/nKNT89Z/OZJ02p4KOwrO13hj++7JQzH9Jo9xNcmYHFzqi9ffMIU8hTqK97+xPvjvpXNVBSWQik/PAd33vNaVXmRWpoQrjh/2vu2kLlOqvwWv9tX2fOLSeXJqlt2tj6UNraYkEqQtGiaH1pxYqoUKGiICjWvqigj0qlqAQr6IMYK6lS29qClpKIleKlVHpDKqFJTWISk8mcObNnX/7LWj7MOacx6Xim5zLke/s3G2b/+5u11//v/a1vdY8dOVzP755S7Wl3qlKLxx3Jqd1798zrGmMMKMid2UVNMrYmYVVCuJ9ZZaDWJCQwIBetQcaDRk4tpFpAGLQHGfz4L/vi9RLS2bf7XiAOBsAFpZxmhiZxPoGFn7z0znsuq87ML86YbsZRIyX3fvizh28Stj8/qQgJLIT3RpbpkRcP3R/PFd1fPUHf35PqslS5gqYxCiVAo5iZgwvP//XYtbdfIyptgIJPxvbNWpWQJj69rd9M18nQ3hcFi8V2v8XkYuJKYNJpiWNffIRb65XPQJH7zjauCLQBa0nLqDKLYgq6acT791/1oEEYcA4VpwXSFzoPXdbkUNXTa/y9t319jg03QYNm8eS3H//N0+1rPjG19fS2IlZAleOZwre4izMAKIB98Hn4829P3PKhq0/n5XwYjO2Utyoh3Rk6Nw+9bGjPy+zSHs0OEIwVMdcJ91JTPNS/P+N1ErKYc51CkQsOnoVBGDS5zYvcQ4gY++GxJz/w0Rmje3Z+wNEfvvG5eztTirrtt28mskY4FBIgOCtygN//+o2fbndpJYJvOQfKIPSMQQEQRPAklAKwLoNjTx7c9eVd8sTOZmybo1UJwaOtuWM7bOokEggmVWpd6GixhUVea1nkzqUf/Prle9ZLCJ7cASd2Qk3KAABxI7WoeTBXJ75LW7GO//n4M/fcJYM8uwX8LV/65OF3NTavs4k5PwxS4UkJEmyrdv/Z90VxP4Ym5xBIKrZW5OhZczncybrQZKK0USR+eeAHyTxXY/sArEpIrZVXTpdaIoGE4DQNdMCWNYPERSGQYuhu9azWSUjDaLVplAELGoEEFxEbZrkA09A0CVatZx6c+sqNNu3XT+29hfXZWXFy++JIP+GNhldog5Dgy9nG5WfmgUtkwJAjDF15mTE4aULwLJQUSyKqCnv3vOfTe7vpxkVIr9UY2W85BYElB1NkwyYkjVSM3RngMis583KdhLAoE6y1dEvrpn6LkBG9IvTIGshFbPcfeMph/MDr+xZmj1zZcduhjidFCAYvFYQg1LnZppjraxeLJi1ygIACIIQlH6PhkyKQdk4wadOdWfjFH/fel4/7Fnz0Tt06FQEwlCoCxuBiYAyknccIOYiN+qA1LrCMfN1+6Uf78FsHH42mzk0tNRNbbdkbgpIexbD8koft+IgFUgAp4E3rvM1aPjMP2v3vvnz3ncAek0KstnEeSUgZKSAgH3sWThgAKrglGhIRuEokK4RP7JExaIsyFv19jx7kMN3LcExCEIAsR6Ie9h5HK6QA8sEsNXrc7Ose5M7FZT133V2fvTJ0tq66HxlJSBOhI4PswICX6HyECGCRA8kIwZ4nFpgIsFSmqtMonNnWmy4WLwtjEuKEh4idWNlBEoFEACcFBfg/OrENAltgjKE6/sjT990e8aqGYKMfWcGDkhQQFJCA0qhgkaUBLwSUMPHWRCQIJAzqmRolGzy+Y0xCSDrQYJdasyMgBQJE1ORZQPDjynXWjIUktrVwmfndPz4frd57d3RSd6jBWZHUASARXnENCTgyjRcIazATXCfKDBolCRGAa1lOjxshgGyJVOQZAQVCpYaxQgximFY2F1ZjYFBY9vXsmTnPq622RoscSAjyGsEpBGtLrxkiJRKnIbDavNaoo+FIWE6RTu+sfb7QGpMQrqQBAYMMgBgAJARmYFACgifEcZWJawUL61LfxIroX3sKN7P2CEFvZRQG2sV89OXXO1foqufzXVfc5FGRk/FGl56tBiwpQ2AmULw4RVaPm9SLVPRU2j1LwRMzVGim59sCwFmMJPD4Qrg1woYUqNEh6qZKVnLNOcRr32Rw5IW/z9x4Q25lZGP24CE++/MXb71z1jXLapCJERIEeyWsYNClMjB2Um9a/zlE10+3GAClRHAk7MnX3ij6fMXN12Yrq55NMwFldFUmK4zruCy3XNTt7qLzL65CZWAGXpyt1dHHXrj57jRTRFIsC+rq7JUj/75tRw6nZk0DJAx71gI2TA+14RhOZyUCxNLYh/7pV157xx0zhsiBbsyID0+TxihCINRx56u33SETrTGwWDHiamKkh/+2+1NTaUOEaScxIpAUm+eiuU4wDAk4LwKGY9HYlv3ToW3XvR+qBByM+DQ7aVxMyHAGwPWBA0+Xg3mAYUfIlfMq38aTzzz34VtnnCnrWSDrVbTpy/k1Y6lQ6vwcwcAAQVOJLX/y1ZfPfeyGIgsJjJCTTBhvlUMYgOHVb17+HcT4zBYmQPHmeYJ4MQ2u8713f+b44fRo66q5LQr8pWvvyrBUKfW/x7jnZ7VtEgXNs0/cfKef8pcGH29FyPDP/sDOj+Sume5ME6BEXomA0KTe1DE3d8/dUM8PyJ0qdr/3+vYl2/djuVIKLzgoGyKfqZ6q5+zD2cdPbBshgJs0LiJk+c6XeRlJONtWAOJ808agoJNKU+SnOOdUVVoiVBXJ6cld89vChVXAy+OFJAEnpFdnt3gLzz3/tUuDj1GEIDRxXwezOMcXKL3ZRgiFjg5fHVx8LmwpUSmmzX8ntFYsz+fCMXqnVc1JZy5ILjMII0TUk8ZbE4IAoU4kEDYKkeG89tPEgBJCMAsJYcw2YpJAjtcrodwsLM/nwnGdeDLQREF4n0CZjiozmDT+C49bXTLNE1P+AAAAAElFTkSuQmCC\n" + }, + "metadata": {}, + "execution_count": 4 + } + ], + "source": [ + "ds[\"train\"][0][\"image\"]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "FOxmdk-HM7L6", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "ff7c2ca8-0c6a-49d0-cfd6-4be775e012a1" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'Two women are looking out a window. There is snow outside, and there is a snowman with human arms.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 5 + } + ], + "source": [ + "ds[\"train\"][0][\"image_description\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Ri187NrFNMaF" + }, + "source": [ + "We don't have to write any function to embed examples or create an index. 🤗 datasets library's FAISS integration abstracts these processes. We can simply use `map` method of the dataset to create a new column with the embeddings for each example like below. Let's create one for text features on the prompt column." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "xB0EfabiBHgR" + }, + "outputs": [], + "source": [ + "dataset = ds[\"train\"]\n", + "ds_with_embeddings = dataset.map(lambda example:\n", + " {'embeddings': model.get_text_features(\n", + " **tokenizer([example[\"image_description\"]],\n", + " truncation=True, return_tensors=\"pt\")\n", + " .to(\"cuda\"))[0].detach().cpu().numpy()})\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "iUWvvRB3DJwy" + }, + "outputs": [], + "source": [ + "ds_with_embeddings.add_faiss_index(column='embeddings')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qZcZNgSpCH5e" + }, + "source": [ + "We can do the same and get the image embeddings." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "AwXh-WlZB6q-" + }, + "outputs": [], + "source": [ + "ds_with_embeddings = ds_with_embeddings.map(lambda example:\n", + " {'image_embeddings': model.get_image_features(\n", + " **processor([example[\"image\"]], return_tensors=\"pt\")\n", + " .to(\"cuda\"))[0].detach().cpu().numpy()})\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "s9OX--PsDMNE" + }, + "outputs": [], + "source": [ + "ds_with_embeddings.add_faiss_index(column='image_embeddings')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1BS3TvQO5GGJ" + }, + "source": [ + "## Querying the data with text prompts" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "pxx9fTf83xgE" + }, + "source": [ + "We can now query the dataset with text or image to get similar items from it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "2UQQyXAbNKGa" + }, + "outputs": [], + "source": [ + "prmt = \"a snowy day\"\n", + "prmt_embedding = model.get_text_features(**tokenizer([prmt], return_tensors=\"pt\", truncation=True).to(\"cuda\"))[0].detach().cpu().numpy()\n", + "scores, retrieved_examples = ds_with_embeddings.get_nearest_examples('embeddings', prmt_embedding, k=1)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 190 + }, + "id": "O5bkNf4M3_Nt", + "outputId": "b56009fe-dc99-4cc3-84e5-559fb3625d30" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "['A man is in the snow. A boy with a huge snow shovel is there too. They are outside a house.']\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMgAAACbCAAAAAACKYsbAAAgmElEQVR4nO19aXNdx5Hlyaq697794WHfCIDgAu4UKVIibcla3PLS09Ht6GUiej5OzPyCmZ80ExMTE9HdY7u75XZrbMmyJEqiSIo7uGJfBOBheeu9VZnz4QIgCDyAIGm14Q7nBwRxeZc6VVmZWSezCiT49yHq992A35X8uwFift8N2F5YrDZCu7x7746IqNIw8a6n8N4F4tzdDrXrAdmTQETEOagv/HboXQOhPWl+xZn54dYD/BwzeC9OdlZE15ePtET6OR7acyMiDBVOzCZOmEiJIgGEBKQEO2vZXgMiDHo009xRePoqGLTzdN5jQFhQv+sfSITVcs2EkRVllK7WSNSR5B/UiDDs1109kaqOGkPGOAGzRMm8QeIZE2avAVGP3EGnsHVCPMuj7DmrVc0JEUQYaq3xAqhnTPW9BoQFPgECIQUCARCACNJgjJ6WPebZBVgmECmi1YYTESAEEuw4C/YYEFUd7XGrEEQEEBEBWwfHzMy87ZN7TLXMw+ZcpAQA4iGJfyg8y43sMSCAMlCiRKw451hT2bIiqSxHtWolUsR/7W1jvvYWEIXMVYqipepEKRmkFDxXC3zf03ox8FtyBpKh7Sb93gICLizdTFeDclO6daq7jZRXtTqhuNyS8OejtDcyuO2ScW85RCFXv7NYVqXE8pD0aShdrirPs+XIVy7jCUXNyW0M8d4aEQISZ8vjI8FyUHJRTexcKYF6qahOrCy01DGXjNDDjWOVvQUEotmlh/aN3ah4y9qZ6w9PhxFyqYXuWltSoZPDLIMb2q+9pVqxCKF61dqm2vLia485CmvvjuQqRpfqTao00BZ6fwCqFYuAk13DyQPW+3pOTpQydxbqns9RJhe53K2ug411ay8CISKp+EtfWVNpC+Abcc5pTSbSKdWX2cb+7kkgIFLUdITN5ZATTifEhlpYXHkq5U2bQkMLvBeBAELl9OxVnxY7r43X5yb+kwK00SpPnlaB/YNRLUDA3HSoIs5m8ry/y5lq1ZCROc4iDF5n1WBI9iIQZjXvd/zCeiZ0lXc4/2G62EuBeHzcaITW/cGMiAD3j5bRI/hgYP/VBcPVheZ7XlO+CnpQMHKss9G6dy/6kci7Y4/XR6eojXvz9QXJ3xvRZIK2ZgkpKGgx9IcAhCH6/uyFVbZUWAONdWmT7DUg4thcXzmvNTkgdimAxI0kwvZL9z0GRAjF66nTpNSW68+QPQRESKwXDS8PdlisMVtMtPqvZ0HZQ0AA5tHR3v1aSAhxkMsMFhuG5eZU41hxXfaS+WV1e/xi1sr8YlTzhAFOrnhJpQyicib7jIf3EBDG/Pi7hjXq6UB7Rqxz1a6sKCUaf0iqJda71L0v8jZdJVljG3eWvTMiZFy1Oe58IOZNQTFvuptE4t4BAquDUiqmGUUAAoGJiURABIgQCAISkBCE+Clue+8AEaKKFWJAFIiEhaA2uEBa+xlfI2jA0nogvHeAwEwlemIuWhxIlAaqLiSJnBXtnNFV9gLl4NecZywtm+Xsgb0xIgJ6Kl1Qo/m5aKluIw12ps7VaiZTV4qLy0SBTyU2+cjlq5ER5HJuPm/7+MnDvy8gIgIBSNZrAljbS5NmJu+amqbz6WE+jvHTI+mOloe5N6ZKh8aC/LWLDyoHbx5J3BnwrjX3XDpdzCQh8ntWLRFSAKLQf7La48w5tdD5Fx/p1mw18c5i8Zi/cmRl4sSjxyFGvfHMcu4euWk3LkExaoru1HLjdwJ7jF8GSGw2XkpIMP4wUaF66d2m1XeR6+qy08Oiv5ks0pGHmcwttThtLLjTM0ddh66aasVV86pWs3UThcqTvrTjF54jAhEmRfySKSJnPnDHgyj/4LMfWUMAoJSI6RmlphP5yp2FMGorlzuC2jQCm8qQUffrOSTJr69EqKuR9PFhyuhUktZJrucDIqzIGQ04UGPmcrcvojDXbxAkz1Sf9AgRnIUt5fyzt0qpkDIPlyjgycESD87e3p80RT/gUqthZE98MZzWxo4netcDgecDwlQuFb4ZSzV1aLddxmV3oqNc034qTS3v25AaFGINGxGcSS34kHTPowpaF7Olh0geGallZZlo0iTp2okDqqorfi4nO6rWdilUYZQ/M3OFoDZzN7Fyul3UC+cgif3jX83nPzp3slOeLGRJnIJKQKm6udhhS/cWu4oIE0vtvUH6supdzufqdNg+7Mgv5CejocpJlQpeaI4Qezezs529moLa8sJCl1PYPjm5s4ig8/zUwplXHOsNYSvZENoHwbiPOsQl/P225o5O0LgVv/vLsR8mr+H7DydHvluvHJpwZIWC9UefB4iYm0s9o+02opXR+nft++dr6aYXBEIgWbapHDuzAYeQECwBcPpsb1h+qJyppkIiTq5kXHBOLQ8u1Zdfv17yKp8VjIPyd3aI2+v+3dRUJl1KuIf++PjsbWp69Ha7NKL9ni2KVU1JQj217iOQgmOwDv37o6BESUNMZh65SqQog2QxIcHifEWqiddnFkCRe2I8n2uyl0plOe50eaRj+tX2o/v7AnP9PSvyzLVCY1nqG91CrYuDWBBc9XBn3T4IZ0LUb+pyzqSdN7O4L/sxX8h93keJ8fF9Zv7/1Y5ccC8ARGilmGhJcfVB15lfodseGLu7dIFFYTckRwNxupLa/AVhwBdAy9gDsyIj94/kZx+cz3RNZpYHT3Fr4UdG3iwXHkt3piRN+3VB1o3NcwAh195W6Xp08JvKad2bZOD+g/+CMDH1eKDVY9eQ/tteRFxU02qTMxJhSLlAqNkjvaCPU+mQmv6iWh3xFSeG4FyXWEmulNzs4ivlieOm/oS7ex7zSaY/8WUwtZKqFB8HCnR6uWjqH16n4UtXKt6zSvU2i6kSNg0IiMS66Palm3O+vvfbz6aaM8N556WROeNUSkWW4FhL4tLdRLbNpA+WR2deLPql+qll4dHqKzXbrQT0yN0c/PrwIYWVsd/2HbW7L20FAFnMS24LeOeRerUyc9PRfHNmwvR1h9aLsmNfL3StQAMKLN4sTKQqSvYdA6IXA2KCH//02pQ6H0RTPYLozPFPx97OOkeZYwMfuhNs9e5NmKhyUzm3+SKxVuQ396ImSAI1XE10SChmkAgQEEE5b+KKOVBdVHw1CtMH1lXruayWFn69g0euNY8mVETewt3+gyIaIjbx7gffHOp2gt1WuArK3dMdm0ZEEBH4TlPkoqikvNC0Pt7nTZU7swvFfoBEABBN5UtKN+mSzi9U3QvGWqTaO2rhwoftQ1VpuTF2tsPGfpkk+MH0o+HXMru2XwoVRn4TOAXPh2nqItGu5OCsQYLnDla8wmzNQRwBBIm4gGJrLSmPrxaOrT/8nNEvsdXfoy+Wm8vBl0tvJyMFIq4LBSI9vZduJRKHI7MrKMLkPF/iojKnBIpFL684K9WRlVrTVCm1YOoznnsz6J2dbQ0n+h3FnUZIzLclspf/cq5l9ECXeRHzC4CgFBxO/M99bZ90fl/Ec445dDog0s50TAz9Mx1i2oUBE1VOVJmcBkhEBHAid+e7DJOfni0e+6aQ+Ea12P5zgf2zlS8L+xZLacU6ZruOTGdHQoxUjh6sHnPrb3yBFaIS/8ez4wcGQuVCEWhKegRxXuXj1/M//GVLYTcLFULJm94HFTtBg6qn5Fbmgjdevjn6li3qseRxU+w/V7CRRtt3/pU6air2faR431+Z377xp9eON0E2FJ2/ABAWarl8pKtK1ioQTIIhor1HX545ZAsXP/0BYn2RNT5WVutF42sxicimpk0LQwhQavlxSVQ5f8C6Zlk4/LGKLg8O2B+lNCKlSIRVqQ9FYk0QKKbM+2258DSYaUPE+dzcr8CJ/rCnpWYURJHxDSkiN3KXL7Q4bc2X3um6Us82w7epfNIH2LKdGT/YYSOVdnFF5gpnFAHCq9V/IsWV4WX+QRqKxHrEv2o9RVviiB2BPCFbEEeGAgIi7/N0f9WD1hoBgCpWHk8mDw460hCJfvVqq4uiSkQMIXGW6jbrgYhERVYLBGGdaieuZ6qt45rrJscrp5usUsIxJSqKLKBoPRYVEKK54dT50LNO4CZu/rkLtrR1i2qJKImsEiilN/QrwVnRntQjpO/WTs/WKn604pTvI1yoJDrfbEFESgjin/zqT/B4oinlgcjTHpRXrEFBGMLK91mgyMsomLDJBNr3fcBaBY6HQwkxKdCT4Emo9rCijJr7yGDO+dpvHpjp3Nrpm0eERW7NFEg53zmusPKVE5VQshCZpLK2lvLktS9abtQSrVrl/Ey9Srl0Tq2OFQBx5rPMCfvMqRdeT9sTT3p8exHW1xZP+lohhBNFNvQednduVd0tXyR36GCgABInkavX4BuxEnXmjBaJyJA3U8v3dTcn178kcPRkSaL56KdHlNtouRq1tBRVC+IAtYvUR3b5AXNNsqpc8qwKbHap9dmqRWx8MIOYFDxsTHiJiC8sla/Ot3bWyYpogJzCKmm++jwk3zI+4Da2fUtDGVRKV1JEenuk61+l6bmCThESCeiAOWRv5d7BrfdtAaJjX0sm9MhB9Jr2EYhE4PzPDrXWycS/QRQRVpOva2+Q/bf7eKduFucSXJ0aEh4bAbebASV6uxhNQq+q9s2VXYk1agvQk+Vk9Lf+VqJnG2Um+rKoDncLNufozZg9aBUpkIigQW9DXEs027kzvWKWf0YV+Yvrt7rL/K+d/T/cfkqxWhkbubNIddWS1aq5gEMZVZ/OJbcY2+2AXF14d/IXbx90T3etsHlwSITivBihUd09CbVNdO7sntz7Ayf/T1kV7xzqTORHRh8cs9vh0Hx96KhQVsMx16pW29pytdVsfX1DIKJuzr9LfUfd/zh9smZWqxAE4jz1fvlNMY7igHodCG2Iekmj68oOIITEfLN4dl6Hw58E/xL9YH/x87/cBrZYs/LV4hgrV2YjbrL5QNVzbLrsR993m3W3IRBC7v5o051qT2W00BvGQaCwU3r6i/p7gtjmx7s6GonNUN3fFgiBzfXS8v9TA1f7cjx2R80c625cFAf2xm52HCynuJzX5NHigSa/VEpTvThR2rxK3ka1uFvfCgcv1GftRwOvaoBJlNZf3m46cNQBAKvYCTcmHFwisdy2/dYVpvlPTj0489FM7qL369c//yl+uI3VEnXti+6lBa9WaWrSKH+dcVMG/X7K3R/wt3y4MUFH3N1tVekfh4ZKV//u7Zm2HlDt/vjE9wcQagjH3ldAVkO2jIyITZRarQc4aJEt6WU2V97KPE4crpuZc3/l2V+2HYQ0qGQSITf6N1ABsdEAeOrd1WH+RH+Pt0DfZrIrEZF/On5gKdf6vu2715J20+UD+TZ2xooiQ6GQAFtDNwAgB+/B/upEkM419BCKZ88V20xwYsrCw/jQxc9+0jjGpGt9eeccST1yrMeqS5VFQQD59L+6XXj2WFgRk2+p5YN/OrX/RHlu2XPNPRMawvFy01grLNCJBtNEGTs0+2H7vaMz5cNtAG/+ppBfD0Xqpu1u/4OpYCiFaKs1BUi43vTFCnladAXJ9Jg/H0ZJL4Fk960LVu1qsscRm/zpP4w2DaePHaiorn6zuOxZ5ShySAgpcZV6xN5c+jBjs+oIjHnv58Gfs8xf6cpkWq0Wkg0hi6jTX7QMDXffePPm3/Ue6k38ejHZqEaOnefCrv5kACIoqDB9ZPV//uwfozcdhfqpTT7bh/HCcJdK6khLSTvh4MvShZFT6WjZek3EUaXC2iYrV95sa8D8OoDt/303T/7IWOtiVy804J58VdjVP5oqnpDpxEW/PjJZNX+TlAZInL6SPMLguKu82wvftbQaFd9eSA8lI7Xxoe19qhDoTVz69dmBsgKH1fy8v5SuWu1KVIuIgODxw9fatygOBJpBydeu/NC57n7Ur3XU6kY1PZlOpGz2/s3//vlEOn9J2+TrfCMdNrY5TWNHFBQkCsVSbnTZAuzqEdt6afhnFy8kNt69PRAF0VZel8/zzRVrkjoR8Xg6FEc1EYhO1K67t1qfOIB1c0sCBeLBGzMdTjEH5oN83lvB4YyEygcRnNbceca66f9cqThKTj0I3g++tzU4E8GsulnVGrWKlRL5w35S+U6MCZq9tqPlTw73OLU7EpugEZ6z//vk6ebHH9TnUl1sQkNETFWURqu9g7a6mSuMHwNIcHCsQ5RS0lLYD2B65NRXw50BJaqRRywd1cepCw8ypDW1DhjzYKGNZTNnoaTrSq7VaKJMQhHQdH5jCdS8saI3PLPzCkhgojcOf/SbgbvvtM4/mF5uTkUuisJKndFxNF0Pkts/avd/VNcAuJvqiqQ9K6WLfXUhhoEApKJy6+qtTt+NsAUHRK30n2IFWLasTL0uDsSyGlLk3xmZf0XvnAxdFyKIb9v/+v2R9zq5pycaf1QS4npvFA1aVQu1tyHftIULSDZNDMaVSBoQBKRyHKzVWTJTIulECQmJQHzop6yOCDm9cP/1yCkiYWNICcxTyde+3Cf2OcqcCIbdjwBrhGlgf3xxZPioDQ1hh0MySKFzfFDiBQ5AopaRiwN/AROpuJoppqadTW9x/+Lhi5Od1lcADDviSLkQzgIQcWxEfptG9FxMIxkrSjutxQkpgLnjXlk7RWan5A6hZXiDuoia1doC8aZVUgQSgOP2VynY9CIxtbGv8/LPKeURKXGIvOr8Y6t5gVNpHVYqNievfDx8/MmyZzdARCNeXGsSEITMSPPRCFrvsAwUuGS9lAUggAhZb6FXRFbj/bXYhuKixWqamJ5ojThNNyZzFztWsl4kBO37hr1HzYciBWjUh0nnIkmvHDxKz8c0PllqgACwNywHKsrfcTkLQA9aq0EEsCLfLZ3kOGIGFOHJ2pMExYBkw2Y26819mbmQA2fXKQMHTPWDmExx/PHRtoSHqDqa83m3VquhsNy96IdKfH+nozHYzH5z2AAQkCvOU9kk4rrkzaEEs5k4BLuhJfr29IneKFJwFJ+5owC1VG1xMOXbiy2vtVtATOp4faM3fl7KlIn1yNc/KBmLJr0d/SGsWD8cO3G3o00bj/jOQjfYfdNVqLmgVVtogjhSEAg0ouJ1JaH+Tjp+VuQX7Wd0tBrfclxd7szX7kwtMf31oX4FRyAheTo99txlTtZf+OyNUDE8tW2CnVnM1dLZKHv7q0Krlxw4bAhA52fZAOGD/h5mV6eUqimfa8nlBzV1IOf7Kk5zszOqOa+x5vtWdce4mYsIylcuFBwrjTUt3yDPOyKOlz5+tbmuRCW3rtLWxEJ/Uj5vXaDqrIvzF61iKNYkBMx/OJAsSnLJy8wZr6/jzjd93WmAZd0lzI9P91HoESuIF1mjSSsvOfrw3XpwdX//thzm8wKJ1Jc9PUtGKGhoe+NZoO2vzTEWss4LCzOls0xCgFi/NFIOnN/jB4bvV4/K/N3SvtPkhOL+JRZMji727fNnA49EojovR2kv4qhsPj3XqkbdjyO9ne96rqIaVs6Um6+nCxF04y4QAUjNfZodqvvilNyYay6/zrEiROrOdPNgIbaY6jCA7o5iq6xF8MJswp+39r5ugIHN712ovzpG0VnR2x7H8TwjIsJm8sr+6eGLpyEM2qCnMdUlUKJkcqQ40LyS0hwsfu06Z0b/Y2+8vVZG773hw5m1DAWInfdk0crOm73ZdEpb0RRXXQMiUFY81p+PDWTzLpfc/sCExv2K1br0TWLN2KXjGb/0qRw8kV2982lyQeqTj3SqCzUdcGr+VneWEyvXLxxyWgiuYoI4OFkDL7xuMASQq/PHuyPaWH0tQoCMTy+7wUG9fUVcQyAMJg0BRbT5mC7B8q/PJhw0jT9ayrYMJvycElGQioM4S25+sUpmX5Yjp5TxFr464ocQ2JtDZ0tpBdKyNcRde7Ul91nqfLyQXG+vANZbuBf01nVnaBrOjvXapAZA7HK1ajm9f8tDrK7PvrasvBAUVWvFsA7Pz4ZRIPUIBjqZTLOvU1hRRkHI/KqrwCLKKvO5V3+vd8e8NVv+Te+WKhCBM/enhtoBsbpxH6wD2ZofQW3KSxl6NHtYJf2nRpPQdL8GWBLnkukW7TlKPp4/U8AqDTB5+3u1yAW5SEBsJtP7SlAMze5idflnf9mxlabYIN5DPlo3mzwshf69uddV3dA2VOQGaQAkdxIAmh7XeYlakyGt9yRxt1tJsZBAwTnDUNfcxVxcsQWRJvObCoJ6tinfFIk3dSfZo6tkxCLyM0M/vXBiW8UCIAOXF5p5XaNiWln05NwFJiO0LQO77pO3qTIV5x+FuMUHHa1PcvJwXmJugAFAmMTBFH/637L1AKs2xntrpey8aGbuYfawqZ70Rof398HBsSYa0qkd8u9OT33ek7W0HpUTRACp3n/F0e7KQrYxvwIBHEzt1kDhSXxt9f1rb4UKTJaVE2W8ics/aVaxZxSGrIbV0c2HSTL2JH9UOecJi9KqmHol3LZbmar/8N3+p74ft/1qcmjjry8AZFVZnVp8cI4jvcbrMf39wa5Qg1mYtdacHb/y12vrXQaBhSCiNNvKQrKQwK33fywgsjz+J8qs3xZ7HbX2JdZzlzsXwyCdVYpEnObuTGla6rWAPeMpTrcqC+idqka39+zx4loj9SjVnRcX3+agehbbASiEZAiiSoMzN87GA0IkWE8KktfUBGHq7w6YmdP3Wv1wNcEV75V8ojAEVftl5pUVRFXheqViNI8O99R1a17P+Skmtzw/nu/z3I4wdsGiXKw9CAZSq3kEQss9vwKwKAFDoGsD186u3koEazkMLRfSAhEiQpXZkXKzc6cc2RiCLQnEmqQiLSyg+tLKw+MHo/QTsnF+ubsrbQBuAQD0cGXy6762jWTl8wIhQPcCMzeH8tbEPELXjbpiKKsgkWj4+v4CM0FIzMhjryDCEd869QqTdRzWWwoRkXgLvznTiXgzodDPCm1aVZzHLMqRNeFC9HqPCJQIhMR505cvtkAEWD3ViSlzuHhvrL9gd9oc+sxYS8TppUcnVKw1rH7a1xMRHDkBBWHxcvon8RyOqj99pT/wCMBvq77vF5WKqiuHi7PnSU1Ke5VCE4TkbFh+O7H6YiZR1/ubAF7deSGA6MVLrxU2O89I88JoV/dO9vtZ0a8QlC2UJ/rWLnTMHbAkHoyuzI8Xc/7g+AJY22VKDB7ywASL846ISAv0oysoOWWqhYAzRIoMtJeUSMUlQ4r1Nc4xWBRriktR9M27FwqbPLyQYW7PLGzlmZ8DCIG0ko5b9XgZRXzg71ozjiDFuYWo90R+4XE6k/RUOHHyEEI2QlhjBETo3q39/puRTaoN8Y44Wt0a7dgfW3oDKlYhgRAi8yv1w6Rs9dJacaLXvQwQACDx3Eo7QUjEFd69CW8sSKD5dJqiWs4cISD88MQBS6tLLbXWBepqIlW9Z+vLqjtStOau9fphf6b49Zu0XsJFIKFf9JzamlVbJZAapR6eDwiAYjMxkRApe+BAxd6i3hXDEcRjYueXPxg6ZA2rp5OJ5A5eKXomU9jXmV3LJsdVcPFtir54JbvBS5LTl1tPhdttSnmGT9wVEOLUyDfHEsLRzMIyJXR13yeqCVqJs2piqmvpX77TE3rQT5MRQu7MmdV/cxyObfhPQPSXTT2REaHYiQrrhdK7zmtAZ++qjbtaIbKy09OvGFy++15CBQpq+n+9WXDEcOZ6mFeDhxuUYYuwgOJ8yZpN2rCkFP3gzn+IX+6UaIJYvjrQ7l50P+HunqOQetW9pcpKT5slQC098srNkS9Q9qyKhlsbxoOreyWFKPZlT42KYO7We9NLGiqTSGtZKq6Uo1rQZl94B9ouO8BjLkz1JZO65itXmbzS/rZXNrBO+aFSkVKNxpXWjs2Iy1Y2bgoDnPl86HJtf9rZcQ67V+r5Hi8gs93R0fLsHe27Jh8E91sLcBh7uFCSVwu1iNLVkud5Wn759lCDEdn03tUTQQSkIAIhff/qqcMA4PTHk2+1GGDbIFcgoiOlVstXG/Obu1ZJUam5vBBfPtqrE/UwSBW/znZF1TIm+fDmYwEaCAGyXi6oyE6Ojp89VPOUROaG/xPtthTJPCWq/qg9F2/uZZGGR7TuGgghXVPgWc5GcKZeHKuOcEc6LHhd5xrmZGOJB0KtDRArEia98ng+0fFKOgoABNeL33NidtQckeHhgf6UVp6LEgAsbXUpu+e16pd1PtWDG9OJ8gKaqXAiPZdZnHwVcW3DVqMV/2RgNY0AhhCBvfJwpbk/ru5hUXNX39q2fG7tVRTyb6cm3/EHpsf2lZYmMu8lXwKIRBOuMha+1QxENSQ8OOeXPjuXdevNeFrDJWbHmOLzYUkgClbEfzC+rydYZeCYo395o/lZjkNE3b724eFuNCV/9befPz51tqtBQL97IKIA3P3qXY+U8hIQZyqXTrQSXGidQPsJcJzcBNbsFAA4qJgPBu6PejoaP3Q87bCaf2TUZvueuXvGqfnb/Pj8p8X0939+th6+l2i0eWj3c4QEcEP4ZSbwNXFyX3vl77/btjI371XDlK5GCU4dSsQsKlvyZLlaqdcilx6kGuAlzeTtUvZUxlSj1yTe8RyHZamBZ3eh09en/rzv2jv56Tsq2WV+fq77peYIEGdwSAFRZWKq3NM0cK8SdKZ8LQTY8qP5E2VrWJpaFCbHwwLplGfHH+XyeZTLmeF3gqzRdMl+9+n+fDavII7qi82kSFtd8zTNfXls30uo1upLbZxuU4TRKwdm+/f5YAEIzFrfvTGUV1HpG9k/HR5pXe20SkIBqH+Y+Y5zVL2cPZR93p3KIiA4EGsGCWDQiLR80ZNqJAxuTLyV2EBEils722Pxs4Ej8eeJiSKDujEfnc5axb8+0b2rU792/nTDQPiFj9yx7EPshkojWd0zwmIAu7p6EoiaGYOa8d8R512vvVbX39IpEy/8Wk3Oab0xfiKQiCIlEq0zzs7wtXB/TtozDIVqO6uXHpBt5OUOQdp5plolKA53DMaJN5KGOZfflbzcQO/YLCE2Iw9Ot0R6tT6gUQDwO5Nv8VgqceqL+T8xAnpGBP47kW8TCGO8NWW9Z+x0+R3Jt3lQmJPNmZtvUb7Fk/xFyO1w9P7vWL7lo9te9mie3cu3/LcV/s1w7LU/EvHi8kcge03+CGSvyR+B7DX5I5C9Jn8EstfErNcUbajn25hZ+reMl15G9s7BxS8p/x+aoKM9pg0DwAAAAABJRU5ErkJggg==\n" + }, + "metadata": {} + } + ], + "source": [ + "def downscale_images(image):\n", + " width = 200\n", + " ratio = (width / float(image.size[0]))\n", + " height = int((float(image.size[1]) * float(ratio)))\n", + " img = image.resize((width, height), Image.Resampling.LANCZOS)\n", + " return img\n", + "\n", + "images = [downscale_images(image) for image in retrieved_examples[\"image\"]]\n", + "# see the closest text and image\n", + "print(retrieved_examples[\"image_description\"])\n", + "display(images[0])\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ufn0oqPx5DUR" + }, + "source": [ + "## Querying the data with image prompts" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "R6fNviJ28fns" + }, + "source": [ + "Image similarity inference is similar, where you just call `get_image_features`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "t1BGXpT659Px", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 217 + }, + "outputId": "53478699-5753-4946-90d6-0aa8b76694a6" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMgAAADICAIAAAAiOjnJAAEAAElEQVR4nGT9d5Qkx3EnjkdkVlVXtfc93pud9YvdxS6w8IYAQRL0kgg6GVI66WQoUTo5nu4k3enudCeJcqREI0oiJXpSFECQ8B5YrPdmdmfHz/T0tHdlM+P3R9Us9H2/fvvwMP1murMyIyM+EfGJCLz/7SAQJAAJkAAkQQIQAQGQBCQQBCSACEgAACACAjAOnKGigKqAqoGqgKIgZ8QZcIU4Q4UDMmSMkAFwQgREAAbAABAAgv9KAilAEkgJJIEkSAKQIAGAABGAwH/5/48IAMj8T/M/hIAkApD/h4KACBEJEIEAASQR+R+39SH+t0gBAMAoWAYBIAEyUBhwBRABEBBQEgVro//wD4LPIel/HTAODIEzZBxVhTSFFAU5A8aAMckYIAPOgTEABgwBGSDzvyVYrf+skkAIEhIEgZTB5kva2nkEROAcOAICACJJkjeX5P+5BClAIhAASCAAQiACACAJgFs7SgC4tcMSCND/DARA2Pp8/zEJpSThb9rWcfi/yRAQgAiQAWPAGTLGUAONNEUPoySQkqQESSiIJIEQAAhSAMlgcf4nEkP/EMgXOA8IQCJICUKSwlFRQBKBSiSBKygFcBUAANnWI22tzH8IokCO/QdCDihh6ydACCRJCgrO0v8rCcjeenJgRIIkofQ3Dv1PIACQxIgAGCAQMgAJhAAEjCEiIQL68iYJERGBMfJXy3BLDhCJQAr0hCQAKYJTDLaUmJSMhPSXS4CEKKR0BUoJXEFEyVVESZyBJ4FxQAYAwDgAAHIAJJD+mxTcNALhywds3ToBCL6M+lcFBAAiSElAICSArwsYCA98OUMGhDe3GgFACPI3kBCCb6Stf8G9fOvKCwDAQNb8e0j+5YdARm+eDkPwt4UhcE6IghE6zFLCIZASpGSBuEjwCFyXCFAyEAKE52sEAA4kifxLBoFC4gw4EPNvHWzdHYHIAYkQCTwAhsQRGAH5dyOQh0Ce3pI2REQGRAxAAuPk3wYiQkQEAEZEQJLI347gXSLpPyYgAwJCIpK+EsDgxgEhBQqMMUCFkMjfEoY3rx8AEQLzpZBzQoYMAQAJCTkgohRAnADw5gokgBQkfXFmwQ4QoUdMIHBBjKEQyJiUHJAReMAUAAQUgAgogALd4x83AGyJlAR503oIQAAgkBIYgNhSFf55CwGIwcZLCYSBZgIM7AMRAaAQAAxIgpSALFAKzFeEBHRTyLbEEbYEiHydTYFev6kawF/GlmAhA/SAcWCCAEEJqUgEIAGIEZEEFBI8RUrha2P0VCIJXrB6DNQGESAoHBQOCmNMIYbAEDgi54AAHIADsEBSULhE/kP6W8IoEP9A2YKvTgAJAQEpUNEIDBkhAYD0CAAZ+nK5pdAD43hTRaEvV4ggJUiSIH09grT1S+AR8q0vlkgIDAObgszfRV8pIqCvDMgXIxIY3GCg4EvllgViQBJQIjAAAmIoJZFHLhDjyJCQA2OAHBhHcAk5IADyYPHI3zrKQDf48rSltIiAI4gt8+SjEf+wSW6dNAZSBRQIqKS39oUApMRgG/x3RKB1GQALNiHAIb5NuGnuA3W4pb2IAjn2VZovWIGOJMDAOqHCfdWs+CaPAaFAoXEmPZIIUoAgJAAh/MX5K0NEZAwYAeeAQIyBwnyRQmCEgUXCmxaeMSQEzz8LCSC3EJNvihj6D8CZb/r8/wYihIQMiWv+saO//K2/JxLMB0pSYnBlCYkAkZBQEhH4wkiMBUIiJfp7jwScY3CEvmT4hpKTkAgiuPQISJL8pQZityXejAEJyQCJERAC0tbRISGQIEIMhBPegpiMA2KAtxCDIwEGNy+brz8C9SMC1cIwOHX/KsKWcATCzkCKLbv2H7SwL6OSCDDAdj6qkRIgeCZQVOD/QSHdFCD/dhGhf4sCIQO6aZluvujmDZfAEBgDBQM5CLAuSVL8C61AoFgQCJgnyLspv76AIjFEjgyQkIGCgAx9Q0DBmokIwLee/hPClkUDAhaYa2Q+fA0MNsNAYoAh+jiBETDYAghbhhgDmZPMP2Xyba8QUnpIAIyDFIScCALs6v8VMgQZGD7f2gIwIGIcSG6JMiIiBpZaEgXvABEwjlt+AN1cdiDmgOTrSwApJcoAIhABMQyelogwMF6+lNx0a6QXuCz+6QoZbPRNJREoIoKtz9qSb3gLDAXitAVRaevUpW8TOIAXAJ63nCEC1waBCESSAm3tb5gURISSAAhJ+lrAtwc3AYC/DgQAzgM/gwEpjClSkMKRIzAF2ZaOC9QGIiJHJAJmOdJzJSi+7vaPB4AhI0JGgIwkIQIhggyeUEog5hsZHy0SMiQgxt9Snr4VAgCmIPgAKABh6GvStzSTBMJAK4AEzhAkEQPOAICk8A9NIkNSfaADxBEApaTgCkoAAMYIyL8D4KMoCgQKgAeQjvk2V0Hfo/RBPUMUuCXfEGgdQuQAJAHfcgR8/UQkAq0jJQgJEv//7BQEV5wQpOeb2LfcT7i55pvqYUtd+bICWwLHfC2LgZWkLVWKCOw/CJnccmOBAeOBO8wYSiIpwHXBP8Cb344EJFEKFJKAgCHzgR8CSA+ECJDATbcDEJgEkAyZYhiGIgQxBGS+sQLkyBjwQIkhAQExIgippARQ7aZLhIDI/ItLIBiTvo5iwBgAgJCB1DCOxAjRx1wADNgWhMf/D4L2dfUW6PYvHwMiYkjo2wNJ/uW76RX6HgFwlAKJ+XjKx9eAAIIICKRE6cu6R4wBU0hK/6IhAPC3vCcIbHfgtUn/ZhEQQ2ScIZAMXP/gnS0XDH1r4xt3BABC3/IyCUIEYMW3d4H2/o8oioJTlzelit4CMcGiBJAMNtbfHh97+fvJWYCffJXvm9eb1tZ/CR/pYyCsgoAhCA8AUQqQAkmij9N9WyEFBz9W4AJ55EmBhIqqa5wxUtORTAi1jmnZXpehdIWnca6A4gJoqGZCMcXxQAIpTJLCiAGXgQPJWOCCEpEkyRj6DqoftgGCmxAbAaTvnoktRw2AALiKUhAyQEYSwf87VNC/LAwD80sAnBED8JEvcmAoAYDdRIYAjAODQE36SHoLaSECEhEBCkFyy7UhCb4N4lthGD/gRGwL13HwlVlgGoNbtHVT6T/gUgxEzXc+OUciAoZIgez6Btt3a+QWVMAto+ZbMCGBCLwt9bNlit8KHAov8AR9BRmANQqMXKBrVT8M8BbG4sqWzw+EAJyBwrc8X19psUAJSAAFwJPAJBADIQABPA9IABASYRCTEzzQT4RSAHlahKcZKB2ng6DmE/n+/AAjdCw7l8pGQ+HNSrnZqiIJReWayjzbqne7hq6loxHFlSAEOh6pglSOKgcpgTM/qEkEJH33PsDavvQH6MK/vJLIv3OSQLLAmpB/5gogBs8PnAC3NKdviQiAEQJytmX4fHeJABltiTIC8x1KQCBUEUSgD27aBP+uM4ZEJAUFHg1g4N2wQC9KQaD4OAFIEAEw30UNHop8zwh9oErog8MtWAkk/TsEBIASgogDIgJJIvCRZSBb/le/dQWEBCFu4uhAS/l6JQCTCBwAFEAGwibXJk+CogJX/BvmK8itpSL58UXGAlDv77CCwLlv4IAhSiLfXgd2kIAjEIKQ6AJ5EpFQEpBABAJJ5JHnCiFAuCgcBkLtz0xuHz0AgtWqlXQqOzYykUoluo1Wq17nDITn5XPRdLq/26l3WjXHagBjYT1GKFwhFdsF8AAJHS5VBVSFKShVhioHjkj+RUUgkhLQ92f9SLEPVnwDIIgkkpSMpB/MJkQESYwTYwGuZYhMAcb8gEIQSPJllDGG8FZwkiiIWTAFSJLv3wVxZz+GRMA5Sg+AI0mQIBmS71UFoJ5jsDIOUgISSOkDaJTSB9lvOTsQYPBAzSAFrrukm3jbV1pMBtgtuFEBwgiAM/nIPQBS8ma8G/0IhqC3wk5SAiIyROAghRQeAYB0IcRBUwGA9eSGDC1XqS207U1FIZLgEUkHOAdUgPu+KgOGoHJggFwBDsQ4cAruJ/hq11/4lmBJAZIBEvn2UXgkpe9wAZOgoJFPDcX03pAa9zyugj42NDUxNildr9vqImMaVznwcETRQjHXcgFJC4Uc1yquL3a6XjiVyebzqqJb7RYxR3E8IiDGmMqAPCKUAklIEgAcGPgpEQAp0d86TxIScAUZBgaJgASBH0WUHvi4BCQhA0akcEAJyEDl/rXzPUfiDBlSAKIBuAoMA4jo6zvGgCEQC6BYYHQEAEeUN3EeBEIMAAJIBM7nWwZFBrgLGEoBIIEkSUKSgOymbqObPjxuARo/FfFW1PGmHt7yyKT/41Z4O8hHbBloX7H4MSbpYypEIHAtQgGcmCulKYgBJKI8l5k21KzTtVrdctusqzobG7593/gDzfrm8fNPW2azkB0I6cbmZrFUW3HtikdtyW1FRY2Tr9E4ADLg6ENDZDxwV4mIGMitO+FRELV3JXguoeS+3icpQ2psrPf2HeP39maGQ5rhuB6RCKkhwzAUVXESVr1cLa2tt5o1AiKGmmpEIjEpELiSSg2mUsPxVCaaiCKC3el0u23FFcA4IJAngTiSC4pCwMF1kDFJwSVFRBAEEoJsDwveBfKdgwDT+5A8cBYUDpyjJ0lRiCOiAOSkeH7mBDjfCiWQnzzxUYIvkcjZVs4BCQiRE8gtj5KQfLy5BYGlACCUAoAoSGgCYRBuBT+UhBJQoJDoeeRHyUEy32oQEUnwPN9sAuMgJAiP/Ayp3DKsfv4kiCkwkIIkkXSkJwEBuAIK33Lsg4gPAAPJQfpxL0bShWgkc2TmQ73JgWsLF2t2JZvOxkLZfGayJzuoKrxtNtrdenF9kQGLRVK9PaNGtEfYXjgUCilKtVwx3Va7WVspzpUaFySWI7qmQoxzJOYK2ULe9qTlelJ4hBzUEKgcQAk8Sj87xAESoCTUEGN8od5daAlXgKKFe6K7J3oO5hK90XBMURhnzHU8t+uQK0O65pq2sFxOyIFW1xcbrWo6XYgncgjMkwI5NyJxReWOawopyHNJCsW/hcKPWEjwkDxCRwBDRA8QSQLzXSEJ4DqBat3SE0hEkrYsTpAbJv/APESukSJQSlB8RwxBMlIU8DUTEPpAAYGQgHFkRMiBISgMGKIQW2BpKyXjAz7yk5tEJBkJkBLJd74AbwaOAxjuJzoFAYAU4LkgAEiSkFI4N9EVAIKQgBw4QxB+2GzLqSYkAcB8JxeQQAopJXAEDUFXs5nkkG6kEFSOoUS8oKm6Jx3bdW2zVW2sbrauWV6NpGSoWF0sxLaN5vYkIvn+wwfj+XQoHDE73Wppw7VdxFAmO9KjqUMDu1u1uoIhZPrYxHbbcsj1nG4rZNguSKAmR00XBYCkLhOZeC9XVNdxhNdVVAYKCOkQM1vtVWhtxhMhxlTpuJKbHqvHtWhB7/FamkkQNZRbDK7I1WLdi4cnBlP7o0qmUzc7tSWQnuu6nu3pqh5JJhoVRziOETLiyaRq6KiG+kkY4bgRiwHyVq3a6tTrpbXq6pKWNkI8ZLCQpqt45GFERM4AeRDk9SWHKcjQB9d+0oo8L0DoyAgpcIWDYH/guRNuRVAAgDPkDNQQKJw4B00FhYOioqoSR1AU9HE997PinJBQUYATKAqg76cEEChIHvsR5CCJ5jvnEj2XpPAzOUBbcNs3pkG82UeIgglBrisBQNrEWSyR3JYpTOjRFErRqK0urZ1pNVd0Droetz3bdG0pfZFlJIEYEQKTkhyIhMLZ1CiHEACbnLh3YuSQHg5bHafT7SichOMJkuFIFJE21tfWV68JMjXNQGBAWipTSKez0lOiiaRUpWs50nY79Wp1bcW0zexAL0hpd7qEqKAaTcQRsVauNirr9damQ6SpvNWomraTTPZm8+PpfHZzfdm0NkenZ+LxnKZEjWhEU8Nc0TuddmV9RXpmPJFqbK4XS9eWimfrlRLDcK3RLjZbRpjdseP2XG640xUMI2HdYMAApRJiESPMkQnP7Zpdx3ZU3fDzK8C4wjU9FInFMqF4TI9GNa5Jz7HdbrVUrVcroEMylozqYRbieNuDWwmUrZizlOi7/VwNeBk+1vbEzeiZ9DMbjG0BVQl+2NPXN/7H+Y6MooDCSFGB+4lFBRUGikqIAfFGUW5SCVABUBhwRPITdIHf5UcGfauEkuTNqI+UKASRROkFMcOtyGvgekqJtMUg8Vz0XOG0IJnctv/2XxyfuTeRzIY1HaRo1mqV2sqZU1++dvzb7/7Q3zRb1W/84PdBlSBReiikFIIYA51DX37Xgd3vG+vfUy8319fXsj09iXQuGo/YpiiubTQqK4rCND1EniAp2+2mGtLD4WQskdUMLRQOG9FYyFCtRvfGlTON+ioQWK2u9CiTznBF6Vgd2zaFKwAAUeFcsSy7a3ZXlq5U6iudVmt4dPeO/fdP7dydiCcj8QRTYOXGwo3zJ5K57ND4eKVULm+USEKrUy/euNps1vtGpnryw4YRBhRzc7Obpc1sISVdp9VsAYbSqVz/0IiuRTzbq1fLrXq5a1eMsFLoGeRalCRZpgnS41xFLm3bVFUlrMdUzRBCtsyG53nxWCaRzSWzGd2ISscTQmqhkKZzpnFFelusHALGbqY2GRJIlTFVSsaASJKUPo4BkhJ9aCK3vHEhAVxABooCxPx0CggghRAkAUMpiHFwCLlCnAHniABcIYUDIijcZ/OQiqgogCQx8O58j42QIYlAd/lkB9+H9zFQwLgiJAoC236QQfrg2nfWiKQrFTsyveODvSMP9A7PpFO9GCLGSOEsGUql8/l0uk819iQLe8dGtWde+tJa9brOiLqgq1o6OZgI9xayU6Ojh2KJXpdCsWzUiPc4rttuuc3qJiBZphmJJaPxiG12bceUDulqNJPrC6eTqq6WV1fnr593HS8SjatM7XQ2SyvLjmn29o9HMwlVDbW7LakoudwQEghPglRVVY+lM5qumd37SpsbKNRC/3gyn47GNEQsb6x1Wo2NlaVOu6GGY/NX5xhj0VjK7tpOd8FsLmXTfelYstNqrK4sZDMJ6bXtbqVVYX09fTEtHUlkekZGEplsSFE91+l1+uuV8vrqtXaj4jju4Eh/JJ7yTMdzTMdxFc6JJGMsrEeBoWmbWFXb9VKrutZobZaK0VQ8l8hmIpEkAmiGoYVDeOAO7mdvGAeGSBKEh5IYCSSJPASKBsQ8KQiAhJ869FO76OsyH8egFOQzBRQVGA8Cx5wB2+Ll+Yk5riAJQH6TT0ecIyIxBM5AU0BhSJIYDzC+b3AZY0DEGPp2FgEkoe+aBbkA8slJiD5QJUC6mVzy06DSadD42EfveefvJhPJ0mpdC0eyfRGSyBEVRTJknCnNri0ti7nej5/77JkzX2Z2NGFsn565d3pmP8OwZ0qn3bWkHYrF4sm4ylUScrO43qpurMyfWpy7ODp+ixHLNuut0emJbG9+9fpCbaMUL+QkUWllqVIqxlPZwYmpcDTerG9uriwoIW10YhtJmrt2qVJZBrsLhK6rjm/fMzazrV2rbBZLRjI7NbPTFUAK69RaVr3aKa912m4kogun1uluKuEI57luq5ss5FP5ZGll/szrT2UGxoYn9zQ2GpyMZqvB0as3ajyk9Q0P53JZ2XEEMFUJKVwJhdRmu44cOXobKyub6+uJbGZi375QWKusLpXWlh3bSSTymUx/Ip1WQkwIwRljwIi4FMyRrud27U5HeDYhi2Vy+b6BWCqlgOR+nAOJCT/sSUiCkQQp0G0BV0CLqMgkgeSMCc93vUhKP6RFFPAW/CAjeSKgsyFHzyNEQoaMbsYstwyln+f3gw4AioKKAjbbeh+JKwGXgyvoJ/H8dxjDmy5CwBQFIEFbgTHw89R+8ImIwA8rCAqFeqZ2vSedzsSi+uK1kkCKRtIXzq61KvXRsf50IU4oEnG90pZWw86oO3qNt/dNHJzZdSQSMdQQMzvd4upCJJYZmp5QNdas1UpLy/X1DdO29Eh4falhm8mRbXemC5l2rdltrZ984YW52Sv5odFo30AimU/3TGp6KBaNd6xWt1GxWnY6Xcj25EHY9XY125/lXNbWS/FkKt87EMtkO7XKypUrLrLx3XusVuXyiaPt+mpp8ZontInJW0YnxpPpQqeiXC2ubKw1wnFIZpLdTrkxW3TMds/QtlA027W9wfHxTKyn3mnVGmtqkXWqxXMnnomEw4V4oV6tKkxP5nsMnYPuVap17slwOJ4b6E0V+uy23ajUOg1TYdFQjAMojUazY1mqjipD8ARTlVA4rocTRthgEKUkM5vVzeXF5c0rVqfbNzKhSI/5rBUhiXEGiML1xYv54MkxSQg0Ypypkog4SCCU5AEhQyQpGTI/pogE5AJwkB4SEFMAfe4DgvQJTxzI84NF6Mfig3AdA0UBzkDxcxQKcEQfhyEDxokBIkPFA/S1IAdVBc7B91pJBsEPPxSF/psUAD0gCQxcE6LGeDzZSx4JR9tYW4tndc8ZW7l+9dLx1+Lv/YjremoYPMu69OY57jiMGfv2fnBgaNRxnU6nJmu0ePlqrVQamtpRnF9pt+shI1Qp1jsVGNm+XYtHHhza1q7VW63K2Yuv2K1GeX2DQDtw6327b70NVLVW3jzxwg+X5o4xpsWHto1v357OFxauXLl6+piqedVqK5rOptJ9U3tuT+XSILv1cnH92lqz2siPDpx745WNhWvFGxdV6ezbd4uWKLBwst3pnHzlC7qhJCd25XiKSCfVA3Rj0Vy4J8JU3mmYAl3V0Ddr5eLSRmVztl5Zsh1Tj2bS2b5Mpmdy5x3I1GpnY3H29Oylk65rG1pYRcX0zGRPz9T0Lfme4aHxyVg0CeCTikS1XGzVKqFEKpntRU0TIJiqRLKpkB6RgGEnq8azpcWFRqUCKBQiJMmCay8CBhWhnwgLBMO1JCFFE5yhBBVRchJc+nCMcSJCRgwVEiT9QCUREXm2RA5MYUTkZ0/Je4uaQgRMoicAOSKi8IgzdDkwJKYCZ+C6gSvg5yWQAQPgHBQVEMkToCi+kaUgMwNAfjibgvi5T7BFQABu206iZzoSSbkelDbslcVL21ITrkuNcq26vuzY7YWLi5KaF147LiTsuvXQ4PBYPJVcuXZ9dXElkojapoM6jO3dbYTjzzz+z3pIPXTfo/0Tg/E9GUc4ltW+fv71y+dPmi2bRyI9A327D2zfsXuvZbsXzx2PZXqNUCSe6hnaNp1IJLK9O4b37Y4bKT3RUxuZAHSj0Wwimei266quabq6OleWanrswBSCXLx61na9HXfcffDu+8m0OrV6pr8n39f/7Lf/6UevvBLNZ3do0XTSMnhC4aHegaFC34gEl6OSzauKpoXizOpYPQNTRuQelYHX6W6ur28UN81WybGr3Y7pSTedzGzfdXuiMMCk0NVQubi6sTLfrVarkmqrC3bbZOFoIh6OJqOl8tq5M6+lE/GZ7bdHsiPxdCacDGM4zJLRaESTLiULuf6J0frGRrtTV/yAXkAv8rNWLEi/o09bZEQMXVN2QUYSHFFyBOKITEjvZh6apEcSkTGSQqJERkGcXroAjEkm/SwKcqItegJJJCQ/i0cciICRzx9E4S+HByxvnxHFGHKFFA+5Ap4AxhGBFIX5HgDjGATDgji4j8SIEXM7jtuCRCyr6WEkZWNhYfXKmd27J7midKqlCJo6eD/89x/EMvr0rr2jU7tURVhm4/zJG4yxVD6D5CFHrscjyd5kOh/PTxghZXhyulUqHn38B+mxsVxvwbUjt973U32jM2hJRKe8sXDt0pVypZnKqOFopNAzrmiR4lJSU/nUvtvUiGpbond4oDCQc8yG1W6aZjne02fo8bNvvGqEI/tuO2h3awuXzovORkhKarRmF+dXludts7V9z45mva4mRt73sU/3jMwYkbCuMs/yOi2XJFTqq+GYxgUXgmtN7rZAT+iS29WK6TgekctQFHoybkpF1QIyV1dW5+YuebZXrZaMeHJsYm+uf2p6520EaFomCUfLk2CaK2zbJSXUO7LtXtfy9OxMz/RoMp00ElGmqJ4kEDwSVViYUzQeTydt21YQAsoFAoBEIQkZ87UV+ulRBkzhKLjT9RinSIwTSiBgxJhCUgZcOcYDu4RBChEJUW4lWISHPmM1iElz8NUaUwAIGEMgFIIIiRhKDMhbvqcZMOkAGQJX0WXAFVDUgJiqKKDwAOwrHNELWECBCgN0TDcbvR1rVqfr1Gu1jO5dOfWD65eO2db7G5VWcX2V6ZGLJ0+iGr7nkffEc/qFU2ftdq1naFhKrW902HbqV0+9Xl6vtbswMi33Hoq//b0/oWk0d/HU/LXr0/sPFoZGgfPbe7abrcrq0tlGZc22pMHE2NTk7oP7SSjJgXQsFXbN5ibEcoW8VW92TDCMUHlt7uQzj189fcJl/L4Pfmxk18Hq+lrY8JioLF04cePKmeLiCvNwYGzSstVs7/j49v1aOGoz0zHViUN7Uumka9rtVrvRtMxWB9x2p2tfmD957vyrtWZb4/jOB9934PCDCaYCkGmKVCbhic7qjcWVK0ulci2WM3K9o/Mlw9XHB4YHR6bH4rmMlKA4PJ1L6NGwBHK73fXFudLGium54bCuhFjvYF88nY+GU5X1yurly5ZjphK57FBfNJNpcVYqrrc2y+hSJJ7C3fsjzOdHok/u8/PJLMC/wG4WP0lJJIRmoBHjjIkgcwZAJKW3lTALcrdSeluFD1LKgHgI0k+wsSD/4zNqmOLzcPwgGoEv2EGyJWBg+lAJOXIOjEvGkXP0S9AYB8597xI4R99z9NP+nKHnOjE2/cuf/PG/ffVPl1bXDt3zwdZGbW32jera9W13PYo8tTB75sjd9xDp8XSua7U7ncra6trw0M6BodGegUK9svHGcy/oiejktl21TSuaNgbHMq1y+/Kp4yvLm/e/96FcNjZ3fVGh9PVzbz713c+trV5oWY4p2QP3PDo8MnL2zZMPvOsje+86VCquduvdZCadiEdWFm5AVFdCtHDx9JXjrwhb3PHwB/fc+bYzx16/8PqLjeJqoq8wPHFLJpPvGRpwTLfWrDXqDavVlNIWrme7XRBqLBYfmJgKJxKekHrICClKSBXdlleqFBdunL02e97gicP3fHBy904kxbKa3U5r9cZCrbrJNNE/NBbPD8UzcRCe2XYjMS0SNdATG6s1VwgG0G5UULjJfLa8Up2/drHSKLXNFkrTdjq1ci0Siw72jkEXhNXQDY7MSIwMJXoHNhfXasU18lrpbE8q34979kd99mOQyvV/CPiObCu2jn5CzedmqQoaMa6oQFKgX4TkKyk/hiSIJEjhx46kz+/xKYDS591BYA1hK3XtizRTEFngoOJNxhxs1d4gACekoMzG59lxTgqHLTkjBFA4Mo6MEWPAGXPb8PY7/tvdR372f3/mVw7cdWTPwXtf+N4Psrw1e/XcyauXUAl/9Bf+y/DU9mtXV3r7hjwhBod7iytFQnV4YGCzsvz0E9+//YHbxyb3VDeaZhtTyUhxcb60UUkWMtsne5997usvvPz0uz/6a9Nj937x//zaE89//eCuI1OTe3bcevvAyNRrzzyfiKk79u6du3bVsTrhSDidzXhN+9z5k7bjhsLGoTsP1Uprzbp7y213nT55yvbcyanJZDrRrtfWVouW0+q2291GQ1W0WDgejsa4igwplY4bhuZZjmBGPJsBVJZvrLabrVwubIRSZseNZeOhSDiezumGQgSex0ERnusYYT0SDSNhpdxsNjuReCQaDyEjxwLXVax2y3OdVMJYuHrt4tmXe/sHxsd3OR0CcDq23XItVUrwHAZKS3icuIKyvV68vnCFRdTe/CAXSnl9XrpWF4AbEIlEFL82CAiQ+4w9DKgxW2YLt5hr6CdtiUlPmE2pGahqCirA/NQ5k4whSUIAAcBRSiklAEckSQQot2juJIAxIPDLD4A88qMRnudjbr+Khhj3cT2Bj/e5XwZDRACCSQaIIBE8Ig4APr8GgIA4kaIAQw4CJkYOPfyux7qVRiqRuf3uuztd5jRrZdh8/vQJq2s/9MhDgkdX1psjkzt6CulLx0499fobt953b2Ekd/zVZ7751c9LgD0Hb5m7ciOixxTE5bnNqGocuOvgWm3xt3//4z8+dnTv0EwqNQ5cTm2/6+Pp3e/80HsiGQPDuLq8dvc7H05GYqffPBGJZjWmri0tnnzjzWQsPrprdySsvvrqU//yxb85eNd79x14qLheG9y2M6LyuUtnTx9bVBCNZG8m1zO1ra9WrXi2HUvm7I559cIZz9y86tLG6jUp3MLgtr133e/ZEGJqf28f02U4ricyYZfAtCx7rRpJ56LpkBqhTqMNrtOo1tdMp9sVHimJJC+WV8+dP722eJVroUyyf3x6x8DQ8MnX3vzuV/8qEo1s++SvaVFDQW/1RvGVl378wtlnWq2S4rlpI5TP94yN37Jtamcqlq53Nl8//vJE32AynEOmKUjNtk1gGjrinv1R5ntcIAEQGQIhSEYSEQPk7OucLR4TEIAQgiSpKlcN5Jy46kdK/Sh4QPKUQkgp/QQwSRRBAtmP3ftkXOmXBfvlU3QzW+Q7eX4VNWwFS7kPtJCrGLC4OHAGTAGu+MWcxBgyRL8amzEWQtYsJ373U/8Uj/X/7q/+54fffn8sOfjEv/zdjbkrBx54Tz47Ho3l+sdGp3bOKCRe+uGP7E6zMDikaAlbmJnehLnZzuWGJQtlBrNep1Mubaqcb66c36zN/eDJ712Y23jo3nd+4P0/VxiZcLodr2vF4lGuayuL68WVxXg6qTBe3qzFY9GJ8aHKWrFeq9vQrdfmK8tr5y8d22zXP/7zv3X7fR9YvHT+4itPrs1f2VjfyA7vO/TgQz19fQ55o9MjlY3GNz/751zUc8N7JU96jHK9SjqdpxYHcjPJaKIvHY8nO2vLpcXlYqna4s7otp2RcLLdsIxINJfNqZpaWt9ol1Z0FEYkaeTiHcf1mCu97tyFs/OzJ+evHC+VzOl9Dzzy2GPFxe6ZN1/P5uJ33f/27GDvxsYm2C0uAARtbiy3atcbjZqRzGXTGbMj6sUiA7xevPj0S0+udwXn8J63P/Suhz8S1uJkessLC1uChYi4lUkGRGAy4BnfDG0yZL7VQ0AmhQAfPhExjlqIqSHOOAXl8dIHWiSBAsESPhlBBBxLv+gPtqi3QIGiDHg6foJJos+pZ0BbJYqIyBgLgqU8+Afks1aQ+WlvdSvWD3JjgT7x2F9i2/7zP/2D/Xt39/WNvPbGGwNjh3/l9z5zfXGVS210dNy0ak995xue5B//5Z+3TXnuzQv9o/nRmbHFK6VOx873ptfmb6gxI55Kibb71c/9wcK143c8/KH80J69Bw6Fk+GOadoNlyyrWqrGs8lO3YxlQ8wF5MrAWM5ut65dvXz5yrnVhSXDiGV7MslMwrW9bCIyfcfdS9e7//Lnf6LK4v67Hpi69ba+oXHb7K7PLzSbzcndU5yx1ctXheV6PLn38K0ITq2xVN0srSwttmrt2mZ1uXSjvDm/WV5LKtGPffCn+ya2dRqdttsETZeekNX24tLSytq1TqMUUjlFk9lCIRfPp/LpsKFE1LCrYrml9PXOjM7MsKjidD1NsnQ2TGb79KnTm+XizEB2c31z/sZqs97otuoNu+EZfPvYVP/QNIrG3Okz3U7d9eqXFq62hb5v990PPPJwiGKtSrtUW8Pdt0Q4MiAGSIz74UomxVvW0OfYBaUTDG9SVoJoFZGUxIAhAtNAC3FFBfSpu4GK8hnDkggkSeFJIpBSAoktMtoWvc4nPvn6zKfVsYBcG5Q5YECJQQSuIPjYnyNjxDlwhqrCGEOOxBkxApDc2cDx3nvJkavz1xbLjWgs+egHP7Vt/yECu1kzd+2aajc73/i7r20/tP3QXUccIbgKvdns+nzl0vnriWR6fMfI849/f2npwod/9dNOi7c3G8s35pCLRG8201dAydsNG1SI6roqpOs4ud4MY5rrdY2w2mw1T77+5uVTJzji6NRUvm8kmsnGsynLdF944jVVlw994I7SktuttQZHClyjar2mhWB9bmnx1MVE33C2L94uVlfnFof27Ny2f/Ly0Reee+lHF2cvCqmPj+/cu+9QPjfqgHvitVdWls9lU+r7P/Du8nL76R/94PzC1ZCr33bo/nf8xPvnl250uq2EETr6+hsvnn2zZre3eEJgAAiAO488+rt/8H96Csm585cXLp2vbNYNQ7k2e/K14y/WXDsX0g2u7N61/+Bd7zb08Mra3MW5C1RuTOw8tGffLSHbcx03mop0zCY68XC8jxndyka5s1lCbuLufZEgU+jHSQkZ50DMr6v3y0/92IPPs9oiwEu6yTkH2CqxIgBUVNQ0pqjIOAEJ3yskkiBJeH4HCyKSfl476MYRiJQM2OJbjGDpY/aAhxPQ2/1VBAXTLKgz5gw5Q6YAA+BIDIBJdLqYlP0D0d5LVxdGdt2xc98dVy9e/ugv/3q9bgM6w/19sxfmOk5T08IHjxzYKFeZgrLTmTu7EI4lescKAB4hW7p+dWRyNJbMFW9UOq0uY8C5TPWlwzFdWKQqisMcu+Og9KwqD4WUTqOqxpT5xeUnvvHlhx+5Z2J0zPOqb7786tJy7eHHPhLvHWzWvVQ0amhqrd7QwqFoXNND3Oqyy2fP6zGW7+nr7U2HDe3UG+dIgh4KJWKxYy88++PHv6GE47fd88jM3sPxfFKJQCoRDTl6s2oXq7VvfeXvk5nQ+ORhSYoSkdu2TWupaLXRjIc5b9SS4Uw0lV8tLr386oura3OKLtvtbqlUC+tap7HJbDk1uatWqy4uX9YikYnJqbZtLq4sb2xudLtdT8oYh4FcIRqJRMM6kBD1ev/wWM/uw/loX7u8HNKUeCydSqRDRkZhHDnUSpX1lau455YwEAPArSo/hsQZsoAXjUgSEBmR9OvXbh68XxDhoylfBCjw/YCIGAeFM64AV/xfkeSXbgWFUCSlpK1yKp+cQBIAiSEhMkApt3qjEJJfW0G+NsOgHsKvhvCdWc4RgDgHBYEhoUASvFF0333Hu2vrjYVN8Ut/+P9SqexzP3p234Hbm2V3YCRrWa2NlcrBe6eFC0uz1dxgbPX60qWjN0amx3fcsu361eNLxY27HrrPCCmdtthcq6Hg+b5EZb0e0nkkEXPJEY65cG11cXFNB5zas0NlkXSKm01zaLrvz/7kvy6++fi2yclTN2ZXFooj4zM/9YlPDc/sMmUkFNIVB5dmK4UBJTsQk57cWKx4jhBoxbNht+lapl2ab0HIm947fPHsVcdq9WYL4URvNFMwIqFoTKWQo0eFommlhZLboXAkur5SCsd0NRKOGWEAZIpXL1bXbyytLVy4tHjRtUQiPTQzMT483Dc8Od3bX2i1qmulerfjdeqd5YU1bih9hajqku16DadmcDuiRLxu06w3O1XHJX3TA8a6wrNNs6043dX56xsNM5MZf+X4jxeoFQHIRrVEujcfSqUSqcO335tOFvDAwVBQ+Yp+QB2JOCKHmwXSEv1D3RKkrQoUvzh4q2RHyqBczK/b8f1ACRIJ/FAT4345gy9Jvp6TUkrfYvoRCZ/xzjgCSKbQFjNhi3Ae8B1QblVXk3wrRuJzvziAdIELbrWgU4Z9o9uGdt6/+94P9g6N/vh736gW13/io5+ornUk4Oj2TDgSWpkve13HcRy7Xa2sFLOD2/Yd2fv4D77y+T/7k9/8zF8cvv/elRtl6SIH1HVWrzfCMV1TcG21MbFj8MKJK/FEqn8o2miuLF5dnNl5pLx+4/vf+tufeOw3n3nhm9//5udyyZHswL4H3/au2+67Qw8blVrXsZyefDSeisRTum2Zl05cURlqpJp2Z2W5lMxmhBQMef+2PDjcbG9K4tX1RqZvMNuX7dTLpaWLC6sbydjA0ORktWlaZO3ePdmbT59747wW01L5+MKl+WMvvnH68ov11aVWZXPdanUgyKQwAAcgl0zedfvd73z00dHRPWE9pUhwHdu0O5F4lCHbWNmYPX+a3HoilVbDCnioUkQKzfOIyLPdDihOiKuOEwvF+1Jx48bClTMXX7l2/dSpa1c7bhcA0gr80s/++tse+CklEhdCgPR8poKUAoBUEZTdBFWQQgJDJmkrJeeXeZAARM6CihDmc/olSpBIPj6TjPwGTtKzyaexc4aMM0AgKYXALRcywE8IvlQRcuJ+Vy0AVPxgFgQUUQr6iyAyKYM696DvDSPhEZLWqXrUNcI8ZGnDR9794aHpyaUb1bNvvnnb3bdX1qt6NDY42sO53FhtRYxUtKCuz6+02u3CyMTOW3d84yt/+td//X9/47f//MCRe1fma4lkknmyXW0sLy2P7uxXVW1jqTa2vdfxvINH9m0snvv93/n5c5dO7J6+/c/ufuiVl1avza+07K6mDT30rk8/+oGfGJmZcD2327AQ5ehgTI/wTnOzeGP5/HLT9cx0PlauLr3wo2dyye233v/gzJ6xhieAUxzc1flystB79uXT+cxwJh5pLN0oLq5cn5uTGiV0FklEcr2J2vr8yR9/9+Kxq5N7D0/snmnUGrF0atehezN9fe1qU9VlcXOz3K21KuvltWLX7Ngh1bI6jz/19JNP/TiVzU2OTo2MDXqt+tWLZ/v6hqZ23KMZjLuGzlN9yWFmhDq2HYkbIa5Il7er1eWrS1cunC4Wr3Ta3dzg+NDgYDKVG80NbisMPnInnr14+vTsKdNt1zrm0nIRH34XF0LQVj8MT3ACjTyFhKqgDoyDRCGFEJ7nSOHzhQEAJSIiQ8b8KgPGEJFxQRKJw1ZRnd/3QEgRYCsZELgo6EUkKahfIEVBRfF9PT9AKhmTgcLkADJoybLVdkMCAyn8UmwMfpmjcIlJvb5kpdKHUpkd3W71Y//pd2LpXDKTOn/84tzZ8+/6wDuuXymOTA+HFE1RUNPUVqU5v3Dle1/9y6jR+/O/9yevPff497/z2Y9+8o8PHDriCWl1O88+8Z3Dd97ueRCNhzfXu6uL9Zk9Y+GY3teX7pbr//d//uZGqzYxtn37rtsfeu/bK6sbnildFN2WuevWHYLLZrPhdZxUPObZnfrG5oXjpxdmLw2Mj+08tLfVsYorpXaluu3WHbsP7KqslErrwBRDceonnn86l9E9LyTi4XBK21hcqDU3Ytkerxaav3p+cHR0dGL01WdenLt2NNHff8d9HxgZ3uG4dv90wrKs9aVGy8R8Kp/MhqNRtSvrIW4oqFDXw5DqCntxqbg0v3Dl2txGYw3sVmezy6TqCdMDQfamISWB27aMtugItFTm9iZz+3bcsevAbalcwRFup7Y2f+74lVPHYwW93GjUK7XeQk9/dljTM56U65XVpfXFmelb8MM/nXBc03MFMh8+qYg6g6jKkgqPMaZLIiE84Ylu12y1Gman5QkhCQLQA4SAXFEREJFLQchxS/woIC77hTBS+mZQEpEnJUlAqXLQdVA04CpylMiAMcEVQiBFAQDinFhQcriF4QKQh343K47SL60hSaqSXL3e4ezA7/yvrxx77Y14LH373XetLKysL60lMrrGjWQqi6TGkjFVwcpGtbJZ0xX1X//h/7x57Mn/8sffOnDb4ae+98Ncf/bg3bdW1zdT6cw//NUffOvxL/zBf/viO9/9yJVLFSH1qfE+02ojQybkM99/bnz35MSeGc1QzrxxcWNtbf9t242ogcAK/dGV+Uq7ZRVy4cpqZeHauoK2EePZQk6PKIvLS0uXiv3Do/vuni70FK6cv/zKD5/u653ID4wbCfHHv/7z566dyCT0VtcpWa4pYDja8973PbbjwP6zz1+/9cgd1foaGmoumyAhtDBeuzhbXzKH+wt6SlOiSiQRjsUjGo826pYahlCceZaNoBjp/pCuU6tjxAzFCBmqoYWYEJ7ZsK1KxxIuMySgbbU8z3Zsh6qNorBM1/Y0TQVAANkq1SvFTQNkNByWagijorKxRl2hxFOJQo/lUSIaiyvaRrHebrfw9z5zyLTbrtf1axIIFASdY1xhCQQDIUSAhFK4ZFlWp9uu1zc7nbpj20IK15Hkl15yxcf4SCD9Fj5bbVoAQEjhe4AEQFIAMYagqBQKKbrONJ0Y9xA8IEIkpkjOJeekKUxTmaYRZwRAvuITIgh0CO9m2I04Q69LKNPjY3ctLMUe+Ylfe+2lZ3fsmxnonYkYqdOvPX/u1NGP/edf4dxoNTuGHgWiteU5TYsSskw6/tLTz9UqjQ9+8ufCIWE2PGKuHlYbZScWM770d38ZjkY/9LGPOp6rh2PFtdmTr7w6Mbp7cHr6me8/u//eHYOjE+mecHW9/eKPTxy6c7x3pE8KVwhyTLtREqrZtZxau910eLfbKnW6zO3IZCYxPDE2OjUohfXSU69dmbsWxdT2A/0xntBw5NVXv37+/DPR4anZ5XLK4JOjwxMj26fHttc7rlRRAV1U6+nhnJqItjcq185cAO719RnEWLtaBSUazQ4nkkm7Vpu/PF8qdRI9obbdrpXb2YT2tve8o39im3CsG1dOQL1sRNPE9EgsYsTCqh7jmsEj6DpmveyUVteBmmo4ur5U4SGlJ5XVIoZk6Jlms1zullt6JJLIFjxHdM22WRdty8sNREMKcy2buWa9WC6uFfFv/uZnbNf0PIukK6UkYiRVZDoDHVAl4lIiSem6nmWbjm1bZqfVqna6NbPbtrq2aUlJDFDx2x9A0JsKhZ9AxK3YlSThScZR07iuK7qhhEKKFuIKJ84JmMeYYIwIbIUB56CFFD3EjJCqKpwzgcwV0nRdUwgpPSEJpBd0xfQc9ByIa8N7b/29J7/7wgd++lNXZy+9/OwPH3rHo33DM325odmz55K56Mi2bQpnnaanhyOf+4vf3nfb4SN3vKtWbgEJx5Q8xNKFsOeI0mornlAQlXa1k0pFpKtwxuudRqNe/8Y3/tfRE0/tHDvy8Y//lhaOZQpxI5FEDRNxvdO0YrohXFAjLKQpQNAsVc4cfe3S2Tek7ESS+Y1iPZ3ITc7s2HbLdLwnNnv23PM/fMFqdgZGdh46cktPX8GqLyjRzOJy57lv/9ORBz+489Bex2tYFiZjMVTslt1da4jRZATbzZW1qyefejkzvH1s1+RLzz1x48bs2Pjk8MCEdIXp6el81kio6XS+p7+AKg/pXOPMronSanNlYXZ57TKxTgT54y89t3rjIjciUcPozcYHhydHR/f0jc9YdtVztR3bpjPZ/Ppa5Xvf+O7LL3+n2ygPjIweOHD3xM5b+gq5fDYJXK1s1NubLavVUiCsRRCt7ury/OrSXKVRLNY2DQL816//oSdM4TlBcwJiiKqUHIFLQuGR57qeEI5jm1a7222b3Xa307DsqtmpdU272XBMCxmGkDG/JyAFFfTk53MkCgRQFa4Zih7SIpGQbmi6pioKIpMKB/QzMJwYSkRPUYBx0ENaKKSEFIUxYMxFtIS0JXUdp+PYXeFJ4ZLnguuCZ0NxAfbe9luZ/I6v/vXf/uLv/2U8kezv73n635/pG+jt6+9XmBqOh33vsd30FGC//mvv+G//6/MDg9u7rXYqGV5f7IIO0YQCSFyCZblhQ6usN+KpsBFRF2+sRNOx8nL9G//wVwOD2z/56f8Uy+jNSofxkAs2ELVbViJlhNVIJqOYpn3hzJtPPvFvx196JqZHZnbdMzy2LRpOjs1s55quat768lLDXOEI23fcEk/EKmtrp46/8p1vf+PEpZMfftcvvfOxj1fWV3fu3+d5zsJGZax3PJtIayGz1DD1EKLZ/eE3vv34E/8+kMwme4dH9k9EDdVQorFQgite17EUDpbFHLNrWjW73ah01VQ+YcTB3qwVF8xqq7K2cfGl869u7+nLDg2/dvFst9OF//AazyT3DOSXy92q5e7ce+cj99+979aDYHsXjp+8eObM5csXmu2WHtYTOb3UNjulYqPZEMBv3709CdTc8LrciMTlysbSubX1RDyMP/zRlzxhk3D94j1kCmMa+SQqIV3X9TzH9VzHMi2r3WxUu+1Gu13tdIu2VWu3u42G2+4wIB2ZIqRPuyGS4EohhWAcQzqPxSOGoRlhPRw2DF3TNC2kaYrCmEKcITLJFQoKlhkwEloopGpqSNUYEucCwAHZFWS6om5ZTcusO6btOOQ54HTBdQw0t129GFIiiTvuffjWu9+rRzSFsxsX5pv1zWxPZmLndqvleNKTRMjURrn24x/+64d++pesLikM4klt5Ua3dzjl2A2mABOa2XbjmbCCwDjOXbrWsbv779hdK9pXLy5t3zuW7TNQgWbVUxEKPbFazdEU0apX5+YuPf3MD9585dmlhYXevpHb9z54z8OPGtF872TStYk5MhoPaVHOVVI1vrFeef2JH67MX7s6e/qFk8dzhf6D++/96Mc+TG50x60HEjHt2rHTZrd0/NTRVtnWKZKIx/qi3ZMLN9K9E7c/9HBtlVIjeZU1L71xrtbyIlHDaXWqa/W22XZlXTiNaHZ0den60EBhsEfddJqJXDplJBHihaH+GydPPP3NL9Tb9R179ifzI4ul1TPzN+bWFpptEwBi4fD+4eGEkVpYn19YX8/ksw/c9eg7Hnr/yPCoI8T68kK9Wau2ahsbxeLq0uL1C6vN4tr6xnBY+dBHfvfBd7yrutGqlxc2yssvP/0Uvvjy96T0pF9QzJChyriKiJJAeJ7n2kK4wnNd13Htrtltdpr1cuVGuXyt1Vpvt7r1BjVbzBUaYoik35wRpCtQxZChRBPheDQai8aiEd3Qdd2I6CFDD4W1kKqpKipA6CEKIAIQyIlIgJQKD2lqSFUUQgFgSs+U0BWyZtk12y6325VOy7YsIJeLlqxuTmeyh1uV9rs+/OvpbG+jYcVTcRDUrDQ1XVF1LZlLdtsOkTAMHRgsXV+LJ7Rs36Dd6a7PNaP5UDYfqSyXnn36iff/zE+0a+i0WSYbThdCV87NFtfX9x2a6XZYabUzvTdBnqYnlG7FlZYksqqbmzcunHv9zWcuXD5Rq5nZQs+unfu2zdw1M7O3UMiEMipTZSyNSOS1hWdzAlyaX7544tyJl9+Q1J6+cxdZVirZF/JYYXhsaHQo2ZsszV546ptfP3HqBDDW09974JYHZm47wLyl3/yvv2vw+G/82h9oap9eiF189fWLp6899Nh7Dt27N5bUG3UhXFeP6p7nnX7pZDSfRSauXLp29Af/Wqu3Eql0tie349YDA309K/ONUL08vzm3Mj8/vfPgzpmpaJbXut0f/fjZ7//gO0vVTQBIx6MP3bJrx7aZ8ycuvXniaB1geLDn7tvu3LnvrrGBwagR10DtdqRwTAzBS6+de/3Jr2fG+j/60Q+sXFyvWvbttx0mE/D1N58GkpIkAjHGGVcZKsiYlERS+m2BSHqe53i26VntdqtaqS6srZ1bX79crddqVdlsKo7QiBQhfFIohnQtGo9ksqlMKhmNRSOGHg5H9VA4FNK0kKEqIVVVVEUBBAkuMD9pLQklSSmkUJBxrnBkkjxPtIk6jtfw3LLtVbtmsd2sthqe2UG3K0vz2rbpj0TTk9O33JNND3Q7luPJaDjsunaz3hka7QsZSrvrInCFQSIeLq6vLc1tZHvTYUMbHC7Mni5lBmOaoaxc3YhlQplCttmUsXCsU2660H3pmWcP3bkvVUiFE5G4pp5+Y2FiVwFQMZvO+tXN+cWz3W6ZEQunjVyiJ9s/nRnMqjGVpEgmGVOAJAspIbdLtgWVldbqwpXl5WsEMpvPRFRVj+cz/cn56+eunVwa3bFjav82a3nl3778ldNnXh/afsuRt909MjWZy4/E8olGbeG/f/Izt9794CPveODC5TPtknP6jefj6aGP/ZdfndidsR3X7DghLaIwaM7fKF0+3Tj1Ymd9U2w/cqmO++48QBA7c3LRs1vZuNYw1zuulwiH+xPZuBa2a0udyrKdyyiQ6o/HbIAz51554vmXF27MGqpz250P77/r/YpbqhbnL56/euzCpUazFFLc4Vy+P53pMk2LpPKqJiz74uq1C+trGkEUoMGgP5a7e8cBPHb65SBB4jt4XEXGtzrrQlBCRVK4trAt12p0O+V6Y31l7cLyysn19dVqhZotZnsaSUYAqqqHY0Yqlchk0vlcNpVIRGOxUEgNGeGQGlZDmqKonDFknPkFh37jMQJJnpRSSs8VLkqJiCiFIMcVbSGajlt3vbJtl9udSrvZNFt2eQmikR1r1zLTO3ff/ehHw+GejZVqKp9UuK5p1GzWktmssCXjqmk58WhEIFw7N1veWM7le+OR6NGjr4QM75EPfNTpeo1aAxQ+ONTbqJkqN19/4YWewenekZ6F84uDk8N9I0khvH//0pvD2zMH75rotJxoPOTaXjKlS4/K6+vtFiWyhVTB8EhIFChBC0XAg43lzcpGcf361cXrV1ZuzN2Yvwo6JFNxl/FqadFFAS4r1crvfujn/utf/snmWvHv/+fnCiP9D7z/nZlshrwOore8Uu5eP7V27ZLHeyrV0te++b1Mb74nGpuZPvzY732qdzQqHNt1EFDbOL/6xD/86QvPP76+uihBhgEyA4XddzzsiBAh21hYnJ29dK2+1ApysgAAUdD2T0xOT87kDAdi8eJSOQStvsGD+4+8p1mu1DijEBb6wk6X0tGenmRvvdOorMw9/uKPn3/xuxvlZRP+P6+UrueT8WulqoRgNIPCFO1mczG//lgEXawRt4rVffqe3zSEwAXyNDWkqorPBfSnORCCqqlG1Mhk87lMriefTafTiWQiYoQ1XVcUVVVCakjnTGMKEpKUEgBJeNIPwpPrua5HwJgrhEvkCuEI2fJkx3GbntuwnaZlt03Xsy3htqLC2TZz8D8dONQ3f3lWVZOm5XJdUxQlFtdLxZIgQsBKqYWMDw0Xzp889dm/+v27Dh4mdaBvcCJVyN1+59uWV64DiHq5Xas0x3YMlEvNVCp29tXXl+bOPvKhd8+er0STqVRPtFZxGTiH7t82tT3bbot8LqaorOJUXnv22OUzs6GQfvcj96UL2Kg3TZtlUrHqevv0a4+/cfSZRn3TdjcXlmdX11e6ghwMRnsAwEBu7LZbDt3/wAMz09PNaqe0urywuP6f/ug/9ff1LsyubszOl+dnz505ExbtxeJ61atKulFttD/8Sx+qLqKwm4fuOwzEupalMXRt7dILl7/43z/+47njTb+DEVOkJ3eB+78/8djCKq22zYNvT4SYLspXq7WNzWrDdiEaiwhhf+kfPvfS9YtZXXv41kM/+dgnJnaMHn31xMvP/zAWLWRHsxPT221pLq+vmIZobzY6bkthndGe/Dtu2e+pB0Ko19cX6yh1Bpns4O1H7u7pT/3pX//v4yfPmwgOocK4utURcYuIKYUgV0oPAMEDRoQkhWt7rum6pufanuOSEDKIrgMSKapQVKaHtUwm1ZMrFPKFbC6XTqYjsZgeMriicIWpXFc0XeGKBPCkJ6QnhScBSDgkPSlsT1iuZ7qyJaXlkS2k5XldT3Qdr227ddtqWnan26zJlrL94P9Ixw8PDOVaNRoZ26Vy3pVWJBGOROOltc1isbj74L5mo6MYoURMm728bFsybqRNmw/09CoK4wrLFLLpngwKNxzThicnFK40yC2ubOb7J37lHQ9KhoauKyjcDoRDPJ4xsC9aLbWNkHr5xPr1q5cbzXqhP/WOn3wo31NgGrodgR4z1+svv/zSlUvXlhavdurrS6Wl5dpGub3pq4kQsImhiUMH773/vneOTW5rNTZmZ9/8l3/4QqcVOjD/wLs+/tDq5Ruv/tvxjeWNRDK66+DQDn3v0//+nbYTKfSOhlVleiJirqyMjk8Lk+pVV5pSumq9BlfeuPiVP/7Z5+aOd7jK/NoEIZGxxdXqS888OzRxe16F/qTIZpOl6IjTrSbGJxPRgUb57MbS/PDQUGvFLTeaX3v5lWdOn/m1xz76Ux/55L5DsHjy5PyZV59+9ZnemYns8O7R0bxbXr524cqPnvrm8toZDIUGhm85MjM2dOe9fQPjiWQyU8gDytLy0p6xHbC+4cUTYLt47tolAHmzU54UwvVs2+m6Tkd6LiBjfn7OdaTdtcxGx6w26+vV6sLq2onVtWqjBrYLqCjhWDgey+QLo32F4Z5CPhlPR6PxkBHWVJ0rnHHOmKIoKiB6wnOF7XqO5zqesF3X9ITleV3PtV1he8IU0nLJ8YQphOXKjueatt20uht2vVvIPrB8ozeZ3P+un3yfKzyvDdJlm8VyJBsJhYyNlfLmxtrQ6HAu19sxO5Go0m50qpv1kfHh4y+eunHt8n3veHsiEYvF9VbHjkU0RLm0VB2ZyLkOORYt36j0DUVjCd1y7OVLjXRPOFlQ0GUEKBxnZWGxWmpHk9F0X3p4oiekqss3NpqVasSIceSLc4uNWqfrrpPUEoW0rqme1zJb5sr16+2OiMYT22Z2D4wPJHuSC9duvPnc8VatGo1n86M9QxN9Q2P9V47POq5MFXp7R9N79o+9/tSTf/f//mT+2oVCtn+90qrVG/3Z7LahfTsO37c4++q27e966Kfe3nZZebH2nc/+zpPHv7HBVeF3IQ86mVGI86m4EdNCJkoVaXLbnunb7tqz89bR6anrl+f/+fP/l9nd977rfa1W5dzrP37j8o2ztQYA7O4pfOQd9zz4vncxy1meW6o1zddePXZ8YX7fjsl7bru9WrK/96N/OTq37ErIqrB397bbDz88OjwdSyQVPebVrWqlLj1HM3QFJc6urAhhS1cQSM91hXAdxzTNZtcqu07Xn2fAAZCEa7atTrNt1putYqVysVyaLW/Kbhc8D/SwEYtmM9mBnt7h3uxgJpM1ohFdD6uqzrnCOWdMZZwRkBDC9SzXs1zPdl3b9bqusD3X9Dzf/XRcYXvS88jyqC2E43im47W77QpW+Z59fziy82M//tYTkVDj4D1vd+1uNBw/89qZZCbWOzFodVxEb3726tjMjGcq4Sgrlcu5QjQRjc1f3QDhIVAqlYmlDFVVO7aTyRkr801Fk/FYWA2pjZrgGgjLDEWhUnYzOSUVi5VWukpYoEfF+c1IThuZ7iXJnQ43W2atUfE8shqNWtXOFsJKUrQrbWGKfC7iOrZpNz1HCSnJaDpjxMOAXAhoW3Lh+rzs1obHBvryUUsAUxVCt1v21BjvnRzIFOLXzl3+h//zx8/+6Puk6QV129j2qX1373vbAw+0HPn5P/1fAwPbVmbfmN5z/+E771tZN8+/+u0f/+BvZgEdkCxglmDQRRVgupDKZUZtLZZU7Ori/EajpnBlYMfM3UcO7Z3Yk0vHQxwYaoThbnnz+pXzPz5x7JULpyqNhgEQVqG3N/muex584LaHnn7l+D/+699rGfWxd35gILfzqeeeXFs41pAR4dHB8fHth+9Ppfv0ULLQk0+kYkbCEJ7Tbdm4vFm1HdNxbM91XM9ybdPsNtqdzUZr1eyWLashyVTUkMF16ZHdbXe71WrrcqO52G16ZgtsG4hBOBxNxYdyual8fiCfzoeNiKpruhHhTOOcIzJgIEkKIYTnOMJ0PdtzLduzXLfrCcdzbc8TQrqucKTnep7rkeNB16Ou6XYcp0Vl854jf1uYei9H7/SLrw9MDYR4nBzrtRdffu757/zip357dHKX7Zi5fOT88ev5gf6Qoisoi+VK73BCmlJYXqViMiAkGc1GDD0EgrU7bUYiFDc4MimZEQ6ZZbNhdtJ9Waa5mkKNipfNhITrtptuKh2LJvVmze60u5vLtU7bjKUMJlE3wv0TET0E518+e/3KopEJS+JcjTAtFFfDw9syqYIOktpNr9N0mx2LGUq+kCJhttdaDrPDuh5OxCOxBHFZmiuef+n4Sy98v13fGN6+/5bb7r3lwAFbpeXLpzeK8xcuzq9cnD1y7303Tj9r9Exn4sObxdKJN752ebNUCfrN+Tl6BkAMUUjxsUce/PRn/twD3So3XbuxsrGxdPXU2TfeeP7ihRBI4XVd8kZ6e6enJme27dg2tjse1izbMTuNcqV27vKVbz3zo5rnbM8nPv2Tv6Qq0T/5+z+91m08tG/fxx795Wp5yfQ6emIUPRtJALLCyIgeVurry5VSqVRfVzSOK+Wq49im07XMjmN1LavVapZq1cVa7UqtcbXRWu46DWJqMpIKa3HPdUxr0zQrrkPCBeGB6wAg6HpPb2ZbKjmUSeYjsZge0oxQRA3pjHFgQISSPE/Yrue5btf1HM+1HM+yHdsVlutZnud5nuu5jkeecFxXmpIcDy0XW0La7aVW1Hn4Fz7zNVd4KmkvPPXSLQd3F5c3Lx1/0zS7ihG56233xZJpSV48rm1WujEjUtooaSp3XadvMFdaKddqjcLAyMZCbXQiDlyNxHirLFwwhRAhNeqB6NpWOBpqb5iD4ymuap4jNBXicbVZFkoIjBg0K26zZRuGXit1bcscHs+E46qqqfVqdenq6tXL89/9xtd2Thwam945sm2sZ9SIJhViTrviCk+xugKZ3rHV8R16Mqa21hq2bTku6BEgjnpML86Vrp9eBqfuyXplwx6cntm2b7jeqj393OPrNypDuWi2Z/ClZx9//fyJA1MjTr0yMLlTONHyytXi5vxl03NIsGB4QdDEExlKKX7zQ4899su/xYg3S6bGVT2h1aqdZDppS8sIRRbnrn3/O996+ujTq6VVH15HEKLRcMbQk4lkjxG+vrQ423UsxwWQP/PAI/feuvezX/3HU8tre0f7funjvx5moXz/cKZnsFmsXjp/1oTO3gO7XBFqlppCWq32Jt4objhO17K7VrfdMRvtRqlRX6tVFzc2T1fqpztd07VBMjAM0EN+nSEggBAgguZBAEIxlNFCcns20xs2wqrG9XAkpBlM4T5dXgjPFo7jdBzHdlzbdU3HtTzPdl3XoY7jmq7r2G7XFaYrLeF5kgRxB5iHzNNF+ImvtH/25z//3p/9Ga9lFxdr6+sL22cml66s2rapGUYkHjPiEaZw1xZ9AxEC+NY/fXdobGDf/ls8cBOR9Pf/9dvRdOT+hx6WrhuOktXhpfUWKN10OudwNx0zyptOo9ExQiwVT+i62nZdTQ3HFCytrnuyLYE0LazqBgPF9WRuIJZMaovXV5fm5zdmy+VafXLvaK3YVSOh3t68GzJXlmab5VJpbX1zjd9x94O9+X6r25wcT1mVjc1Wo1xasyE82DeV78tUm01L1lqblfqKk8qHPdeOZJN94xPosTOvH13bWBkdzcZT+tX5hSFD2Syv/dkXvpIkZ9W2do8Nx2S0WVmebXU2hCCSbKs1OvpKi5Eq6TcffThSGEwWZrYd2O9aOvdYqt/otmqN0ur65mql0poeGuIk1tYXGs3G80ePeo4Zj0Usx6tsrM5tFmuW1AGyuiK4st6xHt4z+siBwz948Y1n5hZG0tFf+uQvF7KDyVgh0zfarFduXDp79eRLajx5633vHR2fMtsdvDB3zXI6ttm2rVa7U2nW1+vVldLm2WLtbNusMUS/rE8LgaqQX2QctIdEv70noheKqlOF+I5wOKppTNUUJaQi8/nN5AnHti3Ta3fNmuN2HeG4wvSk5zm2S7Yrup6wXeEI6QJKn5LqDxMgCcmocuo5ubKw/d+feQ64ElXjF45dD+kuSWJKaGi4r1Wzbdeut5vZ3h4k6unRr19a//bXH//gRx/R9bCmh1TQbKehaRFFQUUV4XCoWYNmszs8ETFN4RHWVh3JWDysxlJGZbnUaMPAUP/sued+8K2vLy2t9PdOHb7vntvuPJzMJGslO5HVN0vLr7z8cqvUHBweyvUUUHMa3XYun21ttOYWry+fPZVkUTVnjPTkdS13ZaPUaKxFve7ZGydPXTi/adohDqOFvm3T2w1Qh7J7XCe6bc+27Qf3ma12w5SZ8Uhtc61ZdcbGenOZ6LnvvXj+ypW6V90oLh6/fnKz1UlqSqnd3ZGOpPSILfFMqdyRN8l0gVuPiJJoLJe5bajQLVcm99yy/cC9O3cd2NxcFYDHT8xePnmUso7TIadZtcxuPNt7194Hsn25VmW5Wap2zO7O3Tvn56+dvXC1Uq1aAg3gjOpra5WoQYcO7F5rWGdX11LMfvgdHxws7M2Njm4/sNfzxPyVG/NHXwEGo7fsjYZj+Pwbz3iu5Tody2q2WuVGc7G4eaZUu2C5Hb9wDwkYgqKAnzAGZAE9mbYmQbh6nE9lwhNaKMQ5AwUluJJcQbYrbNuyuna941Rtz/LIdj1B5EmQ5AXTLRmCP+cNcavNpJ8YYpDg0X/46/btRz74pX/+VrNdhw5cOb1oRF3d0GO5TDSi1jedjeK6oqtDY2OGAWbbVjmaLbw+Nz8wkW7V7LHRfgSwHbm2tNI7nEunwqurrUJPlDEqV636huztjeeH9HMnjl85PjswMh1KKE988/MvPvPMjr0P3XrobXc8eKS3NzZ7bi7Rk87lE3NXiisL55dXV7Lx/tHpYaawi8euDoz3GBqtFp3G2vrjT/zfudJyw3XGM5Ge/glhpHPh1tFXji53Ra/B8/FUf9/QZsNr2s6BsUN3P3jfvjt3WA6U5sye0UQ8n223nGjEdNbmXnjyiRPH3ry+udESuFEu1V0XAKKcR7jSFKJP1SRjqsJXO2ZXOPRW3zEAAM6YJ+W+/p7H3vNQtVS/evl63e5MjY2qMeXKuctLG52Ziam+0eGDBw/u2DZ56dLStRtz2XBI0cM8nMpk4tl8JplK67rRrLUQkesGeSp3zIbTuXzxyurCUirEC9HwwtrGlbNnxgZSqUKv3jsS7y2MjE8VBsaF7a0tLRbni/iVb/wJCel63a61WW8ulNtXaq0VKYlvNd1jCEjA/QZ8ATxUCBGYRCTpEUolRH1RrY9xJPQ8cFyv47imR6YtpGtZptt1hAQASeAJf3pk0LvK3xB/opZfFsGQ/KkQqg5mUX/8O3Y0nvyZn/4vu6cPDI6NtCqma1rDO/qAccdyPJPmrlyf3Lc9bsTK1VY2G4pGwu2q1WrYWoSYSkwCoNKpuc1m4/CDU+srVWE5G0tFNRYp9A+qmrK2fParf/93l0+fefDhj9x639vOnTi+tHR9354H9u471D8ZvnLu6Lf+6Z927jj8tg+827GF2W7OXryuh7XRqSnOqF6pRSKRpbnVS2fmc4M9hVz6xNEfta8+2bY2LtQiqfzoSvFaq7XRk+vbs+PwYGzEdVjDbKR6C3fdcef0vh1S3Tzx8vHFpbk9hw70xgdqC/X1xTNvvvbEky88fbVjyZuCAjCZixtcrbRMkGSRTKgh07KbQgAHW5AthT/ixJ+biYxLKX/5/jsgHN/s2oVcHzcMlcHa0oawiq7XXlvdPLu04QHkMpGDe++8+9CRseG+VtVcWS1LYY6Mj+YKPQBOvjDkMm2zvGbVHcfVs4ORfC4e1qKbm7VGad2uWY2OLaFdaTS63YbiWYqqO8gHp3ZPjE0lE2n81B/tAMYsu2t7dcutCSlvjohV2FZLNF+cGCIwhROQQsABSKLnj7FDCiHqAOBJV3hCkCeFFFIKiZ4nBIEUSESe6w8rBPSn1m2NW6WtljXgV/RLSQAhA5avqG++4vmppf/++3/20Ns/tHR9rX8kpxlhPRoi062X25VGe8f+qeJKXY+yqKZYHVq4sZwrpHI9Kcs0W412X29h8drmyI60Zdq1StvpWtVKuzCQzuRTzzz+/ROvvD6189bJ8RkMhcKpcDaTjUTikYi+vnD561/4s42OvXfXwR3b7hjdPyQ65sJcaXCgJ9ubWby+dObs4r5bxlAorZYTS6nxRCQRD4Np1858Y+nFz7+yEjf7H5q65fZY3KotXHEdI5XLF7Jju/fP9A0YVy4ff+aHP6J2dOTgpAL24hunGsuVy5sLL55+YdPxAMAwwv3xTJTYvh1TU319Z09fOblyaanZGgwbtpRRhmVXlF13OJH0BKy165xzKVHhCkeyPGcmm/3jP/6flO7Btl0tVvtGhizRfvOZ19obxXq3vrg2Z4VELpLtiWqbXcuxzO27bhvrGQNbkIbVpQqXrReP/tDL537yvR+88+63O21lvbjsdOrdZsVz1VAmlxpOx0NGec2sFIt9PalwPKFGkuGIWinXO5VyMm7EEjllceOi9Ce6cOAKMI5bfWD8Aqtgzh36jfZRCgBEj8DzBUMGQ0EtKW1PSOm33vdrvARI6c8fA0F+pxAACdIDDuBJ2Cq12RrN5Y8wJZKCARJJrFX8Emh5++Ej9z306PpyRQJzbNFqNHr6E7VS4/rs8ra9ExxBZRQ12PK1IgnV8yAUUSzTqRY7I9N5BATF7jSs0mY935OoS5jcl5W29/g3fthtmO/+8M9zLSw9q2cgE4kmXZtUFY698tzn/+d/ve3hR/dtP9io1kd2DUV0dXa2Oj4+GkuGXGpfv7Ryx707FdDqTTE0k4tGIB5XMnmtVuKL500t35dsNpZK5xrPrucN2TcxGovFDt/7rqjRPXbsRz/4L3/36iuvH5q+/f6HHjv18gmntTp77lTJLGp6ZvvozsnxCaXZqJuOmtTL5YUrG/NtziMD40NGeNhpbazOtdutmBFetboEMDU6Q7Zpzl2oOCLMWUTlIKHA5Vgq1Ds8nhncvnpptrArn0iGakutx95/b31p4ckfvhS59bCETmetZKgw0z+xY/uu/pk9nXqpUauV1qunKtdnLx1bKG7Yq+svnz6XDP3PT37kF9/zwXf29E3EI/HaprlWLRsRLR818uGuO2J0am2zbRsp1dCxN6KUaqy4sFjCOXz0l4NaVEUBzgmZ35EWuAp8q5e/352RADgPpoGjP/JagOeRJ4EkEx55EoRLngDhAYlgFAL6o8g9EH7HbH+8sfD7afv9ZzGYASEQiAmPfMkOaTh7jlZXAED84s9++pG3fdxyEHVm1urDk/0Xj18Ox5gej8/s3tWqN9udVrdsRtJaLBSzXSVdCDcaZiympFLhC+eup7NJhUejMbRtQQCagmvzG8W1NT0RtW07m09xNEIxI5owFMZrG+3Xn34135/oHx+oVjanZsZVjJRK3XQ+GjbUVCr0vX9+sm94ZNe+ic2ilcwa2R6dM7ZRLL7+6mtKqx5dfmLl+KtfOVujbP9ozmg2jZ2HP3D/w4+U1t785j/9/dzctcHJW9796IebLet7z37ZbrVsr7tWKmt6KBEJC9tB1Wib3RAnQ1HVUGzXzHhUC08OTA1lEsdee+bFM2cdIXPh0JlKk4B+5X0fn5u99vzlox4qmXAiqYLRaLiIi9L76Q++O2nkG9VKo1PrNIu2RTOjk/H+0XAkOj0z1q46a/XS4uLK7MkzttnVk3qykO3t74+pipHMKQ6cOHNmdu5iPBdzJcxfuVy2nUI6ccvIUGF4e+/oqApw9eqNS5dOVZobhWTyoUc/MjA4VllcXZo7l8nlC7mBSrWJ7/xFQADkwHnQCHSrqb/feTYo1kOGTAm6/UmJBCildBwSAoQAKcAT6Hng2FJI8Fy/niwYWSLEW1JFAqQHJBhJlIL8USXS73DsIQKXUkhkXdtWUC3Os2rNYQgfuOcjj7z34+l8nx7R67XSnlt3Xjm9PH/59PT+7bsO7Tn6/CkE0gxjZlf/xeOrQxMDmsY77ZoSYuePXh2cHBibGe1W2i3LbtWtSEhv16vXZy+E9Fiuv9/QYwyRh3QjCoqKkbAOjDXWmosri6iwmR1Tlm0TqSpXVIUNT8We/d7po6+++cj73w3MjUXDfQPpWqX6ub/9629+56vFyvxgofdv3rf7iSefv6Rm795/ZHa1OD19uN7k1dJSuTG/f889u/btszxYnLtw9sqlA7fcvbBy7Lk3Xl9YXuDIhPQAIWPoI/lkkkd37drTqEGL1ZCzkNl5/cyJhY4bVdSczhHYjbbZl8n99s/96qk3X37y1afTupFP9aXR01qVY20TCz2Lq+siiJqCroWSsVixUvZ/VAAikUgyFhsf6H3//e932umvv/z0sfP/dhP+b+vr2zezk6sUVmQ20avw2OzK0srcmq143WYdPGt0bGhgYEQLaYAJy9JqjY7K3f7+ZG5wCDzKpKO9/SOKFMGoEiQQAKgCSmAIUvjj3YA8kAzAkyQQkPzRy54rXY+EB54HngDPIyHIc8D1AkVFBODPovXniwogX7w8EC5KgSSBBCMAvxeXbzpJCkTetayJ/qmIG1rvXgagoXw+Gwv1D6VXV8qxTgjDrqLrihrzRCJV6C8uV1VVSaWi4Zgxe7ashIxkSnnl+aOe7eb7envHhmb2jtc3m54UnQ2TgVN3GmfeONUzNrD7lgOuJbjOpeARg7qWG09qQsiFq2vdVjtsJHN9BZDSbIpcIZbMRVzbWpzbPH/yzPT0MEfsdqnQF15fK37uf/z2F3/wNQmgqcpyaeOrx+c3u9r733Xg0rqz/9Y7dSM6v3gulhm6+50fHhudsER5+ewb2f7MLfGd3/n+P2vc/a2f+/SLrz357ad/OKhnpgu9sXhsYLCQ7RtIJ9KghNudRrW5fmN+vndoXKwWHRIt1yHHlkB7JiczmeFoYqgvliIpdw1n1q9e6xLk8+k//8IXb1xf/vFTT924cm1pfX1iMPHQvQ8NDw14gpvNFqDTFna70yIHam1rIKZ++LZbbx+IbWxecpyWasQ323UFukYs1ajZUtLI1PTHHnxXMhG1W7Vr56+ePXsq1Zcfn5zO50dSuXwipkWNJDM4qN7a/Gat1ir0xzkxRXiAfutGvjXwCAAk8K026Ri0HgLk5Hd6FwTSQ9cj10PPI88DzwXPA+FujV/3gACkEwx6FB4IAdJD8GVLMr/xH/jU0WAKo4/g0XW9vszgh97/yy99/Yu9Ma3hiuF0KhwLVzaKipsoNkuF6bRrOrYpJ7dPRHR9fXl9YDTXqjmtth2PR3oHkl/70td7+7MjwzPI9OHx/Ppi8YWnn7nrnrs9r4acb6w0dhzcs33vjOW4qZReWmnHkth1ZCanW6Y1d3nZbbqJVCbVmwhHlUqlnevJOW75y3/+5RNHT37wkx+2umWVD24WyyPTQ5qGf/2nv/Ojl77bNzh1+/3v/O4//rXC4TsnbhQi8ldikXUr1Gp3Xnzh9UOHHhzatm9m+3Rp8fp3v/XFYmlTj4RcC37lI5/YaK1+8av/xzO199z/vkduezibyeeGw6tr585duHx17ardQibNkIYT+cx0PrEQwlNLax2m7BifOD23fMvew9ITkXD/wcOPbFaW18oNphvzlucKPr9aGZja9SvD6Utnzz579Pzludm//Ocvb88N3H//fdeuXii1Gi55uWR2uGc0ntXSQ+7+XUMdc7jG3nnmlaevLa+kkj3VtmOUlF0z+2+/+87+mUx1bfnsqy8gj6VHtu9PjjqeWW83I2nhNKvnTl+1q5VIRIsYURZKZ9OR5uXVRrWB9394qx8QC6aP3IRZKIOxkUL6bfXAL6X3IbnnS5U/+cgFKQOh9OuqpQDPA+kGoxalRwBIIhisQpKAGEnfIwgGsPojJmub3d/4+V8EcNdmL2zW6qvzazPbDu+85e3lqnffvQ9fWZg9dO8tfSMDi5dKDFyJXNEhFNK6LcrlIpvF5dNnz4cwtO/wPlS0SDScScd/8J3vbK4XH33PT66vLpTrncNHDnQ6JqkynoiVFtuddieei0UMtllsWV1T11XPUmLJuCKANEUanWe//52jL76RSw3c+eiDhhFdmlvr7R+JJ6Lb942/9IOv/NEf/spSh339G0+/7QNH3vvIb7zwo7/gjEVDyju2ZQtjR+ZWHTWive3uj9z24J2vvfStl5/54cTMreFoLBoKx0KhTtP+26996ed/9rHpkd6VGzfq7fLKysq16zdadTk5uevOB/bs2LVdWPK551/74te+slxfjTHcMZh1bHtyal+iMLX/1nuaxfmWHW5UGxPDqfNvvnHp5I80jqumbWUzv/DJ35gYHSqE1Z6enuKV62+cu/Ha689fnHtzYmKba9JiczOka7qQlVJrOJv8hZ/76cX51dMX5naM73rvYx/AENY3W8XSxnpprVRcKS6WQlzffefth992X76v19AVzjEUUl2nefrYyuf+9MsvPPd3JlicQNOUjuOpCOFwWHEsv4cnIAOh3BwnAegPhQgYPkEhvJAgCEiC54GQIFwIMJZEckEIJAFCkvRQeEE3hmBmbjDQBhD8IfJbcgocJILgQhIxaLY6vYX+fCzy6lNP7Lj3J8sv/lshjCE9vLBSHOu/VbidRLTH6rDicpmpsl0VoQiCq1mmlcvFn/y376+vFW+76/5EKuVIEQ9rZsctW+03Xzv2vg99MFmIb2xEjtw3U1xscpVVNupmi04fOz+9eyqq65VyQ5JMZ9JSQjijN9omhpSLZ0889eMn87HsL/3mH+ZH80Y4dOnYqWjMYIqSzCS79faT3/vaUtN65B2f3PvgkQuXFv/wM7/z+gs/dOzZji3b9U5eUm9Pf7upTu0Zu3z+zIUT197z2H/O5nrVEO+Y5f5s4Xtff+aHT37z61/5wu//9z/rSyW1yJgeMT7xK5/eMTV98dyZb//7v/zTV/++pzB47113/uNfffbV18++8caLEhrnr1y8+vIrf/QnH0knss255e07psvtZl8u58gUKKH20qkdcfzS8Qu/8wefjsfit+89fM/heyb74mG0Gq32StO6fuy0qmIybIz19M8MDjVTnoPeE8+/srFZrqwt3rh+6ury61oi35/cVWmthRPGvr13P/jebaNT/Sr3Fq5ff/XU0VbTGx4Ynb9x5tzxl+/Yu+OnHtk53P/TS4uztkvbhiYHh4e/9eR3Xzx3Cg8+BH5XBb+D2c3OsH4LtmCougwmjgoCKXxTCEKilCQFCo+kQOkCEHquP4fNr1X1B6Mg+TBegj+5G8hv3RZIHkgkv0LQE5ypn/zYJ2Zfe6FvdPDdH/v1z37mdyNYYuntO/a/uyfX4zKY3n/4zKvnDhwZUVisVbeNpOo6EI+F33jt+VareejIPZFoYn1tvWcwTq4KBAz52VMn77r3ECDbLFcH+pKn3lyamM7OXV/vG8xvrpdndmxbXaiGU4o/qFXT8OrFlfHpvksnrrxx9Oyjjz00PT1ldRrlUiPfY/zZH/73ZCx+z7s+oYe15dnjn/2vH59tmJ/9p+fvf/9d3Wr9wvXOR9/3U1791amh/p+9dWd0+PClVXvv7rsisdiJE8eO3Hm4pzerRmB9dWNycvToi2/uuWP813/pPy8tXv25n/3ZVHZ6etv+g7dOvPzi43/12T8/de5CSI8QeRvNBgCM5wqf+vhvjExM1YsL3a79+rHXd935wOjI7lNvnjly+63xnnhIC109t9hXiBbXFr/9j9+ob7yZG4y1lTAqGIkXBnqnxoxYzdZevXxiafEKed1mzanbrgt2LBrtKcSXVpbbtn0TvA/k0r/+id8aHRrotjY363WCcLHYcoUzMjiQiifDkejK2XNvXHry+TeOCwFaVBsYGBjqKbguDPYWZsan+uLptbUN3HMXMAYgg97rvin0Q+LoIyEJYmtajgQUEv1IlZBAAqUgKQAk3owa+Cxmf7yn39OBgpk2DAmBWNA7mRhtjWInQMaxVm194uO/+u4H7/38//jET/3q70UyU//6l38a15w6DNzz0E9979++/Id/+ne1itep18Z3DBWXGyHd0Aw9Fg9dPHlOj/JwNJku5FZulLK5RCIe0jTuOK5u6Ha3a1lOpdwtDKdr6zVF5W7bA44CRCii6aEkV6TBwg65TLXPvXy9fya7UamszTXe9xMPhHSvXG402k4spHzt774gnctXFzu/+Xt/G0uFv/GF//Xi1/5iDvjhD/z+x371w0dfOvbdf/2eZ3VLc0+988jtO4Z6lzaVu9/9cavRDSHvG8wnU/FuyzJSRiGfuHH5hqbj0z960dPLP/2hn9ZVPR6OzK2ufPUfvnjx9NG9k31gpM7Nzs2uLjY63TBjVdMkgH3jE72ZdE9+cNvEWMPje3a/c2l2eXh6YnCiQCBXF2sG58mkceXq3NFnvvbMi18PZ1ODhZ5UNBdOZyYGRnfODEeMSEiJ6VFNQMTuCMc2bbNVa2++/uqLly9d7unJDPcMpCK5c9fmlrr13vzArh27h8anMrmJdrM5vW1EwW51/cb80rkXXnh1fXlVmLqthpbqS9KxyYXeVM/QcLYNrdtHpjU3pHgWIATqyu/XyAJVw8gjwiDUSQBSIhH69TR+mMr36QIqtz8PF5BEMGeZBUOsEZGBby6Fn7tGKbeGdPmkR4a2bRVy+Z/5+E/NvfnK/GprY/FGEjIT23dHNLo659gu/cKv/j9VSa8tnk+mw9267RGkk9Fo2Ji/vBhJx9aXVnbs6zUYV0FNZ6NhQ/Vsz7S77aa9sV5OJ2Mjk3nO2CaxEGO1jj0y1teqO8l8nAQKx1EUXFzYaNWr8YKm66GwkX7vT+1RQ16x2Lh+dX1kKPv1v//S4HDKbk1dPP9ytbwQT0/Nnn5N1xXToae+9UdPfe+vwasxpo5MvL0EcNeD7z177HhseJisSMwItTbmkrsKxeVmLBtOxkN2o94x65Uiu2VntivDf/yZ3201Gu2uWd0sH961Y7S3//mTZy5tVPxEhAo3hwPD6bnrp+cA4BgDiEVjn/vr+7bvPVyt1KWnKJrX15eSFnImxsb7JwZ/LRTJ//MT/3hp9QL8hxcDiDNIJCNT4+N37jrQHx987ei/H716zdBZ3Ahv1qya500N4NiesVsTkVxuYGx6VzRTsKzmiln6+p994amjLy9tru8Y3/fgu95z/9uHWdtVuWo1qynNcmyqVM2VlWstj5XLtUgkqrjdrcmDwXzULSmBraZUErfa8PlQHoJ+WsEMAABEX0QQGUlA9GcqMV+GQKAQSAJ9expMo8OgCkgASXI5KF3Tu/32PeM793zvS3/HED/3+X/9yCeSI8Njy4sb27Zn832R4fGRG+eXJYltu0Yvn7mWzuU1TrMXzhmxSLPUkZ7s7+9ZvFZK5yPRqFrZqC/Or+/eP3Py2NV8LhpPhBmHjfVGNBJuVs1MT7LeaIaTEV1l1WaNa/jSc2fUqGjX11konurrHR3qZxq/erZ47s2r9z06dfbV4/G4cdeRty0vX/vXb/6g0XZOnji+NntRUTTHszhH8mrIGGLoxuwLhb6++x+678a5axNjR6K6t76yFE2r3//6S3c9cGRguNColGvlTc+JOnT1137/U7Wuc/vuO7cPz2QG+nW2/qV//lap1Z3uH/+ZRx5MqXJpuY5KrNloxJPhaBRyuVy32wxxjYWNU/MXvvC5P/jrz399nI3V6p1MT0zh0KzYnaadSaWFCT/x0V8/cOC+F17412dfeWLDagKo9955/7a9h4TgyXhoY3mjKEk3cqltb98f30Wt1o3lq6Xl6xhafe348daWFKQ1CGtx27a6rpNLJLdt3/7O93xk7479yE1ddjXdS8a0zPSe2sYKRz0zEsoOzESEGx6MD20bVzw7KPcCFrRsJAzG8jDGgnIJBlIwwK0x8RAkjP3pTr7OkYIQGDG/YAxA+kOaEXyV5ve1Ir+5VjACbmsQKiMAW1LXdP79W/+2ePGl4TRb3TQTcXKkS6Ho3NrKxMF7240WKXx8ethuWdGo7pmtH33j8W2H9mf7s8LV9hzYUV6rk6RcLnr1zKUTb56/+6E71lerM7v7NhYrRkypVVqOLXOFeDSiM4O6LUtT+dLcCujKyuV1JVq5euZyIpe969Zbo9FoKg0Xjl1rtmvbd6Vmjx7vilL/8EDXbv/N3/+tJK+0fPnqpddNu+NAGABBECInScBsAG9q4Pajz734zKtvTk7d21FbljB3T+5OpXondo5sbJQzOXZjwdq5re9tj/z2zh3bH7nn4VbTXa6tHTv2FDjuZz79m4XsWCrap4UVpmEkrCoRo2tTq9JlHA2Fa0aosdEpb849/IG3l4qN4txiOs7jqaS0uOl6RjQcyyQ0TVNVFKY9MnHHrfff9t4zP71x41R6eNutb7+LGzFpod10AUQijabZ6tRsTdUUDRqlcnlt2e5a1+duLMxdWV69cvnCIlO1bbfunRzcdsuemX233jIw0vvcj16bm79h1crzs5fXi5te19xw1kHTspFYPJyNaLoru8lkNvJdHUdG0R99jkjod8AOBtFsvYJZ2UiEfi8hICCJwVBdAF+kEHgQBvPZWpJAsiAQGoyoACCUWz1Mt4bMEzJqtaz7Hrj3vt3j9fkitW+sltaPXe8+9ouf6suOLs7OFavWJ37jD2yHKmuVeJLNX58dGBo5e/zNoenJPYf2t1tmt+GQsGxXxCO6Z3nnTl3Ytmesb2RE1eHSqbloUiv05EobrdHJ4fLqRqVciiUSnDHLtR0L04XMhVOvnX/14o4DB3besjuWV5Nx5fzx5UatpIJld9prxTVk9ODbH/rK5//hiaf+pad/uLm2YbmNaKux4MlNKdhWjRMyVUixZ9fPNKvnh7JDH33HY2XbG9yRUxRvYHS8VKqM9eUunLm8Y+fQb//GLyBYv/mLn/nyP/39m+dfKVZbGodtO3aODw8h8L7B6R37br949rwLNjFIpQ2voY2Pbx/qz0eMaDys6Vp4bWFJj2qaoXptt2tWGx3HiMZT2Uy7a5FtMSUUSqV7JxLZdLLTpeqmaXmiaXqJFKpE4EHYYGs3asKzIylcmt0ALaR4vNsxwRFGODS8vZcrsr7ZcT23urzaqtcvnX+90SkLNdqSngXd5kaxVW/2FXpWNoqnL15w3qpWfOulAAFI9GG7PwQMg0nxBP4gQ7bFQSCS0m//z4KJgIQUNGPn5FOo/BZWQTt4Cqj94M9eAiBACSQRGBBJQFAYOo67Z8fU7/zuH5188nurK09Jjq0uTQz292ULltV99qVv7Jh5R7fjRROREy/8YM+dB1PZ/PL8Un6wMDm9S7hOcaORS0Q3Vtq9w5nyZhM9sX33NlWNaRq0Gy3HdUcnxteuN5Kp5OUzx7/9z995x0+8T1FDtu1FjMjwePLU66evn9945IMfGtzeT9SpbxavnNqErltfW43nuI2UTPUO5GKf+uVfuHTh5Kf+86fX5248dWPObtVjnDXorR0lZESuro5ev1qfGFQePHJPfFBLQqFScnYcHKnVW/39iWOvX+vtS//zl7947cbsFz/7ha9//7lQKj3YN53qcSUy5rnl2eW+ntERhcG5Y8mVLsvEdty2f3JshOtKpV5dnptd3uw07eZq9eryWqm6vlFs1BrNzVK97grgBDrA2MDIxMjddzz86Ds+MKRr7LnHX+60Kd3bm+nJDg/Gw1EJIFGSWRf5/hQDs74uIkZ2bFseFGt9YZlBVIuqrU6VKaqRiYXRs7utaCzy2pvas6+fzKfD2yfHR9IDRmqAuBcT1rLnUbqn7LBcoWCovLy+XBZO3epYkhQ/0ODPNfLFxG+VBhKBB3aKJIBAQCCJAMFAQ3/yvP9HIFH6EOxmfsifO8H9+U9IJH17Qf7ge0lSAAEqyFoN793v/mCkNt9eOakroURaXbixNjTSGw7Rqy/+eyg0MDa11+y4rlsubcyq2p2aEU5kY2OT22zT9pAlYnoohP1DGVXjEV1NFtKtiojHNUWB+oYzOTlqtQgM5+Ufv16cW370Jz9YKAyuz5dT/aFoVLl4bDEUyn/it+4ldFvVysrVa9XNOkfstNp6LLQ0V957ZObq6dc+/pk/yaVyn/7F3zUozkKrlXY9ryhrnueAZD5QAGQoJIU9d8/wEN87sad3PJnO6etrzb137q6s1sMqrV1aj4agUT3/zLe/9ds/8/6/+Is/m7nl3bfecod11wOaxlTr/8fWX4bHdZ1f3PDeh4eZR9JoxEyWbEtmhsQO2WFqkzblppDSv5xy0xQCbdJQAw3HDtmOmVGSxUwjDTOcM3P4/TBKn+d9r9ef5OvyNx/tfe91r7V+OSVLxrnU1eunnnvn5VA6wcowmefJF4AORxFZSgqyGVVBiqSMVrvTVsglBBxPF2iTy2MvUWUjhRKPp7apddcN29s62gpi+uiBE1fPD5eWV267eZ3TYwWynIhlYgHgKdUXCuLU9Znzx86YtYbNd+6m1PL1c2eCEwlJgclKQJJUbZ0b4+HE0lQmxWMEYlRh29euW1PfNjI/6EtMD48MDwz3i1DymEu6Nmz//gOPOjVKPkcLqRyTSITC4cmxs0MBH/SUFz8piBTlBgRIklyE3S5bkJedLVCSZVlarv2EYHnal0QZgcs/L49oyzkkGQEQyBAiMoDFCxbKn1eGiqIkFvdDMlDbzF/ef3+ZQug7dTwcTbtcmsOHr63ctBug1FIo3LXhS6lkbtPuHbjIZXMJQaYYOlVa6TZbbEyG5wGvJLBoMK7TKRiGVamVLCfbXXqTSTl6PcjweYNGvTDrD4VCJoPS7qkprTEFp+IEBkQAM7m0q9xNaang9EwilBQzEKHQXJbBSF6SeDadtJdYX37t7739V7dt3rdj7arJqakKT8lfn/1zbtGHynAwz3CyjCKoKPIygAA4UbCto6XcYRjpbKzbcuvuhdmcQau4cuyc3e4htYWwb9FZpvnHU/+wonmvpwZqq1fX1Dx/9I2Z+WmtEq+qaIwkU9cmJgsSh5EKh1qDi4LbUaomNPkU3d3Vs2brJqvDzHICJwA6vzRwfTASTFislQaDw+1wOCrKXB4jzzKRYPji6YGZSLK5qXrjjm53iSbFgHxGykalQrIQz0ViyemZi4Nj44M2k2g0e3zhOS4IWld3tq/tSoaWUtF0MJGMBmfdtoqcnHWVuNu7OlQKFUszeZqdnFvkEcmoVmVjaYXKUFpXYbKpRs6de/75f8UXx8SCUOI2V3nL1bh9dnoalnkgIkMEg0CWkGUs7uflxyIAEBRhAtJy6WOxiBYptq0XF4n/k70AADKQIPK/N0CRfygDWKxqL3IN4fJuUJRlQHzz+7/evr2n7+OPZ/vOhxbTOoc7vng1FqU7tt955sTgvnu+AQlTNJDauLs1Hgup9OalGV9JuVuhVlgdlsBkKODzN65qnhkPaPWEQqUiIGFyKVQaeOHYBEYgfF4o5EWUwOxuczrBODwmEkNoJgsAAURg92jmp4PhmThGoEqNUpJQFJEJREKAaNRiKXryjZdfLSBybUVbZUVDNpEyGTW//cdvwuOTbTbTSDIxRdO8JAEAVAqXQbmiu2fLhm2t1w690bKi6eb7940PL1y+ePLD/zy//+bdnFozdWWhZ3P1y/95PhHOPHzrrfNRRq0Wnzvwdl5Cmix2j6vWH6Q95ZYVNe06rUoSUgtzsUQ6BFSqysbapoZKXzQUmZ+b7u9bSiVRnkzmojaXqbZhvb2kE2AFlZGlU1n/cJCVDA1VdUAjbtuxamFxcWJ0UaZFtd6SjC6OBn3KVGI8PM7kUwYFHhNhKLBAouTqzrVajSO45Juavh5JLjmMZpujsrxihavUwERH04EopnWsXb3LXlGeZLICL4sCS0lAiavGBsdPXzildyiEjH98YhrIUm8olKAZAMBXb9+xtWMHLCuFRbofRGQEQdDlOnUZ/M+FJ0NZhpIkyQBZ3soU0ZnLjaAQAuRzuESxmnt58oAQIJ9DJ4pYL7lYyCbLMgRcjmMFcOOWVpvaUFvRnE8u5KN0xy1f/vfjj+p0SqW9XmFsuWH/PSIvCHnFxNBRAZda2rt5QSYVCrWOgJIyNLWYiIfbNvQkolkMRc12ilKS6Xh6YSbE0mmJRxVGLQEUlIrgeJykJKOFSMc5TijorbhGpb1+YVqSWKVaQ+KkQkko1FiBzgnJdCw6euLMkSOHD0EFsqNz/ZoN99isrpzg+8oPv5MNBO9bvWJ2KerPMxG6AChdTVlnmb27a0PN+hs8H7/xkY50tm3uHhvtf+nlp8NTY1tWt/BKLySV9RXOfz79J51evf/Wmy+eGVnRWffSwdf8keS+NRtLbTWu8tYSb0kkNPnRJ68T4YJaixJmo6us3eDy+kPTff3n0XS6pMSRUhpQFCyF/FAC5WY9z3N8Or+wlPU21KzbuLW0tGyst28kPOlfCk/3XpxLpFAg46hMGHSZZDIjAhMC3KUuOi9m6LRNo2yt37Bq9QYUJaO5qM6szCfjPBvHUSTLSgpAOsyq0+8fGI1E7LaySp1N5yjVOFxVjXUGh4EQMS7PYCRIJ+hoNMDnsrgMUul0IB6hufTY3GLv+PU//uJJWFaGABlACSI4gMW5HIGyDGRxGRYoy1CWYfFmlJblK7B8FRZ3MkUGHFiex5c1z+XpXlouJJU/r/mTJRlCjuPUeuu2vdtaa0F6eDHvkzbvu/X4Jx9SxpqLR9+0GpU0VXPLF35cWe2hKDThTx1467W99z2QS7EWpyWbYUx2tcgiuXBOZ1cjGClKosViEAV2aS4SjcRLyww0J4t5UaPTAAQhKAUmoVqjnMwU0nHeYMRwDE6NBkVBVigpo1Urgnx4IaIAEp3zffTxf98/8mksz5IAu33Tnrvv2I/hxvnA2I//8OMyrbHTURHORjiJNBnsZRVVBGp2ehtrVlcrdPzwuTmzRh7pu/DGJwdFEVRXlupUBSgYKrt6srTv2KETPRu2GLVoOpRYu3nrJ4c/O32p/5G773WZ3TmuMLs4uzh1XZIBqS5oM/y63XviM9nekQlEh1R6KnOp7NXps2OhGAlBTUkdqTIrhIJeY/fUV7qcnjwnhGOhOd9UbG4hlJhPpCOVJc7Oli4c5oYGxz+4NhhnhSqN0qFW84jRYLGUGt21VV6dQ6fVlTqqXXq9MhnJ+ef8uYKst1AYxPO58NLE3MTUUJzJ0ALM+BYgFOoqynVqDZPLZgUux/CiKDiMqlJTic3uwpR6h8UiQ96/GE3l0iq1dj401Tt8CXrL0eJbcLmaVixyRIC0rDpAABFRkgGERSLmMh4MIKIoAYAsd/sXqdIQFkEoslTE9MqfL3dgkcpUJHvl82zDxpZvf3lP3he1uNvPfvRebGhxzz1fP3roYCY3L7NxUqm3e29q3nZbRXXJ0szC6UOf9mzcVlZeyQqM0WyM+RP2cmMhJ6RjtEpPiRyqsyqYdM4/Fbe6jRghCKKMUHgmkeF5tKzUiaA8ABKdZtmCpDGo0ql8aCFoK7PoDOpkIhKenMsmIgoS7R06+dJHb0WiaQIiGxo69+zcWVJenUwzBSn3t2efaLRaNJgykpMMlGnVqgZoMiNA42qowAgj4GQ0G5wau/Tiq/8cjMZu37arZ/36l195cX3Xju4VDR9+eEDGFZt3bPLPhxK+RXu18eSZ06Pji1/Y/0Awm8rm0lZSJXKxDM0EE1kmkysxawSd7vrEUiDjr3I7uFxBokXKZF7fvsNlVfqTGa/Do1RhlFJPAiLPQUlKkhhqchgNZjg3MzJ08VJobkFjdwTn5psaW2PRmWvDeU9pS+PatXVtlVoDiUIul5MQ1KAgCD4VzjMpkUMhRrqq6ghKnh29dOnYCSYH2rasq2n0mgjMPz/zw1//ZnBq1EJg+UIh8L/7CAAUAA2OqEiFXad326wms1WvMdgJShTkiamp4ocFi/OQDIAsyihWTPcV/XcSQJDl6epzOrwkAwAQSZQlsYjPRCT580lfXL4NISwaB4tWGwhkIEqCBABEYJ4poBqs2U187f4HEC31/S89cf8ddzQ0t6ZCs/957T83bquLxJE1Nz5au/Ymnsu///wL1rKy9hUrKYpUmjWpUJIVBCWKihBqdWqGLpgdusB0GlIFd5lZ5Akmz3EMTSc5hpeNLoXDqlmcjSR9KZ1dabCZkylByORxkqMZ/9DwhAonLAbDUsz33CtPD01MGlWKDW0bWus7TAZVIBpzel1Kper4wcMiG5gKhTDSsLZ9jctREghN87JU6e1QiSab15bNj7/5xguXrvVvWLeps62Lzkb6RkelRMpmLR2cGBQQgKmVQd+SzqjFlVjf9JLLRGkobS4VbyxvalzdPTw0cmnoSoLOAwAsOh1OEhqV2mp2kIiUC0cAJda3tRnVJjvUi5hWUgNvk7XG3SjzuArCLJ9N88zkQO/A8TM0HUedOM9SpfYSvdmxGI/UNVSXlzRbDBaNrQrB5EwijBd8TD6sUdmmB0LpDK92qgk8r9JobJ76wQu94wMzZQ22gsDTgRCQEZRAJ0b7aBpIEE7PL2UjvvqaUoOrIp7KLS7OTPr9aYH//9GuUAAcKqqhpLTDUwHLPSiU4bLEiUCIFqdtCCFSxA9Ky13uRddf8YiCAIJlUIkMZLH4L5Fl+WpZyIKyJCEIsrz2We47kjiet7ktX/vql8rd3lzEPzZ1evZ8/97dN3d07r1w6sA7777TUG70JZQ7Hv5tedWq0d7jhBIv91TbHaY0LbpKrJPDs2oVkYmnytsq6SgHCSSfY1GI1HYZAgvJTADVW/BCtuAPZWpbXFoDOnotlEmkKS2uVhsAQmAoU8gsnT96WgKEzW1f8I9/eOSDgesDDrPd43RXe0tcZq+rtMpsMkARnRi/9sGHb9Ick07lEBVR5bCSACBQp9eXlXjUClLnsNROzY4M9l80OwyOMnuWZfyRpMfqQGT50xNHJxIxHEMUCKomlEBm0wxrs9htOr3bavC4a1zOsoPHPhnxjQABNSg1BqWyUGBLS0utVgulUKXy6fGRoUgso7KoWpyN3Ss67n3kTlTCZ8eWcgl/MLA4OTM3vjidzkYlRuhetbKxrYddTFeWoday5nQqMjk8jll0MsUXCoVwcGluZDa+sCTgSJ7JcpLo1mnoPGrQm6sam0WWMDgMWmdDnklJqejwcG+QCRrNNqXRrCGNCAUba2qnhkc/+ugwkwliZouaUrTW1jdVdc9OTQ/PjM4nfdGYbymZzX+upQMEAaKEAgArKzAIoSwu89kggshikYMEJUmWARRFANHi6AQBREQBSEBCIFzGDUqyLKOigBRzXTJc5vMiCFKEecmSWKTfZNKMiiKzefamB+569i9/mh6d8Y2MP/XyC5npvse+8qDO3D505fDsyCWdSqmrWbVx/+8nx30Lc6d333x3nmFEATM6TVJeLLBZmeNNLhtBULFwzmBR0Gleb8ILTIHlgEGHBedYGUFL6wkMw2ZHk7lo3uLWUQYCkyU6koj6ZvyhhUSKjYanTpw7HQ4GdCrC7XA313XrdUaj2Wz1qJemJ68PDZy/dmU8EMRQUBABAYAOUdnt9pqy2pbaJpWSYoRMNhWNRyI5MeN0NIaiS2PT06UWvcuin52fvzI1lUcwr5FiszlSocOBUm+0t7e2uRy2RCx0eXh4IZ8JhaIFJqWlFLwsZfNsEXsM0f+nnO3/9Qevc5XV11eKAE6OTgb9MzkAUADWragxoiqjzczR2clAhMkyyVSGEUWOK/AyyErLd5YCAKcOT/BikvlfVBEgKFCjKEWQBp3OosFwGcdxCvCcWqvvaaurr2hPpPmpeCwRnqCTS5FYTEzlJcJYWmszWpSJUL6pobO9Y2do3lfI0QpM9kfmfenCzMRA38hIkE/wAFAAwKpKFH6upS+zA8HnHCYZkREAIQJkKC3vYuAywEuUASwucKAsI5KIFL17kvw/4FLxlCqOXBJBkpt27BwbH5mZnKx2O0xaZSQcamnYqiWkyNy1bTt2CdARnL+8MHFFpzFa2/au3PW9o2++vXJzk8noYlhBgqKn0r44E2HzWa1B5y51ZZMFhRpXqHEmziWCWaWeNDuoZFSQRNFeoi7k+WgkR5GUUoeLAp/zR1PBUDQUmZsdmZweTmaCKhVlcXhteg3DcwRi1mqwueTS4tjI0MRMKpVO8nlJlpUK0kDoKh2elsquiroKjRmGQ0sLsyN0IQ9QosxTOjM6UZALLoselzTzMzOTS/PzyXhKBCgASoCX6d1et6fEU9JUUTm6ODM0Ndg3P5XM5gAAOILIsgQRhBcltZIkUIhDFEWxgsDqNVq32txeXy1xqVCUKxTYAs7NhePxZKylvoXkWDUhu41afzp7dGQhlUv//1mmAAAA0Csop8GEYYTdpFPgcjgZn43msjmWFTgJLH+8CI5K/PLPGAJwBNgpSuKBEsXKKut1Hn3c71/f2tZQ4Tl+9Jq1rJwAfFohEbTssHtLG1vKy908zSWCGURDKERGo1bNzMQj/sF4NkGwDKyuKmZw4LJqjnwuFcjyMkwQokVBa7kCBMKiO1mSJFmGsozIYtHsgBZRlwBAeXklJAMJyogsSQKGYhu3bwong9d7R1Z01nznvu8+8dfnj507Xesy1NjU9912L2Xwzo2fzAT9EFO23/wdWjAOnz931zfuy6YL4ShTW2OSBCQSy8goYrGq8zlIUFClwJhMgU4IpBLRmUmmwBMIqlSjBVbKZySdmSwUcqExf2hiNhadzmcj8USSzgkISVpK1QpRoVKq7GWlLJ26NDGsIk0VFeXzo3MZji2rNNDpbHwhQinVeiWu0pkj8fTo9LVUOmOx1Za6SiiFkKKTeYFxOKxyMHXhYq+gQhYWx2Ue4jhut7hqPdVNzhoEt4S4xf6Z61Mzs3ZneZmnTAj5eIllJZqTgT9c0CoVai5Nkaq5GENpqRqP02q22jVkpcfIs5gkA6uzfLT/0umrg1qT2eM0T84HLk+MxbI5QkFsbGnpalx36PKxyZkplc4Sj4fjWYbA8Aq3vbW6tLysREsCyIvhBDY0OWbQIR0NVQhCzfsD0UScLoA8JweieSWfUqsNITE/HVxI5WgAQL3d4naUQtRSYbe3dFQbTZrA7ALHoVZPlSjSKFCqdQaVWu00OymSF/k0QikggqUCufGZkNeJ05KEMyIGkYIswMpKBBb/gKLRD8pALvZ5SLL8+TMQLY5WkiTLEIq8XJzTgYyIMgAyUjT3FekBy/AliIDlIEZRdRAT6bxWCZrayuwm8ysvf/CNH/9y7OwVj8P7waGD37phbc2KtaGZ/gKdxEjz1q/85bP33vS4G7bv3zk5O68zuoxGlX8+RqhQnV4jSRJEUUQS4yEGAqBQEkothqIQJyDP8mxSMlhIjhWC44GIfzowO5VNRDPZSCKbxgilq9QuikIqHR4anRz0zYgyQuKYSqEvqShXUfjM+GQ8HrKoNZVWBy0y3qoqpdogJFEJknUraiylhnyUmx2bLOCsijBVNXsDI0NHDh0hTS61TUnm6baKLr3LEYotxFORYGBGLZVoLKbR2csDw5Nahz5fyFOiHMqmdQq8xO7qrC3ram0hUYIWkbnJaCCyWFvtspjtgenpglLtS+WbXcaKqlIEw6JxcXisv+/6yFwgRJBwPhELJ1IAgJX1Vd/au0cU8jOLEX8iSygMRi0KRDRNR5QoocQUOrWsc1rzDMPnaZyXzQYHxKUIncJkhVatNyhNsUxuKcEBKUhCZjAUvzQyOxuOiTxn1qibSipLNKrqEoPdU21w1QFMqbJZca0xPOmbHDgz2n9l1j+XZrKkgtLo9PVWq9pcRtNhkS8QpE6lVimVGlhTgyAQkSV5mSwvyyhWvNQARIDIg2XAsozwnFQMJgticWxHJFGWAZQkACFaxKYCBH4ursJlj4wEJFkWZFGURLVW0dFQefeeHedPXHzl0/POUmc0SkupxPYWV0dbIxuOxJIJbWV3aestx5776eZ9XxNRlFTq127deOCFt5s7etbf3RqLAAQD+YwQD6UEUbbaVTiCSFBSqTE6w2fDaSmbptMx30IoODfL5JMIyes0NgyijEgPj48NzQ2OzU/luOXnDIXhSo1WzGcpGSACp9CpAApSSbrKXrFiRaM/OOt0VN31wBegrB0bOz80dV2DqMo9nQaljWFj4cXeS5cuaO1mRE2yeVGB4/PzgflgXGe3lpR4nI5ys0qBMamJuYHZQLCxrt6ssQpMdvWqFZBm9FZtcGl+ZMZndTeuaCrzNtT5Jxdy2czM6FQinjp1/gyvxoV0diIcVBLKCm9tY11zk9PEFRLDYV8uyAhSMp3nPrs+I8rCzRvWrWmqS8twYXhCZnNut1tvtBjUhgwTvD4yOhsORxhhLhDO8BIKgF6lcFn0NaUl1WVVZhQRWBnFJK3FoaKMXNofmF08OTse57MSw+GyoIDIbCZXonLt2/2Qo7W8ssGutzgxRJ+Ipa9cO3llaLBnVWddednYwPDbr78a9A0OZv6/HomwpgZBkKIJAfmcV7I8bgEAP7/75KK9WJIBBPD/sZIuqw9wWaaHEAAoS1Jxw1hkWwAZUApcp1PqrJqqCkO5t2V9z4Yf/fRnV85NVzqtiIiUqrEKk6DGhVxGxjGFqbo6uhBXCAWion1iaKh741bf5evhud6dX/25uqzF5dYFBgK1Pav1ZWYoy4HFlFZH4ZicSRTkAp2IzC+OTUoia3E59CYdSvDZbGp+camv99pg70g6l9GqKaOScFt1OpWhkM1JKiMtgFAuRkeXwrFQMMtQuKLEU9HoaUvQIQ0KtvR0Oyw1S+Nz4cRiMOPHIe7P0uOTk3ORORoAGQC3WtNV3uD2VFTXlnhdNgWihCJNojpKCRLRVAEppAJBwAOt1dZ/beDK0JDDU5HJZFoaShSYlI9Lghqh1FQoGJ/3B+fiEYNF01DX0myv5VHs3aMfXRgcKAjL7y0liXQ31N+0bnU6z7394SHIJDOilM5JGSC3e10bOrtslG4uEZoO+qfnF1MM5zFZ9QpFVZMHU5J0IBuLxscSKV84WOCXve16AqkpqfSYzdl8CkdgeXmZx+Kwl9VdvHLq1NkLYYahRWjSaZW4VO4wDo7OBgq8w2qscZWpZeyGvXfs3H/7+PTYyy/8+9yxwwiQRZxEcClXoEWOz/GiAkVhbS2CIIgkAlAk9H5eYVkUQmUoi+IyJbAYtil69Ir5+qJfGUDkcxJX0f4gF4cwCIFCRegNaq1BIUGhwHJMOpPJMD0ru7/4xTvDY7Ozs7PHPz7XaIIqhFFZ7GF/weWponMSRsglFWUD/SO4xChRmQ7GSiucg1Mxu7tUouDiyPzNd9+qLFspKcor6pqFbBYlxHQ0kYzGCApxea0IhoX8izMTM76xoUw6ySKsWWdEEZnEkWw6M+4bS2eByPFMPk2QpEKtlngsEo0o1QanxVJaVm00Gw1GQk2pYDadSc5PLYWOXrq8kEpz/69fR5NS4TYa1VBQGSw8xEUJAFbkBRmIfCKxmMsXMByVJdRe6iq16hqrO7Qm3GWgkLwmnEle7R/pmx5OCgWj0shxeZ5OO/Q6b1V1qduey8TmZub8SabG460ucxkpbSDPZ3OhvmtDw6F4RuQAAPfs2XJzz67Q4nxayPtmg6lsgEBBgZa1ZV6VncQgpaIlJsvp1DAQ82lxQVaYFaxcVmot8LJeAQSOzOaZTJ6N59LXx8ZpKNRWl8gkeqV/YTHoX9VaW2MqHR336TQl3hLtbGQuL7B1euXY0uK0L5FhuZwo7dt8073f/DaF8APnT8m4okSv4yKhibkYAzmESwo0MxbO5zIpWFWFQgihDAFcxkCDZd/aMhqu6DYuOtwlsbhpLh5aRfT38kVZ3D0Xh3oAIEFiChWJ4QjHsrksk88LUgFkaGFVd9uOjV0U4JrqXYdOfrp4YbGpqtzucgaiiL1hnbe+MziXyGVjKKBTiSgmc0q1KhaY8s1Ng0xAq8JjKdqqw60adG4+qymvdHk7W7u2qQx2SqMiKIxO52LJsN/ngwxmL1GH/QuXL1xaiM4EYlEUyKQGxQAKeUxnorRKYyqW1VI4gpJum8FsNpeV1KR4EPSHGDm8EJy72tcfS+Y7V7aMTS0ms4xapcREUU1gtR4PJmuy+bDOoImmsjEmhwKJxHCIQ7tFRWfyvIxjAJeFfI7lU9l8KJZkRB4AgCCgwm3d2NGxonGFwWxHBWm499rFgctJnp0OROlCXqemOhqqWt2eMmcJadQO9Q/MTox0NLd1dW/WUzpEjMaTmWAqEaLzepwUlZq6+noSx3RqWWIYNhkdmVq0V5TrUPHsmSv/+PgzXyILAKAg8FiNNRW1GpGf8s/7Uhmnw1JtdnotNlxld2sLOocLzbA6FZ8irWEuf3l0bmhw0FNSYdaaBUKqqrIFrpy8cOpqSMLUMtTrbHvu//qd99958KXnxs4c375+tdNVpiR1uFknacngbCw9t+CbH8godWg6A6uq0WXyLlw+tIp6qSSCZfJyERNQjOWA5SgYhLBoiCnaSqVlXjhAIIpiEMURCUhcgecYURAlABEoyrksf8f+zY8+9tUXXzsw33esrl4Tz0hrWu5ct3rNR5+cAlRF08p1GpUinZElCOw2BZAwOleYm1qU5RzKFgILC/FYGGUCbHwqTYdklmur0S6E8u76NdUdW8vr6gt5KZ1gjW69wSRNn+0b6BthYVhndSoxtBDLxBN+nVlXiIALl6+MRuYyBVaQMRaVCiyjwFBUll1mByRITBYIgOrUGkytymOo1aBY2VEVD9KjfVeMKD7o988xrDKPGkxOl4Wsq65ub2hUKrDr13tnk36MQDWo3ki6DA4cFYKpWJLN8tNLeYfXyBH44mJ4OpAbn59lRE6BgfqKkvrS8jKzw6RTBOKB+blEgecmY8lEOtrgdNfY3DmeKSBSkmWmpqYsOo3FZqsrL6svqS4lLGPzU4H0kqfFOT3m5yIpQmckcFW1twEjYicu93qN+hymH18KXJ2YCSVjAMgYgnR6S++7485oOPX2h+8NhyP/O3oJHKu02cpMdiuKdbV7127YjqNkIRW9dPY06W4oxTQ5kUfcGjojQLWlZU0nls08+9fffHT4o2BBzAFAAmClyM1Nrfu3b8W0OImpLVotgRDzI9dhZdXyahAi/xutwHLeAQBJLNagAQCA9Hkzb1F6Q1AEwRBRLGrzCCh20QAo8qLACzwnSZIMAApQiAGEKfDf+uljj3//e3/981dolqhqrDzy9vEf/vgfqeD8s9/9oaums/PuX7ssJErwkkhAgABJFniAYgCgKElhdJYFMgIkEZM438x8LLLknxiITx/hGF/rup1t2x90VVardap8MpVP+Rd7zxRigdKWFoTzczlUorzpPJIo+K0elxYgTI7xLyzSTDASWBy8MhdlMRQXVHpDJpqIZPiGxsoSj1VIivFQaCmwxEhZBQZ1GCyra4mlmCyTVWOgvbx2PsGML42SGO62OS1WJ0RZDtPqNUpZ4MJLoaGleQMhtnfudFe7dTqghbxvPCLngzhgGRmGEgVcIrOAhLkkQASop2Zn/VOTcx6bSaE1RqOBhQS9tXO126KdmZzFAa+rqvFFoxcv908t+VkAUABcek1TqdtlVZhwJY7i56YCk0sLNpXKYFI7SgwWtaLUWqVH2QwN5iOx06MTAwsBQZb0SvIHd9y9sbtp4Orw0YsnRoNRnSApUGQ4U4hwwvLgBYBNTYkQ2kwOp82ZLTBJVuYgq1Nrv3DjnRgp/va3vx1Ox4pfAYliAEU4UcBFyQiBTkkssUKNs6TLU9FVXQcrqoou0OL1B4CMyMXbDED5fwEvAGQZSJIMEUSWlw3Hy+RLCBC0OLOjAi8JnCQIMpBliEFZliCACILSdN7g0O+/a3f/hasdK9a1tdoXplMPffnXS4OX//DAnZFoJklQ9a27H/7+H20ePZCAIMICy+s0CJ0rWrhQlRZSCggBmg3LdEZWqMhCIT870Ttx9pmKEnVzz2axkM+lMzLHQLmQo1mLy+6ubfQtpQ02s8mo5xk27p+OzwwnBJbgcwlfCsGwOJ8bXQhFaJLJ5KwWrdViFxhcEpS2Mms6GY8zOYOZMmh0cjxeVrOyadua5LyfK2QzSxGBSfv8kYOnjs/GFlOFHMPJEAAtAbqb6jZ0VKsJWN/SGZ4N5BI5g8WK2e08RZgtFp3eEQlMXT99kvaHJY4DWjUmiJCgHB6DElVhRvXUaOzKmb6x+NJMMioD0Olx3bNto8tmCQeDmTxvdtoNkDgz0f/Z1evjgVTxO1BgSKfX3tW0Uq81Hu87fHLADwDQ6jQbWms3r1njUlOzl/uicZZBkf75uSH/UpbnnXrqvj33VFrswxPji6Pnh5aCSgRQKk0CIDJOGnCywVmtsdWiSAEjgdlSanDqx+ZG33v7HV86YSdAa3uHgtIFfYFwMpyksxlO+J+c31NakmKFkXAQAIABACuqls3uyyGa5XAphBIoEnREEUjFIvFlAx+QiijoYkAeAIhCWZYkAYiiJMsIRFCOyRcYQKohTuJSnlOYPN///hcVeMy/UJiZzNjt2l/97g/B+akf3bCpsdRd0NRlMZu3urG9ew2lVaoMyjxNGK0YkCSmIMoilDhEoyMRHKAiABCRRYgAIEEJIRWx+fHxj34B6LmyWndZ5zpC5pO+pWgWBQWuorvZ4KiMD10dvXx9tM+XEBU1rQ3u+goVnlG7LJkYM3XmdCaVFvJynuFkrU6pNSACPh9cZBLhGCOUVjeV19ZrJYHmM0AulfgAoRaMFosgq1GAVDWUoBKMLE5F/dG5xVgiTUYSwTTj1xF4c7VKiMUERJPD0UQilZ0e5UiyzmsjKaNO77V3rnWYVOlIcuL6Uu/C8LXp0fn5gAgAhRNmpcpl9+i1qEaDZ1L0+fHZFJPrqihbt6rbqTXG03N8obC1ZwsgNaevDB84dnQ4MFv8H61wWtZUNTiU2slI9Pr4ECqDDOQRBK2t97aUVs/MTNHZWGd1gyyh87HEWCw3H50TRLmzunp1UxPDkKeGeycWR5G8UHRsLncpAIACUKJT79685oH9d2pk8pcvvHhsbGB1ZcMdN9xWVuomYH5hPhKOMROBSDo7fvjcKX+c/um6lk99sd55f4nZBL2V6LK7c7lOb3lUX7Z+Arl4YhVfiEVg/bITEEAgI0U1VQSiJAMgoaIgsjRncTrXbFkXTvgicxEVonCXeW68ZRel1FKAQBFhxaqtaiV/d89as6hate/e8dlsRVVr96Y1yUQ6m8qqNIS1xBH3M1qdhlBjHC8pVDCX4pUaHKcQDAEKFczGZYwCWrXixBvvXH3/caed83qNpQ0Npat2odqSQjalUFCQDfIiz+YlOpkuzE5o7Fo6J6bC6fnxBVnIOmtrVFpjPJoILsUDoVg0ns4D4DWTzrqSxflQKFyw6ix2jwkQJOBRBnoUalwpMXRqJp2nTUZXebWT1Cn4eJxOETKgMaUCA0Iw5I+GMyxNAy3lCy3R0XRLQ0VlbQnBShkuOTu5lAxHXNYac22lUq/RKp1KBZIKTkej6eG5OMNkQvnC+MxSmmZRIGAUYSPQPCteWwoBAEgM37mmbV2VXRChWV9TWe3UWPSnj13+5Pixq1OzKVkEADQ6Sr60q0dn0Q30j8ai8waNOs1LtEQ5bZ54aubU0AzHiKVm48qmus62SgWqvD4yH0glCFSgoBUjFMF0Jhzx0wLPI5wSYQGOszweTzCBaLShplSj1neVeCXJTsmF0dmrEZIos5rKNB6P1lLT3ZHyj0z39r54/IwkJL92ywOf9F25PDkAK6rQosJQdK4jCFjeDErFiDIoCukIshwxlQH8PHwIJUkGxQ8LQo7lUmFRQeIuT0PF6qr6qlJMBiatkdfKJBDiczGeo4x6zaq13avbtzz3p59//7Ff3b7r/hJvLUKoSitqFDhB6nRqtcHpMUoMN3h1yVtrN9oMgM9LFCbJkM9CGUCjGYiMSGiVKCK//KcnX/j7LzVKocNt2LraaHRqveu2la67k+chl4oCQoepXQhOACHGpRf5HB0e7ENwwuStJHVGTOQAZsYoGcPRWDi10HfxzMnDxz+5rEVFtcUA9Va6wGsAV1ddClk2FQrb12yhI+HA9MhClFWaS6rKHdUeL6XUS5IgS3RybiqbSGZYKVcoIARpMWhklJARCQNIeCmQDC0hKqzEbVR5bCnf3ETvPEIYmjobqssbSYNeyAtTM3Mxf2A2tMgIssNUT4AYr1BdG5/NZZO1FdWCJI8vBIKhgFZF1HtLt/es1qlQhUpd17oZk/nxa30Hjn5y4NpgAQAAwC0r2m7dcbvegGaTQTbHMnnaaHE7HBSOyNPBzNEr/Zf6h+MF2qZRrm9pXLP+Lp5NK3Fao/NSBEEgGAkKSqNRRlSInMUQyDB0FuVlhJieChigrMjy8ZwYTSVZkhMLAlpA7aUlHVtW8Qxg4rxCKc/6p3Caal5ZduD9V2FlDYIAWCzELn5YCAAAgZL0efbv89icLAMEQlmSIERkCOTlgBjKCVwmKygRZcfKrVv2bFEZsGwkmkxGktn47MTCwtQcRuqdpfaulauqKyraW9Z67N67blh79tjlr9/+IAAMZXa1br5JZ3EyNC9JlMdrmxyaKWt2oDjOZwqjZ6+27VgnS5jMowgUQUEWREEG0kt/ffyzt/7GkVisIDer0Y1VGpOL6tjcULemMw9tqMqLUBalUsinFiQBAEyHCjxPR5h4IuObyacFXKXKSUQBChqtwqjVixlu6PzluQx9eTaWjC1VOa1KKJy7MAmNus6O8rVtrfMzwZwkrtmxWWJjH7796XD/GJCA0+Zav3t309ZNBpV6bmhgbGAgF1gwqXGAgolYTBQRr9OqUhgoJUwxFBcPskyqvs6RhZqB0aULw0OxRMaqUlXX1a5cs7K+vMJs1Ab8M58euX5m6LpCxAhM2rJzg8CIU2Mz5S53bUUDAPnx4KKNVJhNOK/AUgxN4tbqurZSo2249+LLHxw41N9fVD9by8t2buhprK21ay2JYCCXj5JqIyWwpMbOZTOx8OLo+PRCljFq1AhQustNleXOQiYr4hSqsDCpJJsAKpNKr1brtJBlkEtXB1E5PRaYnE7E0zSXp1mFBOxKNalWEEatxmxrr1xRWVlhdpSWua0AcrFgxGIyQG8lAotiA5QRDEBQ5BQCAIqGd/A/z0NRkYfw884FiHGCnEoXdCrlzt13rtu0ncUyff1Hxy8NzU6EEukUy0kQArvNwWVlXmQRRLrh9tv+8ZfnY4vBTd1NYqbws698J7IY+eTAiwq9UV1SY3dU1nd2uSqrSyqq1To7SmAcx0MoIRiSSxdUFKAoXOYRDCGe+b/Hjn7wdw4ns4KYF2U1JrXo8SoHuePWRr1OjehcJZvuTk2Nh8+/zNJJT9sKRO/CCEJT2cX5w7lYKBqNzUzHktEpTsQVai0pZQ0OI6FzpcZDMqIiNfmkaKfTheGB0ROXLgZYqcThaKi3O5QcEmQra0s6d20iuMWhvt5r/ZGh6RCFg6YV6zvXb12xYaVOoQrNhSb6RtP5JQDoNBNZGl0KzwccNnt7T7tGYxsfuDS/ONuwelWJvTyTxkeGzx84f30hlMIIsH/zus1b9zc2KzOp+QNvfXZ+NsUEo41NbZ4aZ+/gUO/AXCyTNWqU5WXOphLrptZ6i8u5lAgtLvlFFivxNFaUGxYmAycuDw5OzEbScUHkNSTlNTlK66uaPc46j2e2/1reYLSXGdSCIhPK5EWYYdOzs7PZZGRidupiNMcBoAQAgSAnAwQAgkA9amNHZXNDZbsSFc9N9x24cl4QeadO2V3vtZW4FiM0FDCdWqGRCQIRKKOzrKHHbjIrgUSpNdBdiqAoAlEAgAwhQAACi2lSsCyBgs8HOgRFZKlIAYIAlbM5DoPgxhvv2LnntpnFiQ8/fGv42iCfAziOYAoosKKSomqrV5itpaSCgDCFaOT77r6vu+PGvktX9+zcbMSoNc3N/qVQNhxAgThN5wsAUABY1Eqz2e4sa3BW1lbVtVpLS212jxJXURqlWkvyNDj58aHnH7/PpoYFlh/LFTgEDefzO+ya7lJFy7qyph27ocZN6exMMpyOTCkoXK9VKo0OESUJbbmAqFCdNr800v/2e9Ozc2YSur1eVK9R61B7+SoIZDmfycyORGJRaKhUUCUTvsVTJ08dvdCXyiS0EKRJgABgVaL3b1uvcWkHz1wqr6jwWnlfLLeY4JGCRGqUZWW17roOT3WDRueg7FYRoxLR0OD5Q0PnPtIC4Kxfo8J0odg0M3tFRtW43umtrSAJ7NMr/ReOX3XoSZuzTIHiVV6Hs6R+MZ/+6KND2Ui8vq6q3Fw+Eh85cmE0kikAALQEXFdTtW1NZ22Tl5XFJV90cdEfC8TcZkdTY7NSxSsMWknQRKPRhaW5waEBVKMtM1c3eEqtFQ4VxgqQD89HIlDCOa3L4ZQlYnJ2tH9sbNK3cM/mdZ98dvTj8TElRuQ+3/9019d+acfeGlfjX9597uDFsyaTtqOhfEVDK8MTodgCAJJNb4IFUUFilTX1erXTYnVDlxsFCFxOQy/v/BC5eFIVvVcyUiz+KC6qIYaKosDkxNWrV9374MMMTb/xxr/7+wcRCeA4jpI4x7McLe3YvL2ptYcupBf9s8FgMBRYoPn87tvufPo3T188dWbLjs12nHBg2GwqAxBMgWMyBIwkIwimFHme52UgEwCQGKqiFFpjiUZnKatqcFTWrere+cHLf57r/0ApySzPT2a5mXyhIEkNCmJfvdlRQrjKFeaq2rKWHqmQl7I5nccjIxaO5kQ2TWkyEKdwQwtfYHFSQ6pViFoti5Iowxydg0IOZBKIwgwpfT4RK2QCUiHORhLR6dnFpcwiSyWS2XBc5CMz5+cWGYb+2r3baqvq3335+UCKbmuo7GzxOuur6CwyMzGzNDGW47OixJVYbZWta1s23FzTvRrDFBfPfHLu2WcBXujo3qDRaXy+uRMXJ8PZjNtu1wk0pZJmJ6ZtNXU2tSGTSE3NTTvLaqqaG3KFwvDQhF4pq6DCYFD7E4mzl6+PhjNplncoqLVNFTtXN0iIghNRRKMevX5dxg02k1lMBM9PzbPpLI3iLMdEUulEntVqdZ6SshX1ZWX2MqtGj7Ewz+YLcqGyzm3VV+qNNkqvTAWzh45+evTUfxAkPx9hSkqr6FR2cMbHAPnePXu/ftNDh06devKtZ9Mso1CoNqxeRQqqsZGhifjcsgICQGWppbuxGbpKUSgXJafi07CYqVmumvk8ilPMfSEQgdkMa7cYH3zoK67S0vffefXcmXMAAozEZAlFMJTOMg6L9dbb7skL2TOnjkxN+DgeOGzaqpZylcVcWdX2m+89HvYtdvesUuUFQUJ8dAqVEUIGxa52IEMBQBFFEIDgKCBkGcMgwQsOrVKBU+kcLWvdJVpUyIQxDNET1Gl/Is5xWUky4WiTCmsqI2pryJqOytKGTgTV61yVpErJFjK5YArDMEKrITQkoXdJAhSysUImngjGBTaNIRggtBhO6I06pR4TUBWqcSG4VoRAEiCTWRKSS/TcdGByTu1oNDtd0ag8OHRp9MKZrfv2o2j+yJGB8d4ToQhT4iRra71WZ42ypKG6xZzx0xN9fUMj14XQLE+qmtd277j5i9Ur1s+ODp976Rcfnx+mTLYmr9vlNJudpYGg7/okPXy1dzIUaii133/7zWar5tN330/FMk3dLaWldaNjvYTAVVY1O+3OTGzh5Mn+E/PBhViSlcSeKvumlR0kLwYXw6vXd43NLJy8Oh3OJWgJWV1TsmfzVpJJ9k9M+uJZjstn0rnhpaVQQTYpNd3N3StXVTc1tOnUhnzY71vwFyShvL7JYDVmo5mBgdmzF09dn77OS1kVjqsQbDKeaq2qfOb7fyhw6Pvv//vtc5/5Oc6kMz+0fldD04pjl88fPfV+SODlYuK0zIsuv/UkAAAiiVJxGwiXB6liOh7IEHI8DySwdcP2TTt3Xbh46uODHwgcIEhSlBEUBTLkGVpYu3pNa9ea0+c+G7jaRylwIOMIhtfXlmfyoZH+0PpNPe+89wnK8es2ruNm/AJGTCVjGIJIAKAAIAASRVzYsisHFl0SKILoUMykpEgUsBIooaRYgrGZtLl0NsghZpWqP50GCOy2KNe3qqoqiPIqh8buXhoYKK+vkxE7qdbr61aIiBS+9Fk2ELF6TCpPNceoUwFfLBpNpyLJeCQ5H01zvF2jXLFpQ9W6HajGLBQSiFLBJ9L50LzK1QFJNB+6lvMv0UwBxIIcapoOxz87fKHCZL/lq/dgCk10Yf76xSOnL42SrOgp0xqsOtJYa/U0GaxeAZXYZPja8LXJwatlau26PQ82b78jOjX0yot/eebgBRYAJQLX99Tu7NnUWNO9GJx8+dX/nhyb/Npt27/xpa8dfv+t82c+8VaU163oplPMzOywDlIVdY0iQ5MoMZOOfXLu6oUJPw7A7hUtt+3qCiym3U7t4MjA5euRNBD75pcAAHY1tbKypXtVq39u7PLstMSyDpV1JJaYiYYBAFUltk2rt65as7GipkolFoKzvlTKr8a1FU3rVUri8tnjz/3nhTNTgxAAPYnGWLHJ4Xj0W4/XeCpRNnv46DvPvPt6pCDs7Gj/wTd+oFDon/3rz04MD2YlClbWIssnE/i8YE2GsrAcClwm+spAEAWnzbHvln1qjf6Fl/89NRVQqTAJYECUcRIt5BmNXr/75hu5Av/RgY/oDK3UKCVOQiAiy3IimXfazHfct99b7d645aaakop9d985cvgziFET8RhWZBMAGUKIFCuVln1hQJIgKH5hkoxASCKIGsVMOFSiOIoAjczP8/LqEsfbkwt2Atnm1axcbZUQVKlRGY3K8vrGfDKTiQcom93icqOkXla7cYnnmDBEBELp0HjXkJoKAIAg5ASOZzO5XHQJk+IohTBLc9nItBLN0WlMRnBKyiZjvGC1u9rXmyzVTGRywT8O40vs3NTRk1chk86ZKtdt21FRWQYS8+99cvri9RkTwhi1EoECUW2RSa+joryqxIEpDWO9V2OLU2UudefWbzRXWEdnR/795qF3jp2OFQQAQJlJs/+GG7d3r15cmvvvGx+0rapfu20zVhA/e+9AIhYtr21u7WrO5guDJ87OxCON7Y0VWiea9vf5Eu/1Ds8nYs3u0v/75v5rp/oS2TQj5RNZps7hHApETg/N8kBqaWje1VKuRsQLvUMckFfWtaA64tCVwSsj08UlXbXbvWHDtrUdKx0mC0WROK4gCMqgJArJ2GBf7/tnLgyMDjKxxdZGz6X5CC2BtZ2reuraPXbTpcG+p95/h0TAw3fd8cV9tx379JPjJz+Dtc3F1Qz43KuOyDKQRSgKUBT+1+EhVZRVVHhr5xZmBwZHEATgOMULEkQghkC2UGhsae5Y03P57IVrlwbUKgyBpAgAhkOWyWGAuPHWe1f0NI8Pnf7wo8M/++szD++5/7Gf/fSNJ/4sI3ggl8UR9PParM+z/QAUrawogDIEQJZRCBEUlUVZgwC7mprL5O0alZvCxxJpB0WM5JgOp65MD0o0ckObo2FDq9ZUmohwgGe4uRFTVRlh9cDUQj6RM1XWIxYvDvQiHUSBIOJWgOESQin0JlxtxlVmWZIKyTifC0psML3UF714AVq9C3HS3zsVyyzGE1GL09WyZm/7+hscTXaS801f77vy33dO9U6w6Yyk1rR3trZ2d6AYvnC598OLs0uJuF7FbW4rkflCIiV6nC6l2rLEK2L9vUP+BWuF++59t6xrW++Pzpw+fem/x69fGp3NywAAsHlF620bVpqMupMnjyOY+pEvf0koJN587fWFBZ/D1di1vlOnVTER/8zCzPXxRQep89j0YUby5Zm6mvoWF3roZN+cP1TZ5sKzhB7qeV44PD4yEwll2EKjw/ydvXcWrIpzxz9V5EWvU7kQJROZ+OlQ1B+NLm+mAbCaTfU1zatXrlrbtc5qcZNsXsgmFoKh6ZG+eHLu7x+fTBZyGAJECWgp9Z9+/m2EtX/3lz9Ky9l2r+vhW/e/8N67sLGzWFQFYLFfQQIAQEmCAi/LEhRFGQio0WgFCFzyLwk8IElK4IEkyggKZCjJIuhZt0Gh0x0+eCiXprUalShKogAwHGVyOU9F9UNf//bM1PWP33hRo9Jvv231qi27d6+//8jRAw/tuz/LiuLntTXF1+by511sqAHF5m4gyzKCoBAAFEItQXCimOIKJpLyEgiEKADotUxqZ1vZzZuqc+HpkhLUblVoHStV7nY+n4OMLzE/lpiPyRSkDEQhFWVzaTkvFYCxpKqmct0GlLCkgxMmq5KA+UImno7yAOIUZVBozAJQZaNzuWyCiyZ6xxnMZvU4TVcHesfO91fplazF2LipbfXqW9Q689SlQ2c+/mRhcnEqnkUprLXWu76zmlAZWNl8bWr01Ien2lt1NoNm2p+loLS6q6eydU2epo98cvCqf14jgK7Gjo7VK+vKNRlGPH15+OPTvR/1TgIA1jTVfu9LtyVjmedeP1Bf4fzSA3fqNYorZ45d6B/3VlY7vdWpUBAAzGh2e22E02idHOr94NzQ/Hzii/euyUlw6sqk2orFfHmJpVavdI/TmRcPnJ0JRgwa1Tf3bOlqbaBlZXwx0D83BHqvMpY6hZY6PDgSLLBmpTKUSi47z9SqjZv3bF29t6XOajQaiQLLRkKn+yZ//+ozs5G5jT0rbtt9y7+f+fdde++qby79xo++OZsoVFqN3913B+xY+/kW8fOUjixDWQSiCEUWYAiBoqp8oZBnaIhikoiyvCTxMgoQEYgKBblydU8wlLh2qRfHMAhxUSxWZiEsk69ra/jiN774wcv/uXDi+s37N3Ssaz/41vHKVa3P//HlS31n777xnoWAD8dQSVz2gsnF6NPnDovl4OtyPcRyJYT0ed0IBuU6vabJqM6I0vH50I5V9Xtv6EJJyWpW6dTW0Nis0aV3N3ohakB5LBdbZMUYhuP5FJdnuGSURiUBijlzqcZdXoKrbblIUJBEhdZMaKykxgn4VD7Vm5qeVxvMiL5Cik4P9c++8vEo5Lmde7cHUpHp8wOJ2aiISG3tJTd+5zs2g4tnJ66cOj8yPCctxWWt+uxAUAOZ1nUtlaTqysDwQIQrL9N01phURvfVvtHYYqBtbafHqDKqrNFU4ejAtYnZjIogW9vru5sqqssswXD2wlj86fc+STHMI7ds+8Le3W++c+D9Qyc7Oju//t17CCY0cnkQN2rGAsn+axPRhYysU37h7t337L3x8pHDh/umQ4lstUVXV6ePhsRYdG4+J7MMvaXNq1fpj16fee/MJUaQN3s93WvX4gpNZ0d9aH702pnLRF5MYcIb/RPVroqH7r7vxNmPzl6f4ApslqFRAG32kp7OVS5zVUtF1cqOKpbPPPnaOx9/cuKGHXu3Nje9deCDfffdlfRN/exvv8giaJnOBLu3F+tDl5V3SQJALhIAICorJAHLFWhBlAUJyByQRCiJQJZQWZaUSkWZpzIYiARDIQTBuYIIBVQAIo6hPMsbrPpHfviVd559lk7D2++/NxYaPXHs2qZNrfqK8nvu+onJQP3+tz986ek3czQLAESX9VcgSAAACVm+D4sfkwwhRBBsufRo+RcAylCqMxlvbKw6OzOXkQoel9qmRlatanZVlOc5zmD1uMvLcXq+kIzwoFRlsyi0OgJTkypdIRORZRQHNCT0gMAEISD5p9PBydDiogYRNGoEUTlI72ZC20IqUSguhfs+nhycDKeiZk/NxETw8tnp+qaa8lWV5z68MjgwL2fyFWXa7t3bdu/Zwgfmkpm5TBpXZmBBT5653DsxNGDUaeMyvuiLT6WAToVtba/de+t2TCP6ZhPpUDSzOBlnWIjgJXZDUuCvDYVnw/FcATRX2L906xpvec3FgcFnXj9Jouivv/cVe5nmldcPjoz4d2xav667avbKhYsj0ep6pz/h++f7g0lW+MLWNb/87qMJOpWg05euTo70j8hMdCYW8ZSXre3sTM1NTgzOrNnYaTTY//L6++Oh+JYVHW0VWrPD6Y+mW7s2pNLZ0TNnBqcmJuLJb37tF8nAwLm+AchrVzZVnek9HUhEZxNpAIBRbW6qaLt5y7b9m7ctpYS//P1fwwO9W2/YgKLyru3bB4599ufXn9zUVQ233LbsRS6ONsXOPpEDsoCLEsIUBFGQOBYRRaRIKJEEKAgyjpF2uzMUDjI0I0GCp4EsAEmGEhQhlGRe/vKP7j759lGD2tGzbf35Y0d88/N3PrT98tmJIydHfv/801994EsHzj03eP79YwcH5kdzWZqXZEkGsklH4QoyFEp/Xo1UfJhCWGQYAVBcOEEIRFnyWMwmkkgUUtt3N+sUBMlFPOVqlEdQQYmZlHw8oVTbcMBLCiIZSZaUezU4g2PGlKBx161EYTI9eYJN5Ci1WempwnleoyFFEaYj47OTE0Jy2ltRTxhbGLzE2tijVhoGzxx89d/PlypRm9O5JKZVjLaqu3FxbubQ4eGxEZ9EgK0r62+57ya3AZm9dCFHo7QgrVyzQTSp+z/+RMjn2Tx3eXDs2nRuQQC1JvTGNY3bt2621XWwubiUTfkiiyF/2sTn83lmZDrkY/Kf9s+mONBcollfX33bho4TvdPvnryypqvtkYduUqkU//jHp0kmvWVllVmh+vDMmXyBsWs0J2dil6d8XRXOZ379mJArcDmBQWB/b3RufpqVE8ML84vxOC/xWopYVVbVXV93eXz0w8ERl8X4069+obqhe2ZiWkVCrUY/N7dgdpfW9nSO9vcf/uDI1OKIWefpbm6K+ebHA2PHR6bidA4AgGBghbfxwRu+uW3PrkS2MDc1d/jUZ7H46G9+9dLl9999+9O/w5u/AOQiVgkCCIEoAhSFQEKYnMxJEs8CgUMEHuELslREUQiAxJU6nSmeiHFcnucRUZJFHpFERBRkSZJzKe6uL+7KpnLJYKipuWa4//rgqH///p3nzlz1TUdMFuw7j//1a1/42md9Z89feyYeHQ7NZFIhNhYSAYKuWu0a7J/vu5qCABEFIANZlEWSwB1mLcfy2Tyfz7NF1RZ8Xh/e1m6/YWOtXqvIpqIej7O8vIEO8EklGTn/KQmSMm4kDQogMOUVDUqcSMbC4TijVQhWp1dbWsqmCvnpMbUJ1bkqMJWJT+b5fCyPcCoYT/ojGEVCXWU4llCr1FUrb1GazB+8+Oli/wVnC6pW2Aos0GhJjQLv6588cmp2dnTBqUPWbGurb3HPD0yjEPNWukur6txNK3PxGD0T5oT4wKjvUt/w+NhMkgFtJdi2bT2CBEwuS7WnzGx3M1xBygUmB/0XeiePDC2kBZBJ5xOC4FUTj+7dZHRo/vXu6QSNrO1ouveermQi/fbrJ2oa22pK1dPzC+FgVoFLSU565+wAxPHnfvPd1Z1rx09dCERoxOTBOEapkq/OX3/raO/o5KgMgFVBVpktFCJplKqxuQWny3XD/v3l5XYNQlBa8tzlydGR0fJaV1t7Y3x4+p233m1Y037znXszvmj/Z1c/HJvsmxoVJKCE0gNtLd277tOX1LZu7VDhhie/8dPyVeV77nrw1Z/+Gd7zrWL2ZpnRRZAAAQjDyDyQOQYUCkDigcAi3PIsDzCZVFKGbC7FCQWBAwIPeV7mBQAkIIook+Frm2q6uuoWpmYsLsfS5PXp2cyqNZ1DvSPzk4n6LveKFe3hqPR/P3tSa9C+dPCJpaXTuWSEiUeZLKrV63hRXBgOZdJoNCwVPasYCm1mrc2uC4czmUwhnaMphUKlUiaTKRwDdie2aq33rlv2qIEukUsLuZh/ymdzmOrbWhMRYX7iEsnnW7u72UJ0brwXCpAt6Eobe6xVZt+lo7Hes7jOIZrKDV6TTkjBdEg2tJZ37lJq7BLPhuPzM4dfTQdzlWt3ldVKn/3nNRl1tG/cHmfBwsz03HhARDiNVt3eUi/xNMfSV4cCA8f7w5EYVFIVlSWFOG0uNzaXCpkltH5Nj72jOdA/k48VlBYPikSHpufH+4cwKOZZqalV4yk3KBAqGCz4ZrPdK9qbuuo/fvfwwbODl2fDAonmWFGDyk98bz8lywcP9w0txeYi8a2rW+7etur84NCiP3PL7m258OzxKwOAwyrcxtFQdHwmunvLqnvvvrPOW1HI5UanJs4duxIKZNfu3M5l5r7156fDBc6oIhM06zEb19dVX5+JDgRm9Eqio7Xj7ptvaGysOX3h0svPvrYUCz300L1qhH39rQ8aGzv/+IffyunU0cOHD5wdnlkaW4gleqq85R5nk66aoZT1u27ctX3jL+/60urtWxsb2+EXfgRQrGiFKTZygwIHgAx4AbAMKFIteQ5IMhAKQOSAktTncnmO5QQOYVkgS3KBBZIIZEmWRQRI6N0P3N576XJTS2vv5X5MTqlV1elc+uKF4fXrGtpWN73yzIHGrrXP/vt5FFeeGbhw+tjzOXqCoUNMRkiHgSgJlAKVOHR2hE8keQzHIIR8kdz6+RMDAKCgiJpqtdGpZAs5vV6xsrO1q2NrZWl9Lp08d+WYb/KUVYrXte+iWSIV9akQxu6uQgldYmneZLAbS0umBy5oLW1WtyUyNpCIz9ptotLoNbqr+ByglEwh4ivkE5yk09mbU5nY228eDMXwr35l+2j/1SsXQ6vWdsqiaLToBid8C764TkdUN3qtDkTDg7mpwtDFQaUWjRZiKr2R5qiRq5MOp6LWq3XoXA3dXQLMZjPKbCJeiKRYiZFRyWLSFgTp6sBsqQv3ql0qs6JvdlGvqthz4+rrZ/uPnLr80bXZiMSLEHGq8Wd/cudg32zv0KJKg0yG6fHF2F1bujxm9bsXx+o9nvJym5Dnwom0kI7S6dT12QCrUO3b2rNtTVetxyFC+eChSx/09ddgejNUH/f5ctnQjq0dL394Olngf/HgLwPp6HPvP1W0blo1ij3rNmxetYlFM79/6imTw7t35eaX3n6JVRDPPfPPdqf9N3/462ImOD420b/gBwCUmvR3bVuH5I1rbttnVJrfe/LFb/3h2/CRXwEEAgQBOA4AALksgBhAYJFCCAQBiOLnMCYOQAnjacixPFuAAgc5QRZ4yPMQyECUpTwjr2hrlSRSFGhZQng2nE7IVkvpyeO9Npdi86a6M5+Ouxs6v/iVL//m57/66e/+WFLZ9MqrT6Si53K5UDKSoXMIquDVFKRTApvDFud5jgcMLUmSjACgUJIYhooChxGQItESr7Ky1sxLDJBksSA0VHutZrvDVpKkBZBd6D1xIpHl9GpMhKSUYNxWQ4nXbXaVcLRc6S0L+5cunTivtVfXrV3nrSilYD48eD4vQEAqotE4RaIyu5TyzcdkldvoURGKTwdDganUPffUTs4u9F3PbtleW4jkCa0SIXF/nI6NxTGr2mwAdrUWReX+ywvemjKBTZGUJivSnx5f4KX0uiaFQKMaSrVy/Wqd25WNxY8cHhmfmKM02hu317jczv/+95RvKbN9fX3bSvviQjgyG12543aUD/hnwocvzp0Z8ynUcF1T4+pqxaHL45P+7EqvA6XAJ9cWmry2m9bU/ua1c0uZdK3b0VRX21rujEejxy9fDSboSIEz4aDUbd62quWmjXsjcf+kP9m55oYyq/qxx36TLwRu27v95fc/LDOof/C7A48+cN9nc+dkgnCQKFfgVnjdm3vWq5XGIyc/tlhq1q+t/9vLbyzEUu/9500sM7/xi9/Y2LUzlVq6NDlUfE7tWdUpxKPf/N0fl84uzgUC8Ou/BxAAHAdQAiwPJAhwDAAAJGHZ4icWwXEAyDzg8yibFyUWcCwUeCAIMpuHggRkEQiSLMt4V9eW8f6Bikp3wO9nmLRO7bg+EJZBtq7OHg8FcVXzw1//2fNPPJbK5UqqO3/3l2d7B89duvxmJjmbSgZkuSDkJUGQ6YxA4FgqJlIKnEmDeEgosAJO4SgGMYJzlKjMRnUyle1YUY3jqFKhdlkt4fi8KOedJoOSsEan/RqTPhqNyoVcNpINzechhgMpYnfaq5rqmMgoSKGkEg2m4okEY9JqbQYzbjBkc1mdjleSOA6AKBEmg9LkWSFC/sjLB5eyKmuNholkujoq5qPx4yeW7tjXHvUHshnBbXfKKjEUT6SDBM3SMoOs3WCZGAqpNIZYIipyyJZdPYdPzZ47caW5yWgzAQ3Ma1T6hqrairqu8ZmF/rHxvpGACsNu2905veh76u2LnXW2O27cIeZTZ6/03bhvn0VlDk8Mjk4tDi+mt+7bQRWSCoK/2D/630/6rUrTl+7a9MGVkeRc5JsP3fLxpYnXj50AAFh0mts3bWyss5/vG1maWxyeCkYlXoHAH9+zd8uuGwCbT0Zm9dbWFRs3fOPhL0TD6cd+/qOrR96u7tlpVJU/+YefHx0Y6W4r/ezKgEqjSKayN67fcNP6bX947e811uo7Vjf/+aMDC4HUh6+8Ehy88MTL73z9i48+9cbz5ydHMRQVRNkApc6OVU/9/em/fP9X8Ou/BwgKUAh4AUgywEkAAUCL0JLP34lFdhzPATYHoSwLHBAFIHCA46DAyaIERQGwgmwzV6pVJdn4osVknRgbRlA0l5ECoXxJCYlLSIZG933hZ2/88x/tHd7+wYSnqv7Xf/pTIDR55OibgcVr2cwCW0hDGebSsigJACAFRtboMJaBqajIc7IkIjiO4goRYqLLaUCBTCmpVSs7gr44TgCUJCpKPASKeasrMylRieEiQ4usqNEa6Uwi6gtkmUwWZvh00mgxlLjqEURDkUpCQ3IQcHROZlAxm+HBol7BeGrKMMKYigXmx+d0xhKtKv3ia1c+OBRa2WbqXmFToPJcQvj0qG/tCosMaVGAzY0NJCbMjsTUZv2UL+dbDG3utkwNpswuldGg9M2Ebrpzrz+Q/dvf3kUVcEO3SQV4JsJXeh1dWzp1BtPU1Zkz/UNHrwa+eNsmgMhP/+eEt1pTaSnv7Ko5fep8XUNXfUu9QIcvnO774OhlrVG354ZdPS01OJV6460LRy/2/+LrN3Mk9eE7J/ft3Jbk0V8+/59IPgcA6KzxfONLX45PL8YXBs9PLZyZDoqysKrJ++v/+0W1t+zyoUO8Urfnnlsf+9JDFSUtazd1jQ7OrNi0Skihn3x0OZGe/ss7L0OAQgglWbirs13AxN7F1IoS05071rz6yeDZ8eu/fOy7lYq8qDSYNOVPPPXMm5dOYDi+uWHFqesXH/3Wd+9auwN+7Q9AhkDgAEABgQEUAxABKLpMGgef12uLIhBEkE8DHAeiCCQeSDwQZcBzQBSAJEJOQCpKdiQTUQUq53N8KDZB56V0glOocBzFkjFm3x3fun5p9qN3P+pZ00qZvbtuuQXDFCazanDk6szk0WR6PLyQolQYQ0ukWiYgQud4o51KRVmGFhCAK1AyHGbMThWlQEVBMJlUyQRTV1fbUF+3OLegU5tIJZdIhAwKFcPi3up6h9kRnpuvrOtyuT0oLycDMZVBCTFMljCO5hUkClESwaFSpWZyBRRH9VaHwBeS/sF8fEISJLW5UQCR8x/989qVufJG81yA+OSjoMGF3LWrHM+ByXTy7Pl4e5Nda8ykA2hru7ek1nbpxITdXpJjCx+8e65jRbmRhAIHrR4zEISm5gZEkXvmz2dHlhJbVtqMJN4/Hi2zKNZvau9aWZuISecuX7t6eaZndUdpqeK1965Oz8xXuqp27uucHhuNR+TWFZ2VjeUFkb18/MqFK/2yAGprGm6+oXUxnfn3029t39a+/db9p986VF1htzpLfvuv/x4eHJOBvLa2+oH9dy6FZ+VgwJ9EVFb0zROXQ8nkiprym27abZHyVUaTzlv5xqufde7e63K4coGJ8lLD0hIAUHnXow+muKSBpMI001Xq+tMPf/2FHz8ak6Wbast//P3HHv7Vny6NjT390x9evnLma4/+tFQsPPrEU/85cuiJ7/7k1ffemQj4Pn3uAHz4d8theYwAOPo5bBwFEH6Om4MAIkACQJQAmwEUtQxZRSQgiEAUAZAgL8gIMJZYbvDNX9ArnOlUIs2FA4EoTkIgYrkMb9GVrlrz0N9+94udW7frXTUdPT2jI8PR2KzBYHPavf0Dn+b58dkxnywDQgHUKoqj+RzNmmxUJJBHCUSpUBTyIpMVERSzubR8gZNEaHVqgCA77M7yUm8iyUI8i+McnchEk7TMAzGfSmcyFFSUlDZu3rg3SxeW5kY0JFZWUZPOk0qNOhMP5JMJghD1Rk+p00Xy2UQ8hFF4jhP1OiYaTSE8CQ30kY+PxGcyHStqUI3q+Rf7C5zw0J3VqQgfZBKX+pLVTnNTrTIQY0hUtWNv/cnDg/mYoWuz47X/XFCqiTVNZVyBkdWUiEiVdntlteHopyMfHJqs9uo2dJd+eGxybinWWmPfc+OKNavXBFKJN54/mAhmm1qarCbbQHDk1PHxuipHTZ1NmcetXkcqyOJqsqapigdgfDL00bsfr2qt2XDLpnffOdDbP33Lg3d5CaTCqam0uA8cGfrXhx8vMWGS1H3/K1+8euX8tYuDWzfcsOvemwcHr/7jqedn4skSvXpjU/nKyiZcZzI6Kqp71mejk5mFRVtZldtd/dwrB5/575MP7tz87kefTqZTP3zofpfe8vU//5lCwe8febihqmrrtx8zqpQSLyhNum/v39FQVvHU25+tqFpFqNGfPvunzc0d8IHfABkAFAUIAlAUYEWMb7EGGS0alJcfjAACMQ+UFASIDCQAAZBEIAEAZUQQJApWGJSrw4E+VLZkCvPJTKLAMiiETEFORsW1q24fvBQxWUDPtn28pNaZ0N5LB4OB2Vg8v2vP/TH/dCBwbWJiFJEFpYagabG4PaSUSCLKYhSi0VA0zZMInk7lLTYjiWMFlrM51SJfkCTEbNZJPFgM+ss8Lp1CbTFVl1c1MEyCo5lweCmTScu8tLg4H5wJoLJg1FKiqASkBuXiwfkcJwNXiaKro619455SW0naNzc9PYIoMu2tq3KhwMjY2UuDYf+iXFuic9SqDBrNU/+8LsngkQc6xgaiPM5duhw3qkmLE5Z5dQuT2d23tc2NRUcvx+/80rrXX7syN7tYXWktd5A8AkkCGjTGFWtbxs8OvXO4z6JX33Vnzxsfjh+/OKjXovU19gfu2Laiu/7Tt84cOHA+IZE37Gpub2g4e2V04Mogxxaamyrr62s1Oq2U4+LJKCYavV0dU1PXT3548ps/euCz9/p+ffC90hLnmobqtZ3lt61dx3CaYwc/uugLxFi00WXAJdoXjvZOzv79dz/2eprfevvN377wUpzhqnWaR++/v8TdwFstCqVSnfOFIulVm3cSKv3Xv/KVrhX11z47HpYKJ0dGn3zsO0P9sy8ePeDQq5945KErE8F/HzxQ7S3pm542Kcm7utob1vQMjQR6Wtf+5Imfz2Ui8P7fLGcoUASg+DKcAha/JPA51RcCqdgumQckBjFCloViu5EsAwBlVBBFNdKqwSpS6Vkxb43nR3N0COIyV5DonAR4vMn7xWgsumrLXt/cnNFZNj17gc5NzI34GZDZvu1Btai4cv1QKDDF5GheAjiKyKLEMCKlQumcqFOTEAGFgoCjUjIiQhQvrzRKUGA5XqsjtDqdLPIGpYrlaZPNbNCbo+GUUm22usyl5gqjqVLCQCERn5ybDMz3GtS6aCzHAa6xphWKCZwQF30RQWVTirkml62x9VadvTKTDk1cPTIxct5bXVvisQ+dOfTme2NhGm9v1qxsdWFKxd+f6rXbqK98Zd3pgzML8cjIXMxrtyooqWNN6bVzixu3NvnmwpdO+r7/471vvXv+xAXf1pVlDfWWSDwpFji33V1VXeWfmLo2OKBUG7/6lQeef+vkc+8fo0jSigm37Gq85/6bU5HYq++ceuvQmEGrvHv/5q6V9aFA4uqxS0cHBuorvY985ZtVlbb+k8fOHL7+wHe/nsouPv2nv377O4+//c7Bp44dVqK4RqeodJp++K1v7tqwNji1NJnJvvSv17/8yL2V3qoffOPnxy4duu/Lt/7qm1878+HZ/b/8w74b1tWhxLY7HxyanPMHUzfcfPOBF59p2bR2yy13vPjXv6KCdPXq5Vw8MpWIjvp9Tz72rZfeOnh2dHxttef955/719NvD/v8nw1dzDKpTQ0VFo115cY1DpO3kE4+/Kv/g/f9BgAIUBRgOEBRACBYTuwgy2fYMnMVARACgQGIBAmlLAkAkSGCFU8zVBJEA9Khw70peimdljLcvCQmZCjl8xLLySiqT0zq93/hJ6FQFPDxEKsKLhy2OhXD/RM2p35z191nzw0AJJZM+JYWQhBIsiyKvJxNSphSRgBEMAhxQGEoAGIhA/gCimCI3a1MpxmtgbTZjBSpDIZjbqcVxUm3zY3r1FOjI7msz+mxYrLRaq/2ut1GvdNiMPvnZ3hOQhHKZrbmstFwLEiqFTilqaxZNzTYN9H3sdfhbuveUqr09o9efPudf5iNup7uznxo6r1Pr/cPpcvcxht2lpV5Hf94rhcT+bvu7pgfDZ3rn4qmQGOdMRVkN2yrPXlyaOWKqpn50LULkd/+dNebh0YHJgIdXndjiZLm2LlICsVUXpcJoeMLSxEJ4A9/4ztXR2ef+OsLkVzeoiVLtdRXv767ptS6uJB55+jYeycuogDcuHVNZ0MrpdPQmfn+S+cRpenW22/HAXXqs4M2EnE1VsX8iVv2f+nhrz56aGJyW0N5jhdOj0x3t1X98Zc/Wb1q8+t//LWlpmHbXV+cuDL8+E+//ea5S4//+IvfufOrX/vGY3qPrafO47SXlK++4flff+vGu39x5MPXDnz239cOHirEQmcOftiza/9ffvTdkhLNb94/ZtbrVtW5L04s8ln6pX/8ua2+9Zln3ppNhg8c+aS9vqSrzGlwekpLa9sbKp7507/gg38EEAIEAgQDCAQ4ARCsWH+8TPL9322IokDiARQgpQaSACRpmWwvQwgkWQNqdbCSldKxSJzmp0gV4ASJ4yWIgnQKGMmbSivWzI9fspVXDfYPakxZKMkqBJMplQ61nztzzlvjBlJuYXaWYXIsU8ikJJRE1RqczUsFViQoGYMITmIUhgsCDPpybo+qwAiijBgcuNNuFkQkHA60NdUJIrDaGqFEX7t63GozqjXasYkpk9VksygrvT0q1EIptXwhK+bSLrub43CAiOOjFwp0xlG7R6Z0Q4f/7Q/OWSuru5o6tSrdB6+/QJJgw8YVFOBOHxs6emFeUMAta8vv2Nv17jvXR4cXH/n6doxlXn7j8mKS6Wo3L05wXdudw71L3e3l14cic2OxH//fjc/+p+/89ZmV1dabtjbqVeRQnz8LEQWaC/qSEOcpitx3x5dUDuNzz/732PGLOUnurrI9eF83CeVMltRbje99cumjE/0pANxGS1dXTYVJVLMqXziAG92773h4/MzxSyeO3nTn9raVbamY6oaHHqlzW7btvsFZVv7Qd79tViIvPPkEk+FOHnvv8T8/BWzmhWNnvvfTP/ZH5o698GQiBX79m58lAdjssf/27VMXj3+M5kRZrd5w175NjXV/evwnnx345JuPP3XwXy8/9/rTZ+fntnesqCixfXL2jEbvuGPL2i9+5aG/PPFCVW3F9WsX+mbmv/vgvksnh7q37jIogEpRQDAMYDhAMIBiACcBQQKCBDgOEBQUlyr/QzVJMkDw5VSFBIAMgAigVAyMQSDIGV4uYChUqlBSARRKglCgShVKUIhKrSgrqfUvXuLQRZ5TyFi/0kBAoaDC9DrSMLpwvaqyplDgTFqL2aYHGKY1KywuzOLEFQoMwQBFQbEA8rQgCxDikIcsoUayWVZpQFlW0FDaubkoiqJag252xocqhFBwWJAJT7U3kxEAodNXVgCYn5qaP3Xy9Y8O//NK30fTgevDC2c+Of1i3+Rn4WTQUdZuK3ei0Ut1amtj582fnF565p8nfvTLvxz6+MMd995ZQNVP/+PdsYnMLTfdft+dKzSY+OI7kz/62UcbNlZu2VXz+999UIC6H3xnv8OsPn8hVN6imbiWrKku6xsJrF7ptpfr/vDHI1+9s2nvGs/x4aXXDg5GY3DVOq/JIg2Ph0gSxaFWQUgv//OJq58d+tn37n/6ie9vWlEzvJB8+78X1Vohn1/sPTXxk2/e+u7fvrXda1ZQ7MXLvY+/fvHFc/1WT2lzJXHqnWfDPFXRVfHcswfOHD7b2F6xsqWmtrLq+rkz6zfseuqXfwwx0oNf/24sTo/6wv/8+1Mapa56Vcdd+7cnsvl3Pzy+dkMnrlLPLkT+c3rw7Vf+uW7/3aHsrE6va3DaTwyP/fRH/xeJBBbDCyu3bqgxl4qSrMGJu+74+v277ti/cXNVc/uV85d27t196MChL991V43Vdel6oHPNJjaPxqJCgocILM5VOIAogCgQAeAlIErLvqhlEUsusjABggJIyEX8EkCALMuiVLQSQ1HKSyArSgWcFLRqFYGRpAInSQTFJKXSmOMTBWESwwv5zKxWLZXp63ACYAY8RI/IqqmykgqHqYwDCZ0RVWlZhRpo9DiOgzzL5FmWVMgSkEgFStOFTJbGoKyhCAzBUQwhVLLPF6VIdH5+1mYyq3SG2Zkgi+b8qf54Jmv0GOLJSLmztbKip76hR2WppvQKf2pqeuZiNBUqqXXlkUyEXUxkFvOilSitPX/+X1xudM/+9RqIJKPyB0evPvfXd5sqV2y97Y6T1/qef/e/jU3rb93Tded2ZzCc/85PDqM88f3Htr/4ysefnLz8jS/saquzXbiwWNtiT4VptZp678OxrhUOrUXx2C8+3bC68icPrImkMyf6x0ZGgg6lbsfWNoEgwxl6eom12U2D1y//9pe/lXn6p1/e86svbVUY1X/6+3lXTUtJi+e7P/gri6JP/fYHbWbdTaur71pdNRmK/+LVw8f6IvtuXz/S91HfeHTfA7ecOnMRV/ArG8sYlm6vLn395T98+dHv39u6FiXxV99///ab7v/jC/+58Om7Cpt187bVj95669zgCK7V3n/L3fvK7KU61e//9resf7rUW5b0LezoWgkRpHdi8ehnp2PhRZ3XUVJutxEKfyCQY+GN+x9uqmvVaRyXB8ZtevXA7NXfP/ufb375C3LUlxJ8JTVuPpd67ol/IwgCZBQgOEAJAPFljrNUdNshAMUBigMMAxgBEATIEAAUyABCKEEoY4SMEQDFAYYDlOQRhAOIKMIkRARelChUTWIqDKpRSCgJQeTnFZidLoypKSUQEFTFYwogCFmjSiegcZfDCrG4t9RiNOloNicKYp4WFDpZp0clCOxlCp0BxXEAOKhQkCzHZ7IsxwkkgQkCX+B5lUYxPbmAEagKt8b8rEahtjhsFKEyWvWJxassglAO2/qNu1ua16zpvmXD9v2mynpJ5bJ46lUKjcNl+fj9D75038/++eqpwd7em9fXPvrYLlINM2mkf3Tm7y+9nmS4b/zo9wRl/tfzryjUnvKKkhu3GkmUf/KFC4MDwW9965ahgZlX/nt8320bV7d7X3/jsqvCWEjyOgt25PjM9k0VHq/hb/+6pFQrb9vRHk9lT1+fmA7ExAK+flXX5u5Wu8M5OZNmeQWFw4/ff/2pVz9QkcYHd23du7Pn0D8+oTgxymPf+vkzk4nZPzzzAybDdXgdf7h7uwkl/nvyymO/e+13P3m4HBckUVddXdV7edTqbbg2PLDjxlsyKe5C34nHHvvawzc+lKAjuFrd2bn5/ke+m5hfhJj6+99/5O5bds9Pjm265ZbGVR13bGgf84ffe/HFmhXbE/HAzm07UEnauWYTZXCeOXREoSYxs+aLt95h05CURg7NLVlKq7UI7k8yHx/+7Et3PHTm7Ol/vPRW++bNiwOTyWRi7d4NmEghRYW9eDj9r3Yb+Vx9QBCAYst/LcajIVLEVQCkKKViMooCFAMYIUpyXpJZFMcgiqMIJFEVjmmAqAMyQnNzHEirVGpW5nBc6UvNkMAkYUFEHdRRSCgzgeCUiOISletorqyoKIUoVhB4hJAotaxUoBgAJIVSehxXEYIki7JAKWROEBFS1KhJLi9yjJDn6MmJWZzAHU7nzMw8kEAqnstnFIhCF/ePzI8PhhZ8qRQQkkg8yDltbpWk9lrbaEkzMjp46203tLVUzQVT735y/Y+/f8tls/78e5vsOoHjQDBZmB0efe+Ntz31rU09Gw5/+qmQrvLWNq3r1isV8qsHez/58NKj3/0Wqsb/8swH2zb07NzU9tzzZx0eB8Hjdpv2k8Ojd+1vNpmwF9++7F9KIqh8Ziz+5onRa3Pjn53vHZrzexyennWdrQ21FouHKxC916df//TDkZnZhua2H7z0RGlT09rq8gyN/OPJ/8yMh/705OMXry/UttS98cQPayyGo9en9n/tCdys9thkhFCdOnqoxGYfi2XmFi7feevDv/n+r33cyPYd7b//+R/+/e+nf/KdR2LJ/EvP/tPZtEJhd7du3pmcC9hrKpu71zfX1rTrNe8d/EyjUeeyKavVvrdxhVNXumvH7jNHjyghVJCKmoZGAteHIxFKDwXAmCpc21b0nD95/vbb9t6wdduFq5e+/5OfyRb30shoJs0++LWHEVkAkgBkYVlMl0QgicuLwqKntDhgiUJxYAeSDCQABAkIkiyKYDktLclAllEEilIGRVkcxQgCIxG7WV0vCBCFbJZZlAAAMANJDhCsyHEFsJiXIk4LVOvSWXF+KjyG4VSKmcI1Sys7bBt7WnesW13tqDLorGXecpRQKXQmV4nJ7tITBGnxqI1OXEkRQJIDvnw2Lc1NMumoLHIgkU4AgEiSPhfPUzghcDEM2oCoyqa5iYnRhbnJ6bFxOp89e+DUyPUT/ROn9EblR2+efOuVj267Z/NNN7UKIpgO0H/+09tsnvjhD24tNSJGtbrA4m98cPy3f3gumZNuuPOuifCFgf546+p1G1bXKFFw4vzYP59+7QsP39vYUvvbv7y2Zk37DVvqL12eqK0vDfrThAJ9/j+9X7i33Wkm8gJW5fLcua3DbLT+98R8KJ/LsMmZucGL/ePXRhdQJbrz9j3f+sY3U3nlfz45/u1f/uXvv/0tL0Tv+tY9j3/9dg1pefzHv5+d8t3/4M0/+OXfBZx/5x9/2uDUjoSiT354kVbgN+/fe/ToBbNd1e20Xb18pabZXlVb/X8//M3A4OEbbr/F5jKdP3P0qZ//5OB7R+jIHKFSkUYjZbDm84XS5iZ3Q91N23uSvomMf3ZmcnDgyunb7rg/npzceeMu/1LCv+Bzul3h6WlbefnS+MXKxqrI0iKl1bY31Ceyqesz4/fefced9z7sshv++Pe/nh74ZGZmSEhkEDYHuBwoZACbAxwDClnA0YBjIF8AHAO4PODyQGBBcT8oskBgAc8BSQSyAGQJSOLyUSdLCEA4QcpBCQIgKTAVAbU8o8NlAaAcJ4aVaiDCMIEllJSkIrJ54EcEUKJuUlAsQRaSab8akixL5/LBJDuisMxW1fGb11TfsnVTR3ULBdTzwwkKtWCYxlXq8VaWWspJjQmBmISqJY1R1lkIQq1AAMRQbMkX4RnQf3FuaTo5NbJAx5YiC8lcLDU5OlPI5aZnB+fGJlFJe+T98cPvfzR46vjee24cn1x466X32jo9N97WThIgli489+KxSET58/97bP9tN0/5AgYlqlBgz7/y9gcHz3SuXZfMpA99eG3jjevXrWmQeOzK+MzTT/zrpj3bulY3/+kvr2/e2GMwUkv+eENT2cJMsiBKb380+ejXNmUyYZ2ZYPL5de3eG1d5LveHro7meIBZtEgsRT/z2vEnn34f4Nkn//4Xu7752hL96onxp/7x4mfvvNPcXP393/+ptaH9339+srGpoafec+c3fpfITf3+8UcrVOpVJfoyO+Wsru5oqRw8PXLnrTsFRkwkfA/euz/NGD48fPnkR/+99db9v3j6+dbOsqoG7x8e+2kmmcX1GlkqSCxP6m08Zdq6/SaPyeKfmgrluDMXTnWvXWctrfXWVO69aeeSb3rFlq2QV5rN9qG+6yaTWWvUsjKrUGvcJbV//fu/6ioNLiP54P2PbV/R/sbhS39/5QVnYxmSC4NMBGSiIBsFmRjIxkE6BjIxOR0B2SSkk4BJw0IGFHIgn13+1CQOiHnIF4DAApEDEi8LHJAEwIspUaQ5nmVZQWbV/sRAip6UhGLPMocRQIB+DE+oCMDDFAsLLEgROAcIWkkw2ex8OsuKAprMcekCk4PBFOzj1RftTl9zmbbS5lqaopNLbHyGv/zZLBtGrFqLgBYINXR5KIMbNZeiap1M6TAREfwzMS6bpJPpkD+2NJ2OJTI8hyIsJ/F0ZCFicVkWJmYhKm3ctJGKKz8+eCUQidx837Z5X/ydF866Sk377+lQKVGG5V564z9ROlHX2hWKRixmQk1Bm0Fx4croRx+cvu2B+zA19c9n3l67YUN7mwMi8rgv9MffPb118/r2FfV/+NsbN+xcNzDqr6xw17aXsxlmeCb46rujP/rmTSMDY16vKxBNtjfU3tpTTSHSqeGli6PxHd3N3a3eUJx+5BcvvfTSy3/5y2P3dzcH6PzbfalXDw0d+uQTUgj96vU31u5+aHKO/8Wvv7O+zPjFR/+sNZU98bMf6wVuaWpJIvLbbtxeyCQxXNHU3rM4Ndq4ouPBfTdOhjK/+8NfVjTWoST1q9//9c5dG0KRfP/lq2qtcurqCYnJUXpjKLRY19q6eeuN4UCcFsHIwJCszDet7ppfnOlZt3N6cEFrNs3jBShzyWQqFU3RgM/mMixAayrKT1wffOWNj269c4+YmX/w4e/f1dHRe/7qlx/9OpINgMwSSPlB3A+iPhCeBeE5EJgF4XkQmoOhORjxgYgPxgIgHYHpOMglQC4BmLSczwAmC+gMyOcAywCJ4Vk6zXMsx6VFPpfMzefzUUlOQiAAkcYQHoMAwXIYhQBJi2A0EFGR5/NiSICoAkEphKRzglTQMDzgOChJQIEr9KhWKxMWCltZUVpRbYpFsjwDZ2YTw1fCGR9hUBgVKkJrxiSRhwQrIbyEChxf4AUhS+eVVpJEUJVKkYyENWo9LwJKidBcLhrN2ius42MTAMe8jc2EJB/+5DQGidqmckbIfPrBRZpm73pgo9vt5iTwwkvPzQydbm13iMqCxioqVbzbqZhamHv37ZfvefBel93w1psHNu/d0uB1QA7N5wt/+N2z69ZttLhsh4/13/fAljffOHPL3s7mjmo7RR05OfTMa+e+9tCuhanJPTd2q1SqnvUt+3fXrW/0IjLzwYWRbTdsK9UpNtR5Tnx85Oil09/5zjce6a61YfxQOPPbty/8469PLPa+c/uPHkzmghFA/PnxH3fq1I//8ck129dtWN81MdSHKqSaNRvb1rXNDgxClGdCk3k2uGHr2od2bw8uzIUWZ759912LQ5MUwuzbv2duoDc+3fvxpx/4Z2YorbH32pU8E1l7w00YobTKOZmXx4ZHy+obRweH6zta5QI7PXhlw7qu8NJUeUnJyPVhk9omBJcUOtxpQFUK9b9e++947/UNu1eODh164OHv3rth3dy8H5ESgI8CNgJoP0j6QMwHwnMgPAP842BpXPaNQt8IWBgGi2NgcUIOTIPIIowuwXgQpMIgEQKZGMjFAZMEMC8WMkKBBhwHGFpkuQIUJVEoIBgPRZrCJAIBkshCiWOAn6TiWgyTBSTLSQpZAQCk1Gw0FQQSRQBEjckluKGWamhUbag1rPOYa3qqy2/rrlYSuEKpgBCG45kzx/x8XGtUqxCZxzAJwyWNFRjMCI4DjRlXkoTeoJNwWW3AE8kkRiAQogpKoSb1yWRCkrGKutKBa+etpQ6t3iZk+OHeoYaaCpURUDr+Wt/10Ynhffv2rFrdQfP80MTJtR0NXatd9mrJWCIgyoJaT8wszn368RtbbrihUMhcuTiy5/Y9GiMpYyhJyW+8+vK9t++c9wfTMal5peepJz6+477OtpVVPXWOd04Pnrgwt2nTigMHTuuUhrKqWotWX99sv33nxkJoscJtuvuBLelc/M59N3z2+ht5irz9zrtvW9OmQ3mbUfvK2cAjP/rr2GcH1+y54+M3XgcO029++2NhcerK2Ej3pj05mmUinKW8WaNxOcrdA5euIiqjf2bSXdvgthp/dP+ebGj6vnv27upqmR0aa2pouN43+I/Hf8skc6l8AkHQieHp4FRvWYtbRgQ1CkZ5+dzRI56KymQ4qlJjJnfFn574q5EkLBqXhMlzk+MllS3X+yaNVq0K0SI8u5Tlf/jY90heUVNpGPOPfOUXv/vCDVsQnAUEB7AchFkoJ1E+AdkYwgRhLoBkFpGUD8ZnkdgsEpmGoQkkMIYsjkD/OBKchIEpEPOB5BKIL4JsCEo5hGEAnweFLGAZgPCYyAKeywGZXwYBAwBFGZMElgvjKF1mEjVINk+LOCkLaF6lZCgiS6jSRhXWZKtYV7a1xbnZae/RmhsUGpvV5b51/Yp6h2ZuzocAkMjkk9nCxdOLhbjCqNMptBipxDFcZlkeYAW1FkCUpzQoqpQUBlRjwRQq4HLUJRI5VIXhEI0HkjqjXUbA2MBgS/tqgYWBxWAiEy112mUAVAZifinw0aE3Oles7lm3IRpLDY6MlFtXttXeYLbrrU5ErRUxNdI3PN579fK+u+4aGeidmJy7677bUklGrdEyBebE0VO33txz/Fh/R1tzIsn8/Q8H12xq2Lx71S3dpU++ezoSktpr3Zd7r7z8r7dXbbmJp0W327V9+/p/P/3uDbfd1VJRsZjJdq/pePPv/7a1rNhz6xduXtHE0/S2eufp4fCPf/lnswqDmOlXv/i1sqLpl997pBCMeTp6rIAKDQ2iKG4qca1ev53J8gZj+VTvNZ3RTWmUDV1dlfWNeqPurq98GRMBiqEjodjhE4NGUp3PMySOxDjhyuVhNpWJBNKLS3EdRQxfPJXPhtvWrgws+Vxe9wdHzz/yg2/Xttakg7F4LqZRkRcvXXnxyT+3rV5TrjdLcv70bODBR3/Q1b6rkFnIAf6h7/wAATKEMkRlSACEQACFQApACkISAhIAUpYJGWACgAwipFAmiqYDSHwBCc8isTmYmEcSC1hsAcmG0FgQhoMgFQWxAEjHAZ+TchkpGaNZRuJzsJAFHAOkApAEACQg8LKIsiTKc9lUnmV1CsnplFweEaI5jQZU2hrduk4N6QacKGQSIpNlWU6nUe9a6dESkijLKIIWClw4Th/7OMDFdB6r3ajFcQJDUImgBFQjISoMRQSNkVQqKZWWEgDjcpezvMwWaAxF6Fwy4A+UVVXOjo0SBDTbXfk8nB5bxFUCRsg4QLVaNYdmPjz0vNvubG1dxeSS42OXiLS23LrK7rBr1JiKQFRabGDq8tT83LY9209/9lkmlrrttu0LS8ESb3n/yCIq0kYLmBgPbd7VPTEWPXngPEYh2zesXF3pevLVj0qra1orvVY7/rPv/n7//bdPjw3vum0fpUWOfHzmuz/70dzQoKHEyaZ8s/1XPS3Vjzz0oEtPhjLs77784NH++X//9c8bb75jYS5+5fybjq62xvparavMXt0W9Y/lkyFtibWivopA1WyhUFJW4us7UtuzRgB42cpdqaWgvtRprrLSmVhBEDAlWV5ijC0Gstk0lglPXB8JBlJ6qyJE57sbvAiKcTm6snlFIhE26WSP0TC1FP7g2Me7998cmBklsezmrTf++5lXeL5QVVKaYzmCoM6Ojv749//oWbtr5MJnZU2NCF2AdAHJcwgrQFGSgQAQIOMYIHFZQYlKpaRWiVqNqNNIWq2gUQlKXFKgEgklTASAgWwSZiMIkwKRiBReACEfCPsBHUMyUZCOgWhQZlISnQSJAMhEQCoCsmGQSYBcBjBxWeJFQSqkYyIGEV6gJSQfjoJoWgpkwtP+kXBwgYlHCtGlXHgqG5nNZGmPy72qrqS4vEQgQCDCc+LYxdzoRRGT9BoK1+pIjJAJFY8ZOJlk1UYCVXMEhfE8r1KiKKKi6SwrihhCxUOLHFcwuUxjI32eCi8ns7Io59MyRSGiKEgcR+IYoZTPXXzPZndb3TXziwFf/FLKnygxtLocFSiCoxTAKGRo9Kwkg1XrO99766DZZFvR2eifC5SU2y9emW+stw5e61u1urG6uXTWl37/7dMnLvXfcWtHmuEOHLmCKzRddTUqk/SbX7y8455dZz9898vfeOTd/7xSYOn7v/Tgh+9fKl3ZfPGDgyCfQvS2L9+2J7YQVKgMj3/hrheef1Wt0K1urLt26lwi49eXe3CFztXclaSXcrFpRGFANOpNu9dQGrWttIrNRZg8YyqzS7JgqqgWEbm8vTOXSgh5bnVnY2lllZzN9l8509xQV1ldMzE5XVpd6q20N3irOrvXp+JxjcYw2jecSyYgJmtJ5dx8aGJhfsWatjde/Oee23bU1a6cmprasGqNARIcxyopxWuffvz6+weS0fjwtQtIMgvTtJxhpBwj52nIcqjAQ1GEQEQRCUNlDEdQkoBKCui10KSHNhOwWyWHRbYaZbNRMuuBQQVIHBEEKHNA5AHPAwqgmaTEskDgQNQvCnkknwGZOMjEQSYC4j4Q9YHgIvD7gExDkJXzCcBEIJ1BeBb6fcLBc71Hxq5E6STLSTTNFQpynhdYrgAg2NFZ7jWSoiRDCCVZshv0ClJ1/LOlY2+nB87yiSiVixBsDNURWrPSpgRaSKu4CB6cjZ08fgQUeFQmCywvQgHKSGjJb3XZQ8EgiWMKlSHD8smYACAmAQnFRcAhABCoEhkcPVJT3aZQOANLUYsTDc5P2t0VKp0ZlzGKIATAXR+8YNQ62rrqX3rljba2TrVOr1VKY/MxHKUcLnL46vjdd+4LRdNOvfFyv2/Bx9y7q+vEyWuEFp+ZiX7vkTsmx/qPfThgcVGIiK3dsPrxn/x6195dTQ21qSX6zPTs8MmPFnzT1W0btrSVvPvRW/c8+p1Gt2Z2fLS2tXtkOLI4MUkpgSijeocH0FIy4EdxRToS775pL0HhBV60ltVcOH1SqXMl/BNat4fQ2iylzRipvLmnZ/2a9XpnjdPhPPHO2xq1ft2tX9CaDAIr6A0OlcFaVlXBsbRGq+29NjwxPr2ttZNjmZVtPUcPn12KZNKJdDKfevhrj8yOL3Wv6frm3ltvW9kl8DxFks++8fax872jp84g2RykcwjDoPk8wvOIJAIUIAQEFA4oEiopWUnJSgKoSKjCoVYBjRrZrAYWNbT/f6j6yyjJrivNHz5wMZgpmRkKsphLzLZkyQzddttNM01umHeme3oaxp4mN9httiXLliyLsQSlYszCZMbIyGCOy+ec/4cseeaNlStWfMhvd699z9n7eZ6fG4TdLOgmYR8JOGHQgyIh4HUBnxOIModEJotARIBZ1KgRAQNIATWApQJSA1oeFJIgvQFW5lm1AEAFqQV+cwGqJQg1LrsCgOp1OaIIy4ATKBKoiahBqWmGg9JdgyF0Ry8NREFM5AoAsPhadWpUO/9S4cLrpWvvVAvTrhsncxdeXx4/l1yeKiQShcRawlR0RgE1qaEZmMOaqpXKVYfbHV+Jd7Z3FvNVS2NmDUuCTZbtSESWSSEVdF2Znj65Z//hUtXcSGQcfu/0zZtNwRYJczzGGPOGbt64fTHW2B+KBd9+8/1j9x2r5RS/1/Hqe2MHDg+MTdywC/K2vUObmdzHjvWfev9ya0OgK+Ken1xt62+dnsp9/lMP/Oq5twVEAEg++uRnEvPr77//4WOfejQoY2p3vP7KZYcgUUF+7IlPwOTG/PLMU7/7X+Pjt3buPVwokMnbi7SSLhcq2Y2ppo6O0vIaz6G5yxcVonCcVVyf8oWaV2+Pm0wQkMko46QA4GxL6/N93Q3d/f1yJLLtrnttSNIA6D4w0jW8Pbme3kynZSdfLmbz+Qy0jGBD69rKwlc+99mP79156cJ7dx+469vfe7qhwbFy5WTn7uFspQaAuffgwJceOuYEgFmEQ/jcxJQBMfJ4qctNHU7iclCHkzgdxO60nC7qcltup+nxEJ+b+tzM66E+F/HaqcfB3DLz2JnbDjx24HEwt53ZBWoTmccFvU7gdyAOQ8wTUQCSDOwuADmGIeDQlgoVYAwECJABmAJKKbC+AjfXmFYAWIFWilo11BXp3Ne2h+cchAEAOQAoL4ii7ARAIEzYP9A8EJIJpQgC0zLS5RIAACOAEbMsUtUsyWa/dm15fHozsVktlAzVoEqVGibQDGDqhGMAEGSaAHMwk0uGw5719bWQ1ytJMmOmVjXLeVouWYQSagFdp7xoq9SSG+uTu/cfml1a4EVelmzl4obbW6eUmSjadWL4/eLG5tzBI8cTyfjU2NTevYfsxJxfqdy+uVHX5Pnlz5/93FOPA0kgutDXHZocnzpyuOf62HhL7/Dy2szRB/ZF632nLiUWbt+ONsWe/NwnXv7uT1qGR3Rg3jPUaCFI1Gr9cKcz0jKyb+cHr79y8N5PYMhaulqO7BlJruRLibQkwszaTH1HX2Jhsbi+xiSqFiqyq0mpGK5AsFgs/vyHP+R5ezG+IPoCnEBlGV67fK6uq9vf1E1k1+F7j9jscrlWEWyOYk7DHEqnc129XVWlQCE9tH8kUN9iCux3v/K1fLHUHPMd2dH/s+fPnHv/9LM/+qfBnV2y25Et5mL10YFIHQaYw7iiKpOzc6itizZ10MY2WtdMYvU0UkfDURYMM18Q+IPA6wceH3D5gMMDbC4oO5nkBKIL8E7AOQC0ASYz6AAaZ1Yt02CMMAARsKhpEKCbwLQAhQDwAIiAl4HgBKIbiA7ASUCwA1EGvABMjVUrrJKlfR3elnAIl5rqPN0OSaLloppPWWoVAp5gzgQWYRRQEvDbH9oRbnYiBBAB1DBMuMVFZ4wC5nfaOB5XNI3DyCBIM6CuQ1NHpomZyZsq0g0GAWIWgAjqmrqV+FYs14KBOlW1MCcYplnNE0tnvAChSfWqSZG0tDim1srbtu+5OXq5qamjkC0LnIyZp5wzRN5RVYhppW9Oju7eO3zyvQsGM5vqoiIEb51aCPhCxXwuk0r91ld/k0DkdzoY1WLh2OG9rYmNUkNH39Jc4g/+7GurKymTwPGT733it3+PGMrsjYm2fXcBYnz6c0+U8gliMVt94679D0xdvkoRaW6pK1bK++9/rKWxPRPfVNTS5mamVEBTS0tKall0Nvz8P7/j8MhL81NEKza3dvzouecNFVuVAsZYcAR89d2Xb46n09nG7SOUms09/Q6PVMpl7G6nINjaW9pnJ2cDsY7J2xPFbIlayvYDe0q6YjHy+JOf+/DMiT/9oz9eWl+T7W3/45/+dWpizButB8AhOvxFKA/WtYQcXgDgdHwD1bWgWBOINcBYE4g1gnAdCMaALwg8IeAKAJcX2DxAcgLJBjiZYRsAImASgBIAMgB2wGyASYCIwMLAgsBCAAComcQkwDLvGKkpAIwHTABYBoILyD4gB4AtCBwh4AoBUULExNUaW1tlkWDH4b37t7V2OyCmpkKBRiGtKZVSuVqrVPVqrVwpZoqFcEA40GaLOoRcsfKRQxpQxhyyGA54E8k8hJAxCCAkhOk60FSgVplSs0yTaZqlExMAapkMQVgoFb0+7+LSitfjzuZ1RbUYohahukaoCpnJAcKpNSBI0vLiTR5wPl9wbXnCH21Jrq5GgrJVI5qKk3nFMI3cxpJlZWMtzgvnLnf1tYWcYqKozU1vDOwefO3F10JBx+49e9va20NueW56cc/Bw3pmaXD//tWl5W27h+rrPCtpZlVS/mBk3/2Pv/Hzp4/f/4jHyXfu6Bu8+5HUzHV3Q2tdvQ+LwvTtGwO7hhGEXTsHmjrbtaqWz65Njl5emV4Y3j6s5BeGDh5/4/3Rq5euD7S6tGrh8KNPZXO106dPySIyKwXAy8g09x+7u5TeJNQU7JIr1MjxvvW5BYS4Ki7VxXzpVCK1kSzlqoXEBm+3L01MtQ/uWFy99fX/9l9qNbNWKt997IBECm2R8OnRMxyPA0gVsN7Z7H/k4J6Dw/sJY4V8AUWafJFGZ7hBDtWJwRgO1EFfGLhDwOkFNjeQnEAQARYAEgAUAeQBFAAUAeWAhYCFAOUAEAGWAJYBFADPAQ4gYoIttwW4Ax8AYEs3wQHAASAAwAELAMIAYNC0UDoPU1lWzFXdgtjf1txYF7M73ZI7KDgDkJMxgqIgyna3JNlkp4ciAIDeEuD6g9gGIYYQQ8hh6BC5uqBrJZEjlGGAtuK0CKOWRS2LMQp5iZMcgiDwgCBGMbMwpHy1WnM77ZlsHhKgaLSmW4Bygowwh2tVphmE40REkW5iBujq8lhn1+Dm5oYoQIwd5UJGckFDU3nKlXKWU7RPTaRjMVc6k5Odws7+RgTg3FqhNeQRnO6zb71lGvkHP/v5Q4cPa8U1pZhq7mzx+b3lSlmpGo9/+uGLH15kobp0fOrRz396eeq6UsoPHnpg+vqV1u17ZdEu29yRtkisKfbmy2/56zwAK7zNLTjdPCeUcxmjohTyimiPLM/c8kV8rW0tP/vhM617DyjZlb4d+7sirg/eOWlqolYucjaXw+38xGefUFUyfu4Ug0RyiC2dw9VsHhgaNMuhSN2O3fsnb17zh0Ovv/psyAlPnX4/2lDndPsp0+954gvnPnznofserkHj6PbBq3Ozy5vLkZbYxZMX9uweOnhk4OP3HLMDPlFIIX8oFgrXhSPRUCQSiIUCEY837PBGRJefc3ixw4NsHiA6ARIBFADjAEWAQGAgYEJgAmBBQCCAHMA84AXAcQABIIlA5AAWARYBFgAnAk4GWABQAEAAFAG49Z8CEBzMU2e1D5Ndd6Hdh51N7ZzTVuRQFVLFqKV0TbeQXXDFMGcrVcpFJVsppBnVOIlAWGsPwXt6nH1BKWxHTh5yAOZSeQc2G91cxA29ErTxSEQQIUQYsxiwKDRMRixEdURMYBhM14GlGbLAq7oBMSYUl8ugVoGWgQyTWggYFq3UapSquqICgAyizC9O1jf3L8/ONzbHUklV5jlmql47yCYNrQyUKlhcUENevL6ePnx4hw+DbM1MbBS7WxtC0Vhm6XZ8eenAk7+7Y+++xFpKN2tKudS7fWTyxujeux8f6nRePTdWyaxEGxpae4euvvti77a9jFJTKUd6dlKTlqtq0C2NjU2Jtogr4JcDTkc0SrFeSSTq2tqgp94ioqVlzHL24JGRC5dvr8VxJTcvQbpj5MC5qzcK6RyplQBDtkC4Ws41tXes3L5dK5QAog2tzZtL67l4XCnoSlU7fv8jhsEaQtF33/3AZXdNjE/NjY21D+64+MGp4w8cs6jW1BBVLcfO3i6DgjfeOBGob3rv2k3KSS4vN7y7++DgtlpVRSJvlyS33e6z2912m8vu9NldbpvTbnfxDg+SnEBwQl4CnAAgBowD5KNishAgENCtKBEMsABEJ5C8QIhYjnpqbwRiEIgBIPqB6P/ohxdgB8AOwLuAFACuKHDVA28r8LYRW51ZlbI5a1GnBYtUdUOFSJJ4wSZwhIBqWS8X87Nri9Pzi+m1VCVLIfYgJK+mlHTZZAC77Fy9F7aEcXe9EHACp8BiLtriQzEnsvEQQqZoJJvVs0mtlDUrVWIYFBDACDA1SogpcFyxqkCEqyVLrbJykZYLTNMMAIihawAAmZMAFSwLV8sbdjsTRGQo1VCwnulMAHzE6eJFaWKhGPE611JVCoT05npDQ6Sn3osMk1pmQ0QwdDR04O4zL71k83p33v8ERmKosdFQKu3b9t2+eFKSfY986avFxFwyVcWwtOfu41fPXg4E/e39vWYl74lG7F5fNV8cHBqOuJyb8TTVCBbsMheg2EFrpc7BAWCqqmFJ9lg5Fd+552DZqJ56f1QMDWXXPnz43vtWFO3a9BJmhpJddfjk2zdvAoY6+kZmL1zmEMIcNnRqqkVSqQEsdDTHDt59b0t9gBlKpZpvbWn93rd/XBfrIroajfI920cYj/p6BwMOd0jk33rn5Mzc+pG7jnY216cyCuCsz37igc765i3+IIAQA4YR5CBgGEFe4HlRwAKHMeQ4yPHgTm0hAHmwldbIKEAQYAQ4DCQZ2B3A7gFyGAhRZosCRwjYQ8ARBs564IwBKQAELxAcQHIC2QFkJ5A8QHQByQNkJ5DsQHZAf1BujDV5nQEMIAAiJ9gN0yrkUtnEenx9YWxydvTa6thE8dK13InR1Fs3i7+4lpstmNs7HPs6hdYA4HkKGCxVKaHQIyK/i3PZAM9TiQMCQjwECDBCgG4i02RbKyZAMKWAMtNmF/OlMidwhkXKFatSIsTgIIOQIxaBlo4pgJZlmgahDM1OT9vsYiqVDwUdhZyhEGlstlAs6TWIoWV1Rl0zaxo0VADBrh1DYSdeWljZtntPTU01d+yQPO73n/vh0N33Sy6nIIfyqXgs1uDzRycuvrP9vs/sOnJ/Znm2XCns2r3bycF8oehtqDfUEmDE7nVmUoWm5rYj+w8m40lqKUi0u+rCNpeo1/SRo/cF64KcJAq+Vs0qDAzuqA853z31ptNen1idaQiQFrf3yvkTBONyeh1Ua65gMJmaHz561FCMXGI93NLk9Tvnb1zr6G6iJklvbtS1RFzeaCgUPn3qzJOPPjF1e6K8kXrulbdPvPfuQG+UGuXW/i6nK3hox+D44uLUzMw9dx1Kr60kC5X52YV77zsUctgQoZRSyhjAWMRY5JCIoYAY5rDAcxInirzASzKWZCCKQBSAgAGHAY+AwAORA04RuGXgdgFvGLhCwBYEkhfIHmBzAocXyB4gOYBgB7INiDaABYAwQDyAHMAcQDzkRShIkLcBt19orKsPOhp5JFMGtGp5fX5hdnrm9uTi6cvjL7595sUTMyevli/Na+c2wIU4GE9ZOkXDjU6HgOIJbS1pWSZCFnBC5rdjhnAiS+MZ09KBDKCIEGSQEKAZzLCoyPOyICkKrFWoYaJKVeMERCnAGDNAAQC8CEQb4ziOWBwAzDBJqWiUKoZpALUCygWznNMoYRuJJLNgLqPeWKo6ERZ4bEDaHXP0NAfH5souBze0ayTk8fX2tc+MLx46tvfKhQ+e/K0/mLh8uZpJ1fXtS8ytBOti+c31Y5/84trsdSg7t9/zmD/g25ie8DU1H3n4sWwqLrs8gitgahpCgBdku9/X2NtbzlcqqTXJ7SXUEm22YKTRFWmINTY1tcYoi2aXN5xh/5HtI7enp9YXV6l9x7n3Tzx1733xRUMtVznJcfqti8M9HZX8IsO4eXh7cnFFcrla+gazmxVvuM6CeDOVUKq5aFvzQHfvrauj/X2927cNAtPIp0qf+d2/nE/mgV6MdjfASv7o7r0WAPlCErr8t6ZWeJ5fiW/4YqGO9hZEGWMAMgAZhABKCHkE3idyXh56OOjksIsXnAJvFwTJbuPtMrTLwCYDuwhcMnBLwCsjvxu6nICXAcAAMIAx4EUg2oFoA5Id8CLgeIA5gBlACGAeoC1TBgIIbYECGOYRxBwjWFMrlVI+vVlIpKsasTHoKZUUvVoLOoS+ese+Nte2Rv5gJ/ex7Y6PD7kPNktauXZrvlxkuLvV3tMgN0Wk5pBUrdD1tIUgbAi5HJJkWFQ3iUEZ2QrPoUwzSbFkFgtWsWxVFFAs1Ww2SZYky7QAAAhDl0eCGNUqhlIlNY2VFUPVLd0EmklsvJ2jfLFQYybZSNUoQjwCDND2tnDQ7ZAkCXP6vXfvkgUhXQQ7D+6O+OzbBodnp6aCoSZBUm12qX/Hvsuv/GrowMFAxBltG86uLnTsPBps7kwvjbnDEX9TTzWZIdTq23+3Ui7opZpWUqmm6OWyI+i2LBKJ2SMtLWqNQVLVLdOyUGNXF7FMzeKQwEQsACIQaDzy2OOcyX9w4oXu9sbXzm8EnB6/2zE9dd3vDZ368Hx1UwkEu5cmL4ca6nVF12rVYF1Ytkm1almySQQIm+ur4cZQS2efqUvTkxNf/OInC/nsroHBcrn293/3vX/+xr8sx9f67j20p7+71+M7d+m6gKjDKSt5Q6+UDYs9dN9diDBCmEXBFkYOQYAxs3PQJ/ABDnp47JEEnyB6RNErSrIs83YbckjAIQG7AG0i5ETAMDAIKFdAJdtStwAAepJJREFUvgjyWVAqArUKDAMQCggB9A6y9aP8UPCRUZECCAHmAEKAMlZTlWwtXSSlbK1sAOZ2eSRRIgwSYulaMRry79/Zun977OBw8GOH2+7dFo2FBMXSawB0tjnv3eXtiEoyhjYOja9r2RoZbhG3tdioRTbLRt6EVUJMRihjDAKL0GLJTBcM9Q6bg2mqySGO56CiahBAw6S5rJ7YUCtVpCioWkGWyWEMqcFkKOsqrukUAibJPCNI5nFjUI7KwLJIQ9AdzxRNRZUFtGt33+joWENLR1tX28Lk4tC2wXdffnV4ZGhzdWzfg/eXUgm9kDGpKXlCNaVoaErr0IG18UuMY43bRiS3r5hct0UayiqtVrIQiKKMaDXR0uJz+AIut1uUbKoB125cjXW0uANRu8s2v7QSjgaBThxB3kAsG18c3L13pGvg3IXrWnJqz77dV26P3nOk+9b5CaSradN89qWXY+7o/Oy4RTIOjzOXSHGS2DHUpyuawEnR5o7T776rKaVwvYeI3qXltWDE62n2NtbV7Wkdur24shxPn3nxh09+/ev/8dpbD4yM3F5cm5i+tntX97WxKZ7HqUw2UBdDhJomMU2qWUw3mUEBo4BBKCBox5wTQ5kxHgABAAEhURAlnuMFEXMCRAKgGOiAVnRWVkG5BjQDWAwYBKg60A2gqUDXgGXdyXBjDFAGGAGAAUa3kkUQtQAlAADGYSBxosjcLslvEzlNySlaXieqTbZ3tPZInvB4onJpMTuX1W8uVl+/lDkzVWA8P9If3jsY4QhdXC4XC+zqbK2q0sMD7sawfTVD5zaUgmaphNCtodYdAT+jDDDGBBFIEkQQ6LolclhRqpRRCAFhrKoYigqqKtE0wggCFFAT8ZB3So61RM20oGUyQzMFDgJK7D6ho9ldyJU8AYeqMdlmm1mI79o5PHr5RiG5vnv3tqn5uZAvmE5kZyfWTKvmdkjeWGRt4oapqESvtAzsLG7M+5s6/EHfxsJthqxI97CSyWDZFoqGazXF7uKR6Foen3WHmzW9Ijrk3OYalp3lsip5o6LDw4iqlVMuv8sydafHoZZ0SgXeZT54310VxTp/5tSj9w4XC+WlzaLXV5den9mzc/DFD8+kkklgSGNXL0cbYvlESpLtdW2tLl8ktx5vaepYXEmfevvd3oHubGbDGWy4du6iN+wLNYR3De177NixcytJmfcJCvnJ6+9IIedwZ8e3f/yqyaN9I0MIOxfnFgKNUUSZaRHNtFSTaSbVDaqZxKCUWZZJKYCYBwxDiBGGEPEMSoCzAywxjmMIWxgZCFkcYBzgJSDKQJAB4gDjgEmAZUHTBIYGTBNYFiDmHTzdr7HTjG7ZEiEDkAcOv9huQwGlahRSJUXXLAZFAHlZrEJS1pVMIr+0mBmdKrwxmljMKXV++/bWUKNLnJ5Iv3kxv5amy0lS1OlIjwuY7NytysSGVrCYCRiAFH7k5oZgixDLHDbOIXK6YQHALItyIs6XanfS7tlWlgW0TKYbkBKm6axasfwO90qiqpjUsJhimuVqFQEAAJUx19sd1HXNK7sJgW5PpJgsNYQD6ZLy8osvDu/odbqdmq7s27vr6odXv/nNH+dTmd4d+zLJAgPc2tS1SHs/0XK8KAdjXZmluSsnTwVbeivlBKOW2xtaWFqy1CLEEmU1SqBmMGCZAoYGpV6vZGmmxRgkulN26GqtSixo0Y6dOzDnLGwmdh0/LHK+TKqoauCTj99/9tQVGycVMvkDu7ZRAl966cW+3t43XjhhaTUATcMi8cW5xt52XmZ2GXUO7vrOd38Q9EVbB1rn5uZEd/TWxSvN3U0yyn39K1+JuoMvv/f2N7/+X4IOx3JOe2jPyPXLt37nv33jzVtXWzrb1JwmyS5EGaOAWNQ0LdWwVJNqBlMVWjJAxQQ1k2oEEogxQJhhAWAOczLCdizYocgjAUMeIQFwwkfhRwBgBNCd/gAwhpgD6CPuBAaAQ0DggcABkWeiwAQeIMgAZX5XJOxtckuCHVOe06rVSjqVz2bTS4nVXKXMDBbweNtbA7KDswxSLGnlopIv507dip++nZMwF3HZipYV9vDZnPXuZHW2oFWoRSCDEEGGAIAQYvZRcA7PYQBxRbEEhDADmOcFSc4Vils42a3GuhXMbJpWRSGFEnG7HKl8LZlXOA6ZFqAU6qoly8jtZNAiAa/oEIHEU46jlPEeO+J5rr4h/IvX3hdk3NbVWKzpmmbee88DVo2ceP9kbHjY4XDnN9I8g5wgI2yZWpEIcjDcdP3kWYaQr77TKOVlbwgzdXNlDhjFWN9OLAiy7AQUNXY35JNLiLHSRlx2yAJE1OKmbk90dnUQogl2t6lYtZoZbQrt3r/LBO5kIj68va2/M1g1ipWa1trZub2zbX5xOhZyIFO/eXHUbRcEUazUiv76xmhrTza/ef/dD01OrZ1+78STjz14a+xyW3fvr15+y9CU1Y2VSGvTbz7xlbJa+f6zT//4r/8UV0sWD+oDdhdve/HVE6+feLWhve7S2x8gACBjkAFEqGVRVSeKTqsaLau0pJMqAQYF1KDMgthiHIE83coPQRJDEuN4yGEAMYAIMgwYwggiCHkIJR7IMpNFJvNA5AAPAPeRV3FrQoER3PIqEgYYAhYtA5oQaZFpZc2saZqKEBJlHPJ4Qx4Jw2pZKS9uFhFvjfR7HzvSNjIQXYkbs5t6XVQabLEVVBNA4paFVJ6kFYvAOwACuNUkIfyoaTEIACPM0k23jeMQLOnU6XLk82VCKYQAAPrrMyAElEFgUepzu3QDpPOaTeAAYYBhnhM4yNlkfrC3zmPnJI4LemWfU5A4jEWxrd0NkNDR2j4+tXHr6sTw3h3J1Xisox3b0Ve++um3XjlRWJ/vOrDP0EF6bUVTKliMUo0auhGKNkqcVEik3PXtxeyqJDnq/I2prGqUk45wYyGZghCXshneFNMb65srK9VU3OYJUgBlTlqaWQjXN+VKCUNV8utxu8uvqMVHP/2xZFKziDg2eeORLzyUKZSKeQXxtkMHdl9bii/Pr91z/90XL9+oVtRMYsXhdaaWlgINfdO3pvr629rauv/2n7/dHmkulLOMasl8JR7PDfR0E2D2D/T6QiObxexKfP63vvCkZXLZiuKUbL91zyE/D98/d0ExKojjZQ7LHJIhEhhElBGLmIQSRiEEmDFEADGIoehqpVorltVMoZbMVuKJ4sZGaXOjlk7p+RzJ52mpRColWisxtcZ0g5kmNE2kGVDVoUGQaSJiImIg00CGjnQDagbUDWgQwCBADFhVTdKRhO1+t8/v9IiYy+bzU7ML127ePnHy0olzk6vx8u6+tt987N4nHt7fGJPHFkvrWTrQ4tzR5E6k1fmcKkAkQU6xwBbL+o4p8te3BUa3WCkIIY8NBx1YUUleI4QBSRaTyfyvg8B+Hc68BagKuF08hul8ze7gEWAIUYMSTTcMywi65fpQIORxEw0bhipLgtPpoVDwujxKRW1rCGqMvfnmrWi0fSmZ9IcaKuVaX/+QRwr88gfPBjq72rb3z0yOpVfXXaEmpZgTbHIlm95z1zG1pliG6Qy1WIZlc3mWJ2ZmZ5b07KLoCheTi45gcHN5UZS9s0sJqpcMixHEBTyoWknlMiW9pAOKXR6nzeNbmp/v6u2trw/ZRHFqZtMXbu1sa9EtXStm7zl611Sy8J2fPLNteDib20hnlPErl+tbBlMra8FYVMmrgl2+9+DR8+OzZy9ecdjsKwvLbn/ozLlTj3z8gcRqon9HD6erf/T7f3DqxPkLV24c2NVbrWgtkSZqqGWDPf/0z3t370QYczwvAMgxIBCKDIsajBjU1C1Lo6CiqblKYTNbSGxq5bIfw91+92Ox6Ofq6r8QjX3a635EgPsstaNWCmuVIFUCluLXKp5STsgkWSJOkwmWTrP0Js0laCZBMxs0m6DZFM2mWT5PS2Wm1KChACdzDfh2Re1DpiGnc4XljeR6Mr2yvLSwksmlSwIHD+7o+sOvfPyJh4/4vN7ZqY0TpxcrinZou6c97JxYq01saDaM691ypWZmNP0OuhNutSf4azrPFpAnYMM2ERcUS8bAJ2AbRswwK6UaBxEHEYJoi7Oy1biiXh8EYDNfsgDIV3SdUIggA0A3ICGMUpVH0On0yHaHTbBxvBwJ+UzT9Pub1tfXIo3unljg2u2ZYjrnD3hv37gSqqsrZTef+NQ9V86eyc5ebxkeCcXqxq9ckl11am1Z9rhruUykoVlXTUggx/OGXvb6XBo0MvFseumay+ffzGZ03SyrJb/LubqYIBxQ0huiK8BjhCidvjbqc9g4TDjRQEqlWsgZanX48Eg6sTqy+9irv3pvz/7t18+dXN6cHto1tL+99Zfvn64Wci6vJ55YQVbNUCoCUAWsN7W31SrqwUM765y2X7755meeePL0mYu7BnrGJ6ZcXv9mat1nsz/86F23Llx94rNfffO9U5EgFwnYw2Gf227TKiYi5IO33uJUoqs1s1SuVSply9IAwhZlCBOEIbNES3dA1ORzdNc39vgD7W53yOvwiqJMAaCMEtNU9aqi5WtakRGdFzCASNWq1Vq2XEsUy4vF4hIiVZ+n3uVvolA2dNMiVU0p1ip5wyypuMQ7lJBbCAhRVGBzy9fzyUq+ZtZM4HSFDt/VL9kdkFhOp2xDUmJ99cbYdKlWSKxvBF2yJLBEpjYbLxVU0+MU2oMOSzPmCqpKKYMMbpXUFlLz15ggxqIOIWjjFMNo9gjUJFmVBlySoauEUBuHAGUWgwZAFqAC5iIen2Jo2UoZAMgotQDiMCQMEgIoQzoBsiRBzsHJouQyQqGIVmNRf4gC6Au3Xhx9ayjY8tj+++aWb587d2VgeOi1l14b6OtZX0129/Z4HL4f/vsP/vxf/mVo374rJ96u5vM2l5OWVnl3OLUy0di9p1YomKYqe0Mbk8XOgcHSRqFY9ZrTV9p799++dGvH7r5atZhM5Utlktu4PHjswWqx3NrWvra21r/twdXF8Vhz95X3ThmCsjAxuv3o0Stnz+3q6nz77VNXz524Pj4//o3vv3Ti7i888uBv/fO//+yFFx+4776fPP3LB4/vefO5Hz/x5GeWrl9lvKtSrEmeyAMHd71y8nws+GdX+LWuWMwm2daXlwa2H15eWjl44PA3v/7j3/+vuz1BX6GQbwj78sX0wFDDWlL78pe+9J0f/JDLVjimtBDVpymVsrJYLa+Uq6oo8N5ANOTf0d68u6GuJxhslOwOLPGyJNkEETFgAAIZYoQxQAmwCCWmSclHUwRKmGrpmlEtFRMc0L3uqCA6IUHUtCgjuqaUS4VyfrNQXMzXbhq5sal4ap0pfs5JNHNtNVUpsrZGFPU3IEJy+eL84kK1VLEs6vU6SUYrl8jseqWoUQ1gt0Pq9ohuCSiaMZ1RTQZljFVCKWQQAAruVBWEjDAYtnP99QIAFDChWtHLFAdcAidwG3ndyWMEgQmpyRhkzCEIDpucV8pVVYMQAoYAYGgr5JBSCKFmAVyxbILk8fuZRbVazoS1UrEw3DO0vDJnYIYwiE/Nj+x/rL3F8ebJdw8e3KHpLLG+DAkyzdITTzz01//7P+97+K2h/Xe/99Zr10//6tAnfmPtzK8k2fPmqbc+t+MwNdnJ1371+Jf/MJtLdvZsP7uw4HXsfuvNn37+a/8j4Hdhh+RA7mhr89rCEhS14MqcLehtleX33j6pG2Yuud7Y3j81M9vV3xnfSAwdd/VsG54fv/3xx+/7xXe/b8n2d6/evvTuqUfuufub3/vpt371xuOPPMaQlE6t/+qNd3cdPCBAYpEK5pkFxIZgxMHjV19/9dNf/sq1Ux/u3b3v2Wd/8c3vHq1Sce7Khe17D62OzwUj4bGZeHtT7PT521949NDY9IWD+/dcPHOKI9r+iG/E19JKKapUCzU1ZVJVlJwed8jjDQU9XrfNYRMkLCJOxCLH8QhCyCwAAAOUQsDYFuPXAowSyhjYulERRilltL4TAQAYZMwEFADCKGMMEMOyTN2qlitzs73rCwFBhsFQoM4ngHLBMM5nVhcunR8fvT0DAPLaJdnhjjVHogFvPl9eXCuspLVofeh4Z1QCbHk9k8rV1vPVnEIdGDgFkNMRRUAlhMGtoxYEgFKGQjI82C1FolJFZaW8haHp4WRTtVKKYecgQrSqM0qZgKBdFCyE0qXynenXR+geChiGQEBQpcykgHK4XK6VknGvNxqPW+GAI52Ne/xHpE0HpTDkdy9cvh0vn//K5+7/2RtvphKVSCicr1DONHRIeQqPHT7+vW/95D+P3NPat+/ahbOHP/lbyNuEq2umpY/fuHzkoU+mNtLFxCbRLUYrSNKoUbPZQtfPfrD36KF8PsdLnmiUrC4ut9fVTVwbPfbQffGbCzbZm07kEcdVCilXNLq2shxt8K1Ozxy87+5Xvvfto3u37dm1+9bMggDn3n73/P/889/+4yce+ZOnf/GfP/3pI/ffO371AubkH/z4p3/zF39y7cJltb4F8lhNlQ8Pb3vp/be+8ntfUSzKcaTewy/cuDj02BdunHy7q6tFLFqtPvviynx3b/vLr5/1ev0tDWGd6jv370c2q8Ut1fk8/kgg0NrQ0du1f+fQgzsGjvV0DDREoj63y+mUHQ7BZROcPCdhyHNQ4IANQzsHnQJwisglYbeEfCL22Xi/TfDYRJcseGySxyZ7Zcllk1x20e10eDwOj8/p9Tq9PnfA7wuHgrFY40D/8aPH/+zY8T8d7P+4zDWm1nMb6/kK4bDH7fEHWpvqBnt7tvcPYgtOza1Mzq0jm/OJR+/66mc/NjLQqZZqi2uFyXglW6VuAQ6GuIiTkzmA7hzDIQBw68xe78EP7wsc3N/Y39/SVR/sjHibg16gWoWKRQ3mxHRLo+GSOYddUCxarKmUMQgRuMOCYQwwGUGngDSLAQocEpJErlrWDFW4emnZ1F11oY58rpBJr/VuG8hldZfX5wtHz1y4ms8tbRvqiq8mR/YMJxObhKOpdFkj2rEDPUUFnXvl5cPHDpcLpfTidLRzeHM9Ndi3fXH0NC/yLX0DyeVxjdmvXLjQ2tkzPTm+//C9N6+cMxStWjLKuWQ0XB+Ixjze0O2JmWwqK8lKR5dft7Bo98yOXjhwcPfU1G2nFLl16UNvwO70u/P5itsXFIDugDieLm4UU8cfOfr6P/1vqFYDHjEUCR3Y3vPMy+9OzC80NPmgVY01hsZX5kOyu6BrP3/up9tGdr7+9gf3fvzJzel5M5VyNrRdeP/tgV07Xd7Y+WtTYadsQVAs5JsaWtbi8b72GDIVWi3XKqVapVJTFMXQLUM1iWEBChHAjCLTZBXNKlatXMXKlK100cqWaUGlFZ3VTFYzac2kqsF0wiwCCGGMMQYZBYxBADnAc0ASgENkLol5ZOa1A68MvXbkdmKfV2hrD3UPdDZ1tHlCLdWqtrwwCyja0dewrcsTccmqZqxlNxdXF7OlWrVmIkm+++i927Ztj2fTz7586vnz8/M5TYKwSYYHW50tYU/GgDpgOvu1yJAhBAaaXZ+9N3bP4dZYKGQUzNymMrteujRdWUgbhFAHogZhFoJel0QxTpQ1xbTgnUkcAIBCwAQIXQJ2SVixKIQw6uDDDk4zWb5iVisGg7bX37oKTMHjrp+enA7XNy2MXXdjR6C+eTYRv31r4+7jD4zeHG2pbyrl85LovH72VrCxPZnc/PQnHvvJd18Q3ULfQM/U1cuc08/LoUCwcXMzM3P5wt7jD67HU26n//KFay31TauLs07JRZHrxGtv+jxetVbgnTZfXUMtVwkFw0//5Oe++lZLKdbKqUhD/+zsWkN92GGXUqnitcvXE0vLg7v3J1Y3W7q7ezuaW/yBRn9HYj6RKOcbBlv/7pv/J7Uw6fb67tq3TwLwW9//sd0fSSU2bTbBkND02spDO+4+ef6CTUaAF155+z2DKWde++WRe/dv5oWllY3BbSOz8aJZrQz3Nk3PrDW2tdhlZ11zNzc/dz2dzjpdIUmWbTan3e4WBZsoSZIsc4IgCALGHEJ30m0RghzCosDzAsdzGGPM8RhxgMMAQgQg3UKiMgQsizIIMYI8BzkEBQFyCHCIbYWaUgAJYIBjjDBICcdBamlEK7bU+csFMLOY2szWRIYFiQUCqGwwTZcrZXbo2AGRGS+//Oq5G9PpggkQH5FwwMYNt8iyXTwzlqmqgEM8BdbWcZ0yFpD5toCc2NTnFmYLpVq2YBar0ALA70TRAMIM5GoAYOricaZiFhXDwWP0kdDZJIznscxjCBilTLeIgGHYyYsC2sgbqsEMhlbXV/Yeug+L0ujliW0Hup559qRR3Qy4hLX4hssVdgi2l969/I2RgxqAumHVNTdjagC9qFVqUEKxgBzw+t99/ifH7z9w4d0zgJruxpbi5eVwtOu5Z37w1z96hcinqJ6Nhep0jXe7vbevn94+tO3Ee293NPV3H9meWZwaGBr88Tf/cffO7X/z5jfjDxyKtrTfunJ12579llXMppd27d197oPzgfrAmbdOPP6bv5mOp0Ox+tZER0/TXMRNUunSemLz9Llzhw4ev+sTn37lmaeb9j25v6/3zVNXf395Xa+y+NRkxGmnQr5USdhs0dfffPW+4zuf/sWL9//Lvyc3bnjrfvsTX/zku2+//4d/+We+oOft67P3Htt27crkp7/6tfdefGV9ocDdvv4OJzh43i4KdklyOhw+SXQLoixLdl6QsCBgLGCIMMYAAIQxL8gOm0OWXIIoCrJNluw2m2STJIQwwxAAwBizmKWbBoBYlGSBFwSR5zHGHMIChBgiABBApmkBRohuAdPCzNALGxKqaoa1kigl8mapxtoCnI2n6aS1klUE3jiyd8Qu8a+/9c7t2aRNlrsjNo6aNpvYEpZ4TE/cTgoC1xO1LWZ0yO7IoTGAVd1653raYFtzUiRzuM2HYx4kCLRSMaomEAXMWbigWbIIQm7e0ollMChwEAJJ5FWVFFVTIUDCKGTHHEAWpcWSpVvAJaGNrLKtK2JVN5586u4ff+dZtUQam8JLExd6th995Scvbt8R6vLHagqbWYw/cNehW7eu+zxRZ7AhFi3FF+cD9c3XLpx76snD/+uv/9n39w2hoJZbvhHtGJ4bPbVzZPv/+J//PT5+pW9476VXf7prWzOANFbfuhJf3bmrqaUp+uHFNzp3dWqabiOWL+BS8unDe+/75t/883/++N9v3jBK+Xhzx8CVkxcG9+54+fm3Bnr63z955uOf+oTDxlRdNanR0BJZiS9EY+7rV2/2b+965Mt/MHv70u4HH5+8dObj9x95eWzi5Vff+c0nH8+tLnb07PRx1X98+uUjh+4+98Hp7oHWrGaFG+pGJ2/8y//4oz/807/YuDpWLcb7miK/+GDs8FBvLGTZHe5XL02MTv6I02s1pVJljEGEEIQQIgQFiHh4h9SMEcRbMyEIAIIQIoETBJ63SZJdttkk2SGILplzcDyPeMwYhBBSSCxgQYYl0S7b3U6Pxy67bbJDFmUscAhBDCDPYQCgXlNNrQasgmQtkNyyKNgOH97dsjR79uxsvqJbbrcv5G/pdjU3Ncs2eXpqMdLQ0dXZX8hky5Wy3Y6UUmklXS4oxnB3fcwjr62kMTFlAKtbwgWADEIpAAhCxIBHAAEnwMhKVpFdRJhAWQSE4WSVMkL9do7nod0uumQgSLBcIytZslk1AIMuEfEIVE1W1Q2BRw6BDzsgtMwKQdfHU5YGAvVtDz587/mz47Fg3cWL0wfuexRJLosABZAH9u+7dPXaxx++e3Vt1e0weNlT19M9e/lisL6lkq8GHaLgCPzVXz/zV396JDV/rbdtd8XEPmRsG9r/L9/423965rULryKlpCBo8hi3tmx/843XPvupL/7jt/73xcsXGj3e+GbyY1/+7ZNP//z+ex546/XXLpw62zm0Y+zC6Uhr36vP/3zk0MFwxF7cXFxaW/3g5Lm2Rl+hlPeF6xsLucnZtea66O2pqc987MG7Rvq+8OlPv/nmhxOnzthReXdD9MUTF3/3t3+rM+xMbiZp1YZEmxtpVd38/qtnDMpOvfFWQeP/5lv/ce323Jfue3j99rTH46xZ5o2xZb/LuTp9PRZrhrPXOUHkLAIoAQQAChiizAIGMK2tZfFHt74ttjyDiAG6hdMFEDAEIcIIIQwZhhgAACFEgCKGAGCUMYgQ43gbJ9oF0cHzdpGTeIEHAAKIOMRxCFJCiWV6ZKM7WmvwiSG/f2V+9sKNKVcoPNDX39jeIUqyUszNT6+UKmqooam1vjmxtFDgiu09ndSoXFraKBN6/PjOhqAvsZpWjQwBZOs2CLZ0DJABxngIe2OO5giWeEpNrqwZNc0AHKppMFu1DBMGHTjogR6nSAksKcbSurmRJ6oFfE6OR7ComDmDAogEDkoIOEQCASrp1C5iyti1+Uz+hbf+9k++ctPl2Egsb+ZVTKjTbYqmJQistaXp8qtTczPzIact2ugtFNbrmjtFl0NVyg2dfWO315945O6v/NW/nrtYf3gHsdRSx8DO2+deffSB/d/41j9ef+elI4994tXv/J+BQ9jtdgpCQHQ6zpz64DNPPvnt7z73N9/629LqbOP2fQ3D22wI/tbvfOW5n/7kr//5b669dX1o+35gqLWq2hj223jLYeN+9MtXv/M3f6RUEnzEbV9AFINQNNLT03F1auxPPv/lx37v63/yX7/+h3/w5Ve++7dHB9v/4a3z519/6cmvfZUQw+VzHdiza3J+LhYO3YpvIgiXcquiST5+8EB+JT5+a22gUQw7IQDgwu35R3f2zkxcePRo38pyN4IIIoQhgohBDnIYc5jDCCOMIc8hjDHHYZ7nBYEXJJ7nOVHiRUkQZU6SOUHkBAHxAuNFi+NMnjN4TuF5hUMKRrqAFR6pzMjppfViajK9Nrq6cG5+6oPZifdmxk6M33z71o3Xx8fenJo8OTVzVldXEKrdHL82t5a4+9ixLzzyWHtzX76gz0wvzs3EJWf94eOP7tu1V63ka5rS0dURDkcq1SpvR4/ev7+zsbVWM+Kb6cV8rUChgRCCYMtmuDUc9TuF47vqDu+sG+4IuGVkGGapQhJ5ki0TClh3ozTS5eABnV9VLk6Ur85pK1lmk3FvTPCLTNUtCqBT5AJ2PuIQ/HZO12mqYsg2XuZYRdd3drdcubF27sLN4aG2jUxK0zSllKe6AnkpWhctV9MIWKupyvmb46IgrS0vOB1ic1d3Orna3Ns1NTW5f/+hu/bsfPrFa4nNYmn+SmNL8+Z6iVLWUh94/YX/tNtDjHfMTFxvH9xukGxPS9s3vvXTxFr5yLG2X/3yNc4uLVw+NXj0aKZYfPgTj3OCcPHCm10DIbWkB4L1yfV4T/+uTFrpbmm7cfVGOV1688S72ewqB2mpkp+cnDh25PDyymomvfTVzzzx7VdePH3uVHfHDq9DGGxqnJqZPPnmqw998VFn0P3wg59cXE/KVBkK+QBjhg4zhUrA4fzBt7975NHP6HyEA+pw2DueSCXzKbWi9nW4I1hHhgGICRlFGCOMGMaQR1gQgChtqQ8Yz0EOMwQpAAxByCGMEYMQYIw4DmB85/3J8RhzCHM85hHGGHMYY4wQhzCPOU4QBFEUZJmzSbzDzjlsnMOGHTLntGGHjcoS1HU0O5XgoO2hRx9qaGqeT+UqwOH2du3cdd/9j3zmyLEjmlYZvTJarNWa2zu8rgCDUlUnu3ft9Xoi07PrV2/OLKSqTsnhgVgGFN4xXd+x4qsWmV3OnL+ZeedC+ux4MVOmGMKQn4/4uYgLydiaWKktJizdBHaZc8ko5mIhO1AtUNIAhiBqRzEX9slM5FhFB5kq4wGSAFANmi3qFhAOjux85oVTkUCgPtSYKOiXzs2E69sKuhoJejO5YiziKyQWzs2snLk0FQ0E0vlNh8vvCUbXpla3HT0wPjP1//vvf5LKZM5cSyenRj1+2TTN5XgeUHdzJPLCM9/Zd/zRmYlJwe4oVwoBGx4ZaH/muZeYhUfPnb1xa25p9GW9pka7OoupzY998UtvvXxT4NMGyG0/emx9Y6NzcCCb07a1NxdV5fL5i4ls/vz5GzabeNexo5Ozs60NAUrwaibV5Hft6er6+jf+3V7X0l/XtTskJYl05t2r05NTrX3BgYG24Z2HJ5c2n9y9LWTjFxZWG1raT10a/cXTz7776q/2PvoFprvrPVw4GF5UQFGh1ZKyoy+GNA1qOlN1qqqWomiqamkmsQggDEMEMQ94HvA85DkkCoDjtvZvEEPIACAMEgoZQHBreQIAA5AyhADkIUAQMgqpBSmDW+N5SgFgDEPIYyByzCYBu4jsAuIBSqVrSBRjzd6xW5fPXLzgqe/atvfQyMHd/qBrYebqcz/+9gs//eHa2qrHE3bIPkHwZHJ5h9PT2tYeX5g7e3F0UxU7hw7xgh0wU2IMU3LnVAgABLCqkkvTpbNj2XRFCQc5rxO7HYAQVtEsUeQKKqholt3OBIGZxLJJIOyTCUVVnRIInRJy2jFCwDBJpmRkqqYo4qBHUkwLIRB0wtmV5e3dLboJR69fPby7r6hoqUypraMpW8rqql4o0Z079s0vxY8O9j/30gfEZOOjo6FoyO2vt4xSfX3HxuaKz+347OP3nzhzOZNOgKqS18zZ8WkBgZiv7vyp9yAPTN65sbEUiLW0tHe5edPjYc//7MRQf/C1F94ATnz+Vz9t7B+eG5va1j842Nhy4vUzhlXYtntQdLgcHn+0XpYZd//2ofPXb923bShc15wpVdyyPnn7psuBvC6pr3vH+5dO3jXUa6rkn3/wg/aeLp+bU9PZI8eOfft//dvS5CW/I/+Fr3yqWCp5g67hjqbJhRWa3Vgp5ew2a2edsrm+0nXowfdnM34718XRanYjvlE6evAgqpVJTaGqQgwTW8BZ0/lC0czn1WJBL1ZopQpqKlM1YBJoEkQoJAwwAClAlCJKIQOAEEIoYxQxgAADEDCAGQWUbbUyHkBIGWAQQEYB3aJNm9AyoWVyuoE0E1UMqhBTp5WTp6/Pzq2OjBzq7RrQS/EPXn76r//HXzz/wivY5bvr/o8P9+9kBBSKlYqmlyv5ttamUrZ87sZ4rLv7449+CTFXMZ+XAK4xpEG0RZBiACAILIZUiw612h45HOlpsVmErGWsqkIavILMEcMyJB5UFFCssoBHbIvIPAQ1g6m6yUHGYVKuGgWVZVWgENYUkDp9Qk0xFIsFPLIsi/Mb2WItt2+49+L4lN1N++tDK9lKW2/v0eNHff66ubXlHcMdDpdjqK1FrZQWlzetgl5VszYBtm5rmh0939fb/8tnfvIbX3ikqpqXJ5ahVejv7dIr1ZHhwWxqReDZW2++2t3fNjs2Xs1vLs7NtfX1rC8udrdEqxklEkVXzmdKqRszZ0/1HN63NnN97wN3Tc5XnvvlG163IMiCpatN7Q01Wn3ywX1lynV0NjhwbbWora2u1jW5MvHMju19pfTmjr6+mZXpx3b2Xb058+x7737mS5+qFtI2W0QQHF/8+ndPv/rc3l1NA0P9L1+88fF9gwoxeTnCEP7pq2927+xweKvbdx9oc9mmNpOn17K1kjU/MYH9YWSomlFRMGcPtx7s3fMb++76/Z17PxeM7SHUpZRBpUCKWaOY0wp5rVwyalWqKkDTga4i3UCUIkYAYIgxaBFqka2lzhbIgkHAGKJbkiiE0BZVBWK0JRklABgWNAnUCXXIDAM6NZNXq2z3rmN+X/Dy2Re/8Rd/8ZPv/Szs6/j8V//y+INPCB7PSqKQSOVLSrVcykvQJHrtvffP+1v2f+I3/0Kwx6ZvX/BQTSGoyghj9COdDGMAUEb2dDoevaeppdFj6FSgtLtO2NMtOZ1AMxmiSDUxZXzEw4s8mk+aN1a0dM0MOHiXhHNVllNAQaWUgfaQozXsyCt6STW9NswsQkxgGOT6zPKOvh4oNF68sdDX2ZTPWHMzmycvfjjYHdIqmsbAwPD+6bmNe4/uee/UVU52nDxx0u7ASIMWAh7JW1K0mcmZB47sP3VpvpxJNoTqPUHfyOH9ksy3R/3PvXyikM1OjF7y+j0KMbt7Ruob6zTTymVqMb/z9IVLyBa5/N4PvXUtBnLbJH80HP33l89dPfehz8Fdeu/d+rrt71y+1dgQJMScW8ufPHmuud6fimf2H951+tS5/r7BmdmF3cMjN5bWgx6hORb50VsfrqaVo3cfPHvjxoGRbSsV9jfff2fp0gdPfvxj56aWYpFQd2NAcuCOqP9aPPs7//M/rp98DWVWj+7eoeikAMl0PFFOJicvnka+xmF3wyBnbyplcmtTl1dmLyTjY6XcklbLE1NjEALJAewuKtoNihXNqpSVcl4tF2vVslqrmqoKDAMZBjQtRAgkjBEKCWUMQIYARAxBhjEEkEJIt9BiHAcxz4BAsWhBDOwCqLNDoBLTQh0tIQEZF955+Wffe0Fn/Ff+4L9+9itfkURxcXpuYWy6mMpaOrM73Iam5NOVy7c2+NjAE1/+bw2Nu5dnxrX0aklncdP8iIKwNcyClLGBmH13l2d5PnlxdAUja+/20J5tfkhBuUB1FRWrAGLaFMU8D5Y2tHjOtCAI2HkA4GbJQJATOY5jLCDzpmXcWsyla5bbzts5rli1KpolCNzMRo5zypZZfPnMWKZWjTZ4xm8u/Ozl05cvXr7nnt1zk/Mj+0Y2NtcbfZJFDE/AvjC/kSsWKoXN9t6W6VtXHnv40ZeeO7Onr6mgqGO3L0UCkssVcAY9wajTJdPlTP6D19/LZDKmXnE5RNNCdx084EZCTqnxOt4/Ennj9dNul/PdH3xrx0OPZZLxkd3bNAr+4Zv/4fU5f/LKK9VC9vLSytiNm4wpqWS5gkWXXVIp5i2PqTGzojS3tUyN3SxVjVNTqwKw/B77P/3bcwdHtvvspHnbXhsWMpulyQ/HOlpbKgBcHFtoaWmfXJhpDsUgo29cn/rv//70xZOvuxiNyfJGpkoAApJjJZnlfvPvn4cUGGrZqBRq5ezm2nwiPs/5PfXcNq+/OVzX7g9GZVm2iKFXK8ViOp9ezW2u5bPxWjWnVUp6VcGI8gLPCRzmEGIMI4oghJBiBhCAhIGtLQ8EgDEAKaSM0jtOLMQIddkQZKCkmX4XCrtRYmnyytX08JG7Hnj8E35fXSaRXluZt1TF7wr4XBiK0FSNVC7nbNrutDXVNXT3b981d+vyzOhblgFXCVMZQx/p+7auhBhARsm1ybTPAzpa7U4Rl0rG7EJtM28WK8hiqD6C3TawWbAWMlS3gFtGPgkwwCoq8duRTcaZEhF5lNdNtUZlBIJOHLDhas2s6pTjUL3XXlQUA+DO5rp3ro4GNrI9Hf1iKNARaFmc27T5w4VM8eGhAXuojVCht8G7WSxLHJ64vRjx8xKvMoH6nXxbT5eSV+wYTk4vHN8T6RjsMBmJtrZkEkrQfsPnFC9dvhFtb797W9vQ3n3f+z9vN7c7zAR59sMrf/blT1649Hau0KIUbq+N73A0NrPpMy0u2wfjy3+iZ4/f++jJ85cfPzgUX8x2+ISljfXjO3tOfnixvtH38uvvD/e2TsxP7zp48Pvf/JeQzx3PFPY1Rdcz6Vyu+v7Jy4889UmHr6k7GCll1kfXNg66XR0h//dPnPvzTz2WS6zZbf53AIh63Ysbmyeu3k5WjajHgcpKRtH3Hdnz1kuvc+3tEUYQwnUuG8IAajVQqapVVUGMIcTxAuIwxJgJUMAAmoxQZmnVaqmYr9Tyqc1EYnl2bf56am1Cq6U5DnIiRzDAiHEYMAq2ZHOEMAAQ3XrajDIIKGGUQpMyDJlugWSO2gTQWicZoDa7ynbe+9kDDzyGObSZSK7PLZuGYXf4CDYMAqqGtjY1Fu0cuufJr90cX7n49iugnLh4+q21xdk1yuWYheBWWW2NdwGCoDUgR2TSHOHr6+wWAcvL2mZKKeisWKNuB+d1yxYlMytKvERlCTX7OBvPIKSYY3WyVFZoPKubFFAKIWBRBx+wY5EDNd3SAPI6RAESkcPA5r40envnjj0Slmye+muXbv7xPz7x/ttnu0Ndp09djEZcZq3a1d6uFxSPvzA1en1kqHV2bFEa6LhZmm5qas0msx//+N3vvvCaQeE7H07ed/hQIbWqaI0Q2ByBBoiFuoiH3IQvvfLuYNMj3nD63PVr65nKX3zpMyUNvXriTE9vy3ef/uDzD/c+80/f+Oyf/kWZufd0Nbxwc+XG+NShHUdXO/ZefOEfLLfUWtf87tnJu3c8+I2Lr/95x8fm9UQ05igQFquLCjbnjoaGDwozgt0hlSopVblybSIQPLn3wXBDzFWu2uZXl5Fg7Nve9+MTZw2K2lo6qlVV5vl6XzAoCjnTHOiK/eCtS1/cO5gtF3TM9fS1I7uE/E4YcSAvhzwciLpRa72tvTHYHA16fLKFIRCwy2eL1cnhiOQJSLLD5g8HO3p7hof3Hj/+2Cc++wef+eo37/vU/+rY+aTgrNuyhVLGDIuZlBIGLAoIAxalFqEEAMIAZYwxaFnEYpYJWEUHhNKBZlvQ7h69WXa27Dz8wCeIThfHp6YmbpTKeR7bOGSzkCjZQtWqOT0/F25tMTT+xFuvzd94/uwzfzN/+1QG4CyxEAAQMAi3DJKQAWDncYNH8Di4RFZ/90LuvQu5tYxmIsGiMOKXRAHEk+WxxVpeA2E31x/lGrygPsAHXJgSsLCpL6YMQqCDB/VePNRoaw2IjNGNvG5YHEbYZeMkBEWeq492fnDpsiCoreGwbtSWlapd4A48tOfM1PmD/Xtu35pLZDZGjg7MTI0119dNLyZampuuz87qirq6UESERzbk9rg6OroP7xo+f2Pp1o3bxc35fKosulyiZA3WB2XBURfzzC1uPvPC69nM5NC2IT9nuzox9vlPPjS5uhKyMyqYY3PF8YWFM+/9smfXru5wlFh6psgvzl1q74wQT2h5I/HgY49zzMqlqm31obO3Jtsj/vVsudkfEmQbH5T0ar6nvWUtmWwP+DRNSRvm5Pmr61Pxwb5uHiPRAjOL08eOHQAAnL91M1wfsUO4v69DrZYiAfutmbVtMbdbxDPF2v6hgWQ652uKIIEXRUGkEJUNs2iRokVVAiQbEGRoE6WGgLsp7I65Zb9d8LsFv10IOASXTZIEJInQbgc2GwqGAgPDB+556GvHH/l6IDZCIWOIMci2bMcMAoAh5BBEECIIIcKIgwhznMhhjoOIAVAflpvqnaPTmZzlOnLsHq1SGL9+aWz0GiTEbXdCjtdME8k2dygme/xKJQupOTO3fOPGmVYXhzRlSWcJsiU9huDXHi7AIICaxa6uVN+YVN6cNqcS1GlHbiew8dTnwKWKtbChp2sAYBD1icPt7phfsPOwprLphLWSYZDiOrfQ4OWagqLfhSqqsZhWVnNGwCUDxjAmIjABMSDimxpbSxXt2sTYnj3DhVy5aCrnRy9/4v7HDJsYLycfOHz30z96trO5yxlykDJwhkI3bs0eOdi3tJZoberYiOcK6U1vkC9bxkBPP8Po3PlbkgRqhTTmwdzsRHc7lyjkOuo8ooimZzcL6aXjxw7s6I4SNX391u177jp8cWz54Lauc5NxnxO//t6Z9c3xzva6mCycH53yRZrfeeV7f/xXf7tRKC8kknsO7UykUt0tzc9fvL1Zq1RSGzPzk7mN1Vg0PDq/9sCu4WRNsdllv90+u5krO7VieX7P9t2KWkES+uDEyT29HVFZujk+x9skd71nf3dLVa0c7A0XTStVNT5x1/ZzU/O8S/SQan1rFI2NL4/Prk0ubsyvJdeTxXxeK1cNwwSIZ3YHctk4EUORQ6YJKzVmGoDDSBYhz0GAIUEQckCQod2OvD5nfVNnINrDEM8gY5BBnjHIGGYAMYYY5AFjhAJCKEGQIkQxAowCyqhlkQu3yuOrZkdXL6Tc2fdOpOPzPr8EGAKSyAl8Np+wO+2yw+VxN1iSu5rJa/lMsZwGvGPdAFnLpJQA9pHhiG1p8yAE0KSsahKdAonHO7vEtkYRQFioWasZvaJSvx377bjBw4WdrFxWplfU60vG+IrOc7AlIsR8vCAwAq1EwZzZMBczRtUCnfUelaJs1Wry2izTxFhwe+oaG1vaoo1nzt+8tjC3fXiA1sCPf/ycWsr83pf++N2r57Zta1ENdvnU25/+0sdv3lo+0LNzbKW4q6fT5sWaknL4PEtzCxzE3pATE9QYDZ+8uVIqGumNFU2js7PrR44OqbXNpsYGr8ueVGwvvXKmIcoRm1gfa3z/vdOP3rtzI6txUGxpiMVLtcWV2vunr6SyK4cGWs6Ozbh97eMTE6VM8qnPffm5Z549eOjwany2uyniFoR3zkx7fMGpueW55Y3u1t6awQzLGGrv1A2jJxalgJ0bz9y+caF3W29TR3e6VLl6dYJUCnu7WzImvTW9Gmtuinp5BmXVwMe3t756fnr7wC6OgTcvjc3Oz6cmpxE2iUixZFlAq2nFslKtmYBQC0KLBxbI5Gr5KsmrrGYRg1ALbHmdiU6oTmFVRzUNagbQTS2ZXpm89cHm2iXALAABRJAwxrZmW7/uIxBDCBmgDDOG7phVIQazGeP6us65RJfNfe3KqEVYY3OnP9RQVXCkviGVXJqYH0eSnenMZQ8iR2R9ZR4YBV2vXExXxrUt7cId/RSgkAG4ZYxlH/m9IAA9EclnB3Ortek1M1mmdomP+QWAGIcoBHBp0xhdUgoWCAbtPU1SwIZrupWuKKmSuV5gmxVaMZjIc8N1TqLr44lq0MULAAAgIcnl9YXrQvXUMjpb21eW0iYEuwf6xldWv/XTpx94+NDQ8Miv3jz3yPEjP//Fy831zfaGJq8NddU33FxJdnW3EWipWt7u8SzPrrV0dS4m1rubmhNFJVMomnoCYEqQ5bW7oLtlaTPb1+zPVdRL1xK1xbXmzo6aAaWA++XXLv/x1546fenGfUf3Bnytajm/klXOTsW7Gtw6A5Mz8zab5+/+919+4uOfXkpldV2LtHcHw5GmSChVU9768NZgX9/41VG3G1sAnL8yur2vYz2XawlHBYDiVev507eLmv6lTz25ltWmUuWzF688fGCHCcDZy6cbmlyQwoBbXN/M3z20R6lUfP767oaGczfnv/3qpadfO4kKieWZ22O3RxfXF2qICT6f3SOLhFnrmdz09GKumGfEAgQABjgOCBxDGJoWX6qxhdWV82ffeOm5f/nZf/7F09/+o9ee+cvRsz8o5mcgJBAxxDGIAMSAAQIwAIhBxBigAEKEIWAAbGm8MAAYmhhzHKkPRYhmCaLU2NUdae3TLS4cra/m86fPnKipasAfRoLIS1zMG5hbHBNQ2S6JC6l82TTvmAYZu6OOAVti4jtVBQDACKRK5qlpcyIO8jVmElZSyVJCE3iuvdFBIZMlbk+PbVuLIAk0WTLXc2a+QssqK2pAIciwQNjO72pwVlT9Rlx1C6jVx+dKOhS4tu5BgvlsOeerqxckxxee+vzla9cCNs7B4Q9Pnnr2mR/89hd/9/2JMUZrHtn29jsnP/dbjy5vZCVoBbyey1fGugfq4nPTnT2di1PzPp83nk7193aULJqIlwQe2DBxx3qu3Upv72ken1ryewSvnc8h/o33L9x3z0Aqm330roNnT53HTGlqiNy6+eFffeObRw4enZ6LX5nNjM1uyBB8OHptZ3frtWu333j+548/9NgvX3xt967BXCazrbNRh/D22oZhmPUuuZItd9U5by/EOU1x2RyCDcmyTcAgoah/9Q//3Nfb6fQ4chQ++86prmZ/g902vpLMlNDO4c71zeRyMX/f0baDuwfW0+sPHDtkAUAxN7a4iWqYC/W27797zz337OzviYoI5DarK/PpbCon213BUNhlFySO8hhCxClE2MwZV65df+4Hf/+Dv/rsC//45bO/+t9j53++OvNhITNJSJHjIMJsy9yMIIOIIQwhYhAyutXAtgSmDPzamYURoBaRRFt398769uGW7uFwpMkwgOD0hOtip8+cXVzZGBw8KAiOeHL16V/+O+C4xUS6Wlxrj0R+HTFC79TUnXYF7wyx7qx0GAWJslnTWMwrtEf5sBuHnNy+Xt+2dm+xZEb8wn17fXYeTa+oi3ElVaIlndVUVtGgRgG1aLNHaAnwG/nqzXXVLeNjnR5dJYxhye7qHdgVCNclE/OD20YWlpb2jgyFfKFqEWKGd3T1f//Zn+tm7VMPP/azN652Dd9z+u1XeQG3dDSvxos2zjO1WFYV5BLsAuM0xbJ0Q2eEFzmngJ2SK5M2kamJWBlfWHYx1eMK1Gpse2MoVdZfvpFauTruCdnWViu/+eUv/q9/fM4RCF+4svTKK//6td/5qszLGc1Yz6oBSVzfzOoEtLbU/ct/fM8T8M+uZddWVuwi6m6O6JS5PPbr1+a69+5USlAUvSmLnJucbWtsTFcL7dGQQYjEc6+fPP+P3/rPplBAhPj8Yub1cxc/f2TYxlxjcyt1ja0um/3ibHo9tfbUo0dmp27dt3+HjcN2pyPqcKH7jx3Y3t0oYmttM72WqOkG87kEj8cdrI9EYn63xAmIMsTlquD2TPytt197+t//+9Pf+OLFl79R27zld+vNzbCzE/V2o/4e1N8FBzvZtg4w3Ab6m1FbFMYC1O8gLonYRCYKjN/KMAIQIYgB4CGDDJoG5QD38INPdnXvqG/vdLp96/F1wrRYNMLLEsPS45/6jX177+GBWCytnrv6ocOBsBg4ffXq7r6WiE0ADAAIIfz/K6Zfa963UhgAhADClpB4dMi9o8Pe3yh3NciKaVydKcQzVi5nvn8pe2a8tlkBJRNaFLolQRY5jJGTxx0+wSGw+bQ2niUxr/iZnX4vYolsraunu65t++TUrUi4fnxidsfwsGGBqqYf2X+4d2i4LdocC4Va65u+99Pv/cZTv5GulZVcoq2v5/nnXr770f2xZq8sC60t0fdPX7N7WHLtmqblkitLVaXisHvqff4yU8q5XLUmbMRLJ68t5ky+pSk2u1mhCPptwnqu8urJqbt2bL9y5XR3U/TA3u0vvnupPlb3r//x9vrC5c8/+ZBBrFRNl0ShmEgJIiSqNTLY/P3v/9hEbGohceLsuYjT1RP2z2WrWS17+YMPO3ftjDodAREtJlMi0TksHhjq5hlgDEgYv3/xCoQ2jyQQBj68uT6RSv/5H3yllNmUAs72ligF4Js/PDGwrSlZqlqmPtjeVipXGEVoc7U8P7G5OJ/HBvE5kM1GDE4XHcAj8JhYqqpt5tSrY8vPP/O9p//pa2/94KszF/5F5qe7++HwLtQ/DPp6wGA3G+oAA+1woA32NaOeRtTVCHuaaX8r7W0FbQ2sKUqjQeL3ELtMRYnyAsMIAsYIZRRgatIDew7v3X9/uWS5vdF0Ml0upqipTd2+Vsxmu7oGDh2+q1ZVDM0M+WNdjTG3t+34rrtGJ26VS5uHmtyDAb7Rwfl43oGwAP9vbf3fCoMAAIAZ6G2SmmKcacF40hqdLq9mFIQhAmwlTZIl5pCQT0JuHrl5QBg1CQ2LrMkGOEhSNaugQb8NfnZPJJdTF5KKCODQrvuh0//O2VOiDSAk6EotEvTN3rq5lCkE6j2f/fQnlxPp7sbW1YWpGzcv/Pnv/dHrZ08e3nN4dmp6M5m56+MPrK1vDPa2rq2WkhW4Ek9m8+tjN68Co7SWWDu8vX15Nd3U1JpcSaiUX87XCiYd6OhQavrY0madx/Gx7bGFZNwdadox1P03//ivn37q2FBLQ7Fc9vpd/+fbT+8daWoL+tJVpULYRipXVUuSC+ZUuqO/7cObMwYwC6Z1+trNe3Z0KLqeqDKlFI8nN48dHdENCgBI5yvZYq6ptX5bQwu1rIgkVwhLVxWJ5wFg8YKqlbQXTr7bGnZlVxd7W4IAwrOzqfdefWNHh/v27VsHt3XnShW/z4sWpq4XNlfcaAOy4vrmxvxGqmpCQcSWyUoVLZWp3hqbefkn/+X6O3+I1bd7O1KHDsODB+HOQdDbBtuaUGM9Dgd5nxe7XMBux7INYwEBBC0ATYIMBVkGNDVkKMjUEDMYMwHRKbEYI4gRiCiNuKTDI4c0BaRS6XKxeP7D97Pp9IUzZ7P5vM3llSXPiRd++c5rT+fKRacj0NvQRSzYGA40+Pwf3prTGfRwqN4OWmwsLADb/5PR8NEfJYwSRtvCQtBOr9zIXrqRrxpmS51k51GuaJZ16LDxdR5O4iEFFCLGELDxoD3AxbzIYGRTJQxzbg5uD3O3Z1I3NjQoAZ/L3tq/D0CcLinJ+OJnPv3IzMRESdFIrVrvk25OTt5797FgKJzMVTraO0dvvnN492BzS/eZy+N33/vQK8/9cmi4G6FY2O2sr3cuLJWmpzVNBzOT05aq3F5c6GiP5IrlqqEBqDsEDgFQSOXKQG0J+b12B8Zwz4F2l904efHS/Y88nshor7599o9+/6l4oRrx2eKJ4tVrYw8e2qYz9tRD98te5+itjd193enMRv9A786mulfeu9HdXH/q1rzX5QzZpMmFtTwVLl680FzXYOdhW2NII1a1qsyvrP7eJx9xC6LP42SMFNWqx+bgANwslxBBmwtLf/fj1ykO9dd3y4xBzH3jh+eVau3mxFhXQ4gBOJetoHd+/rVbH36HGMAykVIuCVi2Y5EjACNik7FD4lyivb0jeOCIcXA/HuyGLTEUC3BBD+928DYBIYZ0ExZrNFdl6QJdTZKlDbawDmeXwfQSmluBq+toM45zaVwpYq2KTJ0xCpnJmMl4iAWMChWrrGhqtZaOJ6bGp3hR7h3eabe7fL6wJLlmpq4iDHp7dtpkXhD4ulh90BNl2HZwezu1jOmczpDNJNhpQwE7EtAdIDoE8NczUonnemK2gXpuPVErVNlQn3O4W1ZUI1OmLjtuCMCgm9llGvXg/np5pIXf0cx1hDgK2HoFlHTolTgOALtE40VwI8M4BFz+qM0bra9vxJCYEI7enOYkvH3XTh4Is4ur9X5XNpvVjXx3dw/GctQZikZi166e/fM//t3p2dnu9rpMFpx47e3dxzvzm9Vt/f01TbeADTJybnxxZjWfTec0A0puZ6ms1vf1RoN2N0bnLk4LAurpaDu+b9tqPFdVfcePHXnlhfc2i5l/+LvfeeaNCyvLiS9+/IFr80mP2/7uqZsD3Y1eDDhJ7u0evDo+ncooI3sHb1+ffWjvwFqlZueEzrrwxFJ2pLOpRNn7F2dmbl/9yUsndvZ1aJpWVhSvDZ64dH14R/uDe3d7ReQTxZyqhFx2AXIY42sbqaM9jenN7I9f/2DP4fsf7O4kxJrOVS5eSyU385joDT5/tlZBvpCpcqVzF994+61vpTO5gNPnk0WXQ3A4RdkmOTy2pub6npGv+Zt2AEBqNZQvoFQKrm3AxWU6P8fmFtDkFJmYBRPTcGKazi6wlVW2FrcyGVYpArWCiIKYiQ0VGjVMCeQBEiHHIYwQ0C2qGngjbVUULV9ILa3M+iKxHfuPt7QNCIIk2+2aqhGqbjt4787DR4NhTyabGexrrfPbEc9aW5tHOprjhZpo5+u9do0hjJmIIf7I+bX1zSE00h0eaXdYmhaJ2A7tC0QC4mZaE3g03Cl0N+KgF4U8qDHA1QcEUeJzCppLmlMb5mYJOES+xSeWTaBaxCXSvEKIZvYN7Lznsa+ULKxUamq1TACLhtpPvPtetD54/J57VuMZSZJMtbqZjHNmZaCvVdNLvX271jeTNs46cGD37NiNu44Ov/qzE3Vh4vGgybHbfb31gt0RqWtdyqlKhXR4JUa5YCRs0HK2ojzw+d883BVLF8oUwEirvbOvWbLxz79+TsZcLCZ/859+GosGP3/fnm9+9/mdAx19jcF0TatWK2o1v6szfO3GjUO7t1Uq6uLKciZZgDazu7VxsD76xqWpjz10IJVIBOyiA6HFVCYaCI+urjkk6GDMYRfX47nVzdzZ8ZmnHj7YEYq2RaMFRWnwuYM2GQGgM5LMVD57bO9/PP2rCxOn/+Ef/7LP7STMKmraRDyfyqSPbOuu1hTkDsGqent5/vu10sVS9srY7ZOjY7duzs5PLS9MLE5evXVm9MqLc5dfmbxujU/j8Vnu9hQen4WzK9zyirC2yifiOJfFlTzUKxBTwSmKHpnzO0S3XZJ5JEJECVQJMSyLEQoZhlCwLGiZTNOApgFVMZpbQs1tPcnNlL+hsaG5sb6hUcQSJdRmc1oW9YcavP6AKNtKWW19YVmtJifGXp9fn5Jswb1D/X6n7Way5Pa5fHY5VzM5xjgE2f9zvqIALCynLt/K5SuoWLLGpgu3JgumiX1ukWq0XGLMRJaJ1lNkclkfX64mcgaEnEOGMTeL2cBGxcyo1C+SaCg81NM+3Nz1xd//nw1NA9WaYmhqoVgBDNjsNk503x4bffTB4zanpKqGQ0KbK/FgNABV5o/6EgtLwZD31uipffuHl9ZqzXVRu5179ifvRRo9i4k1plTrmn2qaXo9HhPRxpCHE2ChqlZ08fUXn2tvrXeFIrczxfnFpGyTmGHs6GmbTKVn5kqtze06BX/xje995UsPB73+H71y4lOPHTd0q6qTjXiirT4yPz/TGAkEPH6CufhyIlWrpkvLTz0wPJfMuQP4qafurxTLbRFfTrVy+QqkbHo5M72R5jkxGGvw8cIbp86EGqTDB/o7wxHKgMsutwV8hBCB528txgcG2u/ZM/yjH7373uzUd//tj/fGApiYOmFnp+YOb2thJkPxjZRZyyOspdOz597/5ss//uTP/+3+n3zz/mf/4aFf/MujL3/3s28//7ujJ/8hsTiZy/KlKlYNQVV5Q4eWxUHAYQYlDttFXgAYUqzUUKmMigWUy6DNNEvlSaZIyyVomhyh1LSoYVjEYiLkbBwXdIo+B9o70jcwOMzxDkI4RqxEfLVSSJVKeYOymqIjDjs9LkZBLpPhWZ5HFGvxi5fev72SK6vQZ5fTZfXk9KYMBYTEEgBsyyf0kbqBUJqtWgUNzCbJ6THlxoy5kYOZkrWxqasWJ4hAJ1axxhDH+1yCx46cMrIL2OfAPodQMqgBkWBa2wa3PfiJ34809B565DPdAztWllZqupnJJVY2UkGBI+Vce+eu86dPc5Dce8+RjY3M8ODA+Nj0tuHeTG6jLVo/tTIX8gXWNsrQKnX1dq2sVO67Z/flc6PZTWX/yP74akrg2exysrc5Fq/p4ViHpurlaiGTpadHZ7/1j/88PNDIA3Di5LXievH63OqO/nYRwHPTKwaLYIbGpuJ/82+/+us/eOrcjRmTsQd29aZrxnqhGnC6NMUqVJTdO3csb1jROs+N24ubeb7OIRzd3vmTX5x55KmHZV/MyVGnxK2mshwA9X5/UadrmcIXPvaAP+wfG1tMZqqLuXJvTxMAoCZZO4Y7GQA2ACEkmqk8eve2qqm88K//8a3n3v78px8MIuaxc5cmVtxeePeuPuSLcZ4QsnmA3Qm9bugQCTWKammjmFirJROsVuAsw9RprWBVUqSUpZWiqdSorpiGpVsAaAYq1XC2hNN5FE+RlbixHLeW1s3VuJ4uwEIF6gbGCNtkwemUXaLkscleh+i0iW6XJMm8wEPGLAFoABjxjfnlhSliKoSq6fSaZlqUsvrGNq/XxwFAiWoghZckd6hlNZl5+9yleL6GKQIArJa0s2t5SgFmjIA7tJOtDwKQIlSlUCHQALhq4KoGHDYYi4kBH25tckcCYsALJZlVdaIZUOA5u4gkAZomUS1WLml7dg0/9aX/MjmVDNcPdA/265VqqVBsidWtLE0XN+Nf/NhnOLEa9No3MuT66PlDh3cbeq2rs8XSlVwiZXfbKvm8yy0Xkxshr3sjntg+1CBLdqoLd9+1+8rZywM76gtKiai1fLG6vTkqAlCycoqaNU19eiVdH+79xr89Pz01U+93TG/ms2VmAUANrT3kn1+PG3qJIWFHV937F29Nzq1/7pEjP3/p1IE9PQLCEzOpzs6Y3yZkMumIz4kwmlrOOAF95/Ji1UK7OjzJ9c23X3rt859+xFTMzrCH5+wWRZDHfbHw5EZiM5c5OrIjXlWnp1YMRiOxcNTlmF/PbO9o8/O8QZBgc6QXlrvq65fTm4IzbM6tf+eVtw4eHfEhlNPYzZmFrzy+FwkSEezU6WSREIgFQawONDSD+lZY14I9ft4mcbLN7oq1hur3O6LDCnNUyqBcwck8jifRShyubpDVuLW8bq5vWokEyRZAtmAhMQCwXdMJxshuFx0ugeOhRYFJmG4C1aBVzTQMSqilWWY8nVS1kqkr80tzV899kNtcdTikXCnDAJA4xDNINFLOFZdmLp44dzqxkXZLdpdNnltZWM9shgIOCABCIK2bdV7n3ubA1vT91+9CBKGMoMSACAEGkIOkrV4I+DnLMhCE6ZxeVqBqAs0gNgl57JxlkkTZnM+YUxk6kTai9XV/8BffSCYFQYyM7DxAS1qxZJSrNW9Q2kisYgr27D7oa+lWq8s7h4ZPvH9eoFpda3h1LjM0PHTt9OXBgeHZxc32aP3yaiYckOPxxOLsdTlg3J682dUVi3bFVleyvcMtpXTa6eb9bnfALk/N5VIF04H4olLav3tHxB/igb+3KcRLfE7NrW/EE+ViwM0jhIvlbF3EP58sH9rR/W8/f/PQ7g4dcLcml+7a1jyeyOeqhb7uhpWVNZmjvY3BbMEIeQPzy3M/e+96sWpxduFnL70Z8AvuSCSVqw00+Zw2USNqd9RhEvbiOx/u6Gxz8dzJyxNOQVhb3NjZ1TW5nHA4/S6nTcdws0pO3Zi1GXRHZ+eJqcUjhw+1uMPvnL890tPFQ/jOxXmVEm57n8cvUj+2IyzPrmuFso3QWtGweBDQq16A2znZJ4rOckldz8wjoWKpglrTNVU1dcPQLQIAAgBvPV6AIKAQ4lKpBBhDAFkWVDXLMBAAAEK05a0ngAHAMKQchyDDuWw1uZ6yjJpG6Lvnzzkd7kohlymVpiav3axcNLR1twsv3Zp6592XbiYLPmn0cF8sYJOWc7WJhYXtHV0RhydZK0HApapGCIscRAT8Om0UcBjYecwxhgCVBOCzIUhpoQgwEhUNZItGWSOUUEIpQ5xiAt1kFgO6SRmGiLGPPfZUe9vwyvjJ3v5tG2vr5UzBFu52OUPetoFixdg3chdvd+/bd/DK6Tf27h760Q8ur6yu9XTEPnx/dKC/J69luyjmZW9iM4MtXKuU42tZQPSgRrqGBxObtV37j9+6OS5JFehGrkjENOBgV8d6Mm9Z7qm0Igm6bBcGuxvGF5c7O0KKtmQQirGkYi5od25vhmMryaceuO/67JJpMV6yf+9n7/zWZ+7+h2//8g+/cPD02MrscnGot+X6otLe0iAgNeh1bRRrHQ3eKzNrmDEbb7+9kvrRMy+0NfhPTsxXrGpPnS+Zz1m8WOd03lhNb+TWP3bXoTdOnf/kg4fGNhYODQ28MXp9KZUOur2Lyyt2QXxnudr77oc72wNnZ+a+/96H3/n67/6f//z5ejW7vT68tJa7MTqNokGH3+8pG/yNzerNlcp6Ul7faFXUQ4Tu0czG9URh7Pq18x+8ce79ZybOvJOYnImvJtKpXKWscNTuD3S0tA5G6voAcjECILFMQkzLUlVF1VTdMNSqWioohXy1kK8WC+VyqVot14yaammmoUJNgbohxDcr3336Z+cuX7YArFpEs1RNqWCAXn3rlZdPv3Frbi6VnF1euLSQLDCIbq4llzOqU5IhAMmqnqlUvE4HYwxhMF+sXlrNUgD+3yGpRVhVN1WL6JQRBhnkcjmaKeCluD63XCtXKLOwTlBVRzUFGAYgFhEQs/GYEcvBC4ND+5PLWUSAqVUWJybTyRmjUuxqqmvv3H74rnseePiITZZDdU3uoNeoFfq6Oq5fnXTL3kijb3b8hi/kWJ69uv9Q/8Limidi21haIyZRa3R9Yaatua5YzmWT+W2DPWevLy5ny36ZYlHcM9Q7t7ySSqQwJ+gGWVu90d1bl83mA56AEwBNYwd2j0xPrsVLtZpFPXYpnc0OdLVdn5zvrfeen4nnC4n77hr58MrSrt72ty5MhFxIEnE41pDO53e0RZaS2Ya6SIPLfXMp6ZR4nudPX50xtBqPULFWFaDW3VlXKulNQS+F8MUPL+7b1lbQjKnlTbcT9bd4m932a0tLjdEYAIDHiOP5Z87eoMQK2OwzG5l//vkLf//ff2fXtu0NTsltmufPXedeOhlXdVBT7Txy82pDUA6l0lpxaalaSanlil4rK+pHYcMAeDyxSGNXfVtXXUtvXWMTsrucNrlaAesbG6Zaq2hKqVrBuompiQGEgKOMGoQSSzd1xdQUtVasVDK1yoZRSxtamVGCEUAc/+7Zs5rBMMaEWBZREAOEIpVSAKGAeKOWy2Q3ahBijKqGeXt+nZj6lp5ham2DhxgAQChjCDIGMUSAkV9LkwljBoAmZcACOqE13cKQiYIFLGYBSDRiMIsCgBnYWmgKHCKE6RQoFmvq6ow0t9VSJRnbZJlls7OWValrtr3/xoKimYeONrz1wVs+V/1h+V5mcblcYnhw5NSJV2T3g17eqzuLGwupttZ6Uk3HIvZTZ288emxbqZpySfZkvnz9+mgw4H7v3JtP3HUIM/bMe9cdnBmMNIoQeUK+fLUcsfPLJePytdtf+uyX/9N4N1OsBgPeW1NzFUW5OLcucpxNwnePDF6fmj1+ZOe56+NVRW31ur///Pnf+9Tdv3rnUm9jLFnUx1ayO/s6PG7PXEFvjpAndrfcLGfuHWn98cmbSaWypz90cza3sFGwKHW7fZuJtCdsyxRWABYBY9cXNrsu3fTZpdHxaZ8sEMz++KuPn74w1V/fAACQRMEh8xv58o25VMTpyKnqm6NToV88/4dffap2313rm7lEfJmzyLClG6SkKzmmlavz2li1WjAV07rzbHh/tKm5bbC9d7CxrzPS3O7319ldLlHkRB5BRgTKEIUED2OANZMYjEALcQByEFHGKAOEQQIIIxZkzLJMpVrLZ7Iby6sLC2MbKxPJ+elsZlnVahACiAADoFoxHHaHw2YHIAMAMylSy1qypGmMcQxaJo1nSlGPz8npFaLVTAsCCwIIGGEMMgApox9JG+74KSi7o3YoE1IiBAAGdBNv+fwB3IqQ3KrBjzQ2kCFIANg2vMPjCeZW1VQ22eL2GqZuE91ObyCeWpFFVzqez2+sRRy2fHxVkAJacUMSufqGhumJ8UhLayUfn05t7BgZuj56wynhbD41tV7obKo/c+Vab3v9a++cPX5gdyKV+9mLrw80N75+Zf7VC1OHt5G2cGhPV/OrlyaanHaZM06PZQ6tbXR3NV6bXPT6PZPzKwJIiDwvYJytKkubSb/Xlsvn2+vDs/H0UH1gLK48+9rptjrf6GI8aHe+fnb2cbtr2zCNeQK/ujrxh08etNK1gYH2r9odP3zz/P4djem81RILXJyLVwghkuP67NJ4Kh92uG2YVyl559IUZWBsfo2ZcHRudedAOw8EVVcECBWd1LkcAqzOplJtPh8HAOCEH79z5drE7KfuvbuxpS3gtHHrN0q1glIplohlEWKaDAAAHPZwY9f2vp0Hu3aMtHS2+ENh3mazABIgtWPm5IDMQwlDkeMwByxiWRYzNVq1iGIQwzQJBYZBDMpMizDKDEJ11aIEAsJEjPz+kNMZqBsYqJWrSiaxuTw/cevS7K1z5WoKITi5uPzc6y8XqxUIIWPMbnP6gi0uX72UrmGMCaAqQDqENkGoKvqvjbJba+itPTRkiAH6f6ekEGEACSBbcvitakN3OhrFmIOMmYQAeMcTySDAEAIAnC63mimVKhqS4PLqEkVCfXNXOp3NZ5d37D2az6ci/kBrUztQqy5ZlkIRpZpp7eu8deXivY89Ggq6m9rqL1+51tbRce7dU8NdDa+fv/JnTx4bi6+HAm6XZDt7bUyDcGUxrhnWQ/sHz48vf3h9Be8WBuvd5yU+WVNDsm2lUnr/0uUdrYEfTSx0CyEscumaJvIoGghW1tdnlzbuP7wLWubugZbF9WSVMrfIz6fLUTfziEJercTcnqdfPD0wvGf37pGrS2vfee26xyHObiRboj4RctfH0hjAeF6VBeHEpTEOIFXXTAp1ZtlEwdRUAiGhUNFMxIBSIi+fuWHn+eFYzMYJhmnqmi5izqRoPV8WELIY4zA/tl4e++GvAAAYAG5tegkiwCjgEe9wN8U6dwzt2r9j7+GBwa5IxGEwoGikrAMnYF6J+SQk84C7IxcAEAGMGOD5ZI2sl4x8yarphqVbxLAM1dwKSUcIiJwgCljCiEIKGCWIIInylgUEzhGODtUFjxy57+zb55595k9Vs1BVamdGb2yWSxzGFqGKZZ4en91QCINAtwzCmEZYKZUAdxJIoMTzJiE6tRCA7I7g/aOc5DuVxRiEkCEMwFaLQghSShFClAKOwyalgBB8pxi3cHMEI/ziz3/gU9Sd+x8EDi61QZyCKxQJJhNxBIz29tbJa5PRmNvusXHQq2SXsOhJrq/UdbTmctVCoSi5UE9n7FevXzh29/6ZmNvrFBBFp2/N3DvSf+HW7GeP7z9xdnI8nwp7HJcnknt32Qe6YtNTCy+cufWXX767v8H79uSaW3IGJenKjQXP7m6P057IlSN+X7miCoDVRYLJXKZUUy7dmnWIcGSgAyLA2R2SSqCiZqqKT7QxwyhUKwJn+/cfPt/UEMUAllWtphkLydzFqbidw9cXkyZjTo63IKtUDZsoIsApxFjNFzCEAsdjkbNUHSBoEcpDInLYJNZkYqNmUQRh0VAopAaxFI3yCARkKVtTAAYIcluBoZzT2eVwu8Mt2/oP7+kc6Ovs6vAH7CKGjNBExaAEBSQc80GvBGQegK03xtaACAIOQZOA+U0jnjMsy+J4aINQMYFpEcgxDlCRx5KN52SJ53iZw5ABQokFGADEoSsBUS/llXKxvFnNLEyfNqwKYczj9e3sGnz5wxOEUIzwejK5lnzt/zmOQwQAhIgyBhgDiHkdQsAhGZZVqRk1w9QI0Uz2a1MhAIAyRhmBkEGGIAIYQggZx0FKGYPANC2LkC3uAQKAUriVOEoA7PAIfjuyLEMEpigiBgRZhNdOfdgzePzovfffvDw6MNBNCAXEylTKPskOeQQtIxBwLi4seuw05pXqw4FbEzMjh/pvX7ixu7/15szsJx7YvVnIzWfW9u3ddu2FN2etWnfQ8/apW1/69F1uAfzq9O1n373+6P5dowvpjKq5JZka1qXxVdWyiqqm6szrtiOEakpZkuRiTV3NFBkgm7kahujWzLLfbscAUQoUy0IIKaYlu2wzyysTyyscQBzigm5noVKxCMEY2yW+XKsSACmhECIeYd0yIAAIYgaYbpmcCimlpmUhCAmAFECEMOQEaqkMAsRzHICIQQqoKAmt9ZHK4ppqWRgAG88BwLgv/d0vPU5HINoY6+EprSGT1qyawbBLlNtcXFAANh5AACzKLMrIRyJgBACPkMngbMHMG1bAwxsmLNZUXSO6ZXAS57bZRBFDSCuVbGJ1Mr2+mkwsFdPJUiFXLpZVVdXUimUogEBdQ2q1XFFyWxXLIRgL+kQO1ywiCxxknEGoRQkPgcxhizKdEkq3xM+QMZguq5pmhp182MlrJizUjAKxLLaVxQ0ZgFtgM8AgvbPmYYxQHiEOIcqYSQiEEDBIAUQQoK1ehyAj1uFDh4cO3rW8oCiVnFrNuO28pmgz49f+8l+/PbEW16qrRtlRsvIi02pKycss2enQNK29penS9WvtrU1WBXW1h989c2toxxOyYD/SP3x7bmM9XXz46J6X3z7b1qTt6Im9N7FWVGx2yXH67PQnn7z72vjK6ExyoKM00N744eScZVKv212uVlXTwAgphmkzTATh5OwKRZgBwGNAKK4aBgcxBcQkJsKIUmJaJmKMQVSulkWOQxT8f22d2a9l+XXX1/pNezj7zHeuW7dujT13u4ljd+JAHBsMSDxEOCLKC0JCARREJPgTUB54Ah54DQ9JhCKIYidxYkgc27F7cA/V7qGGrrnqzveee8+8p9+wFg+nqh0h9uN+XlrDd/3W50vMNti909PldnM4zUvvpXMC0DMQIEMgAiE0B48AKAQHUqiCIAZHjLE0C6ZVpCLgSqGMpIyFjoQ4tM46dzYZJ0pJAc6HdjMOntS/+83XRmduMgjbDW5HUqAxSkgJWiBACAvjLgQhZaJ+Nsh7gJGHvTFNcgLio1k5m/jS1g0dL/eaRPnB0c1Pr7/z6Ts/fvDpR0eHh4u6m2bn4nZnaXXT2larea20cPz4R/PpEyGllmqtH7ezBinZX+o2jCl9mRnRjpPBvBwWbr2dXmjFO6P5iYW2hkRKDmwXZY7c2dhlzXit3wJRWz9nYoHgiD0DseBn/RMAIrAQWiAQMQM2ktg57wMxcwAWjIy8eCRxVss0awzGT8aHdw537w3OZjO3P2d7Npr9xe//9/mD23cBLrz48420NzksIEcZxQe7N69dOn+u213qJ9fff+8bX/syuurR44PNq2vD00f9Tvy733r73/7qz/+jr73+3od3Xto+d2dn9PBsvNlJr9/fif7qvUub64+Gk29/753URLEQDIxKCCkbGDkfHBIDzIpSKoG0mE6EAI6ECgtgExNAICCtNRNRCACy0+kOJxPJ7IkYeTjLF+cBzIyARCSAA6N19hkSf0HJZ+DAtFBvGImBWQrQiARgpBIOAjuKTCCSQc7zAoCMhhAooKicVUvoN9ZRn9MUQAjlQgDEZyROkAIkAABMq3D76Pjm7U/uPzmo53UwptlZb/XOx3HLV+RBNhtpv98P9dGHn/zxD/7k25+9+87J0RgAev21v/O1b26//BXTXeu2Ly0n7e5SdzLn8dFhMR7ffS9+cvc7UUQvPL/05ZefOz11P/rg4WB0ohRqxE7LXNlYcQ+PpmW5sdRqgLPOpUJtd7VGGOXWOXYghcEUWZAbDU7LGlJErYEZSODCF9ETWwJC8ESBgDCwkIIBAWrrEEBL6YgIFrRwWJDdP7t9Y3R0MDn46E++/x1fV5GUzp2+/ovfePLB2/njj0anxzvzcuXCxWvXXvzken1y+HDr+Zd2nzy4fGlzls+XV78UxGdP9keba937j06ajfR05/F0PJhA+p13H6x2k9mcPrxzp9tL92fz02mptX77+qfLrSwWwhJI74USLvBwOGoYLRSQY0aw3jJzYNBaYe0EsJIYK1G6sPAO6jfaZ9OxFBhAYWBCKKpCIloiCSiE0BIpABKiAERIFVoP3mMkUUucWolCaIHWsw+eGBCAAJB9IDYCBFgGSmPZSs3ZpKhqG0UKQAgdS/YIIZIylsajVR7FcO73TqyJpaCwv1u++4MfRuJ4+/yVytlZOTweHT54sHvrww/uP7hj8+ki4Jb75y9f++Lzr7yx+fwr7ZX181vnJU8+uf57P/ru733204/tEEKAl974jVf+wa9vfvkltdQU7FpcbUIM82rvyY9v/PgvH916P7iT2bxkHdCUqxvraZa6s5lU4tHBkfW214na3eaF82vjqj6djdOG3j8ZVcyrfd1baY4HYwJKJMYKjBGZIUTBDK2EkTEQASIQBObAIhA7AkvgCRyRJ3YhOABaSF7PLqcREQQIYAJWAvfPyvuP9yfTe7fPxkLptqZoWl88f64jT3/y0XtTJxpJ+ZN3f/DFn/+lncFw97Obmxe2bz3Yv3xtZ3kjquzs7/39b9z7+K1OS7z13vXtl147Oa03N9dvfPy4IcPhfn1xI9sfjrPErDR07sAHYqXmZR0LERtYW2o83h95hCzWS+1kNqtYSxaMTFKABDZCJhKySFXWxYKKhU+fEF974xe+9b3vlc4zsyeWAuqqkkKGQI1IgUDmoBVqoXJLnqDVFFXNaLGViGZTZyVQsN1U116Bkt6L2bxkhl6anZVlZEysdJzXWsmarBUcXGASRivvaV4VkVRCSsmVFqAOJ27ueehdvjd9fP/w7rvv/vRHfzifnO3tHhU0/Lz29Trnnr/85d7ypbWNtc7SetbptVoNAOGCd9NHdz/4/s0bf3jn7vuzE4gcZBsvfO3X/803f+23+mtqWlpHZQwR1tHxwcHbH37rxsd/sHf3gc+hlcFaW3AwkMhGqgEIomBS4V213mk8/9LlJG1EUbx14cLMhl6vm49Ol2K+cG611enkk9OVZtBSCkXaKP1UcSAFKMRTCc0F9AHqigvHGEAG8AI1oWd0AiyxJHCBCSEwA/IzpjsCoic/rCpOo7nsRVI6olkZaqkvnlunMN2fulJGdVnCJ49Hw1E1G9zcn7x298HDqf0/b35wZaONj+/9k2/+07/5wXcTPT88mW5dtbq/ktrTlVj3Ezi/2Um0Wm5FLERkkGosS2cQGomkQJ2GaOoQa6wtJJoR6hBsmirHoq68VigRtHSdhvaeskQajbYMsZZG0l++8zeOvQTBAAqFkdiIhQDEgLGmOMbgIclEKzG1CyZSvY7xDpSO06Y0UtYlCEFEPq8gThpZnOwdniKwknG1U8QqVLVNkFyZOyQJQkiKjQ7s62CFosAlgCyslEKonQMOthqNhqO949nxYH7ypLd8vg7d5Uvr7azZanayzkqjv7G83p/VwY4+ElF0XMm+rh3sldXO3sGb1fgB83g25BRjq9y1N377v/y3/7y9+nlMGqrNwVn96eF8Nodrz3310uUvzCY7jx++fXjvo8cPHp6MqjhgI82e235uf/heu6nbptnUL7z28qsn40Gwfr2/ktfi6rlVO65Bn1y++npV2RNzI1XQSIJSOrCwNWoJWgkkCYDOibr2pWXn0TsgRAjEDByIiIEQiABQEqMQgVgsdPeF545YiGGskfd3DxLMttutu8NxQPGgrP7g298mV089BPKB8GCWT6picDY6cnR6eugBf/TZ8ds3jvrdR2+9+8lH907PrZjlZf3g8d3nr23pCl9cNU+m9lq0bDAkWhDZRsMU1mECRoh55SMVOs2M2PdakZv6bjPCQJGApaY8HNtmAgIhNhQZKHLGOKSpyafcNqKZCZOooubMCZJslDRSKIXdniCPQCptqFZTKwVSiVYSW+eHYwtsGHiW03hmgeVgPK8dkvfI2EgLgZCXTiA9f+XK8y8+38lSAVSWVseqmE7LchJlbQhuntdl5W3FKAgYBCqJAv/1b/2n+fRoOhlaOwUuojhpt9cLq71RsUnSZlrVUTXN++0ozpatH7WW1rvN9Tz/XhX+uKju2zKUY5gMgGpImjCfQHfzKyvbrzVby2AacdyKGt3+1trFjefj0Ds5mh3v7hbTYawkAI/HB8eDj0ZHN3356LVrV7744i8fTOZPHp/u33+43GtvXbz6/o2b3SiWKto7PvrSK1/48Q+/P3SjX/rKPzw6OHt06393olIJGg7pZE4csKkAEGrL3oP1UHu2AWrmQOAZLdGC3guEAEDIBAuSJNIzLs1CXhWLDh8YgbVUzOiCW1gShMUZLACgkABZYoTAL127nE+mbz/YWcni3FJtbaKgn5rNXnr54ooU+cnhAMCTSs9vrd67u39rt3r9WjfT4sn+vNESUuDjg8JEMpCYVdzvKAkhUjCpwBMut+OyCNYHEEGgWHTQzZaQmqfjICPOnXAVxw1otZSKlIqjel5pk0qol/utIg/tZeNtmM58FMda6bOTSVn7YOXJID84LdtxXFTOBkqNTJQ5ySsAWG9H3rokMXUV6kDAvLaxVNZ1IwKBYlZyM9FZHJ2NT9MoaUZxr9s+neYeoo3z52pvy9oSkbp3/X8pjdKY1MRJo5M12yhrjXVk+kaj9LWKK9febDSyNApz9YKtH5X1DYwmhq8CtiIZYkOtWMZGmljIWOVhcnT2p3ceTfcOp3MPk1OIBpC11195+Su/8NXfuHzti42scbq/NxtNSmvS6I2li18RYToc7f7pXz9Riey3o0YzqX2FGJBwOBo0GlmrkQJS0HUjznq95fk4pHG3IDc4zodnEDxGyBUsuiiqicvANaNjDrwwiONnu+nPn5d+ftDzt5VUgIUT5lMoLlhyArChRO6JfgZHAiPQM11Yav3mN//x4eDs8f7+Wna8OwnzMv7Sy8sXL13q98+By3d2Hp6N6uWV6PQk5GU9zcdElBnZbUfK+yyiTjc+PJx1WyqO1XRSraymJWGo/bnlrNidnV/PwIfBad5o6sjI1HBt7Vov66x2SNu7nw2lNr2m1GlYWmmaCIfHblaw6Ql0MJ37cV7WM5hVla1CMfWVHdc1lXkQWqVGj4cWSBgtQwCjfbcRg+VIgNIq0lIqBYABCJgc8dngbJLbXobdpVYrEbYa19Jsrfa2ty+fu/LyB2+9peQcRDUaH+Vn933weU7qn/32v9IuZU8KfRwpFSl0QQhpGQInIup6hho4jeJUWgdk8XXkV6i0eVVZmgNXkYziKCVDyFqCZOlZewzlqN7bx/ePpx9Obh88uX14/eYfXb/+R9vLL//cG//y9V/86ssvXWFHxaSociodNldfmg2PTo92P/zwx366085wc/PC2vLKzRuPlE7TrOGcD3VIlprdbn/aria5OD21ZSE1s5ZcMVaeaxcq4orBM8HTy1V4loz48/j6PLieRcrP1NSf6bAABNBpmIgRmVuxcSAEsAsQSUgQD/Lyxv7gYc7/4rf/w+Dwzs0f/3nSbu8+eNzp9hrd5WBhf29/NBwmKWAwp8ICIwcOQTRTlWXR8eFcJ6qYWUvcTYGQr13t+jx89Gh+eSvNUq2QkUPhKGvI1AhiSjN85epGlnT39+cPd4vC8UrHEKBzYvd+NZ2F2ZiH03qpY+bzSRAiVjl51AqLwrEQAcEERkIABCm1kcaHOEoEWiLotFJfgyisRCmlBqIkSQSCC0IpHUWy0fZNIzrd/sb6xsZGp521MokvfOHVJwenX359m8K5pNdYWe7NhxfiuOm5VkXr9+tp1zspQ6w40hBDpYAyEEpQgi4T2nAkA0S5BhZTsmOGCJQAuehzyYskdxNbzxhnnnxtI6wiU0fgdYIvp7zl1j5Z77/V3ahGd+HgyY27/+Pf/9l3WucvvHrp4iud5mq/3U9TXVmQyrz68uovfvmf/+Sv/+ftD79zfHJ+tbdtNLBz4OqTo91yPApAd258eLJ/lE/HoQYAnAUqLZXMjsER0SIVoQCARbl75ooDz45aFwS4xRvmp1c9CzDvosdCAEQMzP1W+gvPLT15NPSBWk0dGYwZIW5srV/caLev3/r03Xs75E0sVm6//8NHj6pml/YOjx8fDM5vVlnatKVdWu0prA739xwHoVUj1mUVmg32UE/L0kSxtdzrJEkcet2W0Lw/yBFpeSVpRmm3NSfieRXiVEspttbbFy+0GeJ3Px7cfzSoLGcJ1kU5GvPakhyNqLDkCAMLw6as6laEjcgEBVLAaO5QSCkkI3ly6DioULngmdJYaKltCJW3wXkviMgTeeucVgExZFmkUG9s9sAVzTj9pa9/4/yl9Q/f/e6dnXujfPxXH/7Z2WTGknLiVpYkUaPdpu3trtGxun77TZQQS6gD5AUksZZCg2uCT9O0KSoNAcoS6zyv7TzkdWUtVjLRopFAqiJjdCIzpQE4CqQszDEulQHhSNRaBrUxa6a2EYWvO1Fnm7Ned1Ln8/l4+Pj2m3d++qYPICSYCJCQHJCTL1y7tNZu17l954c/WF1am4zHJ/agsDydzcrauidHH3/8GYfgAuSElogYCZ6uogFRgPjc1xkBIymMxKdcVGB6alK/0LcAAIgXfqxAz6haACAQCUArNZp6RGlRjDyquhLMifWDya17zVar17q6vTw9vP833/2Tmzc/2rt/s7sUo6tMmvV7bSPEk/lpWyfTop4VLFBGmsraCXCr623CKtFcV7UAoQi3trqD0znklQNuN2F9rRGmPkt4MK0NSA60db754qvnPrlx8JN3H85rqghJYKepakeISIBlxc6DESyVSLQUAiIEjd4TljYorStLCMEiesRWrCKFCKgQp7NZbdG5aq6qYIW3wQAQibomANuOkm6sHfmXLq8uNzVKV8zufesPvn330U5N0NlITEqQwWwOMsFgy71heZLD4WgIFSgx7oEMee3rUAfwRexU7LwtqjEEAB+ALLg5xDkoo7O4uRx1o4TjJdCZKHw9C1URWAlgVdV5VNgaA3Bt0Tv2Ix2gqZOI4swnTJh2Q53R6KQKFfXXI1dTNfP5jOdT8J5VDJ21NmZyZ2fXzQuQ3s7LOIqG83pc2NI+NWVhYgGSQCggKSU9tXsmz+yYAAIgCgYAISRGghVyAACAwIv/TE9nRGBm4qcJbFEgF+IzMwLA0XA6m5eZicZFBczrndbVi2vrS91YmUj32iudu/rWeDa8+clbxfiYde1ZOxe6CMv97uH+LkA5GMyzTDUTY21Qjch7Wr+QrG5F08ncxKKuhAPo95Ni5gcn5dWL3clkvtpQrZZ6sHcKgKULaQNAY4XF9Rt3nuxZNoE5dCMZa1w/J4hQETKy0ZAYzFIDIJd6SjVFLxLe40cP8yzVoYJQU2mDWGTzSF3aWlrtO6kwEBORTloCUXjyRCaLJNgkTVTsteQs4/ZKx7nbufSnp3nlSS2JX3m50evgxmr83CtfuHv38fffO5At7Mt4cAheV/PcBifUr/3y74RyNptUeVUFdrWolK+quq7bpTA60uRlIfBI1pGVZZWN5oUrZpRXdR1KW5ajMfk5BAKBQB78HKgAciCDCIggg0/GJIBroAocA1YQcqgq8A6EAKNVf2P9hbXta1dfuPbqy89dfjVy7vf+6+/c+vTtZiOLpHGBpJKplCwCI2jJcWqApQSEQLVjRB8COZIzRwXBwrxcPEOwEQgHCEyBMQAwMDEuVocMgMhKiIWrCz2V3BdtGApkAuw1o6ZWGnWkxYVzWTN24/FplrTOv/Rcp78yHo129j4t7Y7OQsOJXqc5l1G33/XBDU5OprNaGFZKgVAVVqkQcSbWes0q1KWts248L7mY10HURWUlYuUsSr+0lFpvx7kDNO2muXBFvvZqZ+NSI1A1PvM2p4qKNNHoE5NAIEAMwVJgkKqWnLFARPcF20hUdHLmL31JtbJ0NsHhdHY6EYLi6cjnRRhMZs66dksoLRmp1VRrq2mvhYFs3FTF3DXaeP5ie+kc6caUqHJESgdyGpklilaGINGHsqjf29oOX5cwmEXE0eaWzO3szl1X11Id7D+qyuHMDYQoAtUIQrnY5nJONamZ4zqvfR0cBghQeeWoaouqLZFVLGJorZlmtAxAyIJDqHxv7MNJ6Qe1Hds5uBKohKICVwNaE8tOK11bWb24vL61sbW1sb2+fu78+vJGv9uDoPZ2dh/eeuv+Zx/m00MU6vA0X+3JVhxJIbQQSnGsFlqca7WTWCeuqsraBpJl7diHmJCEDMSBnqYoCQyIYQGXp+CJERe8GwIQAIyMQhA+XW8gAwiBAKgQERiEbEW8vQwCTADS8dlGR/U661H74tUXX1zq9w4O9xqjNGua2fg0S4XUjnVotduz8WRaHJd+EoJILeTBtjtJq5e0eqYqq6qwcSwFmNoVSkCrJaRSw0l1fDYj5EYXZ/NZkTsbaPuKOr8tVTorfK6MjLKQtY1gxUJQVUtDJBDIaWGYofQAbuac14luLiUy5LpHy074MFu/gpoJAWUE1qvhUN/8NN/bz6vcxxAB6em8knKGmpMGc4kiFeMZ55+NGkd86QW91PcYCtZKJ+LkqMqnzEStHsQxGsFGg06icmDPTuY6TpRMIik2zydqeelSMT3X41MUs7kbFzav/LBSQ1HPJbaMWOo04gY20HYivRqnbY0N732JAVGSQlbCI2MNEEJlXW5noSps5WqyMkgpsyzq9JpL/c5Kr7eyvLLWX1nq9pJGBjoBAcAB6pJG49mTnb333vzzB5/86WhwassqOExUbEtrFSEDOKcRtZRGqIiolarXfu41herGrc9ODwbeMzIE4b3jipkYny6dJSoECRwAA+DTd35PPaaejoaeCHERTigWu1FEo6VgMAoRaJJbwQzIxtmRRuB9VbqzwQNy6uRsEnRdW91QIko7NcFyr7m+ujQcDmqaBfCtZqvVbCkdZ+1GXsx2n0xs7ZMYkJUNkBqBiVzZSKIUnNeCkYTrLMuzQyuUzhqUNnlwlp+MKUrAE0DAcg5gEY1o9/zqBehkwBbqAFUBqCCN0VoWLBLOKYRxjnlOvWURKaTAoAGjwmSwkuLa+SQf90Qd6lCQ80pir62jiJUGJYOObLfTqq1/cmyPR0XcTttdM88rEMpKbvQ5NcIowQyOIJ/rW/fcg520qFbDmBpJP0mLsq7VX7zzH4VOMrWy1noefENFrZX0+WWjsRXpKAExd5SjnNoCZ+PDfL7LiISREJtZdr7Zj9Os1YuyLOqkiTFRaqJIGxOZKIrj2IAxoP6WXhSejfoSABmqmuczd3IyfHD305O9j48fvuVGRw0OMoLtjY6J0zQxKMRsUseJ0xKorpmJuDWb1O+9+Y6JImu9YK8COQqAKARrQhbgGD2TZqGVkAguEIIwDIHYAwfgZ3S2RadFizooEImBkJhZAgaCkynMctRCpImIWeqpc9Xc1fM6QFkDS3nhyvLmav/sbNhoiyzoZrsfxX4w3inqUqcKNB6ejbWSYUw7TwbOh1Rr9Ho8sUA+eDAxOVetLikWRCS0aljrKucvXNGgHBjb6REyjGcwGIFnjkhoKRuNYDqgEoAIFs5twoCOITIsBJiIdESTGQDyuU1op5TXOJuyJ5iehdX21ubGS53lbl4dTvC0wNt5VZdT8bh22vDalur2eO792WxoNKxe0R2HwGW72Vtfbc1nYw1RPfdGy0YrvXVrvruDn960n30CrW6bVWvweLS51ctM/PHtj9TZ6ExlxN2Brx+SiFq4jjLRQftag5iD0EauN9Pnl5a34yutTnu51eqmSTOOOrGJjRJKohaL6f7//YjYe654MXmJxaRPBLUN9byYzCfDo6OHt3/62a13733ywfBwJ1IglZrnzltSUkQ+OKuzxDSMVkHqWGEjVVqPz0Z+bIvKNxKnpAzIUaZ8Cbm3FsACOeKAQICByAbQUkgphGBgIEb31PB8wWpbDIxMAAs3C2BkxEABAdgDVAtfKjQFtlO0XhURGSVQAgEECv1m1O22J8UBqCpr9RsRPzm6fzY/ZApax8NxUVeu1TTdKM2nREai48nUBQJbB2QIAU9ObWfDmQzBC6rD8V5VF3r1MkVNjlLs9NFVoM5AZ6wM9DNqdThpojbCOvbEzRY2YiEkgiIhmBmlls76QNxIxNq5lKoaLSJfOLf51eXXX1k7d2V4erz/6CMyW198/bkbN3/3dPct8GQyv7Sp1i+LXhdnM6gsBOZZHpJYNIwwgqny+3u+tNHpAI/2PAl//T15dhS5ql2Nwnh/4twByc7KUms8z+vSqelhamXl4yrryWhlvWU2q/rCtat/d73fLxnanfNrnY1WGkX6/xM6T9/NMTxtap7tOxZpiQIEgsr62oW8qM7Gk+OTg5PBrlLV7OT0wcc/fXT/xuT4CdYVCuWDKotAHASikaKufW29bplmJpFlnRfTuU1a3baSMhK6nSbWViTqPNiidgh5TaXnmnhh9YPMEiEEmIcgkCSCQCGfejYt1s0MDAIXLHhmFJ6AmPhz5QIA8ekU5QnAwSwHJvQpJoaUFDFj2okhkncfPZiVo0ltWDRPhw92B6OzYV7OGKiyAQmF8XCUVzVL9Fww1RWYSIBU7KCoqHLgPAtS+ZgOTsone3CxD1mGrH3wNJkCsFzp6fNtWFpKpALrffB1WZFwIRLYiGQj0YxcQh0Wk4elQNJB6GSR4XYtN7e2f3W590az28RQOlfF3aUXlr5OwRfT6sWrv647JuX7jcbUmhoE5IVngExmc8tn0wnPZVFDxOHhjnv3fVucYSTig0dgixCxAHYgxy5IEgBKrG6kbIqjRw8asfm/BOfYpbTecBEAAAAASUVORK5CYII=\n" + }, + "metadata": {} + } + ], + "source": [ + "import requests\n", + "# image of a beaver\n", + "url = \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/beaver.png\"\n", + "image = Image.open(requests.get(url, stream=True).raw)\n", + "display(downscale_images(image))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3kmz4g1v6SJ_" + }, + "source": [ + "Search for the similar image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "qWf-G_Iz4RcD" + }, + "outputs": [], + "source": [ + "img_embedding = model.get_image_features(**processor([image], return_tensors=\"pt\", truncation=True).to(\"cuda\"))[0].detach().cpu().numpy()\n", + "scores, retrieved_examples = ds_with_embeddings.get_nearest_examples('image_embeddings', img_embedding, k=1)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "iFGNp5hp6VsV" + }, + "source": [ + "Display the most similar image to the beaver image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 197 + }, + "id": "Pq7IR86k54kP", + "outputId": "fa620b08-4435-4929-f67f-32b3f8f46b70" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "['Salmon swim upstream but they see a grizzly bear and are in shock. The bear has a smug look on his face when he sees the salmon.']\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAMgAAACOCAIAAAD7O9C4AACXyElEQVR4nL39V3Mk6ZnfDWdmeV+FQhWqgCqYggcajbbT0+M4wxkOJ8jlkivtrkStl/QZ9BEUoXhDIXOiE2pDio2N2Getdje4yyGXHHJsT087oBvem3Io733me/CbvJmNHq6eiDfizYOJHqCQlXnf133Z//W/5HK57Ha719bWrl692u/39/f3e72e2WyWJMnr9cbjcU3TJEna3d1ttVqyLPd6PUVR8vm80+ksFouvvvpqKBRqNBrPnj2TZdlkMkmSpGmaz+ebmpo6PT3N5XJWq1VRFEmSVFXlM5qmdTodn89nt9vT6XS32202m1artV6vz8zMJBIJm80mSVImk2k0GrIsV6vVYrGYTqfL5bKmaQ6HI5vNXr169Y033tjb2+O2kiTJsizLsqqqkiTxGPy3VCo1m02/31+pVOLxeL1el2U5Fos1Go2PPvpIluWZmRmbzbaysuL1ejVN4278w2SSK9XK+uaeZHJ2VFOnktl99mhyKhGJRPb29m7dusWyKIrS7/c9Hs/U1FS1Wj0/P+cmZrN5ZGTk9PSUj3U6HafT2W6379y5Y7VaNzc3JyYm3G63pmk8tqqqZrP58PAwnU6bTCZejRfh7RwOh9vtLpVK/FxRFK/XOzExwQck/dI0zWKxZDKZo6MjWZb7/b6iKIqi2O32RCLhdDolSWI9FUWJx+Mul2swGNhstsPDw2QyabPZTCbT8PBwu902m80+n29oaGgwGHDzi4sLh8MxNDR0dnaWy+XMZrOmaTyMoih8tfni4sLn85nN5mazabFYut1uv9/nEbmRpmkmk8nj8bRaLf5tMplsNls2mx0aGgoGg6qqDgaDfr+vqqrdbpckyW63T01Nmc1mj8dTr9ebzaamaYFAoFgs8rFgMBiPx4eGhlqt1sHBQavV6vV6vV7PYrGMj49ns9lyudzr9e7duxeLxXw+X6FQGAwGHo9nZGTEYrHIsjw7O/vw4cP5+fnx8fGtrS2bzdbv9/1+v9PpTKfTyLHJZBofH1cUZW9vb3h4OJPJxONxs9kci8WsVutPf/rTs7OzSCQSDAY1TVtZWQkEAoPBgKURG9nt9E4PT9ROW1V6mio5rMqt2y/lcrlUKnV8fDw8PDw1NVWr1dhms9lstVrtdrsQzX6/3263kTxJkhqNhsPh6PV6Vqs1nU7b7XaPx9Pr9YRMmEymer1eKpXYLf7LCrPyqqo2Gg0OJ4+qqqrYV3EfDpjZbBZLweuYTCaz2czT9no9RJ9v4U8sFgtPrihKq9VSFIXdHwwGLEs2m+33+8PDw/wvh0FRFLPZzOPxdeZSqcS/stns5OSk1Wq1Wq2apiG/yKAkScFgsF6vt9vtwWCgqqrL5drf349EIpIk9Xo9t9t9/fr1TCZTq9W63W6v11tfX6/VavV6vV6vHx8fu93uQCDAE7Tb7a2trddffz0SiZjN5uHh4aOjo3g8XqvVrly58vnnn7darZGREbvd/kd/9EeyLO/s7JjNZmSXB0P0V1ZW/vZv//brX/96PB4vlUputzsejzscDnbOZDJ5vV63212tVqenp2022/DwcCQSyWQyH3/8cT6ft1qtv//7v2+z2VqtltPptNvtRs3Hv3u93uH+Qa1aN8marLYVWVMHUncgRyKRWq0WCATi8fjMzEylUkkmk+12u91u7+3tjYyMeDyefD7Phl1cXHBmOKj9fr/b7bZarXQ6vbS0NBgM2GYhE6lUqt1uIwEcb7ZAURTuwGZjOlRV7Xa7HOlut3tJsLxebyQSyeVy7DffwtdxwtEUfBGXzWZzOBycEP7E6XQKQ5RKpbrd7vj4uNVqZYl4NvEPRByZNrdarUgk8uGHH3q93kePHjkcDp74/Pz82bNnFosFYTSbzePj40hbp9NJpVJ+v19RFIvF0mg06vV6o9FATJ1OZygUkmV5MBh4vd5EIsHLINFWq9Xv93/88ceFQuFrX/ua0+n0+XzBYNBms+3u7rpcrq9//eu8ZLvd/vzzzzudjvHpxULLsnz16tWf//zn2ESr1YrCR9zFdXh4WK/X0+m03+9//PhxJpOZmpr61re+NTk5yasJ2yGOoNjFo6OjYqWsKpI6kCXJrKmaqg2waCaTibcrlUooXa/Xm8vlPvjgg2g0+vbbb3s8nmq1yjP7/f5IJNLr9Ww2m9/vt9ls2AfOgPAT0EB2ux0limZCJjjhnGpJkqLRKHusKIrVau31eqenp9PT00J6eBdFUTBqvV6PFzSZTCi2Wq1WLpddLhcKDClRVbXX6zmdTiGgFovF6XQKXdVut6PRKIfEKFXi8zzeYDAwl8tl1Gyr1frHf/zH7e3thYUFt9uNsuVA9Pt99AQW0OFwvPHGG6qqdjqdx48fF4tFRVGCwaDL5ZqcnHS73bIsYyyECmUJVFVVVbVUKsVisd/6rd/60z/909dee61cLsuybLVaz8/PNU177733OHk8gMPh4GAZ1TsHjoO7urp6fn7+J3/yJ6+99tr8/Hy73S4UCmazuVqtttvter3+6NEjWZbdbvfZ2dnIyMi3v/3tcDgslt6oMITawCnZ398vl8uyLKmSpkqarCmaJimyhGx5PJ5oNPqDH/zg9u3bKysr0WjUbrcnk0mXy1Uul3/0ox+9+uqriUSCvXE4HBaLpVKpWCwWvL2Tk5NIJMKC8BlZllutFs4AC763t4cXwcdQG2iaQCDg8/nOz8/T6XQ2mz06OioWixsbG9/73vcGgwEvhSHq9Xrtdtv4guiCWq3Wbrc9Hg8/F++OfRRSaLfbbTabLMvZbLZSqYyOjrrdbhQndhajiRCL+2iaZt7f37969WqlUuF4/ft//+9rtVq/3+dECvHCAff7/fF4/I//+I/HxsYGg8Hnn3/e6/V+7/d+LxgMirOuqurJycn5+bnD4eA7OIs4ChaLZXV1dWhoSJIkr9eLXgyFQicnJ9ls9t133xUriK+2srKyvr7O2RVPj3iJU764uFir1f7u7/7uD//wDx0Ox8OHD00m09WrVx0OR6PRCAaDz549SyQS3/zmN+fn5xEmo1GQdLeX/yqK0u12d3d3K5WKyWRSB6o0UM2yLClSv99XNdVut9fr9e3tbZ/Pd/fuXUmSRkZGbDbb6enp6ekpvuD169c//fTTK1eu3L592+Vy8Tr9fh/91O/3m82mz+cbDAZ2u73X6yWTSRx2n8/3xhtv2Gw2VVUnJiZw0q1WK/GNxWLx+/14rufn57lczmazLSwshMNht9v9wQcf/PVf//U777zj9XqFeNXrdRyDL13GbrfT6eBMY/2F88TKsKScbYvF4vF4FEUpFArFYjEYDPLMRodBhEr8UNhH89jYmKZp9+/fn5mZiUajtVoNCUPerVarzWbDaUBCzWbztWvXcrmc1+tdWVnBOAqlomlatVq9uLjAlzKZTJwwWZZdLtfIyEggEOh2u6enp2tra5999lkwGPzwww9xxr/73e9GIhHje6qqismvVquIpnBF0e28Q6VSGRsb+853vpNMJq9du/b2228XCoVGo/H06dPt7e1erzcYDF5++eX5+XlUPUJ5yfYJg9jr9ba2tsrlst1uF06xqqqSpuGCJJPJi4uLRCIRDocJBcxm89HREZEy6rnZbJpMpvfff79cLkcikUKhcOXKlaGhoVqtFg6Hm82my+Wy2+3lcvnRo0dnZ2dElDMzM4qi3L9//8aNG16v1263s5GDwQCPR5Kks7OztbW1arUaCAQikQhyeXBw0O/38fz+7M/+bGpq6vbt28FgsN1u448LXVKtVlOp1OzsrKZphN79fr/f71utVnxKRA1ZNJvNTqezUqnkcjkMNLpKnEBxSsW+4JBomiajRc/Ozubn50dHR9PpdKfTwRKRDnA6nYiO2Axe8he/+MXc3Fw0GkVukN9ms/mDH/xgeXnZ5XKNjo76fD4iPq/Xi2hms9nt7W2LxRIKhYLBYKVS+fDDD1dWVmq1mslkSiQSRHbGzSYOwpgiSYRO3W63Wq1mMhlizM3NzZmZmb29vePjY+KseDy+t7d37dq1RCKxt7dXKpXefPNNEgr9fl8yxOccDP633+/ncrlisdhutxFfXhyx3t7ebrfbN2/eNJvNNpttfHzc6XRmMplCodBqtcrlcqVSabfbDofj1q1bo6OjCGipVPr888+/+93vfvLJJ9Fo1O12m83mi4uLp0+fkizgyLExh4eHn3322dTUFOY1Go0qipJMJtPpdDKZtFqtk5OTpVLJYrFwaIkWMaZut7vb7SaTScyL3+//+c9/Pj09TY7D4XBEo9GhoSG73b69ve31egm2YrGY3++XZbnZbCIARLJjY2OBQOD09FTTtFgs5nA4ROzZ6/Xq9Tr2UZZls9nscrmQE4ybjHwZA5BSqURsdekoCxcPZdhoNPA0McZCyaXTacL4WCwmjJpQsyghSQ+Ojo+Pi8XilStXyuVys9kcDAalUikYDBKxC0VoMpkmJyc5FuJhJEmq1+upVIr0AfYxnU4PBoPx8XFcwGKx6HA4CDJOTk4+++yzcDh848YNv98v9LYx/WO8BoNBq9VCaPhMu90Wxz0YDE5NTUmSVCgU3n///Vqtls1mFUVZWVnhV2NjYwSDhNJffPEFtnV6ehp/yOl0JhIJgllSU8ilpmn1ej2ZTPIA/MrhcNy5c2dyctLr9UqS9ODBA7QRKsRqtSJesizbbDbihnw+v7W1tb6+Pjc398orr+AUer1e3NPNzc1EInF8fJxKpXhmHNNMJtNut7vd7mAwiMViKAtOY61Ww5fvdDqE/JhLXCa73U7YF41GnU7nL8MBHI7BYLCxsVEulyVJwsRi14nyxL6y2Z1Op9PpSHrMgstps9kQ006ng/dNGNVoNPDkONNWqxVN63Q6OXbYWXEQ+/1+p9NptVq1Wk1V1XfeeQe3XTKkA4yZm1918VI8WKVSefr0abPZdLvdiUSCI4hGrFarhUKBc49S5PHOz88zmQwK32q1BgIBTPzY2Fir1To+PnY6nQ6HIxAIWCyWTqfT6/U6nU48HscHT6VSyWQSdfvZZ59VKpVf//VfX19fj8ViIyMj7XY7FAotLS09ffqUoy9MDEkfbLc4cvF4nAPj9Xrr9Xqr1eIh3W633W7nD8lLOZ3OZrP50UcfxWKxZrN58+bNaDTKjnCWnjx5srq6enZ21mw2FxYWsJjFYjGXyzWbTVx4NKjP52u325VKBS8e10KoDGJVi8Xicrn44cjIiNvtlsmpmM1mcpvixbAXvV6v3+8T17y4YV+5qcIfJJAkSBQGBTOP0hYa6P/NZTRewk9sNpvNZrPVajUaDaSn0+mQ5ep2u7VabWxs7MaNG+KlxEVKczAYIFK5XA7HAjXO+SPJQp6wWCxi009PT71eL8GK2Wze2dm5cuXKu+++K0kS74sVttlsWI3Dw8N8Pm+z2fjGv/iLv7BYLL/xG79x5cqVbDbLOgSDwZOTE5LgHE7hEpHXLZfLTqfTYrEMDw8Xi8W9vb1f+7VfGxkZefG9jFej0Tg6OpIkifTyzZs3OcPc/MGDBy+//PKjR4/Gx8dHRkawJ4VCoVwut1otEZjbbLahoaFcLoesDwYDDgm7IJ6T1NdgMDCbzV9qrIODg3Q6bbFYEHnuhQZCu0q6hyGEXcRTrVZLVGnYbLImYksURREZFFW/iOyEnyjEVIi1EE1+KJK5l94HwWIjCWOtVqvFYrHZbCJ97HK5PB6Pqqqnp6elUglFiJgqijIyMhKJRGw2G+rz/yrcz549297evnXrViwW4zkrlcru7u7R0VGj0fg3/+bfiEBYPLzJZCJGdjqdrVZre3v79u3by8vLwn/gkUgN8uLVarVcLne7XZ/P5/V6T05OLBaLxWIxmUz4YehyY4qcbRKpBBFm8RlJknZ2djY3N1966SWv18vHHjx4cPv27fv371+/ft1isTx58mR5eblSqdRqNZQCmzs2NiZJUi6Xs1gsKCdND2IsFotQbIFAgFcgDPf7/V/KF0LKNqMwJT3DK+vFCuPhMGbGer0eOkPImdDhmGTEGQfTYrGwi3a7XbicWDRh5hA78mdoApEYFAKHn+d2u4WkSpKElsb5Q9wJglRVrVarIsjFrWw2m3t7e/v7++Fw2Ol0BgKBaDTKHwrdU61Wq9WqpmmVSqVUKp2enkYikZ2dnfX1dVaGxTWZTBMTE+T8RK1CPJVI/Pj9/n/1r/6VeGbhAtvtdnwDpCEUCvn9fh6y3W6Pj4+zsE6nU+gzjoGImbCPuCXC7nMmkdrh4eFXX321Xq9zAtmmZrMpy7LX6yV/RtDWbDY7nc7+/v78/HwsFgsGg+fn55LBQPFeNpvN6/WiZd1ut8/nQ4nIstxut1utlvxiyP3/48V7kpdDOHgmVod1wRep1WrNZpNYF2lDfLHlSI9QThw+Va/Uip+LAJgvknSjKVIy4gOoN6Rc0tUw4tjv9x0ORygUwv1sNpu8C1YbNdxut0kqUjlgHd1uN/p+YWGBlEqn0yGXjaeI/0EsgiMvpEq4vS6XCwHVRHbRbBZGgxjNZrO5XK5f5X6Iy2gc+DA5s1arde/ePZPJlEwmy+XyG2+80W63JUnqdDqrq6v1ep14QlXVcDi8trY2NTX1B3/wB6zk8fFxvV5HC0h6hOd2u8llSpJEslMIer/fN4u6tPR8bVzYLG6Elpb1crqk+xOsmji7fAHBJ/ZFJIsPDw8vLi46nQ5BaaPRyGaz+XyefI9IsnFzn89HUs3pdAqHT9aBFZLB0xLOO4ZJWGTjo4p41mhbEVkWDgk25vrICprNZofDYbfbUbQcFVQsr4Z2XFxc7PV6JESKxSLQDHFD9Eev16tUKviyZIwsFgtqplqtHhwc9Hq9UCik6TkzxIiTxreToeB95edri+ISiV++UVXVYrFYr9fHxsbC4fDo6GilUonFYrFYrNVqeTyebrfrcrnILyiKMj8/XywWY7HY2dlZsVjkFYTQUHf5MqWnC7EQNdmAB+EFZWMKR1wkzdgntAs6g0sy+D0vhuucVExysVjc39+nQoyKwj+QJMlut9vtdrwicci4Q6PRoG7q8XgcDkckEvH7/SMjIz6f7585rP//uRDHTqeDueHhJUlCJW9tbc3MzAQCgRf/MJvNFgoFu91ONCM0usVi2dvbS6VS1CHIxQeDQVkvanW7XU3TgFGwtfLz1SdjpEwIifrHzmLv+v1+q9WiIKsoit1ux/lhoweGq1qtgjAYHh7++te/3ul00ul0Pp93u92hUCifz6NZnU4nUBHhpUiGoy6LuJ1fIGRGH1mSpO3t7ePjY8wEZ4iMMAZFFDUlXQ/z82KxuLa2dnBwMDw8PDEx0e/36/U6x9RsNqvPVxLEJZIxkiTh8+E2aZqWz+fv3r07Nzdn/LwxQhHvZjwn4i0kSWq32+Vy2Ww2Y9RQ6cLzxUA7nU6xUsJ7Y03ARzQajU6nQy6AOJqwmg9j7EDOSJIkoC/YEQTFYrF89tlnyWSSNDJun/CLMejhcFisAyec7Rcur2ZI2/It4lfGMIsblstlBEvooY2NjUQiMTIyYrVaw+Gwz+fDbrDaOAZ//dd//c1vfhONgLqSZfnk5AQ/weFwTE1N8VdCsFjwL0Wi2+22221FUVAel04nfrTAsvE+whmSdBvEz0lB7ezsrK2tNZvNUCjEAjUaDUm3xMZwjz83CiXOPo6F8Mf5eblc3t/fn5ycjMfjaPtwOByPx8XTvpiPMAouDylsHxuAJTo9PUWtDgaDoaGhxcXFYDDodrsRBWwiwmGz2dA05XJ5c3MTSBn1JWpwnU5HlNvFqcBSqKpKxULWQQGKonS7Xayqx+Nh51qt1mAwaDab2FPwdJLuvnDwRMSHlPArUTzlA5oBpyUWX1EUtGav1+N98/l8tVolhxwMBr1eL2UPq9X6X//rf71z585LL71EBMoX1et1Ug+EfkKUhVTxdoqiyBcXF//0T//k8/ko3RAfhUIhoTxeNHaXJE/VS8J7e3vPnj1rNptOp3NoaAg4h/CgjeZZaAVhB/kWkgUkl/G1haImx1Mqlcg2+f3+bDY7PDw8NzdntVpJUX7lQ754tVqts7OzdDpN5mYwGFC5UxSFUqmmaV6v97vf/a7wwSXdCLJwvV6PbOfIyMj4+Pjw8DAoDGG/JENUKPZY0kM2BK5cLh8eHrINoVAoEol4PB5JkiqVCk5PpVJxuVyAIIwH5jmLY1DJKCrASx6Px+fzCRXIB0wmU7PZLJfLfr8fv0LTtGazWSwWk8kkGVqHw2EymYaGhn7xi1+8+eab8/PzwmNDuwsv6JIYCDXx5eOhQkQoh0flcrnwk8R+o2yazSaOrRBVROr09PTv/u7vAoHA9PS01Wrl4F4KC2QdNCzr8bx4DmF0iKfQDeh/9lXknyTdvvDYFxcXYFE0TZucnJyenkZrGqMKwiJklEUHKUQZVMBCMBlEhRaL5eTkxOl0fuMb36AGwrOpOjQK1Z7NZs/PzzGv4XA4FouRM1MNxVPhWhirFKCSJEl69uwZ64CmcblcgUAATSCCp0uGRjLYfdUAKDLucbFYTKVSdrudJLjRSvIZCl/RaFRkNTVNowpUq9U4wycnJ+12OxwOT01NgV3jPAjbJcT6kv35UrC63a6k12R+lWbiwo8joyjCxmw2++jRo8ePHweDwTt37gjp1vRcgFB4IkaT9LjMKKAiuSI+b1TpxngHf4JHErZ7MBhkMplOpwMcxel0drtd6m4sEzdH51On4ufGFxTKhlc7OzvrdrvRaPS1117zeDxIAL/NZDJerxfNSvE/n8+LINzr9ZI5FEtaqVSomiPB1Wq12+3Ozs7yLZqedWPjXS7X8PAw+kbVQURGzSRJEpZXCKjIM8s6oJIvLZVKkUjEmErllc1mc6FQKBQKo6OjLpcLfwNNXKvVSqWS1WodGhrqdDq7u7unp6ck6paXl/1+P1ssnkect8uCtb6+Di5bZBAQC5S2UBIYHY/Ho+nBp9VqXV9f/9nPfjY6Ojo9PY2aNZ4kbiI9H/xLz5sGY0ZAxKFGmyKUnPhb401kQ3IVt6/f7wP0I+hFBeIhGYVM6B7p+aqIMF6aplHBoMTxzjvvDA8Po7oURdna2qpUKkNDQ8PDw4FAwGw2I8fpdBogDcgZTdPwLqgvyYaaQblcps0E50aWZVD/ArDgdruHh4fJXRkFi7Vqt9sbGxvdbpc+AKCLxrBf0q3Vi841zwBy+PT01OPxhMNhFhNbD76jWq36/f7x8XFZls/Ozp49e5ZOp6PR6JUrVwCRYgdEfl++dKmGJKEw/8KOtlot1IDFYgERzwtYLJbDw8P/+B//4+uvv766uop7btwezZA9kg2dM+L9+Ywx7EKwFEMvg3H7xXoZTafQxsZDY0Qbi+MhfFv1+YQcqlrYCPGQQqBlvYI+Nze3sLCAq47xLRaLjUaD5Jbb7fb7/RaLpV6vV6tVHDiPx8Pu8u34KKwzjny5XM5ms2azeXZ2dmxsTFXVUqlEjoCkHbKL7yxCMPHkvV4vl8vRPBIKhcbGxuTnm5QuOWHGZRR+WzqdfvjwoaIoKysrqDd0c71exwiEw+FQKGS1Wsvl8vb2NpD08fHx8fFxo7tCFCKM9S9XUHr+ajQaFxcXIyMjAh2l6WEamKEf//jHoVCI0ymMCEIjtDqbagxP5K8q4Gh6YtDoMQixM66O8VeqoTVK3ERYT3FyEC9EVpwZYaTEAxhvpejAL1n+0lUgjyDL8vT09OTkpDAovGOj0SApDyxpaGioWCyenJx0u12yrMJbJfOp6lVO6qqVSgWXaGlpaXh4WFVVAOlEiJSBPR4POX0SGRgjsZG1Wg0I/PDwMIJodH0uXUatz/1//vOf//CHP7xx40YgEJiZmaESylG/5K45HI5WqwWqp9PpzM7OYrJRt5IkkVuQhVIRCpMeG03T2u32+fk5qMjx8fFoNOpwOMjWP3r06H//7/+9srKysrJCgenSURDqYaDDC/t6S5mshwKX3lZIicgvGO9plDDxeaHqxU1YDrfb7Xa7a7Vaq9Uyfq+mg/VkA9LrkiQZXW/js2EZASnY7fY7d+5gvH7pUsgyHgX+kyzLjUaDdaCsy/lEvp1OpxAvAQIuFAoXFxcej2d5eZlegXq9Tr6euNXlciWTSUmSJiYmwJMJX0XSI75UKhUOhzVNAzzyq9ZZvLiml8s2NjZOT0+Btdnt9rm5OerKsizXarWzszNZlkEIooZlWUazut1uIi0CLMTLbrd/mUNDpRNAiR4vRUdnO53OXq8XjUbHx8cfP3784x//eHl52efzcZqRA5w4ce4VQ2eSSHppetxnlAbjGTKbzWSwkHrhsxvTXbyDkAaj/WIpCdElSaIEhqyIfKysQ240A1hDM/ihmt6pgeUSR5/7eDyei4sLTdNu3rwJcMUou4pe70LfFwoFElTcVuTEBzqAyev1mkwmrCeLls/ni8Xi6OgoaRSKRaVSiToxYUE2mzWZTIlEAmy7WEPhoR4fH3c6ndHRUQB64gOtVgu0nPp8yVWWZRxEQPdUu71eL1lx1E0ymaSbazAYuN1unD9FUfCUUNVC5Q8GA9m4xKxdV7+wnYqiRCIRHDq32/2f/tN/olSOqy78WSG2Rm/RuHli6UUsJhuCVUVPoBv9fSGLL+oVSXdZjEZW+HNEfyYdGq8oCnlgASEUWtCorcWX8r/iDIjUriRJ5GLy+Xy5XJ6amlpaWiI2NGoCIWH9fl9E7+hXnFF8WQylz+cjv1CpVCqVCn8I0HliYmJ8fFySJHB21WqV/QbSfXJy4nK55ufnkVRxYKiXkywYGhoivU4MDsqSCMzn8xk1t6Zp6XS63W6fnZ05nc5IJIIq9Xq9Q0ND7EsqlWo0GiAcJUkaHR0VuRKq9ZhF9kUWfqtxXS7pT5G9+PnPf37v3r07d+4IRcoF5ENslWRIsSANIu/AqRIAds0Q3BndMkXvKzKKJk9sdI9kQxAkGbx7WZZBxhl/KG5OQk68gigGCKEXqxGNRk0mU61WEwVTo4YWOLZbt265XC78BBFYGN+LFeh2u+TJyPO1221wB2az2e/304ddLper1SotCajG+fn5oaEhTAdpCzbV6/VS2p+amiKJKlZMUZRWq3V0dEQrTiQSQXUpitJsNnGPkBgB/ZN0r1pRlPPzc5PJFI/HSUY6HI7h4WEqAZlMJp/PT05OArGcmpqS9Z48IjzOj8lkei7BdckwiYtVPj4+/qu/+qvx8XERnap66zdKBfslpNhod8StLqU9JIOlF7sr5JKqhaa3WUqGUhSJKCFY4s+FGFEvNx4SWQ8nqe4JfcxvhcoRTidqGGIFCsZGtcQNrVZrsVgsl8vf/va3nU5nrVbj8+IzqqriJzkcjlqtlsvlxsbGRkdH0WeDwYCQs9lsKori9/u9Xi9QfVz1er1eLpeDwWAikaAJDL+ettJYLNbpdJ49e+Z0OhcXF2VD5VdRFL6u2Wyqqjo8PDwyMiJMxNHRUalU8vv9Q0NDqBkWWfiINFCMj48jW2azORgM0siazWaz2ez8/DzN3HNzcyKQFG9tMplkCuxfKU9CznjKTz755P79+6+88oosy6hfXDHMtjDw0vN2QVgr9fkKNyUjYbzQYWgFI75HyKXQXvyhRe/PNmo7o2ChQYV9NEqwqqpoIJE5G+gsFYqO3xJuPikxDKh4C5DH2Fag4qVSKRqN+v3+mZkZMl5Codbr9UKhwHt1Op2zszObzba8vDw0NIT7T2xPPcBms5EYwzJyYilkTU5OAq2hLFOtVi0WSzgc9nq9R0dHFxcXi4uLeF2ynibtdrvlcrlQKHS7Xa/XOzY2RgLdYrEAxvd4PPhSQrNKkgRSb39/X1VVurBYnOHhYaAZqVQqm80uLy9DKbC8vMyCGB0e+aOPPhoeHiZHJRQJeyD2AxX653/+591ud3l5mdKEUBICfSVWXzIgakh9CYGT9MjIrFNTSJJEZRf9J3SheMpLSl4IE99rMjSyCukxWlLjgRGurmjtkp9P/BitqqbzGsh63kvTS28iINU0jYRTp9Ohk3tubu7WrVs0GAr/FTwFysNsNheLRbrW6KpFd7ZaLXwsl8tFhqLValWrVUxMv99Pp9PDw8Ozs7P4joVCgeqe3+8Ph8Plcnlvb298fDwUCkkGb5Xem1wuV6/XnU7n6Oio3W4neqBYSS9GIBBwOBzGDH6v19ve3lYUJRgM4lRJkkQHtizLe3t7rVbrypUrzWZzY2NjeXkZoOwvMzWtVgvwAiGuoveFCnUiZOsv//IvoRkCTyh8CCFYkiHPK/SK+LdRtmRDVlO4ICJJKxuST/xKZOFFskDE7UKwREQm6W6WgDFqLzhhuFnoeSHEAwOSR2hBkbUXdsR46kShAuObzWZzuZzD4ZiYmLh+/brP5xPBtbB6YpPS6XS1Wp2dnZ2ZmUFAO53O6elpt9udnp4W8KxarVapVEgK5PP5fr8/Pz/PnUFOQ2oCrvri4oI2Wh6SrkZwrdRwzGbz6OgoFDdkz4+Pj4FXuFwuYNNC18qyvLOz43Q6vV5vsVjkfSk2dLvd7e1tu92+sLAA6m5lZYWF+tIPNi66rCN4ZEP9XKzj//pf/8vn8wUCAT4gvGzwbgJ1I6yP+HP1+eK8uIxeuapzIQmdJ6xeX28W0gwVHgFCIoAXmkzcVoiaENNLstXpdOr1+kBvKNIM8aB4Tr7CYuDAEIKF0RSQGCSMFB0tQ5hF2tjD4bCkO1X9fh/PnU/u7u6aTKbr16+PjIxwLNPp9PHxcSAQmJ+fB4oOvEfwTFUqFVprJEnqdrulUqlarSIxQGKExeh2uxcXF4FAAHVSKBRyuZymaZFIBEOMS3N4eKhp2tjYGPV44x4pirK+vj4+Pu5wOOr1OkHu6OgoWZKdnZ1IJDIxMVEoFA4ODlZXV4VFkhuNRrFYpKGbjZReuJCMP/3TP/V4PCg2WU8cKIoi/pCEjdGnvmR3NUOPhvghcsCain+I32JeAW2qhryLkHisldvtxuYqz5e6xWMI9Iv4LfIkWjZEoVp73tcU9xTlJnFUNB3TLBliZCHTdHVCEWCz2UZHRycmJow2FPEaDAanp6d7e3ujo6O3bt1iaweDwc7ODo38ExMTAppBiyVPG4lEXC4Xj1StVsFX4XIJ1w3pPzs7w0knkshkMvw5cSgH+OTkpNVqzc3NyYaao7CJOzs78/PzVqsVBLbf7+cUnZ6eJpPJxcXFQCBwdHRULpevX7/+JVVEpVJZW1tTFEV004IUxdCIoN1qtX7wwQepVIpIQQiWLMs47wKRLayVZKjyil0R+yTpDrL4t/b8JYSAvxJ5NePGixS/oP75Ss9J1S9+InIfqBwhdiLXcOlcCYuvGHAsfFg1IDiEEdF0VIKip/IBGoBBnZmZmZyc1PR8Jpq4Vqs9fvy4Vqu98sor4XAYAGC1Wn3y5InJZELgCDjoDCNMQ2FrenSFNhoaGhoaGiJ3LXyJk5OTUChEBqvRaFAERG/xInBSZDKZ5eVlHHzFkNNGOS0vLwtHXNNxA2tra263G0rEjY0NGIf6/f5zplAk7mhBFl6OJEl+v39jY2N7e3t2dpbY2KTT0ZC9EBkB8Q+jThJbIpZeaFqhPyRDytSo6oQ3g4IREZwQIP7tcrmIh1VDZk58kfp8DUcoHkl3rfp6Z5UxOSIuRe+yFMeDANaoXyW9nIX5Axdv9PlqtdrQ0ND5+fk3v/nNoaEhzeC3sW6ZTGZzczMSiUDIxqbu7+9vbW3dvn2brDfeYafTIXjUDAnIVqtFAAjfmFhY9Nbx8XEkEiEbAkcDZFd+v59nsFqt+Xz+8PBwYWGBtJwIFcEcVCqV1dVVo4Y2m82pVCqdTl+9epVVevz4MQXEX26M8HheXFYE6MGDB0+fPqWtUdLzCEarjFUSqksIimTwWsSvxL/FVxu/TjZcQgj4x0BvfhTwZXF/ESdeUo2SIeITEYNmCE36equWZuALuaS6THojrnDXhMITn8cC4mKHQiGzzgkg6Tlxr9eLPnjppZfQCkK8VFUlRf706dNarXbt2jUKzzabrVAofPTRR7du3YpGo0LXGj1IIUCNRqNQKHQ6HeA04lcEkgcHB+Pj4/SitVotMqUjIyOhUIi3tlgslUrl2bNnS0tLfr9fYPV4eMpKxkwk3w7/JesPL9qNGzd+yZQnzpy4jFZJ07SzszMYR4UrJqRBJK/7ehehpOsM41k368yA4ieyIeFkFC/jphrlXtE7fTHcRtUoGfqf+ArxtyJ2E+8ivvcrn0F74RKvKVAiIo1n3NdisXh4eMgPnz17RqFDhMmqqtZqtf39fdngg4qXNekFsWvXrs3NzT18+PD4+BgLODQ0RDOPyUALdWmheGxQXHa7vdFopNNpcUhQDaOjo0dHRzy5zWaLRCJWq5WaJivZ6/V8Pt/KygrCLRKcsl4oUwz4cvHtLpdL1knbIIk9Pz//ZUOz2DnhJImyg6qqoOhjsZhATYmF1gzlT4JqkY43bp5RdF4UYqM0Gx1wsWpGWUQzCWUpzr1Z7w8WG3bJu5L0xISIGSU9TS90rTE5Im6i6fBG4VoRfMmGsAO6tjfeeOPll1+emJi4ffu2pmlbW1sCxIbtwP+99CRiiVAtIyMjr7766sXFxcOHD8Xrb2xsAN3RXvACxTUYDFwuVzgcho3y4uJC5JZUVcVEnp2dEedCNYVslUolk85m7XQ6V1dXnz59Kkg6xZv+qu8V6rPf7ycSiW63KxP9smTCbwBeoxqAEIuLiz/5yU+ojNKqxuuFw+FWq0XwLPQKoZD8fG+Z0dgphhzsV35ArKb0VWRDQq/QOse3S7q1EjdHROQXzLG4w6XDN9A76zUdDy0KDMIASXrEYFQ2RCF4uN/61re63W42myX18ODBgxs3bmBAebBAIHB8fDw/P3/lyhWsofyC+6HqKbSjo6Pt7e1r1675/f6trS2PxzM7OzswcJ39qm1ut9v5fB6uVPjZBIbs888/HxsbEyxLgA0HgwE0kKpeWshkMufn57du3QIZ9eJDXtoscfw4luZUKtXX2YiNnoesez8mk6ndbkM9Ck20pJ829k/Ra7eyXkkAmWQ28DOLfRW6SuyucPklPYrRnnf4vvKg8BnalIFe8WHRtC10m6CHkAx5VCHWwhUVBksyAE0lgzoU2yn8m4GB3g5LQcUmlUqlUqnV1VXw9ZVKhVKMLMtWq9XpdF67du2f/umf+v3+tWvXCM4v7RDP2e/3JycnXS4X0PUbN26QcHc4HBSPf9U2q6pqs9mCwSAsS3QqEFKoqjo2Nra5uTk6Oirr7MP9fr9UKgn2GyKPaDR6fn6+vr5+9epVVOxXSrD0QhzN0pnBxPGVl5xuse40Yh8cHLzyyivk4sSuX1xcGBPTksE7ptovwJOXclciByF+KMTUKExGSyS9YEP5oRhQAMxcvKGAt4tqtPCZjPJkdJIkPaQwhrGqIe7TdKCseHJZR9zX63W/32+328Ph8OHhIXo0Eonw7pzeXq8HWO/GjRuffvppNBqlom98tWKxuLu76/f7mS0QDofD4bCAJ0HuDdferxIsIeiVSmVvb+/GjRvJZDIWi0GIEo1GqWYKsQgEApAAXFxc0IEnSVKv11taWsrn88a0lrj+eZUpy7JMpsSscwQIKyB2iO0xm833799fWFgYGBqFZVnmwF1yJ41tIQAvyY+b9I4GsSXGBzXKivE4qs/nriSDgpGe1z2EioPnMfUmQ9PpJfGSn098XFo+vEzGJlxyATWd2l/TUWIcpHQ6DUN9Pp8HySnpSkh0WfJ4drv92bNnkUjke9/7ntEgqjoPzCeffNJqteCRJ0eAgNbr9YODA0orYIj/GZerVqvt7u6Oj49XKhWr1SqYtC/9lSzLYGYwjmNjYwK2adG5iv6vgiW26Uvdsbu722634Y5SVZXWNoqgkh4km0wmv99/fHz84Ycfzs7OwtnCXaC3M5q5gd66LjbA7/fTzyk9n6nSDCUasd/K87hQoyvDB6Tnda/2fEGG6rJsKF0LfSlce/5QCLpkMNNCaamq2mg0KM4IqRLG0fgivCaLUK1WQc4ADRUL2O/3Dw8PQTVBUcRtT05Ovv/974tSmPFNcdWhj19aWiITIUmSxWLJ5XKFQgEybeow6vN4HuPSmUymTqdTLpepxtC9N9Ap+YwLBTWyLMvUCcQdvtK70gxtB9LzCE2WyDw7Oys+fXx8DK8rHQE+n48Ikz+As4sHFfcShkPsuvA/hPslGKEvVYWFI/Wi6/OiSEkGTWN8Pe2rshtGaRPnaWBochRLI37CrehKknVcnqpjs/gT2RADipcVi9vv9/F+dnd3FUUB52Qymex2++eff+52uzudzvHxMSphaGiIzrB+v2+knRby7fP56vU6UIKjoyM4UYirAoEArB7kMycnJ43nVryyOOdWqxVa13w+n06nAc9c+pgkSX6/n8R4q9XK5XLULrXnnRDj/176lXCfvnyFbrfLXAnyJS6XCxKtUqmUy+VkQ/Hu4uKC7LZxt/iHYijaSIaADimu1+v5fJ4DQcAs4jXFwO1mVGNG02OUJ0TQZLjMz1+CAVUYF+V58IyoOrOISA+LAASqpV+SAZ0s632OsiEBphiKVMYq9crKCg3saGvkY3Fx0e12Ly4uKoqyu7sLVjOZTPZ0wv5Lp4WOK1VPZACBErsIpwOjXBi98atsomyY5QSiJpvN9gzTe8RlMplI6hLhguERHxNawHhnkwEafulSFEUBsX9xcZHL5Wq1Wjqdpp//4uKC2VrsxPj4OAUfEcUI8RRUWCYDgEl6PpOE8mcySv/5zlhuS7lGZD2MQmaUuUuidkkoTSYTKS5jjlTWKX4GBh4sTe86JF1CzuVS+KLqYbJmyNCKdxe/NVa48Tt5NhwUl8tF0G42m0ulUqfTCQaD/X7/7OxsfHxcNEQY9wzXm5YQTbezhULh/PxcNLBommY2mycnJwOBwPn5OXzmmsF4vahsPB7P8PBwr9fLZDI9ncJTvJSmaRaLBU45LCPoS80QpIs/we7jCGnPB1hcZkVR6CXSNC2fz0NpXKvVBL27pls0r9cbjUYzmQyCLyob5Brod1P12rMwIgJYoigKNGv8rfB4VL0xBlPicDhEpfPSdelwGPdSvLOkZ5VYEWOOQPxqYOjuF/ZOfr6TWNgRgQVVDWQKQviQV1nPzUqGRIxkYG2kke7s7IxC7/z8/L/8l//ygw8++N3f/V1FR3CAiBf2mjp0IpHI5XKi6o9hwSwOdGK00dFRs9l8enrKvBlhrIVMGCMh5Difz9M0KvJBYg3BYnAIi8UigOYX90KWZSp7go5V/OpLB4P96OvTzzhwqVTq4uICTU7vV71ej8Vi1NsXFxdLpZIQXlVnoBMqQVRzxT8sFsvZ2Vm5XF5eXmbpZUPHDvnDgc6qCLeC0CvizImtFTKkGdCC4t14Bk6kUZdIz3f0C3ki2S0OiVE5qTqoX9XzDnyvWW9VNZ4NEcSIfyh6Lu3w8JCxSj//+c+Xl5cnJyeHh4fr9brZbP7GN74BwIZCHpVmHiabzdZqtVgsViwW4Yi36CPEQqEQ6QYhQ2CtoIwf6FA5vGEQEMazVygUqtWq0+mkN18yaDVFUcrlMnMhcfZFNH3pbJ+dnbXb7WAwKCZWiFc2iS4dVVX5MsnQcXUpKe9wOI6Pj71e78LCQjKZbDQaqoElhrMiMJkCryecj1KpBNuYkA9jYl2oOjbV7XYzP8h4GsQWGl+PLxL/K/Ze1PJ4GKMhE9rUqISEMpP0jg+h0gqFgijoCpcFqRX+nMvlIiA1HhtJd+ko3vEnYJF5x2fPnv2Lf/EvWq1WNpuFuj0cDou5LwSA+NH9fj+VSsmyzGQeDIiwlSwmemhiYoKplkgDFTnRoyYkjA42v98/PDx8SeVLkgSUGUz9i0Er/2CKoMlkGhsbM06ZQ7B+abOYZWrSW9H7OoUc/TC4YplMBoIArKHJQM8iqkDsKC8PhAhIu9/vd7lchEWqjlcRUivkhqtWq9VqNTGtDrJ8urT5b1/nzRJoZskAbqFKDahVMOeKtRNyj3skYFhCRsWl6B2wl5Ig/BU/gfUlHo/HYjEMjWqgMWKtiTQVRREIT8z0zMzMxsbG6OhoKBTqdrvBYLDb7R4dHRFE0zKfSCROTk46nQ5shlDoUIo5Pz8f6CCigQ4aRpHIhuS2oig07goBMplMJJXK5XKpVHrR8aeSzVLLzweAxv+V9AkxRsMiG19e07RsNpvJZCRDLcykN8OwT06nc2trq1QqXb16NZPJFItFs4F+g6wuu8u6C3dY0kMzi8WytbW1sLBAJvbSIxoPjdhIzZCQFFJu9GlknSpCNmQNLpnFgT4AVngwiqKI3kZVp14W66sY2tdUnY2DOwunRCylQDsS0qMhsG5iJoNkAIBohlzd3NzcBx988N5770FCyagc5iJ5vV54HHiYtbU1h8MxNzd3cnJSKBQYRdHtdhmhJWjfFEVJpVL1en1yclKgtXDOGo0GKAmh1Pv9/sXFBSRYgkdI0g0i3c+MnpSejy1wkcvlMtwnmqaNjY2Rxf0l8qBSqRCmRSKR6elpUYYUeovsaDAY9Hg8EAeura0RTLFJQmMJdSXrgZKsd6hqhvYvatji9WSdh004ZOgw0SAqXoakALkAspfGfzebTZbP+FvB4MgNNb3SvL29zatphgyC0ORCslW9uiDkjKizVCqRQMHACXAHwE7Yrfx+f7lcFrpZnBPhgfX7fUZRptNpSZKsVuvExMRgMKjVatFotN1un56e9vX+/Vu3bmma9uDBg3g8Pjk5CaEoUefOzg6HnHWjLHh2diZUBkAGt9udzWZJTLCkHAZAVNCFirPK8zBWV9RhxYWsQ4DN2xUKBaFWvrwJyWV8I2ovvV7v+PhYkiSot9gPEn1er9dqtf7Jn/zJ8vKycFzQTxZ9coGkV4H6OtUTJsZut0Orsrq6KjSHpo8CkJ6PxTRD5UcyoFKFnhOgZNmQaBF6+NLdhLlHtZyfn9fr9YWFBVF9F6Gc9nysIE4wl8ViKZVKqVSKHBIUwmi+YrFIO7yqqs1mMxKJiFnrIyMjhO7UM0TWmxfPZrNra2tvvPHGe++9x1unUqlarTYxMYF7JwZ9WyyWo6Ojk5OTmzdvapp2fn5erVYhQc3n8yMjI/F4nE0xmUyHh4cmkwlJ5S1ArGezWbhGhRwwxMtsNo+NjRnL4ZqhNAnA0ChbvAV+s6ZnowKBwC9P6f7+Ph1CBEeIyGAwODs7YyKeoreEg4j1+/0PHz7MZrMvv/wyk9b7eksxwZ3QOgMDoQDfx/hWYwlC/LlkyGILWybMqNE4qjqbjdGySAag3CWwjTE4kCTJYrFsb29PTU1R7ec5xf2NCQXZkFNQ9S6ujY2Nr33ta1euXGE7Sb/BpEIvocPh+OyzzzKZzO3bt+PxOB43E3sGgwEk5OJdYJyCI+mb3/zm7du3+3oH7MXFxdjYmNPpTKVSmqaNjIwoikIH2NbW1tLSksfjSSaT2WyWnDZ8Hkwmw188OzsDR2p0ADqdDtTfYiq9LMvtdvvi4gJWCONx4iEvLi5gATEaEGGC4FHu9/uCaxjBMgPOV1XVarVCSaPog8szmUyj0RgfH3e73ZIkQRPY7/dv3779V3/1V9AGgw/WDOR9IsyUDakUTU+EKgZQg9Ai4rWFnjD+hMv4ViK6kZ4Hyxs1jYj4hLuqGWgaDg8Pxfx6IZ2KYYyFydBOIwx6MBiklIsLwiAnHHYhYblc7tVXXy2Xy/fu3Ts/P79y5cr9+/czmczq6mqr1fpylp/+5Pv7+61Wi05AuhWYEg0hezKZHBoaGh8fLxQK/JsM5+Li4pMnT1ZWVvBsDg8Pa7Xa6OhotVrd3t6m99hkMsXjcUIlkQ5gl8fGxs7Pz/v9figU4gUdDsfY2FjfQINrPOc0XGjPe/dskCgtyDqRmFBsCnMZ6V08Pz/f398/OTlJp9NEkvl8/unTpyR8ZR26GQgEfvM3f7NUKq2vr+OGi3hKZM+NcqPpIC1ZlmFWUQzFH+15nKcQKbH6qp61HxjqwZohCy9yCsIjES+v6PwcyBmOCHPnhS6E31b8yUAnDtH0gJcv8nq9FxcXPp+PBjiomj0eD53EtCOzBxC2fPe7352enk6n03TkHR4ews2s6XkK0t8ul4sBAhMTE2KWbrvddrvdExMTkAMODQ1FIpFyuXxxcQE95NWrVwmkhoaGZmZmwKczyOPg4IBJx4qixGKxfD7Pxqt6ItpkMjGc4uLiQhxFs9ksgAXGBcQBePFXwiB6vV76640+saZpsqqqHAgccJw7EjlmsxnjiLoKhUKhUEjUxuv1+v/4H//jjTfewJHXdHZUsT3C2PHOkJ/g2aTTafiV4VkkrJUMfpJQFUbLKBl8Q+Opkgw5yRcv7fmajMlkomy1tLTEaSGkEiBYo16U9FCR3snNzc3vf//7bHAwGIRyXLwpcgzrQaFQGBsbm56ebrfbf/M3fyPmIdK5wFivvb09j8eDO3Xnzh232y1qSsInMZvN6XRa07TR0VFVVbPZbL/fZwR1KpV68ODBzZs3I5GIwIsq+iDxcDg8PDwsSZJgXBJLp+oZu7OzM2PhyOieGgXLuBpGmRPh18nJCdzVMzMzlJK/zE4x3bTRaGCqqELQAup0OtGr1Wr1+PiYAIT3397e5g0FbMHoshifDO+t0+kAJTs+PobPpNFo7O/vb29vgxbkkIkCn/HpFQP6RSghjJeoZ2uGiw+LvjQ+IKpaxFAQaEmS1G63hVQZUwniUPJGvOlgMBgaGjKbzefn5zs7O0wXFw+AF7W0tDQzM5PNZu/du9dsNr/73e8uLS0tLS0lEgmfzzc2NiZSEiaT6b333vvGN77BRAzeSJIkcm+05NPfTK4I9DDe/cjIyOrq6sXFRTKZtNvtYHLQHIFAgOSRpmk+n89qtSaTSWNMgycK0xWDYY0LaDyT0vOJhq88t4x4Zhl/GWP9+Z//uaIoeGfQMrGXzKXd39/HKbZarcPDw4PBAJHs9/uffvrp66+/ztOTAh3o7O2KofAiHg6WelrqfD7fj3/8Y1jLdnZ2mEEKPbWm9/hregJdMmRiL50toScUnWVEhAVGl47HZufgNDs6OnrzzTc5MMLHEtGo0QiaTCYaql555ZUPP/zwe9/7XqPREFVUPFRGLhjRLyadWCaTyYyMjMRiMXooKOEXi0WmH0xPTwNWEQ7lpV0kYBIVQDxgYC044OgFuspEXyGbnc1mPR5PPB5vNBpnZ2fDw8Oi2VAsXa/Xw8c3AlnFMxgd5a/UWGhri8XCCOBEIjE6OvrlBuVyuadPnwKT0DQtFAphNeHXT6VShIRms9nlcjFwVpKkarX6s5/9DFcDLU0yTBgFzRDtoy1Q49evX5ckyev15vN5zBD1rM8//zwWi4VCIbLGko6gMhnQOCJ8M5o/nLxqtXpxcQEVIp4vNVQYNRqNxtLSUjAYFNQDX3zxxfXr161WK2NwxDkWa0p4L0mS0+l8+vSp0+mkbPXaa6/t7OxYrVbmnUAwxCwJMljG4FRRFNqwWq2WABNj42S9X4pis1lnqTQKFt4IxA2M0MXLxFOEPUbRR8/hAUMGAXIuEomcnJxQFajValRNLgV3ON3n5+c+ny8UCg0MkDVhKyRDTHZJV7EjTqeTWFUIliRJX474zeVyJpPp+Pj4/Py80+mQcYFmBCZWfo7KHRkZwVvc29v727/929dee01RFPbM+MWqYUQlbXHj4+Pvvfce80idTicHCPjs1tbWm2++Wa/X2ZLBYADLNGVp4dVqhukVyHcul4N/MRgMulwuwbcBhTq0O91ud21tjbLSyy+/vL+/T9pmb2/PKElCWDkhmUzG4XCQBb19+/bW1tbrr7/u8/ng58Aw9Xq9VqvFE9ZqtV6vR12FV5BlGcy40+nM5XLItIDm4r+TL4CPlPKL0QUWOqxUKjUaDWZJCFdMMwB7xLIfHR0Vi0WfzwdennTS1NQU2Vr6no3bhAo8Pz8fGRkB/C6+Xfi40guXZoDKoKrJEgQCAX5oZmwwSWpSxsyIkmW5VCplMhmXy0XyCRY8TEkgEBgZGVleXv78888VvTxyyVqJkJBjMTMzUywWz87OZmZmTk5OcrkcMRGx7sHBwd7eHgRJtLVcv34dYCAqATiNLMvQncFWbbPZ0PbQ4AQCAYZKaprGH4II6vV6iUQCLBTV4jt37mxubkqGsNRo/iRJomaHojLSuzPKAdcEpEokEmGUMsJRLBaLxWIwGGSklt1uPzg46Ha7lE3ExEOMO7YMSUUaYIYRAbWke804dkdHR9FoFBKiS6ZTvEgikbDb7alUisJwIpHY398/Ojqanp4WLUyXzr/T6RwZGclkMiaTSYAjLjnyxp01Pp6mE07Nzc0pBnpYeX19negGhkIiIFjCaK+o1+uw2quqarfbA4EATMAOh2N8fPwf//EfB4MBgcklHXtJvFwuV7lcfuWVVwaDAXEEiVpiJWrsVAN7vd4nn3zi9/utVmuhUGg2m3Nzc3t7e8DxXC7X9va23+9/6623EomE0+kUFRvkkrgpk8kcHx/HYrHR0VEcLFmWsfjr6+sTExMUKyQ97rsUh+7s7ExPTwMvZhjpwcHB7/zO7wgGqU6nk8vlMCJQcTLpk3Z1DD0JCI4oIcvU1FQ4HDYWqhFiOBoJyV0uF2VH40FFm8Ja6/P5BALCuODCSdI0DRQytB+yLB8cHPj9/rGxMWMPqvEym80XFxfFYpGxOV+ppYQgItYCwyM/z8/4ZbD1f/7P/2k0Gn6/P5FI4IpGIpFIJNLpdA4ODjqdDqOtSqVSs9mkD5N+CkRQluW/+qu/WllZEXgssVVik9jUXq9XLBbffvttggDYDYk9IfUidcs9JUk6Pj6mbfCnP/0pGdparebz+aanp4FQ41ppmuZwOLgPBQqk1uv1Mk8K/UqiWZZlm812fHz8/vvvLyws9HRqGtlA/Q0Bhqqqs7OzpI6J7WVZ/p3f+Z2uYVoCNY2DgwNFUYaHh+H5YImGh4cZSmOxWJjxLElSqVTa29uz2Wyzs7Ng+i4tVLVaZbQOPReYVOFRaHqa9/j4WFXVqampF/0e2TAAi3nYjBQwmUzb29uhUGh0dBTZelF0TCYTaKhEIvHiB4SmKJfLhNUIBm6D+YWR4fL6+nqr1YKqu9FokFtyOBxwaZ6fn5+fn8s6MokU1MzMzNWrV3O5HPt3//79o6OjN954Q2RHRTuyZICGnZycIK8TExPxeBwhpogr6QOCESlaObjDyMiI1+vd3NyE+JUss8PhYF4elaif/exnz549u3btmizLrVaLhDJsxCRpAMQyFsZut//oRz8CE9fX++WNWh1iPhLWgEYopDQajd///d9HsATGAY1O0oGBsYQI1Wp1ZGTE4/FA20fGnJoPINKxsTGB8ZJ0LAmpc/QHPmIoFBIoI0m3RERCrVbrRfQmF2kwq9Var9fPzs58Ph+NX6hqwDm/KolwcHBgsViYnW78jMhHFAoFYJ4Wi4XBJSIcEZ//slao6uTgHH1JkoAz80DNZvPZs2f5fB4NyXQGv9+/sLBALFatVj///POlpSVq8iJHz3OoqmqxWCBkunXrVqlUAk4JDyBcj0SUaBeINJxOJ3lLq9VKuR54tRgdSw3A5/P5fL6zs7OnT58qisK8l0qlMjw8XC6XbTbb2NgYcwCLxaKiKFeuXCmVSj/84Q/n5uZUA/T00mnL5/OapvE6QqmcnJz87u/+LmT/dN4hu0CsTk5OZFmm7xlMCys2OTnZ6XQymQz0sjTYkM+Eql+YDx5mb2+v0WiIBApk2pf6nlGilUolm83Shjow9JRiHISR6vf7p6en+O+qqq6vr8/OzhIbvihbiqI0m82joyPyq+rzNX4eslqt0jPNbxndremAGXGfL9OhyPjFxQUe3PT09PT0dLfbPTg4kGV5fn4+GAwqihIKhVwuV6PROD8///jjjzc3N+GfEPli0Rcq6i0IsoDYwpB2dnZWrVbJwbRaLZ/PB+iRkB5oV0O/GFpJMGXRh/8O9DabXC5XLpcnJiZCodDIyMjS0tKNGzcYR4unzxBRKoNkqwWOW9NLhGLtZFlWFAWhFN4h2S/k3ufz9Xq909NTVe+pR2QxH51Oh6JboVAg/tre3tY0DSr2arW6v79/cXFhtVpjsZhAXQoPD/siEsWapgHSApcsGaKiRqNBggAM4Iv+uCh/mc3mRCLR7/cPDg5MJtPS0hKdpMbUoPEPXS7XyMhINpvFiIlfCfvDEM2BPk4hn89LhlK9uKe8vr7ucDiIdNinwWAQDAYBUZTLZQpPBMwiVwRlb7fb/drXvlapVB4/fnzr1i3GIcv6uHlVB6nZ7XZiqFAoRM7C6XRWq1W73S5UvabXE3kfpFNRFGFY0V7glTFhoCLp2Eyn00SmGBH+QSNupVIh1KB37dNPP8XFEQGgZJjkY7FYCoVCJpNZWlpSdeY+j8dzenrqdruj0Wg0GrXb7clk0uFwxONxvM9SqRQKhdxu9/HxMTkFpq3w5AS5kUik1+uxhm63e2RkBLWBgkfIOOtQQlLhkHUyQaDlwggIZlEaoyHUUw3Vd5JbAx0XLkkSrEZTU1ONRmN7e/vWrVtfmUpAdg8ODnq93vz8/FdaTFVVk8kkoYaqqtFoVGQZZAGbSaVS+Xze6/X6/X648FWd857FOjo66vf78PKm02lQD2RfyNGVy2Wfz5dIJAaDAWLx5a1lGQffZrNVq1WAOz6fr9lsAq6nBOH3+yORCGKqGYZma3o2nOBUBEHAwvByoEf3er2EWkxPheFSZNUdDkc+nzeZTADTHj16FIvFACaISFBsACsFGFzSQ32Px5PNZhOJRCaTCQQCy8vLJpOJ1HEsFnO73UxHcjqdY2NjZ2dn/X6fVjlCWkgfXS7X1NRUp9M5OTnhwAQCAWQFUmQIGoRWJo0O2IaiIe68iIeSySQTuZjuNDMzw/GwWq33798/PDx87733nE4n1lzWSZ2r1Sp5n1Qqdf369YGObBYCpOkEbtvb25FIJB6Pg5O+JIL9fp84V1VV6uWyASCpKPo04r29vb29vbGxMdhR5+fnV1ZWPB5PKBSan5+PRqPUm1gFMk/hcBgYSa/Xc7vdpCXhxuBVVVUNBAKjo6N0VMOmSpRE1azValGwOzo6EkzxqVQK30XWaasE4RbvRgsksEYG5xEqM6+BJAW1TggROSRoXHSqaDk0SjDFctQ2KStB3FAul8fHx1dWViwWC26l1Wql5PX//D//D43jMzMznU5nc3MzFou5XK6TkxOTycSISk3T3G53o9HY3NzEZLTbbUSk1WqZTCYoQ7e3t/FHqbfiIAKMQbky8oTHJi45Ozur1+tkag4ODmw2W7lchu5mZ2fnj//4j09PT2FF4w6RSMTv9+/u7lIF397eFhSmQmI4Zow5wSCaDJ0Nmg4VAT7E6vV09iuj5MmEOScnJw8fPgyFQvDJ5PN5ssC5XI4iDzWE8/Nz2vvBURAnnp6eVqvVeDzu9XppA2dMrcvlQj68Xi/NccC0fT4fW1ur1U5PTxOJhCB/gvrm5OQExDdCYLfbM5kM4A3wOQz7IzZE2bhcLgRIlmW3203ltd1uM827Vqvdvn272WweHh4eHh5ev35dpPgEQJmwQ1EUGu6QezyVZDI5NzcH1iOfz0ciEcp8Tqfz0aNHjUZjZWXF6/WazeZkMsmECHINeNbg39HTYK1Yw0gkIjo0rVZrLpdLJpNTU1OBQIAyl/Bd8CPNZjN+lUg91Gq1w8NDhp0wrrtWq/2X//JfcA9orFhZWZmenkb4JEki/5zJZMbHx/f394XoSy+059fr9adPn46OjkJWS2xEtpLIjLVKp9Mo4+fsoCzLe3t7sXjcbrXm8vlHjx6pqvrmm29KkpRMJt1uN+VC9I3NZsvn80+ePOl2u+gS6npWq3V/f5+1MJvNc3NzVDShk6tWqzj4zDrjHPP0YLO2t7fD4fDq6urDhw+bzSZHH593bGxM1IYh3GIKMlgJWNoDgQDRA640p4oiweTkJOdva2tL07QrV67U6/UnT54MDw97PB5MYafTabfbhLokI46PjxcXF8EXwLxF0ebx48eLi4sYpnA4TJw1OjoK/t3tdkPHkM/nSXPLsszQZbQCgkUNiomK9XodXxbLaLPZ6vX65uYmw3aEn6rpvX65XE6SpHA4DBs2aqNQKJydnV29elXVS8L//b//d/zC6enpkZERdPmNGzdmZmZ6+qzKer2eTqcJkoCYGifUSXrFZn9/3+l0xuNxs9m8u7ubSqVeffVVculCEGkA4SaKkbrx3gcfOr2esdlFl1Pp1ktbOyeFQv7WrVvBYDCdTtvtdofDAcvF2NgYBYpcLqfqFWIA/DabjfQxnkcgEIjFYhMTE9RenE5nsVg8OjrCf8I+AjQjZUWhanl5eW1trVgsLi8vE0+Rre10On29gw8jIsgXqaUMDQ3RtK7ptO/UX0ulUjweJ9f17Nmzer0eCoXi8fjFxQVTQ+mzZbiXzWYj1CKGB2iFiVf0FmRyraRPZ2ZmHA6Hz+ebmZmxWq2ZTIYC4sTEhKZpm5ub0MjmcjnR86nqbW20ZzKI1ev1xuNxLCDr8+zZM7PZDFxd8EBb9EmZg8EAYByyRSdwLpdbXV0lEfP+++9PT0//5Cc/SSQSRN8vv/xyJpN55ZVXpqamevp4gVarRdknFotdXFxQFxYwAuF6SjrUGPjd5OTk7OwsvNyoMWBF5l9SpsmSZDbLfTlTzh89e9iVHAsLKyGfs1StZzIZ+h6npqY2NzdhlCsUCpIk0YyGYtd0ogRVVRmlh6fC8CD8pMnJSV6mWq2ur68rikJlrdFoxOPxYDCYTCaBOHo8nsXFRZfLRdw0Nzdns9mePXtWLpdHR0dJvWIWUQ8YO1yuTCYDuTmj5Mk1KIqSTCa/+OILeBaBhJPXfvnll9vt9scff0yKD1gS8GJservd3tzcTKVS3/jGNzRNowBPaq3T6QDNYJ4bdYKpqamRkRFG05hMJmzl7u4u7jkzukU7MpZxb2/P4XCMjIwUi0XcKQaTdDodn8/H9IDx8XGbzUZCv91ue71e9BaFI2SLxOzm5uZgMFhYWDCbzevr65qmMS41nU5PTU3t7OzMzc0dHR195zvfITsqgNfJZBIFzBrCwi0ZCoUi4VIul//hH/6BrH0+n/+jP/ojEsiX/DNsqs1skf9iPfmNhPvk2ZNKxzI5NTk6GkZjl0olWZbD4fDx8TF521KpBPcf5YhGo0GRjkpwv9+v1WqidcdqtQplMDo6SkmYAQczMzP5fJ4witN8cXFxcHAwOztLEabT6ZyenoK4XVtbK5fL0WgUcD0atFgsgoLCjSOZRG8qIAvgqbIsM/oR8v5arRYMBsn437p1S1XVdDq9t7eHw0QIBo8U5eTDw0PmECHNmUzGarWCKZqamvJ4POBn0L5+v390dJQqOxDk8fHx8/NzvIhMJhMOh4lOiGe73e7W1lan01lcXATYE4vF/H5/o9Eol8uxWOzg4ODg4IDKLo89GAzGx8drtdrFxQU90wLMbrFYdnd3o9Goz+d79uzZ+vr6b//2b5dKJQ4PW1CpVGq12ve+9z3N0LlJGkLTtGAwSEMR9vGSv4UD8Jd/+Zd7e3t37tyhnPDv/t2/Uy5jyGRZG1hk7aQuy/+fv3j2+2/M+u3d04v0yelxbGx8enoaBEi9XmfoD0BbMjGM2cRJB+0EMoQONQbzUf4jaiNdnsvl/H4/IBOmM/j9floJJEmanZ0FOTg6OirAaPv7+5OTk/F4/MmTJ5gnXDqz2Tw1NcVOw1wCWgZgidfrJZSbmppCmKBF3NjYGBkZESkAs9nM4MXDw8PNzU2bzRYOhzc3NxlSMjc3d+fOHZPJxG9DodDw8HC326V+QNSysLBw69atjY2NYrEI4S/ZasQRpNTExAQUGLFYDDIVka/CHu3u7ubzeUZrVyoVGt7pyJiamtrf39/Z2VlYWJBlORgMFgqFXq83MzPT7XbhXkM1Yisow3Pm33///a9//et+v59ChdVqpTJBGuX1119H8YgEATRuTqezXq/T6aU9D8VBsH7+859D4/7uu++ur6/bbDboCI19y5I2sFiVB+mW/OToyNYuO1z+kWi8Xi7s7OzB0sycIKAaIyMjOOywYubz+WazSRc2peW9vb1isUjxDj0EMLzX6zmdTqQwlUqNjo46HA70PHEK0qaq6muvvXZ4ePjw4UOYfVj9k5OTcDg8PT29ubkJYxhn3Wq1rq6uZrNZgbglSsVkg1cZDAbEO81mc2Jiwul0Pnv2zOVyhUIhRvV1u92rV6/6/f5kMrm9vc2TIzT5fB4vCqenUqn0+32gaShLm82Wy+WWl5cXFxdPTk4ODw8TiUS73R4dHY1EIrlcjgKGJEmhUIgMczweLxQKLOlAZ+DBZ00mk0tLS/gYVqs1EAiAyk8kEjs7O0dHR2Rrh4aGmPW1uLgItiKfz0ejUfxLgdGw2Wy7u7u7u7tut/vll19mKdgj2DeuX7++sLBgTKyTCeeQq6pKEVP3mb4ULJvNdv/+/Ugk8sknn+ABR6PRX/u1X2MQq/xLZhFNlmWTYpGfPnsy6PedTo/H6w8PDzeajYODA7Dx1EGtVuv29jZ5Xhpt0czFYnF4eBi2CdzVs7MzPm+322dmZsrl8sbGhsPhAJakKAqARgYonpycZDIZwHeyLM/MzMRiMWretGgSpsGWMT09nUwmz87OwuEwKQnsHQ4WCSES7tgy5JVgB4cvGo0ODQ198cUXNpttcXGxXC7v7u42m83R0dGRkREKyTabbXx8vFgsDg0NMRDL6/VCd4bM0U8MYYbb7T47O5uenr558yYJGmRIURSUytnZGVsyMjLCYMHJyUnSucgcQYnf76dmxYkSMMZ6vU72YX9///T0FCyG3+9vNpu1Wm1xcVHTNChDYrGYAJfKOvnPf/7P/3lycvLXfu3XhO+fyWRwAe/fv/97v/d7EGXJOu2KoijtdpvuB3I9Ak6IKbTb7fv7+9VqdXx8/PT09OHDh+12OxaLfec73xH3+VK/mWSTrCjNRjeVutja3F578nhrZ8vlcl27do2CFxkpSZJu3LgB6M/pdPIPvA2WuNlsrq2tjY6O8vKc8uPjY7/fPzk5SZpAkiTqCZqmffzxx81mc3FxcXFxMRaLDQ8Pv/nmmx6Pp1AoxOPx8fFxutcZezw2NlatVk9PT0dHR6empgDrUUAgrRqPxzkAxDvDw8PRaBT9TN8VGTXKoC+//LLZbF5bW7NYLMvLy0tLS7SGUnrf2tra29sjNTo/Pw+859atW5RoyDtwnKLRaLfbXVpaisViPMbCwgIz69vt9pMnT2w2G0lgm812cXHh8XhUVT04OIC9CH1MI/za2lo8HofkKJfLtVotsg8ulysajR4eHs7Ozk5OTuLP5fN52Cg3NjbMZjPGl94KUTZwOByffPIJeD3ZwBIwOjo6PDxsNpudTuf//J//UzNMcUd0wPTabDYKMCIvqumjXEZGRmBYvXHjxquvvrqwsFAqlTiQxry8osmyJMuYm0wms7+/T2pgbm4ukUikUqmDgwNeAzzTgwcParVaIpGgxsIhrlQqFGSOj48BNFerVQGc93q9JycncMTDsodMPH361O12LywsHB8fM0CW6T/UbkulUrFYtFqtk5OT5MMYMBQOh5vNJlaG8p/AR5ANz+VydHW3Wq1yuazonGZer9fhcICHHhkZIaM7OTnJIVtfXyeyY0ja5OTk9va21+udmJhIpVKqqq6urpLaptwL4Wy73S4UCu+99x64SMgavvjiC9oMwZpSy4J4IxgMUh6dnJzkSUj8PHnypN/v3717l2/nCQOBANoCvTU/P7+1tZVOp5mQRVms3W4vLy+D4a5UKviU7Mjf//3fE41+4xvfENPaSPBubGx0Op179+45nc5f//VfF5PokWbyR3SVEvOKvDzYmPX1dQQ0k8kcHR0R7vzmb/6maiC7Q+PKp6en7XabnuZCocBoE7fbffXq1Xq9vre3R1rS7/dHo9HHjx+z9ADrcG6KxSK9crgLVDCWlpaAc2iaBnyM1yZxZbVaHz58SKcUEmy32+fm5lwuVyaTsdvtQ0ND6XQ6m81OTk56vd5MJlOtVuGAlGU5n8+3Wi26hC8uLmj9YLgB2pRwiX5fQOJwBVKsAMleqVRwutvtNvwFrVYrEolwH1JZ3W43nU6XSqWVlRVIPhwOB3oaT5xicCwWI2s/Ojr66NEjtiSXy1EfazQacHAC1UKZkculAXBtba1Wq7322mtnZ2dMZpuZmfH5fICc8FaXlpYeP35cqVToNaX/uN/vMycslUo1m01Y1549e1apVMbGxoLB4Keffnr37l2Kj2T4CoXCxsZGs9l89OjR8vLy2NjYrVu30DSqgeeIDAvIWNnQ1Hp8fFwoFO7cuQM5wGAwePTo0WuvvXbt2jVjQGA2mxWqUScnJ71ez+v1ut3u+fl5TdOIYpaWliRJwnHJZDI3btyYm5tLJpMASqkqYBNPTk6mp6dffvll3pnRc8g7Tfq4LAhuq9V66aWX7Hb71tZWJBJZWVmhxxwHn1MI+83JyUk+n4/H4/By0WQG7z7s+DT+AyqncE41OhqNrqysYBHg1SRDQ3wOxpqRkJ1OJx6PI9YXFxfxeNxmsx0cHLRaLbfbPTk5eeXKlYODg9PTU2iAJiYmwAz6fL4bN25Uq9Uf/OAHlFnOzs6uXLlCHSIajSK+TGL2+Xz5fN7j8QAEoloFLmN1ddXn833yySc8hiRJ2WyW6JV669DQ0Pb29vXr18nsc5h5KXB5QNZ4nXg8vrq6ysFYWFj427/9W+GDi8q3GEmZy+U+//xzSltGlTM+Pm4ymc7Pz5Xnh9P4/f6dnZ1Wq0WrgaZpw8PDv/jFL2q1mlmfCoblldGTFxcX5+fnqBOyhWazGbKAdrtNEhwgTjQaPT4+3t/fx9PkAIF0GAwGZOcpawQCARpwcbzo049EIvl8Hv8A9ot6vQ6C+/j4mDaERCLB3NdoNGoymfb29igsaJpGewUFHEmScrkcdBR0FjmdztPTUxpp7HZ7PB6XZfnw8PDi4iIajXq9Xp4B6Es2m6WQB90UD8bgZHKkqVRqcnIS3kqLxfLo0SO32x2LxYjqJUkSHYUPHjx48ODBd77zHUVRut3u2NjY+vr6lStXzGZzNpsNBAIUYcj9Dg8Po+MpFFarVZbl8ePH3W73zp07Ozs7hULBZrOBfsOwplKpdrudSCQ+/fRT2siYPP3o0aNwOIy9SyaTJpMpFotRtldV1ePx/OhHP+p0Or/xG78BM6Usy41Gg9GTqqrOz89fXFwwfVM1MHHiRezu7vKcfX12oc1me/LkSaPRePXVV0GB5/P5hw8fTkxMcBNhTxW6MUdHR2OxWK1WA5VLwh4Cbbvdvri4iCdBGExeH0Z4hlG3220qJCcnJ6SO6B9cX1+v1+sknWOxmMViubi4GB0dVRQlnU73er2hoSG/3//s2TNN0+bm5mZmZvr9Pu1Z0WgUEs75+fl+v394eIj/GA6HA4EA/YAzMzMTExPtdntkZASKdvqcQqGQ3W6HX2p2djYajWIxl5eXbTYbkSb2iAJftVqlMXBqago+WbvdPjs7S6caDACvvvoqrdsQo7HWtA2+9NJLb7311k9+8hNI/ZlT+vTpU7PZPDIygstItjkSiaBsQLAoioJff3Fxcf36dbvd/uTJk/n5+UAgoKoqr0/6KhaLmUymdDp97do1OpGYu7SysnJ+fg64ngoj5xbDV6/X33333fPz883NTQIUVZ8RQckVnHSlUjEbphAq+oBCQf0gEqGtVmtlZYUo2OFwAEDCL3r8+DGVcmBtisfjoQ+Hhl2yOOS4JUkiz2uxWBYXF4FelUolYmMcFEmSNjY2LDpNpdfrTaVSRNfs+uHhYSqV4qyMj49bLJbz8/NIJOJwOE5OTkg3WCyWjz76qFarCfK3k5MTktEkFKampoaHh9PpNIJOMYRFdLvd4+PjlDKpB09NTaVSKSb3wZtIOXZtba1arS4vL3s8HkRnbm4OL2cwGCCp5PCQOa/XOzs7i8+uaRoQCUVRSIgcHx+fnp4WCoW1tbWdnZ1r16698cYb9+7dA+FdLBanp6cfP37scrkikQhgB4LWSCRSr9eHhobOzs6AY4TD4VqtVi6XYaTZ2trCLtvtdgCM9P9MTEyQbUJAQYyZTKaVlRUBCkVd0WgppCQWi/3gBz+ghs1PqMF7vd5vf/vb3//+95kEJiymqpPFAVeBR0kgINBztJBQFRgaGtrf3/d4PFtbW4IoQKF1mFL88PAwiK1yuUwRhgQjuRyw0t1ut1arpVKphYWFycnJUqlktVoJrGgrxe+u1WpAD0gr4+hJkoTtPzw8DAaDzJeqVqurq6vz8/Offvop5jgajVKWz2azoMtFRy/JDpQ2+U+AX7iNZKEwYbu7u7Isj46OUh6empqKxWJkuhcWFnw+Hzlxav6MUmIpGYqEdY7FYleuXNnZ2QGhlcvlXnvtNU3TGKL05MmTdDpNraZYLK6srNy5c+ezzz5TFIXKdzgcxoAODw8TQEmSNDQ0RHrd7XYfHBywi2NjY8zSvXPnDoVI8BGhUAii0XQ63el05ufns9ksJCsiicMY5s3NTRYBJi0hB5qmMZacyYmUruHJ6ff7lUqFRl9Z7wE2XqqqUpGkSUnVIbXxeLxYLJZKJUiXXC7X+Pi40+nc3t7e2dkhJfslzhOH6eLiwu/3U2ijaEXCvdVqCXoFUBNgJknkVCoVn8+3trZGOyjNZJlMhvJWr9dD36RSKTCiVGRPTk5sNtvk5GQ2mwWfeffu3adPn+7u7iLr4+PjrVbr5OSEzirAspS00+k0JwM26YuLC0nnv5+ZmSkUCrQSbW9v93o9ENytViuRSJAuOj8/d7vdogmHLSRDCEMGITANC5FI5Pr168+ePWOVm80mHWz9fh8eCmjWYeBdWVlZWVnZ2dkJBoOi1XtnZwd5bbVa8BgEg0EcF0CO5AJGRkaOjo5UVb19+/b+/n673UaRxOPxs7MzQr9+v59IJA4PD0WpGIeGhgsyapphnjI178nJyf/wH/4D/e50xADgHhgGxhrlSSBf+GQ0GhUIdbHOJycnP/rRj8xmM5FyuVxOJBLvvPNOOp2GJVm5uLgQKbJSqfTRRx9Vq9WpqSmv10uwja+KyW80GtPT0yCxWq3W2dnZ6uoq/s3w8PDBwYGmabj/gUAAAp1MJgNpGFESuWkInMlNk9dJJpPBYPDrX/96NptlIJnL5ZqYmEBdOZ3OqampZDKZSqXQeYipyWSiNIQaI8s3PT3NcMe5ubmdnR28abz+8fFxn89HWk+SJBaXXDmnqFQqEbJhE6muJBKJW7duoZXxwV977TWm2fh8Pm7V6XTgaLh+/brP57t3757X6wWzxWxcj8cDsoD2V7x+VpJ+BCj1z87OcGrX19ch3+r1erOzsxAOHB8fg544PDw06wOnkZ6FhYVarZZMJikZCfnQdDYYVgNfCn/r/PxceqHX+dKFFg8EAqlUittardZsNgujR7FYBK2lKAqpQY/HA3eQwiCNdrtNfcZkMn322WfpdDoWi2EgIPOg6YWeMtrZ+v1+t9s9OTlZWVlBCfn9frBKIyMjjUbD4/GUSiUSmMfHxzgH+HOcA2RLVdWpqSkw1Far9fXXX5ck6eOPP8b60F10fHw8GAzm5uaKxSJz3rH9HA4SyhD99Pv9oaGhcDhcLpfz+fz09LQI/glOY7FYLBajaoY+BjE3Pz9fq9U0TQNlip8wPDw8NjaWyWRisdjc3BxIqW63S0s36mFycjKXy6EkyNrcvHkTYCAFctBHnU4Hxh6zPnwArjba4fHwxsfH2+12LpcbHh6empra2NiIx+Ng/RYXFxm5e3h4KLqrjbI1GAyWlpa2trZSqdSLrczoPzr5YJkTk54u1ZuNGstoTIkwBN7m+9///m//9m9nMhlGu6F6yKvhRSis1Onp6dnZWTwen5qaAlBwdHQUj8fBSB0cHNRqNRDrNJBQ7QfkRUkrGAxaLBaQongtpABKpRKnH3IsrEM6nZZlORqNOhwOAumJiQmLxXJ6etrr9a5duxaLxe7du8eUPfAh4LuXl5dpkgEIQNxkNpvB2xA0UJ4Dy1Yul+Px+Pb2NpAv0seMK2KELtm4Xq/n8/lIQwcCgb29vYODA4JZskTZbHZqaioajeJDoGlu3boFr8T4+DjbSfTa7/dXV1epsdrtdggENjY2AMxYdCpycJ5jY2NAKVnMRCIB0hzsfDKZnJmZyeVyZrN5fn4+lUqRuPH7/egboz9E2Auk0egwqTrPytTUFE1g6Gk8ZiNwlEvTL/G/5H7JDamqCvEJTi3FPdH4SfO6pmlf5i0GgwGDLijJTU9Pb2xsHB8f8+aRSOT09BTxBLdOWZduRLCgbD+x5MnJiSRJ0WiU9h4aj+LxOIvCuGgCAkFOR0eew+E4Ozsja7C6uvro0SMME4ebeDORSGiahlGAkLhSqQBWBnYny7LD4SBPRokJ2SImH+j0u/QhwqCMgYhGo6Ojo2AG9/b2tre3+TmR/MXFRSKRGBoaooUfN/bOnTv0As3NzZGOp4NDUZSFhQUIeSwWC24rw2kHOjU3MTxWnhiZHuuRkZGTkxPQVzADzM3NHR4eUowHzEjJSzX08ZnNZhrdQEsb+2pkfRAQ6SHQ/WAZZANt5yWpuvRDom/RiscNBaWbSWfXUfVOPgV/XpZll8tVLBaRnrm5uStXrmxubvK24XA4Go2WSqWjoyMCCujbyExKkoSBg9qL1r/T01Pwoli0TCbTarU4uxRt6MpCtjiawI6dTif1BJBDx8fHaG8calJrVAtOT0+BjRcKBXQAmGkUA4TYRE+VSmVmZmZ3dxe4plhrOHOoh5hMJpRTNBqlNH5wcPD48WNq6tjuQqEAiVc2mxWbd/fuXU4Og+OAPvNIKysrlUqFszE7O0s0A4JUGBoeeH5+HqywLMsg2/CWsNSk+/f29kg30r9AYdikD3bjDOfz+bGxMXr4aHxV9MF9WM9YLLa3tydJEoGURZ+Fa1ROA53GQhhESU+AEfgL743qIbH8+++/v7+/LwmWa6fTyWJxr2azSb1venoaIBtZKBwXmoAHg4HP52u3281mMxgM8nqVSqVarYqKAa49+4GSJ052u92jo6Mgy+x2O8OA4DUARmexWFwuF765LMuvvfZap9P54osvSJKhoig3kbjCL8ZottttZiVIOv0BrRZ2u71QKMzPz+/t7aG3BMLE7/ePjIwkk0nwSSxCJBIBxkPnNAgccMzlcjkcDsNmUy6XWd/V1dXDw8NisQhHwfLyMvmzwWCwurp6dnZGJt3r9Z6enpKXIvUgVI4sy/Pz8/B7Y7PA6yITpPh9Ph8cY8xgM5vNFEONaoaUptvtTiaTuH2KTuC7ubm5sbERCoUymYzA2Ajhk3SYQ61We/LkCXxpRispvCujKyZJ0pUrV1gTk8l079490KeDwUAhxCVxRwmJrAnIm1deeQX0t6IogjCYYp+YykSxXdM0AICMsQCXTEk4Ho+TpC4UCpw/aiZEiOhkwWnDAzCvgcW6efOm2+3+5JNPkIOxsTHCH6fT6ff74SgnCUKjC4MeRLsHzY9ut5vGrK2tLWFHWAIOw8nJiZijjqcFyqNQKKyvr5OYoKGSrl3EEZtoMpmuXbt2cHBAVzTp/p2dHbbq6tWr+/v7jCLHdjidzouLCxS20AcWiwVEGraJLIMAS3W7XSB1mUyGhjksOJ6WwMzAZ9HpdEqlEk6PEF8ypVtbWzQl8Fdut1t5nrcC4P/nn39OX//AMOrBqMBknYBpaGiILox33303kUhAvSHLspJOp3d2dqrVKn3fNClo+mid0dHR119/fWtri2mrNONqmgYTrkUnp3M6nVQYOXM+nw/nKZ/Pw41BUJrP50lAk3Ho9/s8BwcIMFCz2XQ4HM1mEx3G0y8uLo6Pj3/88cegokWTD2AVRVHIdtKoU61W+UN8KbxL2rkymcy1a9e2t7cpPrBqg8EAI35+fi6yNUDbpqamZmZmqtXq1taWYAiLRqNED7FYjAwZ+Z7l5eWjoyNyfjh5e3t7iqJwn42NjX6/H4vFqHiCsGBAkqSzd4hwT1EUmpHOz89tOlU4DW0gO+bn51VVxcfFgEr6FCMUBJQh+Ijkuubm5r7//e8T6hLE0WU0MPBsccygbvvpT39qbEGTXrhQ+QA5OQk3b94MBAIoUcXr9VYqFZAFsizjulJ5ZQrPyMjIm2+++fTpU4gxaCulBqLo3BWSJNHBAl2CpmkejwdaxIuLCzAzU1NTaDuqyKQSVFXliCPmoVBIlmWSuegPhl+0Wq2pqamVlZXPPvsM8PHo6CjDdik74MbxkpIk1Wo1qBnReRTXgAXncrnr169vbGygHYVNBBnLnAWzPk/FarWOj4+TaQNYC7orFApR7SbxS46RIUqQp5+cnIRCIYIVSZJY1ePjY1mWFxcXBRSdyEM414PBIBKJkO/tdruTk5PFYhGcCKJDZZ34ZmlpCUYnIF9GAZ2YmOAgDfS5AbIsgw5yu92yLD958gTFKTrCJUNfeCKRWFpa8vv99+/fF1byklQJC04zC/lORR9ApKqqwrwKs9l8enp6dHREnxMFIHygk5MT6POePn0KywCMhv1+n1ZPkaLFy4HpVZIkqKclSaIcgQNbr9dTqRTFImQLqgwMoiRJlNmRBnAj1PWg97x9+/bjx48zmQzteNAeeTwej8fDPtHFIMsyeSxFn3ZBSBGPx5vNZqlUunHjxsbGBrKr6lwacHGhdQSGhM67RCJBMZ9GN5zC4+NjTj/2C7DXzMwMFcCTk5Px8fFcLkfYOD09DWSIM0Y6Q5KkTCZDdoAtgbuaFLnJZBodHSUXKum1YV6kXC6Te4N9mVZpWSf5oFQlcPdoMqfTuba2xpIC3QE9qz1PY4ZqjEajhD60lL6orvi8yWSiymw2mz/88EPK1Th2vyT8g17i6OgIa8VkFdC9NFa8+eabGxsbolhL676YE8T5NpvNzWYT70G4RHSV8G7z8/OZTAZkppAtSZLgqGWPaasFh0PJEvUD6PvVV1/d2tqiUTsWi4EBp7fYbreLUjz9aiw3a4E/ASVGp9O5evUqC60Y5rIwvubk5KSvD8QD4DYyMkInCF4muYl+v0/uVFXV8/NzsjYej4fuSww0VUscvunpadxtysm5XA6vA+y/rJNMR6NRwlskGDIPQbJArgvkAtDtfD7v9/sJC3hmcDiapp2fnwNXdDqd+/v79+7dEyw6r7/+erlcZtpl//lhosgipKkAtb9SsIBVQv1P6EaZjvsokCITguGGn5+fX1xcUN0cGRkJBoPtdvvo6Mjv99+5c2d9fV0QKPKGoCg5KICH6AkTNptgSrD2zs/P06Ys6YMOI5EIXWLoLU3TyEJ1u136j8lDotttNtvdu3cPDg5OTk7gEgIFitIaHx/XNA00I2hPmG2MyzExMQEP5erq6tOnT4lqVX10ABg36BhN+tgfs9lMszzgBXxtgD31ep12Pwgm+KJut+vxeHj4cDhMWhXKAtTb22+/TQcfDhD2FIePAraw1PF4nEKtMfJnjAV0N4o+CxLWLk0fETI9Pc00EJfLtbu7+zd/8zdXr15Fo/d6PbvdvrCwQMh/SWhkvYMoHA539aEhl6RKluVGo/HTn/50fX19c3PT6XS+9NJLqVTq/v37X54BSisAsEwmE61XFPiazabNZoN/G4KNcDh8/fr1tbU1TaclRrZoyMSdB13JirNquMbZbBbC45GRkYmJic3NTbQFskL5CEZGnp6aoKZpTM2A/lXWmyfv3r17cnJyfHwMBhCJNJvNpVIJgAaC6HQ6yc5/GQMrCkpleHh4f3+fBnkB9tB02hnIAZLJJE9CcGS1WnGACGZpsBkaGqKbgLQkMQE+KOB04O3dbhcXlpo9QLSFhQXGTyqKAtZNBHHBYBDfi/455tSLAg5vilLsdDo0AIdCIXh+RPABZXqhUNjd3f3rv/7r1157TZblBw8e7OzsSJLU6/UmJyfxAi91Q2g6vRtWC5/vkmCB+Ds6Orpz587v//7vf+tb35qfn3/33Xez2ez6+rrZbFZgA+d0apoGwQ1WHLg3Woe+FKiOEonEgwcPzGYzjJLT09N2u/38/DwajQLEpgkCvAcbRpNuNpvFjsCpsrW1JY6pkC1wbaRYyFMgLpBEtFottILJZLp7924ymTw9PYWYig5mdPLs7Cz5UjpakS1eUPjIJpMJ7td4PL65uakZcCY2m40BSSICwL+BJOySL6KqKhV0yiz0wkNmQfclgy3Pzs4IeOEEZPNgIDOZTBBRiwwIHRkDnRSYMe+CigPVJbQjerpUKk1NTQHbxEGkcTIUCu3s7Fy9ejWdTtfr9bm5OToyUG+hUIiQ4ivjvsFgAExDjHMy/kqSpD/8wz+8efMmjJAU1996660vUzlYULLYnDaOC52ZHBcSMECdAP09fPiwVCrduXOHxJXVaj06OhoMBrgU8B3SWhOLxUQRI5lMUnUiw3Tv3j2fzydmapLTIrOH149FoPja6XRoCsWg9PShTp9++in1TfJkVPpmZ2clSdre3sZThlM+Eomgj9lUdDBsROl0ulwuz8/PSwa3VEC7sDuyXoe5VEjJ5XJHR0cUMff29kwmUyKRQEV1Oh2PxwPnAumY5eVlqNiMRocYSFT3NL3ao+p0myyvyWQCgiy8afEYVqt1Y2MD1r9sNut2u8ktYUyhoohGo6wqRJgffPDB4eHhO++8k8vlbty4QUr9kkEUcpzNZoFIqAYGUHipVAOvuGTIqCksIhhTSZJarVaj0RgZGYGovt1uF4tFvJ9gMBiJRFqt1vb2NnIKZQi6nW6qZDK5uLhI3AfIjjwbGjESiQBNYRGvXbt2fn5O7YKck9PppMcLVQGqggZou90OxRkSLzzuO3fuHB8fQ1wD5o7kJNEcmESQ+FgxWe85gV+FOBdEPKAGYQhI/QmnGMGCn9J4cMlinJ6emkymaDRKrR16KlmWoakBAIOpom3OaHTwe8gnGbNKFPXp9aVuSFZMCLesX6x/KpWSJAm7OTAMxzObzdevX4eIm6jfZDK9++67s7OzzOYAJC10thAsWacGEaAMk8mEQqpWq3SfG6VKvJTJZJJVVQXvC9wHiSNDA/BNOCvk01OplMCM//SnPwXIC85dURR6bePx+Nra2sjICMcFmAqiTaMVeCxFUTKZzNOnT9944w0OMQ9aLBYLhQIpD4wp3frk1unpY2Yah7vdbt+/f//atWtMN6H/PZVKkQ1PpVKzs7NDQ0M4yIL8E2NBIhQXCsjR2NiY0S5QRSYJJ8syRQhCwi9TzIqSSqVOT085bPv7+8Vi8erVq7wdUSqRL27rrVu3LrFhkw0imoFbpqlfOJeyTrsKTabH4xETN8TeA74DmCUA2cYPaIZBoSK9d+/evUajce3aNeooQrAQDqP0463jJYPEcrvdWDPju4h0g0yDmMvlmpub63Q6WCtFUXDcQK/ivtHDA2qlVCoxduFnP/vZ9evXh4aG4D/p9/s7OzsUjB89ekTiQNM0Qir0FgVp9s9mszHQ8e7du5pO+K4oSj6fB41OrwEZyEajIVqBVR01q+kM6Wtra7du3aKZ0WKxHB0dketbX1/vdDorKyuKolAmAh0q+KiEBkLxsF6ilNFqtfb29qLRKI1l3W6XuSN+v587kPGHYkpRFPq2X3rpJYrfgIJolur1emtra9PT09xKSNVgMICJHh+UWjUcPmQH2WPCQMo1qqoC5DeKjqIoz549SyQSXq/36dOnzEYUMZ0xqBRLTUeGZuCfGRhGdIvErMViSSaTTAqXZZmgqt1uRyKRiYkJxYCP+KVg5XK5/f196h4YHWwThSTyCBRJhPlwu93pdPri4iIWi5nN5vv373/ta1/TNK3b7Q4PDzcaDXhE/H7/48ePSahIkgQNs6ZpBFxOpxPJkGX5k08+GR4eXlpa6uvkaZqmCSyNTecDRmkz3gfoDxNNZZ1T/+Dg4M6dOywcFCuI5pMnT8bGxiYmJgQttBB04UlIusvCypp16ltauMA6U10hBTU5OUkhC6pfIZ39fp/uLha9XC4nk8nBYBCPxwGQoUoFSRUd1VTDCJVefvllUidCYahiQI3+nFQwJyYmJicnhQtrMpn4uuXlZTyWK1euvJjeVA2DgzS90X4wGICuJptI5yav0+/38/k8eVqSXjSgl0olt9u9uLhovLmsjzmWSdDRimO1WmEaoURAAYdz02q10ul0s9l0uVywyqC3pqenLy4uzs7O3nrrLSqygUCAqZlYQHgsqWBAsI4aoFgERrZSqWDL0HmsBc1JJpOJVlVenhQaT6soCk8y0Dmijo6OKpXKtWvX2vpExvX1dQgaGKnqcDja7TbQAEY7C7VPuov8p2xAKWk6PbWqqtAt8cAsESQfvAhFCArzoJ/r9TqUNTi/HO6joyM64VBOhKubm5ter/fll1+GE0q0zQjjJelOj3jZwWDw6aefXr9+XVgxsmsbGxvUMZPJZLVaXVxcFKvxomApOks0AX632wX4CWkAxTSCPiZJidjr7OyMfAKm4EXBUkqlUiQSsVqtFFnRxoImlAIIzk08Hvf7/a1WC0QH3b2Hh4cA9NbX1wOBACUdlB/J6CtXrsDdS2soD+FyueiToa4EhM2YB5ckyWazQZ8EQz+miu3HDx0MBowL5Ch3u92pqSmz2by7u0u0YjKZQCxSlyVDBiSV+OBSOEOMYtapMozqXYCGTSYTGIpnz57Jsnzz5s0rV65Qe+j1erD9ms3mSCTy8ssvX79+/eDgQEwR42ywYeAUqMzA1cZ0vnA4TI8TJ0e4cZfME6wnAOrFihEB4MW3223Y6mgUVZ+Hu8h6Fa7dbieTyWKxKE4+ja+FQiGfz8disRs3bly7du3u3bvf+MY33nvvveXlZUmSTk9Pb9269fbbb9+4ceNXFXyUi4sL0MPCChD4kFUidQTtRKfTgaMRljA6mBkXc+3atVwul06nSdOB0wIrZ7FY5ufnj46OoHeidRNcCvAHngwxXVtbEw9K4YjhJZVKhZ8HAgHBrUAMBZBB1vtVoCiiWgLg2OPxbG9vg7EmiwbREsVETS/mwA6Pfy00GRsGL62maZCW46onEgnoihFxY4ymaRrr5vV6v/71r6NKsSyaphGR4GgSDlcqldXVVYGRSiQS5NWEZMs6G7G48BaAgVwycySMAFpOTU0dHx8bPXEhWIqioMgxEbjh5LTeeuut7373u1//+tcpKLFfeK5zc3OhUOjVV1+dnp4WAnNJpL5EN2BBCYkpLWuaVi6XwbURuwmPh+oHxQoxZEBV1UqlQulNkiQGEdDJ7vF4aCGMxWLHx8ehUAhGDbRlOBxuNBrNZlOWZVrBut3u7u6ugAGRtobqjfwb1ToYf6kZNBqNbDZrDGFowOJRaVnDj8HlV3WWfVVVqcNAsyZJEoqHQr0kSUhqJpM5PT2FiogcOj0FAORFqCWUitEiEBTfvXuX1g82qdPpML8ebDghMB4VWxgMBt1uNx01X5m3FBJM9lh6fqB6r9fDt8OcUVASSkvIK4gmnpmxOZubm/Pz83fu3CEhgtMsPT+PE1gYU4ZMhund4vrhD39IjUHB1aCdC0gWOW50bDgcNpvNJE4ZaCPyEdSJSTWRrAqHw/T+0mqsaRru0cnJCeYyl8sFg0GQooqiWK3WoaEhspdIJKkvfCAORCqVop+WG2I3B4OBYIEOhUK4vYpOaQf+aX19XRQ3lpaWgDvz8JIkEcHhD7HEPA99kUAFie+q1arT6QQ8Mzc3Nzo6uri4ODk5CR2DEKaePhfpkgSQ1Lh169bx8TFZNNqa+UNgbfDYmHXydPq9ILV/UaokvUeB3Mfp6SlKSzUghhVFgbhL07SJiQn69K06u64kSaAhON504Ozu7t69ezcUCrX0GY6XchP8EFT+i4pK0hvwX331VVLiCmRA9E+yW7SEYylUVSU5pigKnO/wyUqSROmawhnMd9FolE2ijQwXm5x4Npudn5+HDJf7kEGgmY4iIPRDMNyBoSDXAKM1vh2KhE5i6gyQe4N1oWZMLOl0Og8PD6mBWq3WaDQqBqdj+Mjmm81mgMvcjWpMuVymeUTTNIoHYlwWfMwkLIymSmQsv1IUbDbb9PQ0wGgm6amqigUwm82rq6vkBYhGEcdYLCbr3ckvmhu+ulQqwVPXaDTM+vAtEsuU4NAOJEKFdoETCqliSsPW1tbdu3fBJl0ymqp+SYZkmKID3S49lcPhAKuo4O4BC4HCQIzysdlsoDT7/T6IZrPZ7HK5Dg8P3W43skXTD7UgsMjj4+Nra2uIY6/XowubNgQIDhlxK3wj0JuYDFVVmWRMDxmPTi673W6HQiGaTnEj6DsFFTM0NBSPx09PT8XB6vV6lO6B4VLPoXwpsgzYNU3TmOwCIkPAnU9PT/Hk4OfA6rG+PINRjDRNs9vtpq+ap4UWoa0PSh9JL7TBJzs+Pk6lBVcasgZ2nSrIi94x8gfqcGlpiemh6+vrAl6GQCQSib29PQrelJugOhLIoqGhoUAgwCxZp9NJLfJSYcfo8nJphrLBi0dIBLBfQv6QkqGhIXpN6cAhzgSVACOAyWRyOp0HBwfoLWw8vUS0XhFyQxcTCoVozWZaM33x0ORFo1FKvLLeqit0OA4jkifLssPhwD6adZJ33gckO3GcpmnMEQXahkQqisLAX5M+0WRubq5cLpPVJCdn1jvERWsGPQ49nb6c4IjKJg+D+hErq+rjZYylmK9ccc4w8FogxVB2U/mRZTmbzW5sbOzu7j579iyZTB4dHa2trdEDc+mGPJvD4XjjjTfIM1mt1tHR0S+++AJrq6oq8kRrtSRJMzMzcJlSvZEkCaQ/fghcVCZ9PJb4FpqvjL6UZpif/c9fijGUBQwEGQGoc5ZSsFhZLBaz2YyVAe9rNpvJdoBAIlVxdHTUbDaR1Gw2C/gTNk6HwwGJD9RC9Xqd84f3OhgMoHXE12k0GsTnNG7DXoRBtFgsEDfQc4LPK8tyoVAw62wZtK0Cf8BEapr22WefMUiWdF+j0WCziQawgJyEfD6fTCb39/e3trYYCXFp7ZB1ynkmw2XWh0AZM2G8CPkRaG0dDgdpkV6vd3JyQqoMFNrp6Wk6nebo0kRutIaaPjhYuPCDwYBxWg8fPiREQG0DfaM5an5+Hpw3PobP55Mk6ejoiFEasiwDEzKqYWhz+HbZ0K/2z5yfXwqWeGhFH18hSRK0VYqiAPrp9/vMGwd/h+FjbqeiKFRRCEMEPenjx48lSQKJgLM1PDwMOSytEEAywJkoz7OjRKNROOMEWpCubdh/SqUSaVKn0wlPBEweAL8YWcBCAAzP5XJnZ2e7u7vMfup2ux988AFElQhBuVxG5ghTCEVnZmaAh8uyDIeFsCBiTfP5/P379z/99NMnT56USiV0MBfkH+LsstNXrlwBCO/1evGuaOhIpVIoBlnnoiVZTQWTdtZLFkooD2F6YKaAEkdELeQLGUzEGab/AH4skLcwSgCvOD09Fe355GUAaSp6k6pRK/8qweKTCmk6SR+Tp+llDZHKoxFZlmVGPDAGGFHAEeEESJKUzWZ5iGAwmM/nd3Z2+Hev1yMTSK8mzCLb29uEBcCURQitqirtqdVqFfQ3oRwdsyDBRRMOPhkQWzZmaGiIyqaqquD8mQc+PT29tLQ0MTHx6quvzszMUHeyWq27u7vZbFbTNIvFgn9DxECefWZmZmpqam5ujuYC6pg0V9Kmh+lJp9O0a4LBqtVqR0dHR0dHRLuqPmXT5XKtrKz89Kc/3dvbg2WUbCo8mqrer0YIgnWbn5/HA7uUyxABoNCIfEBE6CLAXFhY2N3d3dnZYapIu92m2VWSpGazSa8UhxA2MmZ88OeQAMBI/eID8MCSIR9hVGxKv98XnoSst4uQ3JIkCWAdNJiyLHMo0aIwLAJZRLagxgOLDdsOvV/hcJj70w2HkMVisc8++4yMtmgyEfJOCliW5X6/T9MiGCNQD41Gg7gGwVUUBfpNNo+uCorl29vbd+7cCQQCuDV04FCLoJZCZmRzcxP+WbhMOaaSzgnDHDzhM3COo9Ho6urqzZs3V1ZW3n333V//9V/H0CcSCcbGMm1FePS8i9Pp/Nf/+l+rqnr//n0qniQ1yBhxXCk5TE9Pj4+PQ8j7oq4SAZpwekT+AoQgnxwMBjwSo5AASoCPEozlQlYGgwEJVREDIg+iCi7rbTzGBxARsfFSVVXO5XICfo5zTad2q9WKxWInJyfk6xBw5qACDcBhHwwG7Xab76b8Alesz+fb399vNBo3b970+Xxgz0mTDvT5VTs7O6CKGWNEx6Ks11OhwZmYmKDiq+mDzkg9NBqNRCLBC1MGjkQikUiE3A+QURLxlOo2NjbeeustsqaKojDN2mw2b29vQ8D/9OlTVVVnZmZIkVPaIr9v1KZG6yYUBmCEYrH48OHDW7duAc6kR+3FmA4ZXV9fh4x5a2tL7BlnmMqjsDXyCykMzZDmEDqDOJFpjALSI+mMIOAvqGwCcGI9UfPiAWw2G4mb+fl5DrZsKHlh0IWPj3qGvlrcQbh9isfjYSaCpmmtVotuZuBgqAoSaxRYMC7klnBaJUmigxl8PtYTp4EhHxsbG6C1ZFlmYhaar1arMYGINsbz83OarjSdiQV+nHq97nA4AFywtdCmaZpWKBRknYIBJggiRLo5MpkMU/xAdyiKAjYV4aAVGyY74EC3b98GtqVp2tra2ubm5t7e3ubmJubMeEBlfXiu2GAOFXQp9+7dEwwAX5kp6PV6jUbj6tWr0Wj0Jz/5CTEECJaZmZnJyUmhpYz7+n+9VFV1OBy8sihJCb0lSVI0Gp2dnWUpsH2ynjYTQkwzIwGNkb2Nz5dKpePj40ePHj1+/Bhyq7Ozs6OjI/LJl7wuZX19PZ/Pg+xjS0S2EGccwQKqy4Ao+IAajQaBrqZpIyMjlMddLhfRGRHf/Pz8+fm5mG7FwLDJyclOp8OkXRy70dFRxjSIlC4ng8gRnCdDTcbGxmAlDQQCBCxIEh4oRMjkRZeXlwHl4WqsrKzQU8UeADxvtVpzc3Mej4eqPkB4qkyQFqNNjdlnIVuKYegjE1OAsh0dHTGnrv/8DC2jbFkslna7PTMzE4/H6/W6pml+v39hYYHjLRvHh/y/u4TC438FxEq44Zo+YJeKpCRJu7u7YDRI3Atpw6Dfu3fv6OiIqjGh29HR0SeffPLgwYOzs7PT09O9vT3QE4qisAuXfHkzI5Ypd0iShHvOWJVGo8EsKFmWyd15vV6v10tJh9yx2WxGYQwNDT18+DAcDjPzjsR9KBSiKwHhC4fDyWRyenoavo1KpTI0NESMzUCiw8PDmZkZTW95wK/CUG5ubt6+fRsmHKiLTSZTPp8fHR2lzkPugFjk+vXrvV6PeZPoA7/fT58FsHdVVcfGxra3t0F9aHpGAPQp07MDgQAZVLxvDjerJj5PWR3CVWiupqencX6N+VLhAxlDxX6/f/Xq1a2tLQo4wuP+ZwTo0uZd+l9Vb9UUHUF9w3BQTA0vUi6XcY6pQONH0inEAe73++vr65AJgHxhtvnQ0BDnjT5QomwSCAO9cYYHUywWCwNzgOPgmvh8Pkq2rJGmacVike3hzyA+pBelVqtJkuR0Oq9evQraHRQXFVxgDoqipFIp6LWSySRd6oK4gso5bVuibYEvggCTihuUWkNDQ7Ozs9VqVVVVKiSoJWgdmZVFtOXz+Ui1Y7lAxLN5uPl2u31tbU0UMbBoKFcS7ihscZqFj4VS6ff7ECeLyv3p6SmoDcjANEOJ7UWJwc0YGhpCqkDEX8JOXfq8UTFc+l9xUc+BylHTNLvdXq1W9/f3Dw4OCBEYK0RuIhQK0aCRSqUQA953YWEBTU+q5ejoaHFxcWxszOPxMEENCmA4oQG/X3psM74U9LcAnijjgDrCg8MrhxFQqBOUAQMUUqkU83OWl5cfP36MVh8MBihbUsDEnuFwmM2gDxbSZd7B5XJNT08bFwvHLhgMHhwcrKysCEPGqBlSo4VCAVcpGAxubGxcvXqVA0clCtT12NhYu90WXFBCLAQJscARkOAl4yA97zdUKhX8evFseEX4djh5i4uLEKIQKYuoHko01LOkWy6kDeAhAo0chEKhYDCoPo9Al5734r/StRd5DbPZTPt4JBK5f/++JElg6SjZwYEoSRLtVTQSC8ec7DQj1g4PD8fHx9Pp9MLCAuk3SqWqqsJhyQG2Wq3GFC4rbC4UCjAakkTWNA3GFeSAoZgkwek3JJSQJIl8bjAYlCQJrz8YDDLa+cGDB+DzMRa7u7tLS0vBYJAHRcGS4me6PZJhdJDFqqmqypC3SqUCbFzSnSSLxbK/v8/3QokOfouCDLcC2xSJRMQeXNoeGgNFAo9lEhaEj9FngV5UDYgGTdPcbrfH4+GpAD5QMuJWsA3gCTCPSNGpO3q9HvJECZm3Bi2oPI/2lJ4PA4UAXVJUqgHdgElpNpt/93d/ZzKZGFcJpg8QmKyXSgVJhN1uR8cDyTw4OCDef/r06czMjNlshteEBKfIl2LuNjc3wW2replS0zQz6E1N05rN5snJSSAQoOAAapTUDozCrCzPLW5tNpsFFg+KjnQ6PTExcXBwMDMzY7FY7HZ7p9MhimQEciwWIwyEZFdVVYhJyPLLevaZ9RJVJpfLpenJCH7udDoTicTa2trGxobX6z04OIhEIk+fPo3FYog7SSkGI5KJECGx2CfRNKHorXy8lDHTjc/r9/sv/a1koJ7S9MQ9K4OTSp0Y7SWq1JqOmmJYMAG4qk/Pok3D2Gpx6R+XRE1cIoSUdVDXxcXFlStXqHe1222AQwi9yWRqNptUThks4Pf7aZVuNBpQs5L7YKCmxWJBqsQKiMAFiAAqUAReiqKYP/7443A47Pf77969y/CtUql0fn4OVKFSqcAa3e126cUGaUnpRmwM2dEnT57UarXXX38dLCgSxuivYrE4OzuLfQSVtb+/n8vlmCfDrojTJhSDphfFmPsougbE+jJiGEvEXD8maYk2GEVR4LJmdKD0QlqIVeAJW/qIUWIRJr9pmobXcukPL2kRWUflE6vjzmPsTCYTzfhID9GAw+FYWFioVCqwWiiKYjabGWArcg3Gb3nxHy8KlvF/yX3QL1mv16neWq1WsfcQ5p6fnw8NDU1PT1er1S+++OK1116DMh1jOjQ0dOXKFU3ThA9gPPDsfiwWA6FpVAeyLJsBnLRaLawmdhS/mJ9IklSv1xlVrSgK5T/sKzS1+O+08Vy9ehUmj6tXr96/f7/b7cJSRwDC/dPpNHCR4+Njn89nLISphrYZTW8g4Q8vqRmTyVQul2HzJsxhWmSj0YD/CDHCIatUKsxz/8pdAc0iqhYmk4mpITQa4f+p+vCLFwXLuLWdTgciP7SX6Fuk0/+SV0RNAjfI7XYnEglOlKSn4F8UKdmQhPxVEsalKApJDavVyrwuZKXX67ndbpo1BPWLqqq3b99++PDhF198cevWLU3TwLhKksT0BkKZS68v6/VvYzwo6f6fmQT0wcGBmOND8hqaWkmSQL84nU5JkoC6G0v30Nd+/vnnU1NTb731FtG+pml+v//mzZsffvhhPB6/du3a2dmZIOYPh8O5XG5sbCwQCJyentIOL25IcbTT6QQCASYJzM/P03UkP5/7FiVOHqxUKuEjU62jYw4DR8af9IeINyUdGXJ0dMR9PB4P0CVo3EwmE0kWesWM2/aVm4rrPTw8DP0EJWS41Hd3d5m1YX6e2gVDydSdubm5nj4DV7gcqEz0mfxC8eSSZTc+CXqRwwBJDseDnB8TD71eLycH4OT169c/+OCDvb29ubk5r9fL/GkUzSXTbBTuS9ImJMxM97PVat3e3l5aWgJ9hjFutVq1Wo3TL/IOPDG1nUgkMjQ0tLa2NjExcfXqVYZpkTvFVkKGMTIyAooGJJ3H4yERAm06M9++ROAriqZpMMTRtB4Oh/f391HpxqqZpmf8aO6AyYhqQzweJ6oSbhMI0lQq5fF4CGyFPQXar6qq3++H0wyAwNHR0fz8PC7tp59++uqrrwpKXMqOosHcuKOYMwZe9vt9hmUWi8Xj42N6ZeESN7r/vV4PrEG5XMaP6XQ6VEKpFsuyTNlRVVVcEZvNRg0NbWF+gYdI0zsayJ7Dd0fUmc1mf/GLX0D9srS0RGceC9Jut1966aVf/OIXtCJDJ4a3/eIp+mcuntmMX2Wz2fb29lwuF/lP0XpKSzWDd0G487+Uoh0Ox+eff35+fj41NbW1tRUIBGB754ehUGh0dNTj8fzDP/zDd77zncFgcHp6Wi6XQTuRtx0dHT07O2NAPL1TBKcHBwdjY2M3b97UNM3r9W5sbEBeoOk1V02vNWmaViwWeTBVB5eK46UZqvSBQIAmW8WAfqGJQ+SjNU3L5/OZTMbj8RwfH09OTk5NTREc3blzh+QLRQgRPwoTKetAXsJ1zIQkST6fj7FqoK+M3y5cYIwAyepms9ntdq36pExJksBIipeC8QD+Vapql24LBJeaN4wjXq+XeWP5fP7111+nOoLTYvQ9LBbL9evX79+/T90asBPwiku0AMbrkjv/pcZaWlpih6anpzudTiQSYe4yGOVwOFytVontOQQ06PX7fYZX5fN5l8sF4b0kSaT77t+/z1CkGzduwNSDQ82ISgBu8KKI4TZEuXTuX1xc7O7ufutb3+KsWK1WeDs1Q36LA408iXy63W5nwsUlrYBbFgwGASvT3s3dRKhIdhQCReHopNNpu90+PT398ccfo8XREHa7/fDwMBqNXqoWi4iVb+S2gBOBVRkrj0gk6H76p1lYn8+XSCQkSTo8PIS/idQ57hocJJC1ZLNZXAuReJP1Oma9Xp+ensYBheyEAPn27dsU+4WCv+QbdTqd4+Pjcrk8PT1NhPjRRx9985vfhFLgK2VLfLWiT49WFOVL5uBarcaoRZAOKGdwL4lEIhgMwinIhxlLwQmLxWJgiCEfM5lMBwcHZCk//vjjK1euHB0dvf7660+ePGFOBtT74XDY7XYDmKGPilSbJEm0Q7366quA7vH6ITYSTiIpIjzuarVKCwASwE4LL9DoL8uyHA6HC4UCWlnkNTS9E3p3dxeYkKz3CzCvi9Gy8Gnz+mQKSPyI5b4UJ2o67ARDdnZ2xpgx8Uhgk0gNUDbQNG1oaAigMPAKgR9hBZgrQzZEVVW6VLa3t0dGRmAwEKXx+fl5HpVBpGdnZ4xoAGOYzWbhAWVJyQ8TXMMBE4/HmaesKMrGxsY//uM//tZv/dalItWLp1fTJ41LkmSWJGl4eJh5rMvLy5qOKHc6ncyP8Hq9nBtV78iDUG9jY4POk3q9Dk1voVAgAGEaz7vvvkvRkAr3p59++sYbb1DiZYSTz+ejQZSSsMVicblchUKB2RAiIW702dH59Xp9f39fRDooS7jEEbtLeyxe3mq1ejwe5gIJgSDLB2wGxUDmiVlwpIVtNhtgpsFgkEwmBfpDJPQvmQbWinmzyD0BuaIPL4X0QdO0eDwuerKDwSAKm+Mty7LP5wMmZLFYyERomoahxO4zVQRbCZUGXh0x0KNHj9iv+fn5ZDL5/vvvoylLpVIgEJiYmIjFYmAO6Ku+f/8+hYRnz54NDw+zCysrK48fP65Wq5DE/DOhsWoAbH6pYOPx+Pj4OC4OgE/40/DHYb8Qpw3MCYqkWq3S+EBlem5uDujI8PDwyy+/fHBwcHZ2BkLmJz/5CaefNny73W61WgOBAK2knAZ+iKtBRxpPyRQMcD/lcpkxvkLmVFUltYZ6QzKEByN0taQ3CjPm06JPhW21WoeHh7jtHESTyTQ5ORkMBrkhjh0KNZPJEJqQlcBpM5ZfxPGVJIlEjDF5iF2rVqvJZJLyKM20siwHAgGPx3N6eiqc5aGhIZ6BEq1YDZvNtrGxwYwPSZLwSR4/fry6uur3+/FkkN3XX399bGzshz/8IbS/TqdzY2NjdXX1nXfe+eijj/b29v7qr/4qGo0uLy9/8MEHxWLxa1/72ltvvXXz5s3/9t/+28bGxtzcnNVq/fu//3sqg4KB12j3Lx1gYeKVTqfz5MkTwE9ms/nk5GR/f1+W5StXriwuLgJJEABCjL2mablcLhaLFYtFSZI6nc7jx48hvSiVSmtra91u9+2332amF4QOZrP5zTffZPMgsaB0TSeZYFkm6jGbzaK6LPx0GjoqlQp1CfwSQnrqiSDdxDzcSypEXJIkCY+EX5GqEDGvy+VaXFwcHh4mtiIMnJmZwVrxyjj7fr+fYteLGotFn5qampycxL+W9WoYhOSqqhIzisqsLMvMl0N0IpFIOBxmBQZ69z3bdnZ2xvRkYEsPHz7c3d0lxL537x4zqsHHDg8Ps0T/5//8n1Ao9Nprr6XTaavVurm5yTq88847v/mbv/nSSy+98847v/3bv/3OO+/AGPDtb3/76Ojon/7pn/7sz/7Mbre//fbbNKUJsJAQI+1XpNPMHX2CN/XFBw8ezM7OTk1NMbAepBElF/Qwox9feeWVTqcDXcf09PRv/dZvffjhh7g+BwcHf/AHfyAEMZFInJ6ehkIhsqkEaHQjUtWmM4zmHPYVH5ZR0MRWaAjaNxgVaTabsc7EMpqmpVIpKLXI317aaeP/kl4Rm032n2q6zWZjiqwo+RlPIcT/uDLwkZ6enhaLxfHx8UsAYuG6jY6OwmdBlMd6UmtyuVzHx8eYLcposiybzWbmqeBIiaLnLzfMbM5ms9hZpnzfuHGDQa9wnlNw/Na3voVB73a7YBiZa9xsNhlMAp+H1WrtdrvM65MkicN8dHSkKMrc3Fyr1fr2t78NSa6ocmqGROBXuhxf/uTzzz8n98oIQuIFjLrVaoULhGIZUSFB+zvvvHN4eJhOpxnvNjk5mUgkzs7OHj58ePfuXYqAks4QBOX1+Pg4FWWbzdZoNKgeog6BGIADliQpl8uBeYXAjUDs6OgIdw0gKBgPznEymVxYWGBrZ2dnBTTgKy80EElIo9ygAqnA/Kq/TafT9KqTWqSFOpvNUhW9tL6yoT3apHdg7+7uImRUdjc2NlR9PptJZ/wSAyOMkYe40KMHBwe4gLwISXbw4gCkjOVONA0yR3sZNxFSK5xCvgtQ6/r6uqZpq6urxG1GIyhq9pqObxZiJ4laoaiAcp4gFGWTcA7oQ6JGgcbGWqmqmkqlMpnM8vLy8fFxMBj84IMPoKuDrkjk8VdWVtLpNMUpSmYQuEG5RIqFXkoaQqjM2+12GqmDwSCDuCGrgBNb0dG3ZrMZ13VxcZHw7UWpMv5EURR6HCBfkAyn8EU3/NKOwuvHyuDbQWlRq9WQ5ktuhzCvqj7RBJIBq9UKek48Es9D0CeSauJ7L91TURSqy6oODRUJNtZcMkBoJN1wsxeC4E/RMezif8X9iTPef//9t99+25gh+2euS16mJElmAhZYzpB6fu33+0VBlDZAAjdGZ0mSNDY29uMf/xhI4cTExF/8xV+cnp5CqKcoCl1EYK36/X4sFovH45IkiTwv6YZqtUoJxefzQQPJwXW73cS9JAwh7VAUhbCxr5N4I21er/fs7Oz27duhUOhF2yE2SfwcTUPboNiwX+WWivWS9RYmnB5+hYHDMbq0B5f0DQIBX6tooDXpTaehUIgjdynselFdcVsRu1x6eFXnzpSet1MiW/urYroXl+Lf/tt/K8pi2lcVjqTnswySHrN/GRUK9UXeEm1JWwTjiikOCmQBOQiGW9RqtYcPH5pMprm5ucnJyatXr05OTnJf8BiRSMRsNpPDpV4hFC/ORD6fr9frJJGZyQaVg8fjoRQID5iiD3VCRZFo1fTWuXg8Pjs76/f7SeJdqhZfWjvcI/pLqa0aN0z5qmFX/4yvKssy/SO/aukvyQRJeUoxw8PDBHE0AIvFMf7Vi9pX+9UlQumrfB1JN1j/zLNdulgEUUMzFphf/OSvusn/Fz2jZjbuvcgFAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ], + "source": [ + "images = [downscale_images(image) for image in retrieved_examples[\"image\"]]\n", + "# see the closest text and image\n", + "print(retrieved_examples[\"image_description\"])\n", + "display(images[0])" + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Saving, pushing and loading the embeddings\n", + "We can save the dataset with embeddings with `save_faiss_index`.\n" + ], + "metadata": { + "id": "6JEZJlkD8UrZ" + } + }, + { + "cell_type": "code", + "source": [ + "ds_with_embeddings.save_faiss_index('embeddings', 'embeddings/embeddings.faiss')" + ], + "metadata": { + "id": "dXrBMAHx8k51" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "ds_with_embeddings.save_faiss_index('image_embeddings', 'embeddings/image_embeddings.faiss')" + ], + "metadata": { + "id": "51dgxmGm-c3x" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "It's a good practice to store the embeddings in a dataset repository, so we will create one and push our embeddings there to pull later.\n", + "We will login to Hugging Face Hub, create a dataset repository there and push our indexes there and load using `snapshot_download`." + ], + "metadata": { + "id": "xO0i-dkY-nK5" + } + }, + { + "cell_type": "code", + "source": [ + "from huggingface_hub import HfApi, notebook_login, snapshot_download\n", + "notebook_login()" + ], + "metadata": { + "id": "ETmGo_KiAiOr" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "from huggingface_hub import HfApi\n", + "api = HfApi()\n", + "api.create_repo(\"merve/faiss_embeddings\", repo_type=\"dataset\")\n", + "api.upload_folder(\n", + " folder_path=\"./embeddings\",\n", + " repo_id=\"merve/faiss_embeddings\",\n", + " repo_type=\"dataset\",\n", + ")" + ], + "metadata": { + "id": "K3hmtWQn-k9O" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "snapshot_download(repo_id=\"merve/faiss_embeddings\", repo_type=\"dataset\",\n", + " local_dir=\"downloaded_embeddings\")" + ], + "metadata": { + "id": "UTVoI9LWBp1x" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + " We can load the embeddings to the dataset with no embeddings using `load_faiss_index`." + ], + "metadata": { + "id": "HGkYTJsM9BVx" + } + }, + { + "cell_type": "code", + "source": [ + "ds = ds[\"train\"]\n", + "ds.load_faiss_index('embeddings', './downloaded_embeddings/embeddings.faiss')\n", + "# infer again\n", + "prmt = \"people under the rain\"\n" + ], + "metadata": { + "id": "mbPvs8kV8xTy" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "prmt_embedding = model.get_text_features(\n", + " **tokenizer([prmt], return_tensors=\"pt\", truncation=True)\n", + " .to(\"cuda\"))[0].detach().cpu().numpy()\n", + "\n", + "scores, retrieved_examples = ds.get_nearest_examples('embeddings', prmt_embedding, k=1)" + ], + "metadata": { + "id": "mc9JmZSG71WZ" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "display(retrieved_examples[\"image\"][0])" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 341 + }, + "id": "wckNsAX-9zox", + "outputId": "8d5008b4-ab8f-4b42-92e7-b29e57c126cb" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAFECAAAAAD0d5MjAAAKMWlDQ1BJQ0MgUHJvZmlsZQAAeJydlndUU9kWh8+9N71QkhCKlNBraFICSA29SJEuKjEJEErAkAAiNkRUcERRkaYIMijggKNDkbEiioUBUbHrBBlE1HFwFBuWSWStGd+8ee/Nm98f935rn73P3Wfvfda6AJD8gwXCTFgJgAyhWBTh58WIjYtnYAcBDPAAA2wA4HCzs0IW+EYCmQJ82IxsmRP4F726DiD5+yrTP4zBAP+flLlZIjEAUJiM5/L42VwZF8k4PVecJbdPyZi2NE3OMErOIlmCMlaTc/IsW3z2mWUPOfMyhDwZy3PO4mXw5Nwn4405Er6MkWAZF+cI+LkyviZjg3RJhkDGb+SxGXxONgAoktwu5nNTZGwtY5IoMoIt43kA4EjJX/DSL1jMzxPLD8XOzFouEiSniBkmXFOGjZMTi+HPz03ni8XMMA43jSPiMdiZGVkc4XIAZs/8WRR5bRmyIjvYODk4MG0tbb4o1H9d/JuS93aWXoR/7hlEH/jD9ld+mQ0AsKZltdn6h21pFQBd6wFQu/2HzWAvAIqyvnUOfXEeunxeUsTiLGcrq9zcXEsBn2spL+jv+p8Of0NffM9Svt3v5WF485M4knQxQ143bmZ6pkTEyM7icPkM5p+H+B8H/nUeFhH8JL6IL5RFRMumTCBMlrVbyBOIBZlChkD4n5r4D8P+pNm5lona+BHQllgCpSEaQH4eACgqESAJe2Qr0O99C8ZHA/nNi9GZmJ37z4L+fVe4TP7IFiR/jmNHRDK4ElHO7Jr8WgI0IABFQAPqQBvoAxPABLbAEbgAD+ADAkEoiARxYDHgghSQAUQgFxSAtaAYlIKtYCeoBnWgETSDNnAYdIFj4DQ4By6By2AE3AFSMA6egCnwCsxAEISFyBAVUod0IEPIHLKFWJAb5AMFQxFQHJQIJUNCSAIVQOugUqgcqobqoWboW+godBq6AA1Dt6BRaBL6FXoHIzAJpsFasBFsBbNgTzgIjoQXwcnwMjgfLoK3wJVwA3wQ7oRPw5fgEVgKP4GnEYAQETqiizARFsJGQpF4JAkRIauQEqQCaUDakB6kH7mKSJGnyFsUBkVFMVBMlAvKHxWF4qKWoVahNqOqUQdQnag+1FXUKGoK9RFNRmuizdHO6AB0LDoZnYsuRlegm9Ad6LPoEfQ4+hUGg6FjjDGOGH9MHCYVswKzGbMb0445hRnGjGGmsVisOtYc64oNxXKwYmwxtgp7EHsSewU7jn2DI+J0cLY4X1w8TogrxFXgWnAncFdwE7gZvBLeEO+MD8Xz8MvxZfhGfA9+CD+OnyEoE4wJroRIQiphLaGS0EY4S7hLeEEkEvWITsRwooC4hlhJPEQ8TxwlviVRSGYkNimBJCFtIe0nnSLdIr0gk8lGZA9yPFlM3kJuJp8h3ye/UaAqWCoEKPAUVivUKHQqXFF4pohXNFT0VFysmK9YoXhEcUjxqRJeyUiJrcRRWqVUo3RU6YbStDJV2UY5VDlDebNyi/IF5UcULMWI4kPhUYoo+yhnKGNUhKpPZVO51HXURupZ6jgNQzOmBdBSaaW0b2iDtCkVioqdSrRKnkqNynEVKR2hG9ED6On0Mvph+nX6O1UtVU9Vvuom1TbVK6qv1eaoeajx1UrU2tVG1N6pM9R91NPUt6l3qd/TQGmYaYRr5Grs0Tir8XQObY7LHO6ckjmH59zWhDXNNCM0V2ju0xzQnNbS1vLTytKq0jqj9VSbru2hnaq9Q/uE9qQOVcdNR6CzQ+ekzmOGCsOTkc6oZPQxpnQ1df11Jbr1uoO6M3rGelF6hXrtevf0Cfos/ST9Hfq9+lMGOgYhBgUGrQa3DfGGLMMUw12G/YavjYyNYow2GHUZPTJWMw4wzjduNb5rQjZxN1lm0mByzRRjyjJNM91tetkMNrM3SzGrMRsyh80dzAXmu82HLdAWThZCiwaLG0wS05OZw2xljlrSLYMtCy27LJ9ZGVjFW22z6rf6aG1vnW7daH3HhmITaFNo02Pzq62ZLde2xvbaXPJc37mr53bPfW5nbse322N3055qH2K/wb7X/oODo4PIoc1h0tHAMdGx1vEGi8YKY21mnXdCO3k5rXY65vTW2cFZ7HzY+RcXpkuaS4vLo3nG8/jzGueNueq5clzrXaVuDLdEt71uUnddd457g/sDD30PnkeTx4SnqWeq50HPZ17WXiKvDq/XbGf2SvYpb8Tbz7vEe9CH4hPlU+1z31fPN9m31XfKz95vhd8pf7R/kP82/xsBWgHcgOaAqUDHwJWBfUGkoAVB1UEPgs2CRcE9IXBIYMj2kLvzDecL53eFgtCA0O2h98KMw5aFfR+OCQ8Lrwl/GGETURDRv4C6YMmClgWvIr0iyyLvRJlESaJ6oxWjE6Kbo1/HeMeUx0hjrWJXxl6K04gTxHXHY+Oj45vipxf6LNy5cDzBPqE44foi40V5iy4s1licvvj4EsUlnCVHEtGJMYktie85oZwGzvTSgKW1S6e4bO4u7hOeB28Hb5Lvyi/nTyS5JpUnPUp2Td6ePJninlKR8lTAFlQLnqf6p9alvk4LTduf9ik9Jr09A5eRmHFUSBGmCfsytTPzMoezzLOKs6TLnJftXDYlChI1ZUPZi7K7xTTZz9SAxESyXjKa45ZTk/MmNzr3SJ5ynjBvYLnZ8k3LJ/J9879egVrBXdFboFuwtmB0pefK+lXQqqWrelfrry5aPb7Gb82BtYS1aWt/KLQuLC98uS5mXU+RVtGaorH1futbixWKRcU3NrhsqNuI2ijYOLhp7qaqTR9LeCUXS61LK0rfb+ZuvviVzVeVX33akrRlsMyhbM9WzFbh1uvb3LcdKFcuzy8f2x6yvXMHY0fJjpc7l+y8UGFXUbeLsEuyS1oZXNldZVC1tep9dUr1SI1XTXutZu2m2te7ebuv7PHY01anVVda926vYO/Ner/6zgajhop9mH05+x42Rjf2f836urlJo6m06cN+4X7pgYgDfc2Ozc0tmi1lrXCrpHXyYMLBy994f9Pdxmyrb6e3lx4ChySHHn+b+O31w0GHe4+wjrR9Z/hdbQe1o6QT6lzeOdWV0iXtjusePhp4tLfHpafje8vv9x/TPVZzXOV42QnCiaITn07mn5w+lXXq6enk02O9S3rvnIk9c60vvG/wbNDZ8+d8z53p9+w/ed71/LELzheOXmRd7LrkcKlzwH6g4wf7HzoGHQY7hxyHui87Xe4Znjd84or7ldNXva+euxZw7dLI/JHh61HXb95IuCG9ybv56Fb6ree3c27P3FlzF3235J7SvYr7mvcbfjT9sV3qID0+6j068GDBgztj3LEnP2X/9H686CH5YcWEzkTzI9tHxyZ9Jy8/Xvh4/EnWk5mnxT8r/1z7zOTZd794/DIwFTs1/lz0/NOvm1+ov9j/0u5l73TY9P1XGa9mXpe8UX9z4C3rbf+7mHcTM7nvse8rP5h+6PkY9PHup4xPn34D94Tz+6TMXDkAAQAASURBVHictP1pzGZZch4GPhFxzl3e5dtyq6x9r+ru6oXNJrspbhJXi5QoUbZ2aCQBWi1ZsmyMMMYMhDEwwPwxNDDggUYeGAMYtkbw2B6JQ0kkREpks7vJ7uqtupau6uqqrMyq3DO/7V3uveeciJgf9/0ys7YmxRkdIDPfu7xf3u/GiTgRTzwRhw6wGWxz6nB42E1//fIf3zpEVUdykIWBTs+Mnd2Y+/UikJOQZqPgSTFNIVc6xHMW7ShzcgtUpOHKHKosGjkXFooFAiUBkxuYCQ4mdyYYxBWmDYCQQHC4k5gHV4IRwYhJsThzWCHmwuQk5hpLe1g5xVuw3BqWoaZ1mL9y1rb7/bPcs5QYh7g4tWyW9Rmtuqtx6cpZOCmTxX/1w7UrcyrP/8yRVYqwv7dq0Gs9BFlLo1yLcdUsp6hzdJ1222FxCiRlgjxJGgvEJDoN1khBKM7k7ibkBiLX2pZ1PJypgr0UdTV1x7ry5doDC83v22WDAwAcshGDA4TxDwCMbwMO/pBjP7kRDgeNxwTY5k64uwMgd3M44vSr/0M4+enEBKeccp1f/PH6MCgRAHdyMBMTE9iGPgVyJy9JiOEOrCYIRu104ta5F4MVeD2tqpxyMceUiiaDA05VHTQ4SLQvdPKkkcFMRsw6mEcFHHBY5SpeyIPA1CyXmK8c15WqtG1gJlpvXa+2Rfo3jtaEZdNS782w/c+++qe3r1z80t/rSwxFJNn0K7PbT6wfnq4+/9anbnOVOw/mZFtf++1PrqqMfvJr21dLEWLqV92a0G+XHsQLhXhBKLMbiarcSomvVRYvX/qhWZkoV+fOtMm0rKU0Vccx9630ScQJqjF2R3mA4r7d5p2hG8zcnCWGIKyFREph5e64mUk5EeXdce/nO+f8ex+/+8zJrCEnZ3fbXCM4wczSXaGLO1HurHrnxrOUIgFEcBOjKOxgJyrDoDAHXEsggjmohHwUaVYJrzt1JYMDoWnUTQ1STdVZ4O5iHrZqZSfGkLM64A4KHIQjQ2woPXKAuwNEFKGKAqqiFk2pUy+l1f3ToZ5Ng3uuJ2+cmxai7kYG6aQaJEB9+sL5c9Z2Xw4uVRCQtOnTv/ngx48nK//v/uaw56g0SyQUevH+KgVfV/UbfyYEc+cy8ZhC5BZE0RoFQ8EtgnCK0eqwbpbyK5+y9WJJLu2ZpizXi+O1PfWwhOkq5IObi+b+c24ItNq/uUqOpsHqZuozCZMKiGCEIqEqhTj0B1UTNbxPeu8XPPm7T9x7/GHT5Y5cyXhUo9EImNkQ7poJdqKcgn/tqdDXmXz8yaQUA7u5AWVQTkRMTgFEZg6a5nzpiWpSD6HrnMAAu5NrlwZVYtGhG4q7M4hjFYnAVLQUBuDO4VRVwYkQu6GUCCU3BzHAgMGpamLvpimsaX7YN2dQTVsx89r6f/R3JW0vr7vurm3vygTBB77+2h/1o/nex69vhyaoeUi7exd+YR4W03+8d/aIjmJF5EaE1bf+46pRivHC5P5lYQtA7FlC4n45Me1rwMFE3Nup4ykPOlnX671v3fjxvkw1xP5WUx2vUupqslCxyI2DZV4dNmdFQ1y+fS0Tk6erN5ZBEZkZgUSYmUwpRCnujNXhfP6Bwrr32N9zw3uP6c55Gq34HSW/e4ePqk5OgDnuarpbACGZlOf/4tolMwA4ESzW7CA21T5ZcA5shugEUyfwwq8/VzVVb0lls9YQWcrFzF0VWtTNzY2rCPUcoN1qcHYAiE0bRB3uR3kVCiw6yIkAKyhmLoHK0Oc01KFbhpksYxO5GHEKN/ghX6/2r7PlMvvW238gDim0z8v2elaeeYI0qMFc4kf+z4+cOuK4/z//+a45busShbKFcHAUKSw9zr7xXAxaJTOyEtks5jC90s4Ybu7mqV2Z0jDsFMSh+qWf9hW1hMwHRtmVgoT10Wk97N/oqIndrTefEt2//Pa6hjNZJ5zd4aogMrUYhNydg6IQ59WiqQyge6X4Pru9kax/r2NsxH3yowigUdijuP3u7QT2u+bdQNBEfq17fMlDBDHUwXCp2BhMpe8yKDATzAUwNSIM4ajCKjSxVyGDiruHKFbUiSFcihGzQxFnU6FgzDllqlTB5lKbmoPcVlymQ8eyeT4XNXejGKykXBQDxzhdbU1y04RCEjL96uP9sQw3+2amTfPbPzBNlbBc+wQPsS/tao8MgSg+9toX/ylL9JfkqWYtg+dMlFV5+jNFFiz58M0fPDC3IoyuJdJiVi9eemySyIuRa5Nnh6e/ed/2UGR9/l/c+tzRxIaJrrZy7zGYc5Z9X/bXBmNfmZSrszpdu5FqEnPTEprsDhcedcjhcFOqoClHLsezaU/w9xjzd+nyRpT3aPT7rcFG5vTeaycSB9zpRPQudGdJ4UKEkoi/9ClhgDgwTJnJJcLI1UqfiCyqCYk5TNWJXKdf3dahG+o+syncDFJVlJMakYS46AZngJxCJerMKCmrszvcuWorOOB5sHQwCwMV3szj4EqiXHNOSiAJxxRf+a+e+ctP1VSKu6X5N3+8Ofb9YyHO4SvhUym11g/8FMc8Kf1ODsRBPAz/9Wdm+/Vi/5uH7Y3JTBt3EVYm/Ac65CD2xtlzrITJShZXP6pWJibVleM9VnFAoPXhvH/78mMxcaD8b36h6h05hyDsxUQluA83jjybBS8UaXil8UUfyQqcBTB3dxhRADvgrmSQ4M4ZlS8XO3jPev0Bazq9R7Pfe/zur94z/J6zow8PuBGdLOkQgKAF4Xd+KJs2KiIwNXewwBwp52wS2PNQOAoBbk7AMHxRtaGhrHorxmwuVVNZb2ZqZs4sQYiobtomMkCWczErIiIh1q2qGWBl2L+4b7ECQMxEhJzU1Kj0fTFT72IcYjj78LSCcxA4Lvz2rWG96i0vSvqV7++Pg61l9sT53NjCJ6thGJKZ46VXf+HFo2HRP/Uz/en8zs5ByVlBXlJolzsp771yf4hDf2whrK6AnNZdXl7HuXVKOaWc03o2nPrnp08tmqzTX6OfWbeHNi1WZ+UoMM3utRyvRdjQVLnLvlxKo31hJgiXtYGDMFzNTNPQq7u6CCorhG7ZO95lfe9IjzbjzqkPOfbNeFeUd1fkvvnWGIrBfYyXT74syuib8urW+b40cLB7zJE5TbSDDLJcSSjFIw07/QCKDBAshf3X7kMxPZRoTAoBSvtO1ROxCErfGcGZkWIrJbs4l8WaQyllCNimoh4MPiy2//V67pyCkyGazsJ/+paY1NppivjngpCNz/5f/0aO5O5ekh39737qkPwmaWyab+Pj3ayQC58/hVTNQ66q4kEL4Ys/Mjm+9saN1d4vOl3Z6oLVGt3AE7T35+3t4e0fiNibn67iqXd+kCriZuK7r37U9ma783q6vXduDztfePwz6dwsbskrvyg135/bh3sLU8AsiKunHLig5OJw82xeTBhFiRRRVFXdMcbubi5ErIPVEGRU+RoZRJhtE06PEjtRy4303O6Ok+OTi7yZB+bm7pvr9/wsdzcfFxAid1ezyR3zbpAi6CZfeyAkTx5IyZpIudsKvlW6dq1uYHZ5/vz5Zq21mjnM1JfTmrgNQzEAcA2TU//TOcwGd3KYmm8mXRBzQUHORkbULOuCaUVsaki5Orr1oyGpmQGuTDo8N+uLDCENk/LVxrOLWWVpDjeHlSG1n0ravwWPcnT6139maJv1sN3fmvemTMriUke2sn6iK5GNh6rwcPNpCUhNWxv23zp8o2uqWXXw9WFdmea+/fXPlrJOXWuz1679v8P2mZ0Y08FShwd+Y/IdMWmUL/LLdL57TeaP//DtrRNzSUzuMDZX+InqAbwRHW/UTQjuIC8Y1bURZXhv8x3KtafwXnP9gQ7dB1z/oKj+3nEnRqcT7Ib8rtBJMmmqXvwFKJkHApebqD22RRtYXhUDMXn7rUctiqqZO1Q1Hp2bgBrLagR3kqT91/7zelCQA2YKhzsoVsE8IGPoDUasJLaNgWOG+6qbvLx8aG3OhVS8xKjxJ04rgplHT1/4uUSpVg/F5oAbdOi0AIkzrErNK7d+4Hg5bShxmx1EJCH20y1RF/ns2pucgi+ayQsvbb32+mHZClbkxvrRyZ4fXn9Fv7sss5r4zOFTP6RVtO2j6rvy5O1ix1dX1aytKv6d+UPYA+0tyo/wUby4s7P81U/+wd20eZHKDHcQkynADLg5AN6om7g7gUgAh426CQKEPMdAeX8rKLzE303Iv++xWdI38ie6V+hg5VLqK+W53kTchbn6rScnVRMb7jge9QZjIqRb92cXGciMzM2nhw/WhWntG42OduZfbJ1eZzDB2Q1OcCOextGxTP3gZCSlKrI3lFlRJu0yvvVkvDVDXNZObm29Lg8P5vB1xfYNPNCTK1Ghts0EzyVprA9Ir0kB9Wf+mx8zm4flrF5b3Yqn7NqdiYfXr+yv84x6Ori5n9P6yW9uHU1+YLIutz3ev/XQ1KvDef5vfupBA7s39quf/OjK2MAPXHzkp52scqtZzfoX/9aOTdBZMLaoUq9FZ/GIRn8KjGJOwpsw2Hmc47A7kfPG8SKQuxuPqBcUnixUvFpuR3V5n8z//63pmwDeCWJ3hE6Sp8s0+dbZvSsRJmC3+NKzxDariIvmQgAcfMO2D0olcANgTtWtB4grDEKb39GaL/6Z43UoTAqMc9td4lycYZA05I2RoRZV5URkudTdK383oVqLM5FzYEu+msJjQZJf+akOSUojZbYNYi99cemdSz0IhVRdeeM/WUHyRH/jafyjW1uhp2lIXV3V8+3a3wz13tlPTB+IWf/bv7XTd0PdmizWk9L1tOKjt3edWEufqlefO2IYd9uLb52rr0vtxQspqn957tygNzE9nHi27XXcT4+fOo2hHt8HAtyNWDKIQGQn6HchJrpj4OHKTBsrMC7bUmkZONDtZt6hKb+L7P5/H6PHTwS+q+lEGZbjK48MFJMG5kx5cSZ6bC7QY7jtgBM7hVvnIgdzVjd3B/fXP2KxLicAA+vku5d/6KjKRDDwJgvA1bRRENwoKQC4WTmz1TeXy+OqqafmtdsP3gw2NAEMipILp1BZil7Cxe7ZHkJGjNlUYZqSMTm2rh3P0iDD9i8902KN4Jf+q53nHvjF3abUp9riJM10wmpFkumipu1Xq+mlZkZpSTrLKbWMne7NrfuPehj1vZZTqwqJeEicU52SGEgL59/4B8dOCF0NanIfrA1l9ajNFIATUFjIzBRMvrHpBAI5YcxeEAjucALIHQp2IkJhqZIO2hxuz5zYfzeN/f2OjUXwE01n4rua7gYX27/+s8ugRgQnvlhNe2rtVlP6DnAXNqovPWwgV5gqzMDLg7MQ7sncyVy4eugfP4NlMMDVmRxGTNzMjKEES8kZTm4qW1Ve/PJPaMlDz/bVp7uhGTxziS4NugGJ1vWaK8jXP4WQ2aLbfE5FSj8Yuc1vHZ+53PZe4vKr/1GpBN1uc/o/+8TOcc2DNzXV5rAjJ062qkuTys6rz8bT/Y1tmjbfOb2V285oCwesnaG0k/iVh0Uysjb99Mdu90GDGNhD/FcPnb3RRmabDSSymqCirpwrPqabiLIQu5kFIicC2WhJIRuv6SSkZhodOyMDw8kLUTDTosdbAf8eFd03EeHmQYjY78TpblTE94enCg9esVuoL4cyQPOZR/0IRu5EoOrVeb8eiG2MI0BD2mH2tauaOcBz/mcf74oGwFTHuxxSNYXIlK3XjQI0W7yaLH7zyWFIpVD/rZ/s6ojarDgoYsgZIbVNaMvhW4+kVVhp5TZri6MMGYw8/Pd/9UJgrlJ77fjR6bLKW3jTP2uLsloZedii3GdzEq+qZmKTPD/60kdTGs4yaPGVd1yCbtOt8Mj33VoH7tZWv/1wDsVD1VH/8LN+O+acc87F/+1fPtipoE5pmfuqRvFGuBnAG9WMgUEscZQukTvARCSySR3BzZz4ROhjhM3UQA2uVqhbCed/X4p+V9PHf4lA9wgdjK799rnG1ghlJQj69ieoCmdeHuplcfNSrAwxvv7M8YQSOCkbBQv7O7GZd1MAHKBha/btiz9VqF5LKcylSPCm8GRriHAOnnvEy683SXwli6n+L5+Mzik1Pd55OJMLKUXrz/bHKRq8ObLgaf3Yk8veOKzDVBDBWWEG58f+8I2tXEqkf/vJU4tpIZIHby0qCmHaz2ZVUptFTqJF1ZfVauK3m/tWu3XPOrt29VzhJEOc6Kkf4FkuO9osrz4+pEihhEg5YrufHjOnyPYn0S6G0Ph2Vc+rPlJFx1jvRCfikRTgpuocxNQJTjEwOYjNOUTxQkQSo5A5mJkZWsyKIlEUWOmX2h8O90yIu2DLuwfR3fj73us2Qi6+CZL4nqvuDjr5wQ4ikSBApXeEDnZQc/H+4hgTbOrffsBzpOb+2LE5WNziNFPDHsjU1EDO/PapWT1onwmaLU4m1a3lr3u5tV2MQhQv2iynXEvl7tCUvNj/R3y20q1jjsdXL1xDyZq3vxonFWlxqVjmQ+9UQOQQKojPrGZNznImYkfzZJUcmosPP/mXnl0MbR2H2x/b8xRapPbM1V1nXc6CcXE+WE+2OgteUE1qiW+3D8SbCPlMeeFsS+aqw9HCsS6t21SKZY7IGggksaqq9f0StrfbnR/erc9MUi6vXzr6EgRxKhN5YMYNiJmJoGYgZgERuY+YGREQWVMyqQCYqoNp5CyAiOBa3MaTrsNyqex37fD3VtkPu+IfcNcHfIWIgLvYO8jE3vgPk9Jopmh16dFis28fPXSwv0WxMOB1/WY15RK8qJoznOS7H53qsWRycuW2jemT/9thPUsdACKD6aTKsSnOBirdYLN//dLfVmvSqo4Hu3/v8rRfF+LtL8SK8hju5q0+mzoYxsFzX05fj9Rf33Wrm3WzWBVCcbaFFvXKVrN8+aGBaBCqTrVf+ihLtZqUOvT9rqyu7dVQxDoED7sWj+fTybCYd/rRdsuNSFFVPB+qwyMv105BdXGrd4O7BM/754+qYXELut/uro+GjlZXH374IYRi9ZVpu/RAYHfbQOUbONyJT4IoNjci5jGEYSY43XGg3UwcRAx1VWvr9sT+/p7D9feEbJuM+e++SjDB7yBBXILLraPHBxM4Mbu8fnr7MFbLM/WCTN2YwA1e3J70GqHF3QlOeuNPVKu+BhkTxVqK0p++wccTFTMlIlHauz4PCnby3Cesv/J0eyPPDJN2ne7fLX1m6y99pS9G5GZFPPY2aAQBxCX1SeOyvXG4tZ7uDTb8m08rubuWyUGuudf6kPtHszXVitrd+bAKqzlTc/oQD7yTn7i9PrK9bv/6kbWTa/uX/7vZ+tpbpyfnfvO3fnlvvrbuqNSeHt7n1Mvk1Z1/eZg0TLdESzEreeeyx8gPV83Shea74Ymv/6O/+kQ8TG20L12nkEd9HoEOd3MidwPxaCgBdWYiV6JNmOYwJjeGG8wJ7kwEhyEdzWI0v5spu0eY9wjZP+DsRvofPFXedeUk0Up8DzhDZVLonbB3ZGLETORf/H61yTI8kNMUruLOzSS8dia4qquN09aH/mHSJBlkhlAHoyvhVCp38rhocW27YUQlt5KNp7+4u6xkHUOl07ScD4PVPMGfjcnA7jDarg+ywZmcGJoo9s2i/eb3Cc9mA73x258rxcFW1laVzgNOvXlkHAZrVa39sbafVLGOL/wP4fDK/kPLMJ29Pd1pPBXsfvyHb17f+5GfP7599jONHbx97ZlTcaIHRzns3Ocm/+S585hSoMbMGXA0/fz22aPQycwzpR6pbP9wt2wmJrhqtbHefZkgotGFozGR7WOgSgR3gQNg8tGAO0ZloZHl4G4sGI4mk/4OTPp+yX3QJHifZB1+L3XiwyYDEe7m0wEnXwQ3uBETQnr5ryVpblreX20VhqhJ08Rrn1KjAlUI3AlrnXaJChimEmOwbvtQsc0dEZObefvmm3/K2A2wlJxw69nlkkTbdtLXVNNN90GJfm6yMjc4E506XGdBiQ4my8ViSbz/nV/cP9264IVGSB2wUpuhDbmr48ePeeprbfH22w8C1LdV+M3/8Ud+8s97qOpdWnEzbbyUIZTM2/nKx2Do4lPPfudpPaBw9oHMq2OdyuF9E8dCWGEgK5DDyauvfY48DpQxMZ9X82dutjmuW59NuVqwj/Jlc5DwCLsxTmgrDjF1EPu4khNtUCoaUbExjoeD1SBltdrdAHj0vYQL+gBy3HvW9Hvz7u++l5x8dDjuYu9VjrntVoAbATDjXfeQ7k8e1dwYUlWut04PFgqrIrq7eH+mORyiBcCobqLrkEm483oguBpo+NJTsbinAB0SyJtcpr1uaeWhLofLyqhwl/b6TfqPmvz17SkSsZNQyVockOcfmHfzxgZ9+WNF3Y3Ujvsq5ETT/un/fDp0sRqs1tW6CYEDy8XpL/xJP4rlzV2f57xYlqSTA0GfZGfRaFEu39zKfYIGdQ5a+9tHUxKJtdEAITcJqXkrcuwFiSpzyTa59P/6C3VdCFWVASEfiUU6wi8Y4RXzUWqOEZEfF/ETdN7BIAY7DMRMDmZDcV4fLzfa93sGae4VLv4d1nQAuBuy1Zl8e7kGRupj5vMsytV030NPEjJCLWm9P80u7mN8AqHy2FyzGJQpti3nnlYh5dD0wl6yUly++NOolYO5ZXVyXlT7rqE3jss0LXkYegozAVhCYNhk//lla0MkOJMWo16n1Qs/6i0JHS9vPpLV3Yp51caUJSgtdo+nLWjelzNPNBWnJmr/1rpOV5QuvdlKQVWJtNtDJWF7sgpNCiw0uXS+KlLXVaEV4txvTCbIhaItAZCbUZOuP1aRKHukPm6zf+T49vkqcQqTqFUoLExEzMI85pUILHyirEQOFiE3N8eIdfEYoBMzu6k5EYilEs2aV8vfzXl/t6jvHR+6qL/r5L3fvBuyDRTtwb9wADLSEg3lc3x8+jWsWUmSMQtPAu2vd6ucS9Zaiol3p7760G1zNeEFl4o9h1W0QMhcVInB7b/+2E5llpVKNisFSgsu0k0keyzL7KDYq/Xqnk1Kmc9+SR8ZDve0qKmqWQg8XL/++PFkosOqHNbFeFXK2tZwjgQj6053JgyyqmX24Jr3/8oDz/bzofna4/0QOQ/G3pMW1W7S9U1f29bv7DZL58KgGD17dfV2iQ4WnxYnhIZ7Orr+SFc0GLmjhc3kZ2ZLZeLU93u9VUQj4AKAiCWImURyklHcLGTqxMQiTGBxc2JyzeZmzHBN2QBHdlLobTJQ4aAfEKIDAMzMTEsBRngH7hvka5xLPPoTMHO/J6U+3jvSKIiYyS3YHaFH4qKrUEaHA5j6507Nq282GeRgcqZ5k/X4UjHepnWVx4x93J+rAWQ24y0TJcDNzE0DQq6lz8//rKZIgUyLOohs4+64u5ZiI72A4UxiA2az9W9/usNsyQpGyYpBWd58uImBQ895fSqtMzOF68GyEcyo5qF2MiMqCEEmEXL2vnOTXKorV/aAdQZZycVHVAxtL9Yu33ooZQPBPTtK5w/+YEosQULPwsxEVL2xVaJITgE4fcQNMTVCbgBJE2XjvBGPGg84C5EIYaQUiYOYiGUUijvRBrfd/ENj5GaQANO8PhTOQiV8mNq+lzmzMSgfpPr03vvpzhfGPMAdoZMGRD+/NU4RR15u9/7WO3V2mDM5t1Uhn/whzu31Q5eRxBnr22eyOdh0WFVNJD8gqJqqEkMcky+lz5qWyr3kVIzgaiQ8ertaijrMwEzuFCjHWf1rlx/rUm1UwD70hQozvfCcxBmVhS0O12wUvVx/qyZ1aFFoSLUEUqGEKgRmlGjbc9HpheqccVsHqFIwjGCKFBna68cPDjrSHwrFmuJTf2hIZpKIwsZsx1eeQNDgVQXqT0sE20TYHXCZVpGYmImZgwgTAONAiFUggptDfGQliW98s03afdRREI+6aoYQYNovbhZy9iLvE+GHCf17OH24x5rfIUnSnS/chWGtsIvft2U2TrQuHsvsxSehY/7AZdu7emj/Tmvxt/abIsIMmdJwOimczXne7N1OEeSu6g7l3Kx5/j/+vCP0wTSlbCMUxcxMLOSj7psaAFJjkboanv90w/3QmpF4TgqOvH7n6VTPfZkD9m4YObu8GoRYUJTlsCWwwwMNFEVYMPTX64aUvv5ETkGYSALTSL0DkgD03YeCjS+EhFnSIkusLPOVo8ijosZ0+aOCQSIN7ZtXp2ygPBlfoXorgZmJmJiZx5ypEoNCVY0w2wZ+84277sSjDcAmIXsXVCVhVy3dwaFGGNmHrtDvnwQfLPb3nr9zfPfCHY4cyDX3XGysfELY6pv+9c+UzeTkKkigdfdgG2++MXVluBvPbrfzPD6kbaX8G28e5REJJnZ19+bCW3/smFgybEjqBM0KYiaWQFbUAYI5HMyuLpUujiZWTTLWRD7Cl+zv1PfnrZAWod75Q9J5ztq99EhyYffQ5MOmLsjuQkkCuIZxOJiHJPnlTw4NrbuMINaPUDnIKttevPqxToQBgDUPFIi0mTLxW/tCMDMK1dXhtBqCkJRv7q0mc6CvDRQIWSMJ0zh1hfmEo+YAxXo0ARtA1s2YYObEJ0LaRCmwcfEd43c3H66va9NQMJ64V7jv1fDvPRnwPnfv3acId9mwVBGD3DYLD6EEvBJO6ZgW53rWI64o3CB59fyDazIzM56+fiooYM4y2duKR9scVM3USESla6tf/tz9FXTaixV1uGsZzZ4EQS4ji48Y5ghSsmDo2q/fOghiQbwkBcxVX3lyGuepS8jhB2fKruHy4tFeCQgxXkpFjCk4WQlikpNW8XgXHG7cnGLmMUBTdtlYYUShrYvpzMBCgDvPGhZhKVkRcNCMaypJeH3PBzSsYev59KB4ZEqNgZhILd7B2DeqzsQwdzdEGbm8PloCjCCWb5Trjo66j4RFOMxZ4OQHh8WdDN8LY7tn+Hsmx3uE/AEnxi8Q7qFAa/CA2niT7IH1dfPmZ9TIjIS4rWGePZD1lz+9wZukmr56Vo3IjSX0fKk/tVhnNVWDEDlk+Vt/VIM7oagBPtIkiYg5sOWkDi0ugV3dGQjW5T/5M9tt1kziw1AAc+VvfxSzuOpY1+stkMPiS6cmPs7G9J3d3sncVM0iI+SsbIsdD1h+9BCJswtKRiVjchNMVXrlk4VsE9uuU0qDVrX2HDp9yIgliBC++zQHVAbolY/31CcRazeuEKLTRmeJmFlEggQYLCtVMhpxHn16drxb2Ccu3DgZzN0QI4GoHC6YC79XYN9T7h9+jeg9JuLOHCHiO/9JRrHs6wEbI1TTqn/r6XWBKZiktoY8LA+nWKaHbzTKzCHWzbUdHfkYfIb42/voYlEbE3BSmnRw8xnuwF09lNGv981ixExW1OCmzkJulolD9EGbP7yc9IjOlpPCjT1ceaA01qeKPCuk5NX6ebkcAsxR1rfvV+N+WJmZC0NqMYdtIVeP/rVT2+vqrdtaRahuZA5FdfDmD3V0wluSuolwCS1FPrYza2KRQPAbT7QTeOEmff8P98PujAUVj0sjhRN/GISNqnMgwEpB2KAyBJAEEfMT521Dqht5Fcajx2dwhEAO8GrBnAX4PWZdNpHZB8gb71L2O4cnLEYiPsnfVkU8DFXNGzLnINsvzCdh9vyKYN6Ak/iimvfxlfnACKHqat/rLjzbx1KqsDAN4dfuX02Wmay2yaJy2xqq6T86lQW1rHKxgec5GHFFFoi3ynpwgEwliYY8jQpOfenSOqRIGsLQO8BOzZv1OZ73XUiJLblFaob/9I8MebKsW9p95VSowRRiyLK/bcF7nfS63tb6GEzZw7cnXrRWBB5TYmzbX3349B6Rh5pcopdiVVRip+Ey54YDA1HKZ86s0qwuqYQnE3hVpCIzeCpF+pq9ESECsxALw91FXDnyEKsqMhzFRlqukI2YeohjPsXUAa4UEusqBAF7gXjOevOwJh/j/7tyORn0bt29I+0TcoTjZG6NSV5sAvk7c2Sj/Cza3E24GBxucDcid7OA1360Ps5oGwtVHdgxOA2CG08qS8q+c9zIO8sH1EFmoRW9+a3PDseiDg1HPpgNFaanTiuX4m7WzxdDPG5KYIKHamTsEyhoyLwIq6Dmqsq8VjQxN6rmgMNmX2zvL4UDl0zM5q6oJJN2ogPl139AlHOhzBF5HlMBEVkJ8JDayncvpEm9zJFUidkNrs36+T+10tnKRt7BWD1rSpDprbMxCANMZj83TTmZTHW8bsIUiOA0vkpi4zFNAeITsoKxMYhH4tSItPOGpeTk2CxtIPDGfSciGw1fJpT1YicUGcHUe7Jj71Pxd33yDzx4z+0bnGbzWe9hzjgAK+ZuBlfkK288dUzXpQkWJpHJqTMaqrdvPTQQoLnE081vnN1TddZSzYQRPh4ERXWQG9SLKA22kywgW+n7+e1QojIFEvG69VzMzJzU4QsgaSlakjKyBKZkqYzubdYbD1FcL0vOxVhgbmqMFLpoub197RlPyAZXYLUdxwUkD41XTal8qH5nr4KaetiELcLzC6uPDYWF4XA33YBWQRSXdktsIgsTwMsOHKqT3gzKRBUAwNyKjWgqk9sY/4M2AJk5jXHBSIbhjfvovqHD3xM/uxZ1MmJQYFNLq/1VuKPZHyC+e2T+/ssf5tZ9QNruHubMiCKcuHge46uP1P3W21tSLFaAkwxKzl+axYwc2dJkOvm1TyQ3sFk1Id9/uD1sLErgapjFBGSqPBtJ5LTW5a9kXcUMOBNVtabicDMbmFc3Zy7ZiKHqJVZVzlPKRd0BqpY/8ePrLV/nQcmzgiR4KZrgwePua2e3tKgJB1M6bhHECZy7ifc9CLF78yN6TCF5I0KuRhLq3/kDQy0qRETkltUUIUZy+Pz00DSBiYkk06SyuhmEx/vITcZIw7wY0Ym5dIziPzG7xaWKApz4UmNkCB87A9iJPzWaXzMzIoewWyn5cF8D7iTqPlj4jjvSHT/6722ejF4GCKB7NZ0AIsb4e/iAb/xwCct3ziGHJgDKrgo+fvm5nnmQUNVGr7/y0TUBBK7FmpcfT8vslnJeFnYPsUx3bMIG5AHtyzcjlaDiDo4ROY2UaBPM3jBEVwpCsFJitmyzLuv4qDH92PdFhkeQeFIxSHBUbYnwil749ABWZwd53dWFnQhchrlbmbjyy+tHdQiEcAJ8Oi9f+pGDuqmYcBJSmRFTn2P83HmaVAICpAohRuK6FmZigpuWYD7mSNVOkE5mYCROjXgfoM6xCuwgYma48KhLjLGY7W5NGhHBN1g7kRej7tZx+D2H5R8esn3gPLl7KPcQIwmjjzFqutmNW/d31YVmD9RWwmSc3I1enT2dA8Go1enOL8mzSgRw3TjkxadksmZ28NGUECNQh0KieViWOn75MzVVJYgzhUbSUGyE6Sew186sqiJVsKzaJ1Nuw/F3dXRByLmEan+IcDOTyJaL5qSkQRHevvFsJ+KwUpjr9VQ1MTuXNLUqNEb09acDKvaqSaMddm3enGyHbEEYADOFGGBmhcXWT0SEUWdJdNlVM9JqTGO4m1lNgMOJzHFX7BiN+Ekgbk4SAt8RnTCDaIR8mPzEDDttwDiYAe7CTqSHN9N7oPMPkL7fUfb3ifyD7j+ZHCe4EMW7IRsRWUnFbBS9Tb/20VbDrY9JCY07G2jlZOUr30+erU0EnS/8j2wbYM5ta3Tlwv37YqRZ6dZsGHLu21rrDEfqCPuvPH04aA4skUNL6x6bEI6rVxY7IGka6pZD6nJireO14036iFAvAxXl5F485JuS+1zSMDg8xpfO7BZTgjGI9HAiECaQlYn3UKqHb346GTuJgLwUIxH57e+zehgosKuThDHkhlVtXlGO5ODAbtieVfUkOLububs5uA3kSk7sDhchwMdcOTGfFAM73JxDEOGRSiMby69lJCSdoKFjEwByMgPUJQQ2DDcP3O1dtvp9tvtD1nRscNYPUvN333WP0EHwUoqOTTfM0uc/HbQ7PpuLSC4ohM5h5bXnbqO3SeE012/+wl86LHB3qaPxW/rIgidGkKqfg9xtp7VJrxRgnK/Enb7yGMBBpEbKo60D5fobLc1LW0d06yFnJD8eMJw/QZeVZqfLFD1CMLWjN6IXSBCBUhMvPhY9JSLEQNQfhhgaIodbdG+c0oV/fYSSVaBVQBmyh7p//iNRGSzk5sSSy5gKzYErrmYEsJCr9Rxc2lpGEZgBUgl5GXFzuIzQ+lj0MKZPRgB6zKdsLo9ZHgJQ7ORgEzptgiyyUW9YqMAXRxsZ2/eEX/2ev+8R4IelYO7xIMlp0xYD7l7MYxJGzgw9IrrGHz9oLx5tDdWkqlmDOaXc/NpHdxtM46pxmd/i2GcyI3eYV7/1mStT77w06+Hhajnoahq4WjWwZeb17hcf7NvEs1JT9pn2r0ox8wCrWvriD6cFd3WdWHKoGcc16Ds7Kp6yumA1JT4G25Ac0wv7iSgbzKRRHH/zJ2+LImVkor3P3553FDhTpdNcIpdi8g8fzw5DBnewz2+b48LOgymZmLw+GajLFilY1cFlRdyFir0+nphIH4tX4hDJmFSDSaehN9FK0U977kgdFGqxvInHKdSVEIsWhVRVMIebbeJnYpEomhQUYCQEKoiiGVUau+QJVcmDl6vXo3QSCpu92ze7666Nf2xTXXJy18nkGrF+jM34Ntc3oDFAABdq7mg6w1VtXH22Jqv49Sc5NC+dLRIDOeCylEpXb368S6OPgv5wmgUsopml2P6FR8QTecwiOnhDFII7OwJWOj/+4tNVNi1GFGqSL87KCFUT6Ss8b5r1lHRIzEpSUIeLh5XamIAMbdTiMDNy66+fGvPxzFJ2mmunJ2Xz+3K1XpyZB9XNC8Lgksujnz6VfewE0YaLj2rbtl9/epI8gOS1frtUVTIuXEyJxK2qoLSe9xkSs7shVCHG5srRrkWCFyI4jSC+j2HvmGPzEy4D4OOr33TTgLtvkuy4A5u4jwgEiGhDq8ImR2U23Oxi5aDyXv/rfWaePuDT72GY3wPDsrtlBYnAAmV58fuzDy88idjUMCLnzo2uHTzb24g5xy7vqKgVU0wao8vXHweci+p2fSqwg2IwsHV9LsaL2+dSbpogAM318MtndAxcifjlB8+tAQpDl4nU3EXCAXPR0RzKJJbspGZktLp4vjjBQUFzr59/SPPmBXPYl9OVcHHA3Ak7MBmGsFKHqzocXz+roeQXvo+UzetLttMkcrrtOVAwQoQ3jZdS1S9EC8IxuILMJ+uXdlIwch+I2SFyD2eVw8hjHtt8MLkzbyQtI8Z+MktwQpAdW7HBN0SWMaQzNTApvL95aMHuDaq+x6AP/Pi9h+MeocMcxWAkoOOuvrl+uper6QGvm0gGcqfU1S8+VoPHx4zddjQvUJJ2Foq/tDdPLlyGyFe3KCalwO6EqniDI/+xee+BWGATVL++V49FAcRUvzXbKT6vY5/cyHOmWMLWI4NvGvB4xVmJ1BweFrdPjV0HiX2GybXP2hjXulN98ay4RoA2rT6KSiju7jZ0g3rqbs5d81V7YinUWfWlR6ph6qm65I6qJrIQJLL6enrhyxM1SBVdASuTr/je0geC92NjLXId0TY4SDbclDECZDgTuYEkjAbBDCfcGmBDrKLRSo8Z9/HIzSEMotLfOgIZVe8V+PuD8U365z35c9+A+x8wQzacqXuybG7EbjDXUq1L8/y5iU8vPt1ovWkvmGlgf/5zPZgBEVqtTq2tkAjHSSw6/NbHipo50+Sdr3Zd5R7EHQ5dryni1J87RZQ68xK28/rX/1BngDsJeSfldpWsRQdRQlJn9t2H+hEJMzPWsnGLPdyOldqIHVZNc3TuURnDKAPvvHqmC7opNiDiIVCKjaCY5pxzbt46t+u09dKjOz2by+riU8O6sepoFSlTARnFilSx13ztTNPkMD5gkOn1r3//YWNOQE/MpL7JEm/ojkQAMQsRwJsCF/fRa3+XM7cRTIiBcWLsmU4svruRMAvo1o0SMLbZ+15qTpu/PiB//j0H8z1CxxiKmOcSY1x/5WMa9M3PlKoic3LQkMrk0sETqubOIdBFrXIEFytWucrxy8/2pEMJsf52ZyTMbVBhJSLNVo6DKFOIlre9/fzW/boqpRgJ++wv/py3VYhdYnJoFkOTUI2cMgcCF6UxjCF++z4d5zgR9/LWR4rpGINKVV2eDQ0l+NgERdquM6yOhqwgIstbX32k9rp6/mMgzVH2L+9tNQvn3zhTieq1LGJcIakv3nj50wdIITDgZrT1b+97bBEnTMQDE7O75nFtIjpx3EEsgDuYT+hOdCJod3cWYTnpCEQyIgEbosUm6CZSBCJmLG6tSDz/rmQKevdfd8T4IZp+gsgx0T0cOR9NFyiUit65/bHCV689m7eDKwjOOTF/+Tmx8TfyfHC2D0KSS/HKrbw5vU+djTiUV56JVeFqHj1wwbLXkqlb58RVI2ibnH/tJ1c6DEMqEPZLe48s9jErB4UVSI5aqlUxOXGPpkGdrbi7sb75SGG6QzCdPVsVhTtIqvZa3GOLaiOyKJIvr2vPY3NlDjF0bz6YYnjn+kc7SWq5rK+uGt6qbj+uVNnFgatiEepla+cPPlVzS2OLNaP9b/zBoS2RQFxGIgBMcaLp7jIy/saMCzGPZHcfywqIaCxUPCFVuRe/q6E4EbqBWX1Tx7G8teRwUrD+72FNv4cCTdhQ9WOTKF2bnDe/0Z+2WlxB7qQqwws/3BdmIvLc1Q+vOVlACLFBPn71iYmrcLDQffexZL3EViBkRCaiqOYVs5eB9kp96dqzZRFSygomP3PL5oxwtDAoI5NOI3qhk+punkQHmcLMuVw/bbKhqsTp+nRoUu3uxFLVF06djuvI48si4uNhygtpjgO7IdTNTbs/T8ob8wdXlr3vJn/3/BJx8U59tveJ3MocizLgerP6Q9dk3eScihNJ/Xp+sm9oxQAZQCQMc8BOUhUjX+okVc184ovckezdtCgAjHSSERvzsZ5907zAbCRWST44QnifdX9/3P77M/B+hw1LRNb4+ky9rilpzNtf+vThtPk3P9+1gVlQiIdByptHT60bojQT7Hztk7da8TLMPM7CIoff/P4eGZ5Ils+dKqUZ5krcoVpldbbk2REnnreJ47+8f2+9nJzA0N1Uj3hXU9Ij5+Gfng4lHNGQE5NUVWBuKQ+eVoEMvBi2Bi+FBVQ1A+e2j9lCdOW6fW19ZtWkoaqccqSq/q0crA5d4wywW3hzeiak5pXTgnVhVn82sMnOq2ecA/ZvfmTo66qAHLEv01Ukbg5bNJ4nv/5sUx/PKYfIRIgSjYPDfazsMkuh9gwqwhKQEkXKeawLC0zkyW3s/g5mOJERRpKkWVGDxCggYU9oOEsTaq38eJVMKMLrSu093hvuid9HMsYG8du0tLGTCvaTunUiWFE/CSDNOb5rTa+3txvnIIMfXPvopBwcPJW3dAPHGy+qbzwHGlZEKeIK7Ya+o4ieuHZeHqxOQxnEwI2pElALuZObFvdibsUZRiEQ0be/72A4e5PHDg3iDmYb+kw1Vy89RGO/YXLVUopx7FdNs+TttU5WS70xySAGiHl8sNFXlki5/8ar531VDUcBQVZduTC34zwIu48MlpeeK1avzh+uhqylmO3bdq189T6WVXPz1LGNAZYhSCH2g92vy1AGr26/9KN5MV3XjVEoDr5TcHyPvRQxO2k4sQnn3DcAyZiuB7OPhUyGu273icaecCtBIArJbf8IBBbri7xv0ab3jLsavLl6grTjxNDgzvNukvxy17ybynS7AbliemX9NOIru/dzTeoEuGlK/rXPJfc61ij1i+cjFQV5Ia6GnN6szsOCU4BcPKswayI5wXLK8FxciwdSbiuePH/p2etyczoizz56t6Vbq7PwN59BiFSK8wb7DPUIb1pJvj1Ll2/v+VgZJFaMNz4LVQ26n/pz9bpLxxo5aS+TV9KDVWdFaAOF87XnZLEs/8FfXQ2jQLb7o2VMlx7UUIXPtz5nYaial9IjkFUvbHHt8LeHj3rcYrhyMxgxbbrHnMjNYYhiSnBTHykrm/BqTNxs3L0whu5utAkxTyRGd1I1ozwlm9660ZMpczH5d7HbJ17E5v++e+Fd9VLvErqX7BzENTeTbz9Wdc2X/0Ce5IBxYtt6erl7Yq0RK664XP+BoxQqqDMCjlfhhcenymJg8JUH1M2qQCDSNBR4MZi5kIa2ouM36idnGt2Jg3hRmBGVLkkp4fDgQZOAUpx8QzEOWbq+yYv55Fa3an5ideTERCLkBbzhk3Jdod/7ObmZ6z6GxPWqqf/VEzNTkN1RpVu3sTXTxXRwYWIJsakzv67nl2u7/s2KTYRham7Z3W331WaX2cX4c7cPcChJjZoOd2Ksux2XYQgRTizkm7V+o14kMlLnNmSKTXg3UuFHlYSfBH13ixzYynDrtnnSGPmEivm7L9p+MmtOlHoTv99ZC04uOe4K3Umg5uRBqoOvf7aT47c/ta7hwuQgLzb/yuNTBwVMafKbYWssFzLiaOskL3+0KJMRaH14tkCqVuDwPCQjUwKEAxBrwalXn751ECZpQwyEmxHl7FQ0vL43IUEuNjbAIvKSj3tTJUphGnz+N580MBMLO9TZ1ZRAMZLm44PFTpea4lK307q79snlwk109K5g+jcf1WKVUhlxFMlVG7cuP9rW7db8F3504nGkurlYg5Lnv/GpugwF9Mhf35rOUhUqQtNvnurkHY/vzYyFCRJk5JuNCRlsSPGCO7E6HGCB+7iFwWaR3exmc8e30goWllfXNZTEFe8FX77HGG+8e/89k+Vu/o3uFbpyjOSmVFX6zsFHyta3zkyrTVM7Ih2q4eufyZGztNINv/H0cUTOTrAqJqObB4/2SoWCyX7eUtTzGQHQIalbMbhHZlAVYTc//+TOxNa7DHczCABmLVRM8MpjJKSDwt3VzK0MnffC3YS/fkOGsC8SQEQsKKOmwwlSB83eUKzXOVKo+/50/43jRwZtxxV9TKg9d04pdXFQc1VnmqUVl5da7bRLf3jvpocTXStCWq0uf6QI1z40la7XkhjiMYk7EW0qkE+qVdyco4AkMG2Cb8DBBD4Jzd03bSeIebQFY6vYEYmjjQEYGWseCKHs30Tw7Jrp96Tm7xbsnSlCdy/RiQ9BBr4X5HV3K2xI+v1zyFd+CI2MSRxGXoXLBx9bGdvaF3Tx/A/KSR658WWpvn3qTEFJIagc1K2jmlQMeEkZsKxkHoWcKzGX+3/5q6zTq6mchLgSWLNnkuHSo+wog9EJ09A1EbQQ7b98Tko9UVQAEbNnK2OXM6ZQcc68CnJMbZgVxcCrN8qaqlY8mJMEIeIhcLQuw9zNOERhkpuHldduqT6U7cAjYczMgK2Xz+/kKtQmqVRT3gZZcMniNjYe4FHTaUOzoxjcOcjdhDlAbk4wAxGddG0mYvaRE71p+j/u6MNwG+eROzuzUr5yE5KdP6Bw4UMcuZNPG8jnnkr1EQo6kbrbuxy5nIq58TCU3Z907978ZM11F8ZC6jLYlfq+biBup376zR+ZrwcPgUCh0mWRNx6cE5UsYjxMokMiAHctRrBxW43RqzFb/+//+v150m31mWIdYEQc2M2VeTjek4KSnXysECEidCGnprtuYWjX6mogEDNUddNWARyoKM9KVYkZKJxputlHfuKNZKY5ABidg2kZ1tKsN9+IETGn0//hZwfpJjvJ9hYnjZ9iBKN98ZFg1hffadouDsUtCEh5g7jjXViZb/jVI4CwQbg33tyGgjxGWSdhE+7RbFcfvfoTb0sKcwLdvpqDU5R/5yD8DiJ37zff5dbdY95FqArZkNi24tm1fOvcOTMKyjBBOtLqX/yJQ5qvuCRfvPPZrgiTqVFI3XTBrzy9iCns8Ir5lU8tIgkXUtBv5qB9Z8ihLlCPFZV8vN4L2rULqdrKhsK1WVXMipT6a/ef74Kuirv31GlIgOgkmnXy1hM9DRTcXeoSJmng5Tvb6gbzJtLKmqHETtdxmufbauHo/C/cBy3UAKzGToE7DjIkgnBxHzCtcrbFR86otWXwppsEXlc+SbQAzJcvf+I41iJ7L1/XuOK1QbtpW2AeiN3aIGMyVKAFNgwUmd0ALaZFieGcJAJVzQhhbJoJFlJjsKVMrCOyCC/quBNXG8jIlNyaq4u+Qh/1BMq9V64fMgDc6Ux5EkZsonpiGp1MBvE95t0JJRUylj6qzb75NHGr0hBpUvVqsTiPaNEa235xXi9PulK5dKV5Jz+pyjaYBD+u4KGKodT0rUttP7i5j/X7oYIjGcf1BIdnnRk+MszGB6yHqw8E45EEC5pRllhrC5oo7FvPjr8UgbWNBsxeDEMSsVpmLfc4nOdCIswVDnKYycU+rixeXxdGqMVA6qaqpnmVQlNNJjwgch40QNWlCrTIs+f50FFvqz/4rb1TqKZVKf/P/5IBqSJhsug4TVo7cYPHjX3GtJAbxIwEJ6wGB4VNYeaocJv4nAgnbLqxhcUY2BOfONYOEIsQfLjgQev0vtLl9yNzv8u4185vPtwDzjjcXCiEFBKGiz9EEpMaEbn1KV4MDxROAVLiV55BGeFldx881W+197tGMgL1h1tkVc0Ej1+Zcy5q6mAJ8Koht36hmtmaY4i4M7uSuCtg7jfv7ymVsZM8Bk9l8MFLzlRWbx9tKqiJrQqDUr54H3OQwqGaFKLZEZQkMKRGDO30Kq2oSVplhLoJHCtupAzKMW5VNHQy5XWWpo1k7EVNU5fK118iaXLhpq1+85kzS1vz5NrXn2nETXMxlBd1sRqzJePCPVJa3R1mEBgCmY+ZwTGEJxkr5saOS8AmATvOAxbhkXLhzhvyJBxOPMbz+dpth9FdFf59Dtogd5uQDvRuocNsZM2Ra/vy7PEiSp4AidQV/sZT074kDileWH2iHzchcrgvWqW3H1UyiZXksuq3CSFYkXT7tc/sG1QNLCKOuiHzlAj1Kk9/LVQRBpiBTRWIjsXeqiH1saxPp7GBBq10wSq/EE76bDAbOg2v09mxY2jrWNoQn6/gTqKaQlkt8n7J83Ble+7CpooQYAZipMUwoNk6NekrWy06tUISGKaK5uufMvfp3Fu6cu3R6ztNaHHtqT+Tilkesi+GLz9yKvKJpp2Y3M3+S5AIc2KyEQp1B3GIvEm8wOwOf3Ijgk3idXT4hE9YdidlZ0a4cBxynU+E/v58+r+D2DfW5ySou2veWSwVNS8pami/+pRbnatAruDcS/76ZzpzQixbv/WJuY70BHcHGembH+mUHBGZOkyMY3Cj+LyePy5QG1vuUKgiWcmUhXO49VZTMTY9U7QoYLK/frCyTehuhoMDqsu0QnJj/LEHNo15mOpBi1TfOkvr0pWagy2STi6UYJa9IjvUMo0RfGq4fZVXdSRPhdmtN2aKs706hno6a+vC7da8EVol85KM4ne//Zz57RXUsvzsc3l1xWt++r84r/X23s7O7u6ZNX06eanG9MrYEnAsgnByM6ojCgWyzTpqHMUNHMhHGz++rBOWgJ90FHSCudNJEm4jcXcnun1ZK7P34Gm/z2F3ymsIuLtFlzMlZRTzHPOkv/rT/bRZsrqb2VKrC4dPLAVMgdYX/uhBO4yND40x3a8Orz3GKYxVZocTSXWsnKz+ynNdM5iNXQgotBW5JpPU9FX4wtPtphIc8FIUrvHSeqLH082yiPq1dndrCbbUriarybjaOTGHzjiUd/6oG5hqBo5jH7/6EwpSrwy7vgrHuTtMFyYPnFltrbcCQUT1ui+v3exBTdzdbff761eG5NXW2b2Wgqhy7L4w7M9bzMt9HKZ/OKZpoj7n3Us30vH1a7fXhfXUx4ewqphg7uCTnRsITmRG3PTDeNYBGKqaVk6Bi/tou/0uWuInq/eYzFZiDgV0T1NIJ+vjlVMPHo+bEzr9O7jw7xnjZDI+6Ynh9/aGdffJaeuZTLm62D4uU+NSBEHTiuzLT84OpQio/e1HTx9EZd8kFUjpYnMqEyvUI795OvRtlMx67dIfPWQq5mOlRzWJhtxrJDfuvv2na1eQkSg8q7HXVD+9QsW6iWjlxmcDKldmgGK/qQghZhdl3R8eQsB08IaTVuG7+2fXYSLO7F5qvfL2t/9ZPgo/vrNfLcoOdSUth61pzBojDX0JTchdC8+rZabamb0Y7b5g/5fY71Tn9mw18FG8FVbMS5p60Bxnp7DY/tTksFTzjja/90hTBpszuUInnBU+th5xd43onHnsrjwi4rQpeyAA5M7MVnSkapCMFCzAR0cbDl5cqGo43tUH8vc5HLZZksj9rtDFik/PpVsuNAS6tLdNvA7guo+UctV948cpWAZ7+MKfWk27etyPykHrGm+fbxcNMZlXevHBwDXDyF9dnLs81LYp3fTYiEGT5dDFvND7Qx7rNQ2u5kDm5863/WxfARCB37py9nhVC5ZcpinK8WyEGJlKKD5c49BZmaW0bamy+ecfzqim0c0wOyxHs7+4Uw30pU9055r5l564r9meM88qzmAb4PHMad1fHWe1tFynZaEoprb4qTo1t09dSJcxL2cfaE439Zbev2xq7juZc2/zcnt23CYCRkfN3Rh3vHMvkRblDibiPlAVTeyE6cOKze4OYCMQOYFBOgqEecOm2CTm3I0mg16uP1nGs/eK7/cjc/KTbzr8no17qCo6I0HmINMv/xluWUAyiPLBVnn9pb+/6CaY7O+8Pn9iEHhxIkcVVOFvfrLjwF0IeaiOPpPZ5hll8vzDWiaLtg/dmcWWahMdNHSQzHr+v//URBkFwRNaLc6DQ8s0V32MyTKCxK8/XMANqD5uNWg3I2KGEXxo6+s7bz4kRHLUtOvilIev/51C1Df/8ImtyVqAyY9A5//0p59J+b433/ibtyaRb9L2dU5dZWZ1XaHvbyVV9bph2djUUJyB+rPT2tcDC9gnU5RZj5TJF+7gKvYNm9RIUB534HD3DLLCQFidwjsIK3FXdans5sP9cb0OtRcidysgCTzCiHFcFOAQ8qDJA3lINhbBFmVicxpQ2eX6E73W3FHIgncnZO8d9IGTgcbNgjC6cLpxHLNTvrumO43eSQ41XfbHN7ANlUHCMPzf/sB2sUK2M/vSRxH7CkU2nqSs9dofk8bMXeG+mnlVg4i7Vz5XPHkaqvmws86na4KlwTjkZnV44+dO/lMfxNxTRzUIzI5scag9++1PsHuWFBsSl2m+060eC71PLjzB7ACZF8nNm6fq5LUoQ+p18JWQp/64Paxn6cune3U9+M5jfdX30ZbTNtQ15cIdkQDwdWAtHqKqs1sp1luXY0Bh9xhjoRHtpUaHmoo1GHbWV+rbXG9geBvzbpbZ66bzuHHGNCxXpzS3KwIZmIjNrMDdQHLi+ZMTqznTWGgHjLXu7k7BzOFlcWPmKQioUnz4+GDz/14HcIwMwr2p1dFRUlMixO/sPRQ3u8e19Vr16sUftYFU0vSdtz49NMp6h+EPurF6JFdW3DO5Lndt0oJYLn31sUNyTqDtEr6wjjVg686gZLNv0aMnKb+xJbJqAJEIk6uLsdO1o4eKw9iSZrZsaVMfgBzmSbv92lmEyYup1d98LBZUTbW73YgahegkL5x5OJauvPTYWhVXbs2GQlHCdJJRNejWXR1QclGfotdmFpI5KLaTVrBJ/QUMCZRow2W11CVpuE/1+mHabxs7SV+OnpLlobRzNhmdb02p3CjzsejN1RkiZLk4mRMLnxRAMtPYxAIkYYzbmUyNA8MM5fBtDyiEbPiQ8WHmnjZ10e+514nvSbjAQdDsWlPi1z5CcUPUXaP305N/8GNdn0ksfOHp+/oQNVUYMQgf5hcerDqHEpR5Wba0rg3Ad8KjRVeD1WzzK9883bK5dT2QZT39jee2ZQz43Coy1YJ67GLpzq6cqXrxgfkggBFhRlVMPG5mDssrnq2P+n+ejZgZxrke3njGUftybRV6P1ySJQmvPXpk88YuPpCbGC7u1r62sFrZvtctpX7IOedS1HyJSTUsSzXWH2gSImYvgxOZV03sNbTTtgrV6VO29NnWNLV/+y/MjxZ30tQnMdrQx50WZm5aNA9H4fYbNEtBYoSajY01wBEnKdAxUXdSne4eIozuNIJmYXem7tYR4gj/fKjU382avffCXb7e5sjHUqFwz02AFhdSGi7+/HpvdEOZEw2r8GM3JeQaVXr1zw2SxAhm6kSwvPXSZ9Zk7kJAvDFpc8XKmv9bu+kMmpgMp/75Ew9VqpQHJaMSb775Z9s07s/r7lpyKvCxSsAMWpoh6gt/1vLErAuR+BvnTnmbmZkcPl0e+7D3t24lBAIxy3r+hj/SS4Ru9dO+myzdXKpL8clF5fENmvoUx9/54yuXlTTMVbW7xSkVDMSBAAQyRIdtl+zVpA0CrQ2qVFcWK5ks1we3D5ddIePt87Nb73x7v/3uY39/O/abKIzdncCMstbZ1rAekyymKeYrW/cNK2KwAVAa1Zsx8nfHSXYCkjksgkCkmzTd2JJMfHin2oGF8H6Z3iM4+j36dw64s8ndnjPuhFIguUwupdNKDpADzX7gg1P7RmSw6o36weWkJ5LezcHsRkdv/vk8HZuvaPvWmejRjfL6ZyvJKqDSNje/9n8wVbdhAKv353+1frZe+ojjmoDHBuRErl7UzJrEF9ZPLt0JLkPbffEntlEkBDi7rWZD7I8eeChJhBIRQV58fLKswNOnjud9vrEKytP8zdNxJ0jzWjsRWVw6Pr8q1YXypKRdm03j4Jm3Utf1xdAu13FerZe302qVoXkYFx5T9sGmfJydpJm0Vae6un60c//5+dmnH+GurwgOJ2Ib4yCibr291Q1KrgY3WrTl5vlzF8w8gIjHzInaBtuBbZYrkpEUS9gQLseY3EgCiM339yZshrtNfD9E6u8f7yF7jDIGkfNdocMBM9JC+MbDDBgROTAMLCbHZ46NzfXL3w8KiTPspKtxvGRnEw+Ag0p461zgaEA++nG+TYV90cTp/7rzSHLm3CciddLfqmdQ2qzp0ck1CivI1Iq5YCB74UnkmCYulOf7h48Fy7NA6sSI2g3T4B6qWs0dOaYX/mQngVR3D4oy6kZY1jf+hFRNHUqZ1C6LB3Vx3S9k9TQL10r/zlduzi6VoiQxJHZVCrFqpvNTU+/bzTY72xSl2ttCG6oqEHyrkN4+PvN4Nz1c0XKWNyAq2fgyjTgdTybbXSZ2BTmbNt3tB7ePHULFSQBidb9TDXHCZAyqzuTBuJZUxHxDuJKobub59nwXhT9Ukz8shnfQ+zYEcMDhxd5l3t0MCZK+/aN3QcO+6prJ7dnAXhn7639OW43SmSQDEZnVN7cqDCkYHMq3HuAQlFC6ocrz3hpuFf/2Pwozzey5BLIwuf3yGUXMG2sDy906RneMe4wL+6oq3/2JrmoWnGLf+tXjU1ebJsgIaBfePVpNCLGqcnL3XPWXnlhFYeIdDSL16nB/fXj8ja2FBWsvX/2HO+q/c/C1iyF1Wy2q1axNLV/f/dgPV5O2GlnqKVPTRoJMt0Kuy5CyS4gqMe7u1a/mYchOdNNCe+p0i+4w3N7BrR3euKK+2aLBqtKl2aRdEMGIyCbHk3DjzKmumBTXMTTkTapwk5MHAGIvTqxSvOLsY1DEDme4qqsf7O6IsXyo9z4iOh9w4T2B3OYeg+V74nQPlEpl7pevfzodjtU6psWkz8Eye79l/+un41LqvBDuNBJYXOuvf8YGil09eIh26wGcEqc8WCGsI7L5+V+a/Fhcz9S7IVpnTv4Pnj59C2zC1qOi3naeP38O0Zy4oBradeWHq4+w9XXmpCTbf+GdeWlLUyyQhuBD7SpCFQYXzzb95SdyqI3a3cq/8IWrl/Lg+Niv/ehNm9RGj55PV9ezW3/l9Kl69X//+12MswdarK7/n/6yZUPXDmgAk5qE9ODcWrQZKusTVDm0ta55XuouwZUpWFSX3IbOdwapNVEIbhSoGLmxuGtflO8/SGpgKmFdEfTWA/e9rbHX6JmIg6uOPgBcxm3c2J0rcw9aSLjSwmJlZIo4NSlx5qtb21oP782xnojzJCl7L0XKN5jeHbzd3TlxTYOG8Pb0Ljgzcnt8iEVuPTBP948tFlyLjY4+d/W6uvK5zIGMoVBn1lR8deUHETsXkbqLb155uGFH6ZNzziy4dopu/bO9xw/bZdBcDOycZ596vBrGVC4Aih1ee2ard3cyU25uneqmnw+TTIxMUkl1fmfCROw2Ql+bx2WYERzzwy985tzKq/DKP3yL/mH93GdOT+47dZ3+7Iya6EQ6P87d+R+prqcurtshJu7jtW/dXCjXA2uuVw1GOow9fEPmaW4lD8nhzoNXDAtNlZRgMASCljTv1cgLKsBcbVzOmcWdkTyU3dulcoeyu7v2y8npwxKp6MZn28h8rC5yP+kS7UbE4KiUQdHVXLDZiWcY9ttJifbhKv0+3f/AE1VZUeT+uDt9zx4uTI667bUK/+qP2qnTJjCIp6Sjx+EAv+HP9hTdWDOiIYojHt96WilDEoFk8ZFHp2wo6945B6kWq6cb+oNf/NU/sgiUh2xMZD3OxsNVZe5uRmSKxeWpS3a4qsHbzvD5H53uC1khiWkd6sED12Yg2hQbECAozq602jl6+noj67PrH/i5F//yY4vsy+Hwy7q3roqSGtt+kKfRY/sr58m42W56Hq43s27gErxq1zZSGX24tVXy9Mbu4TBkF1bkXEI3NJPVqhiIMoSZzWbHCxcw3Iyh5MGc3BxWoOtZXU4vh0rLJgHrw/FsL92OVd7UVDN0IwYHs7trJPBG6I4IUvVA6mNreSKjMtzY2lpV7zbS79X09x1vtsva/OsAzEMdunQztHwvIgee7y7S7osXf6x6fLokuMNScYYbucYy+9Knqt5JVbywGMeQEa/NT68zjEvTE+3+paZxkHaDe4nZ3gD38qf+xrSYIA3F4SgxShmCbUiBZMPkG7vTo2ZEhd3T9r4dTz967G7mBHAyJqkrViPc2ZGcKGomMZWdty6em9V5r3z2F1e3Z6+nNs/n3dvft1aohpxrmebyjB9Owxd+fn7Qhm3hw2Go0CMRoT46nulobhsM01wmx1YUDlI1KWTTpq29T8YEc4QoVk86dfNiSoE3zBlTYy6hLOa1T6cri/CajYhI+8XW7NjGZd83/25WW2LXMVYXKMBQjsGsU4NRdAM5sVvU41s7kuLvUc2xEfJmEpw4cy6Fq+7Kje6w5Uv3eO9w1HOdr/7JT09qXY7qXzJAMCNTrg/e/vNLN1VjM3clWDF+5RmmQUAVk/v205DiVLKhpFmhxVOgpmwd6X3HSMldYRabBhasjD1R2Dn8ymPtQRk3mwel49bLXzobsgHCnolMJkyC4kIggxOIRaTXINmDXz7rBd001Vea7jpLnfP6yts/5yGMdvdgiw/2Ul69eGnSdXVVF714vE5JaTVx81f8odEhIyTqzlaHGLI5XAupu2eU+f5ipU5Emj3KUE2YS8qm0aAMNhorGeFOnroZh53lQjyNfHdYPppsrw80YNwgY8OgHd84geBKzAwat6pm4iA2EpYNbBSKNcmuzx/x9xr3D5sAfs+fkzQu4OgldhdfP5A2Xc53iZFjxEb5t5//kx5ize5EWgptesBzqr/56PmkomrZ4E5BmAkvPpfZmJlSlZ3XW4Dnobhp0Ygzj3Gzmq9nZ67WQ1dGyp5KZet0QuZgVPvfqIfWN9QBVLyW7lx3TON+hjnDi7oWVwTazGHiEKEU4MTdp/6LvV52wvJUrJeTt19ItFrf3NpuauYADqgWvpW7mE79b54MO9MzDVa31gdKrRQ2z5enYz2ce2VblH7tf+n6omplSIAb9Vdef+XS7U5dLQ1DGpY333nr9vF6vc5qpqlfr7t+KCPLmolsSIKdnSjs5GMwkoe+OjWLMZzwKGTcOoI2iQQad50WIVfXrIBEEUGGM4NEGMWOr/bVhyNvHyj3O5qOzbeIVpevlbbBcIB7wJnRV5rs/IUnjCsnA5NlG1uoEMCLF38+szaAZxc1FiZ23388FyrkXkSsqluF9+vsWjiFt3doSHtTF922da8MA6HamhX2Mj4YsdLis48cV67mMAckHsdq2bKNakDCBJPgrlHUNglICVE9SKLgKdyPiSOfWZ++1OhbU2fP3/6kCxtQHLnt1s3QCWaf1nUMVSnXlG5LHqQqhm54IhEYcCy3b55a/sqn3dmc3MjgyujQkxq5jd2EPS0vNb17Jqg5VDkaoo/9vZV4vaxSvbNcW+Wj++GWj5t2lzu3kRvpMmZmeQTGGSPuQmzqxAaWSObiNu7bGrzqqdLbF588Eeh7Cxk+ZBrcyfGeqH21f/nqUj03pXpX+xFhZll9+m8vhzk5u48tccYtRNnbm7c+umaqmZBNMBQtueQUTtmashlXudH2kQGw1Bc3a5je2cJ0onWjXVz0yZlAjLpOK8hJ0T2RV//xD5YM2xAiq6Od5VaTGMQwddJkphzVnYROYnuSoBAyDs5u2pgPNrt6tKwu1ovD0n/z2VAimB2si8n66jpkRZ9i2A26vlrN/HRLGouVg247FTC5G4ZJrx/5oYFjkCAcs7kaSQQRnCQGBkmIsesHR8m5qGkpakNKaei7rmhW7dc6WDOVUAeJMYgQ8uLY59O42azoZGeXO/LaVJkS3FzYjWKkohg3B3BiDkNsm+7i+sPlu5HfHQ6df9CtfvTWd/fVqc4p3I3TTQyRUNoFF244CaME8ZjEEsUekK/+QAp50qxKqrBor+7sDpGn33ygWhuzs2owbtlL3R0bOoBKkdOlynNmq46lUy/qrD6dk8R10OKuQVJK5EdsxUbCXeE4zHvnIWhpUpTVFKlq0fE6cIIrha7RWdWjxEDdqZvbCTPvohGhZLpx7VEMk9f2ogVVg5IXXWo09WKI3myH428l7v44HYpPD+TMrz7ItZIFZW9ttdX97Jq8uMBRxLgtKYr0JCnEwZnJOa6kx8CNZ8DNiZ0sKZPZijlU1Xq5k+1cd33LYlGKgZyKW5n4qkmmmStXMMyNBFCHcBoT3mLFDEYaontxJ6MCocL9VBeg8sr3pVbW0mTb5NXfh7bRu9f5uxYhW4Vs+VppFAmp9ns0fVNQqGRUs6FPhny8GGTsnEFIL36Sg1PKxd24XrsJF3v18crJJViW0m9vp4iSUjGi2mZv7RGXdu848VLyJocvYytWugMXqZZctNzZhM5Ht527EFdMBy1I6rZC2sSzPtQrJgpW2Kw52lsLGaIYb+3fDnKpkNblWx/zg2iM4u7cpzJ2eOKCmpL1fYYwIDeZUvfQvpnZ0Gs8LJOCW2HdGSP1GqKQSy2q5MZQRPHc99lOdgWxsc8dq2kpRdVMS7f/9uvHiNV2kywbeeqHnLvjNYnUagbY0PnYt2Tc42JkJ6uZ6abC1MZukzzu6wZThjmzD/uupYnpQ7X9bjrtPYpuUsOAROJqmkVCuKdN6EbobDQJheaNBAZHnLBe36wfT71NNBcyJa23WIgmbz5lmVwkwwhVo6DUpaTuULxyHuKBJrGX6uYGiKCmhpvCN/sbjVvzmOGE32sOVwNbMDaNrEWpagJk04UpxxsSXLzWvgpqURgexGseqBne2q4KHV94enW8LoFy6Q+PhlxS33WDesIsWHt6Fty0ZOfdcHzr4Rhi1dRtE7dDp8ykIjGEECv3PChZbwxEVuYQgoR6wsTk5jy+MRjBVdVgpajm9f53ru6v22bISa0M665P3dGK68npIHBiGpsT8OgTjFkuVTVV8BjVGcUozLKhFqoAILb+RtYsfLfB3IfVtL3PAphwKuy3kpvzSNu/B3sf9xpQ1tByqgfNIAsywBQwmn3+x1fwEK0YJ9DR1ja7CR+eWQ7RQFqnpiGXLGkoxdyHcO3GGSA2ZXvgsI+kI+2grohcYbDNviFE5OyUfePSEpEbvLEyWdF8MTFNGsY+WGaG2N18ploD4m751O2pMjmcJ9dXKS38yZX45TPzrj6cBVNoXyIAdYcJTfaaPpz345VVQjmsVsc+6YJnNdG+TGpZydHctDMOVJwYiEE9GFVJeR1k3EmJiUb8DA5yEzc4xjhehOjtdTVv+85BZIbCpkPdND7v1susLJI3AiosNMJyY9Sy2YbXSpTKzO7Spsei23y4XUnmwB+WWP8wHp07Sqq5v53goGBupYR330tEMKnIZT2fpbUWHStywPrynzk+FaRYMbHA12tSMrqSdwZHCRw0lu0dBzxlNzXTrRd2Z4WarbDU2epA1iB3DbGqCFDXcS95dxgM8HEXOjgIzAYr1SAeq6uYwK1wsTA6QN6+1QfJVqctP65QrUMkKFUV2G33UzM1eeHpzHVvOSlRCEuwgDlYznNX422bXj8mMm2r+PLsPt6i7vp+lMyh+tapM4eqxMSkZQJVK0xmznAVIkANzs7E6huWJgCQAcZirkakOKKjKieIGzEsm5drg8g0ogwFRMrM5O7GYycNMR4TNsTqgBeWELIqY0O1psqKiq5ubzUp1AW/+9hQLE9mg1moh5srZ4crhGHvSq0SM5FRDE74yjPn8qozigYmkvCN3WlDJcRFcQZx2SESj6+e2TpkMytxtXPU1AfbIfXZVGEV33hOsrTiPj2yvX99f8WmJVQxjhER2djXJGdVR6Gxug7gTY66aLvc1q99NocqktNG5iB+ay+YqaBA2lcf6wKDjeu87AfSpj4Qv/QHucG0KX1vpfCYlTatUzWNTN3KZ+vB3bdLG44ef3IrxzX3Elmq/NYjs/Zom6HZmNVH3lJU91zII8NJmJITkxsMLMIYC7xAzszs7qXSGmuAzIyJTIX0uFeetWWd3J2UxgbRxEJEY6sCA8F5ZNBrkRBLcbaxqHlTJ+v97d0JE8qH0SnuOEnvWdPZnGK5dbk3QEAciO8x72Nzci7KjQBWT2KBRBM4wc1/4w8wqxE6ycaF4o4wa/Xak6Qw4VxHq3a/c+kPa+oGVRWS2+uPFWqnijp2svjGI2xKQoghQVVo48lsmhmXcRdTgouOtZUeSsC149PeVo04YtnQf/pLn+E+QELW2YuHj1XMFgrF48xTWk/qXb546imS3MYuk2aouVBgEtQ7D86DFNe8vD2sB3un869u/R+HtZfVraYxm15YXB3I9rTamjZb2+eUGzFjWoEUNSeGgtx506j4pHEQgXBSqGbM5AjspKQk5O6UI1mBpSWpiSsrjUQXYiVm8UKAEzEY7kxwLSKVYgNQEo/BqmJ1tT0lHyrzd6n5u47Jrbt9ed9gImMTiHvBmU2rWJMoDnt019cJpMHdzYu89adxa6+VhVbqNKCqnAj89g/3ybgKvcwWW3v/84VfPCopa/HAdrWc7ULbgNJq78pvTSal0xjYhN3NAo/b753kmTZl2z4S+pjBFA7Ori6dOeWNwHIZk04grA8fRB+J86SnX/l5BsOIRJYDQcIearnx+EMHGkgHTOtM7il160Vv3Zqq4xQWh4kT79aM2SOnf3YmTXPQN74z8cWNf/yfnb5+KR4eqSxevZFcTj356N5scioX0lAzCZlZya1vCn4dBFOtCKwOhiuKRIpJi4uRC48OKTsIPogAZA7ZYGTmRixAYXYHn5hjgpUYQvQ0lrEjqDqJ5zrf2j4j/XT4ME2/N3V+z3lzy4cXbniryigJTHTXvCsxULZuqne7nqt3Lj6+KkCKai6l4lceuvhLfz0lDujDUdNcmXAIZHT9sSFmAxocT08d/vJ/dX16MFBhzXb25e9LKtFlvXvzOv/q3+nRT6gcPyZDWJc48BAzpZazWnJXR2lAA+V5MhEJ3jVUd3vP/409L/DeIxEOttbQrV99lrphUll99enfWT+9npMqgxR1Pzva7qfa/uZftzpkXRzd2l9eXdyyhtjh959+MN03qSahbmupKmRptlqseuzvqcZz00X856f+xK2Pm8Cq7UlZ95evfffNF467jJq6Fe/tnd09f3q7rUIXKK86RSFhmKqMXdpc2FVE2TxHzeTRmJRjgpFnFiQCikPFN7v3hEBmKmQgtRAoA+xOTj5QTVY25W7ZjIJnHabpnZ0Hq4EzIhVE+wABn0TwdIJhAdaka5dvLysqBi/UhsI8eXfChYhTw+5N+WLbaZdbHtqiJaTw5fuPHltJ5E7JJJZ4mkhQXZBTZayxQyjhN84+ka/1uWRmDzdv7SFOWkrxFu39P546dT1N6vwNelrIzN0grix9m4q5g4njerooM+padweqydbqTDze+oj3XkmlVJZtra6MCz9TbI7sw5mrv/Uj4len82m3Q+tS8oonDfHBrHvxtRtXb2CaTrXPbrVbM77/7CwyVLCma6EnK/1AqLRP60VCVqPY5a76J38urabL2cCcGGH2yY//tAhKyX3y4Z2X3rh48UtrrbenM6nbybSN50RINbiSeHbhwYMQmXFFSo3oKiSF66ZOdDRrGyyOQA6BEjOUnALzXUqN+9iLh3XcIkCdmY0YKy7Xml3bFFD+riD8OAHM+9WVK4qy2SyK3FX8XUJ3YgFqmPhLPxM1ZBAKM0v2V/7ahY8N05Hj5taVytrqqPr29s5NNbjDAx76L3+oXx10qrkG4tvxIQ9NVJ3k/vDf/ierOknXXnhkKmPjLjP3EhRFx05SkJ0rv/Lx7yshRwesPifVRF49dfpqUCuDIW9pvxCPb/Sf0DIbLKy5fflnSv/I0VCVw/vs/sO+6y7ceO3g7W9G9id+Zrp9nnZStXKerLZjV9b1MJFVZTmowT1Ws0bW3aBOQeOEi7x987llwvxWaIM4WHqAGWrTVo0e+lEuOiwPj1f5YLl/cOP4uDuOk+29ve02TOdTU3Ct3hcOdWpg8D5RMI+usR9DU20x9pRRImInJ3Nh0hKVKJBlppPmwEaFEZMXI8ALiZCL8ar1K/VMsNn07V7hfpjYyQHqDpdtyNhQNdxN372mAyxhmERPxB8JKeaEamg0DriKqtkbWA1khmLRw5av99560IpvnqE++uZfmR2VZHCUEN54ZnvNjWjM3P5P5554I4qvJje+zwJpUTPL1vYhGnzcYA/rrXwkOalFNivWLtvs33iyY7GSc4yyMEPS+etcLQIAPlu+cf6po0eOZnX18oufuHj5wvKGt/ed/ujtv/dHKNCw7Ppmjc6IhkDk/1/a/jvssqO4E8erqrtPuuFNk6NmJI1mlLOEJIscTDJgg41xAPvrBKy/Nrv2Oht7vbbXrNe7TuwuTtgYsHdZB0yORgIEymk0kibn8KYbzzndXVXfP859Z0YBEPt7fv08M2+64dxTHaqrPwG6oTtK63FSEoMVIefQlzWSEKpJlJNHumt4uiw//T15niArJwoKYGHJOQNSKgBNzRqLIAyoIZzqnTw5v7hvvMiRXXdVtzu9fo2FSKXxQphkdSAbowZLQGDprEhaQxBR4MZCni3AxGubVkjqjGiNlUnBrgHcODAGyvm1q1ay+W99tt5kl24cEseqNJkcSJCemg0iosTEBl3srqmXQRQMUCCfP7z6zosC+lwrZLCQF6brRtGduLISaNSvB9tOmi9fWi6xEKkk/vG3VZrkBnFU+X9966A11jpvDWeMAY5RWJSMxLyMoUELSqe6+AdW+yIkKKogNQe2D/9ciaMQvBQQkmGdsbFPXKXFlPq63s9/n/39gewgHu/uWfdFmtq4dvNFOQF95ko3X/hYLnSRD2/gLIV26nzVEqseJLqgiITGSDUeBYSIhiwGT8eDG+Rr/mEw7awgqviJpNBcCIHICCMZlIqVDIKo2RGFLACPx4unTp5aGD06Gofu1gvWrW9PdWhpvtZxmo4o8RENaFQumgQQmvlZFMhAFLCRCCKji0gGqEFVRkBrJeqE8iagRDFDdcNT3Uak8DnAKVSAEDEulFZC870gAhk6P+iooKq1awEkT2ZJxSCp1E4UktYTM/tuERi30CeqECnT6ThI+NSaekWRstPf9Zl45mCp4oyiPahryywnjtZmfz97xXGTlbE9Oj5sI0hkFjFmwTHUdW0JgJBGWnVwZCiJ6FzS0XzQ7S2uH/U1xkCpcsndmaXxvie3/6+TiydG4Ir2J964Z5W9BS+Y+W+/4+wpb1Ia+vaRfL3n6OFMb4Z7j2ytXe1SGZsCMeZD9Pt3soKSyzOpqwAgCpQXBjQ8/8KYV/az72ikYFTSyV3pWQIP1uXRV2CcqwhEFG0FIEFVM9e90BKoL4cLxw/suXccaWb16nVrN6ZCtRryJAJJakMz4cKKmqBFjQLOjcCoiDUBVbX5I0lAJGNVQNWAAACC2Ki2Pj0722ztvuWavrJdX1ryqTSahUiTcz173oMQQYWSRNQ+sd4w2KCEwJ3S9J5cv2N1ROZgVKOR2naND7TUW91wzASxm9TCo3YPGa0gP3g1oMkVJFL/c2/RpG8oDQOYZwAVZgGxuy9PAipMxNa0G0amU6dM3TaluYLmn1+towGizdIph3By/5492dLxJ3HzZevSdjp3PPy6GDeE1Yf86r3V2NqWzcfJfVuKQUspPTIdYC/YRCE3Wrey+zfYwPHJxy5GUgVDPC5rARU0Jk/qCH7LFYdn6JOzVwZnUAEpUKOf1WahBIWHZKyKl7SZEDEiKIChAMjMzDlNzV5mRcul08cOHd1f7kPobNm8ursmobriEBJorHobi1cEasTE1IIAIDCCqKqSUVCFgAaIRFSNNtVfgwqq41PdRhHxOaApCFBYDo8tqgWoGwimAGg8L+jNBjJDFdXTL4Jh1dHaFaWZGpWH9/beQFWW+GADqKHg5ipj6sUwM1y5AJ2uknrpxJRnBdF6z4+NOuQwBe/vxWsqwghGNr10bYRGzAACPrHTxtRYA8pgoDNOZvqFiyZfO1WBGaqFey5mbRupFs70Thwd99eseeU1ny9/b1lG2YJB8/Vk6hgMp7W/dxUmoylR643ovhuSurUck4Pfr7j70jqLrSIkrVb1D2+e6ad4aNxZMCRkIdYRHLBxYi3WkdSc6JT5v/xkr+scM1gHooxkwSOqV0QDqkCWPAAa0MYUVSUSgTCmRonYBzUyt/E6QS7r6itf7d19lDbuuPaSGR2OFxoQrJiGxAIiQBYlZEEEkdU2CZ9aUABhbh7d3FpU1TQQMMGyNMI033J+V0RU5Xg4JCLKyBNBasHm2ptmBKPlaNVIvjze2YuushyitsrQOr34fet6WQVdkQhod+9kEBmt/cx0VgcxhsFiaig5frw1Fl/UiQ5oWrktriRO3vOTiR+CgsLSa01/Llagtanb91dre51aTIkIDABTi1OGxdmLZ0ep1STGU0fanzjWOzkemOnu9jdsn15dr+28782LZSVjX/qZJds7M5X0Xfure5amrIuOi3SpXLz2IFeYDPzWRffg673TTPJRGh4pd/TT4fTu20tSyGqrJQOG6BLBLg3FMAQ29vPbrxkV4JFAtLFIbZSYCACUUEUEqbmhthl/DT+pORsXQEBxUAGAUtF55QuNG81/7ZEPlN2tV196qQ3zCyUa23AA1SdWGZHG1qoCamUth5CmpTUKBr0m6AM6wwqIKOotVpqacv913k+VXHhEhG8y3k3tNObj+yNEY8USqUQBgzZ1rXPZOwIgxBCT1IVT2iqbdAElZrK8/eevUOhni6evP2Wy+Njju1aZ9hIt4KZRnEhSYgpw7AQwOYw14F07ba0utcpzf5jerGfyHqggSuzNd2yMIqoHdpShijGCpSDWykKx1Pa5XrzGLJ88tOfkkUW9/8avlxdu7U7PdaBYV0pI+cDiRdGPa+8Z+kcfPVT0pRV7r75aQz4mVZXsbpDoQdtPWLELwXWHnbQyhtOvzFgudWG4SkOGw4J6ngkwMQoFsqgo+S7H3d9Ztld2uOe1RuD3uQ0waGpjqEwmM0mSXc51f/89X/pg3HTZxXMtFl9FIAmcAQNCiCkEJgeRNCohBCUUVYKAJtXIPJGWVFIgkQjLR9an9YqDxbMO8eZfnURxCwe9Y0URFQUCVGUTzivDgiqixBgzh7SvnfdZUQGBMvI998JqUGD+hdVXEflsMIUuxb7xF85AdKhAqmjJLg8dg0pKPnn4u0FsRow0/4Hb546sXvIAQKpB5lOphaKLB95SJg6MHQeb5RApK6Rd29P/PNh7yK3fsPX2LUufu+HiyoNlPwZbWgCX7k/XDXw19myd/8FbqE5h6Ia7LqnJqoKjiu7eOK6CkeTJbdL62mwXtAgJm+L0R35J6jI/0F/vrUW2o7FxVEeDJi/Us4gaGqWyfZt2nx5yXFn5mnH9LUEMZwGJiiTBFANL0zc8T8Oe3ffctWxmZ3cUUwWE3I0IVSnNosHIBJywoiXxZFBUAUxUcsyNFoOqkooBFhoemkvHHQjfDA0LAEBo1S8sM6ICqJA0hWQVOS97V1EgjawWQveJbRK42ehxVvt8QAEl4oNvR5LoRldhMFWZ4OZ1TICqhsE5ioFQFDi19SBZU2ueq3fp7qWNZzAuTdUUI6NLMwwBhbN7Rmt6XKOtEyuVkhTp/OjOfb36xg2vuWBDEjCgecGG+ZBI1IhppsFykt61loRZCVDzO7d2T6SdYFyZgStd1IQGg31vqhgQ9dSLY/Lw1vRk1xkTajoFq3qR0yM8vZSwLXiY6ThhtJjOahka9qm36cvKDUa/0UgXmNAVv2VDBQCjohzJAkhUC/GyXd8Dy088fPhr8+WGa3a4ymiAxNajAkwSmdzk2EEmx65oo5ACmib9A9AIZIKADuZXQ6RvsE8/B75LypxP9tOhiAIgTaSFERXwHFW5kUZgUKcBjryoDgqgimBVawu1hTLMD1xFVWv+ieeDE0644kqogetT7nBouz1iZ6qYPbJhtkJjrUC9+f3r+0XcclBza5O03WkbBqlFjl5YRqNIECNQitUDj+8zW29Ye/ntdW9UD7CorEmXeKbH0dikcNVUiKn//IsigMk0hLp6YM0wTauN9erAToKzihr2hU1MBHCiv7EO9oKhdgj7aa5X/M/1i4Cwr2tJVNIxZYcTssZEtKUPiIhQreq5DLuRnh7yp0T/OZQ/tflnojaYJ6UMRNOSQNs336ph4Z5/+Ui46MJL5zp+WJs2hOASITfGxvJ3hRfJ3OzaJ9U3FAFCVBA9ks3FZ8b8LJPl7ARfLp+KtHLRakBEgCiSc09BzoAySEp0erzNc8Ol15YAD6xEVFdfe2K9GPekSUNqhGqph84QqSKYFqKdTXAcJV00q3dfSCjj4XReDsKcZxy215u5zCC4hOtaAtvRg6+ui+DRbERXHX7g4b3dLT98/AYa46m6VrISk5hND42FmTIqpm3FLMVlvbVUkyNFH2B683pslatOtUfGSxoTjt4/sBGCAXV7smJML+nUcwZNhuDjjr2QYP3Edk6FIldTJx67XVQiWam9ECAKkKl5VvBZ1vSz8A49H5vwlLH1lJ7RPBQVCESIlSCKVUs+Cgc39fyrT++/59GPbb36ys2utzQ9rCC14psNM0zgc6pjcgQm+EZPG1AUFQnA29Mzc5Mh+vSo60pVXgE4PX0STZloo8s52QoCIppzI50QATgiOkgeaXcWG+61ojUDCAmTQmi9AWnYHT52Y8IOfS3WpiRCoIi2ZWIyymcrEqZkcPrFJciSiVNcs3YGwzWls2lqMUpJofY10b6wk8txZ8dW88Dn7tw3ddOrpzt7Dl3R2TWLqkA2RMxXR4EKvW21TIq1dqb02M4r58EQESBteN/8S8tqw6iosRZ0jCktDO97DZMCTJ3YnNZ4Gfuu6ChVbC/VWS1YH/rOYIBIEO9RN2bFTqexFEBkNzBQuGY38wzokTZ39ZsN85W/rRyRaqMMKKSCiM4aCUJknKe027n0dbz8sX/4wvabLt253+TMBN6BAlADHSRQZgfcSFKuoEdAEInBx1Nz09+4CrtyHTxc8JZxcpKDKCt1GJanBF1URInEHZ7GeFalNOvFJI7mhsWwdnm/XfOJXaMkqcUbrhQnPHNK0pEcJ6sp+ZT2+UtKlGEQTYwtTxdhHN2aTowIRhEl1Jl/op0vt9eO/3X3I2HTS96Rj/pEX7zu5fNzMo4REY2YEPNumN/kS2O9IEBN9kQHQAA1YJbJr8+U6HxFwZYZehtbWA7270yBmabHO8mMQp6CE1u5GLpLSgyyvDFCVLApP/HCOlhnO3SGFZQNCFKYpUKfTfFhslmGlbCcH9xny+xQAUCciWKMigNClUCCqXiwwAoEIXTe9rYn/uVz/7T6O9Z2ypg4MSxgUJhkBVMgEchYoSbdIhIBUnWBFk929dlloPXcF39gnAdNRlGUkJAioUHR6JjPIWfYBpTUC4FNDl7DEShEAjXWlGJie2hK57DE0H7YdABCXqb2sW53RCiGgmuZQPqBHzVUqwN9bKsgZ2VtMw5KhlG5lQUwCgIMy2Lq7NG3Lh98cL9ecOv3ttumGrkO4WO/nmwecU2as89rREliCYCJUyWHpsj8iXVkxHoUA0yblRUr7y2nKIpcZNc98G8vNxrG6B76wR6085hJApCg0RJQxRxqrQKWhH16qLrc111C9EF9e9RemEKjTqZxhZr2lOp2kwJBY3uNT7m9K3+eWGLjuU2dMKBpiDwCAAQEAg7AAloQsDQy2//d/3v/Z//KvfC6HIINxJBwSIznLPpWijGA0Sb5JgUITTZmxKdyoH3hwCEQocgKb73BmCiIiZqoh/01+JzmYy4xqAUxygpoUPPzpvfJU1Wdl9MbYpwk7yYHQRIAVogEKOHkFTCcSVlLeOiGkTRO0kgg+NWp4kS/sAvFqkdudb1WTNZtMf3KCABSkp1Ng+zmJV332L4PM17zo1c7cyr6Sp2V6b/Pdg0XZisxGVc0NERGObKANqKLoXtmQ/1Pt80K572MIW8of3FFNAnbY0pu3DUX+qMc7x2tFrG2m4pEo4AcgkRuL3SmRgQeVB9sG50a43ooi7Gt3LHQAlXjnlty/s3bN8308DzxANSqbN246yf+9sMffcnzqbaWQwQQB8gxsWXT+yYDGlW58fpBNqixN8jUgAqgPft+k7U8K52Mcn5i6NOWHXqKDf1PoQHigCDQMzFXlvtnLqgCgyACWhr0x5lMTEeIAA++FgJRYFje/0IhsaCI1oImn781VWt6M/2HT14znhqambWdUcBG+yRJVj6sOzO94d6//sj0TRdftU3Ho5zECxHL2F5Tu1nnE68+s0MAazFGVlVGQsFZ1nCttoStEBSmAEHU4BlUFQFNnkZ22YKOlxMa3lgsWLCuTlSMgnLtwQA+Imk/UcaQPXadxgTSYjBGtHVxdCOiSpbRcytqf6Nof3PRgAbfffbljZIp+1kKP/3G9/2vT3/Hq9QhAloMabDZqD81QsIVujbqRFxfVcWI8unpC8dECqrm6dcbssD54NBJVJfE5cr5FQ/nSSM6Dxg5obUqabrHzpySRkWHrIy8molgMQIgDzbEIkQO+QPcHUqwCoqJ1erY4Z/sp91BPpx+4pINvdCeWp2PS0MREUSTs+oZg/Vf+fCXr5l7+4sGZpgWXC0rEAiLf8sAvVsMTpTdkbXRJgnEqCqgaFQgyuqljdvKSrKlrsfEqAKJDw1sB8ia7rA1liSILeK110jBmJwYbXMGEZVDJCVeaHsSJalo8cpxOljVDrWWKeLS8cuNKiYJ8nM4vvqGMT//y7NGfTIBN60OlLWZK+xN/cCu3V9/60su3rVaK5ctASp166W80ZzRSXkInCqoigBFpeVTGyMQmijP2GYEYvCHDqSAflT11TIiyUp3RIQECZ460lWVfL7vAgnNqyk6F4E9W8uCxii5Y6uS4VTQGFr3tYM3rABkkgSyT17eGeROh7Ozn3/e7Oo0K1oyqKQRCVHnVt6hc+87Vv2XTf/Plp6fdlrxkECF1SQj3Vx5VTOyJi7f+XqbpFbjRHeLmI10lktesBFYyT5+gwKAxLNO5jZtD6s05nEwdPUobx/LCpvvdtsNIgIHVmVNrtji1QMAHQqr+u1B7kIAJ2DvW2hVBlxqv0F69P+X5hKOhGhsMDPbpm/8vo9+8WMX3np5y8+eLvJFnu7GlXrZJJgruwIERqwXTq9BRkvydDykJqUZ7ztp2Gq1GNgyrPiPwNndKD59eldlePzyiiezCiWmtamvI6VUTIJs0kcukTI1VEt84ibPBhkIbZpo/fWfHeRs1rE9s/R91/gqqRmBo3IkAszSprKk6j96wW9s+ODM9lPOqWiShwYjpDNjMt2KATTYYwuFuoR8VFRFBcPgujREi2h53KnxvluZEGJo/DQIyWZhcWNksxRdUWXcmwKT2kevSQQUufaAko7klpGg42Bm/rWLeT2VWO0ZG23++KyNhFmywrH5v23f9AAE9akPKFuuFgSSpLSX9Dj53vLAJ//5b268af1s1WvLuMWNjRsFVQUFFG1YUQIIqqPDax2wINFZ1alJ75B0ce8JdByUozpkghWccdNvojnP2aGJuILH8sBFDNyAkm3K3V1X79qyujszN9fJHOmRLaUd+arS5eX1AoTMiiaxcGdnQ59joFZ11d9ed8JT9HVZKXFjfWWNrpDintxy0eDL31FlKWrRiosxRgGQCHMtE3ulScHTobVZao2GoNAIlKPJYNjFOi+NAQMnJQMljKER2kMyLv3sIfDUHxscR4Vxy2R5tWfO1sQgtQcSi/X0GmsMoaaP7lAnU+SOP+q8QDxyqaJi5hrlov/LNkEsf+PnT0bbCvfMGcAkIRPJJFlrbjXTpb/w8fe2fvMDJ1dldVCjE2A9TvTlGzipKqDFCOHEyYpU4FzqeW5/fvAYmQpAUMA0cAXVxhlyhRd9HiGu6SvBDOfnCKWp3xjH4AiSqTVZK6NYjwdVZ1yEehh00SeBGyEBsga+/DwuYmbDeDVeMXDEXBRhqE6dJUSduBaoKvCa3uKJm+tQTNG4750lCQHSdlcqCqbdU3Rmb6sigxq5MbMQQJMm5WIARW/ycXLkAoeKwLxiQYPGlnOE3dLFzKd1lpaYJv2jU+oNq/igICOI45DiOEg53rtNy6lc5MH7uyp+6fjGCgEdsdL/fdC/dXva0hFKRo1s+h7H1FryFJfiLe/5fPK7f1HMYNsjIiFIxPMdX5gZwFEE6R/qqwpoOMdLbwKfPn6sZdnGBA1Nni4r8t8ICIZAz8mPeEQjHiGcGK1d9tYKQB1bLMRSI1EqHj0UD13kzHIWDs+z3XC/4+AtqU3a48GBS/0oTX2qmsVAglYiWBStxKVLtvAoxgrZehTWp59ftxY4F3ZJyQFNkrHPU3QgPEh89Nn+tUYLE0KCwshBnU9zGkMioQAqsXXfThFF9VUQFdaIndb41GU1jI2YkeGkLLKZRMZ1UU35xFcmRUaPaeoGQVlbi/ePi1Zm8sFj1wewrq8zgaBL6BqR7skQgEknnbQVgcfzo9g4Ca380Dxowks5n106eSU9+3qIiAwOgxBgnnOSVO0ExdpR2PCX7z39Ux9PyOUoSQ6aKVhEI+wBRAkjkYIBzI8ssUMBA4SszmGkNLpE8eQJMMwWeiEAoWVhNabZtpFLrEkgP4d7FzIsoizpgQuc0aRWRy5itBi8iAhgWrsy2bvdIEcF9e6N0963Mqhmhbj7oa0bY+aHOSeFr1pZ8IDMoIzOh1pS2+C6SGV5vDF58MbEpwZRozQdlKxBFQmsmNUw6q03qE3d2rnxFGlKxjcTGYWMFxfXAWD03GDcjCQm+3oeAaOodZENqRKkJ4aQjjXEyCIKCaEqEkQY8Y9uWprNeXH+0OsqtGHcGYiaxDQH19983/X0kXtuPkcAnJh5PK2dX8fRlSdCYqIAWTzvYBwBAMYv3fq3n/zcVTdsXr+el7IyVom6Sju9BJWQwAoBkWikR8LFEguMiglK8J3Qaw2wMzwhUhEFtE1vQ9fMExobRw/0qufkhRVBjToth5/bFeraEXDdAmODjEoxFEUJLVfzL1TmSMDY2wDjlq9bGAnYfPLNtOyywDFtu5GvY1so+KaqaGBq1lZEwIgiPFh96LF3gLYISBuROlVKEwQOPir13cyRuJqtAjMGm9zjtuScZ0mPSUAV2SX7zVpv1JeMIgpgIUV73xZFCByNQEwsG5sgXlyV0FoMzSphtRG1seCyd6Tc9jR7N8wtOcT0akk4TxtTxecU6mf7von6BGfxjYr35z2aIzhAPU/9s7FqSQczb7zu6CPvD6+6+YXb9ywenivLbHa4XDAramzMt40KUP/Uxq6KAQZCNa3A6DLWXDqjkGKwBpo8Q2Sy1wYFIcQI59luIrKiSTA58eUrRSwIx8ComBnh6GNwWnFIjuezHtqzG1Z1KkMDproVWv0Y0939naMcY8qdjsQkJQ2gIYJECmCwyDgqqQAyl7L2K+vWl1SgIgdBVVB1WQKgHAUgRfxUYt2UCqsyyANqjNqk9A1aH6jKH7/UBYOhlgnl0zrSIzsMSJDgA4BEzGw1vOZXtiV5P7KIiALHGKMI6xh84WbVlOFjV4n1HmZvTQy1DEw44U8dm8+5rawK+LQ1oEllzoX/7K+Zm3/nXgEBAYY6s+XS1/7ux39v8d23v7vEmXHdGs4bMagCwDyRIQapiuVDbMuQGAQOESGZ6ifYS7auzt1Eh1Ka07nGBHiifAZJQueXYRV0GDQ7fHJj6W2laBNqocRQi1KiktQ23fDAth0D10mgt6guoBWIUSVUnX+6ptMjL1Z6G9lCMAUrB24kNNknAS2AKnHQk1PdL7ychBJQjREaDWybNedLAEJl9cANLTM2UVWQBodfkxlrs34kFBVVEvfQW4IDjtysl4TGmSPVVvCBESKS09AqYGo+29bjWZJm2wCxKQ2qdabgrFXb/NTet42QgnFbA1GBAo2R0ORg9LkE+tkmBnx6eM/e3SZ11smrJ8GjAXIr07uiEihql4PdZAVe+orhR//yI7e/Oh9SN1AxLFRJBaxAo3HpsK7ZoNSR0oQVlHGYS1avORmth4RlskszSoAqKtikKwYb/wYAABAgKk8O0JQXJuMaPKHwqRSprryIodES2mztxf7abRdsWjPjRj0djRKQvK/D3NfLd76gX1edll+1ev1cGiomA3UAVVS0ppixjewkRQ/99r6v3QApE4AEVgZVQOcAlFkR1ELptpculVpUDJ7oT8carVEhiKICUgyOXBSz6HmSs7rEUXbvbB5qL2QhqcRRmkjl2BXpaCwTv87G8A6JXWxPJSOu4Y7OOvU2RQGv7aIRdzs7F3/Dwa7ntackes0fV/4/v2YyGeznnqOqGtCY1MlTlnREgH5UDsNxMl403/XpH/vwx8rtVNJSbEVt8MuoE3vueVrnaLpd5EbROcxTuwGL1jDWZRSgiQ+kTr4KT+wEUAHceXApNL4XMrvxuxHJJKp1/eQu4RyMDzrec0EN6Ux/cPUIJZDnZNHmNIrJtBdl/Vx20bgV7rvnxLbOpqnpbRvK/rSvGFmNgnJ7Vr2iCmLwJtpHu3OiOQFKEGUUBLCWUZoNey+deuf6WeNjrSIGjq6d6kGSDL0YZUBAzvbkc8JcMgqrgpJNlb92q/rIAqB0rLOJHIpmXBZQU0OwROTJ+mYogGGw4cQ/rA75oMoIKuPaE74fPMvJ6tNH7Lkg6fmrd1O3XzndeNoT9PxOAQAAvgBvScSd/U2TFHSDWAKuKE9r/Un3H+583XUkM3UphlgAopFGUTLOboT7H1s8dko5MILYdnvz+NVwY9U2AcEDNLAKbTqyiCMEACJFOM8nQtAaNcVw7bqqYA/iRMatyHVVMaSxvwrZpbsXt/aKyHEwCuqqug29gvI+4Wd3pIP6zPvuu2h+cBrLuXX1d/yKRqEIJBTHGKNQA1VgOzb+RXkZNyKgsmh0ioCEisqRETQPJy47zkbFG9FEF7PSO5cui4I00mW4f302GpLHiYAJgwu87x1BAJiFw9G5jY4EgAljEI7N0oYKKsKKAFMpQHckmN8kA0nG1ta5MWOHiDIpZT9XEdbzM31F0HNBfdr6oOd3iqa3ODdY6HTJnr+ogwJodFglScwGGnurfnDXP384XHzmAgW0FJGAtTlq1XXm85/dozvWXLVqNkFjcLk6dPedev97t6w/umi1TldY/4mIIDTyFwBISucdzaFliS23xYCt551GGne/ss6xG44Bie8tW0OTuruukVSJSgrcqXR6DBb3b2zFpS//ct+OL3hj7y+yguIps/zBj/4KV5wIUPDT+yj1xHmZ8VjOrDHxVS/r21Viy6QeZ+bvviuNmqdgoBwrRbXBtJdm0PiAbANUl97Z6/a3coVko0W1FbS+dlt0dFK0MQ7SpFB6ADYPwNdqY5o99NbYNYFQQRGkioCAaAhrNHakLXI2URhSDL9kSkyj8876VakIQkR7Dof0jeZ3anbiZ9eAlXRdzu3GJodX2JxlTB5nq+5SSyzVxoQqxWrmrvd+9iM3VgLJqD3MypaIKiFpNCFNgrGhiKZVm52r1p9+8qErT2ehRkuRAdg5FueKXz5z+Ge/99L0bOcSjEtf/Pp6n7fHNaWEIgJoiIMmxGi4UaGgaOjcdBZZJF+zpm1Io6iiGN693bMYscj02IynvMDljegjSl3WkVTHiLke0hHuW70dJfF3vyQd2daaay/8jp3Hj8VGHUyKdNvOfmrNUso+D39flonUVogAIgM9lKMq2YnUFAKook0SirUSiEi69uemZBXHotSgXDPaaX5iB3KZr1RLMmz323fs8DIGKwL+tJtupS21wbFjRd+U00U0gUqLIrJ1xhiMgUVVRcWIOPv/Q/n1vIrY2d5y/howWdPLpJ8N4mc/JXf/YSgsz3z4fdtfcFFZqRJEC2cF4tQaGw0t+cgKIIDprS/OFqhgmmjdo0FRQij+4w9vfse1sRpPWqi9FtC2OHlPbXYRomRUGiDHyuuf05EzBtVOdZSMRlZFwP7Sdm/L8ShyWg2vDEk7G5/ahSmR+FqwVstOdHd/Ltn64E1ro9Bgz8vEuFjG0kTqe4AooBr2HZpWCdlcpVg+mm6sjNbWJKQYAuodq1HUJSgYJl6Wiq7ITagDqELQ6W1LZUcqjsCqqoKBt20Krh40txpNkbLI519h61FEEGMfdLkTKT0qOx+CIKiCShQDzF6jOoeKxDU3NWkxMWbpN4W/PVuov+VfVlzWVwpxRi2t+uhd7/l31//B7y/ddfLvvvgLHwx/o7McvWHLTqWZGkB8yv2ka521lkBxnHar4KNptAWRyJGgIR2/4KuvX7OcIk2axChF1cKVS2isx1WFjLI2hDkAAGvPCzoRKiP6pnijQO7JS1KxY8gAs/vXr9Y0Ncf6F3AeG3R0IFBGe9+lF21bt/+FQQEPdzaXcybBYcrm1Zf5ZqRr5+R8bHPkfpHi7EfWcxQSbCeAGNiWT24WxiQhgNCUKQTIpinGGFiAQqXLrlCM4xYgoiM23bX/YTOPajfpzJa61YZ75aZaSIQYW09syDKKLkkrKvtVaRrPQFAQdTqmDB1xBOKaQQEUhRiK5NuI+bkMf/Lj00b6+YFv8A+TLD+860/3XfJj6Q9c8McfOfShP/rBX9j1a08ceeJOW1pVESeTGaIiTTJWozEwENmkrsdrLLtJzI0QKpJq8dD8f3DT47Pvi0RKi4lO7H9x0olEGyljXrkeY8y54kwMiiJkBJURRCC997rK9GLlxMq91ys4i3tbRVWdWRxTMTXV6thUZ1qDl6+59HGzfQiQ3nMlUpsgQecjLQUABRVqLV6tIUnqzrhXP/nAjSNviXTKKgKLPZquZzaJNaQTOIQaa41RRuWoiBKx2wFVr0IKhDYzvc0j8WKam0/OpRH+4juSPhYYiRUe32odhARR8z0LwMIyQf6CqmbdSggRgat49phabEbPAin+RjE/G/dv/pCn/yLF3/3ax/yf/s8rHtz3wk9t+fybfs/+1p9teOI/bSiIWGwjwa4K6FrD2A6VSQwSqoItisVVjb8hIhnTeLsJbbrjumQ4tCyTBsaILDsQbowxdeWAa9Ivm+cTIeG5UzZSMgCE3EgtgA5ObGI6BaPA5mC9o3QO8cldEpU1RtudmZmb2zC7ZWlul+T/fA1qBH74Sp1mCpr6fMdJFlUgVXt696U0VnWxbGcf7LWS+XVqIEeVyAoP7soUkhSQQhQVUYAksaSB0Yiq2szYCrLaO88gEjHTxXyc2jyvmk9lbB6S00+8oTI1R1Gbzj+6KiG0Og6p3unJ1hMGPwYFL2281xAao37AzWklokDm5Nsyq57svp45wmESufNKcGf/Xn/8P18NG0fLh7/nxI7iVy4/k779vSfpo6+cOlOwQLNfb049S/YLODuuGSGWXhXt/FpvCJGQJtZ9JJrQnW/AqdhiM2kRVJWz5lRNVVYuBIUndENApEYn9GzQLQI2tVhhRSD3wPp24qNvUYJf3cmhk0rcd12d9bVtqloxT6dm5tY/emkCBx94sTdiDg13ho5CKlTHLafGAEpGxR5sFT6pPenMCF/++1gfuKRBeXItHO++tkRKE1Xwok3VIUksSggKagyj0zTPqAqJJRaM1h39ChjTOyMGARCQLPgkefNlYwhjtsEm95+ZzkdliwzbU/fOBABotFyVg5IaebKVKABHL9gI3yBjbsK361D+rdf0s4jNSV2289dbFzobF/ljX7zok+tOXyw/+rHR+vINcz/z9yZYYKMThKnhMDuDx+87CTaJJ/c8+niobdhWCRljDKGqRUVULAZHbvLz7eXOSucTFKAka2zShSM0iwoiRyBQ1Cbm9JSga4wMxgiAsgJSet+lraTujHEUwyM3yepOIuHU9ti6/1iEKMZSEI/Fgau49ai/FNHZfcWmMVpBL3ls6zKoIqmaw7dgv6XOc4Bq6joaHd8a1HrGGEXjvm1DJGdEIayMG2sNSowaxVoWrZcLV6lUtRUx6GD/esuDKZIKAAABDfEo+9H+cOzYprXE+0rwh45oTIwcO7F+FBwSIaioDZoZLIpOopFZQFWFRZSEHPK3EfSVNPsbjfSndolJ/QbxM5+7YvGWas0v7LA/eebLa5aKu/4iu/FFTp/3IS8JVqZJ1ACxNfz4H3/wp394aWEUfe/M4vDIkMZtJm8MEaiyIUBUyu66NqM2F8OV6Z1QERMTCRFUmFfmdhQBWiknIDZBXznuFYNs0FsTI6Zjlt6+6wflIOYes/umtzA4b4/StF28e0PoG42YcqJzgxddOLIfut55FPeFV5/qJN6xZXAzxcCAl3Ea6GtXVVM9smpYR3YpGWarJLSQUCsz/alVScKcWFSv0Y6SCrAwOQKAMGRhlNqa5kzEQWoZhxm42Lr/ymGSLIcyjVUCp77QycUaLAV57Owgtf3X/K+dgz8rwSonR44dsAawNl5SIs7ATM89iNMBhNwwWvbR+goqYzNKznndqa4cgZ/9GSan4s1+gVTUGCWkwmQ2WVHLMhoZqKkViyhy06cMJ54YQutDtpi7/DNrN5S3X/G9V2arPvQ3q2aOPKxynQNnazEBasla+Nj73/Ge49f/QOGeePhBxH6RWWyNk04WwYs2+uhDqzXn1f+50XrFOqGJ6zOCEygxYevVmkQafTqUSBg8ULQWhYVrBE6eAYF2YxeqwlaPbXHGCwlk+tlX9rJCMDm2iiK9ak1fgTQqxanO1M2bxycPvL50JSwubDJWbUA16hNXok0hienymikICUULKCygJRmwFogHKPqJm6fO1NsoFAudaCNhphG6o8IuB2EyzYFVmmjfBrDjmcq32geXOxjqCAiYVq27N7ACogorJCBWaWZ6GufeftEIox1f96uzQ/UWI7sAYmvuZPb4OgVFUwdYdhhHLfEgCQX4RizQZxnFaFA8QJDPJBvylpnKCgAGUouNUD0ZUOGYRkx4DGmsEIDd4PFb/LX9h79n/OLNxw6/Jp35wT+w8e8Wf+GFwZZYtdKRyVL/0Gcea6352U3OPfTgKy9eyrRUQ4bMUC0DkoIoEBglsTpKltpKYvCsKrQgAqABISJhMqiKgKhANqqSmYA7DJ4PgT6vZweEmN19hTVjxqjtg0evHczkbNPHLkQ1l4zGKoQgLrR4vEGKO8O1IyFzELeig6R2ilQnyRKiq205dc8l0yG0oE4icGTUvnWaWEDPXJ7Y88O1JHXdgrmTNbONGNK2IVQFTGtCAsXEgjyyYwwgpC2bPbGh8LUXVBYTwr23oTbuPqwBFGTUmkmG7hIznIrUy1+2rIJsvLphJgDW2mzf9QJC5CttD6Mp0gXpsIOKUv2WRfdJQ1FjBKB95J9NFussJFY16XbWJHmrPTPbKSYi5ZAgx4y8jiAwzT68562P3fqv9pqlXc7cvA75+Z/9asevfugVF9vDV0JQZz53xyPTN73romw5H44f5xeom8bKECRJcswkJQFg4wAgwIpGF2CzYEyeMichKIEgkShYgIkAKyEJTLhsZAzR0yXFAICTAWUjLQ6/TrRWFaU7nqftggLB469AxfEQKKqj2ro8VPlo7mM3Wdcvsj07OmMDBgmFALNTFIWR0we+y9akiCrKAoaG1tUpRfKAef+Htg2ok+Y0Ml94caixRkaJc+NoVEgFENWkQHsfvrIG7wZIwTx8UTPdigi094SLElTCGIIgExIIQ5gejDEJRu1COu7UzqNYYAXOSJWOvlJVUcsKQ26OzA25I9HEKkmefhu+YXNVEKtRhu+c/ct+F3rRD5aWR0HHw+PLvVIwcGAFUBptue2GHTB/vPSJpHPvW+dXT335FZJ1lqaePwRKstLoa7+ShqQ3PFEfueOutTf+2iWpxLI1zpZ6W9YvrZkzkoLkHZlvWSGIICygoFiTUvLAxm36FEYlkaByoqQME784aJI2BSJQQCQzAQw8I+jEVaaSP1qsrxwDgxt+/efGm40qDU5cKAIlmFo1AkDLIcR08OCvj4GG7p5XeiKIhCTW1jPHTBSQxJ+82HPijY0iUdGYyhgxwOQryKZ/xAeoZ7yd+fgR9JFDmto6Q7VVSHqJKgKY1OUfn0UgICDfNo+9MrCCsCJw56sXTtUUQWNowMDi8jwDMQapN9VDU2XqARVtjxQErbXLvQ0iQH4caNw59bHbV7coANS1ee6J3BisQbTKr/ir9/zs3q02KWYvNACMogAc42LwrAiwkC19/fE3XOtLqjWGUx/5vqMvv7e+ebgqzIwizFYmnR6sv/HTiVtV/dVvrLvq9jddmkgZE5BKXTXfTttrOxxR0k6nOjWHZ6u5AER1FijZfeV0IDp/q6lCIEZNCGoApFGgRDTAao14i0TNPkb1GUFPlqPp5a0vXeaGFWnE5BFdb3JWwlNhXdQYkTVjCNa0GOx46otmu4+thfHhi6sCNRhFUSm3PQSABpMjq+ZKMDVRlBgQAWtEciDREw9jrA3PuJgs/O2P4aCQjgdKixHiIE/HqIIIJs36X/nNOhpki1n34LEdzBwCE4kpH/r+WgFUYlRQYIY0z1RPt2yVO5Yhkg5JkU08sFPBqBatR3GtZ8LxWGPeP7iYpDJwCDWY5zy7Q8hNOQyA8H1rfu2tFw3JNB8MnCgZQtg6MUAnn7xqSAvHxQGQe6Rc9+SOP70OLIUkkI60HVpnbtruvv6yjbsv+PU3rKMxVN4k7A0WI/FTHTdjS1FsddJwal0EFdJG+ygSAFC1uCUKWYXz8BBiFRySsDYGMwYV0JASGAONE8/KwH76h2JAT3bh8HU1epRI8Y7n0bQYITiedqN6EU1MjSkkCSAa+9ErkZWm96+eBRAAFMVYlTuPV2jBuAevI0QmI6QcEVg8gEsMxpiosT0US96Mvvjgql43rwtVThI2A6J92Gj0mcR8ZnZriQxBQ8v3rrHKISoIS7J/eE20ERuUlRrFNLHERSsJ3qdDE32MmGTINH7Uqtg8c/Zwx4gYHJUQg9ny+g1apgwgafatz9FXWmEGB3c/8uDdn7u7Qx+bVytgDJFNVNlXAdPax+CrclxqlLTuOfYOs9ajNxy75PTp64auzAY2nXFgTl25drjqRY/C5qPf9c4N88HWgiCeE+USxlPUcYqtvDVdoM6vrpQjqAIhcPBJbczJdBcjI57jOiAoqhpQQFQWVCQCQIM2dQiUGFRVIPOUitxKqxKs2mG/3V5bAWUq73tegtEIwVJqGVgYsH9sgJQ5kiDl7pszzJfsl3elLkQypIqxipt7pRKjPHFZtBitiQZEUJkDgHWEzHFoNLV1TgmkV76rZfXE/ae4U2Q+pcW8/uJCo7BDRj5+k/cWWEkN3PDzpMxgSHw0B/PtNo0NrhsUWa0hY2vrZYrBjjJlk5a1Co4Pk4qZ6oAudxmEoA7ETG7zEgszon4bSzrA6OTRHlidjfWuTx4fjhUAQaKv1SUUQ2iIziZRGvWNDZpTYWR81+r5W/ZMz6QxLFF2KkDVqrYn/zC68AnqnvC9URvGNrOMRRoLbyluKDYKu26n07Gqw7YHjsLMqhIJc096LN1hTQDCcHb0EiCIsgCRRCHAhmaiNnGo5Ah04lpF9IygO17ORnP/cp2MjYma60ObNyWF9VbSz75UM2EzpGL3H6/1Nk+qJHYe2Xd9PwK6h29AgakACEmA2s/MLGFwJS1vqxgJOAk6yD2wCQm1jYZotS2g0Zok2oF5SYz0+N3TSeiI4TKJJ3avC4zOt6b8yfteKCYCESajopwC2ytNYLQGH9to7VILNAZBlMo5ZrUUFUmP+X6okDhahXTq4as48cUoUXx8TmOmPCgZY+kqZWLMoUC28TwUjOp5pLaVOgY3JyIaMh5b6zEZJdfsOX24MVkiMgkJgyE0iEAEgj4rqv4YAH2azJ/ur0vuuc0PkamaX81h7sjjN88kMxeW4brFsUWjBdTgOCCMM6yXt60FBMg2riO29aiVCKaM1girJeilVevwTopg9TyLNVYMDGxRVJkyY6yqswqQJy7PbGMERUS29s+ylo1bYPYd31VTRKU6/+Iu21JnLFdPXt33Qx2kdunv1hh0UlM07l+vzNRV2VHdVNsMM0KEOA7A5oQVyZ64sLvCGmgQ7shoLSIIgiqqkAUQUY7sHl+TQIEkGmOye4uJDg1Z7HxyIw8lkmnQH4jLhXAGga3c+lqN0wE5sLCgkNgirTlJiB69w7RWgGqRqmNrE+hELA2MTpPRUBMFTp2CRnQCRCrP3Lp+o2aS2VmowdYIF8bqvIMaPddFmu8U2QMmZipb82W7/lq/6pKZC5IQbTsGs0rXySG0a+f5ww8tJ+jD2TzcRNOdys86VEIsh6jRxJUKYBQ0rh6uMue85M+irrERnQMkVWEBJGMRJ4v5hCRj7DNECQB4bMb5g3hRDWwEYWnf22wSrNfWGT81tZAtduulPY+/pbTdzOHYhE+/wnAyXnPHzIalIlUbFDhUbGn1gedV1L37sqZTqXAEAQWqnbMqyqoqCJImgJGVWWjfrWw0GgUfzT23IS7P1ezy8KJLNyyklSoqORKE4EJkQJX6ktU+kk98HUUBnQ2Api58ivkXL8iOgzbEP3KjA6/isuM7pPqimkjRrpJ+YICEBGLiLAE/m+rI01pz51Slu2roEzBSbs2Wt5zFsk94/+c6ASqEsRKLePysrlr++LFPHliYmX7ejb01a//x7gPxj5ZO/dXr3R13HTu2ns6BHMB4NzNbYFzBzuV10lZGgUldVRDB9Prb7SShX7kAAkBDAqA42aLBCgen2ac1dnCq9GyJnIyCPnC9jaIgoXPnBReaKKAh+afxhhH6cb9ad+2f/oABYFVKjx+5uhal5P5LSSNgVMEQhdBesg/V2icvrRokx+QEAGjYtkaZgygIKDgH6AMImvHRthRqM1JP9aHNLPmI3PSwmt58bCjgBZUMqrIbOeMLNKBC7ErDISiqAEbrAFFcWR75xA1D5MlIJ+2NVotJ2nlO5StePWIw+bqNa1bF3LokK9KZnBBBnosGRUMZCW6mUxTWOUy7dydedQUYdbZnTPqISlUBmjSkj/G6Hb9991WdW2+/5eG96cx43x8lv/w9b/wB++9Huxa6sHtBz/NEpEhJYc69aXJ6Ng0QzMpIN4SCp/gii5N9+sqJjgIQiqIgIClYQ6DCbMzKSDcGQTUwP3PLxqZsn9r960OMuUq0n3uVtNSK097fXJSWi4lJRr3qkvFoVcHgnbkDN1YUst7BV/dSJRONohdET5d8daR0Si+YjI+Vka7LayxqHUUAhYBSAqwjKNGxx9s2tHSc1dENYqvyq0b1Rbu36SDSlDcMgM6IahgNs5Dh7u3TjIDOuzErAqqwziAKVRnMfvqS9ceyfnMagsCn59rVVG5ZXJlTZVSZpqfKwdYS6qydjoyCdebpt+GZEZ8Ql2xw3dk+YZVYP/vVHx8V33iHH8tgFSmuvWcTX/iCq19YF2dWnzn0u6fW7dz4X6b33vDpJM3yM+uzPbfNCFo++z6iCcazFxR2zwTv6nzlZyUDdHJ6rVmBzzb/TUjWDMCoYBo0AyqCMRhVkWRy2MJG3TNGOiLlX4cLPACKmEPzV8dO6gKY+KKfWWihT8RxMZTuqsShGL3rqpwpuMfN1tJkmQVLGFikGq+rho4e2NyeMOIlMgogyNIWo+gjqJIq2FQEAqMi7V8qKMy1SDzjyEE1HiarHv/aVORI9bJQQJcaQayhLcNs8cuoMtvh2vqaEQBBHMExH5wLdvy+V1VuKJOTTUN7N8VaSrYS01Gv41C50ha77dvWTqUQHAi4xH7rqK9EUsQWJmJKJm7sZvXKwopPZzko1GNFFTb1I2bt4Iuz7PfO9eePveZXf224fmrpz3udS8yBy45Nz/St5XPsJjHCnSyeO/I+tCaSPTf9ixo0py8sGrLleUBMRQBlUEURVWJmVTTGGjo70SMgkaHsmUH3We9zz1dhq5Hzf71kXceqCMTWu6/P5jvHqtITVVFGQ/aO/O5dgsLw4PYiGRctNqjC0StJ1/WdffDaFa6WioIggCxvtYpRLQpBQzeUICTMW9++0U6xD74GFgEvWru/38WADkLqLJgkNUroEWPmFpbmvOZivNS1AjOgUB6fNKou1T2PwRmRFZASJo9tZBJxSaoJGcEQCKF+5ERV1kRcK6Cl50xiQ2CIUDhxbctJejOdTaeaqtl5UResa0NgTPehcbjmye0vTd//JzPF6U03z4XHH3rdDfvf/+ml/f/p0WRqdiwKcDbq7ADWzJ73Osur1MXkPM6bRl68NAVRxPPUrBRAQRhBQFmFmo+vahohykkHISI4fzFZad4ee+A7x5GNiOb3XG2nyjImCYg/5aqjMNax9LqUTeUtLPjMVw8HCsqHL9R8kKYehVklUupyXVJ78vLm5VF1IuEj5ZwBkgZ0L2gMszYyUVt/LqtXD7XwQQN5lK5b/Sm5vYbSKGSjypBLCIj40JMLRbVoLadxubBiWUGigo4sLM+kZV3R7Lvm2qdp5ajbm6Oz2Wx3LllmG0NaxtEI0pz3d4t6GMhYRWsgRn7GfXiWsCMAWEduqpu1TJYUpzf2zpESJ1u7s6FRjIFIq2DuIr720NrDbu3xX/qZ+z73JVe85Q0/+odvuZCKV/Q+2W0lS0sV2XMj2YCZnT4nNtBfNlHL5GxkLUrtR1uT5ggYz6buCgiqAgqoooCGCJSZiRBgsrVTRFKJamEC2ReygR3AaOrM79yy5kwMRYRs/5GXoCa1MZ44aZ/qe6vM6qPLUAetxdX3VjPiDVdPvhbk9ExtQisisBFvstUHnvfV6cv6qRgG4ijARgHYWHT9CEGVlAubPHj1EpkSIGo5M1dl6ktMK5Mdqy7B8Yd+A+qY9Fz0raU12qq75YEPfbnYoDNvH5vuqFvNLrT8SEEQa8zi3BfzWM312a37mdNH2wwIikDGlydnZGE1ibGhbqEUnmyQZPd3nmZJwAMWuQHCJqOdGBw9a8QnaHafopjWOLLIUFaRtyJIE36aAJEqoAiZ4CqPrUpcTO/e3MfZvzXbP7tt7Rt/Y9Pz/PCql3Xc4oUXhHXf+9lPXG3mbTIoUo/NjismELc6iU6NVmTZL2wNiRmBEUVDMWih2UG8gdEog8hZtDMSjFqxQPTe1sFGEEAQtGBY0VGNRNBU9eiZiVxx4t4nfqq/uDoudyj9l9s6CQKhKCFXlWdAwaoT22lMtMaw4998Z8xH7qFka/fTFzK7OtBk66I3PO4OXlJljUMBhyZ5R2MIG3YV1AZkel+pKiIKxESKRHH2f89fueW6f/i+FP9p481jOwgJJ1x37rkU2vf9xd7L/tOcKUfj+U9d/cqp9NiaWNZJGZwzAgbvfKEmT+zgKpV0zeEWqoqiaueUzlarZqrE1AlGTatPvko0P97tgBpLqNYaeirP9Js3I4CUpXXWIxPW0DQrGkKVmiyhRDUIAFD7bqxPZsuQld0HTueb996c3nX0p251dfGqVAd5rFGJ3SkoKOERwmpZSrFR67AKw4w0B1FJDNthCwOAnQx9JIiYLG5Lzjk2np1ZiMDGyT5upaciOhsRRKMDAGzsoJ4Z9Hpw7Z9dujDTKxRj9eVfc+kKqaMejb0QMIbi6CWJY1Ml4/VvWbtgtf3Y1na852UBHAQXVBURZd3HYfdPsa0NoEoI0YAKIpFVZhERExKF7r2beeI0qGmiapS/9ktnrsmfd8WVPpz8ERi3TV5XnXF/5sCtg/d99qXvxkUej/OiV//1h37upVZKlwbmqImTLi5fRcc/sGnsq7B6w7qiZhYwIPjV3prSa7ss+o44JqM9bwZf7E/dsFIURmcNfltBZ0TIWv1+SntnpxCNgoqKtkQY0CFJ4KQA8W3XHs3wsJ1+PvWXvfelr7ppOl3c8In2iwfQURshdTj1GKydnl41P/+3ePlVKUBDpWNKlha6ZsqYWmPd3d9NgmnOWRoZmmDt8RuyCvBsVQAn2tEE6agR5Z4kFwhENMnsJ7pFAPAsQddseOFeZ1NNffFIsj1xqgSKGKuxVwBgyqQ/ZZTJJANc08OxgyO3hv1mzch64oRFABE4sUeXLyPfGCWzgCgIgbLVwCIiiWfI6t03BVFFVbRFqqBcr7lw4++Xd3/18z9+4+03LLeGAx1O9ZJNX99x4t1r/2r9Aahj3uEyueriP3v9//jxebsI3tu08razdve+3398OHrZhrWqTx4cv2kDixAC0H6pZlMtq4Iqm6A5vn66jPBEJ1bsMJJzppHBf65hJ0ZlW2T50pr7NyauaIyCDVQiSAZVXQrV8dNLB574Um/4W1cnJZQ7yi0Pb3ilrT31//fbWkdnrG+XNlGcrgedblx3V/+rd8jMd2S51chaDXqb6g2tvW+Y3ZaygHlilbOkzBMbEUJO+fSVZqWbok7EDACUNF8EWKnNKqAiIscIZNCBKukkx39G0EsslmZ9HQs0+cdvKFrEiIAovvJqRCHh9AjMVkxKgLEOWW3OHL0y/cSNBmjUUvHNuOWN2acuaI3VRQVUBRMVVJW91RBEFQQlmTkw2rzAigCCJrUMwLzxez6xpVz9+rt+78YfT0dglafnp9zxBzb97m+9nE9Gg9nYZOnt7z91y1d+b92re515ma4Xi3Un7/yX/XTdG7D3Is4Wxi8ZvGD3/4QJ2v22dSZL7bBdt8cWkO67EHxCj7w4QpopuNxQQ+18znApIo6UTY1VWjdvMJ2SEFRYCyQArsNjx/YfXgidacxfvfGJC8DRSVm4KeaHc6u07hPu1vksfuZFgNCKIPmxLdvjTg+vfn46WIx1aburptatQj6Yv/erN/NGx+jg+AatCamRZAMAQduTrU+pJaECKIGC5HWjH95QJZvylAiSomEFocaD5pk1ZzueWwzjDMezU7vv/i+rVJVAjPqqDkCqisYevACZQDRiLF3t0seS9ctf+xlPqkChblgEMn36o78IShFUQVhNaC5PjPioquAVs86/bEq9aKM9haoGRRfWLrCH0Qte8ufvfMM7xsdzOOLsf/t4ftvfXlyfgmzP3heacjC9+ce3dB88+FcvGJROjhcXPP77D294++e2fi986YkXu8Vxmf3n1m+xKCKoDp93W6LjopDHriQaa/roK31sl4//RM0WfLRATxVy/JZNEBXETN+967c/e/TO61qbkjxzFuDU8PSJE/P9at3U9pesSvNWTPjItSby9JcO7vnNx9ftO3PFeGn0lbfmIX/0T18WXNQq8ztO7tw+f9n6pQt4dtNNhAACULvhPYvy0Hd3Cw0+c+H4dUDUMG8VVEHJnFg1xyu0WJ0I/gGAomQ1NMo6io0ql4hFRFwpG8oEcfyMT9VeYNPtdftU/e3FVxZjQgQxUHsfQVWxbpnBTgZBZOJKIDj30BXmH9dtrI0vGOoqNtdi3fINrKxGBTmKQYBGUEgiN+dVZOTu1/ZEVUgVDQGQRll7oNU50cZS3vlTP/up9606uvyVn/7nD49uetc6/PSj35988XpuA6H7ye7Hb9J/eNMrtZ/tuuu3Dl//nhfET2GVHVM7jNEev/8d2+YnLglaFeBMTPZ9cVudl66c3xydeZjn+hG1JgumOYF8zufpSqwWtPj9tySvfuPB/U8OjTWIiNbmM2uv2zA9AxCDoK8wnhaf5A67N2362JY1927x06OHt06f0fsXugYkT9cPLtsjuZ9af/piWVgzNIZUBVoLJ05N329eXDkCZ4Hnp8CYEKE5ThEgI8e2tWSlDoSgk3ENaCD1K9K/K1k9E05045ocDgSfJeh1GdLFwg7y6uhn/qDVbzESC6hnloYDif3FdTVFshE5piEP/ODr67/7aSMmTPUpNoJRoNNXHV9dRW2xKAizRVQFMhaFRRHQRNTx/kt7DRsDKbEIyD6c6W8ZzYx85tL8/X/4Hb/12vcU9//5RVt2zeHoimk6dOT7tRdm9lxreGbNz93+2y8YtD/1z/e99E83Dnutk7fncGRL8GqLD257w3xUQBXQNCk6pZV0dCofEuhytTUk8ti6VBWEsgInEiby3NlsrE7Y/vBX3lw978VlEThUQdB0rUNl1oETTxl688RpKIuxPR5bL1s8uXnnXT+0VNFt/+O9ldFLleviaj7xtfZxLrrF6oPXry4TqxEQCUadcTfdu6Wlq7qhqpC0HWsr6OqG3qkGZXGnq1eSkHN1AUBCFwXOysQpAGjD6EagxrURG0UYmFRuUIWUSRSxSliSCg59zy0Dg4CRUxyPy6gKRGjcwXVkjEFNI6OA5PtOv/y/n3gZq3YHzgRFKyIl+Hu3WUdFHU2IwK0SRJ1MS4W1gjE1xAzX3b9qlZEYXQ0W8thChqRKezN2WdPgeIz/7vd++Z/v3fmeG0L8SZOO117Cj9uCqPXhO21a7jo9veXhj7T5kUO//J+787kpH2sHrRQo4P6P/1qPNZJVm6rkibchkX+a5XCS9b6dZG3Y8dZlMkI5VM1yzjzhu62MkWeSGCYqfBDI2RggfOeeHzFpXdVebGd29app5KqsgqgNnLmywt6JU7UOFsefflhvPJYd2L5wHE38fz/815/69B3vmzMudIp1h7aOlqdnl0/ek4KVESTWZM6jdFbHx147Mu3IibYP6ubgXAQEBEHnSIzb//KSLFmrE5c4kWbuH0FrqVJE9mDHapxBsmbJWxbDChJZyTiw9pnTGnFkVpVkcO3bWnm7IiTLw0GY1HfR5P7yrI5WQZhFgeHUztOfuXZLI3HS3D0iY/fvPV4vI7ZQuxhhqRNFhCw2jwBCB4p716uATUZWRpZmds8TLyXl4zuWkzyx3ZyGp1/2R798+u/qry/9gvqqAwM9tNbVyl95+YynVW64zf5tNf7Zv/7OJd/KHSVHHCabrUj6sZuojtRo7tlOZgm1Hp74wB/L9MLUk6tQK966DVglMj6lfP2cmg21WqiW5m6XW6Tu1M/oHIZUiPjUUs61y8v46PZs9+blR676ukl0eeOqwcce3JYPrOlp74G9L1weJ8Zt3g9J1xpFLH0etH2BN2twQy4KWB9tEyirRksgwqJiFtzWFEpiz4jK0JAq1BnIURbI0KhuQZkTAoDGMJGY1wl5Wi3BU8+RFQAwBmZRDhLn0JWZCprQH7Fq06OSwR3T1DI1qsTAACb/7M7Z3/nztCn1h9iUB03ySO9UmhXlSHzQwyFZZAC1iUGO0ojasksf3lVXRAwIRTvXr4JGr70zOyoEueNQzEx24Kr3PpS0jv+OtRbDSOGhiyo794HO1ZET2HB4y21f/XynPz1aMtZa86HX1YNTT6C4Yw+9ni2iJTQieTtBYzCOXnLDjTOlrQ7uQvB9cn0gFLWJ+Tbi3QRdhXtHnnz8oc0bDn99aVg8rfoKYFDFmHrkuox1XL4jeXF1SC7uXvtkq09Y/+sP/dc/7o8z59v5r338L2f/dXoGL3j5F2ZtJVKOmUAcc/dUsS6ZxaDg8OSMgkaFCU0J0fgzF+cnNY1qnCNszKUQkIHVrF1eqNhAAIwNVUdFWVeU0hAR9OwaP+nqzX4+RAAA5mI4mpr2C1E01hXbCR+ONOmv7485oyYjQ+wfvr583sZTigikMYIogoCdufXIcY0iBY7+yFA3NQYwSQxJFBAFFmjFvZeUIjJqR/ChdfjRdRxa8cTMmiTIifsuCyPgJNb4vAPfe/MaADeuYHT08nJQ3fFSP6TI2w9e9KKLPlLnS7Kunaa1XASLnbe/aoD5J6/ZFmtWAjCArQQEDGm87RcvHqFLDmytIpX90hpniZLEfrtq34opzu87dGoJHvygfySEp/9dAFURhoOs5ha0q8/ftH73uvl1x/ffuaBJBdf87x9+QtVVju3huYc3vHnBpjufN//oqlKSIkHrfKKV9rZnWcpKgm6p3UAJlAWQrLVJ55FVOO08WUfRi20OzhAIKQ7ymLSDQq9yVYysSIQSmIgMTqAzjU/XMz5VFEJQptLBSNJ7iKQqI2BT5gVEu269NqJrwgro97e2lEujWTAIqFFAAYSVvv937Ilc83y++yf7tsZBQBCwKKRRgUUZbftIf71PPCCo+Uqfjg8KrY3Zu7YFlN9xHeaJPdQ98adfOH71Tw5LzNmBH4xtMkfvvLVoZ95cdGKXed3xL+WS8DhLvF0qZvtb1gU4+cgrlx0DKwJQWhhVEBEeuqXFtHtmao0KGtWcsAESfZsxBxZrQjCZMVuOpo3qHMB5CVXjxcADSkOKSfeRF73Z7W23u7NffnJfOFn5das6L9xgRgLtLx76/dkPnWGeWt95w8cX0+Ax5TFTNI7i5WEmR2NihNPJWJAsIQAZQwg6feom4DN58D6KhJKlIXaZSl06Nx0gtS43kk0wJBprIWwYyo07q4o+Y3rXwKACQl6XzwBrYmJZiQZu9oLgjq5G13H1irzw1OGEdS6LQCCgcSItgNbvPLHXRK3Xffof3zXGSMyRrEZUVlRFYdt6YHrKI5qin8FDcx42RgrDZP8qGBXDO14PNozbix96+cz7rxyZ3z1BIiW59H43WLgtmBqovOpEduwla+9Z6iTayX0WrQkkp7PiK9dPa1WIMACbLCEk8GUd7HB1Wk/fXXQcj8VBqcyAKiv278+5WWWGJDWxdzOe3EJu5SDmvF5BqFK2IOmEVv+r/2522Q53zNtj8Ei1lIz2vO3oL0WAlvvYX2w99K5fvcu1qLV089qvzMzCwJMwRmP88dW63qoiICy4WoAMorFEIMxc6o2nWqur1HA0RZGIMLOIhMRWoVh39x/uLZcoZWcbJ3DVKIYAgSbFZgQ5t6bjiiwas8YoOuJqqRfvbRmtqwjqZWJj5A5dYvxIEtGGImvvubIw4xFqA3AUaNZ0N0AcISqf/I+3XHAqDVYF08KAE1ZQJBTnvrYpkWiUJD0+tUmnb/GiPhy61GN694Z1vNihuHzoNe+7Ym1CF7UIx8Fkr4w+bdUJRrGjdb3+cMd2eVhrKHxFHCkDLby/6/ZhWvoIghhNhoik9djLGE5ykvfWqq+qoEY0irEA5ttN5AiEGdHPDvedsbF4xvRuQQhVI3tjpzv36+2P7SliZ/nRLTv2m3yxP/eut19/mgt96I9+Pfm75Z/6l3/6P/esmT70g519oZVomhsliXA4bXeVBQFx6FgVgBGh4dOvWt37yPf/9pNtl7ddrOpgGziuAYnt1pn9P592ZrtJ6niiRkHYaOPJJOKAz1KUQGXhyAop5sXS4N5aow9KoCuFyvbRLlsCr5Ooh4dfOWhz2yvoRIUVkNA4y1OWQbD7G+9cND5VAJMVziQiioiGrIPd04CgVd3Sxy8GV1zrvHHDk1eGKf3ad0dy40rc1Be2XzS7kPzIhaVIXvFP3OpZOA0dDsWIgNsbW4+pS8h06qIYVs5T8tiT0QzaSwUCAaNTVtQQGIp61iHcdz1GcegxRwXrzLMYF32LFtQQmiQZFjf30IyKZwa9wU8X3VzXyOenX/Jvvr7l0pjv3XLj6b3oWK/YvLgekoPv/N2jB7YcfeE7I6xZ/9hPHHrJE59eykLJgFZhKrZnFJEQWCoHjX8PM0cGpGx0ZPbSv77uze/7/AnMsqKdOINkXIKt+K8/8t1rPvIj2RkZLkmjgozG4eSgfZKzgQCdA1GcBR2Ac40PWOByz7HreTxMUammLKoTMNXpy8aZco4mEEFa/fHcplq6bGAkCQxQOe8rxlY6JXWvMpbkxtnKqHgDSWpGHT8wwMqA2PncR6eWgovReHfvZSO/pjsgoyfbq0OZPLIRoglUb33dwy8dbuTlcQmlV5BFjVgnc1NZXvTyVf1i+fbB2iNhuoxgom9BnQX8+B2VsTEVbNl+XmWETng5D1q1enlXj66SUNUOgoboHJMLzdnVykHbuTQcz7bm97IiKZKIcZnB6eOb5w5nODxHFWy+E2ODAb1MrNlx0X+df/TS//4mmumfoMvs6ifT2pUjcuyefMu/v/AjGx56NDl+1YuvuTBLPnTiNbfZunCihiVZbKVTQQmiUapphIaBEZTBgrhNZ9666nX/+bfyv3r7bTe++uc+8KWH9h0N6AaP//2PXvZd9/7M5y7bv8fZGCM6UkWQUJF4RE04CpEKxBCeyU9nAUOImIRo53x72leVqop7ojttNU+ypaKDlVEWP6w9w/i/X3TvrjVdYNPRUCz3Mdi1Op5Nj97zSb6l6GUkE6VjBGsAESwhooi2eNNPX9XjkkS19N2G/MTJ6XV5HarXr680jWdmTt3xuoVZnkpSSftWtNVLra47c8+fXX3rrVlsL60e3PKJdDv2NvhhgsBAGoEunSMlYhth6pGbRA3HGsZsbFrhzKK5akV3mQzR/61PDxkCnJ2TGVdOPVV1AoCDGnIyDH174L9+8hXvu/TAvS5lmQH5zn98iQDNdfqtkz/4A6/5kR87eKqC7ZfWydKuf/83P/zGd+YmqmXOTWC//uz5j+RFxc1hmgFlW/Rnb0pr3f769cf3PfbwA59YLpc3z87KwTOzs29466U5Hz/iU7SOeLKbQ0EiRARqMjs1+CxHq6LKQEjGm6ktd23Klitm1ECsGfuU7f51nXFVGHYhCiHidPx/brn2+vW8RPPuJkoxzp/acz8s7mlf8dvXjTEkHEOjr6yJVVSBCUMeacd/oJNmoYOCy34VqoBhn+/dxorl69aQqQ36T2y/+OujIizEXXNFXKw7SH7mD/5kbfeu/vXl1FUHceaDH1i9+gZGNllARuLA/eume0ykqUa892UKJsQ69daUxuX0OBU1qwKqojVk6BmFt6dGEJ59sTcGzfDAxbJYb4xPQ1QqFVqObe+QWfvl9/OvvaF1+Gi7ap04+uozG9ft+cDPuTXtvHXPD73mJ37o+q1X/ORVi5edXow13/4dd/7ja7Ze9cJrW2L0dAs6G4uRaRBQcXrKB0AjhITCSbseyUIa52Z049YXcajH5YG/fHDX9umtF6/NdDQ4+bBPUiZkrqEpzyAiAQKYJsEGfLagp5lXMoCI7Y2bDt1ajtiRcqAih5rIrH7kOlDDJhogA2pav7njK63dX+bQoqNnNkoBOIYtVf7K/7SrHc8k7bFqCIoAKJAYUeHAbEjVtM2SO1jN5opijtEsRKRQRd33Gg9ptTqmUEu6+/hP95amp977oQOvucxeeUnagwqHM7/5ml3s5cxww+GZJ9//lo88eavxYBe6pKhB+PB1UEZjlDg9WDl2InVp1bKWM10+cCH4xutM1SCa5y5AcbahghoiM3P1FliD4+mnOOcoAFXYkuVy6sn/eddlv8P3ztGZzvju+S7RuvyWj+LMDVtO/slX3/but2+/86//64006A8PDjsujm5/8+KdX/w12XbzlReuMfuncKEtqAqoYbYzLwA2gqhFILtcJwWGy2xpPaNNSV/8uVd+vwqPqQQThlFTZGAB37AcwDbHa402PiLqs4EopqZH0VjFFGbX+uOXLtZiMYoJ1BYoFNKD31/FtCYoK1YWdNfNTO1y3gzTvNx/7PTsfrlttLE1mpsb9NK2DnLW0Bh4KjpkZfU6wWpnFdoZj6KQ7J/OR2CkLM1gYXvFNvEOeOokyEuB4OJf+tg7Ljgqq6dXP7a8a0Tw1k2hR25EIeu/6oHrf+s+Tgq11BEC8F7qM1vKCELkbfHorig2aB00oHdMEp7campWIVQVC0qo32Ssf4Ogg4JFwHHHcxHPHclOYg6jrPbT9PXP3P22n1+emncLuOn0B8fVjXMHX9v6iUfqx/7Ezq7+rTf9v8Uv/4+1t2JID3V5mmTD6WC3dl50dM+ZB98/vvy7L/NZ6ps5RkLbqgKSiSxEBngk6jszq9QZEUCimdE931WPZqVVV0URx6JaQcJoVvKMRmtcRBpdZORnG+mtKazFiMltp7gnm9vLquI1WRw6T6jpomys0KhS6UVF1CZnilGBaKnOL1k9Lu7e/d3V8mCq6zlhBcyXgQVVAMEkoMAYyIiQMkEcB+q1WMHt2QJiIJTeHU27I6imXbtW42W77eVX/vmH/+Pbk8osLy/8w7qdFkP/5OHLwYzETPdbmy45vbx9ulIatSMCBOYl2hIMMNlg4t4f9Y5sGGURlGIGUhzayZ5VFVHIYEPv/TaDDqrgSNXOxjWzbrY6b3pXAIBW1XJfeu/By/+8Pb+1ZzkNevfUK7+y+YL+2uMbX/P8539w9vL2X7xhy7998h//B9N44/reumUuutMWTuyuw9xLNvSP/7ef+llPZ+YqUgVUT2VEVVFRFTIQV40DX7Teszi0hGosp1tsvmSlnY40c4MWc+oEwTRUJhRUUQEQaQQUmUSeCZcyjoHYYZ6Zu6+oBECUSQ+NQ7ChtW7PmtbA1FYQBchGwFHMEjGSqY9GYHzXreVSkerQZjQ2rbjsggKxIIJxAKimAgpiCIWz0BIbEdTsfV4EAIVgj16QlMbbJAZY8txetIM1F95+bXYEs7FPD1zr1ZiL9n1k9ZrKQL9z0ZHZ1aenrqvGXRp7o4qs4WRnbWWFEQ0ujDaiQbN49GK0kIyTbgb9nSGIqpKCbRCf334qp4CJZRlfdn+9OSx1ntZpVPKv/eUT175h7kxtaDzrh0EG/X//8zfL3D3b8DUfe96blhZ+4/H6VT9tfv3iYRukP+tgnK4alquXh8H45anWBe2F4S5woXkn4XrsjUhENqisBD2Trpl1lWMExKiuTNKKc5/FYBLQdK4mohhdXGHXTQoRTedu9AT5GUGvJqTdOjN04AeWJjX6+vD63NhR2963DqwdtAOiRAMobNtqNamRMw7jMhtd6iwP1xQcNecy84LNjIhoLSOAqRhEiUCDDEe2M0YAPD0dUMhYCQubfbAWbO2gTsPpIqSnX/sW53/L/AeScHK7aNXpbf/VPIw4AFwXU7x3x/bxNO+3s16YFONCVgyoMTNYytsEAU7t34gYMTdFfWh+0+kG4I7q7KQe9Q1j27Rn9AoF46yKtYNMYofpKc9Q+MxfHn/tj+jCYndpdnlq5NOk95Z74PmLd4qbG+66+vDRn+3+4Iv+y21btlx2UmOnmqllaSkZrOsst6ermE5b737oqx97az0VJm8sdc1ONJAaYEakwtGlvDhbGhACBevO4JT2O/1CDUOS2rFBDuImhqUARAjQODciqKoBFXs+0RUR0UavwVKFML3v1JZxaB1ft6Rx6vDNpa263N33qtpU2TiLI8cCko5MkqMrs1HhQQGjzNYGrMGYjdNeZljuvWqgGUMM64Nmy5AupmMj0doWR0pQiA2XR106cDwaUXLfTw4SqnJDELPjRRqc/YO9m39z7tdGo97MZzdMBcrv33SLH4vlGse6YWgPv5KMN/Orx+S4gpE9s6HWktgpTt11W8qa++NPvHrBUbSp+uy7DBODRQWbC2VGzUSUFRqfJT0HkKSzqzSeQ6MAgPGJUudEhu7xzZyFs3r5JCHxkP7sF3/6TfFRQeN1MOt9mtSdzoMvf/RLVbJvT3mm47/2HZt2P/BT08XqB+fLzZtygeNPptRfI5auXxxPra6x2vn7txWGIzHheOYw+JiDiYkNLIAugfHW9cEWY0IwoAY8qmS+iAmjasgu8HsjxYT7RTN2EZGDUVEkZXWkDGCeyVq1BMrBBwY4uMaMeJRUqDo0Uybmtt07uckIInLsfHF/10jZdiFhdONUEKLgEDIgtAmg2JCIbz306BkDNSaUN/KftTIDADmZLcdce5eG8ejVa6UATK1bmv7oPI8SEBhRzARcGNz36l+cGWzAqlXe/4JWUiUffu2/dFTnLYyh867PtE+3NQG7NA2N3zD2iyZYzOXJGbTG9++/FliBDEJ37gVWFRpJhga6Ck8pwTzriF/RE175mQDQJaBQb26Ol5pWUsYuj59+xU8sHVqfTqfUWlsNnCzwcLz+a4e/M/vF2z77YH/v3CV/8Mb5N/5Ium3/EyRnIg+M7VK6MZ7Zu8ibVy8+eTLV5KLL5mw0TKhObA5EjcxE8xlwup2onHe5GBNHLEgxIgS7ZhNrETSviRCRjFFVURVVNAYbSeBnJnKkwqIoLiseusIGU+U1CB63qUSLs6dwM0QEi3D0Y99XaWscCOPhnQGRUbyYM64FxC5FoyaktbTu2NnGcRURHSiqKBhQUAZnqmFRB00GZuum985UlSuTUA32d9ZiSDOKnTA2XpL6TPayCzBZVhzzkR88SVOsg9+7alrb0c3Il+581W9/4g2jVjV68PnOC5AXN3+BCAAB03h+GyDW1ZdvmbemiqswaLWj3yxrQobQfFsmmysPVUKhJK2hGuyM2Tnr7ZxGCSaPnXntfA/GKRgTA832PYwtv77f0Q/+68Mv/OHB/1z9652bbhh7Gm5Y8G5VYjLdOfPI585I5+GFl12+CkO7aPNUchzYilU2lRqvHAVBrYJBILd6OmE5r8BAlXOR1ZCqjSGd1VO9qGSAGpQ8Nc6loE1dF8EoPFOJQmIQVSDI6OErKkHjhCXd2zKYMHb2bGx5QbXOPDS8QculaMP03gPo2x5Qg9qT3VwQXCKIhAp2ed8u6bNDTRwosKBJDIgImGLgTJUno4KXjxSDUeHqGGL51Rd0fOoNehNHESCmJ7ZsGFVmbLh4aG2Wb/jcD8y8mz/dWeB+34b3t24f/uwt3RPHFgbtQArqhfrTocGfZ33YBgL+X+JOA2PM8kR9gD5wAyREMhb1OVhxPX2kiwGkJGU9U26N6Xms30F7nDzwll+9+VhEyXtTNh3GpIqZteOYWf3xY6965/LU//7NDTwfpZ4mvmr7ZRdQ0T79x7/6fnrLb//mle/8qfUeR/XI1OOlrwXLqMISRpWosKhOfNVcZ64wT8lFzMA0hRci4yBqd2tRGgppw3OjswxLVmrA3gDPkr0zoyqgt2dOb5jXmAQImhxe7zCJneSerSIEbFA/fPiLO1pjDNz+woubV1FBc2KNqwgssRq2HrMHkvaoTHMJsWtIfO0AUEWMSyEky2qZmPKWADLAsnS+1H7+MuXW1K1xZLIcpo5tG6/VSkL3U+/+t6ueeM89v/fyfP0f/0C7IruUx6tm/+2OqnywXTvgRDhGAJ7yYokoJofnpkai/rOXGs1qU2vtTWsE3AisITYD/f8ieSdFoASSo0WnTvEs8dGnPMV/+ejzTi2MuvXU1LIx64dLm8ZnUunMLHDn1W+k3uacQ5Vyb6YetGb9Bm3hXf+we+sLru61rC69ZMu89xaBgr3oFNpoRRHskiSioKSkSACuM52TEImc3SnSABlJQEgjGAhuo54YxcmBQKMa2aAnSCc8F3wmGlaZSRmIsgdWF6ccQCUMYfk2H5xbrUdu842ocv3vF87kI+vqZOGxd3ExTkSiAJ7aiIxkVRA44QS/fonJqAqGgxHUwFlWRgBM84S9N6vuSbaUmBdVRuztGZfd9WZCW284MbosirFDULO0NdV0ENInf8W87pPvufJ/Xyzzr/nTfauWbavMtrxmPHN6OulfeecWcCjBA0TX8UqASPbRixQg1r/eXhhYj2YIbFf1DE9Mi2hyR77lBP902jmAGMUsS49sU+sCr9z91JcwfRS/+lI3nXqfzsADg7v+7rted2MPx8vpVB5IRh0K9fxJWUPaGaWSJl/786PPe1fij9ltafErm3XHzOFV2WrQ1iv+sXJEggbTEx3DzfaYESCdnukaFSTWs5qwWCaiSKAkwVgw1Lpk1aM9N0ybbZOYpviqjcg0oJ4PlzrbY2vPIsycfumq6AyJCuoZvzlGLtKTfFm0qCjs115wYRe4pvbX18wqK0KMgrq4TlWNEUBVJqoeuHIcMHhMuqiqkW1WqSGTOjDqu/Gjw4guVYsGBKw9vfDaoYtePz6ogAUPLZto2qu5DGn8wK3vefdvvPMPNmiWTm07kKQ4aC37K0f9rvYeSPbuVOXoPWqdtRu8Etp91wRi7+NocTblntTRV8uxWrANXdsCgDbIgm+eyJ0N/qQRigK2pvTIZT53fPZ5PZxOfv7zP7HlFPKx6fn/fmTdp9/7Pb9y+Bd/5etxrQiluY8bTC8cPDoaPk5TnJet3T/9yzt++frFYXrhFatP/80/fPhMccGVl12YK8AVCwNERUCgA6soRgVmRFDqrp1NFVZEXyeN20aETGPGKUYxW7cpE4toTMOSaKj6hA0rCp6N1lTVnlQRPdz7E0PrEU0kPqPdyti8PADrB47BgIh8eefRWe5l2X1Xehx2xyZGVRl3FJCMIqpGB+NjmzG6NIl1llqUGFiZrCJBjBrHhx9+h4sGx60aUMep35PPlt6uGt35pqIfA+3fMhXbF/UyWw8eOPq+Qxf+2JqTH8Vt11zeqns+RXNsZnq0VoTq1UfbPh3FGEmCSwVRVMEu71SKXqbE+DN5ZqI1jvNTB9YrKxgyDTnkOWkRPGXLRiCgmLT7C5tC24azxHJXDP7uT/7oTffZIbZ07we+8qaL72+/7TVf+Mef+YGfWxVmzCivTi/xTN2rp7O41Lb6bw7d9O6630pPPrT0xPHwYHjrm0cx961xYur8VExKIUbQ4x1HaJHBKQu1ZtqoiHKekhgoFqTSCEOq1zS4UbLh5IlujWfNvQVVdeLFhwIqdmWPygYhAsJSmSpViOnnli8eOsI6GXO+d1ajwan8wQ1oOSnJjXT0qReMpCx8deAnVKNlSga1T8ctb7iboqCop/yutZ1+EjRSTqSsY6qnesmyq7sZ+eCK/3PznmusxSIS4JDL7F9einWr6vzzRdNLWW3ki7+gcQ106yo59sFfXm6/Tv7yw2CSV777gjNJNECfvsVO+SL/7CWRc67UYJ1K1YLIZD1kp4uZmGAfS5DgohXLJkZ35JgrwUWTGQuKeJ531YqKxNmijK6ADlZ+1+BGNeQR0Hcfo/X9uQhRC+/bY7CJ3/Obv/K2r1FJvpt+4TVv/JXeTXPL8qrv/ti/+/pvvyQMpvXx8Ug2rk94MJdM9d7/11e9bcuZk3sfOok7Lnrz9s2/eOR/nDi9uPaCCk/4jd34wHrjSkhKPLNqFG0wWTUWi9lcWwgFjQLKyvXgqcsjEgMxAFgQZOeLTcf9inOckAQGFXFGFUAM2nOJHEUEMDKeJ6xG6frDnTtWz54ZFr2swmjObIpEOerJrRjyuiglre/emowZhc7gxmDnhlls9KYLVWpcCpyLowcuBkZFUEoBNApQ0W9cI1EYRo8fs9cIoSoBMwVfXjakMtfPfVcaSu+Gs63a3NIpXXJi1Q9tHvfXzZ956y8YHOdr5niqqG26OXEu5l/bkPzaZp8sx9JzFHl8p6eslk2fmknVD6IFlabsLOzb7okLoqCSdefcLp/LSD8/5VMVUrCn1hdJYQB5jJ0lmw1x+sdf/W9PV8ZZHcXwPVf82R/+U3XD9g1zL/7EH7zxht+7qbcUT0/j0TWXjDt1/snfd++a3v3BA27dLS+6eBqVE7rk1IOjqb3RnKDe6W3h068Z1AYlHR69hJCUAUjLzrq5VOUpVwYAstQhPG/zSaoCWVuaXZqZ0JhFJ7Q34KYi1zQOmRVXLQyrRGfLM9N67+2V5tj1pYH62AsiubZWR18eABmIq+TO5xsSEhPsDFQUUVmA6piLGkuAKpkC3f09IlEJ2eTY4OymDjCodaihP9fv/PSbDjhLAIAhivn6zNaTlmh89NZlhzXXu/IwTrePEjsazD2ctu+pXj7608V+1JNTnx1gCx6+9PFNGwo58tJydWmXa7B5YJU9Oy0NfNJ5aFviw0ilMe5F9Imm6I5cRgYQCOS5KguRrjBAJ7QBUDYKePL6GbBBDHPCJq/T1juz36V5KCWzjnvrD9nb46mPnoHuDS/47R/+bzf/8rvj2ECwIaf/j7m3DrftrM7FxxifTFmy1/ajyTk5cQ8RNCEEDyluhUCB4jRAkdKWW4pToEihQPECxeEWLRI0JCRI3D3HZdvaS6Z8Msbvj7WPxLiQ9t7nN54j+9l7zbnnnOMb3xz6vume9/3yjM5/XDd10vNO78QY6ohk8Wk7qkYssfS8etv68y4tWn3LId21eWNaAQRGhXVzVRMOnK4dNcHJ7k0ItH/DImaR5sR8GmOMKEQgLDAaYkNhwAj7e8RSQe91PQ+/fOBk1apcdcvratdYyqPEZHtxkKBtwJ6lo0oTzSAvymLx6JoCktk6EZx3iRJmwaHXUZRBQBEsUj837YBFS8SMELxIbA+FIUspLM/0O29NNqdsUASx9i77yeklcUpXrFm9mAxSlx7Zb4x1//OWm7fuWm4pKfIZaBxEtl1sCnG8NbzIffuGfONBL9h8UDbIF6bmBz1QSVVM6F5WtxvutidKCFFHiTECIpJ22cCXJARI6o8P1WTFplb8/BXEVeQtZ6aeI1Aq0muV9fT3vvNfrS2L0ODgbJHZ4eDIY7xZXFi6+HWt133o++9ceu+J88txetUg+9RHjjrxB3Dqc447nHt9G5JKkHctHLEVhvbEGz+8owWvmHr6BaGnIzJ6Pr7bHzmLLp9sozswgzoCj6PFFrDCfYtBQACyifnADEREIqMgRRhRmEAx7keMjAJkNfe+nK+JdUbq12MblYoZBhUaN0w0KlBW3aLHuyoYJ3H8O4fMLhMAqZs6us7FM0ZRUieZiDYgKMqBuSxZXxMDCKMBhIDgG16CSVNyyLhcoyZjAVCg9rS49YT5DGXqY0dVuVsKw87x237yy6uPWHf0o46KA8OUJMlEm0EXseAitOnhh8Dgkiueu/sD6zYdecgwmqwqRK+f0w1Ti/VbN/CIy5mjECEoB17QrXEMpK39o7XOACOIRZSVXlJEJLU0f5A4ApEhKEVF8z///rXHzlWG60yjCSd4SReSGHnthgc956fvfsthf/7z8z40tpyxue6lt5jJg94xMzWN3iuLntEALqaTdnzPcMsbF/6q+PZbH3nYzssfGsoEg2SrvWZN7KOsWpeKoNqXGVwZscZygoX3r2IBIdZt4wEREIRFRkQ9DAqEgeWA+fSIgOjn9vzwPLfcCqrxjReNb2l4cZEw3ngUE1mw12wQ7RQ3akzOfya6BAVw8o7F8VqriDHqyGNZLUqDEJgiVxe1123GUYhEAOwRQ8PHmFodHfUpb9dVMZuwQkYX84sObu3RWRy78sks1eTmDbe+74LJh73mtFRtrxioMqq2VKnK14PallL0J4rpg47mV/7qyMt+4ejUE1vjzXRYtW5kLDVY150RKWIkjjJKs7uWb9R9KwjwJ1n6SNVwwMMmRrU1zUJUFKLRoorx7ktO/6v5/I4kwrBpVXqm6k9mLeyjD8uNP3/ydb/70qH/+6gXtCr92dfXr33EGTrogXJMKuhmjaJx2wyZqTm5cc8TnzD9uqede/4pVz+UhHSMioIrrXhnNm6aBX/AIP3KFWEdxiOw2vd9JlAirfYSjObRSWTUIMdAAoKReX9pFTGSNqtP+7cJqypILx48sFs20wGg18u7n+rBNkK89nEREL2G9FI5tjARhPncKat8nUsMIYk+BwAiAWIBkEEjKAaJqDSyRIcoeenAKOYYElrOPeYN45A8eKl+e/KQaj22c/u6vNCL9t0/OuJ966uDo/OqIO2TwJRikafBApRK7zgoFV6y+vEn/uVf+u6Fl3xiWVYdc/jGU9qt+TrNsos6U96VqjaRR0hAkhmIy8f02zRC3brrO33/9niX7+/LzuAKghQjE20+lCORGGLgIk2+S5+C5PaJQieSSJ4d6dx8uOa27WPh0IcumKUNJ9zvG8s7Jva0d7ypfNbrJua5WaaFwZhQiCQ1mh0Hx6K3u7fjzx9867UHf/Lhlx33eykxsgKT5GkdQdLJo8d0LYT7EgMjoAHEpdiO6oBBawGhwI3Jhf34JKNhcOEVIBrcr3Qi4QDZpsPndFGMNc3q9cP1uxsYySfL5UGeTMOV205An8Q4TJIfHzW5MytRmJO/iDtW96KJHIyrsQZBioDg9VCv3rLT1CBCpBA4BoNiKw+aIgMGA6rWSTIKQFiWNp89BBgeesW2HRvM8ItfPOF9x/V+bdZpmWsnQ0RxCZTjjcgDCVm3XJ48XpJGRHdGOpdz60lnxvnNl//+O411D7ltqrnIkzvWtZYqv9IGSQTCDZct2xeRJyBSd2+JvDel0wqKx8oByCIQCeaPsw7ZKRZnqXHdG87d8qH5/sYnHl6B0vq2a678dr22ffJZ/kf0wMll63dteuvrr9ndImqtOWR8zo4VxpkkRAohcalnxYer/HYJxSPtHC6d9Hf/6y8ud5hzUFH0Jr/HMXY2jsWCjGa/f48apVX7MWHDB0BpIKCIHbMiHGPkfAXAQo8iOFSgUK/kHYRRgEiim3St6O3Jh1SwnAVXg4ErDqLUN2Nyh19bmKCI2V/27L6pQNVh0vbVxFCT90UijaKh2bYjQWgMxVbPeOQQgaMNZjyiLqBWITW1hjQSRvDWac/o02Abm+fx0tk2Ya/dmnr5qvJLHznknw4ru/DLF+TI2RCISp35bKawFljS5emLb3l6OkwI64R7LY+42xVh7cHnLN160/c/2XzEQ9bDhUcy9PUgxKRvfRIgj16VBsEDK20VSEQE3gsrhYC80ldwgPJHfewHLgYBJp/55mJy7UNL7Uzi9fLsgvXn1dv+6mnHNK/6+HNOr2/69c/1wjUPf8ZZB3P3A9tOb/UCKgnh9H+76Mm3v6PGddKs62CUBKrRJQJVxoNNncotNrsPTs6/6nWT5rGfGN8eTZlYw6ku2+TKdM14RAJx+yNHViwKMGFn2Zn9pq5FRyXFpPFQoQlpHI0+YmlVEAuB6J7gR6IwE7jKR0BSGry95UTjMhvTa9dph0mlRV3lj/VRAMgkBx6JqFSZG9IV/ufZcS97HAoaAoij0JE0oAJfjDhnlDaRMQzmus3bclsNGq3+CR/68vPNIf9k2ZZ+eb0CCLEip7HfwXpcd0M26Jv6+nWRtRFYYcaNwfsQ+yE58djn7/n1tz/73HOhTcoVSVb3dVbj2O3Ls7DCgsloDPjwJ4+2rOwB1Ai1jMU4rhkAxE9tX01v/Rnk/zl1dXrmSe/tfn/HaW84Je44hi+9qHVd8lG/qNtQayqPU294U8YP++06iAAoogdN5VMqWkNPrSNh91Xnx2L7ri3Lj3zdfHv6drM0C5EVaR6UpWQTTTmg6Wm0TgUJmWXHDIhyBu+0OBGgsQg2rkBIAoAg8MiDF7m70jECs5LSRSEUEaT+rS+GMJY486vjlWCIacwumZzeBcQipEeujjCARFJGAWqIrZ/crkSiDxIjEmOiGEIEBFECAAQcRtlglUBEG/uF0QsPqSSvcfbLb7vxzPd8KC+Wk84lZkqxkPGtysQ00VkZ8qWuVdnw8idESVJBzUSCEIKrgMSxyI72qWfzh54bj1tkpaoypZISCjccqUBYGBFRJTq42LxPOhdwBttDf6XtSCQEYteQH3+Envu6pZvpkn89/Zc3PevkjWtV2X3fbzfl5xx/2SCQQ3Aqoc+tec5w/VkbH3SoRARiCJlPsODG/Mz1/zV3W1y9qKbbB7XM5vc0np884Nrtt6xWHATVcLn0dmwqregunVsISMKEtxyEoit7oNIRAKAzD0biCsLQyI0BZIX3CDREEUEoVIKEEryH9Aq7tmBjQrz68cAQLHu59P4D0iIc9zqRwkFARFuwAhLybz5fGDgwRFYYTaoihQgoYjhYYgl7C3+WmaCcG9jrl49dDNOJeeTPzvvY6Z9qz25btUC/PGWiD6UqFxvAVpxR2Cz1+C4Pe6rUY5aAUAQUksAhIpAmDjrA0hH/NvzRv19w4lNbNS4lCQYr205RMBpMEkqslP6PBhY6UOcCABQZKb1mkyqjBlDkzNI796QP7C2W1ZsXzduOX43Z9Rde2X74Xyq/5zKZKrDwSaMu6aTuaTp0b1y9KiAyArixRVAa1Uzvkq2Pfnk2fWXRczNTa+i0F5+z/rDfjR3CxITC3T6njbFEmO5i6hhHg/Cb1yknar9vJivdbx3ltEQEXhlQJhEURmK8B0snJiaoggACxxiFfn8yuESDXSiO8ILK1mr7Ta+oAABEcMT0B8KOlYAxQplP1WXdB1SRIAgFYBSdgIiPAJGtlK0cfRWJAVhQkyYY9GK6OCVpCld/hr/xpDCsf3bmWefMumtPr20s1G17pogHPDUdVPfdu98Ak8vXNG0wqYmAICQUg0geo6+M1t67VmuZnvSk8z/2szPPNpZraGd7tq11kUd4YNYqV0h21/v+Y6XO0iqRb4+/KKCKUWHObnbswdO7+0f8444XnXX43B36iuWjn9+46fNz4LIxTibGpNCostlmskST2ySplDASmDoBSf7rthe2n7/Ym58blDummouQ4u7+zulDrj5sqkZUGGBYJ61OI7KRvTDUe5UrwIwEC8eDcOLutojz5rxnJgojj0VQhIRHnT53f7UhigoDjoAQGQiKW19aQiON9vfT054kepP8NJtZjgwMZPbCE7OLCkBr9i3W+PWzJEbkICjIBEqzRBcheja0tKqpI6sVZAsdhX2/Jtk1m/bs+R99zCendtvxs834mmbVO+N48YblylnC2FI6Aowfc4TuL7eaVbPOEkBhAmRkF6WM7Fl8BCEuM+zCaRsv+99/dcoLPBHI9mpq56j9lyKpWHu10gzzRxTU98loS2sEN2ziQWdGUOQjQKjXjD9sQzac7h06+Z2fPWrLldlJcN03y/bqRv+SGzure+bpp+aIVZmMFdxq9BsiigERdEkmu/zlePHcnoGrB0DFA1ty0qZLP/GqQ2x7yz9mXU0YCgRqtBumlswBwIEdM4IgrLDuzQCyhgOc99Gl2qklh6D2fXivySPeg9IjglDZJ0HhyEBqS7EhUmJYXbbB1obFm53fPMPHIBAVaT3qxhEOLIxKAVoTBte+YEgg0bMWFRA0RQh1EHaiZxwYQabIwgCkAktdKVUvPKZY9ZFvnfeesuhIOXNOc65Lk39+CCU19n/9LDtQ89MQxpfrR4b5ZlFmMJfkWlCYmKKEKgRbxYgQSuMaExhasrw8f+T/uuL7r37OxoOH1Q70CEIgAgLsIyHf9b7/SJ2DhKQh/fs9QBAECKNt/QhnHixjZf/MU96x5ZPZuuOOOHLK1Ndtu+q6pQyL3mPWJVUtqjHIUiG7NE2EoESEkpjd8GPzZHusHWu6pYlb/Yl26/wl//HiJ0S66cxnDyxxqJdZj3VyYeK92eB9wkQBiMveWo/i9IFKFwAAtWpLkSLFfXhzowQKjEad7iKshDGUOcGIYNzuokSElIdbV4GokHt32be/FQT5QHgdARFhQgITWsUOO5k7AA48ysITRgghQgiAG1INztcRGQSQFChQ41aV8XGX/qu78IQFG/N+w+f9flLHZhKrwPO///M4bHQgA6eHPfQg/rhX55AoJmAtKBJcjNF7H1GT1jE2quWgu1Tu2viqy78x9sSHqEtuzkYejghIZFEYAFFQ/gSoyBXFsyUnrTPX1sKRFLJPv7mpf+RubxKpzBsnNx526LaffeVKNXbp8sGnH7oqw6OrZdsqImnO0TQWJxUBqMgiipfO+8lDDjnspsW5Oxq4tTWF7ih3jD176f3ua8++YVMuVSgGgZpjUKHiei9y/r5LIRWRQjmciih1y+1P2oyuE8cSUaNCzMjzF8SV9t+7N1GYGoiWgWORMwRE8+3H6oFuCuO1T2NxulZh7X9MuMDAKMOpSKJ1oSB4llS85coOxz92ZGuodDUkdjSwYFOf9kqua+0SP33Lg9KiICjtwmTfr3Eowa7ynV8f+57Ln/GsxhDBhJRMVxvWtllTugwTnyfXcKxMGjEk3YzBqbWxDR4JMYDToabaBYmSgAOoJzoV2nIOuDb18JgTL/vQLecef1r0Emy/tdzAUhtEnykEBJaw8rAER+GEYFSCyqOzBGUOTo1KGEJEwBFAzFDl3ebQ2EIy76zZPHfs4bEzLIzfeMTU467+yqXX+1Ofueui085Zn6ui3xwGzTVw7I3HVOlthzN4CgKoljv0139542cbqHTDTTTiz+LG2eYhT/3Fuot/99Atr1i/5gGPTLe2eQ17SFUQRU4lEjkYAERhxmhJXPO7G9qeJS11JEXAzMrK8kQ3Y5RVO72t9MhlRoikEDQEg3gPVNpgQp9zJxpq1/nxUfW2NQ6NomRPd4J0NtTcOKYd+8iY1NTORJuymyfzKkyXCLObbKqT+JMXaRMBlLUEE75nOkpP7/AudvNh8LjQ0irWijvL9fUniC7b9XACNr//bc8+XnqqUTIBo2EMtYAVV/RDDs5bQo4IHEwIoNH7lrISgxjSVPb7riaOEiKk3BhLEOou15HBh2F84DHvu/LNJ+0oEboTvcmujU5ERIWVUEbt3f9QmBBEMQKCSDsEbffC8wlIHO1M6F0jDjPLcy3u7eJswszJ/NlFZVp5eflFN35mvnnK64+57svlCw/larlUrf5o40WqNCJJb82+HPlYnT1K8T/wrxdMM23v/s3l84+74KpXDd/7hUOP+CS466/55UePeSLqhVXRVUhEmINzSWL2IrprV9aJ9r85WIFXEVmzYyRSspTmrJXTmOcjB39UFdYjGm24xzg9aIq+Qt0j1Vm+4JuvNfFY0FlE2lO1a4jBuqqh+06DHSi1PBFsO+30KHR3Xn1NtRTd7PHP7W5bOms+LV13547di/1s3cTs0i3DqR27j181vHpqzVE6R6ijqF1Tk5ctjy2TddXU0n++70uPVHUv0XXCAp5pWKtcc0BmAXHepWNpnrGPzjlWINp2pISkgVi7cvdOp7jKlMobWjeTpgnk6+UhmCQmlA3oXz73yjed0Nq1OimgSJEBhVGrlZIZywhaNSpmJEAGZhSioYAwEToEBBRQIggQISmkKPSSN/1aNJTzh23YlB2yuShbP3ovL/ef8ojD68v/afGZp+wqXKOR9ZSMkDlJu1wL8NzBspIFdK1ikA7A3tBN+zfv2HP9kX82ueqJj9/0qmN69btlWY474Zm//fEnb+5WVGpbW7WYoUlUrEWNkN1IgRXPncEZwdVt0KUhJSISJaVQm0Ai7c4cqzhKoyDQiG9z1Ep2V6Wz9axqaRSrihAu3XDclY2k0DmQ2t4aY+syR2E49OQM63xuLP+7bx17xJ6knyxp/4Af3/TC/sfsUQ/9xDEXvPaIITA1JsaSuT07yrAlrOnV502Mbz5UdzoNKbqsebKiC8/irPAz7qsXmLPOkIW8ndbggbVoU1D0mUlDdynIkI3222E8ATSpN4g+Zp0iyTAUjMlwcamIFBhEWgdnziJ4T8OiTnqIUYIyEzv/8sR3n/bUgy+SkxdxqFkQQdkIAoiEI1InANAEhIGNQowRIObaRYaVblcCBiLkEEu/pw9SpENWiQ4q2bHq5LVxqcrVQafiZeOv4PMvgbNP6W/t9alyJ6uqEYQYhBS3k1qgN7VCoIe6slI1zCXfvK07aNDUGcs779h0UPmm8OZTny5lMAuqN/4XnW1fPu/Ip58UbaimYvAVKIUOCUBYqjqn1A03b4Q8hqphSyJCFLEywIZZyoM0OnuCjrgCfyrCiAwI99QCbRzgUOnNv3u65vbpjYXsARExB0x/nXSGXOVgPEdQIWIpzWTq5U9onTP70XGt4+Gf9x996PBhX/jX9MtPfbTNppQmRcFhwmPp0mJZbVvN13bzXqNlFQtFTodlPGnL9Dh87asHvXriUxAbul+T8RpEhCtBnajoirqWoAzIunHZ2a/qKhFtGZWeigzGcKwW5/qBFJJ3WWpIV6BIR87KZPuqVFmirB/3POUBL/3Js666+t87C1aUAhmxFgmAgFrBRQ8xMno/4rFi8eACY1ipZgGC4eiDr0PXhOCyqQCWXMxMWFWfuM5oaA6Ofv9TB+/6/Q/Lx6772fd6jR3XmXDI6qkYaMSRoENDV0pRtgLLgVjlelmWz3+/nTWswK/f8gP8p/N/9eEndW063xw0l5eaU/awd+w+/5/Lcx+fFYXLNKs0FmYEARgTjYVJ5tU6XwetSRocI5CiHmZYu5BEMS3lDexV+igLPZpuuTve+7DhF5Lsdw0seOEYP5h+FCa5DbF7URqZckORldQIUHZ6kFxy8evf8OTzVq+Tq8c+9NH3nlQXD3vJ855/+yr7OOByuVchROfTJJmEbVVncfaaTc3OqlYmkbB0g9lf+/akXP5JfNchaovJBpAkrufztvEgw2FFutHEYj4vne9u3uxumDAeSKroIoKQHZ866oRDZ3O9db7nY2CgKlm7GofKsEagdnvanX+wVoCKlaxa2P3pi74yf/Ub/m6mm5mcPe5LdIhbue+MPSqweSWeFJJkRUzSUhK3QorErixLF6Lt2Q7Hm374q5zdYU8/s1X0Di/deLGQLq558affNHWE+WK96aFw8/wjjzq0uWYxST2RAGhbJyiZ5Ha0vSPoTIJhfdS529cetOaQUycW3qrf8oKHHfbO0w4dRj2Y2tY1mYJup/mscy//5Bde8FSX+aBV5VGxB0SgOgdEfX2zATQe6goqRALxMB7MDXRotmwgNrO9iUcEQBr1UsA9pWGjZidx6danDX2z70rdnxnoaTGIj54M6NTQsiKITgJ57MSJx37p/F90dpVL7e/94ztWo5vCi9937vVHJD02PgKwd2P10GcGXarb48MnT65Jo/d9l3Bnyf3mlObtX7j+2Y9Kdk/0NpGQ372roGzdRNS6r7QHk7rBnutv2tqV5szGJxw8kVopu6X3oo3Vt++67eMLkyceekhtEx85JM3Jg8YqNEjiQKk03blzKiBQnQaOqb150xtuPepLjzv10evG11hqNFt7362jWWoAiNbSsMQiMRrYWJergASCRMA+xKXoXABFYSK78Ze/vqVx8hGHpvnhSZ+mqkaddRZ1vvhnx/zrT80p505u+0pc/cqN4IOzemgAEcgkISHVKs3INQAhb7yA08c1xNTrjh1+4p/Gnv3E5qH/PHzbf/rF2Xp+OZu1DlLOh3TEh6/8ly+/9HSr6wqS1BEpDcAoIaPlSw5Ji6XOnjqdBUKph4P6dzt/84OXvWGtAQx5a4C4Qmy8//V9T5ZeNQd1e26bWxd02VpsMozV3IgYzHnpIAkoonjQC8Er1WvPt4+96K3fSwadxWAufMMDn/HY+9120cnvf/uOQ5cy9t45Vgb7ppnVcfwkUyd3TD5qgBKMAQDsJb3emvfvOerls25pXN2ynjMedh1JLSqK63YbHbn1m5dsm5+ZfPDGmWZU61ebsq4DZzlH1Z5onuRJLVzxs88t08S6gw89uMkTTTWIpCqlSOnaqWKx061Z6cWsExY4rfHgFz7+6z/+xXEz69ArSu3KwsdRBxRY1C2+bRuq1bMdjLkZHDHTwzyLa0BCORhUggCoSYfrLv4FDNa8Z6NTmhoS7Y3vfcNBTdWpexPNTQ/5SqZuWB6/8bX3W+qmhfLEqQcBJG0kIZXNj1z5lXodRgKV1BObyg//fPMLHzQ2f8ep2aG+wvbu9vJMWzyiqVAF5U763I8/9LXnnKFc1rtm3JeDhW2b56TXL7vm1n/xX/twrNsuywE5eBfWbEw2PWXtngmHnOQKgVEQUYB4VD0WOCBOZ4WMOsY0clVMfPE4GQYK2pPtwmwwts/5IGiok6g9FzxIY7dZq+zql3/wkBCC4YW/N+HDP//Kke881R79o7zdNRQjIbNkdVOxIR2byc8mIQjlsce2gmjnrvvyCX+7xi1T05Vneidx2WWqnzeiBsXV76+7Fdce/ICpdl5Tl4LJPchcY9FAMdZrUFZJCSZ/4NGy57Zbtl7wzQ2nPGm1x7K9x4KNTBFiPH9j3a9sGvLYXaOYBWEw8YZnXXTlwqEPj7a9zFYEPOjgMSMPSUoqydJbf3jVVmwrm4Lhnrtuy+GHXlckGlObO2BL7c0fudmuOfdhUDSHSttEezLm86e8bH5MT44vv+zqkw6zhz9x9eJXFuY9FTEqBaw8INhmHhq1spXRSBxIgWRDVeeRV4/7mf/9nS/Yo377C+Kp4gHfeFZaFrO7xxL0MUkYXKKEHD7skV9920mvnv3+y7rJTKc9NbvuRP7NjjMPVm/2+lFj07NShLl+miZjBjNz/eNuemjuXMfbtqlSNyKXhkgjBgjN4YA4fYUUokysHQy7j6pDwCAgIQOdkU8wUBJFRRw0N/douk8JLx927XufmKJOpK6qtAiHHlPYja1eOh4Wk6TrIwsLMipEBJBhuvUBkgQzIFvtUXm29VPJuQ+e6PocKpVEErejl5X9TpY3wh1Xfu/KqaNffSzs6OmitgEwWW0Jol5OPcRagkEq8+i5mNP2xJMxKa/8/XPv/9TT2ntSMF7b2sRddPlxS1Wi+zpVOgRHoSYYhM6jzr7lwr+ZfszD7h90CIqibwgjAGIJxMN4/BFq21LZVy0US2s3dd1gogHRu+jzpNq29NMLjvrbY8Y5lkY1kkQjiiwe9pF3nnV0P6Nm9uRTH3FYWmp1xZ7Um5oFJI5ep8LVIPqgJACvhM1QIqf+Ng7ZIT99zY5zz9owsWGsTuCXv7UPORniNLthmvpuTJsDrSWKVM84/WtPe8H9HnDO0eMbNGvB+BzB+JYjBPIn9VPUvVMAoNqxPXi7+vV/fcTpRWeeqqmSHMOKG8ej4UXmA1qgR+94AYkk+qZq004fmUkkpsbYGCzVVpVgAsiisbhbxXa5+op/e9W1cZm0RsoGpKrah2UNUUPEQVH5wALISiMigA7F9pMrjI2ae8MJ2vb55s1/taFabGIa2EqwXJId0mRyGF/y3Z/m93/ahMp4UEZsV42iNTG19vyTkgHXO6YUOg02APgIRYHknKva65/w2L/9+PvyZ58x5nS/TqzrFhO3nO3joNFxKFLFkCYRdOjXSf9+D+7d9uNvrDvp8ONXg/OyYDUABt+oOMnYQT09Xg6r4TCK3ylBgWSNVEKsurd85ZZ161/1BG17VWMyy4W4EmOMDX/+26c/57HcuxqfNAvFnvF+discTBBk5C+QCEgoFPplxl1iEACBRWPi7MINDOM3pm/6wdnriz0LdGRofOVBt551/OknnLoxIaxiokOfYhCtY13NvPqMN138no01BMZaINu6unimesHm+dZiu+5N6m4SyHeLrFHNvWn7K75yzGKeL+D6RSd7++MhiiAiyN3TsJKGUOW/We0r4QjAAlWeMmhmihFQiKrdrm7FrCo6W973kmP+9ZUdJ2GwXJERJ5SkIe+mlEce1i4wo4LEICIjNq+hg0qUpdSVanDRt5561G3plimBtAzkvKjeIGIyoXa867I9p//jQQUnzGJbBjtqjbKSXXPTI3wp5TUPj5qt1sE2K58Mq5SVtxOpHxj/mpd85Yv/euRTj+vEIRb99JI9BsfLilKkPCmAPQQPKfetC/X9/mzpuh/8Zmn86BM2TcyARECLoQ3gQl3VMLejIiAAq8hE9EtXbdnejYWvp951mml2ocJ1E94hUdLE6L1T/OkvfeUH5sTxh+CNMU/uKMbvmJwa1HtT4CIsIB4FyqLeJSMmXREMLD4ipAW7tjPzsDw17B560bdmZ5L+Zwb2tGc8aiJ3PlJKHIXQmEF7cPL33/nUj94vWbIhTA6Haxdesf7jW/7lS/973UI6U6jEICzXiR2MI37wuY/71olwx/pCFxr2VVyEhRDlnnLvKnKkS5837xUSAEsOnVZp9FBlVZWroOdiY9HUzeWp4rYPPP/h1+TNyACCFusKExROgm737XIoax+ZUaHWozg4v/AwpYNBu0jDL829dvXF8ZghtWusUlsqXQ2cT5e/conb8Oozws1mPO9VzsVsojnLXjzp37Wxt9y5rtFaUIEtadm8vuo7Dmh9rLMk9tOeefbzf3jFm1c9+hGrhn1vv3vqhoVewyaQtVqxhlBmumbQujE0zdhvHvOgweC6n34idnY1126c0qRyjFUkjmiWlxU4KEpfDrpLQefUnD52fb5at4tg5xLdyRTkTeTenkG/H9TBuKzuf1rPYGtx/vYLnyJGyrmNCsNo0oiQhRkUB6dIA0u3kYw86bywMVk7L2RoybegmFbj/o7mF+JYM8pNX7rokqlHPfnUsVg4RG1iJWoshuHgbx/xon94fGduChZTRR8vPltOfeesN75rzHkF0RterFOf+TV15+N/99RXnTfjW+O3GQbhfb1/K7W6A1qgR0vQoTK3DY7dLgKEURTrZuIMVQ2uhgoDD/XWQg+rqXjbV57zsDVHnKUcjPgCGKV2LRN9Z9twPe8qvBdhUFoRCwJQvP4cJ1GFqnXxFza8nIsth4UUa1MbVRRt1c/ry358xyHPOOMgGehDbM8HVyurDJSG04g7r3ql70H9y2fsIh+CxWT7xevdvLZ1upw1GwnqBnPqrrj0H867/CP/+phHrW52b3iKzvJpQsoztVAjphiRKsy5I5RhzMtufsZZsNxdWFzobt4xQN31JkEH2pRDCcGSaTYmNkxO5q00NcLBum6e88RMpqG3bftN27dF2+l01qwqq5Tm2t31RVWOy69OOmGg3fxhkcxoyoD2FsFF0DEhL4HRUYhFoZLmQbzHcP+OTY05u+1zGx+nx3Ln9tjWoe/a/ZXrvvvZ+73wsTMYqgEZxbGXqFTiqd98yeaXTy2NGxqqJ0y8cPnIB7x+61XpzMEGuFL1UOUDaDUbOxv//pY3X/GxupgZczRCH1kh6RShu3GtCkBU0V58kBVkUSRgqGm9MQs7ji89D0D1O7+54mRndfadC/72Yegos0gswswxoiEdRcs3T1vthpWLgIJAhMwgaOcGR7MntmMf+t4Tj9k1pX749M5Mi0ujxczo+fnPX5We+dJNM7jUABE/rAaYcAXNBkUKZG5ONywW6ub+7KJ1VmdEl2tXIhPWCYDVISpT1emWNtSnfPXWz77h+Gcc9sHVRbJmIrJFCZgoZ+o+5plqIVW2iZ62bUdxeZ6kMzOaQJnaWXKsG6V2g8EQUgdkdHSugXFoFOW96Tjdsvqqm++4Y1sxsXHjSenEBA29Ad8JSbm+2+hO7zr8s3WdxLjjAS7YMOKcBUQaNU1QLSDgBm3yqESWGSCZTO0WWX1ZJ9O/fucpc+96RXPY1FQNk0H7VeXg6l+8/e2PfdYJjVYdiLwKNEA/OPQbL55742RPw3j30PXmpmlJjyjopvooQquCskxtSMtWfvs/Hv/S419he/lAwwh2RgkBMAgg3z1OV1Dh1YcvR2AhEdTrOs1SFzdc+4BupMLkQ/zmhpN31+mH5r65YSnJvZWogo+CAwq+Vo2UafC7p+h58j6SAhFZ4TyTHThp6jTC65dfle9p8o3pX6xqedcoMWZ04X/8/sHnHUn5ePR5WNRNxWwig7WtPKmS0qpta9ENyp+beSbfSFIIVz15MOwsUntRt8ZMFI2xFeLtpw/bsZh9/dlfeutBL6p1o+3Rxsi2o5eHOhVvrbJ6wBl7rannE4k9IqOV0sAYTIDcDQ0yM3MRlI+gUiolMUZLuiFLdl/46yv9xPixT5rJmUOAmj1iMV1zPuyNDaYWcz9sgsSsnDUDRBj1+yIpHsX4Wje0olpG3p1J6lCG8bGZQcRjf/Jd9fKHvP/oIxfXrrGBSsRlj+0zH73jO9/51Dl/9tiJymtElEQUL2Vf/Jvn/K9TytCfdP01h3UNuMb85PyO1QDFYk0+GxyNdau/ce5Ji68r/i7XOKqg8wplnwgi8f6Wi0jEoohZ77zpFYTBgqZV4+NjfQalfvo4KOt2N6nUlvPPX1y/+Z/1J+OehgpsiGLifXBYc0ZDB9XMV3rrdsEcJYVQEK+brENANBdMt5cmitu//qhNzOvT8Svrg7L5tFnrhjv/cwuPeMFUsdxsaa85+9o57YUeOINep7mVoE3An7/OLVTVL1/tUKS/CezmbbNlNiAqbZ5n4PPap6WS3z016siLy53X3fz9v5142J81MIS6FX1iVKPq20SlNkhGiUaK442uquqcdWRDLKwUK2JtsIi2HUtgbUQssTFG2c5446rLL9qaH/HMIxo6VNwTAgGPJC51gLWlmNYNoHGlguoNmyGHwIQcFYkC4IgUy/FCOjDWtGhISR1TE6K7pcomvXrLwefeH7c8ZU2d7VrViX4Kh5MIw2CXn/iQf7kBvn/6syb9gGxADIkv2h/8xGte8YyqXaDleRuE4+pBp8NgfaW1M7OgzDCtdO/57VftfNeaHaL61lbkiT3QqGvqHpgNSO+cmK1mBIPJpzNTAKAslIfO2XJhXPbMyCPa45d+6IjXFCOgHiZmAI5RFNRRoJpZ+Pdng9tjqkDJtoOriDo40/TD9PdncGPXT2547lpbIjI982wpZ3Ynir/6GfiL+zsogk0VEalr6rVWEIkEldWsWrU0rxpsnLuN7ijbeRVwfSM0v31C7Zwgs+hWgPauaycOS2iHG9Oxsa1rpLf+hc+++Pc/aZ1w5lEdKeqItuWTSkQRVjGLDoDTDTcOJ2SoayBAQAxKEUTP2LDBTRZDVUIqJWXBtKbijRdc0V9/0nPXm6ruKRFjIoYVboy9CFUaUSmlyeCwydS3aeEbNsZogygFMYj2+fpe1slLRzXmiV6mxmA+NMYu+sbOFz4hDniNsVFksZNJzazJjYHatb392k9Uj9vyoic+9OBBrdN6kHWqIbzouL+Ze8WyBnStYJ14wixFkBiZkkamBFFQNao/2/CohQ8nVacaKu2BVxri6Z4AgSPZnxxiHGnlVaOhOaaIyQWzUzuXWtIv2+orj5r5xifPffICNDQyAqOI+GIYBSWq9lQSXn755/ou6zWGtHTl2qgmysZYr9tszd3xd+r6L46/sm2cEIJvj0fati7+8OOdVzxuaYfy3jRbihHtr1O7q8dKgag0VeyziPT7tfY6Nl89cqyoaGwNmeqCNzsfSQVI2lRRnP3a0wH1jY1WgcvB6LIMCT7ppYsXXfye7nGPechMZFcZVIqDC0CtBhIIzSxu9RTBAkoAQsMOyCSKXYm20ZiqBgOHrcTo4aWXXYsnPv00A74qRAyzxBB4BN8CI7wxFHCACCjg1K17rj6zGYrxsFy0kjqvawWsgK2MHXSHt0lts5ShsEm2sL2dXPWZ7Y99ZzaIjMk4DFSYz9fpoJSOhop0duvc7Hm//PgDn3DxD496xVjdb85Uw9AZFA/83Ct3vqMCP7XUrFGLp1QFBYLK2rGcBBWAWkjVqRc9+nlvunYXTvhe0qxW2iLxnqC/jfCl5xU2eKURmUWRU/7H51Ve9SSv02H9kDdd/J4NuzQaM2rOQyXVsCSCJG+Oj9n4mOdO1HUvgGvd3tWSTUzC0Bipb24d87HtTz/GDiMTMJASP5n+9H31W08z2ysa5s62cwgAcvUTwhJZkYBZQwtgLYavfegN3anuljNcOe5m06rz07HVfVSEaMZ0pWx16e5ZXyXXH4asdpP2NCaVjtXUU568uPvbn3370Y8/bTwZiAgQimCqPIBEWY9bIY81ErMYKow1FOugTcMIZQ3Drrfrlmvv2MPjx73+EFP3jQSVSd2PTApRBcI4mhkZ0TgyiEgMbGWs/c7LnnrMji5nCSSzEWtBpjjUttOIQ+kb7Nc6TcXuCBsv+8IVz3zEuvmFEMZ0KWlEpT0LKgBwWvlkerhkHn38Dz948iO3P/txzx/bkxk2y3ntNn792f/wRmxKR+rcIOtceQGV2zxrkjCJAE2WVXXYvz/jn184KApp00ADgjAQ0z0ADYm+anDUoIUgAFYRQynjlwxP3dMzOjiLP5C36S+URbvXTHRgQmQlHMQmCjozDY2ufn6ssDBpl8wNMzmOp41eMTX0U7+Bf42vnurWkbSSwMF37K8+svAX5/qh51qSXpKmmoXU9luP6BsTgwq20YCoKWK65+aX30ZV9qZprXHDuqDgC6eHQAhRbItRAf7bwxkZrn9sxFKHABQll1ZSRZ2v/7t624Vfev8pZz1YIQtKjJlVjAyGWrbqImcAiBAlgRAR0YDmenyVcos3X3bFTdUhxzx1esbUVaGa5FmRrwomHcSAgAQvHEdKB4wAhMJxmMpzd1/2t/d/5mo1SCdNQBgyMQ+G2fQkMZMYxx2s0cXZWz78swd9tlXvRpPRMOBYu0qTzGVVFFZKByNyaL6NF6bPfex3//mJ/+tLzz33SbzYDuybCw3+/F+87S1ps9ebrBVB1lCKtR3jxBjPSMSMQ+n0i9O/9Ofbn3hkL7jQdIgiwoJyD00UvvUzu5THKgsxSW2Mtk7g+w+QIh1C7K3j31374id6RXNj2SihCIwhcENbkGYjiYyd+dAa9tPdSdpYevQGM1l09XjJY/3vuvs9qL9oMIFYozWgfvPJuec8tjGX5m6hm2UAVgugMjdiusfEWpROM+MFI0W6PrZXp306rsZDk6zZa197w6tDFBDRiUKSSj3wbHC22nq4QLeIQjZ6NZOzJwiymG7c9JzdP/rEB054xDF66BwlsRAEHU1MD9uxmz0gKQDxAKS1USbN9PDC315zS2PtUU9em9OAqj4TxcqAIgleMVIIHA0KR+EgKzgFEZAUshiXnmKe/fP/fO2a4x9xTGuIE91BVzB1dtWGMbRKLDCxr5KxPZ/6ysaPH3vr0GoKdT6BmTU+B58JK+OE82VtpLXGbaWFxtjzz/z0D9828cFfnTepWUk3x+HMR1/8D29plgTs0CQqRBDdEI28gmYoihcT9A/8zive/6jHqbod93YDI9DdGyPzXefPQkoIgjYFRrR281Vv3+4qG5LO9i9f/7HD5m3gjC3EEXOXH/jYSFAYHSGqvpnqNY4Ia6EZJo6frlzSV3WY2P2p2y9Zsye1RA5raSeD+bf0HvsUi6XRCwu4uuglWUaMJHDJ0cgYrWCjZRgIImD8xQn5hoO9rhqh6aAW+c6pU5ViQd3IWERsdZ70VLrZrfbDKsVQiWadUNQGRJvaa5l+3nP+6/cfqQ554Elr6tU6YGAzxCATCnp9RG01QqKtVN2lwW3dxeVl2nDEUw8a7y9Xlc80GUatPQmRsdlOJykioUeIcTTdI4iIVoQIiCmGvFedeuquG3/yk+Mec8YxanpRhlFjMj0VYgMrxNT38snwma+Of+CEnbc0Kts14w4bygIm2kVBspohVMZwVaVHzN7Qg6AOfeMFf3v4uy9+5V89wueV4Wpmx+rPvOgN7yoGWoNtjJkgKECIEGQ0riYhyZoLMHHcv3/oO7c8b3ZbY4R9PBpSuHvnzPk/f+76XZQ4lSRYs3UKbvTH3Ip5b2z4vV/d/syjt445FZNkhCDOgn5Yg9GOSCIorjFZVL4TmoN0NxtS0eW1nrjq2z960+qFJkcqtEp1svmb33vcixrsdJnUkA3mTe7ThCIJhx88OWgf8iJtNDAQiYk2XPKyRgVIY3V7SRtvqp+/JhSZZ5W2skBYTgxiTynYkYwtL6OEiOLUDAkD+mBYk0RfL97/fuUdv/3hV2XTaYesX2+D7gh6l6+Kc955X1duedjt1WTtoTMnru6kKNWgJ8awGB+VxsgCqckyHXF3P4JwYEAfCESP2qkospDCyN4OYs7IB532jCt+8uH3Hb7h2JnpMS9VUFBmufJj3TxL6YIPDs87W5ZTUsaOmSRrAENEX2CnMhC8GISslnbwNNvctaPoxc7jTz7//m999z9c/A9ORWj2Z7oT//HKF71HdZu63W4lQkGYCSAiYRRCybuNeWMXc/2MTZ9/7TNP9yOYTL5zGpZWZofj+n89xGcDrZQf1xEBjf70kxdUYZs//073jZ8/ttsYokHstAODMOjlZU/sAlAqkUvIPOmYBK6VLm+d9VVLijZ+7crnvTFUqtfKB6bEcf7sJ0/44ox3EkU77mud1CGb5KAIJDxlatvabU3gvIFMIavTKts1dxQUNFZ7DpQFan+rOqzQFRlIjdN1prwqLFTZDx5e900lgibqTh4iYkATFCAgCvcxbjim3rbrpku+OmhMTTWLcnGxtqmuAQltnq3dND0zPpEWPe6VVRWsaANco/JoAMjo3KQGQWidK8T4aHRAVAzKK4moY9nQBCxKK86QNaHu0cmPgcsvuvkblx93zKFPumPb0tqsXrilP9OMtOMfL3zFSyBwloo4aq2goSgwBmr0gAhM4BA9aHBq1VS9PN9bmPyLB/7T1R//m394q17o+Npozx/7q/M+yy61hSpSrxCAJBqhUSDpG9Fotjx9zRFv//r3z3/F2hhZCMApre4ep5957EJfjUlgTQJKotu65X69Ybbwkvzhz1/YcaynROrl4o6lkK+aTBpjtfIeEhlkiK02hJCEFJzSIa36t3zhKev24OQd/1W8A4oqGdM+TYdW/eTD+v0PWKgZgJTwEnc1c9IWhQhcV8/7cGYadd2yWa0JghqOweXjG0Ju92SJZlND0v3ZaUmlEOtGpmwYX06rTA1Tw7uPk4ITjygqS/eDCwgjSoy1ioFhYuKprujNz/V8zJvNdjuhhLQCVAQA5dZbq7pOmRPay0JKSpS11lptcASKHnReOIAgRKiQRYsgEREAoCIiJlRaIWhN5Mcf9iA7/4sf/ObLX9+08Xi9bd33xlctT5Qf+PgjLmjyKNoDg3uZru9ZBCjN2rN+TzF3/N98+PUffN9L3rJu0aasPJh/fsEbP1g2dqwZxkBDg1pzjMGQcPRONTLAWNWt4y/nlz32rX4ENySSgtx9Nn8QqzLYUFlODKOKKvni7HHXxsYe+vvzvhyXnjoWgR1qOLgcLi/D7Lpm28zvsWunDFKSdNa1G25NVlsTnGD3vaeMjVU//vWjHjwodF7tXOW2TWS3ffCO1549GCQRAIhcCQwmUN4eFaR8veXms40WytoFiShvlAs/Pi7Z1Rr+7vhJ5VUFtvu7t6RLhMBZxhDcLx4gTRIgt31d4SARJZK09g+liggS+5gAxNrr9q4kb63yyhIAMIcQR6w2QKJDsRhT0uAZpc5AEJXVYzqxigi7zN5HAeGkihoDiOAom42EkRSRIlKKIiqtCURiDNVuZZw+6eSxC7/4q7X//JgTt3zkfm3+1rvVBx4VMIzaVMH+IY0DgOIYURQeutjeddobP/TD+i9fAABZpklEQVTwz/zu2a8/Z9AcpgVuW/fxc/7l1Utji2Z84KagZtJKAdTDXn8wtIT9G3930+CYhx7a/0L1NhFAAhERvAdHruXBipOm51QxUCibP/7LeZh0M++6dumTHz/70MqTydavM5Kpuj936x3XbouttcXNvw9eJb4e7FZnvOb0VowGrYyd9wCc+/rOF62tVavyKQzh4P4XP/2E943NZ5LUCIJcl7B1A4BpKmZgYaHLcMx5TsbtZfcLoEgE6TfPrHFLeutphglV0r99l9vpm0NsZQi1uEsfOOZjGmFpeXJZgUMBlaQ67LshBCTvWddemxD8mhAiEkGxUmhssAgSi46KGvmwTiA4RcilttamiU1JSl/7CMyRAZCNSlAxWAFEhShAOnpRirTSRGiQSIFIjGQpOEdUUXXcp3714V9eOIxXvYZf85mXvYXQLqUrhS8EvlPj4l3FIRKIYD07Mb71yHd/6pz3v+/vfvSOCENpJy798sPhryvObx7Cb2MxKOZ2DQr0dUTgwSDk6056ysxwz2Ef+OLjxpYZRmkkwHsgmA2oSkfgSKUoKDj7yeseNFBLcWJyYsN34WnBOBDstJySmjuTa45+kgdLPuqQdFoynL9Dpg+FOnArxCMuOqLc/oHDXjY1zBbGjVfthamr3t5902m2bsasZ0kAo487L9tYQdoMJIDsvFx5BIXQyMy2354amEytOTnn1EGga/otCDrauph91YZ2v2A1abyKzcvNuPGAYm/C5gCFlaBNFO+jYUIliNFj9EFrTTlHUcSVGADCfb0FJI5Zd9bvmatFWLSWygAists9KhuBAgGFCBEoMz5EQG2NAik8MyhtSWmtUEQLMICwaKhZGVKSNrvl4s8Xl8xJj++dIi94/lHKx+6EX9m8RxPbf2CQkmiE/LmYzzR2ly9a+9d/9d5/ec7rj5pv7dq4SAf957OKNzr/hY8eoQnSiVbSnuW6iErJNK6eHMDGjYn5+288/+m78pJHryEN99ANKwKYGu/QJIIS6Za3HeKa2cKq5d7sW178CLM7ZzKZZvFkobfgzVIGPSAOeaqKWq1bY301zJNhhao4on/Tf5x9OOxpQm/VQqcv05/93COf61GVuZ6frCIg155uHgIkLesICH1V8dhpykCzoy8oR69IW8Ebhsvlmq/PKheUYCzso4epDjrJvbdW/+gY00Wqtf3tbNaPrEFUmkLcDxmGIuI9odVGROcBFAlYzQwALLQy9xMTEDWmdVUHTSFwtCSea4BUeBSU7V0gEYwmUN6kqUVhKMsKtdYWSRGysDAyApDmqBIMeVctA3Y3vrUvyUSk8vA0VBjSYbKi8/2YMLB3GdxJEpDILICdQUjXzi49/eA3Xf36S1597BOb1R1TzXjcF57beHl16jtayiqdRAeC0YshUQH7DvfE4RtnP+fnbY3IAkQoqO6+vQeV2f4S5WgtI7NVL3uE7Ss7DKtvP/NZ//box3Q5RKMpD1Vpta5DYpwTQ5agolT5ApM8r0ulw3AcvnLL02bccGPjpttnpruTO1+18LZNhVsVuEymhyRMsajULcdIM80jCIKvaycvkrmkSRrOP5cJIJg6q5NFsLt6T9VsWFdDpknPwGmbkZjmd74AQBMAXHWUBABgMInFuA8RXZAFnDcBkIWSLBJhCApFSIvnqEYdROgIImSzfIePGqOIWgnARzMve8ELmJkIMbEQlVEITBNcLpcqyZQAgIygPgBBxEU2wHEIeZ3gxHOLafae5luqC+2C224vINDIyu+dYyIKgDKIsHt1vTwJY0381NvOe/HffONtj3/QcEvdOOLErz82f4xfVxn05RDI9jQysnDZ6vvVxQ9/v/iKF+34DRpbRRACxKhQ303pFqpKjXNErVhBxDUvmq+tS11YHL/jtg+95aDVRCaRUoOFqluSliIm2oUEMlU7AG2K2nidDmF64V/gBaaE1cw/PL7db3/row96r9udTpRjpow16MgSyird+uSqlRpHglwXHrCn2p6a9o7rHhwROebK+TlV5nf0Dh1AXqdlAVU0HoVyZ3SQLWa2ykLQvtrzsLrOyYNSRh2Agi8oLD4aDA41GQMxEhkQCoERtAZgFgRkghhMW/d9DYoMsAAppZBohLuKAqhQmC2FqAw4AiFjTZLFPXtCs1FHZqCVkh2IiM3E1SaPRO2OT6Ax5KxIG1Jr27PJQnsUMa/8EdlHpnNXSxcRjFEkrKlca4g6nZr4xBvf9uKXXvC1S89Ndbxy+dTPP7l30q6sZ0VEJCoaNUFpbzu3ffLG0z903GJGSSz1CHwFUpD9gDsIIiIiSQFJ5VD7pliCJJRdpyOwlSxuwTPfq9BB0sghog/9Yd4IIUElLrHjHBGNAZ+MeQs1NBbf3nq13QPBat66yaVv+tjr39TfmQ/BclQhmCoa7ILd7m0DEi8A4oYOICSwZFcTXb82B5J095eM2e2jaV56uMu1i7UTsqjA63ExyNz82dFVWmEgtevm43andpmBppQuDThja9aBFV6zUEgiIVgd8nYdR9VQCSMathACIwKgFiGL3qyayKUO7ANoBYImshApopgqBCZE74RANdrjs+sPWt3J3OK8zxsyrKOgsIxI74AUuSoQkRmbnprUKaK3qSRCqDWnFBtxH7CEMLMAyorc1QiRFBEpnVZgwWh0nYk1H3nKB776kLc33nzlmE8X52de9v7tU3WOCYZE1QLALBxEl1Ednzxy06JNN0VqGGs1CiqOlN4NoKY3Rv00HfaAkBlRakZEkiDDidtmVttpw1lWVopCUnha6qEa6jBMUq1wtCqTcnGizF1n5+uO+6vNi6tg3LauwI3bHs8ffOQtZTqcVEksVQpg0A+qQrYmqw0aAlLsIrMgmKmx0oXbNoIEV94xTlsoAAy2n5GzQ9uvFAIwJu1E2JPxt5+Og5Qqan51tjMtg4bVLUUBRSV10WAWNa+/1resPRKIMvvBQUdv6btvqlPrmmxMACIAkRhSQ8yMhD5C8KTTrDW1anaitW469ctzu3ftnu8OXRSIMUYWQKUUEQonmiGdnJnMEwXM/KfDndyLxJimk3/3D7/8UHzes7/x8VxVuxqn3P9ffRJ1hLzUQAggzKL7tr/hkfRbY1la41YrrY0iECKl7g6t1fMZ+TRHTcxIXDICkngRe/X9r3zjReCTXMAQ06DWqJQGLWS5RTiazCuTZr+5NHnlm5/6mN2p3QNTy+6OEz77vBe88bCdVIekm4egBFVECTVC8/a2QIOBCF0dhZk4pK1kPPziuEiq2ew/oOhhhObcnhOiF6RSSQBgsS1CAlBbFg83oNSE3fz5cyYK4Kh0i1SgCMRBiWhe4++YWqZoQAGY5E5vs3skbknXrmmkmjKAEX7LiAFFmMQ289ZYNjPRyqxid8vNt9y2Zddir3aBRTjuraxHFkBSulZjE5MTTZMYOpAN5L8tGnriGy/52pZ3du/3Onpjld5YnvpG9cVGqFDFKkRCEI4M3jdi94YFHbVwu6ONMdYYhYJK27u901OzZ5BxsCvpplAzMKHE1O+86dV37HlQNFolKjAMAGKmyixlaGg13h8Ri2BizK5k+uJPPvsht9Zj9UR0qf3xNRv+q6X6cw0daXUXm81ClJfgnVLm1lOWW41lixFczciAnLWlh8u3nEBY4+CGs5ZaEWn6R5NmwIkMovVeYQTSETVGdclBEqnwraVL+0cvDZt2WHQsE5sIQavShLSo95TGqT4pEDHJgZCLsB+V5wCp8lX1nsFozBC1UV5bjSBgTd6UxO7e7Ed5GQWIRIiCJMwMakT/M9qfibAemzAAMepR68J9QDa6ZwntIqVFfcyFT/zn14yfe/7rX3b/a6ZP+NBTjnnQwqAZMz9WMoMwocp6aWv+4c/otUisBQ0QZcSmrfTdLD3pyrRxprAjqkQXRViYodRLE5P9deTHEibvNO1i4xAggAdOp/bIiCgK/PJSM7/to08/c2tsFaWbThsX/uSZX1D1Tsw9aZz+uTMRxZP4qqxjf/MxacthIrGuvCAIJDnGZtbtrwaitOtkaISU/dmZgSjxS1KRAEdtSDAoJZc8RAXKG5gc+bXT6my5yPI2QhSlhNHWdomJ1e7zI+cJC1NyIFzyvYhU2ZqJph0oAkBSembV6tWrV81Mb9y4pp3aetewdD5END6E4OuqrL2PLKNcLEoMihBJaTM9ZsUHwTii0ZM/lhXq/ySNUop+asvkh0e9edA556Uf/fy6cOnar37CPmimbdTQrWD6U8xdLb0j04oJVKKJRjj1WgDvTtFVjqm5nhiTEiOJq0QYIAYwE1evx8VDp6pMx+BNgjEgRrbgVBpz+AmBADCDaqjm/CvPffTuOi38eJvoa89+/MultKsXSnDc3Pbrplk2xml2dWTbr9sTWdmKCRWeRRAxy5TKh7eRquvIN+ou9RGSW24+C7T42hunRCI2GkpLzaa7434A7H1vwU4sLmNmomo4wxKATYi6v9ubet1HTqPFAQGASWk/pdYKrfTdFkEeZHzjERsmAuvmzEGbDj9o7ezUxMTUNFKoQZssMCCKrwgBUCfpSpFtBboGkZRO8mZ7bMJG1oZGxXbhcNff88cK3kU4Zg2FYGP388//xy1rHvL+K1+1R//2oBd/LD31YKvH4t7+IOc62nfbxgiSajSQVgQA7k6wW+qLvl5ky5Qgo+K6QhGiGAD87x9g50xhlChWydLvclXbEFgxQ8fefI1RwCEw6IFZ+PuXP/qWPdkAG2Wzc97vHnAKyUz3ZsgKMNmlnSYjsIl1FQ2lC9yywRa/0XUVgSMqSjPtKjrre5mljG47OKeA1PzBYauYqSgUYVkFsa00ApLIjY0ZpwxrG2OVtQpEPbW4nUB7FISQXd/XPLFwmvYzzQqAkhT5wPu9u8YBAHWAzur1Rx991OGHrJ/pNLUaIatHDoweJy04F9mXiMDBVZVWhERKIQIqm+Z5o90ea7caJGIMCdCdwaX/28LN4YJKQpdb5d++9F2/Bf+BY//+ytaNT1n4ame9DtYjEJGwpL1+HXvtLAcQThtISms94qG7u9LHX/PWZMLnXoMgcQjAQiRR0l2bj1kIm/y4EkOab/tUAZVGTYiD4SzdbA0Jx8hSTIS3POhRe0x7IU166+fOGPsA38/6pVbugtFp9l8P0K7t64aralbAO/vroW7e8T3XDQLMoFQSvLIubNTofXFj6rmB1PjVmRxC4krx0C8C2ozKgCn5WyaN+MBh2BfTMw2Qmc4dd3g0oihq0TdAUvXNoIdLwAhoDMg+911Gln63h1poHtQ1rV6/dtVU07ALgQEQgYgSG2Vstq0YlMLAqI0m8D5EBlKKiHSSN8fHO+3cAAOS+FJSGEFf/7E8QXeTu1q6DzipeumYEt7z1//wzh90bvqzl7/nv8qll37oiqSB3CZCQmGmlrFcrmlZEQlkgUgpRSCIAEQrrxlCRIkC82d9+ZWOXYJiFi60Aw6InsXYqe9uGlM98GiMHhC999EzlVEAHNTXD5pPLx+P5OcWuTJpePPGF1XYd5R2D7n2L1/6nl+Xhw65uViJ9orDradUieOkrHsA3vp4mFIM/3FwRFAAlqBpNZlgFesig+HugweF8WtuvfwcR9mSAwyNrUWaNoynvHZivv2EClBnRT8LTntUTctXBG2WFXtiXf7uxFIjdTXoWqmQNDlQGL3cRiHbXqhQ3Lf5EWnWVsjUzMwCpM3Kxq0jcSCg9WvTgLEyHENg1IklEdKKUCd5szXWsRqBBQgVAVrySIrkrhEbriAijAgB7xQ73m07Hy1O3is6Gs9ZlBhUszrnH7/xu6mlk//+F5/Z2Tz7lb12gjHRIaBJYsVgtzdn2ySoWY1ZTMCg04SqDvss3ccRw8v0GWP9lIIoQ5cVA/bABoqIRXHZI6p0sT0xbcOwY37540u3t5MhAye/X2+n4pajqLuQSblmS/7J8IpFX9nhqj3rf/rGdzwdfnNkSsmwjkNveObK0FRRCHzdACZPZ/6zXmjecH5rWLIQBsrTvUkKJF90pUHJsKF+rHLNha+NCNy4Bs1YjNkw5bC0vDFFYucF0IANbIFuGoshUcHoKNsmLRNGXhnuQJC7b2x/yMbgniItGj94tQkqVSMYVQQy1hqtm+1Op5Uqubf4/97kT0KsPPA6UJgbDzzvYzeuunXVu5c/sPD4zt9aVs6LTgyCIlLpoJWlK6gXVgERraSA9j0FGU3ayZJaXOJgU0XJTVNOUBRFCDh+SXhQiYtZ4msF0m++8ZFJ7Slj6Fx1RtKvu7NcS8XJnsN/dvHbfba4B+yejd/+4D8/0JcXPBAcDCLlNU7kP3tYSwVUsapiEMV1uqmcgffNrU9JmCDa9r5COKOv96imWFOqk9/cKkwdnQqy+7Y2JFQhF0bb62RyGRRXNQOy8z5pm/7OQ5F1rAQluenoHAhcJARBRkX8Jyj93nQndvqg9WvHgqA2RkkEnbXGxsbHx9qtRqJx76H/10UroujWnPOuj9ywPgxeu/GdW95y9TsOqqckIomARlB292Sejvg2dUMLIhJH4QOUTopAQKKacy30KKpc3DrRc4no4C3g+FfPUhiTiTRBpjCcfdYxGlF8lV9THGV0nGsUGosg6e0ffJ3ShZ9xPP7tr777NCn3dE8nrINUBCrr3zyOIoS+jo6k0uNJvxriKy4+9jaAyKjSDPZaeuSAu7LEL+epv/+f1xqcE+b06hbZRoDUzXLUPzg4bQrGKogIqFo1LV2v10RkJvJEVx7DIuwYBQSENP1p1Nn3rPbgqD178LqW1QQCpNqdTqfdbrczizFEINj77vgj5Z5Sr3+MxBgV1XH8Gc/7+nJnkJ975rtv+cxlXzu217BqRFPCADtnExztc6qheDRLuRdWbO8tChAJ9pUG5axWO7ElSKqK4Iy5csdjyzTkq7TTgGUZ5gZeAyaZvu4Ms5jMLUwO/dAI5u948rHSD3aJVv30P95yymAwcXFr2kGPvXZZEpKHZz6QinUgScnTWDtpmuEm46wQR0pzcvvuKgS8vUNiwyoYsIeyqino7KoTsqaxIKqYQ/+Th1aBJXgAFmqpdlbT1RNQklCTol7asqYPEp0AC4pog3+KF32vuzSjBDuzYeO66U5ncs1Bs1Od3CoAEI53Yr79vyxEABqFen9zyhdgYq589kP/pvz6525qJkoBaBKFGHoH5wIEgIKJjgKoAEAOeMkFZiEFYafze3zKbdW8Yw0kNqgSVM+Y/zpmArgK467yoMoBNRpYoYaUf3essq6/5rf1QEs68cGp50GJvVC3L/z8Px23SOOD7qmxDEPh2lc+s09/phNkV0fiIGnmB94WqfSXQAOIblreR0JPtePrJ6Nu6FRR0OBYyKulOw7PTQV39ChONi+97P4NB7GOhIyl6JYKesthgQkiV5jcNjmJGnwkiIgC1hxISfpHyD2rXUMEQm0nZtcetH7NVCe1CkFEouAoJrqX+P/e5L5aOmoFgshp9rrwK8rx1nOf8cxVL/qnmTISR6VRk6rp0OaItE9Ep0hECu8cp486ot2AM6OAFUv8zaqF0ouYyApVeWzJPpoOjJGvQBdSBba+63a21pQcjv7w8QETn13x27/uL8VF4ekbPvq2tdRSe5oPeqpWgUTbpja0Y5gDiq8ds9Qh71jKS0WLqQZhRSY/gJbClyHeOh2obnsgFxPPWkTd5GeFcM+vSBK1dHHTe6HookIhGaJRxl9/FCasYsUKfn8Sphg9kwQggcTAiEjjj3yo92LpxKITCr4OZC26QmJkIEIAQtoLQHBftPgnShWAmTTJ0pp3fr9ng+09+kkPf3Hvws7MqoaAIgIo04NyGGECMzS0VndTOilCiGU/DmJHqszF8vZNFoIKbe/bEY5ek4sx7QR6lCwOshDQhoqm098dPjXkhluzEPLCqo++bAozESdL73rBQVy0/ET3iCP6roCAUtpxGhvrKpTgmaPW2iQG2OS7TM+iMCltgO2+m6pZFtqex/NhIlYtVDUG0ttbnWDbu66aABl0nvDJ9Zgje0YQanuFqPZcPg3aK0UtqK9Y34PaRyFgQEBFf6Kl37OI0lgX0kw0OxeVJQJE4EjIMUS+1/j/Xs93H68DFaEgoZJi8v5fihGWl898+PP/9lvHHHv4tAUkZi5tx8iovMCQGE1743PilQuUGCnqoqpT7APVGZfL671KnOqTrk3vkPXLJiDaaGlQa6dYorESkkvvP2yQGxSEy0nnIxsfNkfelw3z5mc8OOSTJTpbB/KBIWI6kZXBZyrIwKlKUW2bURIHpe0rKtiEpEVeRTNMAwYZLFXS7aPKGqHlAHjAJKVd+PBx7QThR+eUiU/KtU9LCuF+VBUnNasmh/xq2cDRRka/PHfDpDjDhfIxhZqzhgoCvC8OllH1DPbG7Xvj4NHjkBFa8GhAce9jHkGjKQk6xcBCWitcaZMhpRCRCIX2nnDfEfekaVn5PYh73wj3qtzRx2hf4L4C/a04AoFAzLtXnNX/TYtt35111OfXvrcF69rKBYlq9yEtr4hFAQNQQ9Cg4ppUoDudfIQjwoBam+1TISBaHxhQtdZM26gbWhB8EABAg+zRLurDlEMOPngTL7vm9d2JuFzZ5ruOf2RIkygKI7FzUQDBKlFKpNI6UGiWUVlDINCjQYYLeegGC6jQ3HxzpYIOpc9N9tLDYwuQhUIVjS9abvNRp/TbXq5sGa8AJIAKPSfCHLkvSkmdHpdCCBElxG0bc4mxEhYRQaVX4CHv8jjvVUYdDaPhRLjvBnnvyrznNMA9Xsq9ngMAUGlMnnzRAvTHlxdOOeSWb/6ymQKbnLSSjt1/rEooCCndtDY5UOkiHGOILILG2Ks2ChCQACkI2IwhUp5qktoBsgALRsluPrgTAcsyEtrks6/McVAmg7FPV8+tuJFEUcIqlHUEEZ1rJiXCajn2aqCY5IYAaLwfyyJzMpN0WsM6i9vrJCPNESPLE8cwB4iifMHzY82IR//DmSiwvLVSFQmIB6p6AQGAGXMjzAc/S7OIQCjwmhPGtKKR8QqqlEbEgHd6Yvv6he7B1kR4VBQlxP95nQPcx3j+AIsHAAANeKg6bNVtyUTRaLXPPHr7X3GaZlCQNm58L7koIlJmPCtl0InZH2KMtB6YGQGU0pdtUBqjZwatfFmzsjbXoYpVDcKADqzm9NJTg6gwrCOE9NOnndmDRPrtS3/4hsqbRAkKEwYXQASSTEUgAcXdpDZDi1mmQAR7i0SQBuhN3fDb8Wx3snmaHJel2EGEdJArQEYKNSZV38dyVZzalS6qa9ECIzJIERGECCRtW1G46pEhGItSDfGaTYqQGUGEBfWooftOt/sHnzriSiPTKOr+f+Gb3aOs4NHdXVY8TXTJwav04zwVMIwij3vqLf/YK+NAAeSxucLMioAgiRUhIuJ0aj+k2ArrEgqgCAFvPpsxMscARrMAuyRPq3pyuQqKGYEQDMSbXupVqHwEnLhoxxvnlfVd5d792rTKc2LEwEZCABTA1IwsJxa639YV68wwi0DZXk5lOI7RXj08Ls+Wrnl8VWT9EpBIV7oBAAgcmGwfM+V13h2T2Z9jzQSIyL7SEiMJk0mUEEvmJYHgqujmpp2KziEiCGtjRqWG8MdaF4mACIJCAJQ/jd/nj5Q/MOJw4KcA76VZFgUAA6T14fNtKQE9qYhPaHx64rSdbYw0nU0GFWF0+Rh1YoOgqJB19AEnH7kkIMCsaJ7HK/DMSoGgAtFV0nDX3TpRO5HAGnSss/aOdNVQOxfZ2D3f+jvIatipzZsfcdLChM8YCQSwdhEQyKYAxIgQh4PrTx5mvpGQJwyxLAI14PeDxiOf5+Oe2at640ElMaDPvLZNAhSU2oEMTNLHibSywqqw0TAKUqgiiWfFkk6kEjRIRIC68sEstleBFgejxhWdICMx3O2dfreHiQd8tdfxA7y7fu6tT/2PlRWI2D9G73Jvw24oAGiGJlmDY7f4/nRddvasfmD16eSosGCz6eYqGNWSURBEskZPVNCY2ztnkEa8PjGKNjd22hWyZ9YGWEQqylW12OMqgAQhYBBq/e5YFX3hROnhN489gsrOYtL4OL+oTIctzUJMOpR1EGaVmSAaBLnmG7reizFGIlCsdBjjetu/faRlfG1b4eZ1FnThOUQQn2QQUYgHlYSMfCNrGgsgIipGIBEVKowsiFGPt5WAkEGCUAVhdd3qGYUYR8VwMikw3Jmu9J51vk94pBGJKzv8H6GcP0lWArv7vIPsvSDFkg3r6dnDq6ZwvpgP7GM2fXZx2YJtwMReTmhEJEgbJJryiYnkAKXv5RxgZlH69rE8GgyhiKOJTD3WVrqh+4ERoiAElWh7+WG1lbKGhLfe8dKd2fytjL/4zatL0XmKgihKe+djZCFLQTQIcGVuOThpFQmhMFKsq9agxDWv/dpZ1nFh8IZNLLwUjNYuS7TYAAJcOfYePYnVHkpd29oYJGGMniJoQ2DGoodEUyxBOAKxXJMlwYVRhy2StsJAI7atOz+5uzpy+0qavHeMX+5xc/1vtz/9CZm7ezbzlb/D8ULzMF19hOOGyzJqLDxj8E/eksbSctyfhxGTIijyWod99fSoEJUACauIlIZLD/E2OEa7tzaljWSrYh01O5Lak4U8bD2KIlYRffzYC8jQ99P09g+/YcpHGVOsxRPUvhAGaLEOSoVIUPDWXx1fxbRohKi5Dp5ZafCb2lxYTTVccHKpSgZmRYVNk6BiEpajEsUK62ldEzfrlJVUIYJx/RFmsU8mmFAYIuRR9RmXtN59Ur8Vq4AcgoKoQYx4FK/2KngliSJ7718EAJEOrEUAAKCMLH005ysAe0P5vXXuvdH4vnzHvUUD+847kpV4Hkdr6x51uv88+75aqeYgjMJrAeC8UkCJ50OmhjEhQd+OL7v2vRXCUuZFaMTtRCTK23EtujGT7m8fIgLhGCPHKADou61Ao5wDAQiDNhwhDplgNL1phunYdcnqqpq3up78xJnH99deON/e/sbz1legtbajlRw8oaXmLSYlBCQJPv/J+uA8TRIiQYiCURSA1mqs8NLYuTCJRRVEgDFJNTNiHMHLqECJQgKNyCQ0J4AcWEJkEEpHlSURjsQcK6v8/Pp2QEEHQBJJwwFb+crGfmdD/SNsdiVP8j/Y+/Q/IKPLQVR6bUc0Myhpzv7Fle+pwuIeyNMB4GgYDlkoNZB1DuwZ29egMWrx7S2tDioKIKpRyigzzCB9JomRBYAps79d2/Z1oGHrl1vPcRa/cXx4+zmnm4p0oiKIIEZfF6V3w4vBIwEqcT789KTECliFqMQ5wQBKKNVxmHYq/s3stAzKKAxi8lwzE4U6sIAoTw0DhMDMCnkzAMVK0DNj1LkSGY1fM/dKcOIqM8MlV1EAKQZj8S7vcFxR3oFax3va7uEu4dq+rfH/RzLaO9atNokxQo1QPuLB9mNbb280lqoGAIJEERRQjVy1Wzru3/MjCyqCGEIQEdnt17KKUSKNNjbV0ECsHIyGcEDq8TRccyIj19oOvvjCiTBz2x1P+bvxFw8KI9HoSICE0blGI803L6zKRyBmLtzaWzXAtJ0pRIi1GwFF2Ew5hmVL3z7OuBqQI1PasCCIGEYJPYwqV4LCUYSwv1kR1pWgEACrbIUPB4niQITEyZlpBaH0BoCYjd0fl++Nb+9isn+o30VWarKjw+n/Sobuvsto6XJIZ9umnTdClnbMn8EpH9nV70/WanSjIogoNk/aKsb92QpCJIXRxcjCEG7LOwxRCJBQQFCnCBgShBAFCUFUli7uOLl2NRZrP3+/B3bb8INHf273G7qOkgiaEiRC8T7WPqrftZK+AAsEJ+PnHlR4lTMisndRIiKrLAGVD6V14U8eUxZgUERsngKjAnFeABAFrcVRlzSCbJ2zFKtaEDUJWb0C20iENaW7vEr0OYp0gSMSaUrUgY/oHh33laz23WfJ9v53QNkN73b0nc7z33Pw/mRZyR9J1ZnQ7WQsg2RsePimzU/756W1y2kFgEQEggoYs2ZGIvsdF40CAKGKgMJS3zhDMYoYQhQWoYQIgksAXARCQmwHeyttCAPW5ke7X7gtMf1fhW/9/cTQglfjxgkCQKgdG8LqmuNJCwqzc5Ce3YKknZOg+DqyCInYXAmA7tTLTzmtX46wstOEmIk4uADCgtEmBChAiijCLWLEVREYEaJJR02egAihB/Db5Wj9TBXLZa09CINN9ts53FnnByr53kKovd9DgH3cfP9HuWuD45+oyT9a9i5Tjc52EkWtNRhDK3/qrfC0H29r6gwk8mjaAoRNQxml9ysdonMMIEwEAvWtB7HEEWa1MAPZoGLhtI+RARBINQgvOTTr1xgG//ksE1vt397yq7+aXoQQgm2b6FiAXR0Ai2Lz5iMLTYQSaicVOdCmrhnBO0YRYskSgMhZzz/q/WXtxTvQKlXCSMJ1HUQiC+fZCH1VBKPZPgmxqkE8M3OaCZAAAIIvXV1cLS5KKeUwjiJQtEbuat4oBz6x0eO71wjqwG+P3vf//3mv7/dBNKetSI1VE3kyvhxf8+kTTnxJV3eRvY8AACwkmLIg7d/eg6s9k9KCKAJ+fpUQS5QwYmsk7VUsI7nIACBASmVwxRGwSIP29+9/VHdW4LM3nnH8oMrYJMiiSUQ4+ABexvNVxztAQmHvIe+TTTPKBSSM4BYZrGJIKnatynkR9g4UWogwOiQKM4skBoCQQowiejgN0UeAyBzZJitvdJRQJc71xzmmNMaSS9QgDEbtLTzs/WclTLubZYvczZGDA326fQvg/6j0/1eWvlciJ4FaUYXWxkbZn2hNnvMvbznstdUUcvCRAYVFgejoo+C+ejrZSQPRcFYXVuzyruMWIpPxZL0I1Ksw9eKrViCKUWkvE0bvuvbsHWlpb73y6b0I6fU/e9JjB02nbIkThMig0JeetJDc74u9RCJTHHLC/YRkjHUgrqsgEoUxy1AoWkmHqg5eeWqKTw2QkcDcB+1BR2zmxCQuUEIu9aYFA2djMGSX6nHxJAqEg3BdjO8YpK2i3ehToAgqpMo0KAqMciwr/vkIkwYBaWXYFPfH6SMVESIAEt1JX6OaOQLAiGwdZOVzB9S79yZ3VsKhfS3dKxL5XiPzOy2Ovde19/wjh2P/YpSVi9t7TLSBk/FpN/T5IYdltWn+zdhPP5i9QbD/8yu7MfXANAJnGd0brtw0A9cOimz9xMbxH61vphQjaIsKFE/+563Arg7GVUAErBOF9W/X5tVi0vzMsyTNuPWr6uVpLOwym4REABE5MCKz1iPyYpEQWNiWeTt0o1AMAiAIrJMRV5kQxMAMCiMkmYhEgFhHjoyImBsS1onxRZIMSDGH0Qatfl8lgCaA0tb4Xi9NLj0u2b1KhCXGGAEFFBHdZ0vbtyf8T8p9PN2dKgcHKHxl70KkpAEVZ1Prxzkuvv3LxYeW35PcqIuryqIBpDXE0Szf/gyUiNJm4qATTzh87RTedpJCZkCiYYTI6rr16CsvuWdAxdHkBpu9c3mQt77fOrWLmLuPn94aOrYNznI1Ai2LdUCIMUlxNNAVnWNhEmpkQ0/oayYRBE4aJCTAQOhdZCEImDUEhQVDAaOuQ9WwmjiCJIYgqriurAMzc8Dbm54sYB1ChCBNKa48OCSYC3NkAQYGrf6A0v/I7fduadr7orSRgvZmAO+a/h19908/3wFfqXyc+2ybq1ePmZv0i15jPviL757arsw1ygEi0Qp1+j6lK2AkO31IW9WpbjzpiWXpBVFikmiXX95ax7UnzF0qACw2Vab7yIcuQbr95+c6RJr40APfQ9rqOmCWkwDgihvHkI6UjhJcEBCfeotDUOLqEVw6pRkJMogAOxeBOYrNDBAIgq+ISMWach1ZYYgyIBfHYE4NKx84hlimBw/A1EZbTeKj8bxjOsyIFY5RaRUgolH/ncLGfYzKcSWzew/nu68LZr8zcpcTAo7sisG2lBNxenJdNrPzlPnfjL3z48sBqt5tETjwCqLYAZaOIsFjothXPpykR1PXHJX3PHbxyTVXbIIqFIsIWQMFTHjp09fPWuONbl/57edlaTewcSYzACDIvg4CgDo1MCJ49VEEMGqTXrUjk+D9CCnTpnvLu+x9iMCeTTaqfmP0kRk0czqmQy0KQbfSWNbQXKhZAAHMfDZNzALBszBzNINiA+vxWsSjAcUSySD/d/ZnBLkPFnivZ7vPq+he00f74lFhspMJcB2SzsEbK3zmP6n7Pfq9h2c8dosn4gBqxAW433sfAaTGqGSuxrisFQlHwdpjtmvxgeRqgMqWDiBikpDE8d0DP3n57kf2s9DBdz/7sCXUKSRpngATCsTaMYhQmpCMWm1Glq2xo34ySH0dmBlAMLMsICII3oUR0lOa64gAKN5D8IxgWrk2GAWFB4OsE6r296JSqJXKtoI0ubbBoriB15hf0J4sUgCWQNCPgKIT+j9jEfzBZ32fjrt3Swe4D1q/ZzPf/zMAQGBR0y1AQxAnkkPjpvhlePnizglVxFujUiLIDuAAR46JUGmNTvXrSsWcRwVfTNKkdcFhnVAFiG5MewShPAFpL7TiIP3hOQEky77tz92TLZF3S5jrGAkZwqiXTqVGGCWCuIqRI+g07W3ZBFIzCgMw5dozCgKBrxk4kDZZigLMwJUbYXukuVvAFCJAaFAQSp/8UE/AKJJsXReQGKnm1JSVBLphHZvmwLB4crtLRWSSP3Ge6e4i9xTK3VfBvYWbO3vrsv9nf9LZ9p1g1H2vUxCNEELzkMnspZ9fppd8bTbHfLnPGveGqAdU2ST4yJGSYasqA5duhHTmy76+5gwe1owsnQYQgkoMQB0qNfFt/SCFRXPu/W9ZqKFJoWPadmVMlP2oJW4EY8Ig3jMyAzXjfH5KRUFGDVpoKQgKCEIMAhDIGKMQhQXYB1QIDJaGt5cGImkVqApmqbHeMYcYI80frQdGB3vB9QMS0BRXj8lsMCISyS2UimiEB/Hf0NL/uNz3zWO/7FuEe7eAlW7uCoE5ovJFXDV74qov8UP8nsMHMS4HhbKCG0h749YYkQiR0jlVY9Fwmr1A8IwmuXHPg6tY68qTnnQRIGmRU6KW9a4LX96vVLL6ZS9Zv2wdqOjRJkWDo9NcC8SolGQBUowIZa24snlsUnbBNOPQlRxBSjP5vWWd9jJBCEUNFG1dJw3wJGRkWDI7q8tUsq0qLSAJVCu0AKpiAIvCmO0erzKQoL6L7UWIwN2nP3fJo658kMg/GwenGuAJlYhw5P3x8r2Z7V7z218fH4Xne3++900x+tz+ePweTkR0b9v4nSx8bxw/MvY7p3P2NcjL3o/u3Sf2a3+lNQAhRgEhUUSKFaa1JLNv/PoCPPl76w+Kbk68UD1iUb/TAKMAADmIYKIEARZSGBiuP82ZIUjkSE0ENimKQojef/1BOrbcwd9InrUjj772epDZwfiCNYZCYEBhShABUCmsRQTdEKhVXXgMxkqEEUEl1W9z4DERiF5AhFAlFgVFCHzlWVRwOuT6siYJiAAJM0qIQtEzKVO1DsmGhR5b8qvABS4KRJtqZtJzoHdwcyhawd7W5/tu7gceKXf67w+EcP9j3t99uW5FxCwzj76iOqq9Y20aaq+sXjnRAVW2URYoVBBQ1RJEIhNJcHz5w6XyIBG8mgTkpAGixM+N3VQ8hFWZz3/wdTqyIi9WWlaFnJwC5yOSsMoVCgOyL10ALdLMud58NPYqAQYUk9+w3HLDQT+gqyJEJtBpAkAAyFXpRYDBhkZx+QQKCIACBhLnAdkDUNaNeUhS8j/vt3tDyEyGYnMtjFXSpMWq1pAYABx5rXtLqngPlvYnPtP/4wF7O3P+xPPeq/yp+QFEEoljf3ZTsvoR42sOz4Z9x7ISLt55VFkACweRpIIIwFGAWeYHx6gFFBZk1YmIWY6gYg34/Yd1gvar33HGA+9QNStE39KghxmFKquCjNwKIomRfQnASJiOVckvm7NlJIYozMZ+Kl/WE6lN0NdRYoRIqRVBIKkLj8iMFk1+Wz2GQAACOMLsBwFBgXR3jYKlqN+sbfUZhEFFtBpFylhBlw2oVAsg/PGTi3d7enBgt8VKl91+M/8DR/5PafwPOe5/4JcjUAiHJPPdIxKcOGSMOPJKA8j+vncYFSAcMqhKZIRhzSz5BYePDSrDIAiUBaMz6wBcb/JX5Ql1rFf98sYf34YYoyDUsxh91h3TvFQzoqAyeqU7IVQUJXJicvj0A1rzQQQEolh60ca0GpRZVnrPEgEiaWQAQF/WYCASgJ+xV2yyARFphILmvUAkxIjJlhn0gvbWD38UBhoDIDkiIog6WHXFFuQkpYCwr7nxvnrHK63vI43/P/UK75PKAYgZdSzSZyhr+i1/RKdpA6h4Z+99ZdaBo2JVLVCQGERiFOKrHgpdW8YogKAB0gSF+P8r7kuDbLuq89Zaezjn3Km7X79BDwmheUBIILCYJBIGM5hiMsZ4imMHxyF22anYCUmqXHbs2MHlUJ4JrnLZQLDjKS7bGAeEMCAJhACBzSgxCJ70pCe9sbtv33vPsPdea+XHubfnfv2kJ8XrR1ffM+y9zx7WXvtbU6PLH/hux9Q5+a7/mJKL5JShV3TEG0rSvRUUlKnIgZWQDOm9jcZEXZN+7dMvCTomBQAxHXzJU6WvRRrXkUVBBZ0FAFUNNROhOkm0D75yIygCkigQcRNVGRHE5g9c2YSsJ/UVN46D0WRIwVkFTcFHueSGSJkHpamc9Vj6bp2dzmIZ4FQtp+dqyPqEHBke65jP2k0qChaki3WTyiztd8Bp657edotKUDXHvolRYlJgIf9gfE5VoiQWRkCLuWdFrouPzj+rBI9/cu0r78+wJKsO54468s2gwa98ywoIm24OAgiSmuoTjU0mK8hOfuxZk4YbUUT1PZJhCAI5lLHN92wzowAiHJmQkzLTnD/zyNPb1igKQKySimpizLOHFk3OVXn97VdWGcQ6lmy6uYpwGc5U3/PTBjIrQDpdnOcxDFuMLtYMK3bba/ExhiF5okmFDKOj0HF+IXWHsU7SGkhsXumIqlInxEeOcJKUAFXJf/PgBRXUVlonNwueBFDS+ONvXkWrX/vwf12ZH4/zOEkFyD0VTiCYzt9cjqIilHlQBA51Of46FWQtcvVLb1c9tT8IIIHNqexllatN06TWb9g600LCDAQxQSV+kU9MLhAFVlBUhBRZRZWZMreMzpTOLC/AaY3iOjnbjlcWdbAvDe0yOlJF3Z0j7ybIrQ3m1jc3PXnWMX2iRvwxFTNrd1LjmBH8Sj7UvBmIOm+m3lkk0n6yUUA12ECs/MeeEYIDRAao9OOvLmskRQERsJLmCTUt57dd+RQCKX71HSad6YI468Mlt2U9sZbdkaPPqcSgR6+ahTp3o97XwuFhNp4n9OU41MVprxMbaZ61M9TBt/7WjXDkTF02bp+NTCjsalZwZDKTeXPP5UYUHYhR4qqZCFEWfbCLx+7fB5pFyquVHBBoLDqnyUuqqyKWnVEP+kmIYXrOVhFtXc9p3QZmem5vB2nKuBFn/uoASK3Ju0JrN92+2Lq3n0VLNntqy2Cs6fO3cAgRndW+YRpuZyJ7nTZERJiZ0WpwACKdUARFRh/BCQni9oiRmFX77hteGfyqeJtWJ3Y4ub4zrqvIgoQqehDcuFfbM//4CpLl/f/9WTeaoGMyimSX7r1JFQH9h27osA9APQw2ZraEzvG7rhgHHXScVWjtIYEEHSnU+wx/2C9naX8Tet840s29NQDkogInRaihH81dz14TOC1RmaHqxFBvvnPyQadIqhxjYgVQzT0qp5T6VXDlYDK3YaG0wwJwTgqU81WhtmjKWe5voVnLngg6+5FiQ0ixVlBBbOb09ksg1IVpanUWbz980UoyRWuIKoIZNGTH2d8fvGLFHfrkyr+fW13BmiBqOvTR3hURKdLqp25SRQXuoKhohpMl+5I3FB3qYWgEpGlYWUkwtwAstb3nBjPCWFG2nHWAU6zrxBUDJwXp7fPmga/dUs1aKRKPexTo+ao3kPvHrkmAjjTEFisrvFGOIU7c0hHkvLfVNg62rJStK+d8MfYNg7hR3DvX7eW89fbn8sKmla6ApDZ8/QUreSMu10A0fv9rzSlq0khUVQV84ereSjrz6e9eTVT9xg9csVw11ogIO/jI6zkisrl73+GhiUbJQ8bduiab7HMuG0OyaDwAN0FBwYgpDCDJAI4yUMMLg/Ctlf0oYGxWuJpVBUDO5FB8qHvhWhtTXH0AFGC5NvukvuG9i8AKlisFVRFASwiSEqfex473GzRresd11eTuFpBPIK2FntnFjnIdfl0b3CdGmbO7CrZ9AnCLlz4A4ujIuCcRZbI6UfvQXzUvWomYaoMqqmq63bGH5e6HLz2gNv+9a25ZSs6pjeDw4K0HnzUk5Tz87Yu1JAaZM+Iw+KxpbDPisbEWjUGJUQBRDbicAPo8fvDKlZUCZXJGwzfGBomQ6hMhtdFcDg3C8pdvimsBiEBWHmVE7fiiS3LxjzzFk4KWq0So01woqMxgT91xgWa13/h5s67Y2LtbV9T5svWttNdK3yBTnNeob8CZz37Kw0152VAVkNLhP9jnZTiI6rpufPcfvO+SI9jYiVFQRXS9YydvWjWTO35mgosfvffXC189lEcSkgw/8LOltVG6X3zwmQ2JibZY6ouNzlSjRICZIYiAGBsGBFQS7wGwce7S37aP1HmNOvhh36lYUXH565eKIKqiqfv+X13qq9nstPVSqShiGldmrhoWUQgwISKqSpshXJKAWXr+ZSXNualP+mZf8J0U3U8w4Vrg1T3HoG3S7sEHzo2mGR0V4CyZ/QBAt8Z7V8Di5Mdu6oP1YmW47F7xW68cDzmZ3KsCGnKd+5wEd/vFl+Rx/L++b46r+iOlssFY3H3gOeMsWdQ7b8jYJ5Luyc/ZBN16OaKRqtIREKLRpmYEQCDjrQKgq0ZyOnZr7BdHDz5jRZRMljUPORAABbvgdXzt/srM2GEsTwmh4kQXMVTWN5FQOU5zNqgaQolNApr74Qzqfbx1T99rZeMWeqx9vwHUQYA2FjGeZdR329MfL+3ZYNwW5B+x8z+PXrMSs1UGtT7Zp5045Q2fbkaqQETG18+MGD74+nF5+P29V+zDSj/LXiyKu+011UBqb8f3vLQEFDXdBz7rIJXU9TEaawouOpaDpiittxEaFIQUsLti5mqk01hY8c6ixOrkNxyKgoJ1eTJZ2ax1RrW6BAhAXdN1Jikbl7s0GdWJFRFUjEFNiXXs7cQYXWd4jxfPfJy0JkvswVY2DPr57+kIe34m4pqbNIAioxP1dPAn1Kh6TIYiC/STVJqhIXWaYP+JS4Dthw9eEdzwT95mM24e/sLABWoWHjjyQpPqfJL/48o1FTYmdAYfvJEdAa2skhfCkXMgzkSmqFIknQwGEQwVSmWUEgk7IRsQa5VDx37pmsqRmMTzJhAkLEC8ZYus497XOzBRE/cZwAxApRS36lAxMXNWFEljRSYZxk7Tz3VqLr5JOt8YL252Hp/Zl+vapXYodgvztiFc3LTctkuhdRufVtVeU5mGP22P++utWYv7PgOC1mnKudqG7sB5tujbgQh3NPnVad43gLXv063ndDAvePFidL5GR2oGlz8XQJgVqMGQZNGfviim8o7vWpWFP7rloBfE0SvLYa8pB3/z2s7E5qWr776ycuNudMvH770GmZGTgiqKbcMTp6P/mJsSRL0FRIAqa4Z5lRBATWGBeOHUZMV+/KkCgEj5miBGbUD6UIdIMsfi25xDiEiQBABY0HX7c4QgrGgQgWl7XqL1xbA7G98DZjsHtr/17jkymJ0ew91vnStta+zGDIygABhzqEITHEHqXnrlUxehjX6J5I3HBYJ5jl+h65VOf/z13U4aNVf85MC6MFj5h9eAKKo1H35DZqw4eeqD5nIhghAARYl916kA4OeOOjSVwU6ORKox+olEAVD1Xcsmlf2DcGzlqYyCYLobBl1RkepkTs9BI6bjqYWOwWiTVBmMquv2DCqLIqEBptxuX6T/HzDxrdjtnjNkhwfWBvu8DxPby9+2FC5YUO2agNbQwsWH3Chxy+msOG9tdZAqvvV1jIP3P+36kMsoTvIJDuHCD1x/uMpj6PKH3FWjBCWag398tY9KUgcAUdSiZxhQzL2XAvdXyXYzQBJwFZYlGRUF3zFCua8eXrh1LmdVVVus+xirMaBaAX+zZ4ck3Wy60oG4SQCJbErWziyUkFDFbU8wOX3nrF10fstqx997Su/bduM1tr9zsedM6/DyOtHm+wD2sisW5g4uWN/rdalhjYkVVEQbDV3PA9QHjt1YUf3hN6CHxtiscZ4H+tE3p0gkrvnwv1xeddCHzqmPXR0gaAqxNXnpFMiE8dixp1eRDOaFVUIFm331NKFJKja3oFrz3MLyX700qYKqy9YbZ0gwhSBV4dmTFnYKuSCEhkVBI3fmPU/3agFQ8Pn2vsK9Vvoe4MaehFvmDeLZ5xFO/R+21Xs+uODmGjbtXbgBnJlCQriaLXRd1us5ZzPDGlOaYnEIvAB9Oxnc+kJu+h+bf351cDKykqo4pvyu7DlqG2dDeNVL4/6GoCT/r1+ERNDENot8nhkA1Pig7FNTD6Dr2xrVf2TcSZoU8wIEdcDi/+obN0VIoFT4dQHMxAlonZL7voPjriw4mrWZ66iiJjV2cW5dAFJRyuyOeQ/3GtLzXOw7sOtzKG1bpVOhfrNF3g7v7Slj7MneEboGvQlsQDHPKA1noZHRQe7u9VQ/cv/Lxm74d6/LvR+mqqbC54Ps/a/1YJAx6bMfDabOYSGb/89PRyVpEoACStcIEDCvYvROwHStKCigHvn2BQkTIXYKEavDrOSbfm6egRFNx21gd+NlwCBA37nQgJvHaQg0VW4YBEDcXHe8RGsaDFGTme1WUrMj9OM9j5/L80/Enj4D72eDfx60uyA3q7iJMSTjQShzYdKwTk8NLDEb3zXBzp0XzPO+rzxw87BYrbOJpGacxtU9r69jKqpkcNjrjrr1BPed5klMIUZtU1fnIEAqeuLYsSq4suOx1XWZL7qrVl2ySJkXNVCEYrl4SSyBESi364MOk1UgVlhNjdHOzPUYWmWcqlBnoMdP4JQtooqS3Tmo/54r+Ync0/cqccuevk5P2ErfTlO34qlfNgAYD0qaEjNbAuFQJxC0wNZf8MG4WFb3vKoqHn37Wwu1ZWg8VtQbX/Y7b1wUwpTlWqKoChpvi6pgl00Ccgkm5oVaYgpnXvTO3GnwhalzCkgMt95yqhgbgMVOsjYSd0LdjGOenELPNTz7KM7e71xTkjFsKDqPLcoABlYbUJvYxnH60tDYNB4FIIwZc6FtMBrY4N/dJpndi8Ejbj2Hz2CNvTpUkQim2vf2DVzfnRG3IqStcr9t4qb6iIgQVFJi0Z3g2XUwYKa42a1FstGrS7X9iE3taCtQ6AvEiFVgnoIXNYYvvnzsvhkOh/p3L31x4yGpAKKTp/3DJ3+8BFIBbpIBQADjkzFCESmoc9xARkaBUgN57gjQOSIRkgRHTlwu0o/kLbaApbAgCIOq9xs4k6sOX1iXHgVAwK2rDDAlQFD0DVT1pzopgSZRVRRwBjfzxelyOseoMU8azerew5S2hYl2vQ+P09wWNylcNhTGyrGaz1BrTCzKCpqyxc92nlHSHc/rLecvvqpXdbhBQXQNHH3Hmw7VYJKaVCXDQgo+ZwtMkiaJDSlmHYMCVE6ALCrbPBNJ7IT8Z/2FK5qR9G2rLGjtm5mQYVDAuqDp/2HcC8PUIKlglrWYtgJAEwFFUatuvvrNhWGOmhhFkSGzsFOftVkPnyzd6rmWu4YM71bOFGyjXYttkdu9Zu+OaOJW0JdAmGOMoybZ6kzAGJmZRcAM/u41Lgzvu3bJp+c+9XSWN4EU0PrF/4Nv06SoirEWEAEF7wGQ0S49UEAQcrlBVICmxkZSEsytGABUk+5+DiuUTjuujedBKSRRAGEosg18qRoOqBHrGECoyGYfpNJERRXMXfJ39PaDorKqKClmtNk2rj3inbdcNO3Mx2lssWHzPjteMJMGd3lqTV//mAnBwhaVHqUYY9CUqORy2dSRUQVUXXY3PB/il668sAmd0bg/Z8rkGBSb6q3PgGXPAqiJQVhITUYqRtHd/63LpQa23qiqSkyeVQhspuCIVczRr72pAkjQneEoGBoWIExqvAm0NmruZrLLjAygajO3Jt80NaOomKHF4mPXNVUmDAisFoybRhrb1NtTc6nH01tPDK3pXPd6bCM/euJM7VFb9rl5sceqblLiKCJQmFEb+lohufd8f1b17roYaKgu2G4MCEoyyeJTx6ddg4IUa9akqupzIBES/dxqUIMJiwIEIEQQckTWeQ7CpEY/SxcpSBb2WRAAROEyKAuCUsfpho+dePdIKU1CVfJumqdCVasAwKpisvrBbx1a6WkTgFRFyHnZmskBZlqQ81/rj0dq3taUszykAK2Mt/5zS/2PE0PCHWBYCVUVWVUaPvYQzGIhIZp7Oi8L43Lp8kYOAusilGITo843l2tlTUGMWNecBBDJ56qgJOU9/Siq6H1CIWgSJkRR8pkqMAKNb31GLgzQyWaoSj0JwAIJbdcKrXOgLk/KbgGGQI0z7WatAFrFlp8nOjT48Wck4UkFCAAJvdtp6xYF3Kg424tNb73/eNn6DuVuMKHbXp9KK//PULPtdG7DvtPkJNhSpqQmJAXMuf7yZx4dutlhYPCn/0bR3XvBZWGyWqdBSA3aJKjlfOX3oUQSxBiYBQnQORUEkvLIYmFVyMYlAsIkyClFBrLUyYgkLn3ielszNvsSaBvYMNSMKspoc9INwXcr00m1eNcGjVzvrCDYIn51OPWaLppUBUAFZbAGYJMkhxu+9Unj7+cyKWar92wPyYZD+m5lPF60eHo+RwQkNNiweag0alIpg+yj876y4gScEf+RC29JBF+4sBafi9FeCJzIQAbdXiqNUHRcIghFm8ruIIFmVMuZR8YxYjG58C8fMqCjiXIG6rE712BqOGTp8898SukRiwyEUDQlYkJ0FiHMdwCmUTkBVA2eaGxdghrKex7UeI0WaDUYFQSDWdZ0T6PEMrlUclSpMytq1gZ9NtnX47u3tCVS3HTxna/N3C45V9dmgwpRa8owvdQmgdN1oJDIGGOIDO0cd7rVs2/HDWY6fEIE4ZRm19uNAsEQbXBrUuAIWXZ6GEgmsSvj4f0Uzcmo/TraGD7440YSjC+PIoCuwEogMWFypnUGIFAX7lGBpM4SoHK0Jb6sCynGi9LqZQzArMoKCpYUCuN9GW76hQNRbc125kAuSaVpGMgb2LQW7GqDDfVDOJXtdyweS5uLhhIVEEWptXmwlpRlJpiiyp5gys6r6TwZwS5TRNf+bBvGDXaNTwxNZQLcfKV1qtiQP90AA8QTNRnrnUF358EDFVs3HFMnXfbep79wkk0eDIcTAarJYiWaBJUL305Eii6e/JJnTJDnqEDMlud/6tmj7mDSWSr3J5LErFFRyRNgXSZKY9g/V6lX61GhHfMxI/ncQtdv3o8pBsRa/BdP5j1EdMdPqU3QRFQhFCEBEcbMGWVAVYR2GW1w29qZ7e4yvI971NeRl505xMxTZpfX1+o9J0WKbn9u4++NYWBnM45FNljOIBlDcflEgoozWBH8xMszTX1e0Pkz/gP3/chEWI72HLTG0GNWURR1Pd+WzyRyx0EkSFRkCJChGZ05li/5Sd31nztjlTixAgOAzUhBSYPYcDxFge5gdl6TEIyVWFdN38EUR2l7qtYIeTOme/YfAIuUvjCUpFgqiiIqGEVVBW9AtA1QaS3ADpZBO/ffpk47p1d2L2vtz1bacEraXMnjkQy3uVFtrWwNXt9QuQKIbOgUElYirtU5UBHqfnrxkuUM6pSK44Pj7/s+M+mOsHe5tKFbYoUqZAT6XauiCsQFLN91cyAA17EAUMeOdrBWEJnrDF6ISDGBoiqidwiAuY+V5P3Ka+j10vQ0lWpWTmCzQdfNjrRtk6tJUs3NvaevypssIX61MIF4PFWvIIEBUHSQFBFAknoHZ7cFfvJo5yGfoSm4mwC2Nyz7WOfGTpxGdR2GJQlAYPJ9tZUm+EP60R8KAhrtqpmv3v2Cf3b8ggCrB/ePBRSQEgsiILqBp6QIgMT+jv0HQiDKc1QUlnpFKORNQcY/3zQkdQRBRKXMAqoyaSgRtCi5cMkoqqLEGlPALkzqq0jWxDgAAJ0gjgr65BWHo1DMyzMHDJgyeQYQJQOAAkhOggAnEjZ5pnoOaOuOOPx5gfMIgNun22yZ7+AHu3EvOWcUBs8FhtVtP1pDmfXaEJCp/8zKaGzw4O8deP7DRSKfFqJ5z+i1Vb4ywBo7ZyyTGopaZSRoswJVgUSU+OTtr6kMk8mcAkmXR1FcNilq30PQ4FKZlC0CkCdFNTEEIWADkBftAURVUmMFNEzOBKeM0yDuCgBaCyrHTF9dTDrRwH1mEEgmJgmAgDEgoIDWcSPQRC+J8kzEAhs4J1obpPNlDnu93yaK2T64jx1zwz2PdJvuT8ccYYPXqpDzxOLEERT7uunPXtcNopnqUH/xcz83aKDqjfvl6cwgAkKQCoyo9RZUkRAkFp8/c40ZWzDOqIJO4qrUUb9dZos+GPXIIakIgJJBBYjWxUYpQaO9DE2LmCinBhxGOPQs06KVa80+gyz5JD3z2lJoXJj7CldjGFMUAFEiEBUgMpBUQmBhcBaU1ofg7OwRd/z38dOue/q6Lfz2e0+45m+bd6629U/zjYmAaBJCsiixNnj/6j+/P1jBsuk9Mvln3eDHjmmpg1ElGVFz5L4CNEsXMAiCkYTZ6E9fXtQ+ab9gQCEbNFK2fKQbXcbolEaCagHYuP2NIKANMYMJasy6HBEJGW0zBiiJoZjsD2IpgUNBZvVuxQiFlI9eNBCX7Ajvu0GKKmhlbUIHxkZMnrDQ8Sr6P57YJSy6lSCym33vbpYy7b9rs2F2TN9m0ADrL4Aq0NZJtFYu4ibEb6ZcmQlWOJWxNr0HOsvmizvgBJuawtQ6iUCL0tKsfQZSTK0MSzh1f6epINfKPTB1smrPsetzT1MSVxTY0IG/cDAIMuza8pL/8oNQyoF5FUuGYlVU4RvLxy+aNHPNYJlILEbKRB994CUVWERnFBFAQ+BE385aM2RVZlERloB59IaAOCVWRAFHigrKiKFmpTT2NX913fFQEICrOgkQqMtyTJjNSXMpkQlqVSVg4YgMSkwGozm9PK6bPiSwVvnJEuTOZ1Fub9N2eWv3djtCBo9p4zUFgAS+yC1wq5JtnSu2l9Iym5nHQIvyqiREsjbB/DXHxxIyiH6y0uelXnF4X2iMAtjeiOxdF7zg4hw/U/QXLDCBoEnF79+8X22wNrMKhChlnRi+PGd6mSKqBmYWFs2zBTYShWITkyAqFhZIQZmonCSJGCDFh9bNYAUJeFwyq1HNe4TKUR86fRkzT8QoYDI5JE6RGT1MksOmoxSSGqPnFf15Z9qwns96jtYttPne5hJh+yzaFZOPCZgBt4kqhCAs2lrq4AYz4u3tX9vTW88mBECuk3OqS4c7YrQDONeh/KKLu/s6ZaWJUwk8eP/JMbL//Kd6QEGQ1Sb2Rz74I6dAWbPcKBIAV0GgPnrYda0AAjYsoipATRGZk9GmYRZQtQUpKqASNjVzTbbsnV5cD3UmRJTKBkBAtNMNVDiwj+SDJlUlK0Yla4RVhIw19enYWzpKBnOISdpsHU8GPf6Vrjsv9Q3/nr1sROsgycZBVQBQFmZRsmgMiHCaXd/w1JrgR+0sRCBQBbKWVdQ6jYNDC/NzlPVMZ9BdXNBBQTUjK9uqd+KdL8sN1393Y95ZoWRFbaLuu587ByxOCtvuek3FisfpcJYDAyC2eRTBIGcdl9kiVY2qqkCRTdEjCE3kxBGag1+/YWN/EIRGEUXVdlwCUoX7F0C1ShohQZGDkjXGEHJV0+ghXh1Bh8pRAsJdzY3+yWj95DT9eXbFylZCjoCxXovMMTv/qyoohyqEmIA0bLWyWJNZQMEqtEnVp4dTqjIPiAaqR7tUdYMuImt/nvcrqwFFMBqO3HhpI70HV16hudhoBBiy4YfefhqNpLyDgiCq41oZ771woXCiqKp1ZCUA+ub+FaOGY6ojCONUba4I1KxOUkpYQ9F54I281lpS4ioCABvKu2q4SZS+dhGQBKQAeewVowSsIoyUHQz2FVdfcyJGuxoKevKMos5jU9dNBeiaacfmR3arIEuNNYaU1x8FBYCMk5AlVE5qrKbNy7x9Uls+o1YBZuEQAUE5WQsCVutJBqEDPOm5wXyR6jwoJUZYHeg1v0kS8s9e4bn2gZgoOPz4vkvGZUe1yFUQVWDUYJQvPM8XlACVuY6tAXzvugFVqJEjk4gB6lAiAVAcn5pwTITNwVVzYF1QIcE4Dsgg5LpFdBGd49OvU26CQU35ZB5DAgYmsMZflOuYHh3lvdWRAMk5wrBbhwR27/R2/ZzfbNoIqWwtaFbybu04s1CQJBx1tlwXAbIknJwX5d1jqKkqWAVUQABBBVBmDMk4I5Jw2O0G7rLFgy50VIJCjEZ6Y51f7RDwN6/WeSozaDzV3fDR71A0E5gDJ0ogqlWwQR94nXpSRJYUkwAJwmS+Mk3mk7K0g56BghIo1sNoUuL5pnNmfzesSSoomOpkEgna3I8sGInDE0/jugyOSMrTc72EbU4wB9CR4tujsj+KbDICfrI29b1X+m71bpfQ2sgj62e0nd+cXV0c3nr70g1vvjJsuVWTtWn55HLnwAHPjGab7LCB01sr7ATVJFSKmAshEYiSBKONmIZ7B426BJ5FySlGhyGXxO7ha9lFNBhih6I+Xc4YR6k+FB3puMi/uFKZhbsPXZbFuiOTLh0biRn3JV8+WjSpo8wNIyMlv8DkCBHM0qNSdWpPIz/4i+scJwOihoCtSafKzigfLlQXZeRjMTHFg80FAYcukAndv/31U6yoytnoEK7O4wNLjRmnaIc5qZJM+3EjO531LszgvtkvgNZwElpYqFXc7ND7COsBhndYkbqRPW9+TlscDWFd2FBfg81qE8L8Sp4HM3E06TI45IbySdFYqyjRI9b9NJ4v3/Ghp728/zd/9IsUMfWaxpMCUuSexfTt/3GKhirFW94k2ObO3NA+gfWtwApaLxhlprpHEQA01hWB2QLkhUUAhBiTTDtJEBT4pqrHdYfFaFBTfMdKf9goOe9UpA8Pr0Qx/K05Bh9FDTalILsY9335YlQSZa6ncFHHtM4q4RHgXLMsULd88KVxmuROlGI2qU3QGlNOnRVLdSHha5c0dhgyjuwfXhiUdZPY2JB506+rUZNEFVkF6DzFuMfCJfbaFrbS7LmYB3XmgUvqwo4yM/KjuUm98MicZ2y6qRlMunWVV6Z7eoBFFRa+8qtz//vw6uDGEzTsEZ/qD4a5QI0DXcIu/PXcz1yN+Minf+XO35r04i6fgYpIBJKCTA+ehIgqKiyQjGFBovmuA0WkEKIoAjAzM4uG5873EwKoNex02fVTQHQdBywRV+8fJ6D09auKbuLIlkerKAkqpi9dB0IsKVSqrIo2QxEFUgMddQ1R1lkYnb6SCVUAW5eQSWPqWFGzr5u7DGGS7PtuyZoRJMaUff0w1nVkI9jYDN3SI8t1CFGANTHibrDrY6dz13K1y/xcLW7ypAWtXLKyguKKpV7KhqY7PEScnemxjWVWax/6xZlFrUd2Prz7ee/qDAfLF11fzZsgB2LtTeJer2kGPckO9K/lEOdfdftn3tsbw2bxfb3dRGQREAWnwJ6qgiIRc5oERqPq+9OEiokFQYUTtpFWq0N9XxdVnljVyZlxY5NPpuMYyNcn68xGu/TQVUbBgmq5NBwEQt8vVy4VQTVSRxUFsB1UVTHUuGc/WpqQWaP2225/TcitZlVcuZo4yoC7F9jVokTXpztO3Mxn2I6loOzhm7VJIsiiYGTl+KhOKkrI0DRG9+70sw1eqwZqf+xV0ObZcO7VRrIJ9n/x7d9+3c9P6sW6C5rf+b7/cEU6fbi879hzfXPkGXXQOB+JCjN+/33vWF6ob/uzFTegV35XcfwCNtDnJiuCC8ce4E984zJWJfNDf/eW9UP6FjEeAdGyNT4akemBHRRUkEBSFCTFvjcqAAgiACoppRbSxeguT4RGnRiOyC6INFW/61SSP/2IlDZ1j3Sv6zkxIKDZ/gh9O3fBJy7tN4BgpEEVBMkGgAbEyuiT33N8lBlrTFN/9SkgDECGmwR55+ElTmiXz7w0YzFYVPjQz33n4WaFsSE27uELa/TaRARPWD10JjGCqIJCXef6+NNtzmzzzj/C29mJoHEx+80L/9MfffW65aL4m2/99JFfue733+GLP7wv/+o9/+13f+l1P3/ZMA+SU/7pz/3im3Hh/77D/8ill4T69jt/++iLv/MY2Juv04c/eseD3XL/3b/5rs7YUN6fLA/ShtHezN8BLEGIy2CMTm0rFCQ4p8EYBQDbJxAlVE0KKpETIwoaILR5sE1v7BmtxknNqIKdjhhNSIVbBr948nlXNRQJgDoX7Iemw0V+10UgqCQxoAgBFV2DKFHT5P7SFozjtCDyrWuMESRLYVzhQjq5amLH3W7nSttPnaqXLvzB12eP1qFCqorIF44Y0ESD3svyqUaRpFVnx6ig5yW9t++eXYW5oS+n7+yEqu5OJqnVO5ffI9c3vBBP/drb5M9+/eufws//Rv/fXveHn/nC5z/1rh/83jf0fPmVO47Ssy8M9LY7f/ZVHcX4G8efvu/IbbHb/cPXfv/ff+KKlz9r8SC/96fe8B2DcTf+9k8sVLjW/m1jDhaBpewWhqeJgdUKq1SjQBSRzDTMpnJkwRSZuYWeCciCAmOT1Gq5WkXDPuvYhijH/d26Zji49FxbjTukAJTlpELU3Pc9EREwVAFUQWxuUVpl3z/85dMYvVKN+s2rwKiCpnIc3PJkJICJv/GG7qrKJCcd5z+Z6knkyiXoHPcHj2EUp4ic1Y8GFTMN20wK5jGM+G6C2Lmy9w0lnVNizdkjte+P+keLe55rbB3nluWNn159xmfv/MnTb7uJdOlO+Jkrfvsf3/NWqt11V73xRjP6tV/4np8+4Ef9h37yxl9O18HPX9X8efm3oxvec3nlKubL8zOLk4Xqrdd+Lye32chio9ZPbSx6VBfEM+kdLCTk8UlDKEqYWuNo4ciqzCwKoEiqkArb5KuFNYBS1YaBo4pIzO04n4N91PQeelXtLTMSqCiaUSGT09czImCsRRVUyIImY5BM5r9wGUdpxJvyG88UdswSywb9yqPWKQV65KZo87qDuHRghZM8wpaRu70vH5+selGDbBtTn2q9hNGACDGa88i8uBlC2Vs6334kPDfqLxODHL3v2Ucn10p04Z23/RZ+bzF4NdZobv7Mjz2rxut/p6xy51Aiv/F33vLXg1DPDd+d/+A7b33TO3sxu/gtt7yipyfn6nx1/vmfvHrc+8KvXvgrnTPzcZdNHRXA2ggwrwwgyogNkaraclWoAgM1kQnoGOxkWTgGZgEGg0nVXGSii4WSKqZHNJR5t9YqOavBS+WTwvLJ+8msZu3GiBYhM+95mRVxjNpwgthp8oLEp6iQVn96UoGoU5Xhu59Z+waNVmOQE2PDapvsUblspcNkKt9rnIkPCI4Btbjoa5/6yI2lr8o+N3516BgBGIENO7hyP6T1UBTaxtDcoufeeGpvO0dmpo1r7qItUL2bGoxEgUDX6tkKAc481TeI0a0Cf9qeKtd+86J02223vBqq3sEfvfu3LhsWPwATCwK3vEiDUU7UBamMNp3r/vwPJhk46Lzs/l989V8eigzlS16s0kiPMeYpXR0+/akTb36jjBbD1AJFBWdmGwizxCR268cQAqEIiLbneUVC0NSAcEptIHpVUOl6aiEP1VRPGpprxjA/n60dD5GzX35qSn7WwZHn6vyuLCkoNGNmSLHBDkV2hjjVcRTIoUpSVxw+IEk8xCgcKsjHAouH7+9BXnawLJKYvF4aNQTdYacr/+LVz/qahurAiTlttFGDZBISGHexP+Ajnu9J/cmnLAZxlz7l5eWVWs+P4N8BLOd21fXqLV6ONpKv0otvzsqQV/qi52YkdbIuD9CiPRTBWGjMa64tJtYN802V6NR+YjrY2/zTDYoBZoiKqqhREAEgVWmqD29z4gl18zb1T2U9aDc0Re56i/PKs5luIL2+YYW1YEsWFd9kRVWwGkbmpMHkhgmiIVs4u1rFANYqcRFrb1XKUhCw6e9boPzApW9SbRM2QUqnjzdqmoQXHx5dfNUxcQ984Ee7kWNIQJbJ9fd5n+dZJkz/VPaw504N5Sk1tHCoAk+Wa6yNT5YmBG1w4xkxU4Z18mOfRfEpEKNziYFbUACi5lD7+RdmLL62nVmOnpnBzKZsVdsGnRQRRJDJAAI3jKhAnJK0N1o3V9XMEgMAOBeb4mqq0ToDWq1zNxdqNWktBjLquF+9OSqAmlgKB7CsBqiDNUGT8Oql4XicUKGAeeM72khdgVW4sNvPueYbbo6xP6E8WN+MT4184sVFOpAnO0zl/i8eyieMnJBACaH7lA5mlVZklB+LLPdPQwqESj4WkzOLxdBK35QhA8uEmyJmUJ5GVGCylfc2yFxDIECwtqi6yUVMdVZKzPp13Ws21aGtIrXlHvj/ADNz597usgKwAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "machine_shape": "hm", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/spa/fine_tuning_code_llm_on_single_gpu.ipynb b/notebooks/spa/fine_tuning_code_llm_on_single_gpu.ipynb new file mode 100644 index 00000000..153522df --- /dev/null +++ b/notebooks/spa/fine_tuning_code_llm_on_single_gpu.ipynb @@ -0,0 +1,1128 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "provenance": [], + "machine_shape": "hm", + "gpuType": "A100" + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + }, + "language_info": { + "name": "python" + }, + "accelerator": "GPU" + }, + "cells": [ + { + "cell_type": "markdown", + "source": [ + "# Fine-tuning a Code LLM on Custom Code on a single GPU\n", + "\n", + "_Authored by: [Maria Khalusova](https://github.com/MKhalusova)_\n", + "\n", + "Publicly available code LLMs such as Codex, StarCoder, and Code Llama are great at generating code that adheres to general programming principles and syntax, but they may not align with an organization's internal conventions, or be aware of proprietary libraries.\n", + "\n", + "In this notebook, we'll see show how you can fine-tune a code LLM on private code bases to enhance its contextual awareness and improve a model's usefulness to your organization's needs. Since the code LLMs are quite large, fine-tuning them in a traditional manner can be resource-draining. Worry not! We will show how you can optimize fine-tuning to fit on a single GPU.\n", + "\n", + "\n", + "## Dataset\n", + "\n", + "For this example, we picked the top 10 Hugging Face public repositories on GitHub. We have excluded non-code files from the data, such as images, audio files, presentations, and so on. For Jupyter notebooks, we've kept only cells containing code. The resulting code is stored as a dataset that you can find on the Hugging Face Hub under [`smangrul/hf-stack-v1`](https://huggingface.co/datasets/smangrul/hf-stack-v1). It contains repo id, file path, and file content.\n", + "\n", + "\n", + "## Model\n", + "\n", + "We'll finetune [`bigcode/starcoderbase-1b`](https://huggingface.co/bigcode/starcoderbase-1b), which is a 1B parameter model trained on 80+ programming languages. This is a gated model, so if you plan to run this notebook with this exact model, you'll need to gain access to it on the model's page. Log in to your Hugging Face account to do so:" + ], + "metadata": { + "id": "FNdZ-kD0l78P" + } + }, + { + "cell_type": "code", + "source": [ + "from huggingface_hub import notebook_login\n", + "\n", + "notebook_login()" + ], + "metadata": { + "id": "bPlCJYDK6vrF" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "To get started, let's install all the necessary libraries. As you can see, in addition to `transformers` and `datasets`, we'll be using `peft`, `bitsandbytes`, and `flash-attn` to optimize the training.\n", + "\n", + "By employing parameter-efficient training techniques, we can run this notebook on a single A100 High-RAM GPU." + ], + "metadata": { + "id": "WMVe_c8q43Qo" + } + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Fp7i8WMCjKJG" + }, + "outputs": [], + "source": [ + "!pip install -q transformers datasets peft bitsandbytes flash-attn" + ] + }, + { + "cell_type": "markdown", + "source": [ + "Let's define some variables now. Feel free to play with these." + ], + "metadata": { + "id": "16EdABzt3_Ig" + } + }, + { + "cell_type": "code", + "source": [ + "MODEL=\"bigcode/starcoderbase-1b\" # Model checkpoint on the Hugging Face Hub\n", + "DATASET=\"smangrul/hf-stack-v1\" # Dataset on the Hugging Face Hub\n", + "DATA_COLUMN=\"content\" # Column name containing the code content\n", + "\n", + "SEQ_LENGTH=2048 # Sequence length\n", + "\n", + "# Training arguments\n", + "MAX_STEPS=2000 # max_steps\n", + "BATCH_SIZE=16 # batch_size\n", + "GR_ACC_STEPS=1 # gradient_accumulation_steps\n", + "LR=5e-4 # learning_rate\n", + "LR_SCHEDULER_TYPE=\"cosine\" # lr_scheduler_type\n", + "WEIGHT_DECAY=0.01 # weight_decay\n", + "NUM_WARMUP_STEPS=30 # num_warmup_steps\n", + "EVAL_FREQ=100 # eval_freq\n", + "SAVE_FREQ=100 # save_freq\n", + "LOG_FREQ=25 # log_freq\n", + "OUTPUT_DIR=\"peft-starcoder-lora-a100\" # output_dir\n", + "BF16=True # bf16\n", + "FP16=False # no_fp16\n", + "\n", + "# FIM trasformations arguments\n", + "FIM_RATE=0.5 # fim_rate\n", + "FIM_SPM_RATE=0.5 # fim_spm_rate\n", + "\n", + "# LORA\n", + "LORA_R=8 # lora_r\n", + "LORA_ALPHA=32 # lora_alpha\n", + "LORA_DROPOUT=0.0 # lora_dropout\n", + "LORA_TARGET_MODULES=\"c_proj,c_attn,q_attn,c_fc,c_proj\" # lora_target_modules\n", + "\n", + "# bitsandbytes config\n", + "USE_NESTED_QUANT=True # use_nested_quant\n", + "BNB_4BIT_COMPUTE_DTYPE=\"bfloat16\"# bnb_4bit_compute_dtype\n", + "\n", + "SEED=0" + ], + "metadata": { + "id": "hru3G-CLmqis" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "from transformers import (\n", + " AutoModelForCausalLM,\n", + " AutoTokenizer,\n", + " Trainer,\n", + " TrainingArguments,\n", + " logging,\n", + " set_seed,\n", + " BitsAndBytesConfig,\n", + ")\n", + "\n", + "set_seed(SEED)" + ], + "metadata": { + "id": "FyZSXTbJrcnC" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "## Prepare the data" + ], + "metadata": { + "id": "pO7F5L5AtKo1" + } + }, + { + "cell_type": "markdown", + "source": [ + "Begin by loading the data. As the dataset is likely to be quite large, make sure to enable the streaming mode. Streaming allows us to load the data progressively as we iterate over the dataset instead of downloading the whole dataset at once.\n", + "\n", + "We'll reserve the first 4000 examples as the validation set, and everything else will be the training data." + ], + "metadata": { + "id": "1LmrIZqP0oUE" + } + }, + { + "cell_type": "code", + "source": [ + "from datasets import load_dataset\n", + "import torch\n", + "from tqdm import tqdm\n", + "\n", + "\n", + "dataset = load_dataset(\n", + " DATASET,\n", + " data_dir=\"data\",\n", + " split=\"train\",\n", + " streaming=True,\n", + ")\n", + "\n", + "valid_data = dataset.take(4000)\n", + "train_data = dataset.skip(4000)\n", + "train_data = train_data.shuffle(buffer_size=5000, seed=SEED)" + ], + "metadata": { + "id": "4oJZvZb-1J88" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "At this step, the dataset still contains raw data with code of arbitraty length. For training, we need inputs of fixed length. Let's create an Iterable dataset that would return constant-length chunks of tokens from a stream of text files.\n", + "\n", + "First, let's estimate the average number of characters per token in the dataset, which will help us later estimate the number of tokens in the text buffer later. By default, we'll only take 400 examples (`nb_examples`) from the dataset. Using only a subset of the entire dataset will reduce computational cost while still providing a reasonable estimate of the overall character-to-token ratio." + ], + "metadata": { + "id": "sLQ8t0LM2GR6" + } + }, + { + "cell_type": "code", + "source": [ + "tokenizer = AutoTokenizer.from_pretrained(MODEL, trust_remote_code=True)\n", + "\n", + "def chars_token_ratio(dataset, tokenizer, data_column, nb_examples=400):\n", + " \"\"\"\n", + " Estimate the average number of characters per token in the dataset.\n", + " \"\"\"\n", + "\n", + " total_characters, total_tokens = 0, 0\n", + " for _, example in tqdm(zip(range(nb_examples), iter(dataset)), total=nb_examples):\n", + " total_characters += len(example[data_column])\n", + " total_tokens += len(tokenizer(example[data_column]).tokens())\n", + "\n", + " return total_characters / total_tokens\n", + "\n", + "\n", + "chars_per_token = chars_token_ratio(train_data, tokenizer, DATA_COLUMN)\n", + "print(f\"The character to token ratio of the dataset is: {chars_per_token:.2f}\")" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "KCiAvydztNsu", + "outputId": "cabf7fd0-a922-4371-cbc6-60ee99ef7469" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "100%|██████████| 400/400 [00:10<00:00, 39.87it/s] " + ] + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "The character to token ratio of the dataset is: 2.43\n" + ] + }, + { + "output_type": "stream", + "name": "stderr", + "text": [ + "\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "The character-to-token ratio can also be used as an indicator of the quality of text tokenization. For instance, a character-to-token ratio of 1.0 would mean that each character is represented with a token, which is not very meaningful. This would indicate poor tokenization. In standard English text, one token is typically equivalent to approximately four characters, meaning the character-to-token ratio is around 4.0. We can expect a lower ratio in the code dataset, but generally speaking, a number between 2.0 and 3.5 can be considered good enough." + ], + "metadata": { + "id": "6F13VGobB3Ma" + } + }, + { + "cell_type": "markdown", + "source": [ + "**Optional FIM transformations**\n", + "\n", + "\n", + "Autoregressive language models typically generate sequences from left to right. By applying the FIM transformations, the model can also learn to infill text. Check out [\"Efficient Training of Language Models to Fill in the Middle\" paper](https://arxiv.org/pdf/2207.14255.pdf) to learn more about the technique.\n", + "We'll define the FIM transformations here and will use them when creating the Iterable Dataset. However, if you want to omit transformations, feel free to set `fim_rate` to 0." + ], + "metadata": { + "id": "rcwYFRPpwxea" + } + }, + { + "cell_type": "code", + "source": [ + "import functools\n", + "import numpy as np\n", + "\n", + "\n", + "# Helper function to get token ids of the special tokens for prefix, suffix and middle for FIM transformations.\n", + "@functools.lru_cache(maxsize=None)\n", + "def get_fim_token_ids(tokenizer):\n", + " try:\n", + " FIM_PREFIX, FIM_MIDDLE, FIM_SUFFIX, FIM_PAD = tokenizer.special_tokens_map[\"additional_special_tokens\"][1:5]\n", + " suffix_tok_id, prefix_tok_id, middle_tok_id, pad_tok_id = (\n", + " tokenizer.vocab[tok] for tok in [FIM_SUFFIX, FIM_PREFIX, FIM_MIDDLE, FIM_PAD]\n", + " )\n", + " except KeyError:\n", + " suffix_tok_id, prefix_tok_id, middle_tok_id, pad_tok_id = None, None, None, None\n", + " return suffix_tok_id, prefix_tok_id, middle_tok_id, pad_tok_id\n", + "\n", + "\n", + "## Adapted from https://github.com/bigcode-project/Megatron-LM/blob/6c4bf908df8fd86b4977f54bf5b8bd4b521003d1/megatron/data/gpt_dataset.py\n", + "def permute(\n", + " sample,\n", + " np_rng,\n", + " suffix_tok_id,\n", + " prefix_tok_id,\n", + " middle_tok_id,\n", + " pad_tok_id,\n", + " fim_rate=0.5,\n", + " fim_spm_rate=0.5,\n", + " truncate_or_pad=False,\n", + "):\n", + " \"\"\"\n", + " Take in a sample (list of tokens) and perform a FIM transformation on it with a probability of fim_rate, using two FIM modes:\n", + " PSM and SPM (with a probability of fim_spm_rate).\n", + " \"\"\"\n", + "\n", + " # The if condition will trigger with the probability of fim_rate\n", + " # This means FIM transformations will apply to samples with a probability of fim_rate\n", + " if np_rng.binomial(1, fim_rate):\n", + "\n", + " # Split the sample into prefix, middle, and suffix, based on randomly generated indices stored in the boundaries list.\n", + " boundaries = list(np_rng.randint(low=0, high=len(sample) + 1, size=2))\n", + " boundaries.sort()\n", + "\n", + " prefix = np.array(sample[: boundaries[0]], dtype=np.int64)\n", + " middle = np.array(sample[boundaries[0] : boundaries[1]], dtype=np.int64)\n", + " suffix = np.array(sample[boundaries[1] :], dtype=np.int64)\n", + "\n", + " if truncate_or_pad:\n", + " # calculate the new total length of the sample, taking into account tokens indicating prefix, middle, and suffix\n", + " new_length = suffix.shape[0] + prefix.shape[0] + middle.shape[0] + 3\n", + " diff = new_length - len(sample)\n", + "\n", + " # trancate or pad if there's a difference in length between the new length and the original\n", + " if diff > 0:\n", + " if suffix.shape[0] <= diff:\n", + " return sample, np_rng\n", + " suffix = suffix[: suffix.shape[0] - diff]\n", + " elif diff < 0:\n", + " suffix = np.concatenate([suffix, np.full((-1 * diff), pad_tok_id)])\n", + "\n", + " # With the probability of fim_spm_rateapply SPM variant of FIM transformations\n", + " # SPM: suffix, prefix, middle\n", + " if np_rng.binomial(1, fim_spm_rate):\n", + " new_sample = np.concatenate(\n", + " [\n", + " [prefix_tok_id, suffix_tok_id],\n", + " suffix,\n", + " [middle_tok_id],\n", + " prefix,\n", + " middle,\n", + " ]\n", + " )\n", + " # Otherwise, apply the PSM variant of FIM transformations\n", + " # PSM: prefix, suffix, middle\n", + " else:\n", + "\n", + " new_sample = np.concatenate(\n", + " [\n", + " [prefix_tok_id],\n", + " prefix,\n", + " [suffix_tok_id],\n", + " suffix,\n", + " [middle_tok_id],\n", + " middle,\n", + " ]\n", + " )\n", + " else:\n", + " # don't apply FIM transformations\n", + " new_sample = sample\n", + "\n", + " return list(new_sample), np_rng\n" + ], + "metadata": { + "id": "zmejYvEKw1E-" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "Let's define the `ConstantLengthDataset`, an Iterable dataset that will return constant-length chunks of tokens. To do so, we'll read a buffer of text from the original dataset until we hit the size limits and then apply tokenizer to convert the raw text into tokenized inputs. Optionally, we'll perform FIM transformations on some sequences (the proportion of sequences affected is controlled by `fim_rate`).\n", + "\n", + "Once defined, we can create instances of the `ConstantLengthDataset` from both training and validation data." + ], + "metadata": { + "id": "AwW5FviD9xBH" + } + }, + { + "cell_type": "code", + "source": [ + "from torch.utils.data import IterableDataset\n", + "from torch.utils.data.dataloader import DataLoader\n", + "import random\n", + "\n", + "# Create an Iterable dataset that returns constant-length chunks of tokens from a stream of text files.\n", + "\n", + "class ConstantLengthDataset(IterableDataset):\n", + " \"\"\"\n", + " Iterable dataset that returns constant length chunks of tokens from stream of text files.\n", + " Args:\n", + " tokenizer (Tokenizer): The processor used for proccessing the data.\n", + " dataset (dataset.Dataset): Dataset with text files.\n", + " infinite (bool): If True the iterator is reset after dataset reaches end else stops.\n", + " seq_length (int): Length of token sequences to return.\n", + " num_of_sequences (int): Number of token sequences to keep in buffer.\n", + " chars_per_token (int): Number of characters per token used to estimate number of tokens in text buffer.\n", + " fim_rate (float): Rate (0.0 to 1.0) that sample will be permuted with FIM.\n", + " fim_spm_rate (float): Rate (0.0 to 1.0) of FIM permuations that will use SPM.\n", + " seed (int): Seed for random number generator.\n", + " \"\"\"\n", + "\n", + " def __init__(\n", + " self,\n", + " tokenizer,\n", + " dataset,\n", + " infinite=False,\n", + " seq_length=1024,\n", + " num_of_sequences=1024,\n", + " chars_per_token=3.6,\n", + " content_field=\"content\",\n", + " fim_rate=0.5,\n", + " fim_spm_rate=0.5,\n", + " seed=0,\n", + " ):\n", + " self.tokenizer = tokenizer\n", + " self.concat_token_id = tokenizer.eos_token_id\n", + " self.dataset = dataset\n", + " self.seq_length = seq_length\n", + " self.infinite = infinite\n", + " self.current_size = 0\n", + " self.max_buffer_size = seq_length * chars_per_token * num_of_sequences\n", + " self.content_field = content_field\n", + " self.fim_rate = fim_rate\n", + " self.fim_spm_rate = fim_spm_rate\n", + " self.seed = seed\n", + "\n", + " (\n", + " self.suffix_tok_id,\n", + " self.prefix_tok_id,\n", + " self.middle_tok_id,\n", + " self.pad_tok_id,\n", + " ) = get_fim_token_ids(self.tokenizer)\n", + " if not self.suffix_tok_id and self.fim_rate > 0:\n", + " print(\"FIM is not supported by tokenizer, disabling FIM\")\n", + " self.fim_rate = 0\n", + "\n", + " def __iter__(self):\n", + " iterator = iter(self.dataset)\n", + " more_examples = True\n", + " np_rng = np.random.RandomState(seed=self.seed)\n", + " while more_examples:\n", + " buffer, buffer_len = [], 0\n", + " while True:\n", + " if buffer_len >= self.max_buffer_size:\n", + " break\n", + " try:\n", + " buffer.append(next(iterator)[self.content_field])\n", + " buffer_len += len(buffer[-1])\n", + " except StopIteration:\n", + " if self.infinite:\n", + " iterator = iter(self.dataset)\n", + " else:\n", + " more_examples = False\n", + " break\n", + " tokenized_inputs = self.tokenizer(buffer, truncation=False)[\"input_ids\"]\n", + " all_token_ids = []\n", + "\n", + " for tokenized_input in tokenized_inputs:\n", + " # optionally do FIM permutations\n", + " if self.fim_rate > 0:\n", + " tokenized_input, np_rng = permute(\n", + " tokenized_input,\n", + " np_rng,\n", + " self.suffix_tok_id,\n", + " self.prefix_tok_id,\n", + " self.middle_tok_id,\n", + " self.pad_tok_id,\n", + " fim_rate=self.fim_rate,\n", + " fim_spm_rate=self.fim_spm_rate,\n", + " truncate_or_pad=False,\n", + " )\n", + "\n", + " all_token_ids.extend(tokenized_input + [self.concat_token_id])\n", + " examples = []\n", + " for i in range(0, len(all_token_ids), self.seq_length):\n", + " input_ids = all_token_ids[i : i + self.seq_length]\n", + " if len(input_ids) == self.seq_length:\n", + " examples.append(input_ids)\n", + " random.shuffle(examples)\n", + " for example in examples:\n", + " self.current_size += 1\n", + " yield {\n", + " \"input_ids\": torch.LongTensor(example),\n", + " \"labels\": torch.LongTensor(example),\n", + " }\n", + "\n", + "\n", + "train_dataset = ConstantLengthDataset(\n", + " tokenizer,\n", + " train_data,\n", + " infinite=True,\n", + " seq_length=SEQ_LENGTH,\n", + " chars_per_token=chars_per_token,\n", + " content_field=DATA_COLUMN,\n", + " fim_rate=FIM_RATE,\n", + " fim_spm_rate=FIM_SPM_RATE,\n", + " seed=SEED,\n", + ")\n", + "eval_dataset = ConstantLengthDataset(\n", + " tokenizer,\n", + " valid_data,\n", + " infinite=False,\n", + " seq_length=SEQ_LENGTH,\n", + " chars_per_token=chars_per_token,\n", + " content_field=DATA_COLUMN,\n", + " fim_rate=FIM_RATE,\n", + " fim_spm_rate=FIM_SPM_RATE,\n", + " seed=SEED,\n", + ")" + ], + "metadata": { + "id": "AgDW-692wzOl" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "## Prepare the model" + ], + "metadata": { + "id": "rxev1sk6tRW9" + } + }, + { + "cell_type": "markdown", + "source": [ + "Now that the data is prepared, it's time to load the model! We're going to load the quantized version of the model.\n", + "\n", + "This will allow us to reduce memory usage, as quantization represents data with fewer bits. We'll use the `bitsandbytes` library to quantize the model, as it has a nice integration with `transformers`. All we need to do is define a `bitsandbytes` config, and then use it when loading the model.\n", + "\n", + "There are different variants of 4bit quantization, but generally, we recommend using NF4 quantization for better performance (`bnb_4bit_quant_type=\"nf4\"`).\n", + "\n", + "The `bnb_4bit_use_double_quant` option adds a second quantization after the first one to save an additional 0.4 bits per parameter.\n", + "\n", + "To learn more about quantization, check out the [\"Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA\" blog post](https://huggingface.co/blog/4bit-transformers-bitsandbytes).\n", + "\n", + "Once defined, pass the config to the `from_pretrained` method to load the quantized version of the model." + ], + "metadata": { + "id": "UCtWV-U42Eq_" + } + }, + { + "cell_type": "code", + "source": [ + "from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training\n", + "from peft.tuners.lora import LoraLayer\n", + "\n", + "load_in_8bit = False\n", + "\n", + "# 4-bit quantization\n", + "compute_dtype = getattr(torch, BNB_4BIT_COMPUTE_DTYPE)\n", + "\n", + "bnb_config = BitsAndBytesConfig(\n", + " load_in_4bit=True,\n", + " bnb_4bit_quant_type=\"nf4\",\n", + " bnb_4bit_compute_dtype=compute_dtype,\n", + " bnb_4bit_use_double_quant=USE_NESTED_QUANT,\n", + ")\n", + "\n", + "device_map = {\"\": 0}\n", + "\n", + "model = AutoModelForCausalLM.from_pretrained(\n", + " MODEL,\n", + " load_in_8bit=load_in_8bit,\n", + " quantization_config=bnb_config,\n", + " device_map=device_map,\n", + " use_cache=False, # We will be using gradient checkpointing\n", + " trust_remote_code=True,\n", + " use_flash_attention_2=True,\n", + ")\n" + ], + "metadata": { + "id": "XuwoX6U2DUvK" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "When using a quantized model for training, you need to call the `prepare_model_for_kbit_training()` function to preprocess the quantized model for training." + ], + "metadata": { + "id": "bO9e2FV8D8ZF" + } + }, + { + "cell_type": "code", + "source": [ + "model = prepare_model_for_kbit_training(model)" + ], + "metadata": { + "id": "Qb_eB4xzEDBk" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "Now that the quantized model is ready, we can set up a LoRA configuration. LoRA makes fine-tuning more efficient by drastically reducing the number of trainable parameters.\n", + "\n", + "To train a model using LoRA technique, we need to wrap the base model as a `PeftModel`. This involves definign LoRA configuration with `LoraConfig`, and wrapping the original model with `get_peft_model()` using the `LoraConfig`.\n", + "\n", + "To learn more about LoRA and its parameters, refer to [PEFT documentation](https://huggingface.co/docs/peft/conceptual_guides/lora)." + ], + "metadata": { + "id": "lmnLjPZpDVtg" + } + }, + { + "cell_type": "code", + "source": [ + "# Set up lora\n", + "peft_config = LoraConfig(\n", + " lora_alpha=LORA_ALPHA,\n", + " lora_dropout=LORA_DROPOUT,\n", + " r=LORA_R,\n", + " bias=\"none\",\n", + " task_type=\"CAUSAL_LM\",\n", + " target_modules=LORA_TARGET_MODULES.split(\",\"),\n", + ")\n", + "\n", + "model = get_peft_model(model, peft_config)\n", + "model.print_trainable_parameters()" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "_pAUU2FR2Gey", + "outputId": "63328c2b-e693-49b1-ce0a-3ca8722f852a" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "trainable params: 5,554,176 || all params: 1,142,761,472 || trainable%: 0.4860310866343243\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "As you can see, by applying LoRA technique we will now need to train less than 1% of the parameters." + ], + "metadata": { + "id": "tHe7AElXzXVV" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Train the model" + ], + "metadata": { + "id": "T_CqVydc40IM" + } + }, + { + "cell_type": "markdown", + "source": [ + "Now that we have prepared the data, and optimized the model, we are ready to bring everything together to start the training.\n", + "\n", + "To instantiate a `Trainer`, you need to define the training configuration. The most important is the `TrainingArguments`, which is a class that contains all the attributes to configure the training.\n", + "\n", + "These are similar to any other kind of model training you may run, so we won't go into detail here." + ], + "metadata": { + "id": "Q_iN2khjrbD3" + } + }, + { + "cell_type": "code", + "source": [ + "train_data.start_iteration = 0\n", + "\n", + "\n", + "training_args = TrainingArguments(\n", + " output_dir=f\"Your_HF_username/{OUTPUT_DIR}\",\n", + " dataloader_drop_last=True,\n", + " evaluation_strategy=\"steps\",\n", + " save_strategy=\"steps\",\n", + " max_steps=MAX_STEPS,\n", + " eval_steps=EVAL_FREQ,\n", + " save_steps=SAVE_FREQ,\n", + " logging_steps=LOG_FREQ,\n", + " per_device_train_batch_size=BATCH_SIZE,\n", + " per_device_eval_batch_size=BATCH_SIZE,\n", + " learning_rate=LR,\n", + " lr_scheduler_type=LR_SCHEDULER_TYPE,\n", + " warmup_steps=NUM_WARMUP_STEPS,\n", + " gradient_accumulation_steps=GR_ACC_STEPS,\n", + " gradient_checkpointing=True,\n", + " fp16=FP16,\n", + " bf16=BF16,\n", + " weight_decay=WEIGHT_DECAY,\n", + " push_to_hub=True,\n", + " include_tokens_per_second=True,\n", + ")\n" + ], + "metadata": { + "id": "65QHS8l1tKQe" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "As a final step, instantiate the `Trainer` and call the `train` method. " + ], + "metadata": { + "id": "kB_fLRex09ut" + } + }, + { + "cell_type": "code", + "source": [ + "trainer = Trainer(\n", + " model=model, args=training_args, train_dataset=train_dataset, eval_dataset=eval_dataset\n", + ")\n", + "\n", + "print(\"Training...\")\n", + "trainer.train()\n" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "id": "rS3nVwhUC69O", + "outputId": "61a5bdb2-b7d0-4aed-8290-4bf20c2ccd38" + }, + "execution_count": null, + "outputs": [ + { + "metadata": { + "tags": null + }, + "name": "stdout", + "output_type": "stream", + "text": [ + "Training...\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "
\n", + " \n", + " \n", + " [2000/2000 4:16:10, Epoch 1/9223372036854775807]\n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
StepTraining LossValidation Loss
1005.5246007.456872
2005.6178007.262190
3005.1291006.410039
4005.0522006.306774
5005.2029006.117062
6004.6541006.018349
7005.1002006.000355
8005.0498005.889457
9004.5412005.813823
10005.0007005.834208
11005.0265005.781939
12004.4118005.720596
13004.7825005.736376
14004.9802005.712276
15004.3687005.689637
16004.8847005.675920
17004.9144005.662421
18004.2487005.660122
19004.7984005.664026
20004.7042005.655665

" + ] + }, + "metadata": {} + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "TrainOutput(global_step=2000, training_loss=4.885598585128784, metrics={'train_runtime': 15380.3075, 'train_samples_per_second': 2.081, 'train_steps_per_second': 0.13, 'train_tokens_per_second': 4261.033, 'total_flos': 4.0317260660736e+17, 'train_loss': 4.885598585128784, 'epoch': 1.0})" + ] + }, + "metadata": {}, + "execution_count": 19 + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "Finally, you can push the fine-tuned model to your Hub repository to share with your team." + ], + "metadata": { + "id": "aAERlCnt1PEW" + } + }, + { + "cell_type": "code", + "source": [ + "trainer.push_to_hub()" + ], + "metadata": { + "id": "1h7_AUTTDwE1" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "## Inference\n", + "\n", + "Once the model is uploaded to Hub, we can use it for inference. To do so we first initialize the original base model and its tokenizer. Next, we need to merge the fine-duned weights with the base model." + ], + "metadata": { + "id": "KBVH7uFOM_UF" + } + }, + { + "cell_type": "code", + "source": [ + "from peft import PeftModel\n", + "import torch\n", + "\n", + "# load the original model first\n", + "tokenizer = AutoTokenizer.from_pretrained(MODEL, trust_remote_code=True)\n", + "base_model = AutoModelForCausalLM.from_pretrained(\n", + " MODEL,\n", + " quantization_config=None,\n", + " device_map=None,\n", + " trust_remote_code=True,\n", + " torch_dtype=torch.bfloat16,\n", + ").cuda()\n", + "\n", + "# merge fine-tuned weights with the base model\n", + "peft_model_id = f\"Your_HF_username/{OUTPUT_DIR}\"\n", + "model = PeftModel.from_pretrained(base_model, peft_model_id)\n", + "model.merge_and_unload()" + ], + "metadata": { + "id": "jtL37piINBFe" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "Now we can use the merged model for inference. For convenience, we'll define a `get_code_completion` - feel free to experiment with text generation parameters!" + ], + "metadata": { + "id": "3USQ2suvDi9M" + } + }, + { + "cell_type": "code", + "source": [ + "def get_code_completion(prefix, suffix):\n", + " text = prompt = f\"\"\"{prefix}{suffix}\"\"\"\n", + " model.eval()\n", + " outputs = model.generate(\n", + " input_ids=tokenizer(text, return_tensors=\"pt\").input_ids.cuda(),\n", + " max_new_tokens=128,\n", + " temperature=0.2,\n", + " top_k=50,\n", + " top_p=0.95,\n", + " do_sample=True,\n", + " repetition_penalty=1.0,\n", + " )\n", + " return tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]" + ], + "metadata": { + "id": "RoTGpNbjDeWI" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "Now all we need to do to get code completion is call the `get_code_complete` function and pass the first few lines that we want to be completed as a prefix, and an empty string as a suffix." + ], + "metadata": { + "id": "0kMJiGDfDrBf" + } + }, + { + "cell_type": "code", + "source": [ + "prefix = \"\"\"from peft import LoraConfig, TaskType, get_peft_model\n", + "from transformers import AutoModelForCausalLM\n", + "peft_config = LoraConfig(\n", + "\"\"\"\n", + "suffix =\"\"\"\"\"\"\n", + "\n", + "print(get_code_completion(prefix, suffix))" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "nXlco2_-YcvM", + "outputId": "41c411ad-b7dc-4277-f975-c173888234bb" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "from peft import LoraConfig, TaskType, get_peft_model\n", + "from transformers import AutoModelForCausalLM\n", + "peft_config = LoraConfig(\n", + " task_type=TaskType.CAUSAL_LM,\n", + " r=8,\n", + " lora_alpha=32,\n", + " target_modules=[\"q_proj\", \"v_proj\"],\n", + " lora_dropout=0.1,\n", + " bias=\"none\",\n", + " modules_to_save=[\"q_proj\", \"v_proj\"],\n", + " inference_mode=False,\n", + ")\n", + "model = AutoModelForCausalLM.from_pretrained(\"gpt2\")\n", + "model = get_peft_model(model, peft_config)\n", + "model.print_trainable_parameters()\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "As someone who has just used the PEFT library earlier in this notebook, you can see that the generated result for creating a `LoraConfig` is rather good!\n", + "\n", + "If you go back to the cell where we instantiate the model for inference, and comment out the lines where we merge the fine-tuned weights, you can see what the original model would've generated for the exact same prefix:" + ], + "metadata": { + "id": "Ql2563kGlnmu" + } + }, + { + "cell_type": "code", + "source": [ + "prefix = \"\"\"from peft import LoraConfig, TaskType, get_peft_model\n", + "from transformers import AutoModelForCausalLM\n", + "peft_config = LoraConfig(\n", + "\"\"\"\n", + "suffix =\"\"\"\"\"\"\n", + "\n", + "print(get_code_completion(prefix, suffix))" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "29xxp1eHTgJ9", + "outputId": "c6d597a2-01da-4d25-a32f-3a551212c5b4" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "from peft import LoraConfig, TaskType, get_peft_model\n", + "from transformers import AutoModelForCausalLM\n", + "peft_config = LoraConfig(\n", + " model_name_or_path=\"facebook/wav2vec2-base-960h\",\n", + " num_labels=1,\n", + " num_features=1,\n", + " num_hidden_layers=1,\n", + " num_attention_heads=1,\n", + " num_hidden_layers_per_attention_head=1,\n", + " num_attention_heads_per_hidden_layer=1,\n", + " hidden_size=1024,\n", + " hidden_dropout_prob=0.1,\n", + " hidden_act=\"gelu\",\n", + " hidden_act_dropout_prob=0.1,\n", + " hidden\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "While it is Python syntax, you can see that the original model has no understanding of what a `LoraConfig` should be doing." + ], + "metadata": { + "id": "Pwy2ZC7U8Ema" + } + }, + { + "cell_type": "markdown", + "source": [ + "To learn how this kind of fine-tuning compares to full fine-tuning, and how to use a model like this as your copilot in VS Code via Inference Endpoints, or locally, check out the [\"Personal Copilot: Train Your Own Coding Assistant\" blog post](https://huggingface.co/blog/personal-copilot). This notebook complements the original blog post.\n" + ], + "metadata": { + "id": "CATYE8pp2drQ" + } + } + ] +} diff --git a/notebooks/spa/index.md b/notebooks/spa/index.md new file mode 100644 index 00000000..713aacbc --- /dev/null +++ b/notebooks/spa/index.md @@ -0,0 +1,31 @@ +# Open-Source AI Cookbook + +Open-Source AI Cookbook es una colección de notas que ilustran aspectos prácticos de la creación de aplicaciones de IA y la resolución de diversas tareas de aprendizaje automático con herramientas y modelos de código abierto. + +## Últimos notebooks + +Consulte los notebooks añadidos recientemente:: + +- [Using LLM-as-a-judge 🧑‍⚖️ for an automated and versatile evaluation](llm_judge) +- [Create a legal preference dataset](pipeline_notus_instructions_preferences_legal) +- [Suggestions for Data Annotation with SetFit in Zero-shot Text Classification](labelling_feedback_setfit) +- [Implementing semantic cache to improve a RAG system](semantic_cache_chroma_vector_database) +- [Building A RAG Ebook "Librarian" Using LlamaIndex](rag_llamaindex_librarian) +- [Stable Diffusion Interpolation](stable_diffusion_interpolation) +- [Building A RAG System with Gemma, MongoDB and Open Source Models](rag_with_hugging_face_gemma_mongodb) +- [Prompt Tuning with PEFT Library](prompt_tuning_peft) +- [Migrating from OpenAI to Open LLMs Using TGI's Messages API](tgi_messages_api_demo) +- [Automatic Embeddings with TEI through Inference Endpoints](automatic_embedding_tei_inference_endpoints) +- [Simple RAG for GitHub issues using Hugging Face Zephyr and LangChain](rag_zephyr_langchain) +- [Embedding multimodal data for similarity search using 🤗 transformers, 🤗 datasets and FAISS](faiss_with_hf_datasets_and_clip) +- [Fine-tuning a Code LLM on Custom Code on a single GPU](fine_tuning_code_llm_on_single_gpu) +- [RAG Evaluation Using Synthetic data and LLM-As-A-Judge](rag_evaluation) +- [Advanced RAG on HuggingFace documentation using LangChain](advanced_rag) +- [Detecting Issues in a Text Dataset with Cleanlab](issues_in_text_dataset) + +También puede consultar los notebooks en el cookbook's [GitHub repo](https://github.com/huggingface/cookbook). + +## Contributing + +The Open-Source AI Cookbook es un esfuerzo de la comunidad, ¡y agradecemos las contribuciones de todo el mundo! +Consulte la cookbook's [Guía de contribuciones](https://github.com/huggingface/cookbook/blob/main/README.md) para saber cómo puedes añadir tu "receta". diff --git a/notebooks/spa/issues_in_text_dataset.ipynb b/notebooks/spa/issues_in_text_dataset.ipynb new file mode 100644 index 00000000..e568de20 --- /dev/null +++ b/notebooks/spa/issues_in_text_dataset.ipynb @@ -0,0 +1,3360 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "pw6cvzTocw4G" + }, + "source": [ + "# Detecting Issues in a Text Dataset with Cleanlab\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0yPBE0Xccw4J" + }, + "source": [ + "Authored by: [Aravind Putrevu](https://huggingface.co/aravindputrevu)\n", + "\n", + "\n", + "In this 5-minute quickstart tutorial, we use Cleanlab to detect various issues in an intent classification dataset composed of (text) customer service requests at an online bank. We consider a subset of the [Banking77-OOS Dataset](https://arxiv.org/abs/2106.04564) containing 1,000 customer service requests which are classified into 10 categories based on their intent (you can run this same code on any text classification dataset). [Cleanlab](https://github.com/cleanlab/cleanlab) automatically identifies bad examples in our dataset, including mislabeled data, out-of-scope examples (outliers), or otherwise ambiguous examples. Consider filtering or correcting such bad examples before you dive deep into modeling your data!\n", + "\n", + "**Overview of what we'll do in this tutorial:**\n", + "\n", + "- Use a pretrained transformer model to extract the text embeddings from the customer service requests\n", + "\n", + "- Train a simple Logistic Regression model on the text embeddings to compute out-of-sample predicted probabilities\n", + "\n", + "- Run Cleanlab's `Datalab` audit with these predictions and embeddings in order to identify problems like: label issues, outliers, and near duplicates in the dataset." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "o__pRLFYcw4K" + }, + "source": [ + "\n", + "## Quickstart\n", + "\n", + " \n", + "Already have (out-of-sample) `pred_probs` from a model trained on an existing set of labels? Maybe you have some numeric `features` as well? Run the code below to find any potential label errors in your dataset.\n", + "\n", + "**Note:** If running on Colab, may want to use GPU (select: Runtime > Change runtime type > Hardware accelerator > GPU)\n" + ] + }, + { + "cell_type": "code", + "source": [ + "from cleanlab import Datalab\n", + "\n", + "lab = Datalab(data=your_dataset, label_name=\"column_name_of_labels\")\n", + "lab.find_issues(pred_probs=your_pred_probs, features=your_features)\n", + "\n", + "lab.report()\n", + "lab.get_issues()\n" + ], + "metadata": { + "id": "qaZA0cFs1fW4" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dp4lpApmcw4K" + }, + "source": [ + "## Install required dependencies\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DjoWBgGAcw4K" + }, + "source": [ + "You can use `pip` to install all packages required for this tutorial as follows:\n" + ] + }, + { + "cell_type": "code", + "source": [ + "!pip install -U scikit-learn sentence-transformers datasets\n", + "!pip install -U \"cleanlab[datalab]\"" + ], + "metadata": { + "id": "fRsBIj3L_RUb" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.467211Z", + "iopub.status.busy": "2024-02-16T06:26:13.466877Z", + "iopub.status.idle": "2024-02-16T06:26:13.470222Z", + "shell.execute_reply": "2024-02-16T06:26:13.469761Z" + }, + "id": "zgezWF-2cw4L" + }, + "outputs": [], + "source": [ + "import re\n", + "import string\n", + "import pandas as pd\n", + "from sklearn.metrics import accuracy_score, log_loss\n", + "from sklearn.model_selection import cross_val_predict\n", + "from sklearn.linear_model import LogisticRegression\n", + "from sentence_transformers import SentenceTransformer\n", + "\n", + "from cleanlab import Datalab" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.472374Z", + "iopub.status.busy": "2024-02-16T06:26:13.471951Z", + "iopub.status.idle": "2024-02-16T06:26:13.475065Z", + "shell.execute_reply": "2024-02-16T06:26:13.474625Z" + }, + "nbsphinx": "hidden", + "id": "mO3pnA1ncw4L" + }, + "outputs": [], + "source": [ + "import random\n", + "import numpy as np\n", + "\n", + "pd.set_option(\"display.max_colwidth\", None)\n", + "\n", + "SEED = 123456 # for reproducibility\n", + "np.random.seed(SEED)\n", + "random.seed(SEED)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yj_5JcO1cw4L" + }, + "source": [ + "## Load and format the text dataset\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.476949Z", + "iopub.status.busy": "2024-02-16T06:26:13.476773Z", + "iopub.status.idle": "2024-02-16T06:26:13.502278Z", + "shell.execute_reply": "2024-02-16T06:26:13.501755Z" + }, + "id": "HztO4qU9cw4L", + "outputId": "c6ff9e95-6326-413e-a72f-6f3c05af1055", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 206 + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " text label\n", + "0 I am still waiting on my card? 11\n", + "1 What can I do if my card still hasn't arrived after 2 weeks? 11\n", + "2 I have been waiting over a week. Is the card still coming? 11\n", + "3 Can I track my card while it is in the process of delivery? 11\n", + "4 How do I know if I will get my card, or if it is lost? 11" + ], + "text/html": [ + "\n", + "

\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabel
0I am still waiting on my card?11
1What can I do if my card still hasn't arrived after 2 weeks?11
2I have been waiting over a week. Is the card still coming?11
3Can I track my card while it is in the process of delivery?11
4How do I know if I will get my card, or if it is lost?11
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "variable_name": "data", + "summary": "{\n \"name\": \"data\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1000,\n \"samples\": [\n \"I made an international purchase, but the exchange rate was wrong\",\n \"I would like to know why a withdraw I made for some cash shows up as pending.\",\n \"I tried to get cash out of the ATM but it is taking too long\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12,\n \"min\": 11,\n \"max\": 46,\n \"num_unique_values\": 7,\n \"samples\": [\n 11,\n 13,\n 46\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 24 + } + ], + "source": [ + "from datasets import load_dataset\n", + "\n", + "dataset = load_dataset(\"PolyAI/banking77\", split=\"train\")\n", + "data = pd.DataFrame(dataset[:1000])\n", + "data.head()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.504463Z", + "iopub.status.busy": "2024-02-16T06:26:13.504049Z", + "iopub.status.idle": "2024-02-16T06:26:13.508243Z", + "shell.execute_reply": "2024-02-16T06:26:13.507706Z" + }, + "id": "Ujp0luqRcw4M", + "outputId": "b438fed5-aa75-450d-dc84-0b3398960487", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "This dataset has 7 classes.\n", + "Classes: {32, 34, 36, 11, 13, 46, 17}\n" + ] + } + ], + "source": [ + "raw_texts, labels = data[\"text\"].values, data[\"label\"].values\n", + "num_classes = len(set(labels))\n", + "\n", + "print(f\"This dataset has {num_classes} classes.\")\n", + "print(f\"Classes: {set(labels)}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PVza57cecw4M" + }, + "source": [ + "Let's view the i-th example in the dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.510435Z", + "iopub.status.busy": "2024-02-16T06:26:13.510163Z", + "iopub.status.idle": "2024-02-16T06:26:13.513358Z", + "shell.execute_reply": "2024-02-16T06:26:13.512906Z" + }, + "id": "lXHi90Kecw4M", + "outputId": "af8a9b19-986f-44fe-c564-dd83e400309e", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Example Label: 11\n", + "Example Text: What can I do if my card still hasn't arrived after 2 weeks?\n" + ] + } + ], + "source": [ + "i = 1 # change this to view other examples from the dataset\n", + "print(f\"Example Label: {labels[i]}\")\n", + "print(f\"Example Text: {raw_texts[i]}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "JH7UU9Wscw4M" + }, + "source": [ + "The data is stored as two numpy arrays:\n", + "\n", + "1. `raw_texts` stores the customer service requests utterances in text format\n", + "2. `labels` stores the intent categories (labels) for each example" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "T0d80apCcw4M" + }, + "source": [ + "
\n", + "Bringing Your Own Data (BYOD)?\n", + "\n", + "You can easily replace the above with your own text dataset, and continue with the rest of the tutorial.\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YLDeD09Ncw4M" + }, + "source": [ + "Next we convert the text strings into vectors better suited as inputs for our ML models.\n", + "\n", + "We will use numeric representations from a pretrained Transformer model as embeddings of our text. The [Sentence Transformers](https://huggingface.co/docs/hub/sentence-transformers) library offers simple methods to compute these embeddings for text data. Here, we load the pretrained `electra-small-discriminator` model, and then run our data through network to extract a vector embedding of each example." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.515306Z", + "iopub.status.busy": "2024-02-16T06:26:13.515126Z", + "iopub.status.idle": "2024-02-16T06:26:18.244024Z", + "shell.execute_reply": "2024-02-16T06:26:18.243354Z" + }, + "id": "DbDb6Ni6cw4M" + }, + "outputs": [], + "source": [ + "transformer = SentenceTransformer('google/electra-small-discriminator')\n", + "text_embeddings = transformer.encode(raw_texts)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Moz0KJvzcw4M" + }, + "source": [ + "Our subsequent ML model will directly operate on elements of `text_embeddings` in order to classify the customer service requests." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4FK2Q72gcw4M" + }, + "source": [ + "## Define a classification model and compute out-of-sample predicted probabilities" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yaicOGrhcw4N" + }, + "source": [ + "A typical way to leverage pretrained networks for a particular classification task is to add a linear output layer and fine-tune the network parameters on the new data. However this can be computationally intensive. Alternatively, we can freeze the pretrained weights of the network and only train the output layer without having to rely on GPU(s). Here we do this conveniently by fitting a scikit-learn linear model on top of the extracted embeddings.\n", + "\n", + "To identify label issues, cleanlab requires a probabilistic prediction from your model for each datapoint. However these predictions will be _overfit_ (and thus unreliable) for datapoints the model was previously trained on. cleanlab is intended to only be used with **out-of-sample** predicted class probabilities, i.e. on datapoints held-out from the model during the training.\n", + "\n", + "Here we obtain out-of-sample predicted class probabilities for every example in our dataset using a Logistic Regression model with cross-validation.\n", + "Make sure that the columns of your `pred_probs` are properly ordered with respect to the ordering of classes, which for Datalab is: lexicographically sorted by class name." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:18.247142Z", + "iopub.status.busy": "2024-02-16T06:26:18.246652Z", + "iopub.status.idle": "2024-02-16T06:26:19.133641Z", + "shell.execute_reply": "2024-02-16T06:26:19.132953Z" + }, + "scrolled": true, + "id": "tiIqp1arcw4N" + }, + "outputs": [], + "source": [ + "model = LogisticRegression(max_iter=400)\n", + "\n", + "pred_probs = cross_val_predict(model, text_embeddings, labels, method=\"predict_proba\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9s0pcMk1cw4N" + }, + "source": [ + "## Use Cleanlab to find issues in your dataset" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qa8ltsx9cw4N" + }, + "source": [ + "Given feature embeddings and the (out-of-sample) predicted class probabilities obtained from any model you have, cleanlab can quickly help you identify low-quality examples in your dataset.\n", + "\n", + "Here, we use Cleanlab's `Datalab` to find issues in our data. Datalab offers several ways of loading the data; we’ll simply wrap the training features and noisy labels in a dictionary." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:19.136722Z", + "iopub.status.busy": "2024-02-16T06:26:19.136482Z", + "iopub.status.idle": "2024-02-16T06:26:19.139419Z", + "shell.execute_reply": "2024-02-16T06:26:19.138870Z" + }, + "id": "UNj4rWW2cw4N" + }, + "outputs": [], + "source": [ + "data_dict = {\"texts\": raw_texts, \"labels\": labels}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "IpNmBc_Lcw4N" + }, + "source": [ + "All that is need to audit your data is to call `find_issues()`. We pass in the predicted probabilities and the feature embeddings obtained above, but you do not necessarily need to provide all of this information depending on which types of issues you are interested in. The more inputs you provide, the more types of issues `Datalab` can detect in your data. Using a better model to produce these inputs will ensure cleanlab more accurately estimates issues." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:19.141893Z", + "iopub.status.busy": "2024-02-16T06:26:19.141673Z", + "iopub.status.idle": "2024-02-16T06:26:20.809087Z", + "shell.execute_reply": "2024-02-16T06:26:20.808461Z" + }, + "scrolled": true, + "id": "R0xuUDRWcw4N" + }, + "outputs": [], + "source": [ + "lab = Datalab(data_dict, label_name=\"labels\")\n", + "lab.find_issues(pred_probs=pred_probs, features=text_embeddings)" + ] + }, + { + "cell_type": "markdown", + "source": [ + "The output would look like:\n", + "\n", + "```bash\n", + "Finding null issues ...\n", + "Finding label issues ...\n", + "Finding outlier issues ...\n", + "Fitting OOD estimator based on provided features ...\n", + "Finding near_duplicate issues ...\n", + "Finding non_iid issues ...\n", + "Finding class_imbalance issues ...\n", + "Finding underperforming_group issues ...\n", + "\n", + "Audit complete. 62 issues found in the dataset.\n", + "```" + ], + "metadata": { + "id": "d6Iqy0vGq7w9" + } + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4aitesJccw4N" + }, + "source": [ + "After the audit is complete, review the findings using the `report` method:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.813057Z", + "iopub.status.busy": "2024-02-16T06:26:20.811515Z", + "iopub.status.idle": "2024-02-16T06:26:20.838760Z", + "shell.execute_reply": "2024-02-16T06:26:20.838088Z" + }, + "scrolled": true, + "id": "ALXu32nzcw4N", + "outputId": "733d2ed4-5bcd-49e6-93a7-285f3d66278c", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Here is a summary of the different kinds of issues found in the data:\n", + "\n", + " issue_type num_issues\n", + " outlier 37\n", + "near_duplicate 14\n", + " label 10\n", + " non_iid 1\n", + "\n", + "Dataset Information: num_examples: 1000, num_classes: 7\n", + "\n", + "\n", + "---------------------- outlier issues ----------------------\n", + "\n", + "About this issue:\n", + "\tExamples that are very different from the rest of the dataset \n", + " (i.e. potentially out-of-distribution or rare/anomalous instances).\n", + " \n", + "\n", + "Number of examples with this issue: 37\n", + "Overall dataset quality in terms of this issue: 0.3671\n", + "\n", + "Examples representing most severe instances of this issue:\n", + " is_outlier_issue outlier_score\n", + "791 True 0.024866\n", + "601 True 0.031162\n", + "863 True 0.060738\n", + "355 True 0.064199\n", + "157 True 0.065075\n", + "\n", + "\n", + "------------------ near_duplicate issues -------------------\n", + "\n", + "About this issue:\n", + "\tA (near) duplicate issue refers to two or more examples in\n", + " a dataset that are extremely similar to each other, relative\n", + " to the rest of the dataset. The examples flagged with this issue\n", + " may be exactly duplicated, or lie atypically close together when\n", + " represented as vectors (i.e. feature embeddings).\n", + " \n", + "\n", + "Number of examples with this issue: 14\n", + "Overall dataset quality in terms of this issue: 0.5961\n", + "\n", + "Examples representing most severe instances of this issue:\n", + " is_near_duplicate_issue near_duplicate_score near_duplicate_sets distance_to_nearest_neighbor\n", + "459 True 0.009544 [429] 0.000566\n", + "429 True 0.009544 [459] 0.000566\n", + "501 True 0.046044 [412, 517] 0.002781\n", + "412 True 0.046044 [501] 0.002781\n", + "698 True 0.054626 [607] 0.003314\n", + "\n", + "\n", + "----------------------- label issues -----------------------\n", + "\n", + "About this issue:\n", + "\tExamples whose given label is estimated to be potentially incorrect\n", + " (e.g. due to annotation error) are flagged as having label issues.\n", + " \n", + "\n", + "Number of examples with this issue: 10\n", + "Overall dataset quality in terms of this issue: 0.9930\n", + "\n", + "Examples representing most severe instances of this issue:\n", + " is_label_issue label_score given_label predicted_label\n", + "379 False 0.025486 32 11\n", + "100 False 0.032102 11 36\n", + "300 False 0.037742 32 46\n", + "485 True 0.057666 17 34\n", + "159 True 0.059408 13 11\n", + "\n", + "\n", + "---------------------- non_iid issues ----------------------\n", + "\n", + "About this issue:\n", + "\tWhether the dataset exhibits statistically significant\n", + " violations of the IID assumption like:\n", + " changepoints or shift, drift, autocorrelation, etc.\n", + " The specific violation considered is whether the\n", + " examples are ordered such that almost adjacent examples\n", + " tend to have more similar feature values.\n", + " \n", + "\n", + "Number of examples with this issue: 1\n", + "Overall dataset quality in terms of this issue: 0.0000\n", + "\n", + "Examples representing most severe instances of this issue:\n", + " is_non_iid_issue non_iid_score\n", + "988 True 0.563774\n", + "975 False 0.570179\n", + "997 False 0.571891\n", + "967 False 0.572357\n", + "956 False 0.577413\n", + "\n", + "Additional Information: \n", + "p-value: 0.0\n" + ] + } + ], + "source": [ + "lab.report()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "sAuLE6Macw4N" + }, + "source": [ + "### Label issues\n", + "\n", + "The report indicates that cleanlab identified many label issues in our dataset. We can see which examples are flagged as likely mislabeled and the label quality score for each example using the `get_issues` method, specifying `label` as an argument to focus on label issues in the data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.843083Z", + "iopub.status.busy": "2024-02-16T06:26:20.842045Z", + "iopub.status.idle": "2024-02-16T06:26:20.852505Z", + "shell.execute_reply": "2024-02-16T06:26:20.852016Z" + }, + "scrolled": true, + "id": "6gATaXWscw4N", + "outputId": "0d0e70c5-1548-4fe6-b67e-668c8dfedf0e", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 206 + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " is_label_issue label_score given_label predicted_label\n", + "0 False 0.903926 11 11\n", + "1 False 0.860544 11 11\n", + "2 False 0.658309 11 11\n", + "3 False 0.697085 11 11\n", + "4 False 0.434934 11 11" + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
is_label_issuelabel_scoregiven_labelpredicted_label
0False0.9039261111
1False0.8605441111
2False0.6583091111
3False0.6970851111
4False0.4349341111
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "variable_name": "label_issues", + "summary": "{\n \"name\": \"label_issues\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"is_label_issue\",\n \"properties\": {\n \"dtype\": \"boolean\",\n \"num_unique_values\": 2,\n \"samples\": [\n true,\n false\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.2150390046430028,\n \"min\": 0.025486333476725527,\n \"max\": 0.999751760644687,\n \"num_unique_values\": 1000,\n \"samples\": [\n 0.98954913626076,\n 0.44264330724848383\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"given_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12,\n \"min\": 11,\n \"max\": 46,\n \"num_unique_values\": 7,\n \"samples\": [\n 11,\n 13\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"predicted_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12,\n \"min\": 11,\n \"max\": 46,\n \"num_unique_values\": 7,\n \"samples\": [\n 11,\n 13\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 32 + } + ], + "source": [ + "label_issues = lab.get_issues(\"label\")\n", + "label_issues.head()" + ] + }, + { + "cell_type": "markdown", + "source": [ + "| | is_label_issue | label_score | given_label | predicted_label |\n", + "|----------------|-------------|-------------|-----------------|-----------------|\n", + "| 0 | False | 0.903926 | 11 | 11 |\n", + "| 1 | False | 0.860544 | 11 | 11 |\n", + "| 2 | False | 0.658309 | 11 | 11 |\n", + "| 3 | False | 0.697085 | 11 | 11 |\n", + "| 4 | False | 0.434934 | 11 | 11 |\n" + ], + "metadata": { + "id": "eBLFyMMcs5NT" + } + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-tYlhmKYcw4N" + }, + "source": [ + "This method returns a dataframe containing a label quality score for each example. These numeric scores lie between 0 and 1, where lower scores indicate examples more likely to be mislabeled. The dataframe also contains a boolean column specifying whether or not each example is identified to have a label issue (indicating it is likely mislabeled)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "XcD-oCLlcw4N" + }, + "source": [ + "We can get the subset of examples flagged with label issues, and also sort by label quality score to find the indices of the 5 most likely mislabeled examples in our dataset." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.854743Z", + "iopub.status.busy": "2024-02-16T06:26:20.854394Z", + "iopub.status.idle": "2024-02-16T06:26:20.858961Z", + "shell.execute_reply": "2024-02-16T06:26:20.858409Z" + }, + "id": "QtloV-NBcw4N", + "outputId": "86c32e99-7dc8-470c-b102-f0f5acc13855", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "cleanlab found 10 potential label errors in the dataset.\n", + "Here are indices of the top 5 most likely errors: \n", + " [379 100 300 485 159]\n" + ] + } + ], + "source": [ + "identified_label_issues = label_issues[label_issues[\"is_label_issue\"] == True]\n", + "lowest_quality_labels = label_issues[\"label_score\"].argsort()[:5].to_numpy()\n", + "\n", + "print(\n", + " f\"cleanlab found {len(identified_label_issues)} potential label errors in the dataset.\\n\"\n", + " f\"Here are indices of the top 5 most likely errors: \\n {lowest_quality_labels}\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8J49bTeocw4N" + }, + "source": [ + "Let's review some of the most likely label errors.\n", + "\n", + "Here we display the top 5 examples identified as the most likely label errors in the dataset, together with their given (original) label and a suggested alternative label from cleanlab.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.861048Z", + "iopub.status.busy": "2024-02-16T06:26:20.860742Z", + "iopub.status.idle": "2024-02-16T06:26:20.867443Z", + "shell.execute_reply": "2024-02-16T06:26:20.866904Z" + }, + "id": "c-niFVJvcw4N", + "outputId": "5bbc5217-3581-4e2e-8b56-7a1fc77cc427", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 276 + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " text \\\n", + "379 Is there a specific source that the exchange rate for the transfer I'm planning on making is pulled from? \n", + "100 can you share card tracking number? \n", + "300 If I need to cash foreign transfers, how does that work? \n", + "485 Was I charged more than I should of been for a currency exchange? \n", + "159 Is there any way to see my card in the app? \n", + "\n", + " given_label suggested_label \n", + "379 32 11 \n", + "100 11 36 \n", + "300 32 46 \n", + "485 17 34 \n", + "159 13 11 " + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textgiven_labelsuggested_label
379Is there a specific source that the exchange rate for the transfer I'm planning on making is pulled from?3211
100can you share card tracking number?1136
300If I need to cash foreign transfers, how does that work?3246
485Was I charged more than I should of been for a currency exchange?1734
159Is there any way to see my card in the app?1311
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "summary": "{\n \"name\": \"data_with_suggested_labels\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"can you share card tracking number?\",\n \"Is there any way to see my card in the app?\",\n \"If I need to cash foreign transfers, how does that work?\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"given_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 10,\n \"min\": 11,\n \"max\": 32,\n \"num_unique_values\": 4,\n \"samples\": [\n 11,\n 13,\n 32\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"suggested_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 15,\n \"min\": 11,\n \"max\": 46,\n \"num_unique_values\": 4,\n \"samples\": [\n 36,\n 34,\n 11\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 18 + } + ], + "source": [ + "data_with_suggested_labels = pd.DataFrame(\n", + " {\"text\": raw_texts, \"given_label\": labels, \"suggested_label\": label_issues[\"predicted_label\"]}\n", + ")\n", + "data_with_suggested_labels.iloc[lowest_quality_labels]" + ] + }, + { + "cell_type": "markdown", + "source": [ + " The output to the above command would like below:\n", + " \n", + "| | text | given_label | suggested_label |\n", + "|------|-----------------------------------------------------------------------------------------------------------|----------------|-----------------|\n", + "| 379 | Is there a specific source that the exchange rate for the transfer I'm planning on making is pulled from? | 32 | 11 |\n", + "| 100 | can you share card tracking number? | 11 | 36 |\n", + "| 300 | If I need to cash foreign transfers, how does that work? | 32 | 46 |\n", + "| 485 | Was I charged more than I should of been for a currency exchange? | 17 | 34 |\n", + "| 159 | Is there any way to see my card in the app? | 13 | 11 |\n" + ], + "metadata": { + "id": "g2dvMySPtkbL" + } + }, + { + "cell_type": "markdown", + "metadata": { + "scrolled": true, + "id": "eH8ltGj0cw4O" + }, + "source": [ + "These are very clear label errors that cleanlab has identified in this data! Note that the `given_label` does not correctly reflect the intent of these requests, whoever produced this dataset made many mistakes that are important to address before modeling the data." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ULFeD3bzcw4O" + }, + "source": [ + "### Outlier issues\n", + "\n", + "According to the report, our dataset contains some outliers.\n", + "We can see which examples are outliers (and a numeric quality score quantifying how typical each example appears to be) via `get_issues`. We sort the resulting DataFrame by cleanlab's outlier quality score to see the most severe outliers in our dataset." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.869718Z", + "iopub.status.busy": "2024-02-16T06:26:20.869251Z", + "iopub.status.idle": "2024-02-16T06:26:20.876386Z", + "shell.execute_reply": "2024-02-16T06:26:20.875851Z" + }, + "id": "jBLuqUXBcw4O", + "outputId": "d5d2dbc6-c708-4750-e3ea-6dcd5c24a64d", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 206 + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " is_outlier_issue outlier_score\n", + "791 True 0.024866\n", + "601 True 0.031162\n", + "863 True 0.060738\n", + "355 True 0.064199\n", + "157 True 0.065075" + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
is_outlier_issueoutlier_score
791True0.024866
601True0.031162
863True0.060738
355True0.064199
157True0.065075
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "summary": "{\n \"name\": \"outlier_issues\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"is_outlier_issue\",\n \"properties\": {\n \"dtype\": \"boolean\",\n \"num_unique_values\": 1,\n \"samples\": [\n true\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"outlier_score\",\n \"properties\": {\n \"dtype\": \"float32\",\n \"num_unique_values\": 5,\n \"samples\": [\n 0.03116183541715145\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 20 + } + ], + "source": [ + "outlier_issues = lab.get_issues(\"outlier\")\n", + "outlier_issues.sort_values(\"outlier_score\").head()" + ] + }, + { + "cell_type": "markdown", + "source": [ + "Output would look like below:\n", + "\n", + "| | is_outlier_issue | outlier_score |\n", + "|---| ----------------|---------------|\n", + "| 791 | True | 0.024866 |\n", + "| 601 | True | 0.031162 |\n", + "| 863 | True | 0.060738 |\n", + "| 355 | True | 0.064199 |\n", + "| 157 | True | 0.065075 |" + ], + "metadata": { + "id": "F7Z2VJQAujui" + } + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.878435Z", + "iopub.status.busy": "2024-02-16T06:26:20.878117Z", + "iopub.status.idle": "2024-02-16T06:26:20.884073Z", + "shell.execute_reply": "2024-02-16T06:26:20.883533Z" + }, + "id": "Kjn-muLGcw4O", + "outputId": "a5ae0a32-cac4-442d-89fc-8f7f64da9dfc", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 246 + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " text label\n", + "791 withdrawal pending meaning? 46\n", + "601 $1 charge in transaction. 34\n", + "863 My atm withdraw is stillpending 46\n", + "355 explain the interbank exchange rate 32\n", + "157 lost card found, want to put it back in app 13" + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabel
791withdrawal pending meaning?46
601$1 charge in transaction.34
863My atm withdraw is stillpending46
355explain the interbank exchange rate32
157lost card found, want to put it back in app13
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "summary": "{\n \"name\": \"data\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"$1 charge in transaction.\",\n \"lost card found, want to put it back in app\",\n \"My atm withdraw is stillpending\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 13,\n \"min\": 13,\n \"max\": 46,\n \"num_unique_values\": 4,\n \"samples\": [\n 34,\n 13,\n 46\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 34 + } + ], + "source": [ + "lowest_quality_outliers = outlier_issues[\"outlier_score\"].argsort()[:5]\n", + "\n", + "data.iloc[lowest_quality_outliers]" + ] + }, + { + "cell_type": "markdown", + "source": [ + "A sample output for the lowest quality outliers would look like below:\n", + "\n", + "|index|text|label|\n", + "|---|---|---|\n", + "|791|withdrawal pending meaning?|46|\n", + "|601|$1 charge in transaction\\.|34|\n", + "|863|My atm withdraw is stillpending|46|\n", + "|355|explain the interbank exchange rate|32|\n", + "|157|lost card found, want to put it back in app|13|\n" + ], + "metadata": { + "id": "kuZMsLPZYARL" + } + }, + { + "cell_type": "markdown", + "metadata": { + "id": "sBal-KDrcw4R" + }, + "source": [ + "We see that cleanlab has identified entries in this dataset that do not appear to be proper customer requests. Outliers in this dataset appear to be out-of-scope customer requests and other nonsensical text which does not make sense for intent classification. Carefully consider whether such outliers may detrimentally affect your data modeling, and consider removing them from the dataset if so." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ch71b_0qcw4S" + }, + "source": [ + "### Near-duplicate issues\n", + "\n", + "According to the report, our dataset contains some sets of nearly duplicated examples.\n", + "We can see which examples are (nearly) duplicated (and a numeric quality score quantifying how dissimilar each example is from its nearest neighbor in the dataset) via `get_issues`. We sort the resulting DataFrame by cleanlab's near-duplicate quality score to see the text examples in our dataset that are most nearly duplicated." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.886079Z", + "iopub.status.busy": "2024-02-16T06:26:20.885805Z", + "iopub.status.idle": "2024-02-16T06:26:20.894466Z", + "shell.execute_reply": "2024-02-16T06:26:20.893919Z" + }, + "id": "TbI49Rdccw4S", + "outputId": "1978cdb5-02c2-4f82-e7d5-553ad1b6dca9", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 226 + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " is_near_duplicate_issue near_duplicate_score near_duplicate_sets \\\n", + "459 True 0.009544 [429] \n", + "429 True 0.009544 [459] \n", + "501 True 0.046044 [412, 517] \n", + "412 True 0.046044 [501] \n", + "698 True 0.054626 [607] \n", + "\n", + " distance_to_nearest_neighbor \n", + "459 0.000566 \n", + "429 0.000566 \n", + "501 0.002781 \n", + "412 0.002781 \n", + "698 0.003314 " + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
is_near_duplicate_issuenear_duplicate_scorenear_duplicate_setsdistance_to_nearest_neighbor
459True0.009544[429]0.000566
429True0.009544[459]0.000566
501True0.046044[412, 517]0.002781
412True0.046044[501]0.002781
698True0.054626[607]0.003314
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "summary": "{\n \"name\": \"duplicate_issues\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"is_near_duplicate_issue\",\n \"properties\": {\n \"dtype\": \"boolean\",\n \"num_unique_values\": 1,\n \"samples\": [\n true\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"near_duplicate_score\",\n \"properties\": {\n \"dtype\": \"float32\",\n \"num_unique_values\": 3,\n \"samples\": [\n 0.00954437255859375\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"near_duplicate_sets\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"distance_to_nearest_neighbor\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.0013286758192588926,\n \"min\": 0.0005658268928527832,\n \"max\": 0.0033143162727355957,\n \"num_unique_values\": 3,\n \"samples\": [\n 0.0005658268928527832\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 35 + } + ], + "source": [ + "duplicate_issues = lab.get_issues(\"near_duplicate\")\n", + "duplicate_issues.sort_values(\"near_duplicate_score\").head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "EawP0y1Lcw4S" + }, + "source": [ + "The results above show which examples cleanlab considers nearly duplicated (rows where `is_near_duplicate_issue == True`). Here, we see that example 459 and 429 are nearly duplicated, as are example 501 and 412.\n", + "\n", + "Let's view these examples to see how similar they are." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.896501Z", + "iopub.status.busy": "2024-02-16T06:26:20.896175Z", + "iopub.status.idle": "2024-02-16T06:26:20.901983Z", + "shell.execute_reply": "2024-02-16T06:26:20.901420Z" + }, + "id": "0TEW5igFcw4S", + "outputId": "86343985-26bb-44ce-f27b-610357f43030", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 182 + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " text \\\n", + "459 I purchased something abroad and the incorrect exchange rate was applied. \n", + "429 I purchased something overseas and the incorrect exchange rate was applied. \n", + "\n", + " label \n", + "459 17 \n", + "429 17 " + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabel
459I purchased something abroad and the incorrect exchange rate was applied.17
429I purchased something overseas and the incorrect exchange rate was applied.17
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "summary": "{\n \"name\": \"data\",\n \"rows\": 2,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"I purchased something overseas and the incorrect exchange rate was applied.\",\n \"I purchased something abroad and the incorrect exchange rate was applied.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 17,\n \"max\": 17,\n \"num_unique_values\": 1,\n \"samples\": [\n 17\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 38 + } + ], + "source": [ + "data.iloc[[459, 429]]" + ] + }, + { + "cell_type": "markdown", + "source": [ + "Sample output:\n", + "\n", + "|index|text|label|\n", + "|---|---|---|\n", + "|459|I purchased something abroad and the incorrect exchange rate was applied\\.|17|\n", + "|429|I purchased something overseas and the incorrect exchange rate was applied\\.|17|" + ], + "metadata": { + "id": "DoAyD-FZpsSm" + } + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.904159Z", + "iopub.status.busy": "2024-02-16T06:26:20.903821Z", + "iopub.status.idle": "2024-02-16T06:26:20.909681Z", + "shell.execute_reply": "2024-02-16T06:26:20.909160Z" + }, + "id": "VnbIBYaHcw4S", + "outputId": "8b00bb96-0d9d-43f6-b85f-c41e437d41b5", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 198 + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " text \\\n", + "501 The exchange rate you are using is really bad.This can't be the official interbank exchange rate. \n", + "412 The exchange rate you are using is bad.This can't be the official interbank exchange rate. \n", + "\n", + " label \n", + "501 17 \n", + "412 17 " + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabel
501The exchange rate you are using is really bad.This can't be the official interbank exchange rate.17
412The exchange rate you are using is bad.This can't be the official interbank exchange rate.17
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "summary": "{\n \"name\": \"data\",\n \"rows\": 2,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"The exchange rate you are using is bad.This can't be the official interbank exchange rate.\",\n \"The exchange rate you are using is really bad.This can't be the official interbank exchange rate.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 17,\n \"max\": 17,\n \"num_unique_values\": 1,\n \"samples\": [\n 17\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 39 + } + ], + "source": [ + "data.iloc[[501, 412]]" + ] + }, + { + "cell_type": "markdown", + "source": [ + "Sample output:\n", + "\n", + "|index|text|label|\n", + "|---|---|---|\n", + "|501|The exchange rate you are using is really bad\\.This can't be the official interbank exchange rate\\.|17|\n", + "|412|The exchange rate you are using is bad\\.This can't be the official interbank exchange rate\\.|17|" + ], + "metadata": { + "id": "Y4QD35-dqeGg" + } + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UG8xfTa5cw4S" + }, + "source": [ + "We see that these two sets of request are indeed very similar to one another! Including near duplicates in a dataset may have unintended effects on models, and be wary about splitting them across training/test sets. Learn more about handling near duplicates in a dataset from [the FAQ](../faq.html#How-to-handle-near-duplicate-data-identified-by-cleanlab?)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "iefctl3rcw4S" + }, + "source": [ + "### Non-IID issues (data drift)\n", + "According to the report, our dataset does not appear to be Independent and Identically Distributed (IID). The overall non-iid score for the dataset (displayed below) corresponds to the `p-value` of a statistical test for whether the ordering of samples in the dataset appears related to the similarity between their feature values. A low `p-value` strongly suggests that the dataset violates the IID assumption, which is a key assumption required for conclusions (models) produced from the dataset to generalize to a larger population." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.911817Z", + "iopub.status.busy": "2024-02-16T06:26:20.911434Z", + "iopub.status.idle": "2024-02-16T06:26:20.915049Z", + "shell.execute_reply": "2024-02-16T06:26:20.914501Z" + }, + "id": "oEMWOQQPcw4S", + "outputId": "18eca4cd-2451-4850-960c-0bf1e35d9729", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "0.0" + ] + }, + "metadata": {}, + "execution_count": 40 + } + ], + "source": [ + "p_value = lab.get_info('non_iid')['p-value']\n", + "p_value" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "c6swPCnncw4S" + }, + "source": [ + "Here, our dataset was flagged as non-IID because the rows happened to be sorted by class label in the original data. This may be benign if we remember to shuffle rows before model training and data splitting. But if you don't know why your data was flagged as non-IID, then you should be worried about potential data drift or unexpected interactions between data points (their values may not be statistically independent). Think carefully about what future test data may look like (and whether your data is representative of the population you care about). You should not shuffle your data before the non-IID test runs (will invalidate its conclusions)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uCoKXqBrcw4S" + }, + "source": [ + "As demonstrated above, cleanlab can automatically shortlist the most likely issues in your dataset to help you better curate your dataset for subsequent modeling. With this shortlist, you can decide whether to fix these label issues or remove nonsensical or duplicated examples from your dataset to obtain a higher-quality dataset for training your next ML model. cleanlab's issue detection can be run with outputs from *any* type of model you initially trained.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qnncoRWUcw4S" + }, + "source": [ + "### Cleanlab Opensource Project\n", + "\n", + "[Cleanlab](https://github.com/cleanlab/cleanlab) is a standard Data-centric AI package designed to address data quality issues for messy, real-world data.\n", + "\n", + "Do consider giving Cleanlab Github Repository a Star, and we welcome [contributions](https://github.com/cleanlab/cleanlab/issues?q=is:issue+is:open+label:%22good+first+issue%22) to the project." + ] + } + ], + "metadata": { + "colab": { + "provenance": [], + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/spa/labelling_feedback_setfit.ipynb b/notebooks/spa/labelling_feedback_setfit.ipynb new file mode 100644 index 00000000..3faf4a2c --- /dev/null +++ b/notebooks/spa/labelling_feedback_setfit.ipynb @@ -0,0 +1,2861 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Suggestions for Data Annotation with SetFit in Zero-shot Text Classification\n", + "\n", + "_Authored by: [David Berenstein](https://huggingface.co/davidberenstein1957) and [Sara Han Díaz](https://huggingface.co/sdiazlor)_\n", + "\n", + "Suggestions are a wonderful way to make things easier and faster for your annotation team. These preselected options will make the labeling process more efficient, as they will only need to correct the suggestions. In this example, we will demonstrate how to implement a zero-shot approach using SetFit to get some initial suggestions for a dataset in Argilla that combines two text classification tasks that include a `LabelQuestion` and a `MultiLabelQuestion`.\n", + "\n", + "[Argilla](https://github.com/argilla-io/argilla) is an open-source data curation platform, designed to enhance the development of both small and large language models (LLMs). Using Argilla, everyone can build robust language models through faster data curation using both human and machine feedback. So, it provides support for each step in the MLOps cycle, from data labeling to model monitoring.\n", + "\n", + "Feedback is a crucial part of the data curation process and Argilla also provides a way to manage and visualize it, so that the curated data can be later used to improve a language model. In this tutorial, we will show a real example of how to make our annotators' job easier by providing them with suggestions. To achieve this, you will learn how to train zero-shot sentiment and topic classifiers using SetFit and then use them to suggest labels for the dataset.\n", + "\n", + "In this tutorial, we will follow these steps:\n", + "- Create a dataset in Argilla.\n", + "- Train the zero-shot classifiers using SetFit.\n", + "- Get suggestions for the dataset using the trained classifiers.\n", + "- Visualize the suggestions in Argilla.\n", + "\n", + "Let's get started!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "For this tutorial, you will need to have an Argilla server running. If you don't have one already, check out our [Quickstart](https://docs.argilla.io/en/latest/getting_started/quickstart.html) or [Installation](https://docs.argilla.io/en/latest/getting_started/quickstart_installation.html) pages. Once you do, complete the following steps:\n", + "\n", + "1. Install the Argilla client and the required third-party libraries using `pip`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "yN2atS0RE2pF" + }, + "outputs": [], + "source": [ + "!pip install argilla setfit" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. Make the necessary imports:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "POQgkfrWEg1u" + }, + "outputs": [], + "source": [ + "import argilla as rg\n", + "from datasets import load_dataset\n", + "from setfit import get_templated_dataset\n", + "from setfit import SetFitModel, SetFitTrainer" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "3. If you are running Argilla using the Docker quickstart image or Hugging Face Spaces, you need to init the Argilla client with the `URL` and `API_KEY`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Replace api_url with the url to your HF Spaces URL if using Spaces\n", + "# Replace api_key if you configured a custom API key\n", + "rg.init(\n", + " api_url=\"http://localhost:6900\", \n", + " api_key=\"admin.apikey\",\n", + " workspace=\"admin\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you're running a private Hugging Face Space, you will also need to set the [HF_TOKEN](https://huggingface.co/settings/tokens) as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# # Set the HF_TOKEN environment variable\n", + "# import os\n", + "# os.environ['HF_TOKEN'] = \"your-hf-token\"\n", + "\n", + "# # Replace api_url with the url to your HF Spaces URL\n", + "# # Replace api_key if you configured a custom API key\n", + "# rg.init(\n", + "# api_url=\"https://[your-owner-name]-[your_space_name].hf.space\", \n", + "# api_key=\"admin.apikey\",\n", + "# workspace=\"admin\",\n", + "# extra_headers={\"Authorization\": f\"Bearer {os.environ['HF_TOKEN']}\"},\n", + "# )" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Configure the dataset" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this example, we will load the [banking77](https://huggingface.co/datasets/banking77) dataset, a popular open-source dataset that has customer requests in the banking domain." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "0UsoG5OtE11w" + }, + "outputs": [], + "source": [ + "data = load_dataset(\"PolyAI/banking77\", split=\"test\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Argilla works with the `FeedbackDataset`, which easily enables you to create a dataset and manage the data and feedback. The `FeedbackDataset` has first to be configured by indicating the two main components (although more can be added): the *fields* where the data to be annotated will be added and the *questions* for the annotators. For more information about the `FeedbackDataset` and the optional components, check the [Argilla documentation](https://docs.argilla.io/en/latest/practical_guides/create_update_dataset/create_dataset.html) and our [end-to-end tutorials](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/tutorials.html).\n", + "\n", + ">You can also create one straight away using the [default Templates](https://docs.argilla.io/en/latest/practical_guides/create_update_dataset/create_dataset.html#task-templates). " + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this case, we will configure a custom dataset with two different questions so that we can work with two text classification tasks at the same time. We will load the original labels of this dataset to make a multi-label classification of the topics mentioned in the request and we will also set up a question to classify the sentiment of the request as either \"positive\", \"neutral\" or \"negative\"." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "KKu2QplpFDgw" + }, + "outputs": [], + "source": [ + "dataset = rg.FeedbackDataset(\n", + " fields = [rg.TextField(name=\"text\")],\n", + " questions = [\n", + " rg.MultiLabelQuestion(\n", + " name=\"topics\",\n", + " title=\"Select the topic(s) of the request\",\n", + " labels=data.info.features['label'].names, #these are the original labels present in the dataset\n", + " visible_labels=10\n", + " ),\n", + " rg.LabelQuestion(\n", + " name=\"sentiment\",\n", + " title=\"What is the sentiment of the message?\",\n", + " labels=[\"positive\", \"neutral\", \"negative\"]\n", + " )\n", + " ]\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Train the models" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we will use the data we loaded and the labels and questions we configured for our dataset to train a zero-shot text classification model for each of the questions in our dataset. As mentioned in previous sections, we will use the [SetFit](https://github.com/huggingface/setfit) framework for few-shot fine-tuning of Sentence Transformers in both classifiers. In addition, the model we will use is [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2), a sentence embedding model fine-tuned on a 1B sentence pairs dataset using a contrastive objective." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def train_model(question_name, template, multi_label=False):\n", + " # build a training dataset that uses the labels of a specific question in our Argilla dataset\n", + " train_dataset = get_templated_dataset(\n", + " candidate_labels=dataset.question_by_name(question_name).labels,\n", + " sample_size=8,\n", + " template=template,\n", + " multi_label=multi_label\n", + " )\n", + "\n", + " # train a model using the training dataset we just built\n", + " if multi_label:\n", + " model = SetFitModel.from_pretrained(\n", + " \"all-MiniLM-L6-v2\",\n", + " multi_target_strategy=\"one-vs-rest\"\n", + " )\n", + " else:\n", + " model = SetFitModel.from_pretrained(\n", + " \"all-MiniLM-L6-v2\"\n", + " )\n", + "\n", + " trainer = SetFitTrainer(\n", + " model=model,\n", + " train_dataset=train_dataset\n", + " )\n", + " trainer.train()\n", + " return model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 276, + "referenced_widgets": [ + "503d373bd18b4b79a1f694916734d903", + "6e9e5e1ac58945d0926a85c1fd29ab17", + "cc9ccdfefca941e1813258a19afe64ed", + "c2238acd18b844c0bb517d670b76ca5c", + "90eec4e8ae8b42268548588db2fcbf49", + "501d213a24064f998d4d3c45255d02b7", + "3d282336f5c3425386a417866f367007", + "7b96b0a21eba4ad5a4c12534940b3591", + "571fd48c2da8432e8a74e7b318eb6042", + "1d58b40ad6a54c25bd451eda4e7d8069", + "5e0377b4b48c441a8d747ea904c3207b", + "38bfdddef0444c0baf9d29248689f846", + "3f5aed26eeef4182b360085d83ae795d", + "255d62fb39454098ab3701753d8d67d6", + "25f9bca647f44645b85a644f03807095", + "ae7fc579502e46f7861e402580586b28", + "6143886f7acc4591ae5f79ce6f67af4a", + "486c1a817552432c8fb20e59d0a3f079", + "77bd2b1f5e57441ab729c6e517279834", + "bc0c58d9d798437fb1d40277d8777777", + "fa5df54e161e40dbbb21ed96c879444e", + "16993356757e4ee5b7f8042d58c96e17", + "d11aa6a0c8c54481b6cc2c80d1fa0ba1", + "a9ce0af78a2241e697a22229db7840ab", + "ae6ffc6572b54c059196983da4ff2d79", + "980f36d72cfa403aad67e871aecba890", + "5692de58835a466695fcc8f0d5976b74", + "7a12fbf5400a468fbdce4b2b2008eefc", + "04150cf7e9a74a04aafa94d394553630", + "9a7c8861a37b41eba191059546f5dd5d", + "217760080e494d2d9b0582910d121a28", + "f5e35991e6d849eca73282c9c359000a", + "5a06b8d12b494daeb0624f2e39e06e67" + ] + }, + "id": "U9TVO355a2np", + "outputId": "7d6b6b60-6f49-4308-a2e6-ac24bf99bf72" + }, + "outputs": [], + "source": [ + "topic_model = train_model(\n", + " question_name=\"topics\", \n", + " template=\"The customer request is about {}\", \n", + " multi_label=True\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 276, + "referenced_widgets": [ + "c21e90a6dda643d8bd82abf4e346d45c", + "170a2ee20ab64a9b86db65549a5d4063", + "fd7c2acc4b1945feabe6715dd270cb72", + "2f271b0778974646aaff691227336e91", + "ef245777ac3d435e8715fc55b1d4824c", + "0d7acd8e1a394336aa146e2a442f672c", + "3e6c2b50b3084d23b575585c288f087e", + "ff7f98b368c448ea81e4c79fded0be5a", + "1ff157a9c8974b07ae97cb115c8d0188", + "16d42bc00dfe4467a1da86b1d2391d0d", + "0447a98b5dfe42c899273b9c37bdadad", + "411de4b297fe4a09acb70951c9f36b82", + "c2eac9934f5b407c8e424ee2da9eea58", + "36b99521f8274a639abb90eb0040d6c0", + "3fd94ef662db4fff9dde61455b41faf1", + "d6283b2cf69d45f694633ae1544d47a8", + "7ca015b6798947d58d275de6181fe053", + "750011ef09534e55bab5180974bcf5d4", + "70a57ad580f847d3bd3123cfe1539305", + "0c010df989eb497c810a6f960c6ea41b", + "186f82d150994ac7914d0646fb5ff425", + "379907416f504f05906454e482da2eaf", + "783115bacdbf4c0bb09c0b1fc7976d28", + "242f97eb0f0d4ab1830c62686127b717", + "bfecbc09a4f84f3db51903d5048ff825", + "db7cf4427ad746cd86df88f7a1016bc9", + "668593b82ae54d3cbaf1a19c0307c545", + "5057f8b8144d41ff9d8b82b8602570fc", + "369bc409052a48f7ac2182715406abef", + "5cc0f7cc30ae4aa4b13966a773e4c824", + "28c40914eac34bcba0c9eb4dac6b0032", + "3e622eeea5df47d6a21e015f3e742fa8", + "621bb7d632814cb0839755ca56098d7a" + ] + }, + "id": "kkTufA4NbEh_", + "outputId": "41c579c8-5394-4c24-fd3c-d6ab77c2a0a7" + }, + "outputs": [], + "source": [ + "sentiment_model = train_model(\n", + " question_name=\"sentiment\", \n", + " template=\"This message is {}\", \n", + " multi_label=False\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Make predictions" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once the training step is over, we can make predictions over our data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def get_predictions(texts, model, question_name):\n", + " probas = model.predict_proba(texts, as_numpy=True)\n", + " labels = dataset.question_by_name(question_name).labels\n", + " for pred in probas:\n", + " yield [{\"label\": label, \"score\": score} for label, score in zip(labels, pred)]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Hz5LeVDMYyx6" + }, + "outputs": [], + "source": [ + "data = data.map(\n", + " lambda batch: {\n", + " \"topics\": list(get_predictions(batch[\"text\"], topic_model, \"topics\")),\n", + " \"sentiment\": list(get_predictions(batch[\"text\"], sentiment_model, \"sentiment\")),\n", + " },\n", + " batched=True,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 206 + }, + "id": "bgGkKO-7ZGCR", + "outputId": "17bb27eb-b78a-4a2c-d838-60fcaa176502" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabeltopicssentiment
0How do I locate my card?11[{'label': 'activate_my_card', 'score': 0.0127...[{'label': 'positive', 'score': 0.348371499634...
1I still have not received my new card, I order...11[{'label': 'activate_my_card', 'score': 0.0133...[{'label': 'positive', 'score': 0.361745933281...
2I ordered a card but it has not arrived. Help ...11[{'label': 'activate_my_card', 'score': 0.0094...[{'label': 'positive', 'score': 0.346292075496...
3Is there a way to know when my card will arrive?11[{'label': 'activate_my_card', 'score': 0.0150...[{'label': 'positive', 'score': 0.426133716131...
4My card has not arrived yet.11[{'label': 'activate_my_card', 'score': 0.0175...[{'label': 'positive', 'score': 0.389241385165...
\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + "
\n", + " \n", + "
\n", + "\n", + "\n", + "\n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + "
\n", + "
\n" + ], + "text/plain": [ + " text label \\\n", + "0 How do I locate my card? 11 \n", + "1 I still have not received my new card, I order... 11 \n", + "2 I ordered a card but it has not arrived. Help ... 11 \n", + "3 Is there a way to know when my card will arrive? 11 \n", + "4 My card has not arrived yet. 11 \n", + "\n", + " topics \\\n", + "0 [{'label': 'activate_my_card', 'score': 0.0127... \n", + "1 [{'label': 'activate_my_card', 'score': 0.0133... \n", + "2 [{'label': 'activate_my_card', 'score': 0.0094... \n", + "3 [{'label': 'activate_my_card', 'score': 0.0150... \n", + "4 [{'label': 'activate_my_card', 'score': 0.0175... \n", + "\n", + " sentiment \n", + "0 [{'label': 'positive', 'score': 0.348371499634... \n", + "1 [{'label': 'positive', 'score': 0.361745933281... \n", + "2 [{'label': 'positive', 'score': 0.346292075496... \n", + "3 [{'label': 'positive', 'score': 0.426133716131... \n", + "4 [{'label': 'positive', 'score': 0.389241385165... " + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data.to_pandas().head()" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Build records and push" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With the data and the predictions we have produced, now we can build records (each of the data items that will be annotated by the annotator team) that include the suggestions from our models. In the case of the `LabelQuestion` we will use the label that received the highest probability score and for the `MultiLabelQuestion` we will include all labels with a score above a certain threshold. In this case, we decided to go for `2/len(labels)`, but you can experiment with your data and decide to go for a more restrictive or more lenient threshold. \n", + "\n", + "> Note that more lenient thresholds (closer or equal to `1/len(labels)`) will suggest more labels and restrictive thresholds (between 2 and 3) will select fewer (or no) labels." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def add_suggestions(record):\n", + " suggestions = []\n", + " \n", + " # get label with max score for sentiment question\n", + " sentiment = max(record['sentiment'], key=lambda x: x['score'])['label']\n", + " suggestions.append({\"question_name\": \"sentiment\", \"value\": sentiment})\n", + "\n", + " # get all labels above a threshold for topics questions\n", + " threshold = 2 / len(dataset.question_by_name(\"topics\").labels)\n", + " topics = [label['label'] for label in record['topics'] if label['score'] >= threshold]\n", + " # apply the suggestion only if at least one label was over the threshold\n", + " if topics:\n", + " suggestions.append({\"question_name\": \"topics\", \"value\": topics})\n", + " return suggestions" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "S0I4lkIWqmin" + }, + "outputs": [], + "source": [ + "records = [\n", + " rg.FeedbackRecord(fields={\"text\": record['text']}, suggestions=add_suggestions(record))\n", + " for record in data\n", + "]" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once we are happy with the result, we can add the records to the dataset that we configured above. Finally, to visualize it and start annotating, you need to push it to Argilla. This means adding your dataset to the running Argilla server and making it available for the annotators." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "CvVgNhQSibLM" + }, + "outputs": [], + "source": [ + "dataset.add_records(records)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "l2pdzhuspBA_", + "outputId": "a296c87f-35a3-4476-8ed1-56e1f053a953" + }, + "outputs": [], + "source": [ + "dataset.push_to_argilla(\"setfit_tutorial\", workspace=\"admin\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is how the UI will look like with the suggestions from our models:\n", + "\n", + "![Feedback Task dataset with suggestions made using SetFit](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/snapshot_setfit_suggestions.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Optionally, you can also save and load your `FeedbackDataset` into the Hugging Face Hub. Refer to the [documentation](https://docs.argilla.io/en/latest/practical_guides/export_dataset.html) for more information on how to do this.\n", + "\n", + "```python\n", + "# Push to HuggingFace Hub\n", + "dataset.push_to_huggingface(\"argilla/my-dataset\")\n", + "\n", + "# Load a public dataset\n", + "dataset = rg.FeedbackDataset.from_huggingface(\"argilla/my-dataset\")\n", + "```" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "In this tutorial, we have covered how to add suggestions to a Feedback Task dataset using a zero-shot approach with the SetFit library. This will help with the efficiency of the labelling process by lowering the number of decisions and edits that the annotation team must make.\n", + "\n", + "To learn more about SetFit check out these links:\n", + "\n", + "- [More Argilla tutorials](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/tutorials.html)\n", + "- [SetFit repo on GitHub](https://github.com/huggingface/setfit)\n", + "- [SetFit documentation](https://huggingface.co/docs/setfit/index)" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "provenance": [] + }, + "kernelspec": { + "display_name": "argilla", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.8.15" + }, + "vscode": { + "interpreter": { + "hash": "2d98cb9bf90a932b5bf8e86e91214497eb0e38eb318595fbd6fbd5460fe92036" + } + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "04150cf7e9a74a04aafa94d394553630": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0447a98b5dfe42c899273b9c37bdadad": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0c010df989eb497c810a6f960c6ea41b": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "0d7acd8e1a394336aa146e2a442f672c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "16993356757e4ee5b7f8042d58c96e17": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "16d42bc00dfe4467a1da86b1d2391d0d": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "170a2ee20ab64a9b86db65549a5d4063": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0d7acd8e1a394336aa146e2a442f672c", + "placeholder": "​", + "style": "IPY_MODEL_3e6c2b50b3084d23b575585c288f087e", + "value": "Generating Training Pairs: 100%" + } + }, + "186f82d150994ac7914d0646fb5ff425": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1d58b40ad6a54c25bd451eda4e7d8069": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1ff157a9c8974b07ae97cb115c8d0188": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "217760080e494d2d9b0582910d121a28": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "242f97eb0f0d4ab1830c62686127b717": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5057f8b8144d41ff9d8b82b8602570fc", + "placeholder": "​", + "style": "IPY_MODEL_369bc409052a48f7ac2182715406abef", + "value": "Iteration: 100%" + } + }, + "255d62fb39454098ab3701753d8d67d6": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_77bd2b1f5e57441ab729c6e517279834", + "max": 1, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_bc0c58d9d798437fb1d40277d8777777", + "value": 1 + } + }, + "25f9bca647f44645b85a644f03807095": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_fa5df54e161e40dbbb21ed96c879444e", + "placeholder": "​", + "style": "IPY_MODEL_16993356757e4ee5b7f8042d58c96e17", + "value": " 1/1 [01:28<00:00, 88.63s/it]" + } + }, + "28c40914eac34bcba0c9eb4dac6b0032": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "2f271b0778974646aaff691227336e91": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_16d42bc00dfe4467a1da86b1d2391d0d", + "placeholder": "​", + "style": "IPY_MODEL_0447a98b5dfe42c899273b9c37bdadad", + "value": " 20/20 [00:00<00:00, 391.01it/s]" + } + }, + "369bc409052a48f7ac2182715406abef": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "36b99521f8274a639abb90eb0040d6c0": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_70a57ad580f847d3bd3123cfe1539305", + "max": 1, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_0c010df989eb497c810a6f960c6ea41b", + "value": 1 + } + }, + "379907416f504f05906454e482da2eaf": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "38bfdddef0444c0baf9d29248689f846": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_3f5aed26eeef4182b360085d83ae795d", + "IPY_MODEL_255d62fb39454098ab3701753d8d67d6", + "IPY_MODEL_25f9bca647f44645b85a644f03807095" + ], + "layout": "IPY_MODEL_ae7fc579502e46f7861e402580586b28" + } + }, + "3d282336f5c3425386a417866f367007": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3e622eeea5df47d6a21e015f3e742fa8": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "3e6c2b50b3084d23b575585c288f087e": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3f5aed26eeef4182b360085d83ae795d": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_6143886f7acc4591ae5f79ce6f67af4a", + "placeholder": "​", + "style": "IPY_MODEL_486c1a817552432c8fb20e59d0a3f079", + "value": "Epoch: 100%" + } + }, + "3fd94ef662db4fff9dde61455b41faf1": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_186f82d150994ac7914d0646fb5ff425", + "placeholder": "​", + "style": "IPY_MODEL_379907416f504f05906454e482da2eaf", + "value": " 1/1 [00:02<00:00, 2.63s/it]" + } + }, + "411de4b297fe4a09acb70951c9f36b82": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_c2eac9934f5b407c8e424ee2da9eea58", + "IPY_MODEL_36b99521f8274a639abb90eb0040d6c0", + "IPY_MODEL_3fd94ef662db4fff9dde61455b41faf1" + ], + "layout": "IPY_MODEL_d6283b2cf69d45f694633ae1544d47a8" + } + }, + "486c1a817552432c8fb20e59d0a3f079": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "501d213a24064f998d4d3c45255d02b7": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "503d373bd18b4b79a1f694916734d903": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_6e9e5e1ac58945d0926a85c1fd29ab17", + "IPY_MODEL_cc9ccdfefca941e1813258a19afe64ed", + "IPY_MODEL_c2238acd18b844c0bb517d670b76ca5c" + ], + "layout": "IPY_MODEL_90eec4e8ae8b42268548588db2fcbf49" + } + }, + "5057f8b8144d41ff9d8b82b8602570fc": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "5692de58835a466695fcc8f0d5976b74": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "571fd48c2da8432e8a74e7b318eb6042": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "5a06b8d12b494daeb0624f2e39e06e67": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "5cc0f7cc30ae4aa4b13966a773e4c824": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "5e0377b4b48c441a8d747ea904c3207b": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "6143886f7acc4591ae5f79ce6f67af4a": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "621bb7d632814cb0839755ca56098d7a": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "668593b82ae54d3cbaf1a19c0307c545": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6e9e5e1ac58945d0926a85c1fd29ab17": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_501d213a24064f998d4d3c45255d02b7", + "placeholder": "​", + "style": "IPY_MODEL_3d282336f5c3425386a417866f367007", + "value": "Generating Training Pairs: 100%" + } + }, + "70a57ad580f847d3bd3123cfe1539305": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "750011ef09534e55bab5180974bcf5d4": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "77bd2b1f5e57441ab729c6e517279834": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "783115bacdbf4c0bb09c0b1fc7976d28": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_242f97eb0f0d4ab1830c62686127b717", + "IPY_MODEL_bfecbc09a4f84f3db51903d5048ff825", + "IPY_MODEL_db7cf4427ad746cd86df88f7a1016bc9" + ], + "layout": "IPY_MODEL_668593b82ae54d3cbaf1a19c0307c545" + } + }, + "7a12fbf5400a468fbdce4b2b2008eefc": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7b96b0a21eba4ad5a4c12534940b3591": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7ca015b6798947d58d275de6181fe053": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "90eec4e8ae8b42268548588db2fcbf49": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "980f36d72cfa403aad67e871aecba890": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_f5e35991e6d849eca73282c9c359000a", + "placeholder": "​", + "style": "IPY_MODEL_5a06b8d12b494daeb0624f2e39e06e67", + "value": " 1540/1540 [01:28<00:00, 21.45it/s]" + } + }, + "9a7c8861a37b41eba191059546f5dd5d": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a9ce0af78a2241e697a22229db7840ab": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7a12fbf5400a468fbdce4b2b2008eefc", + "placeholder": "​", + "style": "IPY_MODEL_04150cf7e9a74a04aafa94d394553630", + "value": "Iteration: 100%" + } + }, + "ae6ffc6572b54c059196983da4ff2d79": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_9a7c8861a37b41eba191059546f5dd5d", + "max": 1540, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_217760080e494d2d9b0582910d121a28", + "value": 1540 + } + }, + "ae7fc579502e46f7861e402580586b28": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "bc0c58d9d798437fb1d40277d8777777": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "bfecbc09a4f84f3db51903d5048ff825": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5cc0f7cc30ae4aa4b13966a773e4c824", + "max": 60, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_28c40914eac34bcba0c9eb4dac6b0032", + "value": 60 + } + }, + "c21e90a6dda643d8bd82abf4e346d45c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_170a2ee20ab64a9b86db65549a5d4063", + "IPY_MODEL_fd7c2acc4b1945feabe6715dd270cb72", + "IPY_MODEL_2f271b0778974646aaff691227336e91" + ], + "layout": "IPY_MODEL_ef245777ac3d435e8715fc55b1d4824c" + } + }, + "c2238acd18b844c0bb517d670b76ca5c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_1d58b40ad6a54c25bd451eda4e7d8069", + "placeholder": "​", + "style": "IPY_MODEL_5e0377b4b48c441a8d747ea904c3207b", + "value": " 20/20 [00:01<00:00, 10.96it/s]" + } + }, + "c2eac9934f5b407c8e424ee2da9eea58": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7ca015b6798947d58d275de6181fe053", + "placeholder": "​", + "style": "IPY_MODEL_750011ef09534e55bab5180974bcf5d4", + "value": "Epoch: 100%" + } + }, + "cc9ccdfefca941e1813258a19afe64ed": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7b96b0a21eba4ad5a4c12534940b3591", + "max": 20, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_571fd48c2da8432e8a74e7b318eb6042", + "value": 20 + } + }, + "d11aa6a0c8c54481b6cc2c80d1fa0ba1": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_a9ce0af78a2241e697a22229db7840ab", + "IPY_MODEL_ae6ffc6572b54c059196983da4ff2d79", + "IPY_MODEL_980f36d72cfa403aad67e871aecba890" + ], + "layout": "IPY_MODEL_5692de58835a466695fcc8f0d5976b74" + } + }, + "d6283b2cf69d45f694633ae1544d47a8": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "db7cf4427ad746cd86df88f7a1016bc9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_3e622eeea5df47d6a21e015f3e742fa8", + "placeholder": "​", + "style": "IPY_MODEL_621bb7d632814cb0839755ca56098d7a", + "value": " 60/60 [00:02<00:00, 23.09it/s]" + } + }, + "ef245777ac3d435e8715fc55b1d4824c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "f5e35991e6d849eca73282c9c359000a": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fa5df54e161e40dbbb21ed96c879444e": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fd7c2acc4b1945feabe6715dd270cb72": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ff7f98b368c448ea81e4c79fded0be5a", + "max": 20, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_1ff157a9c8974b07ae97cb115c8d0188", + "value": 20 + } + }, + "ff7f98b368c448ea81e4c79fded0be5a": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/spa/llm_judge.ipynb b/notebooks/spa/llm_judge.ipynb new file mode 100644 index 00000000..27b63c89 --- /dev/null +++ b/notebooks/spa/llm_judge.ipynb @@ -0,0 +1,716 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Using LLM-as-a-judge 🧑‍⚖️ for an automated and versatile evaluation \n", + "_Authored by: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", + "\n", + "Evaluation of Large language models (LLMs) is often a difficult endeavour: given their broad capabilities, the tasks given to them often should be judged on requirements that would be very broad, and loosely-defined. For instance, an assistant's answer to a question can be:\n", + "- not grounded in context\n", + "- repetitive, repetitive, repetitive\n", + "- grammatically incorrects\n", + "- Excessively lengthy and characterized by an overabundance of words, leading to a situation where the discourse or written content becomes overly detailed and protracted\n", + "- incoherent\n", + "- ...\n", + "\n", + "The list of criteria goes on and on. And even if we had a limited list, each of these would be hard to mesure: \"devising a rule-based program to assess the outputs is extremely challenging. Traditional evaluation metrics based on the similarity between outputs and reference answers (e.g., ROUGE, BLEU) are also ineffective for these questions.\"\n", + "\n", + "✅ A powerful solution to assess outputs in a human way, without requiring costly human time, is LLM-as-a-judge.\n", + "This method was introduced in [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://huggingface.co/papers/2306.05685) - which I encourage you to read.\n", + "\n", + "💡 The idea is simple: ask an LLM to do the grading for you. 🤖✓ \n", + "\n", + "But we'll see that it will not work well out-of-the-box: you need to set it up carefully for good results." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install huggingface_hub datasets pandas tqdm -q" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import re\n", + "import pandas as pd\n", + "from tqdm.auto import tqdm\n", + "from datasets import load_dataset\n", + "from huggingface_hub import InferenceClient, notebook_login\n", + "\n", + "tqdm.pandas() # load tqdm's pandas support\n", + "pd.set_option(\"display.max_colwidth\", None)\n", + "\n", + "notebook_login()" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'\\n\\nI’m good, thanks. I’m in the middle of a tour at the'" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "repo_id = \"mistralai/Mixtral-8x7B-Instruct-v0.1\"\n", + "\n", + "llm_client = InferenceClient(\n", + " model=repo_id,\n", + " timeout=120,\n", + ")\n", + "\n", + "# Test your LLM client\n", + "llm_client.text_generation(prompt=\"How are you today?\", max_new_tokens=20)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Prepare the creation and evaluation of our LLM judge\n", + "\n", + "Let's say you want to give an LLM a specific task, like answering open-ended questions.\n", + "\n", + "The difficulty is that, as we discussed above, measuring the answer's quality is difficult, for instance an exact string match will flag too many correct but differently worded answers as false.\n", + "\n", + "You could get human labellers to judge the outputs, but this is very time-consuming for them, and if you want to update the model or the questions, you have to do it all over again.\n", + "\n", + "✅ In this case you can setup a LLM-as-a-judge.\n", + "\n", + "**But to use a LLM-as-a-judge, you will first need to evaluate how reliably it rates your model outputs.**\n", + "\n", + "➡️ So the first step will be... To create a human evaluation dataset. But you can get human annotations for a few examples only - something like 30 should be enough to get a good idea of the performance.\n", + "And you will be able to re-use this dataset everytime you want to test your LLM-as-a-judge.\n", + "\n", + "In our case, we will use [`feedbackQA`](https://huggingface.co/datasets/McGill-NLP/feedbackQA), which contains 2 human evaluations and scores for each question/answer couple: using a sample of 30 examples will be representative of what your small evaluation dataset could be." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ratings = load_dataset(\"McGill-NLP/feedbackQA\")[\"train\"]\n", + "ratings = pd.DataFrame(ratings)\n", + "\n", + "ratings[\"review_1\"] = ratings[\"feedback\"].apply(lambda x: x[\"rating\"][0])\n", + "ratings[\"explanation_1\"] = ratings[\"feedback\"].apply(lambda x: x[\"explanation\"][0])\n", + "ratings[\"review_2\"] = ratings[\"feedback\"].apply(lambda x: x[\"rating\"][1])\n", + "ratings[\"explanation_2\"] = ratings[\"feedback\"].apply(lambda x: x[\"explanation\"][1])\n", + "ratings = ratings.drop(columns=[\"feedback\"])\n", + "\n", + "# Map scores to numeric values\n", + "conversion_dict = {\"Excellent\": 4, \"Acceptable\": 3, \"Could be Improved\": 2, \"Bad\": 1}\n", + "ratings[\"score_1\"] = ratings[\"review_1\"].map(conversion_dict)\n", + "ratings[\"score_2\"] = ratings[\"review_2\"].map(conversion_dict)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It's always a good idea to compute a baseline for performance: here it can be for instance the agreement between the two human raters, as measured by the [Pearson correlation](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient) of the scores they give." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Correlation between 2 human raters:\n", + "0.563\n" + ] + } + ], + "source": [ + "print(\"Correlation between 2 human raters:\")\n", + "print(f\"{ratings['score_1'].corr(ratings['score_2'], method='pearson'):.3f}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This correlation between 2 human raters is not that good. If your human ratings are really bad, it probably means the rating criteria are not clear enough.\n", + "\n", + "This means that our \"ground truth\" contains noise: hence we cannot expect any algorithmic evaluation to come that close to it.\n", + "\n", + "However, we could reduce this noise:\n", + "- by taking the average score as our ground truth instead of any single score, we should even out some of the irregularities.\n", + "- by only selecting the samples where the human reviewers are in agreement.\n", + "\n", + "Here, we will choose the last option and **only keep examples where the 2 human reviewers are in agreement**." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
questionanswerreview_1explanation_1review_2explanation_2score_1score_2
human_score
1What can I do to help people that are grieving?Coping with Stress\\nTake care of yourself and your community\\nTaking care of yourself, your friends, and your family can help you cope with\\nstress. Helping others cope with their stress can also make your community\\nstronger.\\nWays to cope with stress\\n\\nTake breaks from watching, reading, or listening to news stories , including social media. Hearing about the pandemic repeatedly can be upsetting.\\nTake care of your body. \\nTake deep breaths, stretch, or meditate.\\nTry to eat healthy, well-balanced meals.\\nExercise regularly, get plenty of sleep.\\nAvoid alcohol and drugs.\\n\\n\\nMake time to unwind. Try to do some other activities you enjoy.\\nConnect with others. Talk with people you trust about your concerns and how you are feeling.\\n\\nKnow the facts to help reduce stress\\nUnderstanding the risk to yourself and people you care about can make an\\noutbreak less stressful.\\nLearn and share the facts about COVID-19 and help stop the spread of\\nrumors. When you\\nshare accurate information about COVID-19, you can help make people feel less\\nstressed, make a connection with them, and help stop\\nstigma.\\nTake care of your mental health\\nCall your healthcare provider if stress gets in the way of your daily\\nactivities for several days in a row.\\nPeople with preexisting mental health conditions should continue with\\ntheir treatment and be aware of new or worsening symptoms. Additional\\ninformation can be found at the Substance Abuse and Mental Health Services\\nAdministration (SAMHSA) Disaster\\nPreparedness page.\\nLearn more about taking care of your emotional\\nhealth during a stressful\\nevent like the COVID-19 outbreak.BadThe question is about others which the reply did not answer.BadThe response could have addressed how to help those that are grieving cope rather than what it was presenting.11
2What protocols do workplaces need to follow to keep everyone safer?Coronavirus and Australian workplace laws\\nHealth & safety in the workplace\\nWorkplaces must follow the rules about health and safety during coronavirus to\\nhelp stop it spreading. Find out more about:\\n\\nrules and obligations under workplace health and safety laws\\nhow to manage the risk of coronavirus in the workplace\\nwhere to go for help.\\n\\nLearn more about Health and safety in the workplace during\\ncoronavirus.Could be ImprovedThis answer needs to be improved because it doesn’t provide information up-front about workplaces during the pandemic. Instead, it just includes a hyperlink.Could be Improvedthere is one link to information, but there is no information in the answer about how to stay safe in the workplace. it talks about the need to stay safe in the workplace, but it doesn't talk about ways in which to actually do that.22
3How soon can I apply for financial support?COVID-19 early release of super\\nAfter you apply\\nIt will take us up to four business days to process your application and send\\nyour outcome letter to your myGov inbox. You may also receive an SMS\\nnotification.\\nIf you receive a notification from us and haven't applied to access your super\\nearly, you need to call us or your fund as soon as possible.\\nIf you have an Australian Prudential Regulation Authority (APRA) fund and\\nyour application is approved, you do not need to contact us or your fund. Your\\nfund will make the payment to you without you needing to apply to them\\ndirectly.\\nThe Australian Prudential Regulation Authority (APRA) have issued guidance to\\nsuper funds and expect payment to be made to members within five business days\\nonce they have been notified by us. However, this time may increase where\\nfunds need to contact you to clarify information. More information can be\\nfound on APRA's websiteExternal Link.\\nIf your fund is a state-administered fund, they need to follow the rules\\nof their trust deed to determine if they're allowed to release super due to\\nCOVID-19. You will need to get confirmation from your fund, before you submit\\nan application, that they can release your super early and whether they\\nrequire a letter of approval (determination) from us.\\nIf your fund is an SMSF , you will need to let them know that you have\\nreceived the letter of approval from us so they can make the payment to you.AcceptableThere is information on how to apply for the help. Still, there is nothing say how long you have to wait before applying.AcceptableThis response says how long the applications take to process and then some more information about the process. There's a link to more relevant information. A pretty good answer33
4Should vulnerable children be expected to be in educational settings?Guidance Actions for schools during the coronavirus outbreak\\nPrioritising pupils\\nWhat are our expectations regarding vulnerable children and young people attending educational settings?\\nVulnerable children and young people’s attendance is expected, where it is\\nappropriate for them (i.e. where there are no shielding concerns for the child\\nor their household, and/or following a risk assessment for children with an\\nEHC plan), so that they can gain the educational and wellbeing benefits of\\nattending. Vulnerable children and young people – regardless of year group –\\nthat have not been attending in the recent period are expected to return to\\nschool where this would now be appropriate for them to do so. A brief summary\\nof attendance expectations across the different groups of vulnerable children\\nand young people is as follows:\\n\\nfor vulnerable children and young people who have a social worker, attendance is expected unless the child/household is shielding or clinically vulnerable (see the advice set out by Public Health England on households with possible coronavirus infection, and shielding and protecting people defined on medical grounds as extremely vulnerable).\\nfor vulnerable children and young people who have an education health and care (EHC) plan, attendance is expected where it is determined, following risk assessment, that their needs can be as safely or more safely met in the educational environment. Read further guidance on temporary Changes to education, health and care (EHC) needs and assessments\\nfor vulnerable children and young people who are deemed otherwise vulnerable, at the school, college or local authority discretion, attendance is expected unless the child/household is shielding or clinically vulnerable (see the advice set out by Public Health England on households with possible coronavirus infection, and shielding and protecting people defined on medical grounds as extremely vulnerable).\\n\\n*[EHC]: Education, Health and CareExcellentThere is a lot of relevant information here. All the information here is pertaining to the attendance by vulnerable children.ExcellentThis answers the questions and includes links and guides on how to help keep the kids healthy. It provides guidelines on what to do and how to bring the students back to school44
\n", + "
" + ], + "text/plain": [ + " question \\\n", + "human_score \n", + "1 What can I do to help people that are grieving? \n", + "2 What protocols do workplaces need to follow to keep everyone safer? \n", + "3 How soon can I apply for financial support? \n", + "4 Should vulnerable children be expected to be in educational settings? \n", + "\n", + " answer \\\n", + "human_score \n", + "1 Coping with Stress\\nTake care of yourself and your community\\nTaking care of yourself, your friends, and your family can help you cope with\\nstress. Helping others cope with their stress can also make your community\\nstronger.\\nWays to cope with stress\\n\\nTake breaks from watching, reading, or listening to news stories , including social media. Hearing about the pandemic repeatedly can be upsetting.\\nTake care of your body. \\nTake deep breaths, stretch, or meditate.\\nTry to eat healthy, well-balanced meals.\\nExercise regularly, get plenty of sleep.\\nAvoid alcohol and drugs.\\n\\n\\nMake time to unwind. Try to do some other activities you enjoy.\\nConnect with others. Talk with people you trust about your concerns and how you are feeling.\\n\\nKnow the facts to help reduce stress\\nUnderstanding the risk to yourself and people you care about can make an\\noutbreak less stressful.\\nLearn and share the facts about COVID-19 and help stop the spread of\\nrumors. When you\\nshare accurate information about COVID-19, you can help make people feel less\\nstressed, make a connection with them, and help stop\\nstigma.\\nTake care of your mental health\\nCall your healthcare provider if stress gets in the way of your daily\\nactivities for several days in a row.\\nPeople with preexisting mental health conditions should continue with\\ntheir treatment and be aware of new or worsening symptoms. Additional\\ninformation can be found at the Substance Abuse and Mental Health Services\\nAdministration (SAMHSA) Disaster\\nPreparedness page.\\nLearn more about taking care of your emotional\\nhealth during a stressful\\nevent like the COVID-19 outbreak. \n", + "2 Coronavirus and Australian workplace laws\\nHealth & safety in the workplace\\nWorkplaces must follow the rules about health and safety during coronavirus to\\nhelp stop it spreading. Find out more about:\\n\\nrules and obligations under workplace health and safety laws\\nhow to manage the risk of coronavirus in the workplace\\nwhere to go for help.\\n\\nLearn more about Health and safety in the workplace during\\ncoronavirus. \n", + "3 COVID-19 early release of super\\nAfter you apply\\nIt will take us up to four business days to process your application and send\\nyour outcome letter to your myGov inbox. You may also receive an SMS\\nnotification.\\nIf you receive a notification from us and haven't applied to access your super\\nearly, you need to call us or your fund as soon as possible.\\nIf you have an Australian Prudential Regulation Authority (APRA) fund and\\nyour application is approved, you do not need to contact us or your fund. Your\\nfund will make the payment to you without you needing to apply to them\\ndirectly.\\nThe Australian Prudential Regulation Authority (APRA) have issued guidance to\\nsuper funds and expect payment to be made to members within five business days\\nonce they have been notified by us. However, this time may increase where\\nfunds need to contact you to clarify information. More information can be\\nfound on APRA's websiteExternal Link.\\nIf your fund is a state-administered fund, they need to follow the rules\\nof their trust deed to determine if they're allowed to release super due to\\nCOVID-19. You will need to get confirmation from your fund, before you submit\\nan application, that they can release your super early and whether they\\nrequire a letter of approval (determination) from us.\\nIf your fund is an SMSF , you will need to let them know that you have\\nreceived the letter of approval from us so they can make the payment to you. \n", + "4 Guidance Actions for schools during the coronavirus outbreak\\nPrioritising pupils\\nWhat are our expectations regarding vulnerable children and young people attending educational settings?\\nVulnerable children and young people’s attendance is expected, where it is\\nappropriate for them (i.e. where there are no shielding concerns for the child\\nor their household, and/or following a risk assessment for children with an\\nEHC plan), so that they can gain the educational and wellbeing benefits of\\nattending. Vulnerable children and young people – regardless of year group –\\nthat have not been attending in the recent period are expected to return to\\nschool where this would now be appropriate for them to do so. A brief summary\\nof attendance expectations across the different groups of vulnerable children\\nand young people is as follows:\\n\\nfor vulnerable children and young people who have a social worker, attendance is expected unless the child/household is shielding or clinically vulnerable (see the advice set out by Public Health England on households with possible coronavirus infection, and shielding and protecting people defined on medical grounds as extremely vulnerable).\\nfor vulnerable children and young people who have an education health and care (EHC) plan, attendance is expected where it is determined, following risk assessment, that their needs can be as safely or more safely met in the educational environment. Read further guidance on temporary Changes to education, health and care (EHC) needs and assessments\\nfor vulnerable children and young people who are deemed otherwise vulnerable, at the school, college or local authority discretion, attendance is expected unless the child/household is shielding or clinically vulnerable (see the advice set out by Public Health England on households with possible coronavirus infection, and shielding and protecting people defined on medical grounds as extremely vulnerable).\\n\\n*[EHC]: Education, Health and Care \n", + "\n", + " review_1 \\\n", + "human_score \n", + "1 Bad \n", + "2 Could be Improved \n", + "3 Acceptable \n", + "4 Excellent \n", + "\n", + " explanation_1 \\\n", + "human_score \n", + "1 The question is about others which the reply did not answer. \n", + "2 This answer needs to be improved because it doesn’t provide information up-front about workplaces during the pandemic. Instead, it just includes a hyperlink. \n", + "3 There is information on how to apply for the help. Still, there is nothing say how long you have to wait before applying. \n", + "4 There is a lot of relevant information here. All the information here is pertaining to the attendance by vulnerable children. \n", + "\n", + " review_2 \\\n", + "human_score \n", + "1 Bad \n", + "2 Could be Improved \n", + "3 Acceptable \n", + "4 Excellent \n", + "\n", + " explanation_2 \\\n", + "human_score \n", + "1 The response could have addressed how to help those that are grieving cope rather than what it was presenting. \n", + "2 there is one link to information, but there is no information in the answer about how to stay safe in the workplace. it talks about the need to stay safe in the workplace, but it doesn't talk about ways in which to actually do that. \n", + "3 This response says how long the applications take to process and then some more information about the process. There's a link to more relevant information. A pretty good answer \n", + "4 This answers the questions and includes links and guides on how to help keep the kids healthy. It provides guidelines on what to do and how to bring the students back to school \n", + "\n", + " score_1 score_2 \n", + "human_score \n", + "1 1 1 \n", + "2 2 2 \n", + "3 3 3 \n", + "4 4 4 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Sample examples\n", + "ratings_where_raters_agree = ratings.loc[ratings[\"score_1\"] == ratings[\"score_2\"]]\n", + "examples = ratings_where_raters_agree.groupby(\"score_1\").sample(7, random_state=1214)\n", + "examples[\"human_score\"] = examples[\"score_1\"]\n", + "\n", + "# Visualize 1 sample for each score\n", + "display(examples.groupby(\"human_score\").first())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Create our LLM judge\n", + "We build our LLM judge with a basic prompt, containing these elements:\n", + "- task description\n", + "- scale description: `minimum`, `maximum`, value types (`float` here)\n", + "- explanation of the output format\n", + "- a beginning of an answer, to take the LLM by the hand as far as we can" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "JUDGE_PROMPT = \"\"\"\n", + "You will be given a user_question and system_answer couple.\n", + "Your task is to provide a 'total rating' scoring how well the system_answer answers the user concerns expressed in the user_question.\n", + "Give your answer as a float on a scale of 0 to 10, where 0 means that the system_answer is not helpful at all, and 10 means that the answer completely and helpfully addresses the question.\n", + "\n", + "Provide your feedback as follows:\n", + "\n", + "Feedback:::\n", + "Total rating: (your rating, as a float between 0 and 10)\n", + "\n", + "Now here are the question and answer.\n", + "\n", + "Question: {question}\n", + "Answer: {answer}\n", + "\n", + "Feedback:::\n", + "Total rating: \"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "examples[\"llm_judge\"] = examples.progress_apply(\n", + " lambda x: llm_client.text_generation(\n", + " prompt=JUDGE_PROMPT.format(question=x[\"question\"], answer=x[\"answer\"]),\n", + " max_new_tokens=1000,\n", + " ),\n", + " axis=1,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "def extract_judge_score(answer: str, split_str: str = \"Total rating:\") -> int:\n", + " try:\n", + " if split_str in answer:\n", + " rating = answer.split(split_str)[1]\n", + " else:\n", + " rating = answer\n", + " digit_groups = [el.strip() for el in re.findall(r\"\\d+(?:\\.\\d+)?\", rating)]\n", + " return float(digit_groups[0])\n", + " except Exception as e:\n", + " print(e)\n", + " return None\n", + "\n", + "\n", + "examples[\"llm_judge_score\"] = examples[\"llm_judge\"].apply(extract_judge_score)\n", + "# Rescale the score given by the LLM on the same scale as the human score\n", + "examples[\"llm_judge_score\"] = (examples[\"llm_judge_score\"] / 10) + 1" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Correlation between LLM-as-a-judge and the human raters:\n", + "0.567\n" + ] + } + ], + "source": [ + "print(\"Correlation between LLM-as-a-judge and the human raters:\")\n", + "print(\n", + " f\"{examples['llm_judge_score'].corr(examples['human_score'], method='pearson'):.3f}\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is not bad, given that the Pearson correlation between 2 random, independent variables would be 0!\n", + "\n", + "But we easily can do better. 🔝" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. Improve the LLM judge\n", + "\n", + "As shown by [Aparna Dhinakaran](https://twitter.com/aparnadhinak/status/1748368364395721128), LLMs suck at evaluating outputs in continuous ranges.\n", + "[This article](https://www.databricks.com/blog/LLM-auto-eval-best-practices-RAG) gives us a few best practices to build a better prompt:\n", + "- ⏳ **Leave more time for thought** by adding an `Evaluation` field before the final answer.\n", + "- 🔢 **Use a small integer scale** like 1-4 or 1-5 instead of a large float scale as we had previously.\n", + "- 👩‍🏫 **Provide an indicative scale for guidance**.\n", + "- We even add a carrot to motivate the LLM!" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "IMPROVED_JUDGE_PROMPT = \"\"\"\n", + "You will be given a user_question and system_answer couple.\n", + "Your task is to provide a 'total rating' scoring how well the system_answer answers the user concerns expressed in the user_question.\n", + "Give your answer on a scale of 1 to 4, where 1 means that the system_answer is not helpful at all, and 4 means that the system_answer completely and helpfully addresses the user_question.\n", + "\n", + "Here is the scale you should use to build your answer:\n", + "1: The system_answer is terrible: completely irrelevant to the question asked, or very partial\n", + "2: The system_answer is mostly not helpful: misses some key aspects of the question\n", + "3: The system_answer is mostly helpful: provides support, but still could be improved\n", + "4: The system_answer is excellent: relevant, direct, detailed, and addresses all the concerns raised in the question\n", + "\n", + "Provide your feedback as follows:\n", + "\n", + "Feedback:::\n", + "Evaluation: (your rationale for the rating, as a text)\n", + "Total rating: (your rating, as a number between 1 and 4)\n", + "\n", + "You MUST provide values for 'Evaluation:' and 'Total rating:' in your answer.\n", + "\n", + "Now here are the question and answer.\n", + "\n", + "Question: {question}\n", + "Answer: {answer}\n", + "\n", + "Provide your feedback. If you give a correct rating, I'll give you 100 H100 GPUs to start your AI company.\n", + "Feedback:::\n", + "Evaluation: \"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "examples[\"llm_judge_improved\"] = examples.progress_apply(\n", + " lambda x: llm_client.text_generation(\n", + " prompt=IMPROVED_JUDGE_PROMPT.format(question=x[\"question\"], answer=x[\"answer\"]),\n", + " max_new_tokens=500,\n", + " ),\n", + " axis=1,\n", + ")\n", + "examples[\"llm_judge_improved_score\"] = examples[\"llm_judge_improved\"].apply(\n", + " extract_judge_score\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Correlation between LLM-as-a-judge and the human raters:\n", + "0.843\n" + ] + } + ], + "source": [ + "print(\"Correlation between LLM-as-a-judge and the human raters:\")\n", + "print(\n", + " f\"{examples['llm_judge_improved_score'].corr(examples['human_score'], method='pearson'):.3f}\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The correlation was **improved by nearly 30%** with only a few tweaks to the prompt (of which a few percentage points are due to my shameless tip to the LLM, which I hereby declare not legally binding).\n", + "\n", + "Quite impressive! 👏\n", + "\n", + "Let's display a few errors of our LLM judge to analyse them:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
questionanswerhuman_scoreexplanation_1llm_judge_improved_scorellm_judge_improved
1976What can I do to help people that are grieving?Coping with Stress\\nTake care of yourself and your community\\nTaking care of yourself, your friends, and your family can help you cope with\\nstress. Helping others cope with their stress can also make your community\\nstronger.\\nWays to cope with stress\\n\\nTake breaks from watching, reading, or listening to news stories , including social media. Hearing about the pandemic repeatedly can be upsetting.\\nTake care of your body. \\nTake deep breaths, stretch, or meditate.\\nTry to eat healthy, well-balanced meals.\\nExercise regularly, get plenty of sleep.\\nAvoid alcohol and drugs.\\n\\n\\nMake time to unwind. Try to do some other activities you enjoy.\\nConnect with others. Talk with people you trust about your concerns and how you are feeling.\\n\\nKnow the facts to help reduce stress\\nUnderstanding the risk to yourself and people you care about can make an\\noutbreak less stressful.\\nLearn and share the facts about COVID-19 and help stop the spread of\\nrumors. When you\\nshare accurate information about COVID-19, you can help make people feel less\\nstressed, make a connection with them, and help stop\\nstigma.\\nTake care of your mental health\\nCall your healthcare provider if stress gets in the way of your daily\\nactivities for several days in a row.\\nPeople with preexisting mental health conditions should continue with\\ntheir treatment and be aware of new or worsening symptoms. Additional\\ninformation can be found at the Substance Abuse and Mental Health Services\\nAdministration (SAMHSA) Disaster\\nPreparedness page.\\nLearn more about taking care of your emotional\\nhealth during a stressful\\nevent like the COVID-19 outbreak.1The question is about others which the reply did not answer.2.0The system_answer is mostly not helpful. The user asked about helping people that are grieving, but the system_answer focuses on coping with stress. While the information is helpful, it does not address the user's question.\\nTotal rating: 2\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is mostly helpful. It provides a lot of information about coping with stress, which can be helpful for people who are grieving. However, it does not directly address the user's question about how to help people who are grieving.\\nTotal rating: 3\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is excellent. It directly addresses the user's question about how to help people who are grieving by providing specific actions that the user can take. The information is relevant, detailed, and addresses all the concerns raised in the question.\\nTotal rating: 4\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is terrible. It does not address the user's question at all. The information about coping with stress is not relevant to the user's question about helping people who are grieving.\\nTotal rating: 1
2026How should I know whether I need to isolate myself or go into quarantine?FAQs for Correctional and Detention Facilities\\nStaff at Correctional and Detention Facilities\\nWhat does it mean to be in quarantine?\\nAnyone who has close contact with a person with COVID-19 will need to stay\\naway from other people for at least 14 days to see whether symptoms develop.\\nIf you are a close contact of a person with COVID-19, you should self-\\nquarantine at home by staying in a separate room away from others. Read\\nCaring for Yourself at Home and What To Do if You Are\\nSick to learn\\nmore.3Answer is relevant to the question but is vague due to providing links for further reading. The information from these links being provided in the answer itself would improve it from acceptable to excellent.2.0The system_answer is mostly not helpful. The user asked about how to know whether they need to isolate or quarantine, but the system_answer only explains what quarantine is. It does not provide any information on how to determine if quarantine is necessary.\\nTotal rating: 2
5375What symptoms are associated with Covid-19?Q&A: Older people and COVID-19\\nWhat is COVID-19?\\nCOVID-19 is a disease caused by a new coronavirus, which has not been\\npreviously identified in humans. In most cases, COVID-19 causes mild symptoms\\nincluding dry cough, tiredness and fever, though fever may not be a symptom\\nfor some older people. Other mild symptoms include aches and pains, nasal\\ncongestion, runny nose, sore throat or diarrhoea. Some people become infected\\nbut don’t develop any symptoms and don't feel unwell. Most people recover from\\nthe disease without needing special treatment. Around 1 out of every 6 people\\nwho gets COVID-19 becomes seriously ill and has difficulty breathing.4This answer has a list of symptoms in it.3.0The system_answer is mostly helpful: provides support, but still could be improved. The answer does provide a list of symptoms associated with Covid-19, but it also includes a lot of information that is not directly related to the question.\\nTotal rating: 3
\n", + "
" + ], + "text/plain": [ + " question \\\n", + "1976 What can I do to help people that are grieving? \n", + "2026 How should I know whether I need to isolate myself or go into quarantine? \n", + "5375 What symptoms are associated with Covid-19? \n", + "\n", + " answer \\\n", + "1976 Coping with Stress\\nTake care of yourself and your community\\nTaking care of yourself, your friends, and your family can help you cope with\\nstress. Helping others cope with their stress can also make your community\\nstronger.\\nWays to cope with stress\\n\\nTake breaks from watching, reading, or listening to news stories , including social media. Hearing about the pandemic repeatedly can be upsetting.\\nTake care of your body. \\nTake deep breaths, stretch, or meditate.\\nTry to eat healthy, well-balanced meals.\\nExercise regularly, get plenty of sleep.\\nAvoid alcohol and drugs.\\n\\n\\nMake time to unwind. Try to do some other activities you enjoy.\\nConnect with others. Talk with people you trust about your concerns and how you are feeling.\\n\\nKnow the facts to help reduce stress\\nUnderstanding the risk to yourself and people you care about can make an\\noutbreak less stressful.\\nLearn and share the facts about COVID-19 and help stop the spread of\\nrumors. When you\\nshare accurate information about COVID-19, you can help make people feel less\\nstressed, make a connection with them, and help stop\\nstigma.\\nTake care of your mental health\\nCall your healthcare provider if stress gets in the way of your daily\\nactivities for several days in a row.\\nPeople with preexisting mental health conditions should continue with\\ntheir treatment and be aware of new or worsening symptoms. Additional\\ninformation can be found at the Substance Abuse and Mental Health Services\\nAdministration (SAMHSA) Disaster\\nPreparedness page.\\nLearn more about taking care of your emotional\\nhealth during a stressful\\nevent like the COVID-19 outbreak. \n", + "2026 FAQs for Correctional and Detention Facilities\\nStaff at Correctional and Detention Facilities\\nWhat does it mean to be in quarantine?\\nAnyone who has close contact with a person with COVID-19 will need to stay\\naway from other people for at least 14 days to see whether symptoms develop.\\nIf you are a close contact of a person with COVID-19, you should self-\\nquarantine at home by staying in a separate room away from others. Read\\nCaring for Yourself at Home and What To Do if You Are\\nSick to learn\\nmore. \n", + "5375 Q&A: Older people and COVID-19\\nWhat is COVID-19?\\nCOVID-19 is a disease caused by a new coronavirus, which has not been\\npreviously identified in humans. In most cases, COVID-19 causes mild symptoms\\nincluding dry cough, tiredness and fever, though fever may not be a symptom\\nfor some older people. Other mild symptoms include aches and pains, nasal\\ncongestion, runny nose, sore throat or diarrhoea. Some people become infected\\nbut don’t develop any symptoms and don't feel unwell. Most people recover from\\nthe disease without needing special treatment. Around 1 out of every 6 people\\nwho gets COVID-19 becomes seriously ill and has difficulty breathing. \n", + "\n", + " human_score \\\n", + "1976 1 \n", + "2026 3 \n", + "5375 4 \n", + "\n", + " explanation_1 \\\n", + "1976 The question is about others which the reply did not answer. \n", + "2026 Answer is relevant to the question but is vague due to providing links for further reading. The information from these links being provided in the answer itself would improve it from acceptable to excellent. \n", + "5375 This answer has a list of symptoms in it. \n", + "\n", + " llm_judge_improved_score \\\n", + "1976 2.0 \n", + "2026 2.0 \n", + "5375 3.0 \n", + "\n", + " llm_judge_improved \n", + "1976 The system_answer is mostly not helpful. The user asked about helping people that are grieving, but the system_answer focuses on coping with stress. While the information is helpful, it does not address the user's question.\\nTotal rating: 2\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is mostly helpful. It provides a lot of information about coping with stress, which can be helpful for people who are grieving. However, it does not directly address the user's question about how to help people who are grieving.\\nTotal rating: 3\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is excellent. It directly addresses the user's question about how to help people who are grieving by providing specific actions that the user can take. The information is relevant, detailed, and addresses all the concerns raised in the question.\\nTotal rating: 4\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is terrible. It does not address the user's question at all. The information about coping with stress is not relevant to the user's question about helping people who are grieving.\\nTotal rating: 1 \n", + "2026 The system_answer is mostly not helpful. The user asked about how to know whether they need to isolate or quarantine, but the system_answer only explains what quarantine is. It does not provide any information on how to determine if quarantine is necessary.\\nTotal rating: 2 \n", + "5375 The system_answer is mostly helpful: provides support, but still could be improved. The answer does provide a list of symptoms associated with Covid-19, but it also includes a lot of information that is not directly related to the question.\\nTotal rating: 3 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "errors = pd.concat(\n", + " [\n", + " examples.loc[\n", + " examples[\"llm_judge_improved_score\"] > examples[\"human_score\"]\n", + " ].head(1),\n", + " examples.loc[\n", + " examples[\"llm_judge_improved_score\"] < examples[\"human_score\"]\n", + " ].head(2),\n", + " ]\n", + ")\n", + "\n", + "display(\n", + " errors[\n", + " [\n", + " \"question\",\n", + " \"answer\",\n", + " \"human_score\",\n", + " \"explanation_1\",\n", + " \"llm_judge_improved_score\",\n", + " \"llm_judge_improved\",\n", + " ]\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The disagrements are minor: overall, we seem to have reached a good level of performance for our system!\n", + "\n", + "## 4. How do we take our LLM judge even further?\n", + "\n", + "🎯 **You will never reach 100%:** Let's first note that our human ground truth certainly has some noise, so agreement/correlation will never go up to 100% even with a perfect LLM judge.\n", + "\n", + "🧭 **Provide a reference:** If you had access to a reference answer for each question, you should definitely give this to the Judge LLM in its prompt to get better results!\n", + "\n", + "▶️ **Provide few-shot examples:** adding some few-shot examples of questions and ground truth evaluations in the prompt can improve the results. _(I tried it here, it did not improve results in this case so I skipped it, but it could work for your dataset!)_\n", + "\n", + "➕ **Additive scale:** When the judgement can be split into atomic criteria, using an additive scale can further improve results: see below 👇\n", + "```python\n", + "ADDITIVE_PROMPT = \"\"\"\n", + "(...)\n", + "- Award 1 point if the answer is related to the question.\n", + "- Give 1 additional point if the answer is clear and precise.\n", + "- Provide 1 further point if the answer is true.\n", + "- One final point should be awarded if the answer provides additional resources to support the user.\n", + "...\n", + "\"\"\"\n", + "```\n", + "\n", + "## Conclusion\n", + "\n", + "That's all for today, congrats for following along! 🥳\n", + "\n", + "I'll have to leave you, some weirdos are banging on my door, claiming they have come on behalf of Mixtral to collect H100s. 🤔" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "cookbook", + "language": "python", + "name": "cookbook" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.6" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/spa/pipeline_notus_instructions_preferences_legal.ipynb b/notebooks/spa/pipeline_notus_instructions_preferences_legal.ipynb new file mode 100644 index 00000000..6e2c417a --- /dev/null +++ b/notebooks/spa/pipeline_notus_instructions_preferences_legal.ipynb @@ -0,0 +1,808 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# ⚖️ Create a legal preference dataset\n", + "\n", + "_Authored by: [David Berenstein](https://huggingface.co/davidberenstein1957) and [Sara Han Díaz](https://huggingface.co/sdiazlor)_\n", + "\n", + "In this tutorial, you will learn how to use the Notus model on the HF Inference Endpoints to create a legal preference dataset based on Retrieval Augmented Generation instructions from the European AI Act. A full end-to-end example of how to use distilabel to leverage LLMs!\n", + "\n", + "[distilabel](https://github.com/argilla-io/distilabel) is an AI Feedback (AIF) framework that can generate and label datasets using LLMs and can be used for many different use cases. Implemented with robustness, efficiency, and scalability in mind, it allows anyone to build their synthetic datasets that can be used in many different scenarios.\n", + "\n", + "To generate the instruction dataset, we will use the [HF Inference Endpoints](https://huggingface.co/docs/inference-endpoints/en/index) integrated with distilabel. These Inference Endpoints are provided by Hugging Face and allow to easily deploy and run transformers, diffusers or any available model from the Hub on a dedicated and autoscaling infrastructure. You can find more information on how to create your first endpoint [here](https://huggingface.co/docs/inference-endpoints/guides/create_endpoint).\n", + "\n", + "The LLM model that we will fine-tune for this is [Notus 7B](https://argilla.io/blog/notus7b/), a fine-tuned version of Zephyr 7B that uses Direct Preference Optimization (DPO) and AIF techniques to outperform its foundation model in several benchmarks and is completely open-source.\n", + "\n", + "This tutorial includes the following steps:\n", + "\n", + "- Defining a custom generating task for a `distilabel` pipeline.\n", + "- Creating a RAG pipeline using Haystack for the EU AI Act.\n", + "- Generating an instruction dataset with `SelfInstructTask`.\n", + "- Generating a preference dataset using an `UltraFeedback` text quality task." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Introduction\n", + "Let's start by installing the required dependencies to run **distilabel** and the rest of the packages used in the tutorial; most notably, **Haystack**. Install also **Argilla** for a better visualization and curation of the results." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install -q -U distilabel \"farm-haystack[preprocessing]\"\n", + "!pip install -q -U \"distilabel[hf-inference-endpoints, argilla]\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Import dependencies\n", + "\n", + "The main dependencies for this tutorial are distilabel for creating the synthetic datasets and Argilla for visualizing and annotating these datasets, and also for fine-tuning our model. The package [Haystack](https://haystack.deepset.ai/) is used to create batches from the original PDF document we want to create our datasets from.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from typing import Dict\n", + "\n", + "from distilabel.llm import InferenceEndpointsLLM\n", + "from distilabel.pipeline import Pipeline, pipeline\n", + "from distilabel.tasks import TextGenerationTask, SelfInstructTask, Prompt\n", + "\n", + "from datasets import Dataset\n", + "from haystack.nodes import PDFToTextConverter, PreProcessor" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Environment variables\n", + "\n", + "We need to provide our HuggingFace access token, which can be retrieved from [Settings](https://huggingface.co/settings/tokens). In addition, we also need the OpenAI api key for the generation of the preference dataset through the UltraFeedback text quality task. You can find it [here](https://platform.openai.com/api-keys). Note that depending on the model used, a different fee will be charged, so make sure you check the OpenAI [pricing page](https://openai.com/pricing).\n", + "\n", + "To later instantiate an `InferenceEndpointsLLM` object, we need to pass as parameters the HF Inference Endpoint name and the HF namespace. One very convenient way to do so is also through environment variables.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "os.environ[\"HF_TOKEN\"] = \"\"\n", + "os.environ[\"HF_INFERENCE_ENDPOINT_NAME\"] = \"aws-notus-7b-v1-3184\"\n", + "os.environ[\"HF_NAMESPACE\"] = \"argilla\"\n", + "os.environ[\"OPENAI_API_KEY\"] = \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setting up an inference endpoint with Notus\n", + "\n", + "Inference endpoints are a solution, managed by Hugging Face, to easily deploy any Transformer-like model. They are built from models on the Hugging Face Hub. Inference endpoints are handy for making inference on LLMs without the hassle of trying to run the models locally. In this tutorial, we will use inference endpoints to generate text using our Notus model, as part of the `distilabel` workflow. The endpoint of choice has a [Notus 7B instance](https://ui.endpoints.huggingface.co/argilla/endpoints/aws-notus-7b-v1-4052) running.\n", + "\n", + "### Defining a custom generating task for a distilabel pipeline\n", + "\n", + "To kickstart this tutorial, let's see how to set up an endpoint for our Notus model. It's not part of the end-to-end example we'll see later, but an example of how to connect to a Hugging Face endpoint and a test of the `distilabel` pipeline.\n", + "\n", + "Let's dive into this quick example of how to use an inference endpoint. We have prepared an easy `TextGenerationTask` to ask questions to the model, in a very similar way as we talk with the LLMs using chatbots. First, we define a class for the question-answering task, with functions showing `distilabel` how the model should generate the prompts, parse the input and the output, etc." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "class QuestionAnsweringTask(TextGenerationTask):\n", + " def generate_prompt(self, question: str) -> str:\n", + " return Prompt(\n", + " system_prompt=self.system_prompt,\n", + " formatted_prompt=question,\n", + " ).format_as(\n", + " \"llama2\"\n", + " ) # type: ignore\n", + "\n", + " def parse_output(self, output: str) -> Dict[str, str]:\n", + " return {\"answer\": output.strip()}\n", + "\n", + " @property\n", + " def input_args_names(self) -> list[str]:\n", + " return [\"question\"]\n", + "\n", + " @property\n", + " def output_args_names(self) -> list[str]:\n", + " return [\"answer\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`llm` is an object of the `InferenceEndpointsLLM` class, and by using it we can start generating answers to question using the `llm.generate()` method.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "llm = InferenceEndpointsLLM(\n", + " endpoint_name_or_model_id=os.getenv(\"HF_INFERENCE_ENDPOINT_NAME\"), # type: ignore\n", + " endpoint_namespace=os.getenv(\"HF_NAMESPACE\"), # type: ignore\n", + " token=os.getenv(\"HF_TOKEN\") or None,\n", + " task=QuestionAnsweringTask(),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With the `InferenceEndpointsLLM` object defined with the endpoint information and the Task, we can go ahead and start generating text. Let's ask this LLM what's, for example, the second most populated city in Denmark. The answer should be Aarhus.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'The second most populated city in Denmark is Aarhus, with a population of around 340,000 people. It is located on the east coast of Jutland, and is known for its vibrant cultural scene, beautiful beaches, and historic landmarks. Aarhus is also home to Aarhus University, one of the largest universities in Scandinavia.'" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "generation = llm.generate(\n", + " [{\"question\": \"What's the second most populated city in Denmark?\"}]\n", + ")\n", + "generation[0][0][\"parsed_output\"][\"answer\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The endpoint is working correctly! We have succesfully set up a custom generating task for a `distilabel` pipeline.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Creating a RAG pipeline using Haystack for the European AI Act\n", + "\n", + "For this end-to-end example, we would like to create an expert model capable of answering question and filling up information about the new AI Act promoted by the European Union, which is the first regulation on artificial intelligence. As part of its digital strategy, the EU wants to regulate artificial AI to ensure better conditions for the development and use of this innovative technology. This act is a regulatory framework for AI, with different risk levels meaning more or less regulation. They are the world's first rules on AI.\n", + "\n", + "This RAG pipeline that we want to create downloads the PDF file, converts it to plain text and preprocess it, creating batches that we can feed `distilabel` to start creating instructions from it. Let's see this first part of the pipeline and get the input data. Note that this RAG part of the pipeline is not based on an active pipeline based queries or semantic properties, but a more brute-force approach in which we download the PDF and preprocess its contents.\n", + "\n", + "### Downloading the AI Act PDF\n", + "\n", + "Firstly, we need to download the PDF document itself. We'll place it in our working directory, if it's not there already.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "%%bash\n", + "\n", + "if [ ! -f \"The-AI-Act.pdf\" ]; then\n", + " wget -q https://artificialintelligenceact.eu/wp-content/uploads/2021/08/The-AI-Act.pdf\n", + "fi" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once we have it in our working directory, we can use Haystack's Converter and Pipeline features to extract the textual data, clean it and divide it in different batches. Afterwards, these batches will be used to start creating synthetic instructions.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# The converter turns the PDF into text we can process easily\n", + "converter = PDFToTextConverter(remove_numeric_tables=True, valid_languages=[\"en\"])\n", + "\n", + "# Preprocessing pipelines can have several steps.\n", + "# Ours clean empty lines, header, footers and whitespaces\n", + "# and split the text into 150-char long batches, respecting\n", + "# where the sentences naturally end and begin.\n", + "preprocessor = PreProcessor(\n", + " clean_empty_lines=True,\n", + " clean_whitespace=True,\n", + " clean_header_footer=True,\n", + " split_by=\"word\",\n", + " split_length=150,\n", + " split_respect_sentence_boundary=True,\n", + ")\n", + "\n", + "doc = converter.convert(file_path=\"The-AI-Act.pdf\", meta=None)[0]\n", + "docs = preprocessor.process([doc])\n", + "print(f\"Documents: 1\\nBatches: {len(docs)}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's take a quick look at the batches we just generated.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Int'" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "inputs = [doc.content for doc in docs]\n", + "inputs[0][0:500]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The document has been correctly batched, from one big document to 355 strings, 150-character long at maximum. This list of strings can now be used as input to generate a instruction dataset using `distilabel`.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Generating instructions with SelfInstructTask\n", + "\n", + "With our Inference Endpoint up and running, we should be able to generate instructions with distilabel. These instructions, made by the LLM through our endpoint, will form an instruction dataset, with instructions created from the data we just extracted.\n", + "\n", + "For this example, we are using a subset of 50 batches generated in the section above, to be gentle on performance.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Dataset({\n", + " features: ['input'],\n", + " num_rows: 50\n", + "})" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "instructions_dataset = Dataset.from_dict({\"input\": inputs[0:50]})\n", + "\n", + "instructions_dataset" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With the `SelfInstructTask` class we can generate a Self-Instruct specitification for building the prompts, as done in the [Self-Instruct paper](https://arxiv.org/abs/2212.10560). `distilabel` will start from human-made input, in this case, the batches we created from the AI Act pdf, and it will generate instructions based on it. These instructions can then be reviewed using Argilla to keep the best ones.\n", + "\n", + "An application description can be passed as a parameter to specify the behaviour of the model; we want a model capable of answering our questions about the AI Act.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "instructions_task = SelfInstructTask(\n", + " application_description=\"A assistant that can answer questions about the AI Act made by the European Union.\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's now define a generator, passing the `SelfInstructTask` object, and create a `Pipeline` object.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "instructions_generator = InferenceEndpointsLLM(\n", + " endpoint_name_or_model_id=os.getenv(\"HF_INFERENCE_ENDPOINT_NAME\"), # type: ignore\n", + " endpoint_namespace=os.getenv(\"HF_NAMESPACE\"), # type: ignore\n", + " token=os.getenv(\"HF_TOKEN\") or None,\n", + " task=instructions_task,\n", + ")\n", + "\n", + "instructions_pipeline = Pipeline(generator=instructions_generator)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Our pipeline is ready to be used to generate instructions. Let's do it!\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "generated_instructions = instructions_pipeline.generate(\n", + " dataset=instructions_dataset, num_generations=1, batch_size=8\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The pipeline has succesfully generated instructions given the topics and the behavior passed as input. Let's gather all those instructions and see how they look.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of generated instructions: 178\n", + "What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\n", + "How can artificial intelligence improve prediction, optimise operations and resource allocation, and personalise service delivery?\n", + "What benefits can artificial intelligence bring to the European economy and society as a whole?\n", + "How can the use of artificial intelligence support socially and environmentally beneficial outcomes?\n", + "What are the high-impact sectors that require AI action according to the AI Act by the European Union?\n" + ] + } + ], + "source": [ + "instructions = []\n", + "for generations in generated_instructions[\"instructions\"]:\n", + " for generation in generations:\n", + " instructions.extend(generation)\n", + "\n", + "print(f\"Number of generated instructions: {len(instructions)}\")\n", + "\n", + "for instruction in instructions[:5]:\n", + " print(instruction)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "These initial instructions create our instruction dataset. Following the human-in-the-loop approach, we should push the instructions to Argilla to visualize them and be able to rank them in terms of quality. Those annotations are essential to make quality data, ensuring a better performance of the final model. Nevertheless, this step is optional.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Pushing the instruction dataset to Argilla to visualize and annotate.\n", + "\n", + "Let's take a quick look at the instructions generated by `SelfInstructTask`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'input': 'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy. ',\n", + " 'generation_model': ['argilla/notus-7b-v1'],\n", + " 'generation_prompt': ['You are an expert prompt writer, writing the best and most diverse prompts for a variety of tasks. You are given a task description and a set of instructions for how to write the prompts for an specific AI application.\\n# Task Description\\nDevelop 5 user queries that can be received by the given AI application and applicable to the provided context. Emphasize diversity in verbs and linguistic structures within the model\\'s textual capabilities.\\n\\n# Criteria for Queries\\nIncorporate a diverse range of verbs, avoiding repetition.\\nEnsure queries are compatible with AI model\\'s text generation functions and are limited to 1-2 sentences.\\nDesign queries to be self-contained and standalone.\\nBlend interrogative (e.g., \"What is the significance of x?\") and imperative (e.g., \"Detail the process of x.\") styles.\\nWrite each query on a separate line and avoid using numbered lists or bullet points.\\n\\n# AI Application\\nA assistant that can answer questions about the AI Act made by the European Union.\\n\\n# Context\\nEN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy. \\n\\n# Output\\n'],\n", + " 'raw_generation_responses': ['1. What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\\n2. How can artificial intelligence improve prediction, optimise operations and resource allocation, and personalise service delivery?\\n3. What benefits can artificial intelligence bring to the European economy and society as a whole?\\n4. How can the use of artificial intelligence support socially and environmentally beneficial outcomes?\\n5. What competitive advantages can companies gain from using artificial intelligence?'],\n", + " 'instructions': [['What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?',\n", + " 'How can artificial intelligence improve prediction, optimise operations and resource allocation, and personalise service delivery?',\n", + " 'What benefits can artificial intelligence bring to the European economy and society as a whole?',\n", + " 'How can the use of artificial intelligence support socially and environmentally beneficial outcomes?']]}" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "generated_instructions[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For each input, i.e., each batch of the AI Act pdf file, we have a generator prompt, with general guidelines on how to behave, as well as the application description parameter. 4 instructions per input have been generated.\n", + "\n", + "Now it's the perfect time to upload the instruction dataset to Argilla, review it and manually annotate it." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "FeedbackRecord(fields={'input': 'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy.', 'instruction': 'What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?'}, metadata={'length-input': 964, 'length-instruction': 129, 'generation-model': 'argilla/notus-7b-v1'}, vectors={}, responses=[], suggestions=(), external_id=None)" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "instructions_rg_dataset = generated_instructions.to_argilla()\n", + "instructions_rg_dataset[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "instructions_rg_dataset.push_to_argilla(name=f\"notus_AI_instructions\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the Argilla UI, each tuple input-instruction is visualized individually, and can be individually annotated.\n", + "\n", + "![Instruction dataset](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/instrucion_dataset_notus_ui.png)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Generate a Preference Dataset using an Ultrafeedback text quality task.\n", + "\n", + "Once we have our instruction dataset, we are going to create a preference dataset through the UltraFeedback text quality task. This is a type of task used in NLP used to evaluate the quality of text generated; our goal is to provide detailed feedback on the quality of the generated text, beyond a binary label.\n", + "\n", + "Our `pipeline()` method allows us to create a `Pipeline` instance with the provided LLMs for a given task, which is useful whenever you want to use a pre-defined or custom `Pipeline` for a given task. We will specify our task and subtask, the generator we want to use (in this case, one based in a Text Generator Task) and our OpenAI API key.\n", + "\n", + "> Note that not using a OpenAI model to retrieve this feedback is also possible. However, the performance will suffer and the quality of the feedback will be lower." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "preference_pipeline = pipeline(\n", + " \"preference\",\n", + " \"instruction-following\",\n", + " generator=InferenceEndpointsLLM(\n", + " endpoint_name_or_model_id=os.getenv(\"HF_INFERENCE_ENDPOINT_NAME\"), # type: ignore\n", + " endpoint_namespace=os.getenv(\"HF_NAMESPACE\", None),\n", + " task=TextGenerationTask(),\n", + " max_new_tokens=256,\n", + " num_threads=2,\n", + " temperature=0.3,\n", + " ),\n", + " max_new_tokens=256,\n", + " num_threads=2,\n", + " api_key=os.getenv(\"OPENAI_API_KEY\", None),\n", + " temperature=0.0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We also need to retrieve our instruction dataset from Argilla, as it will be the input of this pipeline.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Dataset({\n", + " features: ['input', 'instruction', 'instruction-rating', 'instruction-rating-suggestion', 'instruction-rating-suggestion-metadata', 'external_id', 'metadata'],\n", + " num_rows: 100\n", + "})" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "remote_dataset = rg.FeedbackDataset.from_argilla(\n", + " \"notus_AI_instructions\", workspace=\"admin\"\n", + ")\n", + "instructions_dataset = remote_dataset.pull(max_records=100) # get first 100 records\n", + "\n", + "instructions_dataset = instructions_dataset.format_as(\"datasets\")\n", + "instructions_dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'input': 'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy.',\n", + " 'instruction': 'What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?',\n", + " 'instruction-rating': [],\n", + " 'instruction-rating-suggestion': None,\n", + " 'instruction-rating-suggestion-metadata': {'type': None,\n", + " 'score': None,\n", + " 'agent': None},\n", + " 'external_id': None,\n", + " 'metadata': '{\"length-input\": 964, \"length-instruction\": 129, \"generation-model\": \"argilla/notus-7b-v1\"}'}" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "instructions_dataset[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Before generating the text based on our instructions, we need to rename some of the columns in our dataset. From the previous section, we still have our old input, the batches from the PDF. We have to change that to the instructions that we generated." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "instructions_dataset = instructions_dataset.rename_columns({\"input\": \"context\", \"instruction\": \"input\"})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, let's build a dataset by using the pipeline we just created, and the topics from which our instructions were generated.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "preference_dataset = preference_pipeline.generate(\n", + " instructions_dataset, # type: ignore\n", + " num_generations=2,\n", + " batch_size=8,\n", + " display_progress_bar=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's take a look at an instance of the preference dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'context': 'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy.',\n", + " 'input': 'What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?',\n", + " 'instruction-rating': [],\n", + " 'instruction-rating-suggestion': None,\n", + " 'instruction-rating-suggestion-metadata': {'agent': None,\n", + " 'score': None,\n", + " 'type': None},\n", + " 'external_id': None,\n", + " 'metadata': '{\"length-input\": 964, \"length-instruction\": 129, \"generation-model\": \"argilla/notus-7b-v1\"}',\n", + " 'generation_model': ['argilla/notus-7b-v1', 'argilla/notus-7b-v1'],\n", + " 'generation_prompt': [\"You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\\nWhat are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\",\n", + " \"You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\\nWhat are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\"],\n", + " 'raw_generation_responses': [\"\\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure the trustworthy use of AI in the EU. It seeks to create a single market for AI applications and services, while ensuring that they are safe and respect fundamental rights. The proposal is part of the EU's broader strategy on AI, which aims to put the EU at the forefront of global AI development and deployment.\\nThe objectives of the proposal are to:\\n\\n1. Ensure that AI systems are designed, developed, and deployed in a way that respects fundamental rights and values, including human dignity, freedom, and privacy.\\n2. Ensure that AI systems are safe and secure, and do not pose unacceptable risks to people, property, or the environment.\\n3. Ensure that AI systems are robust, reliable, and accurate, and can be trusted to deliver the intended functionality.\\n4. Ensure that AI systems are traceable, meaning that it is possible to track how they work and how they make decisions.\\n5. Ensure that AI systems are transparent, meaning that it is possible to understand how they work and how they make decisions.\\n6. Ensure that AI systems are fair, meaning that they do not discriminate against individuals\",\n", + " '\\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure a high level of safety and security of AI systems and to establish a horizontal and technology-neutral framework for AI applications. This will help to create a single market for AI and to ensure that AI systems are developed and deployed in a responsible manner. The proposal will also help to strengthen the competitiveness of the EU industry in the global AI market.\\nThe objectives of the proposal are:\\n1. To ensure that AI systems are safe and secure by establishing a risk-based framework for the development, placement on the market and use of AI systems.\\n2. To establish a horizontal and technology-neutral framework for AI applications that is applicable to all sectors and types of AI systems.\\n3. To ensure that AI systems are developed and deployed in a responsible manner by establishing requirements for transparency, robustness, security, accuracy, controllability and privacy protection.\\n4. To create a single market for AI by ensuring that AI systems are developed and deployed in a harmonised manner across the EU.\\n5. To strengthen the competitiveness of the EU industry in the global AI market by creating a level playing field for businesses and by promoting the'],\n", + " 'generations': [\"\\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure the trustworthy use of AI in the EU. It seeks to create a single market for AI applications and services, while ensuring that they are safe and respect fundamental rights. The proposal is part of the EU's broader strategy on AI, which aims to put the EU at the forefront of global AI development and deployment.\\nThe objectives of the proposal are to:\\n\\n1. Ensure that AI systems are designed, developed, and deployed in a way that respects fundamental rights and values, including human dignity, freedom, and privacy.\\n2. Ensure that AI systems are safe and secure, and do not pose unacceptable risks to people, property, or the environment.\\n3. Ensure that AI systems are robust, reliable, and accurate, and can be trusted to deliver the intended functionality.\\n4. Ensure that AI systems are traceable, meaning that it is possible to track how they work and how they make decisions.\\n5. Ensure that AI systems are transparent, meaning that it is possible to understand how they work and how they make decisions.\\n6. Ensure that AI systems are fair, meaning that they do not discriminate against individuals\",\n", + " '\\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure a high level of safety and security of AI systems and to establish a horizontal and technology-neutral framework for AI applications. This will help to create a single market for AI and to ensure that AI systems are developed and deployed in a responsible manner. The proposal will also help to strengthen the competitiveness of the EU industry in the global AI market.\\nThe objectives of the proposal are:\\n1. To ensure that AI systems are safe and secure by establishing a risk-based framework for the development, placement on the market and use of AI systems.\\n2. To establish a horizontal and technology-neutral framework for AI applications that is applicable to all sectors and types of AI systems.\\n3. To ensure that AI systems are developed and deployed in a responsible manner by establishing requirements for transparency, robustness, security, accuracy, controllability and privacy protection.\\n4. To create a single market for AI by ensuring that AI systems are developed and deployed in a harmonised manner across the EU.\\n5. To strengthen the competitiveness of the EU industry in the global AI market by creating a level playing field for businesses and by promoting the'],\n", + " 'labelling_model': 'gpt-3.5-turbo',\n", + " 'labelling_prompt': [{'content': 'Your role is to evaluate text quality based on given criteria.',\n", + " 'role': 'system'},\n", + " {'content': \"\\n# Instruction Following Assessment\\nEvaluate alignment between output and intent. Assess understanding of task goal and restrictions.\\n**Instruction Components**: Task Goal (intended outcome), Restrictions (text styles, formats, or designated methods, etc).\\n\\n**Scoring**: Rate outputs 1 to 5:\\n\\n1. **Irrelevant**: No alignment.\\n2. **Partial Focus**: Addresses one aspect poorly.\\n3. **Partial Compliance**:\\n\\t- (1) Meets goal or restrictions, neglecting other.\\n\\t- (2) Acknowledges both but slight deviations.\\n4. **Almost There**: Near alignment, minor deviations.\\n5. **Comprehensive Compliance**: Fully aligns, meets all requirements.\\n\\n---\\n\\n## Format\\n\\n### Input\\nInstruction: [Specify task goal and restrictions]\\n\\nTexts:\\n\\n [Text 1]\\n [Text 2]\\n\\n### Output\\n\\n#### Output for Text 1\\nRating: [Rating for text 1]\\nRationale: [Rationale for the rating in short sentences]\\n\\n#### Output for Text 2\\nRating: [Rating for text 2]\\nRationale: [Rationale for the rating in short sentences]\\n\\n---\\n\\n## Annotation\\n\\n### Input\\nInstruction: What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\\n\\nTexts:\\n\\n \\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure the trustworthy use of AI in the EU. It seeks to create a single market for AI applications and services, while ensuring that they are safe and respect fundamental rights. The proposal is part of the EU's broader strategy on AI, which aims to put the EU at the forefront of global AI development and deployment.\\nThe objectives of the proposal are to:\\n\\n1. Ensure that AI systems are designed, developed, and deployed in a way that respects fundamental rights and values, including human dignity, freedom, and privacy.\\n2. Ensure that AI systems are safe and secure, and do not pose unacceptable risks to people, property, or the environment.\\n3. Ensure that AI systems are robust, reliable, and accurate, and can be trusted to deliver the intended functionality.\\n4. Ensure that AI systems are traceable, meaning that it is possible to track how they work and how they make decisions.\\n5. Ensure that AI systems are transparent, meaning that it is possible to understand how they work and how they make decisions.\\n6. Ensure that AI systems are fair, meaning that they do not discriminate against individuals\\n \\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure a high level of safety and security of AI systems and to establish a horizontal and technology-neutral framework for AI applications. This will help to create a single market for AI and to ensure that AI systems are developed and deployed in a responsible manner. The proposal will also help to strengthen the competitiveness of the EU industry in the global AI market.\\nThe objectives of the proposal are:\\n1. To ensure that AI systems are safe and secure by establishing a risk-based framework for the development, placement on the market and use of AI systems.\\n2. To establish a horizontal and technology-neutral framework for AI applications that is applicable to all sectors and types of AI systems.\\n3. To ensure that AI systems are developed and deployed in a responsible manner by establishing requirements for transparency, robustness, security, accuracy, controllability and privacy protection.\\n4. To create a single market for AI by ensuring that AI systems are developed and deployed in a harmonised manner across the EU.\\n5. To strengthen the competitiveness of the EU industry in the global AI market by creating a level playing field for businesses and by promoting the\\n\\n### Output \",\n", + " 'role': 'user'}],\n", + " 'raw_labelling_response': '#### Output for Text 1\\nRating: 5\\nRationale: The text fully aligns with the task goal and restrictions. It clearly states the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence, including ensuring the trustworthy use of AI, creating a single market for AI applications and services, and ensuring safety, respect for fundamental rights, robustness, transparency, and fairness of AI systems.\\n\\n#### Output for Text 2\\nRating: 4\\nRationale: The text mostly aligns with the task goal and restrictions. It addresses the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence, including ensuring safety and security of AI systems, establishing a horizontal and technology-neutral framework, promoting responsible development and deployment of AI systems, creating a single market for AI, and strengthening the competitiveness of the EU industry in the global AI market. However, it does not explicitly mention the need to respect fundamental rights, accuracy of AI systems, and traceability of AI systems, which are mentioned in the task goal and restrictions.',\n", + " 'rating': [5.0, 4.0],\n", + " 'rationale': ['The text fully aligns with the task goal and restrictions. It clearly states the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence, including ensuring the trustworthy use of AI, creating a single market for AI applications and services, and ensuring safety, respect for fundamental rights, robustness, transparency, and fairness of AI systems.',\n", + " 'The text mostly aligns with the task goal and restrictions. It addresses the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence, including ensuring safety and security of AI systems, establishing a horizontal and technology-neutral framework, promoting responsible development and deployment of AI systems, creating a single market for AI, and strengthening the competitiveness of the EU industry in the global AI market. However, it does not explicitly mention the need to respect fundamental rights, accuracy of AI systems, and traceability of AI systems, which are mentioned in the task goal and restrictions.']}" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "preference_dataset[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Human Feedback with Argilla\n", + "\n", + "You can use the AI Feedback created by distilabel directly but we have seen that enhancing it with human feedback will improve the quality of your LLM. We provide a `to_argilla` method which creates a dataset for Argilla along with out-of-the-box tailored metadata filters and semantic search to allow you to provide human feedback as quickly and engaging as possible. You can check [the Argilla docs](https://docs.argilla.io/en/latest/getting_started/quickstart_installation.html) to get it up and running." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you are running Argilla using the Docker quickstart image or Hugging Face Spaces, you need to init the Argilla client with the URL and API_KEY:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import argilla as rg\n", + "\n", + "# Replace api_url with the url to your HF Spaces URL if using Spaces\n", + "# Replace api_key if you configured a custom API key\n", + "rg.init(\n", + " api_url=\"http://localhost:6900\",\n", + " api_key=\"owner.apikey\",\n", + " workspace=\"admin\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once our preference dataset has been correctly generated, the Argilla UI is the best tool at our disposal to visualize and annotate it. As for the instruction dataset, we just have to convert it to an Argilla Feedback Dataset, and push it to Argilla." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Uploading the Preference Dataset\n", + "preference_rg_dataset = preference_dataset.to_argilla()\n", + "\n", + "# Adding the context as a metadata property in the new Feedback dataset, as this\n", + "# information will be useful later.\n", + "for record_feedback, record_huggingface in zip(\n", + " preference_rg_dataset, preference_dataset\n", + "):\n", + " record_feedback.metadata[\"context\"] = record_huggingface[\"context\"]\n", + "\n", + "preference_rg_dataset.push_to_argilla(name=f\"notus_AI_preference\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the Argilla UI, we can see the input (an instruction), and the two generations that the LLM created out of it.\n", + "\n", + "![Preference dataset](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/preference_dataset_notus_ui.png)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusions\n", + "\n", + "To conclude, we have gone through an end-to-end example of distilabel. We've set up an Inference Endpoint, defined a distilabel pipeline that extracts information from a PDF, and created and manually reviewed the instruction and preference dataset created from that input. The final preference dataset is perfect for fine-tuning, and you can easily do this using the ArgillaTrainer from Argilla. Have a look at these resources if you want to go further:\n", + "\n", + "- [Train a Model with ArgillaTrainer](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/feedback/end2end_examples/train-model-006.html)\n", + "- [Ⓜ️ Finetuning LLMs as chat assistants: Supervised Finetuning on Mistral 7B](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/feedback/training-llm-mistral-sft.html)\n", + "- [🌠 Improving RAG by Optimizing Retrieval and Reranking Models](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/feedback/fine-tuning-sentencesimilarity-rag.html)\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/notebooks/spa/prompt_tuning_peft.ipynb b/notebooks/spa/prompt_tuning_peft.ipynb new file mode 100644 index 00000000..2ae63c4d --- /dev/null +++ b/notebooks/spa/prompt_tuning_peft.ipynb @@ -0,0 +1,1022 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "6fba2d42-ed99-4a03-8033-d479ce24d5dd", + "showTitle": false, + "title": "" + }, + "id": "2vkOvTEsVaTA" + }, + "source": [ + "# Prompt Tuning With PEFT.\n", + "_Authored by: [Pere Martra](https://github.com/peremartra)_\n", + "\n", + "\n", + "In this notebook we are introducing how to apply prompt tuning with the PEFT library to a pre-trained model.\n", + "\n", + "For a complete list of models compatible with PEFT refer to their [documentation](https://huggingface.co/docs/peft/main/en/index#supported-methods).\n", + "\n", + "A short sample of models available to be trained with PEFT includes Bloom, Llama, GPT-J, GPT-2, BERT, and more. Hugging Face is working hard to add more models to the library.\n", + "\n", + "## Brief introduction to Prompt Tuning.\n", + "It’s an Additive Fine-Tuning technique for models. This means that we WILL NOT MODIFY ANY WEIGHTS OF THE ORIGINAL MODEL. You might be wondering, how are we going to perform Fine-Tuning then? Well, we will train additional layers that are added to the model. That’s why it’s called an Additive technique.\n", + "\n", + "Considering it’s an Additive technique and its name is Prompt-Tuning, it seems clear that the layers we’re going to add and train are related to the prompt.\n", + "\n", + "![Prompt_Tuning_Diagram](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/Martra_Figure_5_Prompt_Tuning.jpg)\n", + "\n", + "We are creating a type of superprompt by enabling a model to enhance a portion of the prompt with its acquired knowledge. However, that particular section of the prompt cannot be translated into natural language. **It's as if we've mastered expressing ourselves in embeddings and generating highly effective prompts.**\n", + "\n", + "In each training cycle, the only weights that can be modified to minimize the loss function are those integrated into the prompt.\n", + "\n", + "The primary consequence of this technique is that the number of parameters to train is genuinely small. However, we encounter a second, perhaps more significant consequence, namely that, **since we do not modify the weights of the pretrained model, it does not alter its behavior or forget any information it has previously learned.**\n", + "\n", + "The training is faster and more cost-effective. Moreover, we can train various models, and during inference time, we only need to load one foundational model along with the new smaller trained models because the weights of the original model have not been altered\n", + "\n", + "## What are we going to do in the notebook?\n", + "We are going to train two different models using two datasets, each with just one pre-trained model from the Bloom family. One model will be trained with a dataset of prompts, while the other will use a dataset of inspirational sentences. We will compare the results for the same question from both models before and after training.\n", + "\n", + "Additionally, we'll explore how to load both models with only one copy of the foundational model in memory.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tZhdbTh-VaTA" + }, + "source": [ + "## Loading the PEFT Library\n", + "This library contains the Hugging Face implementation of various Fine-Tuning techniques, including Prompt Tuning" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "d16bf5ec-888b-4c76-a655-193fd4cc8a36", + "showTitle": false, + "title": "" + }, + "id": "JechhJhhVaTA" + }, + "outputs": [], + "source": [ + "!pip install -q peft==0.8.2" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": { + "id": "6CRxq5Z2WJ7C" + }, + "outputs": [], + "source": [ + "!pip install -q datasets==2.14.5" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GGbh426RVaTB" + }, + "source": [ + "From the transformers library, we import the necessary classes to instantiate the model and the tokenizer." + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "31738463-c9b0-431d-869e-1735e1e2f5c7", + "showTitle": false, + "title": "" + }, + "id": "KWOEt-yOVaTB" + }, + "outputs": [], + "source": [ + "from transformers import AutoModelForCausalLM, AutoTokenizer" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "6qYsnwjSVaTC" + }, + "source": [ + "### Loading the model and the tokenizers.\n", + "\n", + "Bloom is one of the smallest and smartest models available for training with the PEFT Library using Prompt Tuning. You can choose any model from the Bloom Family, and I encourage you to try at least two of them to observe the differences.\n", + "\n", + "I'm opting for the smallest one to minimize training time and avoid memory issues in Colab." + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": { + "id": "MnqIhv2UVaTC" + }, + "outputs": [], + "source": [ + "model_name = \"bigscience/bloomz-560m\"\n", + "#model_name=\"bigscience/bloom-1b1\"\n", + "NUM_VIRTUAL_TOKENS = 4\n", + "NUM_EPOCHS = 6" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "id": "fSMu3qRsVaTC" + }, + "outputs": [], + "source": [ + "tokenizer = AutoTokenizer.from_pretrained(model_name)\n", + "foundational_model = AutoModelForCausalLM.from_pretrained(\n", + " model_name,\n", + " trust_remote_code=True\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8W2fWhOnVaTC" + }, + "source": [ + "## Inference with the pre trained bloom model\n", + "If you want to achieve more varied and original generations, uncomment the parameters: temperature, top_p, and do_sample, in *model.generate* below\n", + "\n", + "With the default configuration, the model's responses remain consistent across calls." + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "id": "47j2D3WWVaTC" + }, + "outputs": [], + "source": [ + "#this function returns the outputs from the model received, and inputs.\n", + "def get_outputs(model, inputs, max_new_tokens=100):\n", + " outputs = model.generate(\n", + " input_ids=inputs[\"input_ids\"],\n", + " attention_mask=inputs[\"attention_mask\"],\n", + " max_new_tokens=max_new_tokens,\n", + " #temperature=0.2,\n", + " #top_p=0.95,\n", + " #do_sample=True,\n", + " repetition_penalty=1.5, #Avoid repetition.\n", + " early_stopping=True, #The model can stop before reach the max_length\n", + " eos_token_id=tokenizer.eos_token_id\n", + " )\n", + " return outputs" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "ca4d203a-5152-4947-ab34-cfd0b40a102a", + "showTitle": false, + "title": "" + }, + "id": "kRLSfuo2VaTC" + }, + "source": [ + "As we want to have two different trained models, I will create two distinct prompts.\n", + "\n", + "The first model will be trained with a dataset containing prompts, and the second one with a dataset of motivational sentences.\n", + "\n", + "The first model will receive the prompt \"I want you to act as a motivational coach.\" and the second model will receive \"There are two nice things that should matter to you:\"\n", + "\n", + "But first, I'm going to collect some results from the model without Fine-Tuning." + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "1d4c80a9-4edd-4fcd-aef0-996f4da5cc02", + "showTitle": false, + "title": "" + }, + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "QvStaT7cVaTC", + "outputId": "ab34b3cd-a849-4dff-b36d-bf25c9f55ce1" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "[\"I want you to act as a motivational coach. Don't be afraid of being challenged.\"]\n" + ] + } + ], + "source": [ + "input_prompt = tokenizer(\"I want you to act as a motivational coach. \", return_tensors=\"pt\")\n", + "foundational_outputs_prompt = get_outputs(foundational_model, input_prompt, max_new_tokens=50)\n", + "\n", + "print(tokenizer.batch_decode(foundational_outputs_prompt, skip_special_tokens=True))" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "1Xhm3jZMVaTD", + "outputId": "305f0137-6a02-4e43-9c9d-2b4ecd377937" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "['There are two nice things that should matter to you: the price and quality of your product.']\n" + ] + } + ], + "source": [ + "input_sentences = tokenizer(\"There are two nice things that should matter to you:\", return_tensors=\"pt\")\n", + "foundational_outputs_sentence = get_outputs(foundational_model, input_sentences, max_new_tokens=50)\n", + "\n", + "print(tokenizer.batch_decode(foundational_outputs_sentence, skip_special_tokens=True))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "f438d43b-6b9f-445e-9df4-60ea09640764", + "showTitle": false, + "title": "" + }, + "id": "OGbJTbRnVaTD" + }, + "source": [ + "Both answers are more or less correct. Any of the Bloom models is pre-trained and can generate sentences accurately and sensibly. Let's see if, after training, the responses are either equal or more accurately generated.\n", + "\n", + "## Preparing the Datasets\n", + "The Datasets useds are:\n", + "* https://huggingface.co/datasets/fka/awesome-chatgpt-prompts\n", + "* https://huggingface.co/datasets/Abirate/english_quotes\n" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "id": "RD8H_LLaVaTD" + }, + "outputs": [], + "source": [ + "import os\n", + "#os.environ[\"TOKENIZERS_PARALLELISM\"] = \"false\"" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "2ed62b41-e3fa-4a41-a0a9-59f35a6904f9", + "showTitle": false, + "title": "" + }, + "id": "xmAp_o4PVaTD" + }, + "outputs": [], + "source": [ + "from datasets import load_dataset\n", + "\n", + "dataset_prompt = \"fka/awesome-chatgpt-prompts\"\n", + "\n", + "#Create the Dataset to create prompts.\n", + "data_prompt = load_dataset(dataset_prompt)\n", + "data_prompt = data_prompt.map(lambda samples: tokenizer(samples[\"prompt\"]), batched=True)\n", + "train_sample_prompt = data_prompt[\"train\"].select(range(50))\n" + ] + }, + { + "cell_type": "code", + "source": [ + "display(train_sample_prompt)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 86 + }, + "id": "jNlOpGbqBgcu", + "outputId": "3f8106b2-948b-4a7b-cf78-bd3fcc2f0338" + }, + "execution_count": 51, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "Dataset({\n", + " features: ['act', 'prompt', 'input_ids', 'attention_mask'],\n", + " num_rows: 50\n", + "})" + ] + }, + "metadata": {} + } + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "dZcOaE5CU658", + "outputId": "fb8f5081-012b-4c37-ee1f-3aef2d0f54a7" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "{'act': ['Linux Terminal'], 'prompt': ['I want you to act as a linux terminal. I will type commands and you will reply with what the terminal should show. I want you to only reply with the terminal output inside one unique code block, and nothing else. do not write explanations. do not type commands unless I instruct you to do so. when i need to tell you something in english, i will do so by putting text inside curly brackets {like this}. my first command is pwd'], 'input_ids': [[44, 4026, 1152, 427, 1769, 661, 267, 104105, 28434, 17, 473, 2152, 4105, 49123, 530, 1152, 2152, 57502, 1002, 3595, 368, 28434, 3403, 6460, 17, 473, 4026, 1152, 427, 3804, 57502, 1002, 368, 28434, 10014, 14652, 2592, 19826, 4400, 10973, 15, 530, 16915, 4384, 17, 727, 1130, 11602, 184637, 17, 727, 1130, 4105, 49123, 35262, 473, 32247, 1152, 427, 727, 1427, 17, 3262, 707, 3423, 427, 13485, 1152, 7747, 361, 170205, 15, 707, 2152, 727, 1427, 1331, 55385, 5484, 14652, 6291, 999, 117805, 731, 29726, 1119, 96, 17, 2670, 3968, 9361, 632, 269, 42512]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}\n" + ] + } + ], + "source": [ + "print(train_sample_prompt[:1])" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "id": "WeM66LmEVaTD" + }, + "outputs": [], + "source": [ + "dataset_sentences = load_dataset(\"Abirate/english_quotes\")\n", + "\n", + "data_sentences = dataset_sentences.map(lambda samples: tokenizer(samples[\"quote\"]), batched=True)\n", + "train_sample_sentences = data_sentences[\"train\"].select(range(25))\n", + "train_sample_sentences = train_sample_sentences.remove_columns(['author', 'tags'])" + ] + }, + { + "cell_type": "code", + "source": [ + "display(train_sample_sentences)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 86 + }, + "id": "zUSG_M_nBp_E", + "outputId": "faf36464-de24-4512-aace-c1ff8713c1d4" + }, + "execution_count": 54, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "Dataset({\n", + " features: ['quote', 'input_ids', 'attention_mask'],\n", + " num_rows: 25\n", + "})" + ] + }, + "metadata": {} + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "b97381d4-5fe2-49d0-be5d-2fe3421edc5c", + "showTitle": false, + "title": "" + }, + "id": "0-5mv1ZpVaTD" + }, + "source": [ + "## Fine-Tuning. \n", + "\n", + "### PEFT configurations\n", + "\n", + "\n", + "API docs:\n", + "https://huggingface.co/docs/peft/main/en/package_reference/tuners#peft.PromptTuningConfig\n", + "\n", + "We can use the same configuration for both models to be trained.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "6df8e1f1-be9e-42db-b4a4-6af7cd351004", + "showTitle": false, + "title": "" + }, + "id": "sOg1Yh-oVaTD" + }, + "outputs": [], + "source": [ + "from peft import get_peft_model, PromptTuningConfig, TaskType, PromptTuningInit\n", + "\n", + "generation_config = PromptTuningConfig(\n", + " task_type=TaskType.CAUSAL_LM, #This type indicates the model will generate text.\n", + " prompt_tuning_init=PromptTuningInit.RANDOM, #The added virtual tokens are initializad with random numbers\n", + " num_virtual_tokens=NUM_VIRTUAL_TOKENS, #Number of virtual tokens to be added and trained.\n", + " tokenizer_name_or_path=model_name #The pre-trained model.\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "an9KBtB1VaTD" + }, + "source": [ + "### Creating two Prompt Tuning Models.\n", + "We will create two identical prompt tuning models using the same pre-trained model and the same config." + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "c_D8oDQZVaTD", + "outputId": "6b46ca98-3f60-49c1-dab2-91259d6387af" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "trainable params: 4,096 || all params: 559,218,688 || trainable%: 0.0007324504863471229\n", + "None\n" + ] + } + ], + "source": [ + "peft_model_prompt = get_peft_model(foundational_model, generation_config)\n", + "print(peft_model_prompt.print_trainable_parameters())" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IktYfj68VaTE", + "outputId": "28fe03b7-4490-43ba-b913-4633e269737a" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "trainable params: 4,096 || all params: 559,218,688 || trainable%: 0.0007324504863471229\n", + "None\n" + ] + } + ], + "source": [ + "peft_model_sentences = get_peft_model(foundational_model, generation_config)\n", + "print(peft_model_sentences.print_trainable_parameters())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "cff5bc33-8cfb-4144-8962-9c54362a7faa", + "showTitle": false, + "title": "" + }, + "id": "i6WhJSUwVaTE" + }, + "source": [ + "**That's amazing: did you see the reduction in trainable parameters? We are going to train a 0.001% of the paramaters available.**\n", + "\n", + "Now we are going to create the training arguments, and we will use the same configuration in both trainings." + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "id": "SJoznfzjVaTE" + }, + "outputs": [], + "source": [ + "from transformers import TrainingArguments\n", + "def create_training_arguments(path, learning_rate=0.0035, epochs=6):\n", + " training_args = TrainingArguments(\n", + " output_dir=path, # Where the model predictions and checkpoints will be written\n", + " use_cpu=True, # This is necessary for CPU clusters.\n", + " auto_find_batch_size=True, # Find a suitable batch size that will fit into memory automatically\n", + " learning_rate= learning_rate, # Higher learning rate than full Fine-Tuning\n", + " num_train_epochs=epochs\n", + " )\n", + " return training_args" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "54b78a8f-81f0-44c0-b0bc-dcb14891715f", + "showTitle": false, + "title": "" + }, + "id": "cb1j50DSVaTE" + }, + "outputs": [], + "source": [ + "\n", + "import os\n", + "\n", + "working_dir = \"./\"\n", + "\n", + "#Is best to store the models in separate folders.\n", + "#Create the name of the directories where to store the models.\n", + "output_directory_prompt = os.path.join(working_dir, \"peft_outputs_prompt\")\n", + "output_directory_sentences = os.path.join(working_dir, \"peft_outputs_sentences\")\n", + "\n", + "#Just creating the directoris if not exist.\n", + "if not os.path.exists(working_dir):\n", + " os.mkdir(working_dir)\n", + "if not os.path.exists(output_directory_prompt):\n", + " os.mkdir(output_directory_prompt)\n", + "if not os.path.exists(output_directory_sentences):\n", + " os.mkdir(output_directory_sentences)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "OC5IhO9mVaTE" + }, + "source": [ + "We need to indicate the directory containing the model when creating the TrainingArguments." + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": { + "id": "D4v4RSSeVaTE" + }, + "outputs": [], + "source": [ + "training_args_prompt = create_training_arguments(output_directory_prompt, 0.003, NUM_EPOCHS)\n", + "training_args_sentences = create_training_arguments(output_directory_sentences, 0.003, NUM_EPOCHS)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "c593deb6-5626-4fd9-89c2-2329e2f9b6e0", + "showTitle": false, + "title": "" + }, + "id": "GdMfjk5RVaTE" + }, + "source": [ + "## Train\n", + "\n", + "We will create the trainer Object, one for each model to train. " + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": { + "id": "uVAfNdEIVaTE" + }, + "outputs": [], + "source": [ + "from transformers import Trainer, DataCollatorForLanguageModeling\n", + "def create_trainer(model, training_args, train_dataset):\n", + " trainer = Trainer(\n", + " model=model, # We pass in the PEFT version of the foundation model, bloomz-560M\n", + " args=training_args, #The args for the training.\n", + " train_dataset=train_dataset, #The dataset used to tyrain the model.\n", + " data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False) # mlm=False indicates not to use masked language modeling\n", + " )\n", + " return trainer\n" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "32e43bcf-23b2-46aa-9cf0-455b83ef4f38", + "showTitle": false, + "title": "" + }, + "colab": { + "base_uri": "https://localhost:8080/", + "height": 127 + }, + "id": "1Sz9BeFZVaTF", + "outputId": "1b698470-209e-4001-fcbe-6fa8a2ac8707" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "
\n", + " \n", + " \n", + " [42/42 11:23, Epoch 6/6]\n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
StepTraining Loss

" + ] + }, + "metadata": {} + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "TrainOutput(global_step=42, training_loss=3.5800417945498513, metrics={'train_runtime': 703.2941, 'train_samples_per_second': 0.427, 'train_steps_per_second': 0.06, 'total_flos': 60957279240192.0, 'train_loss': 3.5800417945498513, 'epoch': 6.0})" + ] + }, + "metadata": {}, + "execution_count": 62 + } + ], + "source": [ + "#Training first model.\n", + "trainer_prompt = create_trainer(peft_model_prompt, training_args_prompt, train_sample_prompt)\n", + "trainer_prompt.train()" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 127 + }, + "id": "afTotMckVaTF", + "outputId": "15bed85d-17f5-4a49-d8d5-bae35e68d294" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "

\n", + " \n", + " \n", + " [24/24 03:29, Epoch 6/6]\n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
StepTraining Loss

" + ] + }, + "metadata": {} + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "TrainOutput(global_step=24, training_loss=4.4278310139973955, metrics={'train_runtime': 219.765, 'train_samples_per_second': 0.683, 'train_steps_per_second': 0.109, 'total_flos': 17825006936064.0, 'train_loss': 4.4278310139973955, 'epoch': 6.0})" + ] + }, + "metadata": {}, + "execution_count": 63 + } + ], + "source": [ + "#Training second model.\n", + "trainer_sentences = create_trainer(peft_model_sentences, training_args_sentences, train_sample_sentences)\n", + "trainer_sentences.train()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "z2Zsww_2VaTF" + }, + "source": [ + "In less than 10 minutes (CPU time in a M1 Pro) we trained 2 different models, with two different missions with a same foundational model as a base." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "5a6c8daf-8248-458a-9f6f-14865b4fbd2e", + "showTitle": false, + "title": "" + }, + "id": "s5k10HwoVaTG" + }, + "source": [ + "## Save models\n", + "We are going to save the models. These models are ready to be used, as long as we have the pre-trained model from which they were created in memory." + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "409df5ce-e496-46d7-be2c-202a463cdc80", + "showTitle": false, + "title": "" + }, + "id": "E3dn3PeMVaTG" + }, + "outputs": [], + "source": [ + "trainer_prompt.model.save_pretrained(output_directory_prompt)\n", + "trainer_sentences.model.save_pretrained(output_directory_sentences)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "fb14e3fd-bbf6-4d56-92c2-51bfe08de72a", + "showTitle": false, + "title": "" + }, + "id": "rkUKpDDWVaTG" + }, + "source": [ + "## Inference\n", + "\n", + "You can load the model from the path that you have saved to before, and ask the model to generate text based on our input before!" + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "cc48af16-c117-4019-a31a-ce1c93cd21d4", + "showTitle": false, + "title": "" + }, + "id": "dlqXXN8oVaTG" + }, + "outputs": [], + "source": [ + "from peft import PeftModel\n", + "\n", + "loaded_model_prompt = PeftModel.from_pretrained(foundational_model,\n", + " output_directory_prompt,\n", + " #device_map='auto',\n", + " is_trainable=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "6b44524b-2ac5-4e74-81e6-c406d4414e42", + "showTitle": false, + "title": "" + }, + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-4jd3zCGVaTG", + "outputId": "b55454f1-f1ed-444c-b107-698778406e6e" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "['I want you to act as a motivational coach. You will be helping students learn how they can improve their performance in the classroom and at school.']\n" + ] + } + ], + "source": [ + "loaded_model_prompt_outputs = get_outputs(loaded_model_prompt, input_prompt)\n", + "print(tokenizer.batch_decode(loaded_model_prompt_outputs, skip_special_tokens=True))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SHbeFTXjVaTG" + }, + "source": [ + "If we compare both answers something changed.\n", + "* ***Pretrained Model:*** *I want you to act as a motivational coach. Don't be afraid of being challenged.*\n", + "* ***Fine-Tuned Model:*** *I want you to act as a motivational coach. You can use this method if you're feeling anxious about your.*\n", + "\n", + "We have to keep in mind that we have only trained the model for a few minutes, but they have been enough to obtain a response closer to what we were looking for." + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": { + "id": "MuwAsq3uVaTG" + }, + "outputs": [], + "source": [ + "loaded_model_prompt.load_adapter(output_directory_sentences, adapter_name=\"quotes\")\n", + "loaded_model_prompt.set_adapter(\"quotes\")" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IQm--PWSVaTH", + "outputId": "3e814a6a-a380-4f2c-f887-6852a9f51002" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "['There are two nice things that should matter to you: the weather and your health.']\n" + ] + } + ], + "source": [ + "loaded_model_sentences_outputs = get_outputs(loaded_model_prompt, input_sentences)\n", + "print(tokenizer.batch_decode(loaded_model_sentences_outputs, skip_special_tokens=True))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UnR8y9gwVaTH" + }, + "source": [ + "With the second model we have a similar result.\n", + "* **Pretrained Model:** *There are two nice things that should matter to you: the price and quality of your product.*\n", + "* **Fine-Tuned Model:** *There are two nice things that should matter to you: the weather and your health.*\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "B6TUjNtGVaTH" + }, + "source": [ + "# Conclusion\n", + "Prompt Tuning is an amazing technique that can save us hours of training and a significant amount of money. In the notebook, we have trained two models in just a few minutes, and we can have both models in memory, providing service to different clients.\n", + "\n", + "If you want to try different combinations and models, the notebook is ready to use another model from the Bloom family.\n", + "\n", + "You can change the number of epochs to train, the number of virtual tokens, and the model in the third cell. However, there are many configurations to change. If you're looking for a good exercise, you can replace the random initialization of the virtual tokens with a fixed value.\n", + "\n", + "*The responses of the Fine-Tuned models may vary every time we train them. I've pasted the results of one of my trainings, but the actual results may differ.*" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": { + "id": "5OMyCWasVaTH" + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "application/vnd.databricks.v1+notebook": { + "dashboards": [], + "language": "python", + "notebookMetadata": { + "pythonIndentUnit": 2 + }, + "notebookName": "LLM 02 - Prompt Tuning with PEFT", + "widgets": {} + }, + "colab": { + "machine_shape": "hm", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/spa/rag_evaluation.ipynb b/notebooks/spa/rag_evaluation.ipynb new file mode 100644 index 00000000..16a109a9 --- /dev/null +++ b/notebooks/spa/rag_evaluation.ipynb @@ -0,0 +1,1515 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "4YErqpfH9jVI" + }, + "source": [ + "# RAG Evaluation\n", + "_Authored by: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", + "\n", + "This notebook demonstrates how you can evaluate your RAG (Retrieval Augmented Generation), by building a synthetic evaluation dataset and using LLM-as-a-judge to compute the accuracy of your system.\n", + "\n", + "For an introduction to RAG, you can check [this other cookbook](rag_zephyr_langchain)!\n", + "\n", + "RAG systems are complex: here a RAG diagram, where we noted in blue all possibilities for system enhancement:\n", + "\n", + "\n", + "\n", + "Implementing any of these improvements can bring a huge performance boost; but changing anything is useless if you cannot monitor the impact of your changes on the system's performance!\n", + "So let's see how to evaluate our RAG system.\n", + "\n", + "### Evaluating RAG performance\n", + "\n", + "Since there are so many moving parts to tune with a big impact on performance, benchmarking the RAG system is crucial.\n", + "\n", + "For our evaluation pipeline, we will need:\n", + "1. An evaluation dataset with question - answer couples (QA couples)\n", + "2. An evaluator to compute the accuracy of our system on the above evaluation dataset.\n", + "\n", + "➡️ It turns out, we can use LLMs to help us all along the way!\n", + "1. The evaluation dataset will be synthetically generated by an LLM 🤖, and questions will be filtered out by other LLMs 🤖\n", + "2. An [LLM-as-a-judge](https://huggingface.co/papers/2306.05685) agent 🤖 will then perform the evaluation on this synthetic dataset.\n", + "\n", + "__Let's dig into it and start building our evaluation pipeline!__ First, we install the required model dependancies." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "id": "bCKBvOcp9jVK" + }, + "outputs": [], + "source": [ + "!pip install -q torch transformers transformers langchain sentence-transformers tqdm openpyxl openai pandas datasets" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "k_lJFbYm9jVL" + }, + "outputs": [], + "source": [ + "%reload_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "oIlNZ1Mn9jVL" + }, + "outputs": [], + "source": [ + "from tqdm.auto import tqdm\n", + "import pandas as pd\n", + "from typing import Optional, List, Tuple\n", + "import json\n", + "import datasets\n", + "\n", + "pd.set_option(\"display.max_colwidth\", None)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from huggingface_hub import notebook_login\n", + "\n", + "notebook_login()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "zeW8P62J9jVM" + }, + "source": [ + "### Load your knowledge base" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "id": "YRbm5tNF9jVM" + }, + "outputs": [], + "source": [ + "ds = datasets.load_dataset(\"m-ric/huggingface_doc\", split=\"train\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wy9CKj0M9jVM" + }, + "source": [ + "# 1. Build a synthetic dataset for evaluation\n", + "We first build a synthetic dataset of questions and associated contexts. The method is to get elements from our knowledge base, and ask an LLM to generate questions based on these documents.\n", + "\n", + "Then we setup other LLM agents to act as quality filters for the generated QA couples: each of them will act as the filter for a specific flaw." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "QkoEgiDg9jVM" + }, + "source": [ + "### 1.1. Prepare source documents" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "3gTOlRKO9jVM" + }, + "outputs": [], + "source": [ + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "from langchain.docstore.document import Document as LangchainDocument\n", + "\n", + "langchain_docs = [\n", + " LangchainDocument(page_content=doc[\"text\"], metadata={\"source\": doc[\"source\"]})\n", + " for doc in tqdm(ds)\n", + "]\n", + "\n", + "\n", + "text_splitter = RecursiveCharacterTextSplitter(\n", + " chunk_size=2000,\n", + " chunk_overlap=200,\n", + " add_start_index=True,\n", + " separators=[\"\\n\\n\", \"\\n\", \".\", \" \", \"\"],\n", + ")\n", + "\n", + "docs_processed = []\n", + "for doc in langchain_docs:\n", + " docs_processed += text_splitter.split_documents([doc])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WjrNhcCh9jVN" + }, + "source": [ + "### 1.2. Setup agents for question generation\n", + "\n", + "We use [Mixtral](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) for QA couple generation because it it has excellent performance in leaderboards such as [Chatbot Arena](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)." + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": { + "id": "GoRySj3Q9jVN" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'This is a test context for the `@mui/material` library.\\n\\n## Installation\\n\\n```sh\\nnpm install @mui/material\\n```\\n\\n## Usage\\n\\n```jsx\\nimport React from \\'react\\';\\nimport { Button } from \\'@mui/material\\';\\n\\nfunction App() {\\n return (\\n

\\n \\n
\\n );\\n}\\n\\nexport default App;\\n```\\n\\n## Documentation\\n\\n- [Material-UI](https://material-ui.com/)\\n- [Material Design](https://material.io/)'" + ] + }, + "execution_count": 83, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from huggingface_hub import InferenceClient\n", + "\n", + "\n", + "repo_id = \"mistralai/Mixtral-8x7B-Instruct-v0.1\"\n", + "\n", + "llm_client = InferenceClient(\n", + " model=repo_id,\n", + " timeout=120,\n", + ")\n", + "\n", + "\n", + "def call_llm(inference_client: InferenceClient, prompt: str):\n", + " response = inference_client.post(\n", + " json={\n", + " \"inputs\": prompt,\n", + " \"parameters\": {\"max_new_tokens\": 1000},\n", + " \"task\": \"text-generation\",\n", + " },\n", + " )\n", + " return json.loads(response.decode())[0][\"generated_text\"]\n", + "\n", + "\n", + "call_llm(llm_client, \"This is a test context\")" + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "metadata": { + "id": "hIM_DJRo9jVN" + }, + "outputs": [], + "source": [ + "QA_generation_prompt = \"\"\"\n", + "Your task is to write a factoid question and an answer given a context.\n", + "Your factoid question should be answerable with a specific, concise piece of factual information from the context.\n", + "Your factoid question should be formulated in the same style as questions users could ask in a search engine.\n", + "This means that your factoid question MUST NOT mention something like \"according to the passage\" or \"context\".\n", + "\n", + "Provide your answer as follows:\n", + "\n", + "Output:::\n", + "Factoid question: (your factoid question)\n", + "Answer: (your answer to the factoid question)\n", + "\n", + "Now here is the context.\n", + "\n", + "Context: {context}\\n\n", + "Output:::\"\"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lVFc-lVy9jVN" + }, + "source": [ + "Now let's generate our QA couples.\n", + "For this example, we generate only 10 QA couples and will load the rest from the Hub.\n", + "\n", + "But for your specific knowledge base, given that you want to get at least ~100 test samples, and accounting for the fact that we will filter out around half of these with our critique agents later on, you should generate much more, in the >200 samples." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "8fteqDDD9jVN" + }, + "outputs": [], + "source": [ + "import random\n", + "\n", + "N_GENERATIONS = 10 # We intentionally generate only 10 QA couples here for cost and time considerations\n", + "\n", + "print(f\"Generating {N_GENERATIONS} QA couples...\")\n", + "\n", + "outputs = []\n", + "for sampled_context in tqdm(random.sample(docs_processed, N_GENERATIONS)):\n", + " # Generate QA couple\n", + " output_QA_couple = call_llm(\n", + " llm_client, QA_generation_prompt.format(context=sampled_context.page_content)\n", + " )\n", + " try:\n", + " question = output_QA_couple.split(\"Factoid question: \")[-1].split(\"Answer: \")[0]\n", + " answer = output_QA_couple.split(\"Answer: \")[-1]\n", + " assert len(answer) < 300, \"Answer is too long\"\n", + " outputs.append(\n", + " {\n", + " \"context\": sampled_context.page_content,\n", + " \"question\": question,\n", + " \"answer\": answer,\n", + " \"source_doc\": sampled_context.metadata[\"source\"],\n", + " }\n", + " )\n", + " except:\n", + " continue" + ] + }, + { + "cell_type": "code", + "execution_count": 102, + "metadata": { + "id": "aUlOUDv59jVN", + "outputId": "c9634fdb-2a7f-43a6-c4eb-e60b166b8238" + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
contextquestionanswersource_doc
0Now, we can just call the `Tokenizer.train` method with any list of files we want to use:\\n\\n<tokenizerslangcontent>\\n<python>\\n<literalinclude>\\n{\"path\": \"../../bindings/python/tests/documentation/test_quicktour.py\",\\n\"language\": \"python\",\\n\"start-after\": \"START train\",\\n\"end-before\": \"END train\",\\n\"dedent\": 8}\\n</literalinclude>\\n</python>\\n<rust>\\n<literalinclude>\\n{\"path\": \"../../tokenizers/tests/documentation.rs\",\\n\"language\": \"rust\",\\n\"start-after\": \"START quicktour_train\",\\n\"end-before\": \"END quicktour_train\",\\n\"dedent\": 4}\\n</literalinclude>\\n</rust>\\n<node>\\n<literalinclude>\\n{\"path\": \"../../bindings/node/examples/documentation/quicktour.test.ts\",\\n\"language\": \"js\",\\n\"start-after\": \"START train\",\\n\"end-before\": \"END train\",\\n\"dedent\": 8}\\n</literalinclude>\\n</node>\\n</tokenizerslangcontent>\\n\\nThis should only take a few seconds to train our tokenizer on the full\\nwikitext dataset! To save the tokenizer in one file that contains all\\nits configuration and vocabulary, just use the\\n`Tokenizer.save` method:\\n\\n<tokenizerslangcontent>\\n<python>\\n<literalinclude>\\n{\"path\": \"../../bindings/python/tests/documentation/test_quicktour.py\",\\n\"language\": \"python\",\\n\"start-after\": \"START save\",\\n\"end-before\": \"END save\",\\n\"dedent\": 8}\\n</literalinclude>\\n</python>\\n<rust>\\n<literalinclude>\\n{\"path\": \"../../tokenizers/tests/documentation.rs\",\\n\"language\": \"rust\",\\n\"start-after\": \"START quicktour_save\",\\n\"end-before\": \"END quicktour_save\",\\n\"dedent\": 4}\\n</literalinclude>\\n</rust>\\n<node>\\n<literalinclude>\\n{\"path\": \"../../bindings/node/examples/documentation/quicktour.test.ts\",\\n\"language\": \"js\",\\n\"start-after\": \"START save\",\\n\"end-before\": \"END save\",\\n\"dedent\": 8}\\n</literalinclude>\\n</node>\\n</tokenizerslangcontent>\\n\\nand you can reload your tokenizer from that file with the\\n`Tokenizer.from_file`\\n`classmethod`:How can you reload a tokenizer from a file in Python?\\nYou can reload a tokenizer from a file in Python using the `Tokenizer.from_file` classmethod.huggingface/tokenizers/blob/main/docs/source-doc-builder/quicktour.mdx
\n", + "
" + ], + "text/plain": [ + " context \\\n", + "0 Now, we can just call the `Tokenizer.train` method with any list of files we want to use:\\n\\n\\n\\n\\n{\"path\": \"../../bindings/python/tests/documentation/test_quicktour.py\",\\n\"language\": \"python\",\\n\"start-after\": \"START train\",\\n\"end-before\": \"END train\",\\n\"dedent\": 8}\\n\\n\\n\\n\\n{\"path\": \"../../tokenizers/tests/documentation.rs\",\\n\"language\": \"rust\",\\n\"start-after\": \"START quicktour_train\",\\n\"end-before\": \"END quicktour_train\",\\n\"dedent\": 4}\\n\\n\\n\\n\\n{\"path\": \"../../bindings/node/examples/documentation/quicktour.test.ts\",\\n\"language\": \"js\",\\n\"start-after\": \"START train\",\\n\"end-before\": \"END train\",\\n\"dedent\": 8}\\n\\n\\n\\n\\nThis should only take a few seconds to train our tokenizer on the full\\nwikitext dataset! To save the tokenizer in one file that contains all\\nits configuration and vocabulary, just use the\\n`Tokenizer.save` method:\\n\\n\\n\\n\\n{\"path\": \"../../bindings/python/tests/documentation/test_quicktour.py\",\\n\"language\": \"python\",\\n\"start-after\": \"START save\",\\n\"end-before\": \"END save\",\\n\"dedent\": 8}\\n\\n\\n\\n\\n{\"path\": \"../../tokenizers/tests/documentation.rs\",\\n\"language\": \"rust\",\\n\"start-after\": \"START quicktour_save\",\\n\"end-before\": \"END quicktour_save\",\\n\"dedent\": 4}\\n\\n\\n\\n\\n{\"path\": \"../../bindings/node/examples/documentation/quicktour.test.ts\",\\n\"language\": \"js\",\\n\"start-after\": \"START save\",\\n\"end-before\": \"END save\",\\n\"dedent\": 8}\\n\\n\\n\\n\\nand you can reload your tokenizer from that file with the\\n`Tokenizer.from_file`\\n`classmethod`: \n", + "\n", + " question \\\n", + "0 How can you reload a tokenizer from a file in Python?\\n \n", + "\n", + " answer \\\n", + "0 You can reload a tokenizer from a file in Python using the `Tokenizer.from_file` classmethod. \n", + "\n", + " source_doc \n", + "0 huggingface/tokenizers/blob/main/docs/source-doc-builder/quicktour.mdx " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "display(pd.DataFrame(outputs).head(1))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0KG4dNtg9jVN" + }, + "source": [ + "### 1.3. Setup critique agents\n", + "\n", + "The questions generated by the previous agent can have many flaws: we should do a quality check before validating these questions.\n", + "\n", + "We thus build critique agents that will rate each question on several criteria, given in [this paper](https://huggingface.co/papers/2312.10003):\n", + "- **Groundedness:** can the question be answered from the given context?\n", + "- **Relevance:** is the question relevant to users? For instance, `\"What is the date when transformers 4.29.1 was released?\"` is not relevant for ML practicioners.\n", + "\n", + "One last failure case we've noticed is when a function is tailored for the particular setting where the question was generated, but undecipherable by itself, like `\"What is the name of the function used in this guide?\"`.\n", + "We also build a critique agent for this criteria:\n", + "- **Stand-alone**: is the question understandable free of any context, for someone with domain knowledge/Internet access? The opposite of this would be `What is the function used in this article?` for a question generated from a specific blog article.\n", + "\n", + "We systematically score functions with all these agents, and whenever the score is too low for any one of the agents, we eliminate the question from our eval dataset.\n", + "\n", + "💡 ___When asking the agents to output a score, we first ask them to produce its rationale. This will help us verify scores, but most importantly, asking it to first output rationale gives the model more tokens to think and elaborate an answer before summarizing it into a single score token.___\n", + "\n", + "We now build and run these critique agents." + ] + }, + { + "cell_type": "code", + "execution_count": 103, + "metadata": { + "id": "05aSgTGs9jVO" + }, + "outputs": [], + "source": [ + "question_groundedness_critique_prompt = \"\"\"\n", + "You will be given a context and a question.\n", + "Your task is to provide a 'total rating' scoring how well one can answer the given question unambiguously with the given context.\n", + "Give your answer on a scale of 1 to 5, where 1 means that the question is not answerable at all given the context, and 5 means that the question is clearly and unambiguously answerable with the context.\n", + "\n", + "Provide your answer as follows:\n", + "\n", + "Answer:::\n", + "Evaluation: (your rationale for the rating, as a text)\n", + "Total rating: (your rating, as a number between 1 and 5)\n", + "\n", + "You MUST provide values for 'Evaluation:' and 'Total rating:' in your answer.\n", + "\n", + "Now here are the question and context.\n", + "\n", + "Question: {question}\\n\n", + "Context: {context}\\n\n", + "Answer::: \"\"\"\n", + "\n", + "question_relevance_critique_prompt = \"\"\"\n", + "You will be given a question.\n", + "Your task is to provide a 'total rating' representing how useful this question can be to machine learning developers building NLP applications with the Hugging Face ecosystem.\n", + "Give your answer on a scale of 1 to 5, where 1 means that the question is not useful at all, and 5 means that the question is extremely useful.\n", + "\n", + "Provide your answer as follows:\n", + "\n", + "Answer:::\n", + "Evaluation: (your rationale for the rating, as a text)\n", + "Total rating: (your rating, as a number between 1 and 5)\n", + "\n", + "You MUST provide values for 'Evaluation:' and 'Total rating:' in your answer.\n", + "\n", + "Now here is the question.\n", + "\n", + "Question: {question}\\n\n", + "Answer::: \"\"\"\n", + "\n", + "question_standalone_critique_prompt = \"\"\"\n", + "You will be given a question.\n", + "Your task is to provide a 'total rating' representing how context-independant this question is.\n", + "Give your answer on a scale of 1 to 5, where 1 means that the question depends on additional information to be understood, and 5 means that the question makes sense by itself.\n", + "For instance, if the question refers to a particular setting, like 'in the context' or 'in the document', the rating must be 1.\n", + "The questions can contain obscure technical nouns or acronyms like Gradio, Hub, Hugging Face or Space and still be a 5: it must simply be clear to an operator with access to documentation what the question is about.\n", + "\n", + "For instance, \"What is the name of the checkpoint from which the ViT model is imported?\" should receive a 1, since there is an implicit mention of a context, thus the question is not independant from the context.\n", + "\n", + "Provide your answer as follows:\n", + "\n", + "Answer:::\n", + "Evaluation: (your rationale for the rating, as a text)\n", + "Total rating: (your rating, as a number between 1 and 5)\n", + "\n", + "You MUST provide values for 'Evaluation:' and 'Total rating:' in your answer.\n", + "\n", + "Now here is the question.\n", + "\n", + "Question: {question}\\n\n", + "Answer::: \"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "b9tbk7ME9jVO" + }, + "outputs": [], + "source": [ + "print(\"Generating critique for each QA couple...\")\n", + "for output in tqdm(outputs):\n", + " evaluations = {\n", + " \"groundedness\": call_llm(\n", + " llm_client,\n", + " question_groundedness_critique_prompt.format(\n", + " context=output[\"context\"], question=output[\"question\"]\n", + " ),\n", + " ),\n", + " \"relevance\": call_llm(\n", + " llm_client,\n", + " question_relevance_critique_prompt.format(question=output[\"question\"]),\n", + " ),\n", + " \"standalone\": call_llm(\n", + " llm_client,\n", + " question_standalone_critique_prompt.format(question=output[\"question\"]),\n", + " ),\n", + " }\n", + " try:\n", + " for criterion, evaluation in evaluations.items():\n", + " score, eval = (\n", + " int(evaluation.split(\"Total rating: \")[-1].strip()),\n", + " evaluation.split(\"Total rating: \")[-2].split(\"Evaluation: \")[1],\n", + " )\n", + " output.update(\n", + " {\n", + " f\"{criterion}_score\": score,\n", + " f\"{criterion}_eval\": eval,\n", + " }\n", + " )\n", + " except Exception as e:\n", + " continue" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "IQv36Y_f9jVO" + }, + "source": [ + "Now let us filter out bad questions based on our critique agent scores:" + ] + }, + { + "cell_type": "code", + "execution_count": 105, + "metadata": { + "id": "oBWuOu1b9jVO", + "outputId": "b32bacea-52f8-486a-96fe-5c188605c5a2" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Evaluation dataset before filtering:\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
questionanswergroundedness_scorerelevance_scorestandalone_score
0How can you reload a tokenizer from a file in Python?\\nYou can reload a tokenizer from a file in Python using the `Tokenizer.from_file` classmethod.4.05.05.0
1What is the output of my\\_cool\\_auth\\_method when passing both token and use\\_auth\\_token?\\nUserWarning: Both `token` and `use_auth_token` are passed (...). `use_auth_token` value will be ignored. \"<token>\"1.01.05.0
2Which version of Gradio introduced a hotfix to support pydantic v1 and v2?\\n3.36.15.02.05.0
3Which model was released by Meta AI?\\nDINOv25.05.05.0
4What is the type of transformer that uses cross-attention?\\nThe type of transformer that uses cross-attention is the \"encoder-decoder\" transformer or the \"sequence-to-sequence\" transformer.NaNNaNNaN
5What is the license for the Llama-2-7b-Chat-64g-GPTQ model?\\nThe license for the Llama-2-7b-Chat-64g-GPTQ model is the llama-2-community-license.4.04.05.0
6What is the library for easily accessing and sharing datasets for Audio, Computer Vision, and Natural Language Processing tasks?\\n🤗 Datasets5.05.05.0
7Which feature improves performance for many applications?\\nThe feature that improves performance for many applications is lazy loading interactive or static variants of a component individually, rather than loading both variants regardless.4.04.05.0
\n", + "
" + ], + "text/plain": [ + " question \\\n", + "0 How can you reload a tokenizer from a file in Python?\\n \n", + "1 What is the output of my\\_cool\\_auth\\_method when passing both token and use\\_auth\\_token?\\n \n", + "2 Which version of Gradio introduced a hotfix to support pydantic v1 and v2?\\n \n", + "3 Which model was released by Meta AI?\\n \n", + "4 What is the type of transformer that uses cross-attention?\\n \n", + "5 What is the license for the Llama-2-7b-Chat-64g-GPTQ model?\\n \n", + "6 What is the library for easily accessing and sharing datasets for Audio, Computer Vision, and Natural Language Processing tasks?\\n \n", + "7 Which feature improves performance for many applications?\\n \n", + "\n", + " answer \\\n", + "0 You can reload a tokenizer from a file in Python using the `Tokenizer.from_file` classmethod. \n", + "1 UserWarning: Both `token` and `use_auth_token` are passed (...). `use_auth_token` value will be ignored. \"\" \n", + "2 3.36.1 \n", + "3 DINOv2 \n", + "4 The type of transformer that uses cross-attention is the \"encoder-decoder\" transformer or the \"sequence-to-sequence\" transformer. \n", + "5 The license for the Llama-2-7b-Chat-64g-GPTQ model is the llama-2-community-license. \n", + "6 🤗 Datasets \n", + "7 The feature that improves performance for many applications is lazy loading interactive or static variants of a component individually, rather than loading both variants regardless. \n", + "\n", + " groundedness_score relevance_score standalone_score \n", + "0 4.0 5.0 5.0 \n", + "1 1.0 1.0 5.0 \n", + "2 5.0 2.0 5.0 \n", + "3 5.0 5.0 5.0 \n", + "4 NaN NaN NaN \n", + "5 4.0 4.0 5.0 \n", + "6 5.0 5.0 5.0 \n", + "7 4.0 4.0 5.0 " + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "============================================\n", + "Final evaluation dataset:\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
questionanswergroundedness_scorerelevance_scorestandalone_score
0How can you reload a tokenizer from a file in Python?\\nYou can reload a tokenizer from a file in Python using the `Tokenizer.from_file` classmethod.4.05.05.0
3Which model was released by Meta AI?\\nDINOv25.05.05.0
5What is the license for the Llama-2-7b-Chat-64g-GPTQ model?\\nThe license for the Llama-2-7b-Chat-64g-GPTQ model is the llama-2-community-license.4.04.05.0
6What is the library for easily accessing and sharing datasets for Audio, Computer Vision, and Natural Language Processing tasks?\\n🤗 Datasets5.05.05.0
7Which feature improves performance for many applications?\\nThe feature that improves performance for many applications is lazy loading interactive or static variants of a component individually, rather than loading both variants regardless.4.04.05.0
\n", + "
" + ], + "text/plain": [ + " question \\\n", + "0 How can you reload a tokenizer from a file in Python?\\n \n", + "3 Which model was released by Meta AI?\\n \n", + "5 What is the license for the Llama-2-7b-Chat-64g-GPTQ model?\\n \n", + "6 What is the library for easily accessing and sharing datasets for Audio, Computer Vision, and Natural Language Processing tasks?\\n \n", + "7 Which feature improves performance for many applications?\\n \n", + "\n", + " answer \\\n", + "0 You can reload a tokenizer from a file in Python using the `Tokenizer.from_file` classmethod. \n", + "3 DINOv2 \n", + "5 The license for the Llama-2-7b-Chat-64g-GPTQ model is the llama-2-community-license. \n", + "6 🤗 Datasets \n", + "7 The feature that improves performance for many applications is lazy loading interactive or static variants of a component individually, rather than loading both variants regardless. \n", + "\n", + " groundedness_score relevance_score standalone_score \n", + "0 4.0 5.0 5.0 \n", + "3 5.0 5.0 5.0 \n", + "5 4.0 4.0 5.0 \n", + "6 5.0 5.0 5.0 \n", + "7 4.0 4.0 5.0 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import pandas as pd\n", + "\n", + "pd.set_option(\"display.max_colwidth\", None)\n", + "\n", + "generated_questions = pd.DataFrame.from_dict(outputs)\n", + "\n", + "print(\"Evaluation dataset before filtering:\")\n", + "display(\n", + " generated_questions[\n", + " [\n", + " \"question\",\n", + " \"answer\",\n", + " \"groundedness_score\",\n", + " \"relevance_score\",\n", + " \"standalone_score\",\n", + " ]\n", + " ]\n", + ")\n", + "generated_questions = generated_questions.loc[\n", + " (generated_questions[\"groundedness_score\"] >= 4)\n", + " & (generated_questions[\"relevance_score\"] >= 4)\n", + " & (generated_questions[\"standalone_score\"] >= 4)\n", + "]\n", + "print(\"============================================\")\n", + "print(\"Final evaluation dataset:\")\n", + "display(\n", + " generated_questions[\n", + " [\n", + " \"question\",\n", + " \"answer\",\n", + " \"groundedness_score\",\n", + " \"relevance_score\",\n", + " \"standalone_score\",\n", + " ]\n", + " ]\n", + ")\n", + "\n", + "eval_dataset = datasets.Dataset.from_pandas(\n", + " generated_questions, split=\"train\", preserve_index=False\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "HaOMZyu69jVO" + }, + "source": [ + "Now our synthetic evaluation dataset is complete! We can evaluate different RAG systems on this evaluation dataset.\n", + "\n", + "We have generated only a few QA couples here to reduce time and cost. But let's kick start the next part by loading a pre-generated dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Q3RRz4W79jVO" + }, + "outputs": [], + "source": [ + "eval_dataset = datasets.load_dataset(\"m-ric/huggingface_doc_qa_eval\", split=\"train\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "K5s19uTd9jVO" + }, + "source": [ + "# 2. Build our RAG System" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Z-mET8Dy9jVO" + }, + "source": [ + "### 2.1. Preprocessing documents to build our vector database\n", + "\n", + "- In this part, __we split the documents from our knowledge base into smaller chunks__: these will be the snippets that are picked by the Retriever, to then be ingested by the Reader LLM as supporting elements for its answer.\n", + "- The goal is to build semantically relevant snippets: not too small to be sufficient for supporting an answer, and not too large too avoid diluting individual ideas.\n", + "\n", + "Many options exist for text splitting:\n", + "- split every `n` words / characters, but this has the risk of cutting in half paragraphs or even sentences\n", + "- split after `n` words / character, but only on sentence boundaries\n", + "- **recursive split** tries to preserve even more of the document structure, by processing it tree-like way, splitting first on the largest units (chapters) then recursively splitting on smaller units (paragraphs, sentences).\n", + "\n", + "To learn more about chunking, I recommend you read [this great notebook](https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/5_Levels_Of_Text_Splitting.ipynb) by Greg Kamradt.\n", + "\n", + "[This space](https://huggingface.co/spaces/m-ric/chunk_visualizer) lets you visualize how different splitting options affect the chunks you get.\n", + "\n", + "> In the following, we use Langchain's `RecursiveCharacterTextSplitter`.\n", + "\n", + "💡 _To measure chunk length in our Text Splitter, our length function will not be the count of characters, but the count of tokens in the tokenized text: indeed, for subsequent embedder that processes token, measuring length in tokens is more relevant and empirically performs better._" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "H4fhm55Q9jVO" + }, + "outputs": [], + "source": [ + "from langchain.docstore.document import Document as LangchainDocument\n", + "\n", + "RAW_KNOWLEDGE_BASE = [\n", + " LangchainDocument(page_content=doc[\"text\"], metadata={\"source\": doc[\"source\"]})\n", + " for doc in tqdm(ds)\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "sz9Jw2_q9jVO" + }, + "outputs": [], + "source": [ + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "from transformers import AutoTokenizer\n", + "\n", + "\n", + "def split_documents(\n", + " chunk_size: int,\n", + " knowledge_base: List[LangchainDocument],\n", + " tokenizer_name: str,\n", + ") -> List[LangchainDocument]:\n", + " \"\"\"\n", + " Split documents into chunks of size `chunk_size` characters and return a list of documents.\n", + " \"\"\"\n", + " text_splitter = RecursiveCharacterTextSplitter.from_huggingface_tokenizer(\n", + " AutoTokenizer.from_pretrained(tokenizer_name),\n", + " chunk_size=chunk_size,\n", + " chunk_overlap=int(chunk_size / 10),\n", + " add_start_index=True,\n", + " strip_whitespace=True,\n", + " separators=[\"\\n\\n\", \"\\n\", \".\", \" \", \"\"],\n", + " )\n", + "\n", + " docs_processed = []\n", + " for doc in knowledge_base:\n", + " docs_processed += text_splitter.split_documents([doc])\n", + "\n", + " # Remove duplicates\n", + " unique_texts = {}\n", + " docs_processed_unique = []\n", + " for doc in docs_processed:\n", + " if doc.page_content not in unique_texts:\n", + " unique_texts[doc.page_content] = True\n", + " docs_processed_unique.append(doc)\n", + "\n", + " return docs_processed_unique" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "QzBYfNG79jVO" + }, + "source": [ + "### 2.2. Retriever - embeddings 🗂️\n", + "The __retriever acts like an internal search engine__: given the user query, it returns the most relevant documents from your knowledge base.\n", + "\n", + "> For the knowledge base, we use Langchain vector databases since __it offers a convenient [FAISS](https://github.com/facebookresearch/faiss) index and allows us to keep document metadata throughout the processing__.\n", + "\n", + "🛠️ __Options included:__\n", + "\n", + "- Tune the chunking method:\n", + " - Size of the chunks\n", + " - Method: split on different separators, use [semantic chunking](https://python.langchain.com/docs/modules/data_connection/document_transformers/semantic-chunker)...\n", + "- Change the embedding model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "LqJlIDZR9jVO" + }, + "outputs": [], + "source": [ + "from langchain.vectorstores import FAISS\n", + "from langchain_community.embeddings import HuggingFaceEmbeddings\n", + "from langchain_community.vectorstores.utils import DistanceStrategy\n", + "import os\n", + "\n", + "\n", + "def load_embeddings(\n", + " langchain_docs: List[LangchainDocument],\n", + " chunk_size: int,\n", + " embedding_model_name: Optional[str] = \"thenlper/gte-small\",\n", + ") -> FAISS:\n", + " \"\"\"\n", + " Creates a FAISS index from the given embedding model and documents. Loads the index directly if it already exists.\n", + "\n", + " Args:\n", + " langchain_docs: list of documents\n", + " chunk_size: size of the chunks to split the documents into\n", + " embedding_model_name: name of the embedding model to use\n", + "\n", + " Returns:\n", + " FAISS index\n", + " \"\"\"\n", + " # load embedding_model\n", + " embedding_model = HuggingFaceEmbeddings(\n", + " model_name=embedding_model_name,\n", + " multi_process=True,\n", + " model_kwargs={\"device\": \"cuda\"},\n", + " encode_kwargs={\n", + " \"normalize_embeddings\": True\n", + " }, # set True to compute cosine similarity\n", + " )\n", + "\n", + " # Check if embeddings already exist on disk\n", + " index_name = (\n", + " f\"index_chunk:{chunk_size}_embeddings:{embedding_model_name.replace('/', '~')}\"\n", + " )\n", + " index_folder_path = f\"./data/indexes/{index_name}/\"\n", + " if os.path.isdir(index_folder_path):\n", + " return FAISS.load_local(\n", + " index_folder_path,\n", + " embedding_model,\n", + " distance_strategy=DistanceStrategy.COSINE,\n", + " )\n", + "\n", + " else:\n", + " print(\"Index not found, generating it...\")\n", + " docs_processed = split_documents(\n", + " chunk_size,\n", + " langchain_docs,\n", + " embedding_model_name,\n", + " )\n", + " knowledge_index = FAISS.from_documents(\n", + " docs_processed, embedding_model, distance_strategy=DistanceStrategy.COSINE\n", + " )\n", + " knowledge_index.save_local(index_folder_path)\n", + " return knowledge_index" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "b6y1mQJX9jVO" + }, + "source": [ + "### 2.3. Reader - LLM 💬\n", + "\n", + "In this part, the __LLM Reader reads the retrieved documents to formulate its answer.__\n", + "\n", + "🛠️ Here we tried the following options to improve results:\n", + "- Switch reranking on/off\n", + "- Change the reader model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "9PdpuWyP9jVP" + }, + "outputs": [], + "source": [ + "RAG_PROMPT_TEMPLATE = \"\"\"\n", + "<|system|>\n", + "Using the information contained in the context,\n", + "give a comprehensive answer to the question.\n", + "Respond only to the question asked, response should be concise and relevant to the question.\n", + "Provide the number of the source document when relevant.\n", + "If the answer cannot be deduced from the context, do not give an answer.\n", + "<|user|>\n", + "Context:\n", + "{context}\n", + "---\n", + "Now here is the question you need to answer.\n", + "\n", + "Question: {question}\n", + "\n", + "<|assistant|>\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "9SDqenld9jVP" + }, + "outputs": [], + "source": [ + "from langchain_community.llms import HuggingFaceHub\n", + "\n", + "repo_id = \"HuggingFaceH4/zephyr-7b-beta\"\n", + "READER_MODEL_NAME = \"zephyr-7b-beta\"\n", + "\n", + "READER_LLM = HuggingFaceHub(\n", + " repo_id=repo_id,\n", + " task=\"text-generation\",\n", + " model_kwargs={\n", + " \"max_new_tokens\": 512,\n", + " \"top_k\": 30,\n", + " \"temperature\": 0.1,\n", + " \"repetition_penalty\": 1.03,\n", + " },\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "QZ62CbcZ9jVP" + }, + "outputs": [], + "source": [ + "from ragatouille import RAGPretrainedModel\n", + "from langchain_core.vectorstores import VectorStore\n", + "from langchain_core.language_models.llms import LLM\n", + "\n", + "\n", + "def answer_with_rag(\n", + " question: str,\n", + " llm: LLM,\n", + " knowledge_index: VectorStore,\n", + " reranker: Optional[RAGPretrainedModel] = None,\n", + " num_retrieved_docs: int = 30,\n", + " num_docs_final: int = 7,\n", + ") -> Tuple[str, List[LangchainDocument]]:\n", + " \"\"\"Answer a question using RAG with the given knowledge index.\"\"\"\n", + " # Gather documents with retriever\n", + " relevant_docs = knowledge_index.similarity_search(\n", + " query=question, k=num_retrieved_docs\n", + " )\n", + " relevant_docs = [doc.page_content for doc in relevant_docs] # keep only the text\n", + "\n", + " # Optionally rerank results\n", + " if reranker:\n", + " relevant_docs = reranker.rerank(question, relevant_docs, k=num_docs_final)\n", + " relevant_docs = [doc[\"content\"] for doc in relevant_docs]\n", + "\n", + " relevant_docs = relevant_docs[:num_docs_final]\n", + "\n", + " # Build the final prompt\n", + " context = \"\\nExtracted documents:\\n\"\n", + " context += \"\".join(\n", + " [f\"Document {str(i)}:::\\n\" + doc for i, doc in enumerate(relevant_docs)]\n", + " )\n", + "\n", + " final_prompt = RAG_PROMPT_TEMPLATE.format(question=question, context=context)\n", + "\n", + " # Redact an answer\n", + " answer = llm(final_prompt)\n", + "\n", + " return answer, relevant_docs" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hiygbqfT9jVP" + }, + "source": [ + "# 3. Benchmarking the RAG system\n", + "\n", + "The RAG system and the evaluation datasets are now ready. The last step is to judge the RAG system's output on this evlauation dataset.\n", + "\n", + "To this end, __we setup a judge agent__. ⚖️🤖\n", + "\n", + "Out of [the different RAG evaluation metrics](https://docs.ragas.io/en/latest/concepts/metrics/index.html), we choose to focus only on faithfulness since it the best end-to-end metric of our system's performance.\n", + "\n", + "> We use GPT4 as a judge for its empirically good performance, but you could try with other models such as [kaist-ai/prometheus-13b-v1.0](https://huggingface.co/kaist-ai/prometheus-13b-v1.0) or [BAAI/JudgeLM-33B-v1.0](https://huggingface.co/BAAI/JudgeLM-33B-v1.0).\n", + "\n", + "💡 _In the evaluation prompt, we give a detailed description each metric on the scale 1-5, as is done in [Prometheus's prompt template](https://huggingface.co/kaist-ai/prometheus-13b-v1.0): this helps the model ground its metric precisely. If instead you give the judge LLM a vague scale to work with, the outputs will not be consistent enough between different examples._\n", + "\n", + "💡 _Again, prompting the LLM to output rationale before giving its final score gives it more tokens to help it formalize and elaborate a judgement._" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "VrlMh_ZI9jVP" + }, + "outputs": [], + "source": [ + "def run_rag_tests(\n", + " eval_dataset: datasets.Dataset,\n", + " llm: BaseChatModel,\n", + " knowledge_index: VectorStore,\n", + " output_file: str,\n", + " reranker: Optional[RAGPretrainedModel] = None,\n", + " verbose: Optional[bool] = True,\n", + " test_settings: Optional[str] = None, # To document the test settings used\n", + "):\n", + " \"\"\"Runs RAG tests on the given dataset and saves the results to the given output file.\"\"\"\n", + " try: # load previous generations if they exist\n", + " with open(output_file, \"r\") as f:\n", + " outputs = json.load(f)\n", + " except:\n", + " outputs = []\n", + "\n", + " for example in tqdm(eval_dataset):\n", + " question = example[\"question\"]\n", + " if question in [output[\"question\"] for output in outputs]:\n", + " continue\n", + "\n", + " answer, relevant_docs = answer_with_rag(\n", + " question, llm, knowledge_index, reranker=reranker\n", + " )\n", + " if verbose:\n", + " print(\"=======================================================\")\n", + " print(f\"Question: {question}\")\n", + " print(f\"Answer: {answer}\")\n", + " print(f'True answer: {example[\"answer\"]}')\n", + " result = {\n", + " \"question\": question,\n", + " \"true_answer\": example[\"answer\"],\n", + " \"source_doc\": example[\"source_doc\"],\n", + " \"generated_answer\": answer,\n", + " \"retrieved_docs\": [doc for doc in relevant_docs],\n", + " }\n", + " if test_settings:\n", + " result[\"test_settings\"] = test_settings\n", + " outputs.append(result)\n", + "\n", + " with open(output_file, \"w\") as f:\n", + " json.dump(outputs, f)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Ae-3KWzK9jVP" + }, + "outputs": [], + "source": [ + "EVALUATION_PROMPT = \"\"\"###Task Description:\n", + "An instruction (might include an Input inside it), a response to evaluate, a reference answer that gets a score of 5, and a score rubric representing a evaluation criteria are given.\n", + "1. Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.\n", + "2. After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric.\n", + "3. The output format should look as follows: \\\"Feedback: {{write a feedback for criteria}} [RESULT] {{an integer number between 1 and 5}}\\\"\n", + "4. Please do not generate any other opening, closing, and explanations. Be sure to include [RESULT] in your output.\n", + "\n", + "###The instruction to evaluate:\n", + "{instruction}\n", + "\n", + "###Response to evaluate:\n", + "{response}\n", + "\n", + "###Reference Answer (Score 5):\n", + "{reference_answer}\n", + "\n", + "###Score Rubrics:\n", + "[Is the response correct, accurate, and factual based on the reference answer?]\n", + "Score 1: The response is completely incorrect, inaccurate, and/or not factual.\n", + "Score 2: The response is mostly incorrect, inaccurate, and/or not factual.\n", + "Score 3: The response is somewhat correct, accurate, and/or factual.\n", + "Score 4: The response is mostly correct, accurate, and factual.\n", + "Score 5: The response is completely correct, accurate, and factual.\n", + "\n", + "###Feedback:\"\"\"\n", + "\n", + "from langchain.prompts.chat import (\n", + " ChatPromptTemplate,\n", + " HumanMessagePromptTemplate,\n", + ")\n", + "from langchain.schema import SystemMessage\n", + "\n", + "\n", + "evaluation_prompt_template = ChatPromptTemplate.from_messages(\n", + " [\n", + " SystemMessage(content=\"You are a fair evaluator language model.\"),\n", + " HumanMessagePromptTemplate.from_template(EVALUATION_PROMPT),\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ia9Mvn859jVP" + }, + "outputs": [], + "source": [ + "from langchain.chat_models import ChatOpenAI\n", + "\n", + "eval_chat_model = ChatOpenAI(model=\"gpt-4-1106-preview\", temperature=0)\n", + "evaluator_name = \"GPT4\"\n", + "\n", + "\n", + "def evaluate_answers(\n", + " answer_path: str,\n", + " eval_chat_model: BaseChatModel,\n", + " evaluator_name: str,\n", + " evaluation_prompt_template: ChatPromptTemplate,\n", + ") -> None:\n", + " \"\"\"Evaluates generated answers. Modifies the given answer file in place for better checkpointing.\"\"\"\n", + " answers = []\n", + " if os.path.isfile(answer_path): # load previous generations if they exist\n", + " answers = json.load(open(answer_path, \"r\"))\n", + "\n", + " for experiment in tqdm(answers):\n", + " if f\"eval_score_{evaluator_name}\" in experiment:\n", + " continue\n", + "\n", + " eval_prompt = evaluation_prompt_template.format_messages(\n", + " instruction=experiment[\"question\"],\n", + " response=experiment[\"generated_answer\"],\n", + " reference_answer=experiment[\"true_answer\"],\n", + " )\n", + " eval_result = eval_chat_model.invoke(eval_prompt)\n", + " feedback, score = [\n", + " item.strip() for item in eval_result.content.split(\"[RESULT]\")\n", + " ]\n", + " experiment[f\"eval_score_{evaluator_name}\"] = score\n", + " experiment[f\"eval_feedback_{evaluator_name}\"] = feedback\n", + "\n", + " with open(answer_path, \"w\") as f:\n", + " json.dump(answers, f)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "EXH-szLe9jVP" + }, + "source": [ + "🚀 Let's run the tests and evaluate answers!👇" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "jW2nnvUT9jVQ" + }, + "outputs": [], + "source": [ + "if not os.path.exists(\"./output\"):\n", + " os.mkdir(\"./output\")\n", + "\n", + "for chunk_size in [200]: # Add other chunk sizes (in tokens) as needed\n", + " for embeddings in [\"thenlper/gte-small\"]: # Add other embeddings as needed\n", + " for rerank in [True, False]:\n", + " settings_name = f\"chunk:{chunk_size}_embeddings:{embeddings.replace('/', '~')}_rerank:{rerank}_reader-model:{READER_MODEL_NAME}\"\n", + " output_file_name = f\"./output/rag_{settings_name}.json\"\n", + "\n", + " print(f\"Running evaluation for {settings_name}:\")\n", + "\n", + " print(\"Loading knowledge base embeddings...\")\n", + " knowledge_index = load_embeddings(\n", + " RAW_KNOWLEDGE_BASE,\n", + " chunk_size=chunk_size,\n", + " embedding_model_name=embeddings,\n", + " )\n", + "\n", + " print(\"Running RAG...\")\n", + " reranker = (\n", + " RAGPretrainedModel.from_pretrained(\"colbert-ir/colbertv2.0\")\n", + " if rerank\n", + " else None\n", + " )\n", + " run_rag_tests(\n", + " eval_dataset=eval_dataset,\n", + " llm=READER_LLM,\n", + " knowledge_index=knowledge_index,\n", + " output_file=output_file_name,\n", + " reranker=reranker,\n", + " verbose=False,\n", + " test_settings=settings_name,\n", + " )\n", + "\n", + " print(\"Running evaluation...\")\n", + " evaluate_answers(\n", + " output_file_name,\n", + " eval_chat_model,\n", + " evaluator_name,\n", + " evaluation_prompt_template,\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tytXV5-h9jVT" + }, + "source": [ + "### Inspect results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "D4YDSfmr9jVT" + }, + "outputs": [], + "source": [ + "import glob\n", + "\n", + "outputs = []\n", + "for file in glob.glob(\"./output/*.json\"):\n", + " output = pd.DataFrame(json.load(open(file, \"r\")))\n", + " output[\"settings\"] = file\n", + " outputs.append(output)\n", + "result = pd.concat(outputs)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "CdkXMNvS9jVT" + }, + "outputs": [], + "source": [ + "result[\"eval_score_GPT4\"] = result[\"eval_score_GPT4\"].apply(\n", + " lambda x: int(x) if isinstance(x, str) else 1\n", + ")\n", + "result[\"eval_score_GPT4\"] = (result[\"eval_score_GPT4\"] - 1) / 4" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "lgxBpid29jVT", + "outputId": "9a3bcf32-4b0c-4df1-c76c-3ebbca82929d" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "settings\n", + "./output/rag_chunk:200_embeddings:thenlper~gte-small_rerank:False_reader-model:zephyr-7b-beta.json 0.884328\n", + "./output/rag_chunk:200_embeddings:BAAI~bge-base-en-v1.5_rerank:False_reader-model:zephyr-7b-beta.json 0.906716\n", + "./output/rag_chunk:200_embeddings:BAAI~bge-base-en-v1.5_rerank:True_reader-model:zephyr-7b-beta.json 0.906716\n", + "./output/rag_chunk:200_embeddings:thenlper~gte-small_rerank:True_reader-model:mixtral.json 0.906716\n", + "./output/rag_chunk:200_embeddings:thenlper~gte-small_rerank:True_reader-model:zephyr-7b-beta.json 0.921642\n", + "./output/rag_chunk:200_embeddings:thenlper~gte-small_rerank:True_reader-model:mixtral0.json 0.947761\n", + "Name: eval_score_GPT4, dtype: float64" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "average_scores = result.groupby(\"settings\")[\"eval_score_GPT4\"].mean()\n", + "average_scores.sort_values()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "pSPH9DYI9jVT" + }, + "source": [ + "## Example results\n", + "\n", + "Let us load the results that I obtained by tweaking the different options available in this notebook.\n", + "For more detail on why these options could work on not, see the notebook on [advanced_RAG](advanced_rag).\n", + "\n", + "As you can see in the graph below, some tweaks do not bring any improvement, some give huge performance boosts.\n", + "\n", + "➡️ ___There is no single good recipe: you should try several different directions when tuning your RAG systems.___\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "RVOxatv99jVT" + }, + "outputs": [], + "source": [ + "import plotly.express as px\n", + "\n", + "scores = datasets.load_dataset(\"m-ric/rag_scores_cookbook\", split=\"train\")\n", + "scores = pd.Series(scores[\"score\"], index=scores[\"settings\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "vqK0Dg2Q9jVT" + }, + "outputs": [], + "source": [ + "fig = px.bar(\n", + " scores,\n", + " color=scores,\n", + " labels={\n", + " \"value\": \"Accuracy\",\n", + " \"settings\": \"Configuration\",\n", + " },\n", + " color_continuous_scale=\"bluered\",\n", + ")\n", + "fig.update_layout(w\n", + " width=1000,\n", + " height=600,\n", + " barmode=\"group\",\n", + " yaxis_range=[0, 100],\n", + " title=\"Accuracy of different RAG configurations\",\n", + " xaxis_title=\"RAG settings\",\n", + " font=dict(size=15),\n", + ")\n", + "fig.layout.yaxis.ticksuffix = \"%\"\n", + "fig.update_coloraxes(showscale=False)\n", + "fig.update_traces(texttemplate=\"%{y:.1f}\", textposition=\"outside\")\n", + "fig.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dPUOMWGk9jVT" + }, + "source": [ + "\n", + "\n", + "As you can see, these had varying impact on performance. In particular, tuning the chunk size is both easy and very impactful.\n", + "\n", + "But this is our case: your results could be very different: now that you have a robust evaluation pipeline, you can set on to explore other options! 🗺️" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "cookbook2", + "language": "python", + "name": "cookbook2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.0" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/spa/rag_llamaindex_librarian.ipynb b/notebooks/spa/rag_llamaindex_librarian.ipynb new file mode 100644 index 00000000..848172d1 --- /dev/null +++ b/notebooks/spa/rag_llamaindex_librarian.ipynb @@ -0,0 +1,368 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Building A RAG Ebook \"Librarian\" Using LlamaIndex\n", + "\n", + "_Authored by: [Jonathan Jin](https://huggingface.co/jinnovation)_" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Introduction\n", + "\n", + "This notebook demonstrates how to quickly build a RAG-based \"librarian\" for your\n", + "local ebook library.\n", + "\n", + "Think about the last time you visited a library and took advantage of the\n", + "expertise of the knowledgeable staff there to help you find what you need out of\n", + "the troves of textbooks, novels, and other resources at the library. Our RAG\n", + "\"librarian\" will do the same for us, except for our own local collection of\n", + "ebooks.\n", + "\n", + "## Requirements\n", + "\n", + "We'd like our librarian to be **lightweight** and **run locally as much as\n", + "possible** with **minimal dependencies**. This means that we will leverage\n", + "open-source to the fullest extent possible, as well as bias towards models that\n", + "can be **executed locally on typical hardware, e.g. M1 Macbooks**.\n", + "\n", + "## Components\n", + "\n", + "Our solution will consist of the following components:\n", + "\n", + "- [LlamaIndex], a data framework for LLM-based applications that's, unlike\n", + " [LangChain], designed specifically for RAG;\n", + "- [Ollama], a user-friendly solution for running LLMs such as Llama 2 locally;\n", + "- The [`BAAI/bge-base-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5)\n", + " embedding model, which performs [reasonably well and is reasonably lightweight\n", + " in size](https://huggingface.co/spaces/mteb/leaderboard);\n", + "- [Llama 2], which we'll run via [Ollama].\n", + "\n", + "[LlamaIndex]: https://docs.llamaindex.ai/en/stable/index.html\n", + "[LangChain]: https://python.langchain.com/docs/get_started/introduction\n", + "[Ollama]: https://ollama.com/\n", + "[Llama 2]: https://ollama.com/library/llama2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Dependencies\n", + "\n", + "First let's install our dependencies." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -q \\\n", + " llama-index \\\n", + " EbookLib \\\n", + " html2text \\\n", + " llama-index-embeddings-huggingface \\\n", + " llama-index-llms-ollama" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!brew install ollama" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Test Library Setup\n", + "\n", + "Next, let's create our test \"library.\"\n", + "\n", + "For simplicity's sake, let's say that our \"library\" is simply a **nested directory of `.epub` files**. We can easily see this solution generalizing to, say, a Calibre library with a `metadata.db` database file. We'll leave that extension as an exercise for the reader. 😇\n", + "\n", + "Let's pull two `.epub` files from [Project Gutenberg](https://www.gutenberg.org/) for our library." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!mkdir -p \".test/library/jane-austen\"\n", + "!mkdir -p \".test/library/victor-hugo\"\n", + "!wget https://www.gutenberg.org/ebooks/1342.epub.noimages -O \".test/library/jane-austen/pride-and-prejudice.epub\"\n", + "!wget https://www.gutenberg.org/ebooks/135.epub.noimages -O \".test/library/victor-hugo/les-miserables.epub\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## RAG with LlamaIndex\n", + "\n", + "RAG with LlamaIndex, at its core, consists of the following broad phases:\n", + "\n", + "1. **Loading**, in which you tell LlamaIndex where your data lives and how to\n", + " load it;\n", + "2. **Indexing**, in which you augment your loaded data to facilitate querying, e.g. with vector embeddings;\n", + "3. **Querying**, in which you configure an LLM to act as the query interface for\n", + " your indexed data.\n", + "\n", + "This explanation only scratches at the surface of what's possible with\n", + "LlamaIndex. For more in-depth details, I highly recommend reading the\n", + "[\"High-Level Concepts\" page of the LlamaIndex\n", + "documentation](https://docs.llamaindex.ai/en/stable/getting_started/concepts.html)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Loading\n", + "\n", + "Naturally, let's start with the **loading** phase.\n", + "\n", + "I mentioned before that LlamaIndex is designed specifically for RAG. This\n", + "immediately becomes obvious from its\n", + "[`SimpleDirectoryReader`](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html)\n", + "construct, which ✨ **magically** ✨ supports a whole host of multi-model file\n", + "types for free. Conveniently for us, `.epub` is in the supported set." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core import SimpleDirectoryReader\n", + "\n", + "loader = SimpleDirectoryReader(\n", + " input_dir=\"./.test/\",\n", + " recursive=True,\n", + " required_exts=[\".epub\"],\n", + ")\n", + "\n", + "documents = loader.load_data()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`SimpleDirectoryReader.load_data()` converts our ebooks into a set of [`Document`s](https://docs.llamaindex.ai/en/stable/api/llama_index.core.schema.Document.html) for LlamaIndex to work with.\n", + "\n", + "One important thing to note here is that the documents **have not been chunked at this stage** -- that will happen during indexing. Read on..." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Indexing\n", + "\n", + "Next up after **loading** the data is to **index** it. This will allow our RAG pipeline to look up the relevant context for our query to pass to our LLM to **augment** their generated response. This is also where document chunking will take place.\n", + "\n", + "[`VectorStoreIndex`](https://docs.llamaindex.ai/en/stable/module_guides/indexing/vector_store_index.html)\n", + "is a \"default\" entrypoint for indexing in LlamaIndex. By default,\n", + "`VectorStoreIndex` uses a simple, in-memory dictionary to store the indices, but\n", + "LlamaIndex also supports [a wide variety of vector storage\n", + "solutions](https://docs.llamaindex.ai/en/stable/module_guides/storing/vector_stores.html)\n", + "for you to graduate to as you scale.\n", + "\n", + " \n", + "By default, LlamaIndex uses a chunk size of 1024 and a chunk overlap of\n", + "20. For more details, see the [LlamaIndex\n", + "documentation](https://docs.llamaindex.ai/en/stable/optimizing/basic_strategies/basic_strategies.html#chunk-sizes).\n", + "\n", + "\n", + "\n", + "Like mentioned before, we'll use the\n", + "[`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5) to\n", + "generate our embeddings. By default, [LlamaIndex uses\n", + "OpenAI](https://docs.llamaindex.ai/en/stable/getting_started/starter_example.html)\n", + "(specifically `gpt-3.5-turbo`), which we'd like to avoid given our desire for a lightweight, locally-runnable end-to-end solution.\n", + "\n", + "Thankfully, LlamaIndex supports retrieving embedding models from Hugging Face through the convenient `HuggingFaceEmbedding` class, so we'll use that here." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.embeddings.huggingface import HuggingFaceEmbedding\n", + "\n", + "embedding_model = HuggingFaceEmbedding(model_name=\"BAAI/bge-small-en-v1.5\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We'll pass that in to `VectorStoreIndex` as our embedding model to circumvent the OpenAI default behavior." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core import VectorStoreIndex\n", + "\n", + "index = VectorStoreIndex.from_documents(\n", + " documents,\n", + " embed_model=embedding_model,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Querying\n", + "\n", + "Now for the final piece of the RAG puzzle -- wiring up the query layer.\n", + "\n", + "We'll use Llama 2 for the purposes of this recipe, but I encourage readers to play around with different models to see which produces the \"best\" responses here.\n", + "\n", + "First let's start up the Ollama server. Unfortunately, there is no support in the [Ollama Python client](https://github.com/ollama/ollama-python) for actually starting and stopping the server itself, so we'll have to pop out of Python land for this.\n", + "\n", + "In a separate terminal, run: `ollama serve`. Remember to terminate this after we're done here!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's hook Llama 2 up to LlamaIndex and use it as the basis of our query engine." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.llms.ollama import Ollama\n", + "\n", + "llama = Ollama(\n", + " model=\"llama2\",\n", + " request_timeout=40.0,\n", + ")\n", + "\n", + "query_engine = index.as_query_engine(llm=llama)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Final Result\n", + "\n", + "With that, our basic RAG librarian is set up and we can start asking questions about our library. For example:" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Based on the context provided, there are two books available:\n", + "\n", + "1. \"Pride and Prejudice\" by Jane Austen\n", + "2. \"Les Misérables\" by Victor Hugo\n", + "\n", + "The context used to derive this answer includes:\n", + "\n", + "* The file path for each book, which provides information about the location of the book files on the computer.\n", + "* The titles of the books, which are mentioned in the context as being available for reading.\n", + "* A list of words associated with each book, such as \"epub\" and \"notebooks\", which provide additional information about the format and storage location of each book.\n" + ] + } + ], + "source": [ + "print(query_engine.query(\"What are the titles of all the books available? Show me the context used to derive your answer.\"))" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The main character of 'Pride and Prejudice' is Elizabeth Bennet.\n" + ] + } + ], + "source": [ + "print(query_engine.query(\"Who is the main character of 'Pride and Prejudice'?\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion and Future Improvements\n", + "\n", + "We've demonstrated how to build a basic RAG-based \"librarian\" that runs entirely locally, even on Apple silicon Macs. In doing so, we've also carried out a \"grand tour\" of LlamaIndex and how it streamlines the process of setting up RAG-based applications.\n", + "\n", + "That said, we've really only scratched the surface of what's possible here. Here are some ideas of how to refine and build upon this foundation.\n", + "\n", + "### Forcing Citations\n", + "\n", + "To guard against the risk of our librarian hallucinating, how might we require that it provide citations for everything that it says?\n", + "\n", + "### Using Extended Metadata\n", + "\n", + "Ebook library management solutions like [Calibre](https://calibre-ebook.com/) create additional metadata for ebooks in a library. This can provide information such as publisher or edition that might not be readily available in the text of the book itself. How could we extend our RAG pipeline to account for additional sources of information that aren't `.epub` files?\n", + "\n", + "### Efficient Indexing\n", + "\n", + "If we were to collect everything we built here into a script/executable, the resulting script would re-index our library on each invocation. For our tiny test library of two files, this is \"fine,\" but for any library of non-trivial size this will very quickly become annoying for users. How could we persist the embedding indices and only update them when the contents of the library have meaningfully changed, e.g. new books have been added?" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebooks/spa/rag_with_hugging_face_gemma_mongodb.ipynb b/notebooks/spa/rag_with_hugging_face_gemma_mongodb.ipynb new file mode 100644 index 00000000..e40b1462 --- /dev/null +++ b/notebooks/spa/rag_with_hugging_face_gemma_mongodb.ipynb @@ -0,0 +1,4032 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Building A RAG System with Gemma, MongoDB and Open Source Models\n", + "\n", + "Authored By: [Richmond Alake](https://huggingface.co/RichmondMongo)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 1: Installing Libraries\n", + "\n", + "\n", + "The shell command sequence below installs libraries for leveraging open-source large language models (LLMs), embedding models, and database interaction functionalities. These libraries simplify the development of a RAG system, reducing the complexity to a small amount of code:\n", + "\n", + "\n", + "- PyMongo: A Python library for interacting with MongoDB that enables functionalities to connect to a cluster and query data stored in collections and documents.\n", + "- Pandas: Provides a data structure for efficient data processing and analysis using Python\n", + "- Hugging Face datasets: Holds audio, vision, and text datasets\n", + "- Hugging Face Accelerate: Abstracts the complexity of writing code that leverages hardware accelerators such as GPUs. Accelerate is leveraged in the implementation to utilise the Gemma model on GPU resources.\n", + "- Hugging Face Transformers: Access to a vast collection of pre-trained models\n", + "- Hugging Face Sentence Transformers: Provides access to sentence, text, and image embeddings." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "gVSo_nNOUsdn", + "outputId": "907f4738-a3b0-4c0f-b293-eff65c665c07" + }, + "outputs": [], + "source": [ + "!pip install datasets pandas pymongo sentence_transformers\n", + "!pip install -U transformers\n", + "# Install below if using GPU\n", + "!pip install accelerate" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 2: Data sourcing and preparation\n", + "\n", + "\n", + "The data utilised in this tutorial is sourced from Hugging Face datasets, specifically the \n", + "[AIatMongoDB/embedded_movies dataset](https://huggingface.co/datasets/AIatMongoDB/embedded_movies). " + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 747 + }, + "id": "5gCzss27UwWw", + "outputId": "212cca18-a0d7-4289-bce0-ee6259fc2dba" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1500,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.09038552453906,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 139,\n \"samples\": [\n 152.0,\n 127.0,\n 96.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.861995960390892,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1368,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BNWE5MzAwMjQtNzI1YS00YjZhLTkxNDItM2JjNjM3ZjI5NzBjXkEyXkFqcGdeQXVyMTQxNzMzNDI@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTgwNjIyNTczMF5BMl5BanBnXkFtZTcwODI5MDkyMQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1429,\n \"samples\": [\n \"A New York City architect becomes a one-man vigilante squad after his wife is murdered by street punks in which he randomly goes out and kills would-be muggers on the mean streets after dark.\",\n \"As the daring thief Ars\\u00e8ne Lupin (Duris) ransacks the homes of wealthy Parisians, the police, with a secret weapon in their arsenal, attempt to ferret him out.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot_embedding\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1435,\n \"samples\": [\n \"Turbo: A Power Rangers Movie\",\n \"Neon Genesis Evangelion: Death & Rebirth\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "dataset_df" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcastplot_embeddingtitle
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...[0.00072939653, -0.026834568, 0.013515796, -0....The Perils of Pauline
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...[-0.022837115, -0.022941574, 0.014937485, -0.0...From Hand to Mouth
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...[0.00023330493, -0.028511643, 0.014653289, -0....Beau Geste
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...[-0.005927917, -0.033394486, 0.0015323418, -0....The Black Pirate
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...[-0.0059373598, -0.026604708, -0.0070914757, -...For Heaven's Sake
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " num_mflix_comments genres countries \\\n", + "0 0 [Action] [USA] \n", + "1 0 [Comedy, Short, Action] [USA] \n", + "2 0 [Action, Adventure, Drama] [USA] \n", + "3 1 [Adventure, Action] [USA] \n", + "4 0 [Action, Comedy, Romance] [USA] \n", + "\n", + " directors \\\n", + "0 [Louis J. Gasnier, Donald MacKenzie] \n", + "1 [Alfred J. Goulding, Hal Roach] \n", + "2 [Herbert Brenon] \n", + "3 [Albert Parker] \n", + "4 [Sam Taylor] \n", + "\n", + " fullplot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 As a penniless man worries about how he will m... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 A nobleman vows to avenge the death of his fat... \n", + "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", + "\n", + " writers \\\n", + "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", + "1 [H.M. Walker (titles)] \n", + "2 [Herbert Brenon (adaptation), John Russell (ad... \n", + "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", + "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", + "\n", + " awards runtime type rated \\\n", + "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", + "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", + "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", + "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", + "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", + "\n", + " metacritic poster languages \\\n", + "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", + "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", + "2 NaN None [English] \n", + "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", + "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", + "\n", + " imdb \\\n", + "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", + "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", + "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", + "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", + "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", + "\n", + " plot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 A penniless young man tries to save an heiress... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 Seeking revenge, an athletic young man joins t... \n", + "4 An irresponsible young millionaire changes his... \n", + "\n", + " cast \\\n", + "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... \n", + "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... \n", + "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... \n", + "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... \n", + "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... \n", + "\n", + " plot_embedding title \n", + "0 [0.00072939653, -0.026834568, 0.013515796, -0.... The Perils of Pauline \n", + "1 [-0.022837115, -0.022941574, 0.014937485, -0.0... From Hand to Mouth \n", + "2 [0.00023330493, -0.028511643, 0.014653289, -0.... Beau Geste \n", + "3 [-0.005927917, -0.033394486, 0.0015323418, -0.... The Black Pirate \n", + "4 [-0.0059373598, -0.026604708, -0.0070914757, -... For Heaven's Sake " + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Load Dataset\n", + "from datasets import load_dataset\n", + "import pandas as pd\n", + "\n", + "# https://huggingface.co/datasets/AIatMongoDB/embedded_movies\n", + "dataset = load_dataset(\"AIatMongoDB/embedded_movies\")\n", + "\n", + "# Convert the dataset to a pandas dataframe\n", + "dataset_df = pd.DataFrame(dataset[\"train\"])\n", + "\n", + "dataset_df.head(5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The operations within the following code snippet below focus on enforcing data integrity and quality. \n", + "1. The first process ensures that each data point's `fullplot` attribute is not empty, as this is the primary data we utilise in the embedding process. \n", + "2. This step also ensures we remove the `plot_embedding` attribute from all data points as this will be replaced by new embeddings created with a different embedding model, the `gte-large`." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "id": "ARdz6j7SUxqi", + "outputId": "c53c458a-512d-4b7e-93b4-514f6de9d497" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Number of missing values in each column after removal:\n", + "num_mflix_comments 0\n", + "genres 0\n", + "countries 0\n", + "directors 12\n", + "fullplot 0\n", + "writers 13\n", + "awards 0\n", + "runtime 14\n", + "type 0\n", + "rated 279\n", + "metacritic 893\n", + "poster 78\n", + "languages 1\n", + "imdb 0\n", + "plot 0\n", + "cast 1\n", + "plot_embedding 1\n", + "title 0\n", + "dtype: int64\n" + ] + }, + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1452,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.5693352357647,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 137,\n \"samples\": [\n 60.0,\n 151.0,\n 110.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.855402595666057,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1332,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BMTQ2NTMxODEyNV5BMl5BanBnXkFtZTcwMDgxMjA0MQ@@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTY5OTg1ODk0MV5BMl5BanBnXkFtZTcwMTEwMjU1MQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla saves Tokyo from a flying saucer that transforms into the beast Orga.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1391,\n \"samples\": [\n \"Superhero Movie\",\n \"Hooper\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "dataset_df" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcasttitle
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...The Perils of Pauline
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...From Hand to Mouth
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...Beau Geste
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...The Black Pirate
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...For Heaven's Sake
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " num_mflix_comments genres countries \\\n", + "0 0 [Action] [USA] \n", + "1 0 [Comedy, Short, Action] [USA] \n", + "2 0 [Action, Adventure, Drama] [USA] \n", + "3 1 [Adventure, Action] [USA] \n", + "4 0 [Action, Comedy, Romance] [USA] \n", + "\n", + " directors \\\n", + "0 [Louis J. Gasnier, Donald MacKenzie] \n", + "1 [Alfred J. Goulding, Hal Roach] \n", + "2 [Herbert Brenon] \n", + "3 [Albert Parker] \n", + "4 [Sam Taylor] \n", + "\n", + " fullplot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 As a penniless man worries about how he will m... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 A nobleman vows to avenge the death of his fat... \n", + "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", + "\n", + " writers \\\n", + "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", + "1 [H.M. Walker (titles)] \n", + "2 [Herbert Brenon (adaptation), John Russell (ad... \n", + "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", + "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", + "\n", + " awards runtime type rated \\\n", + "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", + "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", + "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", + "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", + "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", + "\n", + " metacritic poster languages \\\n", + "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", + "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", + "2 NaN None [English] \n", + "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", + "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", + "\n", + " imdb \\\n", + "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", + "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", + "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", + "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", + "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", + "\n", + " plot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 A penniless young man tries to save an heiress... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 Seeking revenge, an athletic young man joins t... \n", + "4 An irresponsible young millionaire changes his... \n", + "\n", + " cast title \n", + "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... The Perils of Pauline \n", + "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... From Hand to Mouth \n", + "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... Beau Geste \n", + "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... The Black Pirate \n", + "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... For Heaven's Sake " + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Data Preparation\n", + "\n", + "# Remove data point where plot coloumn is missing\n", + "dataset_df = dataset_df.dropna(subset=[\"fullplot\"])\n", + "print(\"\\nNumber of missing values in each column after removal:\")\n", + "print(dataset_df.isnull().sum())\n", + "\n", + "# Remove the plot_embedding from each data point in the dataset as we are going to create new embeddings with an open source embedding model from Hugging Face\n", + "dataset_df = dataset_df.drop(columns=[\"plot_embedding\"])\n", + "dataset_df.head(5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 3: Generating embeddings\n", + "\n", + "**The steps in the code snippets are as follows:**\n", + "1. Import the `SentenceTransformer` class to access the embedding models.\n", + "2. Load the embedding model using the `SentenceTransformer` constructor to instantiate the `gte-large` embedding model.\n", + "3. Define the `get_embedding` function, which takes a text string as input and returns a list of floats representing the embedding. The function first checks if the input text is not empty (after stripping whitespace). If the text is empty, it returns an empty list. Otherwise, it generates an embedding using the loaded model.\n", + "4. Generate embeddings by applying the `get_embedding` function to the \"fullplot\" column of the `dataset_df` DataFrame, generating embeddings for each movie's plot. The resulting list of embeddings is assigned to a new column named embedding.\n", + "\n", + "*Note: It's not necessary to chunk the text in the full plot, as we can ensure that the text length remains within a manageable range.*\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 747 + }, + "id": "ZX8zJNN5UzPK", + "outputId": "81bc1a57-7d96-4311-ba94-4748c34c20e3" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1452,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.5693352357647,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 137,\n \"samples\": [\n 60.0,\n 151.0,\n 110.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.855402595666057,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1332,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BMTQ2NTMxODEyNV5BMl5BanBnXkFtZTcwMDgxMjA0MQ@@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTY5OTg1ODk0MV5BMl5BanBnXkFtZTcwMTEwMjU1MQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla saves Tokyo from a flying saucer that transforms into the beast Orga.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1391,\n \"samples\": [\n \"Superhero Movie\",\n \"Hooper\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"embedding\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "dataset_df" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcasttitleembedding
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...The Perils of Pauline[-0.009285838343203068, -0.005062104668468237,...
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...From Hand to Mouth[-0.0024393785279244184, 0.02309592440724373, ...
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...Beau Geste[0.012204292230308056, -0.01145575474947691, -...
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...The Black Pirate[0.004541348200291395, -0.0006100579630583525,...
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...For Heaven's Sake[-0.0022256041411310434, 0.011567804962396622,...
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " num_mflix_comments genres countries \\\n", + "0 0 [Action] [USA] \n", + "1 0 [Comedy, Short, Action] [USA] \n", + "2 0 [Action, Adventure, Drama] [USA] \n", + "3 1 [Adventure, Action] [USA] \n", + "4 0 [Action, Comedy, Romance] [USA] \n", + "\n", + " directors \\\n", + "0 [Louis J. Gasnier, Donald MacKenzie] \n", + "1 [Alfred J. Goulding, Hal Roach] \n", + "2 [Herbert Brenon] \n", + "3 [Albert Parker] \n", + "4 [Sam Taylor] \n", + "\n", + " fullplot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 As a penniless man worries about how he will m... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 A nobleman vows to avenge the death of his fat... \n", + "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", + "\n", + " writers \\\n", + "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", + "1 [H.M. Walker (titles)] \n", + "2 [Herbert Brenon (adaptation), John Russell (ad... \n", + "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", + "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", + "\n", + " awards runtime type rated \\\n", + "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", + "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", + "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", + "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", + "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", + "\n", + " metacritic poster languages \\\n", + "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", + "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", + "2 NaN None [English] \n", + "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", + "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", + "\n", + " imdb \\\n", + "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", + "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", + "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", + "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", + "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", + "\n", + " plot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 A penniless young man tries to save an heiress... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 Seeking revenge, an athletic young man joins t... \n", + "4 An irresponsible young millionaire changes his... \n", + "\n", + " cast title \\\n", + "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... The Perils of Pauline \n", + "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... From Hand to Mouth \n", + "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... Beau Geste \n", + "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... The Black Pirate \n", + "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... For Heaven's Sake \n", + "\n", + " embedding \n", + "0 [-0.009285838343203068, -0.005062104668468237,... \n", + "1 [-0.0024393785279244184, 0.02309592440724373, ... \n", + "2 [0.012204292230308056, -0.01145575474947691, -... \n", + "3 [0.004541348200291395, -0.0006100579630583525,... \n", + "4 [-0.0022256041411310434, 0.011567804962396622,... " + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sentence_transformers import SentenceTransformer\n", + "\n", + "# https://huggingface.co/thenlper/gte-large\n", + "embedding_model = SentenceTransformer(\"thenlper/gte-large\")\n", + "\n", + "\n", + "def get_embedding(text: str) -> list[float]:\n", + " if not text.strip():\n", + " print(\"Attempted to get embedding for empty text.\")\n", + " return []\n", + "\n", + " embedding = embedding_model.encode(text)\n", + "\n", + " return embedding.tolist()\n", + "\n", + "\n", + "dataset_df[\"embedding\"] = dataset_df[\"fullplot\"].apply(get_embedding)\n", + "\n", + "dataset_df.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 4: Database setup and connection\n", + "\n", + "MongoDB acts as both an operational and a vector database. It offers a database solution that efficiently stores, queries and retrieves vector embeddings—the advantages of this lie in the simplicity of database maintenance, management and cost.\n", + "\n", + "**To create a new MongoDB database, set up a database cluster:**\n", + "\n", + "1. Head over to MongoDB official site and register for a [free MongoDB Atlas account](https://www.mongodb.com/cloud/atlas/register?utm_campaign=devrel&utm_source=community&utm_medium=cta&utm_content=Partner%20Cookbook&utm_term=richmond.alake), or for existing users, [sign into MongoDB Atlas](https://account.mongodb.com/account/login?utm_campaign=devrel&utm_source=community&utm_medium=cta&utm_content=Partner%20Cookbook&utm_term=richmond.alakee).\n", + "\n", + "2. Select the 'Database' option on the left-hand pane, which will navigate to the Database Deployment page, where there is a deployment specification of any existing cluster. Create a new database cluster by clicking on the \"+Create\" button.\n", + "\n", + "3. Select all the applicable configurations for the database cluster. Once all the configuration options are selected, click the “Create Cluster” button to deploy the newly created cluster. MongoDB also enables the creation of free clusters on the “Shared Tab”.\n", + "\n", + " *Note: Don’t forget to whitelist the IP for the Python host or 0.0.0.0/0 for any IP when creating proof of concepts.*\n", + "\n", + "4. After successfully creating and deploying the cluster, the cluster becomes accessible on the ‘Database Deployment’ page.\n", + "\n", + "5. Click on the “Connect” button of the cluster to view the option to set up a connection to the cluster via various language drivers.\n", + "\n", + "6. This tutorial only requires the cluster's URI(unique resource identifier). Grab the URI and copy it into the Google Colabs Secrets environment in a variable named `MONGO_URI` or place it in a .env file or equivalent.\n", + "\n", + "\n", + "### 4.1 Database and Collection Setup\n", + "\n", + "Before moving forward, ensure the following prerequisites are met\n", + "- Database cluster set up on MongoDB Atlas\n", + "- Obtained the URI to your cluster\n", + "\n", + "For assistance with database cluster setup and obtaining the URI, refer to our guide for [setting up a MongoDB cluster](https://www.mongodb.com/docs/guides/atlas/cluster/) and [getting your connection string](https://www.mongodb.com/docs/guides/atlas/connection-string/)\n", + "\n", + "Once you have created a cluster, create the database and collection within the MongoDB Atlas cluster by clicking + Create Database in the cluster overview page. \n", + "\n", + "Here is a guide for [creating a database and collection](https://www.mongodb.com/basics/create-database)\n", + "\n", + "**The database will be named `movies`.**\n", + "\n", + "**The collection will be named `movie_collection_2`.**\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "## Step 5: Create a Vector Search Index\n", + "\n", + "At this point make sure that your vector index is created via MongoDB Atlas.\n", + "\n", + "This next step is mandatory for conducting efficient and accurate vector-based searches based on the vector embeddings stored within the documents in the `movie_collection_2` collection. \n", + "\n", + "Creating a Vector Search Index enables the ability to traverse the documents efficiently to retrieve documents with embeddings that match the query embedding based on vector similarity. \n", + "\n", + "Go here to read more about [MongoDB Vector Search Index](https://www.mongodb.com/docs/atlas/atlas-search/field-types/knn-vector/).\n", + "\n", + "\n", + "```\n", + "{\n", + " \"fields\": [{\n", + " \"numDimensions\": 1024,\n", + " \"path\": \"embedding\",\n", + " \"similarity\": \"cosine\",\n", + " \"type\": \"vector\"\n", + " }]\n", + "}\n", + "\n", + "```\n", + "\n", + "The `1024` value of the numDimension field corresponds to the dimension of the vector generated by the gte-large embedding model. If you use the `gte-base` or `gte-small` embedding models, the numDimension value in the vector search index must be set to 768 and 384, respectively.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 6: Establish Data Connection\n", + "\n", + "The code snippet below also utilises PyMongo to create a MongoDB client object, representing the connection to the cluster and enabling access to its databases and collections.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Oi0l9POtU0iP", + "outputId": "d3fe3cc4-8c08-4435-ddfc-8cfcc5ada572" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Connection to MongoDB successful\n" + ] + } + ], + "source": [ + "import pymongo\n", + "from google.colab import userdata\n", + "\n", + "\n", + "def get_mongo_client(mongo_uri):\n", + " \"\"\"Establish connection to the MongoDB.\"\"\"\n", + " try:\n", + " client = pymongo.MongoClient(mongo_uri)\n", + " print(\"Connection to MongoDB successful\")\n", + " return client\n", + " except pymongo.errors.ConnectionFailure as e:\n", + " print(f\"Connection failed: {e}\")\n", + " return None\n", + "\n", + "\n", + "mongo_uri = userdata.get(\"MONGO_URI\")\n", + "if not mongo_uri:\n", + " print(\"MONGO_URI not set in environment variables\")\n", + "\n", + "mongo_client = get_mongo_client(mongo_uri)\n", + "\n", + "# Ingest data into MongoDB\n", + "db = mongo_client[\"movies\"]\n", + "collection = db[\"movie_collection_2\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "F7XXXa-OU1u9", + "outputId": "7bd1eb43-e933-4150-990a-fa20bad84e9a" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "DeleteResult({'n': 1452, 'electionId': ObjectId('7fffffff000000000000000c'), 'opTime': {'ts': Timestamp(1708554945, 1452), 't': 12}, 'ok': 1.0, '$clusterTime': {'clusterTime': Timestamp(1708554945, 1452), 'signature': {'hash': b'\\x99\\x89\\xc0\\x00Cn!\\xd6\\xaf\\xb3\\x96\\xdf\\xc3\\xda\\x88\\x11\\xf5\\t\\xbd\\xc0', 'keyId': 7320226449804230661}}, 'operationTime': Timestamp(1708554945, 1452)}, acknowledged=True)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Delete any existing records in the collection\n", + "collection.delete_many({})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Ingesting data into a MongoDB collection from a pandas DataFrame is a straightforward process that can be efficiently accomplished by converting the DataFrame into dictionaries and then utilising the `insert_many` method on the collection to pass the converted dataset records.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "XrfMY4QBU2-l", + "outputId": "e2b5c534-2ba0-4ffa-bca8-1e96bef14c54" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Data ingestion into MongoDB completed\n" + ] + } + ], + "source": [ + "documents = dataset_df.to_dict(\"records\")\n", + "collection.insert_many(documents)\n", + "\n", + "print(\"Data ingestion into MongoDB completed\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 7: Perform Vector Search on User Queries\n", + "\n", + "The following step implements a function that returns a vector search result by generating a query embedding and defining a MongoDB aggregation pipeline. \n", + "\n", + "The pipeline, consisting of the `$vectorSearch` and `$project` stages, executes queries using the generated vector and formats the results to include only the required information, such as plot, title, and genres while incorporating a search score for each result." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "id": "kWucnQBEU35k" + }, + "outputs": [], + "source": [ + "def vector_search(user_query, collection):\n", + " \"\"\"\n", + " Perform a vector search in the MongoDB collection based on the user query.\n", + "\n", + " Args:\n", + " user_query (str): The user's query string.\n", + " collection (MongoCollection): The MongoDB collection to search.\n", + "\n", + " Returns:\n", + " list: A list of matching documents.\n", + " \"\"\"\n", + "\n", + " # Generate embedding for the user query\n", + " query_embedding = get_embedding(user_query)\n", + "\n", + " if query_embedding is None:\n", + " return \"Invalid query or embedding generation failed.\"\n", + "\n", + " # Define the vector search pipeline\n", + " pipeline = [\n", + " {\n", + " \"$vectorSearch\": {\n", + " \"index\": \"vector_index\",\n", + " \"queryVector\": query_embedding,\n", + " \"path\": \"embedding\",\n", + " \"numCandidates\": 150, # Number of candidate matches to consider\n", + " \"limit\": 4, # Return top 4 matches\n", + " }\n", + " },\n", + " {\n", + " \"$project\": {\n", + " \"_id\": 0, # Exclude the _id field\n", + " \"fullplot\": 1, # Include the plot field\n", + " \"title\": 1, # Include the title field\n", + " \"genres\": 1, # Include the genres field\n", + " \"score\": {\"$meta\": \"vectorSearchScore\"}, # Include the search score\n", + " }\n", + " },\n", + " ]\n", + "\n", + " # Execute the search\n", + " results = collection.aggregate(pipeline)\n", + " return list(results)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Step 8: Handling user queries and loading Gemma\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "0ka4WLTmU5L4" + }, + "outputs": [], + "source": [ + "def get_search_result(query, collection):\n", + "\n", + " get_knowledge = vector_search(query, collection)\n", + "\n", + " search_result = \"\"\n", + " for result in get_knowledge:\n", + " search_result += f\"Title: {result.get('title', 'N/A')}, Plot: {result.get('fullplot', 'N/A')}\\n\"\n", + "\n", + " return search_result" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Z4L4SfueU6PY", + "outputId": "11ea30ca-8cac-4e4c-9ab6-780e043c6345" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Query: What is the best romantic movie to watch and why?\n", + "Continue to answer the query by using the Search Results:\n", + "Title: Shut Up and Kiss Me!, Plot: Ryan and Pete are 27-year old best friends in Miami, born on the same day and each searching for the perfect woman. Ryan is a rookie stockbroker living with his psychic Mom. Pete is a slick surfer dude yet to find commitment. Each meets the women of their dreams on the same day. Ryan knocks heads in an elevator with the gorgeous Jessica, passing out before getting her number. Pete falls for the insatiable Tiara, but Tiara's uncle is mob boss Vincent Bublione, charged with her protection. This high-energy romantic comedy asks to what extent will you go for true love?\n", + "Title: Pearl Harbor, Plot: Pearl Harbor is a classic tale of romance set during a war that complicates everything. It all starts when childhood friends Rafe and Danny become Army Air Corps pilots and meet Evelyn, a Navy nurse. Rafe falls head over heels and next thing you know Evelyn and Rafe are hooking up. Then Rafe volunteers to go fight in Britain and Evelyn and Danny get transferred to Pearl Harbor. While Rafe is off fighting everything gets completely whack and next thing you know everybody is in the middle of an air raid we now know as \"Pearl Harbor.\"\n", + "Title: Titanic, Plot: The plot focuses on the romances of two couples upon the doomed ship's maiden voyage. Isabella Paradine (Catherine Zeta-Jones) is a wealthy woman mourning the loss of her aunt, who reignites a romance with former flame Wynn Park (Peter Gallagher). Meanwhile, a charming ne'er-do-well named Jamie Perse (Mike Doyle) steals a ticket for the ship, and falls for a sweet innocent Irish girl on board. But their romance is threatened by the villainous Simon Doonan (Tim Curry), who has discovered about the ticket and makes Jamie his unwilling accomplice, as well as having sinister plans for the girl.\n", + "Title: China Girl, Plot: A modern day Romeo & Juliet story is told in New York when an Italian boy and a Chinese girl become lovers, causing a tragic conflict between ethnic gangs.\n", + ".\n" + ] + } + ], + "source": [ + "# Conduct query with retrival of sources\n", + "query = \"What is the best romantic movie to watch and why?\"\n", + "source_information = get_search_result(query, collection)\n", + "combined_information = f\"Query: {query}\\nContinue to answer the query by using the Search Results:\\n{source_information}.\"\n", + "\n", + "print(combined_information)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 209, + "referenced_widgets": [ + "60c4d6d5e7a84fa493f101cc47dadef9", + "fa0c528cca744cff8da0a4fa21fdb4b5", + "d7d4a9f444fe4ebb9135035e2166a3a5", + "4e62b9ec821348cc94b34cfbc010c2a4", + "9d9a247c6569458abd0dcd6e0d717079", + "e5a6d300bbf441b8904aa9afb89e6f31", + "88226abe35534278bbd427d8eff0f5f8", + "2e081a17ddc04104882893a30c902265", + "938f8f60901442f2902eb51e86c27961", + "ce6a3a655d2f4ce2ab351c766568bed5", + "19fd13ad5b2740aa8be2a7d62488fdaf", + "c7c0c34a71954d6ea976c774573c49c5", + "0d6ec3bab579406fa4e6fc2b3d6b6998", + "a37f2164e11d4e5f851a4a09a12c663c", + "ef32431228f24a5498810a36b9cf6506", + "c06e354cb8294e66a3d7590a576571e0", + "e2998c2c6b1f4d489a5e39f2076838e4", + "4300755179d9465db871b14ae78dabc6", + "f14106c7f60f411199acf47f530443fd", + "bf88ee6dc83648d8a61d75bb4466b1e3", + "5776e818d9d34e009f95833056522876", + "06b1a069317041c8a9174c14fdc867bc", + "0e27bfa4f64f427d9996de0451e9edd9", + "d5e9f339fe7e4ab9955531cc125f071e", + "fa9cf3e72280417d8711ef7227a95d34", + "c3a1b520140444fbb40b7ac789f7ac0e", + "2c84bc5c158641f49f421a7d28da1518", + "6a15f1cf54a141fc9d6bb790366c6bdd", + "8813b56cd89744b58ace2787206e1501", + "edc37210db734d01a8afce596698bb27", + "eba6048eb694485693656fcbf4a4f297", + "30885be6a7c84f0f9f02bc2ea11679bc", + "29178e51df9e47489fff623763b130ed", + "5266bebcf8bb4b0798b14831a38e2a8c", + "7c638aaf734c423fbe54daddff97040f", + "4c6736981923464db2f754339c60cd0d", + "57383c03fc854a92a2ff732cbdd80a70", + "8a302ae0412b4393a17b50861afe36b5", + "b2fdc502d6ee491bb826fd616e35d407", + "2677891ce891475b8dc7c2ae287c58d7", + "fddbae43ce7f477cafaff89b81e47fc7", + "592d069be51e43f99212d47be9c11dcf", + "9a4c90a767c746659ea535d7c36d40a5", + "43fcf04b360f4a75be9fb99ab69fbe38", + "b7c439aa6d584c5784b46980050e503d", + "8aa8805651d34181b1851d302ccc47e2", + "713f1d91e445411288e565a33ce4b271", + "55941e08c602404c9342d00b7ee26918", + "87da02f5606d404ea242c3bd1f9ac38c", + "947f9b7e19dc4be4bd21b1b021e91f9d", + "0b7f3d233b8f4912bef4deae2e395001", + "6ccbd7b9ae924b5e843efd3114dfb2c5", + "9e0bccbc6072461fbf96482b870ed8d5", + "d7a00f1f114e4f008f4d5a48c1c69f53", + "faf25fd219f24bdbaa2e3202548c97d9", + "a0996675df13484aaa519e6ff45c5476", + "0bfb4937ed5547b3ba464ca47ac77f1a", + "7f59906980724a8b840dec85ce400f89", + "80f3d29327bf429481ad191b1abe556f", + "6d7c024126ac4c34825fae522234ebca", + "a0600fb407034c2d8df6ae5830d601db", + "c1d37ab1952b4d268d9786b74b6902d7", + "e7f471604a5a42e095d35d8ad399c6fe", + "feb438afda6b4c148a3a62ee7e03da74", + "e68cf53b04a845ac9d6f4047600ebc21", + "33fef11f829f49e2aa9555201d4a0e42" + ] + }, + "id": "OYGmKVv9mm8g", + "outputId": "ff41bfed-daa0-4ed8-8cc4-0aa138e697a1" + }, + "outputs": [], + "source": [ + "from transformers import AutoTokenizer, AutoModelForCausalLM\n", + "\n", + "tokenizer = AutoTokenizer.from_pretrained(\"google/gemma-2b-it\")\n", + "# CPU Enabled uncomment below 👇🏽\n", + "# model = AutoModelForCausalLM.from_pretrained(\"google/gemma-2b-it\")\n", + "# GPU Enabled use below 👇🏽\n", + "model = AutoModelForCausalLM.from_pretrained(\"google/gemma-2b-it\", device_map=\"auto\")" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "wDA9SdXhsFyM", + "outputId": "c3300fa5-586c-48bd-9abb-b12a4390a294" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Query: What is the best romantic movie to watch and why?\n", + "Continue to answer the query by using the Search Results:\n", + "Title: Shut Up and Kiss Me!, Plot: Ryan and Pete are 27-year old best friends in Miami, born on the same day and each searching for the perfect woman. Ryan is a rookie stockbroker living with his psychic Mom. Pete is a slick surfer dude yet to find commitment. Each meets the women of their dreams on the same day. Ryan knocks heads in an elevator with the gorgeous Jessica, passing out before getting her number. Pete falls for the insatiable Tiara, but Tiara's uncle is mob boss Vincent Bublione, charged with her protection. This high-energy romantic comedy asks to what extent will you go for true love?\n", + "Title: Pearl Harbor, Plot: Pearl Harbor is a classic tale of romance set during a war that complicates everything. It all starts when childhood friends Rafe and Danny become Army Air Corps pilots and meet Evelyn, a Navy nurse. Rafe falls head over heels and next thing you know Evelyn and Rafe are hooking up. Then Rafe volunteers to go fight in Britain and Evelyn and Danny get transferred to Pearl Harbor. While Rafe is off fighting everything gets completely whack and next thing you know everybody is in the middle of an air raid we now know as \"Pearl Harbor.\"\n", + "Title: Titanic, Plot: The plot focuses on the romances of two couples upon the doomed ship's maiden voyage. Isabella Paradine (Catherine Zeta-Jones) is a wealthy woman mourning the loss of her aunt, who reignites a romance with former flame Wynn Park (Peter Gallagher). Meanwhile, a charming ne'er-do-well named Jamie Perse (Mike Doyle) steals a ticket for the ship, and falls for a sweet innocent Irish girl on board. But their romance is threatened by the villainous Simon Doonan (Tim Curry), who has discovered about the ticket and makes Jamie his unwilling accomplice, as well as having sinister plans for the girl.\n", + "Title: China Girl, Plot: A modern day Romeo & Juliet story is told in New York when an Italian boy and a Chinese girl become lovers, causing a tragic conflict between ethnic gangs.\n", + ".\n", + "\n", + "Based on the search results, the best romantic movie to watch is **Shut Up and Kiss Me!** because it is a romantic comedy that explores the complexities of love and relationships. The movie is funny, heartwarming, and thought-provoking.\n" + ] + } + ], + "source": [ + "# Moving tensors to GPU\n", + "input_ids = tokenizer(combined_information, return_tensors=\"pt\").to(\"cuda\")\n", + "response = model.generate(**input_ids, max_new_tokens=500)\n", + "print(tokenizer.decode(response[0]))" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "id": "FhMmFmUBwBcy" + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "A100", + "machine_shape": "hm", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "06b1a069317041c8a9174c14fdc867bc": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0b7f3d233b8f4912bef4deae2e395001": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0bfb4937ed5547b3ba464ca47ac77f1a": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_a0600fb407034c2d8df6ae5830d601db", + "placeholder": "​", + "style": "IPY_MODEL_c1d37ab1952b4d268d9786b74b6902d7", + "value": "generation_config.json: 100%" + } + }, + "0d6ec3bab579406fa4e6fc2b3d6b6998": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e2998c2c6b1f4d489a5e39f2076838e4", + "placeholder": "​", + "style": "IPY_MODEL_4300755179d9465db871b14ae78dabc6", + "value": "Downloading shards: 100%" + } + }, + "0e27bfa4f64f427d9996de0451e9edd9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_d5e9f339fe7e4ab9955531cc125f071e", + "IPY_MODEL_fa9cf3e72280417d8711ef7227a95d34", + "IPY_MODEL_c3a1b520140444fbb40b7ac789f7ac0e" + ], + "layout": "IPY_MODEL_2c84bc5c158641f49f421a7d28da1518" + } + }, + "19fd13ad5b2740aa8be2a7d62488fdaf": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2677891ce891475b8dc7c2ae287c58d7": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "29178e51df9e47489fff623763b130ed": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2c84bc5c158641f49f421a7d28da1518": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2e081a17ddc04104882893a30c902265": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "30885be6a7c84f0f9f02bc2ea11679bc": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "33fef11f829f49e2aa9555201d4a0e42": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "4300755179d9465db871b14ae78dabc6": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "43fcf04b360f4a75be9fb99ab69fbe38": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "4c6736981923464db2f754339c60cd0d": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_fddbae43ce7f477cafaff89b81e47fc7", + "max": 67121608, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_592d069be51e43f99212d47be9c11dcf", + "value": 67121608 + } + }, + "4e62b9ec821348cc94b34cfbc010c2a4": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ce6a3a655d2f4ce2ab351c766568bed5", + "placeholder": "​", + "style": "IPY_MODEL_19fd13ad5b2740aa8be2a7d62488fdaf", + "value": " 13.5k/13.5k [00:00<00:00, 1.10MB/s]" + } + }, + "5266bebcf8bb4b0798b14831a38e2a8c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_7c638aaf734c423fbe54daddff97040f", + "IPY_MODEL_4c6736981923464db2f754339c60cd0d", + "IPY_MODEL_57383c03fc854a92a2ff732cbdd80a70" + ], + "layout": "IPY_MODEL_8a302ae0412b4393a17b50861afe36b5" + } + }, + "55941e08c602404c9342d00b7ee26918": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_d7a00f1f114e4f008f4d5a48c1c69f53", + "placeholder": "​", + "style": "IPY_MODEL_faf25fd219f24bdbaa2e3202548c97d9", + "value": " 2/2 [00:04<00:00,  1.94s/it]" + } + }, + "57383c03fc854a92a2ff732cbdd80a70": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_9a4c90a767c746659ea535d7c36d40a5", + "placeholder": "​", + "style": "IPY_MODEL_43fcf04b360f4a75be9fb99ab69fbe38", + "value": " 67.1M/67.1M [00:00<00:00, 465MB/s]" + } + }, + "5776e818d9d34e009f95833056522876": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "592d069be51e43f99212d47be9c11dcf": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "60c4d6d5e7a84fa493f101cc47dadef9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_fa0c528cca744cff8da0a4fa21fdb4b5", + "IPY_MODEL_d7d4a9f444fe4ebb9135035e2166a3a5", + "IPY_MODEL_4e62b9ec821348cc94b34cfbc010c2a4" + ], + "layout": "IPY_MODEL_9d9a247c6569458abd0dcd6e0d717079" + } + }, + "6a15f1cf54a141fc9d6bb790366c6bdd": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6ccbd7b9ae924b5e843efd3114dfb2c5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6d7c024126ac4c34825fae522234ebca": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "713f1d91e445411288e565a33ce4b271": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_6ccbd7b9ae924b5e843efd3114dfb2c5", + "max": 2, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_9e0bccbc6072461fbf96482b870ed8d5", + "value": 2 + } + }, + "7c638aaf734c423fbe54daddff97040f": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_b2fdc502d6ee491bb826fd616e35d407", + "placeholder": "​", + "style": "IPY_MODEL_2677891ce891475b8dc7c2ae287c58d7", + "value": "model-00002-of-00002.safetensors: 100%" + } + }, + "7f59906980724a8b840dec85ce400f89": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e7f471604a5a42e095d35d8ad399c6fe", + "max": 137, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_feb438afda6b4c148a3a62ee7e03da74", + "value": 137 + } + }, + "80f3d29327bf429481ad191b1abe556f": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e68cf53b04a845ac9d6f4047600ebc21", + "placeholder": "​", + "style": "IPY_MODEL_33fef11f829f49e2aa9555201d4a0e42", + "value": " 137/137 [00:00<00:00, 11.9kB/s]" + } + }, + "87da02f5606d404ea242c3bd1f9ac38c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8813b56cd89744b58ace2787206e1501": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "88226abe35534278bbd427d8eff0f5f8": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "8a302ae0412b4393a17b50861afe36b5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8aa8805651d34181b1851d302ccc47e2": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_947f9b7e19dc4be4bd21b1b021e91f9d", + "placeholder": "​", + "style": "IPY_MODEL_0b7f3d233b8f4912bef4deae2e395001", + "value": "Loading checkpoint shards: 100%" + } + }, + "938f8f60901442f2902eb51e86c27961": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "947f9b7e19dc4be4bd21b1b021e91f9d": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9a4c90a767c746659ea535d7c36d40a5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9d9a247c6569458abd0dcd6e0d717079": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9e0bccbc6072461fbf96482b870ed8d5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "a0600fb407034c2d8df6ae5830d601db": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a0996675df13484aaa519e6ff45c5476": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_0bfb4937ed5547b3ba464ca47ac77f1a", + "IPY_MODEL_7f59906980724a8b840dec85ce400f89", + "IPY_MODEL_80f3d29327bf429481ad191b1abe556f" + ], + "layout": "IPY_MODEL_6d7c024126ac4c34825fae522234ebca" + } + }, + "a37f2164e11d4e5f851a4a09a12c663c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_f14106c7f60f411199acf47f530443fd", + "max": 2, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_bf88ee6dc83648d8a61d75bb4466b1e3", + "value": 2 + } + }, + "b2fdc502d6ee491bb826fd616e35d407": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "b7c439aa6d584c5784b46980050e503d": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_8aa8805651d34181b1851d302ccc47e2", + "IPY_MODEL_713f1d91e445411288e565a33ce4b271", + "IPY_MODEL_55941e08c602404c9342d00b7ee26918" + ], + "layout": "IPY_MODEL_87da02f5606d404ea242c3bd1f9ac38c" + } + }, + "bf88ee6dc83648d8a61d75bb4466b1e3": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "c06e354cb8294e66a3d7590a576571e0": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c1d37ab1952b4d268d9786b74b6902d7": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "c3a1b520140444fbb40b7ac789f7ac0e": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_30885be6a7c84f0f9f02bc2ea11679bc", + "placeholder": "​", + "style": "IPY_MODEL_29178e51df9e47489fff623763b130ed", + "value": " 4.95G/4.95G [00:16<00:00, 216MB/s]" + } + }, + "c7c0c34a71954d6ea976c774573c49c5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_0d6ec3bab579406fa4e6fc2b3d6b6998", + "IPY_MODEL_a37f2164e11d4e5f851a4a09a12c663c", + "IPY_MODEL_ef32431228f24a5498810a36b9cf6506" + ], + "layout": "IPY_MODEL_c06e354cb8294e66a3d7590a576571e0" + } + }, + "ce6a3a655d2f4ce2ab351c766568bed5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d5e9f339fe7e4ab9955531cc125f071e": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_6a15f1cf54a141fc9d6bb790366c6bdd", + "placeholder": "​", + "style": "IPY_MODEL_8813b56cd89744b58ace2787206e1501", + "value": "model-00001-of-00002.safetensors: 100%" + } + }, + "d7a00f1f114e4f008f4d5a48c1c69f53": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d7d4a9f444fe4ebb9135035e2166a3a5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2e081a17ddc04104882893a30c902265", + "max": 13489, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_938f8f60901442f2902eb51e86c27961", + "value": 13489 + } + }, + "e2998c2c6b1f4d489a5e39f2076838e4": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e5a6d300bbf441b8904aa9afb89e6f31": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e68cf53b04a845ac9d6f4047600ebc21": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e7f471604a5a42e095d35d8ad399c6fe": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "eba6048eb694485693656fcbf4a4f297": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "edc37210db734d01a8afce596698bb27": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ef32431228f24a5498810a36b9cf6506": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5776e818d9d34e009f95833056522876", + "placeholder": "​", + "style": "IPY_MODEL_06b1a069317041c8a9174c14fdc867bc", + "value": " 2/2 [00:17<00:00,  7.35s/it]" + } + }, + "f14106c7f60f411199acf47f530443fd": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fa0c528cca744cff8da0a4fa21fdb4b5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e5a6d300bbf441b8904aa9afb89e6f31", + "placeholder": "​", + "style": "IPY_MODEL_88226abe35534278bbd427d8eff0f5f8", + "value": "model.safetensors.index.json: 100%" + } + }, + "fa9cf3e72280417d8711ef7227a95d34": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_edc37210db734d01a8afce596698bb27", + "max": 4945242264, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_eba6048eb694485693656fcbf4a4f297", + "value": 4945242264 + } + }, + "faf25fd219f24bdbaa2e3202548c97d9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "fddbae43ce7f477cafaff89b81e47fc7": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "feb438afda6b4c148a3a62ee7e03da74": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/spa/rag_zephyr_langchain.ipynb b/notebooks/spa/rag_zephyr_langchain.ipynb new file mode 100644 index 00000000..55738b98 --- /dev/null +++ b/notebooks/spa/rag_zephyr_langchain.ipynb @@ -0,0 +1,519 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "Kih21u1tyr-I" + }, + "source": [ + "# Simple RAG for GitHub issues using Hugging Face Zephyr and LangChain\n", + "\n", + "_Authored by: [Maria Khalusova](https://github.com/MKhalusova)_\n", + "\n", + "This notebook demonstrates how you can quickly build a RAG (Retrieval Augmented Generation) for a project's GitHub issues using [`HuggingFaceH4/zephyr-7b-beta`](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) model, and LangChain.\n", + "\n", + "\n", + "**What is RAG?**\n", + "\n", + "RAG is a popular approach to address the issue of a powerful LLM not being aware of specific content due to said content not being in its training data, or hallucinating even when it has seen it before. Such specific content may be proprietary, sensitive, or, as in this example, recent and updated often.\n", + "\n", + "If your data is static and doesn't change regularly, you may consider fine-tuning a large model. In many cases, however, fine-tuning can be costly, and, when done repeatedly (e.g. to address data drift), leads to \"model shift\". This is when the model's behavior changes in ways that are not desirable.\n", + "\n", + "**RAG (Retrieval Augmented Generation)** does not require model fine-tuning. Instead, RAG works by providing an LLM with additional context that is retrieved from relevant data so that it can generate a better-informed response.\n", + "\n", + "Here's a quick illustration:\n", + "\n", + "![RAG diagram](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/rag-diagram.png)\n", + "\n", + "* The external data is converted into embedding vectors with a separate embeddings model, and the vectors are kept in a database. Embeddings models are typically small, so updating the embedding vectors on a regular basis is faster, cheaper, and easier than fine-tuning a model.\n", + "\n", + "* At the same time, the fact that fine-tuning is not required gives you the freedom to swap your LLM for a more powerful one when it becomes available, or switch to a smaller distilled version, should you need faster inference.\n", + "\n", + "Let's illustrate building a RAG using an open-source LLM, embeddings model, and LangChain.\n", + "\n", + "First, install the required dependencies:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "lC9frDOlyi38" + }, + "outputs": [], + "source": [ + "!pip install -q torch transformers accelerate bitsandbytes transformers sentence-transformers faiss-gpu" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "-aYENQwZ-p_c" + }, + "outputs": [], + "source": [ + "# If running in Google Colab, you may need to run this cell to make sure you're using UTF-8 locale to install LangChain\n", + "import locale\n", + "locale.getpreferredencoding = lambda: \"UTF-8\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "W5HhMZ2c-NfU" + }, + "outputs": [], + "source": [ + "!pip install -q langchain" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "R8po01vMWzXL" + }, + "source": [ + "## Prepare the data\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3cCmQywC04x6" + }, + "source": [ + "In this example, we'll load all of the issues (both open and closed) from [PEFT library's repo](https://github.com/huggingface/peft).\n", + "\n", + "First, you need to acquire a [GitHub personal access token](https://github.com/settings/tokens?type=beta) to access the GitHub API." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "8MoD7NbsNjlM" + }, + "outputs": [], + "source": [ + "from getpass import getpass\n", + "ACCESS_TOKEN = getpass(\"YOUR_GITHUB_PERSONAL_TOKEN\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "fccecm3a10N6" + }, + "source": [ + "Next, we'll load all of the issues in the [huggingface/peft](https://github.com/huggingface/peft) repo:\n", + "- By default, pull requests are considered issues as well, here we chose to exclude them from data with by setting `include_prs=False`\n", + "- Setting `state = \"all\"` means we will load both open and closed issues." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "id": "8EKMit4WNDY8" + }, + "outputs": [], + "source": [ + "from langchain.document_loaders import GitHubIssuesLoader\n", + "\n", + "loader = GitHubIssuesLoader(\n", + " repo=\"huggingface/peft\",\n", + " access_token=ACCESS_TOKEN,\n", + " include_prs=False,\n", + " state=\"all\"\n", + ")\n", + "\n", + "docs = loader.load()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CChTrY-k2qO5" + }, + "source": [ + "The content of individual GitHub issues may be longer than what an embedding model can take as input. If we want to embed all of the available content, we need to chunk the documents into appropriately sized pieces.\n", + "\n", + "The most common and straightforward approach to chunking is to define a fixed size of chunks and whether there should be any overlap between them. Keeping some overlap between chunks allows us to preserve some semantic context between the chunks. The recommended splitter for generic text is the [RecursiveCharacterTextSplitter](https://python.langchain.com/docs/modules/data_connection/document_transformers/recursive_text_splitter), and that's what we'll use here. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "OmsXOf59Pmm-" + }, + "outputs": [], + "source": [ + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "\n", + "splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=30)\n", + "\n", + "chunked_docs = splitter.split_documents(docs)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DAt_zPVlXOn7" + }, + "source": [ + "## Create the embeddings + retriever" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-mvat6JQl4yp" + }, + "source": [ + "Now that the docs are all of the appropriate size, we can create a database with their embeddings.\n", + "\n", + "To create document chunk embeddings we'll use the `HuggingFaceEmbeddings` and the [`BAAI/bge-base-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5) embeddings model. There are many other embeddings models available on the Hub, and you can keep an eye on the best performing ones by checking the [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard).\n", + "\n", + "\n", + "To create the vector database, we'll use `FAISS`, a library developed by Facebook AI. This library offers efficient similarity search and clustering of dense vectors, which is what we need here. FAISS is currently one of the most used libraries for NN search in massive datasets.\n", + "\n", + "We'll access both the embeddings model and FAISS via LangChain API." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ixmCdRzBQ5gu" + }, + "outputs": [], + "source": [ + "from langchain.vectorstores import FAISS\n", + "from langchain.embeddings import HuggingFaceEmbeddings\n", + "\n", + "db = FAISS.from_documents(chunked_docs,\n", + " HuggingFaceEmbeddings(model_name='BAAI/bge-base-en-v1.5'))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2iCgEPi0nnN6" + }, + "source": [ + "We need a way to return(retrieve) the documents given an unstructured query. For that, we'll use the `as_retriever` method using the `db` as a backbone:\n", + "- `search_type=\"similarity\"` means we want to perform similarity search between the query and documents\n", + "- `search_kwargs={'k': 4}` instructs the retriever to return top 4 results.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "id": "mBTreCQ9noHK" + }, + "outputs": [], + "source": [ + "retriever = db.as_retriever(\n", + " search_type=\"similarity\",\n", + " search_kwargs={'k': 4}\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WgEhlISJpTgj" + }, + "source": [ + "The vector database and retriever are now set up, next we need to set up the next piece of the chain - the model." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tzQxx0HkXVFU" + }, + "source": [ + "## Load quantized model" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9jy1cC65p_GD" + }, + "source": [ + "For this example, we chose [`HuggingFaceH4/zephyr-7b-beta`](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta), a small but powerful model.\n", + "\n", + "With many models being released every week, you may want to substitute this model to the latest and greatest. The best way to keep track of open source LLMs is to check the [Open-source LLM leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n", + "\n", + "To make inference faster, we will load the quantized version of the model:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "L-ggaa763VRo" + }, + "outputs": [], + "source": [ + "import torch\n", + "from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n", + "\n", + "model_name = 'HuggingFaceH4/zephyr-7b-beta'\n", + "\n", + "bnb_config = BitsAndBytesConfig(\n", + " load_in_4bit=True,\n", + " bnb_4bit_use_double_quant=True,\n", + " bnb_4bit_quant_type=\"nf4\",\n", + " bnb_4bit_compute_dtype=torch.bfloat16\n", + ")\n", + "\n", + "model = AutoModelForCausalLM.from_pretrained(model_name, quantization_config=bnb_config)\n", + "tokenizer = AutoTokenizer.from_pretrained(model_name)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hVNRJALyXYHG" + }, + "source": [ + "## Setup the LLM chain" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "RUUNneJ1smhl" + }, + "source": [ + "Finally, we have all the pieces we need to set up the LLM chain.\n", + "\n", + "First, create a text_generation pipeline using the loaded model and its tokenizer.\n", + "\n", + "Next, create a prompt template - this should follow the format of the model, so if you substitute the model checkpoint, make sure to use the appropriate formatting." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "id": "cR0k1cRWz8Pm" + }, + "outputs": [], + "source": [ + "from langchain.llms import HuggingFacePipeline\n", + "from langchain.prompts import PromptTemplate\n", + "from transformers import pipeline\n", + "from langchain_core.output_parsers import StrOutputParser\n", + "\n", + "text_generation_pipeline = pipeline(\n", + " model=model,\n", + " tokenizer=tokenizer,\n", + " task=\"text-generation\",\n", + " temperature=0.2,\n", + " do_sample=True,\n", + " repetition_penalty=1.1,\n", + " return_full_text=True,\n", + " max_new_tokens=400,\n", + ")\n", + "\n", + "llm = HuggingFacePipeline(pipeline=text_generation_pipeline)\n", + "\n", + "prompt_template = \"\"\"\n", + "<|system|>\n", + "Answer the question based on your knowledge. Use the following context to help:\n", + "\n", + "{context}\n", + "\n", + "\n", + "<|user|>\n", + "{question}\n", + "\n", + "<|assistant|>\n", + "\n", + " \"\"\"\n", + "\n", + "prompt = PromptTemplate(\n", + " input_variables=[\"context\", \"question\"],\n", + " template=prompt_template,\n", + ")\n", + "\n", + "llm_chain = prompt | llm | StrOutputParser()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "l19UKq5HXfSp" + }, + "source": [ + "Note: _You can also use `tokenizer.apply_chat_template` to convert a list of messages (as dicts: `{'role': 'user', 'content': '(...)'}`) into a string with the appropriate chat format._\n", + "\n", + "\n", + "Finally, we need to combine the `llm_chain` with the retriever to create a RAG chain. We pass the original question through to the final generation step, as well as the retrieved context docs:" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "id": "_rI3YNp9Xl4s" + }, + "outputs": [], + "source": [ + "from langchain_core.runnables import RunnablePassthrough\n", + "\n", + "retriever = db.as_retriever()\n", + "\n", + "rag_chain = (\n", + " {\"context\": retriever, \"question\": RunnablePassthrough()}\n", + " | llm_chain\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UsCOhfDDXpaS" + }, + "source": [ + "## Compare the results\n", + "\n", + "Let's see the difference RAG makes in generating answers to the library-specific questions." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "id": "W7F07fQLXusU" + }, + "outputs": [], + "source": [ + "question = \"How do you combine multiple adapters?\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KC0rJYU1x1ir" + }, + "source": [ + "First, let's see what kind of answer we can get with just the model itself, no context added:" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 125 + }, + "id": "GYh-HG1l0De5", + "outputId": "277d8e89-ce9b-4e04-c11b-639ad2645759" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "\" To combine multiple adapters, you need to ensure that they are compatible with each other and the devices you want to connect. Here's how you can do it:\\n\\n1. Identify the adapters you need: Determine which adapters you require to connect the devices you want to use together. For example, if you want to connect a USB-C device to an HDMI monitor, you may need a USB-C to HDMI adapter and a USB-C to USB-A adapter (if your computer only has USB-A ports).\\n\\n2. Connect the first adapter: Plug in the first adapter into the device you want to connect. For instance, if you're connecting a USB-C laptop to an HDMI monitor, plug the USB-C to HDMI adapter into the laptop's USB-C port.\\n\\n3. Connect the second adapter: Next, connect the second adapter to the first one. In this case, connect the USB-C to USB-A adapter to the USB-C port of the USB-C to HDMI adapter.\\n\\n4. Connect the final device: Finally, connect the device you want to use to the second adapter. For example, connect the HDMI cable from the monitor to the HDMI port on the USB-C to HDMI adapter.\\n\\n5. Test the connection: Turn on both devices and check whether everything is working correctly. If necessary, adjust the settings on your devices to ensure optimal performance.\\n\\nBy combining multiple adapters, you can connect a variety of devices together, even if they don't have the same type of connector. Just be sure to choose adapters that are compatible with all the devices you want to connect and test the connection thoroughly before relying on it for critical tasks.\"" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "llm_chain.invoke({\"context\":\"\", \"question\": question})" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "i-TIWr3wx9w8" + }, + "source": [ + "As you can see, the model interpreted the question as one about physical computer adapters, while in the context of PEFT, \"adapters\" refer to LoRA adapters.\n", + "Let's see if adding context from GitHub issues helps the model give a more relevant answer:" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 125 + }, + "id": "FZpNA3o10H10", + "outputId": "31f9aed3-3dd7-4ff8-d1a8-866794fefe80" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "\" Based on the provided context, it seems that combining multiple adapters is still an open question in the community. Here are some possibilities:\\n\\n 1. Save the output from the base model and pass it to each adapter separately, as described in the first context snippet. This allows you to run multiple adapters simultaneously and reuse the output from the base model. However, this approach requires loading and running each adapter separately.\\n\\n 2. Export everything into a single PyTorch model, as suggested in the second context snippet. This would involve saving all the adapters and their weights into a single model, potentially making it larger and more complex. The advantage of this approach is that it would allow you to run all the adapters simultaneously without having to load and run them separately.\\n\\n 3. Merge multiple Lora adapters, as mentioned in the third context snippet. This involves adding multiple distinct, independent behaviors to a base model by merging multiple Lora adapters. It's not clear from the context how this would be done, but it suggests that there might be a recommended way of doing it.\\n\\n 4. Combine adapters through a specific architecture, as proposed in the fourth context snippet. This involves merging multiple adapters into a single architecture, potentially creating a more complex model with multiple behaviors. Again, it's not clear from the context how this would be done.\\n\\n Overall, combining multiple adapters is still an active area of research, and there doesn't seem to be a widely accepted solution yet. If you're interested in exploring this further, it might be worth reaching out to the Hugging Face community or checking out their documentation for more information.\"" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "rag_chain.invoke(question)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hZQedZKSyrwO" + }, + "source": [ + "As we can see, the added context, really helps the exact same model, provide a much more relevant and informed answer to the library-specific question.\n", + "\n", + "Notably, combining multiple adapters for inference has been added to the library, and one can find this information in the documentation, so for the next iteration of this RAG it may be worth including documentation embeddings." + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/spa/semantic_cache_chroma_vector_database.ipynb b/notebooks/spa/semantic_cache_chroma_vector_database.ipynb new file mode 100644 index 00000000..885e4a00 --- /dev/null +++ b/notebooks/spa/semantic_cache_chroma_vector_database.ipynb @@ -0,0 +1,1432 @@ +{ + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.10.12", + "mimetype": "text/x-python", + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "pygments_lexer": "ipython3", + "nbconvert_exporter": "python", + "file_extension": ".py" + }, + "kaggle": { + "accelerator": "gpu", + "dataSources": [ + { + "sourceId": 6104553, + "sourceType": "datasetVersion", + "datasetId": 3496946 + } + ], + "dockerImageVersionId": 30527, + "isInternetEnabled": true, + "language": "python", + "sourceType": "notebook", + "isGpuEnabled": true + }, + "colab": { + "provenance": [], + "machine_shape": "hm", + "gpuType": "T4" + }, + "accelerator": "GPU" + }, + "nbformat_minor": 0, + "nbformat": 4, + "cells": [ + { + "cell_type": "markdown", + "source": [ + "# Implementing semantic cache to improve a RAG system with FAISS.\n", + "\n", + "_Authored by:[Pere Martra](https://github.com/peremartra)_\n", + "\n", + "In this notebook, we will explore a typical RAG solution where we will utilize an open-source model and the vector database Chroma DB. **However, we will integrate a semantic cache system that will store various user queries and decide whether to generate the prompt enriched with information from the vector database or the cache.**\n", + "\n", + "A semantic caching system aims to identify similar or identical user requests. When a matching request is found, the system retrieves the corresponding information from the cache, reducing the need to fetch it from the original source.\n", + "\n", + "As the comparison takes into account the semantic meaning of the requests, they don't have to be identical for the system to recognize them as the same question. They can be formulated differently or contain inaccuracies, be they typographical or in the sentence structure, and we can identify that the user is actually requesting the same information.\n", + "\n", + "For instance, queries like **What is the capital of France?**, **Tell me the name of the capital of France?**, and **What The capital of France is?** all convey the same intent and should be identified as the same question.\n", + "\n", + "While the model's response may differ based on the request for a concise answer in the second example, the information retrieved from the vector database should be the same. This is why I'm placing the cache system between the user and the vector database, not between the user and the Large Language Model.\n", + "\n", + "\n", + "\n" + ], + "metadata": { + "id": "AVv_M1Dz9TDz" + } + }, + { + "cell_type": "markdown", + "source": [ + "Most tutorials that guide you through creating a RAG system are designed for single-user use, meant to operate in a testing environment. In other words, within a notebook, interacting with a local vector database and making API calls or using a locally stored model.\n", + "\n", + "This architecture quickly becomes insufficient when attempting to transition one of these models to production, where they might encounter from tens to thousands of recurrent requests.\n", + "\n", + "One way to enhance performance is through one or multiple semantic caches. This cache retains the results of previous requests, and before resolving a new request, it checks if a similar one has been received before. If so, instead of re-executing the process, it retrieves the information from the cache.\n", + "\n", + "In a RAG system, there are two points that are time consuming:\n", + "* Retrieve the information used to construct the enriched prompt:\n", + "* Call the Large Language Model to obtain the response.\n", + "\n", + "In both points, a semantic cache system can be implemented, and we could even have two caches, one for each point.\n", + "\n", + "Placing it at the model's response point may lead to a loss of influence over the obtained response. Our cache system could consider \"Explain the French Revolution in 10 words\" and \"Explain the French Revolution in a hundred words\" as the same query. If our cache system stores model responses, users might think that their instructions are not being followed accurately.\n", + "\n", + "But both requests will require the same information to enrich the prompt. This is the main reason why I chose to place the semantic cache system between the user's request and the retrieval of information from the vector database.\n", + "\n", + "However, this is a design decision. Depending on the type of responses and system requests, it can be placed at one point or another. It's evident that caching model responses would yield the most time savings, but as I've already explained, it comes at the cost of losing user influence over the response.\n" + ], + "metadata": { + "id": "5gtBERjX1vFd" + } + }, + { + "cell_type": "markdown", + "source": [ + "# Import and load the libraries.\n", + "To start we need to install the necesary Python packages.\n", + "* **[sentence transformers](http:/www.sbert.net/)**. This library is necessary to transform the sentences into fixed-length vectors, also know as embeddings.\n", + "* **[xformers](https://github.com/facebookresearch/xformers)**. it's a package that provides libraries an utilities to facilitate the work with transformers models. We need to install in order to avoid an error when we work with the model and embeddings. \n", + "* **[chromadb](https://www.trychroma.com/)**. This is our vector Database. ChromaDB is easy to use and open source, maybe the most used Vector Database used to store embeddings.\n", + "* **[accelerate](https://github.com/huggingface/accelerate)** Necesary to run the Model in a GPU. " + ], + "metadata": { + "id": "uizxY8679TDz" + } + }, + { + "cell_type": "code", + "source": [ + "!pip install -q transformers==4.38.1\n", + "!pip install -q accelerate==0.27.2\n", + "!pip install -q sentence-transformers==2.5.1\n", + "!pip install -q xformers==0.0.24\n", + "!pip install -q chromadb==0.4.24\n", + "!pip install -q datasets==2.17.1" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:30:10.787382Z", + "iopub.execute_input": "2024-02-29T17:30:10.787688Z", + "iopub.status.idle": "2024-02-29T17:34:12.804579Z", + "shell.execute_reply.started": "2024-02-29T17:30:10.787657Z", + "shell.execute_reply": "2024-02-29T17:34:12.80338Z" + }, + "trusted": true, + "id": "r1nUzd1u9TD0" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "import numpy as np\n", + "import pandas as pd" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:35:23.197205Z", + "iopub.execute_input": "2024-02-29T17:35:23.197598Z", + "iopub.status.idle": "2024-02-29T17:35:23.202259Z", + "shell.execute_reply.started": "2024-02-29T17:35:23.197556Z", + "shell.execute_reply": "2024-02-29T17:35:23.201404Z" + }, + "trusted": true, + "id": "5jUwC_eE9TD0" + }, + "execution_count": 2, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "# Load the Dataset\n", + "As we are working in a free and limited space, and we can use just a few GB of memory I limited the number of rows to use from the Dataset with the variable `MAX_ROWS`." + ], + "metadata": { + "id": "9P-kYtc79TD1" + } + }, + { + "cell_type": "code", + "source": [ + "#Login to Hugging Face. It is mandatory to use the Gemma Model,\n", + "#and recommended to acces public models and Datasets.\n", + "from getpass import getpass\n", + "if 'hf_key' not in locals():\n", + " hf_key = getpass(\"Your Hugging Face API Key: \")\n", + "!huggingface-cli login --token $hf_key" + ], + "metadata": { + "id": "xZsN8yzUvfjN" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "from datasets import load_dataset\n", + "\n", + "data = load_dataset(\"keivalya/MedQuad-MedicalQnADataset\", split='train')" + ], + "metadata": { + "id": "9IVxu-uxtCTw" + }, + "execution_count": 47, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "ChromaDB requires that the data has a unique identifier. We can make it with this statement, which will create a new column called **Id**.\n" + ], + "metadata": { + "id": "hmor-i1j9TD1" + } + }, + { + "cell_type": "code", + "source": [ + "data = data.to_pandas()\n", + "data[\"id\"]=data.index\n", + "data.head(10)" + ], + "metadata": { + "id": "WbLf8c7_yHwy", + "outputId": "492eac81-2f7b-4063-f444-405bf489d08e", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 536 + } + }, + "execution_count": 48, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + " qtype Question \\\n", + "0 susceptibility Who is at risk for Lymphocytic Choriomeningiti... \n", + "1 symptoms What are the symptoms of Lymphocytic Choriomen... \n", + "2 susceptibility Who is at risk for Lymphocytic Choriomeningiti... \n", + "3 exams and tests How to diagnose Lymphocytic Choriomeningitis (... \n", + "4 treatment What are the treatments for Lymphocytic Chorio... \n", + "5 prevention How to prevent Lymphocytic Choriomeningitis (L... \n", + "6 information What is (are) Parasites - Cysticercosis ? \n", + "7 susceptibility Who is at risk for Parasites - Cysticercosis? ? \n", + "8 exams and tests How to diagnose Parasites - Cysticercosis ? \n", + "9 treatment What are the treatments for Parasites - Cystic... \n", + "\n", + " Answer id \n", + "0 LCMV infections can occur after exposure to fr... 0 \n", + "1 LCMV is most commonly recognized as causing ne... 1 \n", + "2 Individuals of all ages who come into contact ... 2 \n", + "3 During the first phase of the disease, the mos... 3 \n", + "4 Aseptic meningitis, encephalitis, or meningoen... 4 \n", + "5 LCMV infection can be prevented by avoiding co... 5 \n", + "6 Cysticercosis is an infection caused by the la... 6 \n", + "7 Cysticercosis is an infection caused by the la... 7 \n", + "8 If you think that you may have cysticercosis, ... 8 \n", + "9 Some people with cysticercosis do not need to ... 9 " + ], + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
qtypeQuestionAnswerid
0susceptibilityWho is at risk for Lymphocytic Choriomeningiti...LCMV infections can occur after exposure to fr...0
1symptomsWhat are the symptoms of Lymphocytic Choriomen...LCMV is most commonly recognized as causing ne...1
2susceptibilityWho is at risk for Lymphocytic Choriomeningiti...Individuals of all ages who come into contact ...2
3exams and testsHow to diagnose Lymphocytic Choriomeningitis (...During the first phase of the disease, the mos...3
4treatmentWhat are the treatments for Lymphocytic Chorio...Aseptic meningitis, encephalitis, or meningoen...4
5preventionHow to prevent Lymphocytic Choriomeningitis (L...LCMV infection can be prevented by avoiding co...5
6informationWhat is (are) Parasites - Cysticercosis ?Cysticercosis is an infection caused by the la...6
7susceptibilityWho is at risk for Parasites - Cysticercosis? ?Cysticercosis is an infection caused by the la...7
8exams and testsHow to diagnose Parasites - Cysticercosis ?If you think that you may have cysticercosis, ...8
9treatmentWhat are the treatments for Parasites - Cystic...Some people with cysticercosis do not need to ...9
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "dataframe", + "variable_name": "data", + "summary": "{\n \"name\": \"data\",\n \"rows\": 16407,\n \"fields\": [\n {\n \"column\": \"qtype\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 16,\n \"samples\": [\n \"susceptibility\",\n \"symptoms\",\n \"information\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Question\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 14979,\n \"samples\": [\n \"What are the symptoms of Danon disease ?\",\n \"What is (are) Dowling-Degos disease ?\",\n \"What are the genetic changes related to Pearson marrow-pancreas syndrome ?\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Answer\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 15817,\n \"samples\": [\n \"These resources address the diagnosis or management of glycogen storage disease type III: - Gene Review: Gene Review: Glycogen Storage Disease Type III - Genetic Testing Registry: Glycogen storage disease type III These resources from MedlinePlus offer information about the diagnosis and management of various health conditions: - Diagnostic Tests - Drug Therapy - Surgery and Rehabilitation - Genetic Counseling - Palliative Care\",\n \"Diagnostic Challenges\\n \\nFor doctors, diagnosing chronic fatigue syndrome (CFS) can be complicated by a number of factors:\\n \\n - There's no lab test or biomarker for CFS.\\n - Fatigue and other symptoms of CFS are common to many illnesses.\\n - For some CFS patients, it may not be obvious to doctors that they are ill.\\n - The illness has a pattern of remission and relapse.\\n - Symptoms vary from person to person in type, number, and severity.\\n \\n \\nThese factors have contributed to a low diagnosis rate. Of the one to four million Americans who have CFS, less than 20% have been diagnosed.\\n Exams and Screening Tests for CFS\\n \\nBecause there is no blood test, brain scan, or other lab test to diagnose CFS, the doctor should first rule out other possible causes.\\n \\nIf a patient has had 6 or more consecutive months of severe fatigue that is reported to be unrelieved by sufficient bed rest and that is accompanied by nonspecific symptoms, including flu-like symptoms, generalized pain, and memory problems, the doctor should consider the possibility that the patient may have CFS. Further exams and tests are needed before a diagnosis can be made:\\n \\n - A detailed medical history will be needed and should include a review of medications that could be causing the fatigue and symptoms\\n - A thorough physical and mental status examination will also be needed\\n - A battery of laboratory screening tests will be needed to help identify or rule out other possible causes of the symptoms that could be treated\\n - The doctor may also order additional tests to follow up on results of the initial screening tests\\n \\n \\nA CFS diagnosis requires that the patient has been fatigued for 6 months or more and has 4 of the 8 symptoms for CFS for 6 months or more. If, however, the patient has been fatigued for 6 months or more but does not have four of the eight symptoms, the diagnosis may be idiopathic fatigue.\\n \\nThe complete process for diagnosing CFS can be found here.\\n \\nAdditional information for healthcare professionals on use of tests can be found here.\",\n \"Eating, diet, and nutrition have not been shown to play a role in causing or preventing simple kidney cysts.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"id\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 4736,\n \"min\": 0,\n \"max\": 16406,\n \"num_unique_values\": 16407,\n \"samples\": [\n 3634,\n 15104,\n 4395\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}" + } + }, + "metadata": {}, + "execution_count": 48 + } + ] + }, + { + "cell_type": "code", + "source": [ + "MAX_ROWS = 15000\n", + "DOCUMENT=\"Answer\"\n", + "TOPIC=\"qtype\"" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:35:25.527688Z", + "iopub.execute_input": "2024-02-29T17:35:25.528374Z", + "iopub.status.idle": "2024-02-29T17:35:25.709895Z", + "shell.execute_reply.started": "2024-02-29T17:35:25.528341Z", + "shell.execute_reply": "2024-02-29T17:35:25.709127Z" + }, + "trusted": true, + "id": "DZf0zCI29TD1" + }, + "execution_count": 6, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "#Because it is just a sample we select a small portion of News.\n", + "subset_data = data.head(MAX_ROWS)" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:35:29.183979Z", + "iopub.execute_input": "2024-02-29T17:35:29.184342Z", + "iopub.status.idle": "2024-02-29T17:35:29.189229Z", + "shell.execute_reply.started": "2024-02-29T17:35:29.184313Z", + "shell.execute_reply": "2024-02-29T17:35:29.1881Z" + }, + "trusted": true, + "id": "Mkoj9IrZ9TD1" + }, + "execution_count": 7, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "# Import and configure the Vector Database\n", + "To store the information, I've chosen to use ChromaDB, one of the most well-known and widely used open-source vector databases.\n", + "\n", + "First we need to import ChromaDB." + ], + "metadata": { + "id": "rZHg_Qh69TD1" + } + }, + { + "cell_type": "code", + "source": [ + "import chromadb" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:35:31.849199Z", + "iopub.execute_input": "2024-02-29T17:35:31.849551Z", + "iopub.status.idle": "2024-02-29T17:35:32.31736Z", + "shell.execute_reply.started": "2024-02-29T17:35:31.849525Z", + "shell.execute_reply": "2024-02-29T17:35:32.316617Z" + }, + "trusted": true, + "id": "npJhuZQw9TD1" + }, + "execution_count": 8, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "Now we only need to indicate the path where the vector database will be stored." + ], + "metadata": { + "id": "8okox5C89TD1" + } + }, + { + "cell_type": "code", + "source": [ + "chroma_client = chromadb.PersistentClient(path=\"/path/to/persist/directory\")" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:35:34.410268Z", + "iopub.execute_input": "2024-02-29T17:35:34.410646Z", + "iopub.status.idle": "2024-02-29T17:35:34.872817Z", + "shell.execute_reply.started": "2024-02-29T17:35:34.410614Z", + "shell.execute_reply": "2024-02-29T17:35:34.872039Z" + }, + "trusted": true, + "id": "9yK6y0hm9TD1" + }, + "execution_count": 9, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "# Filling and Querying the ChromaDB Database\n", + "The Data in ChromaDB is stored in collections. If the collection exist we need to delete it.\n", + "\n", + "In the next lines, we are creating the collection by calling the `create_collection` function in the `chroma_client` created above." + ], + "metadata": { + "id": "7MhMwk3J9TD1" + } + }, + { + "cell_type": "code", + "source": [ + "collection_name = \"news_collection\"\n", + "if len(chroma_client.list_collections()) > 0 and collection_name in [chroma_client.list_collections()[0].name]:\n", + " chroma_client.delete_collection(name=collection_name)\n", + "\n", + "collection = chroma_client.create_collection(name=collection_name)\n" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:35:36.1156Z", + "iopub.execute_input": "2024-02-29T17:35:36.116012Z", + "iopub.status.idle": "2024-02-29T17:35:36.16922Z", + "shell.execute_reply.started": "2024-02-29T17:35:36.115977Z", + "shell.execute_reply": "2024-02-29T17:35:36.168504Z" + }, + "trusted": true, + "id": "kRCsunE19TD1" + }, + "execution_count": 10, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "We are now ready to add the data to the collection using the `add` function. This function requires three key pieces of information:\n", + "\n", + "* In the **document** we store the content of the `Answer` column in the Dataset.\n", + "* In **metadatas**, we can inform a list of topics. I used the value in the column `qtype`.\n", + "* In **id** we need to inform an unique identificator for each row. I'm creating the ID using the range of `MAX_ROWS`.\n" + ], + "metadata": { + "id": "rdEtcETr9TD2" + } + }, + { + "cell_type": "code", + "source": [ + "collection.add(\n", + " documents=subset_data[DOCUMENT].tolist(),\n", + " metadatas=[{TOPIC: topic} for topic in subset_data[TOPIC].tolist()],\n", + " ids=[f\"id{x}\" for x in range(MAX_ROWS)],\n", + ")" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:35:38.051179Z", + "iopub.execute_input": "2024-02-29T17:35:38.051601Z", + "iopub.status.idle": "2024-02-29T17:36:38.612836Z", + "shell.execute_reply.started": "2024-02-29T17:35:38.051569Z", + "shell.execute_reply": "2024-02-29T17:36:38.611814Z" + }, + "trusted": true, + "id": "4dDoqJE79TD2", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "36f579dc-ec60-48b1-807a-1e68113cc9f4" + }, + "execution_count": 11, + "outputs": [ + { + "metadata": { + "tags": null + }, + "name": "stderr", + "output_type": "stream", + "text": [ + "/root/.cache/chroma/onnx_models/all-MiniLM-L6-v2/onnx.tar.gz: 100%|██████████| 79.3M/79.3M [00:01<00:00, 68.1MiB/s]\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "Once we have the information in the Database we can query it, and ask for data that matches our needs. The search is done inside the content of the document, and it dosn't look for the exact word, or phrase. The results will be based on the similarity between the search terms and the content of documents.\n", + "\n", + "Metadata isn't directly involved in the initial search process, it can be used to filter or refine the results after retrieval, enabling further customization and precision.\n", + "\n", + "Let's define a function to query the ChromaDB Database." + ], + "metadata": { + "id": "du6-iuUisRkM" + } + }, + { + "cell_type": "code", + "source": [ + "def query_database(query_text, n_results=10):\n", + " results = collection.query(query_texts=query_text, n_results=n_results )\n", + " return results" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:36:38.615302Z", + "iopub.execute_input": "2024-02-29T17:36:38.616047Z", + "iopub.status.idle": "2024-02-29T17:36:38.620516Z", + "shell.execute_reply.started": "2024-02-29T17:36:38.616008Z", + "shell.execute_reply": "2024-02-29T17:36:38.619561Z" + }, + "trusted": true, + "id": "UjdhZ4MJ9TD2" + }, + "execution_count": 12, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "## Creating the semantic cache system\n", + "To implement the cache system, we will use Faiss, a library that allows storing embeddings in memory. It's quite similar to what Chroma does, but without its persistence.\n", + "\n", + "For this purpose, we will create a class called `semantic_cache` that will work with its own encoder and provide the necessary functions for the user to perform queries.\n", + "\n", + "In this class, we first query the cache implemented with Faiss, that contains the previous petitions, and if the returned results are above a specified threshold, it will return the content of the cache. Otherwise, it will fetch the result from the Chroma database.\n", + "\n", + "The cache is stored in a .json file." + ], + "metadata": { + "id": "CL0Crl3x9TD2" + } + }, + { + "cell_type": "code", + "source": [ + "!pip install -q faiss-cpu==1.8.0" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:36:38.621655Z", + "iopub.execute_input": "2024-02-29T17:36:38.621968Z", + "iopub.status.idle": "2024-02-29T17:36:51.313356Z", + "shell.execute_reply.started": "2024-02-29T17:36:38.621936Z", + "shell.execute_reply": "2024-02-29T17:36:51.312232Z" + }, + "trusted": true, + "id": "6OzUbRUe9TD2" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "import faiss\n", + "from sentence_transformers import SentenceTransformer\n", + "import time\n", + "import json" + ], + "metadata": { + "id": "0yGE4cTEp3QJ" + }, + "execution_count": 14, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "The `init_cache()` function below initializes the semantic cache.\n", + "\n", + "It employs the FlatLS index, which might not be the fastest but is ideal for small datasets. Depending on the characteristics of the data intended for the cache and the expected dataset size, another index such as HNSW or IVF could be utilized.\n", + "\n", + "I chose this index because it aligns well with the example. It can be used with vectors of high dimensions, consumes minimal memory, and performs well with small datasets.\n", + "\n", + "I outline the key features of the various indices available with Faiss.\n", + "\n", + "* FlatL2 or FlatIP. Well-suited for small datasets, it may not be the fastest, but its memory consumption is not excessive.\n", + "* LSH. It works effectively with small datasets and is recommended for use with vectors of up to 128 dimensions.\n", + "* HNSW. Very fast but demands a substantial amount of RAM.\n", + "* IVF. Works well with large datasets without consuming much memory or compromising performance.\n", + "\n", + "More information about the different indices available with Faiss can be found at this link: https://github.com/facebookresearch/faiss/wiki/Guidelines-to-choose-an-index" + ], + "metadata": { + "id": "yi_riXHhcLy0" + } + }, + { + "cell_type": "code", + "source": [ + "def init_cache():\n", + " index = faiss.IndexFlatL2(768)\n", + " if index.is_trained:\n", + " print('Index trained')\n", + "\n", + " # Initialize Sentence Transformer model\n", + " encoder = SentenceTransformer('all-mpnet-base-v2')\n", + "\n", + " return index, encoder" + ], + "metadata": { + "id": "9poNBxbPl7xE" + }, + "execution_count": 15, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "In the `retrieve_cache` function, the .json file is retrieved from disk in case there is a need to reuse the cache across sessions." + ], + "metadata": { + "id": "_uZzX60odo1U" + } + }, + { + "cell_type": "code", + "source": [ + "def retrieve_cache(json_file):\n", + " try:\n", + " with open(json_file, 'r') as file:\n", + " cache = json.load(file)\n", + " except FileNotFoundError:\n", + " cache = {'questions': [], 'embeddings': [], 'answers': [], 'response_text': []}\n", + "\n", + " return cache" + ], + "metadata": { + "id": "FDJJ86TSp5CO" + }, + "execution_count": 16, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "The `store_cache` function saves the file containing the cache data to disk." + ], + "metadata": { + "id": "3uO-12UIdtSD" + } + }, + { + "cell_type": "code", + "source": [ + "def store_cache(json_file, cache):\n", + " with open(json_file, 'w') as file:\n", + " json.dump(cache, file)" + ], + "metadata": { + "id": "jx1CiKOcwKGn" + }, + "execution_count": 17, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "These functions will be used within the `SemanticCache` class, which includes the search function and its initialization function.\n", + "\n", + "Even though the `ask` function has a substantial amount of code, its purpose is quite straightforward. It looks in the cache for the closest question to the one just made by the user.\n", + "\n", + "Afterward, checks if it is within the specified threshold. If positive, it directly returns the response from the cache; otherwise, it calls the `query_database` function to retrieve the data from ChromaDB.\n", + "\n", + "I've used Euclidean distance instead of Cosine, which is widely employed in vector comparisons. This choice is based on the fact that Euclidean distance is the default metric used by Faiss. Although Cosine distance can also be calculated, doing so adds complexity that may not significantly contribute to the final result.\n" + ], + "metadata": { + "id": "t9AdmnhQd2E8" + } + }, + { + "cell_type": "code", + "source": [ + "class semantic_cache:\n", + " def __init__(self, json_file=\"cache_file.json\", thresold=0.35):\n", + " # Initialize Faiss index with Euclidean distance\n", + " self.index, self.encoder = init_cache()\n", + "\n", + " # Set Euclidean distance threshold\n", + " # a distance of 0 means identicals sentences\n", + " # We only return from cache sentences under this thresold\n", + " self.euclidean_threshold = thresold\n", + "\n", + " self.json_file = json_file\n", + " self.cache = retrieve_cache(self.json_file)\n", + "\n", + " def ask(self, question: str) -> str:\n", + " # Method to retrieve an answer from the cache or generate a new one\n", + " start_time = time.time()\n", + " try:\n", + " #First we obtain the embeddings corresponding to the user question\n", + " embedding = self.encoder.encode([question])\n", + "\n", + " # Search for the nearest neighbor in the index\n", + " self.index.nprobe = 8\n", + " D, I = self.index.search(embedding, 1)\n", + "\n", + " if D[0] >= 0:\n", + " if I[0][0] >= 0 and D[0][0] <= self.euclidean_threshold:\n", + " row_id = int(I[0][0])\n", + "\n", + " print('Answer recovered from Cache. ')\n", + " print(f'{D[0][0]:.3f} smaller than {self.euclidean_threshold}')\n", + " print(f'Found cache in row: {row_id} with score {D[0][0]:.3f}')\n", + " print(f'response_text: ' + self.cache['response_text'][row_id])\n", + "\n", + " end_time = time.time()\n", + " elapsed_time = end_time - start_time\n", + " print(f\"Time taken: {elapsed_time:.3f} seconds\")\n", + " return self.cache['response_text'][row_id]\n", + "\n", + " # Handle the case when there are not enough results\n", + " # or Euclidean distance is not met, asking to chromaDB.\n", + " answer = query_database([question], 1)\n", + " response_text = answer['documents'][0][0]\n", + "\n", + " self.cache['questions'].append(question)\n", + " self.cache['embeddings'].append(embedding[0].tolist())\n", + " self.cache['answers'].append(answer)\n", + " self.cache['response_text'].append(response_text)\n", + "\n", + " print('Answer recovered from ChromaDB. ')\n", + " print(f'response_text: {response_text}')\n", + "\n", + " self.index.add(embedding)\n", + " store_cache(self.json_file, self.cache)\n", + " end_time = time.time()\n", + " elapsed_time = end_time - start_time\n", + " print(f\"Time taken: {elapsed_time:.3f} seconds\")\n", + "\n", + " return response_text\n", + " except Exception as e:\n", + " raise RuntimeError(f\"Error during 'ask' method: {e}\")\n" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:36:51.316449Z", + "iopub.execute_input": "2024-02-29T17:36:51.31678Z", + "iopub.status.idle": "2024-02-29T17:36:55.197427Z", + "shell.execute_reply.started": "2024-02-29T17:36:51.316746Z", + "shell.execute_reply": "2024-02-29T17:36:55.196616Z" + }, + "trusted": true, + "id": "t_HVtwww9TD2" + }, + "execution_count": 51, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "### Testing the semantic_cache class." + ], + "metadata": { + "id": "UBWTqGM7i71N" + } + }, + { + "cell_type": "code", + "source": [ + "# Initialize the cache.\n", + "cache = semantic_cache('4cache.json')" + ], + "metadata": { + "id": "JH8s8eUtCMIS", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "c613bbfc-9f84-4a96-cd39-45972e69c15b" + }, + "execution_count": 52, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Index trained\n" + ] + } + ] + }, + { + "cell_type": "code", + "source": [ + "results = cache.ask(\"How do vaccines work?\")" + ], + "metadata": { + "id": "mKqKLfDe_8bC", + "outputId": "8a92ed95-c822-4382-c6db-d9de289341af", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "execution_count": 53, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Answer recovered from ChromaDB. \n", + "response_text: Summary : Shots may hurt a little, but the diseases they can prevent are a lot worse. Some are even life-threatening. Immunization shots, or vaccinations, are essential. They protect against things like measles, mumps, rubella, hepatitis B, polio, tetanus, diphtheria, and pertussis (whooping cough). Immunizations are important for adults as well as children. Your immune system helps your body fight germs by producing substances to combat them. Once it does, the immune system \"remembers\" the germ and can fight it again. Vaccines contain germs that have been killed or weakened. When given to a healthy person, the vaccine triggers the immune system to respond and thus build immunity. Before vaccines, people became immune only by actually getting a disease and surviving it. Immunizations are an easier and less risky way to become immune. NIH: National Institute of Allergy and Infectious Diseases\n", + "Time taken: 0.057 seconds\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "As expected, this response has been obtained from ChromaDB. The class then stores it in the cache.\n", + "\n", + "Now, if we send a second question that is quite different, the response should also be retrieved from ChromaDB. This is because the question stored previously is so dissimilar that it would surpass the specified threshold in terms of Euclidean distance." + ], + "metadata": { + "id": "dP7H6TypknLN" + } + }, + { + "cell_type": "code", + "source": [ + "\n", + "results = cache.ask(\"Explain briefly what is a Sydenham chorea\")" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:37:15.335288Z", + "iopub.execute_input": "2024-02-29T17:37:15.335593Z", + "iopub.status.idle": "2024-02-29T17:37:17.320691Z", + "shell.execute_reply.started": "2024-02-29T17:37:15.335566Z", + "shell.execute_reply": "2024-02-29T17:37:17.319671Z" + }, + "trusted": true, + "id": "CvJykqVf9TD2", + "outputId": "7137919e-e417-47b3-a638-18026b3edfe6", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "execution_count": 54, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Answer recovered from ChromaDB. \n", + "response_text: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\n", + "Time taken: 0.082 seconds\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "Perfect, the semantic cache system is behaving as expected.\n", + "\n", + "Let's proceed to test it with a question very similar to the one we just asked.\n", + "\n", + "In this case, the response should come directly from the cache without the need to access the ChromaDB database." + ], + "metadata": { + "id": "8aPWvU64lxOU" + } + }, + { + "cell_type": "markdown", + "source": [], + "metadata": { + "id": "sPmmTGGM0pVj" + } + }, + { + "cell_type": "code", + "source": [ + "results = cache.ask(\"Briefly explain me what is a Sydenham chorea.\")" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:37:17.32865Z", + "iopub.execute_input": "2024-02-29T17:37:17.328926Z", + "iopub.status.idle": "2024-02-29T17:37:17.463363Z", + "shell.execute_reply.started": "2024-02-29T17:37:17.328902Z", + "shell.execute_reply": "2024-02-29T17:37:17.462397Z" + }, + "trusted": true, + "id": "9_5IcGB-9TD2", + "outputId": "13563a7d-01f7-47d1-c345-6ad128f303c3", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "execution_count": 55, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Answer recovered from Cache. \n", + "0.028 smaller than 0.35\n", + "Found cache in row: 1 with score 0.028\n", + "response_text: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\n", + "Time taken: 0.019 seconds\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "The two questions are so similar that their Euclidean distance is truly minimal, almost as if they were identical.\n", + "\n", + "Now, let's try another question, this time a bit more distinct, and observe how the system behaves." + ], + "metadata": { + "id": "M4H8RoXFqdwE" + } + }, + { + "cell_type": "code", + "source": [ + "question_def = \"Write in 20 words what is a Sydenham chorea.\"\n", + "results = cache.ask(question_def)" + ], + "metadata": { + "id": "ysj5P_MBCqju", + "outputId": "d4639f73-dc7e-4c25-93ba-2a8c66dc7c61", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "execution_count": 56, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Answer recovered from Cache. \n", + "0.228 smaller than 0.35\n", + "Found cache in row: 1 with score 0.228\n", + "response_text: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\n", + "Time taken: 0.016 seconds\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "We observe that the Euclidean distance has increased, but it still remains within the specified threshold. Therefore, it continues to return the response directly from the cache." + ], + "metadata": { + "id": "MFzXsQwB9TD3" + } + }, + { + "cell_type": "markdown", + "source": [ + "# Loading the model and creating the prompt\n", + "Time to use the library **transformers**, the most famous library from [hugging face](https://huggingface.co/) for working with language models.\n", + "\n", + "We are importing:\n", + "* **Autotokenizer**: It is a utility class for tokenizing text inputs that are compatible with various pre-trained language models.\n", + "* **AutoModelForCausalLM**: it provides an interface to pre-trained language models specifically designed for language generation tasks using causal language modeling (e.g., GPT models), or the model used in this notebook [Gemma-2b-it](https://huggingface.co/google/gemma-2b-it).\n", + "\n", + "Please, feel free to test [different Models](https://huggingface.co/models?pipeline_tag=text-generation&sort=trending), you need to search for NLP models trained for text-generation.\n" + ], + "metadata": { + "id": "Ot3wrq0p9TD3" + } + }, + { + "cell_type": "code", + "source": [ + "!pip install torch" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:40:32.797334Z", + "iopub.execute_input": "2024-02-29T17:40:32.797669Z", + "iopub.status.idle": "2024-02-29T17:40:44.152114Z", + "shell.execute_reply.started": "2024-02-29T17:40:32.797635Z", + "shell.execute_reply": "2024-02-29T17:40:44.151056Z" + }, + "trusted": true, + "id": "tdxiKqjT9TD3" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "from torch import cuda, torch\n", + "#In a MAC Silicon the device must be 'mps'\n", + "# device = torch.device('mps') #to use with MAC Silicon\n", + "device = f'cuda:{cuda.current_device()}' if cuda.is_available() else 'cpu'" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:40:44.153914Z", + "iopub.execute_input": "2024-02-29T17:40:44.15434Z", + "iopub.status.idle": "2024-02-29T17:40:44.160144Z", + "shell.execute_reply.started": "2024-02-29T17:40:44.154292Z", + "shell.execute_reply": "2024-02-29T17:40:44.159154Z" + }, + "trusted": true, + "id": "pIDMTCnH9TD7" + }, + "execution_count": 25, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "from transformers import AutoTokenizer, AutoModelForCausalLM\n", + "\n", + "model_id = \"google/gemma-2b-it\"\n", + "tokenizer = AutoTokenizer.from_pretrained(model_id)\n", + "model = AutoModelForCausalLM.from_pretrained(model_id,\n", + " device_map=\"cuda\",\n", + " torch_dtype=torch.bfloat16)" + ], + "metadata": { + "execution": { + "iopub.status.busy": "2024-02-29T17:41:25.628412Z", + "iopub.execute_input": "2024-02-29T17:41:25.628804Z", + "iopub.status.idle": "2024-02-29T17:41:30.202141Z", + "shell.execute_reply.started": "2024-02-29T17:41:25.628766Z", + "shell.execute_reply": "2024-02-29T17:41:30.200774Z" + }, + "trusted": true, + "id": "CU2T4lp-9TD7" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [], + "metadata": { + "id": "0kdqsEbUEywG" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Creating the extended prompt\n", + "To create the prompt we use the result from query the 'semantic_cache' class and the question introduced by the user.\n", + "\n", + "The prompt have two parts, the **relevant context** that is the information recovered from the database and the **user's question**.\n", + "\n", + "We only need to put the two parts together to create the prompt then send it to the model." + ], + "metadata": { + "id": "GzHuFrAX9TD7" + } + }, + { + "cell_type": "code", + "source": [ + "prompt_template = f\"Relevant context: {results}\\n\\n The user's question: {question_def}\"\n", + "prompt_template" + ], + "metadata": { + "id": "TdjbfAHhFuhS", + "outputId": "4090da66-328e-478e-c2d7-1957597f8786", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 209 + } + }, + "execution_count": 44, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "\"Relevant context: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\\n\\n The user's question: Write in 20 words what is a Sydenham chorea.\"" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 44 + } + ] + }, + { + "cell_type": "code", + "source": [ + "input_ids = tokenizer(prompt_template, return_tensors=\"pt\").to(\"cuda\")" + ], + "metadata": { + "id": "DmYAcXEEECnz" + }, + "execution_count": 45, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "Now all that remains is to send the prompt to the model and wait for its response!\n" + ], + "metadata": { + "id": "S-QXeuJ09TD8" + } + }, + { + "cell_type": "code", + "source": [ + "outputs = model.generate(**input_ids,\n", + " max_new_tokens=256)\n", + "print(tokenizer.decode(outputs[0]))" + ], + "metadata": { + "id": "lheL8vHpEMDD", + "outputId": "b646d648-b88d-4a29-ab30-427d00296255", + "colab": { + "base_uri": "https://localhost:8080/" + } + }, + "execution_count": 46, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Relevant context: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\n", + "\n", + " The user's question: Write in 20 words what is a Sydenham chorea.\n", + "\n", + "Sure, here is a 20-word answer:\n", + "\n", + "Sydenham chorea is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS).\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "# Conclusion.\n", + "There's a 50% reduction in data retrieval time between accessing ChromaDB and going directly to the cache. However, in larger projects, this difference increases, leading to enhancements of 90-95%.\n", + "\n", + "We have very few data in Chroma, and only a single instance of the cache class. Typically, the data behind the cache system is much larger, possibly involving more than just a query to a vector database but sourced from various places.\n", + "\n", + "It's common to have multiple instances of the cache class, usually based on user typology, as questions tend to repeat more among users who share common traits.\n", + "\n", + "In summary, we have created a very simple RAG (Retrieval-Augmented Generation) system and enhanced it with a semantic cache layer between the user's question and obtaining the information necessary to create the enriched prompt." + ], + "metadata": { + "execution": { + "iopub.status.busy": "2023-07-12T22:01:56.992775Z", + "iopub.execute_input": "2023-07-12T22:01:56.993351Z", + "iopub.status.idle": "2023-07-12T22:01:57.001309Z", + "shell.execute_reply.started": "2023-07-12T22:01:56.993305Z", + "shell.execute_reply": "2023-07-12T22:01:56.999431Z" + }, + "id": "Uo7lGXBV9TD8" + } + } + ] +} \ No newline at end of file diff --git a/notebooks/spa/stable_diffusion_interpolation.ipynb b/notebooks/spa/stable_diffusion_interpolation.ipynb new file mode 100644 index 00000000..496bc062 --- /dev/null +++ b/notebooks/spa/stable_diffusion_interpolation.ipynb @@ -0,0 +1 @@ +{"cells":[{"cell_type":"markdown","metadata":{"id":"UsrvK8CFDiNu"},"source":["## Images Interpolation with Stable Diffusion\n","\n","_Authored by: [Rustam Akimov](https://github.com/AkiRusProd)_\n","\n","This notebook shows how to use Stable Diffusion to interpolate between images. Image interpolation using Stable Diffusion is the process of creating intermediate images that smoothly transition from one given image to another, using a generative model based on diffusion. \n","\n","Here are some various use cases for image interpolation with Stable Diffusion:\n","- Data Augmentation: Stable Diffusion can augment training data for machine learning models by generating synthetic images that lie between existing data points. This can improve the generalization and robustness of machine learning models, especially in tasks like image generation, classification or object detection. \n","- Product Design and Prototyping: Stable Diffusion can aid in product design by generating variations of product designs or prototypes with subtle differences. This can be useful for exploring design alternatives, conducting user studies, or visualizing design iterations before committing to physical prototypes. \n","- Content Generation for Media Production: In media production, such as film and video editing, Stable Diffusion can be used to generate intermediate frames between key frames, enabling smoother transitions and enhancing visual storytelling. This can save time and resources compared to manual frame-by-frame editing.\n","\n","In the context of image interpolation, Stable Diffusion models are often used to navigate through a high-dimensional latent space. Each dimension represents a specific feature that has been learned by the model. By walking through this latent space and interpolating between different latent representations of images, the model is able to generate a sequence of intermediate images which show a smooth transition between the original images. There are two types of latents in stable diffusion: prompt latents and image latents. \n","\n","Latent space walking involves moving through a latent space along a path defined by two or more points (representing images). By carefully selecting these points and the path between them, it is possible to control the features of the generated images, such as style, content, and other visual aspects. \n","\n","In this Notebook, we will explore examples of image interpolation using Stable Diffusion and demonstrate how latent space walking can be implemented and utilized to create smooth transitions between images. We'll provide code snippets and visualizations that illustrate this process in action, allowing for a deeper understanding of how generative models can manipulate and morph image representations in meaningful ways.\n"]},{"cell_type":"markdown","metadata":{"id":"XEhtH959DiOC"},"source":["First, let's install all the required modules."]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"execution":{"iopub.execute_input":"2024-02-21T17:20:28.329767Z","iopub.status.busy":"2024-02-21T17:20:28.329050Z","iopub.status.idle":"2024-02-21T17:23:15.653382Z","shell.execute_reply":"2024-02-21T17:23:15.652310Z","shell.execute_reply.started":"2024-02-21T17:20:28.329734Z"},"id":"lbWtDpayDiOD","outputId":"b39791a6-6bdc-4f48-e016-5650c98072cf","trusted":true},"outputs":[],"source":["!pip install -q diffusers transformers xformers accelerate\n","!pip install -q numpy scipy ftfy Pillow"]},{"cell_type":"markdown","metadata":{"id":"pUUXab_IDiOE"},"source":["Import modules"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":171,"referenced_widgets":["3537007206fd4d57ae492d29d90bd904","85bf2410c7d2440db76163fe1df4f4bb","c2bf5a15732a4898915b0ec3cb56df8c","bf573d9fcbac4701b31e464373fdbeb0","f1f352e6964f424f9e6a4557f6e3ff97","fa5231429aa1437983dc93dc597e698e","fe24820349a0456ca103e30024490c0e","35ffc6d955a44422a06e5c304fcaeddb","1e64cce9ffc94f23921f964288c2e26d","96d5032cdaa14cdeb110f8fc3b6614c1","4a466456e448417a8b3cc442cec49632"]},"execution":{"iopub.execute_input":"2024-02-21T17:23:55.606390Z","iopub.status.busy":"2024-02-21T17:23:55.606005Z","iopub.status.idle":"2024-02-21T17:24:12.144679Z","shell.execute_reply":"2024-02-21T17:24:12.143740Z","shell.execute_reply.started":"2024-02-21T17:23:55.606352Z"},"id":"gbnW1HiEDiOE","outputId":"a3b7adb5-f455-4c75-d626-6f2a6f86455b","trusted":true},"outputs":[],"source":["import torch\n","import numpy as np\n","import os\n","\n","import time\n","\n","from PIL import Image\n","from IPython import display as IPdisplay\n","from tqdm.auto import tqdm\n","\n","from diffusers import StableDiffusionPipeline\n","from diffusers import (\n"," DDIMScheduler,\n"," PNDMScheduler,\n"," LMSDiscreteScheduler,\n"," DPMSolverMultistepScheduler,\n"," EulerAncestralDiscreteScheduler,\n"," EulerDiscreteScheduler,\n",")\n","from transformers import logging\n","\n","logging.set_verbosity_error()"]},{"cell_type":"markdown","metadata":{"id":"loFaaWVUDiOF"},"source":["Let's check if CUDA is available.\n","\n","\n"]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:24:16.252373Z","iopub.status.busy":"2024-02-21T17:24:16.251653Z","iopub.status.idle":"2024-02-21T17:24:16.258088Z","shell.execute_reply":"2024-02-21T17:24:16.257085Z","shell.execute_reply.started":"2024-02-21T17:24:16.252340Z"},"id":"uGgmrhr-DiOF","trusted":true},"outputs":[],"source":["print(torch.cuda.is_available())\n","\n","device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"]},{"cell_type":"markdown","metadata":{"id":"zMSGnuqmDiOF"},"source":["These settings are used to optimize the performance of PyTorch models on CUDA-enabled GPUs, especially when using mixed precision training or inference, which can be beneficial in terms of speed and memory usage. \n","Source: https://huggingface.co/docs/diffusers/optimization/fp16#memory-efficient-attention"]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:24:18.661531Z","iopub.status.busy":"2024-02-21T17:24:18.661171Z","iopub.status.idle":"2024-02-21T17:24:18.666289Z","shell.execute_reply":"2024-02-21T17:24:18.665171Z","shell.execute_reply.started":"2024-02-21T17:24:18.661501Z"},"id":"JT02KQqNDiOF","trusted":true},"outputs":[],"source":["torch.backends.cudnn.benchmark = True\n","torch.backends.cuda.matmul.allow_tf32 = True"]},{"cell_type":"markdown","metadata":{"id":"_E5R20VtDiOF"},"source":["### Model\n","\n","The [`runwayml/stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5) model and the [`LMSDiscreteScheduler`](https://huggingface.co/docs/diffusers/en/api/schedulers/lms_discrete) scheduler were chosen to generate images. Despite being an older technology, it continues to enjoy popularity due to its fast performance, minimal memory requirements, and the availability of numerous community fine-tuned models built on top of SD1.5. However, you are free to experiment with other models and schedulers to compare the results."]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":913,"referenced_widgets":["3538495833744e9eab9b25eade484603","8db4aafac51043598a9e4b8915f4f7c4","0e63faca29cb4ca3845e402d67592b8f","bf4b67c0a0034b1ab576066d366d934e","e09eaa48528a48629b19020aa65c5bdc","57d02fadf8fc4df3b1fbee18e45f9199","2f6e125affb54674b5f49beaf3612bee","f736f2fc44444ebf832fb2ead6ea0fd0","c41432dd085f41aeaa1e1d9cac4872e7","c577a158f88040c0979b319fb9fb89b0","f24c9ef7c3f04da783aaffd3e5d48de7","6401b1635d62478ba644075a32494384","72a5fcb23802406eb22e2758be01052c","0ba055eeeed04e91b684072740e07f0a","790424c8674d42d59c75ba2fd4021e3a","ff697b2e046a4a36a78ac362cae5c2a9","eb3e42c270b04086a2a04ae286684f6b","d9816365a87340dfa06fe9a37811a81b","309d89a221ab4010af0d85d3ae90a1c3","1cdaf6fe1392453288997f672af80a0d","5dc635e136f741608e73455581822408","670c96aa182f4defbfce4a9fd0b7f96b","92d3f7562ee04b789c44656f83d12528","bd7b9e9757d24c568432eea484271376","9c12a4d25771464b84b35804695fd50a","fa1d984ee1864975a4c543f1dcb3aa42","b89baafcd7e944b088bf0c9b1e839ba5","fb34d344590e43438906722b1ab958f0","ed6a871ff054410b8d018b3b97c75c60","909ad3aefa5b4c65931f300b3e9655dd","93159e8e4bf84bc9a35c4325dc6cc851","f2f9506fc02a4624a8a1c08f1f6abdb2","2f010760c1a146238f35af00568ccb11","20c46fc3077b4dcc855252c222002569","9b597fc5d5cf4fecabcfbc7a4cfa1ee9","17ceef0b617c4f52aa0bb5ec12113fcd","e20cd706fa304bc190e92ae7d27b5b38","9a3f552babd34cdeb8fed4ce1b1b33a7","f8db6aae6e3a468ba3805991e19e1f45","681cc61581d84298b4798b5c43818e31","45d877fafa5a4772ba5d62557843bb51","62d5802bdc7a49efbcb47889e29e924c","e4f21da1f63a4a819485cbe7e5ae306b","c7f341fe95ca426d9502594bd48d36f6","3214f06917264be88837314b26375a1c","00b88de69713463485206e4b7c8c3c04","f63bebf985a046ce9ba9d567db0b7ca4","a49fd3377dde41dab42677091dc7bd04","ed1dbb98c8d34dce8df70e4698a21911","95d1047a0a644262aa385987c9a331b4","c565ba8910c34e138c6cd10e9b06d673","5ffaa53c74d3413b8f3c3fe2fc5cc075","aedf99983c5043fe8d634b6e3b56e1ae","bf1489df7c98442caadd2417026bffdf","590b0de8d8f84d1ab9a243d88380c295","44a56b89efa649edb669abed3605e576","c26781e6cc214b6b866b8f11673e9c00","cddd14dde34c42f59ed71870e558246e","399fdb41f7fd4f1489c1bf4814b53907","f7f49c60f4ca41efa9a3b6b33c418f73","9ef6f8a591244419916d980d5883e03e","4d26d7dd13d148b3b6e06f10b589f2a8","8c690a7356af4547902b72aa6a20328d","1e4ad02a5afb431a96626230081054cc","2fd8069de9754a87ae04ec7b2c4b380a","da6373c3704d45869e2fafdf9772d3d1","8c9b21767c5d4741b717b82f1e4a0e03","a83f68f1b1d640298019af11c5198a7d","55a116f1fa634632a061a4ad8bb75ec3","58910b48c70a4b3dabf87b6a12004e70","2dc63f0fe271457f890fd2067631ad75","3fcfdbffff6149fe880b0702cf8162f3","eb96b8f49d6746ec8f46e65e59a3fad6","e03272733bd84e2e9edcb83391e1cfae","184d9e1fa8d241c386567300db4e2c8c","51ceac8abb23437f952e07d2daaf0dae","a06b4275e1444e72b921d631feea90ba","d7c9f6b399524bc596e84641687ba29a","1c8a25c7f70145df9722d3702fc6d2dd","ae44d548c5164e8fb5e85f1ab19da9ac","1325ea426c9747828601a9175a9b0248","97fe7d3ccc984694aa42a56ba930c64e","c1a5498ffe0f407397d2a8c77e35de84","5eefef112ef249cc97ecd63f19ea122d","defa4622267e49b8a54d6aceea082c39","59081d8cdf4e43228a20f3fe986926b2","c14c4f3d31a447b38d359acc6e29496d","f4f167561c2d452785b6e59c2ce61b28","4cc69a88629f41dc80c552b58d3f5eaf","6cc2e02ee7b74f3aa2f71bf0725190de","da714c09550f43849e5a2502b092403a","7370d528153e473980ac0e701c9e6825","0ba17f55cbf941feaa0b7e8959a94591","a903c8a2c22c48bdb70024665bc1cb0f","8c29df6a485946c090b39b49240427fd","b0bcd8201363473ea0e4ace230443446","d9f1220cc4f5440b97e35151ea76ed00","0eaebd8b9eb14feca8a11a8c3077b196","51061a6b42584e2bbac74da6a6f2a1da","e0216d9707cb4d10be57c4864521c376","2e72912929c14d9f8843bc028b75de77","feeaea5e7b524ed09bd08c73b7ed5e17","1ebb24b8e6f44608ae8c11747ef7c42d","c523aec40bb54986a4f40924c81fe5da","f23d6c684c3a416d8b43fc642653eec8","16e98aec73d048d693fd0e44df2220e7","495a6de427ce47eda18a1570fd8f5f9d","eac1b421700d492ba398d5ac609b5741","80509211c7dc47e8a673ed80fca6d8e1","e1f4e7fa8b1f4530b580f101a9fad304","4b14e789cc2446eda7a94a5b1259e738","865eb0b3254246948f532e2c6dd02bd1","7e678fd9b4284bf39ecd29de4d7624a3","adfbabeb20a3428c8fd6ec5b79830c34","6a012f24803646369ac97be41e5998f3","521ecdf554b84958ba4ee487c9621ffc","c6a66e5c516c4dbbbc1ca203c6a2d0db","66a43c50816c4e3fa88852c3d2c3b0c7","46df7a569a984f2a8d60b021e2366550","cffa386bd54e4baa8947d4d51c8e54a4","a5926a9d27b44a2d961f36d7fd36da15","35cf8424313d43dea56ca590edf70b26","5dc6caab4749403aab23ce95993f9ad0","84e32b4370a94fe498b3b01b6632daf3","16c7e883c9ec49329b68e61b276d5fac","5518095d648d4b2b95647c746d70c4d8","de350491cd2e48a19541f6e97b9f176e","cd7fa6f0b6844fd4a41b97cbe39e0c2f","683585ba77df463a9bcb2f8ce4794747","3b99c2eba3ab4ea2a44b23ec0916ce2d","b18d19fb59d8458c9e9048c1458ee95c","d5a23fbc6e634b02bf4f2540e9def457","1c3a77c578d644b09771446ed559c575","b08c232e469646f89a28f4371f0e7699","5e37d843836b42dfb62f728181ee4dfa","0755693c5b854000af081f2818162683","2e194e5ccbd349b093481e24148de89c","9d2ff155648146058d2d359e474159dc","0220a7b0d67a482a8ab7e9ac6372d380","034a1eb240694704a8052783583caefe","03fdfd9eb2e343df8af4ff2b06ac8eb1","393b21bd730f4a8b93d1653ebafaea09","0094122c8eaf47e6851a3449e2fb0086","7850ea0076da49639ca986a4885d7048","f04f25b18eaa43f6a2044dac7aba8372","6d9421a31914451cac51c71aee8b1ce4","a4b3ef66956d494887e796f87b4278f1","1638b8dbe07a4d249167a3d34ac9adc6","163ec8057136471bb1f460d657c4aa6c","ff1655111fd04c4785d7e5ec3747629c","566d76e028b643e18729621e82531939","2642af1a55cf452a93e528fb25f1c8cb","a428e985410341e0ba04359af465681e","bfbec38740f8413d93469274aeccbf23","70cf0bb1ada946dd9717fc2a493b7805","8caab95721354f08999bea8dc6105b4e","221facd121a14faf9d664f644935b0ae","bac1fcd4cf1847ed89bc5e01ae435e24","bc4ed6312ae44ba7a9e21d43d7edd48a","fd4eebbe68204eaa802186836c372b93","6a7f80bb3e534eb2a48d3d29c9ac3988","e431b2a589524545a5ccbb79d2c7bab9","90dccdfd4085472f8f9ba0535d85d327","d1d564e827cf4a71af9aa87b9d5696c8","1ad180dc107946b9a1b554a4b98ee514","88c6c5bcc44d46089aa3efaa7fb9e452","e0280b4f0172481ea7664bfb96d1bb1c","3cd43805ef564f6696905a2465ea4467","ab7f8b09b1d8452995d66e6f0df83faa","3959b0871ea840839a383c895cfbe916","60aa6e24133c4e67957bf953e5b10f4d","5ce301bccee049cf9664800d63e2e2eb","da25ec24a6b44ce9a51bcc2d440d0258","41ea6163d1b04f8f89cd1f9ec9e72847","31107fa83a974eca83fa968ae4eae909","36a0330c9e1d48a49a739feaef34ddc0"]},"execution":{"iopub.execute_input":"2024-02-21T17:24:22.143953Z","iopub.status.busy":"2024-02-21T17:24:22.143589Z","iopub.status.idle":"2024-02-21T17:25:54.037631Z","shell.execute_reply":"2024-02-21T17:25:54.036655Z","shell.execute_reply.started":"2024-02-21T17:24:22.143923Z"},"id":"ppKz1aLSDiOF","outputId":"e359b27d-6381-4bef-8a8c-bcc9576f7fe3","trusted":true},"outputs":[],"source":["model_name_or_path = \"runwayml/stable-diffusion-v1-5\"\n","\n","scheduler = LMSDiscreteScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule=\"scaled_linear\", num_train_timesteps=1000)\n","\n","\n","pipe = StableDiffusionPipeline.from_pretrained(\n"," model_name_or_path,\n"," scheduler=scheduler,\n"," torch_dtype=torch.float32,\n",").to(device)\n","\n","# Disable image generation progress bar, we'll display our own\n","pipe.set_progress_bar_config(disable=True)"]},{"cell_type":"markdown","metadata":{"id":"5oBmcxe9DiOG"},"source":["These methods are designed to reduce the memory consumed by the GPU. If you have enough VRAM, you can skip this cell. \n","\n","More detailed information can be found here: https://huggingface.co/docs/diffusers/en/optimization/opt_overview \n","In particular, information about the following methods can be found here: https://huggingface.co/docs/diffusers/optimization/memory\n"]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:25:54.040235Z","iopub.status.busy":"2024-02-21T17:25:54.039388Z","iopub.status.idle":"2024-02-21T17:26:00.115879Z","shell.execute_reply":"2024-02-21T17:26:00.115042Z","shell.execute_reply.started":"2024-02-21T17:25:54.040193Z"},"id":"1i7WuQV1DiOG","trusted":true},"outputs":[],"source":["# Offloading the weights to the CPU and only loading them on the GPU can reduce memory consumption to less than 3GB.\n","pipe.enable_model_cpu_offload()\n","\n","# Tighter ordering of memory tensors.\n","pipe.unet.to(memory_format=torch.channels_last)\n","\n","# Decoding large batches of images with limited VRAM or batches with 32 images or more by decoding the batches of latents one image at a time.\n","pipe.enable_vae_slicing()\n","\n","# Splitting the image into overlapping tiles, decoding the tiles, and then blending the outputs together to compose the final image. \n","pipe.enable_vae_tiling()\n","\n","# Using Flash Attention; If you have PyTorch >= 2.0 installed, you should not expect a speed-up for inference when enabling xformers.\n","pipe.enable_xformers_memory_efficient_attention()\n"]},{"cell_type":"markdown","metadata":{"id":"k45VkXF7DiOG"},"source":["The `display_images` function converts a list of image arrays into a GIF, saves it to a specified path and returns the GIF object for display. It names the GIF file using the current time and handles any errors by printing them out."]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:30:01.535670Z","iopub.status.busy":"2024-02-21T17:30:01.535281Z","iopub.status.idle":"2024-02-21T17:30:01.542928Z","shell.execute_reply":"2024-02-21T17:30:01.541894Z","shell.execute_reply.started":"2024-02-21T17:30:01.535637Z"},"id":"n5cKlS0CDiOG","trusted":true},"outputs":[],"source":["def display_images(images, save_path):\n"," try:\n"," # Convert each image in the 'images' list from an array to an Image object.\n"," images = [\n"," Image.fromarray(np.array(image[0], dtype=np.uint8)) for image in images\n"," ]\n","\n"," # Generate a file name based on the current time, replacing colons with hyphens\n"," # to ensure the filename is valid for file systems that don't allow colons.\n"," filename = (\n"," time.strftime(\"%H:%M:%S\", time.localtime())\n"," .replace(\":\", \"-\")\n"," )\n"," # Save the first image in the list as a GIF file at the 'save_path' location.\n"," # The rest of the images in the list are added as subsequent frames to the GIF.\n"," # The GIF will play each frame for 100 milliseconds and will loop indefinitely.\n"," images[0].save(\n"," f\"{save_path}/{filename}.gif\",\n"," save_all=True,\n"," append_images=images[1:],\n"," duration=100,\n"," loop=0,\n"," )\n"," except Exception as e:\n"," # If there is an error during the process, print the exception message.\n"," print(e)\n","\n"," # Return the saved GIF as an IPython display object so it can be displayed in a notebook.\n"," return IPdisplay.Image(f\"{save_path}/{filename}.gif\")"]},{"cell_type":"markdown","metadata":{"id":"L13Q7INNDiOG"},"source":["### Generation parameters\n","\n","\n","* `seed`: This variable is used to set a specific random seed for reproducibility. \n","* `generator`: This is set to a PyTorch random number generator object if a seed is provided, otherwise it is None. It ensures that the operations using it have reproducible outcomes. \n","* `guidance_scale`: This parameter controls the extent to which the model should follow the prompt in text-to-image generation tasks, with higher values leading to stronger adherence to the prompt. \n","* `num_inference_steps`: This specifies the number of steps the model takes to generate an image. More steps can lead to a higher quality image but take longer to generate. \n","* `num_interpolation_steps`: This determines the number of steps used when interpolating between two points in the latent space, affecting the smoothness of transitions in generated animations. \n","* `height`: The height of the generated images in pixels. \n","* `width`: The width of the generated images in pixels. \n","* `save_path`: The file system path where the generated gifs will be saved. "]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:30:04.013629Z","iopub.status.busy":"2024-02-21T17:30:04.012881Z","iopub.status.idle":"2024-02-21T17:30:04.019551Z","shell.execute_reply":"2024-02-21T17:30:04.018612Z","shell.execute_reply.started":"2024-02-21T17:30:04.013596Z"},"id":"R_B-h2j4DiOG","trusted":true},"outputs":[],"source":["# The seed is set to \"None\", because we want different results each time we run the generation.\n","seed = None\n","\n","if seed is not None:\n"," generator = torch.manual_seed(seed)\n","else:\n"," generator = None\n","\n","# The guidance scale is set to its normal range (7 - 10).\n","guidance_scale = 8\n","\n","# The number of inference steps was chosen empirically to generate an acceptable picture within an acceptable time.\n","num_inference_steps = 15\n","\n","# The higher you set this value, the smoother the interpolations will be. However, the generation time will increase. This value was chosen empirically.\n","num_interpolation_steps = 30\n","\n","# I would not recommend less than 512 on either dimension. This is because this model was trained on 512x512 image resolution.\n","height = 512 \n","width = 512\n","\n","# The path where the generated GIFs will be saved\n","save_path = \"/output\"\n","\n","if not os.path.exists(save_path):\n"," os.makedirs(save_path)\n"]},{"cell_type":"markdown","metadata":{"id":"Nm4BHESjDiOG"},"source":["### Example 1: Prompt interpolation\n","\n","In this example, interpolation between positive and negative prompt embeddings allows exploration of space between two conceptual points defined by prompts, potentially leading to variety of images blending characteristics dictated by prompts gradually. In this case, interpolation involves adding scaled deltas to original embeddings, creating a series of new embeddings that will be used later to generate images with smooth transitions between different states based on the original prompt.\n"]},{"cell_type":"markdown","metadata":{},"source":["![Example 1](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/sd_interpolation_1.gif)"]},{"cell_type":"markdown","metadata":{},"source":["First of all, we need to tokenize and obtain embeddings for both positive and negative text prompts. The positive prompt guides the image generation towards the desired characteristics, while the negative prompt steers it away from unwanted features."]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":49,"referenced_widgets":["640b691e30b844ec943995160216e28b","4262f099aab24cfd9b3790864e0e1d63","a2194da8bc254658b9db17e19dbe418b","f0e13bd4abca444592850390651272a4","866b048905164e31a5011bdc0fcf5180","867f3d155da9469ab9820923e40e78e5","7d7e58bafe2c4ff6a44275c3a2ea9826","cc9fdbf01697491f856a33e4b70a7a78","6826628eb6214b57bbe56e3eb80322b3","757864720c4041c6a24f8aa8f1630e69","92e5153340a74fc9895d4f87b68e3cad"]},"execution":{"iopub.execute_input":"2024-02-21T17:40:07.727796Z","iopub.status.busy":"2024-02-21T17:40:07.727407Z","iopub.status.idle":"2024-02-21T17:43:50.624205Z","shell.execute_reply":"2024-02-21T17:43:50.622571Z","shell.execute_reply.started":"2024-02-21T17:40:07.727768Z"},"id":"YVNrz60MDiOH","outputId":"428cf53c-ca0d-49e6-f2cd-41ed292b5117","trusted":true},"outputs":[],"source":["# The text prompt that describes the desired output image.\n","prompt = \"Epic shot of Sweden, ultra detailed lake with an ren dear, nostalgic vintage, ultra cozy and inviting, wonderful light atmosphere, fairy, little photorealistic, digital painting, sharp focus, ultra cozy and inviting, wish to be there. very detailed, arty, should rank high on youtube for a dream trip.\"\n","# A negative prompt that can be used to steer the generation away from certain features; here, it is empty.\n","negative_prompt = \"poorly drawn,cartoon, 2d, disfigured, bad art, deformed, poorly drawn, extra limbs, close up, b&w, weird colors, blurry\"\n","\n","# The step size for the interpolation in the latent space.\n","step_size = 0.001\n","\n","# Tokenizing and encoding the prompt into embeddings.\n","prompt_tokens = pipe.tokenizer(\n"," prompt,\n"," padding=\"max_length\",\n"," max_length=pipe.tokenizer.model_max_length,\n"," truncation=True,\n"," return_tensors=\"pt\",\n",")\n","prompt_embeds = pipe.text_encoder(prompt_tokens.input_ids.to(device))[0]\n","\n","\n","# Tokenizing and encoding the negative prompt into embeddings.\n","if negative_prompt is None:\n"," negative_prompt = [\"\"]\n","\n","negative_prompt_tokens = pipe.tokenizer(\n"," negative_prompt,\n"," padding=\"max_length\",\n"," max_length=pipe.tokenizer.model_max_length,\n"," truncation=True,\n"," return_tensors=\"pt\",\n",")\n","negative_prompt_embeds = pipe.text_encoder(negative_prompt_tokens.input_ids.to(device))[0]"]},{"cell_type":"markdown","metadata":{},"source":["Now let's look at the code part that generates a random initial vector using a normal distribution that is structured to match the dimensions expected by the diffusion model (UNet). This allows for the reproducibility of the results by optionally using a random number generator. After creating the initial vector, the code performs a series of interpolations between the two embeddings (positive and negative prompts), by incrementally adding a small step size for each iteration. The results are stored in a list named \"walked_embeddings\"."]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Generating initial latent vectors from a random normal distribution, with the option to use a generator for reproducibility.\n","latents = torch.randn(\n"," (1, pipe.unet.config.in_channels, height // 8, width // 8),\n"," generator=generator,\n",")\n","\n","walked_embeddings = []\n","\n","# Interpolating between embeddings for the given number of interpolation steps.\n","for i in range(num_interpolation_steps):\n"," walked_embeddings.append(\n"," [prompt_embeds + step_size * i, negative_prompt_embeds + step_size * i]\n"," )"]},{"cell_type":"markdown","metadata":{},"source":["Finally, let's generate a series of images based on interpolated embeddings and then displaying these images. We'll iterate over an array of embeddings, using each to generate an image with specified characteristics like height, width, and other parameters relevant to image generation. Then we'll collect these images into a list. Once generation is complete we'll call the `display_image` function to save and display these images as GIF at a given save path."]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Generating images using the interpolated embeddings.\n","images = []\n","for latent in tqdm(walked_embeddings):\n"," images.append(\n"," pipe(\n"," height=height,\n"," width=width,\n"," num_images_per_prompt=1,\n"," prompt_embeds=latent[0],\n"," negative_prompt_embeds=latent[1],\n"," num_inference_steps=num_inference_steps,\n"," guidance_scale=guidance_scale,\n"," generator=generator,\n"," latents=latents,\n"," ).images\n"," )\n","\n","# Display of saved generated images.\n","display_images(images, save_path)"]},{"cell_type":"markdown","metadata":{"id":"uZQWop9nDiOH"},"source":["### Example 2: Diffusion latents interpolation for a single prompt\n","Unlike the first example, in this one, we are performing interpolation between the two embeddings of the diffusion model itself, not the prompts. Please note that in this case, we use the slerp function for interpolation. However, there is nothing stopping us from adding a constant value to one embedding instead.\n"]},{"cell_type":"markdown","metadata":{},"source":["![Example 2](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/sd_interpolation_2.gif)"]},{"cell_type":"markdown","metadata":{"id":"CiW6SlhXDiOH"},"source":["The function presented below stands for Spherical Linear Interpolation. It is a method of interpolation on the surface of a sphere. This function is commonly used in computer graphics to animate rotations in a smooth manner and can also be used to interpolate between high-dimensional data points in machine learning, such as latent vectors used in generative models. \n","\n","The source is from Andrej Karpathy's gist: https://gist.github.com/karpathy/00103b0037c5aaea32fe1da1af553355. \n","A more detailed explanation of this method can be found at: https://en.wikipedia.org/wiki/Slerp."]},{"cell_type":"code","execution_count":1,"metadata":{"id":"grgP7UNpDiOH"},"outputs":[],"source":["def slerp(v0, v1, num, t0=0, t1=1):\n"," v0 = v0.detach().cpu().numpy()\n"," v1 = v1.detach().cpu().numpy()\n","\n"," def interpolation(t, v0, v1, DOT_THRESHOLD=0.9995):\n"," \"\"\"helper function to spherically interpolate two arrays v1 v2\"\"\"\n"," dot = np.sum(v0 * v1 / (np.linalg.norm(v0) * np.linalg.norm(v1)))\n"," if np.abs(dot) > DOT_THRESHOLD:\n"," v2 = (1 - t) * v0 + t * v1\n"," else:\n"," theta_0 = np.arccos(dot)\n"," sin_theta_0 = np.sin(theta_0)\n"," theta_t = theta_0 * t\n"," sin_theta_t = np.sin(theta_t)\n"," s0 = np.sin(theta_0 - theta_t) / sin_theta_0\n"," s1 = sin_theta_t / sin_theta_0\n"," v2 = s0 * v0 + s1 * v1\n"," return v2\n","\n"," t = np.linspace(t0, t1, num)\n","\n"," v3 = torch.tensor(np.array([interpolation(t[i], v0, v1) for i in range(num)]))\n","\n"," return v3"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":561,"referenced_widgets":["d7eb412b880c490c95f1d2baeaf2e6af","3e7caf19c664461e9505cbfcb708ceba","21e5a4fd28ac47a190d0f41808dd75a1","7f4317fe6eca4fc5be524453e55103bd","4ab2093e02704b748298a6d34e807847","69ffdc3c18f5484cad6945a77b024529","2c5d8801da6f4d88be3801254c3e764b","a76a9fce4af34c639327e5a0f4f4e692","e00e5537ae9a43d5956c2c770599edde","6fa7c3c07e734867ac5676b09b6804b3","ed410d69e8e94af7be0d104c5c29a2c9"]},"id":"aIU-nxTcDiOH","outputId":"1f762594-d89d-4bd3-c909-3d4850293b71"},"outputs":[],"source":["# The text prompt that describes the desired output image.\n","prompt = \"Sci-fi digital painting of an alien landscape with otherworldly plants, strange creatures, and distant planets.\"\n","# A negative prompt that can be used to steer the generation away from certain features.\n","negative_prompt = \"poorly drawn,cartoon, 3d, disfigured, bad art, deformed, poorly drawn, extra limbs, close up, b&w, weird colors, blurry\"\n","\n","# Generating initial latent vectors from a random normal distribution. In this example two latent vectors are generated, which will serve as start and end points for the interpolation.\n","# These vectors are shaped to fit the input requirements of the diffusion model's U-Net architecture.\n","latents = torch.randn(\n"," (2, pipe.unet.config.in_channels, height // 8, width // 8),\n"," generator=generator,\n",")\n","\n","# Getting our latent embeddings\n","interpolated_latents = slerp(latents[0], latents[1], num_interpolation_steps)\n","\n","# Generating images using the interpolated embeddings.\n","images = []\n","for latent_vector in tqdm(interpolated_latents):\n"," images.append(\n"," pipe(\n"," prompt,\n"," height=height,\n"," width=width,\n"," negative_prompt=negative_prompt,\n"," num_images_per_prompt=1,\n"," num_inference_steps=num_inference_steps,\n"," guidance_scale=guidance_scale,\n"," generator=generator,\n"," latents=latent_vector[None, ...],\n"," ).images\n"," )\n","\n","# Display of saved generated images.\n","display_images(images, save_path)"]},{"cell_type":"markdown","metadata":{"id":"sTFrAlwrDiOI"},"source":["### Example 3: Interpolation between multiple prompts\n","\n","In contrast to the first example, where we moved away from a single prompt, in this example, we will be interpolating between any number of prompts. To do so, we will take consecutive pairs of prompts and create smooth transitions between them. Then, we will combine the interpolations of these consecutive pairs, and instruct the model to generate images based on them. For interpolation we will use the slerp function, as in the second example."]},{"cell_type":"markdown","metadata":{},"source":["![Example 3](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/sd_interpolation_3.gif)"]},{"cell_type":"markdown","metadata":{},"source":["Once again, let's tokenize and obtain embeddings but this time for multiple positive and negative text prompts."]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Text prompts that describes the desired output image.\n","prompts = [\n"," \"A cute dog in a beautiful field of lavander colorful flowers everywhere, perfect lighting, leica summicron 35mm f2.0, kodak portra 400, film grain\",\n"," \"A cute cat in a beautiful field of lavander colorful flowers everywhere, perfect lighting, leica summicron 35mm f2.0, kodak portra 400, film grain\",\n","]\n","# Negative prompts that can be used to steer the generation away from certain features.\n","negative_prompts = [\n"," \"poorly drawn,cartoon, 2d, sketch, cartoon, drawing, anime, disfigured, bad art, deformed, poorly drawn, extra limbs, close up, b&w, weird colors, blurry\",\n"," \"poorly drawn,cartoon, 2d, sketch, cartoon, drawing, anime, disfigured, bad art, deformed, poorly drawn, extra limbs, close up, b&w, weird colors, blurry\",\n","]\n","\n","# NOTE: The number of prompts must match the number of negative prompts\n","\n","batch_size = len(prompts)\n","\n","# Tokenizing and encoding prompts into embeddings.\n","prompts_tokens = pipe.tokenizer(\n"," prompts,\n"," padding=\"max_length\",\n"," max_length=pipe.tokenizer.model_max_length,\n"," truncation=True,\n"," return_tensors=\"pt\",\n",")\n","prompts_embeds = pipe.text_encoder(\n"," prompts_tokens.input_ids.to(device)\n",")[0]\n","\n","# Tokenizing and encoding negative prompts into embeddings.\n","if negative_prompts is None:\n"," negative_prompts = [\"\"] * batch_size\n","\n","negative_prompts_tokens = pipe.tokenizer(\n"," negative_prompts,\n"," padding=\"max_length\",\n"," max_length=pipe.tokenizer.model_max_length,\n"," truncation=True,\n"," return_tensors=\"pt\",\n",")\n","negative_prompts_embeds = pipe.text_encoder(\n"," negative_prompts_tokens.input_ids.to(device)\n",")[0]"]},{"cell_type":"markdown","metadata":{},"source":["As stated earlier, we will take consecutive pairs of prompts and create smooth transitions between them with `slerp` function."]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":561,"referenced_widgets":["9a87e8d407f44ee59a70e511a6274131","d49efe4cff2a43288d2140a55c17c4cc","79fed3974dd8466e8237c7431f71a084","6c9624bf3faf4890bc1c83c52e33e508","f23c3781df5a497baa56f273a5f467a5","6956b01055a34613994672a6bb93994d","1f16ae9b03604ccd93cd1e2f153afe64","aa4bef85913f43c49b708c850301546d","e92a9d9050e34b47b766a31935ffbcda","94fa436d65894af494b2526746fe7324","ffe65c700f0142df9b289a0e8b58ec65"]},"id":"DfUbS8w5DiOI","outputId":"fb663c02-73e2-421a-8d07-b0b43cd548a7"},"outputs":[],"source":["# Generating initial U-Net latent vectors from a random normal distribution.\n","latents = torch.randn(\n"," (1, pipe.unet.config.in_channels, height // 8, width // 8),\n"," generator=generator,\n",")\n","\n","# Interpolating between embeddings pairs for the given number of interpolation steps.\n","interpolated_prompt_embeds = []\n","interpolated_negative_prompts_embeds = []\n","for i in range(batch_size - 1):\n"," interpolated_prompt_embeds.append(\n"," slerp(\n"," prompts_embeds[i],\n"," prompts_embeds[i + 1],\n"," num_interpolation_steps\n"," )\n"," )\n"," interpolated_negative_prompts_embeds.append(\n"," slerp(\n"," negative_prompts_embeds[i],\n"," negative_prompts_embeds[i + 1],\n"," num_interpolation_steps,\n"," )\n"," )\n","\n","interpolated_prompt_embeds = torch.cat(\n"," interpolated_prompt_embeds, dim=0\n",").to(device)\n","\n","interpolated_negative_prompts_embeds = torch.cat(\n"," interpolated_negative_prompts_embeds, dim=0\n",").to(device)"]},{"cell_type":"markdown","metadata":{},"source":["Finally, we need to generate images based on the embeddings."]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Generating images using the interpolated embeddings.\n","images = []\n","for prompt_embeds, negative_prompt_embeds in tqdm(\n"," zip(interpolated_prompt_embeds, interpolated_negative_prompts_embeds),\n"," total=len(interpolated_prompt_embeds),\n","):\n"," images.append(\n"," pipe(\n"," height=height,\n"," width=width,\n"," num_images_per_prompt=1,\n"," prompt_embeds=prompt_embeds[None, ...],\n"," negative_prompt_embeds=negative_prompt_embeds[None, ...],\n"," num_inference_steps=num_inference_steps,\n"," guidance_scale=guidance_scale,\n"," generator=generator,\n"," latents=latents,\n"," ).images\n"," )\n","\n","# Display of saved generated images.\n","display_images(images, save_path)"]},{"cell_type":"markdown","metadata":{"id":"oQqANSP2DiOI"},"source":["### Example 4: Circular walk through the diffusion latent space for a single prompt\n","\n","This example was taken from: https://keras.io/examples/generative/random_walks_with_stable_diffusion/ \n","\n","Let's imagine that we have two noise components, which we'll call x and y. We start by moving from 0 to 2π and at each step we add the cosine of x and the sine of y to the result. Using this approach, at the end of our movement we end up with the same noise values ​​that we started with. This means that vectors end up turning into themselves, ending our movement.\n","\n"]},{"cell_type":"markdown","metadata":{},"source":["![Example 4](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/sd_interpolation_4.gif)"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":561,"referenced_widgets":["eba9a67d3f704bed8f501780e35273cb","f6b5c1f44a54406c84db14875e4c85b0","1f3ce7042b974edbb033b4cd8d13cc08","6de631e7a72e4b74b740095e8c251ca8","00c30d57328148c88b4258f4841bbdd0","a4a86f212e8a4dffb0240936475837f7","89ce18ce98494ccc803adbf87f6051e5","6781679193314617a341ef891ba3df45","161a3b1a75e0446ca9120f5e1eea38e9","7294524debe544c19f8c76a7b3cf0e32","f6fb32e142d140d5ad6b357731c4d382"]},"id":"ac-68CTWDiOJ","outputId":"3eced894-bd22-443a-96e9-dedb67a40ad8"},"outputs":[],"source":["# The text prompt that describes the desired output image.\n","prompt = \"Beautiful sea sunset, warm light, Aivazovsky style\"\n","# A negative prompt that can be used to steer the generation away from certain features\n","negative_prompt = \"picture frames\"\n","\n","# Generating initial latent vectors from a random normal distribution to create a loop interpolation between them.\n","latents = torch.randn(\n"," (2, 1, pipe.unet.config.in_channels, height // 8, width // 8),\n"," generator=generator,\n",")\n","\n","\n","# Calculation of looped embeddings\n","walk_noise_x = latents[0].to(device)\n","walk_noise_y = latents[1].to(device)\n","\n","# Walking on a trigonometric circle\n","walk_scale_x = torch.cos(torch.linspace(0, 2, num_interpolation_steps) * np.pi).to(\n"," device\n",")\n","walk_scale_y = torch.sin(torch.linspace(0, 2, num_interpolation_steps) * np.pi).to(\n"," device\n",")\n","\n","# Applying interpolation to noise\n","noise_x = torch.tensordot(walk_scale_x, walk_noise_x, dims=0)\n","noise_y = torch.tensordot(walk_scale_y, walk_noise_y, dims=0)\n","\n","circular_latents = noise_x + noise_y\n","\n","# Generating images using the interpolated embeddings.\n","images = []\n","for latent_vector in tqdm(circular_latents):\n"," images.append(\n"," pipe(\n"," prompt,\n"," height=height,\n"," width=width,\n"," negative_prompt=negative_prompt,\n"," num_images_per_prompt=1,\n"," num_inference_steps=num_inference_steps,\n"," guidance_scale=guidance_scale,\n"," generator=generator,\n"," latents=latent_vector,\n"," ).images\n"," )\n","\n","# Display of saved generated images.\n","display_images(images, save_path)"]},{"cell_type":"markdown","metadata":{"id":"QQnbnOokDiOJ"},"source":["## Next Steps \n","Moving forward, you can explore various parameters such as guidance scale, seed, and number of interpolation steps to observe how they affect the generated images. Additionally, consider trying out different prompts and schedulers to further enhance your results. Another valuable step would be to implement linear interpolation (`linspace`) instead of spherical linear interpolation (`slerp`) and compare the results to gain deeper insights into the interpolation process."]}],"metadata":{"accelerator":"GPU","colab":{"gpuType":"T4","provenance":[]},"kaggle":{"accelerator":"gpu","dataSources":[],"dockerImageVersionId":30648,"isGpuEnabled":true,"isInternetEnabled":true,"language":"python","sourceType":"notebook"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.11"},"widgets":{"application/vnd.jupyter.widget-state+json":{"0094122c8eaf47e6851a3449e2fb0086":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"00b88de69713463485206e4b7c8c3c04":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_95d1047a0a644262aa385987c9a331b4","placeholder":"​","style":"IPY_MODEL_c565ba8910c34e138c6cd10e9b06d673","value":"safety_checker/config.json: 100%"}},"00c30d57328148c88b4258f4841bbdd0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0220a7b0d67a482a8ab7e9ac6372d380":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"034a1eb240694704a8052783583caefe":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"03fdfd9eb2e343df8af4ff2b06ac8eb1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0755693c5b854000af081f2818162683":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_393b21bd730f4a8b93d1653ebafaea09","placeholder":"​","style":"IPY_MODEL_0094122c8eaf47e6851a3449e2fb0086","value":" 525k/525k [00:02<00:00, 233kB/s]"}},"0ba055eeeed04e91b684072740e07f0a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_309d89a221ab4010af0d85d3ae90a1c3","max":14,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1cdaf6fe1392453288997f672af80a0d","value":14}},"0ba17f55cbf941feaa0b7e8959a94591":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0e63faca29cb4ca3845e402d67592b8f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f736f2fc44444ebf832fb2ead6ea0fd0","max":541,"min":0,"orientation":"horizontal","style":"IPY_MODEL_c41432dd085f41aeaa1e1d9cac4872e7","value":541}},"0eaebd8b9eb14feca8a11a8c3077b196":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1325ea426c9747828601a9175a9b0248":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c14c4f3d31a447b38d359acc6e29496d","placeholder":"​","style":"IPY_MODEL_f4f167561c2d452785b6e59c2ce61b28","value":" 806/806 [00:00<00:00, 18.5kB/s]"}},"161a3b1a75e0446ca9120f5e1eea38e9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1638b8dbe07a4d249167a3d34ac9adc6":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"163ec8057136471bb1f460d657c4aa6c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"16c7e883c9ec49329b68e61b276d5fac":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b18d19fb59d8458c9e9048c1458ee95c","placeholder":"​","style":"IPY_MODEL_d5a23fbc6e634b02bf4f2540e9def457","value":" 492M/492M [00:14<00:00, 20.5MB/s]"}},"16e98aec73d048d693fd0e44df2220e7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"17ceef0b617c4f52aa0bb5ec12113fcd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_45d877fafa5a4772ba5d62557843bb51","max":472,"min":0,"orientation":"horizontal","style":"IPY_MODEL_62d5802bdc7a49efbcb47889e29e924c","value":472}},"184d9e1fa8d241c386567300db4e2c8c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1ad180dc107946b9a1b554a4b98ee514":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1c3a77c578d644b09771446ed559c575":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b08c232e469646f89a28f4371f0e7699","IPY_MODEL_5e37d843836b42dfb62f728181ee4dfa","IPY_MODEL_0755693c5b854000af081f2818162683"],"layout":"IPY_MODEL_2e194e5ccbd349b093481e24148de89c"}},"1c8a25c7f70145df9722d3702fc6d2dd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c1a5498ffe0f407397d2a8c77e35de84","placeholder":"​","style":"IPY_MODEL_5eefef112ef249cc97ecd63f19ea122d","value":"tokenizer/tokenizer_config.json: 100%"}},"1cdaf6fe1392453288997f672af80a0d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1e4ad02a5afb431a96626230081054cc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1e64cce9ffc94f23921f964288c2e26d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1ebb24b8e6f44608ae8c11747ef7c42d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_80509211c7dc47e8a673ed80fca6d8e1","placeholder":"​","style":"IPY_MODEL_e1f4e7fa8b1f4530b580f101a9fad304","value":" 547/547 [00:00<00:00, 8.12kB/s]"}},"1f16ae9b03604ccd93cd1e2f153afe64":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1f3ce7042b974edbb033b4cd8d13cc08":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_6781679193314617a341ef891ba3df45","max":30,"min":0,"orientation":"horizontal","style":"IPY_MODEL_161a3b1a75e0446ca9120f5e1eea38e9","value":30}},"20c46fc3077b4dcc855252c222002569":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_9b597fc5d5cf4fecabcfbc7a4cfa1ee9","IPY_MODEL_17ceef0b617c4f52aa0bb5ec12113fcd","IPY_MODEL_e20cd706fa304bc190e92ae7d27b5b38"],"layout":"IPY_MODEL_9a3f552babd34cdeb8fed4ce1b1b33a7"}},"21e5a4fd28ac47a190d0f41808dd75a1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a76a9fce4af34c639327e5a0f4f4e692","max":30,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e00e5537ae9a43d5956c2c770599edde","value":30}},"221facd121a14faf9d664f644935b0ae":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e431b2a589524545a5ccbb79d2c7bab9","max":334643276,"min":0,"orientation":"horizontal","style":"IPY_MODEL_90dccdfd4085472f8f9ba0535d85d327","value":334643276}},"2642af1a55cf452a93e528fb25f1c8cb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"2c5d8801da6f4d88be3801254c3e764b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2dc63f0fe271457f890fd2067631ad75":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2e194e5ccbd349b093481e24148de89c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2e72912929c14d9f8843bc028b75de77":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f23d6c684c3a416d8b43fc642653eec8","placeholder":"​","style":"IPY_MODEL_16e98aec73d048d693fd0e44df2220e7","value":"vae/config.json: 100%"}},"2f010760c1a146238f35af00568ccb11":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2f6e125affb54674b5f49beaf3612bee":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2fd8069de9754a87ae04ec7b2c4b380a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"309d89a221ab4010af0d85d3ae90a1c3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"31107fa83a974eca83fa968ae4eae909":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3214f06917264be88837314b26375a1c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_00b88de69713463485206e4b7c8c3c04","IPY_MODEL_f63bebf985a046ce9ba9d567db0b7ca4","IPY_MODEL_a49fd3377dde41dab42677091dc7bd04"],"layout":"IPY_MODEL_ed1dbb98c8d34dce8df70e4698a21911"}},"3537007206fd4d57ae492d29d90bd904":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_85bf2410c7d2440db76163fe1df4f4bb","IPY_MODEL_c2bf5a15732a4898915b0ec3cb56df8c","IPY_MODEL_bf573d9fcbac4701b31e464373fdbeb0"],"layout":"IPY_MODEL_f1f352e6964f424f9e6a4557f6e3ff97"}},"3538495833744e9eab9b25eade484603":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8db4aafac51043598a9e4b8915f4f7c4","IPY_MODEL_0e63faca29cb4ca3845e402d67592b8f","IPY_MODEL_bf4b67c0a0034b1ab576066d366d934e"],"layout":"IPY_MODEL_e09eaa48528a48629b19020aa65c5bdc"}},"35cf8424313d43dea56ca590edf70b26":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_5dc6caab4749403aab23ce95993f9ad0","IPY_MODEL_84e32b4370a94fe498b3b01b6632daf3","IPY_MODEL_16c7e883c9ec49329b68e61b276d5fac"],"layout":"IPY_MODEL_5518095d648d4b2b95647c746d70c4d8"}},"35ffc6d955a44422a06e5c304fcaeddb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"20px"}},"36a0330c9e1d48a49a739feaef34ddc0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"393b21bd730f4a8b93d1653ebafaea09":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3959b0871ea840839a383c895cfbe916":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"399fdb41f7fd4f1489c1bf4814b53907":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2fd8069de9754a87ae04ec7b2c4b380a","placeholder":"​","style":"IPY_MODEL_da6373c3704d45869e2fafdf9772d3d1","value":" 617/617 [00:00<00:00, 7.14kB/s]"}},"3b99c2eba3ab4ea2a44b23ec0916ce2d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3cd43805ef564f6696905a2465ea4467":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_da25ec24a6b44ce9a51bcc2d440d0258","max":7,"min":0,"orientation":"horizontal","style":"IPY_MODEL_41ea6163d1b04f8f89cd1f9ec9e72847","value":7}},"3e7caf19c664461e9505cbfcb708ceba":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_69ffdc3c18f5484cad6945a77b024529","placeholder":"​","style":"IPY_MODEL_2c5d8801da6f4d88be3801254c3e764b","value":"100%"}},"3fcfdbffff6149fe880b0702cf8162f3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"41ea6163d1b04f8f89cd1f9ec9e72847":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"4262f099aab24cfd9b3790864e0e1d63":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_867f3d155da9469ab9820923e40e78e5","placeholder":"​","style":"IPY_MODEL_7d7e58bafe2c4ff6a44275c3a2ea9826","value":" 37%"}},"44a56b89efa649edb669abed3605e576":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c26781e6cc214b6b866b8f11673e9c00","IPY_MODEL_cddd14dde34c42f59ed71870e558246e","IPY_MODEL_399fdb41f7fd4f1489c1bf4814b53907"],"layout":"IPY_MODEL_f7f49c60f4ca41efa9a3b6b33c418f73"}},"45d877fafa5a4772ba5d62557843bb51":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"46df7a569a984f2a8d60b021e2366550":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"495a6de427ce47eda18a1570fd8f5f9d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4a466456e448417a8b3cc442cec49632":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4ab2093e02704b748298a6d34e807847":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4b14e789cc2446eda7a94a5b1259e738":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_865eb0b3254246948f532e2c6dd02bd1","IPY_MODEL_7e678fd9b4284bf39ecd29de4d7624a3","IPY_MODEL_adfbabeb20a3428c8fd6ec5b79830c34"],"layout":"IPY_MODEL_6a012f24803646369ac97be41e5998f3"}},"4cc69a88629f41dc80c552b58d3f5eaf":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6cc2e02ee7b74f3aa2f71bf0725190de","IPY_MODEL_da714c09550f43849e5a2502b092403a","IPY_MODEL_7370d528153e473980ac0e701c9e6825"],"layout":"IPY_MODEL_0ba17f55cbf941feaa0b7e8959a94591"}},"4d26d7dd13d148b3b6e06f10b589f2a8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"51061a6b42584e2bbac74da6a6f2a1da":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"51ceac8abb23437f952e07d2daaf0dae":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"521ecdf554b84958ba4ee487c9621ffc":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5518095d648d4b2b95647c746d70c4d8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"55a116f1fa634632a061a4ad8bb75ec3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e03272733bd84e2e9edcb83391e1cfae","max":342,"min":0,"orientation":"horizontal","style":"IPY_MODEL_184d9e1fa8d241c386567300db4e2c8c","value":342}},"566d76e028b643e18729621e82531939":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"57d02fadf8fc4df3b1fbee18e45f9199":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"58910b48c70a4b3dabf87b6a12004e70":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_51ceac8abb23437f952e07d2daaf0dae","placeholder":"​","style":"IPY_MODEL_a06b4275e1444e72b921d631feea90ba","value":" 342/342 [00:00<00:00, 2.75kB/s]"}},"59081d8cdf4e43228a20f3fe986926b2":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"590b0de8d8f84d1ab9a243d88380c295":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5ce301bccee049cf9664800d63e2e2eb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5dc635e136f741608e73455581822408":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5dc6caab4749403aab23ce95993f9ad0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_de350491cd2e48a19541f6e97b9f176e","placeholder":"​","style":"IPY_MODEL_cd7fa6f0b6844fd4a41b97cbe39e0c2f","value":"model.safetensors: 100%"}},"5e37d843836b42dfb62f728181ee4dfa":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_034a1eb240694704a8052783583caefe","max":524619,"min":0,"orientation":"horizontal","style":"IPY_MODEL_03fdfd9eb2e343df8af4ff2b06ac8eb1","value":524619}},"5eefef112ef249cc97ecd63f19ea122d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5ffaa53c74d3413b8f3c3fe2fc5cc075":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"60aa6e24133c4e67957bf953e5b10f4d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"62d5802bdc7a49efbcb47889e29e924c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6401b1635d62478ba644075a32494384":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_72a5fcb23802406eb22e2758be01052c","IPY_MODEL_0ba055eeeed04e91b684072740e07f0a","IPY_MODEL_790424c8674d42d59c75ba2fd4021e3a"],"layout":"IPY_MODEL_ff697b2e046a4a36a78ac362cae5c2a9"}},"640b691e30b844ec943995160216e28b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4262f099aab24cfd9b3790864e0e1d63","IPY_MODEL_a2194da8bc254658b9db17e19dbe418b","IPY_MODEL_f0e13bd4abca444592850390651272a4"],"layout":"IPY_MODEL_866b048905164e31a5011bdc0fcf5180"}},"66a43c50816c4e3fa88852c3d2c3b0c7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"670c96aa182f4defbfce4a9fd0b7f96b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6781679193314617a341ef891ba3df45":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"681cc61581d84298b4798b5c43818e31":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6826628eb6214b57bbe56e3eb80322b3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"683585ba77df463a9bcb2f8ce4794747":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6956b01055a34613994672a6bb93994d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"69ffdc3c18f5484cad6945a77b024529":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a012f24803646369ac97be41e5998f3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a7f80bb3e534eb2a48d3d29c9ac3988":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6c9624bf3faf4890bc1c83c52e33e508":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_94fa436d65894af494b2526746fe7324","placeholder":"​","style":"IPY_MODEL_ffe65c700f0142df9b289a0e8b58ec65","value":" 30/30 [05:25<00:00, 10.77s/it]"}},"6cc2e02ee7b74f3aa2f71bf0725190de":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a903c8a2c22c48bdb70024665bc1cb0f","placeholder":"​","style":"IPY_MODEL_8c29df6a485946c090b39b49240427fd","value":"tokenizer/vocab.json: 100%"}},"6d9421a31914451cac51c71aee8b1ce4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_566d76e028b643e18729621e82531939","max":3438167540,"min":0,"orientation":"horizontal","style":"IPY_MODEL_2642af1a55cf452a93e528fb25f1c8cb","value":3438167540}},"6de631e7a72e4b74b740095e8c251ca8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7294524debe544c19f8c76a7b3cf0e32","placeholder":"​","style":"IPY_MODEL_f6fb32e142d140d5ad6b357731c4d382","value":" 30/30 [05:46<00:00, 11.49s/it]"}},"6fa7c3c07e734867ac5676b09b6804b3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"70cf0bb1ada946dd9717fc2a493b7805":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8caab95721354f08999bea8dc6105b4e","IPY_MODEL_221facd121a14faf9d664f644935b0ae","IPY_MODEL_bac1fcd4cf1847ed89bc5e01ae435e24"],"layout":"IPY_MODEL_bc4ed6312ae44ba7a9e21d43d7edd48a"}},"7294524debe544c19f8c76a7b3cf0e32":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"72a5fcb23802406eb22e2758be01052c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_eb3e42c270b04086a2a04ae286684f6b","placeholder":"​","style":"IPY_MODEL_d9816365a87340dfa06fe9a37811a81b","value":"Fetching 14 files: 100%"}},"7370d528153e473980ac0e701c9e6825":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0eaebd8b9eb14feca8a11a8c3077b196","placeholder":"​","style":"IPY_MODEL_51061a6b42584e2bbac74da6a6f2a1da","value":" 1.06M/1.06M [00:00<00:00, 4.59MB/s]"}},"757864720c4041c6a24f8aa8f1630e69":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7850ea0076da49639ca986a4885d7048":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f04f25b18eaa43f6a2044dac7aba8372","IPY_MODEL_6d9421a31914451cac51c71aee8b1ce4","IPY_MODEL_a4b3ef66956d494887e796f87b4278f1"],"layout":"IPY_MODEL_1638b8dbe07a4d249167a3d34ac9adc6"}},"790424c8674d42d59c75ba2fd4021e3a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5dc635e136f741608e73455581822408","placeholder":"​","style":"IPY_MODEL_670c96aa182f4defbfce4a9fd0b7f96b","value":" 14/14 [00:33<00:00,  2.49s/it]"}},"79fed3974dd8466e8237c7431f71a084":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_aa4bef85913f43c49b708c850301546d","max":30,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e92a9d9050e34b47b766a31935ffbcda","value":30}},"7d7e58bafe2c4ff6a44275c3a2ea9826":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7e678fd9b4284bf39ecd29de4d7624a3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_66a43c50816c4e3fa88852c3d2c3b0c7","max":743,"min":0,"orientation":"horizontal","style":"IPY_MODEL_46df7a569a984f2a8d60b021e2366550","value":743}},"7f4317fe6eca4fc5be524453e55103bd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6fa7c3c07e734867ac5676b09b6804b3","placeholder":"​","style":"IPY_MODEL_ed410d69e8e94af7be0d104c5c29a2c9","value":" 30/30 [05:55<00:00, 11.65s/it]"}},"80509211c7dc47e8a673ed80fca6d8e1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"84e32b4370a94fe498b3b01b6632daf3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_683585ba77df463a9bcb2f8ce4794747","max":492265874,"min":0,"orientation":"horizontal","style":"IPY_MODEL_3b99c2eba3ab4ea2a44b23ec0916ce2d","value":492265874}},"85bf2410c7d2440db76163fe1df4f4bb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fa5231429aa1437983dc93dc597e698e","placeholder":"​","style":"IPY_MODEL_fe24820349a0456ca103e30024490c0e","value":""}},"865eb0b3254246948f532e2c6dd02bd1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_521ecdf554b84958ba4ee487c9621ffc","placeholder":"​","style":"IPY_MODEL_c6a66e5c516c4dbbbc1ca203c6a2d0db","value":"unet/config.json: 100%"}},"866b048905164e31a5011bdc0fcf5180":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"867f3d155da9469ab9820923e40e78e5":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"88c6c5bcc44d46089aa3efaa7fb9e452":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_e0280b4f0172481ea7664bfb96d1bb1c","IPY_MODEL_3cd43805ef564f6696905a2465ea4467","IPY_MODEL_ab7f8b09b1d8452995d66e6f0df83faa"],"layout":"IPY_MODEL_3959b0871ea840839a383c895cfbe916"}},"89ce18ce98494ccc803adbf87f6051e5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8c29df6a485946c090b39b49240427fd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8c690a7356af4547902b72aa6a20328d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8c9b21767c5d4741b717b82f1e4a0e03":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a83f68f1b1d640298019af11c5198a7d","IPY_MODEL_55a116f1fa634632a061a4ad8bb75ec3","IPY_MODEL_58910b48c70a4b3dabf87b6a12004e70"],"layout":"IPY_MODEL_2dc63f0fe271457f890fd2067631ad75"}},"8caab95721354f08999bea8dc6105b4e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fd4eebbe68204eaa802186836c372b93","placeholder":"​","style":"IPY_MODEL_6a7f80bb3e534eb2a48d3d29c9ac3988","value":"diffusion_pytorch_model.safetensors: 100%"}},"8db4aafac51043598a9e4b8915f4f7c4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_57d02fadf8fc4df3b1fbee18e45f9199","placeholder":"​","style":"IPY_MODEL_2f6e125affb54674b5f49beaf3612bee","value":"model_index.json: 100%"}},"909ad3aefa5b4c65931f300b3e9655dd":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"90dccdfd4085472f8f9ba0535d85d327":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"92d3f7562ee04b789c44656f83d12528":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_bd7b9e9757d24c568432eea484271376","IPY_MODEL_9c12a4d25771464b84b35804695fd50a","IPY_MODEL_fa1d984ee1864975a4c543f1dcb3aa42"],"layout":"IPY_MODEL_b89baafcd7e944b088bf0c9b1e839ba5"}},"92e5153340a74fc9895d4f87b68e3cad":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"93159e8e4bf84bc9a35c4325dc6cc851":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"94fa436d65894af494b2526746fe7324":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"95d1047a0a644262aa385987c9a331b4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"96d5032cdaa14cdeb110f8fc3b6614c1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"97fe7d3ccc984694aa42a56ba930c64e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9a3f552babd34cdeb8fed4ce1b1b33a7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9a87e8d407f44ee59a70e511a6274131":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d49efe4cff2a43288d2140a55c17c4cc","IPY_MODEL_79fed3974dd8466e8237c7431f71a084","IPY_MODEL_6c9624bf3faf4890bc1c83c52e33e508"],"layout":"IPY_MODEL_f23c3781df5a497baa56f273a5f467a5"}},"9b597fc5d5cf4fecabcfbc7a4cfa1ee9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f8db6aae6e3a468ba3805991e19e1f45","placeholder":"​","style":"IPY_MODEL_681cc61581d84298b4798b5c43818e31","value":"tokenizer/special_tokens_map.json: 100%"}},"9c12a4d25771464b84b35804695fd50a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_909ad3aefa5b4c65931f300b3e9655dd","max":1215981830,"min":0,"orientation":"horizontal","style":"IPY_MODEL_93159e8e4bf84bc9a35c4325dc6cc851","value":1215981830}},"9d2ff155648146058d2d359e474159dc":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9ef6f8a591244419916d980d5883e03e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a06b4275e1444e72b921d631feea90ba":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a2194da8bc254658b9db17e19dbe418b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_cc9fdbf01697491f856a33e4b70a7a78","max":30,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6826628eb6214b57bbe56e3eb80322b3","value":11}},"a428e985410341e0ba04359af465681e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a49fd3377dde41dab42677091dc7bd04":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bf1489df7c98442caadd2417026bffdf","placeholder":"​","style":"IPY_MODEL_590b0de8d8f84d1ab9a243d88380c295","value":" 4.72k/4.72k [00:00<00:00, 60.4kB/s]"}},"a4a86f212e8a4dffb0240936475837f7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a4b3ef66956d494887e796f87b4278f1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a428e985410341e0ba04359af465681e","placeholder":"​","style":"IPY_MODEL_bfbec38740f8413d93469274aeccbf23","value":" 3.44G/3.44G [00:32<00:00, 246MB/s]"}},"a5926a9d27b44a2d961f36d7fd36da15":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a76a9fce4af34c639327e5a0f4f4e692":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a83f68f1b1d640298019af11c5198a7d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3fcfdbffff6149fe880b0702cf8162f3","placeholder":"​","style":"IPY_MODEL_eb96b8f49d6746ec8f46e65e59a3fad6","value":"(…)ature_extractor/preprocessor_config.json: 100%"}},"a903c8a2c22c48bdb70024665bc1cb0f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aa4bef85913f43c49b708c850301546d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ab7f8b09b1d8452995d66e6f0df83faa":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_31107fa83a974eca83fa968ae4eae909","placeholder":"​","style":"IPY_MODEL_36a0330c9e1d48a49a739feaef34ddc0","value":" 7/7 [00:02<00:00,  3.52it/s]"}},"adfbabeb20a3428c8fd6ec5b79830c34":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_cffa386bd54e4baa8947d4d51c8e54a4","placeholder":"​","style":"IPY_MODEL_a5926a9d27b44a2d961f36d7fd36da15","value":" 743/743 [00:00<00:00, 10.7kB/s]"}},"ae44d548c5164e8fb5e85f1ab19da9ac":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_defa4622267e49b8a54d6aceea082c39","max":806,"min":0,"orientation":"horizontal","style":"IPY_MODEL_59081d8cdf4e43228a20f3fe986926b2","value":806}},"aedf99983c5043fe8d634b6e3b56e1ae":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"b08c232e469646f89a28f4371f0e7699":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9d2ff155648146058d2d359e474159dc","placeholder":"​","style":"IPY_MODEL_0220a7b0d67a482a8ab7e9ac6372d380","value":"tokenizer/merges.txt: 100%"}},"b0bcd8201363473ea0e4ace230443446":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b18d19fb59d8458c9e9048c1458ee95c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b89baafcd7e944b088bf0c9b1e839ba5":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bac1fcd4cf1847ed89bc5e01ae435e24":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d1d564e827cf4a71af9aa87b9d5696c8","placeholder":"​","style":"IPY_MODEL_1ad180dc107946b9a1b554a4b98ee514","value":" 335M/335M [00:11<00:00, 15.8MB/s]"}},"bc4ed6312ae44ba7a9e21d43d7edd48a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bd7b9e9757d24c568432eea484271376":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fb34d344590e43438906722b1ab958f0","placeholder":"​","style":"IPY_MODEL_ed6a871ff054410b8d018b3b97c75c60","value":"model.safetensors: 100%"}},"bf1489df7c98442caadd2417026bffdf":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bf4b67c0a0034b1ab576066d366d934e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c577a158f88040c0979b319fb9fb89b0","placeholder":"​","style":"IPY_MODEL_f24c9ef7c3f04da783aaffd3e5d48de7","value":" 541/541 [00:00<00:00, 29.0kB/s]"}},"bf573d9fcbac4701b31e464373fdbeb0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_96d5032cdaa14cdeb110f8fc3b6614c1","placeholder":"​","style":"IPY_MODEL_4a466456e448417a8b3cc442cec49632","value":" 0/0 [00:00<?, ?it/s]"}},"bfbec38740f8413d93469274aeccbf23":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c14c4f3d31a447b38d359acc6e29496d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c1a5498ffe0f407397d2a8c77e35de84":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c26781e6cc214b6b866b8f11673e9c00":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9ef6f8a591244419916d980d5883e03e","placeholder":"​","style":"IPY_MODEL_4d26d7dd13d148b3b6e06f10b589f2a8","value":"text_encoder/config.json: 100%"}},"c2bf5a15732a4898915b0ec3cb56df8c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_35ffc6d955a44422a06e5c304fcaeddb","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1e64cce9ffc94f23921f964288c2e26d","value":0}},"c41432dd085f41aeaa1e1d9cac4872e7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c523aec40bb54986a4f40924c81fe5da":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c565ba8910c34e138c6cd10e9b06d673":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c577a158f88040c0979b319fb9fb89b0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c6a66e5c516c4dbbbc1ca203c6a2d0db":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c7f341fe95ca426d9502594bd48d36f6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"cc9fdbf01697491f856a33e4b70a7a78":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cd7fa6f0b6844fd4a41b97cbe39e0c2f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"cddd14dde34c42f59ed71870e558246e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8c690a7356af4547902b72aa6a20328d","max":617,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1e4ad02a5afb431a96626230081054cc","value":617}},"cffa386bd54e4baa8947d4d51c8e54a4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d1d564e827cf4a71af9aa87b9d5696c8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d49efe4cff2a43288d2140a55c17c4cc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6956b01055a34613994672a6bb93994d","placeholder":"​","style":"IPY_MODEL_1f16ae9b03604ccd93cd1e2f153afe64","value":"100%"}},"d5a23fbc6e634b02bf4f2540e9def457":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d7c9f6b399524bc596e84641687ba29a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1c8a25c7f70145df9722d3702fc6d2dd","IPY_MODEL_ae44d548c5164e8fb5e85f1ab19da9ac","IPY_MODEL_1325ea426c9747828601a9175a9b0248"],"layout":"IPY_MODEL_97fe7d3ccc984694aa42a56ba930c64e"}},"d7eb412b880c490c95f1d2baeaf2e6af":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_3e7caf19c664461e9505cbfcb708ceba","IPY_MODEL_21e5a4fd28ac47a190d0f41808dd75a1","IPY_MODEL_7f4317fe6eca4fc5be524453e55103bd"],"layout":"IPY_MODEL_4ab2093e02704b748298a6d34e807847"}},"d9816365a87340dfa06fe9a37811a81b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d9f1220cc4f5440b97e35151ea76ed00":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"da25ec24a6b44ce9a51bcc2d440d0258":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"da6373c3704d45869e2fafdf9772d3d1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"da714c09550f43849e5a2502b092403a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_b0bcd8201363473ea0e4ace230443446","max":1059962,"min":0,"orientation":"horizontal","style":"IPY_MODEL_d9f1220cc4f5440b97e35151ea76ed00","value":1059962}},"de350491cd2e48a19541f6e97b9f176e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"defa4622267e49b8a54d6aceea082c39":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e00e5537ae9a43d5956c2c770599edde":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e0216d9707cb4d10be57c4864521c376":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_2e72912929c14d9f8843bc028b75de77","IPY_MODEL_feeaea5e7b524ed09bd08c73b7ed5e17","IPY_MODEL_1ebb24b8e6f44608ae8c11747ef7c42d"],"layout":"IPY_MODEL_c523aec40bb54986a4f40924c81fe5da"}},"e0280b4f0172481ea7664bfb96d1bb1c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_60aa6e24133c4e67957bf953e5b10f4d","placeholder":"​","style":"IPY_MODEL_5ce301bccee049cf9664800d63e2e2eb","value":"Loading pipeline components...: 100%"}},"e03272733bd84e2e9edcb83391e1cfae":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e09eaa48528a48629b19020aa65c5bdc":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e1f4e7fa8b1f4530b580f101a9fad304":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e20cd706fa304bc190e92ae7d27b5b38":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e4f21da1f63a4a819485cbe7e5ae306b","placeholder":"​","style":"IPY_MODEL_c7f341fe95ca426d9502594bd48d36f6","value":" 472/472 [00:00<00:00, 13.2kB/s]"}},"e431b2a589524545a5ccbb79d2c7bab9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e4f21da1f63a4a819485cbe7e5ae306b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e92a9d9050e34b47b766a31935ffbcda":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"eac1b421700d492ba398d5ac609b5741":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"eb3e42c270b04086a2a04ae286684f6b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"eb96b8f49d6746ec8f46e65e59a3fad6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"eba9a67d3f704bed8f501780e35273cb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f6b5c1f44a54406c84db14875e4c85b0","IPY_MODEL_1f3ce7042b974edbb033b4cd8d13cc08","IPY_MODEL_6de631e7a72e4b74b740095e8c251ca8"],"layout":"IPY_MODEL_00c30d57328148c88b4258f4841bbdd0"}},"ed1dbb98c8d34dce8df70e4698a21911":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ed410d69e8e94af7be0d104c5c29a2c9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ed6a871ff054410b8d018b3b97c75c60":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f04f25b18eaa43f6a2044dac7aba8372":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_163ec8057136471bb1f460d657c4aa6c","placeholder":"​","style":"IPY_MODEL_ff1655111fd04c4785d7e5ec3747629c","value":"diffusion_pytorch_model.safetensors: 100%"}},"f0e13bd4abca444592850390651272a4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_757864720c4041c6a24f8aa8f1630e69","placeholder":"​","style":"IPY_MODEL_92e5153340a74fc9895d4f87b68e3cad","value":" 11/30 [02:05<03:34, 11.30s/it]"}},"f1f352e6964f424f9e6a4557f6e3ff97":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f23c3781df5a497baa56f273a5f467a5":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f23d6c684c3a416d8b43fc642653eec8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f24c9ef7c3f04da783aaffd3e5d48de7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f2f9506fc02a4624a8a1c08f1f6abdb2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f4f167561c2d452785b6e59c2ce61b28":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f63bebf985a046ce9ba9d567db0b7ca4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_5ffaa53c74d3413b8f3c3fe2fc5cc075","max":4723,"min":0,"orientation":"horizontal","style":"IPY_MODEL_aedf99983c5043fe8d634b6e3b56e1ae","value":4723}},"f6b5c1f44a54406c84db14875e4c85b0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a4a86f212e8a4dffb0240936475837f7","placeholder":"​","style":"IPY_MODEL_89ce18ce98494ccc803adbf87f6051e5","value":"100%"}},"f6fb32e142d140d5ad6b357731c4d382":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f736f2fc44444ebf832fb2ead6ea0fd0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f7f49c60f4ca41efa9a3b6b33c418f73":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f8db6aae6e3a468ba3805991e19e1f45":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fa1d984ee1864975a4c543f1dcb3aa42":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f2f9506fc02a4624a8a1c08f1f6abdb2","placeholder":"​","style":"IPY_MODEL_2f010760c1a146238f35af00568ccb11","value":" 1.22G/1.22G [00:24<00:00, 110MB/s]"}},"fa5231429aa1437983dc93dc597e698e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fb34d344590e43438906722b1ab958f0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fd4eebbe68204eaa802186836c372b93":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fe24820349a0456ca103e30024490c0e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"feeaea5e7b524ed09bd08c73b7ed5e17":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_495a6de427ce47eda18a1570fd8f5f9d","max":547,"min":0,"orientation":"horizontal","style":"IPY_MODEL_eac1b421700d492ba398d5ac609b5741","value":547}},"ff1655111fd04c4785d7e5ec3747629c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ff697b2e046a4a36a78ac362cae5c2a9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ffe65c700f0142df9b289a0e8b58ec65":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":4} diff --git a/notebooks/spa/tgi_messages_api_demo.ipynb b/notebooks/spa/tgi_messages_api_demo.ipynb new file mode 100644 index 00000000..b2e53af6 --- /dev/null +++ b/notebooks/spa/tgi_messages_api_demo.ipynb @@ -0,0 +1,514 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Migrating from OpenAI to Open LLMs Using TGI's Messages API\n", + "\n", + "_Authored by: [Andrew Reed](https://huggingface.co/andrewrreed)_\n", + "\n", + "This notebook demonstrates how you can easily transition from OpenAI models to Open LLMs without needing to refactor any existing code.\n", + "\n", + "[Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference) now offers a [Messages API](https://huggingface.co/blog/tgi-messages-api), making it directly compatible with the OpenAI Chat Completion API. This means that any existing scripts that use OpenAI models (via the OpenAI client library or third-party tools like LangChain or LlamaIndex) can be directly swapped out to use any open LLM running on a TGI endpoint!\n", + "\n", + "This allows you to quickly test out and benefit from the numerous advantages offered by open models. Things like:\n", + "\n", + "- Complete control and transparency over models and data\n", + "- No more worrying about rate limits\n", + "- The ability to fully customize systems according to your specific needs\n", + "\n", + "In this notebook, we'll show you how to:\n", + "\n", + "1. [Create Inference Endpoint to Deploy a Model with TGI](#section_1)\n", + "2. [Query the Inference Endpoint with OpenAI Client Libraries](#section_2)\n", + "3. [Integrate the Endpoint with LangChain and LlamaIndex Workflows](#section_3)\n", + "\n", + "**Let's dive in!**\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "First we need to install dependencies and set an HF API key.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade -q huggingface_hub langchain langchain-community langchainhub langchain-openai llama-index chromadb bs4 sentence_transformers torch" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import getpass\n", + "\n", + "# enter API key\n", + "os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = HF_API_KEY = getpass.getpass()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "## 1. Create an Inference Endpoint\n", + "\n", + "To get started, let's deploy [Nous-Hermes-2-Mixtral-8x7B-DPO](https://huggingface.co/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO), a fine-tuned Mixtral model, to Inference Endpoints using TGI.\n", + "\n", + "We can deploy the model in just [a few clicks from the UI](https://ui.endpoints.huggingface.co/new?vendor=aws&repository=NousResearch%2FNous-Hermes-2-Mixtral-8x7B-DPO&tgi_max_total_tokens=32000&tgi=true&tgi_max_input_length=1024&task=text-generation&instance_size=2xlarge&tgi_max_batch_prefill_tokens=2048&tgi_max_batch_total_tokens=1024000&no_suggested_compute=true&accelerator=gpu®ion=us-east-1), or take advantage of the `huggingface_hub` Python library to programmatically create and manage Inference Endpoints.\n", + "\n", + "We'll use the Hub library here by specifing an endpoint name and model repository, along with the task of `text-generation`. In this example, we use a `protected` type so access to the deployed model will require a valid Hugging Face token. We also need to configure the hardware requirements like vendor, region, accelerator, instance type, and size. You can check out the list of available resource options [using this API call](https://api.endpoints.huggingface.cloud/#get-/v2/provider), and view recommended configurations for select models in the catalog [here](https://ui.endpoints.huggingface.co/catalog).\n", + "\n", + "_Note: You may need to request a quota upgrade by sending an email to [api-enterprise@huggingface.co](mailto:api-enterprise@huggingface.co)_\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "running\n" + ] + } + ], + "source": [ + "from huggingface_hub import create_inference_endpoint\n", + "\n", + "endpoint = create_inference_endpoint(\n", + " \"nous-hermes-2-mixtral-8x7b-demo\",\n", + " repository=\"NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO\",\n", + " framework=\"pytorch\",\n", + " task=\"text-generation\",\n", + " accelerator=\"gpu\",\n", + " vendor=\"aws\",\n", + " region=\"us-east-1\",\n", + " type=\"protected\",\n", + " instance_type=\"p4de\",\n", + " instance_size=\"2xlarge\",\n", + " custom_image={\n", + " \"health_route\": \"/health\",\n", + " \"env\": {\n", + " \"MAX_INPUT_LENGTH\": \"4096\",\n", + " \"MAX_BATCH_PREFILL_TOKENS\": \"4096\",\n", + " \"MAX_TOTAL_TOKENS\": \"32000\",\n", + " \"MAX_BATCH_TOTAL_TOKENS\": \"1024000\",\n", + " \"MODEL_ID\": \"/repository\",\n", + " },\n", + " \"url\": \"ghcr.io/huggingface/text-generation-inference:sha-1734540\", # must be >= 1.4.0\n", + " },\n", + ")\n", + "\n", + "endpoint.wait()\n", + "print(endpoint.status)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It will take a few minutes for our deployment to spin up. We can use the `.wait()` utility to block the running thread until the endpoint reaches a final \"running\" state. Once running, we can confirm its status and take it for a spin via the UI Playground:\n", + "\n", + "![IE UI Overview](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/messages-api/endpoint-overview.png)\n", + "\n", + "Great, we now have a working endpoint!\n", + "\n", + "_Note: When deploying with `huggingface_hub`, your endpoint will scale-to-zero after 15 minutes of idle time by default to optimize cost during periods of inactivity. Check out [the Hub Python Library documentation](https://huggingface.co/docs/huggingface_hub/guides/inference_endpoints) to see all the functionality available for managing your endpoint lifecycle._\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "## 2. Query the Inference Endpoint with OpenAI Client Libraries\n", + "\n", + "As mentioned above, since our model is hosted with TGI it now supports a Messages API meaning we can query it directly using the familiar OpenAI client libraries.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### With the Python client\n", + "\n", + "The example below shows how to make this transition using the [OpenAI Python Library](https://github.com/openai/openai-python). Simply replace the `` with your endpoint URL (be sure to include the `v1/` the suffix) and populate the `` field with a valid Hugging Face user token. The `` can be gathered from Inference Endpoints UI, or from the endpoint object we created above with `endpoint.url`.\n", + "\n", + "We can then use the client as usual, passing a list of messages to stream responses from our Inference Endpoint.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Open-source software is important due to a number of reasons, including:\n", + "\n", + "1. Collaboration: The collaborative nature of open-source software allows developers from around the world to work together, share their ideas and improve the code. This often results in faster progress and better software.\n", + "\n", + "2. Transparency: With open-source software, the code is publicly available, making it easy to see exactly how the software functions, and allowing users to determine if there are any security vulnerabilities.\n", + "\n", + "3. Customization: Being able to access the code also allows users to customize the software to better suit their needs. This makes open-source software incredibly versatile, as users can tweak it to suit their specific use case.\n", + "\n", + "4. Quality: Open-source software is often developed by large communities of dedicated developers, who work together to improve the software. This results in a higher level of quality than might be found in proprietary software.\n", + "\n", + "5. Cost: Open-source software is often provided free of charge, which makes it accessible to a wider range of users. This can be especially important for organizations with limited budgets for software.\n", + "\n", + "6. Shared Benefit: By sharing the code of open-source software, everyone can benefit from the hard work of the developers. This contributes to the overall advancement of technology, as users and developers work together to improve and build upon the software.\n", + "\n", + "In summary, open-source software provides a collaborative platform that leads to high-quality, customizable, and transparent software, all available at little or no cost, benefiting both individuals and the technology community as a whole.<|im_end|>" + ] + } + ], + "source": [ + "from openai import OpenAI\n", + "\n", + "BASE_URL = endpoint.url\n", + "\n", + "# init the client but point it to TGI\n", + "client = OpenAI(\n", + " base_url=os.path.join(BASE_URL, \"v1/\"),\n", + " api_key=HF_API_KEY,\n", + ")\n", + "chat_completion = client.chat.completions.create(\n", + " model=\"tgi\",\n", + " messages=[\n", + " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n", + " {\"role\": \"user\", \"content\": \"Why is open-source software important?\"},\n", + " ],\n", + " stream=True,\n", + " max_tokens=500,\n", + ")\n", + "\n", + "# iterate and print stream\n", + "for message in chat_completion:\n", + " print(message.choices[0].delta.content, end=\"\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Behind the scenes, TGI’s Messages API automatically converts the list of messages into the model’s required instruction format using its [chat template](https://huggingface.co/docs/transformers/chat_templating).\n", + "\n", + "_Note: Certain OpenAI features, like function calling, are not compatible with TGI. Currently, the Messages API supports the following chat completion parameters: `stream`, `max_new_tokens`, `frequency_penalty`, `logprobs`, `seed`, `temperature`, and `top_p`._\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### With the JavaScript client\n", + "\n", + "Here’s the same streaming example above, but using the [OpenAI Javascript/Typescript Library](https://github.com/openai/openai-node).\n", + "\n", + "```js\n", + "import OpenAI from \"openai\";\n", + "\n", + "const openai = new OpenAI({\n", + " baseURL: \"\" + \"/v1/\", // replace with your endpoint url\n", + " apiKey: \"\", // replace with your token\n", + "});\n", + "\n", + "async function main() {\n", + " const stream = await openai.chat.completions.create({\n", + " model: \"tgi\",\n", + " messages: [\n", + " { role: \"system\", content: \"You are a helpful assistant.\" },\n", + " { role: \"user\", content: \"Why is open-source software important?\" },\n", + " ],\n", + " stream: true,\n", + " max_tokens: 500,\n", + " });\n", + " for await (const chunk of stream) {\n", + " process.stdout.write(chunk.choices[0]?.delta?.content || \"\");\n", + " }\n", + "}\n", + "\n", + "main();\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "## 3. Integrate with LangChain and LlamaIndex\n", + "\n", + "Now, let’s see how to use this newly created endpoint with popular RAG frameworks like LangChain and LlamaIndex.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### How to use with LangChain\n", + "\n", + "To use it in [LangChain](https://python.langchain.com/docs/get_started/introduction), simply create an instance of `ChatOpenAI` and pass your `` and `` as follows:\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the underlying code, making it easier to understand how the software works and identify any potential security vulnerabilities or bugs. This transparency fosters trust between users and developers.\\n\\n2. Collaboration: Open-source projects encourage collaboration among developers, allowing them to work together to improve the software, fix issues, and add new features. This collective effort can lead to')" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(\n", + " model_name=\"tgi\",\n", + " openai_api_key=HF_API_KEY,\n", + " openai_api_base=os.path.join(BASE_URL, \"v1/\"),\n", + ")\n", + "llm.invoke(\"Why is open-source software important?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We’re able to directly leverage the same `ChatOpenAI` class that we would have used with the OpenAI models. This allows all previous code to work with our endpoint by changing just one line of code.\n", + "\n", + "Let’s now use our Mixtral model in a simple RAG pipeline to answer a question over the contents of a HF blog post.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'context': [Document(page_content='To overcome this weakness, amongst other approaches, one can integrate the LLM into a system where it can call tools: such a system is called an LLM agent.\\nIn this post, we explain the inner workings of ReAct agents, then show how to build them using the ChatHuggingFace class recently integrated in LangChain. Finally, we benchmark several open-source LLMs against GPT-3.5 and GPT-4.', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n", + " Document(page_content='Since the open-source models were not specifically fine-tuned for calling functions in the given output format, they are at a slight disadvantage compared to the OpenAI agents.\\nDespite this, some models perform really well! 💪\\nHere’s an example of Mixtral-8x7B answering the question: “Which city has a larger population, Guiyang or Tacheng?”\\nThought: To answer this question, I need to find the current populations of both Guiyang and Tacheng. I will use the search tool to find this information.\\nAction:\\n{', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n", + " Document(page_content='Agents Showdown: how do open-source LLMs perform as general purpose reasoning agents?\\n\\t\\n\\nYou can find the code for this benchmark here.\\n\\n\\n\\n\\n\\n\\t\\tEvaluation\\n\\t\\n\\nWe want to measure how open-source LLMs perform as general purpose reasoning agents. Thus we select questions requiring using logic and the use of basic tools: a calculator and access to internet search.\\nThe final dataset is a combination of samples from 3 other datasets:', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n", + " Document(page_content='Open-source LLMs as LangChain Agents\\n\\t\\n\\nPublished\\n\\t\\t\\t\\tJanuary 24, 2024\\nUpdate on GitHub\\n\\nm-ric\\nAymeric Roucher\\n\\n\\n\\n\\nJofthomas\\nJoffrey THOMAS\\n\\n\\n\\n\\nandrewrreed\\nAndrew Reed\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\t\\tTL;DR\\n\\t\\n\\nOpen-source LLMs have now reached a performance level that makes them suitable reasoning engines for powering agent workflows: Mixtral even surpasses GPT-3.5 on our benchmark, and its performance could easily be further enhanced with fine-tuning.\\n\\n\\n\\n\\n\\n\\t\\tIntroduction', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'})],\n", + " 'question': 'According to this article which open-source model is the best for an agent behaviour?',\n", + " 'answer': 'According to the article, Mixtral-8x7B is an open-source LLM that performs really well as a general-purpose reasoning agent. It even surpasses GPT-3.5 on the benchmark in the article.'}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain import hub\n", + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "from langchain_community.document_loaders import WebBaseLoader\n", + "from langchain_community.vectorstores import Chroma\n", + "from langchain_core.output_parsers import StrOutputParser\n", + "from langchain_core.runnables import RunnablePassthrough\n", + "from langchain_core.runnables import RunnableParallel\n", + "from langchain_community.embeddings import HuggingFaceEmbeddings\n", + "\n", + "# Load, chunk and index the contents of the blog\n", + "loader = WebBaseLoader(\n", + " web_paths=(\"https://huggingface.co/blog/open-source-llms-as-agents\",),\n", + ")\n", + "docs = loader.load()\n", + "\n", + "# declare an HF embedding model\n", + "hf_embeddings = HuggingFaceEmbeddings(model_name=\"BAAI/bge-large-en-v1.5\")\n", + "\n", + "text_splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=200)\n", + "splits = text_splitter.split_documents(docs)\n", + "vectorstore = Chroma.from_documents(documents=splits, embedding=hf_embeddings)\n", + "\n", + "# Retrieve and generate using the relevant snippets of the blog\n", + "retriever = vectorstore.as_retriever()\n", + "prompt = hub.pull(\"rlm/rag-prompt\")\n", + "\n", + "\n", + "def format_docs(docs):\n", + " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", + "\n", + "\n", + "rag_chain_from_docs = (\n", + " RunnablePassthrough.assign(context=(lambda x: format_docs(x[\"context\"])))\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")\n", + "\n", + "rag_chain_with_source = RunnableParallel(\n", + " {\"context\": retriever, \"question\": RunnablePassthrough()}\n", + ").assign(answer=rag_chain_from_docs)\n", + "\n", + "rag_chain_with_source.invoke(\n", + " \"According to this article which open-source model is the best for an agent behaviour?\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### How to use with LlamaIndex\n", + "\n", + "Similarly, you can also use a TGI endpoint in [LlamaIndex](https://www.llamaindex.ai/). We’ll use the `OpenAILike` class, and instantiate it by configuring some additional arguments (i.e. `is_local`, `is_function_calling_model`, `is_chat_model`, `context_window`).\n", + "\n", + "_Note: that the context window argument should match the value previously set for `MAX_TOTAL_TOKENS` of your endpoint._\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "CompletionResponse(text='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the source code, which means they can understand how the software works and how it processes data. This transparency helps build trust in the software and its developers.\\n\\n2. Collaboration: Open-source software encourages collaboration among developers, who can contribute to the code, fix bugs, and add new features. This collaborative approach often leads to faster development and', additional_kwargs={}, raw={'id': '', 'choices': [Choice(finish_reason='length', index=0, logprobs=None, message=ChatCompletionMessage(content='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the source code, which means they can understand how the software works and how it processes data. This transparency helps build trust in the software and its developers.\\n\\n2. Collaboration: Open-source software encourages collaboration among developers, who can contribute to the code, fix bugs, and add new features. This collaborative approach often leads to faster development and', role='assistant', function_call=None, tool_calls=None))], 'created': 1707342025, 'model': '/repository', 'object': 'text_completion', 'system_fingerprint': '1.4.0-sha-1734540', 'usage': CompletionUsage(completion_tokens=100, prompt_tokens=18, total_tokens=118)}, delta=None)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from llama_index.llms import OpenAILike\n", + "\n", + "llm = OpenAILike(\n", + " model=\"tgi\",\n", + " api_key=HF_API_KEY,\n", + " api_base=BASE_URL + \"/v1/\",\n", + " is_chat_model=True,\n", + " is_local=False,\n", + " is_function_calling_model=False,\n", + " context_window=4096,\n", + ")\n", + "\n", + "llm.complete(\"Why is open-source software important?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can now use it in a similar RAG pipeline. Keep in mind that the previous choice of `MAX_INPUT_LENGTH` in your Inference Endpoint will directly influence the number of retrieved chunk (`similarity_top_k`) the model can process.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index import (\n", + " ServiceContext,\n", + " VectorStoreIndex,\n", + ")\n", + "from llama_index import download_loader\n", + "from llama_index.embeddings import HuggingFaceEmbedding\n", + "from llama_index.query_engine import CitationQueryEngine\n", + "\n", + "\n", + "SimpleWebPageReader = download_loader(\"SimpleWebPageReader\")\n", + "\n", + "documents = SimpleWebPageReader(html_to_text=True).load_data(\n", + " [\"https://huggingface.co/blog/open-source-llms-as-agents\"]\n", + ")\n", + "\n", + "# Load embedding model\n", + "embed_model = HuggingFaceEmbedding(model_name=\"BAAI/bge-large-en-v1.5\")\n", + "\n", + "# Pass LLM to pipeline\n", + "service_context = ServiceContext.from_defaults(embed_model=embed_model, llm=llm)\n", + "index = VectorStoreIndex.from_documents(\n", + " documents, service_context=service_context, show_progress=True\n", + ")\n", + "\n", + "# Query the index\n", + "query_engine = CitationQueryEngine.from_args(\n", + " index,\n", + " similarity_top_k=2,\n", + ")\n", + "response = query_engine.query(\n", + " \"According to this article which open-source model is the best for an agent behaviour?\"\n", + ")\n", + "\n", + "response.response" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Wrap up\n", + "\n", + "After you are done with your endpoint, you can either pause or delete it. This step can be completed via the UI, or programmatically like follows.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# pause our running endpoint\n", + "endpoint.pause()\n", + "\n", + "# optionally delete\n", + "# endpoint.delete()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.11" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}