|
| 1 | +{ |
| 2 | + "cells": [ |
| 3 | + { |
| 4 | + "cell_type": "markdown", |
| 5 | + "metadata": {}, |
| 6 | + "source": [ |
| 7 | + "<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/docs/examples/llm/lmstudio.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" |
| 8 | + ] |
| 9 | + }, |
| 10 | + { |
| 11 | + "cell_type": "markdown", |
| 12 | + "metadata": {}, |
| 13 | + "source": [ |
| 14 | + "# LM Studio" |
| 15 | + ] |
| 16 | + }, |
| 17 | + { |
| 18 | + "cell_type": "markdown", |
| 19 | + "metadata": {}, |
| 20 | + "source": [ |
| 21 | + "## Setup" |
| 22 | + ] |
| 23 | + }, |
| 24 | + { |
| 25 | + "cell_type": "markdown", |
| 26 | + "metadata": {}, |
| 27 | + "source": [ |
| 28 | + "1. Download and Install LM Studio\n", |
| 29 | + "2. Follow the steps mentioned in the [README](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/llms/llama-index-llms-lmstudio/README.md)." |
| 30 | + ] |
| 31 | + }, |
| 32 | + { |
| 33 | + "cell_type": "markdown", |
| 34 | + "metadata": {}, |
| 35 | + "source": [ |
| 36 | + "If not already installed in collab, install *llama-index* and *lmstudio* integration." |
| 37 | + ] |
| 38 | + }, |
| 39 | + { |
| 40 | + "cell_type": "code", |
| 41 | + "execution_count": null, |
| 42 | + "metadata": {}, |
| 43 | + "outputs": [], |
| 44 | + "source": [ |
| 45 | + "%pip install llama-index-core llama-index llama-index-llms-lmstudio" |
| 46 | + ] |
| 47 | + }, |
| 48 | + { |
| 49 | + "cell_type": "markdown", |
| 50 | + "metadata": {}, |
| 51 | + "source": [ |
| 52 | + "Fix for \"RuntimeError: This event loop is already running\"" |
| 53 | + ] |
| 54 | + }, |
| 55 | + { |
| 56 | + "cell_type": "code", |
| 57 | + "execution_count": null, |
| 58 | + "metadata": {}, |
| 59 | + "outputs": [], |
| 60 | + "source": [ |
| 61 | + "import nest_asyncio\n", |
| 62 | + "\n", |
| 63 | + "nest_asyncio.apply()" |
| 64 | + ] |
| 65 | + }, |
| 66 | + { |
| 67 | + "cell_type": "code", |
| 68 | + "execution_count": null, |
| 69 | + "metadata": {}, |
| 70 | + "outputs": [], |
| 71 | + "source": [ |
| 72 | + "from llama_index.llms.lmstudio import LMStudio\n", |
| 73 | + "from llama_index.core.base.llms.types import ChatMessage, MessageRole" |
| 74 | + ] |
| 75 | + }, |
| 76 | + { |
| 77 | + "cell_type": "code", |
| 78 | + "execution_count": null, |
| 79 | + "metadata": {}, |
| 80 | + "outputs": [], |
| 81 | + "source": [ |
| 82 | + "llm = LMStudio(\n", |
| 83 | + " model_name=\"Hermes-2-Pro-Llama-3-8B\",\n", |
| 84 | + " base_url=\"http://localhost:1234/v1\",\n", |
| 85 | + " temperature=0.7,\n", |
| 86 | + ")" |
| 87 | + ] |
| 88 | + }, |
| 89 | + { |
| 90 | + "cell_type": "code", |
| 91 | + "execution_count": null, |
| 92 | + "metadata": {}, |
| 93 | + "outputs": [ |
| 94 | + { |
| 95 | + "name": "stdout", |
| 96 | + "output_type": "stream", |
| 97 | + "text": [ |
| 98 | + "The result of 2 + 2 is 4.\n" |
| 99 | + ] |
| 100 | + } |
| 101 | + ], |
| 102 | + "source": [ |
| 103 | + "response = llm.complete(\"Hey there, what is 2+2?\")\n", |
| 104 | + "print(str(response))" |
| 105 | + ] |
| 106 | + }, |
| 107 | + { |
| 108 | + "cell_type": "code", |
| 109 | + "execution_count": null, |
| 110 | + "metadata": {}, |
| 111 | + "outputs": [ |
| 112 | + { |
| 113 | + "name": "stdout", |
| 114 | + "output_type": "stream", |
| 115 | + "text": [ |
| 116 | + "The result of 7 + 3 is 10." |
| 117 | + ] |
| 118 | + } |
| 119 | + ], |
| 120 | + "source": [ |
| 121 | + "# use llm.stream_complete\n", |
| 122 | + "response = llm.stream_complete(\"What is 7+3?\")\n", |
| 123 | + "for r in response:\n", |
| 124 | + " print(r.delta, end=\"\")" |
| 125 | + ] |
| 126 | + }, |
| 127 | + { |
| 128 | + "cell_type": "code", |
| 129 | + "execution_count": null, |
| 130 | + "metadata": {}, |
| 131 | + "outputs": [], |
| 132 | + "source": [ |
| 133 | + "messages = [\n", |
| 134 | + " ChatMessage(\n", |
| 135 | + " role=MessageRole.SYSTEM,\n", |
| 136 | + " content=\"You an expert AI assistant. Help User with their queries.\",\n", |
| 137 | + " ),\n", |
| 138 | + " ChatMessage(\n", |
| 139 | + " role=MessageRole.USER,\n", |
| 140 | + " content=\"What is the significance of the number 42?\",\n", |
| 141 | + " ),\n", |
| 142 | + "]" |
| 143 | + ] |
| 144 | + }, |
| 145 | + { |
| 146 | + "cell_type": "code", |
| 147 | + "execution_count": null, |
| 148 | + "metadata": {}, |
| 149 | + "outputs": [ |
| 150 | + { |
| 151 | + "name": "stdout", |
| 152 | + "output_type": "stream", |
| 153 | + "text": [ |
| 154 | + "assistant: The number 42 has been significant in various contexts throughout history and across different cultures, often holding symbolic or philosophical meanings. \n", |
| 155 | + "\n", |
| 156 | + "1. In mathematics: 42 is a relatively simple but still interesting whole number with no factors other than 1 and itself.\n", |
| 157 | + "\n", |
| 158 | + "2. In popular culture: Douglas Adams' science fiction series \"The Hitchhiker's Guide to the Galaxy\" presents the ultimate answer to the meaning of life as 42, which has become a well-known joke and meme since its introduction in the first book published in 1979.\n", |
| 159 | + "\n", |
| 160 | + "3. In religion and mythology: The number 42 appears in various religious texts or myths with different meanings, such as the Biblical Book of Numbers where Moses spent 42 years tending to his father-in-law's flock before receiving the call from God, or in Norse mythology when Odin spent 42 nights suspended on Yggdrasil (the World Tree) to gain knowledge.\n", |
| 161 | + "\n", |
| 162 | + "4. In sports: In baseball, a perfect game is considered to be an immaculate game with no hits, errors, or runners allowed to reach base; only 15 players can achieve this in Major League Baseball history, and the number of their names added together equals 42 (6 + 2 = 8, 3 + 4 = 7).\n", |
| 163 | + "\n", |
| 164 | + "5. In music: The English rock band Coldplay's popular song \"42\" is about lead singer Chris Martin reflecting on his age during the time it took for the band to gain success.\n", |
| 165 | + "\n", |
| 166 | + "The significance of the number 42 varies depending on the context and cultural background. It has often been used symbolically or metaphorically, making it a versatile and intriguing number in various aspects of human life.\n" |
| 167 | + ] |
| 168 | + } |
| 169 | + ], |
| 170 | + "source": [ |
| 171 | + "response = llm.chat(messages=messages)\n", |
| 172 | + "print(str(response))" |
| 173 | + ] |
| 174 | + }, |
| 175 | + { |
| 176 | + "cell_type": "code", |
| 177 | + "execution_count": null, |
| 178 | + "metadata": {}, |
| 179 | + "outputs": [ |
| 180 | + { |
| 181 | + "name": "stdout", |
| 182 | + "output_type": "stream", |
| 183 | + "text": [ |
| 184 | + "The number 42 has various significances in different contexts:\n", |
| 185 | + "\n", |
| 186 | + "1. In popular culture: The famous \"Answer to the Ultimate Question of Life, the Universe and Everything\" from Douglas Adams' science fiction series \"The Hitchhiker's Guide to the Galaxy\" is 42. This has led to widespread recognition of the number as something meaningful or profound.\n", |
| 187 | + "\n", |
| 188 | + "2. Mathematics: The number 42 is a highly composite number with many divisors (1, 2, 3, 6, 7, 14, 21, and 42). In mathematics, the study of factors and divisors plays an essential role in various concepts such as prime factorization and greatest common denominators.\n", |
| 189 | + "\n", |
| 190 | + "3. Christianity: According to a story from The Book of Kells (an illuminated manuscript), it is said that St. Patrick used the number 42 to calculate when to begin his mission to convert Ireland to Christianity.\n", |
| 191 | + "\n", |
| 192 | + "4. Astrology: In astrology, the 42nd day after the Winter Solstice marks the beginning of the new astrological year and the start of a 13-month cycle in some traditions.\n", |
| 193 | + "\n", |
| 194 | + "5. Literature: The number 42 is mentioned several times throughout William Shakespeare's plays, such as \"Hamlet\" and \"Henry IV.\" It appears as a coincidence or possibly with symbolic intent in these works.\n", |
| 195 | + "\n", |
| 196 | + "6. In the field of computer science, the popular programming language 'Python' uses 42 as its \"magic number\" to represent the start-up code for the interpreter.\n", |
| 197 | + "\n", |
| 198 | + "Each context assigns a different significance to the number 42, making it multi-faceted and culturally relevant in various ways." |
| 199 | + ] |
| 200 | + } |
| 201 | + ], |
| 202 | + "source": [ |
| 203 | + "response = llm.stream_chat(messages=messages)\n", |
| 204 | + "for r in response:\n", |
| 205 | + " print(r.delta, end=\"\")" |
| 206 | + ] |
| 207 | + } |
| 208 | + ], |
| 209 | + "metadata": { |
| 210 | + "kernelspec": { |
| 211 | + "display_name": "llama_index", |
| 212 | + "language": "python", |
| 213 | + "name": "python3" |
| 214 | + }, |
| 215 | + "language_info": { |
| 216 | + "codemirror_mode": { |
| 217 | + "name": "ipython", |
| 218 | + "version": 3 |
| 219 | + }, |
| 220 | + "file_extension": ".py", |
| 221 | + "mimetype": "text/x-python", |
| 222 | + "name": "python", |
| 223 | + "nbconvert_exporter": "python", |
| 224 | + "pygments_lexer": "ipython3" |
| 225 | + } |
| 226 | + }, |
| 227 | + "nbformat": 4, |
| 228 | + "nbformat_minor": 2 |
| 229 | +} |
0 commit comments