Agents/Together_Open_Deep_Research_CookBook.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jrhNq1xFndwM"
      },
      "source": [
        "# Together's Open Deep Research Cookbook: An Efficient and Open Source Implementation with Multi-Step Web Search\n",
        "\n",
        "Authors: Shang Zhu, Federico Bianchi, Zhichao Li, Ben Athiwaratkun, Albert Meixner, James Zou\n",
        "\n",
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/togethercomputer/together-cookbook/blob/main/Agents/Together_Open_Deep_Research_CookBook.ipynb)\n",
        "\n",
        "## Introduction\n",
        "\n",
        "Welcome to the Together's Open Deep Research Cookbook! This guide demonstrates how to answer complex research questions with comprehensive, evidence-based reports by combining the power of large language models (LLMs) with structured web search techniques.\n",
        "\n",
        "We break down the research process into clear, modular steps that can be understood, customized, and improved independently. This modularity makes advanced research capabilities accessible and adaptable to diverse research needs.\n",
        "\n",
        "### Together's Open Deep Research Workflow\n",
        "\n",
        "<img src=\"https://github.com/vinid/data/blob/master/loop.png?raw=true\">\n",
        "\n",
        "\n",
        "Our Deep Research framework offers several key advantages:\n",
        "\n",
        "1. **Progressive Exploration**: Starting with initial queries derived from your research topic, the system intelligently explores information, identifies gaps, and refines its search strategy.\n",
        "\n",
        "2. **Evidence-Based Synthesis**: All findings are grounded in verified sources with proper citations, reducing the hallucinations common in pure LLM-based question and answering.\n",
        "\n",
        "3. **Adaptive Search with Completeness Evaluation**: The system evaluates information gaps against research goals and generates targeted follow-up queries for missing specifics. This ensures resources focus precisely on completing your research rather than collecting redundant information.\n",
        "\n",
        "4. **Source Filtering**: Not all information is equally valuable - our system evaluates and prioritizes the most relevant and reliable sources.\n",
        "\n",
        "5. **Structured Output**: Results are presented as well-organized reports with clear sections, cohesive narratives, and proper citations.\n",
        "\n",
        "### Cookbook Contributions\n",
        "\n",
        "- Set up a flexible research environment with configurable parameters\n",
        "- Process complex topics through a multi-stage research pipeline\n",
        "- Generate high-quality reports for the given research topic\n",
        "\n",
        "The functional design pattern we showcase makes each step transparent and customizable, allowing you to adapt the system to specific domains or research styles. Each function handles a specific part of the research process, from query generation to answer synthesis, with explicit inputs and outputs that can be tested and improved independently.\n",
        "\n",
        "Whether you're a researcher, student, analyst, or curious explorer, this cookbook provides the tools to transform your research questions into comprehensive, evidence-based reports that would normally require hours of manual work.\n",
        "\n",
        "Let's begin our exploration of AI-powered research!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "peuxZFsYndwO"
      },
      "source": [
        "## Install packages"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "c1HSzyRSndwP"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "!pip install tavily-python together markdown"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S_epEg0yndwP"
      },
      "source": [
        "## Initialize necessary functions and configuration"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YF4SSAGV2dw0"
      },
      "outputs": [],
      "source": [
        "from dataclasses import dataclass\n",
        "from typing import Literal, Optional\n",
        "from pydantic import BaseModel, Field\n",
        "import json\n",
        "import asyncio\n",
        "from typing import List\n",
        "from together import AsyncTogether, Together\n",
        "from tavily import AsyncTavilyClient"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_4zf_TsOndwP"
      },
      "source": [
        "### Data Model Definitions\n",
        "\n",
        "Before diving into the research functionality, we need to define the data structures that will represent our research plan, search results, and source evaluation. These models provide a structured way to handle the different types of data flowing through our research pipeline.\n",
        "\n",
        "The data model consists of several key components:\n",
        "\n",
        "1. **ResearchPlan**: Structures the initial research plan with search queries generated by the LLM\n",
        "2. **SourceList**: Captures the filtered list of relevant sources chosen by the LLM\n",
        "3. **SearchResult**: Represents an individual search result with its metadata and content\n",
        "4. **SearchResults**: A collection of search results with utilities for manipulation and display\n",
        "\n",
        "These structured data models allow us to maintain organization throughout the research process, from query generation to final report synthesis.\n",
        "\n",
        "The first two will be pydantic base classes, and it will be used for JSON mode. The latter two will instread be more general dataclasses."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wQ0jnHXqndwQ"
      },
      "outputs": [],
      "source": [
        "class ResearchPlan(BaseModel):\n",
        "    \"\"\"\n",
        "    Structured representation of a research plan with search queries.\n",
        "\n",
        "    Used to parse the LLM's planning output into a structured format\n",
        "    that can be easily processed by the research pipeline.\n",
        "    \"\"\"\n",
        "    queries: list[str] = Field(description=\"A list of search queries to thoroughly research the topic\")\n",
        "\n",
        "class SourceList(BaseModel):\n",
        "    \"\"\"\n",
        "    Structured representation of filtered source indices.\n",
        "\n",
        "    Used to parse the LLM's source evaluation output into a structured\n",
        "    format that identifies which search results should be retained.\n",
        "    \"\"\"\n",
        "    sources: list[int] = Field(description=\"A list of source numbers from the search results\")\n",
        "\n",
        "@dataclass\n",
        "class SearchResult:\n",
        "    \"\"\"\n",
        "    Container for an individual search result with its metadata and content.\n",
        "\n",
        "    Holds both the original content and the filtered/processed content\n",
        "    that's relevant to the research topic.\n",
        "    \"\"\"\n",
        "    title: str\n",
        "    link: str\n",
        "    content: str\n",
        "    filtered_raw_content: Optional[str] = None\n",
        "\n",
        "    def __str__(self):\n",
        "        \"\"\"(For Report Generation and Completeness Evaluation) String representation with title, link and refined content.\"\"\"\n",
        "        return (\n",
        "            (f\"Title: {self.title}\\n\" f\"Link: {self.link}\\n\" f\"Refined Content: {self.filtered_raw_content}\")\n",
        "            if self.filtered_raw_content\n",
        "            else (f\"Title: {self.title}\\n\" f\"Link: {self.link}\\n\" f\"Raw Content: {self.content[:1000]}\")\n",
        "        )\n",
        "\n",
        "    def short_str(self):\n",
        "        \"\"\"(For Filtering ONLY) Abbreviated string representation with truncated raw content.\"\"\"\n",
        "        return f\"Title: {self.title}\\nLink: {self.link}\\nRaw Content: {self.content[:1000]}\"\n",
        "\n",
        "\n",
        "@dataclass\n",
        "class SearchResults:\n",
        "    \"\"\"\n",
        "    Collection of search results with utilities for manipulation and display.\n",
        "\n",
        "    Provides methods for combining result sets, deduplication, and\n",
        "    different string representations for processing and display.\n",
        "    \"\"\"\n",
        "    results: list[SearchResult]\n",
        "\n",
        "    def __str__(self):\n",
        "        \"\"\"Detailed string representation of all search results with indices.\"\"\"\n",
        "        return \"\\n\\n\".join(f\"[{i+1}] {str(result)}\" for i, result in enumerate(self.results))\n",
        "\n",
        "    def __add__(self, other):\n",
        "        \"\"\"Combine two SearchResults objects by concatenating their result lists.\"\"\"\n",
        "        return SearchResults(self.results + other.results)\n",
        "\n",
        "    def short_str(self):\n",
        "        \"\"\"Abbreviated string representation of all search results with indices.\"\"\"\n",
        "        return \"\\n\\n\".join(f\"[{i+1}] {result.short_str()}\" for i, result in enumerate(self.results))\n",
        "\n",
        "    def dedup(self):\n",
        "        \"\"\"\n",
        "        Remove duplicate search results based on URL.\n",
        "\n",
        "        Returns a new SearchResults object with unique entries.\n",
        "        \"\"\"\n",
        "        def deduplicate_by_link(results):\n",
        "            seen_links = set()\n",
        "            unique_results = []\n",
        "\n",
        "            for result in results:\n",
        "                if result.link not in seen_links:\n",
        "                    seen_links.add(result.link)\n",
        "                    unique_results.append(result)\n",
        "\n",
        "            return unique_results\n",
        "\n",
        "        return SearchResults(deduplicate_by_link(self.results))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S8iB6cOpndwQ"
      },
      "source": [
        "## Deep Research Configuration Parameters\n",
        "\n",
        "This section defines the key parameters that control the Deep Research process. These configuration settings allow you to customize the research behavior, model selection, and output format according to your specific needs.\n",
        "\n",
        "The parameters include:\n",
        "\n",
        "- **Model Selection**: Defines which LLMs to use for different stages of the research process\n",
        "- **Research Budget**: Controls the number of research cycles performed, with higher values enabling more thorough but time-consuming exploration\n",
        "- **Query Limits**: Sets boundaries on how many search queries to generate and execute\n",
        "- **Source Management**: Configures the maximum amount of sources to be considered in the final synthesis (avoid ultra-long context that may break the context window)\n",
        "- **Prompt Templates**: Defines the instruction prompts utilized for selected models.\n",
        "\n",
        "\n",
        "Adjust these parameters to balance depth, speed, and resource usage according to your research priorities.\n",
        "\n",
        "# Disclaimers ⚠️\n",
        "\n",
        "**For readability**, here we provide rather simple prompts but we encourage you to optimize them as needed.\n",
        "\n",
        "**This workflow involves multiple LLM calls and search API calls**, so we remind the user to be mindful about the resource management as defined in the following code block.\n",
        "\n",
        "**Be sure** to understand that this code will make API requests to other services. Specifically, you will need Tavily API key (get it [here](https://docs.tavily.com/documentation/quickstart)) and Together API key (get it [here](https://docs.together.ai/docs/quickstart))."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PdE_TIBsndwQ"
      },
      "outputs": [],
      "source": [
        "raise Exception(\"Please remove this after you read the disclaimers above ☝️\")\n",
        "\n",
        "# Research Configuration\n",
        "# ======================\n",
        "\n",
        "# Model Selection\n",
        "# --------------\n",
        "# Specialized models for different stages of the research pipeline\n",
        "planning_model = \"Qwen/Qwen2.5-72B-Instruct-Turbo\"  # Used for research planning and evaluation\n",
        "json_model = \"meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo\"  # Used for structured data parsing\n",
        "summary_model = \"meta-llama/Llama-4-Scout-17B-16E-Instruct\"  # Used for web content summarization\n",
        "answer_model = \"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8\"  # Used for final answer synthesis\n",
        "\n",
        "# Resource Allocation\n",
        "# ------------------\n",
        "# Parameters controlling research depth and breadth\n",
        "budget = 2  # Number of research refinement cycles to perform (in addition to the initial search operation)\n",
        "max_queries = 2  # Maximum number of search queries per research cycle\n",
        "max_sources = 10  # Maximum number of sources to include in final synthesis\n",
        "max_tokens = 8192 # Maximum number of tokens in the generated report\n",
        "\n",
        "# System Prompts\n",
        "# -------------\n",
        "# Instructions for each stage of the research process\n",
        "prompts = {\n",
        "    # Planning: Generates initial research queries\n",
        "    \"planning_prompt\": \"\"\"You are a strategic research planner with expertise in breaking down complex\n",
        "                         questions into logical search steps. Generate focused, specific, and self-contained queries that\n",
        "                         will yield relevant information for the research topic.\"\"\",\n",
        "\n",
        "    # Plan Parsing: Extracts structured data from planning output\n",
        "    \"plan_parsing_prompt\": \"\"\"Extract search queries that should be executed.\"\"\",\n",
        "\n",
        "    # Content Processing: Identifies relevant information from search results\n",
        "    \"raw_content_summarizer_prompt\": \"\"\"Extract and synthesize only the information relevant to the research\n",
        "                                       topic from this content. Preserve specific data, terminology, and\n",
        "                                       context while removing irrelevant information.\"\"\",\n",
        "\n",
        "    # Completeness Evaluation: Determines if more research is needed\n",
        "    \"evaluation_prompt\": \"\"\"Analyze these search results against the original research goal. Identify\n",
        "                          specific information gaps and generate targeted follow-up queries to fill\n",
        "                          those gaps. If no significant gaps exist, indicate that research is complete.\"\"\",\n",
        "\n",
        "    # Evaluation Parsing: Extracts structured data from evaluation output\n",
        "    \"evaluation_parsing_prompt\": \"\"\"Extract follow-up search queries from the evaluation. If no follow-up queries are needed, return an empty list.\"\"\",\n",
        "\n",
        "    # Source Filtering: Selects most relevant sources\n",
        "    \"filter_prompt\": \"\"\"Evaluate each search result for relevance, accuracy, and information value\n",
        "                       related to the research topic. At the end, you need to provide a list of\n",
        "                       source numbers with the rank of relevance. Remove the irrelevant ones.\"\"\",\n",
        "    # Source Filtering: Selects most relevant sources\n",
        "    \"source_parsing_prompt\": \"\"\"Extract the source list that should be included.\"\"\",\n",
        "\n",
        "    # Answer Generation: Creates final research report\n",
        "    \"answer_prompt\": \"\"\"Create a comprehensive, publication-quality markdown research report based exclusively\n",
        "                       on the provided sources. The report should include: title, introduction, analysis (multiple sections with insights titles)\n",
        "                       and conclusions, references. Use proper citations (source with link; using \\n\\n \\\\[Ref. No.\\\\] to improve format),\n",
        "                       organize information logically, and synthesize insights across sources. Include all relevant details while\n",
        "                       maintaining readability and coherence. In each section, You MUST write in plain\n",
        "                       paragraghs and NEVER describe the content following bullet points or key points (1,2,3,4... or point X: ...)\n",
        "                       to improve the report readability.\"\"\"\n",
        "}\n",
        "\n",
        "# Initialize Clients\n",
        "# ------------------\n",
        "# Initialize the Together and Tavily clients with your API keys\n",
        "together_client = AsyncTogether(api_key=\"--your--together-api-key--\")\n",
        "tavily_client = AsyncTavilyClient(api_key=\"--your--tavily-api-key--\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wF4E4RhundwR"
      },
      "source": [
        "## Building the Research Pipeline\n",
        "\n",
        "### STEP 1: Query Generation - Breaking Down the Research Question\n",
        "\n",
        "In this first stage, we transform the broad research topic into specific, targeted search queries. This crucial step determines what information we'll gather and sets the foundation for the entire research process.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WKPrJ8mZndwR"
      },
      "outputs": [],
      "source": [
        "async def generate_initial_queries(topic: str, together_client: AsyncTogether, max_queries: int, planning_model: str, json_model: str, prompts: dict) -> List[str]:\n",
        "    \"\"\"Step 1: Generate initial research queries based on the topic\"\"\"\n",
        "    queries = await generate_research_queries(topic, together_client, planning_model, json_model, prompts)\n",
        "    if max_queries > 0:\n",
        "        queries = queries[:max_queries]\n",
        "    print(f\"\\n\\nInitial queries: {queries}\")\n",
        "\n",
        "    if len(queries) == 0:\n",
        "        print(\"ERROR: No initial queries generated\")\n",
        "        return []\n",
        "\n",
        "    return queries\n",
        "\n",
        "async def generate_research_queries(topic: str, together_client: AsyncTogether, planning_model: str, json_model: str, prompts: dict) -> list[str]:\n",
        "    \"\"\"Generate research queries for a given topic using LLM\"\"\"\n",
        "    PLANNING_PROMPT = prompts[\"planning_prompt\"]\n",
        "\n",
        "    planning_response = await together_client.chat.completions.create(\n",
        "        model=planning_model,\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": PLANNING_PROMPT},\n",
        "            {\"role\": \"user\", \"content\": f\"Research Topic: {topic}\"}\n",
        "        ]\n",
        "    )\n",
        "    plan = planning_response.choices[0].message.content\n",
        "\n",
        "    print(f\"Generated plan: {plan}\")\n",
        "\n",
        "    SEARCH_PROMPT = prompts[\"plan_parsing_prompt\"]\n",
        "\n",
        "    json_response = await together_client.chat.completions.create(\n",
        "        model=json_model,\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": SEARCH_PROMPT},\n",
        "            {\"role\": \"user\", \"content\": f\"Plan to be parsed: {plan}\"}\n",
        "        ],\n",
        "        response_format={\"type\": \"json_object\", \"schema\": ResearchPlan.model_json_schema()}\n",
        "    )\n",
        "\n",
        "    response_json = json_response.choices[0].message.content\n",
        "    plan = json.loads(response_json)\n",
        "    return plan[\"queries\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N1-ZmGiYndwR"
      },
      "source": [
        "**In `generate_research_queries` function:** We use a two-model approach where the planning model creates comprehensive research strategies, then the JSON model structures these into precisely formatted queries. This ensures both creative planning and reliable structured output.\n",
        "\n",
        "**The `generate_initial_queries` function** wraps this process, applying query limits and providing additional validation and logging before returning the final set of targeted search queries.\n",
        "\n",
        "Let's demonstrate this process with a practical example, using the research topic: \"The impact of artificial intelligence on the manufacturing industry\" This multifaceted subject provides an excellent case study for showing how our system breaks down complex topics into targeted queries.\n",
        "\n",
        "**Note: In rare cases we saw none initial queries were generated, where rerunning the cell would typically resolve the issue.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dmxqFukPndwR",
        "outputId": "0af754d8-2422-4ecf-cefe-8a57cb047a0a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Generated plan: To effectively research the impact of artificial intelligence (AI) on the manufacturing industry, you can break down the topic into several focused, specific, and self-contained queries. Each query will help gather relevant information that contributes to a comprehensive understanding of the topic. Here are the queries:\n",
            "\n",
            "1. **Overview of AI in Manufacturing:**\n",
            "   - What are the key applications of AI in the manufacturing industry?\n",
            "   - How has the adoption of AI in manufacturing evolved over the past decade?\n",
            "\n",
            "2. **Economic Impact:**\n",
            "   - What are the economic benefits of AI in manufacturing, such as cost reduction and efficiency gains?\n",
            "   - How has AI influenced the job market in the manufacturing sector, including job creation and displacement?\n",
            "\n",
            "3. **Operational Efficiency:**\n",
            "   - How does AI improve production efficiency and quality control in manufacturing?\n",
            "   - What are the specific AI technologies (e.g., machine learning, robotics, predictive maintenance) used to enhance operational efficiency?\n",
            "\n",
            "4. **Supply Chain Management:**\n",
            "   - How does AI optimize supply chain management in manufacturing?\n",
            "   - What are the best practices for integrating AI into supply chain operations?\n",
            "\n",
            "5. **Product Innovation:**\n",
            "   - How does AI drive product innovation in the manufacturing industry?\n",
            "   - What are some examples of AI-driven product developments in manufacturing?\n",
            "\n",
            "6. **Challenges and Barriers:**\n",
            "   - What are the main challenges and barriers to AI adoption in manufacturing?\n",
            "   - How do regulatory and ethical considerations impact the use of AI in manufacturing?\n",
            "\n",
            "7. **Case Studies:**\n",
            "   - Provide case studies of successful AI implementations in manufacturing companies.\n",
            "   - What lessons can be learned from these case studies?\n",
            "\n",
            "8. **Future Trends:**\n",
            "   - What are the emerging trends in AI for manufacturing?\n",
            "   - How is the future of manufacturing likely to be shaped by AI?\n",
            "\n",
            "9. **Stakeholder Perspectives:**\n",
            "   - What are the perspectives of key stakeholders (e.g., manufacturers, policymakers, employees) on the impact of AI in manufacturing?\n",
            "   - How are these perspectives influencing the adoption and development of AI technologies?\n",
            "\n",
            "10. **Comparative Analysis:**\n",
            "    - How does the impact of AI in manufacturing compare to other industries?\n",
            "    - What are the unique aspects of AI's impact on the manufacturing sector?\n",
            "\n",
            "By addressing these queries, you can build a robust and detailed research report on the impact of artificial intelligence on the manufacturing industry. Each query is designed to provide a specific piece of the puzzle, contributing to a comprehensive understanding of the topic.\n",
            "\n",
            "\n",
            "Initial queries: ['What are the key applications of AI in the manufacturing industry?', 'How has the adoption of AI in manufacturing evolved over the past decade?']\n"
          ]
        }
      ],
      "source": [
        "research_topic=\"The impact of artificial intelligence on the manufacturing industry\"\n",
        "\n",
        "initial_queries = await generate_initial_queries(\n",
        "    topic=research_topic,\n",
        "    together_client=together_client,\n",
        "    max_queries=max_queries,\n",
        "    planning_model=planning_model,\n",
        "    json_model=json_model,\n",
        "    prompts=prompts\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "etSRHL8NndwR"
      },
      "source": [
        "Below are the targeted queries generated by our system, demonstrating how the general research topic has been decomposed into specific, searchable questions. Each query addresses a distinct aspect of AI's impact on the manufacturing industry:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Kvux0q0CndwR",
        "outputId": "82542623-ef82-4c77-8a0e-38608ce4ca14"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['What are the key applications of AI in the manufacturing industry?',\n",
              " 'How has the adoption of AI in manufacturing evolved over the past decade?']"
            ]
          },
          "execution_count": 33,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "initial_queries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zffTU5HzndwS"
      },
      "source": [
        "### STEP 2: Information Gathering - Searching and Processing Web Content\n",
        "\n",
        "Now that we have our targeted queries, we move to the information gathering phase. This step involves two critical processes:\n",
        "\n",
        "1. **Web Search**: Executing our queries against web sources to retrieve relevant information\n",
        "2. **Content Processing**: Filtering and summarizing the raw content to extract what's most relevant to our research, using an LLM (`summary_model`).\n",
        "\n",
        "#### Let's first examine a single search operation (using [Tavily Search API](https://tavily.com/) as an example) to understand the core mechanics:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GT4Y8uV5ndwS"
      },
      "outputs": [],
      "source": [
        "async def tavily_search(query: str, tavily_client: AsyncTavilyClient, prompts: dict, together_client: AsyncTogether, summary_model: str) -> SearchResults:\n",
        "    \"\"\"Perform a single Tavily search\"\"\"\n",
        "    print(f'Perform Tavily search with query: {query}')\n",
        "\n",
        "    response = await tavily_client.search(query, include_raw_content=True)\n",
        "\n",
        "    print(f\"Tavily Responded with {len(response['results'])} results (Tavily returning None will be ignored for summarization)\")\n",
        "\n",
        "    RAW_CONTENT_SUMMARIZER_PROMPT = prompts[\"raw_content_summarizer_prompt\"]\n",
        "\n",
        "    # Create tasks for summarization\n",
        "    summarization_tasks = []\n",
        "    result_info = []\n",
        "    for result in response[\"results\"]:\n",
        "        if result[\"raw_content\"] is None:\n",
        "            continue\n",
        "        task = summarize_content(result[\"raw_content\"], query, RAW_CONTENT_SUMMARIZER_PROMPT, together_client, summary_model)\n",
        "        summarization_tasks.append(task)\n",
        "        result_info.append(result)\n",
        "\n",
        "    summarized_contents = await asyncio.gather(*summarization_tasks)\n",
        "\n",
        "    formatted_results = []\n",
        "    for result, summarized_content in zip(result_info, summarized_contents):\n",
        "        formatted_results.append(\n",
        "            SearchResult(\n",
        "                title=result[\"title\"],\n",
        "                link=result[\"url\"],\n",
        "                content=result[\"raw_content\"],\n",
        "                filtered_raw_content=summarized_content,\n",
        "            )\n",
        "        )\n",
        "    return SearchResults(formatted_results)\n",
        "\n",
        "async def summarize_content(raw_content: str, query: str, prompt: str, together_client: AsyncTogether, summary_model: str) -> str:\n",
        "    \"\"\"Summarize content asynchronously using the LLM\"\"\"\n",
        "    print(\"Summarizing content asynchronously using the LLM\")\n",
        "\n",
        "    summarize_response = await together_client.chat.completions.create(\n",
        "        model=summary_model,\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": prompt},\n",
        "            {\"role\": \"user\", \"content\": f\"<Raw Content>{raw_content}</Raw Content>\\n\\n<Research Topic>{query}</Research Topic>\"}\n",
        "        ]\n",
        "    )\n",
        "    return summarize_response.choices[0].message.content\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pqVu70XVndwS"
      },
      "source": [
        "To demonstrate the search and content processing mechanism, let's execute a single search using the first query from our generated list. This allows us to examine in detail how the system retrieves and processes information before we implement the full multi-query research workflow:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wBk4YwjBndwS",
        "outputId": "6db00441-d9e5-40d6-9f2a-16aea42d1a4f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Perform Tavily search with query: What are the key applications of AI in the manufacturing industry?\n",
            "Tavily Responded with 10 results (Tavily returning None will be ignored for summarization)\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n"
          ]
        }
      ],
      "source": [
        "search_results=await tavily_search(initial_queries[0], tavily_client, prompts, together_client, summary_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gi_bPEIFndwS"
      },
      "source": [
        "Below is an example of the first retrieved and summarized result. Note how the system has not only found relevant content but also processed it to extract information specifically related to our research topic. This demonstrates the content filtering capabilities that make our research pipeline more effective than simple search retrieval:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TxDJdSdNndwS",
        "outputId": "3c13773d-a7ef-4550-f4c4-fed0f6938bb7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Title: How Is AI Used in Manufacturing: Examples, Use Cases, and Benefits\n",
            "\n",
            "Link: https://www.azumuta.com/blog/how-is-ai-used-in-manufacturing-examples-use-cases-and-benefits/\n",
            "\n",
            "Content: **Key Applications of AI in the Manufacturing Industry**\n",
            "\n",
            "Artificial intelligence (AI) is transforming the manufacturing landscape by enhancing efficiency, innovation, and productivity. The five main AI applications in manufacturing include:\n",
            "\n",
            "1. **Predictive Maintenance**: AI analyzes data from sensors, equipment telemetry, and other sources to forecast equipment failures, allowing for proactive maintenance scheduling, minimizing downtime, and reducing costs.\n",
            "2. **AI-Enhanced Quality Control**: AI-powered computer vision systems detect defects or anomalies in products, ensuring quality and safety. Machine learning algorithms are trained on labeled datasets to recognize patterns associated with defects.\n",
            "3. **Supply Chain Optimization**: AI predicts demand fluctuations, optimizes inventory, and identifies potential disruptions. It analyzes large volumes of data, including sales data, customer behavior, economic indicators, and external factors.\n",
            "4. **Intelligent Automation**: AI combines [...]\n"
          ]
        }
      ],
      "source": [
        "print(f\"Title: {search_results.results[0].title}\\n\\nLink: {search_results.results[0].link}\\n\\nContent: {search_results.results[0].filtered_raw_content[:1000]}[...]\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oZY3YBWnndwS"
      },
      "source": [
        "#### Executing the Complete Initial Search Process\n",
        "\n",
        "Now that we understand how a single search works, let's implement the parallel search process that executes all our initial queries simultaneously. This approach significantly improves efficiency by:\n",
        "\n",
        "1. Running multiple search operations concurrently\n",
        "2. Processing all results in parallel\n",
        "3. Combining findings into a comprehensive result set\n",
        "\n",
        "\n",
        "**Disclaimer: This step involves multiple LLM runs processing long web texts, so we remind the user to be mindful about the resource management as defined at the beginning.**\n",
        "\n",
        "This demonstrates a key advantage of our functional pipeline architecture:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BaJdNpn5ndwS"
      },
      "outputs": [],
      "source": [
        "async def perform_search(queries: List[str], tavily_client: AsyncTavilyClient, prompts: dict, together_client: AsyncTogether, summary_model: str) -> SearchResults:\n",
        "    \"\"\"Execute searches for all queries in parallel\"\"\"\n",
        "    tasks = [tavily_search(query, tavily_client, prompts, together_client, summary_model) for query in queries]\n",
        "    results_list = await asyncio.gather(*tasks)\n",
        "\n",
        "    combined_results = SearchResults([])\n",
        "    for results in results_list:\n",
        "        combined_results = combined_results + results\n",
        "\n",
        "    combined_results_dedup=combined_results.dedup()\n",
        "    print(f\"Search complete, found {len(combined_results_dedup.results)} results after deduplication\")\n",
        "    return combined_results_dedup"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7m5HOKxlndwS",
        "outputId": "c9ce3276-9e3a-4312-ade5-e7cd0aa066bc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Perform Tavily search with query: What are the key applications of AI in the manufacturing industry?\n",
            "Perform Tavily search with query: How has the adoption of AI in manufacturing evolved over the past decade?\n",
            "Tavily Responded with 10 results (Tavily returning None will be ignored for summarization)\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Tavily Responded with 10 results (Tavily returning None will be ignored for summarization)\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Search complete, found 9 results after deduplication\n"
          ]
        }
      ],
      "source": [
        "initial_results=await perform_search(initial_queries, tavily_client, prompts, together_client, summary_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ERJxOhQ8ndwS"
      },
      "source": [
        "Below is the complete set of results from our parallel search operations, consolidated into a single collection. This demonstrates how our system gathers diverse information across multiple queries while maintaining organization. The results include sources addressing different aspects of our research topic, providing a foundation for comprehensive analysis."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3lm84AcAndwS",
        "outputId": "b2262b5c-0840-4582-ab03-8856f9e57c85"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "First 10000 characters of 9 results:\n",
            "\n",
            " [1] Title: How Is AI Used in Manufacturing: Examples, Use Cases, and Benefits\n",
            "Link: https://www.azumuta.com/blog/how-is-ai-used-in-manufacturing-examples-use-cases-and-benefits/\n",
            "Refined Content: **Key Applications of AI in the Manufacturing Industry**\n",
            "\n",
            "Artificial intelligence (AI) is transforming the manufacturing landscape by enhancing efficiency, innovation, and productivity. The five main AI applications in manufacturing include:\n",
            "\n",
            "1. **Predictive Maintenance**: AI analyzes data from sensors, equipment telemetry, and other sources to forecast equipment failures, allowing for proactive maintenance scheduling, minimizing downtime, and reducing costs.\n",
            "2. **AI-Enhanced Quality Control**: AI-powered computer vision systems detect defects or anomalies in products, ensuring quality and safety.\n",
            "3. **Supply Chain Optimization**: AI predicts demand fluctuations, optimizes inventory, and identifies potential disruptions, enabling manufacturers to make informed decisions about production, procurement, and resource allocation.\n",
            "4. **Intelligent Automation**: AI automates tasks beyond repetition by combining intelligent software and robotic equipment, improving efficiency, flexibility, and ergonomics in manufacturing operations.\n",
            "5. **AI-Driven Training and Assistance**: AI-powered digital tools create clear and accurate work instructions, improving operational efficiency and productivity.\n",
            "\n",
            "**AI Technologies Used in Manufacturing**\n",
            "\n",
            "Some key AI technologies used in manufacturing include:\n",
            "\n",
            "1. **Machine Learning**: enables machines to learn from data, identify patterns, and make decisions.\n",
            "2. **Deep Learning**: a subset of machine learning that recognizes patterns using many processing layers, useful for image and speech recognition.\n",
            "3. **Natural Language Processing (NLP)**: enables machines to understand, interpret, and generate human language.\n",
            "4. **Computer Vision**: allows machines to interpret and understand visual information from images or videos.\n",
            "5. **Robotics**: combines AI with mechanical engineering to create machines that perform tasks autonomously or with minimal human intervention.\n",
            "\n",
            "**Real-World Examples of AI in Manufacturing**\n",
            "\n",
            "1. **BMW Group**: uses AI to automate quality processes in the conveyor belt, analyzing data recorded by cameras and sensor technology.\n",
            "2. **Ford**: integrates AI into assembly lines with robot arms that learn the most efficient way to assemble parts.\n",
            "3. **Rolls Royce**: uses digital twins and AI to consolidate data from produced engines, monitoring performance, predicting potential issues, and optimizing maintenance schedules.\n",
            "\n",
            "**Benefits of AI in Manufacturing**\n",
            "\n",
            "1. **Cost Savings**: AI reduces operational costs through optimized processes and decreased downtime.\n",
            "2. **Data-Driven Decision-Making**: AI analyzes vast amounts of data to identify trends and patterns, providing valuable insights for optimizing production processes.\n",
            "3. **Supply Chain Optimization**: AI optimizes supply chain logistics, inventory management, and procurement processes.\n",
            "4. **Improved Safety**: AI improves workplace safety by combining automation, real-time monitoring, and predictive analytics.\n",
            "5. **Enhanced Product Quality**: AI-powered vision systems inspect products with greater accuracy and speed than human inspectors.\n",
            "\n",
            "[2] Title: The Rise of AI in Manufacturing: 2025 Trends, Tools & Real-World Impact\n",
            "Link: https://www.authentise.com/post/the-rise-of-ai-in-manufacturing-2025-trends-tools-real-world-impact\n",
            "Refined Content: Here is the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**Key Applications of AI in Manufacturing:**\n",
            "\n",
            "1. **Predictive Maintenance**: AI systems can predict equipment failures before they occur, reducing unplanned downtime by up to 50% and cutting maintenance costs by 30%.\n",
            "2. **Quality Control**: Advanced machine vision systems, powered by deep learning algorithms, detect defects and deviations with unprecedented accuracy, ensuring higher product quality and reducing waste.\n",
            "3. **Automation of Documentation and Compliance**: AI can automate documentation, traceability, and compliance, reducing manual entry and enabling proactive quality checks.\n",
            "4. **Materials Management**: Predictive intelligence can help manufacturers plan, track, and reclaim materials more effectively, cutting costs and reducing waste.\n",
            "5. **Engineering Collaboration**: AI-powered platforms capture and contextualize intent from conversations, drawings, and data streams, ensuring decisions don't get lost.\n",
            "6. **Human-Machine Collaboration**: AI has led to the development of humanoid robots capable of performing complex tasks alongside human workers.\n",
            "7. **Smart Manufacturing**: AI enables real-time decision-making, near-autonomous systems, and seamless human-machine collaboration, enhancing flexibility and responsiveness in production environments.\n",
            "\n",
            "**Benefits of AI in Manufacturing:**\n",
            "\n",
            "* Reduced conversion costs by up to 20%\n",
            "* Improved workforce productivity\n",
            "* Enhanced troubleshooting processes\n",
            "* Increased efficiency and reduced costs\n",
            "* Improved product quality and reduced waste\n",
            "\n",
            "**Companies Using AI in Manufacturing:**\n",
            "\n",
            "* Siemens\n",
            "* Microsoft (Factory Operations Agent)\n",
            "* Nvidia\n",
            "* Figure AI\n",
            "* Authentise (integrating AI into core manufacturing workflows)\n",
            "\n",
            "**Trends and Future Directions:**\n",
            "\n",
            "* The emergence of \"AI factories\" where companies operate dual production lines - one for traditional products and another dedicated to AI model training and deployment.\n",
            "* The need for continuous upskilling of the workforce to effectively collaborate with intelligent systems.\n",
            "* The importance of addressing ethical considerations surrounding data privacy and job displacement.\n",
            "\n",
            "[3] Title: AI in Manufacturing: The Smart Revolution in Industry\n",
            "Link: https://sigmatechnology.com/articles/the-application-of-ai-in-manufacturing/\n",
            "Refined Content: **Key Applications of AI in the Manufacturing Industry**\n",
            "\n",
            "Based on the provided content, the key applications of AI in the manufacturing industry are:\n",
            "\n",
            "1. **Predictive Maintenance**: AI-powered predictive maintenance helps prevent downtime by monitoring machine performance, detecting anomalies, and scheduling repairs during planned downtime.\n",
            "2. **Quality Control and Inspection**: AI-driven computer vision systems enable automated defect detection, ensuring high-quality output and reducing waste.\n",
            "3. **Supply Chain Efficiency**: AI analyzes vast datasets to anticipate demand fluctuations, optimize inventory levels, and automate order fulfillment, leading to operational efficiency and reduced waste.\n",
            "4. **Collaborative Robots (Cobots)**: AI-powered cobots work safely alongside human employees, taking over repetitive tasks and freeing up skilled workers to focus on complex and creative aspects of the manufacturing process.\n",
            "5. **Digital Twins**: AI-driven digital twins simulate production processes, enhancing predictive analytics, anticipating equipment failures, and optimizing workflows.\n",
            "6. **Machine Learning (ML) and Deep Learning**: ML algorithms detect inefficiencies in production processes, predict demand, and optimize supply chains, while deep learning enables visual inspection and quality control.\n",
            "7. **Sensor Data Analytics (IoT + AI)**: AI processes data from IoT sensors to enable predictive maintenance, optimize energy consumption, and ensure quality control.\n",
            "8. **Computer Vision**: AI-powered computer vision systems enable automated defect detection, assembly line monitoring, and workplace safety.\n",
            "9. **Edge AI**: Edge AI brings AI processing directly to manufacturing devices, enabling real-time decision-making, autonomous manufacturing systems, and instant quality control.\n",
            "\n",
            "**Transformative Benefits of AI in Manufacturing**\n",
            "\n",
            "The integration of AI in manufacturing offers several benefits, including:\n",
            "\n",
            "1. **Cost Reduction and Increased Operational Efficiency**: AI-powered automation streamlines processes, reducing the need for human intervention in repetitive tasks and minimizing costly errors.\n",
            "2. **Customized Production and Dynamic Supply Chain Management**: AI enables manufacturers to offer customized products efficiently, blending the advantages of mass production with the appeal of one-off design.\n",
            "3. **Improved Sustainability**: AI optimizes energy consumption, minimizes environmental impact, and enhances the skills of the workforce.\n",
            "\n",
            "**Challenges and Lessons Learned**\n",
            "\n",
            "The content also highlights the challenges of AI implementation in manufacturing, including:\n",
            "\n",
            "1. **Managing Data Privacy and Security**: Protecting sensitive data from breaches and misuse is critical.\n",
            "2. **Bridging the AI Skills Gap**: Manufacturers need targeted reskilling and upskilling initiatives to close the skills gap.\n",
            "3. **Overcoming Cultural Resistance**: Creating an inclusive and supportive culture around AI implementation is essential.\n",
            "4. **Ensuring Ethical and Transparent AI Use**: Clear ethical guidelines and transparent AI processes build trust among employees, partners, and customers.\n",
            "\n",
            "By understanding these key applications, benefits, and challenges of AI in manufacturing, companies can successfully leverage AI's full potential, securing a competitive advantage in an increasingly digital and sustainability-focused market.\n",
            "\n",
            "[4] Title: Manufacturing AI: Top 15 tools & 13 real-life use cases ['25] - AIMultiple\n",
            "Link: https://research.aimultiple.com/manufacturing-ai/\n",
            "Refined Content: Here is the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**Key Applications of AI in the Manufacturing Industry:**\n",
            "\n",
            "1. **Predictive Maintenance**: AI analyzes sensor data to forecast equipment failures, allowing for scheduled maintenance and reducing downtime. (Example: PepsiCo’s Frito-Lay plants saved costs and improved equipment performance using AI-driven predictive maintenance.)\n",
            "2. **Generative Design**: AI algorithms generate multiple design options based on parameters, enabling manufacturers to quickly produce thousands of designs. (Example: Airbus reduced aerodynamics prediction times from \n",
            "\n",
            "...\n",
            "\n",
            " Last 10000 characters of 9 results:\n",
            "\n",
            " ce/article/pii/S000785062400115X\n",
            "Refined Content: Here is the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**Evolution of AI in Manufacturing over the past decade:**\n",
            "\n",
            "The adoption of Artificial Intelligence (AI) in manufacturing has seen accelerated growth since the beginning of the 21st century, particularly with the advent of Industry 4.0. Over the past decade, AI has been increasingly applied in various manufacturing areas, including:\n",
            "\n",
            "1. **Production system design and planning**: AI has been used to optimize production planning and design, enabling more efficient and flexible manufacturing systems.\n",
            "2. **Process modeling and optimization**: AI techniques, such as machine learning, have been applied to model and optimize manufacturing processes, improving their efficiency and effectiveness.\n",
            "3. **Quality assurance**: AI has been used to enhance quality control and assurance in manufacturing, enabling real-time monitoring and prediction of product quality.\n",
            "4. **Maintenance**: AI-powered predictive maintenance has become more prevalent, allowing for proactive maintenance scheduling and reducing equipment downtime.\n",
            "5. **Automated assembly and disassembly**: AI has been applied to automate assembly and disassembly processes, improving manufacturing efficiency and reducing labor costs.\n",
            "\n",
            "**Current State and Future Directions:**\n",
            "\n",
            "The current state of AI in manufacturing is characterized by the development of **smart manufacturing** systems, which leverage AI, machine learning, and data analytics to optimize production processes. Future research directions are expected to focus on:\n",
            "\n",
            "1. **Leveraging AI towards smart manufacturing**: Further developing AI solutions to realize smart manufacturing systems that are highly efficient, flexible, and autonomous.\n",
            "2. **Matching AI solutions to manufacturing problems**: Identifying representative manufacturing problems and matching them with suitable AI solutions to drive innovation and improvement.\n",
            "\n",
            "Overall, the adoption of AI in manufacturing has evolved significantly over the past decade, with a growing focus on applications such as machine learning, smart manufacturing, and process optimization.\n",
            "\n",
            "[7] Title: The Evolution of AI in Manufacturing - Augmentir\n",
            "Link: https://www.augmentir.com/blog/the-evolution-of-ai-in-manufacturing\n",
            "Refined Content: **Evolution of AI in Manufacturing: A Decade of Progress**\n",
            "\n",
            "The adoption of AI in manufacturing has undergone significant evolution over the past decade, transforming from a focus on automation and replacement of human workers to augmenting and supporting frontline workers. Here's a synthesis of the relevant information:\n",
            "\n",
            "**Early Adoption (1960s-2019)**\n",
            "\n",
            "1. **Automation and Robotics**: AI was initially used in manufacturing to automate manual, repetitive tasks such as assembly, parts handling, and sorting.\n",
            "2. **Machine Vision Systems**: AI-enabled machine vision systems were used to automate visual inspections, improving quality control and precision.\n",
            "3. **Industrial Internet of Things (IIoT)**: AI was integrated with IIoT to enable predictive analytics for machine health monitoring and optimize operations.\n",
            "\n",
            "**Recent Advancements (2020-2023)**\n",
            "\n",
            "1. **Connected Worker Solutions**: AI-powered connected worker solutions emerged, enabling manufacturers to digitize and optimize frontline processes, including autonomous maintenance, quality, safety, and assembly.\n",
            "2. **Generative AI (GenAI)**: The introduction of GenAI assistants, like Augmentir's Augie, which uses enterprise-wide data to provide instant access to relevant information, closes skills gaps, and identifies opportunities for continuous improvement.\n",
            "3. **AI-First Approach**: Companies like Augmentir pioneered an AI-first approach, designing platforms with AI capabilities in mind to support frontline workers.\n",
            "\n",
            "**Current State and Future Directions (2024 and beyond)**\n",
            "\n",
            "1. **Augmenting Human Workers**: AI is now being used to augment and directly support frontline workers, rather than replace them.\n",
            "2. **Increased Focus on Workforce Optimization**: AI-powered solutions aim to optimize workforce development, training, and productivity, addressing the manufacturing workforce crisis.\n",
            "3. **Predictions and Future Developments**: The future of AI in manufacturing is expected to involve measuring signals to detect actual engagement of industrial workers and deriving useful insights to enhance HR and manufacturing processes.\n",
            "\n",
            "**Key Statistics and Trends**\n",
            "\n",
            "1. **86% of manufacturing executives** believe AI-based factory solutions will drive competitiveness in the next five years (Deloitte).\n",
            "2. **1.7 Petabytes** of connected machine data is collected yearly, which can be used to optimize operations and frontline work processes.\n",
            "3. **Generative AI assistants** can enhance operational efficiency, problem-solving, and decision-making for frontline industrial workers.\n",
            "\n",
            "The evolution of AI in manufacturing has transformed from a focus on automation and replacement of human workers to augmenting and supporting frontline workers. The current state of AI adoption focuses on optimizing workforce development, training, and productivity, with future directions expected to involve measuring worker engagement and deriving useful insights.\n",
            "\n",
            "[8] Title: Adopting AI in manufacturing at speed and scale | McKinsey\n",
            "Link: https://www.mckinsey.org/capabilities/operations/our-insights/adopting-ai-at-speed-and-scale-the-4ir-push-to-stay-competitive\n",
            "Refined Content: Here is the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**Adoption of AI in Manufacturing: Evolution over the Past Decade**\n",
            "\n",
            "The adoption of AI in manufacturing has undergone significant evolution over the past decade, particularly among leading companies, known as Lighthouses, that are part of the Global Lighthouse Network. These companies have made substantial progress in implementing AI at scale, driven by advancements in machine intelligence technologies.\n",
            "\n",
            "**Key Developments:**\n",
            "\n",
            "1. **Maturity of AI**: AI has reached unprecedented levels of maturity, enabling machines to perform complex tasks in the cyber-physical world of production.\n",
            "2. **Scaling Impact**: Leading companies are redefining the concept of a pilot by using entire factories as pilots for network-wide deployment at scale.\n",
            "3. **Gen AI**: Generative AI (gen AI) has emerged as a significant development, enabling use of exceptionally large neural nets that can learn abstract patterns.\n",
            "4. **Increased Adoption**: AI-based use cases make up over 60% of the use cases presented by new Lighthouse applicants, up from just 11% in 2019.\n",
            "\n",
            "**Changes in Adoption over the Past Decade:**\n",
            "\n",
            "1. **Early Use Cases**: In 2018, cutting-edge use cases focused on localized applications of advanced analytics and autonomous vehicles.\n",
            "2. **Rapid Implementation**: Today, Lighthouses can implement use cases more rapidly, with 75% able to deploy a new advanced use case in under six months, and 30% in less than three months.\n",
            "3. **Scaling Slump**: Companies are overcoming the scaling slump, achieving network-level impact, and setting standards for industries.\n",
            "\n",
            "**Strategic Responses**:\n",
            "\n",
            "Manufacturers must consider three types of strategic responses:\n",
            "\n",
            "1. **Innovate**: Take a risk on and prove the next big thing at the factory level.\n",
            "2. **Accelerate**: Focus on network-level impact, changing the landscape for an entire industry.\n",
            "3. **Follow Fast**: Embrace the playbook already written by innovators and accelerators, capturing value while skipping the costs and tribulations of the learning curve.\n",
            "\n",
            "Overall, the adoption of AI in manufacturing has evolved significantly over the past decade, with leading companies making rapid progress in implementing AI at scale, driving impact, and setting standards for industries.\n",
            "\n",
            "[9] Title: Study reveals high AI adoption in manufacturing sector\n",
            "Link: https://www.cohnreznick.com/insights/manufacturing-checkup-artificial-intelligence\n",
            "Refined Content: Here's the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**High AI Adoption in Manufacturing Sector**\n",
            "\n",
            "* A recent study by CohnReznick, titled the Manufacturing Checkup, reveals that 92% of manufacturers are very familiar with their company's use of artificial intelligence (AI), while only 8% are somewhat familiar.\n",
            "* The study found that **three-fourths of manufacturers are currently investing in and applying AI**, with most functions being AI priorities for a majority of firms.\n",
            "* **64% of manufacturers** cited customer growth as a driver for their willingness to consider using AI, followed by improved performance (51%), increased productivity (50%), and competitive advantage (36%).\n",
            "\n",
            "**AI Usage and Priorities**\n",
            "\n",
            "* Manufacturers are using or considering AI for various activities, including:\n",
            "\t+ Analyzing transportation performance for logistics (57%)\n",
            "\t+ Analyzing fraud reports for finance (53%)\n",
            "\t+ Analyzing production performance for production and warehouses (52%)\n",
            "* The top concerns for manufacturers when implementing AI are:\n",
            "\t+ Cybersecurity and privacy (60%)\n",
            "\t+ Cost (46%)\n",
            "\t+ Employee concerns about AI (42%)\n",
            "\n",
            "**AI Impact and Investment**\n",
            "\n",
            "* Most manufacturers report a medium or high impact of AI on their businesses.\n",
            "* **Over the next three years**, manufacturers expect to invest significantly in AI, with many making significant investments in AI for different functions.\n",
            "\n",
            "**Key Findings and Recommendations**\n",
            "\n",
            "* The study suggests that manufacturers should develop detailed plans for what they hope to accomplish with AI and the expected returns on investment.\n",
            "* Companies should identify and resolve AI support gaps and establish policies and practices to minimize concerns about AI's unintended consequences.\n",
            "* CohnReznick has worked with manufacturers to develop and implement AI governance structures and deploy processes to achieve AI objectives.\n",
            "\n",
            "**Methodology**\n",
            "\n",
            "* The research study was conducted by The MPI Group, an independent research firm, and was fielded in October 2024.\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "print(f\"First 10000 characters of {len(initial_results.results)} results:\\n\\n {str(initial_results)[:10000]}\\n\\n...\\n\\n Last 10000 characters of {len(initial_results.results)} results:\\n\\n {str(initial_results)[-10000:]}\\n\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QTDIOnTIndwT"
      },
      "source": [
        "### STEP 3: Iterative Refinement - Addressing Information Gaps\n",
        "\n",
        "One of the most powerful aspects of our research pipeline is its ability to identify information gaps and adaptively refine the research process. Rather than relying solely on initial queries, the system:\n",
        "\n",
        "1. Evaluates the completeness of currently gathered information\n",
        "2. Identifies specific knowledge gaps related to the research topic\n",
        "3. Generates targeted follow-up queries to fill those gaps\n",
        "4. Iteratively expands the research until sufficient information is obtained\n",
        "\n",
        "**Disclaimer: This step involves multiple LLM runs processing long web texts, so we remind the user to be mindful about the resource management as defined at the beginning.**\n",
        "\n",
        "This iterative approach mimics how expert researchers work - continuously assessing what's known, what's missing, and how to fill those gaps systematically. Let's examine how this process works:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fw96wiLrndwT"
      },
      "outputs": [],
      "source": [
        "async def conduct_iterative_research(topic: str, initial_results: SearchResults, all_queries: List[str],\n",
        "                                  budget: int, max_queries: int, tavily_client: AsyncTavilyClient, together_client: AsyncTogether,\n",
        "                                  planning_model: str, json_model: str, summary_model: str, prompts: dict) -> tuple[SearchResults, List[str]]:\n",
        "    \"\"\"\n",
        "    Conduct iterative research within budget to refine results.\n",
        "\n",
        "    Args:\n",
        "        topic: The research topic\n",
        "        initial_results: Results from initial search\n",
        "        all_queries: List of all queries used so far\n",
        "        budget: Maximum number of follow-up iterations\n",
        "        max_queries: Maximum number of queries to use per iteration\n",
        "        tavily_client: The Tavily client for web search\n",
        "        together_client: The Together AI client for LLM operations\n",
        "        planning_model: Model to use for evaluation\n",
        "        json_model: Model to use for JSON parsing\n",
        "        summary_model: Model to use for summarization   \n",
        "        prompts: Dictionary of prompt templates\n",
        "\n",
        "    Returns:\n",
        "        Tuple of (final results, all queries used)\n",
        "    \"\"\"\n",
        "    results = initial_results\n",
        "\n",
        "    for _ in range(0, budget):\n",
        "        # Evaluate if more research is needed using the independent function\n",
        "        additional_queries = await evaluate_research_completeness(\n",
        "            topic, results, all_queries, together_client, planning_model, json_model, prompts\n",
        "        )\n",
        "\n",
        "        # Exit if research is complete\n",
        "        if not additional_queries:\n",
        "            print(\"No need for additional research\")\n",
        "            break\n",
        "\n",
        "        # Limit the number of queries if needed\n",
        "        if max_queries > 0:\n",
        "            additional_queries = additional_queries[:max_queries]\n",
        "        print(\"================================================\\n\\n\")\n",
        "        print(f\"Additional queries from evaluation parser: {additional_queries}\\n\\n\")\n",
        "        print(\"================================================\\n\\n\")\n",
        "\n",
        "        # Expand research with new queries\n",
        "        new_results = await perform_search(\n",
        "            additional_queries,\n",
        "            tavily_client,\n",
        "            prompts,\n",
        "            together_client,\n",
        "            summary_model\n",
        "        )\n",
        "\n",
        "        results = results + new_results\n",
        "        all_queries.extend(additional_queries)\n",
        "\n",
        "    return results, all_queries\n",
        "\n",
        "async def evaluate_research_completeness(topic: str, results: SearchResults, queries: List[str],\n",
        "                                       together_client: AsyncTogether, planning_model: str, json_model: str, prompts: dict) -> list[str]:\n",
        "    \"\"\"\n",
        "    Evaluate if the current search results are sufficient or if more research is needed.\n",
        "\n",
        "    Args:\n",
        "        topic: The research topic\n",
        "        results: Current search results\n",
        "        queries: List of queries already used\n",
        "        together_client: The Together AI client for LLM operations\n",
        "        planning_model: Model to use for evaluation\n",
        "        json_model: Model to use for JSON parsing\n",
        "        prompts: Dictionary of prompt templates\n",
        "\n",
        "    Returns:\n",
        "        List of additional queries needed or empty list if research is complete\n",
        "    \"\"\"\n",
        "    # Format the search results for the LLM\n",
        "    formatted_results = str(results)\n",
        "\n",
        "    EVALUATION_PROMPT = prompts[\"evaluation_prompt\"]\n",
        "\n",
        "    evaluation_response = await together_client.chat.completions.create(\n",
        "        model=planning_model,\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": EVALUATION_PROMPT},\n",
        "            {\"role\": \"user\", \"content\": (\n",
        "                f\"<Research Topic>{topic}</Research Topic>\\n\\n\"\n",
        "                f\"<Search Queries Used>{queries}</Search Queries Used>\\n\\n\"\n",
        "                f\"<Current Search Results>{formatted_results}</Current Search Results>\"\n",
        "            )}\n",
        "        ]\n",
        "    )\n",
        "    evaluation = evaluation_response.choices[0].message.content\n",
        "\n",
        "    print(\"================================================\\n\\n\")\n",
        "    print(f\"Evaluation:\\n\\n {evaluation}\")\n",
        "\n",
        "    EVALUATION_PARSING_PROMPT = prompts[\"evaluation_parsing_prompt\"]\n",
        "\n",
        "    json_response = await together_client.chat.completions.create(\n",
        "        model=json_model,\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": EVALUATION_PARSING_PROMPT},\n",
        "            {\"role\": \"user\", \"content\": f\"Evaluation to be parsed: {evaluation}\"}\n",
        "        ],\n",
        "        response_format={\"type\": \"json_object\", \"schema\": ResearchPlan.model_json_schema()}\n",
        "    )\n",
        "\n",
        "    response_json = json_response.choices[0].message.content\n",
        "    evaluation = json.loads(response_json)\n",
        "    return evaluation[\"queries\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9BIJNnF0ndwT",
        "outputId": "e9de75de-7d9a-4ffb-c336-597839c0864a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "================================================\n",
            "\n",
            "\n",
            "Evaluation:\n",
            "\n",
            " ### Analysis of Search Results\n",
            "\n",
            "#### Key Findings:\n",
            "1. **Key Applications of AI in Manufacturing:**\n",
            "   - **Predictive Maintenance**: Commonly mentioned across multiple sources, highlighting its importance in reducing downtime and maintenance costs.\n",
            "   - **Quality Control**: AI-powered computer vision systems are widely used for defect detection and ensuring product quality.\n",
            "   - **Supply Chain Optimization**: AI helps in demand forecasting, inventory management, and logistics.\n",
            "   - **Intelligent Automation**: Combines software and robotics to automate complex tasks.\n",
            "   - **Digital Twins**: Virtual replicas of physical systems for simulation and optimization.\n",
            "   - **Generative Design**: AI algorithms generate multiple design options.\n",
            "   - **Edge Analytics**: Real-time data processing at the edge of the network.\n",
            "   - **Workforce Management**: AI aids in workforce planning and training.\n",
            "\n",
            "2. **Benefits of AI in Manufacturing:**\n",
            "   - **Cost Reduction**: Through optimized processes and reduced downtime.\n",
            "   - **Improved Efficiency**: Automation and data-driven decision-making.\n",
            "   - **Enhanced Safety**: AI-powered systems handle hazardous tasks.\n",
            "   - **Sustainability**: Optimized resource use and reduced waste.\n",
            "   - **Customization**: Mass customization of products.\n",
            "\n",
            "3. **Challenges and Concerns:**\n",
            "   - **Data Quality and Availability**: High-quality data is essential for AI effectiveness.\n",
            "   - **Operational Risks**: AI models can lack precision in some applications.\n",
            "   - **Skills Shortages**: Lack of professionals with AI expertise.\n",
            "   - **Cybersecurity**: Increased digital connectivity brings new security risks.\n",
            "   - **Cultural Resistance**: Employee concerns about job security and AI integration.\n",
            "\n",
            "4. **Evolution of AI in Manufacturing:**\n",
            "   - **Early Adoption (1960s-2019)**: Focus on automation and machine vision.\n",
            "   - **Recent Advancements (2020-2023)**: Connected worker solutions, generative AI, and AI-first platforms.\n",
            "   - **Current State (2024 and beyond)**: Augmenting human workers, workforce optimization, and measuring worker engagement.\n",
            "\n",
            "5. **High AI Adoption Rates:**\n",
            "   - **92% of manufacturers** are familiar with AI.\n",
            "   - **75% are currently investing in AI**.\n",
            "   - **64% cite customer growth** as a driver for AI adoption.\n",
            "\n",
            "### Information Gaps:\n",
            "1. **Detailed Case Studies:**\n",
            "   - While there are examples of companies using AI, more detailed case studies with specific metrics and outcomes would provide a deeper understanding of the impact of AI in different manufacturing contexts.\n",
            "\n",
            "2. **Regulatory and Ethical Considerations:**\n",
            "   - The search results mention ethical concerns but lack detailed information on regulatory frameworks and ethical guidelines specific to AI in manufacturing.\n",
            "\n",
            "3. **Long-Term Impact:**\n",
            "   - There is limited information on the long-term impact of AI on the manufacturing industry, including potential job displacement, economic effects, and societal changes.\n",
            "\n",
            "4. **Regional Differences:**\n",
            "   - The search results do not provide a comparative analysis of AI adoption and impact across different regions or countries.\n",
            "\n",
            "5. **Technological Barriers:**\n",
            "   - More information on the specific technological barriers to AI adoption in manufacturing, such as integration with existing systems and interoperability issues, would be valuable.\n",
            "\n",
            "### Targeted Follow-Up Queries:\n",
            "1. **Detailed Case Studies:**\n",
            "   - \"Detailed case studies of AI implementation in manufacturing with specific metrics and outcomes\"\n",
            "   - \"Examples of successful AI projects in manufacturing with long-term impact analysis\"\n",
            "\n",
            "2. **Regulatory and Ethical Considerations:**\n",
            "   - \"Regulatory frameworks for AI in manufacturing\"\n",
            "   - \"Ethical guidelines for AI use in manufacturing\"\n",
            "\n",
            "3. **Long-Term Impact:**\n",
            "   - \"Long-term economic and societal impacts of AI in manufacturing\"\n",
            "   - \"Job displacement and workforce transformation due to AI in manufacturing\"\n",
            "\n",
            "4. **Regional Differences:**\n",
            "   - \"Comparative analysis of AI adoption in manufacturing across different regions\"\n",
            "   - \"Regional policies and initiatives supporting AI in manufacturing\"\n",
            "\n",
            "5. **Technological Barriers:**\n",
            "   - \"Technological barriers to AI adoption in manufacturing\"\n",
            "   - \"Best practices for integrating AI with existing manufacturing systems\"\n",
            "\n",
            "### Conclusion:\n",
            "While the current search results provide a comprehensive overview of the key applications, benefits, and challenges of AI in manufacturing, there are specific areas that require further exploration to fill the identified information gaps. Conducting follow-up research using the targeted queries will provide a more holistic understanding of the impact of AI on the manufacturing industry.\n",
            "================================================\n",
            "\n",
            "\n",
            "Additional queries from evaluation parser: ['Detailed case studies of AI implementation in manufacturing with specific metrics and outcomes', 'Examples of successful AI projects in manufacturing with long-term impact analysis']\n",
            "\n",
            "\n",
            "================================================\n",
            "\n",
            "\n",
            "Perform Tavily search with query: Detailed case studies of AI implementation in manufacturing with specific metrics and outcomes\n",
            "Perform Tavily search with query: Examples of successful AI projects in manufacturing with long-term impact analysis\n",
            "Tavily Responded with 5 results (Tavily returning None will be ignored for summarization)\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Tavily Responded with 5 results (Tavily returning None will be ignored for summarization)\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Search complete, found 7 results after deduplication\n",
            "================================================\n",
            "\n",
            "\n",
            "Evaluation:\n",
            "\n",
            " ### Analysis of Search Results\n",
            "\n",
            "The search results provide a comprehensive overview of the impact of artificial intelligence (AI) on the manufacturing industry. The information covers key applications, evolution over the past decade, detailed case studies, and long-term impact analysis. However, there are a few specific information gaps that need to be addressed to ensure a complete and well-rounded research report.\n",
            "\n",
            "### Key Findings from Search Results\n",
            "\n",
            "1. **Key Applications of AI in Manufacturing:**\n",
            "   - Predictive Maintenance\n",
            "   - Quality Control and Inspection\n",
            "   - Supply Chain Optimization\n",
            "   - Intelligent Automation\n",
            "   - Digital Twins\n",
            "   - Generative Design\n",
            "   - Edge Analytics\n",
            "   - Workforce Management\n",
            "   - Energy Management\n",
            "\n",
            "2. **Evolution of AI in Manufacturing Over the Past Decade:**\n",
            "   - Early adoption focused on automation and robotics.\n",
            "   - Recent advancements include connected worker solutions, generative AI, and AI-first approaches.\n",
            "   - Leading companies (Lighthouses) have made significant progress in scaling AI at the factory level and network-wide.\n",
            "\n",
            "3. **Detailed Case Studies:**\n",
            "   - Siemens: Reduced downtime and maintenance costs through predictive maintenance.\n",
            "   - General Electric (GE): Improved diagnostics and patient care with AI.\n",
            "   - Toyota: Enhanced production efficiency and reduced defects.\n",
            "   - Boeing: Increased manufacturing efficiency and product consistency.\n",
            "   - Intel: Boosted semiconductor output and reduced waste.\n",
            "   - Nissan: Improved assembly quality and production efficiency.\n",
            "   - Harley-Davidson: Enhanced customer satisfaction and market agility.\n",
            "   - ABB: Increased production efficiency and reduced energy consumption.\n",
            "   - LG Electronics: Improved production efficiency and environmental sustainability.\n",
            "   - Honda: Enhanced production flexibility and reduced environmental impact.\n",
            "   - BASF: Higher yields and lower waste in chemical production.\n",
            "   - Samsung: Increased production efficiency and product consistency.\n",
            "   - Volvo: Improved production efficiency and reduced environmental impact.\n",
            "   - Caterpillar: Enhanced machinery quality and durability.\n",
            "   - Philips: Maintained high standards and achieved sustainability goals.\n",
            "\n",
            "4. **Long-term Impact Analysis:**\n",
            "   - Increased operational efficiency\n",
            "   - Improved quality control\n",
            "   - Enhanced supply chain management\n",
            "   - Cost savings\n",
            "   - Increased competitiveness\n",
            "   - Sustainability and resilience\n",
            "\n",
            "### Information Gaps\n",
            "\n",
            "1. **Regulatory and Ethical Considerations:**\n",
            "   - The search results do not provide detailed information on the regulatory and ethical considerations surrounding the implementation of AI in manufacturing. This includes data privacy, job displacement, and ethical use of AI.\n",
            "\n",
            "2. **Challenges and Barriers to AI Adoption:**\n",
            "   - While some challenges are mentioned, such as data quality and cybersecurity, a more comprehensive list of barriers to AI adoption in manufacturing is needed. This should include issues like initial investment costs, resistance to change, and the need for specialized skills.\n",
            "\n",
            "3. **Impact on Workforce:**\n",
            "   - The impact of AI on the manufacturing workforce, including job displacement, reskilling, and upskilling, is not thoroughly covered. More detailed information on how companies are addressing these issues would be beneficial.\n",
            "\n",
            "4. **Case Studies from Smaller Manufacturers:**\n",
            "   - Most case studies focus on large, well-known companies. Including examples from smaller and medium-sized manufacturers (SMMs) would provide a more balanced view of AI adoption across the industry.\n",
            "\n",
            "5. **Long-term Sustainability and Environmental Impact:**\n",
            "   - While some case studies mention sustainability, a more detailed analysis of the long-term environmental impact of AI in manufacturing is needed. This should include energy consumption, waste reduction, and carbon footprint.\n",
            "\n",
            "### Targeted Follow-up Queries\n",
            "\n",
            "1. **Regulatory and Ethical Considerations:**\n",
            "   - \"What are the key regulatory and ethical considerations for implementing AI in manufacturing?\"\n",
            "   - \"How are manufacturers addressing data privacy and ethical use of AI in their operations?\"\n",
            "\n",
            "2. **Challenges and Barriers to AI Adoption:**\n",
            "   - \"What are the main challenges and barriers to AI adoption in manufacturing, and how are companies overcoming them?\"\n",
            "   - \"What are the initial investment costs and return on investment (ROI) for AI projects in manufacturing?\"\n",
            "\n",
            "3. **Impact on Workforce:**\n",
            "   - \"How is the adoption of AI affecting the manufacturing workforce, and what strategies are companies using to reskill and upskill their employees?\"\n",
            "   - \"What are the long-term implications of AI on job displacement in the manufacturing industry?\"\n",
            "\n",
            "4. **Case Studies from Smaller Manufacturers:**\n",
            "   - \"Can you provide examples of successful AI projects in small and medium-sized manufacturing companies?\"\n",
            "   - \"What are the unique challenges and benefits of AI adoption for smaller manufacturers?\"\n",
            "\n",
            "5. **Long-term Sustainability and Environmental Impact:**\n",
            "   - \"What is the long-term environmental impact of AI in manufacturing, and how are companies measuring and reducing their carbon footprint?\"\n",
            "   - \"How are manufacturers using AI to achieve sustainability goals and reduce energy consumption?\"\n",
            "\n",
            "### Conclusion\n",
            "\n",
            "The current search results provide a solid foundation for understanding the impact of AI on the manufacturing industry. However, to ensure a comprehensive and well-rounded research report, the identified information gaps need to be addressed through targeted follow-up queries. This will provide a more holistic view of the benefits, challenges, and long-term implications of AI in manufacturing.\n",
            "================================================\n",
            "\n",
            "\n",
            "Additional queries from evaluation parser: ['What are the key regulatory and ethical considerations for implementing AI in manufacturing?', 'How are manufacturers addressing data privacy and ethical use of AI in their operations?']\n",
            "\n",
            "\n",
            "================================================\n",
            "\n",
            "\n",
            "Perform Tavily search with query: What are the key regulatory and ethical considerations for implementing AI in manufacturing?\n",
            "Perform Tavily search with query: How are manufacturers addressing data privacy and ethical use of AI in their operations?\n",
            "Tavily Responded with 5 results (Tavily returning None will be ignored for summarization)\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Tavily Responded with 5 results (Tavily returning None will be ignored for summarization)\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Summarizing content asynchronously using the LLM\n",
            "Search complete, found 10 results after deduplication\n"
          ]
        }
      ],
      "source": [
        "results, all_queries=await conduct_iterative_research(topic=research_topic, initial_results=initial_results, all_queries=initial_queries, budget=budget, max_queries=max_queries, tavily_client=tavily_client, together_client=together_client, planning_model=planning_model, json_model=json_model, summary_model=summary_model, prompts=prompts)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Z0aiao9ndwT"
      },
      "source": [
        "Below is the comprehensive collection of all queries executed and results gathered throughout our research process. This dataset represents:\n",
        "\n",
        "1. Initial queries generated from the research topic\n",
        "2. Follow-up queries identified through gap analysis\n",
        "3. All retrieved content after preprocessing and relevance filtering\n",
        "4. Source metadata for proper attribution\n",
        "\n",
        "This consolidated view demonstrates the breadth and depth of information our system has gathered, providing the foundation for our final research synthesis:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RP1ruU1andwT",
        "outputId": "c175b37c-29ca-4a29-a2a8-bdcffa6b9b60"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['What are the key applications of AI in the manufacturing industry?',\n",
              " 'How has the adoption of AI in manufacturing evolved over the past decade?',\n",
              " 'Detailed case studies of AI implementation in manufacturing with specific metrics and outcomes',\n",
              " 'Examples of successful AI projects in manufacturing with long-term impact analysis',\n",
              " 'What are the key regulatory and ethical considerations for implementing AI in manufacturing?',\n",
              " 'How are manufacturers addressing data privacy and ethical use of AI in their operations?']"
            ]
          },
          "execution_count": 42,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "all_queries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pDBICxQlndwT",
        "outputId": "8116a45e-692d-491c-db33-f62c8b3fd564"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "First 10000 characters of 26 results:\n",
            "\n",
            " [1] Title: How Is AI Used in Manufacturing: Examples, Use Cases, and Benefits\n",
            "Link: https://www.azumuta.com/blog/how-is-ai-used-in-manufacturing-examples-use-cases-and-benefits/\n",
            "Refined Content: **Key Applications of AI in the Manufacturing Industry**\n",
            "\n",
            "Artificial intelligence (AI) is transforming the manufacturing landscape by enhancing efficiency, innovation, and productivity. The five main AI applications in manufacturing include:\n",
            "\n",
            "1. **Predictive Maintenance**: AI analyzes data from sensors, equipment telemetry, and other sources to forecast equipment failures, allowing for proactive maintenance scheduling, minimizing downtime, and reducing costs.\n",
            "2. **AI-Enhanced Quality Control**: AI-powered computer vision systems detect defects or anomalies in products, ensuring quality and safety.\n",
            "3. **Supply Chain Optimization**: AI predicts demand fluctuations, optimizes inventory, and identifies potential disruptions, enabling manufacturers to make informed decisions about production, procurement, and resource allocation.\n",
            "4. **Intelligent Automation**: AI automates tasks beyond repetition by combining intelligent software and robotic equipment, improving efficiency, flexibility, and ergonomics in manufacturing operations.\n",
            "5. **AI-Driven Training and Assistance**: AI-powered digital tools create clear and accurate work instructions, improving operational efficiency and productivity.\n",
            "\n",
            "**AI Technologies Used in Manufacturing**\n",
            "\n",
            "Some key AI technologies used in manufacturing include:\n",
            "\n",
            "1. **Machine Learning**: enables machines to learn from data, identify patterns, and make decisions.\n",
            "2. **Deep Learning**: a subset of machine learning that recognizes patterns using many processing layers, useful for image and speech recognition.\n",
            "3. **Natural Language Processing (NLP)**: enables machines to understand, interpret, and generate human language.\n",
            "4. **Computer Vision**: allows machines to interpret and understand visual information from images or videos.\n",
            "5. **Robotics**: combines AI with mechanical engineering to create machines that perform tasks autonomously or with minimal human intervention.\n",
            "\n",
            "**Real-World Examples of AI in Manufacturing**\n",
            "\n",
            "1. **BMW Group**: uses AI to automate quality processes in the conveyor belt, analyzing data recorded by cameras and sensor technology.\n",
            "2. **Ford**: integrates AI into assembly lines with robot arms that learn the most efficient way to assemble parts.\n",
            "3. **Rolls Royce**: uses digital twins and AI to consolidate data from produced engines, monitoring performance, predicting potential issues, and optimizing maintenance schedules.\n",
            "\n",
            "**Benefits of AI in Manufacturing**\n",
            "\n",
            "1. **Cost Savings**: AI reduces operational costs through optimized processes and decreased downtime.\n",
            "2. **Data-Driven Decision-Making**: AI analyzes vast amounts of data to identify trends and patterns, providing valuable insights for optimizing production processes.\n",
            "3. **Supply Chain Optimization**: AI optimizes supply chain logistics, inventory management, and procurement processes.\n",
            "4. **Improved Safety**: AI improves workplace safety by combining automation, real-time monitoring, and predictive analytics.\n",
            "5. **Enhanced Product Quality**: AI-powered vision systems inspect products with greater accuracy and speed than human inspectors.\n",
            "\n",
            "[2] Title: The Rise of AI in Manufacturing: 2025 Trends, Tools & Real-World Impact\n",
            "Link: https://www.authentise.com/post/the-rise-of-ai-in-manufacturing-2025-trends-tools-real-world-impact\n",
            "Refined Content: Here is the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**Key Applications of AI in Manufacturing:**\n",
            "\n",
            "1. **Predictive Maintenance**: AI systems can predict equipment failures before they occur, reducing unplanned downtime by up to 50% and cutting maintenance costs by 30%.\n",
            "2. **Quality Control**: Advanced machine vision systems, powered by deep learning algorithms, detect defects and deviations with unprecedented accuracy, ensuring higher product quality and reducing waste.\n",
            "3. **Automation of Documentation and Compliance**: AI can automate documentation, traceability, and compliance, reducing manual entry and enabling proactive quality checks.\n",
            "4. **Materials Management**: Predictive intelligence can help manufacturers plan, track, and reclaim materials more effectively, cutting costs and reducing waste.\n",
            "5. **Engineering Collaboration**: AI-powered platforms capture and contextualize intent from conversations, drawings, and data streams, ensuring decisions don't get lost.\n",
            "6. **Human-Machine Collaboration**: AI has led to the development of humanoid robots capable of performing complex tasks alongside human workers.\n",
            "7. **Smart Manufacturing**: AI enables real-time decision-making, near-autonomous systems, and seamless human-machine collaboration, enhancing flexibility and responsiveness in production environments.\n",
            "\n",
            "**Benefits of AI in Manufacturing:**\n",
            "\n",
            "* Reduced conversion costs by up to 20%\n",
            "* Improved workforce productivity\n",
            "* Enhanced troubleshooting processes\n",
            "* Increased efficiency and reduced costs\n",
            "* Improved product quality and reduced waste\n",
            "\n",
            "**Companies Using AI in Manufacturing:**\n",
            "\n",
            "* Siemens\n",
            "* Microsoft (Factory Operations Agent)\n",
            "* Nvidia\n",
            "* Figure AI\n",
            "* Authentise (integrating AI into core manufacturing workflows)\n",
            "\n",
            "**Trends and Future Directions:**\n",
            "\n",
            "* The emergence of \"AI factories\" where companies operate dual production lines - one for traditional products and another dedicated to AI model training and deployment.\n",
            "* The need for continuous upskilling of the workforce to effectively collaborate with intelligent systems.\n",
            "* The importance of addressing ethical considerations surrounding data privacy and job displacement.\n",
            "\n",
            "[3] Title: AI in Manufacturing: The Smart Revolution in Industry\n",
            "Link: https://sigmatechnology.com/articles/the-application-of-ai-in-manufacturing/\n",
            "Refined Content: **Key Applications of AI in the Manufacturing Industry**\n",
            "\n",
            "Based on the provided content, the key applications of AI in the manufacturing industry are:\n",
            "\n",
            "1. **Predictive Maintenance**: AI-powered predictive maintenance helps prevent downtime by monitoring machine performance, detecting anomalies, and scheduling repairs during planned downtime.\n",
            "2. **Quality Control and Inspection**: AI-driven computer vision systems enable automated defect detection, ensuring high-quality output and reducing waste.\n",
            "3. **Supply Chain Efficiency**: AI analyzes vast datasets to anticipate demand fluctuations, optimize inventory levels, and automate order fulfillment, leading to operational efficiency and reduced waste.\n",
            "4. **Collaborative Robots (Cobots)**: AI-powered cobots work safely alongside human employees, taking over repetitive tasks and freeing up skilled workers to focus on complex and creative aspects of the manufacturing process.\n",
            "5. **Digital Twins**: AI-driven digital twins simulate production processes, enhancing predictive analytics, anticipating equipment failures, and optimizing workflows.\n",
            "6. **Machine Learning (ML) and Deep Learning**: ML algorithms detect inefficiencies in production processes, predict demand, and optimize supply chains, while deep learning enables visual inspection and quality control.\n",
            "7. **Sensor Data Analytics (IoT + AI)**: AI processes data from IoT sensors to enable predictive maintenance, optimize energy consumption, and ensure quality control.\n",
            "8. **Computer Vision**: AI-powered computer vision systems enable automated defect detection, assembly line monitoring, and workplace safety.\n",
            "9. **Edge AI**: Edge AI brings AI processing directly to manufacturing devices, enabling real-time decision-making, autonomous manufacturing systems, and instant quality control.\n",
            "\n",
            "**Transformative Benefits of AI in Manufacturing**\n",
            "\n",
            "The integration of AI in manufacturing offers several benefits, including:\n",
            "\n",
            "1. **Cost Reduction and Increased Operational Efficiency**: AI-powered automation streamlines processes, reducing the need for human intervention in repetitive tasks and minimizing costly errors.\n",
            "2. **Customized Production and Dynamic Supply Chain Management**: AI enables manufacturers to offer customized products efficiently, blending the advantages of mass production with the appeal of one-off design.\n",
            "3. **Improved Sustainability**: AI optimizes energy consumption, minimizes environmental impact, and enhances the skills of the workforce.\n",
            "\n",
            "**Challenges and Lessons Learned**\n",
            "\n",
            "The content also highlights the challenges of AI implementation in manufacturing, including:\n",
            "\n",
            "1. **Managing Data Privacy and Security**: Protecting sensitive data from breaches and misuse is critical.\n",
            "2. **Bridging the AI Skills Gap**: Manufacturers need targeted reskilling and upskilling initiatives to close the skills gap.\n",
            "3. **Overcoming Cultural Resistance**: Creating an inclusive and supportive culture around AI implementation is essential.\n",
            "4. **Ensuring Ethical and Transparent AI Use**: Clear ethical guidelines and transparent AI processes build trust among employees, partners, and customers.\n",
            "\n",
            "By understanding these key applications, benefits, and challenges of AI in manufacturing, companies can successfully leverage AI's full potential, securing a competitive advantage in an increasingly digital and sustainability-focused market.\n",
            "\n",
            "[4] Title: Manufacturing AI: Top 15 tools & 13 real-life use cases ['25] - AIMultiple\n",
            "Link: https://research.aimultiple.com/manufacturing-ai/\n",
            "Refined Content: Here is the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**Key Applications of AI in the Manufacturing Industry:**\n",
            "\n",
            "1. **Predictive Maintenance**: AI analyzes sensor data to forecast equipment failures, allowing for scheduled maintenance and reducing downtime. (Example: PepsiCo’s Frito-Lay plants saved costs and improved equipment performance using AI-driven predictive maintenance.)\n",
            "2. **Generative Design**: AI algorithms generate multiple design options based on parameters, enabling manufacturers to quickly produce thousands of designs. (Example: Airbus reduced aerodynamics prediction times from \n",
            "\n",
            "...\n",
            "\n",
            " Last 10000 characters of 26 results:\n",
            "\n",
            " ment, focusing on privacy, fairness, and transparency, and actively addresses biases.\n",
            "\n",
            "### Collaboration for Ethical AI Development\n",
            "\n",
            "Developing ethical AI requires collaboration among various stakeholders:\n",
            "\n",
            "* **Academics**: Research and develop theories and statistics to guide AI ethics.\n",
            "* **Governments**: Facilitate AI ethics through regulations and public outreach.\n",
            "* **Intergovernmental Entities**: Raise global awareness and draft international agreements on AI ethics.\n",
            "* **Non-Profit Organizations**: Advocate for diverse representation and create guidelines.\n",
            "\n",
            "### Best Practices for Manufacturers\n",
            "\n",
            "To address data privacy and ethical use of AI, manufacturers can:\n",
            "\n",
            "1. **Establish AI ethics boards** to ensure accountability and transparency.\n",
            "2. **Implement robust data protection measures** to respect individual privacy.\n",
            "3. **Invest in bias detection and mitigation** to eliminate unfair outcomes.\n",
            "4. **Prioritize human-centric design**, focusing on AI that augments rather than replaces human capabilities.\n",
            "5. **Consider the environmental impact** of AI technologies and strive for sustainable practices.\n",
            "\n",
            "By adopting these best practices and collaborating with various stakeholders, manufacturers can ensure that their use of AI is both responsible and beneficial to society.\n",
            "\n",
            "[24] Title: AI Safety and Ethical Considerations | SpringerLink\n",
            "Link: https://link.springer.com/chapter/10.1007/978-3-031-86091-1_6\n",
            "Refined Content: **Relevant Information for Research Topic: How are manufacturers addressing data privacy and ethical use of AI in their operations?**\n",
            "\n",
            "**Key Findings:**\n",
            "\n",
            "1. **Adoption of AI Ethics Policies:** As of 2021, 89% of industrial manufacturers have reported the adoption of AI ethics policies, indicating a significant effort to address ethical considerations in AI deployment.\n",
            "2. **Ethical Challenges:** Manufacturers face key ethical challenges, including:\n",
            "\t* Algorithmic biases\n",
            "\t* Cybersecurity threats\n",
            "\t* Job displacement\n",
            "\t* Compliance with global regulations\n",
            "3. **Strategies for Ethical AI Implementation:** To address these challenges, manufacturers are focusing on:\n",
            "\t* Ensuring transparency\n",
            "\t* Interpretability\n",
            "\t* Human oversight\n",
            "\t* Continuous monitoring\n",
            "4. **Governance Framework:** A governance framework for responsible AI is necessary, built on the pillars of:\n",
            "\t* People\n",
            "\t* Processes\n",
            "\t* Technology\n",
            "5. **Regulatory Variations:** Manufacturers must comply with diverse legal landscapes, including:\n",
            "\t* EU's AI Act\n",
            "\t* China's stringent compliance requirements\n",
            "6. **Actionable Insights:** To foster responsible AI use, manufacturers should:\n",
            "\t* Design fairness-aware algorithms\n",
            "\t* Align corporate policies with international standards\n",
            "\n",
            "**Specific Data and Terminology:**\n",
            "\n",
            "1. **Algorithmic Fairness:** The absence of discrimination or bias in automated decision-making systems.\n",
            "2. **Human Oversight:** Human monitoring and intervention in AI-driven processes to ensure ethical and responsible operation.\n",
            "3. **Interpretability:** The degree to which a human can understand the cause of a decision or prediction made by an AI model.\n",
            "4. **Responsible AI:** The practice of designing, developing, and deploying AI systems in a manner that empowers people and businesses and fairly impacts customers and society.\n",
            "5. **Transparency:** The clarity and openness about how AI systems operate and make decisions.\n",
            "\n",
            "**References:**\n",
            "\n",
            "1. **Statista (2021):** Worldwide artificial intelligence implementation ethics policies.\n",
            "2. **Copper Digital:** Exploring ethical AI decision making in manufacturing.\n",
            "3. **Ryan et al. (2021):** Research and practice of AI ethics: A case study approach.\n",
            "4. **Princeton Dialogues on AI and Ethics:** Case studies on AI ethics.\n",
            "5. **Brookings (2023):** The EU and U.S. diverge on AI regulation: A transatlantic comparison and steps to alignment.\n",
            "\n",
            "[25] Title: How Businesses Can Ethically Embrace Artificial Intelligence - Forbes\n",
            "Link: https://www.forbes.com/councils/forbesbusinesscouncil/2023/05/12/how-businesses-can-ethically-embrace-artificial-intelligence/\n",
            "Refined Content: Here is the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**Addressing Data Privacy and Ethical Use of AI in Operations**\n",
            "\n",
            "Manufacturers are addressing data privacy and ethical use of AI in their operations by implementing guidelines and regulations to prevent misuse. Some of the measures being taken include:\n",
            "\n",
            "1. **Disclosing information about AI systems**: Companies are disclosing information about their AI systems to identify potential risks and ensure accountability.\n",
            "2. **Establishing guidelines for acceptable data types and collection methods**: Businesses are creating guidelines that outline acceptable data types and personal information collection methods to ensure data security and prevent misuse.\n",
            "3. **Ensuring data security**: Companies are taking steps to ensure data security and prevent unauthorized access or misuse of data.\n",
            "4. **Creating laws and regulations**: Governments are being lobbied to introduce laws and regulations to safeguard personal data and establish independent bodies to monitor AI's use and investigate potential misuse.\n",
            "\n",
            "**Ethical Concerns and Potential Risks**\n",
            "\n",
            "Some of the ethical concerns raised about AI include:\n",
            "\n",
            "1. **Plagiarism and propaganda**: AI can be used to create false information or propaganda to influence public opinion.\n",
            "2. **Job loss**: AI can automate tasks, potentially replacing human jobs.\n",
            "3. **Decline in creativity**: AI may rely on existing data, limiting its ability to provide innovative solutions.\n",
            "4. **Data privacy**: AI systems can collect and use personal data, raising concerns about data protection and security.\n",
            "\n",
            "**Best Practices for Ethical AI Use**\n",
            "\n",
            "To address these concerns, manufacturers are:\n",
            "\n",
            "1. **Implementing restrictions**: Companies are implementing restrictions to maintain the well-meant intention of AI use.\n",
            "2. **Lobbying for safeguards**: Business leaders are lobbying for safeguards at a higher level within their governments.\n",
            "3. **Adhering to guidelines**: Companies are adhering to guidelines that prohibit misuse, outline acceptable data types and personal information collection methods, and ensure data security.\n",
            "\n",
            "By taking these steps, manufacturers can ensure the safe and ethical integration of AI in their operations, addressing concerns around data privacy and ethical use of AI.\n",
            "\n",
            "[26] Title: Top 10 companies with ethical AI practices\n",
            "Link: https://aimagazine.com/ai-strategy/top-10-companies-with-ethical-ai-practices\n",
            "Refined Content: Here is the extracted and synthesized information relevant to the research topic:\n",
            "\n",
            "**Companies Addressing Data Privacy and Ethical Use of AI:**\n",
            "\n",
            "1. **Accenture**: \n",
            "   - Established a generative AI and LLM Centre of Excellence (CoE) with 1600 employees.\n",
            "   - Trained 40,000 employees in AI to support the CoE.\n",
            "   - Aims to design and deploy responsible AI solutions that are ethical, transparent, and trustworthy.\n",
            "\n",
            "2. **Deloitte**: \n",
            "   - Offers an AI Risk Management Framework for effective operating models and crisis management.\n",
            "   - Provides services to help organizations govern AI responsibly.\n",
            "   - Conducts roundtable workshops and training for senior leaders on AI ethics.\n",
            "\n",
            "3. **Google DeepMind**: \n",
            "   - Merged with Google Brain to tackle AI research and products that improve human lives.\n",
            "   - Achieved scientific breakthroughs, such as AlphaFold for predicting protein structures.\n",
            "\n",
            "4. **TCS (Tata Consultancy Services)**: \n",
            "   - Uses the Machine First Delivery Model (MFDM) to address ethics in AI services and solutions.\n",
            "   - Announced plans to train 25,000 engineers to be certified on Microsoft Azure Open AI.\n",
            "\n",
            "5. **Apple**: \n",
            "   - Joined the Partnership on AI research group to demonstrate its commitment to AI ethics.\n",
            "   - Explores AI to solve real-world problems through deep learning, reinforcement learning, and research.\n",
            "\n",
            "6. **AWS (Amazon Web Services)**: \n",
            "   - Committed to developing fair and accurate AI/ML services.\n",
            "   - Provides tools and guidance for building AI applications responsibly.\n",
            "   - Offers education and training through programs like the AWS Machine Learning University.\n",
            "\n",
            "7. **IBM**: \n",
            "   - Aims to help people and organizations adopt AI responsibly via ethical principles.\n",
            "   - Pillars of ethical AI: explainability, fairness, robustness, transparency, and privacy.\n",
            "   - Established a Center of Excellence (CoE) for Generative AI.\n",
            "\n",
            "8. **Meta**: \n",
            "   - Grounds its work in five pillars of responsibility: privacy and security, fairness and inclusion, robustness and safety, transparency and control, and accountability and governance.\n",
            "   - Addresses AI fairness through research and diverse datasets.\n",
            "\n",
            "9. **Microsoft**: \n",
            "   - Created the Microsoft Responsible AI Standard principles to guide AI system design, build, and testing.\n",
            "   - Collaborates with researchers and academics to advance responsible AI practices.\n",
            "\n",
            "10. **Google AI**: \n",
            "    - Works towards eliminating biases in AI teams through human-centered design and data examination.\n",
            "    - Will not pursue AI applications in technologies like weapons or surveillance that violate human rights.\n",
            "\n",
            "**Key Terminology and Context:**\n",
            "\n",
            "* **Ethical AI**: Prioritizing data protection and preventing data bias.\n",
            "* **Responsible AI**: Ensuring AI systems are trustworthy, transparent, and fair.\n",
            "* **AI Risk Management Framework**: A structure for effective operating models and crisis management.\n",
            "* **Machine First Delivery Model (MFDM)**: TCS's approach to addressing ethics in AI services and solutions.\n",
            "* **Center of Excellence (CoE)**: A centralized unit for expertise and innovation, e.g., Accenture's generative AI and LLM CoE.\n",
            "\n",
            "**Specific Data:**\n",
            "\n",
            "* 35% of global consumers trust how AI is being implemented by organizations (Accenture).\n",
            "* 77% of consumers think organizations must be held accountable for AI misuse (Accenture).\n",
            "* 25,000 engineers to be certified on Microsoft Azure Open AI by TCS.\n",
            "* 1600 employees in Accenture's generative AI and LLM Centre of Excellence.\n",
            "* 40,000 employees trained in AI by Accenture.\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "print(f\"First 10000 characters of {len(results.results)} results:\\n\\n {str(results)[:10000]}\\n\\n...\\n\\n Last 10000 characters of {len(results.results)} results:\\n\\n {str(results)[-10000:]}\\n\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yXiR66R7ndwT"
      },
      "source": [
        "### STEP 4: Content Filtering and Prioritization - Optimizing Information Quality\n",
        "\n",
        "After gathering comprehensive information through our iterative search process, we face a new challenge: not all content is equally valuable or relevant. This step addresses the critical task of quality control by:\n",
        "\n",
        "1. Evaluating each source for relevance, credibility, and information value\n",
        "2. Prioritizing the most pertinent content while filtering out noise\n",
        "3. Organizing information to create an optimal foundation for synthesis\n",
        "\n",
        "This filtering process improves the quality of our final research output by ensuring we synthesize only the most valuable information. The system acts as a discerning research assistant, making informed judgments about which sources deserve attention:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2NG0amc7ndwT"
      },
      "outputs": [],
      "source": [
        "async def filter_results(topic: str, results: SearchResults, together_client: AsyncTogether, json_model: str, max_sources: int, prompts: dict) -> tuple[SearchResults, SourceList]:\n",
        "    \"\"\"\n",
        "    Filter and rank search results based on relevance to the research topic.\n",
        "\n",
        "    Args:\n",
        "        topic: The research topic\n",
        "        results: Search results to filter\n",
        "        together_client: The Together AI client for LLM operations\n",
        "        json_model: Model to use for filtering\n",
        "        max_sources: Maximum number of sources to keep (-1 for unlimited)\n",
        "        prompts: Dictionary of prompt templates\n",
        "\n",
        "    Returns:\n",
        "        Tuple of (filtered results, source list with indices)\n",
        "    \"\"\"\n",
        "    # Format the search results for the LLM, without the raw content\n",
        "    formatted_results = results.short_str()\n",
        "\n",
        "    FILTER_PROMPT = prompts[\"filter_prompt\"]\n",
        "\n",
        "    SOURCE_PARSING_PROMPT = prompts['source_parsing_prompt']\n",
        "\n",
        "    llm_filter_response = await together_client.chat.completions.create(\n",
        "        model=json_model,\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": FILTER_PROMPT},\n",
        "            {\"role\": \"user\", \"content\": (\n",
        "                f\"<Research Topic>{topic}</Research Topic>\\n\\n\"\n",
        "                f\"<Current Search Results>{formatted_results}</Current Search Results>\"\n",
        "            )}\n",
        "        ]\n",
        "    )\n",
        "\n",
        "    llm_filter_response_content = llm_filter_response.choices[0].message.content\n",
        "    print(f'filter response: {llm_filter_response_content}')\n",
        "\n",
        "    json_response = await together_client.chat.completions.create(\n",
        "        model=json_model,\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": SOURCE_PARSING_PROMPT},\n",
        "            {\"role\": \"user\", \"content\": f\"<FILTER_RESPONSE>{llm_filter_response_content}</FILTER_RESPONSE>\"}\n",
        "        ],\n",
        "        response_format={\"type\": \"json_object\", \"schema\": SourceList.model_json_schema()}\n",
        "    )\n",
        "\n",
        "    response_json = json_response.choices[0].message.content\n",
        "    evaluation = json.loads(response_json)\n",
        "    sources = evaluation[\"sources\"]\n",
        "\n",
        "    print(f'sources ranked by relevance {sources} (we will keep maximum of {max_sources} sources, as defined by the user)')\n",
        "\n",
        "    if max_sources > 0:\n",
        "        sources = sources[:max_sources]\n",
        "\n",
        "    # Filter the results based on the source list\n",
        "    filtered_results = [results.results[i] for i in sources if i < len(results.results)]\n",
        "\n",
        "    return SearchResults(filtered_results), sources\n",
        "\n",
        "async def process_search_results(topic: str, results: SearchResults, together_client: AsyncTogether, json_model: str, max_sources: int, prompts: dict) -> SearchResults:\n",
        "    \"\"\"Step 4: Process search results by deduplicating and filtering\"\"\"\n",
        "    # Deduplicate results\n",
        "    results = results.dedup()\n",
        "    print(f\"Deduplication complete, kept {len(results.results)} results\")\n",
        "\n",
        "    # Filter results\n",
        "    filtered_results, sources = await filter_results(topic, results, together_client, json_model, max_sources, prompts)\n",
        "    print(f\"LLM Filtering complete, kept {len(filtered_results.results)} results\")\n",
        "\n",
        "    return filtered_results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VI6PD9MjpL8S"
      },
      "source": [
        "**Note: In rare cases we saw none results were kept after LLM filtering, where rerunning the cell would typically resolve the issue.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y2eI087SndwT",
        "outputId": "ca299d43-00e1-471b-e785-0c22ddef9256"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Deduplication complete, kept 26 results\n",
            "filter response: Based on the research topic \"The impact of artificial intelligence on the manufacturing industry,\" I have evaluated each search result for relevance, accuracy, and information value. Here is the list of source numbers with the rank of relevance:\n",
            "\n",
            "**Highly Relevant (1-5)**\n",
            "\n",
            "1. [5] Title: How is AI being used in Manufacturing | IBM\n",
            "2. [6] Title: Artificial Intelligence in manufacturing: State of the art ...\n",
            "3. [4] Title: Manufacturing AI: Top 15 tools & 13 real-life use cases ['25] - AIMultiple\n",
            "4. [14] Title: AI in Manufacturing: 12 key Use Cases, Examples, and more - MultiQoS\n",
            "5. [16] Title: Artificial Intelligence in Manufacturing: Real World Success Stories ...\n",
            "\n",
            "**Relevant (6-10)**\n",
            "\n",
            "6. [1] Title: How Is AI Used in Manufacturing: Examples, Use Cases, and Benefits\n",
            "7. [2] Title: The Rise of AI in Manufacturing: 2025 Trends, Tools & Real-World Impact\n",
            "8. [3] Title: AI in Manufacturing: The Smart Revolution in Industry\n",
            "9. [7] Title: The Evolution of AI in Manufacturing - Augmentir\n",
            "10. [8] Title: Adopting AI in manufacturing at speed and scale | McKinsey\n",
            "\n",
            "**Somewhat Relevant (11-15)**\n",
            "\n",
            "11. [9] Title: Study reveals high AI adoption in manufacturing sector\n",
            "12. [10] Title: How can AI be Used in Manufacturing? [15 Case Studies] [2025]\n",
            "13. [11] Title: Eight AI Case Studies Demonstrate the Potential of AI in Manufacturing\n",
            "14. [12] Title: AI in the Real World: 4 Case Studies of Success in Industrial Manufacturing\n",
            "15. [15] Title: AI Transformation Chronicles: AI in Manufacturing Industry\n",
            "\n",
            "**Not Relevant (16-25)**\n",
            "\n",
            "16. [17] Title: Three Key Considerations for Companies Implementing Ethical AI\n",
            "17. [18] Title: (PDF) Ethical Considerations in the Deployment and Regulation of ...\n",
            "18. [19] Title: Ethical Considerations and Responsible Implementation of AI - Verbat\n",
            "19. [20] Title: How Ethics, Regulations And Guidelines Can Shape Responsible AI - Forbes\n",
            "20. [21] Title: 5 Ethical Considerations of AI in Business - Harvard Business School Online\n",
            "21. [22] Title: AI's Data Dilemma: Privacy, Regulation, and the Future of Ethical AI\n",
            "22. [23] Title: 13 Ethical AI Companies Leading the Way in Responsible Innovation\n",
            "23. [24] Title: AI Safety and Ethical Considerations | SpringerLink\n",
            "24. [25] Title: How Businesses Can Ethically Embrace Artificial Intelligence - Forbes\n",
            "25. [26] Title: Top 10 companies with ethical AI practices\n",
            "\n",
            "The top 5 sources are highly relevant to the research topic, providing in-depth information on the impact of AI on the manufacturing industry. The next 5 sources are relevant, offering insights into the use of AI in manufacturing, its benefits, and challenges. The remaining sources are somewhat relevant or not relevant, focusing on ethical considerations, regulations, and guidelines for AI implementation, which are not directly related to the research topic.\n",
            "sources ranked by relevance [5, 6, 4, 14, 16, 1, 2, 3, 7, 8, 9, 10, 11, 12, 15] (we will keep maximum of 10 sources, as defined by the user)\n",
            "LLM Filtering complete, kept 10 results\n"
          ]
        }
      ],
      "source": [
        "processed_results=await process_search_results(research_topic, results, together_client, json_model, max_sources, prompts)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SJohln_-ndwT"
      },
      "source": [
        "### STEP 5: Research Synthesis - Generating the Comprehensive Final Report\n",
        "\n",
        "We've now reached the culmination of our research process. After carefully gathering, refining, and filtering information through multiple stages, we're ready to synthesize everything into a cohesive, publication-quality report. This final step:\n",
        "\n",
        "1. Integrates information from all selected sources into a unified narrative\n",
        "2. Structures content logically with clear sections and progression of ideas\n",
        "3. Provides proper citations for all factual claims and insights\n",
        "4. Balances breadth and depth to create a thorough yet readable analysis\n",
        "5. Preserves nuance and context from the original sources\n",
        "\n",
        "The synthesis process uses our specialized answer model, which excels at long-form, coherent content generation while maintaining factual accuracy. Let's generate our final research report:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6KD_V6jXndwT"
      },
      "outputs": [],
      "source": [
        "def remove_thinking_tags_from_answer(answer: str) -> str:\n",
        "    \"\"\"Remove content within <think> tags\"\"\"\n",
        "    while \"<think>\" in answer and \"</think>\" in answer:\n",
        "        start = answer.find(\"<think>\")\n",
        "        end = answer.find(\"</think>\") + len(\"</think>\")\n",
        "        answer = answer[:start] + answer[end:]\n",
        "    return answer\n",
        "\n",
        "async def generate_research_answer(topic: str, results: SearchResults, together_client: AsyncTogether, answer_model: str, prompts: dict, max_tokens: int, remove_thinking_tags: bool = True) -> str:\n",
        "    \"\"\"\n",
        "    Generate a comprehensive answer to the research topic based on the search results.\n",
        "\n",
        "    Args:\n",
        "        topic: The research topic\n",
        "        results: Filtered search results to use for answer generation\n",
        "        together_client: The Together AI client for LLM operations\n",
        "        answer_model: Model to use for answer generation\n",
        "        prompts: Dictionary of prompt templates\n",
        "        max_tokens: Maximum number of tokens in the answer\n",
        "        remove_thinking_tags: Whether to remove <think> tags from the answer\n",
        "\n",
        "    Returns:\n",
        "        Detailed research answer as a string\n",
        "    \"\"\"\n",
        "    formatted_results = str(results)\n",
        "\n",
        "    ANSWER_PROMPT = prompts[\"answer_prompt\"]\n",
        "\n",
        "    answer_response = await together_client.chat.completions.create(\n",
        "        model=answer_model,\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": ANSWER_PROMPT},\n",
        "            {\"role\": \"user\", \"content\": f\"Research Topic: {topic}\\n\\nSearch Results:\\n{formatted_results}\"}\n",
        "        ],\n",
        "        max_tokens=max_tokens\n",
        "    )\n",
        "\n",
        "    answer = answer_response.choices[0].message.content\n",
        "\n",
        "    # Remove <think> tokens for reasoning models\n",
        "    if remove_thinking_tags:\n",
        "        answer = remove_thinking_tags_from_answer(answer)\n",
        "\n",
        "    # Handle potential error cases\n",
        "    if answer is None or not isinstance(answer, str):\n",
        "        print(\"ERROR: No answer generated\")\n",
        "        return \"No answer generated\"\n",
        "\n",
        "    return answer.strip()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YttHKYXRA4ZO"
      },
      "outputs": [],
      "source": [
        "research_answer = await generate_research_answer(research_topic, processed_results, together_client, answer_model, prompts, max_tokens)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "8M_zmm-PndwU",
        "outputId": "26bcebb4-4857-4c8c-ab45-f728dc5434df"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "# The Impact of Artificial Intelligence on the Manufacturing Industry\n",
              "The manufacturing industry has witnessed significant transformations over the past decade, driven in part by the adoption of Artificial Intelligence (AI). As AI technologies continue to evolve, their impact on manufacturing is becoming increasingly profound, influencing various aspects of production, operations, and management.\n",
              "\n",
              "## Introduction\n",
              "The integration of AI in manufacturing has been a gradual process, with early applications focusing on automation and robotics. However, with advancements in machine learning, data analytics, and other AI-related technologies, the scope of AI in manufacturing has expanded dramatically. Today, AI is being used to optimize production processes, improve product quality, and enhance operational efficiency.\n",
              "\n",
              "## Evolution of AI in Manufacturing\n",
              "Over the past decade, the adoption of AI in manufacturing has seen accelerated growth, particularly with the advent of Industry 4.0 \\[1\\]. Initially, AI was used primarily for automation and replacing human workers. However, recent advancements have shifted the focus towards augmenting and supporting frontline workers \\[2\\]. The current state of AI in manufacturing is characterized by the development of smart manufacturing systems, which leverage AI, machine learning, and data analytics to optimize production processes.\n",
              "\n",
              "## Key Applications of AI in Manufacturing\n",
              "AI is being applied in various areas of manufacturing, including predictive maintenance, quality control, supply chain optimization, and workforce management \\[3\\]. Predictive maintenance, for instance, uses AI to analyze sensor data from machinery to forecast failures before they occur, reducing unexpected downtimes and maintenance costs. Quality control is another significant application, where AI-powered computer vision systems scan products in real-time to identify defects, improving product quality and reducing waste.\n",
              "\n",
              "## Benefits of AI in Manufacturing\n",
              "The benefits of AI in manufacturing are multifaceted. AI-driven automation accelerates production by taking over repetitive tasks, reducing human error, and optimizing workflows. This leads to increased efficiency, cost reduction, and improved decision-making. Additionally, AI enhances safety by enabling collaborative robots to handle strenuous or hazardous tasks alongside human workers. Sustainability is also improved as AI optimizes resource allocation, reduces energy use, and limits waste \\[3\\].\n",
              "\n",
              "## Challenges and Future Directions\n",
              "Despite the numerous benefits, the adoption of AI in manufacturing is not without challenges. Key concerns include data quality and availability, operational risks associated with AI models, skills shortages, and cybersecurity concerns \\[3\\]. To address these challenges, manufacturers must invest in data infrastructure, develop AI expertise, and cultivate visionary leadership. Furthermore, aligning AI initiatives with business goals and implementing continuous measurement and improvement are crucial for long-term success.\n",
              "\n",
              "## Successful AI Projects and Long-term Impact\n",
              "Several manufacturing companies have successfully implemented AI projects, achieving significant long-term impacts. For example, Siemens implemented an AI-driven predictive maintenance system, reducing unplanned downtime and maintenance costs. Foxconn deployed AI-driven visual inspection systems, significantly improving product quality and reducing inspection times. These examples demonstrate the transformative potential of AI in driving efficiency, innovation, and competitive advantage in manufacturing \\[4\\].\n",
              "\n",
              "## Ethical Considerations and Regulatory Frameworks\n",
              "As AI becomes increasingly integral to manufacturing, ethical considerations and regulatory frameworks are gaining importance. Manufacturers must ensure that AI systems are transparent, trustworthy, and comply with regulatory requirements. The EU Artificial Intelligence Act 2024 and other regulatory frameworks aim to ensure that AI is used responsibly and ethically \\[5\\].\n",
              "\n",
              "## Conclusion\n",
              "The impact of AI on the manufacturing industry is profound and multifaceted. As AI technologies continue to evolve, their applications in manufacturing are expected to expand, driving further improvements in efficiency, productivity, and innovation. To fully leverage the potential of AI, manufacturers must address the associated challenges and ensure that AI is implemented in a responsible and ethical manner. By doing so, manufacturers can achieve significant competitive advantages and contribute to a more sustainable and efficient manufacturing ecosystem.\n",
              "\n",
              "## References\n",
              "\\[1\\] https://www.sciencedirect.com/science/article/pii/S000785062400115X  \n",
              "\\[2\\] https://www.augmentir.com/blog/the-evolution-of-ai-in-manufacturing  \n",
              "\\[3\\] https://www.ibm.com/think/topics/ai-in-manufacturing  \n",
              "\\[4\\] https://digitopia.co/blog/ai-in-manufacturing-industry/  \n",
              "\\[5\\] https://www.silicon.co.uk/expert-advice/three-key-considerations-for-companies-implementing-ethical-ai  \n",
              "\\[6\\] https://www.authentise.com/post/the-rise-of-ai-in-manufacturing-2025-trends-tools-real-world-impact  \n",
              "\\[7\\] https://sigmatechnology.com/articles/the-application-of-ai-in-manufacturing/  \n",
              "\\[8\\] https://research.aimultiple.com/manufacturing-ai/  \n",
              "\\[9\\] https://www.mckinsey.org/capabilities/operations/our-insights/adopting-ai-at-speed-and-scale-the-4ir-push-to-stay-competitive  \n",
              "\\[10\\] https://www.cohnreznick.com/insights/manufacturing-checkup-artificial-intelligence"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from IPython.display import Markdown\n",
        "display(Markdown(research_answer))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SpzFOqCandwU"
      },
      "source": [
        "#### Exporting and Sharing Research Results\n",
        "\n",
        "The final step in our research pipeline is to format and export the results for sharing, preservation, and presentation (e.g., html, pdf, slides, etc). Enjoy!\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6l86Thrnxof3",
        "outputId": "784d9739-1527-423a-dc19-fdd7fb329be2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Research report saved as research_report.html\n"
          ]
        }
      ],
      "source": [
        "import markdown\n",
        "\n",
        "# Convert to HTML\n",
        "report_html = markdown.markdown(research_answer)\n",
        "\n",
        "filename=\"research_report.html\"\n",
        "with open(filename, \"w\", encoding=\"utf-8\") as f:\n",
        "    f.write(report_html)\n",
        "\n",
        "print(f\"Research report saved as {filename}\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bxtTiuQSndwY"
      },
      "source": [
        "## Limitations and Considerations\n",
        "\n",
        "While the Deep Search approach offers powerful research capabilities, it's important to understand its limitations and constraints:\n",
        "\n",
        "### Information Constraints\n",
        "- **Recency Gap**: Search-based research is limited by the indexing latency of search engines. Very recent events or publications may not be captured.\n",
        "- **Access Limitations**: Some valuable information sources may be behind paywalls or otherwise inaccessible to the search system.\n",
        "- **Domain Specificity**: Highly specialized or technical domains may have limited indexing in general search engines.\n",
        "\n",
        "### Processing Considerations\n",
        "- **Query Dependency**: The quality of results is heavily influenced by the initial and follow-up queries generated. Poorly formulated queries can lead to information gaps.\n",
        "- **Source Reliability**: While the system attempts to prioritize credible sources, it cannot perfectly assess the accuracy or bias of all content.\n",
        "- **Content Extraction Challenges**: Complex formats like tables, charts, or specialized notation may not be accurately captured in the content processing stage.\n",
        "\n",
        "### Resource Requirements\n",
        "- **API Costs**: The system makes multiple calls to search and LLM APIs, which can incur significant costs for extensive research topics.\n",
        "- **Time Efficiency**: The iterative, multi-step nature of the process means it takes longer than simple RAG or direct LLM approaches.\n",
        "- **Model Dependencies**: The quality of outputs depends on having access to capable LLM models for different stages of the pipeline.\n",
        "\n",
        "### Ethical Considerations\n",
        "- **Source Attribution**: While the system attempts to provide proper citations, it may not always perfectly attribute information to original creators.\n",
        "- **Content Bias**: The system may inadvertently amplify biases present in the indexed content or search ranking algorithms.\n",
        "- **Privacy Implications**: Research on certain topics may involve information with privacy sensitivities.\n",
        "\n",
        "Understanding these limitations helps set appropriate expectations and identify situations where this approach should be complemented with other research methodologies. For critical research needs, human review and validation remain essential."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qrAPRcNendwY"
      },
      "source": [
        "## Acknowledgements\n",
        "\n",
        "This cookbook relies on several key technologies and services that make advanced research capabilities possible:\n",
        "\n",
        "**[Tavily Search API](https://tavily.com/)**: Our cookbook leverages the Tavily API for efficient, high-quality web search functionality. Tavily provides comprehensive search results with raw content access, enabling our system to perform detailed content analysis. We are grateful to the Tavily team for providing this essential capability that forms the foundation of our information gathering process.\n",
        "\n",
        "**[Together AI](https://together.ai/)**: The LLM-powered components of our system are built using Together AI's API, which provides access to state-of-the-art language models optimized for different aspects of the research pipeline. Together AI's infrastructure enables the sophisticated analysis, planning, and synthesis capabilities that drive our research system.\n",
        "\n",
        "**Open Source Community**: This work builds upon numerous open-source libraries, tools, and research that have advanced the fields of natural language processing, information retrieval, and AI-assisted research. We acknowledge the broader community whose ongoing contributions make projects like this possible.\n",
        "\n",
        "If you use this cookbook in your own work or research, please consider acknowledging both Tavily and Together AI for their enabling technologies.\n",
        "\n",
        "\n",
        "## Related\n",
        "\n",
        "Open source developers across the tech landscape have launched several meaningful attempts to recreate the capabilities of OpenAI's Deep Researcher. Some notable ones are as follows:\n",
        "\n",
        "* [Huggingface](https://huggingface.co/blog/open-deep-research)\n",
        "* [Open Deep Research](https://github.com/nickscamara/open-deep-research)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": ".venv",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.7"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}