{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "HJsHqaTksDHt" }, "source": "# Tutorial: Building a Chat Agent with Function Calling\n\n- **Level**: Advanced\n- **Time to complete**: 20 minutes\n- **Components Used**: [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder), [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [Agent](https://docs.haystack.deepset.ai/docs/agent)\n- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and be familiar with [creating pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines)\n- **Goal**: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI's function calling feature." }, { "cell_type": "markdown", "metadata": { "id": "PWXXqq_MPn7y" }, "source": [ "## Overview\n", "\n", "\n", "\n", "📚 Useful Sources:\n", "* [OpenAIChatGenerator Docs](https://docs.haystack.deepset.ai/docs/openaichatgenerator)\n", "* [OpenAIChatGenerator API Reference](https://docs.haystack.deepset.ai/reference/generators-api#openaichatgenerator)\n", "* [Agent Docs](https://docs.haystack.deepset.ai/docs/agent)\n", "* [🧑‍🍳 Cookbook: Function Calling with OpenAIChatGenerator](https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/function_calling_with_OpenAIChatGenerator.ipynb)\n", "\n", "[OpenAI's function calling](https://platform.openai.com/docs/guides/function-calling) connects large language models to external tools. By providing a `tools` list with functions and their specifications to the OpenAI API calls, you can easily build chat assistants that can answer questions by calling external APIs or extract structured information from text.\n", "\n", "In this tutorial, you'll learn how to convert a Haystack pipeline into a function-calling tool and how to implement chat applications that show agent-like behavior. You'll first see the raw tool-calling mechanics through `OpenAIChatGenerator`, and then use Haystack's [`Agent`](https://docs.haystack.deepset.ai/docs/agent) component, which automatically runs the tool-calling loop for you." ] }, { "cell_type": "markdown", "metadata": { "id": "K04cnh_IleMV" }, "source": "## Setting up the Development Environment\n\nInstall Haystack and [sentence-transformers-haystack](https://pypi.org/project/sentence-transformers-haystack/) using pip:" }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "zNyqNVFaPN1A", "scrolled": true }, "outputs": [], "source": "%%bash\n\npip install haystack-ai\npip install sentence-transformers-haystack" }, { "cell_type": "markdown", "metadata": { "id": "2owelT4NpXtw" }, "source": [ "Save your OpenAI API key as an environment variable:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "WM-sVkYonutA", "outputId": "7895e2b4-97a3-4cfe-e2cc-80bceac53b3e" }, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ "Enter OpenAI API key: ········\n" ] } ], "source": [ "import os\n", "from getpass import getpass\n", "\n", "if \"OPENAI_API_KEY\" not in os.environ:\n", " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter OpenAI API key:\")" ] }, { "cell_type": "markdown", "metadata": { "id": "0_mGdadLrcNr" }, "source": [ "## Learning about the OpenAIChatGenerator\n", "\n", "[OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator) is a component that supports the function calling feature of OpenAI through Chat Completion API. In contrary to `OpenAIGenerator`, the way to communicate with `OpenAIChatGenerator` is through [`ChatMessage`](https://docs.haystack.deepset.ai/docs/data-classes#chatmessage) list. Read more about the difference between them in [Generators vs Chat Generators](https://docs.haystack.deepset.ai/docs/generators-vs-chat-generators).\n", "\n", "\n", "To start working with the `OpenAIChatGenerator`, create a `ChatMessage` object with \"SYSTEM\" role using `ChatMessage.from_system()` and another `ChatMessage` with \"USER\" role using `ChatMessage.from_user()`. Then, pass this messages list to `OpenAIChatGenerator` and run:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "yzuTzWpJFVbH", "outputId": "9ea4c85b-b533-4bc1-dff3-304d30a24021" }, "outputs": [ { "data": { "text/plain": [ "{'replies': [ChatMessage(_role=, _content=[TextContent(text='Natural Language Processing (NLP) ist ein Teilbereich der Künstlichen Intelligenz, der sich mit der Interaktion zwischen Computern und Menschen durch natürliche Sprache beschäftigt. Ziel ist es, Computern zu ermöglichen, menschliche Sprache zu verstehen, zu interpretieren und darauf zu reagieren.')], _name=None, _meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 58, 'prompt_tokens': 33, 'total_tokens': 91, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}})]}" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from haystack.dataclasses import ChatMessage\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "\n", "messages = [\n", " ChatMessage.from_system(\"Always respond in German even if some input data is in other languages.\"),\n", " ChatMessage.from_user(\"What's Natural Language Processing? Be brief.\"),\n", "]\n", "\n", "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\")\n", "chat_generator.run(messages=messages)" ] }, { "cell_type": "markdown", "metadata": { "id": "cbl0xs0MJ76Z" }, "source": [ "### Basic Streaming\n", "\n", "`OpenAIChatGenerator` supports streaming, provide a `streaming_callback` function and run the `chat_generator` again to see the difference." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "D0nEEd5PJ1X2", "outputId": "74613af0-282b-497a-c76a-4bee69b41d97" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[ASSISTANT]\n", "Natural Language Processing (NLP) ist ein Teilbereich der Künstlichen Intelligenz, der sich mit der Interaktion zwischen Computern und Menschen in natürlicher Sprache beschäftigt. Ziel ist es, Maschinen das Verstehen, Interpretieren und Generieren von menschlicher Sprache zu ermöglichen.\n", "\n" ] } ], "source": [ "from haystack.components.generators.chat import OpenAIChatGenerator\n", "from haystack.components.generators.utils import print_streaming_chunk\n", "\n", "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\", streaming_callback=print_streaming_chunk)\n", "response = chat_generator.run(messages=messages)" ] }, { "cell_type": "markdown", "metadata": { "id": "FW91cHchCw_W" }, "source": [ "## Creating a Function Calling Tool from a Haystack Pipeline\n", "\n", "To use the function calling of OpenAI, you need to introduce `tools` to your `OpenAIChatGenerator`.\n", "\n", "For this example, you'll use a Haystack RAG pipeline as one of your tools. Therefore, you need to index documents to a document store and then build a RAG pipeline on top of it." ] }, { "cell_type": "markdown", "metadata": { "id": "FWkDXKbeoNqZ" }, "source": "### Index Documents with a Pipeline\n\nCreate a pipeline to store the small example dataset in the [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore) with their embeddings. You will use [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder) to generate embeddings for your Documents and write them to the document store with the [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter).\n\nAfter adding these components to your pipeline, connect them and run the pipeline.\n\n> If you'd like to learn about preprocessing files before you index them to your document store, follow the [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline) tutorial." }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 171, "referenced_widgets": [ "81e44362f359418f96a7fb064124e04c", "b96969f943d640a699feaa3a4ab28dad", "cabee03b724449678cffb53c9e46f630", "f59509f2462a4ba387a5c2ff6a6f911a", "2ec4c8a5f88e485abd8ce946239b59c5", "c495e6700dc3452fa76538ad1d9c1ed8", "ec769506baec4b5ca99c05d049940a6d", "a40c2224a30a43718f86a08e16c59d7e", "3f29fa86eea2420587cf3039908cc2e6", "47e025333d33466f802122517f3a584d", "7c23fd14ca8d48c2b63e13c5b599472e" ] }, "id": "ZE0SEGY92GHJ", "outputId": "5d7d3003-971b-453f-fc47-de612993d1aa" }, "outputs": [], "source": "from haystack import Pipeline, Document\nfrom haystack.document_stores.in_memory import InMemoryDocumentStore\nfrom haystack.components.writers import DocumentWriter\nfrom haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersDocumentEmbedder\n\ndocuments = [\n Document(content=\"My name is Jean and I live in Paris.\"),\n Document(content=\"My name is Mark and I live in Berlin.\"),\n Document(content=\"My name is Giorgio and I live in Rome.\"),\n Document(content=\"My name is Marta and I live in Madrid.\"),\n Document(content=\"My name is Harry and I live in London.\"),\n]\n\ndocument_store = InMemoryDocumentStore()\n\nindexing_pipeline = Pipeline()\nindexing_pipeline.add_component(\n instance=SentenceTransformersDocumentEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\"), name=\"doc_embedder\"\n)\nindexing_pipeline.add_component(instance=DocumentWriter(document_store=document_store), name=\"doc_writer\")\n\nindexing_pipeline.connect(\"doc_embedder.documents\", \"doc_writer.documents\")\n\nindexing_pipeline.run({\"doc_embedder\": {\"documents\": documents}})" }, { "cell_type": "markdown", "metadata": { "id": "dwC6fWQy-7pI" }, "source": "### Build a RAG Pipeline\n\nBuild a basic retrieval augmented generative pipeline with [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder) and [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator).\n\n> For a step-by-step guide to create a RAG pipeline with Haystack, follow the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial." }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "23JuUuql7PZ8", "outputId": "1ab36b2a-5f95-4580-ab3b-e60c8807795a" }, "outputs": [], "source": "from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder\nfrom haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\nfrom haystack.components.builders import ChatPromptBuilder\nfrom haystack.dataclasses import ChatMessage\nfrom haystack.components.generators.chat import OpenAIChatGenerator\n\ntemplate = [\n ChatMessage.from_system(\n \"\"\"\nAnswer the questions based on the given context.\n\nContext:\n{% for document in documents %}\n {{ document.content }}\n{% endfor %}\nQuestion: {{ question }}\nAnswer:\n\"\"\"\n )\n]\nrag_pipe = Pipeline()\nrag_pipe.add_component(\"embedder\", SentenceTransformersTextEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\"))\nrag_pipe.add_component(\"retriever\", InMemoryEmbeddingRetriever(document_store=document_store))\nrag_pipe.add_component(\"prompt_builder\", ChatPromptBuilder(template=template))\nrag_pipe.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n\nrag_pipe.connect(\"embedder.embedding\", \"retriever.query_embedding\")\nrag_pipe.connect(\"retriever\", \"prompt_builder.documents\")\nrag_pipe.connect(\"prompt_builder.prompt\", \"llm.messages\")" }, { "cell_type": "markdown", "metadata": { "id": "3vjtLi5A-XAi" }, "source": [ "### Run the Pipeline\n", "Test this pipeline with a query and see if it works as expected before you start using it as a function calling tool." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 101, "referenced_widgets": [ "d2e3e46dec734d848382ccde2f753a2c", "31585686320941e79444270287fa3bbf", "5944ad9184ab49aa999b62d152f81fc0", "3c191fd5ff34400fa703b24fc09d57e3", "480a7a302d1e45bcb2d54573f3568ad6", "1bdb07730cb146b79d0c0bfd1c9284dd", "3be0dedaea4d4a86adf089699d0814a8", "bd6e62494b8745399905a713169f5957", "1f450b7f47124f8196b3b7714ec77c2a", "825074d7daf54c8f92a0c4637246f904", "f20aca19ac55499089519e86c8c63d49" ] }, "id": "h4bXXC2Z7eaj", "outputId": "805cd637-e7f6-4099-baa0-93257b7e8aa0" }, "outputs": [], "source": [ "query = \"Where does Mark live?\"\n", "rag_pipe.run({\"embedder\": {\"text\": query}, \"prompt_builder\": {\"question\": query}})" ] }, { "cell_type": "markdown", "metadata": { "id": "xeQf1QT0_OA0" }, "source": [ "### Convert the Haystack Pipeline into a Tool\n", "\n", "Wrap the `rag_pipe.run` call inside a function called `rag_pipeline_func`. This function should take a `query` as input and return the response generated by the LLM in the RAG pipeline you previously built. Next, initialize a `Tool` by following the steps in [Tool Initialization](https://docs.haystack.deepset.ai/docs/tool#tool-initialization); define `parameters`, set a `name` and `description` for the tool. \n", "\n", "The `Tool` abstraction in Haystack enables seamless function calling/tool calling with multiple ChatGenerators. As of Haystack 2.9, this includes support for [AnthropicChatGenerator](https://docs.haystack.deepset.ai/docs/anthropicchatgenerator), [OllamaChatGenerator](https://docs.haystack.deepset.ai/docs/ollamachatgenerator), and [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), with more integrations expected in the future." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "id": "hCUFdKV5_Z7k" }, "outputs": [], "source": [ "from haystack.tools import Tool\n", "\n", "\n", "def rag_pipeline_func(query: str):\n", " result = rag_pipe.run({\"embedder\": {\"text\": query}, \"prompt_builder\": {\"question\": query}})\n", " return {\"reply\": result[\"llm\"][\"replies\"][0].text}\n", "\n", "\n", "parameters = {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"query\": {\n", " \"type\": \"string\",\n", " \"description\": \"The query to use in the search. Infer this from the user's message. It should be a question or a statement\",\n", " }\n", " },\n", " \"required\": [\"query\"],\n", "}\n", "\n", "rag_pipeline_tool = Tool(\n", " name=\"rag_pipeline_tool\",\n", " description=\"Get information about where people live\",\n", " parameters=parameters,\n", " function=rag_pipeline_func,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "zJt-mzb4oHxj" }, "source": [ "## Creating a Tool from Function\n", "\n", "In addition to the `rag_pipeline_tool`, create a new tool called `get_weather_tool` to be used to get weather information of cities.\n", "\n", "First, create a function that simulates an API call to an external weather service. Instead of passing parameters as JSON (like in the previous tool), use [`create_tool_from_function`](https://docs.haystack.deepset.ai/docs/tool#create_tool_from_function). This function requires additional details using the `Annotated` type to describe tool parameters. However, based on this information, `create_tool_from_function` can automatically infer the parameters and generate a JSON schema, so you don't need to define `parameters` separately." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "id": "XaQgnmPoJD13" }, "outputs": [], "source": [ "from typing import Annotated, Literal\n", "from haystack.tools import create_tool_from_function\n", "\n", "WEATHER_INFO = {\n", " \"Berlin\": {\"weather\": \"mostly sunny\", \"temperature\": 7, \"unit\": \"celsius\"},\n", " \"Paris\": {\"weather\": \"mostly cloudy\", \"temperature\": 8, \"unit\": \"celsius\"},\n", " \"Rome\": {\"weather\": \"sunny\", \"temperature\": 14, \"unit\": \"celsius\"},\n", " \"Madrid\": {\"weather\": \"sunny\", \"temperature\": 10, \"unit\": \"celsius\"},\n", " \"London\": {\"weather\": \"cloudy\", \"temperature\": 9, \"unit\": \"celsius\"},\n", "}\n", "\n", "\n", "def get_weather(\n", " city: Annotated[str, \"the city for which to get the weather\"] = \"Berlin\",\n", " unit: Annotated[Literal[\"Celsius\", \"Fahrenheit\"], \"the unit for the temperature\"] = \"Celsius\",\n", "):\n", " \"\"\"A simple function to get the current weather for a location.\"\"\"\n", " if city in WEATHER_INFO:\n", " return WEATHER_INFO[city]\n", " else:\n", " return {\"weather\": \"sunny\", \"temperature\": 21.8, \"unit\": \"fahrenheit\"}\n", "\n", "\n", "weather_tool = create_tool_from_function(get_weather)" ] }, { "cell_type": "markdown", "metadata": { "id": "bkRPp3JKpZgf" }, "source": [ "## Running OpenAIChatGenerator with Tools\n", "\n", "To use the tool calling feature, you need to pass the list of tools to `OpenAIChatGenerator` as `tools`. \n", "\n", "Instruct the model to use provided tools with a system message and then provide a query that requires a tool call as a user message:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "OEScMyqctzFN" }, "outputs": [], "source": [ "from haystack.dataclasses import ChatMessage\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "from haystack.components.generators.utils import print_streaming_chunk\n", "\n", "user_messages = [\n", " ChatMessage.from_system(\n", " \"Use the tool that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\"\n", " ),\n", " ChatMessage.from_user(\"Can you tell me where Mark lives?\"),\n", "]\n", "\n", "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\", streaming_callback=print_streaming_chunk)\n", "response = chat_generator.run(messages=user_messages, tools=[rag_pipeline_tool, weather_tool])" ] }, { "cell_type": "markdown", "metadata": { "id": "zYwb-SqaGL3O" }, "source": [ "As a response, you'll get a `ChatMessage` with information about the tool name and arguments as a [ToolCall](https://docs.haystack.deepset.ai/reference/data-classes-api#toolcall) object:" ] }, { "cell_type": "markdown", "metadata": { "id": "EHAs6kU5OPWe" }, "source": [ "```python\n", "{'replies': [\n", " ChatMessage(\n", " _role=,\n", " _content=[TextContent(text=''), ToolCall(tool_name='rag_pipeline_tool', arguments={'query': 'Where does Mark live?'}, id='call_xHEPMFrkHEKsi7tFB8FItkFC')],\n", " _name=None,\n", " meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'tool_calls', 'usage': {}}\n", " )\n", " ]\n", "}\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "_ulFfHcnGfsw" }, "source": [ "## Letting the Agent Run the Tool-Calling Loop\n", "\n", "As you saw above, the OpenAI Chat Completions API doesn't call the tool itself. It only tells you *which* tool to call and with *which* arguments. To get a final answer, you'd have to execute the tool yourself, append the result to the message list, and call the generator again, possibly several times, until the model stops requesting tools.\n", "\n", "Instead of writing this loop by hand, use Haystack's [`Agent`](https://docs.haystack.deepset.ai/docs/agent) component. The `Agent` wraps a `ChatGenerator` together with a list of `tools` and automatically runs the full tool-calling loop for you: it calls the LLM, invokes any requested tools, feeds the results back to the LLM, and repeats until the model returns a plain text answer.\n", "\n", "Create an `Agent` with the two tools you defined and run it on the same query:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 86, "referenced_widgets": [ "466461528ca1456481e860f6202df70c", "30d9aba7738f4d79a5c4d0593d9b6faa", "27424a72441e465ab03699702bf08e75", "e2046da9ab2a4f14b81ac130abf43393", "85ad1c88c416446fb1df56789c2c65a9", "c29df52ee29d4602878f502f14ec121d", "23b57a518b2a4724a807e7d87d84ed6f", "b29f84659637486c97bf97023d2d9c3a", "68457ab6ef80409abc692c2591e282c9", "f548b43c34f14120b1f5344a2b5216b7", "ce7ddd5617a44e6eb8fbaf510717d1a6" ] }, "id": "sijMc9fhR7ej", "outputId": "bd126d0f-3204-4089-a786-f0b233621149" }, "outputs": [], "source": [ "from haystack.components.agents import Agent\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "from haystack.dataclasses import ChatMessage\n", "\n", "agent = Agent(\n", " chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", " tools=[rag_pipeline_tool, weather_tool],\n", " system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n", ")\n", "\n", "result = agent.run(messages=[ChatMessage.from_user(\"Can you tell me where Mark lives?\")])\n", "print(result[\"messages\"][-1].text)" ] }, { "cell_type": "markdown", "metadata": { "id": "I6zam1cfbuZv" }, "source": [ "Besides the final answer, the `Agent` returns the full list of messages exchanged during the run under the `\"messages\"` key. This lets you see the tool call and the tool result that the `Agent` handled for you under the hood, exactly the steps you would have had to orchestrate manually:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "TdwkbMPLPO_L", "outputId": "0ee96a46-16f3-4a03-ce54-a69bedde24ae" }, "outputs": [], "source": [ "for message in result[\"messages\"]:\n", " print(message)\n", " print(\"-\" * 10)" ] }, { "cell_type": "markdown", "metadata": { "id": "5UkjGYEfwPzS" }, "source": [ "## Building the Chat Application\n", "\n", "Now let's turn the `Agent` into an interactive, multi-turn chat application. Because the `Agent` already handles the tool-calling loop internally, all you need to do to hold a conversation is keep track of the message history: append each new user message, run the `Agent`, and carry the returned `messages` over to the next turn so the model remembers the context.\n", "\n", "To build a nice UI for your application, you can use [Gradio](https://www.gradio.app/) that comes with a chat interface. Install `gradio`, run the code cell below and use the input box to interact with the chat application that has access to the two tools you've created above.\n", "\n", "Example queries you can try:\n", "* \"***What is the capital of Sweden?***\": A basic query without any tool calls\n", "* \"***Can you tell me where Giorgio lives?***\": A basic query with one tool call\n", "* \"***What's the weather like in Berlin?***\", \"***Is it sunny there?***\": To see the messages are being recorded and sent\n", "* \"***What's the weather like where Jean lives?***\": To force two tool calls\n", "* \"***What's the weather like today?***\": To force OpenAI to ask more clarification\n", "\n", "> Keep in mind that OpenAI models can sometimes hallucinate answers or tools and might not work as expected." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "mmOBVDvmOPWe", "scrolled": true }, "outputs": [], "source": [ "%%bash\n", "\n", "pip install -U gradio pytz" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sK_JeKZLhXcy" }, "outputs": [], "source": "import gradio as gr\n\nfrom haystack.components.agents import Agent\nfrom haystack.components.generators.chat import OpenAIChatGenerator\nfrom haystack.dataclasses import ChatMessage\n\nagent = Agent(\n chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n tools=[rag_pipeline_tool, weather_tool],\n system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n)\nagent.warm_up()\n\nmessages = [ChatMessage.from_system(agent.system_prompt)]\n\n\ndef chatbot_with_tc(message, history):\n global messages\n messages.append(ChatMessage.from_user(message))\n # The Agent runs the full tool-calling loop and returns all messages exchanged during the run\n result = agent.run(messages=messages)\n # Carry the updated conversation history over to the next turn\n messages = result[\"messages\"]\n return result[\"messages\"][-1].text\n\n\ndemo = gr.ChatInterface(\n fn=chatbot_with_tc,\n examples=[\n \"Can you tell me where Giorgio lives?\",\n \"What's the weather like in Madrid?\",\n \"Who lives in London?\",\n \"What's the weather like where Mark lives?\",\n ],\n title=\"Ask me about weather or where people live!\",\n)\n\n## Uncomment the line below to launch the chat app with UI\n# demo.launch()" }, { "cell_type": "markdown", "metadata": { "id": "rgGKjm7cpv1_" }, "source": [ "## What's next\n", "\n", "🎉 Congratulations! You've learned how to build chat applications that demonstrate agent-like behavior using OpenAI function calling, Haystack Pipelines, and the `Agent` component.\n", "\n", "If you liked this tutorial, there's more to learn about Haystack:\n", "- [Building a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)\n", "- [Create a Swarm of Agents](https://haystack.deepset.ai/cookbook/swarm)\n", "- [Evaluating RAG Pipelines](https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines)\n", "\n", "To stay up to date on the latest Haystack developments, you can [sign up for our newsletter](https://landing.deepset.ai/haystack-community-updates) or [join Haystack discord community](https://discord.gg/Dr63fr9NDS).\n", "\n", "Thanks for reading!" ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.11" } }, "nbformat": 4, "nbformat_minor": 4 }