🦾 Building Autonomous Assistants with GPT-4o: A Step-by-Step Guide

Introduction to ChatGPT Update GPT-4o

OpenAI just released new ChatGPT Update: GPT-4o, their latest and greatest model, about a week ago. This new model is really impressive because it can handle audio, vision, and text all at once and do it in real-time. Imagine having a tool that can understand and respond to spoken language, recognize and describe images, and read and write text, all seamlessly.

In this article, I'm going to show you how to build autonomous assistants, also known as agents, from scratch using GPT-4o.

Get ready to be blown away by what this new AI model can do! GPT-4o just dropped, and we're going to put it to work building the handiest virtual assistants you've ever seen.

I. Get the OpenAI API Key

You can find your Secret API key on the API key page.

Check out our Best Practices for API Key Safety to learn how you can keep your API key protected.

Reminder: Do not share you API key with anyone!

II. About Phidata - A Framework for AI Assistant

Phidata is a framework for building Autonomous Assistants (aka Agents) that have long-term memory, contextual knowledge, and the ability to take actions using function calling.

Why Phidata?

1. Problem

• LLMs (Large Language Models) have a small memory and can't take actions on their own.

2. Solution

• Phidata gives LLMs memory, knowledge, and tools.

3. Key Features

1. Memory:

   • Stores chat history in a database, so LLMs can remember long conversations.

2. Knowledge:

   • Keeps information in a vector database, giving LLMs the context they need.

3. Tools:

   • Lets LLMs do tasks like fetching data from an API, sending emails, or querying a database.

4. Installation

pip install -U phidata
pip install -U duckdb
pip install -U duckduckgo-search

III. Build the Basic Assistant

1. Web Search Agent

Here’s the code I use:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.tools.duckduckgo import DuckDuckGo

assistant = Assistant(
    llm=OpenAIChat(model="gpt-4o", api_key="your_openai_key_here"),
    tools=[DuckDuckGo()],
    show_tool_calls=True,
    markdown=True,
)
assistant.print_response("Search for news from France and write a short poem about it.")

This’s what I got:

2. Finance Agent

You just need to apply this code:

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.tools.yfinance import YFinanceTools

assistant = Assistant(
    llm=OpenAIChat(model="gpt-4o", api_key="your_openai_key_here"),
    tools=[YFinanceTools(stock_price=True, analyst_recommendations=True, company_info=True, company_news=True)],
    show_tool_calls=True,
    markdown=True,
)
assistant.print_response("What is the stock price of NVDA")
assistant.print_response("Write a comparison between NVDA and AMD, use all tools available.")

Here’s what I got:

3. Hackernews Agent

This’s the code you can refer:

import json
import httpx

from phi.assistant import Assistant


def get_top_hackernews_stories(num_stories: int = 10) -> str:
    """Use this function to get top stories from Hacker News.

    Args:
        num_stories (int): Number of stories to return. Defaults to 10.

    Returns:
        str: JSON string of top stories.
    """

    # Fetch top story IDs
    response = httpx.get("https://hacker-news.firebaseio.com/v0/topstories.json")
    story_ids = response.json()

    # Fetch story details
    stories = []
    for story_id in story_ids[:num_stories]:
        story_response = httpx.get(f"https://hacker-news.firebaseio.com/v0/item/{story_id}.json")
        story = story_response.json()
        if "text" in story:
            story.pop("text", None)
        stories.append(story)
    return json.dumps(stories)


assistant = Assistant(tools=[get_top_hackernews_stories], show_tool_calls=True, markdown=True, debug_mode=True)
assistant.print_response("Summarize the top stories on hackernews?")

Here’s the result:

4. Data Analysis Agent

You just need to apply this code:

import json

from phi.llm.openai import OpenAIChat
from phi.assistant.duckdb import DuckDbAssistant

data_analyst = DuckDbAssistant(
    llm=OpenAIChat(model="gpt-4o", api_key="your_openai_key_here"),
    semantic_model=json.dumps(
        {
            "tables": [
                {
                    "name": "movies",
                    "description": "Contains information about movies from IMDB.",
                    "path": "https://phidata-public.s3.amazonaws.com/demo_data/IMDB-Movie-Data.csv",
                }
            ]
        }
    ),
)

data_analyst.print_response("What is the average rating of movies? Show me the SQL.", markdown=True)
data_analyst.print_response("Show me a histogram of ratings. Choose a bucket size", markdown=True)

Here’s the result:

5. Research Agent

You just need to apply this code:

import os
from pathlib import Path
from textwrap import dedent
from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
from phi.tools.exa import ExaTools

# Set the EXA_API_KEY environment variable
os.environ["EXA_API_KEY"] = "your_exa_api_key"

cwd = Path(__file__).parent.resolve()
scratch_dir = cwd.joinpath("scratch")
if not scratch_dir.exists():
    scratch_dir.mkdir(exist_ok=True, parents=True)

assistant = Assistant(
    llm=OpenAIChat(model="gpt-4o", api_key="your_openai_key_here"),
    tools=[ExaTools()],
    description="You are a senior NYT researcher writing an article on a topic.",
    instructions=[
        "For the provided topic, search for the top 10 links.",
        "Read the results carefully and prepare a NYT worthy article.",
    ],
    add_datetime_to_instructions=True,
    expected_output=dedent(
        """\
        An engaging, informative, and well-structured article in the following format:
        ## Engaging Article Title

        ### Overview
        {give a brief introduction of the article and why the user should read this report}
        {make this section engaging and create a hook for the reader}

        ### Section 1
        {break the article into sections}
        {provide details/facts/processes in this section}

        ... more sections as necessary...

        ### Takeaways
        {provide key takeaways from the article}

        ### References
        - [Title](url)
        - [Title](url)
        - [Title](url)
        """
    ),
    markdown=True,
    save_output_to_file=str(scratch_dir.joinpath("new_article.md")),
)

assistant.print_response("OpenAI GPT-4o")

Here’s what I got:

Conclusion

Wow, that was awesome! We just learned how to create some incredibly useful virtual assistants using GPT-4o - the brand new AI model that can understand audio, images, and text. By following the steps, we built assistants that can search the web, look up stock prices, analyze data, and even write full articles for us. It's mind-blowing what this technology can do. The best part is, with the code provided, it was really easy to make these assistants ourselves. I can't wait to keep exploring GPT-4o and all the cool virtual assistants we can build with it. This new AI is going to change everything!

If you are interested in other topics and how AI is transforming different aspects of our lives, or even in making money using AI with more detailed, step-by-step guidance, you can find our other articles here:

• Meta's Llama 3 AI: A Game Changer in the Tech Race

• Creating Realistic AI Avatar Videos For Free with This AI Tool: A Simple Guide

• Try GPT2 LLM: Is This OpenAI's Hidden GPT-2 Model?*

• Free GPT-4 Alternative: How to run Llama 3 on Your Local Computer*

• How to Easily Run Any Large Language Models on Your Computer*

_{*indicates a premium content, if any}