AI Agents: Build an Agent from Scratch using JavaScript

Build and understand AI agents with LLMs and web search

Introduction

Agentic systems allow software to perform complex tasks autonomously by making decisions and using external tools when necessary. In this article, we will build an agentic system from scratch in JavaScript, without relying on a framework. This will give us deep insights into how AI agents work under the hood.

We will implement a system that:
- Uses a large language model (LLM) via Replicate API
- Integrates a web search tool powered by SerpAPI (up to 100 Request are free)
- Dynamically registers and uses tools based on the query

At the end, you’ll have a fully functional agent that can search the web, process information, and generate insights. Let’s get started!

Project Setup

Prerequisites

Ensure you have Node.js installed, and install ts-node globally if you haven’t already:

npm install -g ts-node

Next, initialize a TypeScript project:

mkdir agentic-system && cd agentic-system
npm init -y
npm install typescript ts-node axios dotenv cheerio
npx tsc — init

Create a .env file and add your SerpAPI key:

SERPAPI_KEY=your-serpapi-key

Now, let’s create our TypeScript files.

The complete code for this tutorial is available here.

Understanding the AI Agent

How Does an AI Agent Work?

An AI agent is a system that can reason about user input, decide when to use tools and generate responses based on external information. In our case, we need to:

• Accept a user query
• Decide if we can respond directly or need external tools (like web search)
• Retrieve additional data if needed
• Generate a structured response

Step 1: Creating the AI Agent (Agent.ts)

This agent will manage tools and interact with the LLM. The `Agent` class maintains a list of tools and determines whether to call the LLM or use a tool.

import { LLMProvider } from "../llm/LLMProvider";
import { createSystemPrompt } from "../utils/createSystemPrompt";
import { Tool } from "./Tool";

export class Agent {
    tools: Record<string, Tool>;
    private llmProvider: LLMProvider;

    constructor(llmProvider: LLMProvider) {
        this.tools = {};
        this.llmProvider = llmProvider;
    }

    addTool(tool: Tool): void {
        this.tools[tool.name] = tool;
    }

    getAvailableTools(): string[] {
        return Object.values(this.tools).map(
            (tool) => `${tool.name}: ${tool.description}`
        );
    }

    async useTool(toolName: string, args: Record<string, any>): Promise<string> {
        const tool = this.tools[toolName];
        if (!tool) {
            throw new Error(
                `Tool '${toolName}' not found. Available: ${Object.keys(this.tools)}`
            );
        }
        const orderedKeys = Object.keys(tool.parameters);
        const argValues = orderedKeys.map(key => args.hasOwnProperty(key) ? args[key] : undefined);

        return await tool.call(...argValues);
    }


    createSystemPrompt(): string {
        const toolsArray = Object.values(this.tools);
        return JSON.stringify(createSystemPrompt(toolsArray), null, 2);
    }

    /**
     * Calls the LLM using the provided LLMProvider.
     *
     * It builds the input object (using a system prompt, user prompt, and other parameters)
     * and passes it to the LLMProvider.
     */
    async callLLM(userPrompt: string): Promise<any> {
        const systemPrompt = this.createSystemPrompt();
        const promptTemplate = "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
        const input = {
            prompt: userPrompt,
            system_prompt: systemPrompt,
            max_new_tokens: 10000,
            prompt_template: promptTemplate,
        };

        return await this.llmProvider.callLLM(input);
    }
}

Explanation:

1. The Agent class stores a set of available tools.
2. addTool() registers a new tool, allowing the agent to use it later.
3. useTool() checks if a tool is registered and executes it with the given parameters.
4. callLLM() queries the language model, providing instructions to use tools when needed.

Step 2: Creating a System Prompt (createSystemPrompt.ts)

import {Tool} from "../agent/Tool";
import {SystemPrompt} from "../interfaces/SystemPrompt";

/**
 * Generates the system prompt based on available tools.
 *
 * @param tools Array of registered tools.
 * @returns The system prompt object.
 */
export function createSystemPrompt(tools: Tool[]): SystemPrompt {
    return {
        role: "AI Assistant",
        capabilities: [
            "Use provided tools when needed to answer user queries",
            "Provide direct responses when tool usage is unnecessary",
            "Plan efficient tool usage sequences"
        ],
        instructions: [
            "Only use tools when necessary",
            "If the answer can be provided directly, do not use a tool",
            "Plan the steps needed if tool usage is required",
            "Only respond the JSON and nothing else"
        ],
        tools: tools.map(tool => ({
            name: tool.name,
            description: tool.description,
            parameters: tool.parameters,
        })),
        response_format: {
            type: "json",
            schema: {
                requires_tools: {
                    type: "boolean",
                    description: "whether tools are needed for this query"
                },
                direct_response: {
                    type: "string",
                    description: "response when no tools are needed",
                    optional: true
                },
                thought: {
                    type: "string",
                    description: "reasoning on how to solve the query (when tools are needed)",
                    optional: true
                },
                plan: {
                    type: "array",
                    items: { type: "string" },
                    description: "steps to solve the query (when tools are needed)",
                    optional: true
                },
                tool_calls: {
                    type: "array",
                    items: {
                        type: "object",
                        properties: {
                            tool: { type: "string", description: "name of the tool" },
                            args: { type: "object", description: "parameters for the tool" }
                        }
                    },
                    description: "tools to call in sequence (if required)",
                    optional: true
                }
            }
        }
    };
}

Explanation:

1. This function generates a structured system prompt based on available tools.
2. The capabilities and instructions define how the agent should behave.
3. The response_format ensures responses are always in JSON format.
4. Example tool usage scenarios help guide the LLM’s decision-making process.

Step 3: Implementing LLM Calls (ReplicateLLMProvider.ts)

This module interacts with an external LLM API, simulating a call to Replicate’s Llama 3 model.

import Replicate from "replicate";
import { LLMProvider } from "./LLMProvider";
import {writeToLog} from "../utils/writeToLog";
import {isValidJson} from "../utils/isValidJson";

export class ReplicateLLMProvider implements LLMProvider {
    private replicate: Replicate;
    private readonly modelName: `${string}/${string}` | `${string}/${string}:${string}`;

    constructor(modelName: `${string}/${string}` | `${string}/${string}:${string}`) {
        this.replicate = new Replicate();
        this.modelName = modelName;
    }

    async callLLM(input: any): Promise<any> {
        await writeToLog('llm_input.log', JSON.stringify(input))

        let fullResponse = "";
        for await (const event of this.replicate.stream(this.modelName, { input })) {
            fullResponse += event;
        }

        await writeToLog('llm_response.log', fullResponse)

        if (isValidJson(fullResponse)) {
            return JSON.parse(fullResponse);
        } else {
            return fullResponse;
        }
    }
}

Explanation:

1. ReplicateLLMProvider wraps Replicate’s API, allowing our agent to generate responses.
2. It stores a model name (`meta/meta-llama-3–8b-instruct`).
3. callLLM() sends a query to Replicate and returns a response.

Step 4: Web Search Tool Using SerpAPI (WebSearchTool.ts)

import { ToolDecorator } from "../decorators/ToolDecorator";
import axios from "axios";
import * as cheerio from "cheerio";
import 'dotenv/config';

// Ensure you have the API key set in your environment variables
const SERPAPI_KEY = process.env.SERPAPI_KEY!;
const SERPAPI_URL = "https://serpapi.com/search";

export class WebSearchTool {
    @ToolDecorator({
        docstring: `Performs a web search using Google and summarizes the top 5 results.

Parameters:
- query: The search query to look up information.`,
        parameters: {
            query: { type: "string", description: "Search query for the web search" }
        },
    })
    async searchWeb(query: string): Promise<string> {
        try {
            const searchResponse = await axios.get(SERPAPI_URL, {
                params: {
                    q: query,
                    api_key: SERPAPI_KEY,
                    num: 5, // Fetch top 5 results
                },
            });

            const results = searchResponse.data.organic_results;
            if (!results || results.length === 0) {
                return "No relevant results found.";
            }

            // Fetch summaries for all 5 search results
            let summaries: string[] = [];
            for (const result of results.slice(0, 5)) { // Limit to top 5 results
                if (!result.link) continue; // Skip if no link is available

                console.log(`Fetching content from: ${result.link}`);

                const summary = await this.summarizeWebPage(result.link);
                summaries.push(`**${result.title}**\n${summary}\n[Read more](${result.link})\n`);
            }

            // Join all summaries into a structured response
            return `### Web Search Summary:\n\n` + summaries.join("\n---\n");
        } catch (error) {
            console.error('Error:', error);
            return `Error fetching search results.`;
        }
    }

    /**
     * Fetches and summarizes content from a webpage.
     * Uses cheerio to extract readable text and limit its length.
     */
    private async summarizeWebPage(url: string): Promise<string> {
        try {
            const response = await axios.get(url, { timeout: 8000 }); // Fetch webpage content
            const html = response.data; // Get raw HTML as string
            const $ = cheerio.load(html); // Load HTML into Cheerio

            // Extract readable text from <p> elements
            let pageTextArray: string[] = [];
            $("p").each((index, element) => {
                const paragraphText = $(element).text().trim();
                if (paragraphText.length > 50) { // Ignore very short paragraphs (e.g., ads, navigation)
                    pageTextArray.push(paragraphText);
                }
            });

            // Join extracted text into a single summary
            let pageText = pageTextArray.join(" ");

            if (!pageText || pageText.length < 100) {
                return "No substantial content found.";
            }

            // Limit the output to the first 1000 characters
            return pageText.substring(0, 1000) + "...";
        } catch (error) {
            console.error(`Failed to fetch/summarize page: ${url}`, error);
            return "Error retrieving page summary.";
        }
    }
}

Explanation:

1. searchWeb() queries Google using SerpAPI and retrieves the top 5 results.
2. It calls summarizeWebPage(), which fetches and extracts meaningful text from each page using Cheerio.

Step 5: Bringing Everything Together (index.ts)

import { WebSearchTool } from "./providers/WebSearchTool";  // Import new tool
import { ToolsManager } from "./agent/ToolsManager";
import { Agent } from "./agent/Agent";
import { ReplicateLLMProvider } from "./llm/ReplicateLLMProvider";

(async () => {
    const searchProvider = new WebSearchTool();

    const searchTools = ToolsManager.registerToolsFrom(searchProvider);

    const llmProvider = new ReplicateLLMProvider("meta/meta-llama-3-8b-instruct");
    const agent = new Agent(llmProvider);

    // Register all tools with the agent
    [...searchTools].forEach(tool => agent.addTool(tool));

    const userPrompt = "Find the latest trends in AI for 2025 specifically related to Large Language Models and summarize them search the web. Just return Json no additional text.";
    const initialLLMResponse = await agent.callLLM(userPrompt);

    if (!initialLLMResponse.requires_tools) {
        console.log('no tools')
        console.log(initialLLMResponse.direct_response);
    } else {
        const toolResults: Record<string, string> = {};
        for (const toolCall of initialLLMResponse.tool_calls) {
            toolResults[toolCall.tool] = await agent.useTool(toolCall.tool, toolCall.args);
        }

        const finalPrompt = `
            User Query: ${userPrompt}
            
            Tool Outputs:
            ${JSON.stringify(toolResults, null, 2)}
            
            Instructions:
            1. Read and analyze the content summaries extracted from the top 5 websites in the "Tool Outputs" section.
            2. For each website, generate a **short summary (2-3 sentences)** that captures the most important insights from that source.
            3. Then, provide a **final summary** that combines the key takeaways from all 5 sources into a **concise, well-structured overview** of the topic.
            4. Use clear, informative language. Do not generate any additional text beyond the required summaries.
            No tool usage required.
            Respond in JSON no additional text.
            
            Format:
            ### Source Summaries:
            1. **[Title of Source 1]** - (Summary of Source 1)
            2. **[Title of Source 2]** - (Summary of Source 2)
            3. **[Title of Source 3]** - (Summary of Source 3)
            4. **[Title of Source 4]** - (Summary of Source 4)
            5. **[Title of Source 5]** - (Summary of Source 5)
            
            ### Final Summary:
            (Concise summary synthesizing key insights from all sources)
        `;

        const finalLLMResponse = await agent.callLLM(finalPrompt);
        console.log("Final AI Summary:");
        console.log(finalLLMResponse.direct_response);
    }
})();

Explanation:

Tool Registration: We instantiate WebSearchTool, register it with ToolsManager and add it to the Agent instance.

User Query Processing: The agent receives a prompt asking for the latest AI trends.

Decision Making:

• If the agent determines that no external tools are needed, it provides a direct response.
• Otherwise, it invokes searchWeb using SerpAPI to fetch relevant results.

Summarization: The LLM is instructed to:

• Analyze the top 5 search results
• Summarize each individually
• Create a final, structured summary

Final Output: The AI agent outputs a JSON-formatted summary containing structured insights.

Final Thoughts

By the end of this tutorial, you have built an AI agent from scratch that:

• Uses an LLM API
• Performs web searches
• Decides whether to use external tools
• Generates structured responses

This provides a deep understanding of how agentic systems work under the hood. You can now extend this by integrating more tools, adding memory or refining the decision-making process. 🚀