Advisors API

The NestJS AI Advisors API provides a flexible and powerful way to intercept, modify, and enhance AI-driven interactions in your NestJS applications. By leveraging the Advisors API, developers can create more sophisticated, reusable, and maintainable AI components.

The key benefits include encapsulating recurring Generative AI patterns, transforming data sent to and from Large Language Models (LLMs), and providing portability across various models and use cases.

You can configure existing advisors using the ChatClient API as shown in the following example:

import { ChatClient } from '@nestjs-ai/client-chat';
import { MessageChatMemoryAdvisor } from '@nestjs-ai/client-chat';
import { QuestionAnswerAdvisor } from '@nestjs-ai/advisors-vector-store';
import { ChatMemory } from '@nestjs-ai/model';

const chatMemory = ...; // Initialize your chat memory store
const vectorStore = ...; // Initialize your vector store

const chatClient = ChatClient.builder(chatModel)
  .defaultAdvisors(
    new MessageChatMemoryAdvisor({ chatMemory }), // chat-memory advisor
    QuestionAnswerAdvisor.builder(vectorStore).build(), // RAG advisor
  )
  .build();

const conversationId = '678';

const response = await chatClient
  .prompt()
  // Set advisor parameters at runtime
  .advisors((advisor) => advisor.param(ChatMemory.CONVERSATION_ID, conversationId))
  .user(userText)
  .call()
  .content();

It is recommend to register the advisors at build time using builder’s defaultAdvisors() method.

Advisors also participate in the Observability stack, so you can view metrics and traces related to their execution.

Core Components

The API consists of CallAdvisor and CallAdvisorChain for non-streaming scenarios, and StreamAdvisor and StreamAdvisorChain for streaming scenarios. It also includes ChatClientRequest to represent the unsealed Prompt request, ChatClientResponse for the Chat Completion response. Both hold an advise-context to share state across the advisor chain.

Advisors API Classes

The adviseCall() and the adviseStream() are the key advisor methods, typically performing actions such as examining the unsealed Prompt data, customizing and augmenting the Prompt data, invoking the next entity in the advisor chain, optionally blocking the request, examining the chat completion response, and throwing exceptions to indicate processing errors.

In addition the order getter determines advisor order in the chain, while the name getter provides a unique advisor name.

The Advisor Chain, created by the NestJS AI framework, allows sequential invocation of multiple advisors ordered by their order values. The lower values are executed first. The last advisor, added automatically, sends the request to the LLM.

Following flow diagram illustrates the interaction between the advisor chain and the Chat Model:

Advisors API Flow
  1. The NestJS AI framework creates an ChatClientRequest from user’s Prompt along with an empty advisor context object.

  2. Each advisor in the chain processes the request, potentially modifying it. Alternatively, it can choose to block the request by not making the call to invoke the next entity. In the latter case, the advisor is responsible for filling out the response.

  3. The final advisor, provided by the framework, sends the request to the Chat Model.

  4. The Chat Model’s response is then passed back through the advisor chain and converted into ChatClientResponse. Later includes the shared advisor context instance.

  5. Each advisor can process or modify the response.

  6. The final ChatClientResponse is returned to the client by extracting the ChatCompletion.

Advisor Order

The execution order of advisors in the chain is determined by the order getter. Key points to understand:

  • Advisors with lower order values are executed first.

  • The advisor chain operates as a stack:

    • The first advisor in the chain is the first to process the request.

    • It is also the last to process the response.

  • To control execution order:

    • Set the order close to HIGHEST_PRECEDENCE to ensure an advisor is executed first in the chain (first for request processing, last for response processing).

    • Set the order close to LOWEST_PRECEDENCE to ensure an advisor is executed last in the chain (last for request processing, first for response processing).

  • Higher values are interpreted as lower priority.

  • If multiple advisors have the same order value, their execution order is not guaranteed.

The seeming contradiction between order and execution sequence is due to the stack-like nature of the advisor chain:

  • An advisor with the highest precedence (lowest order value) is added to the top of the stack.

  • It will be the first to process the request as the stack unwinds.

  • It will be the last to process the response as the stack rewinds.

As a reminder, here are the semantics of the Ordered interface from @nestjs-port/core:

export interface Ordered {
  /**
   * Get the order value of this object.
   *
   * Higher values are interpreted as lower priority. As a consequence, the object
   * with the lowest value has the highest priority.
   */
  get order(): number;
}

/**
 * Useful constant for the highest precedence value.
 * Equivalent to Java `Integer.MIN_VALUE`.
 */
export declare const HIGHEST_PRECEDENCE = -2147483648;

/**
 * Useful constant for the lowest precedence value.
 * Equivalent to Java `Integer.MAX_VALUE`.
 */
export declare const LOWEST_PRECEDENCE = 2147483647;

For use cases that need to be first in the chain on both the input and output sides:

  1. Use separate advisors for each side.

  2. Configure them with different order values.

  3. Use the advisor context to share state between them.

API Overview

The main Advisor interfaces are exported from the @nestjs-ai/client-chat package. Here are the key interfaces you’ll encounter when creating your own advisor:

import type { Ordered } from '@nestjs-port/core';

export interface Advisor extends Ordered {
  /**
   * Return the name of the advisor.
   */
  get name(): string;
}

The two sub-interfaces for synchronous and reactive Advisors are

export interface CallAdvisor extends Advisor {
  adviseCall(
    chatClientRequest: ChatClientRequest,
    callAdvisorChain: CallAdvisorChain,
  ): Promise<ChatClientResponse>;
}

and

import type { Observable } from 'rxjs';

export interface StreamAdvisor extends Advisor {
  adviseStream(
    chatClientRequest: ChatClientRequest,
    streamAdvisorChain: StreamAdvisorChain,
  ): Observable<ChatClientResponse>;
}

To continue the chain of Advice, use CallAdvisorChain and StreamAdvisorChain in your Advice implementation:

The interfaces are

export interface CallAdvisorChain extends AdvisorChain {
  /**
   * Invokes the next CallAdvisor in the CallAdvisorChain with the given request.
   */
  nextCall(chatClientRequest: ChatClientRequest): Promise<ChatClientResponse>;

  /**
   * Returns the list of all the CallAdvisor instances included in this chain at
   * the time of its creation.
   */
  get callAdvisors(): CallAdvisor[];

  copy(after: CallAdvisor): CallAdvisorChain;
}

and

import type { Observable } from 'rxjs';

export interface StreamAdvisorChain extends AdvisorChain {
  /**
   * Invokes the next StreamAdvisor in the StreamAdvisorChain with the given request.
   */
  nextStream(chatClientRequest: ChatClientRequest): Observable<ChatClientResponse>;

  /**
   * Returns the list of all the StreamAdvisor instances included in this chain
   * at the time of its creation.
   */
  get streamAdvisors(): StreamAdvisor[];

  copy(after: StreamAdvisor): StreamAdvisorChain;
}

Implementing an Advisor

To create an advisor, implement either CallAdvisor or StreamAdvisor (or both). The key method to implement is adviseCall() for non-streaming or adviseStream() for streaming advisors, calling nextCall() / nextStream() to continue the chain.

For advisors that simply augment the request before, and the response after, the call to the next entity, you can extend the BaseAdvisor abstract class and implement only the before() and after() methods.

Examples

We will provide few hands-on examples to illustrate how to implement advisors for observing and augmenting use-cases.

Logging Advisor

We can implement a simple logging advisor that logs the ChatClientRequest before and the ChatClientResponse after the call to the next advisor in the chain. Note that the advisor only observes the request and response and does not modify them. This implementation support both non-streaming and streaming scenarios.

import {
  ChatClientMessageAggregator,
  type CallAdvisor,
  type CallAdvisorChain,
  type ChatClientRequest,
  type ChatClientResponse,
  type StreamAdvisor,
  type StreamAdvisorChain,
} from '@nestjs-ai/client-chat';
import { LoggerFactory, type Logger } from '@nestjs-port/core';
import type { Observable } from 'rxjs';

export class SimpleLoggerAdvisor implements CallAdvisor, StreamAdvisor {
  private readonly logger: Logger = LoggerFactory.getLogger(
    SimpleLoggerAdvisor.name,
  );

  private readonly aggregator = new ChatClientMessageAggregator();

  get name(): string { (1)
    return this.constructor.name;
  }

  get order(): number { (2)
    return 0;
  }

  async adviseCall(
    chatClientRequest: ChatClientRequest,
    callAdvisorChain: CallAdvisorChain,
  ): Promise<ChatClientResponse> {
    this.logRequest(chatClientRequest);

    const chatClientResponse = await callAdvisorChain.nextCall(chatClientRequest);

    this.logResponse(chatClientResponse);

    return chatClientResponse;
  }

  adviseStream(
    chatClientRequest: ChatClientRequest,
    streamAdvisorChain: StreamAdvisorChain,
  ): Observable<ChatClientResponse> {
    this.logRequest(chatClientRequest);

    const chatClientResponses = streamAdvisorChain.nextStream(chatClientRequest);

    return this.aggregator.aggregateChatClientResponse( (3)
      chatClientResponses,
      (chatClientResponse) => this.logResponse(chatClientResponse),
    );
  }

  private logRequest(request: ChatClientRequest): void {
    this.logger.debug(`request: ${JSON.stringify(request)}`);
  }

  private logResponse(chatClientResponse: ChatClientResponse): void {
    this.logger.debug(`response: ${JSON.stringify(chatClientResponse)}`);
  }
}
1 Provides a unique name for the advisor.
2 You can control the order of execution by setting the order value. Lower values execute first.
3 The ChatClientMessageAggregator is a utility class that aggregates the streamed responses into a single ChatClientResponse. This can be useful for logging or other processing that observe the entire response rather than individual items in the stream. Note that you can not alter the response in the aggregator as it is a read-only operation.

A SimpleLoggerAdvisor is already provided as a built-in advisor by the @nestjs-ai/client-chat package, so you typically do not need to implement your own.

Re-Reading (Re2) Advisor

The "Re-Reading Improves Reasoning in Large Language Models" article introduces a technique called Re-Reading (Re2) that improves the reasoning capabilities of Large Language Models. The Re2 technique requires augmenting the input prompt like this:

{Input_Query}
Read the question again: {Input_Query}

Implementing an advisor that applies the Re2 technique to the user’s input query can be done like this:

import {
  BaseAdvisor,
  type AdvisorChain,
  type ChatClientRequest,
  type ChatClientResponse,
} from '@nestjs-ai/client-chat';
import { PromptTemplate } from '@nestjs-ai/model';

export class ReReadingAdvisor extends BaseAdvisor {
  private static readonly DEFAULT_RE2_ADVISE_TEMPLATE = `{re2_input_query}
Read the question again: {re2_input_query}`;

  private readonly re2AdviseTemplate: string;

  private _order = 0;

  constructor(
    re2AdviseTemplate: string = ReReadingAdvisor.DEFAULT_RE2_ADVISE_TEMPLATE,
  ) {
    super();
    this.re2AdviseTemplate = re2AdviseTemplate;
  }

  override async before( (1)
    chatClientRequest: ChatClientRequest,
    _advisorChain: AdvisorChain,
  ): Promise<ChatClientRequest> {
    const augmentedUserText = PromptTemplate.builder()
      .template(this.re2AdviseTemplate)
      .variables({
        re2_input_query: chatClientRequest.prompt.userMessage.text ?? '',
      })
      .build()
      .render();

    return chatClientRequest
      .mutate()
      .prompt(chatClientRequest.prompt.augmentUserMessage(augmentedUserText))
      .build();
  }

  override async after(
    chatClientResponse: ChatClientResponse,
    _advisorChain: AdvisorChain,
  ): Promise<ChatClientResponse> {
    return chatClientResponse;
  }

  override get order(): number { (2)
    return this._order;
  }

  withOrder(order: number): this {
    this._order = order;
    return this;
  }
}
1 The before method augments the user’s input query applying the Re-Reading technique.
2 You can control the order of execution by setting the order value. Lower values execute first.

NestJS AI Built-in Advisors

NestJS AI provides several built-in advisors to enhance your AI interactions. Here’s an overview of the available advisors:

Chat Memory Advisors

These advisors manage conversation history in a chat memory store:

  • MessageChatMemoryAdvisor (from @nestjs-ai/client-chat)

    Retrieves memory and adds it as a collection of messages to the prompt. This approach maintains the structure of the conversation history. Note, not all AI Models support this approach.

  • PromptChatMemoryAdvisor (from @nestjs-ai/client-chat)

    Retrieves memory and incorporates it into the prompt’s system text.

  • VectorStoreChatMemoryAdvisor (from @nestjs-ai/advisors-vector-store)

    Retrieves memory from a VectorStore and adds it into the prompt’s system text. This advisor is useful for efficiently searching and retrieving relevant information from large datasets.

Question Answering Advisor
  • QuestionAnswerAdvisor (from @nestjs-ai/advisors-vector-store)

    This advisor uses a vector store to provide question-answering capabilities, implementing the Naive RAG (Retrieval-Augmented Generation) pattern.

  • RetrievalAugmentationAdvisor (from @nestjs-ai/rag)

    Advisor that implements common Retrieval Augmented Generation (RAG) flows using the building blocks defined in the @nestjs-ai/rag package and following the Modular RAG Architecture.

Content Safety Advisor
  • SafeGuardAdvisor (from @nestjs-ai/client-chat)

    A simple advisor designed to prevent the model from generating harmful or inappropriate content.

The ReReadingAdvisor (RE2 reasoning advisor) is not yet available as a built-in advisor in NestJS AI. You can implement it yourself by extending BaseAdvisor as shown in the Re-Reading (Re2) Advisor example above.

Streaming vs Non-Streaming

Advisors Streaming vs Non-Streaming Flow
  • Non-streaming advisors work with complete requests and responses.

  • Streaming advisors handle requests and responses as continuous streams, using reactive programming concepts (e.g., RxJS Observable for responses).

import { asyncScheduler, map, mergeMap, observeOn, of, type Observable } from 'rxjs';

adviseStream(
  chatClientRequest: ChatClientRequest,
  chain: StreamAdvisorChain,
): Observable<ChatClientResponse> {
  return of(chatClientRequest).pipe(
    observeOn(asyncScheduler),
    map((request) => {
      // This runs on the scheduled execution context.
      // Advisor before next section
      return request;
    }),
    mergeMap((request) => chain.nextStream(request)),
    map((response) => {
      // Advisor after next section
      return response;
    }),
  );
}

Best Practices

  1. Keep advisors focused on specific tasks for better modularity.

  2. Use the advisor context to share state between advisors when necessary.

  3. Implement both streaming and non-streaming versions of your advisor for maximum flexibility.

  4. Carefully consider the order of advisors in your chain to ensure proper data flow.