The Upstreet Agents SDK is now in public beta 🎉 Get started →
bg-pattern
📚 API ReferenceContexts

<VideoPerception>

Allow your agent to perceive images and video.

Import

import { VideoPerception } from 'react-agents';

Usage

The VideoPerception component enables your agent to perceive and analyze images and videos in the conversation.

  1. Add the component to your agent:
    import { VideoPerception } from 'react-agents';
     
    return (
      <Agent>
        {/* ... */}
        <VideoPerception
          hint="Optional guidance for the agent's video perception behavior"
        />
        {/* ... */}
      </Agent>
    );

The passed-in hint will be used during inference to determine what aspects of the video the agent should focus on.

The supported media types are:

  • image/jpeg
  • image/png
  • image/webp

On this page

Facing an issue? Add a ticket.