LLM Tool step
Use a LLM as a step in your Tools
The LLM Tool Step is a versatile Tool step in Relevance AI that allows you to inject the power of large language models into your tools. It supports complex reasoning, classification, summarization, and text generation tasks — tailored to your use case.
Add the LLM Tool step to your Tool
You can add the LLM Tool step to your Tool by:
- Creating a new Tool, then clicking ‘LLM’ under Steps
- Click ‘Expand’ to see the full Tool step
- Add the Prompt you want to the Prompt field
- Choose the model you want to use
- Click the Settings icon if you want to access Advanced Settings
Advanced Settings
If you’re a pro at using LLMs, you can also access these Advanced Settings.
Fallback Model
You can select a Fallback Model if the model you’ve selected fails for any reason. This is a good setting to select to make sure your Tool works!
System prompt
A system prompt is composed of notes, instruction and guides normally guiding AI to assume a certain role, or to follow a specific format, or limitations.
An example of a System prompt would be: You are an expert on the solar system. Answer the following questions in a concise and informative manner.
Temperature
Temperature is a hyperparameter, ranging in (0,1), that affects the randomness (sometimes referred to as creativity) of the LLMs’ response.
Higher randomness/creativity/diversity is expected of higher temperature values.
However, responses might also lose the right context.
Conversation history
Click on + Add row and you can add lines of conversation taking place between a “user” and “ai”.
Conversation history is useful in conversational agents and help AI to know more about the situation.
Thinking / Reasoning Configuration
For OpenAI, Claude and Gemini models, you can select a thinking configuration. Enabling this will make the model ‘think’ more before answering, which is useful for complex problems.
Reasoning / thinking tokens are charged as output tokens and will be added to the cost of your LLM Tool step.
Force response format
For certain models such as OpenAI, response format can be forced using this field.