How to use Cohere's Rerank in an LLM Chain

Cohere have just launched their Rerank endpoint designed to return the most relevant results based on an input of documents. They claim that customers have seen a 50% improvement over existing search systems by using it. When working with LLM chains, it's often required to reduce a large amount of context to be fed into the LLM prompt. To do this, content is split up and then reranked to feed in the top N results into the LLM.

Here's how you can add Cohere's Rerank to your Relevance AI Chain. Make sure first to create an account with Cohere and grab an API key.

Add Cohere Rerank as a step in your LLM chain

Head to the chain in Relevance AI that you'd like to add Rerank to or create a new one.

Add a new "API request" transformation to your chain
Set the URL to https://api.cohere.ai/v1/rerank
Set the method to POST
Set the headers to { "Content-Type": "application/json", "Authorization": "Bearer your-api-key" }
Set the body to { "query": "{{params.query}}", "documents": {{steps.split_text.output.chunks}}, "top_n": 3, "return_documents": true }Make sure to switch the variable to the relevant one for query and documents.

Once you run this step, you'll see in the `response_body` the results from Cohere. You can now feed in the text into your LLM or any other step in the chain.

Here's a video adding to a chain that takes a PDF, converts it to text, splits it by number of tokens and reranks it with Cohere.