Integrate Orquesta with Cohere using Python SDK
Post date :
Aug 28, 2023
Orquesta provides your product teams with no-code collaboration tooling to experiment, operate, and monitor LLMs and remote configurations within your SaaS. Using Orquesta, you can easily perform prompt engineering, prompt management, experiment in production, push new versions directly to production, and roll back instantly.
Cohere, on the other hand, is an API that offers language processing to any system. It trains massive language models and puts them behind a very simple API.
This article guides you through integrating your SaaS with Orquesta and Cohere using our Python SDK. By the end of the article, you'll know how to set up a prompt in Orquesta, perform prompt engineering, request a prompt variant using our SDK code generator, map the Orquesta response with Cohere, send a payload to Cohere, and report the response back to Orquesta for observability and monitoring.
For you to be able to follow along in this tutorial, you will need the following:
Jupyter Notebook (or any IDE of your choice)
Orquesta Python SDK
Follow these steps to integrate the Python SDK with Cohere.
Step 1 - Install SDK and create a client instance
To create a client instance, you need to have access to the Orquesta API key, which can be found in your workspace
Copy it and add the following code to your notebook to initialize the Orquesta client.
timemodule to calculate the total time for the program to run
We also import
cohere, to be able to use the API
OrquestaClientOptionsclasses that are already defined in the
orquesta_sdkmodule, are imported.
The Orquesta SDK has helper functions that map and interface between Orquesta and specific LLM providers. For this integration, we will make use of the
To log all the interactions with Cohere, we use the
We create the instance of the
OrquestaClientOptionsand configure it with the
ttl(Time to Live) in seconds for the local cache; by default, it is 3600 seconds (1 hour)
Finally, an instance of the
OrquestaClient class is created and initialized with the previously configured options object. This
client instance can now interact with the Orquesta service using the provided API key for authentication.
Step 2 - Enable Cohere models in Model Garden
Head over to Orquesta's Model Garden and enable the Cohere models you want to use.
Step 3 - Set up a completion prompt and variants
The next step is to set up your completion prompt; ensure it is completion and not chat to use Cohere.
To create a prompt, click on
Add Prompt, provide a
prompt key, a
Domain (optional) and select
Once that is set up, create your first completion, give it a name prompt, add all the necessary information, and click Save.
Step 4 - Request a variant from Orquesta using the SDK
Our flexible configuration matrix allows you to define multiple prompt variants based on custom context. This allows you to work with different prompts and hyperparameters with, for example, environment, country, locale or user segment. The Code Snippet Generator makes it easy to request a prompt variant.
Once you open the Code Snippet Generator, copy the code snippet and paste it into your editor.
Step 5 - Map the Orquesta response to Cohere using a Helper
We have already established at the beginning of this tutorial that for us to be able to integrate these two technologies, we will make use of a Helper provided by Orquesta, which is
We start the
timeusing the time module
An instance of the Cohere
generate()endpoint, we can generate realistic text conditioned on a given input
generate()endpoint also receives other body parameters, such as the
promptas a required string, the model, the num_generations, max_tokens, temperature, etc. For simplicity, we are only working with model and prompt
We end the
Step 6 - Report analytics back to Orquesta
After each query, Orquesta generates a log with a Trace ID. Using the
add_metrics() method, you can add additional information, such as the
With these easy steps, you have successfully integrated Orquesta with Cohere, and this is just the tip of the iceberg because, as of the time of writing this article, Orquesta only supports the
generate() endpoint, but in the future, you can use the other endpoints, such as