Skip to content

Deploying OrcaDB Instances#

REDESIGN IN PROGRESS

We are currently redesigning OrcaDB to make it easier to interact with your memorysets and allow directly hosting retrieval-augmented models with Orca. Please get in touch if you want to learn more about how to deploy your model to production with Orca.

This guide will show you how to deploy OrcaDB instances to AWS and GCP (1) via the Orca App and connect to them from your Python code using OrcaLib which will communicate with the OrcaDB instance via HTTP requests.

  1. If you need to deploy to a different cloud provider or on-premises, please get in touch to discuss your requirements. Orca is designed to be cloud-agnostic and can be deployed to any cloud provider or on-premises.

To manage OrcaDB instances, you need to have an account in the Orca App and be an owner of your organization. See Account & Organization Management Guide for more info.

Managing OrcaDB Instances#

OrcaDB instances can be deployed and managed via the Orca App on the Cloud Tab. The tab will show you a list of all your OrcaDB instances by default.

Cloud Tab Page Light Cloud Tab Page Dark

Deploy an OrcaDB Instance#

To deploy a new instance, navigate to the Cloud Tab of the Orca App. Click the deploy button in the top right which will bring you to the deploy screen.

Deploy Instance Page Light Deploy Instance Page Dark

Select the following options to configure your instance:

  • Cloud Provider: Instances can be hosted on GCP or AWS. Select the cloud provider you use at your company.
  • Cloud Region: We offer instance hosting in a few regions. Choose the region closest to where you have your model deployed to reduce latency.
  • OrcaDB Version: Select the latest version unless you have identified specific versioning restrictions.

  • Machine Type: We currently offer 5 machine types.

    • Shared GPU 1 vCPU 4GB RAM
    • Shared GPU 2 vCPU 8GB RAM
    • Shared GPU 4 vCPU 16GB RAM
    • Shared GPU 8 vCPU 32GB RAM
    • T4 Tesla GPU 4 vCPU 16GB RAM

    Shared GPU instances call a separate shared service for embedding generation to minimize cost. This adds some latency of course. They come in a few configuration options for the number of cores and RAM. We recommend using them only for low volume or low utilization scenarios.

    For production scenarios, we recommend deploying an instance with a dedicated Nvidia Tesla T4 GPU to maximize performance and availability of your system.

  • Storage Limit: Select an appropriate capacity based on your intended data volume.

Once you have selected the configuration for your instance, click the deploy button. You will be redirected to the instance screen. It takes a few minutes for the instance to become available.

Check Status and Get Credentials#

Navigate to the Cloud Tab which will show a list of all your OrcaDB instances with a colored dot indicating their status. To view more info about an instance, click the arrow icon to the far right. On the Instance Details Page, you can see the status and configuration of the instance as well as a list of all databases on the instance. If you are an owner, you can get the API key and secret by clicking on the key icon next to the instance URL. This will open a dialog with the credentials and a button to regenerate credentials for that instance.

Copy Credentials Page Light Copy Credentials Page Dark

There are 4 possible statuses for each instance:

Deploying: Instance is being deployed and not yet available.

Available: Instance has been deployed successfully and is available for use.

Unavailable: Instance is unavailable due to an unsuccessful deployment or other issue.

Rebooting: Instance is in the middle of being rebooted and is unavailable for use at this time.

Connect to Your OrcaDB#

To connect to your OrcaDB from OrcaLib, you just need to store the credentials in environment variables. You can use a tool like dotenv to load them into your environment from a .env file that should contain the following values:

.env
ORCADB_URL=<your-endpoint>
ORCADB_API_KEY=<your-api-key>
ORCADB_SECRET_KEY=<your-secret-key>

Then simply make sure to call dotenv.load_dotenv in your script before using any OrcaLib functionality. Orca memoryset and database objects will automatically connect to the database specified by the environment variables when instantiated.

Delete an OrcaDB Instance#

Navigate to the Cloud Tab which will show a list of all your OrcaDB instances. Then, click the trash icon to the right of the instance you wish to delete and confirm in the dialog.

DELETION IS IRREVERSIBLE

Delete Instance Light Delete Instance Dark