orcalib.client

PagedResponse

Bases: TypedDict

The response from a paged query

Attributes:

• page_index (int) –

  The index of the current page

• page_size (int) –

  The size of the current page

• total_size (int) –

  The total number of items

• num_pages (int) –

  The total number of pages

• has_next (bool) –

  If there is a next page

• items (list[Any]) –

  The items on the current page

ColumnSpec dataclass

ColumnSpec(name, notnull, unique, dtype)

Schema of a column in a table

Attributes:

• name (str) –

  The name of the column

• notnull (bool) –

  If the column is allowed to contain null values

• unique (bool) –

  If each value in the column must be unique

• dtype (str) –

  The data type of the column

OrcaClient

The OrcaClient class is used to make requests to the Orca web service

set_credentials staticmethod

set_credentials(*, api_key, secret_key, base_url=None)

Set the api and secret key for the session

Parameters:

• api_key (str) –

  The api key for the OrcaDB instance

• secret_key (str) –

  The secret key for the OrcaDB instance

• base_url (str | None, default: None ) –

  The base url of the OrcaDB instance

check_version_compatibility staticmethod

check_version_compatibility()

Check if the OrcaLib version is compatible with the OrcaDB instance version and log a warning if not.

Returns:

• bool | None –

  True if the versions match, False if they do not, None if the version check is skipped

get_server_version staticmethod

get_server_version()

Get the version of the OrcaDB server.

Returns:

• str | None –

  the version string

create_database staticmethod

create_database(db_name)

Create a new database on the instance

Parameters:

• db_name (str) –

  The name of the database

drop_database staticmethod

drop_database(db_name, ignore_db_not_found=False)

Drop a database from the instance

Parameters:

• db_name (str) –

  The name of the database

• ignore_db_not_found (bool, default: False ) –

  If True, ignore the error if the database is not found

Returns:

• bool –

  True if the database was dropped, False if it was not found

database_exists staticmethod

database_exists(db_name)

Check if the database exists

Parameters:

• db_name (str) –

  The name of the database

Returns:

• bool –

  True if the database exists, False if it does not

restore_backup staticmethod

restore_backup(target_db_name, backup_name, checksum=None)

Restore a database from a backup

Parameters:

• target_db_name (str) –

  The name of the target database

• backup_name (str) –

  The name of the backup

• checksum (str | None, default: None ) –

  The checksum of the backup file (optional)

Returns:

• None –

  Restore database response

create_backup staticmethod

create_backup(db_name)

Create a backup of the database

Parameters:

• db_name (str) –

  The name of the database

Returns:

• Response –

  Create backup response

download_backup staticmethod

download_backup(
    backup_file, download_path="./data", overwrite=False
)

Download a backup from the server and save it to a file

Parameters:

• backup_file (str) –

  The name of the backup file

• download_path (str, default: './data' ) –

  The path of the folder to save the backup file

• overwrite (bool, default: False ) –

  If True, overwrite the file if it already exists

upload_backup staticmethod

upload_backup(file_path)

Upload a backup to the server

Parameters:

• file_path (str) –

  The path to the backup file

delete_backup staticmethod

delete_backup(backup_file_name)

Delete a backup from the server

Parameters:

• backup_file_name (str) –

  The name of the backup file

list_databases staticmethod

list_databases()

List all the databases on the server

Returns:

• list[str] –

  List of database names

list_tables staticmethod

list_tables(db_name)

List all the tables in the database

Parameters:

• db_name (str) –

  The name of the database

Returns:

• list[str] –

  List of table names

table_info staticmethod

table_info(db_name, table_name)

Get the information about a table

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

Returns:

• list[ColumnSpec] –

  List with schema information for each column in the table

healthcheck staticmethod

healthcheck()

Perform a healthcheck on the server

Returns:

• dict[str, Any] –

  Dictionary with healthcheck information

create_table staticmethod

create_table(
    db_name,
    table_name,
    table_schema,
    if_table_exists=TableCreateMode.ERROR_IF_TABLE_EXISTS,
)

Create a new table in the database

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

• table_schema (list[ColumnSpec]) –

  The schema of the table

• if_table_exists (TableCreateMode, default: ERROR_IF_TABLE_EXISTS ) –

  What to do if the table already exists

insert staticmethod

insert(db_name, table_name, rows, files)

Insert rows into the table

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

• rows (list[RowDict]) –

  The rows to insert

• files (list[tuple[str, IO[bytes]]]) –

  The files to upload

update staticmethod

update(db_name, table_name, row, filter, files)

Update a row in the table

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

• row (RowDict) –

  The row to update

• filter (Any) –

  The filter to apply

• files (list[tuple[str, IO[bytes]]]) –

  The files to upload

Returns:

• None –

  Update response

upsert staticmethod

upsert(db_name, table_name, rows, key_columns, files)

Upsert rows into the table

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

• rows (list[RowDict]) –

  The rows to upsert

• key_columns (list[str]) –

  The key columns to use for the upsert

• files (list[tuple[str, IO[bytes]]]) –

  The files to upload

delete staticmethod

delete(db_name, table_name, filter)

Delete rows from the table

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

• filter (Any) –

  The filter to apply

count staticmethod

count(db_name, table_name)

Count the number of rows in a table

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

Returns:

• int –

  The number of rows in the table

add_column staticmethod

add_column(
    db_name, table_name, new_col, dtype, notnull, unique
)

Add a new column to the table

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

• new_col (list[str]) –

  The name of the new column

• dtype (list[str]) –

  The data type of the new column

• notnull (list[bool]) –

  If the new column is not null

• unique (list[bool]) –

  If the new column is unique

drop_column staticmethod

drop_column(db_name, table_name, col_names)

Drop a column from the table

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

• col_names (list[str]) –

  The name of the column to drop

drop_table staticmethod

drop_table(db_name, table_name, error_if_not_exists=True)

Drop a table from the database

Parameters:

• db_name (str) –

  The name of the database

• table_name (str) –

  The name of the table

• error_if_not_exists (bool, default: True ) –

  If True, raise an error if the table does not exist

drop_index staticmethod

drop_index(db_name, index_name)

Drop an index from the database

Parameters:

• db_name (str) –

  The name of the database

• index_name (str) –

  The name of the index

select staticmethod

select(
    table,
    columns=None,
    limit=None,
    filter=None,
    order_by_columns=None,
    default_order=Order.ASCENDING,
)

Perform a select query on the table

Parameters:

• table (TableHandle) –

  The TableHandle for the table we’re querying

• columns (list[ColumnName] | None, default: None ) –

  The columns to select. If None, all columns are selected

• limit (int | None, default: None ) –

  The maximum number of rows to return

• filter (ApiFilter | None, default: None ) –

  The filter to apply to the query

• order_by_columns (OrderByColumns | None, default: None ) –

  The columns to order by. If None, no order is applied.

• default_order (Order, default: ASCENDING ) –

  The default order to use if no order is specified. Defaults to ascending.

Returns:

• TableSelectResponse –

  The response from the select query

create_index staticmethod

create_index(
    db_name,
    index_name,
    table_name,
    column,
    index_type,
    ann_index_type="hnswlib",
    embedding_model=EmbeddingModel.SENTENCE_TRANSFORMER,
)

Create a new index

Parameters:

• db_name (str) –

  The name of the database

• index_name (str) –

  The name of the index

• table_name (str) –

  The name of the table

• column (str) –

  The name of the column to index

• index_type (str) –

  The type of the index

• ann_index_type (str, default: 'hnswlib' ) –

  The type of the approximate nearest neighbors index

• embedding_model (EmbeddingModel, default: SENTENCE_TRANSFORMER ) –

  The name of the embedding model

Returns:

• IndexHandle –

  handle for the created index

get_index_status staticmethod

get_index_status(db_name, index_name)

Get the status of an index

Parameters:

• db_name (str) –

  The name of the database

• index_name (str) –

  The name of the index

Returns:

• CatchupStatus –

  Get index status response

get_index staticmethod

get_index(db_name, index_name)

Get the details of an index

Parameters:

• db_name (str) –

  The name of the database

• index_name (str) –

  The name of the index

Returns:

• IndexHandle –

  The index handle

scan_index staticmethod

scan_index(
    db,
    index_name,
    query,
    limit,
    columns=None,
    filter=None,
    drop_exact_match=False,
    exact_match_threshold=EXACT_MATCH_THRESHOLD,
)

Scan an index

Parameters:

• db (OrcaDatabase) –

  The OrcaDatabase object

• index_name (str) –

  The name of the index

• query (Any) –

  The query to apply

• limit (int) –

  The maximum number of rows to return

• columns (list[str] | None, default: None ) –

  The columns to return (optional)

• filter (str | None, default: None ) –

  The filter to apply (optional)

• drop_exact_match (bool, default: False ) –

  Drops the exact match from the results, if it’s found

• exact_match_threshold (float, default: EXACT_MATCH_THRESHOLD ) –

  The minimum distance threshold for the exact match

Returns:

• Any –

  The response from the scan index query

vector_scan_index staticmethod

vector_scan_index(
    table,
    index_name,
    query,
    limit,
    columns=None,
    filter=None,
    drop_exact_match=False,
    exact_match_threshold=EXACT_MATCH_THRESHOLD,
    curate_run_ids=None,
    curate_layer_name=None,
)

Performs a vector scan index query

Parameters:

• table (TableHandle) –

  The TableHandle for the table we’re querying

• index_name (str) –

  The name of the index

• query (Any) –

  The query to apply

• limit (int) –

  The maximum number of rows to return

• columns (list[str] | None, default: None ) –

  The columns to return (optional)

• filter (str | None, default: None ) –

  The filter to apply (optional)

• drop_exact_match (bool, default: False ) –

  The flag to drop exact matches

• exact_match_threshold (float, default: EXACT_MATCH_THRESHOLD ) –

  The threshold for exact matches

• curate_run_ids (list[int] | None, default: None ) –

  The curate run ids to apply (optional)

• curate_layer_name (str | None, default: None ) –

  The curate layer name to apply (optional)

Returns:

• BatchedScanResult –

  The response from the scan index query

full_vector_memory_join staticmethod

full_vector_memory_join(
    *,
    db_name,
    index_name,
    memory_index_name,
    num_memories,
    query_columns,
    page_size=100,
    page_index=0,
    drop_exact_match=False,
    exact_match_threshold=EXACT_MATCH_THRESHOLD,
    shuffle_memories=False
)

Perform a full vector memory join

Parameters:

• db_name (str) –

  The name of the database

• index_name (str) –

  The name of the index

• memory_index_name (str) –

  The name of the memory index

• num_memories (int) –

  The number of memory indexes

• query_columns (list[str] | str) –

  The columns to query (or a single column)

• page_size (int, default: 100 ) –

  The size of the page to return

• page_index (int, default: 0 ) –

  The index of the page to return

Returns:

• PagedResponse –

  The response from the full vector memory join query

get_index_values staticmethod

get_index_values(db_name, index_name)

Get the values of an index

Parameters:

• db_name (str) –

  The name of the database

• index_name (str) –

  The name of the index

Returns:

• dict[int, list[float]] –

  The response from the get index values query

get_index_values_paginated staticmethod

get_index_values_paginated(
    db_name, index_name, page_index=0, page_size=100
)

Get the values of an index paginated

Parameters:

• db_name (str) –

  The name of the database

• index_name (str) –

  The name of the index

• page_index (int, default: 0 ) –

  The index of the page to return

• page_size (int, default: 100 ) –

  The size of the page to return

Returns:

• PagedResponse –

  A paged response with dict[int, list[float]]

get_index_table staticmethod

get_index_table(db_name, index_name)

Get the table of an index

Parameters:

• db_name (str) –

  The name of the database

• index_name (str) –

  The name of the index

Returns:

• str –

  The name of the table the index is associated with

run_sql staticmethod

run_sql(db_name, query, params=[])

Run a raw SQL select query

Parameters:

• db_name (str) –

  The name of the database

• query (str) –

  The SQL query

• params (list[None | int | float | bytes | str], default: [] ) –

  The parameters for the query

Returns:

• list[dict[str, Any]] –

  The response from the SQL select query

encode_text staticmethod

encode_text(
    strings, model=EmbeddingModel.SENTENCE_TRANSFORMER
)

Encode a list of strings using the Roberta model

Parameters:

• strings (list[str]) –

  The list of strings to encode

Returns:

• list[list[float]] –

  A list of encoded strings with shape (len(strings), embedding_size)

index_embed_text staticmethod

index_embed_text(
    db_name, index_name, strings, result_format="pt"
)

Encode text values using the embedding model of a specific index.

Parameters:

• db_name (str) –

  The name of the database that the index belongs to

• index_name (str) –

  The name of the index whose embedding will be used to encode the text

• strings (list[str]) –

  The list of strings to encode

• result_format (Literal['list', 'pt'], default: 'pt' ) –

  If "list", return the results as a list of lists. If "pt", return the results as a list of PyTorch tensors.

Returns:

• list[list[float]] | Tensor –
  • If result_format is "list", this is a list[list[float]]
• list[list[float]] | Tensor –
  • If result_format is "pt", this is a Tensor with shape (batch_size, mem_count, embedding_dim)

init_forward_pass staticmethod

init_forward_pass(
    db_name,
    model_id,
    batch_size,
    model_version=None,
    seq_id=None,
    tags=None,
    metadata=None,
)

Generate run ids for a batch of forward passes

Parameters:

• db_name (str) –

  The name of the database

• model_id (str) –

  The id of the model

• batch_size (int) –

  The batch size

• model_version (str | None, default: None ) –

  The version of the model (optional)

• seq_id (UUID | None, default: None ) –

  The sequence id for the forward pass (optional)

• tags (set[str] | None, default: None ) –

  The tags for the forward pass (optional)

• metadata (OrcaMetadataDict | None, default: None ) –

  The metadata for the forward pass (optional)

Returns:

• list[int] –

  The ids for the model runs

record_memory_weights staticmethod

record_memory_weights(
    db_name, layer_name, run_ids, memory_ids, memory_weights
)

Record the memory weights for a batch of forward passes

Parameters:

• db_name (str) –

  The name of the database

• run_ids (list[int]) –

  The ids of the model runs

• layer_name (str) –

  The name of the lookup layer that the memory weights are for

• memory_weights (list[float]) –

  The memory weights for the model runs

delete_model_runs staticmethod

delete_model_runs(db_name, model_id, filters)

Delete curate tracking data for model runs

Parameters:

• db_name (str) –

  The name of the database

• model_id (str) –

  The id of the model to delete runs for

• filters (list[ApiFilter]) –

  Filters to select runs to delete

record_model_feedback staticmethod

record_model_feedback(db_name, run_ids, values, name, kind)

Record model feedback

Parameters:

• db_name (str) –

  The name of the database

• run_ids (list[int]) –

  A list of the run ids

• values (list[float]) –

  A list of the feedback values - type should match the kind

• name (str) –

  The name of the feedback - name and run_id combo must be unique

• kind (str) –

  The kind of feedback as a string

record_model_input_output staticmethod

record_model_input_output(
    db_name, run_ids, inputs, outputs
)

Record input and output data for model runs

Parameters:

• db_name (str) –

  The name of the database

• run_ids (list[int]) –

  The ids of the model runs as returned by init_forward_pass

• inputs (list[Any]) –

  The inputs for the model runs

• outputs (list[Any]) –

  The outputs for the model runs