C API

This API provides a very low-level interface to FoundationDB. It is primarily intended for use in implementing higher level APIs, rather than for direct use. If you are new to FoundationDB, you are probably better served by reading one of the other APIs first.

Installation

FoundationDB’s C bindings are installed with the FoundationDB client binaries (see Installing FoundationDB client binaries).

On Linux,
fdb_c.h is installed into /usr/include/foundationdb/
libfdb_c.so is installed into /usr/lib/
On macOS,
fdb_c.h is installed into /usr/local/include/foundationdb/
libfdb_c.dylib is installed into /usr/local/lib/

Linking

The FoundationDB C bindings are provided as a shared object which may be linked against at build time, or dynamically loaded at runtime. Any program that uses this API must be able to find a platform-appropriate shared library at runtime. Generally, this condition is best met by installing the FoundationDB client binaries (see Installing FoundationDB client binaries) on any machine where the program will be run.

Linux

When linking against libfdb_c.so, you must also link against libm, libpthread and librt. These dependencies will be resolved by the dynamic linker when using this API via dlopen() or an FFI.

macOS

When linking against libfdb_c.dylib, no additional libraries are required.

API versioning

Prior to including fdb_c.h, you must define the FDB_API_VERSION macro. This, together with the fdb_select_api_version() function, allows programs written against an older version of the API to compile and run with newer versions of the C library. The current version of the FoundationDB C API is 710.

#define FDB_API_VERSION 710
#include <foundationdb/fdb_c.h>
fdb_error_t fdb_select_api_version(int version)

Must be called before any other API functions. version must be less than or equal to FDB_API_VERSION (and should almost always be equal).

Language bindings implemented in C which themselves expose API versioning will usually pass the version requested by the application, instead of always passing FDB_API_VERSION.

Passing a version less than FDB_API_VERSION will cause the API to behave as it did in the older version.

It is an error to call this function after it has returned successfully. It is not thread safe, and if called from more than one thread simultaneously its behavior is undefined.

Note

This is actually implemented as a macro. If you are accessing this API via dlopen() or an FFI, you will need to use fdb_select_api_version_impl().

Warning

When using the multi-version client API, setting an API version that is not supported by a particular client library will prevent that client from being used to connect to the cluster. In particular, you should not advance the API version of your application after upgrading your client until the cluster has also been upgraded.

fdb_error_t fdb_select_api_version_impl(int runtime_version, int header_version)

This is the actual entry point called by the fdb_select_api_version() macro. It should never be called directly from C, but if you are accessing this API via dlopen() or an FFI, you will need to use it. fdb_select_api_version(v) is equivalent to fdb_select_api_version_impl(v, FDB_API_VERSION).

It is an error to call this function after it has returned successfully. It is not thread safe, and if called from more than one thread simultaneously its behavior is undefined.

runtime_version

The version of run-time behavior the API is requested to provide. Must be less than or equal to header_version, and should almost always be equal.

Language bindings which themselves expose API versioning will usually pass the version requested by the application.

header_version

The version of the ABI (application binary interface) that the calling code expects to find in the shared library. If you are using an FFI, this must correspond to the version of the API you are using as a reference (currently 710). For example, the number of arguments that a function takes may be affected by this value, and an incorrect value is unlikely to yield success.

Warning

When using the multi-version client API, setting an API version that is not supported by a particular client library will prevent that client from being used to connect to the cluster. In particular, you should not advance the API version of your application after upgrading your client until the cluster has also been upgraded.

int fdb_get_max_api_version()

Returns FDB_API_VERSION, the current version of the FoundationDB C API. This is the maximum version that may be passed to fdb_select_api_version().

Network

The FoundationDB client library performs most tasks on a singleton thread (which usually will be a different thread than your application runs on). These functions are used to configure, start and stop the FoundationDB event loop on this thread.

fdb_error_t fdb_network_set_option(FDBNetworkOption option, uint8_t const *value, int value_length)

Called to set network options. If the given option is documented as taking a parameter, you must also pass a pointer to the parameter value and the parameter value’s length. If the option is documented as taking an Int parameter, value must point to a signed 64-bit integer (little-endian), and value_length must be 8. This memory only needs to be valid until fdb_network_set_option() returns.

type FDBNetworkOption

Please see fdb_c_options.g.h for a definition of this type, along with documentation of its allowed values.

fdb_error_t fdb_setup_network()

Must be called after fdb_select_api_version() (and zero or more calls to fdb_network_set_option()) and before any other function in this API. fdb_setup_network() can only be called once.

fdb_error_t fdb_add_network_thread_completion_hook(void (*hook)(void*), void *hook_parameter)

Must be called after fdb_setup_network() and prior to fdb_run_network() if called at all. This will register the given callback to run at the completion of the network thread. If there are multiple network threads running (which might occur if one is running multiple versions of the client, for example), then the callback is invoked once on each thread. When the supplied function is called, the supplied parameter is passed to it.

fdb_error_t fdb_run_network()

Must be called after fdb_setup_network() before any asynchronous functions in this API can be expected to complete. Unless your program is entirely event-driven based on results of asynchronous functions in this API and has no event loop of its own, you will want to invoke this function on an auxiliary thread (which it is your responsibility to create).

This function will not return until fdb_stop_network() is called by you or a serious error occurs. It is not possible to run more than one network thread, and the network thread cannot be restarted once it has been stopped. This means that once fdb_run_network has been called, it is not legal to call it again for the lifetime of the running program.

fdb_error_t fdb_stop_network()

Signals the event loop invoked by fdb_run_network() to terminate. You must call this function and wait for fdb_run_network() to return before allowing your program to exit, or else the behavior is undefined. For example, when running fdb_run_network() on a thread (using pthread), this will look like:

pthread_t network_thread; /* handle for thread which invoked fdb_run_network() */
int err;

...

err = fdb_stop_network();
if ( err ) {
    /* An error occurred (probably network not running) */
}
err = pthread_join( network_thread, NULL );
if ( err ) {
    /* Unknown error */
}
exit(0);

This function may be called from any thread. Once the network is stopped it cannot be restarted during the lifetime of the running program.

Future

Most functions in the FoundationDB API are asynchronous, meaning that they may return to the caller before actually delivering their result. These functions always return FDBFuture*. An FDBFuture object represents a result value or error to be delivered at some future time. You can wait for a Future to be “ready” – to have a value or error delivered – by setting a callback function, or by blocking a thread, or by polling. Once a Future is ready, you can extract either an error code or a value of the appropriate type (the documentation for the original function will tell you which fdb_future_get_() function you should call).

To use the API in a synchronous way, you would typically do something like this for each asynchronous call:

// Call an API that returns FDBFuture*, documented as returning type foo in the future
f = fdb_something();

// Wait for the Future to be *ready*
if ( (fdb_future_block_until_ready(f)) != 0 ) {
    // Exceptional error (e.g. out of memory)
}

if ( (err = fdb_future_get_foo(f, &result)) == 0 ) {
    // Use result
    // In some cases, you must be finished with result before calling
    // fdb_future_destroy() (see the documentation for the specific
    // fdb_future_get_*() method)
} else {
    // Handle the error. If this is an error in a transaction, see
    // fdb_transaction_on_error()
}

fdb_future_destroy(f);

Futures make it easy to do multiple operations in parallel, by calling several asynchronous functions before waiting for any of the results. This can be important for reducing the latency of transactions.

See Programming with futures for further (language-independent) discussion.

type FDBFuture

An opaque type that represents a Future in the FoundationDB C API.

void fdb_future_cancel(FDBFuture *future)

Cancels an FDBFuture object and its associated asynchronous operation. If called before the future is ready, attempts to access its value will return an operation_cancelled error. Cancelling a future which is already ready has no effect. Note that even if a future is not ready, its associated asynchronous operation may have succesfully completed and be unable to be cancelled.

void fdb_future_destroy(FDBFuture *future)

Destroys an FDBFuture object. It must be called exactly once for each FDBFuture* returned by an API function. It may be called before or after the future is ready. It will also cancel the future (and its associated operation if the latter is still outstanding).

fdb_error_t fdb_future_block_until_ready(FDBFuture *future)

Blocks the calling thread until the given Future is ready. It will return success even if the Future is set to an error – you must call fdb_future_get_error() to determine that. fdb_future_block_until_ready() will return an error only in exceptional conditions (e.g. deadlock detected, out of memory or other operating system resources).

Warning

Never call this function from a callback passed to fdb_future_set_callback(). This may block the thread on which fdb_run_network() was invoked, resulting in a deadlock. In some cases the client can detect the deadlock and throw a blocked_from_network_thread error.

fdb_bool_t fdb_future_is_ready(FDBFuture *future)

Returns non-zero if the Future is ready. A Future is ready if it has been set to a value or an error.

fdb_error_t fdb_future_set_callback(FDBFuture *future, FDBCallback callback, void *callback_parameter)

Causes the FDBCallback function to be invoked as callback(future, callback_parameter) when the given Future is ready. If the Future is already ready, the call may occur in the current thread before this function returns (but this behavior is not guaranteed). Alternatively, the call may be delayed indefinitely and take place on the thread on which fdb_run_network() was invoked, and the callback is responsible for any necessary thread synchronization (and/or for posting work back to your application event loop, thread pool, etc. if your application’s architecture calls for that).

Note

This function guarantees the callback will be executed at most once.

Warning

Never call fdb_future_block_until_ready() from a callback passed to this function. This may block the thread on which fdb_run_network() was invoked, resulting in a deadlock.

type FDBCallback

A pointer to a function which takes FDBFuture* and void* and returns void.

void fdb_future_release_memory(FDBFuture *future)

Note

This function provides no benefit to most application code. It is designed for use in writing generic, thread-safe language bindings. Applications should normally call fdb_future_destroy() only.

This function may only be called after a successful (zero return value) call to fdb_future_get_key(), fdb_future_get_value(), or fdb_future_get_keyvalue_array(). It indicates that the memory returned by the prior get call is no longer needed by the application. After this function has been called the same number of times as fdb_future_get_*(), further calls to fdb_future_get_*() will return a future_released error. It is still necessary to later destroy the future with fdb_future_destroy().

Calling this function is optional, since fdb_future_destroy() will also release the memory returned by get functions. However, fdb_future_release_memory() leaves the future object itself intact and provides a specific error code which can be used for coordination by multiple threads racing to do something with the results of a specific future. This has proven helpful in writing binding code.

fdb_error_t fdb_future_get_error(FDBFuture *future)

Returns zero if future is ready and not in an error state, and a non-zero error code otherwise.

fdb_error_t fdb_future_get_int64(FDBFuture *future, int64_t *out)

Extracts a 64-bit integer from a pointer to FDBFuture into a caller-provided variable of type int64_t. future must represent a result of the appropriate type (i.e. must have been returned by a function documented as returning this type), or the results are undefined.

Returns zero if future is ready and not in an error state, and a non-zero error code otherwise (in which case the value of any out parameter is undefined).

fdb_error_t fdb_future_get_key_array(FDBFuture *f, FDBKey const **out_key_array, int *out_count)

Extracts an array of FDBKey from an FDBFuture* into a caller-provided variable of type FDBKey*. The size of the array will also be extracted and passed back by a caller-provided variable of type int future must represent a result of the appropriate type (i.e. must have been returned by a function documented as returning this type), or the results are undefined.

Returns zero if future is ready and not in an error state, and a non-zero error code otherwise (in which case the value of any out parameter is undefined).

fdb_error_t fdb_future_get_key(FDBFuture *future, uint8_t const **out_key, int *out_key_length)

Extracts a key from an FDBFuture into caller-provided variables of type uint8_t* (a pointer to the beginning of the key) and int (the length of the key). future must represent a result of the appropriate type (i.e. must have been returned by a function documented as returning this type), or the results are undefined.

Returns zero if future is ready and not in an error state, and a non-zero error code otherwise (in which case the value of any out parameter is undefined).

The memory referenced by the result is owned by the FDBFuture object and will be valid until either fdb_future_destroy(future) or fdb_future_release_memory(future) is called.

fdb_error_t fdb_future_get_value(FDBFuture *future, fdb_bool_t *out_present, uint8_t const **out_value, int *out_value_length)

Extracts a database value from an FDBFuture into caller-provided variables. future must represent a result of the appropriate type (i.e. must have been returned by a function documented as returning this type), or the results are undefined.

Returns zero if future is ready and not in an error state, and a non-zero error code otherwise (in which case the value of any out parameter is undefined).

*out_present

Set to non-zero if (and only if) the requested value was present in the database. (If zero, the other outputs are meaningless.)

*out_value

Set to point to the first byte of the value.

*out_value_length

Set to the length of the value (in bytes).

The memory referenced by the result is owned by the FDBFuture object and will be valid until either fdb_future_destroy(future) or fdb_future_release_memory(future) is called.

fdb_error_t fdb_future_get_string_array(FDBFuture *future, const char ***out_strings, int *out_count)

Extracts an array of null-terminated C strings from an FDBFuture into caller-provided variables. future must represent a result of the appropriate type (i.e. must have been returned by a function documented as returning this type), or the results are undefined.

Returns zero if future is ready and not in an error state, and a non-zero error code otherwise (in which case the value of any out parameter is undefined).

*out_strings

Set to point to the first string in the array.

*out_count

Set to the number of strings in the array.

The memory referenced by the result is owned by the FDBFuture object and will be valid until either fdb_future_destroy(future) or fdb_future_release_memory(future) is called.

fdb_error_t fdb_future_get_keyvalue_array(FDBFuture *future, FDBKeyValue const **out_kv, int *out_count, fdb_bool_t *out_more)

Extracts an array of FDBKeyValue objects from an FDBFuture into caller-provided variables. future must represent a result of the appropriate type (i.e. must have been returned by a function documented as returning this type), or the results are undefined.

Returns zero if future is ready and not in an error state, and a non-zero error code otherwise (in which case the value of any out parameter is undefined).

*out_kv

Set to point to the first FDBKeyValue object in the array.

*out_count

Set to the number of FDBKeyValue objects in the array.

*out_more

Set to true if (but not necessarily only if) values remain in the key range requested (possibly beyond the limits requested).

The memory referenced by the result is owned by the FDBFuture object and will be valid until either fdb_future_destroy(future) or fdb_future_release_memory(future) is called.

type FDBKeyValue

Represents a single key-value pair in the output of fdb_future_get_keyvalue_array().

typedef struct {
    const void* key;
    int         key_length;
    const void* value;
    int         value_length;
} FDBKeyValue;
key

A pointer to a key.

key_length

The length of the key pointed to by key.

value

A pointer to a value.

value_length

The length of the value pointed to by value.

Database

An FDBDatabase represents a FoundationDB database — a mutable, lexicographically ordered mapping from binary keys to binary values. Modifications to a database are performed via transactions.

type FDBDatabase

An opaque type that represents a database in the FoundationDB C API.

fdb_error_t fdb_create_database(const char *cluster_file_path, FDBDatabase **out_database)

Creates a new database connected the specified cluster. The caller assumes ownership of the FDBDatabase object and must destroy it with fdb_database_destroy().

A single client can use this function multiple times to connect to different clusters simultaneously, with each invocation requiring its own cluster file. To connect to multiple clusters running at different, incompatible versions, the multi-version client API must be used.

cluster_file_path

A NULL-terminated string giving a local path of a cluster file (often called ‘fdb.cluster’) which contains connection information for the FoundationDB cluster. If cluster_file_path is NULL or an empty string, then a default cluster file will be used.

*out_database

Set to point to the newly created FDBDatabase.

void fdb_database_destroy(FDBDatabase *database)

Destroys an FDBDatabase object. It must be called exactly once for each successful call to fdb_create_database(). This function only destroys a handle to the database – your database will be fine!

fdb_error_t fdb_database_set_option(FDBDatabase *database, FDBDatabaseOption option, uint8_t const *value, int value_length)

Called to set an option an on FDBDatabase. If the given option is documented as taking a parameter, you must also pass a pointer to the parameter value and the parameter value’s length. If the option is documented as taking an Int parameter, value must point to a signed 64-bit integer (little-endian), and value_length must be 8. This memory only needs to be valid until fdb_database_set_option() returns.

type FDBDatabaseOption

Please see fdb_c_options.g.h for a definition of this type, along with documentation of its allowed values.

fdb_error_t fdb_database_open_tenant(FDBDatabase *database, uint8_t const *tenant_name, int tenant_name_length, FDBTenant **out_tenant)

Opens a tenant on the given database. All transactions created by this tenant will operate on the tenant’s key-space. The caller assumes ownership of the FDBTenant object and must destroy it with fdb_tenant_destroy().

tenant_name

The name of the tenant being accessed, as a byte string.

tenant_name_length

The length of the tenant name byte string.

*out_tenant

Set to point to the newly created FDBTenant.

fdb_error_t fdb_database_create_transaction(FDBDatabase *database, FDBTransaction **out_transaction)

Creates a new transaction on the given database without using a tenant, meaning that it will operate on the entire database key-space. The caller assumes ownership of the FDBTransaction object and must destroy it with fdb_transaction_destroy().

*out_transaction

Set to point to the newly created FDBTransaction.

FDBFuture *fdb_database_reboot_worker(FDBDatabase *database, uint8_t const *address, int address_length, fdb_bool_t check, int duration)

Reboot the specified process in the database.

Returns an FDBFuture which will be set to a int64_t which represents whether the reboot request is sent or not. In particular, 1 means request sent and 0 means failure (e.g. the process with the specified address does not exist). You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_int64() to extract the result, and then destroy the FDBFuture with fdb_future_destroy().

address

A pointer to the network address of the process.

address_length

The length of the parameter specified by address.

check

whether to perform a storage engine integrity check. In particular, the check-on-reboot is implemented by writing a check/validation file on disk as breadcrumb for the process to find after reboot, at which point it will eat the breadcrumb file and pass true to the integrityCheck parameter of the openKVStore() factory method.

duration

If positive, the process will be first suspended for duration seconds before being rebooted.

FDBFuture *fdb_database_force_recovery_with_data_loss(FDBDatabase *database, uint8_t const *dcId, int dcId_length)

Force the database to recover into the given datacenter.

This function is only useful in a fearless configuration where you want to recover your database even with losing recently committed mutations.

In particular, the function will set usable_regions to 1 and the amount of mutations that will be lost depends on how far behind the remote datacenter is.

The function will change the region configuration to have a positive priority for the chosen dcId, and a negative priority for all other dcIds.

In particular, no error will be thrown if the given dcId does not exist. It will just not attempt to force a recovery.

If the database has already recovered, the function does nothing. Thus it’s safe to call it multiple times.

Returns an FDBFuture representing an empty value. You must first wait for the FDBFuture to be ready, check for errors, and then destroy the FDBFuture with fdb_future_destroy().

FDBFuture *fdb_database_create_snapshot(FDBDatabase *database, uint8_t const *snapshot_command, int snapshot_command_length)

Create a snapshot of the database.

uid

A UID used to create snapshot. A valid uid is a 32-length hex string, otherwise, it will fail with error_code_snap_invalid_uid_string.

It is the user’s responsibility to make sure the given uid is unique.

uid_length

The length of the parameter specified by uid

snapshot_command

A pointer to all the snapshot command arguments.

In particular, if the original fdbcli command is snapshot <arg1> <arg2> <argN>, then the string snapshot_command points to is <arg1> <arg2> <argN>.

snapshot_command_length

The length of the parameter specified by snapshot_command

Note

The function is exposing the functionality of the fdbcli command snapshot. Please take a look at the documentation before using (see Disk snapshot backup and Restore).

double fdb_database_get_main_thread_busyness(FDBDatabase *database)

Returns a value where 0 indicates that the client is idle and 1 (or larger) indicates that the client is saturated. By default, this value is updated every second.

Tenant

FDBTenant represents a FoundationDB tenant. Tenants are optional named transaction domains that can be used to provide multiple disjoint key-spaces to client applications. A transaction created in a tenant will be limited to the keys contained within that tenant, and transactions operating on different tenants can use the same key names without interfering with each other.

type FDBTenant

An opaque type that represents a tenant in the FoundationDB C API.

void fdb_tenant_destroy(FDBTenant *tenant)

Destroys an FDBTenant object. It must be called exactly once for each successful call to fdb_database_create_tenant(). This function only destroys a handle to the tenant – the tenant and its data will be fine!

fdb_error_t fdb_tenant_create_transaction(FDBTenant *tenant, FDBTronsaction **out_transaction)

Creates a new transaction on the given tenant. This transaction will operate within the tenant’s key-space and cannot access data outside the tenant. The caller assumes ownership of the FDBTransaction object and must destroy it with fdb_transaction_destroy().

*out_transaction

Set to point to the newly created FDBTransaction.

Transaction

In FoundationDB, a transaction is a mutable snapshot of a database. All read and write operations on a transaction see and modify an otherwise-unchanging version of the database and only change the underlying database if and when the transaction is committed. Read operations do see the effects of previous write operations on the same transaction. Committing a transaction usually succeeds in the absence of conflicts.

Applications must provide error handling and an appropriate retry loop around the application code for a transaction. See the documentation for fdb_transaction_on_error().

Transactions group operations into a unit with the properties of atomicity, isolation, and durability. Transactions also provide the ability to maintain an application’s invariants or integrity constraints, supporting the property of consistency. Together these properties are known as ACID.

Transactions are also causally consistent: once a transaction has been successfully committed, all subsequently created transactions will see the modifications made by it.

type FDBTransaction

An opaque type that represents a transaction in the FoundationDB C API.

void fdb_transaction_destroy(FDBTransaction *transaction)

Destroys an FDBTransaction object. It must be called exactly once for each successful call to fdb_database_create_transaction(). Destroying a transaction which has not had fdb_transaction_commit() called implicitly “rolls back” the transaction (sets and clears do not take effect on the database).

fdb_error_t fdb_transaction_set_option(FDBTransaction *transaction, FDBTransactionOption option, uint8_t const *value, int value_length)

Called to set an option on an FDBTransaction. If the given option is documented as taking a parameter, you must also pass a pointer to the parameter value and the parameter value’s length. If the option is documented as taking an Int parameter, value must point to a signed 64-bit integer (little-endian), and value_length must be 8. This memory only needs to be valid until fdb_transaction_set_option() returns.

type FDBTransactionOption

Please see fdb_c_options.g.h for a definition of this type, along with documentation of its allowed values.

void fdb_transaction_set_read_version(FDBTransaction *transaction, int64_t version)

Sets the snapshot read version used by a transaction. This is not needed in simple cases. If the given version is too old, subsequent reads will fail with error_code_transaction_too_old; if it is too new, subsequent reads may be delayed indefinitely and/or fail with error_code_future_version. If any of fdb_transaction_get_*() have been called on this transaction already, the result is undefined.

FDBFuture *fdb_transaction_get_read_version(FDBTransaction *transaction)

Returns an FDBFuture which will be set to the transaction snapshot read version. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_int64() to extract the version into an int64_t that you provide, and then destroy the FDBFuture with fdb_future_destroy().

The transaction obtains a snapshot read version automatically at the time of the first call to fdb_transaction_get_*() (including this one) and (unless causal consistency has been deliberately compromised by transaction options) is guaranteed to represent all transactions which were reported committed before that call.

FDBFuture *fdb_transaction_get(FDBTransaction *transaction, uint8_t const *key_name, int key_name_length, fdb_bool_t snapshot)

Reads a value from the database snapshot represented by transaction.

Returns an FDBFuture which will be set to the value of key_name in the database. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_value() to extract the value, and then destroy the FDBFuture with fdb_future_destroy().

See fdb_future_get_value() to see exactly how results are unpacked. If key_name is not present in the database, the result is not an error, but a zero for *out_present returned from that function.

key_name

A pointer to the name of the key to be looked up in the database. The value does not need to be NULL-terminated.

key_name_length

The length of the parameter specified by key_name.

snapshot

Non-zero if this is a snapshot read.

FDBFuture *fdb_transaction_get_estimated_range_size_bytes(FDBTransaction *tr, uint8_t const *begin_key_name, int begin_key_name_length, uint8_t const *end_key_name, int end_key_name_length)

Returns an estimated byte size of the key range.

Note

The estimated size is calculated based on the sampling done by FDB server. The sampling algorithm works roughly in this way: the larger the key-value pair is, the more likely it would be sampled and the more accurate its sampled size would be. And due to that reason it is recommended to use this API to query against large ranges for accuracy considerations. For a rough reference, if the returned size is larger than 3MB, one can consider the size to be accurate.

Returns an FDBFuture which will be set to the estimated size of the key range given. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_int64() to extract the size, and then destroy the FDBFuture with fdb_future_destroy().

FDBFuture *fdb_transaction_get_range_split_points(FDBTransaction *tr, uint8_t const *begin_key_name, int begin_key_name_length, uint8_t const *end_key_name, int end_key_name_length, int64_t chunk_size)

Returns a list of keys that can split the given range into (roughly) equally sized chunks based on chunk_size.

Note

The returned split points contain the start key and end key of the given range

Returns an FDBFuture which will be set to the list of split points. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_key_array() to extract the array, and then destroy the FDBFuture with fdb_future_destroy().

FDBFuture *fdb_transaction_get_key(FDBTransaction *transaction, uint8_t const *key_name, int key_name_length, fdb_bool_t or_equal, int offset, fdb_bool_t snapshot)

Resolves a key selector against the keys in the database snapshot represented by transaction.

Returns an FDBFuture which will be set to the key in the database matching the key selector. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_key() to extract the key, and then destroy the FDBFuture with fdb_future_destroy().

key_name, key_name_length, or_equal, offset

The four components of a key selector.

snapshot

Non-zero if this is a snapshot read.

FDBFuture *fdb_transaction_get_addresses_for_key(FDBTransaction *transaction, uint8_t const *key_name, int key_name_length)

Returns a list of public network addresses as strings, one for each of the storage servers responsible for storing key_name and its associated value.

Returns an FDBFuture which will be set to an array of strings. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_string_array() to extract the string array, and then destroy the FDBFuture with fdb_future_destroy().

key_name

A pointer to the name of the key whose location is to be queried.

key_name_length

The length of the parameter specified by key_name.

FDBFuture *fdb_transaction_get_range(FDBTransaction *transaction, uint8_t const *begin_key_name, int begin_key_name_length, fdb_bool_t begin_or_equal, int begin_offset, uint8_t const *end_key_name, int end_key_name_length, fdb_bool_t end_or_equal, int end_offset, int limit, int target_bytes, FDBStreamingMode mode, int iteration, fdb_bool_t snapshot, fdb_bool_t reverse)

Reads all key-value pairs in the database snapshot represented by transaction (potentially limited by limit, target_bytes, or mode) which have a key lexicographically greater than or equal to the key resolved by the begin key selector and lexicographically less than the key resolved by the end key selector.

Returns an FDBFuture which will be set to an FDBKeyValue array. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_keyvalue_array() to extract the key-value array, and then destroy the FDBFuture with fdb_future_destroy().

begin_key_name, begin_key_name_length, begin_or_equal, begin_offset

The four components of a key selector describing the beginning of the range.

end_key_name, end_key_name_length, end_or_equal, end_offset

The four components of a key selector describing the end of the range.

limit

If non-zero, indicates the maximum number of key-value pairs to return. If this limit was reached before the end of the specified range, then the *more return of fdb_future_get_keyvalue_array() will be set to a non-zero value.

target_bytes

If non-zero, indicates a (soft) cap on the combined number of bytes of keys and values to return. If this limit was reached before the end of the specified range, then the *more return of fdb_future_get_keyvalue_array() will be set to a non-zero value.

mode

One of the FDBStreamingMode values indicating how the caller would like the data in the range returned.

iteration

If mode is FDB_STREAMING_MODE_ITERATOR, this parameter should start at 1 and be incremented by 1 for each successive call while reading this range. In all other cases it is ignored.

snapshot

Non-zero if this is a snapshot read.

reverse

If non-zero, key-value pairs will be returned in reverse lexicographical order beginning at the end of the range. Reading ranges in reverse is supported natively by the database and should have minimal extra cost.

type FDBStreamingMode

An enumeration of available streaming modes to be passed to fdb_transaction_get_range().

FDB_STREAMING_MODE_ITERATOR

The caller is implementing an iterator (most likely in a binding to a higher level language). The amount of data returned depends on the value of the iteration parameter to fdb_transaction_get_range().

FDB_STREAMING_MODE_SMALL

Data is returned in small batches (not much more expensive than reading individual key-value pairs).

FDB_STREAMING_MODE_MEDIUM

Data is returned in batches between _SMALL and _LARGE.

FDB_STREAMING_MODE_LARGE

Data is returned in batches large enough to be, in a high-concurrency environment, nearly as efficient as possible. If the caller does not need the entire range, some disk and network bandwidth may be wasted. The batch size may be still be too small to allow a single client to get high throughput from the database.

FDB_STREAMING_MODE_SERIAL

Data is returned in batches large enough that an individual client can get reasonable read bandwidth from the database. If the caller does not need the entire range, considerable disk and network bandwidth may be wasted.

FDB_STREAMING_MODE_WANT_ALL

The caller intends to consume the entire range and would like it all transferred as early as possible.

FDB_STREAMING_MODE_EXACT

The caller has passed a specific row limit and wants that many rows delivered in a single batch.

void fdb_transaction_set(FDBTransaction *transaction, uint8_t const *key_name, int key_name_length, uint8_t const *value, int value_length)

Modify the database snapshot represented by transaction to change the given key to have the given value. If the given key was not previously present in the database it is inserted.

The modification affects the actual database only if transaction is later committed with fdb_transaction_commit().

key_name

A pointer to the name of the key to be inserted into the database. The value does not need to be NULL-terminated.

key_name_length

The length of the parameter specified by key_name.

value

A pointer to the value to be inserted into the database. The value does not need to be NULL-terminated.

value_length

The length of the parameter specified by value.

void fdb_transaction_clear(FDBTransaction *transaction, uint8_t const *key_name, int key_name_length)

Modify the database snapshot represented by transaction to remove the given key from the database. If the key was not previously present in the database, there is no effect.

The modification affects the actual database only if transaction is later committed with fdb_transaction_commit().

key_name

A pointer to the name of the key to be removed from the database. The value does not need to be NULL-terminated.

key_name_length

The length of the parameter specified by key_name.

void fdb_transaction_clear_range(FDBTransaction *transaction, uint8_t const *begin_key_name, int begin_key_name_length, uint8_t const *end_key_name, int end_key_name_length)

Modify the database snapshot represented by transaction to remove all keys (if any) which are lexicographically greater than or equal to the given begin key and lexicographically less than the given end_key.

The modification affects the actual database only if transaction is later committed with fdb_transaction_commit().

Range clears are efficient with FoundationDB – clearing large amounts of data will be fast. However, this will not immediately free up disk - data for the deleted range is cleaned up in the background. For purposes of computing the transaction size, only the begin and end keys of a clear range are counted. The size of the data stored in the range does not count against the transaction size limit.

begin_key_name

A pointer to the name of the key specifying the beginning of the range to clear. The value does not need to be NULL-terminated.

begin_key_name_length

The length of the parameter specified by begin_key_name.

end_key_name

A pointer to the name of the key specifying the end of the range to clear. The value does not need to be NULL-terminated.

end_key_name_length

The length of the parameter specified by end_key_name_length.

void fdb_transaction_atomic_op(FDBTransaction *transaction, uint8_t const *key_name, int key_name_length, uint8_t const *param, int param_length, FDBMutationType operationType)

Modify the database snapshot represented by transaction to perform the operation indicated by operationType with operand param to the value stored by the given key.

An atomic operation is a single database command that carries out several logical steps: reading the value of a key, performing a transformation on that value, and writing the result. Different atomic operations perform different transformations. Like other database operations, an atomic operation is used within a transaction; however, its use within a transaction will not cause the transaction to conflict.

Atomic operations do not expose the current value of the key to the client but simply send the database the transformation to apply. In regard to conflict checking, an atomic operation is equivalent to a write without a read. It can only cause other transactions performing reads of the key to conflict.

By combining these logical steps into a single, read-free operation, FoundationDB can guarantee that the transaction will not conflict due to the operation. This makes atomic operations ideal for operating on keys that are frequently modified. A common example is the use of a key-value pair as a counter.

Warning

If a transaction uses both an atomic operation and a strictly serializable read on the same key, the benefits of using the atomic operation (for both conflict checking and performance) are lost.

The modification affects the actual database only if transaction is later committed with fdb_transaction_commit().

key_name

A pointer to the name of the key whose value is to be mutated.

key_name_length

The length of the parameter specified by key_name.

param

A pointer to the parameter with which the atomic operation will mutate the value associated with key_name.

param_length

The length of the parameter specified by param.

operation_type

One of the FDBMutationType values indicating which operation should be performed.

type FDBMutationType

An enumeration of available opcodes to be passed to fdb_transaction_atomic_op()

FDB_MUTATION_TYPE_ADD

Performs an addition of little-endian integers. If the existing value in the database is not present or shorter than param, it is first extended to the length of param with zero bytes. If param is shorter than the existing value in the database, the existing value is truncated to match the length of param. In case of overflow, the result is truncated to the width of param.

The integers to be added must be stored in a little-endian representation. They can be signed in two’s complement representation or unsigned. You can add to an integer at a known offset in the value by prepending the appropriate number of zero bytes to param and padding with zero bytes to match the length of the value. However, this offset technique requires that you know the addition will not cause the integer field within the value to overflow.

FDB_MUTATION_TYPE_AND

Performs a bitwise “and” operation. If the existing value in the database is not present, then param is stored in the database. If the existing value in the database is shorter than param, it is first extended to the length of param with zero bytes. If param is shorter than the existing value in the database, the existing value is truncated to match the length of param.

FDB_MUTATION_TYPE_OR

Performs a bitwise “or” operation. If the existing value in the database is not present or shorter than param, it is first extended to the length of param with zero bytes. If param is shorter than the existing value in the database, the existing value is truncated to match the length of param.

FDB_MUTATION_TYPE_XOR

Performs a bitwise “xor” operation. If the existing value in the database is not present or shorter than param, it is first extended to the length of param with zero bytes. If param is shorter than the existing value in the database, the existing value is truncated to match the length of param.

FDB_MUTATION_TYPE_COMPARE_AND_CLEAR

Performs an atomic compare and clear operation. If the existing value in the database is equal to the given value, then given key is cleared.

FDB_MUTATION_TYPE_MAX

Sets the value in the database to the larger of the existing value and param. If the existing value in the database is not present or shorter than param, it is first extended to the length of param with zero bytes. If param is shorter than the existing value in the database, the existing value is truncated to match the length of param.

Both the existing value and param are treated as unsigned integers. (This differs from the behavior of atomic addition.)

FDB_MUTATION_TYPE_BYTE_MAX

Performs lexicographic comparison of byte strings. If the existing value in the database is not present, then param is stored. Otherwise the larger of the two values is then stored in the database.

FDB_MUTATION_TYPE_MIN

Sets the value in the database to the smaller of the existing value and param. If the existing value in the database is not present, then param is stored in the database. If the existing value in the database is shorter than param, it is first extended to the length of param with zero bytes. If param is shorter than the existing value in the database, the existing value is truncated to match the length of param.

Both the existing value and param are treated as unsigned integers. (This differs from the behavior of atomic addition.)

FDB_MUTATION_TYPE_BYTE_MIN

Performs lexicographic comparison of byte strings. If the existing value in the database is not present, then param is stored. Otherwise the smaller of the two values is then stored in the database.

FDB_MUTATION_TYPE_SET_VERSIONSTAMPED_KEY

Transforms key using a versionstamp for the transaction. This key must be at least 14 bytes long. The final 4 bytes will be interpreted as a 32-bit little-endian integer denoting an index into the key at which to perform the transformation, and then trimmed off the key. The 10 bytes in the key beginning at the index will be overwritten with the versionstamp. If the index plus 10 bytes points past the end of the key, the result will be an error. Sets the transformed key in the database to param.

A versionstamp is a 10 byte, unique, monotonically (but not sequentially) increasing value for each committed transaction. The first 8 bytes are the committed version of the database (serialized in big-endian order). The last 2 bytes are monotonic in the serialization order for transactions (serialized in big-endian order).

A transaction is not permitted to read any transformed key or value previously set within that transaction, and an attempt to do so will result in an accessed_unreadable error. The range of keys marked unreadable when setting a versionstamped key begins at the transactions’s read version if it is known, otherwise a versionstamp of all 0x00 bytes is conservatively assumed. The upper bound of the unreadable range is a versionstamp of all 0xFF bytes.

Warning

At this time, versionstamped keys are not compatible with the Tuple layer except in Java, Python, and Go. Note that this implies versionstamped keys may not be used with the Subspace and Directory layers except in those languages.

FDB_MUTATION_TYPE_SET_VERSIONSTAMPED_VALUE

Transforms param using a versionstamp for the transaction. This parameter must be at least 14 bytes long. The final 4 bytes will be interpreted as a 32-bit little-endian integer denoting an index into the parameter at which to perform the transformation, and then trimmed off the key. The 10 bytes in the parameter beginning at the index will be overwritten with the versionstamp. If the index plus 10 bytes points past the end of the parameter, the result will be an error. Sets key in the database to the transformed parameter.

A versionstamp is a 10 byte, unique, monotonically (but not sequentially) increasing value for each committed transaction. The first 8 bytes are the committed version of the database (serialized in big-endian order). The last 2 bytes are monotonic in the serialization order for transactions (serialized in big-endian order).

A transaction is not permitted to read any transformed key or value previously set within that transaction, and an attempt to do so will result in an accessed_unreadable error. The range of keys marked unreadable when setting a versionstamped key begins at the transactions’s read version if it is known, otherwise a versionstamp of all 0x00 bytes is conservatively assumed. The upper bound of the unreadable range is a versionstamp of all 0xFF bytes.

Warning

At this time, versionstamped values are not compatible with the Tuple layer except in Java, Python, and Go. Note that this implies versionstamped values may not be used with the Subspace and Directory layers except in those languages.

FDBFuture *fdb_transaction_commit(FDBTransaction *transaction)

Attempts to commit the sets and clears previously applied to the database snapshot represented by transaction to the actual database. The commit may or may not succeed – in particular, if a conflicting transaction previously committed, then the commit must fail in order to preserve transactional isolation. If the commit does succeed, the transaction is durably committed to the database and all subsequently started transactions will observe its effects.

It is not necessary to commit a read-only transaction – you can simply call fdb_transaction_destroy().

Returns an FDBFuture representing an empty value. You must first wait for the FDBFuture to be ready, check for errors, and then destroy the FDBFuture with fdb_future_destroy().

Callers will usually want to retry a transaction if the commit or a prior fdb_transaction_get_*() returns a retryable error (see fdb_transaction_on_error()).

As with other client/server databases, in some failure scenarios a client may be unable to determine whether a transaction succeeded. In these cases, fdb_transaction_commit() will return a commit_unknown_result error. The fdb_transaction_on_error() function treats this error as retryable, so retry loops that don’t check for commit_unknown_result could execute the transaction twice. In these cases, you must consider the idempotence of the transaction. For more information, see Transactions with unknown results.

Normally, commit will wait for outstanding reads to return. However, if those reads were snapshot reads or the transaction option for disabling “read-your-writes” has been invoked, any outstanding reads will immediately return errors.

fdb_error_t fdb_transaction_get_committed_version(FDBTransaction *transaction, int64_t *out_version)

Retrieves the database version number at which a given transaction was committed. fdb_transaction_commit() must have been called on transaction and the resulting future must be ready and not an error before this function is called, or the behavior is undefined. Read-only transactions do not modify the database when committed and will have a committed version of -1. Keep in mind that a transaction which reads keys and then sets them to their current values may be optimized to a read-only transaction.

Note that database versions are not necessarily unique to a given transaction and so cannot be used to determine in what order two transactions completed. The only use for this function is to manually enforce causal consistency when calling fdb_transaction_set_read_version() on another subsequent transaction.

Most applications will not call this function.

FDBFuture *fdb_transaction_get_approximate_size(FDBTransaction *transaction)

Returns an FDBFuture which will be set to the approximate transaction size so far in the returned future, which is the summation of the estimated size of mutations, read conflict ranges, and write conflict ranges. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_int64() to extract the size, and then destroy the FDBFuture with fdb_future_destroy().

This can be called multiple times before the transaction is committed.

FDBFuture *fdb_transaction_get_versionstamp(FDBTransaction *transaction)

Returns an FDBFuture which will be set to the versionstamp which was used by any versionstamp operations in this transaction. You must first wait for the FDBFuture to be ready, check for errors, call fdb_future_get_key() to extract the key, and then destroy the FDBFuture with fdb_future_destroy().

The future will be ready only after the successful completion of a call to fdb_transaction_commit() on this Transaction. Read-only transactions do not modify the database when committed and will result in the future completing with an error. Keep in mind that a transaction which reads keys and then sets them to their current values may be optimized to a read-only transaction.

Most applications will not call this function.

FDBFuture *fdb_transaction_watch(FDBTransaction *transaction, uint8_t const *key_name, int key_name_length)

A watch’s behavior is relative to the transaction that created it. A watch will report a change in relation to the key’s value as readable by that transaction. The initial value used for comparison is either that of the transaction’s read version or the value as modified by the transaction itself prior to the creation of the watch. If the value changes and then changes back to its initial value, the watch might not report the change.

Until the transaction that created it has been committed, a watch will not report changes made by other transactions. In contrast, a watch will immediately report changes made by the transaction itself. Watches cannot be created if the transaction has set the READ_YOUR_WRITES_DISABLE transaction option, and an attempt to do so will return an watches_disabled error.

If the transaction used to create a watch encounters an error during commit, then the watch will be set with that error. A transaction whose commit result is unknown will set all of its watches with the commit_unknown_result error. If an uncommitted transaction is reset or destroyed, then any watches it created will be set with the transaction_cancelled error.

Returns an FDBFuture representing an empty value that will be set once the watch has detected a change to the value at the specified key. You must first wait for the FDBFuture to be ready, check for errors, and then destroy the FDBFuture with fdb_future_destroy().

By default, each database connection can have no more than 10,000 watches that have not yet reported a change. When this number is exceeded, an attempt to create a watch will return a too_many_watches error. This limit can be changed using the MAX_WATCHES database option. Because a watch outlives the transaction that creates it, any watch that is no longer needed should be cancelled by calling fdb_future_cancel() on its returned future.

key_name

A pointer to the name of the key to watch. The value does not need to be NULL-terminated.

key_name_length

The length of the parameter specified by key_name.

FDBFuture *fdb_transaction_on_error(FDBTransaction *transaction, fdb_error_t error)

Implements the recommended retry and backoff behavior for a transaction. This function knows which of the error codes generated by other fdb_transaction_*() functions represent temporary error conditions and which represent application errors that should be handled by the application. It also implements an exponential backoff strategy to avoid swamping the database cluster with excessive retries when there is a high level of conflict between transactions.

On receiving any type of error from an fdb_transaction_*() function, the application should:

  1. Call fdb_transaction_on_error() with the returned fdb_error_t code.

  2. Wait for the resulting future to be ready.

  3. If the resulting future is itself an error, destroy the future and FDBTransaction and report the error in an appropriate way.

  4. If the resulting future is not an error, destroy the future and restart the application code that performs the transaction. The transaction itself will have already been reset to its initial state, but should not be destroyed and re-created because state used by fdb_transaction_on_error() to implement its backoff strategy and state related to timeouts and retry limits is stored there.

Returns an FDBFuture representing an empty value. You must first wait for the FDBFuture to be ready, check for errors, and then destroy the FDBFuture with fdb_future_destroy().

void fdb_transaction_reset(FDBTransaction *transaction)

Reset transaction to its initial state. This is similar to calling fdb_transaction_destroy() followed by fdb_database_create_transaction(). It is not necessary to call fdb_transaction_reset() when handling an error with fdb_transaction_on_error() since the transaction has already been reset.

void fdb_transaction_cancel(FDBTransaction *transaction)

Cancels the transaction. All pending or future uses of the transaction will return a transaction_cancelled error. The transaction can be used again after it is reset.

Warning

Be careful if you are using fdb_transaction_reset() and fdb_transaction_cancel() concurrently with the same transaction. Since they negate each other’s effects, a race condition between these calls will leave the transaction in an unknown state.

Warning

If your program attempts to cancel a transaction after fdb_transaction_commit() has been called but before it returns, unpredictable behavior will result. While it is guaranteed that the transaction will eventually end up in a cancelled state, the commit may or may not occur. Moreover, even if the call to fdb_transaction_commit() appears to return a transaction_cancelled error, the commit may have occurred or may occur in the future. This can make it more difficult to reason about the order in which transactions occur.

fdb_error_t fdb_transaction_add_conflict_range(FDBTransaction *transaction, uint8_t const *begin_key_name, int begin_key_name_length, uint8_t const *end_key_name, int end_key_name_length, FDBConflictRangeType type)

Adds a conflict range to a transaction without performing the associated read or write.

Note

Most applications will use the strictly serializable isolation that transactions provide by default and will not need to manipulate conflict ranges.

begin_key_name

A pointer to the name of the key specifying the beginning of the conflict range. The value does not need to be NULL-terminated.

begin_key_name_length

The length of the parameter specified by begin_key_name.

end_key_name

A pointer to the name of the key specifying the end of the conflict range. The value does not need to be NULL-terminated.

end_key_name_length

The length of the parameter specified by end_key_name_length.

type

One of the FDBConflictRangeType values indicating what type of conflict range is being set.

type FDBConflictRangeType

An enumeration of available conflict range types to be passed to fdb_transaction_add_conflict_range().

FDB_CONFLICT_RANGE_TYPE_READ

Adds a range of keys to the transaction’s read conflict ranges as if you had read the range. As a result, other transactions that write a key in this range could cause the transaction to fail with a conflict.

FDB_CONFLICT_RANGE_TYPE_WRITE

Adds a range of keys to the transaction’s write conflict ranges as if you had cleared the range. As a result, other transactions that concurrently read a key in this range could fail with a conflict.

Snapshot reads

Snapshot reads selectively relax FoundationDB’s isolation property, reducing conflicts but making it harder to reason about concurrency.

By default, FoundationDB transactions guarantee strictly serializable isolation, resulting in a state that is as if transactions were executed one at a time, even if they were executed concurrently. Serializability has little performance cost when there are few conflicts but can be expensive when there are many. FoundationDB therefore also permits individual reads within a transaction to be done as snapshot reads.

Snapshot reads differ from ordinary (strictly serializable) reads by permitting the values they read to be modified by concurrent transactions, whereas strictly serializable reads cause conflicts in that case. Like strictly serializable reads, snapshot reads see the effects of prior writes in the same transaction. For more information on the use of snapshot reads, see Snapshot reads.

In the C API, snapshot reads are performed by passing a non-zero value to the snapshot parameter of any of fdb_transaction_get_* (see for example fdb_transaction_get()). Snapshot reads also interact with transaction commit a little differently than normal reads. If a snapshot read is outstanding when transaction commit is called that read will immediately return an error. (Normally, transaction commit will wait until outstanding reads return before committing.)

Key selectors

FoundationDB’s lexicographically ordered data model permits finding keys based on their order (for example, finding the first key in the database greater than a given key). Key selectors represent a description of a key in the database that could be resolved to an actual key by fdb_transaction_get_key() or used directly as the beginning or end of a range in fdb_transaction_get_range().

For more about how key selectors work, see Key selectors.

In the FoundationDB C API, key selectors are not represented by a structure of any kind, but are instead expressed as sequential parameters to fdb_transaction_get_key() and fdb_transaction_get_range(). For convenience, the most common key selectors are available as C macros that expand to the appropriate parameters.

type FDB_KEYSEL_LAST_LESS_THAN(key_name, key_name_length)
type FDB_KEYSEL_LAST_LESS_OR_EQUAL(key_name, key_name_length)
type FDB_KEYSEL_FIRST_GREATER_THAN(key_name, key_name_length)
type FDB_KEYSEL_FIRST_GREATER_OR_EQUAL(key_name, key_name_length)

To use one of these macros, simply replace the four parameters in the function with one of FDB_KEYSEL_*:

future = fdb_transaction_get_key(transaction, "key", 3, 0, 2, 0);

could instead be written as:

future = fdb_transaction_get_key(transaction, FDB_KEYSEL_FIRST_GREATER_THAN("key", 3)+1, 0);

Miscellaneous

type fdb_bool_t

An integer type representing a boolean. A value of 0 is false and non-zero is true.

type fdb_error_t

An integer type representing an error. A value of 0 is success and non-zero is an error.

const char *fdb_get_error(fdb_error_t code)

Returns a (somewhat) human-readable English message from an error code. The return value is a statically allocated null-terminated string that must not be freed by the caller.

fdb_bool_t fdb_error_predicate(int predicate_test, fdb_error_t code)

Evaluates a predicate against an error code. The predicate to run should be one of the codes listed by the FDBErrorPredicate enum defined within fdb_c_options.g.h. Sample predicates include FDB_ERROR_PREDICATE_RETRYABLE, which can be used to determine whether the error with the given code is a retryable error or not.