libqdb

A library for easy C databases (disk backed if needed). Built on top of libdb but optimized for simpler usage.

I'll start this documentation with the most common functions, but know that there are others which can be useful sometimes which you can find later on.

Common functions

qdb_init

void qdb_init(void);

This initializes the system.

Please run it before anything else!

qdb_open

int qdb_open(char *database, char *key_tid, char *value_tid, unsigned flags);

Open a database without having to specify too much stuff.

key_tid and value_tid are strings which indicated the usual data type for keys and values. Why? Because we don't want to specify lengths all the time, just pointers. And if we know about your data types in advance we can do just that.

But how do we register types of data? (you might ask) Let's answer that right away.

PS: More information in the section "Less common functions / qdb_openc".

qdb_reg

void qdb_reg(char *key, size_t len);

Register a new data type using only a length

qdb_put

unsigned qdb_put(unsigned hd, void *key, void *value);

Put a key value pair into the database

Yeah, now that the database is type aware, we can easily put and retrieve values.

If you use NULL as the key, it will generate an automatic index if the database is configured for it, and returns it.

qdb_get

int qdb_get(unsigned hd, void *value, void *key);

Get a value from a key

See how easy that makes it?

qdb_del

void qdb_del(unsigned hd, void *key, void *value);

Delete a key-value pair. If value is NULL, delete all values from the key

qdb_close

void qdb_close(unsigned hd, unsigned flags);

Closing a database

qdb_exists

int qdb_exists(unsigned hd, void *key);

Check for key's existence

With those out of the way, let's get into cursors and iteration!

qdb_iter

qdb_cur_t qdb_iter(unsigned hd, void *key);

Start an iteration

Use this to start an iteration of all values in the given key. You might use NULL as key to not filter out any values.

qdb_next

int qdb_next(void *key, void *value, qdb_cur_t *cur);

Get the next key / value in the iteration

I recommend you use it like so:

char person[BUFSIZ], pet[BUFSIZ];
qdb_cur_t c = qdb_iter(person_pets_hd, "Joe");

while (qdb_next(person, pet, &c))
    fprint("Joe has pet: %s\n", pet);

qdb_fin

void qdb_fin(qdb_cur_t *cur);

Stop an iteration early

Please call this before you might break or return from a while loop like that early. So that the cursor is cleanly closed. Otherwise you might have problems later on.

Well. That gets the most common functions out of the way. We even included some more uncommon ones. I guess we can call that an early exit.

Less common functions

qdb_openc

unsigned qdb_openc(const char *file, const char *database, int mode, unsigned flags, int type, char *key_tid, char *value_tid);

Open a database, but be more specific.

What if you want to specify a file, for disk-based database? And maybe you want to specify that you want a DB_BTREE instead of a DB_HASH. Well, you have another way of doing that by using qdb_config, which is an object with the defaults for open operations. But in case you want to specify everything in one go, you have this option.

qdb_regc

void qdb_regc(char *key, qdb_type_t *type);

Register a new data type, but we might want to calculate the size dynamically. Or provide it a print callback, or something.

qdb_putc

int qdb_putc(unsigned hd, void *key, size_t key_len, void *value, size_t value_len);

A low-level way to put keys and values that is not type-aware.

qdb_getc

void *qdb_getc(unsigned hd, size_t *size, void *key_r, size_t key_len);

And a low-level way to get items from the database that is not type-aware.

qdb_pget

int qdb_pget(unsigned hd, void *pkey, void *key);

Get the primary key corresponding to a key of a secondary database

Now that we mention it:

qdb_assoc

void qdb_assoc(unsigned hd, unsigned link, qdb_assoc_t assoc);
typedef void (*qdb_assoc_t)(void **data, uint32_t *len, void *key, void *value);

Associate a secondary database to a primary one.

If you use NULL as the callback, a simple mapping of the primary's key will be done.

qdb_cdel

int qdb_cdel(qdb_cur_t *cur);

Delete the item under the current iteration of the cursor

qdb_drop

int qdb_drop(unsigned hd);

Drop everything in a database (except metadata)

qdb_sync

void qdb_sync(unsigned hd);

Sync a database to disk without closing it.

qdb_existsc

int qdb_existsc(unsigned hd, void *key, size_t key_len)

A low-level way to check if a key exists. Not type-aware.

qdb_piter

qdb_cur_t qdb_piter(unsigned hd, void *key, unsigned reverse);

Just a little helper to iterate THRICE databases.

qdb_len

void qdb_len(unsigned hd, unsigned type, void *thing);

Return the length of a key (QDB_KEY) or a value (QDB_VALUE)

qdb_print

void qdb_print(unsigned hd, unsigned type, void *thing);

Print a key or a value

Here and when checking types for THRICE, you might use the least significant bit or QDB_REVERSE to check the inverse.

Logging

void qdb_set_logger(log_t logger);
typedef void (*log_t)(int type, const char *fmt, ...);

This is how you configure how logging is made. It defaults to printing to stderr, but you might as well use syslog for example.

Environments and transactions (Rarer)

qdb_env_create

DB_ENV *qdb_env_create(void);

Create a database environment, and set it to some good defaults.

qdb_env_open

void *qdb_env_open(DB_ENV *env, char *dir);

Open it and all ops will use it by default

qdb_begin

DB_TXN *qdb_begin(void);

Begin a transaction

qdb_commit

void qdb_commit(void);

Commit the upmost transaction on the stack.

qdb_abort

void qdb_abort(DB_TXN *txn);

Abort the upmost transaction on the stack.

qdb_checkpoint

void qdb_checkpoint(unsigned kbytes, unsigned min, unsigned flags);

This creates a checkpoint. You'll need the libdb docs for some more detail on some of this.

transaction stack

Although it is unlikely, you might need to copy the qdb_config.txnl object. And replace it temporarily in case you are working with multiple databases and each needs its own transaction stack.

Types

These are the built-in types we provide. You can add more if you like.

"s" - char *

The first to arrive is the odd one! A type of variable length!

"u" - unsigned

Pretty self explanatory

"p" - void *

Just a pointer. If what it points to doesn't change, that's all you need.

"ul" - unsigned long

You know, at this point.

Flags

These are the flags you can provide when opening databases:

QH_AINDEX

Use automatic indexes (don't forget to use 'u' as the key type).

QH_RDONLY

Don't change the database, we're just interested in reading.

This avoids having to have write permissions.

QH_SEC

This should be a secondary database

QH_TXN

Use transaction support

QH_DUP

Allow duplicate values for the same key

QH_THRICE

We want to have forward and reverse lookup (one primary two secondary).

Reusable indexes

We also export some features for automatic reusable indexes. We're internally interested in them. But if you also want to use them, you are free to.

struct idml

Is a singly-linked list of unsigned numbers, basically.

struct idm

Well, this is what we really need for reusable indexes. Just an idml of free numbers, and the biggest number in the db..

idm_init

struct idm idm_init(void);

Just initialize one of these.

idm_del

void idm_del(struct idm *idm, unsigned id);

Delete (in other words consider free) a certain id.

idm_new

unsigned idm_new(struct idm *idm);

Get an id we can use.

FILO(name, TYPE, INVALID)

This is just a macro to easily declare a FIFO. We use it for idml and txnl.

I'm going to be brief describing the provided features.

name_init(void)

Initializes a stack of this kind.

name_push(&stack, TYPE thing)

Pushes an element into it.

name_peek(&stack)

Returns at the element at the top without popping it.

name_pop(&stack)

Returns at the element at the top and pops it out of the stack.

name_iter(&stack)

Starts iterating over the stack's elements.

name_next(&slot, &stack)

Gets the next iteration and copies the element into the slot.

The cli tool

This executable is a way to make indexes easily right from the shell.

It allows for a few different kinds of databases to be created, queried and changed easily and with flexibility. You should be able to find documentation using:

qdb -?

But we still want to give you some examples.

First of all, there are some things you should know.

Initialization

The first thing you do is to put something into a database. That is very easy! Look:

qdb -p hi a.db

But what about the types envolved? Well... It defaults to unsigned to string. With automatic indexes, so you can easily put a value like that. How would you do another one?

qdb -p 3 b.db:a:u

This specifies that you want unsigned-type values. Dandy. But how to specify value types? Easy again! Just add another colon, like a roman! They fix everything.

qdb -p 3 b.db:s:u # You guessed it. String keys, unsigned values.

Key types

There are some key types built-in in this first version. Here they are:

• 'a' means unsigned but with (optional) automatic indexes - that's the default.
• 't<what>' means type 'what' and possibly duplicate keys!
• 'u' means unsigned.
• 's' means string.

For value types, remove 'a' and 't' from that list.

Examples

Put a person into the owner database:

qdb -p Mathew owners.db # Output: 4

List owners!

qdb -l owners.db

Insert pets into the pet database:

qdb -p cat -p dog pets.db # Output: 2 and 3

Let's associate them!

qdb -p 4:2 -p 4:3 assoc.db:t:u

Let's see all of Mathew's pets (show their names):

qdb -a pets.db -g4 assoc.db

Get a random one:

qdb -q owners.db -a pets.db -RMathew assoc.db

More information

See, those -q and -a flags can be handy. Since with them you can print (-a) or query (-q) using values in another database. We also have a -r flag that can reverse the lookups. And more! Be sure to check the help (-?).

Hope this is useful for you!