CBFS Vault 2022 Go Edition
Version 22.0 [Build 8460]

Introduction

Welcome to CBFS Vault, an all-in-one file storage solution that allows applications to create and embed secure, fully-functional filesystems anywhere. These filesystems, known as Vaults, are completely self-contained and can be stored in files, memory, or any other location your application has read/write access to.

Each vault can store as many files, directories, custom file attributes and tags, alternate streams, and symbolic links as desired; your application has complete control over its contents and capacity. To ensure your data is stored efficiently and securely, CBFS Vault also offers built-in Compression and Encryption functionality, and allows you to provide your own algorithms if desired.

Included Components

CBVaultThe CBVault component lets applications create a vault and manipulate its contents.

Additional Information

You will always find the latest information about CBFS Vault at our web site: www.callback.com. We offer free, fully-functional 30-day trials for all of our products, and our technical support staff are happy to answer any questions you may have during your evaluation.

Please direct all technical questions to support@callback.com. To help support technicians assist you as quickly as possible, please provide an detailed and accurate description of your problem, the results you expected, and the results that you received while using our product. For questions about licensing and pricing, and all other general inquiries, please contact sales@callback.com.

Thank You!

Thank you for choosing CBFS Vault for your development needs. We realize that you have a choice among development tools, and that by choosing us you are counting on us to be a key component in your business. We work around the clock to provide you with ongoing enhancements, support, and innovative products; and we will always do our best to exceed your expectations!

Deployment

The user mode library comes as dynamic libraries, named

  • Windows: gocbfsvault22.dll (available for 32-bit (x86) and 64-bit (x64) processor architectures)
  • Linux: libgocbfsvault.so.22.0 (available for 32-bit (x86) and 64-bit (x64) processor architectures)
  • macOS: libgocbfsvault22.0.dylib (available for x64 and ARM64 processor architectures)

Windows: When deploying the application, copy the dynamic library to the target system and place it next to the application's executable file, (on Windows, it has the .exe extension.

Alternatively, the native dynamic library may be placed into one of directories, pathes to which are contained in the

  • Windows: PATH environment variable, such as C:\Windows\System32 (or C:\Windows\SysWOW64 when deploying a 32-bit application on a 64-bit Windows system)
  • Linux: LD_LIBRARY_PATH environment variable
  • macOS: DYLD_LIBRARY_PATH environment variable

General Information

The topics in this section provide general information about various aspects of the product's functionality.

Topics

Buffer Parameters

Some events include one or more parameters intended for use as a binary data buffer. Depending on the event, these parameters may contain data when the event is fired, or it may be expected that the application populates them with the desired amount of data during the event handler. Some events combine both paradigms, and expect the application to modify the data already present when the event is fired.

The documentation for such events will describe which of the above cases applies to each buffer parameter. Note that, in all cases, buffer parameters point to a pre-allocated block of unmanaged memory, the size of which is specified by the parameter immediately following the buffer parameter. In cases where data is to be written, be sure to write it directly to the pointed-to memory, don't change the value of the buffer parameter itself.

Callback Mode

As discussed in the Vaults topic, the default behavior of CBVault component is to create a vault using a real file on a local disk. However, the filesystem engine behind these components doesn't require a vault to be a local file; it can be a remote file, a memory region, or anything else that the application can provide random read/write access to.

Applications that wish to use something other than a local file to store a vault must enable callback mode using the component's CallbackMode property. When callback mode is enabled, applications must handle the following events (which map closely to the Windows File API) in order for the component to interact with the vault. For brevity, vaults created and accessed using callback mode are typically referred to as "callback mode vaults".

  • VaultClose: Fires when the currently-open vault should be closed.
  • VaultDelete: Fires when a callback mode vault (that is not open) should be deleted.
  • VaultFlush: Fires when any buffered vault data should be flushed out to storage.
  • VaultGetParentSize: Fires when the component needs to know how much free space is available for the currently-open vault to use for automatic growth.
  • VaultGetSize: Fires when the component needs to know the size of the currently-open vault.
  • VaultOpen: Fires when a callback mode vault should be opened (and, if necessary, created).
  • VaultRead: Fires when the component needs to read one or more pages of vault data.
  • VaultSetSize: Fires when the component needs to resize (i.e., shrink or grow) the currently-open vault.
  • VaultWrite: Fires when the component needs to write one or more pages of vault data.

Callback mode is an extremely powerful feature for applications that want to fine-tune performance. For example, consider the following scenario: a help authoring tool that keeps a compound file in memory for fast operations. In this scenario, the vault itself would be a help project which the help authoring tool loads into memory and uses the above events to access. When a user needs to save the project, the vault is flushed and the data in memory is copied back to the help project on disk.

Note: An application should not attempt to call component's methods from handlers of the listed events. Doing this is guaranteed to cause a deadlock.

Encryption

The CBVault component includes strong built-in data encryption support, which can be applied to individual files and alternate streams, entire vaults, or both. Each file, alternate stream, and vault can have its own encryption key. (Note: the API members discussed in this topic are available in all listed components, unless otherwise noted.)

Encrypting Vaults

To specify a default encryption mode and password to use when creating new vaults, applications can set the VaultEncryption and VaultPassword properties. To change the encryption mode and/or password of an existing vault, use the UpdateVaultEncryption method.

When opening an existing vault, VaultEncryption is updated to reflect the vault's encryption mode; and if the vault is encrypted, the password specified by VaultPassword is used to access it.

Encrypting Files and Alternate Streams

To specify a default encryption mode and password for files and alternate streams, applications can set the DefaultFileEncryption and DefaultFilePassword properties. Additionally, the following methods allow applications to set a file or alternate stream's encryption mode and/or password explicitly:

When a file or alternate stream is encrypted, its encryption password must be provided in order to access it; many methods in the component's API provide a Password parameter for this purpose. If the application doesn't explicitly specify a password when calling such a method, then the DefaultFilePassword will be used, if possible.

Using Custom Encryption

The component's built-in encryption implementation uses 256-bit AES encryption in XTS mode with PBKDF2 key derivation based on a HMAC-SHA256 key hash. However, applications also can choose to provide their own custom encryption and key derivation implementations. This flexibility allows applications to support more sophisticated security techniques, such as PKI-based encryption, or Digital Rights Management. To get started, do the following:

  1. Choose a custom encryption mode to implement (i.e., one of the VAULT_EM_CUSTOM* options from the table below). This choice will determine:
    • Whether the custom encryption implementation uses a 256-bit, 512-bit, or 1024-bit block size; and,
    • Whether to use built-in key derivation, custom key derivation, or no key derivation.
  2. Implement the DataEncrypt and DataDecrypt events.
  3. If a VAULT_EM_CUSTOM*_CUSTOM_KEY_DERIVE mode was chosen, implement the KeyDerive event.
  4. If a VAULT_EM_CUSTOM*_DIRECT_KEY mode was chosen, implement the HashCalculate event.

Supported Encryption Modes

The component support the following encryption modes:

VAULT_EM_NONE0x0Don't use encryption.

VAULT_EM_DEFAULT0x1Use default encryption (VAULT_EM_XTS_AES256_PBKDF2_HMAC_SHA256).

VAULT_EM_XTS_AES256_PBKDF2_HMAC_SHA2560x2Use AES256 encryption with PBKDF2 key derivation based on a HMAC_SHA256 key hash.

VAULT_EM_CUSTOM256_PBKDF2_HMAC_SHA2560x3Use event-based custom 256-bit encryption with PBKDF2 key derivation based on a HMAC_SHA256 key hash.

256-bit (32-byte) block size.

VAULT_EM_CUSTOM512_PBKDF2_HMAC_SHA2560x4Use event-based custom 512-bit encryption with PBKDF2 key derivation based on a HMAC_SHA256 key hash.

512-bit (64-byte) block size.

VAULT_EM_CUSTOM1024_PBKDF2_HMAC_SHA2560x5Use event-based custom 1024-bit encryption with PBKDF2 key derivation based on a HMAC_SHA256 key hash.

1024-bit (128-byte) block size.

VAULT_EM_CUSTOM256_CUSTOM_KEY_DERIVE0x23Use event-based custom 256-bit encryption with custom key derivation.

256-bit (32-byte) block size.

VAULT_EM_CUSTOM512_CUSTOM_KEY_DERIVE0x24Use event-based custom 512-bit encryption with custom key derivation.

512-bit (64-byte) block size.

VAULT_EM_CUSTOM1024_CUSTOM_KEY_DERIVE0x25Use event-based custom 1024-bit encryption with custom key derivation.

1024-bit (128-byte) block size.

VAULT_EM_CUSTOM256_DIRECT_KEY0x43Use event-based custom 256-bit encryption with no key derivation.

256-bit (32-byte) block size. Useful for cases where the password is an identifier for an external key and should not be used for key derivation.

VAULT_EM_CUSTOM512_DIRECT_KEY0x44Use event-based custom 512-bit encryption with no key derivation.

512-bit (64-byte) block size. Useful for cases where the password is an identifier for an external key and should not be used for key derivation.

VAULT_EM_CUSTOM1024_DIRECT_KEY0x45Use event-based custom 1024-bit encryption with no key derivation.

1024-bit (128-byte) block size. Useful for cases where the password is an identifier for an external key and should not be used for key derivation.

VAULT_EM_UNKNOWN0xFFUnidentified or unknown encryption.

Error Reporting and Handling

Error Codes

The CBFS Vault component APIs communicate errors using the error codes defined on the Error Codes page.

Reporting Errors to the Component from Event Handlers

If the event has a ResultCode parameter, the event handler can use it to return the result code of the operation to the component. The ResultCode parameter is set to 0 by default, which indicates the operation was successful.

If the event handler panics, the component will recover from an error and fire the OnError event.

In some events, the OS doesn't expect the error code to be returned and either the component or the OS ignores the returned error code. Please, refer to the description of a particular event for more information.

How to Handle Errors Reported by the Component

If an error occurs, the functions of the component will return an error structure. The code field of the structure will contain an error code, and the message field will contain an error message (if available).

File Features

The topics in this section provide information about file-related features.

Topics

Alternate Streams

Every file stored in a CBFS Vault vault contains a primary stream of data that holds the file's contents. In addition to this primary stream, files in a vault may also contain one or more alternate streams of data whose contents are determined by the application. By taking advantage of the flexibility that alternate streams offer, applications can support a wide range of use-cases; for example:

  • Storing a file's metadata and/or security information.
  • Saving supplementary information associated with a file (e.g., song lyrics for music files).
  • Maintaining a history of file content revisions.
  • Providing multiple representations of the same file (e.g., HTML, RTF, and plain versions of text content).

Alternate streams are addressed using names like <FileName>:<StreamName>, so an alternate stream named "altstream" could be addressed as \path\to\filename.ext:altstream. Alternate streams can be created and accessed just like files using OpenFile, OpenFileEx, DeleteFile, etc.; and can even be compressed and/or encrypted individually if an application desires.

To enumerate a file's alternate streams, call the FindFirst method with a mask like <FileName>:<StreamNameMask>. The <StreamNameMask> part can be * to enumerate all streams in a file. A file's main stream, which is always nameless, can be accessed explicitly using the name <FileName>: (note the trailing colon).

Compression

The CBFS Vault filesystem stores data in a vault as a series of one or more pages. To reduce space usage, CBFS Vault can compress files and alternate streams with an application-selected compression algorithm using the following mechanism, which is optimized to provide optimal performance for both sequential and random data access:

  1. A block of data composed of a specific number of pages is passed to the compression routine, which attempts to compress the data.
  2. If the compressed data can be stored using fewer pages than before, it is written to the vault. Otherwise, the original (uncompressed) data is written instead.
  3. Steps 1 and 2 are repeated until all of the data pages associated with the file or alternate stream have been processed.

Compressing Files and Alternate Streams

To specify a default compression mode for files and alternate streams, applications can set the DefaultFileCompression property (and, if applicable, the DefaultFileCompressionLevel configuration setting). Additionally, the following methods allow applications to set a file or alternate stream's compression mode explicitly:

Using Custom Compression

CBFS Vault includes built-in support for zlib and RLE data compression. However, applications can also choose to provide their own custom compression implementation using the DataCompress and DataDecompress events.

Supported Compression Modes

CBFS Vault supports the following compression modes:

VAULT_CM_NONE0Don't use compression.

VAULT_CM_DEFAULT1Use default compression (zlib).

VAULT_CM_CUSTOM2Use event-based custom compression.

Compression level is not used.

VAULT_CM_ZLIB3Use zlib compression.

Valid compression levels are 1-9.

VAULT_CM_RLE4Use RLE compression.

Compression level is not used.

File Tags

CBFS Vault allows applications to attach arbitrary metadata to any file, directory, or alternate stream using file tags. There are two kinds of file tags, both of which are stored as key-value pairs:

  1. Raw file tags use numeric Ids as keys and store raw binary data.
    • Valid Id values are those in the range 0x0001 to 0xCFFF (inclusive).
    • A tag should contain at least one (1) byte of data.
    • The maximum size of a raw file tag's binary data is 65531 bytes.
  2. Typed file tags use string keys and store typed values.
    • Names may be up to 4095 characters long (not including the null terminator), and are stored in UTF-16LE format internally.
    • The maximum size of a typed file tag's value is 65529 - (name_length * 2) bytes (where name_length is measured in characters, including the null terminator).

Each file, directory, and alternate stream can have up to 1024 typed file tags and 53247 raw file tags attached to it at once. The following methods are used to manage and interact with file tags:

Applications can also use the FindFirstByQuery method to search for files and directories whose file tags match a specified query; please refer to that method's documentation, as well as the Query Language topic, for more information. Note that the query language only works with typed file tags; it does not support raw file tags.

Query Language

In addition to searching by name, applications can search for files and directories based on their File Tags (metadata) using the CBFS Vault query language.

The query language includes a wide variety of Language Elements and supports all common Data Types. Search queries are interpreted as UTF-16LE strings, and may contain any valid arrangement of language elements, typed file tag names, and constants. For example:

  • From = 'John Smith': Selects all files received from John Smith.
    • From is the name of a file tag.
    • = is the equality operator (== is also supported).
    • 'John Smith' is a string constant.
  • SendData - Today > 3: Selects all files which were sent over 3 days ago.
    • SendData is the name of a file tag.
    • - is the subtraction operator.
    • Today is an intrinsic constant which returns the current system date.
    • > is the greater than operator.
    • 3 is a numeric constant.

When parsing an expression from a search query, the query engine converts all of its operands to the same data type using a specific set of rules; please refer to the Type Conversion topic for more information.

To find the first match for a query, call the FindFirstByQuery method, passing the desired search query for the Query parameter; and then call FindNext to find other matches, if necessary. Be sure to call FindClose when finished so that the component can release the resources allocated for the search operation.

Please refer to the File Tags topic for more information about how to manage file tags, and please refer to the other topics in this section for more information about the query language:

Language Elements

The CBFS Vault query language supports a wide variety of language elements, all of which are described below.

Logical Operators

Operator Operand type(s) Description
NOT, !, ~ Boolean Logical negation (NOT)
NOT, !, ~ Number Bitwise NOT
AND, & Boolean Logical AND
AND, & Number Bitwise AND
OR, | Boolean Logical OR
OR, | Number Bitwise OR

Arithmetic Operators

Operator Operand type(s) Description
+ Number, DateTime Addition
+ String String concatenation
- Number Negation
- Number, DateTime Subtraction
* Number Multiplication
/ Number Division (attempting to divide by zero will cause an exception)

Addition and subtraction operations involving DateTime operands behave as follows:

  • When adding a Number (n) and a DateTime, the result is a DateTime whose value has increased by n whole days.
  • When subtracting a Number (n) from a DateTime, the result is a DateTime whose value has decreased by n whole days.
  • When subtracting a DateTime from another DateTime, the result is a Number which reflects the difference as a number of whole days. The query evaluator converts both operands to whole days before performing the subtraction; "leftover" time is truncated as part of the conversion.

Relational Operators

Operator Operand type(s) Description
=, == All types Equal to
<>, != All types Not equal to
< All types Less than
> All types Greater than
<= All types Less than or equal to
>= All types Greater than or equal to

Conditions

Condition Operand type Description
IS [NOT] NULL All types Returns True if the value is/isn't NULL, and False otherwise.
IS [NOT] True Boolean Returns True if the value is/isn't True, and False otherwise.
IS [NOT] False Boolean Returns True if the value is/isn't False, and False otherwise.
[NOT] LIKE '...' [ESCAPE '...'] String Returns True if the value does/doesn't match the specified pattern; see notes below.

Keep the following in mind when using the LIKE condition:

  • Two kinds of wildcards are supported: %, which matches a string of any length; and _, which matches any single character. For example:
    • From LIKE '% Smith': Selects all files received from people with the last name "Smith".
    • From LIKE 'John Sm_th': Selects all files received from people with the first name "John" and a last name that is five characters long, begins with "Sm", and ends with "th" (Smith, Smyth, Smeth, etc.).
  • To search for values which include wildcard characters, the optional ESCAPE parameter can be used to specify a wildcard escape character. For example:
    • From LIKE 'John!_Smith' ESCAPE '!': Selects all files received from "John_Smith".
    • From LIKE 'John!_%' ESCAPE '!': Selects all files received from a name that begins with "John_".

File Variables

File variables represent some piece of information about the current file the query is being evaluated against.

Variable Type Description
FileName String The name of the current file.
FullName String The fully-qualified name of the current file, starting from the root directory /.
Path String The full path to the current file, including the final path separator (not including the file name).
IsFile Boolean True if the current file is not a directory, and False otherwise.
IsDirectory Boolean True if the current file is a directory, and False otherwise.
IsLink Boolean True if the current file is a symbolic link, and False otherwise.
LinkDestination String If the current file is a symbolic link, the link's target; otherwise acts the same as FullName.
CreationTime DateTime The current file's creation date and time.
LastAccessTime DateTime The current file's last access date and time.
ModificationTime DateTime The current file's last modification date and time.
Size Number The size of the current file (always 0 for directories).
Attributes Number The current file's attribute, encoded as a number.
IsEncrypted Boolean True if the current file is encrypted, and False otherwise.
IsCompressed Boolean True if the current file is compressed, and False otherwise. May be True for directories that contain files compressed by default.

Intrinsics

"Intrinsics" are the functions and constants built into the query language.

Intrinsic Operand type(s)Return typeDescription
D(value) String DateTime Converts a String to a DateTime; please refer to the Type Conversion topic for more information.
IsNull(value) All types Boolean Returns True if the value is NULL, and False otherwise.
IsNotNull(value) All types Boolean Returns True if the value is not NULL, and False otherwise.
Min(value1, value2) All types All types Returns the smaller of the two values.
Max(value1, value2) All types All types Returns the larger of the two values.
Now DateTime Returns the current system date and time.
Today DateTime Returns the current system date.
True Boolean Boolean True.
False Boolean Boolean False.

Precedence

The table below lists the query language's elements in order of descending precedence. Any legal expression within a query string may be surrounded with parentheses () in order to override precedence or increase readability.

PrecedenceLanguage Elements
1

All File Variables

All Intrinsics (except D(value); see note)

2

-: Arithmetic negation

NOT, !, ~: Logical/bitwise negation

D(value): Explicit String to DateTime conversion

3

*: Multiplication

/: Division

4

+: Addition/string concatenation

-: Subtraction

5

=, ==: Equal to

<>, !=: Not equal to

<: Less than

>: Greater than

<=: Less than or equal to

>=: Greater than or equal to

IS [NOT], [NOT] LIKE: All Conditions

6AND, &: Logical/bitwise AND
7OR, |: Logical/bitwise OR

Note: The query engine treats the D(value) function as an operator, so its precedence is lower than the other intrinsics.

Data Types

The CBFS Vault query language supports the following operand data types:

Type Description
NULL Empty value. Operations with NULL operand(s) always result with NULL.
Boolean Boolean; either False or True (and False < True).
String String of UTF-16LE (2-byte Unicode) characters.
DateTime Describes the date and time.
Number Signed 64-bit integer.

Type Conversion

When CBFS Vault parses a query, it will attempt to convert operands to the same type before evaluating an expression. The right-hand operand is converted to match the type of the left-hand operand if possible; otherwise, if the right-hand operand is of the String type, the left-hand operand is converted to String.

Supported Data Type Conversions

Convert To Convert From Notes
String Number, Boolean, NULL Typical conversion rules apply; Boolean values become "True" or "False".
String DateTime The format string used by the conversion is YYYY-MM-DD hh:mm:ss.fff.
Boolean String "True" and "False" are the recognized string values.
DateTime String The parsing pattern used by the conversion is YYYY[-]MM[-]DD[[tT ]hh[[:]mm[[:]ss[.fff]]]]; see notes below.
Number String The conversion recognizes numbers formatted as signed base-10 integers.

In addition to the implicit conversion mentioned above, a String can be converted to a DateTime explicitly using the intrinsic function D(value). The implicit and explicit conversions both use the parsing pattern shown above, which has a number of optional parts:

  • The date separators - may be omitted if the month and day are both two-digit values. They must both be present if the month and/or day is a single-digit value.
  • The time portion may be omitted; if present, it must be specified as one of the following: hours only; hours and minutes; hours, minutes, and seconds; or hours, minutes, seconds, and milliseconds.
    • The time separators : may be omitted if all included time elements are two-digit values. They must be present if any time elements are single-digit values.
    • Milliseconds, if present, must always be separated by a . character, and must always be a three-digit value.
  • When the time portion is present, it may immediately follow the date portion (i.e., with no separator), or it may be separated from the date portion using a T, a t, or a single space.

Vaults

What is a Vault?

The key functionality CBFS Vault provides is the ability to create and store an entire filesystem (complete with files, directories, metadata, and much more) in a standalone container called a vault.

A vault is typically stored as a real file on a local disk (similar to, e.g., an SQLite database file), but applications can technically store it using any data location by using Callback Mode.

Internally, a vault's storage space is divided up into chunks of equal size called pages. A vault's page size is specified at creation-time, and cannot be changed later. Applications do not have direct access to vault pages, but awareness of their existence is helpful for understanding certain component APIs.

Please refer to the other topics in this section for more information about vaults:

Multipart Vaults

CBFS Vault is capable of storing a single vault across multiple files on disk; this is known as a multipart vault. To create a multipart vault, set the PartSize configuration setting to a non-zero value before the vault is created. CBFS Vault will automatically create, resize, and delete individual part files as necessary over time (please refer to the Vault Size topic for more information).

Multipart vaults are typically used by applications which operate in environments with file size constraints. For example, if an application needed to store a 16GB vault on a FAT32 filesystem, it could use a multipart vault with a 4GB part size.

Existing vaults cannot be converted between multipart and non-multipart, and a multipart vault's size cannot be changed after creation. Also, multipart vaults are not supported in Callback Mode (since it already gives applications full control over how/where a vault is stored); the PartSize configuration setting is simply ignored.

Using RootData

All CBFS Vault vaults contain a special data stream called RootData which can be used for application-defined purposes. A vault's RootData stream can only be accessed using the OpenRootData method, because the stream itself is not part of the vault's filesystem hierarchy. The standalone nature of the RootData stream means that:

The RootData stream is also exempt from whole-vault encryption. This exemption is intentional; it allows applications which utilize whole-vault encryption to store information about said encryption (e.g., encrypted session keys, certificates, access control lists (ACLs), etc.) within the vault itself, thus simplifying application design.

Vault Corruption

CBFS Vault vaults have a complex internal structure which may become corrupted if a vault is not closed properly or if some operation is interrupted. Typically, such things are caused by an application crash, a system crash, or (when operating in Callback Mode) due to an error in an event handler. Corruption can also occur if a vault's raw data is modified externally, either intentionally or due to storage failure.

When a vault is open, the IsCorrupted property can be queried to determine if has been corrupted. If a vault is corrupted, any operation may fail with a VAULT_ERR_VAULT_CORRUPTED error code.

Applications can attempt to fix a corrupted vault by calling the CheckAndRepair method. Always create a vault backup before calling CheckAndRepair, as it's possible for data to be lost during the repair process in cases of severe corruption.

Journaling

To reduce the chances of vault corruption in the event of a crash, CBFS Vault can make use of journaling. Journaling works by wrapping vault modification operations in transactions, as follows:

  1. A new transaction is opened by writing information about a change to a journal located within the vault.
  2. The changes themselves are written to the vault.
  3. The transaction is committed by writing another entry in the journal.

If a crash occurs, any interrupted modification operations will appear in a vault's journal as pending transactions. The next time CBFS Vault opens the vault, it will discover any pending transactions and automatically to recover them. During the transaction recovery process, each transaction is either committed or rolled back, depending on its last known state.

Overall, journaling is an effective technique for maintaining data integrity. However, keep the following considerations in mind:

  • When journaling is enabled, all file data changes incur additional write operations; this has a significant impact on overall write performance.
  • Journaling does not provide any kind of data redundancy or consistency; it cannot protect against corruption caused by bit-rot, storage failures, or external modification of a vault's physical data.

CBFS Vault implements journaling as an operational mode rather than a vault attribute, so there is no such thing as a "journaled vault" or a "non-journaled vault". Applications control whether journaling mode is used by setting the JournalingMode parameter of the OpenVault methodwhen opening a vault. Therefore, the same vault might be opened with journaling enabled at one point, and opened without journaling enabled at another point.

The filesystem engine will always perform transaction recovery when a vault is opened (if there are pending transactions in its journal), even if journaling is disabled.

Vault Size

By default, a vault grows automatically as more data is written to it, and shrinks automatically when its free space percentage reaches the threshold defined by the AutoCompactAt property.

Applications can use the following properties to both control, and obtain information about, a vault's size. Please refer to each one's documentation for more information.

  • VaultSizeMax: Specifies the maximum size a vault can be; 0 (unlimited) by default.
  • VaultSizeMin: Specifies the minimum size a vault can be; 0 by default.
  • VaultSize: Reflects a vault's actual size; and can also be used to explicitly resize a vault, keeping in mind the following:
    • A vault cannot shrink more than its available free space allows (i.e., not by more than VaultFreeSpace bytes).
    • A vault cannot shrink beyond VaultSizeMin bytes.
    • If VaultSizeMax is not 0 (unlimited), a vault cannot grow beyond VaultSizeMax bytes.
    • If a vault grows enough to reach/exceed its AutoCompactAt threshold, it will automatically shrink again when the next automatic compaction occurs.
  • VaultFreeSpace: Reflects the actual amount of free space a vault has available.
  • PossibleSize: Reflects the maximum size a vault could possibly be.
  • PossibleFreeSpace: Reflects the maximum amount of free space a vault could possibly have available.

Constants

All constants are defined in the cbfsvault package.

Error Codes

VAULT_ERR_INVALID_VAULT_FILE -1 The specified file is not a CBFS Vault vault.

VAULT_ERR_INVALID_PAGE_SIZE -2 The specified page size is not valid.

VAULT_ERR_VAULT_CORRUPTED -3 The vault is corrupted.

VAULT_ERR_TOO_MANY_TRANSACTIONS -4 Too many transactions active.

VAULT_ERR_FILE_ALREADY_EXISTS -5 A file, directory, symbolic link, or alternate stream with the specified name already exists.

VAULT_ERR_TRANSACTIONS_STILL_ACTIVE -6 One or more transactions are still active.

VAULT_ERR_TAG_ALREADY_EXISTS -7 The specified file tag already exists.

VAULT_ERR_FILE_NOT_FOUND -8 The specified file, directory, symbolic link, or alternate stream was not found.

VAULT_ERR_PATH_NOT_FOUND -9 The specified path was not found.

VAULT_ERR_SHARING_VIOLATION -10 The specified file or alternate stream is already open in an exclusive access mode.

VAULT_ERR_SEEK_BEYOND_EOF -11 Cannot seek beyond the end of a file or alternate stream.

VAULT_ERR_NO_MORE_FILES -12 There are no other files, directories, symbolic links, or alternate streams that match the search criteria.

VAULT_ERR_INVALID_FILE_NAME -13 The specified name is not valid.

VAULT_ERR_VAULT_ACTIVE -14 The requested operation cannot be performed while a vault is open.

VAULT_ERR_VAULT_NOT_ACTIVE -15 A vault must be open before the requested operation can be performed.

VAULT_ERR_INVALID_PASSWORD -16 The specified password is incorrect.

VAULT_ERR_VAULT_READ_ONLY -17 The requested operation cannot be performed; the vault is open in read-only mode.

VAULT_ERR_NO_ENCRYPTION_HANDLERS -18 Can't use custom encryption; no custom encryption event handlers provided.

VAULT_ERR_OUT_OF_MEMORY -19 Out of memory.

VAULT_ERR_SYMLINK_DESTINATION_NOT_FOUND -20 A symbolic link's destination file could not be found.

VAULT_ERR_FILE_IS_NOT_SYMLINK -21 The specified file is not a symbolic link.

VAULT_ERR_BUFFER_TOO_SMALL -22 The specified buffer is too small to hold the requested value.

VAULT_ERR_BAD_COMPRESSED_DATA -23 Decompression failed (possible due to corruption).

VAULT_ERR_INVALID_PARAMETER -24 Invalid parameter.

VAULT_ERR_VAULT_FULL -25 The vault is full (and cannot be automatically resized).

VAULT_ERR_INTERRUPTED_BY_USER -26 Operation interrupted by user.

VAULT_ERR_TAG_NOT_FOUND -27 The specified file tag was not found.

VAULT_ERR_DIRECTORY_NOT_EMPTY -28 The specified directory is not empty.

VAULT_ERR_HANDLE_CLOSED -29 The file or alternate stream was closed unexpectedly; the handle is no longer valid.

VAULT_ERR_INVALID_STREAM_HANDLE -30 Invalid file or alternate stream handle.

VAULT_ERR_FILE_ACCESS_DENIED -31 Access denied.

VAULT_ERR_NO_COMPRESSION_HANDLERS -32 Can't use custom compression; no custom compression event handlers provided.

VAULT_ERR_NOT_IMPLEMENTED -33 Not implemented in this version of CBFS Vault.

VAULT_ERR_DRIVER_NOT_INSTALLED -35 The CBFS Vault system driver has not been installed.

VAULT_ERR_NEW_VAULT_VERSION -37 The specified vault cannot be opened, it was created using a newer version of CBFS Vault.

VAULT_ERR_FILE_IS_NOT_DIRECTORY -38 The specified file is not a directory.

VAULT_ERR_INVALID_TAG_DATA_TYPE -39 The specified file tag data type is not valid.

VAULT_ERR_VAULT_FILE_DOES_NOT_EXIST -40 The specified vault storage file does not exist.

VAULT_ERR_VAULT_FILE_ALREADY_EXISTS -41 The specified vault storage file already exists.

VAULT_ERR_CALLBACK_MODE_FAILURE -42 Some callback mode event handler has returned an unidentified error.

VAULT_ERR_EXTERNAL_ERROR -43 External library could not be initialized or used.

File Attributes

VAULT_FATTR_FILE 0x00000001 The entry is a file.

VAULT_FATTR_DIRECTORY 0x00000002 The entry is a directory.

VAULT_FATTR_DATA_STREAM 0x00000004 The entry is an alternate data stream.

VAULT_FATTR_COMPRESSED 0x00000008 The file or stream is compressed.

VAULT_FATTR_ENCRYPTED 0x00000010 The file or stream is encrypted.

VAULT_FATTR_SYMLINK 0x00000020 The entry is a symbolic link.

VAULT_FATTR_READONLY 0x00000040 The file is read-only.

This attribute is not used by CBFS Vault, but it can be set and retrieved.

VAULT_FATTR_ARCHIVE 0x00000080 The file requires archiving.

This attribute is not used by CBFS Vault, but it can be set and retrieved.

VAULT_FATTR_HIDDEN 0x00000100 The file is hidden.

This attribute is not used by CBFS Vault, but it can be set and retrieved.

VAULT_FATTR_SYSTEM 0x00000200 The file is a system file.

This attribute is not used by CBFS Vault, but it can be set and retrieved.

VAULT_FATTR_TEMPORARY 0x00000400 The file is temporary.

This attribute is not used by CBFS Vault, but it can be set and retrieved.

VAULT_FATTR_DELETE_ON_CLOSE 0x00000800 The file should be deleted when the last handle to the file is closed.

This attribute is currently not supported by CBFS Vault.

VAULT_FATTR_RESERVED_0 0x00001000 Reserved.

VAULT_FATTR_RESERVED_1 0x00002000 Reserved.

VAULT_FATTR_RESERVED_2 0x00004000 Reserved.

VAULT_FATTR_RESERVED_3 0x00008000 Reserved.

VAULT_FATTR_NO_USER_CHANGE 0x0000F03F A mask which includes all attributes that cannot be changed.

Applications cannot use the SetFileAttributes method to directly change any of the following attributes: FILE, DIRECTORY, DATA_STREAM, COMPRESSED, ENCRYPTED, SYMLINK, RESERVED_0, RESERVED_1, RESERVED_2, RESERVED_3.

VAULT_FATTR_USER_DEFINED 0x7FF00000 A mask for application-defined attributes.

Applications can use the SetFileAttributes method to set custom attributes, so long as their values are covered by this mask.

VAULT_FATTR_ANY_FILE 0x7FFFFFFF A mask which includes any and all attributes.

Check and Repair Flags

VAULT_CR_CHECK_ONLY 0x00000001 Check only, don't attempt any repairs.

VAULT_CR_CHECK_ALL_PAGES 0x00000002 Check all vault pages, including empty ones.

(When this flag is not present, only the vault pages which are marked as occupied are checked.)

Format Flags

VAULT_FMF_FAST_FORMAT 0x00000001 Perform a fast format; only initialize the pages necessary for storing the filesystem structure.

When this flag is not set, all pages of the new vault are initialized.

Vault Journaling Modes

VAULT_JM_NONE 0 No journaling is used.

This mode ensures the fastest operations but if the application crashes, corruption of the vault is possible.

VAULT_JM_METADATA 1 Journaling is used only for metadata (filesystem structure and directory contents).

This mode is a balance between speed and reliability.

VAULT_JM_FULL 2 Journaling is used for both filesystem structure and file data and metadata.

This is the slowest but the most reliable option.

Search Flags

VAULT_FF_NEED_NAME 0x00000001 Include entry names (without paths) when returning search results.

VAULT_FF_NEED_FULL_NAME 0x00000002 Include fully-qualified entry names when returning search results.

VAULT_FF_NEED_ATTRIBUTES 0x00000004 Include entry attributes when returning search results.

VAULT_FF_NEED_SIZE 0x00000008 Include entry sizes when returning search results.

VAULT_FF_NEED_METADATA_SIZE 0x00000010 Include entry metadata sizes when returning search results.

VAULT_FF_NEED_TIMES 0x00000020 Include entry times when returning search results.

VAULT_FF_NEED_LINK_DEST 0x00000040 Include symbolic link destinations when returning search results.

VAULT_FF_EMULATE_FAT 0x00001000 Inserts . and .. pseudo-entries into search results for all directories except the root one.

VAULT_FF_RECURSIVE 0x00002000 Search recursively in all subdirectories.

VAULT_FF_CASE_INSENSITIVE 0x00004000 Forces case-insensitive search, even if the vault is case-sensitive.

Vault Open Modes

VAULT_OM_CREATE_NEW 0 Creates a new vault if possible, failing if one already exists.

VAULT_OM_CREATE_ALWAYS 1 Creates a new vault, overwriting an existing one if necessary.

VAULT_OM_OPEN_EXISTING 2 Opens a vault if it exists; fails otherwise.

VAULT_OM_OPEN_ALWAYS 3 Opens a vault if it exists; creates a new one otherwise.

Vault State Flags

VAULT_ST_FIXED_SIZE 0x00000001 The vault is fixed-size.

VAULT_ST_READ_ONLY 0x00000002 The vault was opened in read-only mode.

Please refer to the ReadOnly property for more information.

VAULT_ST_CORRUPTED 0x00000004 The vault is corrupted.

Applications can use the CheckAndRepair method to try to repair vault corruption. Please refer to the Vault Corruption topic for more information.

VAULT_ST_TRANSACTIONS_USED 0x00000008 The vault was opened in journaling mode.

Please refer to the UseJournaling property for more information.

VAULT_ST_ACCESS_TIME_USED 0x00000010 Last access times are being tracked.

Please refer to the UseAccessTime property for more information.

VAULT_ST_ENCRYPTED 0x00000020 The vault is encrypted with whole-vault encryption.

Please refer to the Encryption topic for more information.

VAULT_ST_VALID_PASSWORD_SET 0x00000040 The correct whole-vault encryption password has been provided.

Please refer to the Encryption topic for more information.

VAULT_ST_PARTED 0x00000100 The vault's contents are split across multiple files on disk.

Please refer to the Multipart Vaults topic for more information.

Tag Data Types

VAULT_TDT_RAWDATA 0x0 The tag is untyped and must be addressed by Id.

VAULT_TDT_BOOLEAN 0x1 The tag contains Boolean data and must be addressed by name.

VAULT_TDT_STRING 0x2 The tag contains String (UTF-16LE) data and must be addressed by name.

VAULT_TDT_DATETIME 0x3 The tag contains DateTime data and must be addressed by name.

VAULT_TDT_NUMBER 0x4 The tag contains numeric (signed 64-bit) data and must be addressed by name.

VAULT_TDT_ANSISTRING 0x5 The tag contains AnsiString (8-bit string) data and must be addressed by name.

Path Separator Characters

VAULT_PSC_BACKSLASH 92 Backslash ('\\').

Windows path separator.

VAULT_PSC_SLASH 47 Forward slash ('/').

Unix-style path separator.

Compression Modes

VAULT_CM_NONE 0 Don't use compression.

VAULT_CM_DEFAULT 1 Use default compression (zlib).

VAULT_CM_CUSTOM 2 Use event-based custom compression.

Compression level is not used.

VAULT_CM_ZLIB 3 Use zlib compression.

Valid compression levels are 1-9.

VAULT_CM_RLE 4 Use RLE compression.

Compression level is not used.

Encryption Modes

VAULT_EM_NONE 0x0 Don't use encryption.

VAULT_EM_DEFAULT 0x1 Use default encryption (VAULT_EM_XTS_AES256_PBKDF2_HMAC_SHA256).

VAULT_EM_XTS_AES256_PBKDF2_HMAC_SHA256 0x2 Use AES256 encryption with PBKDF2 key derivation based on a HMAC_SHA256 key hash.

VAULT_EM_CUSTOM256_PBKDF2_HMAC_SHA256 0x3 Use event-based custom 256-bit encryption with PBKDF2 key derivation based on a HMAC_SHA256 key hash.

256-bit (32-byte) block size.

VAULT_EM_CUSTOM512_PBKDF2_HMAC_SHA256 0x4 Use event-based custom 512-bit encryption with PBKDF2 key derivation based on a HMAC_SHA256 key hash.

512-bit (64-byte) block size.

VAULT_EM_CUSTOM1024_PBKDF2_HMAC_SHA256 0x5 Use event-based custom 1024-bit encryption with PBKDF2 key derivation based on a HMAC_SHA256 key hash.

1024-bit (128-byte) block size.

VAULT_EM_CUSTOM256_CUSTOM_KEY_DERIVE 0x23 Use event-based custom 256-bit encryption with custom key derivation.

256-bit (32-byte) block size.

VAULT_EM_CUSTOM512_CUSTOM_KEY_DERIVE 0x24 Use event-based custom 512-bit encryption with custom key derivation.

512-bit (64-byte) block size.

VAULT_EM_CUSTOM1024_CUSTOM_KEY_DERIVE 0x25 Use event-based custom 1024-bit encryption with custom key derivation.

1024-bit (128-byte) block size.

VAULT_EM_CUSTOM256_DIRECT_KEY 0x43 Use event-based custom 256-bit encryption with no key derivation.

256-bit (32-byte) block size. Useful for cases where the password is an identifier for an external key and should not be used for key derivation.

VAULT_EM_CUSTOM512_DIRECT_KEY 0x44 Use event-based custom 512-bit encryption with no key derivation.

512-bit (64-byte) block size. Useful for cases where the password is an identifier for an external key and should not be used for key derivation.

VAULT_EM_CUSTOM1024_DIRECT_KEY 0x45 Use event-based custom 1024-bit encryption with no key derivation.

1024-bit (128-byte) block size. Useful for cases where the password is an identifier for an external key and should not be used for key derivation.

VAULT_EM_UNKNOWN 0xFF Unidentified or unknown encryption.

File Open Modes

VAULT_FOM_CREATE_NEW 0 Creates a new file or alternate stream if possible, failing if one already exists.

VAULT_FOM_CREATE_ALWAYS 1 Creates a new file or stream, overwriting an existing one if necessary.

VAULT_FOM_OPEN_EXISTING 2 Opens a file or stream if it exists; fails otherwise.

VAULT_FOM_OPEN_ALWAYS 3 Opens a file or stream if it exists; creates a new one otherwise.

Progress Operations

VAULT_PO_FORMATTING 0 Formatting a vault.

VAULT_PO_CHECKING_1 1 Checking a vault (stage 1).

VAULT_PO_CHECKING_2 2 Checking a vault (stage 2).

VAULT_PO_CHECKING_3 3 Checking a vault (stage 3).

VAULT_PO_CHECKING_4 4 Checking a vault (stage 4).

VAULT_PO_CHECKING_5 5 Checking a vault (stage 5).

VAULT_PO_PAGE_CORRUPTED 8 Processing a corrupted vault page.

VAULT_PO_PAGE_ORPHANED 9 Processing an orphaned vault page.

VAULT_PO_COMPRESSING 10 Compressing a file or alternate stream.

VAULT_PO_DECOMPRESSING 11 Decompressing a file or alternate stream.

VAULT_PO_ENCRYPTING 12 Encrypting a vault, file, or alternate stream.

VAULT_PO_DECRYPTING 13 Decrypting a vault, file, or alternate stream

VAULT_PO_COMPACTING 14 Compacting a vault.

VAULT_PO_RESIZING 15 Resizing a vault.

VAULT_PO_CALCULATING_SIZE 16 Calculating a vault's size.

VAULT_PO_COPYING_FILES_TO_VAULT 17 Copying files to a vault.

VAULT_PO_COPYING_FILES_FROM_VAULT 18 Copying files from a vault.

Copy Flags

VAULT_CFF_OVERWRITE_NONE 0x00000000 Never overwrite destination files.

VAULT_CFF_OVERWRITE_IF_NEWER 0x00000001 Overwrite a destination file only if the source file is newer.

VAULT_CFF_OVERWRITE_ALL 0x00000002 Always overwrite destination files.

VAULT_CFF_INCLUDE_SUBDIRS_WITH_CONTENTS 0x00010000 Include all subdirectories in source directory, and their contents, recursively.

VAULT_CFF_INCLUDE_SUBDIRS_NO_CONTENTS 0x00020000 Include all subdirectories in the source directory, without their contents.

VAULT_CFF_FIRE_COPY_EVENTS 0x40000000 Fire events related to file copying.

When the flag is set, the component fires the FileBeforeCopy and FileAfterCopy events.

Copyright (c) 2023 Callback Technologies, Inc. - All rights reserved.
CBFS Vault 2022 Go Edition - Version 22.0 [Build 8460]