Deployment

The topics in this section provide information regarding the deployment of applications built with CBFS Storage.

Topics

• Windows-specific section
• User Mode Library

Windows-specific

The topics in this section provide information relevant to the deployment of applications built with the components that include a kernel-mode driver or make use of a third-party kernel driver: CBVaultDrive or CBMemoryDrive.

This information should be reviewed carefully when designing a deployment strategy for such an application, since CBFS Storage's kernel mode drivers and other supplementary DLLs must be distributed along with the application in order for it to function correctly.

Topics

• Prerequisites
• Driver Installation in Windows

Prerequisites

The components create a virtual drive, visible to other processes, which requires a certain level of integration between the components and the system itself. In order for an application that uses such component to function correctly, the following prerequisites must first be met on the target machine:

• Windows: The kernel mode drivers must be installed; please refer to the Driver Installation in Windows topic for more information.
• macOS: macFUSE together with its "FUSE Compatibility Layer" must be installed in the target system (including the final end-users' systems). Note that the product does not include macFUSE, it must be downloaded and installed separately.
• Linux: The system's kernel must have been compiled with support for FUSE. Also, FUSE 2.9 user-mode libraries must be installed in the development system. This can be achieved using the following commands:
  • RedHat/CentOS and derivative Linux distributions: sudo yum install fuse-devel
  • Debian/Ubuntu and derivative Linux distributions: sudo apt-get install libfuse-dev
  FUSE 2 must be installed in target systems, where your application is deployed. Modern versions of Linux do not include FUSE 2 by default, but it can be installed using these commands:
  • RedHat/CentOS and derivative Linux distributions: sudo yum install fuse-libs
  • Debian/Ubuntu and derivative Linux distributions: sudo apt-get install libfuse2

Driver Installation in Windows

This topic describes the functionality, available in CBVaultDrive and CBMemoryDrive components when the component is used in Windows operating system.

At a high level, CBFS Storage consists of a kernel mode driver, a helper DLL, and a user mode library; all of which work together in tandem to provide the product's functionality. Therefore, it is necessary to install the CBFS Storage kernel mode driver and helper DLL when deploying an application built with the CBFS Storage user mode library.

The functionality needed to install the above-mentioned modules is included in the user mode library itself, as well as in a separate installer DLL. The drivers directory, located within the product's installation directory, contains the following files:

Windows: Note: When the user-mode library is installed or updated on end-user systems, it is required to ensure that the kernel-mode drivers already present in the system are updated to match the version of the installed user-mode library.

NuGet Notes

If CBFS Storage is installed using NuGet, the drivers directory described above will be located within NuGet's global-packages directory at the following path (assuming NuGet's global-packages directory has not been changed): %USERPROFILE%\.nuget\packages\callback.cbfsstorage\24.0.xxxx\contentFiles\any\any\drivers.

Installation and Uninstallation via User Mode Library Methods

The component includes the following methods to install and uninstall the required files; please refer to their documentation for more information:

• Install
• Uninstall
• GetDriverStatus
• GetModuleVersion
• RegisterIcon
• UnregisterIcon

Important: Uninstall must only be used when completely removing the driver. When updating the driver, this method must not be used as it may cause the OS to incorrectly remove the driver on reboot. Please refer to the "Updating the Driver" section, below, for more information.

Installation and Uninstallation via Installer DLL Functions

The installer DLL is a lightweight, stand-alone library that contains only the functionality required for installing and uninstalling the required files. It is available in both 32-bit and 64-bit versions (each of which is capable of installing both 32-bit and 64-bit drivers and helper DLLs); and may be used as desired in installation scripts, setup applications, or any other executable capable of loading dynamically-linked libraries (DLLs).

The functions exposed by the installer DLL mirror the component methods listed above. Each function is available in two forms: those with an *A suffix, which can be used with ANSI/UTF8 strings; and those with a *W suffix, which can be used with Unicode (UTF16) strings.

Updating the Driver

To update the driver, call the Install method. The new version of the driver will replace the older version. Please do not call the Uninstall method when updating the driver.

Uninstalling the Driver

To uninstall the driver completely, call the Uninstall method. If the driver cannot be immediately uninstalled, it will be marked for removal and uninstalled on the next reboot.

Use caution when calling Uninstall ; if it gets called and the driver cannot be uninstalled immediately, and then Install is subsequently called to install a new version, then upon reboot, the OS will end up uninstalling the newly-installed driver.

Important: The driver should only be uninstalled when the intent is to completely remove it from the system. Do not uninstall the driver to update it.

Reboot Requirements

Depending on the current state of the system, as well as the options chosen when installing or uninstalling the driver, the OS may need to reboot to complete the operation.

For example, the helper DLL must be loaded by Windows File Explorer when it starts, and a reboot or restart of Explorer is required for this to occur. When installing or uninstalling the Plug-and-Play (PnP) drivers, a reboot is almost always requested by Windows.

Always check the return value of the Install and Uninstall methods/functions; it will indicate whether a reboot is required (and if so, which module(s) required it).

Additional Notes

The OS treats major versions of the driver as separate products; they can operate in parallel and do not share any resources. Old major versions may optionally be removed from the system when calling Install by passing the appropriate value for its Flags parameter.

For each major version of the product, only one copy of the driver can be installed at any time. When the driver is being installed, its version is checked, and one of the following three things occurs:

• If no driver with the same major version is currently installed, then the install procedure installs the driver as a new product.
• If a driver with the same major version and an older minor version is currently installed, then the install procedure updates the existing driver with the new one.
• If a driver with the same major version and a newer minor version is currently installed, then the install procedure leaves the existing driver unchanged.

When deploying files to a target system, the CAB file must remain present on the system. This file is required for uninstallation of the driver at a later time.

The product's installation code maintains a ProductGUID-based record of driver installations in the Windows Registry, creating a separate registry entry for each different ProductGUID. When the driver is "uninstalled", the corresponding registry entry is removed. The driver is only removed from the system if there are no entries left in the registry that reference the driver.

Windows 7 and Windows 2008 Server R2

Kernel-mode drivers are signed using the SHA2 algorithm. The original releases of Windows 7 and Windows 2008 Server R2 didn't support SHA2. To be able to load the newest versions of the drivers, the system needs to have certain updates installed. The updates are KB976932 (Service Pack 1 of the mentioned systems) and KB4474419 (Security Update).

Required Permissions

By default, Windows only allows installation and uninstallation of the CBFS Storage system files (kernel mode drivers and helper DLLs) to be performed from a user account which is a member of the Administrators group.

On systems where UAC is enabled, the process responsible for installing or uninstalling the system files must run with elevated permissions. Detection of current privileges and elevation of permissions is not within the scope of the component itself.

Some examples of obtaining the required permissions for driver installation and uninstallation are below.

• Starting the application which uses the component with the "Run as administrator" option.
• Modifying the Load and unload device drivers setting in the Local Security Policy under the User Rights Assignment section.
• Including a manifest alongside the application indicating the requirement for elevated permissions. For instance, if a file MyApp.exe.manifest with the content below exists next to the application MyApp.exe, it will prompt for elevated permissions when started (if required).

  <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0"> <assemblyIdentity version="1.0.0.0" processorArchitecture="X86" name="ExeName" type="win32"/> <description>elevate execution level</description> <trustInfo xmlns="urn:schemas-microsoft-com:asm.v2"> <security> <requestedPrivileges> <requestedExecutionLevel level="requireAdministrator" uiAccess="false"/> </requestedPrivileges> </security> </trustInfo> </assembly>

User Mode Library

Windows Only: The user-mode library must be deployed to end-user systems along with the kernel-mode drivers; the version of the kernel-mode drivers on the end-user systems must be equal to or newer than the version of the user-mode library. Thus, when the user-mode library is installed or updated on end-user systems, it is required to ensure that the kernel-mode drivers already present in the system are updated to match the version of the installed user-mode library.

The user-mode library comes in two pieces, both of which must be deployed along with the application:

1. A .NET assembly (managed), named callback.CBFSStorage.dll (or callback.CBFSStorage.NetStd.dll, for .NET Standard).
2. A native dynamic library (unmanaged), named cbfsstorage24.dll, available for both 32-bit (x86) and 64-bit (x64, ARM64) processor architectures.

When deploying the application, copy both the .NET assembly and the native library to the target system and place them next to the application's executable file (on Windows, it has the .exe extension).

Alternatively, the native library may be placed into one of directories, the paths to which are contained in the

• Windows: PATH environment variable, such as C:\Windows\System32 (or C:\Windows\SysWOW64 when deploying a 32-bit application on a 64-bit Windows system)
• Linux: LD_LIBRARY_PATH environment variable
• macOS: DYLD_LIBRARY_PATH environment variable

Windows:

The .NET assembly may be deployed to the Global Assembly Cache (GAC).

Loading of Native Library

Usually, .NET finds the native library when it is placed next to the assembly. But on some systems, it skips the directory of the assembly or even of the main application module. To ensure that this doesn't happen on a user's system when an application keeps the native library next to the .NET assembly, in the .NET Core 3.1 and .NET 5+ applications, you can use the DllImportResolver function as shown in the following snippet: public static IntPtr CBVaultDriveImportResolver(string libraryName, Assembly assembly, DllImportSearchPath? searchPath) { bool loaded = false; IntPtr libHandle = IntPtr.Zero; if (!libraryName.Contains("callback.CBFSStorage")) return IntPtr.Zero; //Look for the native library next to the .NET assembly loaded = NativeLibrary.TryLoad(libraryName, assembly, DllImportSearchPath.AssemblyDirectory, out libHandle); //Look for the native library in the application's main directory if (!loaded) loaded = NativeLibrary.TryLoad(libraryName, assembly, DllImportSearchPath.ApplicationDirectory, out libHandle); //Look for the native library using teh default .NET behavior if (!loaded) loaded = NativeLibrary.TryLoad(libraryName, assembly, searchPath, out libHandle); return loaded ? libHandle : IntPtr.Zero; } ... NativeLibrary.SetDllImportResolver(Assembly.GetAssembly(typeof(CBVaultDrive)), CBVaultDriveImportResolver); This code attempts to load the native library in the directory of the managed .NET assembly, and if not found then attempts to find the library in the application's main directory, and if not found then revert to the default .NET behavior. Only one DllImportResolver function may be set for the assembly, so doing the above for one component is sufficient.

In the .NET Framework 4.x applications, one can use AppDomain.CurrentDomain.AssemblyResolve event for the same purpose.

NuGet Notes

After the NuGet package is added to a project, both the managed .NET assembly and the unmanaged native library will be copied to the project's output directory anytime the project is built. However, the exact files copied to the output directory for the native library will vary based on the project type:

• For .NET Core projects, a runtimes directory will be created in the output directory (if it does not already exist), and versions of the native library for each supported runtime identifer (RID) (e.g., win-x64) will be placed in the appropriate subdirectories. When the .NET Core application is distributed, the entire runtimes directory should be deployed alongside it.
• For other types of projects (.NET Framework, UWP), only the native library version specific to the currently selected platform target (e.g., x64) will be copied to the output directory.
  • For .NET Framework projects specifically, please note that the project's platform target (Project > Properties > Build Tab > Platform target) must be set to a real architecture. If it is set to "Any CPU", no native library will be copied to the output directory.

Windows:

The native library may alternatively be installed to the Windows system directory. This approach allows deploying both the 32-bit and 64-bit versions of the native library simultaneously, because each gets placed into the system directory that corresponds to the appropriate processor architecture.

Windows Only: Remember to deploy the drivers too, as they are an integral part of CBFS Storage.

General Information

The topics in this section provide general information about various aspects of the product's functionality.

Topics

• Buffer Parameters
• Callback Mode
• Encryption
• Error Handling
• Windows-specific section
• File Features section
• Query Language section
• Vaults section

Buffer Parameters

Some events include one or more parameters intended for use as a binary data buffer. Depending on the event, these parameters may contain data when the event is fired, or it may be expected that the application populates them with the desired amount of data during the event handler. Some events combine both paradigms and then expect the application to modify the data already present when the event is fired.

The documentation for such events will describe which of these cases applies to each buffer parameter. In all cases, buffer parameters point to a preallocated block of unmanaged memory, the size of which is specified by the parameter immediately following the buffer parameter. In cases in which data are to be written, be sure to write it directly to the pointed-to memory, do not change the value of the buffer parameter itself.

Buffer parameters are always of the IntPtr type; use the .NET Marshal.Copy() method to read and write data from and to the unmanaged memory region. When targeting newer .NET versions, such as .NET Standard 2.1 and later, applications can use the Span<T> and ReadOnlySpan<T> classes to access and modify the unmanaged memory region without extra data copying.

Callback Mode

As discussed in the Vaults topic, the default behavior of CBVaultDrive and CBMemoryDrive components is to create a vault using a real file on a local disk. However, the filesystem engine behind these components does not require a vault to be a local file: it can be a remote file, a memory region, or anything else that the application can provide random read/write access to.

Applications that wish to use something other than a local file to store a vault must enable callback mode using the component's CallbackMode property. When callback mode is enabled, applications must handle the following events (which map closely to the Windows File API) for the component to interact with the vault. For brevity, vaults created and accessed using callback mode are typically referred to as "callback mode vaults".

• VaultClose: Fires when the currently open vault should be closed.
• VaultDelete: Fires when a callback mode vault (that is not open) should be deleted.
• VaultFlush: Fires when any buffered vault data should be flushed out to storage.
• VaultGetParentSize: Fires when the component needs to know how much free space is available for the currently open vault to use for automatic growth.
• VaultGetSize: Fires when the component needs to know the size of the currently-open vault.
• VaultOpen: Fires when a callback mode vault should be opened (and, if necessary, created).
• VaultRead: Fires when the component needs to read one or more pages of vault data.
• VaultSetSize: Fires when the component needs to resize (i.e., shrink or grow) the currently open vault.
• VaultWrite: Fires when the component needs to write one or more pages of vault data.

Callback mode is an extremely powerful feature for applications that want to fine-tune performance. For example, consider the following scenario: a help-authoring tool that keeps a compound file in memory for fast operations. In this scenario, the vault would be a help project that the help-authoring tool loads into memory and uses the above events to access. When a user needs to save the project, the vault is flushed and the data in memory are copied back to the help project on disk.

Note: An application should not attempt to call component's methods from handlers of the listed events. Doing this is guaranteed to cause a deadlock.

Encryption

The CBVaultDrive, CBMemoryDrive, and CBVAULT components include strong built-in data encryption support, which can be applied to individual files and alternate streams, entire vaults, or both. Each file, alternate stream, and vault can have its own encryption key.

Note: The API members discussed in this topic are available in all listed components, unless otherwise noted.

Encrypting Vaults

To specify a default encryption mode and password to use when creating new vaults, applications can set the VaultEncryption and VaultPassword properties. To change the encryption mode or password of an existing vault, use the UpdateVaultEncryption method.

When opening an existing vault, VaultEncryption is updated to reflect the vault's encryption mode; and if the vault is encrypted, the password specified by VaultPassword is used to access it.

Encrypting Files and Alternate Streams

To specify a default encryption mode and password for files and alternate streams, applications can set the DefaultFileEncryption and DefaultFilePassword properties. Additionally, the following methods allow applications to set a file or alternate stream's encryption mode or password explicitly:

• OpenFile (when creating a new file or alternate stream)
• OpenFileEx (when creating a new file or alternate stream)
• SetFileEncryption
• CopyToVault (CBVAULT component only)

When a file or alternate stream is encrypted, its encryption password must be provided to access it; many methods in the component's API provide a Password parameter for this purpose. If the application does not explicitly specify a password when calling such a method, then the DefaultFilePassword will be used, if possible.

Using Custom Encryption

The component's built-in encryption implementation uses 256-bit AES encryption in XTS mode with PBKDF2 key derivation based on a HMAC-SHA256 key hash. However, applications also can choose to provide their own custom encryption and key derivation implementations. This flexibility allows applications to support more sophisticated security techniques, such as PKI-based encryption or Digital Rights Management. To get started, do the following:

1. Choose a custom encryption mode to implement (i.e., one of the VAULT_EM_CUSTOM* options from the table below). This choice will determine:
   • Whether the custom encryption implementation uses a 256-bit, 512-bit, or 1024-bit block size; and,
   • Whether to use built-in key derivation, custom key derivation, or no key derivation.
2. Implement the DataEncrypt and DataDecrypt events.
3. If a VAULT_EM_CUSTOM*_CUSTOM_KEY_DERIVE mode was chosen, implement the KeyDerive event.
4. If a VAULT_EM_CUSTOM*_DIRECT_KEY mode was chosen, implement the HashCalculate event.

Supported Encryption Modes

The component supports the following encryption modes:

Error Handling

Error Codes

The CBFS Storage component APIs communicate errors using the error codes defined in their Error Codes pages (available for each component). The CBVaultDrive component also communicate errors using OS-specific error codes; for example, on Windows they use Win32 error codes defined in WinError.h, which is part of the Windows Platform SDK.

Reporting Errors to the Component from Event Handlers

If an event has a ResultCode parameter, an event handler can use it to return the result code of the operation to the component. The ResultCode parameter is set to 0 by default, which indicates the operation was successful.

If an unhandled exception occurs in the event handler, it will be caught by the component, which will fire the OnError event.

In some events, the OS does not expect the error code to be returned and either the component or the OS ignores the returned error code. Please refer to the description of a particular event for more information.

How to Handle Errors Reported by the Component

If an error occurs, the component will throw an exception. The Code property of the exception object will contain an error code, and the Message property will contain an error message (if available).

Extended Logging in Windows (Driver-based Components Only)

Some component methods in CBFS Storage are capable of writing extended information about reported errors to the Windows event logs, which can be viewed using the system's eventvwr.exe tool. The user mode part of the component writes to the "Windows Logs \ Application" folder, while the kernel mode part writes to the "Windows Logs \ System" folder.

The information written in the extended logs is meaningful to the Callback Technologies development team, but not to end-users, so extended logging is disabled by default. If issues occur during the installation of the CBFS Storage system drivers, or while using the component, please do the following:

1. Enable extended logging (see below).
2. Replicate the issue.
3. Using Event Viewer (eventvwr.exe), export the event log entries from the locations mentioned above in native format (please restrict the scope of the export to just those entries related to CBFS Storage).
4. Submit an issue report that includes the exported file.

There are two ways to toggle extended logging for a component:

1. By toggling the component's LoggingEnabled configuration setting.
2. By adding a DWORD-typed value named Enabled to the HKEY_LOCAL_MACHINE\SOFTWARE\Callback Technologies\{ComponentName}\EventLog registry key and setting it to 0 (disabled) or 1 (enabled).
   • Replace the {ComponentName} part of the registry key path with the name of the applicable component.
   • If this registry key, one of its parents, or the actual value does not exist, please create it manually.

   Note: If your code runs in emulated mode (x86 mode on x64 or ARM64 architecture), you need to add the value to the HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Callback Technologies\{ComponentName}\EventLog registry key in addition to the "main" registry key.

The system must be rebooted anytime extended logging is enabled or disabled to make the changes take effect.

Windows-specific

The topics in this section provide information specific to the CBVaultDrive and CBMemoryDrive components on Windows.

Topics

• Custom Drive Icons
• Mounting Points
• Helper DLL
• Threading and Concurrency
• Event Handling
• PID Re-use
• Troubleshooting

Custom Drive Icons

This topic describes the functionality, available in CBVaultDrive and CBMemoryDrive components when the component is used in Windows operating system.

Virtual drives created with the CBVaultDrive component can have a custom icon associated with them to better distinguish them in Windows File Explorer. There are a few different ways to accomplish this:

Using Additional Files

If placing additional files into the virtual drive itself is an acceptable condition, Windows provides a couple of file-based mechanisms for specifying a custom icon.

To specify a custom icon for the virtual drive itself, an autorun.inf file can be created based on the information in Microsoft's Autorun.inf article.

Additionally, custom icons can be specified for subdirectories of the virtual drive using desktop.ini files, which can be created based on the information in Microsoft's Desktop.ini article. Note that desktop.ini files cannot be used to specify a custom icon for the root directory of the virtual drive (i.e., they cannot be used to change the icon of the virtual drive itself).

Using Registry Keys

If the virtual drive is assigned a persistent drive letter, using registry keys to assign a custom icon may be a good option. To specify a custom icon using the registry, create a subkey like {DriveLetter}\DefaultIcon (e.g., K\DefaultIcon) under one of the following keys:

• HKEY_CURRENT_USER\SOFTWARE\Classes\Applications\Explorer.exe\Drives, if the custom icon should only be used for the current user.
• HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\DriveIcons, if the custom icon should be used for all users. (Note that manipulating anything under the HKEY_LOCAL_MACHINE registry hive requires administrative rights.)

Regardless of which key it is created under, the {DriveLetter}\DefaultIcon subkey's (Default) (i.e., "unnamed") value should then be set to the absolute path of the icon file.

Please note that custom icons specified in this manner are only effective so long as the drive letter assigned to the virtual drive remains unchanged over time; if its drive letter changes, the registry keys used to specify the custom icon will need to be updated accordingly.

Using the Component and its Shell Helper DLL

As long as the Helper DLL has been installed to the system using the Install method, custom icons can be assigned to a virtual drive directly using the component. This method of specifying custom icons is especially valuable when project constraints preclude placing additional files into the drive or modifying the registry.

Custom icons assigned in this manner function a bit differently than those assigned using the two methods described above, as they are implemented using Windows' icon overlay mechanism. Consequently, the custom icons are restricted to 25% of the original icon's area (except for 16x16 icons); the tables below describe the required sizes and color levels of the assets in the icon file.

Overlay icon sizes map as follows:

Icon assets must have the following color levels:

Because it's possible to specify multiple different overlay icons (e.g., to represent different drive states), icons are assigned through the component using a two-step process:

1. Register the desired icon(s) using the RegisterIcon method. (Note that administrative rights are required to execute this method successfully.)
2. Switch between the registered icon(s) using the SetIcon and ResetIcon methods.

Icons are copied to a temporary location when registered; and removed from said location when unregistered using the UnregisterIcon method.

It is important to keep in mind that Windows limits the number of registered overlay icons to 15 (this is a global limit for the entire system, and it cannot be changed). Since other applications on the system (e.g., OneDrive, Dropbox, etc.) may have registered multiple overlay icons, it's not uncommon to get into a situation where various applications are competing to have their overlay icons registered.

Overlay icons are registered by placing values in the following keys in the Registry:

• HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\ShellIconOverlayIdentifiers
• HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\explorer\ShellIconOverlayIdentifiers (64-bit Windows only)

If necessary, it's up to the application (or better yet, the user) to decide whether or not to remove other entries; however, doing so too aggressively will likely have a negative impact on the user's experience with other applications.

Mounting Points

A mounting point is a name that can be used to access a volume. When the filesystem driver mounts a volume, it must make that volume accessible by creating one or more mounting points for it.

Windows:

Mounting points can be global (visible in all user sessions) or local (visible only to a specific user session). The AddMountingPoint method creates global mounting points by default; applications must include the STGMP_LOCAL flag in the Flags parameter value to create local mounting points. (Note: The STGMP_MOUNT_MANAGER flag is not compatible with the STGMP_LOCAL flag.)

When creating a local mounting point, applications can specify a specific user session for it to be visible in by passing that session's Authentication ID for the AuthenticationId parameter (retrieval of Authentication IDs is discussed in a later section). If no Authentication ID is provided (i.e., 0 is passed), the local mounting point is created in the current user session; and if the application does this while running with elevated rights, then the local mounting point will only be visible in the elevated session, and consequently won't be available to applications in other sessions (such as, e.g., Windows File Explorer).

When mounting points are added or removed, a system message (WM_DEVICECHANGE) is broadcast. It instructs Windows File Explorer to refresh the list of drives available. However, these messages cannot cross user session boundaries; so if, for example, the application is running as a service, Windows File Explorer may not receive the broadcast and thus fail to refresh the list of drives. To address this issue, CBFS Storage includes a Helper DLL which, among other things, helps ensure that Windows File Explorer always refreshes the list of drives regardless of which user session the application is running in; please refer to that topic for more information.

Types of Mounting Points

There are a handful of different mounting point types, each of which exposes volumes in a slightly different manner:

• Drive letter mounting points
• Folder mounting points
• Network mounting points
• UNC path mounting points

Each type of mounting point is discussed in more detail below.

Drive Letter Mounting Points

Drive letter mounting points are one of the more commonly-used mounting point types thanks to users' familiarity with them. To create a drive letter mounting point, pass a string composed of a single character in the A-Z range followed by a colon (e.g., Z:) for the AddMountingPoint method's MountingPoint parameter.

If the value passed for the AddMountingPoint method's Flags parameter includes the STGMP_AUTOCREATE_DRIVE_LETTER flag, the component will assign a drive letter automatically. In this case, the value passed for the MountingPoint parameter must not include a drive letter.

Folder Mounting Points

A folder mounting point makes a volume accessible through a folder located on another (pre-existing) NTFS volume. Folder mounting points are always visible to all users in the system, and their creation requires administrative privileges.

To create a folder mounting point using the AddMountingPoint method, include the STGMP_MOUNT_MANAGER flag in the Flags parameter , and pass the target folder's absolute path for the MountingPoint parameter (e.g., C:\MountedDrives\MyMountingPoint). The target folder must already exist, must reside on an NTFS volume, and must be empty; otherwise, the call will fail.

Authentication IDs

An Authentication ID is a locally unique identifier (LUID) assigned to a logon session (or, "user session"), retrievable through the access token that represents said session. Applications can obtain the Authentication ID of a session from an access token or by enumerating logon sessions.

To obtain an Authentication ID from an access token, call the Windows API's GetTokenInformation function and pass either TokenGroupsAndPrivileges or TokenStatistics for the TokenInformation parameter. The resulting value will be a reference to a structure (TOKEN_GROUPS_AND_PRIVILEGES or TOKEN_STATISTICS, respectively) containing the needed Authentication ID.

To enumerate logon sessions, use the Windows API's LsaEnumerateLogonSessions function, which returns a list of existing logon session IDs (that is, Authentication IDs). To obtain additional information about a particular logon session (e.g., in order to determine if it's the desired one), use the Windows API's LsaGetLogonSessionData function. Network Mounting Points

Network mounting points are similar to other mounting point types, except that the system treats them as "remote devices". This distinction is useful since:

• Windows File Explorer makes fewer requests for files located on remote devices.
• Some applications are more tolerant of timeouts and delays when working with remote devices.

Therefore, when an application is designed to work with some slow or remote storage medium, it's recommended that it use a network mounting point. When using network mounting points, it's important that the Helper DLL be used so that Windows File Explorer displays the correct drive status.

To create a network mounting point using the AddMountingPoint method, include the STGMP_NETWORK flag in the Flags parameter, and pass a string of the form <Local Name>;<Server Name>;<Share Name> for the MountingPoint parameter.

• <Local Name> is the name to use for the mounting point on the local system; it can be a drive letter or a name for use in a UNC path. Alternatively, it can be left empty, in which case the volume will only be accessible via the network path (see below) or the drive letter will be assigned automatically if the STGMP_AUTOCREATE_DRIVE_LETTER flag is used.
  • Note: This "local name" is not related to the concept of "local and global mounting points" discussed in the overview.
• <Server Name> and <Share Name> are used to create a network path of the form \\<Server Name>\<Share Name>. This network path is not shared by default (see notes following examples below).

The set of characters allowed in server names, is defined in this document. The set of characters allowed in share names, is defined in this document.

With the above information in mind, here are some examples of valid MountingPoint parameter values when creating network mounting points:

• Y:;MyServer;VirtualShare: Creates a network mounting point accessible both via the drive letter Y: and via the network path \\MyServer\VirtualShare.
• MyMountingPoint;MyServer;VirtualShare: Creates a network mounting point accessible both via the UNC path \\.\MyMountingPoint and via the network path \\MyServer\VirtualShare
• ;MyServer;VirtualShare: Creates a network mounting point accessible only via the network path \\MyServer\VirtualShare.

As stated above, the network paths created for network mounting points are not shared (i.e., visible to other computers on the network) by default. To have the component create an actual network share when AddMountingPoint is called, applications must include either the STGMP_NETWORK_READ_ACCESS or the STGMP_NETWORK_WRITE_ACCESS flag in the Flags parameter value, and use empty string for the <Server Name> segment of the MountingPoint parameter value (the local computer's name is used). Note that when the mounting point is shared in this way, a local resource is created and then shared. The name of the resource is derived from the Share Name defined above. However, the set of allowed characters for such name is not strictly defined. Additionally, sharing is done using a call to NetShareAdd Windows API function, which can be called by Administrators, System Operators, and Power Users.

UNC Path Mounting Points

UNC path mounting points make a volume available via a specific name, and unlike other mounting points types, they are not displayed anywhere in Windows File Explorer; the UNC path must already be known.

UNC path mounting points consist of the \\.\ prefix, followed by a name (e.g., \\.\CBDrive1). The mounting-point-related component methods expect just the name (i.e., the UNC path with the \\.\ prefix omitted). So to add a new UNC path mounting point like, e.g., \\.\CBDrive1, call the AddMountingPoint method and pass CBDrive1 for the MountingPoint parameter.

Linux and macOS:

A virtual drive / filesystem is mounted to a directory, which must exist at the time of mounting and be empty; otherwise, the call will fail.

Helper DLL

This topic describes the functionality, available in CBVaultDrive and CBMemoryDrive components when the component is used in Windows operating system.

The Helper DLL is integrated into Windows File Explorer and offers functionality designed to provide users with a consistent and pleasant experience. It is recommended that the Helper DLL be installed alongside the system driver, which can be accomplished by including the MODULE_HELPER_DLL flag when calling the Install method.

The Helper DLL is distributed in the same .cab file as the system driver; its name is CBVaultDriveShellHelper24.dll, and it is shipped in both 32-bit and 64-bit variants. Its functionality is described below.

Mounting Point Change Broadcasts

Anytime a mounting point is added or removed, the system driver will send a notification to the Helper DLL, which then broadcasts a system message instructing Windows File Explorer to refresh the list of drives. Without this functionality, Windows File Explorer will not refresh the list of drives if a mounting point is added or removed from a Windows service or another user session.

Network Mounting Point

When a network mounting point is used, the Helper DLL provides the functionality that allows Windows File Explorer to correctly display the current status of, and interact with, the virtual drive. Without this functionality, the virtual drive will display as "Disconnected", which may result in unexpected behavior.

Custom Icons

When custom icons are used for a virtual drive, the Helper DLL ensures that they are properly displayed in Windows File Explorer.

Threading and Concurrency

Through the use of multithreading, the CBVaultDrive component provides powerful concurrency features to help applications maximize their performance. For data integrity purposes, the component also strictly enforces the order in which events fire in certain situations, and allows applications to specify the extent to which events should be fired concurrently.

Please note that, even when configured for minimal concurrency, the component always fires events in the context of some worker thread, not in the thread the component was originally created on. Therefore, applications must be sure to synchronize operations between event handlers and other threads as necessary (including, but not limited to, calls to the component instance, unless a method is explicitly documented as callable within events).

Configuring Event Concurrency

Generally speaking, CBVaultDrive will always enforce per-file event serialization; that is, it always fires events relating to the same file in sequence (though technically-speaking, there is one optional exception to this behavior, discussed at the end of this section). For example, if there are multiple read or write operations pending against a given file, then an event will be fired for the first operation, and after its event handler has returned, another event will be fired for the second operation; and so on.

When the SerializeEvents property is set to seOnMultipleThreads, the MinWorkerThreadCount and MaxWorkerThreadCount configuration settings control the minimum and maximum number of worker threads the component can use for firing events. By default, both are set to 0, which indicates that the component's system driver should automatically choose appropriate values based on how many CPU cores the system has. These settings are both ignored if SerializeEvents is set to values other than seOnMultipleThreads.

Event Handling

In CBFS Storage, the purpose of most events is to let an application provide custom data storage, encryption, or compression routines. Events are also used to communicate the progress of long-running operations.

Since the CBDriveVault component's events are typically tied directly to requests from the OS or kernel-mode actors, it's critical that event handlers complete quickly in order to prevent the system from being blocked. To help prevent such blocking, the CBFS Storage system driver enforces request timeouts on a per-virtual-drive basis.

Any event handler has 30 seconds to finish its operations; after this, if an event handler didn't return, the driver completes the request handling and reports an error to the OS.

Events are easy to understand and to use, but keep the following things in mind when implementing an application's event handlers:

1. Event handlers must not perform any operations, explicitly or implicitly, against files opened in buffered mode. Any file accessed from an event handler must be opened using the Windows API's FILE_FLAG_NO_BUFFERING flag.
2. Event handlers must not perform any asynchronous procedure calls (APCs).
3. If the currently-open vault is stored in a file on an NTFS-formatted disk, neither the file containing the vault nor the disk said file resides on can be compressed or encrypted using NTFS features.

Events handlers which violate any of the restrictions described above will cause a system-wide deadlock. To ensure stable operation, it is critical to avoid accessing drives and filesystems recursively. Essentially, this means that event handlers must not perform any operations involving the drive or filesystem that the event fired for (i.e., don't read from/write to files on it, don't unmount the media, etc.).

PID Re-use

This topic describes the functionality, available in CBVaultDrive and CBMemoryDrive components when the component is used in Windows operating system.

When using various rules that are based on process IDs (PID), you need to be aware that Windows tends to reuse PID numbers. Once the process with a certain PID is finished in any way, Windows can re-use this PID for another process being started. And it does reuse PIDs quite frequently for the purpose of keeping PID numbers low.

Such reuse can cause unexpected and sometimes unpleasant consequence for your application. To counteract it, you can take one or both actions:

1. Open a handle to the process with the needed PID and not close it as long as your rule exists. Windows documentation states that as long as there exists an open handle to a process, its PID is not reused.
2. Track completion of the process with the given PID (either by monitoring the state of the process by its handle or using CBProcess component of the CBFS Filter product) and once the process is finished, delete the corresponding rule.

Troubleshooting

This topic describes the functionality, available in CBVaultDrive and CBMemoryDrive components when the component is used in Windows operating system.

Note: The below information applies to the operations of the component on Windows .

CBFS Storage is a complex product that operates in both user mode and kernel mode simultaneously; so when a serious issue occurs, it's critical that we are able to obtain sufficient information about the circumstances of the failure.

In order to help us assist you in a more expedient manner, please collect the information described in the instructions below when reporting a serious issue (i.e., one that causes the system to crash or hang). Our development team cannot effectively diagnose such issues without this information.

Also, please note that these sorts of issues commonly involve environmental differences and other factors that are either unforeseen or otherwise out of our control. It is also not unheard-of for a crash to appear attributable to one thing while in fact being caused by something completely different. Rest assured that we are committed to assisting you as best we can, and we thank you ahead-of-time for your patience and understanding throughout the support process.

System Crashes (BSODs)

If you encounter a consistently-reproducible system crash (BSOD) that you suspect may be due to CBFS Storage, please obtain a crash dump and include it when reporting the issue to us. Our development team is unable to diagnose system crashes without the information these dumps contain.

Ensure that your system is set up to generate crash dumps, and to not restart automatically after a crash, by following the steps found in Microsoft's Enabling a Kernel-Mode Dump File article. The options available in the memory dump dropdown vary depending on your version of Windows; please choose the first one from the following list that is present in yours:

• Complete
• Full
• Automatic
• Kernel

Once your system is set up to generate crash dumps, perform the same action that caused the BSOD originally to trigger the crash again. When it occurs, be sure to copy the information on the BSOD screen exactly so that it can be included in your submission (a picture of the screen in which all of the information is legible is also acceptable). Here are some examples of the specific information we're looking for:

Recent versions of Windows:

What failed: cb***24.sys Stop Code: FILE_SYSTEM

Older versions of Windows:

STOP: 0x00000022 (0x00240076, 0xF7A07AA8, 0xF7A077A8, 0xF7800C82) cb***24.sys - Address F7800C82 base at F77CD000, DateStamp 447d6975

After you've copied this information, reboot and check that the memory dump file was created at %SYSTEMROOT%\MEMORY.DMP (typically this is C:\Windows\MEMORY.DMP; if you changed the dump file location in the crash dump settings, check the location you specified instead). It will be a very large file that is too big to attach to an email, so please upload it to a file sharing site of your choice and generate a sharing link that our development team can use to download it.

Finally, submit a support issue to us that includes the link to your dump file, all of the information from the BSOD screen (if you took a picture, attach it or provide another sharing link), a description of how the BSOD was triggered, and any other information that you feel is relevant.

System Hangs

If you encounter a consistently-reproducible system hang that you suspect may be due to CBFS Storage, you'll need to collect the same information as described above. But in order to obtain a crash dump, you'll first need to configure your system so that you can trigger a crash from the keyboard once it hangs. To make this possible, follow these steps (adapted from Microsoft's Forcing a System Crash from the Keyboard article):

1. First, using the instructions provided in the section above, configure your system to generate crash dumps, and to not restart automatically after a crash.
2. Next, you must enable keyboard-initiated crashes in the registry by creating a new value named CrashOnCtrlScroll, and setting it equal to a REG_DWORD value of 0x01, in all of the following registry keys:
   • HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\i8042prt\Parameters
   • HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\kbdhid\Parameters
   • HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\hyperkbd\Parameters
3. Finally, you must restart the system in order for these settings to take effect.

After these steps are complete, you'll be able to trigger a keyboard-initiated crash by using the following hotkey sequence: hold down the Right CTRL key, and press the SCROLL LOCK key twice.

At this point, you can perform the same action that caused the system to hang originally to trigger the hang again. Once the system hangs, use the hotkey sequence to force it to crash, and then follow the rest of the instructions from the section above to collect and submit the necessary information.

Method Never Returns

If your application makes a call to some CBFS Storage method and that method never returns, there's a high chance that the driver deadlock has occurred. This can be caused by various factors; to determine the reason and possible solutions, we need a memory dump, as described above. If your system remains functional, you may also use easier ways to initiate a crash:

1. Use the NotMyFault tool by SysInternals to initiate a system crash. This is the preferred way because it generates Complete memory dumps. This tool has a self-explanatory GUI.
2. Use the LiveKD tool by SysInternals to create a live memory dump without crashing a system. The disadvantage of this method is that it creates Kernel dumps that are missing user-mode information. To use LiveKD on 32-bit systems, use this command: "livekd.exe -ml -o memory.dmp". On 64-bit systems, the command line would be "livekd64.exe -ml -k .\kd64.exe -o memory.dmp".

Application Crashes

Sometimes, an application crashes while the OS continues to operate, and the name of one of the modules of CBFS Storage is present in the crash information. A crashing application can be the one that uses CBFS Storage or some third-party process. If the crash occurs repeatedly, it is possible to make use of a User-Mode Crash Dump to locate or narrow down the source of the crash. Generation of crash dumps is disabled by default. Before you reproduce the crash, you need to Enable Collecting User-Mode Crash Dumps.

After you enable the crash dump, you don't need to reboot, you can proceed to reproduction of the crash immediately. After the crash re-occurs, you can pick the dump file from its location. The default locations of user-mode dump files are:

• For regular applications: %LOCALAPPDATA%\CrashDumps
• For System services: %WINDIR%\System32\Config\SystemProfile
• For Network and Local services: %WINDIR%\ServiceProfiles

If you changed the dump file location in the crash dump settings in the Registry, check the location you specified instead.

A crash dump can be a large file (depending on the settings) that is too big to attach to an email, so please upload it to a file sharing site of your choice and generate a sharing link that our development team can use to download it.

Finally, submit a support issue to us that includes the link to your dump file, a description of how the BSOD or a manual crash was triggered, and any other information that you feel is relevant.

File Features

The topics in this section provide information about file-related features.

Topics

• Alternate Streams
• Compression
• File Tags

Alternate Streams

Every file stored in a CBFS Storage vault contains a primary stream of data that holds the file's contents. In addition to this primary stream, files in a vault may also contain one or more alternate streams of data whose contents are determined by the application. By taking advantage of the flexibility that alternate streams offer, applications can support a wide range of use-cases, including the following:

• Storing a file's metadata or security information.
• Saving supplementary information associated with a file (e.g., song lyrics for music files).
• Maintaining a history of file content revisions.
• Providing multiple representations of the same file (e.g., HTML, RTF, and plain versions of text content).

Alternate streams are addressed using names like <FileName>:<StreamName>, so an alternate stream named "altstream" could be addressed as \path\to\filename.ext:altstream. Alternate streams can be created and accessed just like files using OpenFile, OpenFileEx, and DeleteFile, and can even be compressed or encrypted individually if an application desires.

To enumerate a file's alternate streams, call the FindFirst method with a mask like <FileName>:<StreamNameMask>. The <StreamNameMask> part can be * to enumerate all streams in a file. A file's main stream, which is always nameless, can be accessed explicitly using the name <FileName>: (note the trailing colon).

Compression

The CBFS Storage filesystem stores data in a vault as a series of one or more pages. To reduce space usage, CBFS Storage can compress files and alternate streams with an application-selected compression algorithm using the following mechanism, which is optimized to provide optimal performance for both sequential and random data access:

1. A block of data composed of a specific number of pages is passed to the compression routine, which attempts to compress the data.
2. If the compressed data can be stored using fewer pages than before, it is written to the vault. Otherwise, the original (uncompressed) data are written instead.
3. Steps 1 and 2 are repeated until all of the data pages associated with the file or alternate stream have been processed.

Compressing Files and Alternate Streams

To specify a default compression mode for files and alternate streams, applications can set the DefaultFileCompression property (and, if applicable, the DefaultFileCompressionLevel configuration setting). Additionally, the following methods allow applications to set a file or alternate stream's compression mode explicitly:

• OpenFileEx (when creating a new file or alternate stream)
• SetFileCompression
• CopyToVault

Using Custom Compression

CBFS Storage includes built-in support for zlib and RLE data compression. However, applications can also choose to provide their own custom compression implementation using the DataCompress and DataDecompress events.

Supported Compression Modes

CBFS Storage supports the following compression modes:

File Tags

CBFS Storage allows applications to attach arbitrary metadata to any file, directory, or alternate stream using file tags. There are two kinds of file tags, both of which are stored as key-value pairs:

1. Raw file tags use numeric Ids as keys and store raw binary data.
   • Valid Id values are those in the range 0x0001 to 0xCFFF (inclusive).
   • A tag should contain at least one (1) byte of data.
   • The maximum size of a raw file tag's binary data is 65531 bytes.
2. Typed file tags use string keys and store typed values.
   • Names may be up to 4095 characters long (not including the null terminator), and are stored in UTF-16LE format internally.
   • The maximum size of a typed file tag's value is 65529 - (name_length * 2) bytes (where name_length is measured in characters, including the null terminator).

Each file, directory, and alternate stream can have up to 1024 typed file tags and 53247 raw file tags attached to it at once. The following methods are used to manage and interact with file tags:

• DeleteFileTag
• FileTagExists
• GetFileTag, GetFileTagAsAnsiString, GetFileTagAsBoolean, GetFileTagAsDateTime, GetFileTagAsNumber, GetFileTagAsString
• GetFileTagDataType
• GetFileTagSize
• SetFileTag, SetFileTagAsAnsiString, SetFileTagAsBoolean, SetFileTagAsDateTime, SetFileTagAsNumber, SetFileTagAsString

Applications can also use the FindFirstByQuery method to search for files and directories whose file tags match a specified query; please refer to that method's documentation, as well as the Query Language topic, for more information.

Note: The query language works only with typed file tags; it does not support raw file tags.

Query Language

In addition to searching by name, applications can search for files and directories based on their File Tags (metadata) using the CBFS Storage query language.

The query language includes a wide variety of Language Elements and supports all common Data Types. Search queries are interpreted as UTF-16LE strings, and they may contain any valid arrangement of language elements, typed file tag names, and constants. For example:

• From = 'John Smith': Selects all files received from John Smith.
  • From is the name of a file tag.
  • = is the equality operator (== is also supported).
  • 'John Smith' is a string constant.
• SendData - Today > 3: Selects all files that were sent more than three days ago.
  • SendData is the name of a file tag.
  • - is the subtraction operator.
  • Today is an intrinsic constant that returns the current system date.
  • > is the greater than operator.
  • 3 is a numeric constant.

When parsing an expression from a search query, the query engine converts all of its operands to the same data type using a specific set of rules; please see the Type Conversion topic for more information.

To find the first match for a query, call the FindFirstByQuery method, passing the desired search query for the Query parameter; and then call FindNext to find other matches, if necessary. Be sure to call FindClose when finished so that the component can release the resources allocated for the search operation.

Please refer to the File Tags topic for more information about how to manage file tags and see the other topics in this section for more information about the query language:

• Data Types
• Language Elements
• Type Conversion

Language Elements

The CBFS Storage query language supports a wide variety of language elements, all of which are described below.

Logical Operators

Arithmetic Operators

Addition and subtraction operations involving DateTime operands behave as follows:

• When adding a Number (n) and a DateTime, the result is a DateTime whose value has increased by n whole days.
• When subtracting a Number (n) from a DateTime, the result is a DateTime whose value has decreased by n whole days.
• When subtracting a DateTime from another DateTime, the result is a Number that reflects the difference as a number of whole days. The query evaluator converts both operands to whole days before performing the subtraction; "leftover" time is truncated as part of the conversion.

Relational Operators

Conditions

Keep the following in mind when using the LIKE condition:

• Two kinds of wildcards are supported: %, which matches a string of any length; and _, which matches any single character. For example:
  • From LIKE '% Smith': Selects all files received from people with the last name "Smith".
  • From LIKE 'John Sm_th': Selects all files received from people with the first name "John" and a last name that is five characters long, begins with "Sm", and ends with "th" (e.g., Smith, Smyth, Smeth).
• To search for values that include wildcard characters, the optional ESCAPE parameter can be used to specify a wildcard escape character. For example:
  • From LIKE 'John!_Smith' ESCAPE '!': Selects all files received from "John_Smith".
  • From LIKE 'John!_%' ESCAPE '!': Selects all files received from a name that begins with "John_".

File Variables

File variables represent some piece of information about the current file the query is being evaluated against.

Intrinsics

"Intrinsics" are the functions and constants built into the query language.

Precedence

The following table lists the query language's elements in order of descending precedence. Any legal expression within a query string may be surrounded with parentheses () to override precedence or increase readability.

Note: The query engine treats the D(value) function as an operator, so its precedence is lower than the other intrinsics.

Data Types

The CBFS Storage query language supports the following operand data types:

Type Conversion

When CBFS Storage parses a query, it will attempt to convert operands to the same type before evaluating an expression. The right-hand operand is converted to match the type of the left-hand operand if possible; otherwise, if the right-hand operand is a String type, the left-hand operand is converted to String.

Supported Data Type Conversions

In addition to the implicit conversion mentioned above, a String can be converted to a DateTime explicitly using the intrinsic function D(value). The implicit and explicit conversions both use the parsing pattern shown above, which has a number of optional parts:

• The date separators - may be omitted if the month and day are both two-digit values. They must both be present if the month and/or day is a single-digit value.
• The time portion may be omitted; if present, it must be specified as one of the following: hours only; hours and minutes; hours, minutes, and seconds; or hours, minutes, seconds, and milliseconds.
  • The time separators : may be omitted if all included time elements are two-digit values. They must be present if any time elements are single-digit values.
  • Milliseconds, if present, must always be separated by a . character, and must always be a three-digit value.
• When the time portion is present, it may immediately follow the date portion (i.e., with no separator), or it may be separated from the date portion using a T, a t, or a single space.

Vaults

What Is a Vault?

The key functionality CBFS Storage provides is the ability to create and store an entire filesystem (complete with files, directories, metadata, and much more) in a standalone container called a vault.

A vault is typically stored as a real file on a local disk (similar to, e.g., an SQLite database file), but applications can technically store it using any data location by using Callback Mode.

Internally, a vault's storage space is divided into chunks of equal size called pages. A vault's page size is specified at creation time, and it cannot be changed later. Applications do not have direct access to vault pages, but awareness of their existence is helpful for understanding certain component APIs.

Vaults can be created and accessed using both the CBVAULT and CBVaultDrive components, but only the latter allows a vault to be mounted as a virtual drive. Please refer to the other topics in this section for more information about vaults:

• Multipart Vaults
• Using RootData
• Vault Corruption
• Vault Size

Multipart Vaults

CBFS Storage is capable of storing a single vault across multiple files on disk; this is known as a multipart vault. To create a multipart vault, set the PartSize configuration setting to a nonzero value before the vault is created. CBFS Storage will automatically create, resize, and delete individual part files as necessary over time (please refer to the Vault Size topic for more information).

Multipart vaults are typically used by applications that operate in environments with file size constraints. For example, if an application needed to store a 16 GB vault on a FAT32 filesystem, it could use a multipart vault with a 4 GB part size.

Existing vaults cannot be converted between multipart and non-multipart, and a multipart vault's size cannot be changed after creation. Also, multipart vaults are not supported in Callback Mode (because it already gives applications full control over how/where a vault is stored); the PartSize configuration setting is simply ignored.

Using RootData

All CBFS Storage vaults contain a special data stream called RootData that can be used for application-defined purposes. A vault's RootData stream can be accessed only using the OpenRootData method, because the stream is not part of the vault's filesystem hierarchy. The standalone nature of the RootData stream means the following:

• It does not have a path in the filesystem, which prevents it from being found using FindFirst or FindFirstByQuery, or accessed via a CBVaultDrive-based virtual drive.
• It cannot have any file tags, attributes, times, or alternate streams associated with it.
• It cannot be compressed using the SetFileCompression method.
• It cannot be encrypted using the SetFileEncryption method.

The RootData stream is also exempt from whole-vault encryption. This exemption is intentional; it allows applications that utilize whole-vault encryption to store information about that encryption (e.g., encrypted session keys, certificates, access control lists [ACLs]) within the vault, thus simplifying application design.

Vault Corruption

CBFS Storage vaults have a complex internal structure that may become corrupted if a vault is not closed properly or if some operation is interrupted. Typically, such things are caused by an application crash or a system crash, or (when operating in Callback Mode) as a result of an error in an event handler. Corruption can also occur if a vault's raw data are modified externally, either intentionally or because of storage failure.

When a vault is open, the IsCorrupted property can be queried to determine if has been corrupted. If a vault is corrupted, any operation may fail with a VAULT_ERR_VAULT_CORRUPTED error code.

Applications can attempt to fix a corrupted vault by calling the CheckAndRepair method. Always create a vault backup before calling CheckAndRepair, because in cases of severe corruption, it is possible for data to be lost during the repair process.

Journaling

To reduce the chances of vault corruption in the event of a crash, CBFS Storage can make use of journaling. Journaling works by wrapping vault modification operations in transactions, as follows:

1. A new transaction is opened by writing information about a change to a journal located within the vault.
2. The changes themselves are written to the vault.
3. The transaction is committed by writing another entry in the journal.

If a crash occurs, any interrupted modification operations will appear in a vault's journal as pending transactions. The next time CBFS Storage opens the vault, it will discover any pending transactions and automatically try to recover them. During the transaction recovery process, each transaction is either committed or rolled back, depending on its last known state.

Overall, journaling is an effective technique for maintaining data integrity. However, keep the following considerations in mind:

• When journaling is enabled, all file data changes incur additional write operations; this has a significant impact on overall write performance.
• Journaling does not provide any kind of data redundancy or consistency; it cannot protect against corruption caused by bit-rot, storage failures, or external modification of a vault's physical data.

CBFS Storage implements journaling as an operational mode rather than a vault attribute, and there is no such thing as a "journaled vault" or a "nonjournaled vault". Applications control whether the journaling mode is used by setting the JournalingMode parameter of the OpenVault method when opening a vault. Therefore, the same vault might be opened with journaling enabled at one point and may be opened without journaling enabled at another point.

The filesystem engine will always perform transaction recovery when a vault is opened (if transactions are pending in its journal), even if journaling is disabled.

Vault Size

By default, a vault grows automatically as more data are written to it, and it shrinks automatically when its free space percentage reaches the threshold defined by the AutoCompactAt property.

Applications can use the following properties to both control, and obtain information about, a vault's size. Please refer to each one's documentation for more information.

• VaultSizeMax: Specifies the maximum size a vault can be; 0 (unlimited) by default.
• VaultSizeMin: Specifies the minimum size a vault can be; 0 by default.
• VaultSize: Reflects a vault's actual size; and also can be used to explicitly resize a vault, keeping in mind the following:
  • A vault cannot shrink more than its available free space allows (i.e., not by more than VaultFreeSpace bytes).
  • A vault cannot shrink beyond VaultSizeMin bytes.
  • If VaultSizeMax is not 0 (unlimited), a vault cannot grow beyond VaultSizeMax bytes.
  • If a vault grows enough to reach or exceed its AutoCompactAt threshold, it will automatically shrink again when the next automatic compaction occurs.
• VaultFreeSpace: Reflects the actual amount of free space a vault has available.
• PossibleSize: Reflects the maximum size a vault could possibly be.
• PossibleFreeSpace: Reflects the maximum amount of free space a vault could possibly have available.

Note: For CBVaultDrive, the size of a vault backed by a storage volume or partition (i.e., one created or opened using the FormatVolume/OpenVolume methods) is identical to the size of said storage volume or the partition. Such vaults, once created, cannot be resized using any of the properties or methods discussed above.