Introduction

Welcome to CBFS Connect, an all-in-one solution for creating and managing virtual filesystems in Windows applications. CBFS Connect's robust and performant event-driven API can help you expose any data source as a fully featured Windows filesystem, including database records, remote files, dynamically generated content, or something else entirely. Without writing a single line of driver code, you will be able to present your data as "just another drive full of files and folders." This solution makes it easy for users to view and interact with the filesystem and also gives other applications the ability to access and manipulate it via standard file APIs.

Included Libraries

The .NET Edition includes several libraries (.dll files) which are supported in different environments. The following libraries are present in the lib directory after installation:

  • lib\callback.CBFSConnect.dll is designed for use in .NET Framework 4.0 and up. This is the default library which maintains a familiar API in line with previous versions of the product.

  • lib\net6.0\callback.CBFSConnect.dll is designed for use in .NET 6 and up. The .NET 6 library maintains the same API as the default library and can be used in projects that are based on .NET 6 and later.

  • lib\netstandard2.0\callback.CBFSConnect.dll is designed for use in .NET Standard 2.0 and up. The .NET Standard library maintains the same API as the default library and can be used in projects that are based on .NET Core and .NET 5 and later.

  • lib\net20\callback.CBFSConnect.dll is designed for use in .NET Framework 2.0 and up. This library targets .NET Framework 2.0 and is maintained for legacy projects.

Included Components

CBCacheThe CBCache component allows applications to easily cache remote file data locally.
CBFSThe CBFS component gives applications the ability to create a virtual filesystem.
FUSEThe FUSE component gives applications the ability to create a virtual filesystem using a FUSE-like API.

Additional Information

You will always find the latest information about CBFS Connect at our web site: www.callback.com. We offer free, fully-functional 30-day trials for all of our products, and our technical support staff are happy to answer any questions you may have during your evaluation.

Please direct all technical questions to support@callback.com. To help support technicians assist you as quickly as possible, please provide an detailed and accurate description of your problem, the results you expected, and the results that you received while using our product. For questions about licensing and pricing, and all other general inquiries, please contact sales@callback.com.

Thank You!

Thank you for choosing CBFS Connect for your development needs. We realize that you have a choice among development tools, and that by choosing us you are counting on us to be a key component in your business. We work around the clock to provide you with ongoing enhancements, support, and innovative products; and we will always do our best to exceed your expectations!

Deployment

The topics in this section provide information regarding the deployment of applications built with CBFS Connect.

Topics

Driver-specific

The topics in this section provide information relevant to the deployment of applications built with the components that include a kernel-mode driver or make use of a third-party kernel driver: CBFS or FUSE.

This information should be reviewed carefully when designing a deployment strategy for such an application, since CBFS Connect's kernel mode drivers and other supplementary DLLs must be distributed along with the application in order for it to function correctly.

Topics

Prerequisites

CBFS and FUSE components:

The components create a virtual drive, visible to other processes, which requires a certain level of integration between the components and the system itself. In order for an application that uses such component to function correctly, the following prerequisites must first be met on the target machine:

  • Windows: The kernel mode drivers must be installed; please refer to the Driver Installation topic for more information.
  • macOS: macFUSE together with its "FUSE Compatibility Layer" must be installed in the target system (including the final end-users' systems). Note that the product does not include macFUSE, it must be downloaded and installed separately.
  • Linux: The system's kernel must have been compiled with support for FUSE. Also, FUSE 2.9 user-mode libraries must be installed in the development system. This can be achieved using the following commands:
    • RedHat/CentOS and derivative Linux distributions: sudo yum install fuse-devel
    • Debian/Ubuntu and derivative Linux distributions: sudo apt-get install libfuse-dev
    FUSE 2 must be installed in target systems, where your application is deployed. Modern versions of Linux do not include FUSE 2 by default, but it can be installed using these commands:

    • RedHat/CentOS and derivative Linux distributions: sudo yum install fuse-libs
    • Debian/Ubuntu and derivative Linux distributions: sudo apt-get install libfuse2

Driver Installation

The topics in this section provide information regarding the deployment of applications built with CBFS or FUSE components on Windows. The information in these topics should be reviewed carefully when designing a deployment strategy, because CBFS Connect's kernel mode drivers and supplementary dynamic link libraries (DLLs) must be distributed along with the application for it to function correctly.

At a high level, CBFS consists of a kernel mode driver, a helper DLL, and a user mode library; all of which work together in tandem to provide the product's functionality. Therefore, it is necessary to install the CBFS kernel mode driver and helper DLL when deploying an application built with the CBFS Connect user mode library.

The functionality needed to install the above-mentioned modules is included in the user mode library itself, as well as in a separate installer DLL. The drivers directory, located within the product's installation directory, contains the following files:

{ComponentName}.cab Contains the main drivers, PnP bus drivers, helper DLLs, and the supplementary installation/uninstallation files.
installer/{ComponentName}Inst.h A header file for the installer DLL. The installer DLL may be used on the target system to install (or uninstall) the items within {ComponentName}.cab.
installer/x64/{ComponentName}Inst.dll The C/C++ installer DLL for the x64 (AMD64) processor architecture.
installer/x86/{ComponentName}Inst.dll The C/C++ installer DLL for 32-bit x86 processor architecture.
installer/ARM/{ComponentName}Inst.dll The C/C++ installer DLL for 32-bit ARM processor architecture.
installer/ARM64/{ComponentName}Inst.dll The C/C++ installer DLL for 64-bit ARM processor architecture.
*Note: The FUSE component uses the same driver as the CBFS component.

Windows: Note: When the user-mode library is installed or updated on end-user systems, it is required to ensure that the kernel-mode drivers already present in the system are updated to match the version of the installed user-mode library.

NuGet Notes

If CBFS Connect is installed using NuGet, the drivers directory described above will be located within NuGet's global-packages directory at the following path (assuming NuGet's global-packages directory has not been changed): %USERPROFILE%\.nuget\packages\callback.cbfsconnect\22.0.xxxx\contentFiles\any\any\drivers.

Installation and Uninstallation via User Mode Library Methods

The component includes the following methods to install and uninstall the required files; please refer to their documentation for more information:

Important: Uninstall must only be used when completely removing the driver. When updating the driver, this method must not be used as it may cause the OS to incorrectly remove the driver on reboot. Please refer to the "Updating the Driver" section, below, for more information.

Installation and Uninstallation via Installer DLL Functions

The installer DLL is a lightweight, stand-alone library that contains only the functionality required for installing and uninstalling the required files. It is available in both 32-bit and 64-bit versions (each of which is capable of installing both 32-bit and 64-bit drivers and helper DLLs); and may be used as desired in installation scripts, setup applications, or any other executable capable of loading dynamically-linked libraries (DLLs).

The functions exposed by the installer DLL mirror the component methods listed above. Each function is available in two forms: those with an *A suffix, which can be used with ANSI/UTF8 strings; and those with a *W suffix, which can be used with Unicode (UTF16) strings.

Updating the Driver

To update the driver, call the Install method. The new version of the driver will replace the older version. Please do not call the Uninstall method when updating the driver.

Uninstalling the Driver

To uninstall the driver completely, call the Uninstall method. If the driver cannot be immediately uninstalled, it will be marked for removal and uninstalled on the next reboot.

Use caution when calling Uninstall ; if it gets called and the driver cannot be uninstalled immediately, and then Install is subsequently called to install a new version, then upon reboot, the OS will end up uninstalling the newly-installed driver.

Important: The driver should only be uninstalled when the intent is to completely remove it from the system. Do not uninstall the driver to update it.

Reboot Requirements

Depending on the current state of the system, as well as the options chosen when installing or uninstalling the driver, the OS may need to reboot to complete the operation.

For example, the helper DLL must be loaded by Windows Explorer when it starts, and a reboot or restart of Explorer is required for this to occur. When installing or uninstalling the Plug-and-Play (PnP) drivers, a reboot is almost always requested by Windows.

Always check the return value of the Install and Uninstall methods/functions; it will indicate whether a reboot is required (and if so, which module(s) required it).

Additional Notes

The OS treats major versions of the driver as separate products; they can operate in parallel and do not share any resources. Old major versions may optionally be removed from the system when calling Install by passing the appropriate value for its Flags parameter.

For each major version of the product, only one copy of the driver can be installed at any time. When the driver is being installed, its version is checked, and one of the following three things occurs:

  • If no driver with the same major version is currently installed, then the install procedure installs the driver as a new product.
  • If a driver with the same major version and an older minor version is currently installed, then the install procedure updates the existing driver with the new one.
  • If a driver with the same major version and a newer minor version is currently installed, then the install procedure leaves the existing driver unchanged.

When deploying files to a target system, the CAB file must remain present on the system. This file is required for uninstallation of the driver at a later time.

The product's installation code maintains a ProductGUID-based record of driver installations in the Windows Registry, creating a separate registry entry for each different ProductGUID. When the driver is "uninstalled", the corresponding registry entry is removed. The driver is only removed from the system if there are no entries left in the registry that reference the driver.

Windows 7 and Windows 2008 Server R2

Kernel-mode drivers are signed using the SHA2 algorithm. The original releases of Windows 7 and Windows 2008 Server R2 didn't support SHA2. To be able to load the newest versions of the drivers, the system needs to have certain updates installed. The updates are KB976932 (Service Pack 1 of the mentioned systems) and KB4474419 (Security Update).

Required Permissions

By default, Windows only allows installation and uninstallation of the CBFS Connect system files (kernel mode drivers and helper DLLs) to be performed from a user account which is a member of the Administrators group.

On systems where UAC is enabled, the process responsible for installing or uninstalling the system files must run with elevated permissions. Detection of current privileges and elevation of permissions is not within the scope of the component itself.

Some examples of obtaining the required permissions for driver installation and uninstallation are below.

  • Starting the application which uses the component with the "Run as administrator" option.
  • Modifying the Load and unload device drivers setting in the Local Security Policy under the User Rights Assignment section.
  • Including a manifest alongside the application indicating the requirement for elevated permissions. For instance, if a file MyApp.exe.manifest with the content below exists next to the application MyApp.exe, it will prompt for elevated permissions when started (if required).

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0"> <assemblyIdentity version="1.0.0.0" processorArchitecture="X86" name="ExeName" type="win32"/> <description>elevate execution level</description> <trustInfo xmlns="urn:schemas-microsoft-com:asm.v2"> <security> <requestedPrivileges> <requestedExecutionLevel level="requireAdministrator" uiAccess="false"/> </requestedPrivileges> </security> </trustInfo> </assembly>

User Mode Library

Windows:

Note: (CBFS and FUSE on Windows only): The user-mode library must be deployed to end-user systems along with the kernel-mode drivers; the version of the kernel-mode drivers on the end-user systems must be equal to or newer than the version of the user-mode library. Thus, when the user-mode library is installed or updated on end-user systems, it is required to ensure that the kernel-mode drivers already present in the system are updated to match the version of the installed user-mode library.

The user-mode library comes in two pieces, both of which must be deployed along with the application:

  1. A .NET assembly (managed), named callback.cbfsconnect.dll (or callback.cbfsconnect.NetStd.dll, for .NET Standard).
  2. A native dynamic library (unmanaged), named cbfsconnect22.dll, available for both 32-bit (x86) and 64-bit (x64, ARM64) processor architectures.

When deploying the application, copy both the .NET assembly and the native library to the target system and place them next to the application's executable file (on Windows, it has the .exe extension).

Windows Only:

The .NET assembly may alternatively be deployed to the Global Assembly Cache (GAC).

Alternatively, the native dynamic library may be placed into one of directories, the paths to which are contained in the

  • Windows: PATH environment variable, such as C:\Windows\System32 (or C:\Windows\SysWOW64 when deploying a 32-bit application on a 64-bit Windows system)
  • Linux: LD_LIBRARY_PATH environment variable
  • macOS: DYLD_LIBRARY_PATH environment variable

NuGet Notes

After the NuGet package is added to a project, both the managed .NET assembly and the unmanaged native library will be copied to the project's output directory anytime the project is built. However, the exact files copied to the output directory for the native library will vary based on the project type:

  • For .NET Core projects, a runtimes directory will be created in the output directory (if it does not already exist), and versions of the native library for each supported runtime identifer (RID) (e.g., win-x64) will be placed in the appropriate subdirectories. When the .NET Core application is distributed, the entire runtimes directory should be deployed alongside it.
  • For other types of projects (.NET Framework, UWP), only the native library version specific to the currently selected platform target (e.g., x64) will be copied to the output directory.
    • For .NET Framework projects specifically, please note that the project's platform target (Project > Properties > Build Tab > Platform target) must be set to a real architecture. If it is set to "Any CPU", no native library will be copied to the output directory.

Windows:

The native library may alternatively be installed to the Windows system directory. This approach allows deploying both the 32-bit and 64-bit versions of the native library simultaneously, because each gets placed into the system directory that corresponds to the appropriate processor architecture.

For CBFS and FUSE components, Windows Only: Remember to deploy the drivers too, as they are an integral part of CBFS Connect.

General Information

The topics in this section provide general information about various aspects of the product's functionality.

Topics

CBFS Topics

The topics in this section provide additional information specific to the CBFS component.

Topics

Thread Safety of the API

The properties and methods provided by CBFS Connect's APIs fall into one of the following categories when it comes to thread safety:

  • Properties and methods that are intended to be accessed from event handlers (i.e., those documented as such) are thread-safe, since event handlers always execute in the context of worker threads.
  • Methods related to installation, uninstallation, and status retrieval are not thread-safe, since they are expected to be used during the application installation process. (Methods in this category are those which are documented as being "available in both the component API and the Installer DLL".)
  • Any helper methods documented in the component's API (i.e., those used to convert error codes, or provide similar supplementary functionality) don't use any explicit thread synchronization, but also don't access any other component properties or methods, and thus are inherently thread-safe.
  • All other properties and methods are not thread-safe, so applications must employ proper thread synchronization techniques when using them on multiple threads (including, but not limited to, during event handlers).

Event Handling

The topics in this section provide information about event handling.

Note: When events are fired, the system and the kernel modules, such as filter drivers or a filesystem driver, are in the middle of handling a filesystem request (the I/O request packets, or IRP). In this state, it is dangerous to initiate new operations on the drives or filesystems - both those handled by the product and other drives and filesystems. If you need to write data to the file, it is better to do this asynchronously by offloading the task to a separate worker thread and returning immediately. If you need to read data from the local file, in-memory caching of data, performed by a worker thread, is a good idea (if it is possible in your application).

In general, it is relatively safe to access other filesystems than the one to which the request in progress is directed. However, some third-party filesystem filters may be not prepared for such recursive operations and will deadlock. Such filters must be dealt with on a case-by-case basis (usually by disabling them).

To ensure stable operation, it is critical to avoid accessing drives and filesystems recursively. Essentially, this means that event handlers must not perform any operations involving the drive or filesystems that the event fired for (i.e., do not read from/write to files on it, do not unmount the media).

If an event handler detects that the file, for which this handler was called, has been changed externally (not through the virtual drive), it may return the ERROR_FILE_INVALID error code to tell the driver about the change.

Additional Topics

Contexts

It is often necessary for an application to associate certain information with a given file/directory, file handle, or enumeration operation. To assist developers in doing so in a convenient and performant manner, the CBFS component provides context parameters in a number of events.

A context carries an application-defined value that identifies or points to some application-defined data, and each file/directory, file handle, and enumeration operation has a separate context associated with it. The CBFS component treats context values as opaque; it stores the context values passed to it by the application and ensures that the correct values are exposed again whenever some event fires for a particular file, handle, or enumeration; but it does not otherwise attempt to use said values in any way.

Context Lifetimes

Contexts can be grouped into a few categories, each of which is subject to a different lifetime:

  • File contexts and directory contexts, which are associated with an open file or directory.
  • Handle contexts, which are associated with a specific open file or directory handle.
  • Enumeration contexts, which are associated with an ongoing enumeration.

File/directory contexts are created the first time a file or directory is opened, and they live until the last handle to that file or directory is closed. Handle contexts, conversely, are created every time a file or directory is opened, and they live only until the associated file handle is closed. For example, consider the following sequence of operations:

Operation on File X Context Creations/Deletions Active Contexts
1. Opened by process A File context FX and handle context HXA created FX, HXA
2. Opened by process B Handle context HXB created FX, HXA, HXB
3. Closed by process B Handle context HXB deleted FX, HXA
4. Opened by process C Handle context HXC created FX, HXA, HXC
5. Closed by process A Handle context HXA deleted FX, HXC
6. Closed by process C File context FX and handle context HXC deleted

Note: When the FireAllOpenCloseEvents property is disabled, the OpenFile and CloseFile events are both fired only once (for steps 1 and 6, respectively), which renders handle contexts useless.

File/directory contexts are available in all events corresponding to operations performed on some open file or directory. Handle contexts have similar availability, with the notable exception of the ReadFile and WriteFile events because of behavior caused by the system's memory mapping manager and cache manager. Because these managers act as proxies for file I/O requests, said requests may be buffered initially, and then combined and submitted to the filesystem later in a single request. It is not possible to determine which processes contributed to these "aggregate requests", nor is there any guarantee that any or all of them are still alive, hence, the lack of handle context parameters in the aforementioned events.

Enumeration contexts are created any time a new enumeration operation begins, and they live until the enumeration operation ends.

All contexts, when created, are created before their corresponding "first event" fires (e.g., OpenFile, EnumerateDirectory); and when deleted, they are deleted after their corresponding "last event" fires (e.g., CloseFile, CloseDirectoryEnumeration). If, however, a context's "first event" fails, whether expectedly (e.g., due to Security Checks) or otherwise (see Error Reporting and Handling), then that context's value is immediately discarded because its corresponding "last event" will not ever fire.

Additionally, for file and handle contexts, if the RouteToFile method is called at some point, then depending on which flags are passed to its Flags argument, the current event may be the "last event" for the current handle context. In such cases, the application must be sure to perform the necessary cleanup actions for the handle context before the event handler finishes. If additional events are expected, it is also reasonable to pass the CBFS_ROUTE_CLOSE_EVENT flag when calling RouteToFile so that the CloseFile event will fire again later (at which point, the application can perform the necessary cleanup actions).

Context Use Cases

Contexts are most helpful when used to store information associated with a file/directory or file handle. Typically, contexts are initialized during their corresponding "first event", and then are used in subsequent events to speed up those operations.

For example, when an OpenFile event arrives, an application might wish to obtain, for example, a stream for some real file and to store it (directly or indirectly) in the file context. Then, when the application needs to handle subsequent requests like, for example, ReadFile, it can do so quickly using the stream stored in the file context, instead of having to go through the process of obtaining that stream a second time.

As another example, consider the first time an EnumerateDirectory event fires. In this case, a full list of files can be built, and a reference to it will be stored in the enumeration context. This list can then be used to speed up subsequent firings.

Using Contexts

In .NET, context parameters are IntPtr-typed. You can store either an Int64 value or a reference to the object in those parameters. Storing Int64 values is used to keep keys of dictionary records, so that the stored values act as "handles" to the real data in the dictionary.

To store an integer value, use the IntPtr(Int64) constructor. To retrieve the stored value, use the IntPtr.ToInt64 method.

To store a reference to an object in a context, create a GCHandle instance using said object (so that it will not be garbage collected), and then use it to obtain an IntPtr: Stream stream; // The object whose reference will be stored in the context. Context = GCHandle.ToIntPtr(GCHandle.Alloc(stream));

Then, to access the object in a later event using the IntPtr stored in the context, do the following: // Guard against empty context. Stream stream = (Context != IntPtr.Zero) ? (Stream) GCHandle.FromIntPtr(Context).Target : null;

Finally, before a Context is discarded, be sure to free the GCHandle instance so that the referenced object can be garbage collected, if needed. // Guard against empty context. if (Context != IntPtr.Zero) GCHandle.FromIntPtr(Context).Free();

This scheme also can be used to store primitives (via "boxing") and other value types (nonobjects).

Security Checks

Filesystems play a key role in securing access to the files and directories they contain; in addition to storing the security information associated with each filesystem object, they must also enforce access restrictions based on that information. CBFS-based virtual filesystems, if desired, should generally handle security using one of the following options:

  • For virtual filesystems that identify themselves (via the FileSystemName property) as NTFS, security should be handled using standard Windows security mechanisms: access control lists (ACLs).
  • Virtual filesystems handle security by implementing additional custom security checks.

Each security handling method is described in more detail below, followed by additional information relevant for both.

Windows Security Mechanisms (ACLs)

The CBFS component can optionally advertise support for NTFS security attributes by enabling the UseWindowsSecurity property (and, as implied earlier, this is highly recommended for virtual filesystems that identify as NTFS). When UseWindowsSecurity is enabled, the GetFileSecurity and SetFileSecurity events will fire any time Windows needs to read or write security information. To correctly implement these events, applications must store and retrieve the OS-provided security information alongside the associated filesystem objects.

Note: Windows does not handle the task of actually checking or enforcing access rights when sending requests to create or open filesystem objects. It is up to the virtual filesystem to do so. Therefore, when such requests arrive, the application must retrieve the previously stored security information for the applicable file or directory, and then use that information to validate whether the process attempting access has sufficient rights to do so.

The aforementioned validation can be done easily using the Windows API's AccessCheck function, which validates a process's access rights against some object's security attributes. Please refer to its documentation for more information. Once the application has performed the access validation, it should either allow or deny the request according to the results.

Custom Security Checks

The CBFS component also offers a number of methods that applications can use to implement their own custom security checks. The two most notable methods are GetOriginatorProcessName, which returns the name of the process that initiated the request, and GetOriginatorToken, which returns the system-defined security token of the process that initiated the request.

The latter is particularly useful when used with various methods in the Windows API, such as the GetTokenInformation function, which can be used to obtain various pieces of information about the object the token is associated with.

Implementing Security Checks

Regardless of which security handling method is used, a number of factors must be considered when implementing security checks in general. First and foremost, for an application to effectively enforce file security, it is important that the FireAllOpenCloseEvents property is enabled so that all file open requests are surfaced (and thus given a chance to be allowed or denied).

Second, consider that effective security enforcement does not require that all event handlers perform security checks. Applications are technically free to deny almost any file-related event because of a failed security check. However, it is not actually meaningful to perform security checks in all event handlers in the first place, and unnecessary security checks will decrease an application's overall performance.

For example, it makes sense to validate access rights in the CreateFile and OpenFile event handlers, but not in the ReadFile or WriteFile event handlers. If an application denies a file create or open request made by some process, then that process will not be able to make a subsequent read/write request. Therefore, applications can safely assume that all read/write requests come from processes whose access rights they have already verified. Similar logic can be applied for directories and directory enumerations. Note: This is not an exhaustive set of use cases; each application's needs will differ.

Third, as mentioned earlier, an application may allow or deny any operation based on security checks. However, it must not alter (or "selectively report") filesystem information based on the results of the validation process (it is "all or nothing"). Said another way: at any given time, and for each CBFS-based virtual filesystem associated with it, an application must expose a single, consistent representation of the filesystem's state to all processes accessing it. The following clarifying examples should help guide implementation:

  • A filesystem object cannot be reported as existent to process A, but as nonexistent to process B.
  • A filesystem object cannot be reported as a file to process A, but as a directory to process B.
  • A filesystem object's metadata, absent of any changes, must be reported consistently between requests; for example, if a file's size is reported as 1 KB to process A, then it must also be reported as 1 KB to process B.
  • A file's contents, absent of any changes, must also be reported consistently between requests; for example, for a file whose size is reported as 1 KB, the application must return exactly 1024 bytes when the file is read, and those 1024 bytes must be exactly the same (including ordering) regardless of which process is doing the reading.

Timeouts

Because the component's events are typically tied directly to requests from the OS itself, it is critical that event handlers complete requests quickly to prevent the system from being blocked. To help prevent such blocking, the CBFS Connect system driver can enforce request timeouts on a per-virtual-drive basis.

A virtual drive's request timeout interval is specified by passing the desired number of milliseconds to the Timeout parameter when calling the MountMedia method. The value passed to the Timeout parameter must either be a positive value greater than or equal to 3000, or 0, indicating that timeouts should not be enforced, in which case events may take as long as necessary to execute.

When timeout enforcement is in effect, and an event executes long enough for its timeout to expire, the driver cancels the underlying request by reporting an error to the OS. The tardy event still runs to completion, but any results it returns once finished are ignored because the underlying request has already been handled.

If an event handler knows it will require additional time to complete an operation, it can call the ResetTimeout method (before the current timeout expires) to restart the timeout timer.

Applications should always strive to ensure that all event handlers complete quickly, even if request timeouts are disabled. Do not perform time-consuming work, especially network operations, within event handlers; offload such work to background threads. For example, a viable strategy for writing to a remote file would be to pass the data to be written to a background thread, and then to immediately finish the event handler. Similarly, to read from a remote file, an application could pre-cache some of its data with the help of a worker thread, and then return the cached data when requested.

Threading and Concurrency

Through the use of multithreading, the CBFS and FUSE components provide powerful concurrency features to help applications maximize their performance. For data integrity purposes, the component also strictly enforces the order in which events fire in certain situations and allows applications to specify the extent to which events should be fired concurrently.

Note: Even when configured for minimal concurrency, the component always fires events in the context of some worker thread, and not in the thread the component was originally created on. Therefore, applications must be sure to synchronize operations between event handlers and other threads as necessary (including, but not limited to, calls to the component instance, unless a method is explicitly documented as callable within events).

Configuring Event Concurrency

Generally speaking, the CBFS and FUSE components will always enforce per-file event serialization; that is, it always fires events relating to the same file in sequence (although technically speaking, there is one optional exception to this behavior, discussed at the end of this section). For example, if there are multiple read or write operations pending against a given file, then an event will be fired for the first operation, and after its event handler has returned, another event will be fired for the second operation, and so on.

With per-file event serialization already ensured, the most important concurrency-related consideration for CBFS- and FUSE-based applications is whether to enforce multifile event serialization as well. The components' SerializeEvents property controls whether events relating to different files should be serialized on a single worker thread, or fired in parallel on several threads; by default, this property is set to seOnMultipleThreads, and events for different files are allowed to fire in parallel.

CBFS-specific features

When the SerializeEvents property is set to seOnMultipleThreads, the MinWorkerThreadCount and MaxWorkerThreadCount configuration settings control the minimum and maximum number of worker threads the component can use for firing events. By default, both are set to 0, which indicates that the component's system driver should automatically choose appropriate values based on how many CPU cores the system has. These settings are both ignored if SerializeEvents is set to values other than seOnMultipleThreads.

As mentioned previously, the CBFS component offers one feature that alters its otherwise strictly enforced per-file event serialization model: the ability to handle nonoverlapping read operations on a single file in parallel. This functionality is controlled by the SerializeAccess property, which is enabled by default.

Mounting Points

A mounting point is a name that can be used to access a volume. When the filesystem driver mounts a volume, it must make that volume accessible by creating one or more mounting points for it.

Windows:

Mounting points can be global (visible in all user sessions) or local (visible only to a specific user session). The AddMountingPoint method creates global mounting points by default; applications must include the STGMP_LOCAL flag in the Flags parameter value to create local mounting points. (Note: The STGMP_MOUNT_MANAGER flag is not compatible with the STGMP_LOCAL flag.) Local mounting points are not supported by the FUSE component.

When creating a local mounting point, applications can specify a specific user session for it to be visible in by passing that session's Authentication ID for the AuthenticationId parameter (retrieval of Authentication IDs is discussed in a later section). If no Authentication ID is provided (i.e., 0 is passed), the local mounting point is created in the current user session; and if the application does this while running with elevated rights, then the local mounting point will only be visible in the elevated session, and consequently won't be available to applications in other sessions (such as, e.g., Windows Explorer).

When mounting points are added or removed, a system message (WM_DEVICECHANGE) is broadcast. It instructs Windows Explorer to refresh the list of drives available. However, these messages cannot cross user session boundaries; so if, for example, the application is running as a service, Windows Explorer may not receive the broadcast and thus fail to refresh the list of drives. To address this issue, CBFS Connect includes a Helper DLL which, among other things, helps ensure that Windows Explorer always refreshes the list of drives regardless of which user session the application is running in; please refer to that topic for more information.

Types of Mounting Points

There are a handful of different mounting point types, each of which exposes volumes in a slightly different manner:

  • Drive letter mounting points
  • Folder mounting points
  • Network mounting points (not supported by the FUSE component)
  • UNC path mounting points

Each type of mounting point is discussed in more detail below.

Drive Letter Mounting Points

Drive letter mounting points are one of the more commonly-used mounting point types thanks to users' familiarity with them. To create a drive letter mounting point, pass a string composed of a single character in the A-Z range followed by a colon (e.g., Z:) for the AddMountingPoint method's MountingPoint parameter (when using FUSE component, pass the value to the Mount method).

If the value passed for the AddMountingPoint method's Flags parameter includes the STGMP_AUTOCREATE_DRIVE_LETTER flag, the component will assign a drive letter automatically. In this case, the value passed for the MountingPoint parameter must not include a drive letter.

Folder Mounting Points

A folder mounting point makes a volume accessible through a folder located on another (pre-existing) NTFS volume. Folder mounting points are always visible to all users in the system, and their creation requires administrative privileges.

To create a folder mounting point using the AddMountingPoint method, include the STGMP_MOUNT_MANAGER flag in the Flags parameter (in FUSE, this flag is always set), and pass the target folder's absolute path for the MountingPoint parameter (e.g., C:\MountedDrives\MyMountingPoint). The target folder must already exist, must reside on an NTFS volume, and must be empty; otherwise, the call will fail.

Authentication IDs

(not supported by FUSE component)

An Authentication ID is a locally unique identifier (LUID) assigned to a logon session (or, "user session"), retrievable through the access token that represents said session. Applications can obtain the Authentication ID of a session from an access token or by enumerating logon sessions.

To obtain an Authentication ID from an access token, call the Windows API's GetTokenInformation function and pass either TokenGroupsAndPrivileges or TokenStatistics for the TokenInformation parameter. The resulting value will be a reference to a structure (TOKEN_GROUPS_AND_PRIVILEGES or TOKEN_STATISTICS, respectively) containing the needed Authentication ID.

To enumerate logon sessions, use the Windows API's LsaEnumerateLogonSessions function, which returns a list of existing logon session IDs (that is, Authentication IDs). To obtain additional information about a particular logon session (e.g., in order to determine if it's the desired one), use the Windows API's LsaGetLogonSessionData function. Network Mounting Points (not supported by FUSE component)

Network mounting points are similar to other mounting point types, except that the system treats them as "remote devices". This distinction is useful since:

  • Windows Explorer makes fewer requests for files located on remote devices.
  • Some applications are more tolerant of timeouts and delays when working with remote devices.

Therefore, when an application is designed to work with some slow or remote storage medium, it's recommended that it use a network mounting point. When using network mounting points, it's important that the Helper DLL be used so that Windows Explorer displays the correct drive status.

To create a network mounting point using the AddMountingPoint (for the FUSE component, Mount) method, include the STGMP_NETWORK flag in the Flags parameter, and pass a string of the form <Local Name>;<Server Name>;<Share Name> for the MountingPoint parameter.

  • <Local Name> is the name to use for the mounting point on the local system; it can be a drive letter or a name for use in a UNC path. Alternatively, it can be left empty, in which case the volume will only be accessible via the network path (see below) or the drive letter will be assigned automatically if the STGMP_AUTOCREATE_DRIVE_LETTER flag is used.
    • Note: This "local name" is not related to the concept of "local and global mounting points" discussed in the overview.
  • <Server Name> and <Share Name> are used to create a network path of the form \\<Server Name>\<Share Name>. This network path is not shared by default (see notes following examples below).

The set of characters allowed in server names, is defined in this document. The set of characters allowed in share names, is defined in this document.

With the above information in mind, here are some examples of valid MountingPoint parameter values when creating network mounting points:

  • Y:;MyServer;VirtualShare: Creates a network mounting point accessible both via the drive letter Y: and via the network path \\MyServer\VirtualShare.
  • MyMountingPoint;MyServer;VirtualShare: Creates a network mounting point accessible both via the UNC path \\.\MyMountingPoint and via the network path \\MyServer\VirtualShare
  • ;MyServer;VirtualShare: Creates a network mounting point accessible only via the network path \\MyServer\VirtualShare.

As stated above, the network paths created for network mounting points are not shared (i.e., visible to other computers on the network) by default. To have the component create an actual network share when AddMountingPoint is called, applications must include either the STGMP_NETWORK_READ_ACCESS or the STGMP_NETWORK_WRITE_ACCESS flag in the Flags parameter value, and use empty string for the <Server Name> segment of the MountingPoint parameter value (the local computer's name is used). Note that when the mounting point is shared in this way, a local resource is created and then shared. The name of the resource is derived from the Share Name defined above. However, the set of allowed characters for such name is not strictly defined. Additionally, sharing is done using a call to NetShareAdd Windows API function, which can be called by Administrators, System Operators, and Power Users.

UNC Path Mounting Points

UNC path mounting points make a volume available via a specific name, and unlike other mounting points types, they are not displayed anywhere in Windows Explorer; the UNC path must already be known.

UNC path mounting points consist of the \\.\ prefix, followed by a name (e.g., \\.\CBDrive1). The mounting-point-related component methods expect just the name (i.e., the UNC path with the \\.\ prefix omitted). So to add a new UNC path mounting point like, e.g., \\.\CBDrive1, call the AddMountingPoint (for the FUSE component, Mount) method and pass CBDrive1 for the MountingPoint parameter.

Linux and macOS:

A virtual drive / filesystem is mounted to a directory, which must exist at the time of mounting and be empty; otherwise, the call will fail.

Error Reporting and Handling

Error Codes

The CBFS and FUSE on Windows communicate errors using the Win32 error codes defined in WinError.h, which is part of the Windows Platform SDK. The CBFS and FUSE on Windows system drivers, however, use NT native error codes, which do not have one-to-one mappings with Win32 error codes. The component includes logic for converting between the two as necessary, and that logic can be updated if any codes are not covered.

Additionally, CBFS and FUSE on Windows use certain error codes in a special manner. For more information about such error codes, please refer to the Error Codes page. In the FUSE component on Linux, errors are communicated using negative Unix/Linux system error codes (e.g., the absence of some file, normally indicated in Linux using the ENOENT error code, will be reported as -ENOENT and so on).

Reporting Errors to the Component from Event Handlers

If an event has a ResultCode parameter, an event handler can use it to return the result code of the operation to the component. The ResultCode parameter is set to 0 by default, which indicates the operation was successful. In the FUSE component, on all supported platforms, you may use negative Unix/Linux system error codes to report errors (on Windows, Unix system codes will be converted to Win32 error codes implicitly). Also, on Windows, you may return Win32 error codes.

If an unhandled exception occurs in the event handler, it will be caught by the component, which will fire the OnError event.

In some events, the OS does not expect the error code to be returned and either the component or the OS ignores the returned error code. Please refer to the description of a particular event for more information.

How to Handle Errors Reported by the Component

If an error occurs, the component will throw an exception. The Code property of the exception object will contain an error code, and the Message property will contain an error message (if available).

Extended Logging in Windows

Some component methods in CBFS Connect are capable of writing extended information about reported errors to the Windows event logs, which can be viewed using the system's eventvwr.exe tool. The user mode part of the component writes to the "Windows Logs \ Application" folder, while the kernel mode part writes to the "Windows Logs \ System" folder.

The information written in the extended logs is meaningful to the Callback Technologies development team, but not to end-users, so extended logging is disabled by default. If issues occur during the installation of the CBFS Connect system drivers, or while using the component, please do the following:

  1. Enable extended logging (see below).
  2. Replicate the issue.
  3. Using Event Viewer (eventvwr.exe), export the event log entries from the locations mentioned above in native format (please restrict the scope of the export to just those entries related to CBFS Connect).
  4. Submit an issue report that includes the exported file.

There are two ways to toggle extended logging for a component:

  1. By toggling the component's LoggingEnabled configuration setting.
  2. By adding a DWORD-typed value named Enabled to the HKEY_LOCAL_MACHINE\SOFTWARE\Callback Technologies\{ComponentName}\EventLog registry key and setting it to 0 (disabled) or 1 (enabled).
    • Replace the {ComponentName} part of the registry key path with the name of the applicable component.
    • If this registry key, one of its parents, or the actual value does not exist, please create it manually.

    Note: If your code runs in emulated mode (x86 mode on x64 or ARM64 architecture), you need to add the value to the HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Callback Technologies\{ComponentName}\EventLog registry key in addition to the "main" registry key.

The system must be rebooted anytime extended logging is enabled or disabled to make the changes take effect.

Troubleshooting

Note: The below information applies to the operations of the component on Windows .

CBFS Connect is a complex product that operates in both user mode and kernel mode simultaneously; so when a serious issue occurs, it's critical that we are able to obtain sufficient information about the circumstances of the failure.

In order to help us assist you in a more expedient manner, please collect the information described in the instructions below when reporting a serious issue (i.e., one that causes the system to crash or hang). Our development team cannot effectively diagnose such issues without this information.

Also, please note that these sorts of issues commonly involve environmental differences and other factors that are either unforeseen or otherwise out of our control. It is also not unheard-of for a crash to appear attributable to one thing while in fact being caused by something completely different. Rest assured that we are committed to assisting you as best we can, and we thank you ahead-of-time for your patience and understanding throughout the support process.

System Crashes (BSODs)

If you encounter a consistently-reproducible system crash (BSOD) that you suspect may be due to CBFS Connect, please obtain a crash dump and include it when reporting the issue to us. Our development team is unable to diagnose system crashes without the information these dumps contain.

Ensure that your system is set up to generate crash dumps, and to not restart automatically after a crash, by following the steps found in Microsoft's Enabling a Kernel-Mode Dump File article. The options available in the memory dump dropdown vary depending on your version of Windows; please choose the first one from the following list that is present in yours:

  • Complete
  • Full
  • Automatic
  • Kernel

Once your system is set up to generate crash dumps, perform the same action that caused the BSOD originally to trigger the crash again. When it occurs, be sure to copy the information on the BSOD screen exactly so that it can be included in your submission (a picture of the screen in which all of the information is legible is also acceptable). Here are some examples of the specific information we're looking for:

Recent versions of Windows:

What failed: cbfs***22.sys Stop Code: FILE_SYSTEM

Older versions of Windows:

STOP: 0x00000022 (0x00240076, 0xF7A07AA8, 0xF7A077A8, 0xF7800C82) cbfs***22.sys - Address F7800C82 base at F77CD000, DateStamp 447d6975

After you've copied this information, reboot and check that the memory dump file was created at %SYSTEMROOT%\MEMORY.DMP (typically this is C:\Windows\MEMORY.DMP; if you changed the dump file location in the crash dump settings, check the location you specified instead). It will be a very large file that is too big to attach to an email, so please upload it to a file sharing site of your choice and generate a sharing link that our development team can use to download it.

Finally, submit a support issue to us that includes the link to your dump file, all of the information from the BSOD screen (if you took a picture, attach it or provide another sharing link), a description of how the BSOD was triggered, and any other information that you feel is relevant.

System Hangs

If you encounter a consistently-reproducible system hang that you suspect may be due to CBFS Connect, you'll need to collect the same information as described above. But in order to obtain a crash dump, you'll first need to configure your system so that you can trigger a crash from the keyboard once it hangs. To make this possible, follow these steps (adapted from Microsoft's Forcing a System Crash from the Keyboard article):

  1. First, using the instructions provided in the section above, configure your system to generate crash dumps, and to not restart automatically after a crash.
  2. Next, you must enable keyboard-initiated crashes in the registry by creating a new value named CrashOnCtrlScroll, and setting it equal to a REG_DWORD value of 0x01, in all of the following registry keys:
    • HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\i8042prt\Parameters
    • HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\kbdhid\Parameters
    • HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\hyperkbd\Parameters
  3. Finally, you must restart the system in order for these settings to take effect.

After these steps are complete, you'll be able to trigger a keyboard-initiated crash by using the following hotkey sequence: hold down the Right CTRL key, and press the SCROLL LOCK key twice.

At this point, you can perform the same action that caused the system to hang originally to trigger the hang again. Once the system hangs, use the hotkey sequence to force it to crash, and then follow the rest of the instructions from the section above to collect and submit the necessary information.

Method Never Returns

If your application makes a call to some CBFS Connect method and that method never returns, there's a high chance that the driver deadlock has occurred. This can be caused by various factors; to determine the reason and possible solutions, we need a memory dump, as described above. If your system remains functional, you may also use easier ways to initiate a crash:

  1. Use the NotMyFault tool by SysInternals to initiate a system crash. This is the preferred way because it generates Complete memory dumps. This tool has a self-explanatory GUI.
  2. Use the LiveKD tool by SysInternals to create a live memory dump without crashing a system. The disadvantage of this method is that it creates Kernel dumps that are missing user-mode information. To use LiveKD on 32-bit systems, use this command: "livekd.exe -ml -o memory.dmp". On 64-bit systems, the command line would be "livekd64.exe -ml -k .\kd64.exe -o memory.dmp".

Application Crashes

Sometimes, an application crashes while the OS continues to operate, and the name of one of the modules of CBFS Connect is present in the crash information. A crashing application can be the one that uses CBFS Connect or some third-party process. If the crash occurs repeatedly, it is possible to make use of a User-Mode Crash Dump to locate or narrow down the source of the crash. Generation of crash dumps is disabled by default. Before you reproduce the crash, you need to Enable Collecting User-Mode Crash Dumps.

After you enable the crash dump, you don't need to reboot, you can proceed to reproduction of the crash immediately. After the crash re-occurs, you can pick the dump file from its location. The default locations of user-mode dump files are:

  • For regular applications: %LOCALAPPDATA%\CrashDumps
  • For System services: %WINDIR%\System32\Config\SystemProfile
  • For Network and Local services: %WINDIR%\ServiceProfiles
If you changed the dump file location in the crash dump settings in the Registry, check the location you specified instead.

A crash dump can be a large file (depending on the settings) that is too big to attach to an email, so please upload it to a file sharing site of your choice and generate a sharing link that our development team can use to download it.

Finally, submit a support issue to us that includes the link to your dump file, a description of how the BSOD or a manual crash was triggered, and any other information that you feel is relevant.

Helper DLL

The Helper DLL is integrated into Windows Explorer and offers functionality designed to provide users with a consistent and pleasant experience. It is recommended that the Helper DLL be installed alongside the system driver, which can be accomplished by including the MODULE_HELPER_DLL flag when calling the Install method.

The Helper DLL is distributed in the same .cab file as the system driver; its name is CBFSShellHelper22.dll, and it is shipped in both 32-bit and 64-bit variants. Its functionality is described below.

Mounting Point Change Broadcasts

Anytime a mounting point is added or removed, the system driver will send a notification to the Helper DLL, which then broadcasts a system message instructing Windows Explorer to refresh the list of drives. Without this functionality, Windows Explorer will not refresh the list of drives if a mounting point is added or removed from a Windows service or another user session.

Network Mounting Point

When a network mounting point is used, the Helper DLL provides the functionality that allows Windows Explorer to correctly display the current status of, and interact with, the virtual drive. Without this functionality, the virtual drive will display as "Disconnected", which may result in unexpected behavior.

Custom Icons

When custom icons are used for a virtual drive, the Helper DLL ensures that they are properly displayed in Windows Explorer.

Caching

Caching is a mechanism used to increase performance by optimizing access to commonly used data or information. The CBFS component provides caching functionality for three distinct areas:

  • File data caching
  • File metadata caching
  • Nonexistent file information caching

Each virtual drive can be individually configured to use all, some, or none of these caches; please refer to the following sections for more information about each one.

File Data Caching

The CBFS component can employ a file data cache to help decrease the ReadFile and WriteFile events' rate of fire. Three different file data cache implementations are available: (1) the Windows file management system cache, which is built into Windows; and the internal (2) kernel-mode and (3) user-mode caches, which are implemented using the CBFS component and its system driver.

At a basic level, all three cache implementations in CBFS work by storing blocks of file data in memory pages (which may, or may not, be swappable), and use some sort of least recently used (LRU)-based cache replacement algorithm to decide which pages to discard first (when necessary). Implementation-specific information is discussed in more detail in the following.

Additionally, an application may use the CBCache component for caching and buffering file data on the disk for further transfer to and from external or remote locations.

For each virtual drive, the FileCache property controls whether file data caching is enabled and specifies which file data cache implementation to use; by default, file data caching is enabled, and the system's cache implementation is used.

The FileCachePolicyWriteThrough configuration setting controls the behavior of the driver with regards to writing file data. When the setting is enabled, the driver will fire WriteFile as soon as the data have been cached to ensure that it is written out to storage as soon as the write request is received. When the setting is disabled (default), the data are cached, and flushed out to storage later; this causes write operations to appear to finish quickly, but since some operations (e.g., file copying) require a flushing step, they can appear to get "stuck" at the last moment as flushing occurs.

If the writing process opens a file with FILE_FLAG_WRITE_THROUGH option, the system and the driver behave as if FileCachePolicyWriteThrough were set.

System Cache Details

By default, the CBFS component leverages Windows' built-in file management system cache, offloading the job of file data caching to the OS entirely. For CBFS-based virtual drives configured to use the system cache, Windows automatically caches file data in a dedicated area of the system memory region. This caching operates on a per-file basis, and the overall cache size is managed automatically by the system based on current memory pressure. The system automatically balances the rate at which cached data are flushed to storage for optimal system performance. Together, these attributes ensure access to frequently read file data are as fast as possible. Please refer to Microsoft's File Caching article for more information about the system cache.

The system cache, by virtue of being a built-in Windows mechanism, is faster, more robust, and more flexible than either of the CBFS-implemented file data caches. Therefore, unless there are specific reasons to do otherwise, applications should always prefer the system cache over either of the internal implementations.

Kernel Mode Cache Details

The internal kernel mode file data cache uses a nonpaged (i.e., nonswappable) memory region to cache file data. As with the system cache, this approach guarantees high-speed access to cached data because it is guaranteed to remain in-memory. However, there are two caveats to be aware of regarding this memory region:

  • The memory region is limited to a finite size that cannot be increased. On modern systems, this is typically 300 MB, but the exact amount depends on how much physical memory the system has.
  • Because the memory region is allocated by the CBFS system driver, it is shared between all CBFS-based virtual drives on the system configured to use kernel mode file data caching.

If the kernel mode cache's size limit is reached, pages are freed in LRU order. Pages are also freed if they have not been used for 120 seconds.

User-Mode Cache Details

The internal user-mode file data cache uses pageable (i.e., swappable) memory regions, allocated per-process, to cache file data. Caching the file data in each process's memory space means the cache size can be as large as desired. As with the internal kernel mode cache, however, there are some caveats to keep in mind:

  • Because the memory regions are allocated per-process, all CBFS-based virtual drives within a given process (that are configured to use user-mode file data caching) share that process' memory region.
  • Because the memory is pageable, the OS can swap any page out to storage at any time, causing the next access of such pages' data to be slower.
  • CBFS locks recently used pages to prevent them from being swapped out, but the total size of all locked pages' data is limited to about 300 MB (and this limit is shared among all of the CBFS-based virtual drives on the system that are configured to use user-mode file data caching).

The UserModeFileCacheSize configuration setting controls the size of the user-mode file data cache. Because the memory is allocated per-process, this setting's value is synced between all CBFS component instances in a process and thus can be changed only when the user-mode cache is not in use by any virtual drive in the process (please refer to its documentation for details).

If the user-mode cache's size limit is reached, pages are freed in LRU order.

File Metadata Caching

The CBFS component can employ a file metadata cache to help decrease the GetFileInfo events' rate of fire. The file metadata cache stores entries containing file names, times, and attributes, which the component uses to automatically respond to file information requests (rather than firing the aforementioned events).

The MetadataCacheEnabled property controls whether file metadata should be cached for a virtual drive; it is enabled by default. The MetadataCacheSize property specifies the maximum number of cache entries.

While a file or directory is open, its metadata is kept available in a special record called a File Control Block, regardless of whether or not the metadata cache is enabled.

Nonexistent File Information Caching

The CBFS component can also employ a "nonexistent files" cache to further decrease the GetFileInfo event's rate of fire. The nonexistent files cache stores entries representing files that are known to be nonexistent (based on previous responses to the GetFileInfo event) so that the component can automatically respond to information requests for such files.

The NonExistentFilesCacheEnabled property controls whether nonexistent file information should be cached for a virtual drive. The NonExistentFilesCacheSize property specifies the maximum number of cache entries.

Docker Containers

CBFS Connect can be used to create virtual drives within Docker containers on Windows. To operate correctly the following conditions must be met:

  • The CBFS Connect drivers are installed on the host machine.
  • The container uses Process isolation (Hyper-V isolation is not supported).

Isolation Modes and Windows Containers

Docker containers on Windows run in two isolation modes: Process isolation and Hyper-V isolation. These isolation modes are similar in that containers do not share resources with one another, but they differ in how much interaction the containers have with the host kernel. Process isolation containers share their kernel with the host system and allow for the isolation boundary to be crossed upon a user request. Containers running in Hyper-V isolation mode do not share a kernel with the host and are not supported.

The CBFS Connect drivers need to be installed only on the host. The application creating the virtual drive can run either in the container or on the host.

Virtual Drives

Virtual drives can be created in a variety of ways. Common scenarios include creating virtual drives from within a container, or while running on the host, creating a virtual drive for use within a container.

To create and use a virtual drive within a container no special settings are required, as shown in the following example:

//Create a virtual drive within a container //This code runs in a container. cbfs.AddMountingPoint("Z:", STGMP_SIMPLE, 0);

When mounting a drive within a container, the host will not have access to the drive. If, however, a network mounting point with a UNC (universal naming convention) path is created, the drive will be accessible as a regular network share to applications running on the host or in other process-isolated containers in the same system.

When running on the host, set the DockerContainerId setting to the container Id to specify the container where the drive will be created. The container Id is either the 12-character short Id or the full 64-character Id of the container. See the following example:

//Create a virtual drive form the host, for use by a container //This code runs on the host, and must specify a container Id cbfs.Config("DockerContainerId=4c01db0b339c"); cbfs.AddMountingPoint("Z:", STGMP_SIMPLE, 0);

Mounting Point Restrictions

Certain mounting point types are not supported depending on where the mount operation takes place. An application that is running on the host cannot create a network mounting point (STGMP_NETWORK) for a virtual drive that has been created in a container. STGMP_MOUNT_MANAGER mounting points are not supported in containers, regardless of whether the process performing the mount operation is running on the host or within the container.

Advanced Features

The topics in this section provide information about various advanced features of CBFS Connect.

Topics

Custom Drive Icons

Virtual drives created with the CBFS component can have a custom icon associated with them to better distinguish them in Windows Explorer. There are a few different ways to accomplish this:

Using Additional Files

If placing additional files into the virtual drive itself is an acceptable condition, Windows provides a couple of file-based mechanisms for specifying a custom icon.

To specify a custom icon for the virtual drive itself, an autorun.inf file can be created based on the information in Microsoft's Autorun.inf article.

Additionally, custom icons can be specified for subdirectories of the virtual drive using desktop.ini files, which can be created based on the information in Microsoft's Desktop.ini article. Note that desktop.ini files cannot be used to specify a custom icon for the root directory of the virtual drive (i.e., they cannot be used to change the icon of the virtual drive itself).

Using Registry Keys

If the virtual drive is assigned a persistent drive letter, using registry keys to assign a custom icon may be a good option. To specify a custom icon using the registry, create a subkey like {DriveLetter}\DefaultIcon (e.g., K\DefaultIcon) under one of the following keys:

  • HKEY_CURRENT_USER\SOFTWARE\Classes\Applications\Explorer.exe\Drives, if the custom icon should only be used for the current user.
  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\DriveIcons, if the custom icon should be used for all users. (Note that manipulating anything under the HKEY_LOCAL_MACHINE registry hive requires administrative rights.)

Regardless of which key it is created under, the {DriveLetter}\DefaultIcon subkey's (Default) (i.e., "unnamed") value should then be set to the absolute path of the icon file.

Please note that custom icons specified in this manner are only effective so long as the drive letter assigned to the virtual drive remains unchanged over time; if its drive letter changes, the registry keys used to specify the custom icon will need to be updated accordingly.

Using the Component and its Shell Helper DLL

As long as the Helper DLL has been installed to the system using the Install method, custom icons can be assigned to a virtual drive directly using the component. This method of specifying custom icons is especially valuable when project constraints preclude placing additional files into the drive or modifying the registry.

Custom icons assigned in this manner function a bit differently than those assigned using the two methods described above, as they are implemented using Windows' icon overlay mechanism. Consequently, the custom icons are restricted to 25% of the original icon's area (except for 16x16 icons); the tables below describe the required sizes and color levels of the assets in the icon file.

Overlay icon sizes map as follows:

Main Icon SizeOverlay Icon Size
16x1610x10
32x3216x16
48x4824x24
256x256128x128

Icon assets must have the following color levels:

Icon SizeColor Level
16x1616 colors
32x3216 colors
48x48256 colors
256x25632-bit color

Because it's possible to specify multiple different overlay icons (e.g., to represent different drive states), icons are assigned through the component using a two-step process:

  1. Register the desired icon(s) using the RegisterIcon method. (Note that administrative rights are required to execute this method successfully.)
  2. Switch between the registered icon(s) using the SetIcon and ResetIcon methods.

Icons are copied to a temporary location when registered; and removed from said location when unregistered using the UnregisterIcon method.

It is important to keep in mind that Windows limits the number of registered overlay icons to 15 (this is a global limit for the entire system, and it cannot be changed). Since other applications on the system (e.g., OneDrive, Dropbox, etc.) may have registered multiple overlay icons, it's not uncommon to get into a situation where various applications are competing to have their overlay icons registered.

Overlay icons are registered by placing values in the following keys in the Registry:

  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\ShellIconOverlayIdentifiers
  • HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\explorer\ShellIconOverlayIdentifiers (64-bit Windows only)
If necessary, it's up to the application (or better yet, the user) to decide whether or not to remove other entries; however, doing so too aggressively will likely have a negative impact on the user's experience with other applications.

Hard Links

For filesystems that support them, hard links are essentially arbitrary names used to expose a particular file's data at one or more locations within the filesystem's directory structure. For example, although not typically referred to as such, a file's "original" file name is technically a hard link in such a filesystem.

Expanding on the previous example, hard links could be thought of as "additional" names for the "original" file (although a file's "original" name technically is not any more important than any "additional" names for it that may exist elsewhere in the filesystem). Regardless of the order in which they are created and deleted, or their location within the directory structure, all of the names (i.e., hard links) for a particular file are equivalent because they all reference the exact same file data.

Because of this equivalence, a CBFS-based virtual filesystem that indicates support for hard links (see below for details) must handle file deletion requests carefully. When such a request arrives, the application must check how many hard links to the actual file data remain after removing the requested one; the file data should not actually be deleted until the last hard link has been deleted.

Note: Hard links, by definition, can refer only to files. Directories only ever have one name, in one location, within a CBFS-based virtual filesystem (to avoid recursion).

Hard links are a powerful and complex feature of Windows filesystems; and for clarity and brevity, this overview omits many details not relevant to understanding how the CBFS component interacts with them. To learn more about hard links, please refer to Microsoft's Hard Links article.

Supporting Hard Links with CBFS

In CBFS, the UseHardLinks property is used to indicate whether a virtual filesystem supports hard links; it is disabled by default. Virtual filesystems that do advertise hard link support must also support file Ids and Id-to-name resolution.

When hard link support is enabled, the following events must be handled by the component:

Additionally, the GetFileInfo event handler must always be sure to return the number of hard links a file has by setting the HardLinkCount event parameter appropriately.

Finally, as described previously, the application must be careful to delete file data only after the last hard link to a file has been deleted.

Reparse Points

For filesystems that support them, reparse points are essentially storage containers for some application-specific data associated with a file or directory. The format of the data held by a reparse point is determined by the application that stored it; the data are treated as opaque by the OS, which in turn delegates the interpretation of it to some filesystem filter. The notable exceptions are symbolic links and NFS links, implemented using reparse points and handled by the OS and a filesystem.

To assist the OS in determining which filesystem filter to use, each reparse point includes a reparse tag that uniquely identifies the data's format. When the filesystem encounters a reparse point, it attempts to find the filesystem filter associated with the data format identified by this tag. If a matching filesystem filter is found, it is invoked to process the file/directory using the reparse point's data.

Some reparse tags are predefined by Microsoft, and others are third-party-defined (although they are still assigned by Microsoft to ensure uniqueness). All Microsoft-defined tags are listed in this MSDN article; there is no central list of third-party tags.

A complete reparse point (e.g., tag, data) is always represented as either a REPARSE_GUID_DATA_BUFFER structure, or (for certain Microsoft-reserved tags) a REPARSE_DATA_BUFFER structure. That said, the CBFS component interacts with reparse points in a limited fashion, and its API exposes reparse point structures as nothing more than opaque blobs of data for the application to store and retrieve.

Reparse points are a powerful and complex feature of Windows filesystems; and for clarity and brevity, this overview omits many details not relevant to understanding how the CBFS component interacts with them. To learn more about reparse points, please refer to Microsoft's Reparse Point articles.

Supporting Reparse Points with CBFS

In CBFS, the UseReparsePoints property is used to indicate whether a virtual filesystem supports reparse points; it is disabled by default. When a virtual filesystem has reparse point support enabled, the associated CBFS-based application is responsible for storing reparse points created within the virtual filesystem.

When reparse point support is enabled, the following events must be handled by the component:

Folling is the generalized flow for reparse point support:

  1. When a reparse point is created somewhere within the virtual filesystem, the SetReparsePoint event will fire. To properly handle this event, the application must store the data provided in the ReparseBuffer parameter in some persistent manner (this data should not replace other file/directory data; it should be stored in addition to such data). In addition, it must record (in some manner) the fact that the file/directory specified by the event has a reparse point associated with it.
  2. Any time information about a file/directory is requested via the EnumerateDirectory and/or GetFileInfo events, the application must check whether a reparse point is associated with it. If so, the FILE_ATTRIBUTE_REPARSE_POINT (0x00000400) flag must be included in the returned Attributes event parameter value.
  3. Any time the OS encounters a file/directory with the aforementioned file attribute flag set, it will request the reparse point data before continuing so that it knows how to process the file/directory correctly. Such requests cause the GetReparsePoint event to fire, at which point the application must return the requested reparse point data.
  4. When a reparse point is deleted somewhere within the virtual filesystem, the DeleteReparsePoint event will fire. At this point, the application should delete the data it had stored for that reparse point and should update its records for the file/directory accordingly (to indicate that a reparse point is no longer associated with it).

Please refer to each of the above events' documentation for more information about how to properly handle each scenario.

File IDs

File Ids are used by some system components (e.g., NFS sharing) and third-party applications as an alternative to file names. Ids associated with files must be unique within a filesystem and should not change over time.

The root directory (\) always uses the predefined Id 0x7FFFFFFFFFFFFFFF.

For proper File Id support, an application must do all of the following:

  • enable the UseFileIds property;
  • create a mounting point with either STGMP_NETWORK or STGMP_MOUNT_MANAGER flag;
  • return File IDs during directory enumeration and retrieval of file information;
  • handle the GetFileNameByFileId event and return the appropriate file name and path.

If an Id, returned during directory enumeration, is 0, the component will generate an Id. This generated Id is not guaranteed to be unique, especially under a heavy load. File Ids, including the generated ones, are kept in the metadata cache. If the metadata cache is disabled and Ids are not provided by an application, the driver will fail to provide proper File Ids and will open files by Id, which can prevent operations of components and applications that rely on File Id support.

Short File Names

The CBFS component includes optional support for short filenames (also known as 8.3 filenames), which allows files to be accessed using the 8.3 name format: an eight-character name followed by a three-character extension, separated by a period, containing no spaces. A CBFS-based virtual filesystem may need to enable this feature, for example, for its files to be accessible to some older, third-party software that only knows of the 8.3 name format.

In CBFS, the UseShortFileNames property is used to indicate whether a virtual filesystem supports short filenames; it is disabled by default. Virtual filesystems that do advertise short filename support must also:

  • Be able to generate short filenames that uniquely identify any and all files.
  • Be able to map between files' normal (i.e., long) and short names, in both directions.
  • Be sure to return files' short names when handling the EnumerateDirectory and GetFileInfo events.
  • Be able to handle a short filename being provided (instead of a normal one) in any event.

For more information about short filenames, please refer to this section of Microsoft's file naming article.

Symbolic Links

Symbolic links are implemented in Windows as reparse points with a dedicated reparse tag. Such reparse points' data contains the name of the target of the symbolic link (i.e., the file or directory referenced by the symbolic link).

To support symbolic links, an application must do the following things:

  1. Enable reparse points by setting the UseReparsePoints property to true and implementing the corresponding events:
  2. For files and directories that are symbolic links, include the FILE_ATTRIBUTE_REPARSE_POINT flag in the attributes of the corresponding file or directory when handling the EnumerateDirectory and GetFileInfo events.
  3. When handling the OpenFile event, check whether the Attributes event parameter contains the FILE_FLAG_OPEN_REPARSE_POINT flag. If the flag is present, the application must open the target instead of the file itself. If the flag is not present, the application must act as if the file did not have a reparse point attached to it.

Additionally, for Network-type mounting points (i.e., those for which STGMP_NETWORK was included in the value passed for the AddMountingPoint method's Flags parameter), symbolic links will work only if support for symbolic links on network drives is enabled in the system. This can be done by running the following commands in an elevated command prompt:

  • fsutil behavior set symlinkEvaluation R2R:1
  • fsutil behavior set symlinkEvaluation L2R:1
  • fsutil behavior set symlinkEvaluation R2L:1

Re-use of Process IDs

When using various rules that are based on process IDs (PID), you need to be aware that Windows tends to reuse PID numbers. Once the process with a certain PID is finished in any way, Windows can re-use this PID for another process being started. And it does reuse PIDs quite frequently for the purpose of keeping PID numbers low.

Such reuse can cause unexpected and sometimes unpleasant consequence for your application. To counteract it, you can take one or both actions:

  1. Open a handle to the process with the needed PID and not close it as long as your rule exists. Windows documentation states that as long as there exists an open handle to a process, its PID is not reused.
  2. Track completion of the process with the given PID (either by monitoring the state of the process by its handle or using CBProcess component of the CBFS Filter product) and once the process is finished, delete the corresponding rule.

CBCache Topics

The topics in this section provide additional information specific to the CBCache component.

Topics

File Ids

Each file in the cache must have a unique file Id that identifies it. File Ids are strings of unlimited length that may contain any character; the CaseSensitive parameter of the CBCache component's OpenCache method determines whether the component will treat file Ids in a case-sensitive or a case-insensitive manner.

File Sizes

For each file in the cache, the CBCache component keeps track of an application-defined file size value that indicates the total size of the file (i.e., the size of the real file outside of the cache, not the amount of data cached for the file).

The CBCache component will not change this value by itself. It is up to the application to set it. If true is passed for the FetchMissingData parameter when a file's size is changed using the FileSetSize method, the CBCache component will fire the ReadData event to fetch additional data from external storage so that it may be cached.

File Tags

CBFS Connect allows applications to attach arbitrary metadata to any cached file as tags. Such metadadata can be used to store supplementary information (e.g., file metadata or some file handle).

File tags are stored as key-value pairs. They use numeric Ids as keys and store raw binary data.

  • Valid Id values are those in the range 0x0001 to 0xCFFF (inclusive).
  • A tag should contain at least one (1) byte of data.
  • The maximum size of a file tag's binary data is 65531 bytes.

Each cached file can have up to 53247 raw file tags attached to it at one time. The following methods are used to manage and interact with file tags:

Local Files

Local files are files created in the cache that do not have corresponding files in an external storage. Local files are stored wholly within the cache, and the cache will never attempt to read data into them from external storage, or flush their data out to an external storage.

Typically, applications make use of local files in cases in which certain files are known to be temporary or for some other reason that they should not be written to an external storage. For example, the Windows Thumbs.db and desktop.ini files and the macOS .DS_Store files. It is up to the application to determinate the state of, and subsequently manage, all local files.

Orphan Files

Files that are not currently open, yet have unflushed data, are referred to as orphan files. Typically, orphan files appear when an application is closed or terminated before all file modifications have a chance to be flushed to external storage.

To recover orphan files, applications should enumerate them by passing the ENUM_MODE_ORPHAN flag to the EnumerateCachedFiles method. The application is then free to open, flush, or delete these files as needed.

Reporting Transfer Errors

If the ReadData or WriteData event reports a RWRESULT_FILE_FAILURE result, the component will not send any more requests for the specified file until the application calls ResetErrorState to reset the error state of the file.

If the ReadData or WriteData event reports a RWRESULT_PERMANENT_FAILURE result, the component will not send any more requests for any file until the application calls ResetErrorState to reset the global error state.

Cache Corruption

Vault files used by CBCache have a rather complex internal structure that may become corrupted if the component is not deactivated properly or if some operation is interrupted. Typically, such things are caused by an application crash or a system crash. Corruption also can occur if a cache's raw data is modified externally, either intentionally or because of storage failure.

Journaling

To reduce the chances of cache corruption in the event of a crash, CBCache can make use of journaling. Journaling works by wrapping vault modification operations in transactions, as follows:

  1. A new transaction is opened by writing information about a change to a journal located within the vault.
  2. The changes themselves are written to the vault.
  3. The transaction is committed by writing another entry in the journal.

If a crash occurs, any interrupted modification operations will appear in a cache vault's journal as a pending transaction. The next time CBFS Connect opens the cache vault, it will discover any pending transactions and automatically recover them. During the transaction recovery process, each transaction is either committed or rolled back, depending on its last known state.

Overall, journaling is an effective technique used to maintain data integrity. However, keep the following considerations in mind:

  • When journaling is enabled, all file data changes incur additional write operations; this has a significant impact on overall write performance.
  • Journaling does not provide any kind of data redundancy or consistency; it cannot protect against corruption caused by bit-rot, storage failures, or external modification of a vault's physical data.

CBFS Connect implements journaling as an operational mode rather than a cache vault attribute, so there is no such thing as a "journaled cache vault" or a "non-journaled cache vault". Applications control whether the journaling mode is used by setting the JournalingMode configuration setting before activating the component.

The component will always perform transaction recovery when the cache is opened (if there are pending transactions in its journal) , even if journaling is disabled.

Error Reporting and Handling

Reporting Errors to the Component from Event Handlers

If the event has a ResultCode parameter, the event handler should use it to return the result code of the operation to the component. The ResultCode parameter is set to 0 by default, which indicates the operation was successful. If a problem occurs while handling the event, set the ResultCode parameter to one of the following values to inform the component about it:

RWRESULT_SUCCESS0x00000000Success.

Returning this code indicates that all data were successfully transferred.

RWRESULT_PARTIAL0x00000080Partial success.

Returning this code indicates that some, but not all, data were successfully transferred. i.e., when BytesRead < BytesToRead (for ReadData), or BytesWritten < BytesToWrite (for WriteData).

RWRESULT_RANGE_BEYOND_EOF0x00000081Specified range is beyond the end of the file (EOF).

For the ReadData event, returning this code indicates to the cache that there is no need to request data beyond (Position + BytesRead), and that the tail should be zeroed instead.

RWRESULT_FILE_MODIFIED_EXTERNALLY0x00000082The specified file's size was modified externally.

Returning this code indicates that the operation was not completed successfully because of this conflicting situation.

RWRESULT_FILE_MODIFIED_LOCALLY0x00000083The requested file's size was modified locally.

RWRESULT_FAILURE0x0000008DThe operation failed for some transient reason.

Returning this code indicates that, even though the current request failed, it is safe to continue sending requests for both the specified file and others. (Other operations may continue.)

RWRESULT_FILE_FAILURE0x0000008EThe operation failed for some external-file-related reason.

Returning this code indicates that some failure related to the external file has occurred, and it is expected that any further requests made against that file will also fail. The component will not send any more requests for the specified file until the file-specific error is reset as described in Reporting Transfer Errors.

RWRESULT_PERMANENT_FAILURE0x0000008FThe operation failed for some external-storage-related reason.

Returning this code indicates that some failure related to the external storage has occurred, and it is expected that all further requests will also fail. The component will not send any more requests for any file until the global error is reset as described in Reporting Transfer Errors.

If an unhandled exception occurs in the event handler, it will be caught by the component, which will fire the OnError event.

How to Handle Errors Reported by the Component

The CBCache component APIs communicate errors to applications using the error codes defined on the Error Codes page.

If an error occurs, the component will throw an exception. The Code property of the exception object will contain an error code, and the Message property will contain an error message (if available).

Recursive Calls

To ensure stable operation, it is critical to avoid accessing drives and filesystems recursively. Essentially, this means that event handlers must not perform any operations involving the drive or filesystem that the event fired for (i.e., don't read from/write to files on it, don't unmount the media, etc.).

Buffer Parameters

Some events include one or more parameters intended for use as a binary data buffer. Depending on the event, these parameters may contain data when the event is fired, or it may be expected that the application populates them with the desired amount of data during the event handler. Some events combine both paradigms and then expect the application to modify the data already present when the event is fired.

The documentation for such events will describe which of these cases applies to each buffer parameter. In all cases, buffer parameters point to a preallocated block of unmanaged memory, the size of which is specified by the parameter immediately following the buffer parameter. In cases in which data are to be written, be sure to write it directly to the pointed-to memory, do not change the value of the buffer parameter itself.

Buffer parameters are always of the IntPtr type; use the .NET Marshal.Copy() method to read and write data from and to the unmanaged memory region. When targeting newer .NET versions, such as .NET Standard 2.1 and later, applications can use the Span<T> and ReadOnlySpan<T> classes to access and modify the unmanaged memory region without extra data copying.

Constants

All constants are accessible through the callback.CBFSConnect.Constants class.

Route Cache Flags

CBFS_ROUTE_FILE_READ_ONLY 0x00000001 Causes files to be treated as read-only; all write operations will be automatically denied.

CBFS_ROUTE_OPEN_EVENT 0x00000020 Prevents 'open' requests from being routed automatically.

If set, the OpenFile event will fire as usual when such requests arrive.

Note: This flag is implied if any of the other CBFS_ROUTE_*_EVENT flags are present.

CBFS_ROUTE_CLEANUP_EVENT 0x00000040 Prevents 'cleanup' requests from being routed automatically.

If set, the CleanupFile event will fire as usual when such requests arrive.

CBFS_ROUTE_CLOSE_EVENT 0x00000080 Prevents 'close' requests from being routed automatically.

If set, the CloseFile event will fire as usual when such requests arrive.

Note: This flag is implied if any of the other CBFS_ROUTE_*_EVENT flags are present.

CBFS_ROUTE_ENUMERATE_DIRECTORY_EVENT 0x00000200 Prevents 'enumerate directory' requests from being routed automatically.

If set, the EnumerateDirectory event will fire as usual when such requests arrive.

CBFS_ROUTE_SET_SECURITY_EVENT 0x00000400 Prevents 'set security' requests from being routed automatically.

If set, the SetFileSecurity event will fire as usual when such requests arrive.

CBFS_ROUTE_GET_SECURITY_EVENT 0x00000800 Prevents 'get security' requests from being routed automatically.

If set, the GetFileSecurity event will fire as usual when such requests arrive.

CBFS_ROUTE_SET_FILE_ATTRIBUTES_EVENT 0x00002000 Prevents 'set file attributes' requests from being routed automatically.

If set, the SetFileAttributes event will fire as usual when such requests arrive.

CBFS_ROUTE_SET_FILE_SIZES_EVENT 0x00004000 Prevents 'set file size' requests from being routed automatically.

If set, the SetFileSize event will fire as usual when such requests arrive.

CBFS_ROUTE_SET_VALID_DATA_LENGTH_EVENT 0x00008000 Prevents 'set valid data length' requests from being routed automatically.

If set, the OpenFile event will fire as usual when such requests arrive.

CBFS_ROUTE_CREATE_HARD_LINK_EVENT 0x00020000 Prevents 'create hard link' requests from being routed automatically.

If set, the CreateHardLink event will fire as usual when such requests arrive.

CBFS_ROUTE_QUERY_QUOTA_EVENT 0x00040000 Prevents 'query quota' requests from being routed automatically.

If set, the QueryQuotas event will fire as usual when such requests arrive.

CBFS_ROUTE_SET_QUOTA_EVENT 0x00080000 Prevents 'set quota' requests from being routed automatically.

If set, the SetQuotas event will fire as usual when such requests arrive.

CBFS_ROUTE_CAN_FILE_BE_DELETED_EVENT 0x00200000 Prevents 'can file be deleted' requests from being routed automatically.

If set, the CanFileBeDeleted event will fire as usual when such requests arrive.

CBFS_ROUTE_IS_DIRECTORY_EMPTY_EVENT 0x00400000 Prevents 'is directory empty' requests from being routed automatically.

If set, the IsDirectoryEmpty event will fire as usual when such requests arrive.

CBFS_ROUTE_RENAME_EVENT 0x00800000 Prevents 'rename/move' requests from being routed automatically.

If set, the RenameOrMoveFile event will fire as usual when such requests arrive.

Notify Change Flags

CBFS_NOTIFY_FLAG_ADDED 0x00000001 The specified file or directory has been created.

CBFS_NOTIFY_FLAG_REMOVED 0x00000002 The specified file or directory has been removed.

For directories, all handles to files in the directory are immediately closed.

CBFS_NOTIFY_FLAG_MODIFIED 0x00000003 The specified file or directory has been modified.

For files, use this option when the file's contents have changed. For directories, use this option when a change is made in alternate data streams that belong to the directory.

If a directory is reported as modified, all handles to files in the directory are immediately invalidated.

CBFS_NOTIFY_FLAG_METADATA_MODIFIED 0x00000004 The timestamp or other attributes of the specified file or directory have been modified.

CBFS_NOTIFY_FLAG_ALLOCATION_SIZE_MODIFIED 0x00000005 The allocation size for the specified file or directory has been modified.

This option is applicable only if CorrectAllocationSizes is false.

CBFS_NOTIFY_FLAG_MODIFIED_NOT_INVALIDATE 0x00000006 The specified file or directory has been modified, but handles should not be invalidated.

Note: Using this option can cause other applications to operate using stale data if they keep the specified file or directory open, because they have no way to know about the external changes.

CBFS_NOTIFY_FLAG_RENAMED 0x00000007 The specified file or directory has been renamed.

This flag updates the internally stored name and does not invalidate existing handles (if any exist).

CBFS_NOTIFY_FLAG_SKIP_LOCKED_FILE 0x00001000 Ignore the request if the file has locked ranges.

This supplementary flag may be used together with the CBFS_NOTIFY_FLAG_MODIFIED_NOT_INVALIDATE flag to tell the driver that if the file has locked ranges, the notification should be ignored.

Install Flags

INSTALL_REMOVE_OLD_VERSIONS 0x00000001 Uninstall drivers and helper DLLs from previous component versions (e.g., 2017).

INSTALL_KEEP_START_TYPE 0x00000002 Keep the driver's current start type setting in the registry.

If this flag is not set (default), the installation logic will reset the driver's start type setting in the Windows registry to the default value. Setting this flag causes the installation logic to preserve the current value, which may be necessary if the user (or the application itself) set it previously.

INSTALL_OVERWRITE_SAME_VERSION 0x00000004 Install files when their version is the same as the version of already installed files.

If this flag is not set (default), the installation logic will overwrite the existing file only if the version number of the file being installed is larger than the version of the file being overwritten. Setting this flag causes the installation logic to overwrite the file even when it has the same version.

INSTALL_FORCE_SHA1_DRIVERS 0x00000008 Tell the installer to use SHA1-signed drivers regardless of the OS.

If this flag is not set (default), the installation logic will use SHA1 drivers only on Windows 7. However, some systems based on Windows 8.x, despite supporting SHA2 driver signatures, refuse to install the Plug-n-play drivers signed by Microsoft using SHA256 (error 0xe0000244 is reported). To work around this problem, you may use this flag - a system should accept SHA1-signed drivers then. Note that the flag should be used only as a fallback when the system returns the above-mentioned error code during the regular installation procedure.

Uninstall Version Flags

UNINSTALL_VERSION_PREVIOUS 0x00000001 Uninstall modules from previous product versions.

UNINSTALL_VERSION_CURRENT 0x00000002 Uninstall modules from the current product version.

UNINSTALL_VERSION_ALL 0x00000003 Uninstall modules from all product versions.

Module Flags

MODULE_PNP_BUS 0x00000001 PnP Bus Driver (.sys file).

This module must be installed if the application wishes to make use of Plug-and-Play (PnP) storage features. PnP storage devices are those visible as disks in the Device Manager, and the system treats such storage devices differently from other purely virtual devices.

The filesystem driver must be reinstalled any time this module is added or removed.

MODULE_DRIVER 0x00000002 Core Product Driver (.sys file).

The product's filesystem driver module provides the core of its functionality; it must be installed for the product to function correctly.

MODULE_HELPER_DLL 0x00010000 Shell Helper DLL (CBFSShellHelper22.dll)

This module provides supplementary functionality; please refer to the Helper DLL topic for more information.

Note: This module is not applicable when calling the GetDriverStatus method.

Module Status Flags

MODULE_STATUS_NOT_PRESENT 0x00000000 The specified module is not present on the system.

MODULE_STATUS_STOPPED 0x00000001 The specified module is in the Stopped state.

MODULE_STATUS_RUNNING 0x00000004 The specified module is loaded and running.

Mounting Point Flags in Windows

STGMP_SIMPLE 0x00010000 Create a simple mounting point.

Simple mounting points may be local or global; and when local, can be made visible in either the current user session or another one.

This flag cannot be combined with STGMP_MOUNT_MANAGER or STGMP_NETWORK, and is implied if neither of those flags are present.

STGMP_MOUNT_MANAGER 0x00020000 Create a mounting point that appears to the system as a physical device.

When the StorageType property is set to STGT_DISK_PNP, mounting points created using the system mount manager appear as physical devices in the Disk Management snap-in of the Microsoft Management Console (mmc.exe).

This flag is a necessary prerequisite for creating a folder mounting point, which makes a drive accessible via an otherwise empty directory on another NTFS volume.

This flag cannot be combined with STGMP_SIMPLE, STGMP_NETWORK, or STGMP_LOCAL.

Only one mounting point of this type can be added to a virtual drive.

An application cannot add a STGMP_MOUNT_MANAGER mounting point for a virtual drive created in a Docker container.

STGMP_NETWORK 0x00040000 Create a network mounting point.

Network mounting points can be further configured using the various STGMP_NETWORK_* flags described below. Applications that plan to make use of network mounting points must be sure to install the Helper DLL before doing so, otherwise Windows Explorer will not correctly recognize the "network" drive.

This flag cannot be combined with STGMP_SIMPLE or STGMP_MOUNT_MANAGER.

When a virtual drive is created in a Docker container by an application running on the host, it cannot add a mounting point of STGMP_NETWORK type. If an application is running in the container, STGMP_NETWORK type may be used.

STGMP_LOCAL 0x10000000 Specifies that a local mounting point should be created.

This flag specifies that a local mounting point should be created rather than a global one. When this flag is set, applications must also pass an appropriate value for the AddMountingPoint method's AuthenticationId parameter.

Passing 0 for AuthenticationId will make the mounting point visible in the current user session. To make the mounting point visible in a different user session instead, pass the target session's Authentication ID.

This flag is valid when combined with STGMP_SIMPLE or STGMP_NETWORK; it cannot be combined with STGMP_MOUNT_MANAGER. Please note that a mounting point can be made available to other computers as a network share, and network shares are always globally visible on the local machine, even if this flag is set.

STGMP_NETWORK_ALLOW_MAP_AS_DRIVE 0x00000001 Indicates that users may assign a drive letter to the share (e.g., using the 'Map network drive...' context menu item in Windows Explorer).

STGMP_NETWORK_HIDDEN_SHARE 0x00000002 Indicates that the share should be skipped during enumeration.

Such shares are only accessible when their name is already known to the accessor.

STGMP_NETWORK_READ_ACCESS 0x00000004 Makes a read-only share available for the mounting point.

When this flag is specified, the <Server Name> part of the MountingPoint parameter value must be empty. Please refer to the Mounting Points topic for more information. This flag makes the component use the Windows API's NetShareAdd function. As per MSDN, "Only members of the Administrators, System Operators, or Power Users local group can add file shares with a call to the NetShareAdd function."

This flag cannot be used together with STGMP_NETWORK_CLAIM_SERVER_NAME.

STGMP_NETWORK_WRITE_ACCESS 0x00000008 Makes a read/write share available for the mounting point.

When this flag is specified, the <Server Name> part of the MountingPoint parameter value must be empty. Please refer to the Mounting Points topic for more information. This flag makes the component use the Windows API's NetShareAdd function. As per MSDN, "Only members of the Administrators, System Operators, or Power Users local group can add file shares with a call to the NetShareAdd function."

This flag cannot be used together with STGMP_NETWORK_CLAIM_SERVER_NAME.

STGMP_NETWORK_CLAIM_SERVER_NAME 0x00000010 Specifies that the server name is unique.

When this flag is specified, the driver handles IOCTL_REDIR_QUERY_PATH[_EX] requests by instructing the OS to direct all requests going to the <Server Name> part of the MountingPoint parameter's value to the driver instead.

This flag should be used when the <Server Name> is unique within the local system (e.g., when the application's name is used). Using this flag allows the system to avoid delays caused by certain network requests made by various processes.

This flag is also required for "net view" command to be able to show the share in the list.

This flag cannot be used together with STGMP_NETWORK_READ_ACCESS or STGMP_NETWORK_WRITE_ACCESS.

STGMP_DRIVE_LETTER_NOTIFY_ASYNC 0x20000000 Causes the method to return immediately without waiting for mounting notifications to be sent to the system.

STGMP_AUTOCREATE_DRIVE_LETTER 0x40000000 Tells the component that it should assign the drive letter automatically.

When this flag is specified, the component will automatically assign a drive letter from the list of available letters. The assigned letter is added to the end of the list of mounting points, and can be retrieved from there.

Do not include a drive letter in the MountingPoint parameter's value when specifying this flag.

RequestedInfo Flags

FILEINFO_REAL_NAME 0x0040 File's actual name is requested.

In an EnumerateDirectory event handler, store the name of the file or directory in the FileName parameter. In a GetFileInfo event handler, store the name of the file or directory in the RealFileName parameter. Refer to the corresponding event's help topic for more details.

FILEINFO_SHORT_NAME 0x0200 Short name is requested.

FILEINFO_TIME 0x0004 File times are requested.

FILEINFO_SIZE 0x0008 File size is requested.

FILEINFO_ATTR 0x0010 File attributes are requested.

FILEINFO_FILEID 0x0020 File Id is requested.

This flag is used when UseFileIds is true.

FILEINFO_REPARSE_TAG 0x0080 File Reparse Tag is requested.

This flag is used when UseReparsePoints is true. The reparse tag must be provided if file attributes indicate that the file or directory is a reparse point.

Desired Access Flags

STG_DACCESS_READ 0x00000001 Grant/deny read access.

STG_DACCESS_WRITE 0x00000002 Grant/deny write access.

STG_DACCESS_READWRITE 0x00000003 Grant/deny read and write access.

File Attributes

FILE_SYS_ATTR_READ_ONLY 0x00000001 The file is read-only.

Applications can read the file, but cannot write to it or delete it. This attribute is not honored on directories.

FILE_SYS_ATTR_HIDDEN 0x00000002 The file or directory is hidden.

The file is not included in an ordinary directory listing.

FILE_SYS_ATTR_SYSTEM 0x00000004 A file or directory that the operating system uses a part of, or uses exclusively.

FILE_SYS_ATTR_DIRECTORY 0x00000010 The entry is a directory.

FILE_SYS_ATTR_ARCHIVE 0x00000020 The entry is an archive file or directory.

Applications typically use this attribute to mark files for backup or removal.

FILE_SYS_ATTR_NORMAL 0x00000080 A file doesn't have other attributes set.

This attribute is valid only when used alone.

FILE_SYS_ATTR_TEMPORARY 0x00000100 A file that is being used for temporary storage.

File systems avoid writing data back to mass storage if sufficient cache memory is available, because typically, an application deletes a temporary file after the handle is closed. In that scenario, the system can entirely avoid writing the data. Otherwise, the data are written after the handle is closed.

FILE_SYS_ATTR_SPARSE_FILE 0x00000200 A file that is a sparse file.

FILE_SYS_ATTR_REPARSE_POINT 0x00000400 A file that is a reparse point or a symbolic link.

FILE_SYS_ATTR_COMPRESSED 0x00000800 A file or directory that is compressed.

For a file, all of the data in the file are compressed. For a directory, compression is the default for newly created files and subdirectories. A filesystem implementation can make use of this attribute by setting the SupportCompressedAttribute property to true and then properly handling the GetFileInfo, EnumerateDirectory, and SetFileAttributes events.

FILE_SYS_ATTR_OFFLINE 0x00001000 The data of a file are not available immediately.

This attribute indicates that the file data are physically moved to offline storage.

FILE_SYS_ATTR_NOT_CONTENT_INDEXED 0x00002000 The file or directory is not to be indexed by the content indexing service.

FILE_SYS_ATTR_ENCRYPTED 0x00004000 A file or directory that is encrypted.

For a file, all data streams in the file are encrypted. For a directory, encryption is the default for newly created files and subdirectories.

Note: This flag is used by NTFS and the OS sends undocumented requests to the filesystem based on this flag. The flag should not be used for files in custom filesystem implementations.

FILE_SYS_ATTR_VIRTUAL 0x00010000 Reserved.

Note: This flag is reserved by the OS and should not be used for files in custom filesystem implementations.

FILE_SYS_ATTR_RECALL_ON_OPEN 0x00040000 The file or directory has no physical representation on the local system; the item is virtual.

Opening the item will be more expensive than normal (e.g., it will cause at least some of it to be fetched from a remote store). This flag is reported by filesystems during directory enumerations.

File Desired Access Flags

DESIRED_ACCESS_FILE_LIST_DIRECTORY 0x00000001 For a directory, the right to list the contents of the directory.

DESIRED_ACCESS_FILE_READ_DATA 0x00000001 For a file object, the right to read the corresponding file data.

For a directory object, the right to read the corresponding directory data.

DESIRED_ACCESS_FILE_ADD_FILE 0x00000002 For a directory, the right to create a file in the directory.

DESIRED_ACCESS_FILE_WRITE_DATA 0x00000002 For a file object, the right to write data to the file.

For a directory object, the right to create a file in the directory

DESIRED_ACCESS_FILE_ADD_SUBDIRECTORY 0x00000004 For a directory, the right to create a subdirectory.

DESIRED_ACCESS_FILE_APPEND_DATA 0x00000004 For a file object, the right to append data to the file.

(For local files, write operations will not overwrite existing data if this flag is specified without FILE_WRITE_DATA.) For a directory object, the right to create a subdirectory (FILE_ADD_SUBDIRECTORY).

DESIRED_ACCESS_FILE_READ_EA 0x00000008 The right to read extended file attributes.

DESIRED_ACCESS_FILE_WRITE_EA 0x00000010 The right to write extended file attributes.

DESIRED_ACCESS_FILE_EXECUTE 0x00000020 For a native code file, the right to execute the file.

This access right given to scripts may cause the script to be executable, depending on the script interpreter.

DESIRED_ACCESS_FILE_DELETE_CHILD 0x00000040 For a directory, the right to delete a directory and all the files it contains, including read-only files.

DESIRED_ACCESS_FILE_READ_ATTRIBUTES 0x00000080 The right to read file attributes.

DESIRED_ACCESS_FILE_WRITE_ATTRIBUTES 0x00000100 The right to write file attributes.

DESIRED_ACCESS_READ_CONTROL 0x00020000 The right to read the information in the file or directory object's security descriptor.

This does not include the information in the SACL.

DESIRED_ACCESS_STANDARD_RIGHTS_READ 0x00020000 Includes READ_CONTROL, which is the right to read the information in the file or directory object's security descriptor.

This does not include the information in the SACL.

DESIRED_ACCESS_STANDARD_RIGHTS_WRITE 0x00020000 Same as STANDARD_RIGHTS_READ

DESIRED_ACCESS_STANDARD_RIGHTS_EXECUTE 0x00020000 Same as STANDARD_RIGHTS_READ

DESIRED_ACCESS_SYNCHRONIZE 0x00100000 The right to use the object for synchronization.

This enables a thread to wait until the object is in the signaled state. Some object types do not support this access right.

DESIRED_ACCESS_FILE_ALL_ACCESS 0x001F01FF All possible access rights for a file.

DESIRED_ACCESS_FILE_GENERIC_READ 0x00120089 A combinarion of flags that allow reading of the file.

Note: Don't match received values against this flag. Instead, use flags that specify the rights that you want to verify or add/remove.

DESIRED_ACCESS_FILE_GENERIC_WRITE 0x00120116 A combinarion of flags that allow modifications to the file.

Note: Don't match received values against this flag. Instead, use flags that specify the rights that you want to verify or add/remove.

DESIRED_ACCESS_FILE_GENERIC_EXECUTE 0x001200A0 A combinarion of flags that allow execution of the file.

Note: Don't match received values against this flag. Instead, use flags that specify the rights that you want to verify or add/remove.

Creation Disposition flags

FILE_DISPOSITION_CREATE_NEW 0x00000001 Creates a new file, only if it does not already exist.

If the specified file exists, the operation fails with an "already exists" error.

FILE_DISPOSITION_CREATE_ALWAYS 0x00000002 Creates a new file, always.

If the specified file exists and is writable, the system overwrites the file. If the specified file does not exist and is a valid path, a new file is created.

FILE_DISPOSITION_OPEN_EXISTING 0x00000003 Opens a file, only if it exists

If the specified file does not exist, opening fails.

FILE_DISPOSITION_OPEN_ALWAYS 0x00000004 Opens a file, always.

If the specified file exists, the operation succeeds. If the specified file does not exist and is a valid path to a writable location, the a file is created.

FILE_DISPOSITION_TRUNCATE_EXISTING 0x00000005 Opens a file and truncates it so that its size is zero bytes, only if it exists.

If the specified file does not exist, the operation fails with a "file not found" error.

Share Modes

FILE_SYS_SHARE_READ 0x00000001 Enables subsequent open operations on a file to request read access.

Otherwise, other processes cannot open the file if they request read access. If this flag is not specified, but the file has been opened for read access, file creation or opening fails.

FILE_SYS_SHARE_WRITE 0x00000002 Enables subsequent open operations on a file to request write access.

Otherwise, other processes cannot open the file if they request write access. If this flag is not specified, but the file has been opened for write access or has a file mapping with write access, file creation or opening fails.

FILE_SYS_SHARE_DELETE 0x00000004 Enables subsequent open operations on a file to request delete access.

Otherwise, other processes cannot open the file if they request delete access. If this flag is not specified, but the file has been opened for delete access, the function fails.

Note: Delete access allows both delete and rename operations.

Storage Type Values

STGT_DISK 0x00000000 Create a regular disk device.

STGT_CDROM 0x00000001 Create a CD-ROM or DVD device.

STGT_DISK_PNP 0x00000003 Create a plug-and-play storage device.

Important: The CBFS Connect system driver must be installed in PnP mode for this option to function properly.

Storage Characteristics Flags

STGC_FLOPPY_DISKETTE 0x00000001 The storage is a floppy disk device.

This flag is not supported when StorageType is set to STGT_DISK_PNP.

STGC_READONLY_DEVICE 0x00000002 The storage is a read-only device.

STGC_WRITE_ONCE_MEDIA 0x00000008 The storage device's media can only be written to once.

This flag is not supported when StorageType is set to STGT_DISK_PNP.

STGC_REMOVABLE_MEDIA 0x00000010 The storage device's media is removable.

Users may remove the storage media from the virtual drive at any time. (Note that this flag does not indicate that the virtual drive itself is removable.)

STGC_AUTOCREATE_DRIVE_LETTER 0x00002000 The system should automatically create a drive letter for the storage device.

Deprecated: Include the STGMP_AUTOCREATE_DRIVE_LETTER flag in the value passed for the AddMountingPoint method's Flags parameter instead.

When this flag is present, the StorageGUID property must be set. This flag only works when StorageType is set to STGT_DISK_PNP.

STGC_SHOW_IN_EJECTION_TRAY 0x00004000 The storage device should be shown in the 'Safely Remove Hardware and Eject Media' menu in the system notification area (system tray).

This flag only works when StorageType is set to STGT_DISK_PNP.

STGC_ALLOW_EJECTION 0x00008000 The storage device can be ejected.

Users may eject the virtual drive at any time. When the virtual drive is ejected, it is destroyed.

This flag only works when StorageType is set to STGT_DISK_PNP.

STGC_RESERVED_1 0x00010000 Reserved, do not use.

STGC_RESERVED_2 0x00020000 Reserved, do not use.

Possible Lock operation commands

FUSE_GETLK 0x0005 The lock owner is to be retrieved.

This command is not used in Windows. In Linux, the process identifier (PID) of the lock owner must be returned. The constant is the same as the F_GETLK constant used in Linux.

FUSE_SETLK 0x0006 Set lock.

If the lock cannot be set immediately, return immediately with an error (EAGAIN and EACCES are recommended). The constant is the same as the F_SETLK constant used in Linux.

FUSE_SETLKW 0x0007 Set lock and wait.

If the lock cannot be set immediately, wait until this becomes possible. The constant is the same as the F_SETLKW constant used in Linux.

Possible Lock operation types

FUSE_RDLCK 0x0000 Read lock should be acquired.

Other applications may read from the locked range but not write to it. The constant is the same as the F_RDLCK constant used in Linux.

FUSE_WRLCK 0x0001 Write lock should be acquired.

No other application may write to the locked range. The constant is the same as the F_WRLCK constant used in Linux.

FUSE_UNLCK 0x0002 The lock should be released.

The constant is the same as the F_UNLCK constant used in Linux.

Cache Check and Repair Flags

CACHE_CHECK_NO_RECREATE_VAULT 0x00000000 If the vault is damaged beyond repair, report an error.

When this flag is used, the CheckAndRepair will return an error if the vault could not be repaired.

CACHE_CHECK_RECREATE_VAULT 0x00000001 If the vault is damaged beyond repair, recreate the vault automatically.

When this flag is used, the CheckAndRepair will delete the damaged vault and create a new one instead.

Cache Cache Operation

CACHE_OP_IDLE 0 The cache is idle.

CACHE_OP_CLEANUP 2 Cache cleanup is being performed.

CACHE_OP_COMPACT 3 The cache's storage file is being compacted.

CACHE_OP_READ 4 The cache is reading some file's data.

CACHE_OP_WRITE 5 The cache is writing some file's data.

CBCache Enumeration Mode

ENUM_MODE_ALL 0 Enumerate all files.

ENUM_MODE_CHANGED 1 Enumerate files with changed blocks.

ENUM_MODE_UNCHANGED 2 Enumerate files with no changed blocks.

ENUM_MODE_LOCAL 4 Enumerate local files.

Refer to the Local Files topic for more information.

ENUM_MODE_ORPHAN 8 Enumerate orphan files.

Refer to the Orphan Files topic for more information.

ENUM_MODE_PINNED 16 Enumerate pinned files.

Cache File Status Kind

FILE_STATUS_CHANGED 1 Retrieves the value of the Changed state of the file.

FILE_STATUS_UNCHANGED 2 Retrieves the value of the Unchanged state of the file.

FILE_STATUS_LOCAL 3 Retrieves the value that indicates whether the file is Local.

Refer to the Local Files topic for more information.

FILE_STATUS_ORPHAN 4 Retrieves the value that indicates whether the file is Orphan.

Refer to the Orphan Files topic for more information.

FILE_STATUS_OPEN 5 Retrieves the value of the Open state of the file.

Cache Flush Action

FLUSH_DELAYED 0 Flush as usual, taking into account any specified delay.

The FlushAfterCloseDelay property specifies the delay. Passing this value to FileClose or FileCloseEx will cause the method to return immediately; flushing is performed in the background. Passing this value to CloseCache has no effect (i.e., the cache is not flushed before being closed).

FLUSH_NONE 1 Do not flush anything.

FLUSH_IMMEDIATE 2 Flush immediately.

Passing this value to FileClose, FileCloseEx, and CloseCache will cause the method to return only after all flushing operations are complete.

Cache File Flushing Mode

FLUSH_MODE_SYNC 0 Flush file data synchronously, blocking until finished.

FLUSH_MODE_ASYNC 1 Flush file data in the background; do not wait until the flush operation completes.

FLUSH_MODE_TEST 2 Check to see if file data exists and needs to be flushed, but do not actually flush anything.

Cache Flushing Result

FLUSH_RESULT_NOTHING 0 The file has no data that need flushing.

FLUSH_RESULT_DIRTY 1 Data in the file need flushing, but no flush operation is running yet.

FLUSH_RESULT_RUNNING 2 Data in the file are in the process of being flushed.

FLUSH_RESULT_SUCCESS 3 Data in the file have been flushed successfully.

Cache Prefetch Constants

PREFETCH_NOTHING 0 Do not prefetch any file data.

PREFETCH_ALL -1 Prefetch all file data.

Cache Progress Operations

CACHE_PO_CLEANUP 1 Cache cleanup

Cache Purge Action

PURGE_DELAYED 0 Purge as usual, taking into account any specified delay.

The PurgeAfterCloseDelay property specifies the delay. Passing this value to FileClose or FileCloseEx will cause the method to return immediately; purging is performed in the background. Passing this value to CloseCache has no effect; the cache is not purged before being closed.

PURGE_NONE 1 Do not purge anything.

PURGE_IMMEDIATE 2 Purge immediately.

Passing this value to FileClose, FileCloseEx, or CloseCache will cause the method to return only after all purging operations are complete.

ReadData/WriteData Event Flags

RWEVENT_IN_PROGRESS 0x00000001 A file's data are being transferred into or out of the cache.

RWEVENT_CANCELED 0x00000002 A data transfer has been canceled for some reason.

RWEVENT_WHOLE_FILE 0x00000004 The entire file is being transferred.

RWEVENT_CONTINUOUS_STARTED 0x00000010 A continuous transfer has started.

RWEVENT_CONTINUOUS_RESTARTED 0x00000020 A continuous transfer has restarted from the beginning.

RWEVENT_CONTINUOUS_FINISHED 0x00000040 A continuous transfer is finishing (i.e., the current block is the final one).

RWEVENT_RANDOM_STARTED 0x00000100 A random-access transfer has started.

RWEVENT_RANDOM_FINISHED 0x00000200 A random-access transfer is finishing (i.e., the current block is the final one).

RWEVENT_BEFORE_END 0x00001000 The current transfer will finish before the end of the file.

RWEVENT_TIL_END 0x00002000 The current transfer will last until the end of the file.

Reading and Writing Capability Flags (Position Related)

RWCAP_POS_SEQUENTIAL_FROM_BOF 0x00000001 No random access allowed; read/write operations must start at the beginning of a file.

When this flag is used in writing capabilities, the component can initiate reading of missing data before writing the changes to build the complete dataset, which is required for correct write operations.

RWCAP_POS_BLOCK_MULTIPLE 0x00000004 Reading/writing is possible at positions that are multiple of the corresponding block size.

The read and write block sizes are specified by the ReadBlockSize and WriteBlockSize properties . When this flag is used in writing capabilities, the component can initiate reading of missing data before writing the changes to build the complete dataset, which is required for correct write operations.

RWCAP_POS_RANDOM 0x00000008 Reading/writing is possible at any position.

When this flag is used, other flags can still alter the positions at which reading/writing are possible.

RWCAP_POS_SUPPL_ONLY_WITHIN_FILE 0x00000010 File position must remain in the range: 0 <= FilePos < FileSize

Reading and Writing Capability Flags (Size Related)

RWCAP_SIZE_WHOLE_FILE 0x00000100 Only whole-file reads/writes are possible; partial reads/writes are not supported.

Note:

  • The ReadBlockSize and WriteBlockSize properties should still be set to reasonable values when using this flag.
  • Use one of the other RWCAP_SIZE_* flags instead of this one if the external storage supports reading subsets of the file.
When this flag is used in writing capabilities, the component can initiate reading of missing data before writing the changes to build the complete dataset, which is required for correct write operations.
RWCAP_SIZE_FIXED_BLOCKS_WITH_TAIL 0x00000200 Reads/writes must be done in blocks of a fixed size, except for the last block, which may be of any size.

When this flag is used in writing capabilities, the component can initiate reading of missing data before writing the changes to build the complete dataset, which is required for correct write operations.

RWCAP_SIZE_FIXED_BLOCKS_NO_TAIL 0x00000400 Reads/writes must be done in blocks of a fixed size, including the last block.

Note: Real file sizes, if needed, must be stored elsewhere when using the cache with an external storage medium that has this characteristic.

When this flag is used in writing capabilities, the component can initiate reading of missing data before writing the changes to build the complete dataset, which is required for correct write operations.

RWCAP_SIZE_ANY 0x00000800 Reads/writes may be done in blocks of any size.

Reading and Writing Capability Flags (Write Related)

RWCAP_WRITE_APPEND_ONLY 0x00000002 Append only; write operations must start at the end of a file.

To overwrite a file, it must first be truncated, and then written to. (Truncation behavior is defined by the corresponding flags.)

RWCAP_WRITE_SUPPL_NOT_BEYOND_EOF 0x00000020 File position remain in the range: 0 <= FilePos <= FileSize

When writing, appending to the end of the file is supported, but writing at positions past the end of the file is not.

RWCAP_WRITE_TRUNCATES_FILE 0x00001000 When writes occur, the last data written becomes the new end of the file.

Any overwritten data, and any data previously present after the written range, are lost.

RWCAP_WRITE_KEEPS_FILESIZE 0x00002000 Normal writing behavior (i.e., writes do not alter a file's size, except possibly to expand it).

RWCAP_WRITE_NOT_BEYOND_EOF 0x00004000 Writes may not extend past the end of a file's current size (i.e., cannot cause a file to grow).

Applications may still adjust the file size beforehand.

Resizing Capability Flags

RSZCAP_GROW_TO_ANY 0x00000001 Files can grow to any size.

RSZCAP_SHRINK_TO_ANY 0x00000002 Files can shrink to any size.

RSZCAP_GROW_TO_BLOCK_MULTIPLE 0x00000004 Files can grow to sizes that are a multiple of the block size.

RSZCAP_SHRINK_TO_BLOCK_MULTIPLE 0x00000008 Files can shrink to sizes that are a multiple of the block size.

RSZCAP_TRUNCATE_ON_OPEN 0x00000010 The external storage (or application itself) supports truncating a file when it is opened for writing.

RSZCAP_TRUNCATE_AT_ZERO 0x00000020 The external storage (or application itself) supports truncating a file to zero at any time (not just when it is being opened).

ReadData/WriteData Event Result Codes

RWRESULT_SUCCESS 0x00000000 Success.

Returning this code indicates that all data were successfully transferred.

RWRESULT_PARTIAL 0x00000080 Partial success.

Returning this code indicates that some, but not all, data were successfully transferred. i.e., when BytesRead < BytesToRead (for ReadData), or BytesWritten < BytesToWrite (for WriteData).

RWRESULT_RANGE_BEYOND_EOF 0x00000081 Specified range is beyond the end of the file (EOF).

For the ReadData event, returning this code indicates to the cache that there is no need to request data beyond (Position + BytesRead), and that the tail should be zeroed instead.

RWRESULT_FILE_MODIFIED_EXTERNALLY 0x00000082 The specified file's size was modified externally.

Returning this code indicates that the operation was not completed successfully because of this conflicting situation.

RWRESULT_FILE_MODIFIED_LOCALLY 0x00000083 The requested file's size was modified locally.

RWRESULT_FAILURE 0x0000008D The operation failed for some transient reason.

Returning this code indicates that, even though the current request failed, it is safe to continue sending requests for both the specified file and others. (Other operations may continue.)

RWRESULT_FILE_FAILURE 0x0000008E The operation failed for some external-file-related reason.

Returning this code indicates that some failure related to the external file has occurred, and it is expected that any further requests made against that file will also fail. The component will not send any more requests for the specified file until the file-specific error is reset as described in Reporting Transfer Errors.

RWRESULT_PERMANENT_FAILURE 0x0000008F The operation failed for some external-storage-related reason.

Returning this code indicates that some failure related to the external storage has occurred, and it is expected that all further requests will also fail. The component will not send any more requests for any file until the global error is reset as described in Reporting Transfer Errors.