CBFS Connect 2020 Python Edition

Questions / Feedback?

CBFS Configuration

The class accepts one or more of the following configuration settings. Configuration settings are similar in functionality to properties, but they are rarely used. In order to avoid "polluting" the property namespace of the class, access to these internal properties is provided through the config method.

CBFS Configuration Settings

AllowOriginatorBufferMapping:   Whether read file, write file, and enumerate directory requests may pass original buffers to event handlers.

When firing the on_read_file, on_write_file, and on_enumerate_directory events, the driver can do one of two things:

  1. Map the original buffer from the underlying request into the application's address space, and pass it to the event handler directly.
  2. Allocate an intermediary buffer to pass to the event handler, and copy the data between it and the original buffer as needed.

The first option is faster since no data copying occurs. However, it also causes the event timeout mechanism to be disabled for particular event fired for the request. The lack of event timeout means that the driver cannot complete the underlying request until the event handler returns, which can lead to situations where the filesystem (or the system itself) becomes "stuck" due to being blocked by the event handler.

By default, this setting is enabled, and the driver will map the original buffer into the application's address space and pass it to the event handler directly if possible. (An intermediary buffer is always used if the original buffer's data length is not a multiple of the memory page size, or if the offset of the buffer is not aligned to the page border.)

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

AllowReadOutsideEof:   Whether read requests beyond the end of a file are surfaced, or automatically denied.

This setting specifies whether read requests beyond the end of a file are surfaced via the on_read_file event, or automatically denied by the driver.

By default, this setting is disabled, and the driver will automatically deny read requests beyond the end of a file by returning the appropriate error.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

AsyncDeleteStorageNotifications:   Whether system broadcasts for virtual drive deletion are sent asynchronously.

This setting specifies whether the WM_DEVICECHANGE broadcast is sent asynchronously (True) or synchronously (False) when the virtual drive is deleted using delete_storage.

By default, this setting is enabled, and the broadcast is sent asynchronously. This is typically sufficient, but applications may disable this setting if they find that Windows Explorer is still presenting virtual drives as available after they've been deleted (which may occur if the application exits immediately after deleting a virtual drive).

ClusterSize:   The cluster size to create the virtual drive with.

This setting specifies the number of bytes each cluster on the virtual drive contains; it must either be set to a multiple of SectorSize, or 0 to use the same value as SectorSize.

All filesystems in Windows organize hard drive storage space based on cluster size (also known as "allocation unit size"). Cluster size represents the smallest amount of storage space that can be used to hold a file. Common cluster sizes for FAT and NTFS filesystems are 4096, 8192, and 16384 bytes.

The class does not use this value for any purpose besides passing it to the OS. Some applications use a drive's cluster size to calculate the size of the buffer used to transfer data in read/write operations; increasing the cluster size may reduce the number of times the on_read_file and on_write_file events fire for such applications.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

CorrectAllocationSizes:   Whether to perform automatic allocation size correction.

This setting specifies whether the class should perform automatic allocation size correction. Allocation size measures how many bytes of storage are actually used to hold a file, and typically a file's allocation size is greater than or equal to its actual size.

However, thanks to the flexibility of CBFS-based virtual filesystems, it's possible for a file's data to be stored remotely, with only a portion of it available locally. In such cases, a file's allocation size may indeed be less than its actual size, since the allocation size tracks the amount of local storage that has been allocated for a file.

The class internally caches the allocation size of each file that's currently open. If this setting is enabled (default), the class automatically corrects the known allocation size to be greater than the file's actual size when necessary; the on_set_allocation_size event is fired anytime this occurs.

Applications can disable this setting to prevent the class from automatically adjusting the allocation sizes they report. When this setting is disabled, and an application changes a file's allocation size, it must call the notify_directory_change method using the CBFS_NOTIFY_FLAG_ALLOCATION_SIZE_MODIFIED flag to inform the class and the OS of the change.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

DeviceChangeBroadcastsEnabled:   Whether the user-mode API broadcasts WM_DEVICECHANGE messages.

This setting specifies whether the class's user-mode API should broadcast WM_DEVICECHANGE messages to top-level windows when necessary.

By default, this setting is enabled, causing the user-mode API to broadcast such messages. However, if the Helper DLL is installed, then it will also send such broadcasts upon being notified by the driver, causing a duplicate message to be sent; applications should disable this setting to prevent the duplicate broadcast.

FastRenameMove:   Whether open files are closed and re-opened during a rename or move operation.

This setting specifies whether, for open files, the class should close and re-open them as part of the overall rename or move operation. (This setting doesn't affect the process of renaming or moving closed files.)

When this setting is enabled, the class performs rename and move operations on open files the same way the system does: the file is opened, renamed or moved, and then closed. The sequence of events is:

  1. The file is opened one or more times by the OS, causing the on_create_file and/or on_open_file events to fire.
  2. <...Other events may fire...>
  3. The file is renamed, causing the on_rename_or_move_file event to fire. If the operation succeeds, the file gets a new name.
  4. <...Other events may fire...>
  5. The file is closed, causing the on_cleanup_file and on_close_file events to fire.

When this setting is disabled, the class performs rename and move operations on open files by closing all open handles to the file, executing the rename/move operation, then re-opening the file using its new name. The file is reopened once for each handle that was closed before the rename/move operation. The sequence of events is:

  1. The file is opened one or more times by the OS, causing the on_create_file and/or on_open_file events to fire.
  2. <...Other events may fire...>
  3. A request to rename or move the file arrives. The class performs (emulates) closing all of the currently-open file handles, causing the on_cleanup_file and on_close_file events to fire (once each for each open handle). All handle closing occurs within a single worker thread without context switches (so the overall operation is fairly fast, but not multithreaded).
  4. The on_rename_or_move_file event fires for the "completely closed" file.
  5. Finally, the file is re-opened by the class using its new name, causing the on_open_file event to fire once for each of the handles that was closed previously. Again, all opens are performed sequentially on a single worker thread, without context switches.
  6. <...Other events may fire...>
  7. The file is closed, causing the on_cleanup_file and on_close_file events to fire.

By default, this setting is disabled, which is the slower but more conservative option; an application whose backend storage supports renames/moves for open resources can try enabling this setting to improve performance.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

FileCachePolicyPurgeOnClose:   Whether file data should be purged from the cache when the file is closed.

This setting specifies whether the class should purge a file's data from the file data cache as soon as the file is closed. By default, this setting is disabled, and the class keeps the data from closed files cached for future use.

Please refer to the Caching topic for more information.

Note: This setting cannot be changed within events.

FileCachePolicyWriteThrough:   Whether file data should be written to storage as it is written to the cache.

This setting specifies whether the class should write file data out to storage (via on_write_file) as it is written to the cache. By default, this setting is disabled, and data is written to the cache first and flushed out to storage later (assuming file caching is enabled).

Please refer to the Caching topic for more information.

Note: This setting cannot be changed within events.

FireSetFileSizeOnWrite:   Whether to fire SetFileSize before WriteFile if writing would expand the file.

This setting specifies whether the on_set_file_size event should be fired prior to firing the on_write_file event if the write operation in question would cause the file's size to increase.

When this setting is enabled (default), and the on_set_file_size operation is successful, then either the data is written to the file data cache or the on_write_file event is subsequently fired. It is also possible for the application to deny the request to expand the file (e.g., due to insufficient storage space).

When this setting is disabled, on_set_file_size won't be fired when a write request would require the file to expand, which results in fewer events being fired and thus increased performance. In this case, if the data is written to the file data cache first and on_write_file is fired later (when the cache gets flushed), then any file expansion errors that occur at that time won't be reported to the originator of the write request. Instead, the OS will report them via a notification on the active desktop.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

FlushFileBeforeUnlock:   Whether file buffers are flushed before completion of UnlockFile operation.

Normally, the cached data of the file are managed by the cache manager automatically. If the application implements a network filesystem and handles file lock operations by locking the file beyond the local enironment, it is desired that the data written under the lock are flushed during unlock to let the application send them to remote locations. Enable this setting for such flushing to occur automatically.

By default, this setting is disabled.

Note: This setting cannot be changed when active is True, and cannot be changed within events.

ForceFileClose:   Whether the driver should force files to close after the last handle to them is closed.

A file is not closed after the last handle to it is closed (i.e., after the last on_cleanup_file event fires); it is closed only after the last on_close_file event fires. If a file is cached or memory-mapped, the system may hold it open for quite a while (minutes, or even hours) after the last handle to it is closed.

If this setting is enabled, then after the last handle to a file is closed, the driver will wait several seconds, and then it will attempt to flush and purge any in-memory file data, allowing it to be formally closed. By default, this setting is disabled, which improves performance.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

LoggingEnabled:   Whether extended logging is enabled.

This setting specifies whether extended logging is enabled for this class; it is disabled by default. Please refer to the Error Reporting and Handling topic for more information.

This setting's value is stored in the registry and is persistent; it requires administrative rights to be changed.

MaxReadBlockSize:   The maximum buffer size allowed for the ReadFile event.

This setting specifies the maximum size, in bytes, of the buffer passed to the on_read_file event. The default value of 0 means "no limit", in which case the class will always pass a buffer large enough to hold all of the requested data in a single on_write_file event.

Please note that unlimited buffer sizes may not be suitable for use-cases where the virtual filesystem is backed by network operations or other "slow" storage, since attempting to handle large (100MB+) blocks may take longer than the configured event timeout duration.

Applications with constraints such as those described above can use this setting to limit how much data they have to process in a single on_read_file event. If a read request arrives requesting more data than the limit allows, the class will "split up" the overall buffer into smaller chunks and fire the on_read_file event individually for each one (and reset the timeout timer between each event).

Note: This setting cannot be changed within events.

MaxWorkerThreadCount:   The maximum number of worker threads to use to fire events.

This setting specifies the maximum number of worker threads the class may create to fire events on when the serialize_events property is disabled. (If serialize_events is enabled, this setting does not apply.)

By default, this setting is set to 0, and the driver automatically chooses an optimal number of threads using this equation: 4 * number_of_processors.

MaxWriteBlockSize:   The maximum buffer size allowed for the WriteFile event.

This setting specifies the maximum size, in bytes, of the buffer passed to the on_write_file event. The default value of 0 means "no limit", in which case the class will always pass a buffer containing all of the data to write in a single on_write_file event.

Please note that unlimited buffer sizes may not be suitable for use-cases where the virtual filesystem is backed by network operations or other "slow" storage, since attempting to handle large (100MB+) blocks may take longer than the configured event timeout duration.

Applications with constraints such as those described above can use this setting to limit how much data they have to process in a single on_write_file event. If a write request arrives carrying more data than the limit allows, the class will "split up" the overall buffer into smaller chunks and fire the on_write_file event individually for each one (and reset the timeout timer between each event).

Note: This setting cannot be changed within events.

MinWorkerThreadCount:   The minimum number of worker threads to use to fire events.

This setting specifies the minimum number of worker threads the class should create to fire events on when the serialize_events property is disabled. (If serialize_events is enabled, this setting does not apply.)

By default, this setting is set to 0, and the driver automatically chooses an optimal number of threads using this equation: max(number_of_processors, 4). If this setting's value exceeds the MaxWorkerThreadCount value, the latter is used instead.

NrDeviceQueryInfoHandling:   Whether the network redirector module should respond to file-specific requests sent to non-file entries.

Some filter drivers send file-specific requests to our network device object. This behavior can lead to a "Device does not recognize the command" error when attempting to create or mount a filesystem. By default, this setting is disabled to prevent unexpected conflicts with other software.

This setting changes the default behavior of our network redirector module to handle such requests.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

SectorSize:   The sector size to create the virtual drive with.

This setting specifies the number of bytes each "sector" on the virtual drive contains. This setting must be set to a value that is a multiple of 512, a power of 2, and not larger than 4096 (e.g., 512, 1024, 2048, 4096).

By default, this setting is set to 512; using a different value is not recommended.

SupportPosixStyleDeletion:   Whether the driver should close all open handles to a file being deleted.

This setting specifies that the driver should handle the FILE_DISPOSITION_POSIX_SEMANTICS flag, when it is set on a file object.

When this setting is enabled, the driver forcefully closes all open handles to a file which is being deleted, if these handles include SHARE_DELETE flag. By default, this setting is disabled to prevent unexpected behavior in applications that own handles.

Note: This setting cannot be changed within events.

SupportSearchIndexer:   Specifies whether the driver must take additional measures to support indexing by Windows Search.

The Search Indexer of Windows 10 has been recently modified in the way that Search Indexer stopped indexing virtual disks. This happens because of the missing mounting point when the disk is created.

This setting, when enabled, tells the driver to create a fake mounting point and use it to work around the Search Indexer bug. By default, this setting is disabled.

Note: This property cannot be changed within events.

SupportUnlockAllRequests:   Whether the driver should fire the UnlockFile event for IRP_MN_UNLOCK_ALL requests or process them internally.

When this setting is disabled, the driver handles an IRP_MN_UNLOCK_ALL request internally and fires the on_unlock_file event for each locked range, passing its offset and length to the event handler.

When this setting is enabled (default), the driver fires on_unlock_file once with offset and length of the locked region being set to -1.

Note: This setting cannot be changed within events.

SynchronousCleanupFile:   Whether the driver should perform cleanup operations synchronously.

This setting specifies that the driver should wait until on_cleanup_file event returns before completing the IRP_MJ_CLEANUP request.

When this setting is enabled, the driver does not complete the IRP_MJ_CLEANUP request until the user mode processing of the on_cleanup_file event returns or a timeout occurs. By default, this setting is disabled to prevent deadlocks due to some error in the user mode processing.

Note: This setting cannot be changed within events.

UpdateFileMetadataOnOpen:   Whether the class should request file info and update internal records during file open.

When enabled, this setting tells the class to fire the on_get_file_info event right after on_open_file and pass the obtained information to the driver, which will use the updated information for the newly opened file. The driver in turn will pass this inormation the OS. By default, this setting is disabled to speed up file opening.

Note: This setting cannot be changed when active is True, and cannot be changed within events.

UseFileIds:   Whether the virtual filesystem supports file IDs.

This setting specifies whether the virtual filesystem should indicate to the system that it supports File IDs.

When this setting is enabled (default), the application must properly handle the on_get_file_info and on_enumerate_directory events and return proper file IDs for files and directories. The on_get_file_name_by_file_id event must be handled as well.

Note: This setting cannot be changed when active is True, and cannot be changed within events.

UseObjectIds:   Whether the virtual filesystem supports volume object ID.

This setting specifies whether the virtual filesystem should indicate to the system that it supports Volume Object ID.

When this setting is enabled (default), the application must properly handle the on_get_volume_object_id and on_set_volume_object_id events. Disable this setting to tell the OS that the Volume Object ID is not supported.

Note: This setting cannot be changed when active is True, and cannot be changed within events.

UseProtectiveFSFilter:   Whether the driver should use a filesystem filter to try and prevent deadlocks.

This setting specifies whether the driver should try to prevent certain kinds of deadlocks by using a filesystem filter that tracks circular access to the virtual filesystem. This may be necessary in order for some third-party applications and drivers to function correctly.
If this setting is enabled, the driver will attempt to activate the filesystem filter and use it to prevent deadlocks caused by cyclical access (if the activation fails, the driver will simply write an entry to the system log). If this setting is disabled, the filter is not used at all.

By default, this setting is disabled; enable it if system deadlocks are encountered.

Note: This setting cannot be changed when active is True, and cannot be changed within events.

UserModeFileCacheSize:   The size of the user mode file data cache.

When the file_cache property is set to fcInternalUserMode (3), this setting specifies the size of the file data cache. This setting cannot be used to change the size of the other file data cache implementations.

This setting's value is specified in bytes and has a granularity of 64 KB, so it must be a multiple of 65536. By default, this setting is set to 0, and the driver uses a default size (currently, 300 MB; i.e. 314572800).

The user mode file data cache's memory region is allocated on a per-process basis. Consequently, this setting's value is synced between all CBFS class instances in the current process, and all associated virtual drives (that are configured to use the user mode file data cache) share that memory region allocated for the current process.

Please refer to the Caching topic for more information.

Note: This setting cannot be changed within events. Additionally, this setting can only be changed if, for all virtual drives in the current process, at least one of the following is true:

  • The drive is not currently mounted.
  • The file_cache property is not set to fcInternalUserMode (3).

USNJournalPolicy:   How USN Journal requests should be handled.

For virtual filesystems that identify as NTFS (via file_system_name), this setting specifies how USN Journal requests should be handled. Possible values are:

  • 0 - Emulate (default): Causes the driver to automatically handle USN Journal requests by emulating them internally in order to satisfy the OS.
  • 1 - Bypass: Causes USN Journal requests to be passed to the application via the on_fsctl event for it to handle.
  • 2 - NotSupportedThe driver does not announce USN Journal support to the system.

Note that the driver does not truly emulate all journaling functionality, it simply tells the OS that the journal is empty. Applications that want more robust functionality should set this setting to 1 - Bypass and implement the desired logic themselves.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

VolumeGuidName:   The GUID of the mounted volume.

Use this setting to obtain the guild of the created disk device. The value is returned as a string in the "Volume{GUID}" format, where GUID is the actual GUID.

WorkerInitialStackSize:   The initial stack size to create worker threads with.

This property specifies the initial size of the stack each worker thread is created with. The system rounds this value to the nearest page.

By default, this setting is set to 0, and the driver uses a default stack size (currently, 1 MB).

Note: This setting cannot be changed when active is True, and cannot be changed within events.

ZeroOriginatorBufferBeforeMapping:   Whether the allocated buffer should be zeroed before it's passed to the event handler.

When an intermediary buffer is allocated for use with the on_read_file and on_enumerate_directory events, it will contain whatever data was previously stored in that memory region, including old data leftover from other processes. This can potentially lead to sensitive information being leaked to the process that initiated the filesystem request (e.g., if the event handler doesn't completely fill the buffer during its execution).

To guard against the possibility of data leakage, this setting can be enabled to instruct the driver to zero intermediary buffers before they are passed to event handlers. Since the zeroing of such buffers takes requires additional processing time, this setting is disabled by default.

Note: This setting cannot be changed after a virtual drive is created, and cannot be changed within events.

Base Configuration Settings

BuildInfo:   Information about the product's build.

When queried, this setting will return a string containing information about the product's build.

LicenseInfo:   Information about the current license.

When queried, this setting will return a string containing information about the license this instance of a class is using. It will return the following information:

  • Product: The product the license is for.
  • Product Key: The key the license was generated from.
  • License Source: Where the license was found (e.g. RuntimeLicense, License File).
  • License Type: The type of license installed (e.g. Royalty Free, Single Server).

 
 
Copyright (c) 2021 Callback Technologies, Inc. - All rights reserved.
CBFS Connect 2020 Python Edition - Version 20.0 [Build 7880]