CBFS Connect 2020 Python Edition

Questions / Feedback?

Contexts

It is often necessary for an application to associate certain information with a given file/directory, file handle, or enumeration operation. To assist developers in doing so in a convenient and performant manner, the CBFS class provides context parameters in a number of events.

A context carries an application-defined value that points to some application-defined data, and each file/directory, file handle, and enumeration operation has a separate context associated with it. The CBFS class treats context values as opaque; it stores the context values passed to it by the application, and ensures that the correct values are exposed again whenever some event fires for a particular file, handle, or enumeration; but does not otherwise attempt to use said values in any way.

Context Lifetimes

Contexts can be grouped into a few categories, each of which is subject to a different lifetime:

  • File contexts and directory contexts, which are associated with an open file or directory.
  • Handle contexts, which are associated with a specific open file or directory handle.
  • Enumeration contexts, which are associated with an ongoing enumeration.

File/directory contexts are created the first time a file or directory is opened, and live until the last handle to that file or directory is closed. Handle contexts, on the other hand, are created every time a file or directory is opened, and only live until the associated file handle is closed. For example, consider the following sequence of operations:

Operation on File X Context Creations/Deletions Active Contexts
1. Opened by process A File context FX and handle context HXA created FX, HXA
2. Opened by process B Handle context HXB created FX, HXA, HXB
3. Closed by process B Handle context HXB deleted FX, HXA
4. Opened by process C Handle context HXC created FX, HXA, HXC
5. Closed by process A Handle context HXA deleted FX, HXC
6. Closed by process C File context FX and handle context HXC deleted

Note that when the fire_all_open_close_events property is disabled, the on_open_file and on_close_file events are both only fired once (for steps 1 and 6 above, respectively), which renders handle contexts useless.

File/directory contexts are available in all events corresponding to operations performed on some open file or directory. Handle contexts have similar availability, with the notable exception of the on_read_file and on_write_file events due to behavior caused by the system's memory mapping manager and cache manager. Since these managers act as proxies for file I/O requests, said requests may be buffered initially, and then combined and submitted to the filesystem later in a single request. There's no way to determine which processes contributed to these "aggregate requests", nor is there any guarantee that any/all of them are still alive, hence the lack of handle context parameters in the aforementioned events.

Enumeration contexts are created anytime a new enumeration operation begins, and live until the enumeration operation ends.

All contexts, when created, are created before their corresponding "first event" fires (e.g., on_open_file, on_enumerate_directory, etc.); and when deleted, are deleted after their corresponding "last event" fires (e.g., on_close_file, on_close_directory_enumeration, etc.). However, if a context's "first event" fails, whether expectedly (e.g., due to Security Checks) or otherwise (see Error Reporting and Handling), then that context's value is immediately discarded since its corresponding "last event" won't ever fire.

Additionally, for file and handle contexts, if the route_to_file method is called at some point, then depending on which flags are passed to its Flags argument, the current event may be the "last event" for the current handle context. In such cases, the application must be sure to perform the necessary cleanup actions for the handle context before the event handler finishes. If additional events are expected, it is also reasonable to pass the CBFS_ROUTE_CLOSE_EVENT flag when calling route_to_file so that the on_close_file event will fire again later (at which point the application can perform the necessary cleanup actions).

Context Use-Cases

Contexts are most helpful when used to store information associated with a file/directory or file handle. Typically, contexts are initialized during their corresponding "first event", and then used in subsequent events to speed up those operations.

For example, when an on_open_file event arrives, an application might wish to obtain, e.g., a stream for some real file and store it (directly or indirectly) in the file context. Then, when the application needs to handle subsequent requests like, e.g., on_read_file, it can do so quickly using the stream stored in the file context, instead of having to go through the process of obtaining that stream a second time.

Another example: the first time an on_enumerate_directory event fires, a full list of files can be built, and a reference to it stored in the enumeration context. This list can then be used to speed up subsequent firings.

Using Contexts

In Python, there is no completely safe way to store object references in contexts either directly or indirectly, which is why all context parameters are integers. To emulate such capabilities, the following approach is recommended:

  1. Create a global dictionary instance for the application (i.e., a singleton), with keys that are integers and values of whatever type is desired.
  2. When the application needs to create a context object in an event handler, a "key" can be created using the hash of the full file/directory name (including path), potentially mixed with additional information.
    • For file contexts, a hash of the full file/directory name is sufficiently unique since the context is exposed in all events pertaining to that file/directory.
    • For handle and enumeration contexts, additional information must be mixed in since multiple handle and/or enumeration contexts may be present at once for any given file/directory.
  3. Using the created key, add the object to the dictionary.
  4. Set the Context parameter to the key used in the previous step.
  5. To access the object in a later event, use the key stored by the context to retrieve the object from the dictionary.

Notes:

  • Applications must take care to enforce proper thread synchronization when accessing the dictionary since events are always fired using worker threads. Please refer to the Threading and Concurrency topic for more information.
  • In 32-bit applications, contexts are stored in 32-bit variables internally, thus the higher 32 bits of 64-bit values are lost.

 
 
Copyright (c) 2021 Callback Technologies, Inc. - All rights reserved.
CBFS Connect 2020 Python Edition - Version 20.0 [Build 7880]