CBFS Storage 2020 Node.js Edition

Questions / Feedback?

Vault Corruption

CBFS Storage vaults have a complex internal structure which may become corrupted if a vault is not closed properly or if some operation is interrupted. Typically, such things are caused by an application crash, a system crash, or (when operating in Callback Mode) due to an error in an event handler. Corruption can also occur if a vault's raw data is modified externally, either intentionally or due to storage failure.

When a vault is open, the IsCorrupted property can be queried to determine if has been corrupted. If a vault is corrupted, any operation may fail with a CBFSSTORAGE_ERR_VAULT_CORRUPTED error code.

Applications can attempt to fix a corrupted vault by calling the CheckAndRepair method. Always create a vault backup before calling CheckAndRepair, as it's possible for data to be lost during the repair process in cases of severe corruption.

Journaling

To reduce the chances of vault corruption in the event of a crash, CBFS Storage can make use of journaling. Journaling works by wrapping vault modification operations in transactions, as follows:

  1. A new transaction is opened by writing information about a change to a journal located within the vault.
  2. The changes themselves are written to the vault.
  3. The transaction is committed by writing another entry in the journal.

If a crash occurs, any interrupted modification operations will appear in a vault's journal as pending transactions. The next time CBFS Storage opens the vault, it will discover any pending transactions and automatically to recover them. During the transaction recovery process, each transaction is either committed or rolled back, depending on its last known state.

Overall, journaling is an effective technique for maintaining data integrity. However, keep the following considerations in mind:

  • When journaling is enabled, all file data changes incur additional write operations; this has a significant impact on overall write performance.
  • Journaling does not provide any kind of data redundancy or consistency; it cannot protect against corruption caused by bit-rot, storage failures, or external modification of a vault's physical data.

CBFS Storage implements journaling as an operational mode rather than a vault attribute, so there is no such thing as a "journaled vault" or a "non-journaled vault". Applications control whether journaling mode is used by setting the UseJournaling property before opening a vault. Therefore, the same vault might be opened with journaling enabled at one point, and opened without journaling enabled at another point.

Note that even when journaling is disabled via the above-mentioned UseJournaling property, "partial journaling" by default takes place: the filesystem still uses transactions and writes metadata (filesystem structure and directory information) as soon as possible. You can disable such partial journaling using the AlwaysJournalMetadata configuration setting.

The filesystem engine will always perform transaction recovery when a vault is opened (if there are pending transactions in its journal), even if UseJournaling is disabled.

(Note: All API members discussed in this topic are available in both CBDrive, CBMemDrive, and CBVault, unless otherwise noted.)

Copyright (c) 2021 Callback Technologies, Inc. - All rights reserved.
CBFS Storage 2020 Node.js Edition - Version 20.0 [Build 7986]