Cloud Folder sample

Source repository: https://github.com/aelyo-softworks/ShellBoost-Samples

The sample needs ShellBoost 1.6.0.0 or higher and is composed of two .NET Core projects:

A .NET 5 Console application (“Cloud Folder”). This application connects using HTTP(S) to the second project.

A ASP.NET Core 5 Web application (“Cloud Folder Site”). This application uses SQL Server or a local Windows NTFS path as a storage backend, and ASP.NET Core SignalR for publishing events to clients.

Optionally, a .NET Core Winforms application (“Cloud Folder Client”) demonstrated in the Cloud Folder sample chapter. This is only used for tests.

Here is the sample’s technical architecture and demonstrated features:

The client application shows the content of the Web Server (the SQL Server database or another backend storage) as a regular folder. It adds two dynamic columns:

The “Server Identifier” column, which is a Guid that uniquely identify all items in the CloudFolder. The Guid also serves as the primary key of all items in the SQL Server Storage case.

The “Sync State” column, which is a text that describes the current synchronization state of a Shell Item.

To maximize Open and Save As Common Dialog scenarios, Shell Items and Shell Folders shown to the end user are all internally associated with a corresponding physical directory or file that will be filled with the content (binary data ultimately from the SQL Server database in the SQL Server implementation case) of Shell Items. All these local files and directories are stored locally in a root folder that’s generally hidden from end-user’s view.

However, it means all applications (Windows applications such as notepad, Microsoft Office apps, etc.) will read and save files in a folder that’s not located, and sometimes, the real physical file path can be shown to the end-user. This is mitigated by the fact the folder where files are really saved to and opened from is customized using desktop.ini techniques which prevents end-user from exploring it using standard UIs.

Other points worth to mention:

The console app keeps an in-memory cache of calls made to the Web Server back end. This cache is automatically cleared using notifications coming from the Web Server, using ASP.NET Core’s SignalR technology. This communication relies on WebSockets and is limited to the Server => Client direction. Size of messages is kept very small to minimize WebSockets calls.

Recycle bin support uses an intermediary physical folder (located in %LocalAppData%\ClouldFolder\RecycleBin). So, when the end-user deletes a file, this file is first downloaded (if needed) from the Web Server, placed into this special folder, then this file from that special folder is recycled using standard Recycle Bin Shell API (FileOperation). After that, the file is deleted from the Web Server. It means if the end-user restores that file from the Windows Recycle Bin, it will go back into that special folder (no UI is provided in this sample to restore the file from that folder to the CloudFolder). A Web Server Recycle Bin could also be implemented and would certainly make sense, but this is beyond ShellBoost use cases.

Since the whole system is asynchronous at all levels, race conditions can (and will) happen during the normal course of operations. This is by design but means errors can be reported by various log and trace systems. These errors (like database locks or file system sharing violation) are handled in most cases.

For the same kind of reasons, users can be sometimes presented with a “Retry” dialog box. For example, if a file not yet locally present is dragged and dropped quickly from the Shell Folder to the physical disk. In this case, if the drop event happens before the file has been completely downloaded, the target will not be able to copy it, yet.

Depending on applications using Shell Items in the ClouldFolder, some local files may not be uploaded immediately when they have been modified. That’s why the LocalEvents class also has a periodic timer that tries to synchronize all files that need to be.

The code doesn’t use milliseconds from all dates to ensure consistent DateTime comparison precision across all involved platforms (.NET, SQL Server, etc.) during synchronization.

The code resolves conflicts without the help of the end-user. The last one to write generally wins.

Installation of the SQL Server database

For the web server to run, if you want to use the SQL Server implementation, you will need to create an SQL Server database. In the screenshot below we used SQL Server Express LocalDB and named the database “CloudFolder”:

The database name must correspond to what’s defined in the appsettings.json file of the ShellBoost.Samples.CloudFolderSite project:

{
  "AllowedHosts": "*",
  "CloudFolder": {
    "FileSystems": [
      {
        "TypeName": "ShellBoost.Samples.CloudFolderSite.FileSystem.Sql.SqlFileSystem",
        "Description": "SQL Server implementation",
        "Properties": {
          "ConnectionString": "Server=(localdb)\\mssqllocaldb;Database=CloudFolder;Integrated Security=True" // TODO: you must create the database, or update this to your context
        }
      }
    ]
  }
}

You don’t need to run a SQL Script to create the required tables and indices, they will be added automatically by the code in SqlFileSystem.cs once ran the first time:

Note: starting with ShellBoost version 1.6.0.4, a ‘Change’ table has been added and some modifications were applied to previous database schema, so it’s possible you’ll have to delete the database (the sample will recreate the tables and indices automatically).

The ASP.NET Core Web Application is configured for ASP.NET Kestrel hosting for demonstration and simplicity purposes but you’re free to use any other host.

Configuring the Web Site

Starting with ShellBoost version 1.8.0.0, you can choose the backend implementation between the SQL Server implementation and the Windows NTFS path implementation. The configuration file looks like this:

{
  "Logging": {
    "LogLevel": {
      "Default": "Trace",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "AllowedHosts": "*",
  "CloudFolder": {
    "ShowChoice": true,
    "FileSystems": [
      {
        "TypeName": "ShellBoost.Samples.CloudFolderSite.FileSystem.Sql.SqlFileSystem",
        "Description": "SQL Server implementation",
        "Properties": {
          "ConnectionString": "Server=(localdb)\\mssqllocaldb;Database=CloudFolder;Integrated Security=True" // TODO: you must create the database, or update this to your context
        }
      },
      {
        "TypeName": "ShellBoost.Samples.CloudFolderSite.FileSystem.Local.LocalFileSystem",
        "Description": "Local Path implementation",
        "Properties": {
          "DirectoryPath": "d:\\cf" // TODO: choose a path where your backend files will be stored
        }
      }
    ]
  }
}

When you start the Web Site sample, if it contains more than one entry in the FileSystems array, it will ask in the console to choose one implementation. It nothing is entered to the console, after 5 seconds, the default choice will be selected automatically, and the sample will start. You can also add other implementations of yours and define a default one, like this:

{
  "AllowedHosts": "*",
  "CloudFolder": {
    "ShowChoice": true,
    "FileSystems": [
      {
        "TypeName": "ShellBoost.Samples.CloudFolderSite.FileSystem.Sql.SqlFileSystem",
        "Description": "SQL Server implementation",
        "Properties": {
          "ConnectionString": "Server=(localdb)\\mssqllocaldb;Database=CloudFolder;Integrated Security=True" // TODO: you must create the database, or update this to your context
        }
      },
      {
        "IsDefault": true, // indicates this is the default choice
        "TypeName": "MyCompany.MyFileSystem",
        "Description": "My implementation",
        "Properties": {
          "Property1": "property value 1" // TODO: update this to your context. Any name/value pair can be used
          "Property2": "property value 2"
        }
      }
    ]
  }
}

Running the sample

To test the sample:

Run the Web Site project, choose an implementation if the program asks for one.

Run the Console app project, Register the namespace extension (1) and run it (2).

Now you can open an explorer view and go to the Samples.CloudFolder shell folder.

If everything went well, you should see an empty folder as we have not added any item yet:

If you right click on the column’s header, you can observe the two dynamic columns we’ve added. You can add folders and files using the Explorer’s command bar:

As you can see the new file has been created to the database, but the “Sync State” columns says there’s “no local presence” because there’s no physical file created locally yet. The database contains two rows: the root item row and the new file item row:

If you double-click on the file, it will 1) ensure the file is local, download it from the server if it’s not, and 2) open the associated application (Microsoft Word for example in the .rtf case). Now the “Sync State” has now been changed to “In Sync”. The various state values are located in the WebItem.cs file.

In the previous screenshot, you can also notice the context menu for an item has an extra “Cloud Folder” menu with two items:

“Free up space” delete the associated local file.

“Download on this device” ensures the associated local file is present on the physical disk.

Winforms client

Starting with ShellBoost version 1.7.0.1, a companion sample Cloud Folder Client Winforms application is provided that can connect to the Cloud Folder Site sample. Its purpose is mostly to be able to create, modify, delete items, and upload files to the sample site, independently from the other Cloud Folder (Shell Namespace extension) and Cloud Folder Sync (On-Demand synchronizing processes) samples. You can also test large file downloads.