Introduction

Welcome to S3 Drive, a powerful solution for accessing remote resources as if they were local drives, eliminating the need to download and upload files you need to work with. Easily manage remote files stored on any S3-compatible service from your favorite file management utility or the command line. Choose from the list of built-in providers or add your own provider to get started. Configure the access credentials and click the "Start" button to start browsing the filesystem.

Features

• Work with a remote file system as if it were a local disk drive.
• Support for multiple drive configurations.
• Run as a Windows service or a desktop application.
• Upload and download files via your favorite file manager, such as Windows Explorer.
• Supports all common directory and file operations such as move, copy, and rename.
• Support for all common providers including:
  • Amazon S3
  • Backblaze B2
  • Cloudflare R2
  • DigitalOcean Spaces
  • IBM Cloud Object Storage
  • IDrive e2
  • Linode
  • MinIO
  • Oort DSS
  • Oracle Cloud Storage
  • Scality
  • Seagate Lyve Cloud
  • Wasabi
  • Any S3 Compatible Service (Custom Provider)

Additional Information

You will always find the latest information about S3 Drive at our web site: www.callback.com. We offer free, fully-functional 30-day trials for all of our products, and our technical support staff are happy to answer any questions you may have during your evaluation.

Please direct all technical questions to support@callback.com. To help support technicians assist you as quickly as possible, please provide a detailed and accurate description of your problem, the results you expected, and the results that you received while using our product. For questions about licensing and pricing, and all other general inquiries, please contact sales@callback.com.

Thank You!

Thank you for choosing S3 Drive. We realize that you have a choice among tools, and that by choosing us you are counting on us to be a key component in your business. We work around the clock to provide you with ongoing enhancements, support, and innovative products; and we will always do our best to exceed your expectations!

Running S3 Drive

S3 Drive supports running on Windows, Linux and macOS. On Windows the application can be configured from the main window. On Linux and macOS the application can be controlled using the command line.

Windows

The application can be started directly from the application's main window, via command line, or configured to run as a Windows service. Please take a look at the Quick Start Guide for a step-by-step introduction. Drives may be managed from the Drives tab (see Drive Management for more information).

Prerequisites

S3 Drive is supported on Windows 7 and Server 2008 R2 with SHA2 code-signing updates and above. The system needs to have KB976932 (Service Pack 1 of the mentioned systems) and KB4474419 (Security Update).

Application

The buttons in the main application window or the context menu in the System Tray may be used to manage the application. To start the enabled drives, click the Start button. Once started, the options to Stop or Restart the application are enabled. Unmount all drives by clicking the Stop button.

All the configured drives will be displayed in list of drives. Click Save Changes to store changes across stopping and starting. For more information on configuring a drive please see the Drive Configuration topic.

Starting as a Windows Service

The application may also be configured to run as a Windows service. To enable running as a Windows service, navigate to the Service tab and check the Run as a Windows Service checkbox. Click Save Changes to save the changes.

After Save Changes is pressed, no user interaction is required to start the application. The Windows service will be configured with a startup type of automatic. The user interface does not need to remain open when running as a Windows service.

Command Line Parameters

The application may also be controlled via the command line. The command line values allow management of the application from unattended configurations such as scripts. The following command line parameters are available:

Linux and macOS

Prerequisites

In order for the application to function properly, the following prerequisites must be met on the target machine.

• .NET: The application requires that .NET 6 or higher is installed to the system. Please refer to the Microsoft documentation for instructions to install .NET on your version of Linux. In most cases, this can be achieved using the following commands.
  • For RedHat/CentOS derivative distributions: sudo yum install dotnet-runtime-6.0
  • For Debian/Ubuntu derivative distributions: sudo apt-get install dotnet-runtime-6.0
• FUSE 2: The application requires that the system's kernel must have been compiled with support for FUSE 2. Additionally, the FUSE 2 libraries must be installed. Support for FUSE 3 is coming soon. In most cases, one of the following commands can be used:
  • For RedHat/CentOS derivative distributions: sudo yum install fuse-libs
  • For Debian/Ubuntu derivative distributions: sudo apt-get install libfuse2

Installing a License

To get started with S3 Drive go ahead and install a license. Run the below command and specify a product key to install a full version license. Optionally run the command without a product key to install a free version license.

  dotnet s3drive.dll install-license [PRODUCT_KEY]

Start the Drive via Command Line

The command line arguments allow managing the application in unattended and interactive configurations. The application allows specifying all connection settings with the start command. Please see the Quick Start Guide for a step-by-step introduction.

dotnet s3drive.dll start --mount-point /path/to/mount/point --provider amazons3 [--region us-east-1] --bucket test --access-key VALUE --secret-key VALUE

The following parameters are available with explicit drive creation.

Start the Drive via Configuration File

A configuration file may be used to specify connection settings and other advanced options. To start the drive with a configuration file, use the following command.

dotnet s3drive.dll start -c /path/to/config/file.conf

The following parameters are available with configuration files.

Stop the Drive

The application can be stopped using the following command.

dotnet s3drive.dll stop

Status of the Drive

The drive status can be queried using the following command.

dotnet s3drive.dll status

Drive Help

For information on the available commands and parameters, use the following command.

dotnet s3drive.dll help

Example Configuration Settings

Below is a copy of the s3drive_example.conf file that is also included with the application. This file provides details about possible settings.

# S3 Drive Example Configuration
#
# This is the S3 Drive configuration file. See the online 
# documentation for more information.
#
#   https://cdn.callback.com/help/NBH/app/
# 
# The syntax used for specifying options in this file is 
# one setting per line in a NAME=VALUE pattern. Uncommented
# options will override the default value. 

[Global]
# Determines whether or not passwords will be masked in the log.
#MaskSensitive=True
#MaxLogLines=1000

# Determines the level of detail in the log output. Valid values range from 
# 0 (off) to 5 (debug):
# 0 Off - Nothing will be logged.
# 1 Error - Only errors will be logged. 
# 2 Warning - Additional warning information will be logged. 
# 3 Info - General information about the status of the connection will be logged. 
# 4 Verbose - logs additional information bout the connection and detailed protocol information.
# 5 Debug - Logs detailed debug information, including raw data.  
#LogMode=3
#LogToFile=True
#LogFile="/root/.callback/s3drive/logs/app.log"

# The logs may be rotated after a specified number of days or 
# log file size. When a log is rotated the existing log file will 
# be renamed to include the last date for which the log applies. 
# Logs may also be deleted after a specified number of days.
#RotateLogDays=0
#RotateLogSize=0
#DeleteLogDays=0

# Defines the number of seconds to wait for the mount operation to complete. 
#MountTimeout=20

# Specifies the number of reconnection attempts when a disconnect 
# occurs. Use a value of -1 to reconnect indefinitely.
#ReconnectAttempts=4

# Specifies the number of seconds to wait before reconnecting.
#ReconnectInterval=5

# Defines the number of seconds to wait for a response to a request. If 
# no response from the server is received after Timeout seconds an error occurs.
#Timeout=60

# Enables compression to be used on the SSH connection if the server supports it.
#UseCompression=False

# Determines whether files that start with a dot or period (.) are 
# marked as hidden when returned in a directory listing.
#HideDotFiles=False

# Determines whether files that start with a dot or period (.) are 
# returned in a directory listing.
#ShowDotFiles=True

# Specifies a comma-separated list of folders that cannot be created on 
# the virtual drive so will not be returned in directory listings. 
#    NOTE: Use with caution. When the system asks us to create these folders
#    we will respond with SYSERR_CANNOT_MAKE. Applications that rely on these
#    operations may behave unexpectedly. Specify a folder relative to the 
#    root of the virtual drive, for example: "/DoNotCreate"
#IgnoredDirs=""

# Specifies a comma-separated list of files the drive should ignore requests for. 
# Requests for the specified files will not be made to the server.
#    NOTE: For each file in the list we will respond with FILE_DOES_NOT_EXIST. 
#IgnoredFiles="autorun.inf,bootmgr,bootnxt,hiberfil.sys,pagefile.sys,desktop.ini"

# Specifies whether the application will use caching for file metadata 
# and directory enumeration entries.
#UseMetadataCache=True

# Defines the proxy connection details. 
#ProxyHost=""
#ProxyPassword=""
#ProxyPort=0

# Determines the type of proxy. (0 - HTTP Tunnel, 1 - SOCKS4, 2 - SOCKS5)
#ProxyType=2
#ProxyUsername=""

[AppInfo]
Name="S3Drive"

# The Drives section contains options for each drive. Multiple 
# drives may be specified, please scroll down for more details. 
[Drives/ExampleDrive]

# Determines whether the drive will be connected.
#Enabled=True

# Defines a bucket or object on the server that will serve as
# a root folder of the drive. If needed, a path portion can be 
# specified after the bucket name. For example: bucket/path/to/object.
Bucket="bucket"

# Defines the location on the local disk where the drive will be 
# mounted. Note the folder must exist and be empty.
MountPoint="/path/to/mount/point"

# Defines the provider and region to use when connecting to the S3 storage provider. 
# To use a custom provider, define the URL configuration setting below. 
# Valid options are :
#   amazons3 (default)
#   backblazeb2
#   cloudflarer2
#   digitalocean
#   ibm
#   linode
#   oortdss
#   oracle
#   seagatelyvecloud
#   wasabi
#Provider="amazons3"
#Region="us-east-1"

# Defines credentials to connect to the S3 storage provider. 
AccessKey="YOUR_ACCESS_KEY"
SecretKey="YOUR_SECRET_KEY"

# Optionally define the URL to connect to the provider.
# By default the application will use the appropriate URL for the 
# provider and region specified. This setting may be used to override
# the default value.
#URL="https://s3.us-east-1.amazonaws.com"

# Defines an account id for authenticating to certain service providers. 
# This is only applicable to CloudflareR2.
#AccountID="" 

# Defines the settings to authenticate via role-based session with the Amazon
# provider using SAML identity provider for authentication. 
# The endpoint should be the ADFS endpoint to make the SAML request. When 
# set, the application will make a GET request to this endpoint to get a SAML
# response. This must be set alongside AssumeRoleARN, ADFSRelyingParty.
#ADFSSTSEndpoint=""
#ADFSRelyingParty=""
#AssumeRoleARN=""
#AssumeRoleDuration=0
#AssumeRolePrincipalARN=""

# Defines the settings to allow authentication via credentials file. The profile 
# for authentication will be parsed from the "credentias" file managed by AWS CLI.
#   NOTE: This setting is only applicable when UseAWSProfile is enabled. By default, 
#   the AWSProfileDir is stored under $HOME/.aws
#AWSProfile="default"
#AWSProfileDir=""

# Specifies the time in seconds for which the directory information will
# be considered valid. This will improve performance by decreasing enumeration requests.
#   NOTE: Specifying a value of 0 means that directory information will not be 
#   cached and will result in a separate request to the S3 server. 
#CacheInfoValidityTime=30

# Specifies the number of allowed concurrent operations.
#CacheConcurrentDownloadFilesMax=8
#CacheConcurrentUploadFilesMax=8

# Specifies the number of allowed connections per file.
#CacheConcurrentDownloadThreadsPerFile=8
#CacheConcurrentUploadThreadsPerFile=8

# Specifies the location on disk where bits and pieces of files will be stored 
# and uploaded at a later time. This allows random writing to be supported.
#   NOTE: By default, the cache is stored in $HOME/.callback/S3Drive/cache/cli_provider_bucket 
#   where "provider" and "bucket" will be replaced with the values provided to the corresponding settings in this file. 
#   If a drive name is supplied, the cache directory will be $HOME/.callback/S3Drive/cache/drivename.
#CacheDir=""

# Specifies whether file content should be cached locally when an application reads
# content from the virtual drive. This is only applicable if caching is enabled. 
#   NOTE: If the remote files are large this could significantly increase the
#   amount of disk space used by the cache. 
#CacheEnbledForRead=True

# Specifies the number of seconds to wait before removing the file from the cache. 
# After the time has elapsed the file will be considered unused. 
#CacheFileDeleteDelay=5

# Specifies the number of seconds to wait before uploading the file. After time has
# elapsed the file will be uploaded to the server. This only applies to large files.
#CacheFileUploadDelay=5

# Specifies the maximum size in bytes of a file that will be uploaded immediately 
# without waiting for the upload delay. This is only applicable when caching is enabled.
#   NOTE: Specifying a value of 0 means the upload delay will apply to all files. 
#CacheFileUploadDelayThreshold=1048576

# Specifies the minimim size of a file in megabytes for which the application should 
# initiate a multipart upload. 
#CacheLargeFileSizeMinimum=5

# Specifies the maximum number of retries for the upload operation. This is only
# applicable when caching is enabled. 
#CacheMaxUploadRetries=3

# Specifies a comma-separated list of filemasks that will not be created. Any file
# which matches the pattern will not be uploaded but will remain accessible on the drive.
#   NOTE: The value provided in this setting will affect only new files, will not affect 
#   files already on the storage, or renamed, or deleted. 
#CacheOnlyFiles=""

# Specifies the number of small files to pre-download. Files that are small enough
# will be pre-downloaded alphabetically. 
#   NOTE: Specifying a value of 0 means files will not be pre-downloaded. 
#CachePreDownloadSmallFileCount=4

# Specifies the size in bytes of a file that will be considered for pre-download. 
# This is only applicable when pre-downloading files is enabled. 
#CachePreDownloadSmallFileSizeLimit=1048576

# Specifies whether the application should authenticate by using the IAM role attached
# to the instance and obtain credentials from the Instance Metadata Service. 
#   NOTE: When IMDSVersion is 1 then the older request/response scheme will be used. The
#   settings are only applicable when the UseEC2RoleCredentials setting is enabled.
#IMDSVersion=2
#IMDSSessionDuration=21600

# Defines a namespace for authenticating to certain service providers. 
# This is only applicable to Oracle Cloud.
#Namespace=""

# Determines whether the drive is accessible by other users of the system.
# When enabled, the drive will be shared with all users of the system. 
# This setting is disabled by default. 
#   NOTE: This setting also needs the setting user_allow_other set in /etc/fuse.conf.
#Shared=False

# Determines the signing standard in outbound requests. Specify a 
# value of 2 to use the older signing standard. 
#SignatureVersion="4"

# Defines whether authentication will be attempted as an unauthenticated user.
#UseAnonymousAuthentication=False

# Defines whether authentication will be attempted using an Amazon credentials file. 
# See AWSProfile and AWSProfileDir for more information. 
#UseAWSProfile=False

# When the application is running within an Amazon EC2 instance, this can be set 
# to authenticate requests using the IAM role attached to the instance.
#UseEC2RoleCredentials=False

# Specifies whether to use server side encryption when objects are uploaded. By 
# default server side encryption will not be used. 
#UseServerSideEncryption=False

# Specifies whether to use TLS to communicate with the server. 
#UseSSL=True

# Specifies that the application should use the accelerated endpoint for 
# transfers to the Amazon S3 provider.
#   NOTE: The bucket must be configured to allow transfer acceleration.
#UseTransferAcceleration=False

# Defines whether the buckets are addresses in the URL using the S3 Virtual
# Hosting format. When set to False, path-style addressing is used instead. 
#UseVirtualHosting=True

# Controls which IP version is used for the connection. These are the valid options:
#    0 - Use IPv4 (default)
#    1 - Use IPv6
#    2 - Try IPv6, but fallback to IPv4 on failure.
#UseIPv6=0

Drive Management

The Start and Stop buttons provide the basic functionality of S3 Drive. Enabled drives are mounted when Start is pressed and all drives are unmounted when Stop is pressed. Start and Stop buttons can also be found in the system tray. The Restart button unmounts and subsequently mounts the currently running drives.

Press the Exit button to unmount any connected drives and stop the application. When running as a Windows Service, the Exit button will exit the graphical application and leave the service running. The service can be stopped from the application interface by clicking the Stop button or stopped manually in the Windows Service Manager.

The Drives tab is the control center for the application. Once a drive is configured it is added to the drive list. Select a drive from the drive list and click the control buttons on the right panel to change it or open it. Alternatively, right-click a particular drive and manage it from the context menu.

Working with Drives

• The Edit... button opens the Drive Configuration window.
• The Duplicate... button creates a new drive using the settings of the selected drive.
• The Delete button permanently removes the drive from the drive list.
• The Enable button designates a drive to be mounted when the Start button is pressed or the service is started.
• The Disable button disables the drive so it cannot be mounted.
• The Open button opens the remote folder associated with the selected drive.
• The Connect button mounts the selected drive without affecting other drives.
• The Disconnect button unmounts the selected drive without affecting other drives.

Configuration details about each drive are included in drive list. Click on the column headers to sort the drives by the respective category.

Drive Configuration

The drive configuration window is used to configure new drives and edit existing drives. It can be accessed by clicking the New... button in the Drives tab or clicking the Edit... button when a drive is selected in the drive list. Drive-specific configuration for example, the connection settings, the drive name, the drive selection settings, and the startup behavior are specified in this window (additional configuration can be specified in the Advanced tab).

Please be aware that configuration changes will not take effect until you click the Save Changes button in the S3 Drive toolbar, which will save the settings to the Windows registry.

Adding a Drive

• To add a drive, click the New... button on the Drives tab of the main window.
• Enter a name for the drive in the Drive Name field, and if needed, choose the desired drive letter. This will be the drive letter where the remote file system will be mounted.
• Select the storage provider, such as Amazon S3.
• Enter your access key in the Access Key field.
• Enter your secret key in the Secret Key field.
• Use the Bucket dropdown list to select the desired bucket.
• The Test S3 Connection button will verify the connection to the S3 provider can be successfully established. When configuration is complete press OK to complete the drive configuration.
• Optional Settings:
  • The Region field shows the current bucket's region and may optionally be set. In most cases there is no need to modify this value.
  • The Sub Folder setting may optionally be set to a folder within the bucket to be used as the drive root. For instance MyPictures/Birthday. This setting is not required.
  • Open Remote Folder On Connect specifies which folder to open in Explorer when the drive is mounted (if any). If set, each time the drive is connected an Explorer window will open at the specified path. Uncheck this box to disable this functionality.

After clicking OK the drive will be added to the drive list. Use the Advanced Settings to set additional configuration settings at the drive level and at the global level.

Provider and Region Management

The ... buttons next to the Provider and Region settings open a separate dialog to manage the available list of providers and their supported regions. When using custom providers, for instance MinIO or Ceph, this dialog may be used to add the necessary connection information. This dialog may also be used to add new regions to existing providers. For more details see the Provider Management page.

Provider Management

The Provider Management dialog provides a way to add additional S3-compatible providers, add new regions to existing providers, and otherwise manage the available providers and regions within the application.

To access the Provider Management dialog, from the Drives tab of the application select a drive and press the Edit... button. From the drive configuration page, click the ... button next to the Provider or Region property to open the Provider Management dialog.

Managing Providers

The Provider section of the Provider Management page defines the services available to the application. Both the built-in providers and the custom providers are shown here. All the providers on this list will be available in the Provider dropbox of the drive configuration page.

The Region section of the Provider Management page lists the regions of the selected provider. The regions will available in the Region dropbox of the drive configuration page. The corresponding New..., Edit..., and Delete buttons may be used to manage providers or regions.

Adding a New Provider

To add a new provider click New... and specify the friendly name of the provider and the endpoint format.

Provider is the friendly name of the service. This name will be added to the Provider dropdown list on the drive configuration page.

Endpoint Format specifies the URL of the service. If the provider supports multiple regions, use the %region% macro in the URL to represent the location where the region name should be substituted. If the service does not contain multiple regions simply do not include the placeholder, for instance a value of 10.0.1.15:9000 for a MinIO or Scality service is valid.

Adding a New Region

To add a new region to an existing provider click New... in the Region section.

Display Name is the friendly name of the region. This name will be added to the Region dropdown list on the drive configuration page.

Region is the region code which will be used in the URL when making requests. For instance us-east-1.

The Endpoint field displays the endpoint for the specific region, based on the endpoint format defined for the provider. This field is informational only and cannot be set.

Advanced

The Advanced tab of the drive settings dialog provides access additional settings which are not commonly used.

Allow all users access to the drive

This option controls whether drives are visible to other users on the system. When checked (default), drives are accessible by all users on the system. When unchecked, the drive is only accessible by the account under which S3 Drive is running.

Note: this setting should not be disabled when running as a Windows Service. When running as a Windows Service with sharing disabled only the identity of the Windows Service itself will be able to access the drive (typically the Network Service identity). For more information see Shared Drives.

Use SSL/TLS

This setting specifies whether SSL/TLS is enabled for the connection. The default value is enabled.

Use Virtual Hosting

This setting specifies whether the buckets are addresses in requests using the S3 Virtual Hosting format. This is enabled by default, and recommended unless there is a specific reason to disable this setting (for instance when using a custom provider).

Cache Settings

See Caching Information for more information.

Cache Directory

This setting specifies the location on disk where data will be stored. This is populated automatically, but may be changed if desired.

Max Concurrent Uploads

This setting specifies the maximum number of simultaneous upload operations that can be in progress at any time. The default value of 8 should work well in most cases. Large values in this setting will increase the number of simultaneous threads in the upload operation.

Max Concurrent Downloads

This setting specifies the maximum number of simultaneous downloads that can be in progress at any time. The default value of 8 should work well in most cases. Large values for this setting will increase the number of simultaneous threads in the download operation. Note: Using a large value may result in intermittent errors from the S3 service which will result in the upload/download operation being retried.

Enable Cache For Reads

This setting controls whether the requested content will be stored locally to speed up subsequent read operations against the files.

Directory (Metadata) caching

This setting controls the lifetime of files and directories returned during directory enumeration. Additionally, it controls the lifetime of metadata serviced from the cache. Low values for this setting means the cache contents expire quickly and will be fetched from remote storage more frequently. High values for this setting will provide a more responsive experience browsing the virtual drive. The default vaule of 30 seconds should work well in most cases.

Import and Export

S3 Drive offers convenient ways to migrate application settings from one system to another. Please note that this migration pertains exclusively to your client settings and does not involve transferring the contents of the virtual drive.

Exporting Settings

To export your settings, follow these steps:

1. Navigate to the Settings tab within the application.
2. Click the Export link to initiate the settings export process.
3. Specify the location and filename where settings will be saved. By default, the settings will be stored in INI file format.

Importing Settings

To import settings from an existing file, follow these steps:

1. Navigate to the Settings tab within the application.
2. click Import link to start the import process.
3. Specify the filename of the .ini file you want to load your settings from.

Note on Sensitive Information

Sensitive fields such as passwords will be decrypted as part of the export process and stored in plaintext in the .ini file.

Additional Configuration

The Settings tab of the defines application-wide settings. Settings such as log settings, reconnection logic, etc. are controlled here.

Advanced settings are available through the Windows Registry which allow setting infrequently used values.

Settings

The Settings tab is used to configure application-wide settings. Please note that configuration changes will not take effect until you click the Save Changes button in the S3 Drive toolbar. Options specific to Drives are configured via the Drives tab.

Logging Options

These options can be used to configure the application's logging features.

• Write Log to a File checkbox defines whether logs are written to a file. By default logs are not written to a file and are only available in the Service tab.
• Log Mode defines the verbosity of the logs generated by the application. The following Log Mode values are supported:
  • Off - Nothing will be logged.
  • Error - Only errors will be logged.
  • Warning - Additional warning information will be logged.
  • Info - General information about the status of the connection will be logged.
  • Verbose - Logs additional information about the connection and detailed protocol information.
  • Debug - Logs detailed debug information, including raw data.
• Log Rotation is available when logging to a file. The logs may be rotated after a specified number of days, and the application can also be configured to automatically delete log files older than a specified number of days. When a log is rotated the existing log file will be renamed to include the last date for which the log applies. For instance myfile.log may be renamed to myfile-2020-07-15.log. The date portion of the rotated log name is in the format yyyy-MM-dd.

Connection Settings

The settings in this section allow granular control over the connection management. In most cases the settings specified here should not be adjusted. The default settings enable automatic reconnection in the case where the connection is dropped.

• Automatically Reconnect specifies whether the application will attempt to automatically reconnect to the SFTP server if the connection is lost. When enabled, the application will attempt to reconnect indefinitely every 5 seconds by default.
• Timeout defines the number of seconds to wait for a response when a request is made. If no response from the server is received after Timeout seconds an error occurs.

For more advanced connection settings, please see Advanced Settings.

Proxy Settings

If a proxy is required to be used when connecting to the SFTP server, the proxy connection details may be specified here. The application supports the following types of proxies:

• HTTP Tunnel (most common)
• SOCKS4
• SOCKS5

Specify the Proxy Type, Address, Port, and, if applicable, the Username and Password. Note that these are global proxy settings and will be used when establishing a connection for each drive. However, you may customize the proxy settings for specific drives to override the global settings within the Advanced tab.

Caching Information

All our drive products use various types of caching. Caching is a complex topic and the actual observed behavior can vary. Depending on the specifics of a situation, multiple different actors may be involved. In S3 Drive, caching will help to ensure file content is uploaded to the remote machine.

The S3 REST API does not support writing a few bytes to a specific offset on an existing file. To support randomly writing files and the propagating changes to S3, we created a file-based cache which holds pieces of files on disk and uploads them once the modifications are complete.

Cache Overview

The cache is a storage location for file content and metadata. When applications interact with files on the virtual drive, they might behave differently. The cache mediates access to allow complete files to be uploaded regardless of how they were written. It also helps reduce the overhead of transmitting all operations received from the OS.

S3 Drive will store cache files in the location specified by the Cache Directory setting in the advanced tab of the drive configuration page. The default location is %ProgramData%\S3Drive\<drive> .

File Read

When an application reads a file on the drive, we will start multiple download threads to download the requested byte range and subsequent chunks. If the file is small then the first thread will download the entire content. Once the file is closed, it is removed from the cache.

The CacheEnabledForRead setting in the registry controls whether the requested content from read operations will be stored in the S3 Drive file cache. The CacheConcurrentDownloadThreadsPerFile config defines how many concurrent download threads can be active at the same time (one thread per chunk).

If the size of the file being read is smaller than CachePreDownloadSmallFileSizeLimit, we will pre-download other small files in alphabetical order in the background. The maximum size for files that will be pre-downloaded is controlled by the CachePreDownloadSmallFileSizeLimit setting. The maximum count of small files that will be pre-downloaded is controlled by the CacheConcurrentDownloadFileCount setting.

File Write

Once the file system driver receives a request to write a file, a corresponding cache file is created with the same size as the original. The modified byte ranges are noted in the cache metadata.

If an application writes to a file on the drive and the file is smaller than CacheFileUploadDelayThreshold and the entire file content is modified, then the file will be uploaded as soon as the file is closed. If only a piece is modified, then after the file is closed the entire content will be downloaded and the modifications applied to it. Then the entire file will be uploaded.

If an application writes to a file on the drive and file is larger than the CacheFileUploadDelayThreshold, then the cache will first ensure the chunks of the file are ready to be uploaded. Any chunks of the file which are partially modified will be first downloaded then modifications are applied to them. All modified chunks are uploaded after the file has been closed and CacheFileUploadDelay seconds have passed without any additional operations on the file.

If the file has any chunks which have not been read or modified before, then S3 Drive uses the CopyParts S3 API call to fill in the content of the new file with existing file data on S3.

File Information

The OS will ask for file information frequently. S3 Drive will store file metadata such as size, creation time, and last modified time for a short time specified by the "Directory (Metadata) caching" setting which corresponds with the CacheInfoValidityTime configuration setting in the registry. If the OS sends subsequent requests for file information during this validity interval, the request will be serviced from the cache without provoking another round trip. This is primarily helpful at the S3-protocol level for deletion performance.

Separate from application-level caching, the filesystem driver will cache file information and service requests from the OS to improve the responsiveness of the virtual drive. This is controlled by the UseMetadataCache configuration setting.

Directory Information

S3 Drive will store the results of previous directory listing requests for a short time specified by the "Directory (Metadata) caching" setting which corresponds with the CacheInfoValidityTime configuration setting in the registry. If the OS sends subsequent requests during this validity interval, the request will be serviced from the cache without provoking another round trip.

Shared Drives

S3 Drive allows drives to be shared among all local users or restricted to the current user. This is controlled by the "Allow all users access to the drive" setting found in the Advanced tab of the Drive Configuration window. It is also controlled by the Shared registry setting for a particular drive (see Drives for more information). When this setting is enabled, the drive is accessible to all local users on the machine. When disabled, the drive is only accessible by the account under which S3 Drive is running.

Accessible to All Users

When "Allow all users access to the drive" is enabled S3 Drive creates a virtual drive accessible to all user sessions. The application maintains one connection to the server per drive and all file system operations that occur use the existing connection. Any user that interacts with the drive inherits the credentials of the connection to work with files on the drive. Settings for the application and drives can be specified in a subkey of the HKEY_LOCAL_MACHINE hive to indicate the settings apply system-wide to any user interacting with the drive.

Restricted to the Current User

When "Allow all users access to the drive" is disabled the drive is only accessible in the current user session. Attempting to access the drive from another user session will cause Windows to throw a "Cannot find drive." error message.

See Also: Isolated Mode

Isolated Mode

S3 Drive can be launched with a command line flag to indicate all settings should be completely separate for each local user account. The /isolated flag controls this behavior (see Running S3 Drive for usage examples). When this flag is specified each local user will be able to manage their own drive list and application settings. The user's list of drives and application settings will persist across logging in and logging out.

This flag will tell S3 Drive to read application configuration and drive configuration from the HKEY_CURRENT_USER hive instead of the HKEY_LOCAL_MACHINE hive. When this flag is specified and a new instance is created, it is called an "isolated instance" or "HKCU instance". When a new instance is created without this flag it is called a "global instance" or "HKLM instance".

The default values of configuration settings are the same regardless of whether you are running an HKCU or HKLM instance. For example, drives are shared among all local users by default. It is a common pitfall to think an isolated instance will only create drives that are private - but that is not the case. To create a private drive in isolated mode turn off the "Allow all users access to the drive" setting as discussed in Shared Drives. Below is a table to illustrate the interaction between shared drives and isolated mode.

Isolated mode is useful in environments where each user needs to mount a drive with their own credentials. Drives can be configured and started manually by each end user or via an automated logon script. This allows many isolated instances to be running at the same time but only one global instance can ever be running at a time. One important note: running as a Windows Service is not possible in an isolated instance.

Advanced Settings

Advanced options for S3 Drive are stored in the Windows registry in HKEY_LOCAL_MACHINE\SOFTWARE\callback\S3Drive\24. This registry key holds settings that are available for S3 Drive globally. Sub-keys at this path hold settings for individual drives, and S3 providers.

The following keys hold configuration information:

The following values can be configured within the root HKEY_LOCAL_MACHINE\SOFTWARE\callback\S3Drive\24 registry key:

Drives

The following values can be configured independently for each drive, at HKEY_LOCAL_MACHINE\SOFTWARE\callback\S3Drive\24\Drives\{Drive Name}:

Providers

S3 Drive stores provider and region information in a tree in the registry. In most cases there is no need to manage these settings directly within the registry. Providers and regions can be managed from the application directly from the Provider Management dialog. See Provider Management for details.

Providers

A registry key is automatically created for each provider supported by the application. These are created at the following path:

HKEY_LOCAL_MACHINE\SOFTWARE\callback\S3Drive\20\Providers\{ProviderName}

The following values are applicable to the providers at this path:

Regions

Regions for each supported provider are specified in a sub-key of the provider key. These are created at the following path:

HKEY_LOCAL_MACHINE\SOFTWARE\callback\S3Drive\24\Providers\{ProviderName}\Regions

Each region is listed as a separate value in the above path. The value should be a String type, where the value name is the region to use in the URL and the value data is the friendly name which will be displayed to the user in the dropdown when creating the drive. For instance:

Content Types

S3 Drive automatically sets the MIME Content-Type on files during upload based on the file extension. For instance a file ending in .txt will use a Content-Type of text/plain.

If desired, the Content-Type mapping may be modified for an existing extension, or a new mapping for a new extension may be added. Note that extensions must include the '.', for example .txt. If a mapping is not found for the file extension the Content-Type will not be set explicitly. The mapping between file extensions and Content-Types is defined by name value pairs in the registry at the following path:

HKEY_LOCAL_MACHINE\SOFTWARE\callback\S3Drive\24\ContentTypes

The following values are applicable at this path:

Trusted Certificates

S3 Drive stores information about trusted SSL certificates at the location noted below. In most cases there is no need to manage these settings directly within the registry.

When establishing a connection to the configured host the trust of the SSL certificate is determined by the Windows security libraries. If the certificate is not trusted on the system a dialog is displayed and the certificate may be explicitly accepted, thereby overriding the system trust. In this case the server's certificate is base64 encoded and stored in the registry for future connections. These are created at the following path:

HKEY_LOCAL_MACHINE\SOFTWARE\callback\S3Drive\24\TrustedCertificates

The following values are applicable at this path:

Troubleshooting

If you are having problems with S3 Drive, our support team is here to help. Visit https://www.callback.com/support/submit.aspx to submit a ticket. Please be as detailed as possible about your issue. It is also helpful if you generate and send a log file to support@callback.com.

How to Generate a Log File

1. Navigate to the Settings tab.
2. Enable Write Log to a File and specify a local file.
3. Increase the Log Mode.
4. Click the Save Changes button.
5. Reproduce the issue.
6. Click the Stop button.

Note that the length of time it takes for any operation is increased when Debug logging is enabled. The amount of data logged with this option enabled is substantial. We recommend enabling these settings for debugging purposes only.

Common Issues

Below you will find a list of common issues reported by our customers that you may find useful in troubleshooting a problem.

The Run as a Windows Service checkbox is enabled but drives are not connected upon startup.

All enabled drives should be connected automatically when the machine turns on, but there can be external factors which can cause this to fail. For instance, the system resources are not ready, the machine is multi-homed, policy of anti-virus software, etc.

When troubleshooting behavior with automatic start first ensure that the startup type is set to Automatic (Delayed Start) by opening the Service Control Manager, right-click the S3 Drive service, and open the service properties.

If you have VPN software installed or the machine is multi-homed, then use Advanced Settings to configure the LocalHost setting in the registry. This setting should be set to the IP address of the network interface to use when connecting.

If neither suggestions help please gather an event log from the system which is failing and send it to us for analysis. This can be done by opening the Event Viewer, expand Windows Logs, then save the Application and System logs by right-click and Save All Events As....

Copying certain files causes a prompt specifying "The file XXX has properties that can't be copied to the new location"

The virtual drive created by S3 Drive is of type exFAT. Certain NTFS specific functionality such as alternate data streams are not currently supported and may result in the warning above. Choosing to proceed with the operation when prompted by Windows is safe and does not affect the file content.

Does regularly clearing the cache improve upload and download speeds?

The S3 Drive cache does not behave like the cache in a web browser, and "clearing the cache" is not something that will help with transfer speeds. S3 Drive manages the cache for you, so you should never need to perform maintenance on the cache to keep it working efficiently. Slow transfer rates could stem from environmental factors outside of S3 Drive. See Caching Information for more details.

S3 connection failed: Connection failed: No connection could be made because the target machine actively refused it.

A connection refused error indicates a TCP-reset flag was received when the connection was initiated. This can mean no process is listening on the specified port, the port is being blocked by a firewall, or the IP address is incorrect.

S3 connection failed: Timeout.

A timeout error is thrown when the remote host does not respond after the specified amount of time in the Settings tab. It could indicate a firewall, antivirus, or proxy is dropping the connection.

S3 connection failed: Could not bind to LocalAddress.

A bind error indicates the application was not able to bind to the default networking interface. This can happen if the machine has multiple networking interfaces. To resolve the issue, set the LocalHost value of the Advanced Settings to the IP address of your default network interface.