Troubleshooting guide

Here are some resources for troubleshooting issues with ShellBoost.

Issue: the ShellBoost Namespace Extension is not visible

This is usually a registration issue. Registration serves two purposes: 1) register the ShellBoost native proxy as a COM object and 2) register the ShellBoost native proxy as a Shell Namespace Extension, including its location in the Shell namespace icon, name, etc. which are defined in the ShellBoost project characteristics on the ShellBoost website.

Check the Project characteristics in the ShellBoost web site. Setting the SFGAO_HIDDEN characteristic for example will, as the name suggests, hides your extension. You should only change these for valid reasons and if you know what you’re doing. The default settings are the following: SFGAO_STORAGE, SFGAO_DROPTARGET, SFGAO_STREAM, SFGAO_STORAGEANCESTOR, SFGAO_FOLDER, SFGAO_HASSUBFOLDER.

If you reset those characteristics, you must unregister what you have registered first, download the new binaries, and register again.

The Shell/Explorer keeps many things (icons, names, etc.) in a private cache, so sometimes, you’ll have to reset it. You can kill all the explorer.exe processes (make sure they are all gone using a tool such as the Task Manager, Details tab) or use the hidden “Exit Explorer” menu trick described here (note this trick can unfortunately keep some rogue explorer.exe instances running, so always check explorer.exe running using the Windows’ Task Manager) https://www.howtogeek.com/198815/use-this-secret-trick-to-close-and-restart-explorer.exe-in-windows/ . Sometimes you’ll have to restart your session or even reboot, but that should be exceptional.

End users (or a testing developer) can use the Windows Explorer UI to delete a Shell Namespace Extension, for example using the DELETE key. In this case, the extension will not be shown anymore. To check that, you can verify there’s no key at HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\CLSID\<Your Folder Class Id> or HKLM\Software\Microsoft\Windows\CurrentVersion\Explorer\CLSID\<Your Folder Class Id>. Your Folder Class Id is visible in the ShellBoost web site project characteristics or available with the ShellBoost.Core.ShellFolderServer.FolderId static property (valid only after the ShellFolderServer is started).

Make sure your namespace extension is not Disallowed. For that, check there’s no <Your Folder Class Id> value in the HKCU\Software\Microsoft\Windows\CurrentVersion\Shell Extensions\Disallowed registry key (or HKLM if you registered for all users).

Make sure your namespace extension is not Blocked. For that, check there’s no <Your Folder Class Id> value in the HKLM\Software\Microsoft\Windows\CurrentVersion\Shell Extensions\Blocked registry key.

Shell extension handlers such as the ShellBoost native proxy run in the Shell process. Because it is a system process, the administrator of a system can limit Shell extension handlers to those on an approved list by setting the EnforceShellExtensionSecurity value of the Explorer key to 1. If this is the case, make sure your namespace extension is Approved. For that, check there’s a <Your Folder Class Id> value in the HKLM\Software\Microsoft\Windows\CurrentVersion\Shell Extensions\Approved registry key. See Microsoft official documentation Registering Shell Extension Handlers for more on this.

Please pay attention to 32-bit vs 64-bit issues, and Release vs Debug (and possibly other combinations) compilation options. If you always compiled as 64-bit, that shouldn’t be an issue, but if you register for both worlds, you can also face multiple registration issues: Release x86, Release x64, Debug x86, Debug x64, etc.

Issue: the ShellBoost Namespace Extension is visible but doesn’t answer Shell requests

This could be a license issue. In the trial version case, the license could be expired. You can check your account and projects in the ShellBoost website to verify if your license has expired or not.

The ShellBoost binaries could have been tampered. Tampered ShellBoost binaries won’t run. If you think that’s the case, you can always download again the ShellBoost binaries from the ShellBoost website.

To check the license is ok, you can also programmatically subscribe to the ShellFolderServer’s Licensing .NET event. This is demonstrated in the Overview sample. Using this sample code, the output for a commercial license (no expiration date) should be something like this:

LicenseDataIsValid: True
LicenseExpirationDate: 31/12/9999 23:59:59
LicenseRegisteredCompany: My Company Name Goes Here

This could be a UAC (“User Account Control”) issue. If your .NET server runs at a different UAC level than the Explorer, of a 3^rd party application hosting Common Controls, then communication may not work. For example, if you run your .NET server as administrator and Explorer is run as normal (not as Administrator).

This could be a disabled UAC (“User Account Control”) issue. See Merged View of HKEY_CLASSES_ROOT and Application Compatibility: UAC: COM Per-User Configuration Microsoft articles about this: “If an application is run with administrator rights and User Account Control is disabled, the COM runtime ignores per-user COM configuration and accesses only per-machine COM configuration. Applications that require administrator rights should register dependent COM objects during installation to the per-machine COM configuration store” which is typically the case when running on default Windows Server or in some Windows Sandbox configurations. By default, when running on Windows Server, you must register a ShellBoost Namespace Extension for the machine, not for the user.

Issue: the ShellBoost Namespace Extension is not visible in Common Dialogs

Unfortunately, there are not always solutions to this as, ultimately, applications host the Common Dialog; The loading process is the application one, not explorer.exe. They can therefore customize it the way they want and prevent any namespace extension to run.

One common case of issues is applications that restrict the Common Dialog views to file system Shell Items only. They can do this with the IFileDialog::SetOptions method: https://docs.microsoft.com/en-us/windows/win32/api/shobjidl_core/nf-shobjidl_core-ifiledialog-setoptions using the FOS_FORCEFILESYSTEM flag. In this case, your extension will only be shown if it’s declared as a file system one using its project characteristics. All folders under the root will also have to be file system compatible. If you own the application using the Common Dialog, then you can always adapt your code to make sure a custom namespace extension is shown.

Even if your extension is fully file system compliant (exposes what ShellBoost call “physical items”), some applications may still fail for various reason purely depending on the way they are coded.

You will have to test your extension with all applications you wish to target (Office applications, notepad, Adobe Reader, etc.) as they can behave differently. Some application even hook (as in “Windows hook”) the Common Dialogs to deeply change their behavior.

In general, Drag & Drop and Copy & Paste operations are better alternatives to Common Dialogs “Save As” because the use a more standard clipboard protocol. For example, Drag & Drop outlook message or attachment from message is demonstrated in the Physical Overview sample.

Another issue you can face is some applications are compiled as 32-bit binaries and therefore use Common Dialogs in 32-bit as well. If your extension runs in a 64-bit OS and you have not registered your extension for the 32-bit world, the Common Dialog will not be able to reach your Shell Folder Server application (even if it’s compiled as “Any Cpu”) and will not display its items. Check the “Intel 32 and 64-bit support” chapter for more on this.

Issue: preview images (from preview handlers) are not displayed for items in a ShellBoost Namespace Extension

Preview images are supported through the Windows Shell’s native IPreviewHandler COM interface. Some preview handlers are provided by Windows (image, simple text, etc.) while others are provided by 3rd party binaries (PDF files, etc.). Handlers are also supposed to implement one of the 3 following native COM interfaces: IInitializeWithStream (strongly preferred and recommended by Microsoft), IInitializeWithFile, or IInitializeWithItem.

Unfortunately, some handlers are not capable of loading Shell Items that are not physical file because they simply don’t implement IInitializeWithStream (uses a stream that can come from anywhere) but IInitializeWithFile (uses a physical file) instead which is the legacy interface.

For these legacy handlers, you’ll have to provide physical Shell Items for the handler to work.

To check if your namespace extension works properly with a preview handler, you can install and test the Microsoft official “Recipe Preview Handler Sample” https://docs.microsoft.com/en-us/windows/win32/shell/samples-recipepreviewhandler . Just add a .recipe file (or virtual file) corresponding to the expected format in a folder in your namespace extension and test this item has a correct preview image.

Another issue you can face is some handlers are compiled as 32-bit binaries. If your extension runs in a 64-bit OS and you have not registered your extension for the 32-bit world, the preview handler will not be able to reach your Shell Folder Server (even if it’s compiled as “Any Cpu”). Check the “Intel 32 and 64-bit support” chapter for more on this.

Issue: what’s displayed in the namespace extension is not expected, icons are wrong, or drag & drop doesn’t work properly (shows stop icon)

When you are developing, like stopping your .NET server and starting it again repeatedly, depending on your context, if you keep Explorer Views opened on your Shell Folder, it’s possible that some discrepancy installs between what your .NET server gives back (notably PIDLs) and what these Explorer Views expect, or what they remember or keep in memory / cache. To make sure you are not in this case, it’s often necessary to close Explorer views that are connected to your server and reopen them. The F5 (Refresh) key pressed in Explorer Views can also reset memory.

Issue: you see too many calls to your .NET implementation

Most of the calls you can see from ShellBoost are in fact initiated by the Shell itself and proxied back to your code. The number and order of calls is not contractual, can change depending on Windows version, machine setup and all sort of other out-of-our-reach reasons.

Any 3^rd party (other Shell Extensions, Context Menu extensions, etc.) can call into your extension indirectly by using the Shell API.

In general, don’t assume you can relate what you can do as an end-user vs what happens technically at your code level. For example, the context menu is asked and used to determine what is the result of a double-click on a Shell Item, although no context menu is shown to the end-user on double-click.

Depending on your context, caching techniques are often used between the Shell calling and your final code executing.

If you think there are too many calls to the ShellFolder’s EnumItems method, make sure you have overridden the two direct access GetItems methods as explained here Items enumeration.