How to solve Components Scanner reporting thousands of missing registry keys

COMPONENTS 96 MB, SOFTWARE 120 MB, SYSTEM 78 MB, DRIVERS 5 MB.

After writing and running another script to find (p) entries in CBS.log, verify the files are still missing and if so collect them from the repair server, and create an SFXFix.zip; then apply the fix; and then run DISM with /source pointing to '\\repairserver\c$\windows\winsxs' (which replaced two additional files):

DISM /analyzecomponentstore now shows no store corruption. DISM /restorehealth now completes. SFC /scannow fails at 40%. Windows Update was able to install the latest LCU and Defender update on its own but failed to install the latest SSU. There is now 45 GB free on the 127 GB OS partition.

CBS.log has no (p) or (f) errors. There are still some errors reporting CBS_E_NEW_SERVICINGSTACK_REQUIRED.

The failing server will still fail to boot with BSOD 0xc000021a unless I select disable driver signing. Sigverif reports 3 drivers are not signed (agilevpn.sys, intelppm.sys and ndiswan.sys) and no catalog is identified for these items. But Get-WmiObject -class win32_systemdriver | foreach-object { get-authenticodesignature $_.pathname } reports several more that are not signed (amdk8.sys, amdppm.sys, bowser.sys, cdfs.sys, exfat.sys, intelppm.sys, mrxsmb10.sys, ndiswan.sys, peauth.sys, processr.sys, rdpdr.sys, srv.sys, srv2.sys, srvnet.sys, and tsusbhub.sys) or where the systemdriver objects with a null pathname (mpksldc46c7a6) or an invalid pathname that begins with '\??\C' (semav6msr64.sys, mpksldrv.sys). Running the same command on the repairserver VM shows none of these failures with file signatures. The drivers that begin with 'mp' seems to be windows defender related which hasn't been updated on the repairserver VM and that could explain those. But, it seems to me that the other signatures and associated missing catalogs could be part of the problem causing the system to BSOD unless I disable driver signing checks. I don't know how to source or resolve the signatures or catalogs.

ComponentsScanner now only shows catalog error and some mismatched f! entries.

Since DISM completes, should the next step be to do a component store cleanup to, hopefully, get rid of any old entries that are no longer needed? Or will that cause further problems by creating orphaned files and registry entries?

What should my next step be?
 
It's been quite a while since I posted. Trying a bunch of things. Many would not persist a reboot or corruption would come back after trying to install any update even after dism restorehealth succeeds. This hardest part was sourcing some of the KBs, because some are no longer in the catalog listed for various reasons and I had to go to internet archive and other sources to find direct links to those udpates. This weekend I had a major breakthrough. Here's what ultimately worked for this breakthrough...

  • Make a new 'repairserver' VM with no network so it cannot access the internet.
    • Install WSE 2016, cancelling the WSE customization part whenever it pops up.
    • Disable the tasks DiskCleanup\SilentCleanup and servicing\StartComponentCleanup so that it will not remove any superseded update items.
      • Get-ScheduledTask -TaskName 'SilentCleanup' | Disable-ScheduledTask
      • Get-ScheduledTask -TaskName 'StartComponentCleanup' | Disable-ScheduledTask
    • Use sconfig to set it to manual updates only and change its name to repairserver.
    • Reboot.
    • Change the VM network switch to internal switch and enable PSRemoting.
  • On the Hyper-V host
    • Use PSDesiredStateConfiguration to asssure the repairserver has the various WindowsFeatures that match what is currently in use on the damaged server. (possibly not necessary).
    • Use the powershell module kbupdate to remotely install EVERY servicing stack update that was ever released, in their original release order. (KBs are not necessarily numbered in a way that the larger number is always a later release).
    • Use kbupdate to remotely install the most recent two cumulative updates.
    • Reboot repairserver
  • On the damaged server
    • Use sconfig to set it to manual updates only.
    • Use dism /online /cleanup-image /restorehealth /source:\\repairserver\c$\windows\winsxs /limitaccess on the damaged server. Dism completes to 100%
    • Verify the cbs.log indicates that all damaged packages with missing sources were (fixed).
    • Use extract -f:* <path_to_msu> to extract each of the servicing stack updates that are on the repairserver.
    • Use dism /online /add-package /packagepath:<path> to add the extracted .cab of the each of those directly to the image, in the same order as they were added on the repairserver.
    • Reboot.
    • Use dism /online /cleanup-image /restorehealth to confirm there are no corruptions that have missing sources. In my case dism completes to 100%.
    • Use dism /online /cleanup-image /scanhealth to confirm it completes to 100%.
So next, I will shut down the damaged server and create a new snapshop just in case. After bringing it back up I will be try sfc /scannow to make sure it completes to 100% before trying to install the latest cumulative update.
 
DISM now completes. SFC fails at 44%.

So I've seen that the vast majority of Windows\WinSxS\Catalog\*.cat files are not hard linked to the Windows\servicing\packages\*.cat or Windows\Systems32\CatRoot{<guid>}\*.cat. After some research, I think this probably happened when using Windows Server 2016's built-in backup but restoring using the installation media.

Is there a tool that will correctly relink these, or should I write a script to do it? (i.e. calculate the hash of all the servicing cat files, verify the winsxs cat file with the name of the hash has the same contents, find the catroot file with the same contents, and create the hard links?)
 
Please help. I've sourced all the updates, built a repair server, used it as a source for dism restore health and SFC still fails. I'm at a loss for what to do next.

There are no more missing packages with mismatched f! values. No errors at all in ComponentsScanner other than warnings about catalogs that exist but don't have CanonicalData\Catalogs registry entries.

DISM /online /cleanup-image /restorehealth completes. SFC scannow fails at 44%. An in-place upgrade of Windows Server 2016 Essentials to Windows Server 2016 Essentials fails because it loses all the Essentials Role settings such as users, computers, Office 365 integration, shares, etc.

An in-place upgrade guide I saw suggested that WSE 2016 to WSE 2016 would work and that the 'Configure Windows Server Essentials' wizard that comes up on the first reboot would just have a done button since it's already configured. Since the server is in a VM I tried it. But that isn't what happens in my case. And since FSC fails at 44% I can't rule out that as a possible cause. So, I reverted the VM to before I tried it.

I don't want to upgrade or migrate to Essentials 2019 since the Server Essentials role is deprecated (the client computer backups feature is what I really need because I need the archival backup history of old XP client computers). However, I did try this, but it also fails with the error message "0x8007001F - 0x20006 The installation failed in the SAFE_OS phase with an error during REPLICATE_OC operation".

Do I need to extract every MSU and do a file by file binary comparison of each assembly's contents and manifests with the assemblies and manifests on the damaged server?
 
Back
Top