3.0.0

If multiple kd runs each utilize 47% per core (meaning 2 = 96% total CPU), my Core2Duo would likely freeze.

If option to run consecutive rather than concurrent, all would be fine.
 
writhziden said:
I tried to learn inno, but I did not have much success. All I could get it to do was install to Program Files (some users like having it in the user directory and do not want it moved to Program Files), or to the Documents directory. Those seemed rather restrictive. I was unable to find a way to have it install to the userprofile directory, and I could not figure out how to edit the script to allow ngen.exe to run after the apps installed. If you have a good guide or you can provide some more advanced script help so I can do what needs to be done with the apps, I would appreciate it.

The documentation I could find online was rather lacking. I could find nothing through Google to help me with writing an inno installer script to run another process when finished... :-{


Here is a list of what I need from the installer:

  1. Install for specific users and not just the user running the installer or the administrator running the installer. Some people use standard accounts and do not have administrative privileges.
  2. Try to run ngen.exe at the end (this requires administrative privileges, so if it fails, the installer still needs to complete).
  3. Set up a file to confirm that the license agreement was accepted.
  4. Set up the directories for the apps settings: backup dmpOptions, download, driverStatistics, forumSettings, and parms.
Click to expand...

I am glad you have resolve the problem with GZ's help now. This is slightly academic for any future, unrelated projects you might have.

I will have a look when I get time. I made some pretty advanced installers with Inno, and found it extremely powerful. I once did a large conditional installer with it (which is why I chose Inno rather than anything simpler for that project). And somewhere, there was excellent documentation, although I think it was in an offline .chm...

Anyway, don't spend any more time on this, but for future projects, I am pretty sure that it does all that you need & much more. I will try to dig out some of my old scripts as examples.

Richard
 
OK, guys, maybe there's something wrong with my installation, but basic analysis in multiple threads is _much_ faster on my system. The app (+kd.exe) spends most of the time on I/O so CPU is rather idle, even for more than 20 threads. I wrote simple test app that invokes some commands: ".reload, !analyze -v, r, kv, lmtn, lmtsmn, .bugcheck, !peb, !sysinfo machineid, !sysinfo cpuspeed, !sysinfo smbios" (every command invoked separatedly) for each of the .dmp files in directory containing the app. I utilize the PowerDbg via automation (which, in fact, uses the cdb.exe debugger), so it's quite heavy solution, but still much faster than current version of the SysnativeBSODApps. Of course, it's only debugger part of the app, but I suspect, that all other activity (downloading drivers.txt file, parsing the text files and combining into result files) is rather negligible. For example, analysis of 90 sample minidumps in 20 threads takes about 6 minutes, which in SysnativeBSODApps takes more than 100 minutes.

m.g.
 
mgrzeg said:
OK, guys, maybe there's something wrong with my installation, but basic analysis in multiple threads is _much_ faster on my system. The app (+kd.exe) spends most of the time on I/O so CPU is rather idle, even for more than 20 threads. I wrote simple test app that invokes some commands: ".reload, !analyze -v, r, kv, lmtn, lmtsmn, .bugcheck, !peb, !sysinfo machineid, !sysinfo cpuspeed, !sysinfo smbios" (every command invoked separatedly) for each of the .dmp files in directory containing the app. I utilize the PowerDbg via automation (which, in fact, uses the cdb.exe debugger), so it's quite heavy solution, but still much faster than current version of the SysnativeBSODApps. Of course, it's only debugger part of the app, but I suspect, that all other activity (downloading drivers.txt file, parsing the text files and combining into result files) is rather negligible. For example, analysis of 90 sample minidumps in 20 threads takes about 6 minutes, which in SysnativeBSODApps takes more than 100 minutes.

m.g.
Click to expand...

What version of kd.exe are you using?

Are you using kd.exe or cdb .exe? I am confused by your message.

kd.exe 6.2.9200.16384 is where I see the kd.exe performance hit. It may be different through PowerDbg since it uses WinDbg, which does not use kd.exe for its analysis.


Why are you just now telling us about this after I've spent hundreds of hours putting together apps that you feel are obsolete? :banghead:


Sorry, but some of the comments were a little upsetting considering how much work I've put in. There are a lot more features to the apps than just the debugging portion. If I had known how to speed that up, I would have, but I was not in the loop with what you had discovered.


Reading up on PowerDbg: This does not look like a viable way to set up the SysnativeBSODApps for the average user. I could create a plugin for it for advanced users, but I think beginning analysts would find learning three different programs at once a bit overwhelming.

Can you supply the version of PowerDbg you use, the version of WinDbg you use, whether the latest version of WinDbg works, whether this works in XP, Vista, 7, and 8, etc.? This needs to be backwards and forwards compatible with all operating systems to be a workable solution. Maintaining the current apps for all four systems has been a challenge, but it has been done, and I do not want to limit the use of the apps for people running XP or 8 (especially 8).
 
Last edited:
Can't seem to get PowerDbg working.
Code: 
The term 'New-DbgSession' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:15
+ New-DbgSession <<<<
    + CategoryInfo          : ObjectNotFound: (New-DbgSession:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
Click to expand...

Anyone with experience know what I might be doing wrong? The module is in C:\Users\writhziden\Documents\WindowsPowerShell\Modules\PowerDbg\PowerDbg.psm1

The documentation for this software is horrible. I'll have to see if I can find some user documentation/tutorials for it to figure it out. Just thought I'd ask here in case others know how to use it. Never even heard of it before today...


I think my knowledge, or lack thereof, of powershell is the limiting factor. Never really used powershell. I've only borrowed others' scripts before.
 
Last edited:
writhziden said:
What version of kd.exe are you using?
Click to expand...

6.2.9200.16384 - I remember that this version is slower, but AFAIK this is the latest one.

writhziden said:
Are you using kd.exe or cdb .exe?
Click to expand...

When running SysnativeBSODApps - kd.exe, but when using my tool - cdb.exe (from PowerDbg)

writhziden said:
Why are you just now telling us about this after I've spent hundreds of hours putting together apps that you feel are obsolete? :banghead:
Sorry, but some of the comments were a little upsetting considering how much work I've put in. There are a lot more features to the apps than just the debugging portion. If I had known how to speed that up, I would have, but I was not in the loop with what you had discovered.
Click to expand...

I do use your app and love this tool. Whenever I can, I write my suggestions (vide my previous posts). I knew that I have to wait for the output from the tool, but few days ago I had to analyze ~20 dumps having not to much time, so I had to do some analysis 'manually' and decided to see if I can speed up a bit by parallelizing the app. This is why I created this simple app... I didn't want to upset you, sorry for that.

writhziden said:
Reading up on PowerDbg: This does not look like a viable way to set up the SysnativeBSODApps for the average user. I could create a plugin for it for advanced users, but I think beginning analysts would find learning three different programs at once a bit overwhelming.
Click to expand...

Forget about PowerDbg. I used it, because it was the simplest solution for me and I don't think it's better in any way comparing to your approach.
You may try my app, where I changed a bit PowerDbg.psm1 and provided some sample .dmps. Steps:
1. Download the tool [CLICK] and unpack to any folder (but please, full path shouldn't contain spaces);
2. Run PowerShell (x86) (and (x64)) and Set-ExecutionPolicy Unrestricted;
3. If you don't have _NT_SYMBOL_PATH, the symbols will be downloaded to the \symbols directory - first run should be longer.
4. I hardcoded x64 version of the tools (Tools\DbgTools\x64), but feel free to put there x86 files if you use x86 OS;
5. Run cmd, cd to the dir containing the app and simply run the DumpInterpreter.exe.

The tool should create 20 threads and spawn cdb (6.2.9200.16384) for each dump and collect results in .dmp.txt files. Please, treat this app as a PoC and test tool, so exceptions may appear. The app is written in .NET, so feel free to use any .NET disasm to see what's going on inside.

m.g.
 
mgrzeg said:
writhziden said:
What version of kd.exe are you using?
Click to expand...

6.2.9200.16384 - I remember that this version is slower, but AFAIK this is the latest one.

writhziden said:
Are you using kd.exe or cdb .exe?
Click to expand...

When running SysnativeBSODApps - kd.exe, but when using my tool - cdb.exe (from PowerDbg)

writhziden said:
Why are you just now telling us about this after I've spent hundreds of hours putting together apps that you feel are obsolete? :banghead:
Sorry, but some of the comments were a little upsetting considering how much work I've put in. There are a lot more features to the apps than just the debugging portion. If I had known how to speed that up, I would have, but I was not in the loop with what you had discovered.
Click to expand...

I do use your app and love this tool. Whenever I can, I write my suggestions (vide my previous posts). I knew that I have to wait for the output from the tool, but few days ago I had to analyze ~20 dumps having not to much time, so I had to do some analysis 'manually' and decided to see if I can speed up a bit by parallelizing the app. This is why I created this simple app... I didn't want to upset you, sorry for that.

writhziden said:
Reading up on PowerDbg: This does not look like a viable way to set up the SysnativeBSODApps for the average user. I could create a plugin for it for advanced users, but I think beginning analysts would find learning three different programs at once a bit overwhelming.
Click to expand...

Forget about PowerDbg. I used it, because it was the simplest solution for me and I don't think it's better in any way comparing to your approach.
You may try my app, where I changed a bit PowerDbg.psm1 and provided some sample .dmps. Steps:
1. Download the tool [CLICK] and unpack to any folder (but please, full path shouldn't contain spaces);
2. Run PowerShell (x86) (and (x64)) and Set-ExecutionPolicy Unrestricted;
3. If you don't have _NT_SYMBOL_PATH, the symbols will be downloaded to the \symbols directory - first run should be longer.
4. I hardcoded x64 version of the tools (Tools\DbgTools\x64), but feel free to put there x86 files if you use x86 OS;
5. Run cmd, cd to the dir containing the app and simply run the DumpInterpreter.exe.

The tool should create 20 threads and spawn cdb (6.2.9200.16384) for each dump and collect results in .dmp.txt files. Please, treat this app as a PoC and test tool, so exceptions may appear. The app is written in .NET, so feel free to use any .NET disasm to see what's going on inside.

m.g.
Click to expand...

.dmps took 390 seconds in your program, took 1058 seconds with the apps. A disparity, but not as large as you had said on your system.

Could you do me a favor and save a .zdn file through the apps to check settings. 90 .dmps should not take 100 minutes.


  1. Click Change Settings

  2. Click File

  3. Click Save As

  4. Choose a name for the .zdn file and click Save

  5. Upload the .zdn file here so I can check your settings.
 
Last edited:
Your time is very close to mine in my app - but what version of the kd.exe and dbgeng.dll do you use in your app? I use the same for both apps, so I'd like to test my app against your version of the dbg tools... I think that's the problem.
I tried different things - reinstaled your app, cleared all the directories, etc.

m.g.
 
Both your program and the SysnativeBSODApps were run using your C:\Tests\Tools\DbgTools\x64\cdb.exe and C:\Tests\Tools\DbgTools\x64\dbgeng.dll files.

I ran the apps with local symbols only checking for symbols if the symbols are found to be wrong by the debugger.
 
Ran with your settings in 1208 seconds... :confused2:

Not sure why it would be ~5 times slower on your system. Do you have an outputDmps directory from the last time you ran the 90 where the output .txt files are stored? If so, could you .zip it up so I can see what the logging information says?
 
Aha, yours always gets symbols online, probably due to your setup with _NT_SYMBOL_PATH, so even though you have the apps set to run locally, the _NT_SYMBOL_PATH is overriding it.

That's my hypothesis, anyway. Regardless, for online symbols, parallelization is very helpful. Both cdb.exe and kd.exe use very little of the processor during symbols downloads, so I plan on building parallelization into the apps with an option for it within the Options portion of the menu bar in the apps.
 
You may be right, but when I run your (or mine) app, I see (using Process Hacker) that kd (or cdb) loads into memory all of the required symbols from local cache just after the kd (cdb) was started. Then it unloads them, reloads, and gets from ms online store only missing (I see only these connections on the network tab in Process Hacker) and in second run doesn't repeat this.
Are you sure, that you run your app with .dlls and .exes provided by me? Please check in Process Explorer or Process Hacker, that your app uses mine (6.2.9200.16384) libs, not yours from your system.

m.g.
 
mgrzeg said:
You may be right, but when I run your (or mine) app, I see (using Process Hacker) that kd (or cdb) loads into memory all of the required symbols from local cache just after the kd (cdb) was started. Then it unloads them, reloads, and gets from ms online store only missing (I see only these connections on the network tab in Process Hacker) and in second run doesn't repeat this.
Are you sure, that you run your app with .dlls and .exes provided by me? Please check in Process Explorer or Process Hacker, that your app uses mine (6.2.9200.16384) libs, not yours from your system.

m.g.
Click to expand...

I am pretty sure it does not matter whether it has to download the symbols or not. Having the path setup to check online is enough to slow things down significantly due to just the check. I will run the apps with the same path you use: SRV*c:/websymbols*http://msdl.microsoft.com/download/symbols and let you know how long the apps take even after I have already downloaded all necessary symbols.
 
The internet connection is not an issue (but, for your information I have ~130 Mb/s). I played a bit with symbol path, as you suggested. That's the key!!! See what happens when you run my tool with local-only symbol srv:

1. Run cmd and cd to the directory with my tool. Then set the _NT_SYMBOL_PATH for your local cache of ms public symbols (mine-c:\websymbols).
set _NT_SYMBOL_PATH=srv*C:\websymbols
2. Then, still in the same cmd run
DumpInterpreter.exe

:)

According to this I've changed a bit my app:
1. At the beginning I changed the env var _NT_SYMBOL_PATH to "" to make sure, that cdb won't use it.
2. Then I set .sympath to online store (but caching in the Symbols subdirectory of my app dir): .sympath srv*c:\PATH_TO_MY_APP\Symbols*http://msdl.microsoft.com/downloads/symbols
3. Next .reload, which downloaded all of the missing symbols
4. Then pointed symsrv to local cache only: .sympath srv*c:\PATH_TO_MY_APP\Symbols
5. The rest of the analysis.

First run against 90 .dmps: 97.462 sec
Second run: 63.426 sec

Total size of downloaded symbols: 170.5 MB

I played with your tool changing my _NT_SYMBOL_PATH and settings in your app and last run against 24 dumps (you have them) was about 600 secs (but I can't find the output in the outputDmps - strange - I saw the .txt files, but can't find them now ;))

m.g.
 
To use your own locally cached symbols (which you may have in your _NT_SYMBOL_PATH, for "srv*c:\websymbols*http://msdl.microsoft.com/downloads/symbols" it is c:\websymbols), you can use the junction:

mklink /j Symbols c:\path_to_your_symbols
(mklink /j Symbols c:\websymbols)

where Symbols is the subdirectory of the directory containing dumpinterpreter.exe.

The version of the app with changes made yesterday is in the attachment to this post. I tried different sets of .dmps and for 100 dumps the time of full analysis ( ".reload, !analyze -v, r, kv, lmtn, lmtsmn, .bugcheck, !peb, !sysinfo machineid, !sysinfo cpuspeed, !sysinfo smbios") is ~75 secs (so, less than a second for one .dmp file, for next runs it is close to 0.5 sec/dump). The analysis time differs from dump to dump and usage of multiple threads helps in optimizing the total time of the analysis - one dump doesn't stop entire process and total time is not just a sum of timings, so the number of threads can be 10 for one or two core systems (with multithreading) and more for other (20 threads is just ok for my quad core - so 8 logical processors). As I mentioned before, the analysis takes lot of I/O time, so I don't feel that my system is busy. Cdb.exe uses ~50-70 MB for every dump and dumpinterpreter.exe uses ~0.4GB for analysis with 20 running threads (about 20MB for one thread).

Did you try to make some changes to your app regarding symbol paths?

m.g.
 

Attachments

I didn't know I needed to make changes to the apps regarding symbols paths. You and one other person are the only people I know who have hard coded the _NT_SYMBOL_PATH and caused problems as a result. It is much safer to run the debugger with the -y flag and enter your symbols path manually or though your program. I cannot protect all users from themselves and make things completely idiot proof. The apps would be a 24/7 job for me if I were to try to do that.

Also, changing the _NT_SYMBOL_PATH environment variable through the apps would defeat the purpose of it, which is to allow users to override other symbols path options through other applications. If you do not want the _NT_SYMBOL_PATH to always go online, then do not set it up to run with the SRV* string and html path. Environment variables are designed to override other variables on the system. Other apps should not touch them. Only the application that the environment variable is designed for should have access to and be able to change that variable (in this case, the Windows Debugging Tools software). Other programmers may disagree, but to me, it is bad programming practice to interfere with how users set up global variables for applications that I did not write.

I already have a lot on my plate for re-designing the apps for multi-threading and redesigning the output to allow users to choose what order to open them in, what method to open them with, etc. I am also working on the new 3.0.0 interface.

I currently have a petition in progress to extend my Ph.D. for another semester because I've had to change my hypothesis twice. I need to get some initial results to talk about within the petition, so that is my number one priority at the moment. The company I work for was also given two new projects last week that I will likely be helping with.
 
Last edited:
mgrzeg said:
You may be right, but when I run your (or mine) app, I see (using Process Hacker) that kd (or cdb) loads into memory all of the required symbols from local cache just after the kd (cdb) was started. Then it unloads them, reloads, and gets from ms online store only missing (I see only these connections on the network tab in Process Hacker) and in second run doesn't repeat this.
Are you sure, that you run your app with .dlls and .exes provided by me? Please check in Process Explorer or Process Hacker, that your app uses mine (6.2.9200.16384) libs, not yours from your system.

m.g.
Click to expand...

Does the same occur in 6.11 & 6.12 - or just 6.2.9200?

EDIT: If the latter, it would answer a lot of questions re: kd
 

Has Sysnative Forums helped you? Please consider donating to help us support the site!

Back
Top