[SOLVED] Another high DPC latency issue with sound distortion

[kallistibrc]

Hey folks,

Got another fun issue that's been extremely frustrating. The short story is that while playing many different games, the audio will suddenly develop a horrid popping reverb that will last for awhile (ten to fifteen minutes usually) and eventually go away.

I've had this happen with: Diablo3 (pretty consistently), Civ5, Borderlands2, and basically everything I've played lately.

With the DPC Latency Checker running on the second monitor everything is green, around the 80us range until the problem occurs. As soon as the problem starts, the bars go solid red, to the top of the window ( > 16000us) and stay there until it stops happening. As soon as the audio returns to normal, the red lines instantly drop away and it goes back to the 80us green range.

If I start xperf running after it redlines, the summary charts aren't that helpful. Nothing has a Max duration above 0.04, and all the avg durations are around 0.002. I'm going to run another test in a bit with xperf running the whole time, and I'll return with more info.

Now for the system details:
OS: Windows 7, 64
MB: Gigabyte GA-X58-UD5 (GIGABYTE - Motherboard - Socket 1366 - GA-X58A-UD5 (rev. 2.0))
Sound: Onboard Realtek
Video: HD5870 (two monitors attached to this card)

Until this morning I had a SB X-Fi in the machine, with the Realtek disabled in the device manager. I yanked the SB out and turned the Realtek Audio on and the problem still occurred. I've disabled a bunch of devices (and left them disabled) and the problem still occurs. I'm down to hard drive controllers and things like that which I'm hesitant to disable.

 

[kallistibrc]

A little more info...

I was attempting to cause the problem to occur again, with xperf running. As these things go, it was, of course, not happening. So since the log was getting rather large, i decided to generate up the .etl file as a "this is an ok log" and then hopefully be able to generate up a bad log for comparison.

As I started running the xperf -d, the HD started churning like normal and then blammo, DPC spiked and stayed through the roof. So I figred off xperf again and -d'd it to a different file for comparison. The "good" log has USBPORT.SYS at the top, with max duration of 0.08 and avg of 0.009. So that looks fine I think. The "bad" log has ATAPORT.SYS at the top, Max duration of 37.7 and avg of 0.071.

So I'm thinking this might have something to do with the HD or controllers therein? The disks in the system are:

Main Disk: (OS + Storage) Western Digital 1TB 7200 RPM SATA 6.0Gb/s
- This is on the ATA Location 0 (Channel 1, Target 0)

Game Disk: OCZ-VERTEX4 ATA SSD
- This is on Location 1 (Channel 0, Target 1)

I recently added the SSD, but the problem was occurring long before that.

I know it's probably not a whole lot to go on, but any ideas would be appreciated. If there's any other logs or files that would help, please let me know and I'll be glad to generate em up.

 

[Digerati]

I would disconnect any unused USB devices. If using EasyTune - Gigabytes hardware monitor and clocking tweaker - I would disable that as that has been known to cause these problems.

 

[kallistibrc]

I would disconnect any unused USB devices. If using EasyTune - Gigabytes hardware monitor and clocking tweaker - I would disable that as that has been known to cause these problems.

The only things plugged in right now are my mouse and keyboard. I went ahead and disconnected the front case USB connectors from the motherboard to totally isolate those out. I had EasyTune installed, but not running. For kicks, I turned it on and watched the DPC Latency become an issue right away, but not the way my problem normally is. The monitor would have occasional spikes, every other sampling pool actually, but only into the 4000us range. Not the issue I'm having, but at least I can confirm that program does in fact cause some issues.

Anyway, I went ahead and formatted the machine, and reinstalled the OS. The only thing on it are all of the Windows Updates, Diablo3, Starcraft2, and Steam. (Had to get some things to test with of course *wink*) At first I was all excited and I thought the problem was gone. But after about 40 minutes it showed up again, exactly as before, same durations, etc. I figure with my system a LOT cleaner, it should help to diagnose and find the causes. I'll be back with some more updates once I run xperf some more on the clean system.

Thanks for your suggestion Digerati. I'm still looking. :)

 

[kallistibrc]

Ok, so still not fixed, but here's some more info:

I did a ton of research into my motherboard and peripherals. Turns out I had my USB mouse plugged into a eSata/USB port on the back. Not a big deal, but I went ahead and disabled the eSata capability in BIOS and moved the mouse to a USB 3.0 port. USBPORT.SYS no longer shows up, but the problem still persists.

So the next thing I did was to unplug both the bluray and cdrom drives. No power or connectors going to either device. I also realized that I had both my SSD and non SSD HD plugged into the 3gb/s SATA connectors (SATA2_0 and SATA2_1). I moved them both over to the 6gb/s connector running on the Marvell 9128 chip (GSATA3_6, GSATA3_7). The effect of this is that I no longer have ATAPORT.SYS showing up in my DPC charts. I've also disabled the other GSATA controllers in BIOS (GSATA2_8, GSATA2_9), but left all the SATA2_* controllers on (basically because I couldn't find an option to turn em off) Now, instead...

Sometimes the DPC chart tops out the xperf charts with dxgkrnl.sys and other times it's SCSIPORT.SYS. Once the problem happens from one of those two, if I leave LatencyMon running while the DPC Latency Checker tool is redlined, then I'll end up with a bunch of different drivers all at 37ish in LatencyMon. I think this is probably because whatever is causing the problem in the first place grabs a lock or something like that and all the others have to wait, and as soon as it releases they all release too.

Anyway, what I'm thinking of doing now is to go ahead and move the non SSD HD back to one of the SATA2 controllers. I'm hoping that by seeing which of SCSIPORT.SYS and ATAPORT.SYS tops the charts I'll be able to see if it's one of the HDs causing the issues.

Any suggestions on other things to continue trying would be much appreciated, as well as any other ideas or explanations that ya'll might have.

TLDR: Trying a few more things to isolate the cause, looks like it might be hard drive related. One question: Why would SCSIPORT.SYS be acting up if there are no SCSI devices plugged in? Does that driver somehow deal with SATA drives?

 

[Digerati]

SCSI is often used in the name drivers that are not EIDE (PATA). Don't know why.

Do make sure after these changes you still have audio problems, not just high DPC readings. I personally would not worry if I could not see or hear any performance problems.

 

[satrow]

Are there not Intel SATA connectors on that 'board? The downloads offered for it suggest that there are.

I'd fully uninstall EasyTune (and any other motherboard/graphics 'utilities') to test (assuming they were installed this time); note that even if you don't have them running, it doesn't mean that the drivers are not loaded (similar applies to antivirus software/suites and firewalls, disabling them won't completely disable them, drivers are still loaded and frequently still 'blocking').

The intermittent nature of the trigger makes me wonder if some Scheduled Task or power-saving feature isn't behind this.

 

[Digerati]

note that even if you don't have them running, it doesn't mean that the drivers are not loaded

Or services running. I agree, uninstall it while testing.

 

[kallistibrc]

Do make sure after these changes you still have audio problems, not just high DPC readings. I personally would not worry if I could not see or hear any performance problems.

Yeah, the sound artifacts are the first thing I notice, then I look over to the other monitor at the DPC monitor tool and it's pegged.

Are there not Intel SATA connectors on that 'board? The downloads offered for it suggest that there are.

I'd fully uninstall EasyTune (and any other motherboard/graphics 'utilities') to test (assuming they were installed this time); note that even if you don't have them running, it doesn't mean that the drivers are not loaded (similar applies to antivirus software/suites and firewalls, disabling them won't completely disable them, drivers are still loaded and frequently still 'blocking').

The intermittent nature of the trigger makes me wonder if some Scheduled Task or power-saving feature isn't behind this.

Yeah, when I did the reinstall I didn't install any of Gigabyte's tools (EasyTune, DES2, etc etc). I also haven't installed a virus scanner yet. Windows Firewall is on, but that's it.

As for the Intel SATA connectors, there are 6 of them, and thats what I had everything plugged into at first. My recent experiment of putting the SSD on the Marvell controller and the older Western Digital HD back on the Intel Sata2 has resulted in the following. The SCSIPORT.sys driver has dropped completely off the chart, and only ATAPORT.SYS now shows issues.

So yeah, I'm *thinking* this means one of a few things:
1) There's something up with that older HD.
2) There's something up with that controller (it's in a different SATA2 port, but still the same controller)
3) There's something else up and it only manifests when reading from the slower HD.

I'm going to try moving the older HD onto the GSATA2 controller (currently haven't used this controller at all) to see if I can isolate if it's #1 or #2 above. I'm leaning towards #1 unfortunately.

The issue popped up right at system boot this time, and what's interesting is that right around the same time, this showed up in the Kernel-EventTracing log: "Session "NT Kernel Logger" failed to start with the following error: 0xC0000035" Not sure if that's related or not. Going to see if it occurs again. There are no other errors like that from other times I've had the issue occur (like *right now* as I'm typing this)

I'm also considering putting both controllers in AHCI mode (currently in IDE I believe). Pretty sure I'm going to have to do the whole registry edit, multi reboot nonsense to get that set though. Meh.

Thanks for all your advice folks.

 

[satrow]

The main reason I queried the SATA connections was that it looks like there are 3x different sets, GigaByte/JRaid, Marvell and Intel. I would double-check with the 'boards manual against a visual of the 'board, also check exactly which drivers (3rd party only) are loading for the SATA connection(s) currently in use.

I would certainly enable AHCI and disable anything else that you can from the BIOS.

Can you test with a single drive, maybe copy a game onto the SSD to speed up the trigger?

 

[kallistibrc]

The main reason I queried the SATA connections was that it looks like there are 3x different sets, GigaByte/JRaid, Marvell and Intel. I would double-check with the 'boards manual against a visual of the 'board, also check exactly which drivers (3rd party only) are loading for the SATA connection(s) currently in use.

You're exactly right. So the history is that initially I had everything (cdrom, bluray, both HDs) plugged into the Intel connections (of which there are 6). I've long since unplugged both optical drives completely. After some research I moved both HDs to the Marvell connections (only 2 of em) and thats when the SCSIPORT.SYS started acting up. I left the SSD in the Marvell and moved the HD back to the Intel and ATAPORT.SYS is the culprit again.

I *just* now enabled the GigaByte/Jraid in bios (those are the GSATA2_9 and GSATA2_8 ports listed in the manual). After putting the HD onto that controller (which is in IDE mode) the machine BSODs on windows boot. I haven't updated the drivers for that port or anything yet so I'm not too shocked. I put the HD back on the Intel port for now. I'm probably gonna skip testing that port for now and proceed with the AHCI tests.

Also, as for the drivers... the Marvell driver is the latest I could find on Gigabyte's website. 3/17/2010, version 1.0.0.1036. As for the Intel drivers... Device Manager is showing something that is a bit confusing actually. There are 6 things listed there:
ATA Channel 0
ATA Channel 0
ATA Channel 1
ATA Channel 1
Intel(R) ICH10 Family 2 port Serial ATA Storage Controller2 - 3A26
Intel(R) ICH10 Family 2 port Serial ATA Storage Controller1 - 3A20

Not sure why the ATA Channels are listed twice. I currently have both Channel 1 listings disabled. The controllers listed as Intel are both using the same driver, 6/4/2009 version 9.1.1.1013. I'm strongly suspecting that these need to be updated, but unfortunately I can't find a good driver on Gigabyte's site to use. There is an Intel SATA Preinstall driver listed, and I'm wondering if I can use that even though it's not during preinstall? Should be fine I think. Trying to install the "Intel Rapid Storage Technology" link just results in some error about my not having the proper hardware.

Can you test with a single drive, maybe copy a game onto the SSD to speed up the trigger?

I can test with just the old HD (which seems to be where the issues come up) but not with just the SSD. The OS is installed on the old HD. I'm seriously contemplating driving to Frys and picking up another SSD to install the OS on and using the old HD for backup storage or something. Before I do that though I'm going to do the AHCI switch. Of course, I have to do the registry tomfoolery first.

Aaaand it's time to pick up my daughter from daycare, so I'm likely done tinkering with this for today. I'll do more work tomorrow and let y'all know what I find.

 

[kallistibrc]

Soooo I haven't picked up my daughter yet. But what I have done is enable AHCI mode. The first thing I've noticed is that, of course, my machine is SOO much faster (as it should be. Can't believe I was in IDE mode with an SSD for all this time). Also the other thing is that in the IDE ATA/ATAPI controllers now instead of having Channel 0 and 1 both listed twice, there are now Channel 0-5 (each only once) and "Standard AHCI 1.0 Serial ATA Controller" listed. So that makes a LOT more sense to me.

Ok... now I'm really going to get her. Take care all and again, thanks for all your help. (Wish I could send you both some Scotch or something)

 

[satrow]

I'll see if I can find a URL for some cask strength Talisker now

Keep us in touch with your progress.

 

[kallistibrc]

So I thought I would go ahead and post my current BIOS settings here for y'all and see if there's anything odd that you can see. I've put a few notes on some of the lines, in brackets at the end [like this]. Also, as for the format, each nested menu item gets tabbed in one spot. And the main menu options are separated by the dashed line thing. If there's anything here that my shorthand is confusing on let me know and I'll clarify. Thanks again.****[The formatting was horrible, I'm including it in a text file]And, as for Talisker... I am a fan, but I'm leaning towards my favorite Scotch being the Macallan Cask. It's cheap enough (usually around 50 to 60 a bottle), and just amazingly good. I'm not usually a Macallan fan, but the cask is fantastic. :)

 

[kallistibrc]

OK, here's the post with the BIOS settings attached in a text file.

 

[kallistibrc]

Well... I think I finally figured it out.

Drove on over to Fry's today and bought another 90 gig SSD to put the OS on. Popped it in, changed the boot order of the hds, and then installed Win7 on the new 90 gig. It booted up fine, ran all my updates and driver configs etc. I still had the old Western Digital in there, but empty. Oh, and I had to plug in my DVD drive to do the install. Both the DVD and the WD-HD were on the old Sata2 controller. Everything seemed to be running fine for awhile...

So I decided to run the Windows Experience Index tool with LatencyMon and the DPC monitor running . It tends to hit all the system components and can cause the problem to occur pretty reliably. There was some chugging as it ran the intensive CPU tests which I expected, and then when it got to the HDs it redlined again. Not solid, but pretty close for awhile. So I just went ahead and disabled the WD-HD in the device manager and reran the WEI tool. Same few lines during the CPU test but the hard drive stuff went through clean.

I've now since played all my "trouble" games and got no issues. I then left one of my games booted up and went and watched a movie. Nothing in LatencyMon is above 0.18.

Sooooooo I'm thinking it's probably just something faulty/failing with the old WD-HD. I'm leaving it plugged in but disabled for now. I'm wondering if I can enable it for just long term storage and backup. It's a 1 TB drive so I'm reluctant to just 'lose' it.

Anyway, I'm gonna go ahead and call this one "cured" for now. Hopefully I won't have to call jinx on myself. Thanks again for your help. :)

 

[kallistibrc]

I'm still gonna call this fixed but...

Was playing some D3 with a buddy and plugged in my headphones which required a driver install. After setting up the voice chat and starting the game up, blammo the problem popped up again. This bugged me a lot, so I opened up the device manager to take a look at stuff and somehow the HD Audio on my video card was enabled again (i think from the driver update somehow?). I disabled it and rebooted and then fired it all up, met him in game, and played for about 4 hours error free. Been using the computer pretty much constantly ever since and no issues.

I'm slowly starting to add things back into the machine, so if the problem all of a sudden becomes an issue again I'll be back, but for now I think it's solved.

 

[Digerati]

I don't think I would disable SMART monitoring. While SMART errors can be annoying and may result in "boy crying wolf" problems, SMART errors can be invaluable in reporting drive problems BEFORE data loss.

It is interesting the problem was there when HD Audio was enabled and gone when disabled. That's where I would focus my attention. We've been looking at high DPC readings causing audio problems, maybe it is audio problems causing high DPC readings.

 

[kallistibrc]

Yeah I was contemplating turning off SMART monitoring as I was thinking maybe it was sending a notification and that's what was causing the interrupt.

So now that things have been stable (barring that one weird issue with the driver re-enable) I'm going to go ahead and install the other Radeon HD 5800 and put em in SLI mode (both with the HD Audio disabled in the control panel) and make sure that all is well. And then, for kicks, I'm going to turn the HD Audio back on for both of them and see if I can't make the issue occur. If so, then I plan on disabling them both, then reenabling the 1 TB drive. Just some more diagnostic junk basically. If there is any issue I'm going to take the 1 TB drive back out.

Also, even though the video card's HD Audio gets disabled from the device manager, I'm wondering if the card doesn't do something funky itself. I'm leaning towards Digerati's suggestion that maybe it's the Audio causing the high DPC and thereby causing it's own stuttering. More specifically, I'm wondering if perhaps the slower 1TB HD drive causes a tiny hiccup but then the video card goes wonky with it's Audio and then gets in some sort of really bad state where it just keeps the high DPC going.

And for more fun, I plugged my guitar into the USB 2.0 slot and played for an hour or so and couldn't get it to pop up again. :)


BTW, do I change the subject to SOLVED (now that I'm pretty sure it is) or does somebody else do that?

Thanks again,

B

 

[Digerati]

I'm going to go ahead and install the other Radeon HD 5800

Remember, graphics cards are often the most power hungry devices in our system, often much more so than CPUs. So you MUST ensure your power supply is capable of supporting the added demands BEFORE installing that power hungry card.

If you feel the problem is resolved and if the site lets you mark it as solved, you can change it.