Yes, avoid PSUs that come with cases. I recommend starting with the CPU first, then PSU, then mobo.
I have no idea what VG means by this, and I imagine you don't either.
I'll clarify a little on PSUs though: stick with brands such as XFX, SeaSonic and Corsair TX.
I think you should get all the replacement parts at the same time, since that'll save time and you'll get a smooth transition into hopefully a stable computer in the event that there are more than one devices at fault.