PCI cards fail when using 4 gig ram

"It could be that your computer is set to prioritize RAM over allocation for the cards. I would go to the BIOS and change the setting of PNP OS Installed (it'll be different for every BIOS but the idea is that the OS can change the settings of your PCI cards). If it's set to disabled or off, enable it. This will let Windows change the settings of your cards and could allow them to work. If it is already enabled go ahead and disable it since it could be Windows that is reallocating things in the first place.

Neither change will harm your computer and can easily be set back."

"Your problem is that the first video card is using the same resources that the second video card is requesting. You're probably getting Device Manager error code 12, for which Microsoft counsels to disable the conflicting device.

You can verify this in the device manager / Display Adapters : right-click on each video card, then Properties and check the Resources tab of both cards for conflict.

This problem won't be solved by going to Vista 64-bits.
Some advice I've found when searching:

Update the video driver
Upgrade or downgrade the BIOS (dangerous!)
This might not apply, but BIOS changes to either (a) limit the AGP Aperture to 256MB, or (b) change the video Share Memory size.
Move the video card to another slot, for new resources to be allocated.
In this thread the solution was to edit the boot options by entering

bcdedit /set CONFIGACCESSPOLICY
DISALLOWMMCONFIG which can be
undone by: bcdedit /set
CONFIGACCESSPOLICY default

Clean install of the O/S, since resources are allocated during the installation of Windows.

Needless to say, make sure you have good backups before doing any of the above."