What parts might I need to replace

"Can you access it via the AirPort Utility?

Often an Amber LED is a bad config Vs needing a reset. Using AirPort Utility you should be able to see the error condition and make the needed correction. I would recommend if you have done a hard reset to make sure you re-establish a password for the device as well as reconfigure the WiFi connection settings from the default fully open to WPA2 with a strong password.

Update (10/06/2016)

OK, lets try this again...

Yes, I fully understand your the reset button is not resetting, lets put that aside for now. Watch this YouTube: AirPort Utility 6.0 Walkthrough and download this onto your Mac: Download AirPort Utility 6.3.1 for Mac. If you don't have a Mac Apple also has a iPhone/iPad version on their App Store and they also offer a MS Windows version.

With the app you can also reset the unit. but as I stated the Amber LED is a mis-config of the unit. Which is why you need the app to fix. If you can't gain access to the AirPort Utility then you do need to do a hard reset. Follow this Apple T/N: Resetting an AirPort base station FAQ

If the reset button won't reset it's likely damaged. You'll need to open the unit to gain access to the logic board to patch in a second switch."