Does the power still go to that recepticle?

The APC UPS should have designated protected outlets which are turned off when you turn off the UPS, as well as some which remain 'live' no matter what. Read the doentation that came with the UPS. Your computer technician is well intended but an ogy is, do you leave your car running while you're in the house for hours and hours?