when you drop 100 horsepower, you're not making boost. If you're not making boost, the bypass valve is not closing. It's spring operated, so the default position is to make boost. You should be able to move the valve by hand. When you open it and let it go it should snap shut. If the valve is loose, because it does have generous adjustment slots, that's probably your issue. If the spring inside the valve is defective, it may not close the valve.

The other issue is a solenoid on the roush supercharges that allows vacuum to reach the bypass valve. If that solenoid is not doing it's job, the bypass valve could be held open. You should be able hook the vacuum line going to the solenoid directly to the by-pass valve and make boost, unless your by-pass isn't properly adjusted as stated above. A boost gauge would go a long way to telling you if you're making full boost. It that by-pass allows some air flow, you won't be making maximum boost.