You're fine. As long as your RPM's are staying steady and not jumping around. On mine the torque converter "unlocks" and you'll see the rpm's jump up. When it starts doing that I turn OD off. I try not to let it downshift to 3rd. I pull a 6500-7000# trailer though, with a similar truck setup as you (02 King Ranch).
Does your rig have the towing package? The reason I ask is it comes with the tranny cooler. If you don't have that I would suggest having one put on for good measure. Especially if you're towing in hot weather.