Finally figured it out. Thought Id post it if anyone else had the same problem. The one thing I missed is a small screw at the front of the assembly, next to the main housing.
There is a clip that can be seen when you turn the mirror all the way in. Push it in until it pops down. Figure out how get the screw out[I never did,hence the broke tab but does not affect anything] and the plastic assembly pops out. Pics