Assalamu alaikum,
Let me first preface this question by saying that I personally don't always lower my gaze. Given the type of work that I do, it would be inappropriate not to address men directly. I do so as much as I can outside of work.
However, my partner does lower his gaze, and lately he...