How can I address this gap between tile wall and tub diverter?

I am having an issue with my tub diverter. There is a small gap where I can’t get the diverter to sit flush with the trim piece. Maybe an 1/8 of an inch if that. I loosed the nut that holds the push on diverter in place and tried to get it to sit flush to no avail. The pipe doesn’t seem to be too long when I removed the diverter. Maybe it isn't perfectly perpendicular to the wall? I was thinking of putting some black silicon in the gap. Or do you think I should get a plumber out to address it? I just don’t want to get any water seeping into the gap and finding its way behind the wall. Thanks for your suggestions.

Images: https://imgur.com/a/k05zhRk