Men can't marry other men on my Earth. [ or more like in his own time but whatever the distant future holds, he doesn't expect to see it. ] So that we love each other doesn't matter.

[ and that's pretty much it. he doesn't sound too upset about it (aka not at all because it's Childermass). ]

Though you have the right of it. He's never needed an excuse for that.