What would you do if your significant other and his father wanted to take a 3-week trip to Japan?

Let him go. If he cheats on you, dump him. You don't have much of a choice here. You've already told him how you feel. That's basically all you can do for now. Maybe tell him to call you a few times a week to let you know how things are going?