I think that he is the one with the problem and that is alcohol related. If you don't want to spend time with him then don't go - if he wants to then let him spend time on his own too - perhaps that is what you both need You need to come clean and admit that his drinking is upsetting you. Make a stand now and either he will turn his life around or he will get drunk in Portugal and realise what he is doing and how he is about to lose quite a lot.
Sorry, I can't be more positive but he does have a problem.