My father is troublesome, what should I do?

I am from a well off family, and so is my husband. When we married, he seemed to be in love. Very soon it became evident that he did not care about me at all, and married me for money. He kept numerous women openly, drank, gambled and spent freely on his dissipations. Now, I live in a catholic country, and divorce here is very difficult and scandalous to a woman; a divorced woman is always wrong, her husband keeps her money and children, and she is a pariah. So, when I met that wonderful, handsome, smart man (he is deeply interested in chemistry and is constantly experimenting, in his free time), an army captain Godin de Sainte-Croix and fell in love, I didn't think it was wrong. At first we hid our love, then it seemed ridiculous to pretend, and we went out together. My husband didn't mind since he still had access to my money, and no one else, too. But my father makes a fuss. He says that my life is a scandal, that, while my husband amuses himself and spends my money, I must stay at home and pray for him, and so on. My father is high in civil service and even threatened to arrange an arrest of Godin. And it's not as if I had a string of boyfriends, I am faithful to Godin. I just want to have a little happiness in my life. But my father is growing more and more menacing, what should I do?