That depends on why you don't want to go.
I have never attended any school play, nativity, concert, choir recital or any other event. I have attended a very few "open evenings" but both wives have preferred my absence to my company on such occasions as they both say I am "argumentative and aggressive" to the teachers. I say I am only seeking truthful answers to my concerns.
The reason I do not attend any other events at schools is that I found them acutely embarrassing as a child and still find watching children reluctantly performing embarrassing. My heart goes out to the kids and I want to shout "STOP" and take my children home. Both boys agree that they hated the whole performance thing and confess that they felt threatened by teachers when they tried to protest that they didn't want to take part.
If you feel that going would compromise your convictions and if your convictions are so anti religion that you think it worth taking a stance that we all know will not sit well with Mrs Nick then don't go. If on the other hand you can suffer in silence because The Boy wants you to be there then I think you should go. Attendance at a ritual does not bind you to the religion involved. I have attended Hindu weddings but that doesn't mean I am now a Hindu. I do believe they have a right to hold such beliefs as they wish.
I personally believe in God but I have a strong dislike of organised religion which is, I feel, responsible for more trouble in this world than any other single factor. That does not stop me going to Harvest Festival if my kids want me there. It did not stop me "giving away" my daughter at her church wedding because it was what she wanted me to do (that and pay for the bloody thing)