By saying yes to this job, this activity, this deal, what are you saying no to?
Are you saying no to time with your wife or kids? Are you saying no to more money? Are you saying no to further opportunities or that project you always wanted to start? Are you saying no to a vacation you always wanted to take? Are you saying no to your happiness, your health or well-being?
If this thing brings you closer to the person you want to be, and the change you are seeking to make, by all means say yes.
If no, well, then you have your answer.
By saying yes to something, you have to say no somewhere else (and visa-versa).