But you must. Obviously your most important role in life is being a dad but you're also other things. A brother (if you have siblings), a son, colleague, friend. You need to pursue your own interests, go out with mates for a pint etc or whatever your interests are. You need to be the best version of you ready for when your kids can see you.
You're still important and you can't let the system take you down.