You really just have to be as fair as possible, without spreading yourselves too thin. If your parents don't live near one another (making it difficult or impossible to spend time with both every holiday), you'll just have to work out a compromise. Either celebrate with one family the following week, or rotate years.
Personally, we only have this problem with Christmas, as we don't live near enough to our families to celebrate 'smaller' holidays with them. For Christmas, we spend Xmas Eve with one family and Xmas day with the other. Sometimes that means we're driving either late on Xmas Eve or early on Xmas morning (like this past year, when DH's brother hosted Xmas instead of his parents - but it was only a 2-hour drive, so no big deal). We try to keep it as fair as possible, so that everybody gets to see us, but we don't try to kill ourselves seeing everybody, either. (Made that mistake in the past).
Vice President and Guardian of the Toilet Brush of POOP: People Offended by Offended People