[subject] + 来 / 去 + [place] [subject] + 来 / 去 + [verb]
来 is movement towards the speaker, 去 is movement away. The place follows the verb directly, with no word for "to".
A second verb after 来 or 去 says what the movement is for. A place, if there is one, goes between the two.
不去 and 不来 say someone won't or doesn't go; 没去 and 没来 say they didn't.
来 and 去 also go after another verb to show which way it points: 进来, 出去, 回来, 上去.
回 is going back and 到 is arriving: 回家 is to go home, 到家 is to get there.