つい + [verb て-form] + しまう
つい means "without meaning to", "couldn't help it": you did something out of habit or impulse, often knowing you shouldn't. It goes before the verb, and the verb almost always ends in きごう〜てしまう (casually 〜ちゃう), which adds the regret.
In the present tense it describes a habit you can't break.
Because つい is about what you did not choose, it does not go with a plan, an intention or a request: ×つい食べよう, ×つい食べてください.
Both translate as "accidentally". つい is impulse or habit: you half knew and did it anyway. うっかり is carelessness: your attention slipped and you didn't notice.
Before a word for a time or a distance, つい means "only just": ついせんじつ先日"just the other day", ついさっき "just a moment ago". This つい has nothing to do with impulse.