除了…以外【chúle…yǐwài】

除了…以外 takes one thing out of the picture: "apart from X". Whether X is then left out or added to depends on the adverb before the verb in the second half.

With 都: "except"

都 says everything else shares something, and X is left out of it.

With 都不 or 都没, X is the only one it applies to.

With 也 or 还: "besides"

也 or 还 adds to X: X is included, and there's more.

Leaving out the adverb

Without 都, 也 or 还, the sentence doesn't say which reading you mean.

• × 除了小雨，我们去了 → 除了小雨，我们都去了

The 除了 phrase comes first. English can put "except X" at the end; Chinese can't.

• × 我们都去了除了小雨 → 除了小雨，我们都去了

以外 is optional: 除了星期天 and 除了星期天以外 mean the same. In writing, 之外 often replaces it.