map_err_when sounds like it's related to map_err, which changes one Err(x) into a different Err(f(x)) and leaves Ok(_) untouched. Your operation here does neither of this, it conditionally changes Ok(_) into Err(e) and it leaves Err(x) untouched. Option's filter does conditionally change Some(_) into None and it leaves s None untouched, so that's a better fit for deriving the name from.
Of course the conditional would be reversed compared to your proposed map_error_when where a filter_or method would turn the value into Errwhen the condition closure returns false, not true.
W. r. t. naming, my initial. thoughts would be either filter_or/filter_or_else or filter/filter_with, where for each naming pair, the first method would be taking an error value, and the second one an error-producing closure.
Arguably, there should indeed really be a closure version, too, to avoid constructing unused error values that are too expensive.
That's then a method (self, impl FnOnce(&T) -> bool, impl FnOnce() -> E) -> Self or perhaps, since there's always an inner value T available when the second closure is called, a method (self, impl FnOnce(&T) -> bool, impl FnOnce(T) -> E) -> Self.
OTOH... in this latter case, we have two closures with almost the same input producing first a bool, and then conditionally an E, so that might be better served by taking a single closure returning Result<T, E> (need to return the T because now it's an owned parameter even for determining whether or not to filter), though that then gives us the signature of and_then. In other words, we arrive back at the original question of whether or not writing foo.filter_or_else(|x| cond(x), |x| err_value(x)) as foo.and_then(|x| if cond(&x) { Ok(x) } else { Err(err_value(x)) } is too much effort. Or for foo.filter_or(|x| cond(x), e), there's also the option of writing foo.and_then(|x| cond(&x).then(|| x).ok_or(e)).