This is – at present, at least – the expected behavior. If you do not supply the move keyword, then the closure accesses all the variables from the stack by reference, unless it cannot do so (i.e., because it must be moved).
Keep in mind that both copying and using in place have legitimate meanings when mutability is involved. There is a kind of ambiguity that arises:
let mut i = 1;
let x = || i;
i += 1;
let j = x(); // is `x()` going to return `1` or `2` here?
Right now, x will return 2, because – since you did not write move – the closure is “attached” to the creating stack frame and uses its values in place. If Copy types were moved by default, it would return 1.
You could imagine taking the binding of the variable into account (i.e., here we would use in place, because i is declared mut, but if it were not mut, we would copy). That makes me uncomfortable because marking a variable as mut or not mut currently has no affect whatsoever on the semantics of your program, but in this case it would change the meaning of the code quite drastically.
In a way, I feel we picked the wrong keyword for closures. I prefer to think of move closures as “detached” from the surrounding stack frame (in which case they must copy all the data they must use into themselves), whereas non-move closures always use the variables “in place”.