FWIW you probably already know this but @ruuda might now. Right now you can do:
#[cfg(not(target_feature = "avx"))]
compile_error!("This library assumes availability of AVX and must be compiled with -C target-feature=+avx");
For everything else I pretty much agree with what @alexcrichton says.
If the problem is that RUSTFLAGS are not easily discoverable, or not very ergonomic, we could think about adding options to cargo that allow you to pass features, like cargo build --target-features=avx or similar, that get shown by the cargo build --help. Or maybe cargo build --help should just tell you about RUSTFLAGS and about how to query the available RUSTFLAGS.