Cargo rebuilds the package if the command line arguments aren't the same, which makes it test the wrong binary.
uname -m