Voters won't trust policy shift: Labor MP