you can change preferences by doing RLHF or changing the prompt. there's a whole field on it: alignment.