Let $x_n = f(x_{n-1})$ for all $n > 1$, where $f(x) = x^2-4x+3$. Suppose $x_n$ for $n \ge 1$ only takes distinct two values. Find all possible values of $x_1$.