Torsion of elliptic curves with rational $j$-invariant defined over number fields of prime degree

Let $[K:\mathbb{Q}]=p$ be a prime number and let $E/K$ be an elliptic curve with $j(E) \in \mathbb{Q}$. We determine the all possibilities for $E(K)_{tors}$. We obtain these results by studying Galois representations of $E$ and of it's quadratic twists.