Let $u_{pi\alpha}$ be the displacement of atom $\alpha$ in the basis located in supercell with position $\mathbf{R}_p$ and in Cartesian direction $i$. With this "Cartesian" description of the motion of the atoms, it then becomes very simple to understand whether an atom moves out of plane (zero amplitude $x$ and $y$ components), or in plane (zero amplitude $z$ component), or in any other direction you may be interested in. So how do you figure out these $u_{pi\alpha}$ displacements for a given phonon mode?
A phonon mode is labelled by quantum numbers $(\mathbf{q},\nu)$, where $\mathbf{q}$ is the momentum and $\nu$ the branch index. This phonon is characterized by a frequency $\omega_{\mathbf{q}\nu}$ and eigenvector $v_{\mathbf{q}\nu;i{\alpha}}$, which are obtained by diagonalizing the dynamical matrix (I outlined these calculations in this answer). Phonon modes provide an alternative basis in which you can characterize the motion of the atoms in the system by the so-called normal modes, of amplitude $u_{\mathbf{q}\nu}$. These correspond to "collective" motions of atoms, which have the advantage that lead to a Hamiltonian that is a set of uncoupled simple harmonic oscillators (so very beneficial computationally), but the motion is not always easy to see.
The general formula relating the normal modes coordinates and the Cartesian coordinates is (again, see my answer here for further details):
$$
u_{pi\alpha}=\frac{1}{\sqrt{N_pm_{\alpha}}}\sum_{\mathbf{q},\nu}u_{\mathbf{q}\nu}e^{i\mathbf{q}\cdot\mathbf{R}_p}v_{\mathbf{q}\nu;i\alpha},
$$
where $N_p$ is the number of primitive cells in the periodic supercells and $m_{\alpha}$ is the mass of atom $\alpha$. Therefore, what you want to do once you have found the phonon frequencies and eigenvectors is to pick a particular phonon mode $(\mathbf{q},\nu)$ and calculate what the corresponding $u_{pi\alpha}$ is. In the equation above, this means that all $u_{\mathbf{q}\nu}$ apart from the one corresponding to the mode you are interested in will be zero.
Most codes that calculate phonons will have some way of directly outputting the Cartesian atomic displacements $u_{pi\alpha}$ so that you can directly visualize them.