I see in many sources the exfoliation energy defined with the unit $meV/A^2$ I frankly don't understand how it's calculated and I couldn't find any mathematical formula for that. its unit confuses me. I was wondering if anyone could explain the concept mathematically. And how can I convert it into the interlayer binding energy ($meV/Atom$)
Unfortunately, these appear to be the same quantity (a somewhat disappointing answer). They only differ in units, you can convert between the atomwise and area based units by simply calculating the area per atom in the cell. This is commonly seen for surface energies as well. You can see it reported either way in literature.
Interestingly, we never really see this sort of unit used in adsorbate calculations where it tends always be reported atomwise but when analyzing a coverage I imagine you could also treat it the same way.