Let's say I optimized geometry with PBE+D3, does it matter if I make single point bandstructure calculation with just PBE or PBE+D3 since D3 is a additive method.
It doesn't matter. The self-consistent calculation will use the Hamiltonian constructed from the relaxed structure, in which the D3 has been applied and should not be considered again.
Take bilayer MoSe2 as an example:
This structure is optimized with PBE+D3 using the VASP package. With this relaxed structure, I have calculated the band structure with and without the consideration of D3 in the INCAR file. One can not see any difference between the two band structures.
A PBE vs PBE+D3 calculation will have the same band structure if the same geometry is used. If the geometry is optimized, however, they can differ but the D3 correction does not directly influence the band structure just the geometry.
You can validate this method by considering that these corrections can often be done by an external step such as combining VASP with D3 using ASE rather than VASP's internal tools. To VASP, nothing about these calculations is different, the correction happens afterward.