Hi Anji,
To me 3mm sounds a lot!, (0.120in.)
Presuming that the faces of the cam bushes (shoulders) look ok and are not worn away?
What Josh says is correct,
As a final check before completing the assembly I would check that the cams are running fairly centerally on the followers, as this is the most important point,
It would not do if one of the cam lobes was touching the follower next to it

Regards
John O R