Add shims as or if needed until caster is out of base.
One drop of oil where the caster stem is in the base is very helpful.