What you are experiencing is just one of the issues with the parts store kits.
Carter produced 505 different AFB carburetors, the parts store kits have 1 exploded diagram (one size fits none

).
But you guessed correctly. As the AFB was used one many different size engines, pump discharge check valves of different mass are necessary to work correctly with the different engines. Carter used at least 6 different square needles and at least three different balls as pump discharge check valves.
Hopefully, the correct one for your carb is in the kit you acquired.
I have yet to see a good book on the Carter AFB carbs. Best existing information to my knowledge:
(A) Carter factory Master Parts & Service pages
(B) Rebuilding section in factory shop manual
(C) Carburetor section in the car manufacturer Master Parts Book
(D) Some of the guys that answer on these forums
Jon.
(B)
Jon.
__________________
"Good carburetion is fuelish hot air".
"The most expensive carburetor is the wrong one given to you by your neighbor".
If you truly believe that "one size fits all" try walking a mile in your spouse's shoes!
Owner of The Carburetor Shop, LLC (of Missouri).
Current caretaker of the remains of Stromberg Caburetor, and custodian of the existing Carter and Kingston carburetor drawings.