Does an "automatic policy" start blocking right away?

yes the automatic policy is in blocking mode from the beginning. But all the parts of your policy should be in staging, which means the policy does not block anything. Also you must enforce a policy entry that the policy blocks anything.