Do you need seperate Synch Groups one for LTM and another for APM modules.

You should have a Sync-Only group to handle APM synchronizations. That's the reason for the Access Policy Sync section under the Access Profiles config.

Same thing goes for PSM or GTM (though GTM is done differently, it's the same concept of having a specific sync group for the individual module configuration).