asm practices for CPU/ memory tuning

You are correct about Data Guard because ASM has to process HTTP responses. The same applies to applying attack signatures to responses. Also be aware that logging can affect performance under heavy load. So the recommendation is to not log data locally. Generally, activity that generates learning suggestions will cause more CPU, especially if the application is changing frequently and/or there are many requests.

You may see an increase in CPU resource consumption after adding brute force/web scraping or DoS profile protection. Memory is consumed for tracking sessions, clients, and users so it will be proportional to the number of concurrent clients in your application. What happens during an attack will depend on if you are using count-based mitigation (connections) or rate-based mitigation (cap connections or transactions from a specific IP address or for a specific URL). Either way, if you're getting hammered by bots, you will detect an increase in CPU during the attack.

If the device hypothetically would have reached 80% CPU usage, with configurations like: attack signatures in responses, local logging, data guard, DoS protection, anomaly detection, some new policies within the enforcement readiness period. What would you recommend to disable and in which order?

I would first recommend that you not log security events locally. You don't specify the version of ASM you are using, but anything post-11.5 no longer logs security events in /var/log/asm by default. Definitely move your logging off the active unit. Second would be response checking--Data Guard counts as response checking, so determine if you are at risk of leaking data before enabling it. Third would be applying attack signatures to responses. Ensure that your rationale for applying attack signatures to responses makes sense. Your remaining examples are tough to assess without knowledge of normal traffic patterns. If you have many policies, and they are complex--meaning more than just applying attack signatures, checking RFC compliance, and detecting evasion techniques, then the resources they consume will be directly related to how much learning you are asking ASM to do. High traffic volume, with learning enabled for file types, parameters, URLs, cookies, etc. will be more expensive than low traffic volume with no learning. This is especially true during the Enforcement Readiness Period, because that's when the the bulk of ASM's heuristic work is being done. Try to build your policy based on the risks you think are the most likely to be encountered in live traffic.