RDS Farm without RDS Connection Broker

You need the connection broker to direct your client to the server where they have an existing session. If you use msrdp persistence with session directory enabled, LTM will honor the session token and forward traffic to the correct server. Without connection broker, you'll just get load balanced according to the selected LB method and likely end up on the wrong server most of the time.