APM httpagent401 in clientless mode?

You are correct. Clientless-mode is disabling this "disruptive" agent. A workaround would be to use an iRule to send the 401 (in the absence of an existing Authorization) header. That iRule might look something like this:

when HTTP_REQUEST {    
    if { not ( [HTTP::header exists Authorization] ) } {
        HTTP::respond 401 WWW-Authenticate "Basic realm=\"myrealm\""
    }
}

I may have my Authorization header syntax a little off, but you should get the idea.