Just throwing this out there...
Considering you want to have systems and processes querying an API and you wanted to implement the BIG-IP as the Identity Provider using SAML but you also have a requirement where smart card authentication is a requirement. I've never seen anyone successfully query an API where on-demand cert auth was implemented.
Example: Process queries API -> API redirects to BIG-IP as Idp -> BIG-IP asks client for certificate (on-demand cert auth) -> Creds on certificate validated with LDAP query -> BIG-IP sends client back to API -> Client gets data
When client uses a browser, we implement that today with services but not sure about programmatic processes and APIs if that is even possible.
the process for a browser is the same as for a service. A client (browser or service) tries to access an API. The SP will redirect the client to the IdP. The IdP (F5 APM) will ask for a client certificate and validate the certificate. The IdP will then issue a SAML token and redirect the client back to the SP. The SP will accept the token.
It think the difference is in the capability of the client to follow the SAML authentication flow. A browser can follow the flow described above for sure. A service might need to be re-programmed.
Whilst, in my personal opinion, using JWT or Opaque tokens are better suited for API authentication or server-to-server authorization than SAML, I don't see a reason why this should not work.