When is the floating IP or anything used?

Floating self IP addresses are used for lots of things on a BIG-IP system, including as a possible default gateway for your pool members, as YossiV indicated. They are also used for source address translation addresses (SNAT) when Auto Map is specified.

Why do you need floating self IP addresses? In an HA configuration, a floating self IP is shared between device group members, and will always point to the device that is active for its associated traffic group. Static (non-floating) self IP addresses are dedicated to a particular device and never point to anything but that device. For example, non-floating self IP addresses are used for monitor tests, as each device does its own monitoring, whether it's active or standby. Non-floating self IPs are also used for HA communication between devices in an HA device group. As a result, the MAC address of a non-floating self IP never changes. But you wouldn't want the default gateway on back-end servers (nodes) pointing to a non-floating self IP on a BIG-IP system as then traffic might get directed to a standby device. Same for SNATs. To support traffic processing through a BIG-IP HA configuration, the MAC address of a floating self IP address changes during failover to a MAC address on the active device. (This is why we say the IP address is "shared" between the device group members.)

While you can initially set up an HA configuration without floating self IP addresses, you cannot successfully process traffic without floating self IP addresses. Defining an HA (sync-failover) device group simply involves identifying the non-floating self IP addresses that will be used by each device for HA communication. Eventually, you must define the floating self IP addresses that will be associated with traffic processing.

floating ip is actully the VIP of the unit. and in some cases it also can be your pool members defult GW

Floats are used by virtual servers to talk to pool members (nodes). It is useful when you have have long running connections, so the mirrored sessions can persist after a failover. Otherwise, if you had to change IP addresses, session mirroring would be useless.

Another situation is when you use BIG-IP as a network gateway, so you can have only one destination in you routing table being served by any active device in your cluster.

Virtual addresses are always floating.

Monitors use local Self IPs. Some management tasks can use local SelF IPs or the management port depending on your configuration.

The Pool Members uses the Float IP as their gateway.

Can we set multiple floating self IPs per vlan?

A common use case for multiple self IP addresses on a single VLAN is to prevent port exhaustion when SNAT automap is in use. K7820: Overview of SNAT features

describes this condition.