Cannot run configuration utility through serial console port

I assume you've quit out of the AOM menu, and hit enter ?

AOM Command Menu:
B --- Set console baud rate
C --- Capture blade console
I --- Display platform information
P --- Power on/off blade
R --- Reset blade
Q --- Quit menu and return to console

Enter Command: q

BIG-IP 11.4.1 Build 608.0
Kernel 2.6.32-220.el6.f5.x86_64 on an x86_64
VIP2400.local login:

If you're not seeing the unix login prompt as above, then it is possible that the console baud rate was set on the blade and mismatches the AOM.

It might also be that the blade is powered off - try the P option in the menu, though it should be on by default.

If you power cycle the blade (either through the AOM menu, or by reseating the card), while connected to the serial port (and quit out of the AOM menu), do you see the BIOS and linux kernel messages ?

Version 2.14.1219. Copyright (C) 2011 American Megatrends, Inc.
BIOS Date: 05/02/2014 08:34:27 Ver: A113106
F5 Platform: A113 OBJ-0552-xx BIOS (Build: 1.06.043.0) Date: 05/02/2014
Press  to enter setup.
Press  to launch F5 Disk Erase Utility.
                                                                            Red Hat nash version 5.1.19.6 starting

Reading all physical volumes.      This may take a while...
  Found volume group "vg-db-sda" using metadata type lvm2
  12 logical volume(s) in volume group "vg-db-sda" now active
/sysroot/sbin/setfiles:  labeling files, pretending /sysroot is /
/sysroot/sbin/setfiles:  labeling files under /sysroot/sbin/init
matchpathcon_filespec_eval:  hash table stats: 1 elements, 1/65536 buckets used, longest chain length 1
/sysroot/sbin/setfiles:  Done.
                Welcome to BIG-IP 11.4.1 Build 608.0

If you do not see those messages, then I suggest you open a support ticket with F5.