Whenever I execute the following code, iControlPortal.cgi crahses. The error in the syslog is pid 381 (iControlPortal.c), uid 1: exited on signal 6 (core dumped). The code is as follows: my $soap = SOAP::Lite -> uri('urn:iControl:ITCMLocalLB/Connections') -> proxy("https://$sHost:$sPort/iControl/iControlPortal.cgi" ); my $resp = $soap->get_all_active_connections( SOAP::Data->name(mirror_flag=> 0), ); Am I doing something wrong here? I am able to execute this method against a BigIP that does not have any connections, but consistently get a 500 Internal Server Error when executing against the production SLB. Thanks, Will

Currently, *most* connection and persistence information can be mirrored between a redundant pair. In the past, I wrote an emulator for shining a little light on the mirroring process. Lately, it has occurred to me that this same sort of thing--connection add and delete events--might be useful for customers for data mining purposes. If I were architecting something like this, I'd mirror the traffic to a machine setup for traffic monitoring* and put some logic on *it* to create classes** that I then passed to bigip via iControl. Furthermore, you could even use the existing aol class on BIGIP to help alleviate the monster proxy issue. *I assume there are existing products with this capability. If not, this sort of filtering would be uber-trivial (it's easy since the IP header contains addressing information *as well as* the packets total length) to implement on top of Linux' SOCK_PACKET stuff. NB: "easy" depends, I guess, on how much traffic you have since you might have difficulty keeping up. That being said, the amount of processing you'd need to do for classification is tiny. Someone with higher performance requirements might do something less naive and put classification and preliminary aggregation in the kernel and have a corresponding user-space process grab data periodically via an ioctl. This is significantly more work. **external or internal. Small benefit: the rule is constant. BIG benefit: classes are well-equipped for large datasets. Lookups are currently O(log n) for internal classes and O(1) [NB: at the expense of memory] for external ones.

Joe, Thanks for helping out. Are the methods to get connection information for single IP's reliable enough to use in production? Will

We're trying to sniff out client IP's that have (what we deem to be) excessively high connection and transfer rates (e.g. web spiders). . Once these hosts are identified, we wish to be able to persist them all to a single node so one machine will handle their processing requirements w/o affecting "normal" traffic. Given your reply, my current thought process is to ssh in, run bigpipe conn dump, parse that to identify potential "suspects" and then use iControl to track individual IP addresses. Problem is, I don't want to have to deal w/ all of the overhead and atomitcity (is that even a word?) of relying on an external shell, not to mention dealing w/ ssh keys and ssh-agent. Any other way I can obtain a list of all client IP's and the total number of connections they have? Thanks for your help, Will

Absolutely, the get_all type methods that can return an enormous list are the ones that tend to cause problems. It's a classic chicken and egg problem. App developers don't want to make 10000 calls for 10000 objects, they want to make 1. But for large result sets, the 1 method call causes problems. We are working at segmenting these types of operations in new platforms but that brings with it it's own set of problems.. Again, I've pinged our dev group to see of any alternate solutions for you and if there is any thing on the 4.x platform in the near future that can help. We really appreciate your feedback and knowing that you need this functionaility will help us priortize for future development efforts. -Joe

This was an example of us providing too much functionality early on. Basically the connection table is too large in a production environment and the memory overhead for building the SOAP response. We've put limits on the memory overhead of the iControl processes to help limit the overall memory usage and it seems that this causes that limit to be reached. If you look at it, let's say there are 10000 active connections, and each connections requires ~1k of XML data for the SOAP response, we are looking at 1k*10000b=10M Bytes of payload. This would more than likely cause the client undo hardship in parsing as well.. The real issue is that on our 4.x platforms we have a CORBA interface that the SOAP interfaces act as a proxy to and this causes another level of memory allocation which isn't quite as large as the SOAP usage, but close. We've optimized the way we handle large result sets in the next generation of products but are somewhat limited by the architecture in the 4.x platform. It is our recommendation that you do not use the Connections interface in it's present form on the 4.x series of products. What is your need for viewing the connections table? What are the requirements for your application? -Joe

Forum Discussion

gmtiwill_127449

Nimbostratus

Jul 09, 2004

get_al_active_connections() crashes iControlPortal.cgi

Whenever I execute the following code, iControlPortal.cgi crahses. The error in the syslog is pid 381 (iControlPortal.c), uid 1: exited on signal 6 (core dumped). The code is as follows:

my $soap = SOAP::Lite

-> uri('urn:iControl:ITCMLocalLB/Connections')

-> proxy("https://$sHost:$sPort/iControl/iControlPortal.cgi"

);

my $resp = $soap->get_all_active_connections(

SOAP::Data->name(mirror_flag=> 0),

);

Am I doing something wrong here? I am able to execute this method against a BigIP that does not have any connections, but consistently get a 500 Internal Server Error when executing against the production SLB.

Thanks,

Will

dev

devops

iControl

11 Replies

Joe_Pruitt
Jul 10, 2004
No problem. I just sent of a message to one of the developers asking about the best way to alter individual persistence records. Odd are that you will have to use a Class within a Rule to accomplish this. I'll post back when I have an answer.

-Joe
bknotwell_12713
Historic F5 Account
Jul 10, 2004
[quote:85423441dc="Joe"]No problem. I just sent of a message to one of the developers asking about the best way to alter individual persistence records. Odd are that you will have to use a Class within a Rule to accomplish this. I'll post back when I have an answer.

-Joe[/quote:85423441dc]

Beyond clearing sticky records, manipulating persistence records via a user interface isn't available. As Joe alluded to above, you'd do something like the following instead (untested so I'll undoubtedly get the rule syntax wrong):

if (client_addr one of spider_class) { use (pool spider_pool) } else { use (pool happy_customer_pool) }

It's not persistence per se, but it's essentially the same thing. Furthermore, as I stated above, spider_class should behave well with large N.

NB: I suggest querying bigip for the aol class so you don't accidentally add the AOL mega-proxies into spider_class.
bknotwell_12713
Historic F5 Account
Jul 13, 2004
[quote:e72e26c272="gmtiwill"]What Interface / Method should be used for querying the BigIP for AOL mega proxies? I don't see this in the SDK.

Will[/quote:e72e26c272]

It's get_address_class in ITCMLocalLB/Class. I've included a snippet below:

!/usr/bin/env python from F5Proxy import ProxyFactory,loadEnv loadEnv('./iControl.env',globals(),locals()) factory = ProxyFactory(user,passwd,host,protocol) print factory('ITCMLocalLB/Class').get_address_class(class_name = 'aol')
gmtiwill_127449
Nimbostratus
Jul 13, 2004
What Interface / Method should be used for querying the BigIP for AOL mega proxies? I don't see this in the SDK.

Will