/var/log/ has only 0% free | Big-IP PEM
Hi all, Unit: Big-IP Policy Enforcement Manager (PEM) Software: BIG-IP 13.1.1 Build 0.76.4 Engineering Hotfix Issue: /var/log/ has reached full capacity. We have started receiving Broadcast messages when connected via SSH, stating that Disk partition /var/log (slot 1) has only 0% free. As our regular F5 administrators are away I am unsure how to proceed. Broadcast message from root@hostname (Wed Jan 9 13:01:02 2019): 011d0004:3: Disk partition /var/log (slot 1) has only 0% free Here is the disk usage from /var/log/ and the culprit looks to be wr_urldbd.out. As shown below it's at a staggering 34G. We have another identical unit at another site which has this file @ 8.4M. [user@hostname:/S1-green-P::Active:Standalone] log ls -Slh | head -10 total 6.8G -rw-r--r--. 1 root root 34G 2019-01-09 13:04 wr_urldbd.out -rw-r--r--. 1 root root 4.7G 2019-01-09 13:01 wr_urldbd.out.1 -rw-------. 1 root root 2.1G 2018-10-27 21:45 zebos.log -rw-r--r--. 1 root root 1.1M 2018-10-24 14:37 restjavad-gc.log.0 -rw-r--r--. 1 root root 978K 2018-10-27 21:45 restjavad.1.log -rw-r--r--. 1 root root 688K 2019-01-09 12:53 restjavad-gc.log.0.current -rw-------. 1 root root 680K 2019-01-09 03:26 hitless_upgrade.log -rw-------. 1 root root 672K 2019-01-09 13:03 ltm -rw-r--r--. 1 root root 496K 2019-01-09 12:48 restjavad.0.log Tailing wr_urldbd.out yields - `[user@hostname:/S1-green-P::Active:Standalone] log tail wr_urldbd.out nGetRtuUpdateInfo: failed sConnectSocket to service.brightcloud.com failed connect to 570354308 on 8888, err = -1 nGetRtuUpdateInfo: failed sConnectSocket to service.brightcloud.com failed connect to 570354308 on 8888, err = -1 nGetRtuUpdateInfo: failed sConnectSocket to service.brightcloud.com failed connect to 570354308 on 8888, err = -1 nGetRtuUpdateInfo: failed sConnectSocket to service.brightcloud.com failed connect to 570354308 on 8888, err = -1 nGetRtuUpdateInfo: failed sConnectSocket to service.brightcloud.com failed connect to 570354308 on 8888, err = -1 As this unit is relatively new to our network, I suspect that configuration is yet to be finalised which is why it's being blocked - and I'm not too worried about that. What'd I'd like to know is the purpose of this file and if it would be safe to delete it? Thanks in advance. First time poster so please let me know if any additional information is required.
PEM: Subscriber-Aware Policy and Why Every Large Network Needs One
Previous post “PEM: Key Component of the Next Generation University Network” provided a high-level overview of several Policy Enforcement Manager features which help K-12 Schools, Colleges and Universities transform their Networks into agile, user-focused “Data Delivery Fabrics” which redefine the way Educational Institutions provide data connectivity services to students, faculty, staff, and guests. As with all networks, schools provide access to internal resources as well as the Internet. Typically, internal network (LAN) traffic is not a major concern for network admins (although at some point WiFi saturation prompts infrastructure expansion), but Internet link saturation is a much more common and serious issue. Since any expansion of Internet access is associated with increased ongoing operating expense (opex) and, in many cases, infrastructure expansion resulting in upfront capital expense (capex). Even when an institution can afford a larger ISP link, regional Internet service providers (ISPs) may not offer the required bandwidth, or the ISP lacks sufficient infrastructure to support and/or provide increased bandwidth resources. Nobody likes slow Internet. From myriad apps constantly pulling data in the background to the always-connected lifestyle of millennial students, the need for a fast, reliable, and low-latency connection is now more critical than ever. In the environment with limited resources, such as a school’s ISP link, it is critical to have the ability to control and distribute these resources according to priorities which maximize user’s experience while still providing a healthy mix of QoS for different types of traffic. F5’s Policy Enforcement Manager (PEM) has a number of facilities to enable schools to achieve the optimal balance between performance and traffic priority. Policies, bandwidth controllers, traffic intelligence categories, and presets are among those facilities. Today we will talk about the core PEM functionality - Enforcementpolicies. There are 3 main types of PEM Enforcement policies: Pic 1. PEM Enforcement policy types Global Policy: Applied to all users: known and unknown Subscriber Policy: Applied to known users: provisioned statically or discovered via DHCP, Radius or Access Profiles & iRules Unknown Subscriber Policy: Applied to unknown users PEM uses various subscriber discovery methods which usually differ by implementation. RADIUS and DHCP “sniffing” are among the configurable discovery methods. When PEM sees traffic, it checks whether the source IP address belongs to any known user (previously-discovered subscriber). If the user is known, traffic is classified and appropriate action is taken according to Subscriber Policy of that user. However, if the source IP address is not known to PEM, the Unknown Subscriber Policy is used until that user is discovered. Global Policy is applied to all users and may contain high-level rules applicable to all users in the network (e.g. blocking of malicious URLs, suppression of certain P2P applications, etc.). Pic 2. PEM Policies example Each user can be assigned a Subscriber Policy, and as long as the user is known to PEM, all traffic associated with that user will be analyzed and given priorities according to the policy rules. Among other functions, rules are used to provide application visibility by categorizing both encrypted and unencrypted traffic into categories. URL filtering and blocking actions are also provisioned using PEM Policy rules. PEM can associate a rule with the traffic using any of the following: Classification URL category Flow Custom Classification Pic 3. Policy rule Classification example The Classification tab in enforcement policy rules has a flexible definition to match an Application or Category from the extensive list provided in drop-down menu. PEM uses signatures to detect the applications. These signatures are updated periodically by F5 and PEM can be configured to check for Signature updates automatically Daily, Weekly or Monthly. Matching criteria can provide a positive or negative matching, allowing for granular actions like QoS/bandwidth control, reporting or TCP optimization to be applied to various classified traffic types. URL category Pic 4. URL categories and URLDB URLs can be categorized according to pre-defined or custom definitions. PEM can also use external URLDB/Feed list which makes it easy to extend pre-defined Categories list and maintain central reference for Categorized URLs. URLDB is a CSV file that contains website URL and associated category ID Pic 5. Custom URLDB content example Flow PEM can use flow information as a condition to apply an enforcement policy rule. There are various types of flow-specific properties that can be configured as a matching condition: DSCP Value, Protocol, IP Type, Source/Destination Address/Port, VLAN, etc. Pic 6. Flow condition rule example Like any other BIG-IP module, PEM functionality can be extended and customized using iRules.Custom tab allows user to configure a specific condition not covered by built-in PEM functionality. As always, iRules are a powerful and flexible way to extend platform functionality. Please refer to DevCentraliRules API Wiki for PEM-specific iRules syntax. Enforcement policy rules are defined to perform a specific action within policy: limit bandwidth, close the “Gate” (block the traffic), redirect, insert HTML content, log messages etc. Some items may only be applicable to service providers - i.e. Application reporting and Rating Groups, therefore we will focus on configuration items that will be most commonly used by Education network admins. Reporting: usage, QoE, TCP Analytics Gate Status Forwarding Modify Header Insert Content QoS TCP Optimization Congestion Detection Custom Action (iRule) Rather than describing each feature separately, let’s consider a few common use cases for these rules. For example, we can create 2 rules that block all traffic classified as “Phishing and other Frauds” by assigning a Gate Status “Disabled” and limit the bandwidth of Skype to 10Mbps max system-wide and 1Mbps max per user. The Classification rule will look similar to: Pic 7. Flow condition rule example Bandwidth limiting rule uses Bandwidth controllers within QoS section: The resulting Enforcement Policy will protect users from phishing and other fraudulent sites while limiting the bandwidth of Skype (including Video calls) to 1Mbps per user (and 10 Mbps total allocated for Skype application traffic). Flexible, user-aware classifications and a variety of traffic actions can be taken by individual rules to create the intelligent environment of flexible micro-granular control. This approach balances apps and services by both speed and priority, protects users (on-campus and remote students, staff, and visitors) from fraudulent and malicious activities and enhances overall quality of user experience by optimizing TCP and pacing video by preventing congestion on the ISP link. Institutions of any size can immediately start enjoying the incredible benefits that come with introduction of PEM Policies into their network. F5 engineers are available to make every project a success, helping customers from inception to a successful deployment. Next, we will dive into how PEM can save ISP link bandwidth by forcing streaming video to fallback to lower resolution while supporting the encrypted QUIC protocol. Stay tuned!PEM: Key Component of the Next Generation University Network
In recent years, higher education institutions have become significant providers of digital services and content, ranging from mesh WiFi access to virtual-classroom services featuring high-bandwidth real-time collaboration experiences for on-campus and remote students alike. In fact, many Universities’ IT networks have become so large, they now compete with some regional Service Providers based on the amount of data they process and route within their IT infrastructure. Students, classrooms, staff, and guests all need to have reliable access to Campus LAN and Internet services simultaneously. However, with growing number of consumers, internal and outbound routes can become quickly saturated and oversubscribed, resulting in slow response times and degraded performance of the entire university network. To prevent chaos and limit data-hungry devices from clogging up data links, Universities have begun to employ certain services usually found in Service Provider (SP) networks. In particular, Policy and Charging Control (PCC) elements that: Are subscriber - aware Assign QoS to applications and services Perform application layer data inspection Enforce subscriber and application policies Ensure compliance with State and Federal laws Prevent access to inappropriate content Provide visibility and reporting So, how does the modern University achieve this without having to build a full-blown Evolved Packet Core inside their IT Network? Some have implemented the list in parts using different network elements, but this approach offers limited centralized visibility and/or traffic control, while others use the aging Cisco SCE, which will be End-of-Life on September 30th of 2018. The most progressive University IT teams quickly realized the benefits of having a subscriber-aware policy enforcement device, and turned to F5 Policy Enforcement Manager(PEM) as a full and integral solution that optimizes network resources and allows for optimal channel utilization, ultimately leading to improved user experience and substantial financial savings for Universities due to much more efficient use of available bandwidth. Pic 1. F5 Policy Enforcement Manager Any school or other organization which implemented PEM in their network can achieve a “subscriber” (in SP terms) or end-user (in enterprise terms) granularity. That means every user connecting to the School or University network can be assigned a Policy with certain rules which dictate how this user will be treated by the network. For example, some students may be given a preferential access to certain network resources and applications while faculty members may have an unrestricted Internet access with higher priority during classes and post-class activities. By categorizing users and applications network can achieve better utilization, ensure fair resource consumption and provide the best experience for all users Pic 2. Per-Subscriber Policy In addition to subscribers, PEM also implements a “per-application” concept. It provides the most comprehensive and agile configuration of policies when combined with subscriber and global policy scopes. This capability enables the University to limit or block certain application types - i.e. P2P Torrent traffic, various messengers, or social networks. Pic 3. Per-Application Policy SSL Visibility is a crucial part of Network monitoring and content filtering in Public Networks. By terminating the SSL (or TLS) connection from users and establishing new SSL connection to application servers, PEM makes it possible to perform: SNI analysis and classification Traffic content inspection and manipulation Detailed reporting and data visualization Pic 4. SSL Forward Proxy URL Classification and Filtering is another important aspect of managing IT network in Schools or Universities. Age-appropriate content must be enforced for students and other users, while maintaining the up-to-date list of blacklisted and malicious websites. PEM utilizes a Webroot-provided DB for precise URL categorization. With more than 80 URL categories available including live updates and custom categories, URL classification and enforcement becomes an effortless and efficient automatic routine. PEM also enables custom HTML content insertion into HTTP traffic, which can be used to warn users about a potentially harmful website or blocked internet resource by URL Filtering engine. Pic 5. URL Classification and Enforcement Schools can also realize significant savings on bandwidth by using Policy Enforcement Manager’s Video Pacing feature. PEM ensures that video content is pre-loaded at the same or similar pace as consumed by the user. By doing so it eliminates wasted bandwidth and traffic spikes that are produced by multiple users accessing video resources at the same time. Without video pacing, video pre-loading is triggered when a user starts watching content, making entire length of content available for viewing. Sometimes users stop watching the content before the end of video file, effectively throwing out unconsumed portion of pre-loaded video. PEM ensures that no unnecessary content is pre-loaded, so that no bandwidth is wasted. Pic 6. Video Pre-loaded, no pacing used Pic 7. PEM uses video pacing Network Visibility and Reporting plays a significant role in the Network Management domain. By knowing exactly what is happening in near real-time, Network Administrators are empowered to identify violations and fix issues before they impact other users in school network. PEM provides both on-the-box analytics and exported data to be used for reporting and visualization using third party tools. Pic 8. Data export options Policy Enforcement Manager enables Schools or Universities to implement alternative, Service Provider-oriented network architectures delivering: More granular control and visibility Optimized user experience Savings on Internet Services. Next in this series, we will be diving into deeper detail on how Universities can best leverage the various features of PEM covered here. Stay tuned!
PEM Http-Redirect not working
Hi, we're currently facing a problem in redirecting a website to a custom block page. We managed to block the facebook but it doesn't redirecting to the block page. We created a policy on pem and add fb-block-rule: We use this iRule and applied it to a virtual server 192.168.0.23. when HTTP_REQUEST { HTTP::respond 200 content " FORBIDDEN SITE Hi there! Thanks for stopping by. But this page is restricted.. and you are being monitored. ===This is from your DCMD Administrator=== " } This is what we would like to show: We are hoping for your kind assistance. Thanks!
