Understanding how Linux reacts to memory pressure using vmstat
Introduction This article is just a quick hands-on practical example to show how a typical Linux system usually reacts when memory pressure builds up quickly. I'll usevmstatto watchfree,buffersandcachedcounters to watch how they react to a sharp memory increase. FYI, my test box has 4 GB of memory: To quickly increase memory usage, I wrote a simple C program that allocates 3 GB of RAM to an array of integers. It then waits 5 seconds before freeing up memory to give us some time to look at memory usage before program frees up memory. Here's the C code: I then compiled the code usinggcccommand: On the left terminal, I'm executingwatch -n 1 vmstatcommand and on the right terminal I'm running my C script./mem: I recorded a couple of seconds after program finished running, i.e. after we free up memory so we can see how free counters recover from memory pressure. I encourage you to go through the above animation 2 times. Once observing free and a 2nd time observing buff/cache. free For the first time, let's focus onfree. freeshows the amount of memory that is 100% physically free/idle. Notice that it starts out with about 2.7G (2759796 KB) and sharply decreases to just 107 MB (107876 KB) in 8 seconds: As soon as memory is freed, it sharply rises back to 3 GB (3039684 KB): buff/cache buff/cache are kept high in a healthy Linux system with plenty of memory available to make the most of available RAM. Cache is the size of memory page cache and buffers is the size of in-memory block I/O buffers. Since 2.4.10, both page and buffer cache are unified so we look at them together here. This is also the reason why top command shows both of them together: For Linux, idle RAM is a waste when it can be used for faster access to read frequently used data or to buffer data for faster retrieval. For this reason, Linux uses a lot of buff/cache so in reality, the total amount of free memory on Linux is the sum offree+buff/cache. In this particular case, the fact that buff/cache is reducing significantly in size, indicates that Linux no longer has the luxury of usingbuff/cachefor other purposes that much and is now prioritising our./memprogram as shown below: They immediately start building up again and 6 seconds later, values are much higher: If memory continued to be a problem, Linux could opt to use its swap address and as memory is swapped in/out of disk, we should see the values ofsi/soincreasing. Appendix - OOM Killer Since we're talking about memory here, it's worth mentioning that when memory is critically low or threatening the stability of the system, Linux can trigger the Out of Memory (OOM) Killer functionality. Its job is to kill process(es) until enough memory is freed so that the system can function smoothly. The OOM Killer selects process(es) that It considers as least important when it comes to damaging system's functionality. The kernel maintains anoom_scorefor each process which can be found in /proc: The lower the score, the less likely OOM killer will kill it as less memory process is using. The higher theoom_scorevalue, the higher the chances of process getting killed by OOM Killer in an OOM situation. As most processes on my Linux box are not using memory, most of them will get a score of 0 (143 out of 147 processes): The last command lists all the scores in my system. I usedsort -uto sort the entries and remove the duplicates in a sorted manner.
F5 LTM increasing "TMM Used Memory"
Hi. I have a BIGIP run LTM 11.5.1 HF2 only. My TMM Used Memory increasing indefinitely. I believe that the Subsystem xdata is responsible. What is the xdata? I run the command: tmsh show /sys memory | egrep 'Max\ Alloc|xdata' SubSystem Alloc (bytes) Max Alloc (bytes) Object Size xdata 1.1G 1.1G 2048 Thanks, Jayme
How to calculation memory in F5.
Hi, I would like to know how the F5 calculation it's memory cause when I use command 'tmsh show sys performance system detail' I got the following images attached. you can look at yellow text (other memory used in percentage) it's about 54% from all memory in F5. I would like to know what is the formular to calculation that number? If you look at the images in the section memory breakdown Other used is just 6.8GB/50GB it's about 13.5% of all memory. Thank you.Time It Takes the Fingers to Remember a New Password? About 3 days
Recently I changed some of my passwords. Some due to typical rotation time and a couple due to potential breaches and encouragement from the affected site. No, I’m not going to tell you which ones or how I go about it but I noticed that it took about 3 days for my fingers to key the correct combination. This has probably happened to you too, where after changing a password, you inadvertently enter the old password a number of times since that is what the fingers and hands remember. Yes, I’m sure many of you have password keepers (which have also been breached) locked by a master and I use one too, but for many of my highly sensitive passwords, I keep those in my head. As I continued to enter the old password for a couple days only to correct myself, I started thinking about habits and muscle memory. Some adages talk about it taking about 30 days (66 days in this study) to either pick up or drop a habit if done daily. Want to keep an exercise routine? Do it daily for a month and you are more than likely to continue...barring any unforeseen circumstances. And then there’s muscle memory. Things like riding a bike, signing your name, catching a ball or any repetitious, manual activity that you complete often. Your muscles already know how to do it since they’ve been trained over time. You do not need to think about, ‘OK, as it gets closer, bring your hands together to snag it from the air,’ it just happens. This is one of the reasons why people change or update certain exercise or resistance routines – the muscles get used to it and need a different approach to reach the next plateau. I wondered if anyone else had thought of this and a quick search proved that it is a bona fide technique for password memory. Artists like musicians use repetitive practice for scale patterns, chords, and melodic riffs and this trains the muscles in the fingers to 'remember' those patterns. It is the same notion with passwords. Choose a password that alternates between left and right hands that have some rhythm to it. After a bit, the hands remember the cadence on the keyboard and you really do not need to remember the random, committed numbers, letters or Shift keys pounced while typing your secret. This is ideal since only your fingers remember not necessarily your mind. Granted, depending on how your head works this technique might not work for everyone but it is still an interesting way to secure your secrets. And you can brag, 'If you break my fingers, it'll wipe the device.' ps Related: Memorizing Strong Passwords Muscle memory passphrases and passwords Muscle Memory: Scientists May Have Unwittingly Uncovered Its Mystery Muscle Memory Solved
TMOS v14.1.2.1
We upgraded our vCMP pairs from v13.1.1.4 to v14.1.2 and got "out of memory" and "tmm reload" issue. I spent a month with F5 support did not resolve issue. Now we have 2 core and LTM only (we removed AVR) and swap group up to 100% in couple weeks. F5 recommend use 4 cores for LTM only. Most of pair have less then 10Mbits traffic throughput. We are losing faith in F5 :( Does anybody have same issue with v14 code?sys db provision.memory.asm
Does anyone know how the sys db keys for provision are used? Can they be custom tuned for host/tmm memory allocations? (BIGIP1-NPRD)(cfg-sync In Sync)(/S1-green-P::Active)(/Common)(tmos) list sys db provisioned.memory.asm sys db provisioned.memory.asm { value "2286" } (BIGIP1-NPRD)(cfg-sync In Sync)(/S1-green-P::Active)(/Common)(tmos) list sys db provision.memory.asm sys db provision.memory.asm { value "2286" }healthiness of F5 - help with errors
Dear Gents, I am reviewing /var/log/ltm logs and KPI statistics on two high available F5s running on release 12.1.3. I am looking for root cause of high memory or CPU usages. There are several problems bellow. Q1 First of all other memory utilization is hitting 70 % (tmm memory just 20%). Also CPU usage is hitting over 70% in peaks. Is there any hint how or where to start high memory or CPU usage issue troubleshooting? F5 is used as loadbalancer. There is very low traffic flowing across F5s around 50 Mbps. Q2 There is seen in ltm logs following error: mcpd[7118]: Sync of device group /Common/device-group-failover-2530624eebc7 to commit id 6666 Is it something on which keep eye? Q3 There is error with unreachable pool member and failing monitor ( err f5-A tmm[12726]: No members available for pool /poolX/pool_A ltm 07-31 05:43:50 notice f5-A mcpd[7118]: Pool /part_X/pool_A member /part_www/10.XX.YY.ZZ:80 monitor status down). I can fix it easily but can such unreachable pool member use high CPU or Memory usage? Any advice helping troubleshooting errors above would be highly appreciated. Many thanks Regards Andy
