There’s a particular kind of panic that sets in when you SSH into a production server and see this:
load average: 45.63, 38.37, 28.93On a 2-core machine, that’s not just high — it’s catastrophic.
I usually help one of my friends with LAMP servers hosted on DigitalOcean that run WooCommerce. The site brings in good sales for his business. Recently, he reached out to me to say that some of his customers reported slow order placement. When I logged into the server, I found an interesting pattern.
This post walks through a real debugging session using a symptoms → diagnostics → solution approach. Along the way, we’ll uncover multiple overlapping issues (not just one), fix them step by step, and explain why architectural changes like PHP-FPM and Nginx matter.
Symptoms: What went wrong
The server started showing:
- Extremely high load averages (45+ on a 2-core system)
- Slow or unresponsive web requests
- CPU is constantly maxed out
- Intermittent recovery followed by spikes
Initial snapshot:
# uptime
load average: 5.95, 25.07, 25.33
# nproc
2Even after partial recovery, the load remained unstable.
Diagnostics: What the system revealed
1. Top CPU consumers
# ps aux --sort=-%cpu | head -20Output (trimmed):
root 92 35.8 0.0 0 0 ? S 12:40 82:49 [kswapd0]
mysql 198808 18.6 10.9 1821488 439632 ? Ssl 16:29 0:31 /usr/sbin/mysqld
www-data 197164 5.6 5.1 504092 205036 ? S 16:16 0:51 /usr/sbin/apache2The key observation from this is that the process kswapd0 is consuming 35% CPU. This is not normal. It means the kernel is struggling with memory pressure.
2. Apache process explosion
# ps aux | grep apache | wc -l
14RSS is the actual physical RAM a process is using right now, measured in KB. It does NOT include swapped-out memory, so it represents memory currently resident in RAM. It is the single most important metric for sizing concurrency.
In the output, I saw that the RSS is approximately 200MB – 260MB for each Apache process.
So for 14 processes it is:
14 processes × ~220MB ≈ ~3GB RAM
On a 4GB system, that’s quite high.
3. MySQL check (surprisingly clean)
When I checked the full process list on the MySQL
mysql> SHOW FULL PROCESSLIST;I found it clean, with a few sleep connections and no long-running queries. I verified it with
# mysqladmin processlistand found a similar output. So MySQL wasn’t the bottleneck.
4. Network state – hidden problem
The netstat revealed a hidden problem that may be contributing to the sluggishness.
# netstat -ant | awk '{print $6}' | sort | uniq -c
.....
121 SYN_RECV
.....This indicates:
- Many half-open TCP connections
- Likely bot traffic or SYN flood behavior
5. System pressure via vmstat
In this case, vmstat was the most powerful tool run. In the output,
- r is the number of runnable processes (waiting for CPU). Ideally, it should have a value less than or equal to the number of CPU cores. A value exceeding the number of available CPU cores on the machine would indicate CPU contention.
- id indicates a percentage of CPU that is idle. A value typically in the range of 70-100% indicate a relaxed system. A low value (say 0-20%) indicates a busy CPU. However, 0% means it is fully saturated.
- si and so are swapped in and out. A value of 0 indicates no swapping and is considered good. Occasionally, a value > 0 indicates mild pressure. But if this value remains above 0 continuously, it may indicate memory problems.
So when I ran:
# vmstat 1 5Output (trimmed):
r b swpd free si so us sy id
14 0 0 399400 0 0 34 29 35
15 0 0 362864 0 0 87 12 0r with a value of 14-15 indicates too many runnable processes, and id with 0 means CPU is fully saturated.
After initial fixes, when I ran vmstat again, I saw the new numbers:
r b swpd free si so us sy id
1 0 12120 2554084 0 0 34 29 35
0 0 12120 2554084 0 0 0 1 99So, now a value of r between 0-2 indicates a healthy condition, an id of 86-89% indicate idle CPU, and a si/so of 0 indicates no swapping.
r = 0–2→ healthyid = 86–99%→ CPU idlesi/so = 0→ no swapping
Three Root Causes
This wasn’t a single issue. It was a stacked failure:
1. Apache (mod_php) memory bloat
- Each request = full Apache process
- Each process ≈ 200MB+
- Too many workers → RAM exhaustion
2. Swap thrashing (kswapd0)
- Memory filled up
- Kernel started reclaiming memory
- CPU burned by swap management
3. Connection pressure (SYN_RECV flood)
- 121 half-open connections
- Apache workers are tied up waiting
Solutions Applied
1. SYN flood mitigation (UFW + kernel)
I enabled:
net.ipv4.tcp_syncookies=1And:
ufw limit 80/tcp
ufw limit 443/tcp2. Apache concurrency control
Reduced workers:
MaxRequestWorkers 6This helped stabilize the CPU with no process pile-up
3. KeepAlive tuning
KeepAlive On
MaxKeepAliveRequests 50
KeepAliveTimeout 24. OPcache verification and tuning
When PHP runs a script, it parses PHP code, compiles it into bytecode, and executes it. Without OPcache, this happens on every request.
With OPcache enabled, compiled bytecode is stored in memory so that future requests can reuse it. Without OPcache, high CPU usage and slower response times are expected. With OPcache, 30-35% less CPU is used, and execution is faster.
When I checked, I found that OPcache (opcache.enable) was already enabled in the php.ini.
I improved it with more cache:
opcache.memory_consumption=192
opcache.interned_strings_buffer=16
opcache.max_accelerated_files=20000Additional Changes I would like to make
1. Replace mod_php with PHP-FPM
I would want to replace mod_php with php-fpm. In mod_php, each Apache process embeds PHP, leading to high memory usage (~200 MB per worker). This results in poor scalability and a lack of separation of concerns.
PHP-FPM, on the other hand, runs as a separate service and has lightweight workers (~20-40 MB), providing better process control and supporting pooling and scaling. This will result in lower memory usage, better CPU efficiency, and more predictable performance.
2. Prefer Nginx Over Apache
Now, this is not about nginx hype; it’s about an architectural choice. I have been using Apache for quite some time and love it. The pre-fork model of Apache has a process/thread per connection, is memory-heavy, and struggles under concurrency.
Nginx, with its event-driven model, can handle thousands of connections with a few processes and non-blocking I/O, making it an ideal choice for modern web workloads.
Finally
What looked like a “CPU problem” turned out to be:
- Memory exhaustion
- Connection pressure
- Poor process model
Fixing it required layered thinking, not just tweaking one parameter.
And the biggest lesson?
One can tune one’s way out of trouble temporarily, but the real win comes from choosing the right architecture.
So, now, if you’ve ever seen load averages that made no sense, this pattern might look familiar. And now you know exactly how to break it down.
