Linux commands to monitor a FCSDK system
Linux commands that can be used by sysadmins to monitor a FCSDK deployment on their network.
1. Confirm the Web Gateway is running on a given host:
netstat -anp | grep 5060 (to confirm the WG process is up & has bound to the port)
e.g. success return value:
tcp 0 0 192.168.9.34:5060 0.0.0.0:* LISTEN 24624/java
udp 0 0 192.168.9.34:5060 0.0.0.0:* 24624/java2. Confirm the Media Broker is running on a given host:
netstat -anp | grep 16000 (to confirm the MB process is up & has bound to the port.)
e.g. success return value:
udp 0 0 ::ffff:192.168.9.34:16000 :::* 11134/java3. There should be 6 java processes for WG & MB running on same box.
ps -ef | grep java (to confirm desired number of java processes are running)
e.g. success return value:
[root@cs-fcsdk21 ~]# ps -fC java
UID PID PPID C STIME TTY TIME CMD
root 11134 11123 3 Oct24 ? 15:50:49 /opt/java/bin/java -DTRACE=true -Djava.awt.headless=true -Xmx512M -Xms512M -cp . -jar rtp-proxy.jar
root 24537 24488 0 Oct28 ? 00:13:48 /opt/java/bin/java -D[Process Controller] -server -Xms64m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi
root 24553 24537 0 Oct28 ? 00:53:29 /opt/java/bin/java -D[Host Controller] -Dorg.jboss.boot.log.file=/opt/cafex/FAS-2.1.17/domain/log/host-controller.log -Dcom.aliceca
root 24618 24537 1 Oct28 ? 04:03:09 /opt/java/bin/java -D[Server:appserver-cs-fcsdk21] -XX:PermSize=256m -XX:MaxPermSize=256m -Xms256m -Xmx512m -server -XX:+UseG1GC -X
root 24624 24537 0 Oct28 ? 01:59:23 /opt/java/bin/java -D[Server:loadbalancer-cs-fcsdk21] -XX:PermSize=128m -XX:MaxPermSize=128m -Xms256m -Xmx512m -server -XX:+UseG1GC
root 24679 24537 0 Oct28 ? 00:46:50 /opt/java/bin/java -D[Server:management] -Xms64m -Xmx512m -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=34. Monitoring logs for Exceptions **OPTIONAL - Some exceptions will be expected**
Only run this script if you think you have a problem that requires active monitoring.
Your server will need to be able to send outbound email.
Create a bash script to check for exceptions such as:
cat /root/check-for-exceptions.sh:
#!/bin/bash
# Script to check for exceptions and send email to user
grep "Exception" /opt/cafex/FAS-/domain/servers/appserver-cs-/log/server.log | mail -s "Exceptions in server.log" sparfitt@cafex.com
Then add to crontab:
*/30 * * * * /root/check-for-exceptions.sh
5. Checking CPU & Memory usage
Use the TOP command to monitor CPU & memory usage, if a CPU is at 100% it can lead to unexpected behaviour. Ensure the server has plenty of free CPU and memory. E.g
root@cs-fcsdk21 ~]# top
top - 10:34:10 up 55 days, 23:13, 1 user, load average: 0.00, 0.00, 0.00
Tasks: 134 total, 1 running, 133 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.7%us, 0.6%sy, 0.0%ni, 98.0%id, 0.3%wa, 0.0%hi, 0.0%si, 0.4%st#
Mem: 3922700k total, 3805288k used, 117412k free, 167968k buffers
Swap: 8184k total, 8184k used, 0k free, 1334000k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
20884 root 20 0 3067m 302m 14m S 10.0 7.9 1637:21 java
30045 root 20 0 3599m 651m 14m S 2.0 17.0 383:53.64 java
28920 root 20 0 3218m 405m 10m S 0.3 10.6 106:11.47 java
28977 root 20 0 2884m 396m 10m S 0.3 10.3 44:50.85 java
1 root 20 0 19360 1220 984 S 0.0 0.0 0:01.06 init<
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:13.13 migration
4 root 20 0 0 0 0 S 0.0 0.0 6:10.56 ksoftirqd
The above command shows us the CPU is at 0.7% so there is plenty available for FCSDK. It shows the server has 4GB of memory available of which 1GB is free, so again plenty of resources available to the FCSDK