Monitor critical temperatures in Ubuntu Server - v.12.04.2

Discussion and Technical Support for general software applications and utilities including OS related issues.
danhansen@denmark
Member
Member
Posts: 14
Joined: May 27th, 2013, 3:43 pm

Monitor critical temperatures in Ubuntu Server - v.12.04.2

Post by danhansen@denmark »

Hi everybody...

I'm new on this board so I am not quite sure if this is the right place to post this.. Sorry if it is not..


I have a Ubuntu Server 12.04.2 with ISPConfig on an older PC / Intel P4 / 2Gb RAM. its running quite good, but I would really like to have this security function which I found on this nice site. The "Monitor Critical Temp. in Ubuntu" function that is. But,

After having tried to follow the ToDo and seaching the sites I still haven't solved the problem. Therefore I hope some of you experts in here would like to help out a Linux not quite newbie, but allmost...

The problem is that I have tried to run "sensors-detect" quite some times and every time with the same result.. It finds:
"Found `ITE IT8712F Super IO Sensors' Success! (address 0x290, driver `it87')"

But afterwards when I am trying to run "sensors" to se some results, nothing appears... Only:
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.


Then of course, when I am running the script I get errors like:
JOB RUN AT Mon May 27 18:55:25 CEST 2013
=======================================

CPU Warning Limit set to => 35
CPU Shutdown Limit set to => 50


No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.


No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
./CPUTempShutdown.sh: line 46: [: -ge: unary operator expected
./CPUTempShutdown.sh: line 56: [: -ge: unary operator expected
Temperature temp1 OK at =>

CPU Core is within limits


I have prepared listing, info and files in case anybody will be able to help me.. And I'm sitting right here, just in case anybody see's this ;)

Anybody who can help me? Please help out a stupid dane ;)

Kind Regards,
Dan Hansen, Denmark
User avatar
Ian
Moderator
Posts: 752
Joined: January 1st, 2011, 7:00 am

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by Ian »

Hi Dan and welcome to the forums :thumbup:

I'm not familiar with that issue but a bit of goggling suggests you can fix it as follows:

Issue the following command:

Code: Select all

sudo vim /etc/default/grub
On or around line 7 of the file which opens you should see the following line:
GRUB_CMDLINE_LINUX=""
Change it to read as follows:
GRUB_CMDLINE_LINUX="acpi_enforce_resources=lax"
Save and close the file then run

Code: Select all

update-grub
Then reboot and see if that helps.

Good luck ;)

Ian.
danhansen@denmark
Member
Member
Posts: 14
Joined: May 27th, 2013, 3:43 pm

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by danhansen@denmark »

Hi Ian,

You are a g.. d... genius!!! No more and no less! Genius! :clap:

Thanks my friend, I have been fighting this 10337 hours and this is really important to me...
Now I will go on with the script and see if I can modify it to work.. But the main error has been solved!

Thanks again!!!

Kind Regards
DanHansen,Denmark
User avatar
Ian
Moderator
Posts: 752
Joined: January 1st, 2011, 7:00 am

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by Ian »

Glad it helped ;)

Ian.
danhansen@denmark
Member
Member
Posts: 14
Joined: May 27th, 2013, 3:43 pm

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by danhansen@denmark »

Hi Ian,

Sorry to disturb you again ;)

1.
>>/home/'user'/Desktop/CPUWarning.Log
"Desktop" folder? I don't understand why they chosen "Desktop"? There is no "Desktop" on the Ubuntu Server Edition 12.04.2 LTS, or am I clowning around again??? Is it perhaps a way to keep it simple/keep it in order?:
I made a folder named MonitorHardware where the logs can be located. But is there a better way? I mean is there a way to place these log files, so that it is possible to see them online perhaps? Or is this risky? Symlinks maybe? Any ideas?

2.
I tried running the script and it works.. I have been testing with different settings to trigger an email warning and a shutdown. Until now I succeeded in triggering a warning onscreen by the script and a log entry... This is all good. I also succeeded in getting the system to shutdown due to to high temp.! But, the email is not being sent - and an error occurs:
"Line 56" "No such file or directory"]

This is Line 56:
/usr/sbin/ssmtp myemail@myaddress.com </home/'user'/MyScripts/msg.txt[/i]

Here's how it's done/shown in the script:
if [ ${newstr} -ge $2 ]
then
echo '============================'
echo ''
echo 'CRITICAL: TEMPERATURE CORE' $i 'EXCEEDED' $2 '=>' $newstr
echo ''
echo '============================'
/sbin/shutdown -h now
/usr/sbin/ssmtp myemail@myaddress.com </home/'user'/MyScripts/msg.txt
echo 'Email Sent.....'
exit
else
echo ' Temperature Core '$i' OK at =>' $newstr
echo ''
fi


a. Isn't suppose to make a textfile itself and add the echo part into this file and mail it? or how does it work?
b. isn't getting a shutdown command before sending the email.. Sorry for the newbie question. Last time I programmed other than PHP/Html etc., was Basicv.7.0 and Comal80 ;)
c. I can't see what the variabel "$i" is used for nthis modified script. Here's the original script:
http://www.havetheknowhow.com/scripts/C ... utdown.txt
It's a script for a computer with Duo Core, I get that, but I can't see why it's kept in the altered script which is the one I am using:
http://forum.havetheknowhow.com/viewtop ... sors#p2086

3.
What I am trying to do is:
a. to alter the sqript to log and mail if "CPU Warning Limit" has exceeded --> CPUwarning.Log ... And, to log and mail and shutdown if "CPU Critical Limit" has exceeded --> CPUwarning.Log - but with the "CRITICAL...." text or mabye this is stupid.. I get the warning and the temp..
This way I am warned by mail if it is running hot before it gets even hotter and shut's down ;)
b. the last "loop" I guess is a confirmation regardless of the temp. as long as it doesn't exceed critical limit???
else
echo ' Temperature temp1 OK at =>' $newstr
echo ''
fi


I will try to make the script myself and if I succeed, will you then take a look at it???


Kind Regards,
DanHansen
User avatar
Ian
Moderator
Posts: 752
Joined: January 1st, 2011, 7:00 am

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by Ian »

Hi Dan, you're not disturbing me at all ;)


danhansen@denmark wrote:1.
>>/home/'user'/Desktop/CPUWarning.Log
"Desktop" folder? I don't understand why they chosen "Desktop"? There is no "Desktop" on the Ubuntu Server Edition 12.04.2 LTS, or am I clowning around again??? Is it perhaps a way to keep it simple/keep it in order?:
I made a folder named MonitorHardware where the logs can be located. But is there a better way? I mean is there a way to place these log files, so that it is possible to see them online perhaps? Or is this risky? Symlinks maybe? Any ideas?
Good point. I use VNC on my server so do have a Desktop folder. You can put the log anywhere, the folder you have created is ideal.
danhansen@denmark wrote:2.
I tried running the script and it works.. I have been testing with different settings to trigger an email warning and a shutdown. Until now I succeeded in triggering a warning onscreen by the script and a log entry... This is all good. I also succeeded in getting the system to shutdown due to to high temp.! But, the email is not being sent - and an error occurs:
"Line 56" "No such file or directory"]

This is Line 56:
/usr/sbin/ssmtp myemail@myaddress.com </home/'user'/MyScripts/msg.txt[/i]

Here's how it's done/shown in the script:
if [ ${newstr} -ge $2 ]
then
echo '============================'
echo ''
echo 'CRITICAL: TEMPERATURE CORE' $i 'EXCEEDED' $2 '=>' $newstr
echo ''
echo '============================'
/sbin/shutdown -h now
/usr/sbin/ssmtp myemail@myaddress.com </home/'user'/MyScripts/msg.txt
echo 'Email Sent.....'
exit
else
echo ' Temperature Core '$i' OK at =>' $newstr
echo ''
fi


a. Isn't suppose to make a textfile itself and add the echo part into this file and mail it? or how does it work?
b. isn't getting a shutdown command before sending the email.. Sorry for the newbie question. Last time I programmed other than PHP/Html etc., was Basicv.7.0 and Comal80 ;)
c. I can't see what the variabel "$i" is used for nthis modified script. Here's the original script:
http://www.havetheknowhow.com/scripts/C ... utdown.txt
It's a script for a computer with Duo Core, I get that, but I can't see why it's kept in the altered script which is the one I am using:
http://forum.havetheknowhow.com/viewtop ... sors#p2086
The textfile referred to in the script is simply a "email template" which reads:

Code: Select all

To: ToEmail@SomeAddress.com
From: FromEmail@SomeAddress.com
Subject: CPU/Drive Temperature Exceeded



The server has shut down!
a. I thought I'd included a copy in my instructions but obviously not :oops: You're the first person to spot this mistake!

b. The reason the shutdown command comes before the command to send the email is to ensure the server gets shutdown. I didn't want it staying alive if the email command made the whole script hang. The server does not shut down instantly and so has enough time to run the other lines in the script.

c. Good spot once more! You can just remove that variable.
danhansen@denmark wrote: 3.
What I am trying to do is:
a. to alter the sqript to log and mail if "CPU Warning Limit" has exceeded --> CPUwarning.Log ... And, to log and mail and shutdown if "CPU Critical Limit" has exceeded --> CPUwarning.Log - but with the "CRITICAL...." text or mabye this is stupid.. I get the warning and the temp..
This way I am warned by mail if it is running hot before it gets even hotter and shut's down ;)
b. the last "loop" I guess is a confirmation regardless of the temp. as long as it doesn't exceed critical limit???
else
echo ' Temperature temp1 OK at =>' $newstr
echo ''
fi


I will try to make the script myself and if I succeed, will you then take a look at it???
Just add the email command to the first "if" (the warning "if") and it should work as you want it to. You would need to create a separate "msg.txt" file for the "warning if" and change the text in it to suit.

Good luck and, yes, please send me the script if you get stuck :thumbup:

Ian.
danhansen@denmark
Member
Member
Posts: 14
Joined: May 27th, 2013, 3:43 pm

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by danhansen@denmark »

Hi Ian,

OK, I had to change the whole server park to RACK mounted computers instead. There was no place left here ;)
This is why I haven't posted any feedback other than my last post..

I hope you will help me one more time, since it is now time for me to finish my project "CPUSingleTempShutdown.sh" and "CPUMultipleTempShutdown.sh"
I will repeat my wish from last time so that you can see what the issues were:



1.
Regarding the sending of the mail/txt file --> "/usr/sbin/ssmtp myemail@myaddress.com </home/'user'/MyScripts/msg.txt"
I Ubuntu Server 12.04.2 it's "sendmail" which I have to use, is it not? If, please show me the path/or the full code line ;)

2.
What I am trying to do is:
a. to log and mail if "CPU Warning Limit" has exceeded - to log the temp./date etc. into "CPUwarning.Log" and mail it as well! Headline/Subject "CPU TEMP. WARNING" Not just the standard text file but a warning with info about the temp., date etc!
b. to log and mail and shutdown if "CPU Critical Limit" has exceeded - to log the temp./date etc. into "CPUwarning.Log"and mail it as well! Headline/Subject "CPU TEMP. CRITICAL - COMPUTER SHUTDOWN" Not just the standard text file but a warning with info about the temp., date etc. and SHUTDOWN!

This way I am warned by mail if it is running hot before it gets even hotter and shut's down ;)

I will try to make the script myself and post it for you to see, but is it maybe possible, that you could do a suggestion/point me in the right direction?

Last time you wrote:
Just add the email command to the first "if" (the warning "if") and it should work as you want it to. You would need to create a separate "msg.txt" file for the "warning if" and change the text in it to suit.
Good luck and, yes, please send me the script if you get stuck :thumbup:
Ian.

But I was not sure exactly what you meant ;)

Hoping to hear from you - Thanks in advance ;)

Kind Regards,
Dan Hansen
Denmark
anp
Member
Member
Posts: 5
Joined: November 13th, 2013, 9:13 pm

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by anp »

Hi everybody,

I also decided to get into server building stuff :)
Right now I am in the middle - the computer is up and running Ubuntu 12.04.03 LTS server with Open SSH, VNC, Webmin, ssmtip, lmsensors. As you might have noticed I am just following the manual step by step - and thank you very much to the author :clap:

So, my question is about the CPU temperature monitoring.
First some observation. I did not pay much attention to ">>" in echo opearators in "warning" section, and I was quite concerned when nothing would appear on the screen no matter what warning thresholds I tried, and I thought the script did not work... Until I noticed "CPU warning log.txt" :oops:
Now I know it does!

My question is about the second section - critical temperature:

Code: Select all

if [ ${newstr} -ge $2 ]
  then
    echo '============================'
    echo ''
    echo 'CRITICAL: TEMPERATURE CORE' $i 'EXCEEDED' $2 '=>' $newstr
    echo ''
    echo '============================'
    /sbin/shutdown -h now
    /usr/sbin/ssmtp myemail@myaddress.com </home/htkh/MyScripts/msg.txt
    echo 'Email Sent.....'
    exit
Well, the quesiton is about shutdown command. When I test the script with low shutdown threshold, it would say "shutdown: Need to be root" and only send an e-mail to me. :?

Is it possible to make it work executing the script as "user", or it must be executed as "root"?

Thank you!
Anton.
New Zealand
anp
Member
Member
Posts: 5
Joined: November 13th, 2013, 9:13 pm

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by anp »

Hi again,

I think I found the answer myself, thank Google and man shutdown
Looks like shutdown must be run as root only, so I assigned the script in Webmin to root.

Also, when I tested the script, I found out the following:
1. If CPU temperature reaches warning limit, the script nevertheless reports at the end that both cores are OK, not sure if it was a bug or a feature, but I wanted to change that.
2. When I test-run the script as a cron job in Webmin, I would always get a message "Bad substitution". Turned out the very first line of each script should be #!/bin/bash, but I did not copy it :oops: When the line was added, the message dissapeared.
3. after shutdown now command e-mail will not be sent. Actually nothing will be done (at least on my computer). I tried to add time delay as shutdown +1 which suppose to shutdown in 1 minute. However, it did not help. The server indeed would switch itself off in 1 minute, but no other commands after shutdown would be executed. So I moved shutdown to be last command after sending an e-mail.
4. I wanted the script to send me e-mail also in case temperature reaches a warning limit, not just record that in a log.
5. Also I thought it might be a good idea if the message that is sent to me would contain info about when the temperature was high, and what the temperature actually was.

So, I changed the script a bit ;) to do all that stuff:

Code: Select all

#!/bin/bash
# PURPOSE: Script to check temperature of CPU cores and report/shutdown if spe$
#
# AUTHOR: feedback[AT]HaveTheKnowHow[DOT]com; modified by anp :)

# Expects two arguments:
#    1. Warning temperature
#    2. Critical shutdown temperature
#    eg. using ./CPUTempShutdown.sh 30 40
#        will warn when temperature of one or more cores hit 30degrees and shu$

# NOTES:
# Change the strings ">>/home/htkh" as required
# Substitute string "myemail@myaddress.com" with your own email address in the$

# Assumes output from sensors command is as follows:
#
# coretemp-isa-0000
# Adapter: ISA adapter
# Core 0:      +35.0 C  (high = +78.0 C, crit = +100.0 C)  
#
# coretemp-isa-0001
# Adapter: ISA adapter
# Core 1:      +35.0 C  (high = +78.0 C, crit = +100.0 C) 
#
# if not then modify the commands str=$(sensors | grep "Core $i:") & newstr=${$

echo "JOB RUN AT $(date)"
echo "======================================="

echo ''
echo 'CPU Warning Limit set to => '$1
echo 'CPU Shutdown Limit set to => '$2
echo ''
echo ''

#sensors

echo ''
echo ''

for i in 0 1
do

  str=$(sensors | grep "Core $i:") #get string with Core from sensors
  newstr=${str:15:2} #truncate the string to first two digits of temperature

  #First check if the temperature is critical.
  if [ ${newstr} -ge $2 ]
  then
    echo '============================'
    echo ''
    echo 'CRITICAL: TEMPERATURE CORE' $i 'EXCEEDED' $2 '=>' $newstr
    echo ''
    echo '============================'
    #
    #Creating message:
    echo 'From: YourMail@gmail.com' >/home/USER_NAME/MyScripts/msg.txt
    echo 'Subject: CRITICAL server temperature' >>/home/USER_NAME/MyScripts/msg.txt
    echo '' >>/home/USER_NAME/MyScripts/msg.txt
    echo "CPU core "$i" reached critical temperature of "$newstr" at $(date). Server was shut down." >>/home/USER_NAME/MyScripts/msg.txt
    /usr/sbin/ssmtp YourMail@gmail.com </home/USER_NAME/MyScripts/msg.txt
    echo 'Email Sent.....'
    /sbin/shutdown -P now
    exit
  #Now check if the temperature just exeeds normal
    elif [ ${newstr} -ge $1 ]
    then
      echo 'From: YourMail@gmail.com' >/home/USER_NAME/MyScripts/msg.txt
      echo 'Subject: Warning Server temperature' >>/home/USER_NAME/MyScripts/msg$
      echo '' >>/home/USER_NAME/MyScripts/msg.txt
      echo "CPU core "$i" reached high temperature of "$newstr" at $(date)." >>/home/USER_NAME/MyScripts/msg.txt
      /usr/sbin/ssmtp YourMail@gmail.com </home/USER_NAME/MyScripts/msg.txt
      echo 'Email Sent.....'
    else
  #OK, temperature within normal limit 
      echo ' Temperature Core '$i' OK at =>' $newstr
      echo ''
  fi

done

echo 'Script completed'
echo ''
anp
Member
Member
Posts: 5
Joined: November 13th, 2013, 9:13 pm

Re: Monitor critical temperatures in Ubuntu Server - v.12.04

Post by anp »

Hi there,

Ian, again huge THANK YOU!!! for such a great site. :clap:

My server works as a clock! It serves for Win7 and Ubuntu machines, and I actually converted one of my old laptops into a media-player. A player laptop runs Ubuntu (well, actually Lubuntu) desktop, and connected to a TV and wirelessly (only 48 Mb/s, as it is too old) to the server (using NFS). The bandwidth is not great, but this is enough to watch even DVDs (provided no one skypes or streams anything at the same time :) With wireless mouse I have full control I need and can do whatever I want :crazy:

Also my media server works as a gateway router, having two LAN cards, and monitors total daily internet traffic, restricting it to 4 Gb per day, so I am always sure my inet will not dissapear too suddenly :D

However, now, I just wanted to post yet another modification of the original critical temperatures script. Since output of sensors command also gives (in my case) CPU FAN Speed: 2500 rpm, I thought it might be a nice idea to monitor this parameter as well, and shut down the server in case the fan brakes down. It accepts critical CPU fan speed as a third paramenter, and in Webmin it should be run as e.g. /home/htkh/MyScripts/CPUTempShutdown.sh 35 45 1900 >/dev/null

I hard-tested it by pulling out CPU-fan plug from a socket on the motherboard - in a minute the server shut down, and I got e-mail :)

Well, here is the script:

Code: Select all

#!/bin/bash
# PURPOSE: Script to check temperature of CPU cores and CPU fan speed and report/shutdown if specified temperatures exceeded
# or CPU fan does not work.

# AUTHOR: feedback[AT]HaveTheKnowHow[DOT]com; modified by anp :)
#
# Expects two arguments:
#    1. Warning temperature
#    2. Critical shutdown temperature
#    3. Critical low fan speed
#    eg. using ./CPUTempShutdown.sh 30 40 1900
#        will warn when temperature of one or more cores hit 30degrees
#        and shutdown when either hits 40degrees, or CPU fan speed is lower
#	 than 1900 rpm.


# NOTES:
# Change the strings ">>/home/htkh" as required
# Substitute string "myemail@myaddress.com" with your own email address in the string which starts "/usr/sbin/ssmtp myemail@myaddress.com"

# Assumes output from sensors command is as follows:
#
# coretemp-isa-0000
# Adapter: ISA adapter
# Core 0:      +35.0 C  (high = +78.0 C, crit = +100.0 C)  
#
# coretemp-isa-0001
# Adapter: ISA adapter
# Core 1:      +35.0 C  (high = +78.0 C, crit = +100.0 C) 
#
# if not then modify the commands str=$(sensors | grep "Core $i:") & newstr=${str:14:2} below accordingly

echo "JOB RUN AT $(date)"
echo "======================================="

echo ''
echo 'CPU Warning Limit set to => '$1
echo 'CPU Shutdown Limit set to => '$2
echo 'CPU fan rpm limit set to =>' $3
echo ''
echo ''

#sensors

#check CPU fan speed:
cpufan=$(sensors | grep "CPU FAN Speed:")
cpufan_t=${cpufan:20:4}

if [ ${cpufan_t} -lt $3 ]
  then
    #Creating message:
    echo 'From: myemail@myaddress.com' >/home/htkh/MyScripts/msg.txt
    echo 'Subject: CRITICAL CPU fan speed' >>/home/htkh/MyScripts/msg.txt
    echo '' >>/home/htkh/MyScripts/msg.txt
    echo "CPU fan speed was "$cpufan_t "at $(date). Server was shutdown" >>/home/htkh/MyScripts/msg.txt
    /usr/sbin/ssmtp myemail@myaddress.com </home/htkh/MyScripts/msg.txt
    echo 'Email Sent.....'
    /sbin/shutdown -P now

    echo '============================'
    echo ''
    echo 'CPU fan speed is too low at' $cpufan_t
    echo ''
    echo '============================'
    exit
  else
    echo ' CPU fan speed is OK at' $cpufan_t
    echo ''
fi

#check CPU temperature
for i in 0 1
do

  str=$(sensors | grep "Core $i:") #get string with Core from sensors
  newstr=${str:15:2} #truncate the string to first two digits of temperature

  #First check if the temperature is critical.
  if [ ${newstr} -ge $2 ]
  then
    echo '============================'
    echo ''
    echo 'CRITICAL: TEMPERATURE CORE' $i 'EXCEEDED' $2 '=>' $newstr
    echo ''
    echo '============================'
    #
    #Creating message:
    echo 'From: myemail@myaddress.com' >/home/htkh/MyScripts/msg.txt
    echo 'Subject: CRITICAL CPU temperature' >>/home/htkh/MyScripts/msg.txt
    echo '' >>/home/htkh/MyScripts/msg.txt
    echo "CPU core "$i" reached critical temperature of "$newstr" at $(date). Server was shut down." >>/home/htkh/MyScripts/msg.txt
    /usr/sbin/ssmtp myemail@myaddress.com </home/htkh/MyScripts/msg.txt
    echo 'Email Sent.....'
    /sbin/shutdown -P now
    exit

  #Now check if the temperature just exeeds normal
    elif [ ${newstr} -ge $1 ]
    then
      echo 'From: myemail@myaddress.com' >/home/htkh/MyScripts/msg.txt
      echo 'Subject: Warning High CPU temperature' >>/home/htkh/MyScripts/msg.txt
      echo '' >>/home/htkh/MyScripts/msg.txt
      echo "CPU core "$i" reached high temperature of "$newstr" at $(date)." >>/home/htkh/MyScripts/msg.txt
      /usr/sbin/ssmtp myemail@myaddress.com </home/htkh/MyScripts/msg.txt
      echo 'Email Sent.....'
    else

  #OK, temperature within normal limit 
      echo ' Temperature Core '$i' OK at =>' $newstr
      echo ''
  fi

done

echo 'Script completed'
echo ''

Cheers, Anton (NZ)
Post Reply