Basic webserver for alive check

[x21]

Hello,

I was wondering if there's any basic webserver mod to implement a check to see if a server is alive or has hung

[Gizzy]

skw (I believe) made a tool that kept a server running and would automatically reboot it if it crashed, not sure how extensively it logs crashes though

summoning bob who probably has it archived somewhere

[x21]

Currently a simple batch loop can restart the server if it crashes, but it won't help if the server hangs. I want a more robust solution to the problem. I suppose web admin could be abused to do the job... But that's a clumsy way of doing things.

[Smirftsch]

guess it could work to use a masterserver entry as a health beacon

[BobIsUnreal]

Gizzy , this is what you talked about.
http://www.bobisunreal.com/Random/Forum/guard.zip
its not to much more then a batch file with colors and timestamps in batch saving,
witch is unneccesary with 227's logtimestamp argument.

detecting the hang via the webserver is kinda a noval idea althought i am not it would work, but it might.
you will have a issue tho - mapswitches. the webserver isnt "always" running. it dosnt run intill the map starts. so while you map is switching , you can get a false detection of failure, map switch time increases with package load/unloads and # of players.

if it was me i would create somthing with udplink and a python script that checks the server every minute or somthing, but you would have the same issue. to avoid the mp switch issue you would have to create a timeout , check multiple cycles. fail more then once, or programaticly acknoledge the map switch , but then you could very well hang during switch anyway.....

[x21]

@Bob
I understand what you mean by the server going down during a map switch, however this is usually quite short and could be handled by multiple checks rather than just a single check. This should allow it to detect a hang quite easily.

guess it could work to use a masterserver entry as a health beacon

Can this be implemented without using large amounts of memory?

[.:..:]

Easiest solution would be to send a simple ping query to the server uplink (using udp connection).

Send line "\echo\0\final\", and server responds with same line.

[x21]

Interestingly the output is not the same as the input I get back "\echo\0\queryid\49.1"
when i send "\echo\0\final\"

[x21]

I suppose I could try a simple solution like this

Code: Select all

ECHO \echo\0\final\|NC -u -w 1 %SVRIP% %SVRPORT%|FIND "\echo\0\queryid\"> NUL & IF ERRORLEVEL 1 (
    GOTO BAD
) ELSE (
    GOTO GOOD
)

[BobIsUnreal]

Here is a simple implementation of the upd/python idea  curiously of Casey.

Ini file:
Serveractors=watchdog.watchdog




echo only udplink src.
---------------------------------------------------------------

[code]class watchdog extends udplink
        transient;

function PostBeginPlay()
{
        linkmode = MODE_TEXT;
        bindport(1888,false);
}

event ReceivedText( IpAddr Addr, string Text )
{
        SendText(Addr,Text); //Just echo it back.
}[/code]


Python script. Meant to work only on linux.

Code: Select all

#!/usr/bin/env python

import socket
import time
import psutil #sudo apt-get install python-psutil

PROCNAME = "UCCLinux.bin"
IP = "127.0.0.1"

PORT = 1888


sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.settimeout(30)
sock.bind((IP,PORT-1))

counter = 0

while True:
        sock.sendto(":|",(IP,PORT))
        try:
                data, addr = sock.recvfrom(1024)
                counter = 0
                time.sleep(1)
        except socket.timeout:
                counter = counter + 1
                print "Counter = " + str(counter)
                if counter >= 10:
                        print "Ok attempting to kill..."
                        for proc in psutil.process_iter():
                                if proc.name( == PROCNAME:
                                        print proc.name( + "getting killed!!!"
                                        proc.kill()
                        counter = 0

this will check every 30 seconds 10 times for a reply , after the last attempt with no response, it will kill the proccess.

time for me to maybe go shoehorn this into the chatlinkbot so it messages the teamspeak everytime the server hangs.

[x21]

This is great. I ended up using the server query and searching for a string in the reply. The Python solution should work well for Windows too with some modifications. This script plus NSSM would almost certainly keep a server running.

[medor]

ServerWatch is just excellent ... it work with unreal and ut
http://unrealtournament.99.free.fr/utfi ... h3.5.8.rar

You can set sounds for server down
sounds joint players leave players
Set your @ for advertise server down
Use the web admin with it

[x21]

ServerWatch is just excellent ... it work with unreal and ut
http://unrealtournament.99.free.fr/utfi ... h3.5.8.rar

You can set sounds for server down
sounds joint players leave players
Set your @ for advertise server down
Use the web admin with it

Does this query the server?

EDIT: I had a look at serverwatch and it apears to be useless for my purpose.

[medor]

Yes it do that
May be you not set it right
for the port server if you ave 7777 You must put 7778

[x21]

Yes it do that
May be you not set it right
for the port server if you ave 7777 You must put 7778

I was looking for a solution that automatically recovers from a hang. I ended up writing a script to manage this.

Code: Select all

================================================================
SERVER NOT RESPONDING TO QUERY at 16:42:55.06 - Tue 06/20/2017
RESTARTING SERVER 
----------------------------------------------------------------
SERVER RESTARTED AT 16:43:02.74 - Tue 06/20/2017 
================================================================

As you can see it detects a server hang based on failed queries and automatially restarts the server. This way I don't have to worry about the server as much. I know it's running 

[medor]

Oh ok

i ave some script here but not test http://medor.no-ip.org/index.php?dir=En ... r_scripts/

[x21]

Python script. Meant to work only on linux.

Code: Select all

#!/usr/bin/env python

import socket
import time
import psutil #sudo apt-get install python-psutil

PROCNAME = "UCCLinux.bin"
IP = "127.0.0.1"

PORT = 1888


sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.settimeout(30)
sock.bind((IP,PORT-1))

counter = 0

while True:
        sock.sendto(":|",(IP,PORT))
        try:
                data, addr = sock.recvfrom(1024)
                counter = 0
                time.sleep(1)
        except socket.timeout:
                counter = counter + 1
                print "Counter = " + str(counter)
                if counter >= 10:
                        print "Ok attempting to kill..."
                        for proc in psutil.process_iter():
                                if proc.name( == PROCNAME:
                                        print proc.name( + "getting killed!!!"
                                        proc.kill()
                        counter = 0

I was thinking of doing something like this using win32serviceutil in Windows and this tool called nssm that allows ucc to run as a windows service.

https://nssm.cc/

=================================================

Oh ok

i ave some script here but not test http://medor.no-ip.org/index.php?dir=En ... r_scripts/

These are all linux scripts except for the telnet server.

[x21]

[code]class watchdog extends udplink
        transient;

function PostBeginPlay()
{
        linkmode = MODE_TEXT;
        bindport(1888,false);
}

event ReceivedText( IpAddr Addr, string Text )
{
        SendText(Addr,Text); //Just echo it back.
}[/code]

Do you think this can be ran using TCPLink instead of UDPLink? If so this can be implemented quite easily with something like this https://www.microsoft.com/en-us/downloa ... x?id=17148

[[]KAOS[]Casey]

Late, but there is no specific reason why it TCP link couldn't be used.

[x21]

Is there a way to abuse the local servers server query?