ssh_tunnel statistics: napoleon to zippy

Time: 2018-Feb-24 00:10:09 Sat

Current tunnel state: disabled          Current tunnel condition: DOWN

loader stats

------ loader (pid: 30176 ) ------------ (v3.26 2006/04/15) ----- State: disabled DOWN
Time:                2018-Jan-25 22:50:04 Thu
client:              napoleon
server:              zippy
client heartbeat:    Client Heartbeat
server heartbeat:    Server Heartbeat
server heartbeat 1:  Server Heartbeat 1
server heartbeat 2:  Server Heartbeat 2
server heartbeat 3:  Server Heartbeat 3
server heartbeat 4:  Server Heartbeat 4
Sleep time:          238
loader loop counter: 0
bad port tests:      0     last: none yet
failed file updates: 0     last: none yet
tunnels killed:      3     last: 2018-Jan-25 22:50:04 Thu
emails sent:         0     last: none yet
email addresses:     john@larsen-family.us
loader started:      YYYY-MMM-DD HH:MM:SS DOW
debug flags:         0x0

tunnel stats

------ tunnel (pid: 7728 ) ------------ (v3.26 2006/04/15) ----- State: enabled UP
Time:                2017-Aug-04 21:09:43 Fri
client:              napoleon
server:              zippy
client heartbeat:    Fri Aug 4 21:09:39 MDT 2017
server heartbeat:    Fri Aug 4 23:09:39 EDT 2017
Sleep time:          238
tunnel loop counter: 36
bad port tests:      0     last: none yet
emails sent:         1     last: 2017-Aug-04 18:50:47 Fri
email addresses:     john@larsen-family.us
tunnel started:      2017-Aug-04 18:50:49 Fri
debug flags:         0x0

Current users, uptime, and load average

 00:10:09 up 14 days,  7:31,  0 users,  load average: 0.00, 0.00, 0.00
USER     TTY        LOGIN@   IDLE   JCPU   PCPU WHAT

Current processes

UID        PID  PPID  C STIME TTY          TIME CMD
cyg_ser+  6632  5636  0 Feb09 ?        00:00:00 /usr/sbin/sshd -D
jlarsen   6640  5716  0 Feb09 ?        00:00:03 /usr/sbin/cron -n
cyg_ser+  5636     1  0 Feb09 ?        00:00:00 /usr/bin/cygrunsrv
jlarsen  28340 25624  0 00:10 ?        00:00:00 /usr/bin/procps -edf
jlarsen   5716     1  0 Feb09 ?        00:00:00 /usr/bin/cygrunsrv
jlarsen  25624     1  0 00:10 ?        00:00:00 /usr/bin/perl -w /home/jlarsen/ssh_tunnel/zippy/ssh_tunnel.pl
jlarsen  42760 18236  3 00:10 ?        00:00:00 
jlarsen  18236  6640  0 00:10 ?        00:00:00 /usr/sbin/cron -n

Current netstat


Active Connections

  Proto  Local Address          Foreign Address        State
  TCP    192.168.1.4:53526      243:4070               ESTABLISHED
  TCP    192.168.1.4:53565      13.89.185.175:https    ESTABLISHED
  TCP    192.168.1.4:53592      pf-in-f188:5228        ESTABLISHED
  TCP    192.168.1.4:53597      media-router-brb1:https  ESTABLISHED
  TCP    192.168.1.4:53612      e2:https               ESTABLISHED
  TCP    192.168.1.4:53622      t1:https               ESTABLISHED
  TCP    192.168.1.4:53653      a23-210-98-44:https    ESTABLISHED
  TCP    192.168.1.4:53654      e2:https               ESTABLISHED
  TCP    192.168.1.4:53657      130:https              TIME_WAIT
  TCP    192.168.1.4:53688      pr-bh:https            ESTABLISHED
  TCP    192.168.1.4:53689      162.248.16.26:https    ESTABLISHED
  TCP    192.168.1.4:53698      server-52-84-237-225:https  ESTABLISHED
  TCP    192.168.1.4:53699      a23-213-146-133:https  ESTABLISHED
  TCP    192.168.1.4:53702      a96-17-216-167:https   ESTABLISHED
  TCP    192.168.1.4:53712      ec2-54-183-5-177:https  ESTABLISHED
  TCP    192.168.1.4:53724      162.248.16.50:https    TIME_WAIT
  TCP    192.168.1.4:53726      104.19.194.102:https   ESTABLISHED
  TCP    192.168.1.4:53749      server-52-84-237-41:https  ESTABLISHED
  TCP    192.168.1.4:53791      68.67.129.190:https    ESTABLISHED
  TCP    192.168.1.4:53795      151.101.198.49:https   ESTABLISHED
  TCP    192.168.1.4:53811      a104-95-181-180:https  ESTABLISHED
  TCP    192.168.1.4:53812      151.101.198.49:https   ESTABLISHED
  TCP    192.168.1.4:53818      r-199-59-148-84:https  ESTABLISHED
  TCP    192.168.1.4:53824      104.16.14.243:https    ESTABLISHED
  TCP    192.168.1.4:53840      151.101.198.49:https   ESTABLISHED
  TCP    192.168.1.4:53850      a104-95-210-201:https  ESTABLISHED
  TCP    192.168.1.4:53854      204.144.140.22:https   ESTABLISHED
  TCP    192.168.1.4:53862      204.144.141.26:https   ESTABLISHED
  TCP    192.168.1.4:53864      199.187.193.1:https    ESTABLISHED
  TCP    192.168.1.4:53867      151.101.198.49:http    ESTABLISHED
  TCP    192.168.1.4:53868      laxadvip2:https        ESTABLISHED
  TCP    192.168.1.4:53870      a104-95-212-135:https  ESTABLISHED
  TCP    192.168.1.4:53875      8.39.36.143:https      ESTABLISHED
  TCP    192.168.1.4:53971      ec2-52-20-152-198:https  ESTABLISHED
  TCP    192.168.1.4:53979      ec2-34-207-45-197:https  ESTABLISHED
  TCP    192.168.1.4:54001      pr-west:https          ESTABLISHED
  TCP    192.168.1.4:54177      a104-95-221-151:https  ESTABLISHED
  TCP    192.168.1.4:54178      a104-95-221-151:https  ESTABLISHED
  TCP    192.168.1.4:54182      a104-95-219-158:https  ESTABLISHED
  TCP    192.168.1.4:54185      a104-95-221-151:https  ESTABLISHED
  TCP    192.168.1.4:54227      s-prd-req-adcom-scd-blue-b:https  ESTABLISHED
  TCP    192.168.1.4:54228      a104-95-214-212:https  ESTABLISHED
  TCP    192.168.1.4:54244      s-prd-umpxl-adcom-scd-blue-b:https  ESTABLISHED
  TCP    192.168.1.4:54245      s-prd-umpxl-adcom-scd-a:https  ESTABLISHED
  TCP    192.168.1.4:54246      ec2-34-199-214-243:https  ESTABLISHED
  TCP    192.168.1.4:54247      s-prd-umpxl-adcom-scd-a:https  ESTABLISHED
  TCP    192.168.1.4:54309      sea15s11-in-f10:https  ESTABLISHED
  TCP    192.168.1.4:54318      a104-95-196-214:https  ESTABLISHED
  TCP    192.168.1.4:54354      151.101.198.2:https    ESTABLISHED
  TCP    192.168.1.4:54355      a104-95-199-80:https   ESTABLISHED
  TCP    192.168.1.4:54356      a104-95-199-80:https   ESTABLISHED
  TCP    192.168.1.4:54372      64.4.54.254:https      TIME_WAIT
  TCP    192.168.1.4:54375      a23-210-119-94:https   ESTABLISHED
  TCP    192.168.1.4:54378      64.4.54.254:https      TIME_WAIT
  TCP    192.168.1.4:54401      a23-209-176-13:https   ESTABLISHED
  TCP    192.168.1.4:54416      a96-17-216-167:https   ESTABLISHED
  TCP    192.168.1.4:54417      a96-17-216-167:https   ESTABLISHED
  TCP    192.168.1.4:54422      64.4.54.254:https      TIME_WAIT
  TCP    192.168.1.4:54426      64.4.54.254:https      TIME_WAIT
  TCP    192.168.1.4:54467      64.4.54.254:https      TIME_WAIT
  TCP    192.168.1.4:54496      64.4.54.254:https      TIME_WAIT
  TCP    192.168.1.4:54497      t1:https               ESTABLISHED
  TCP    192.168.1.4:54583      l3dsr-cserv-um-20:https  TIME_WAIT
  TCP    192.168.1.4:54584      l3dsr-cserv-um-20:https  TIME_WAIT
  TCP    192.168.1.4:54608      a23-209-176-13:https   ESTABLISHED
  TCP    192.168.1.4:54609      192.229.173.67:https   ESTABLISHED
  TCP    192.168.1.4:54610      ec2-52-205-55-68:https  ESTABLISHED
  TCP    192.168.1.4:54613      t2:https               ESTABLISHED
  TCP    192.168.1.4:54634      s-prd-umpxl-adcom-scd-blue-b:https  TIME_WAIT
  TCP    192.168.1.4:54635      s-prd-umpxl-adcom-scd-blue-b:https  TIME_WAIT
  TCP    192.168.1.4:54642      192.168.1.2:8080       TIME_WAIT
  TCP    192.168.1.4:54643      192.168.1.2:8080       TIME_WAIT
  TCP    192.168.1.4:54645      l3dsr-cserv-um-20:https  TIME_WAIT
  TCP    192.168.1.4:54646      l3dsr-cserv-um-20:https  TIME_WAIT
  TCP    192.168.1.4:54649      mpr1:https             ESTABLISHED
  TCP    192.168.1.4:54654      mpr1:https             ESTABLISHED
  TCP    192.168.1.4:54657      162.248.16.24:https    TIME_WAIT
  TCP    192.168.1.4:54659      192.168.1.2:8080       TIME_WAIT
  TCP    192.168.1.4:54660      192.168.1.2:8080       TIME_WAIT
  TCP    192.168.1.4:54666      ec2-52-52-164-145:https  ESTABLISHED
  TCP    192.168.1.4:54667      a104-95-196-214:https  ESTABLISHED
  TCP    192.168.1.4:54668      a104-95-196-214:https  ESTABLISHED
  TCP    192.168.1.4:54669      a104-95-196-214:https  ESTABLISHED
  TCP    192.168.1.4:54672      gw-vlan105:https       ESTABLISHED
  TCP    192.168.1.4:54673      gw-vlan105:https       ESTABLISHED
  TCP    192.168.1.4:54674      gw-vlan105:https       ESTABLISHED
  TCP    192.168.1.4:54675      66.6.33.149:https      ESTABLISHED
  TCP    192.168.1.4:54691      a23-209-177-64:https   ESTABLISHED
  TCP    192.168.1.4:54692      ec2-52-88-185-201:https  ESTABLISHED
  TCP    192.168.1.4:54693      ec2-54-245-121-55:https  ESTABLISHED
  TCP    192.168.1.4:54694      ec2-52-33-194-200:https  ESTABLISHED
  TCP    192.168.1.4:54704      ec2-54-241-161-164:https  ESTABLISHED
  TCP    192.168.1.4:54721      a23-210-119-94:https   ESTABLISHED
  TCP    192.168.1.4:54724      ec2-54-214-243-189:https  ESTABLISHED
  TCP    192.168.1.4:54725      ec2-35-160-154-114:https  ESTABLISHED
  TCP    192.168.1.4:54726      ec2-35-160-154-114:https  ESTABLISHED
  TCP    192.168.1.4:54727      ec2-35-160-154-114:https  ESTABLISHED
  TCP    192.168.1.4:54728      a23-210-119-94:https   ESTABLISHED
  TCP    192.168.1.4:54729      a23-210-119-94:https   ESTABLISHED
  TCP    192.168.1.4:54730      a23-210-119-94:https   ESTABLISHED
  TCP    192.168.1.4:54731      ec2-34-211-132-194:https  ESTABLISHED
  TCP    192.168.1.4:54732      ec2-34-211-132-194:https  ESTABLISHED
  TCP    192.168.1.4:54733      ec2-34-211-132-194:https  ESTABLISHED
  TCP    192.168.1.4:54734      ec2-54-245-121-55:https  ESTABLISHED
  TCP    192.168.1.4:54735      ec2-54-245-121-55:https  ESTABLISHED
  TCP    192.168.1.4:54736      ec2-54-214-243-189:https  ESTABLISHED
  TCP    192.168.1.4:54737      ec2-54-214-243-189:https  ESTABLISHED
  TCP    192.168.1.4:54739      ec2-34-211-132-194:https  ESTABLISHED
  TCP    192.168.1.4:54740      ec2-34-211-132-194:https  ESTABLISHED
  TCP    192.168.1.4:54741      ec2-34-211-132-194:https  ESTABLISHED
  TCP    192.168.1.4:54742      ec2-52-33-194-200:https  ESTABLISHED
  TCP    192.168.1.4:54743      ec2-52-33-194-200:https  ESTABLISHED
  TCP    192.168.1.4:54744      mpr1:https             ESTABLISHED
  TCP    192.168.1.4:54745      mpr1:https             ESTABLISHED
  TCP    192.168.1.4:54746      mpr1:https             ESTABLISHED
  TCP    192.168.1.4:54747      mpr1:https             ESTABLISHED
  TCP    192.168.1.4:54748      ox-173-241-250-143:https  CLOSE_WAIT
  TCP    192.168.1.4:54749      l3dsr-cserv-um-20:https  CLOSE_WAIT
  TCP    192.168.1.4:54751      173-14-166-5-NewEngland:ssh  TIME_WAIT

crontab file

# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.5aDY8Lykbe installed on Fri Aug  4 21:05:41 2017)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
MAILTO=john@larsen-family.us
#* * * * * echo "`date`: This is an email test"; echo `date` > /tmp/crontest.$LOGNAME.txt

# For now do diskusage by itself until restore of nashua files is completed
#0 15,21 * * * /home/jlarsen/diskusage/diskusage -n 50 -e john@larsen-family.us

## This script restores files from zippy back to napoleon
#10,40 * * * * /home/jlarsen/napoleon_restore/napoleon_restore.sh /c/napoleon_nashua
#34 22 * * * /home/jlarsen/napoleon_restore/napoleon_restore.sh /c/napoleon_nashua

# Check if the ssh_tunnel is up and running
0,10,20,30,40,50 * * * * /home/jlarsen/ssh_tunnel/zippy/ssh_tunnel.pl

# Perform the nightly backup --------------------------------------
2 17,21 * * * /home/jlarsen/backup/backup.cron
7 21 * * * /home/jlarsen/backup/backup.cron

# UPDATE LOCATE DATABASE ---------------------------------------
02 15,18,21 * * * /home/jlarsen/bin/updatedb.cron

loader.conf file

# Filename:  loader.conf
# Description:  This is the configuration file for the loader portion of the 
# ssh_tunnel program.  The contents of this file never change.  This insures
# that the loader will always operate.  It is important not to change the 
# formatting of this file because it is read by the ssh_tunnel and processed 
# using perl which is expecting things to be in certain locations.
#
# loader.conf has two sections:
# 1. Configuration information such as the names of the ssh and sshd
#    hosts, the working directories on each host, and other data as needed.
# 2. The ssh host configuration for the loader program is in this file.
#    That configuration should never change.  It needs to be at the end
#    of the file because ssh uses everything between "Host" entries as
#    configuration.  There is only one Host section in this file.
#
########################################################################
# The following section contains host information 
# SSH_CLIENT      napoleon
# SSH_CLIENT_DIR  /home/jlarsen/ssh_tunnel/zippy
# SSH_SERVER      zippy
# SSH_SERVER_DIR  /home/tunnels/ssh_tunnel/napoleon
#
########################################################################
# The following section is the ssh config file for the loader program.
# IMPORTANT!  The path on the client MUST have the SSH_SERVER name in
# it.  The script uses this name is search strings and it MUST be there.
Host loader
HostName sml.dnsalias.org
Port = 22
UserKnownHostsFile = /home/jlarsen/ssh_tunnel/zippy/known_hosts
User = tunnels
IdentityFile = /home/jlarsen/ssh_tunnel/zippy/id_rsa.rlarsen

tunnel.conf file

# Filename:  tunnel.conf
#
# Description:  This is the configuration file for the tunnel portion of the 
# ssh_tunnel program.  The contents of this file can be changed during program execution.
# It is important not to change the formatting of the file because it is read by the
# tunnel program and processed using awk which is expecting things to be in certain
# locations.
#
# tunnel.conf has three sections:
# 1. Configuration information such as email addresses and thresholds.
# 2. The ssh host configuration for the tunnel.  This consists of two
#    host definitions, tunnel and heartbeat.  The tunnel is the permanent
#    connection and defines port forwarding.  The heartbeat uses one of 
#    the forwarded ports to write and copy heartbeat files between the
#    ssh client and the ssh server.  Changing heartbeats indicate that the
#    tunnel is functioning.
# 3. The webpage configuration section as described below.  This has the
#    ssh host configuration for updating a webpage on a webserver.
#
################################################################################
# The following section contains configuration information
#
# The tunnel is either "enabled" or "disabled" based on the value given
# below.  If disabled, then no processes are running on either the
# client or the server.  A cron job on the client runs ssh_tunnel.pl periodically
# to check if tunnel.conf has changed on the server.  If a change is detected
# then the new tunnel.conf file is transfered over.  If the tunnel state
# becomes "enabled" then the tunnel is activated.
# TUNNEL_STATE disabled
#
# The email addresses below receive diagnostic messages.  Separate
# multiple addresses with commas and no white space.  The word "none"
# turns off email sending and is the default.  The same email address
# is used by cron, loader, tunnel, and pulse.
# EMAIL_ADDRESS     john@larsen-family.us
#
# The threshold defined below is how many port failures are required
# before an email is sent.  Set this to "none" to turn this off.  If
# the ssh_server's ssh port isn't accessible then the loader kills 
# the tunnel.  This periodic email serves as a reminder that the tunnel
# is down.
# EMAIL_THRESHOLD   50
#
# The loader, tunnel, and pulse programs all have the same sleep value.
# The sleep time can be changed here.  It should be in the range of
# 60 to 3600 seconds.  Shorter sleep times increase system loading.
# The sleep time must be less than half the crontab time for ssh_tunnel.pl.
# SLEEP_TIME        238
#
#
################################################################################
# The following section is the ssh config file for the tunnel program.
# IMPORTANT!  The path on the client MUST have the SSH_SERVER name in
# it.  The script uses this name is search strings and it MUST be there.
#
# Important info about port numbers used in the "ssh_tunnel" and "heartbeat"
# sections below.  The "heartbeat" Port number must be the same as the last
# two port numbers in the "ssh_tunnel" section.  In this example 50022 is
# used.  The port number used must be unused by anything else on the two
# machines.  If you have multiple tunnels connecting to a single ssh_server
# be sure to use different port numbers.  
#
# Another requirement is that the ssh_server must be configured to
# allow remote port forwarding.  This is normally off by default in the 
# ssh_server's sshd_config file.  Set "GatewayPorts yes" in sshd_config.
#
Host tunnel
HostName sml.dnsalias.org
Port = 22
UserKnownHostsFile = /home/jlarsen/ssh_tunnel/zippy/known_hosts
User = tunnels
IdentityFile = /home/jlarsen/ssh_tunnel/zippy/id_rsa.rlarsen
Compression = yes
RemoteForward = 13999 localhost:5900
RemoteForward = 13222 localhost:22
RemoteForward = 13022 localhost:22
LocalForward = 13022 localhost:22

################################################################################
Host heartbeat
HostName localhost
Port = 13022
UserKnownHostsFile = /home/jlarsen/ssh_tunnel/zippy/known_hosts
User = tunnels
IdentityFile = /home/jlarsen/ssh_tunnel/zippy/id_rsa.rlarsen

################################################################################
Host pulse
HostName localhost
Port = 13022
UserKnownHostsFile = /home/tunnels/ssh_tunnel/napoleon/known_hosts
User = jlarsen
IdentityFile = /home/tunnels/ssh_tunnel/napoleon/id_rsa.rlarsen

################################################################################
# Note: This section must be last in tunnel.conf.
# A webpage will be copied to the directory set by WEBSERVER_DIR below.  The
# default value "none" turns this feature off.  Change "none" to the directory
# path where the webpage should be copied.  The "webpage" host information 
# below must also be changed to valid values to make this work.
# WEBSERVER_DIR     /home/tunnels/ssh_tunnel_html
#
# The filename of the webpage copied to the webserver defaults to
# SSH_CLIENT_to_SSH_SERVER.html.  To choose a different name replace the
# word "default" on the line below with the desired name.
# WEBPAGE_FILENAME  default
#
# A copy of the webpage can be put in the directory define by WEBPAGE_LOCAL_DIR.
# Change "none" to a valid directory to enable this feature.
# WEBPAGE_LOCAL_DIR  none
#
# The webpage update rate defaults to once per sleep time. Change the value below
# to select a different rate.  The update rate is the number of times through the
# loader while loop before updating the webpage.  This value is ignored when events
# occur such as the tunnel going down or during startup.  The webpage is updated
# for all non normal runtime events.
# WEBPAGE_UPDATE_RATE   1
#
Host webpage
HostName sml.dnsalias.org
Port = 22
UserKnownHostsFile = /home/jlarsen/ssh_tunnel/zippy/known_hosts
User = tunnels
IdentityFile = /home/jlarsen/ssh_tunnel/zippy/id_rsa.rlarsen