Get Social

Script for domain names monitoring

Script for domain names monitoring image

Often, who have multiple sites, think about – how to control when a domain name renewal term will expire. Especially if domain names are served by different registrars. So it was in the companies where I worked: one and a half to two dozen domain names were always was used. I wanted to find out about domain expiration and pay it on time.

I’ll share the script on Bash, which I sketched for this. In fact, it’s a parser.
What he does, you can understand from the comments, but I will describe a little more:

So the task: the script, runs on a schedule; it check domains; sends email report.

[1] Where do we get data from?

This script uses whois command results. Sometimes console query: “whois some kind of domain” does not give the expected result. This is due to the fact that the whois command does not poll suitable servers that contain complete information. Therefore, we can add such servers to the configuration file whois.conf

[2] How it works?

1. We see an array of domain names that need to be checked (fill it by yourself).
2. Next – create a temporary file, which is beautifully filled with a parser
3. File contents are sent to the mail and the file is deleted.

[3] Source?

Here it is:

#!/bin/bash

## Script for monitoring domain names -> checking domain expiration date
## url: http://rocknroot.pp.ua/en/blog.script-for-domain-names-monitoring.html


# DON'T forget add to /etc/whois.conf:
# \.ua$ whois.com.ua
# \.by$ whois.cctld.by
#(if needed)
# Or place somewhere your whois.conf and specify a parameter to the whois command

export LC_ALL=en_US.utf8

# domain names array for polling
ARR_DOMAIN+=(
yahoo.com
google.com
)

# If there is no possibility to edit whois.conf - then you can try to enter the following
# value in the bash array: '-h whois.cctld.by tut.by' (in quotation marks)

# Create a file for temporary data
echo "" > tmp_file_domain_renew

# Writes table header there 
printf "%-35s\t %-25s\t %-25s\n" "Domain     " "< 50 days" "> 50 days" >> tmp_file_domain_renew
printf "%-35s\t %-25s\t %-25s\n" "---------------" "----------" "----------" >> tmp_file_domain_renew

# Parses information about the validity period of the domain
for DOMAIN_ in "${ARR_DOMAIN[@]}"
 do
 sleep 0.2
 dateexp_=`whois $DOMAIN_ | \
 egrep -i 'Expiration Date:|paid-till:|OK-UNTIL|Domain Currently Expires:|Record expires on|expires:|Registry Expiry Date:'| \
 head -n 1 | \
 sed 's/\(OK-UNTIL\) \(.\{,8\}\).*/\2/i' | \
 sed 's/.*Currently //' | \
 sed 's/.*Expiration Date:/ExpDate: /' | \
 sed 's/.*Registry Expiry Date:/RegExpDate: /' | \
 sed 's/T[0-9][0-9]\:.*//'| \
 awk '{$1=""; print $0}' | \
 sed 's/^\ //' | \
 sed 's/\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\)/\1\2\3/i'`

# Calculate how many days remain from the current date
dateexp_s=`date -d "${dateexp_}" +"%s"`
datenow_s=`date +"%s"`
diff_s=`expr $dateexp_s - $datenow_s`
diff_d=`expr $diff_s / 3600 / 24`

# Fill in the columns of the table, depending on whether
# More or less, in this case, 50 days
if [[ $diff_d -ge 50 ]];
then
  printf "%-35s\t %-25s\t %-25s\n" "$DOMAIN_"  "-" "$diff_d" >> tmp_file_domain_renew
else
  printf "%-35s\t %-25s\t %-25s\n" "$DOMAIN_" "$diff_d" "-" >> tmp_file_domain_renew
fi

done

# send to email
mail -s DomainRenewCheck yourmail@example.com < tmp_file_domain_renew

# cleaning
rm tmp_file_domain_renew

[4] Checking.

Do not forget to make executable with the command:

chmod +x domain_renew.sh

The result should be something like this (in your mailbox):

Domain                          < 50 days               > 50 days                
—————                  ———-              ———-            
yahoo.com                               –                         320                      
google.com                             16                        –                           

[5] How to add schedule task?

Run “crontab -e” in the console and prescribe script path and time when it needs to run.
That’s all. Now it’s a little easier to track when the domain expires. 🙂

Installing Debian Wheezy on an SSD

So, what do we have:

Intel i5-3470
16 Gb DDR3-1600Mhz
60Gb Kingston SSD
1500Gb Samsung Hard Disk

[1] Analysis and preparation

Google search gives the first 3 links to wiki.debian.org
That’s what we need.
The article “SSD Installation” tells that the use of SSD will increase the speed of the system by 200%. This is good news. However, it also shows that there is a problem – a limited number of rewriting cycles.Further explains how to transfer an existing installation to a solid-state drive – this is not what we need, since we will put a clean system on it. Therefore, proceed to the article “SSDOptimization”. Here we get the following recommendations:
– use the kernel version of at least 3.2 (in Debian 7 is just 3.2)
– use the ext4 file system (and select it during installation)

[2] Installing a clean system

At this preparatory stage is over. Now, in any convenient way, we run the Debian Wheezy installation. Having preliminarily read various articles about which I will go further, I decided not to change anything particularly in how logical disks are partitioning, and all the necessary transformations I will make after the installation. Moreover, it would take a lot of time – the main place on the 1.5TB drive is the /home partition on the xfs file system, which does not support the reduction. That is, it would be necessary to transfer more than 1 TB of data somewhere, and then copy again … I will put the operation system on an SSD drive, and then I will make symbolic links for such directories as /var and others that I deem it necessary to place next to /home on the existing HDD.
I will not even install the swap partition.

[3] Setting up and optimizing

So, the system is installed. Further on the links to other articles we find still valuable recommendations.
Thanks https://wiki.archlinux.org/index.php/SSD, http://help.ubuntu.ru/wiki/ssd, and other articles, we can see that the partitions and file systems on the SSD need to be customized. What came in handy for me personally:

-> Let’s check the TRIM support (http://ru.wikipedia.org/wiki/TRIM) (technology that allows to distribute load on SSD)

hdparm -I /dev/sda | grep TRIM
        *    Data Set Management TRIM supported (limit 1 block)
        *    Deterministic read data after TRIM

-> Boot from the LiveCD / LiveUSB and transfer the / var partition from the root located on the SSD to the HDD (the one that is mounted in the home). Then make a symbolic link from /home/var to /var.
This will significantly reduce the number of write cycles on the SSD, since this directory is actively used by operationg system for recording (log files and others…).

-> We also edit /etc/fstab – add the “discard” option (includes use of TRIM) for the root file system, “noatime” (turn off file access time), “nodiratime”

-> I decided to use one more tip – apt cache put to RAM
To do this, add the following line in /etc/fstab:

tmpfs   /var/cache/apt/archives tmpfs   defaults        0       0

In this case, the apt cache will not be stored on harddisk. For reinstalling packages, its can always be downloaded from the network. But it will now be possible to speed up packages installation in a clean system due to the large volume of RAM.

[4] Further analysis

Install utility iotop:

aptitude install iotop

Now you can see which processes are most actively use disk for writing:

iotop -oPa

After that, perhaps, you can think about what other folders can be transfered from application profiles to hdd.

p.s. This configuration has been working since January 2014. There are no problems 🙂

Installing proxmox on debian wheezy

I’ve heard about this system of virtualization earlier (http://en.wikipedia.org/wiki/Proxmox)…

This is a system using KVM and OpenVZ as hypervisors and having a convenient web interface for management. So, we have a pumped-up home computer with Debian Wheezy 7.3 (amd64) on board. When I finally decided to install it, it turned out that now, with the release of the new version of Proxmox 3.1, the policy of granting access to repositories has changed.

Here (https://pve.proxmox.com/wiki/Package_repositories) we can find descriptions of the repositories. I see that Proxmox VE Enterprise Repository is not suitable – it’s paid subscription.

Official guide (http://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Wheezy) strongly recommends Proxmox VE No-Subscription Repository, but description of this repository scares me: “The pve-no-subscription repo can be used for testing and non-production use. Its not recommended to run on production servers as these packages are not always heavily tested and validated. As the name suggests, there is no Proxmox VE Subscription Key needed to access this repository.

What to do? I decided to try to put version 3.0 – a stable previous release.

Add a line to the package manager:

echo "deb http://download.proxmox.com/debian wheezy pve" >> /etc/apt/sources.list

Further, we will do, as described in the documentation. Put the repository key:

wget -O- "http://download.proxmox.com/debian/key.asc" | apt-key add -

Then update packages list and install currently available updates (I’m using aptitude):

aptitude update && aptitude safe-upgrade

Install kernel:

aptitude install pve-firmware pve-kernel-2.6.32-26-pve

At the same time, the system shown error message::

The following packages have unsatisfied dependencies:
 pve-firmware : Conflicts: firmware-linux-free but 3.2 installed
                Conflicts: firmware-linux-free:i386 which is a virtual package.
                Conflicts: firmware-realtek but 0.36+wheezy.1 installed
                Conflicts: firmware-realtek:i386 which is a virtual package.
The following actions will allow dependencies:

     Remove next packages:                                          
1)     firmware-linux-free                                              
2)     firmware-realtek                                                 

     Leave the following dependencies unresolved:                     
3)     linux-image-3.2.0-4-amd64 recommends firmware-linux-free (>= 3~)


Will you accept this decision? [Y/n/q/?]

I took this decision, so proxmox developers propose to remove the kernel linux-image-3.2.0-4-amd64
After checking after rebooting, that everything works with kernel pve-kernel-2.6.32-26-pve, safely delete the default kernel:

aptitude remove linux-image-amd64 linux-image-3.2.0-4-amd64

We check the bootloader configuration:

update-grub

Now we install Proxmox VE packages:

aptitude install proxmox-ve-2.6.32 ntp ssh lvm2 postfix ksm-control-daemon vzprocps open-iscsi bootlogd

Enter https://your-hostname:8006/ and rejoice!

Errors when installing Proxmox on Debian

[1] Proxmox: initscript rrdcached, action “start” failed

The installation of proxmox-ve-2.6.32 and its dependencies was interrupted by a specific error:

Configures package rrdcached (1.4.7-2) …
Starting RRDtool data caching daemon: rrdcachedBase directory (-b) resolved via file system links!
Please consult rrdcached '-b' documentation!
Consider specifying the real directory (/media/all/var/lib/rrdcached/db)
invoke-rc.d: initscript rrdcached, action "start" failed.
dpkg: Error while processing parameter rrdcached (--configure):
 Subprocess installed post-installation script returned error code 5
dpkg: Dependencies do not allow to configure package pve-cluster:
 pve-cluster depends on rrdcached, hovewer:
  Package rrdcached not yet configured.

This was due to the fact that I made /var directory as symbolic link from the directory located on same hard drive, while the root of the system is on the SSD. I could, of course, mount it, but I was too lazy to create a separate partition for /var, so I used the same partition as /home. You should not have such a problem.

Overcame it so:

1) Determine where the configuration is stored

dpkg -L rrdcached

...
/etc/default/rrdcached
...

2) Open file and uncomment the parameter, specifying folder actual location without symbolic links:

nano /etc/default/rrdcached

...
OPTS="-b /media/all/var/lib/rrdcached/db"
...

[2] Proxmox: Unable to get local IP address

Configures package pve-cluster (3.0-8) …
[....] Starting pve cluster filesystem : pve-cluster[main] crit: Unable to get local IP address
 (warning).
invoke-rc.d: initscript pve-cluster, action "start" failed.

How I overcame it (rocknroothost -this is my hostname, substitute it with your one here):

cp -av /etc/hosts /etc/hosts.original

Then we edit /etc/hosts for next view:

---
127.0.0.1 localhost.localdomain localhost
192.168.99.99 rocknroothost.pp.ua rocknroothost pvelocalhost
---

Also /etc/hostname for such view:

---
rocknroothost
---

after that we need execute:

apt-get install -f

And installation process will continue

/etc/init.d/hostname.sh start

[3] Proxmox: failed to load

Finally, having worn out the installation, I enter the browser:
http://rocknroothost:8006/ => get error: failed to load, and I check service

lsof -i tcp:8006

I break my head, google … but it’s just ridiculous – I had to watch carefully:
https , no http 🙂