Category Archives: Web Hosting

Amazon AWS EC2 Automatic Snapshot Daily Cron Job

There seems to be very poor documentation about setting up an automatic backup, daily snapshot with Amazon AWS EC2. I am currently running Ubuntu 14.04 with ISPConfig. Here is how to setup a daily cron job for backup.

First, you must get your Access Key ID and Secret Access Key from Amazon. There is plenty of documentation about this. You may want to setup an IAM user so this access key has limited permissions, as Amazon recommends. I’ve linked to the Amazon support page if you are not familiar with this. This is so that Amazon knows you are authorized to make this request.

Second, log in to your AWS console and click on ‘Volumes’ under the EBS (elastic block storage) section. Make note of your Volume-ID, which you will need so that Amazon knows which volume to backup.

Next, you must install the Amazon command line tools. All of the sources should already be setup for you, if you used an Amazon image. Just execute the following code from the command shell and install.

sudo apt-get install ec2-api-tools

Here is the command used for your cron job:

/usr/bin/ec2-create-snapshot -O aws-secret-key -W aws-access-key-id  volume-id -d "Daily Backup $(date +'%m/%d/%y')"

For a daily backup, use the hour and minutes that you want the job to run. Use a * for days-of-month, months, and days-of-week. This means it will run at a specific time of day, but every day.

Breakdown:

  • -O AWS Secret Access Key
  • -W AWS Access Key ID
  • -d Snapshot Name (in our case, it will be Daily Backup mm/dd/yy). Of course, you can change this as needed.

ISPConfig Configuration:

First, check to make sure your permissions are setup for Reseller and Client to allow for you to create cron jobs. Also, you will want to setup the maximum number of cron jobs (-1 is unlimited) and the type of cron job to ‘Full Cron’. To setup a cron job with ISPConfig, you must link to a .sh file. Below, replace ‘clientXX’ and ‘webXX’ with your actual client number.

Create a directory in your client folder called ‘cron’. This should be located at: /var/www/clients/clientXX/cron/

cd /var/www/clients/clientXX
mkdir cron

Inside, create a file called daily-snapshot.sh.

vi cron/daily-snapshot.sh

This file should include the following:

#!/bin/bash

# content of your script

/usr/bin/ec2-create-snapshot -O aws-secret-key -W aws-access-key-id  volume-id -d "Daily Backup $(date +'%m/%d/%y')"

Also, you much execute the following commands to give the correct permissions to this file:

chown -R webXX:clientXX /var/www/clients/clientXX/cron/
chmod 755 /var/www/clients/clientXX/cron/daily-snapshot.sh

Finally, you are now ready to create your cron job in ISPConfig. Go to the ‘Cron Jobs’ section under the sites tab. Create a new cron job. Select your site. For a daily backup, use the hour and minutes that you want the job to run. Use a * for days-of-month, months, and days-of-week. This means it will run at a specific time of day, but every day. Enter the following for the command to run:

/var/www/clients/clientXX/cron/daily-snapshot.sh

Links:

SSI on Apache 2.4+, Conditional Statements – new format

I moved my site to a new server, and all of my conditional SSI statements stopped working. There is plenty of documentation for using SSILegacyExprParser, but very little documentation on the new method.

I used SSI to deliver a different html file depending on the domain visited. Originally, this was to get past my web host restrictions for total domains. However, when a particular client has many domains with very simple websites, I prefer to have all files in the same folder.

After the update, I was getting the following errors in my error.log folder:
[Tue Feb 17 01:58:08.865059 2015] [include:error] [pid 4855] [client xxx.xxx.xxx.xxx:yyyyy]: Could not parse expr "$HTTP_HOST = "mysite.com"" in /var/www/mysite.com/web/index.shtml: Parse error near '$'
and
[Tue Feb 17 01:52:11.580714 2015] [include:error] [pid 4294] [client xxx.xxx.xxx.xxx:yyyyy]: Could not parse expr "%{HTTP_HOST}=mysite.com" in /var/www/mysite.com/web/index.shtml: syntax error, unexpected T_OP_CONCAT, expecting '('

This is a result of the new syntax requirements for conditional expressions in Apache 2.4 and newer. Here is the code that worked for me:


<!--#if expr='%{HTTP_HOST} = "site1.com" || %{HTTP_HOST} = "www.site1.com"' -->
<!--#include virtual="page1.html" -->
<!--#elif expr='%{HTTP_HOST} = "site2.com" || %{HTTP_HOST} = "www.site2.com"' -->
<!--#include virtual="page2.html" -->
<!--#elif expr='%{HTTP_HOST} = "site3.com" || %{HTTP_HOST} = "www.site3.com"' -->
<!--#include virtual="page3.html" -->
<!--#else-->
<!--#include virtual="others.html" -->
<!--#endif-->

Fix “HTTP request length 134926 (so far) exceeds MaxRequestLen” error on Linux Ubuntu

If you get a 500 error in a webpage with apache webserver and fastcgi, take a look into the apache error.log file. This can either be the global error.log or the error.log of the website where you got the error. If you find a error similar to this one:

[Fri Apr 9 02:38:12 2014] [warn] [client 100.200.100.10] mod_fcgid: HTTP request length 134926 (so far) exceeds MaxRequestLen (131072), referer: http://www.example.tld/administrator/index.php?option=com_installer

then the MaxRequestLen setting of mod_fccgid is too low. To fix that, edit the file /etc/apache2/mods-available/fcgid.conf

# vi /etc/apache2/mods-available/fcgid.conf

and add or edit the line “MaxRequestLen 15728640″ to set the Request Limit to 15MB. The resulting file should contain these settings:

AddHandler fcgid-script .fcgi
FcgidConnectTimeout 20
MaxRequestLen 15728640

Save the changes and restart apache:

# /etc/init.d/apache2 restart

Deleting/viewing mail in postfix queue (mailq)

Postfix stores mails in a queue before sending it. Postfix has a command line called ‘postsuper’ which can be used to delete unsent mail from the queue. Before removing mail from the queue it is good idea to list all mail in the queue.

List all of the mail which is in the queue.
Code:

mailq

View the contents of a specific message.
Code:

postcat -q MESSAGE_ID

Remove a message from the “mailq”.
Code:

postsuper -d mailID

where mailID is the ID of the mail in the mail queue.

Delete all queued mail from the mailq.
Code:

postsuper -d ALL

Upload files with ssh / scp in Linux

The following code will transfer myfile.zip to the folder ‘myfolder’ on the server:

Code:
scp /home/username/documents/myfile.zip username@myserver.com:/home/username/myfolder/

scp = Linux command to transfer files ‘Secure Copy’
/home/username/documents/ = the directory where myfile.zip resides
myfile.zip = the file to transfer
username@myserer.com = username on myserver.com
/home/username/myfolder/ = the directory on myserver.com where myfile.zip should be transferred

You can also do the same in reverse:

Code:
scp username@myserver.com:/home/username/myfolder/myfile.zip /home/username/documents/

If you want to transfer a directory, just leave the trailing ‘/’ and add the ‘-r’ flag:

Code:
scp -r username@myserver.com:/home/username/myfolder/ /home/username/documents/

If you are connecting to Amazon hosting or need to use an identity file, try the following:

Code:
scp -v -r  -i ~/.ssh/identity.pem ~/Documents/myfolder/ username@myserver.com:/home/username/

-v Verbose – see the file transfer progress
-r Recursive – transfer a directory and all files inside
-i ~/.ssh/identity.pem Identity – use the identity file located at ~/.ssh/identity.pem
username@myserer.com = username on myserver.com
/home/username/ = the directory on myserver.com where myfolder should be transferred

Convert Pixles to REM: WordPress calculator for responsive design

Instructions

Just type the value in pixels and hit Enter. You will get two results – one to be used for font-size, width, height, margin, padding and the other result for line-height.

REM SIZE CALCULATOR

Rem
Line-height

Tip: A single click then selects the whole result, after that you can copy and paste it to your CSS.

Use rm to delete a directory

To delete a file or directory in terminal (command line) use the rm command.

Here is how to delete an entire directory:
rm -dfr directory

OPTIONS
-d, –directory
unlink FILE, even if it is a non-empty directory (super-user only)
-f, –force
ignore nonexistent files, never prompt
-i, –interactive
prompt before any removal
-r, -R, –recursive
remove the contents of directories recursively
-v, –verbose
explain what is being done
–help
display this help and exit
–version
output version information and exit

To remove a file whose name starts with a ‘-‘, for example `-foo’, use one of these commands:
rm -- -foo
rm ./-foo

To remove a file whose name contains a ‘*’ (or other special character), you will need to escape it. For example, I wanted to delete all directories starting with ‘~*w’:
rm -dfr ~\*w*

rsync via SSH

To copy files from one server to another via ssh:

First login to the destination server and cd to the directory that you want to copy files into.
Then type rsync -ave ssh username@sourceserver:directory/path/to/copy/ .
NOTE: don’t forget the ‘.’at the end. This is important.

On TextDrive (old FreeBSD servers), the exact path is:
rsync -ave ssh username@server.textdrive.com:domains/mydomain.com/public_html/ .

Expand tar file from Linux command line

If you have a file.tar.gz that you need to expand from the command line, type the following:
tar xfv file.tar.gz

Why doesn’t my cron job run?

Possible reasons your cron job won’t execute:

  • Check your script
  • Type the exact content of your script in your terminal window and verify that is is working manually. Then, if your server supports it, check if it works when you execute manually from web interface.
  • Use full paths
  • Many servers will run scripts without full paths both from the command line and in the web interface; however, they may not run on schedule.
  • If you are using:
  • * * * * * cd /users/home/username/domains/mydomain.tld/web/public/folder; php -f cron.php > /dev/null 2>&1
  • Try this, with full paths:
  • * * * * *cd /users/home/username/domains/mydomain.tld/web/public/folder; /usr/local/bin/php -f cron.php > /dev/null 2>&1
  • The time is not set correctly
  • The time is usually GMT/UTC on your server. Don’t look at your system clock. Click here for instructions to see the date/time on your server. Also, verify that the correct minutes, hours, days, months, and weekdays are active.
  • Reference files have been moved or deleted
  • Check any files used in your script and make sure they are in the location referenced by your script.
  • Your cron job is not active
  • Make sure your cron job is marked active (not disabled)