How To Setup An Apache Forward Proxy Server on Ubuntu 14.04


Introduction : –

A Forward Proxy Server is a server that sits between you, aka client, and your remote server. Lets put it in a simple way 🙂

Lets say you want to access Facebook from your laptop using your favorite browser. Your browser is a client here. Facebook obviously runs on a server 😉 Now, all HTTP requests made from your browser to Facebook will contain your laptop’s IP address as well. But you don’t want your IP address to be tracked. What you will do now?

Yes, the answer is setting up a Forward Proxy Server. This proxy server will sit between you & Facebook server. Whatever HTTP requests your browser will initiate will be relayed/proxied via this proxy server. Data packets flow like this. Your laptop –> proxy server –> Facebook server.

It means, Facebook will see that the request came from the proxy server & it will never know that it actually originated from your laptop.

It is just a simple explanation of Forward Proxy Server. There are many other uses & explanations. But I tend to keep things simple 😉

Lab Description : –

  • Forward Proxy Server – An Ubuntu 14.04 Server running Apache with IP 192.168.0.62
  • Client  – An Ubuntu 14.04 Server with IP 192.168.0.51
  • Remote Server : – An Ubuntu 14.04 Server with IP 192.168.0.50. This server runs my Jenkins, so I will be accessing Jenkins from my client via proxy server.

Steps to Perform : –

Configure Forward Proxy Server.

1. Install core product.

Install Apache, if not already done. It is as simple as issuing –

root@shashank-forward-proxy-server:/home/shashank# apt-get install-y apache2 apache2-doc apache2-utils

2. Install necessary modules.

Issue below command to install modules required for proxy server.

root@shashank-forward-proxy-server:/home/shashank# apt-get installlibapache2-mod-proxy-html libxml2-dev

3. Enable Apache modules.

Issue below command to enable all the required Apache modules.

root@shashank-forward-proxy-server:/home/shashank# a2enmod proxy proxy_ajp proxy_http rewrite deflate headers proxy_balancer proxy_connect proxy_html

4. Configure proxy configuration file.

We will now create an Apache proxy configuration file that will hold information required for proxying. Start with creating a file like this.

root@shashank-forward-proxy-server:/home/shashank# vim /etc/apache2/mods-available/proxy.conf

Please check my GitHub gist to see the contents of this file. HTML tags are interfering with the tags in configuration file. You can see that nothing much is happening here except enabling the proxy.

5. Define port for proxy server.

Now we need to define the port on which our proxy server must run. Issue this command after backing up the original file.

root@shashank-forward-proxy-server:/home/shashank# sed -i -e 's/80/8889/g' /etc/apache2/ports.conf

You can see that I have replaced default port 80 with 8889. Choose any port that you like & is available.

6. Define a Virtual Host.

Its now time to define a Virtual Host which is a separate instance of your web-server(remember you can host multiple sites on a single Apache server). We are defining it to enable more fine-grained logs & port. Start with backing up the original default Virtual Host.

root@shashank-forward-proxy-server:/home/shashank# cp -p /etc/apache2/sites-enabled/000-default.conf /etc/apache2/sites-enabled/000-default.conf.orig

Now edit this /etc/apache2/sites-enabled/000-default.conf file to define logs location & port. Here, I am running it on the same port 8889 as this is my only instance.

Please check my GitHub gist to see the contents of this file.

As you can see, we have defined the location for logs specific to proxy.

7. Enable the Virtual Host.

Time to enable our newly created Virtual Host.

root@shashank-forward-proxy-server:/home/shashank# a2ensite 000-default.conf

8. Restart Apache.

Configuration is now done & we must restart Apache to load these new settings.

root@shashank-forward-proxy-server:/home/shashank# service apache2 restart
 * Restarting web server apache2 [Fri Jun 23 09:09:32.982307 2017] [proxy_html:notice] [pid 2940:tid 140143966525312] AH01425: I18n support in mod_proxy_html requires mod_xml2enc. Without it, non-ASCII characters in proxied pages are likely to display incorrectly.
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using node2.shashank.com. Set the 'ServerName' directive globally to suppress this message

With a success message, you are confident that your Apache configuration is correct 🙂 Above is just a warning & not an error, so no need to worry 😉

Lets move on to client-side configuration.

Client Side Configuration : –

With a working Apache Forward Proxy Server configuration, lets move on to our client machine & configure it to use our proxy server. For this, log-in to the machine & go to System Settings. Then select Network & then Proxy.

Enter your HTTP proxy server IP or FQDN or hostname & the port. Remember we configured our proxy server to run on port 8889.

Configuring_Client_to_use_Apache_Forward_Proxy_Server-Shashank_Srivastava.png

Test proxy connections.

You are all set now 🙂 Time to test the settings. Lets now open our browser & try to access Facebook & also Jenkins on a remote server (IP 192.168.0.50. See Introduction) in the same network. If your configuration is correct, you will be able to browse Facebook or internet without any issue. You will also be able to access your Jenkins server. You can check your proxy in action in below logs. Remember we had configured log location in step # above. You need to log-in to your proxy server & check the log file which happens to be /var/log/apache2/access_forward_proxy.log . If your location is different, make sure you check that file.

You can see that requests made from client 192.168.0.51 to remote server 192.168.0.50 are proxying through our proxy server.

192.168.0.51 - - [23/Jun/2017:09:33:17 +0530] "GET http://192.168.0.50:8080/static/50cbf35e/images/16x16/warning.png HTTP/1.1" 200 761 "http://192.168.0.50:8080/static/50cbf35e/css/style.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36"
192.168.0.51 - - [23/Jun/2017:09:33:17 +0530] "GET http://192.168.0.50:8080/static/50cbf35e/images/16x16/error.png HTTP/1.1" 200 817 "http://192.168.0.50:8080/static/50cbf35e/css/style.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36"
192.168.0.51 - - [23/Jun/2017:09:33:17 +0530] "GET http://192.168.0.50:8080/static/50cbf35e/images/top-sticker-bottom-edge.png HTTP/1.1" 200 605 "http://192.168.0.50:8080/static/50cbf35e/css/style.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36"
192.168.0.51 - - [23/Jun/2017:09:33:03 +0530] "CONNECT fonts.gstatic.com:443 HTTP/1.1" 200 4806 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36"
192.168.0.51 - - [23/Jun/2017:09:33:17 +0530] "GET http://192.168.0.50:8080/opensearch.xml HTTP/1.1" 200 6997 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36"

I hope you found this post interesting, informative & easy 🙂 Do let me know if it needs improvements or you have suggestions 🙂

Advertisements

Bash Shell Script To Create A Rich, Colorful Server Health Report(in HTML)


As you know,  my previous shell script produces an HTML Server Health Report displaying all vital stats in a plain format & I thought its magical to have a shell script create an HTML report even though it was a rather simple looking report 😉 Then I wrote a web-app that helps you track how much money you have saved & came up with the idea to enhance my script to output a rich, vibrant, vividly colored HTML report that has interactive 3D charts & other colorful visual elements 🙂 Sneak a peek below!

You can grab the script from my GitHub gist page.

Server-Health-Report-Shashank_Srivastava.png

So, I modified my script & made it fetch more server data and display that data in different tables & panels. These tables & panels include : –

Directories that eat up most of your disk space.

Server-Health-Report-Shashank_Srivastava.png

Dynamic table showing the directories that take up most of your disk-space.

Top Memory consuming processes.

Server-Health-Report-Shashank_Srivastava.png

Dynamic table displaying Top Memory consuming processes.

Top CPU consuming processes.

Server-Health-Report-Shashank_Srivastava.png

Dynamic table displaying Top CPU consuming processes.

2 Speedometers displaying # of processes running & # of logged in users.

Server-Health-Report-Shashank_Srivastava.png

Speedometer 🙂

A bars system that displays the resources utilisation (RAM/CPU/Filesystem/INodes) & these bars change their color to red if threshold is crossed 🙂

Server-Health-Report-Shashank_Srivastava.png

Bars displaying vital system stats.

3D interactive pie-chart showing the break-up of filesystem utilisation.

Server-Health-Report-Shashank_Srivastava.png

3D chart with break-up of filesystem usage.

How To Receive Emails From Your PHP OpenShift Application Using Swift Mailer & GMail


There are times when we need emails to be sent/received from our PHP applications hosted on OpenShift. My particular example is based on a feedback form that I have put on my own PHP based web-application hosted on OpenShift http://www.howmuchisaved.in. Using this form, I can receive email whenever users submit their feedback to me. If your applications is hosted on your own server, you can easily setup Postfix to start receiving mails but things start to get tricky when you have little to no control on the web-server which runs your application. OpenShift doesn’t allow you to configure Postfix or any such utilities. It even doesn’t give you root access to install & configure dependencies. So, you are left with very little options. And this is where Swift Mailer comes to rescue. In this tutorial I will explain how you can leverage Swift Mailer PHP library to be able to receive mails. Please note that this tutorial can be followed for other restrictive hosting solutions as well.

Requirements : –

For this tutorial, you will need one GMail account (more on it below) & one PHP application hosted on OpenShift or any cloud platform.

Steps to be performed : –

Configure Google Security settings

Since we are using GMail as our SMTP, all emails will be delivered/relayed via smtp.gmail.com. By default, Google doesn’t allow less-secure apps (such as your OpenShift app) to access your account. So, go to https://www.google.com/settings/security/lesssecureapps and turn it on. Its better to create a new Google account & turn this setting on for that particular account. Choice is yours, so act accordingly.

Download Swift Mailer PHP Library

Download it from here https://github.com/swiftmailer/swiftmailer & place it inside your project directory.

Or if you have git installed on your machine, you can always use git clone https://github.com/swiftmailer/swiftmailer inside your project directory. This will create a new directory inside your project. Rename it so that its easy to reference Swift Mailer inside your code. I renamed mine as swiftmailer.

Edit PHP code

Now, with Swift Mailer library in place, edit your PHP script that handles e-mail functionality. Below is the sample code snippet from my GitHub repository’s file.

https://github.com/shashank-ssriva/HowMuchISaved/blob/openshift-version/howmuchisaved/send_feedback_mail.php

You may fork or download my entire project if you so wish 🙂 This version is fully functional and is already hosted on OpenShift (as told in introduction above).

require_once 'swiftmailer/lib/swift_required.php';
      //allow less secured app in Gmail settings for this to work.
      //Also use port 465 and ssl if it doesn't work.
      $transport = Swift_SmtpTransport::newInstance('smtp.gmail.com', 587, "tls")
      ->setUsername('your gmail username')
      ->setPassword('your gmail password');

You can see in the snippet above how you need to organise your Swift Mailer directory. You also need to enter your GMail credentials there.

Test your application

After making changes to the code, fire up your application in a browser & check if you received email from this page or not. Please note the line #22 in above
script. It mentions the email-address where you will be receiving e-mails from your application. Change this to yours if not already done.

That’s it 🙂 If everything has been configured correctly, you will start to receive e-mails from this application. You may use my repository for fully functional application & its code.

Host a PHP/MySQL Application/Dynamic Website on RedHat OpenShift


I had no idea that I can host my PHP/MySQL based application on OpenShift for free until I had to do some work on it a few weeks back. While working on some Automation with OpenShift, I came up with the idea to host my own app on it 😉

If you have a small PHP application or some dynamic PHP based website that you want to host on OpenShift for free, please read on.

So, here is how to do it.

Go to OpenShift Console

Login to your OpenShift account & go to console https://openshift.redhat.com/app/console/applications. You will see a button called “Add Application” like below. Click on it.

Applications___OpenShift_Online_by_Red_Hat.jpg

Choose PHP

Choose any of the PHP versions.

Create_a_New_Application___OpenShift_Online_by_Red_Hat.jpg

Enter a Public URL

Use a unique URL for your application. If you have not created a domain earlier, create one before this step.

Create_a_New_Application___OpenShift_Online_by_Red_Hat.png

Choose Scaling

If you want an HAProxy Loadbalancing, choose Scale with web-traffic as seen above. Please bear in mind that if your application is based on MySQL or any other Database, don’t scale it and go with default. Otherwise you will need to use HAProxy server’s IP or FQDN in your code for database connection string.

Choose Region

Free tier of OpenShift doesn’t allow to choose anything except first 2 options.

Create_a_New_Application___OpenShift_Online_by_Red_Hat.png

Create Application

Click “Create Application” button to create your application. This may take a few minutes.

Get_Started___OpenShift_Online_by_Red_Hat.png

Install Git client for your OS

Install Git client by following instructions shown there on the screen. Also, make note of the git clone command. Using these details, you will clone a repository from OpenShift to your machine. From this repository, you can push to OpenShift after making changes to code. Follow the instructions you see after clicking “Create Application” button.

Add Database gear

Add any of the Database gears. I have personally chosen MySQL 5.5. Make note of the details like database user & password. You will have to use these values in your code.

php___OpenShift_Online_by_Red_Hat.png

Push/Publish Code

Add your code project to this repository (created above). Edit your database connection string in code using the values you got from last step. Then run below commands to push it to OpenShift & restart Apache there. Whenever you make changes to code or add a new file/directory to your local repository, don’t forget to follow below steps.

git add .

git commit -am "adding project data"

git push

You are done 🙂 You will see your application/site hosted on OpenShift now 🙂

I hope you liked this post. Catch you soon with another post!

Create a Server Health Report (HTML) Using Shell Script


Shell scripts are insanely powerful & convenient. We all know it 😉 Much of the beauty in shell scripts lies in the way they can be used to automate many aspects of System Administration. As a SysAdmin, you might have been asked to prepare health-reports on a regular basis. Today, I wrote one such script that will generate an HTML health-report containing some vital system information. Lets see how it works 🙂

Lab Description : –

Ubuntu 14.04 Server. Environment : – Bash shell

Instructions : –

Download or clone my GitHub repository from below location.

https://github.com/shashank-ssriva/Linux-Server-HTML-Health-Report-Using-Shell-Script

Place the syshealth.sh file anywhere you want. I prefer keeping it under my home-directory but you may keep it anywhere.

Make it executable (if not already).

You may either run/execute it manually or you may also put it in a CRON job. I have chosen to generate the report twice a day, but its entirely upto you 🙂

Video Tutorial : –

To see the script in action, watch the video below on my YouTube Channel.

Additional Notes : –

I have kept the script & report minimal since I wrote it today only. You may customize it further so as to suit your needs. Sky is the limit 😉

Installing git 1.9.0 on RHEL


Installing git was quite some pain today 😦 Was trying to find the RPM but couldn’t. Tried using the YUM, but no success once again 😦 So I went for the manual install of (g)it 😉

Below are the steps I followed.

  • Download git tarball & its  signature file from this link.
  • Place this tarball inside /usr/src.
  • Run tar xzf git-1.9.0.tar.gz
  • cd git-1.9.0
  • make prefix=/usr/bin/git
  • make prefix=/usr/bin/git install
  • echo “export PATH=$PATH:/usr/bin/git/bin” >> /etc/bashrc
  • source /etc/bashrc
  • Restart your session or open a new session.
  • Verify by issuing git –version

[root@server git-1.9.0]# git –version
git version 1.9.0
[root@server git-1.9.0]# which git
/usr/local/bin/git

P.S. You may receive below errors while installing git. So, make sure you have installed these packages before trying to install git :- 

libcurl-devel.x86_64
expat-devel.x86_64
zlib-devel.x86_64
openssl-devel.x86_64

Errors :-

* new build flags cc credential-store.o in file included from credential-store.c:1: cache.h:19:18: warning: zlib.h: no such file or directory in file included from credential-store.c:1: cache.h:21: error: expected specifier-qualifier-list before ‘z_stream’ make: *** [credential-store.o] error 1

http.c:1578: error: ‘struct http_object_request’ has no member named ‘slot’

P.P.S. – If somehow, above steps fail. Try below steps : –

  • cd /usr/src/git-1.9.0
  • ./configure
  • make
  • make install
  • which git
  • Then export the path using the same method as described above, just change that to /usr/local/bin/git

This is the default setting 😉