Awk – Part 1 – Printing the 5th word in a line of text to std output


To kick off ‘Project Bash’ located here I’m going to be talking about Awk. Awk is a text processing tool that can be used to manipulate text in a line in bash. There are many uses of awk so I have called this one Part 1. In Part 1 we are going to use awk to select a word in a line of text, this can be very useful if we want a specific value in a line of text like the 5th word and want to iterate that through each line in a file. Take the following example:

So we have just used Nmap to ping sweep a subnet and want to create a target IP list from the results, we send the output to a file called alive.txt. check out the below:

Awk Part 1

If we break the command down:

The first command ‘Cat alive.txt’ prints out the contents of the file alive.txt to std output (ie into bash). We then use the command ‘grep “report”‘ to find all lines with the word ‘report’ in:

Then finally using ‘awk ‘{print $5}’ to print the 5th word in the line using the variable $5.

Changing the variable in the awk command to ‘$2’ prints the 2nd word in the line:



Sed – Part 2 – To remove text from a line.

Sed – To remove text from a line.

To remove specific characters or portion of text from a line in Linux in bash we can use sed. In the below example we want to specifically remove ‘BARRY\’ from a line everything within the square brackets [text to remove] is removed, so we use:



Over the wire – bandit

War games!

I just thought I would share with you all I found this through ‘The hackers playbook 2’ book of pentesting tips and tricks found here

Over The Wire - War Games - Bandit

Over the wire is a set of text based games, brought to you in a ‘capture the flag’ type style where you ssh into there servers and crack the puzzle or code. The first series called ‘bandit’ is relatively straight forward with 26 levels. The bandit series is a great intro in its own right to bash as you have to use and combine tools and functions like grep, sort, base64  etc in order to crack the code. Each level finds you figuring out how to view/capture a text string which is the password to the next level. Head over to and check it out.


Information Security Defenders

We need more Information Security defenders!

InfoSec DefendersAs an industry we need more information security defenders to play the role of the blueteam in the Enterprise. All too often the IT Team receive a pentest report with a list of red criticals and are immediately overwhelmed with issues to fix. Whilst an organization has regular pentests it has the danger of lulling itself into a feeling of ‘doing the right thing’ however if all they are doing is getting a pentest and not re-mediating the issues then its a waste of time and money. This is a pessimistic view however it is usually the case that more vulnerabilities are released that the IT know what to do with. The IT team tend to either not have enough resource or they don’t have the skills to fix the issues, usually the first being being the case. Penetration testing needs to be more than just a test, it needs to be a more proactive engagement with after test help and support for the enterprise should they need it.

Its easy to criticise the IT department. As Information Security Professionals, we are doped daily with the next new exploit or zero day from the various blogs and podcasts that we listen to. This isn’t a small IT departments main concern or top of their list of to do tasks when they arrive at work, (although it could be argued it should be, with a little more security awareness training). The IT team tend to prioritise keeping those critical business apps or switches up and running (I’ve been that person).

As Information Security Professionals we need to be more involved in fixing and explaining the risks and supporting organisations so that they are better equipped to defend themselves.

The next new shiny security toy in the enterprise may not necessarily be the answer, the existing tools already installed around us could be better used and more creatively used to defend the enterprise.  Getting the latest IDS or implementing the next new NextGen Firewall can tend to be a distraction to getting down to what needs to be done i.e. Getting that MS08-067 patched on that business critical server that’s been there for years or tightening up the existing firewall. Hardening existing systems with group policy, tuning WSUS or SCCM to deploy timely updates and utilising windows logging to alert when a domain admin group is modified for example will generally go far further, and are free.

Hopefully sites like this will help contribute to securing enterprises, and assisting system administrators to better defend their existing environments. Enterprises don’t necessarily need to spend huge amounts of money on new technology, the answer could be in being more creative with what you already have.


Lets Encrypt!

Lets Encrypt! Free Certificate!

Lets Encrypt

I thought I would run through the process of obtaining a certificate for a web server from Lets Encrypt the open certificate authority by the Internet Security Research Group (ISRG) , for free!

The Lets Encrypt initiative has been setup to simplify the process of obtaining a certificate and make it more accessible for people to use encryption on their websites you can read more about it here: Certificates are free and valid for 90 days and need to be renewed  for continuation of the service. (which can be automated). Those who have been through the process of provisioning certificates will understand the involved process to get it working, this is a breath of fresh air in comparison.

Sounds great so what do we need to do? Well the below example runs through the process for an already created standard HTTP apache debian based system.

OK So Lets Encrypt!

First we need to obtain the Lets Encrypt files.  We will pull the files across from the Official GitHub repository. We’ll do this through git. This ensures that we have the latest version of the Lets Encrypt files. We will drop it under the /opt folder.

If you don’t have git installed… install it with:

Then pull across the Lets Encrypt folder with:

(If you want to update in the future run ‘git pull’ in the same folder.)

Move into the lets encrypt folder:

We then need to install the Lets Encrypt client through the ‘letsencrypt-auto’ command, when this is run it will essentially pull down all the related dependencies from your OS repositories and update the client through either apt-get or yum for example. From there on you can run either ‘letsencrypt’ or ‘letsencrypt-auto’ command for new certs of renewals.

Next we will obtain our certificate and bundle for our test domain and subdomain with the letsencrypt-auto command. This is the recommended method from the Lets Encrypt website.

There are a number of plugins that can be used with letsencrypt run command, and a number of command line parameters you can pass, these are all designed to help ease the process along. We are going to be running the Apache plugin and so will pass it the ‘–apache’ command. This plugin is designed to be used with Apache (funnily enough!) which automates the process of obtaining and installing the certificate with Apache2.4 on Debian based systems. This effectively sorts all the certificate configuration within Apache and then restarts the service.

After this is executed you should see the updates scroll through pulling content from your repositories, after a while you will presented with the following screen asking you to confirm your email. This is used as a reminder for renewal:

Lets Encrypt Confirm Email
Lets Encrypt Confirm Email

You can automate this process passing the –email parameter at the command line if you wish. So ‘–email’

You will next be asked to agree to the terms of service:

Lets Encrypt ToS
Lets Encrypt ToS

Again you can automate this by passing the –agree-tos in the command line.

Next you will be asked where you want to access your site through http and https or just through https.

You will then be presented with the ‘Congratulations!’ screen saying you have successfully configured your certificate and enabled https:

Lets Encrypt! Congratulations
Lets Encrypt! Congratulations


To renew your certificate run:

This will renew all your certificates with all previously used parameters for certificates that are due to expire within 30 days. passing the -d parameter will renew per domain.

Allows you to renew before 30 days.

You can also run:

This will renew your certificate with a key size of 4096 bit.

You can also rerun the existing command you ran earlier in which case you will be prompted that you have already run the command and to either re-run the install or renew your certificates.

All that is left to do is to automate the renewal with a cron job, one for another post.

Amazing that’s it! This makes obtaining a certificate for a website very accessible to people due to the way the scripts/plugins automate the apache configuration.

**Thumbs up Lets Encrypt this is great project!**


Server Hardening: HTTP TRACE TRACK Methods Allowed – Part1 Apache

HTTP TRACE / TRACK Methods AllowedMany vulnerability scanners will often bring back HTTP TRACE TRACK Methods Allowed against Apache and Microsoft web servers of the older generation. TRACE is usually associated with Apache and TRACK for Microsoft. This has a CVSS score of 4.3 and is a relatively easy fix. Clearly the older generation operating systems should be migrated to a supported platform, both the later distributions of Ubuntu and Microsoft 2012 R2 do not allow these methods to be used. However a simple way to validate this finding is to use telnet to connect to the web server on port 80, once connected you can type something similar to the following for each method. The ‘Host’, ‘TestA’ and ‘TestB’ aren’t needed however if you use some custom text you will be sure to see it echoed back by the web server if trace is enabled.

Tap return twice to send.

Which would look something like the below as you can see the user input was returned, the web server accepting the method:

HTTP Trace enabled on Apache
HTTP Trace enabled on Apache


As I said the HTTP TRACK / TRACE issue is this is relatively straight forward to fix, simple add ‘TraceEnable off’ somewhere in your main Apache config file outside of the vhost configuration.

Once implemented retesting should reveal that the method is not allowed:

after adding 'TraceEnable off' HTTP Trace disabled on Apache
After adding ‘TraceEnable off’

Server Hardening: Securing SSH part 2

Secure up your SSH service

This is Part two of Securing SSH in the Server Hardening Series. In this post we will continue to walk through the remaining hardening options for SSH.

In Part 1 of Securing SSH located here we discussed:

  • Disabling SSH Protocol 1 and using 2.
  • Limiting the users who can login.
  • Disable root login and unsing only a standard user account.
  • Run SSH on a different port.

In Part 2 of Securing SSH we will now cover the remaining techniques:

  • Use Public Private keys for Authentication
  • Filtering SSH with iptables
  • Setting strong Cryptographic Algorithms

Use Public Private keys for Authentication

As we know passwords can be subject to brute force attacks and given enough time simple passwords can be broken. Using SSH public private key pairs for authentication is a more secure method. Its a more secure method of logging in than using a password due to the computational power and time needed to decipher the public private key pair through brute force. The private key is used on the machine from which you are logging in from and the public key is used on the server or machine which you are establishing an SSH session with. In addition to using the key pair you can use a passphrase to secure the key pair. Should the key pair become compromised you have a an additional fallback and time in which to remove the public key from server it is being used on.

First we need to create the key pair, you can create either an RSA (Rivest-Shamir-Adleman) or DSA (Digital Signature Algorithm) key pair. The default key size in Ubuntu is currently 2048 however you can specify the -b parameter for a high keysize for example ‘-b 4096’. In this example we are creating a keypair for the root login, you should ideally do this for a standard user however this is just to demonstrate the procedure.

In this example we are creating the key pair on the same test ubtunu 14.04  server as the one which we are going to be using the keys to actually access. In reality there are a number of ways and platforms from which to create the key pair.

After the above command you will be prompted with some further questions, what and where the keys will be stored. Press enter and move on to accept the default files name, you can specify your own file name, the files will be created in the current working directory.

ssh file name

You will next be prompted to enter a passphrase, you have the option to enter one here. Doing so adds a layer of complexity for anyone that has already obtained the key pair and thus further hardens your configuration. If you enter a passphrase you will need to enter it in every time you ssh to the server.

The whole process will look like this:

SSH Key Generation using RSA 4096 key
SSH Key Generation using RSA 4096 key

If we now look in the ssh folder in the home user directory for root (or where ever you chose to save the files) we will see our key pair. The private key is called id_rsa and the public key is

SSH Key files
SSH Key files

At this point it would be wise to backup both sets of keys. You will need to copy off the private key for you to use from your local machine that you will be ssh’ing in from, we could do this in many ways either using winscp from a windows operating system alternatively we can view the private key and use copy and paste the contents from an existing session. Remember the folder is hidden, with the period in front of the folder name (.ssh) if your looking for it in winscp ;-).

We will need to move the public key into the ‘authorized_keys’ file on the server you are going to be using the keys for (the same server in our case), and restart SSH. The .pub file should be copied over to the correct profile you will be ssh’ing into.

At this point you are ready to test the rsa key authentication method. I would strongly recommend you test this at this stage whilst you still have regular username and password ssh authentication enabled, in case you have made a mistake and can’t get back in. If you going to be testing from putty or winscp you will need to use puttygen to convert your private key file into a format that putty can read which is a ppk file. This is a straight forward task.

Once tested and you are ready to start using your key pair you will want to remove the ability for ‘root’ (and other user in this example) to login with a username and password. Only do this after you have successfully used your key pair and are comfortable with the process. In the ‘sshd_config’ amend the ‘PermitRootLogin’ to read ‘PermitRootLogin without-password’ and add a line that reads ‘PasswordAuthentication no’ like below and then restart ssh one final time. This will deny access via username and password.



Filtering SSH with iptables

I demonstrated the use of iptables in one of my previous posts here, so won’t be going into this again.

Setting strong cryptographic algorithms

In this particular section of the post we are going to look at how we can customize the cryptographic algorithms that SSH can use and further optimize these so we are only using the most secure ones available. To dig a little bit deeper here there are two specific issues we are going address. The first being the type of encryption mode that is being used, and the second being the use of weak MAC algorithms.

At the time of writing (as this will change) your average vulnerability scanner will detect ssh on port 22 and will try to negotiate a session with the service. In doing so it will detect the cryptographic properties that the server would like to use, in your typical out of the box setup CBC (Cipher Block Chaining) encryption mode and MD5 or 96-bit MAC (Message Authentication Code) algorithms will be configured, both of which are considered weak. The use of CBC encryption mode for SSH is currently scored as CVSS Base Score 2.6 equating generally speaking as a Low risk. We can correct this by disabling CBC encryption mode, and enabling  a different type of mode such as CTR or GCM. The use of weak MAC is also scored with a CVSS Base Score of 2.6 equating again generally speaking as a Low risk. This can be correct by removing the use of MD5 and 96 bit MACs.

Amend or appending the following lines to the /etc/ssh/sshd_conf file will resolve this issue, there are other variations that you could use however this will rectify the issue:

I hope this has been informative.


Server Hardening: Securing SSH Part 1

This will be the first of two posts in the server hardening Series where we will discuss the Secure Shell (SSH) service.  I will cover the options available to us for hardening SSH. SSH is a cryptographic network protocol used for remote management over an insecure (or less secure) communication channel. Whether this is a web server, an appliance or a firewall, out on the internet or in your local subnet. For example most remote Linux based servers have SSH enabled in order for administrators to remotely manage them rather than being directly in front of the server/appliance with a monitor at the console. According to arstechnica (2015) Microsoft is also to introduce native support for SSH in 2015 – so watch this space.

There are a number of hardening techniques that we can undertake to further secure the SSH service from an out of the box typical install. These include:

  • Disabling SSH Protocol 1 and using Protocol 2. (Part 1)
  • Limit the the users who can login. (Part 1)
  • Disable root login and use a standard user account. (Part 1)
  • Run SSH on a different port to 22. (Part 1)
  • Use Public Private keys for Authentication. (Part 2)
  • Filter SSH with iptables (demonstrated in one of my previous posts here) (Part 2)
  • Setting strong cryptographic algorithms (Part 2)

I will be demonstrating these tasks on Ubtuntu 14.04. however the options and configurations will be very similar across the different Linux distributions.

Disabling SSH Protocol 1 and using Protocol 2

Disabling SSH Protocol 1 is done in the following file ‘/etc/ssh/sshd_config’, so using your favorite text editor nano, vi, leafpad etc (mines nano) open up the ‘sshd_config’ file and find the protocol line and ensure it has a the ‘2’ parameter next to it like the below:

Limit the the users who can login

Locate the authentication section again in the file ‘/etc/ssh/sshd_config’, and add in the line ‘AllowUsers Adam Mark’ and any other usernames as needed. This should be used in conjunction with disabling the root login. This will only let ‘Adam’ and ‘Mark’ login for example.

Disable root login and use a standard user account

Locate the authentication section and specifically the ‘PermitRootLogin yes’ and either comment the line out with a # and add a new line in or change the parameter to ‘no’.

Run SSH on a different port to 22

This can clearly be achieved in a number of different ways depending on how your infrastructure is configured. For example you could change the port on your SSH Service in the sshd_config file or adjust port forwarding/translation rules on your firewall or router. The main goal of this exercise is to obfuscate the SSH service to a potential attacker, we must be clear here this doesn’t protect the port in any way however does distract from the fact port 22 is not open for business. I say again this will not protect you against an attacker with enough intent, an attacker with enough experience will have scanned all 65k ports and verified all services on all open ports. Never the less if somebody has only scanned the top 1k ports its still an option, but should not be relied upon. I’m not going to get into the debate of ‘security by obscurity’ in this post. From the following file ‘/etc/ssh/sshd_config’, locate the ‘port 22’ line and change the number parameter to the unused port of your choice

I have added a comment in the line above as a reminder here to allow the next person to see what I have done for any troubleshooting purpose. You never know when you will move on and the next person needs to administer the box, this is useful to them. 😉

In most cases the SSH service will need to be restarted in order for the changes to take effect. From a terminal: (I always add a ‘-v’, optional for verbose)

As always I would suggest you try these out first in a test environment making one change at a time before changing your corporate machines.


Web Server Hardening: Removing Server and Software version information

All too often web servers are setup with fairly standard configuration. The HTTP Headers display various information from time stamps, cookie info and also server version.

Server version information especially should be removed from the HTTP headers as it allows an attacker to identify what the underlying server and web server version is. If vulnerabilities lie in the stated version, an attacker can concentrate there efforts towards that version identified on your system more easily.

The below configurations should be set for minimal server version info.


In the /etc/php5/apache2/php.ini file find the line ‘expose_php = On’ and set the parameter from ‘On’ to ‘Off’ as below:

This will remove the ‘X-Powered-by’ option from the HTTP header thus removing your PHP version and OS version information.

In the /etc/apache2/conf-available/security.conf locate the ‘Server Tokens Full’ line and change the parameter from ‘Full’ to ‘Prod’ this will give the least amount of information. Unfortunately without changing this hard-coded parameter and recompiling apache yourself there is no way to reduce this information any further.

In the same file locate the ‘ServerSignature On’ line and change the parameter from ‘On’ to ‘Off’, or comment out the existing line and add a new one in with the ‘Off’ option as below.

The ServerSignature isn’t actually information from or displayed in the HTTP headers, it is however information that is displayed at the bottom of for example a 404, 403 default page, which again will give away information about your system. Better still use a custom 404 or 403 page however if you don’t have custom pages this is the next best thing.

And as usual you should test these configurations out in a test environment first before your main production web servers.


Patch Management

Having good Patch managment is essential, and being able to keep on top of your microsoft patching is paramount to good security. It is all too easy to get caught behind in keeping systems upto date, since almost all software needs patching these days. Clawing your way back from out of date patches on servers can be a nightmare however automating patch managment if setup correctly with correctly configured maintenance windows makes life easier. Using products like WSUS, or better still SCCM 2012 R2 and having well built resiliant system architecture, can remove the pain from this task. Having a robust patching policy, and management buy in from the business is also essential. This enables the IT team to bring servers down at an appropriate time for that inevitable reboot is just  as important and can make the process run far more effectively. Does this bring into question whether this is an IT issue, a resourcing issue or a business strategy issue? Getting down time approval from a section of the business can be tricky without managment buy in, however, not letting the IT team take down that all important business critical system for patching is in itself a risk. A risk assement needs to be carried out by the business as to whether they delay remediating that zero day vulnerabilty vs letting the IT team patch the server and losing potential revenue whilst the server is down vs patching the system which subsequntly causes a system failure. All businesses should be asking themselves ‘what are my vulnerabilities?’. Subsequently what is the impact vs likelihood of this resulting in my overall risk? Of course this will be a case by case decision, with multiple factors, ie what is the patch fixing, the system archetecture etc. This needs to be weighed up against the consenquenses of not applying a patch ie can you afford to be hacked…

What are your thoughts?