Chaining Operations and Operators in Linux

This is my list of my most used chaining operators in Ubuntu.

1 .   Semi-colon operator ;


The semi-colon allows you to chain multiple commands so that they run in-order.

# apt-get update ; apt-get upgrade ; echo 'upgrades are done'

 

2 . Single Ampersand &


The single ampersand will execute a command in the background and can chain commands to run in the background. You use the command followed by a space and the ampersand per command.
To run a single command:

# ping www.slsmk.com &

To run more than 1 command in the background:

# ping www.slsmk.com & cp ~/* . & apt-get update &

 

3 . Double Ampersand AND Operator &&


The && symbol, also called the AND Operator, links and executes commands in order only if the previous command is successful.
Technically a command is successful if it completes with exit status 0.
For example, I want to create a directory and file, but I only want to create the file is the directory is created correctly.

# mkdir ~/test && touch ~/test/tempfile1

 

4. Single PIPE |


The PIPE operator is used when you want the output of one command to be the input of a following command.
For example, this will list installed packaged then search for lines with ‘java’.

# dpkg -l | grep java

 

5. OR Operator ||


The OR operatorm || is similar to the AND operator, only here it will execute the following command only if the previous command failed. A command fails if it exits with status code 1.

# mkdir ~/test || echo 'The command failed'

 

6. NOT Operator !


The NOT operator ! is used in a command to identify those items that should be exempt from the command.
For example, imagine a directory with various filetypes and you wanted to remove all files except the PDFs.

# rm -r !(*.pdf)  

 

7. AND OR Operator && ||


This combination of the AND && and the OR || operators delivers what is basically an if-else statement based on the exit status code of the 1st command. BASH Shell has other command to get an if-else result, but this is using just the Operators.
For Example:

# mkdir ~/testdir && echo 'Directory Created' || echo 'Directory Creation Failed'

 

8. Precedence ()


When using && and ||, the exit codes determine whether or not to execute the following commands. Also, it is important to understand that the && and || only evaluate the 2 commands preceding and following the operators. So when using multiple operators, setting groups and precedence comes in handy when you want to ensure groups of commands complete or fail in a certain way.
In this example, Both commands in the 1st set () must exit with 0 in order for the next () set to execute.

# (ls *.pdf > pdf-files.list && cp *.pdf ~\) && (ls *.tar > tar-tiles.list && cp *.tar ~\) || echo 'Needs attention' 

 

9. IF THEN ELSE


Precedence can become very unreadable very quickly. I prefer using BASH’s IF THEN ELSE commands. These work just like any programming languange… IF something is true, THEN execute this command, otherwise (ELSE) run this command. Note that in BASH the if is concluded with ‘fi’.
There are pages of options for IF THEN ELSE which you should explore.
For example, this is a very basic example:

# if ls *.pdf ; then echo 'There are PDFs here' ; else echo 'There are no PDFs here' ; fi 

How to Install OwnCloud to Ubuntu 14.04 LTS

This quick how-to steps you through a simple installation of OwnCloud to a Ubuntu 14.04 server.

First, you need some prereqs:

 sudo apt-get install php5-gd mysql-server

To begin, you need to add the repository for ownCloud for ubuntu 14.04.

wget -nv https://download.owncloud.org/download/repositories/stable/xUbuntu_14.04/Release.key -O Release.key
apt-key add - < Release.key

Next, update the lists and install the package.

sh -c "echo 'deb http://download.owncloud.org/download/repositories/stable/xUbuntu_14.04/ /' >> /etc/apt/sources.list.d/owncloud.list"
apt-get update
apt-get install owncloud

Once the package is installed, access the ownCloud interface at http://SERVERNAME/owncloud

The first time you launch it, it will prompt you to create an admin id and password. Optionally, you can pick the Data folder location and choose MySQL vs SQLite.

owncloud setup 1404

 

After you create the admin user and sign into the OwnCloud Interface, if you are installing this for home use, you will probably want to enable some basic plug-ins.   If you plan on syncing calendars and contacts, then you will need at least those 2 add-ons.

Click on the Files entry on the upper left for the pull down menu shown below.   Click on the + to add new apps.

owncloud setup 1404

owncloud setup 1404

 

Select ‘Productivity’ from the left hand menu and Enable the Calendar and Contacts Applications.

owncloud setup 1404

owncloud setup 1404

 

Once completed, you will be taken to the web interface. Here you can add users and adjust settings as needed. I create the users here and that will pretty much complete the basic install.

ownCloud Admin menu

Last thing to do is load a desktop client available from ownCloud’s web page http://owncloud.org/sync-clients/

Next Steps that you should consider:
1) Enforce https connectivity to the owncloud, this is done through the admin menu selection.
2) Turn on Antivirus. Enabling this app in the Admin->apps menu sets up ClamAV to scan all uploaded content.
3) Make sure you are backing up the Owncloud Data Directory and the MySQL database.

 

There are additional options here for LDAP authentication, email alerts, and much more beyond this basic setup. Explore the add-on applications as well.

StrongSwan Ipsec VPN for Remote Users with Certificate Based Authentication

This is a working strongswan ipsec config that can be used for a roadwarrior setup for remote users utilizing certificate based authentication instead of id/pw.

This is a pure IPSEC with ESP setup, not L2tp.
This is not 2 factor, it is cert only.

To get started:

sudo apt-get install strongswan

You need is a CA that is capable of registering AltNames in a cert. OpenSSL can do this easily. I used this guide to setup the basic openssl CA. http://www.freebsdmadeeasy.com/tutorials/freebsd/create-a-ca-with-openssl.php

Once the CA is ready and you have generated your ca cert and ca private key, you next need to create a cert for the ipsec host and a cert for the end user.

For the Server:
Since I need the Alt Names in the certs, make a copy of /etc/ssl/openssl.cnf to be used for the Server.

cp /etc/ssl/openssl.cnf /etc/ssl/openssl-for-server.cnf
# Extension copying option: use with caution. copy_extensions = copy [ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment subjectAltName=@alt_names [alt_names] DNS.1 = ipsecvpn.mydomain.com DNS.2 = another_name.mydomain.com

Now, using Openssl, create the request for the server, fill in the details of the req as needed, then sign the request.

openssl req -new -nodes -out ipsechost-req.pem -keyout private/ipsechost-key.pem -config /etc/ssl/openssl-for-server.cnf
openssl ca -config /etc/ssl/openssl-for-server.cnf -out ipsechost-cert.pem -in ipsechost-req.pem

Copy the certs to the correct locations for strongswan to use.

cp cacert.pem /etc/ipsec.d/cacerts
cp ipsechost-cert.pem /etc/ipsec.d/certs
cp private/ipsechost-key.pem /etc/ipsec.d/private/

Stongswan is configured using the /etc/ipsec.conf and /etc/ipsec.secrets files.
This is a very simple config that will work for providing access to remote users:
Edit /etc/ipsec.conf

 
# /etc/ipsec.conf - strongSwan IPsec configuration file
config setup

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1

conn common
        left=IP_OF_IPSEC_HOST          # Ip of the host
        leftcert=ipsechost-cert.pem    # the cert we just created and copied
        leftid=@ipsecvpn.mydomain.com  # the Alt name in the Cert we just created
        leftsubnet=172.16.31.0/24      # The internal subnet the remote user wants to access
        right=%any                     # Connections can come from anywhere
        rightsourceip=192.168.1.0/24   # Use this pool of IPs to assign to these inbound connections
        auto=add

conn ikev2
        keyexchange=ikev2 
        also=common

Edit the /etc/ipsec.secrets file

: RSA ipsechost-key.pem

Restart/Reload IPsec.

 
ipsec restart

I like to watch logs just to be sure there are no errors:

 
tail -f /var/log/syslog /var/log/auth.log

Next we create a client cert. We need another copy of the openssl config file for user requests since the Alt Name changes from DNS to Email.

cp /etc/ssl/openssl-for-server.cnf /etc/ssl/openssl-for-users.cnf
[ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment subjectAltName=email:copy #[alt_names] #DNS.1 = ipsecvpn.mydomain.com #DNS.2 = another_name.mydomain.com

Create the request, fill in the details as needed for the user especially the email address, and Sign the request. The email address specified in the request prompts will be used in the cert for the Alt name and in the config for the user’s side of the tunnel.

openssl req -new -nodes -out user1-req.pem -keyout private/user1-key.pem -config /etc/ssl/openssl.cnf 
openssl ca -config /etc/ssl/openssl-for-users.cnf -out user1-cert.pem -in user1-req.pem 

You need to copy the user1-cert.pem, user1-key.pem, and the cacert.pem to the user’s machine. We will need these file to complete the VPN.

On the User’s Side:

sudo apt-get install strongswan

Copy the files into the proper directories
user1-cert.pem to /etc/ipsec.d/certs
user1-key.pem to /etc/ipsec.d/private
cacert.pem to /etc/ipsec.d/cacerts

Edit the client side ipsec.conf. This is a working config:

conn %default
     ikelifetime=60m
     keylife=20m
     rekeymargin=3m
     keyingtries=1
     keyexchange=ikev2

conn roadwarrior
     leftsourceip=%config                # This will take an IP from the ip pool on server
     leftcert=ipsecuser1-cert.pem        # The user cert we copied in
     leftid=ipsecuser1@openpeak.org      # This is the email included in the Alt Name in the user cert
     leftfirewall=yes
     right=cert-ipsec-vpn.openpeak.org   # The location of the host, FQDN or IP 
     rightid=cert-ipsec-vpn.openpeak.org # the Altname used by the ipsec host
     rightsubnet=172.16.31.0/24          # the subnet on the servers side you want to access. 
     auto=start

Edit the ipsec.secrets file

: RSA ipsecuser1-key.pem

On the client, issue an “ipsec restart” and it should attempt to build the tunnel with that you are done.

Troubleshooting:
Use “ipsec statusall” to get details on the tunnels. From the server, a healthy tunnel looks like this:

Security Associations (1 up, 0 connecting):
       ikev2[11]: ESTABLISHED 3 minutes ago, HOST_IP[vpnhost.mydomain.com]...REMOTE_IP[ipsecuser1@mydomain.com]
       ikev2[11]: IKEv2 SPIs: 49c4512b56436e5b_i 6276554588ce1803_r*, public key reauthentication in 50 minutes
       ikev2[11]: IKE proposal: AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_2048
       ikev2{9}:  INSTALLED, TUNNEL, ESP in UDP SPIs: cd1e015f_i cd3cb1c1_o
       ikev2{9}:  AES_CBC_128/HMAC_SHA1_96, 0 bytes_i, 0 bytes_o, rekeying in 12 minutes
       ikev2{9}:   HOST_INTERNAL_SUBNET=== POOL_IP_ASSIGNED_TO_REMOTEUSER

Use “ipsec listall” for details on the host’s certs and configs. Here we want to be sure Alt Names are good, and the CA and certs are loaded correctly.

List of X.509 End Entity Certificates:
  altNames:  ipsecvpn.mydomain.com
  subject:  "C=US, ST=NY, O=OpenPeak, OU=IT, CN=ipsecvpn.mydomain.com, E=ipsechost@mydomain.com"
  issuer:   "C=US, ST=NY, L=NY, O=mydomain, OU=IT, CN=ipsecserver-ca, E=ca@mydomain.com"
  serial:    10:03
  validity:  not before Mar 18 20:44:25 2015, ok
             not after  Nov 24 20:44:25 2028, ok 
  pubkey:    RSA 2048 bits, has private key
  keyid:     10:15:77:ae:2e:a4:e8:3f:cc:1f:6d:a9:d9:80:bd:9f:41:fb:63:e5
  subjkey:   3f:0c:bf:01:2f:c7:16:be:d4:83:5c:76:81:56:a9:f1:3a:84:b4:5f
  authkey:   b7:61:d7:32:19:65:c3:10:1a:43:23:27:bc:46:29:e5:ff:df:03:1c

List of X.509 CA Certificates:

  subject:  "C=US, ST=NY, L=NY, O=mydomain, OU=IT, CN=ipsecserver-ca, E=ca@mydomain.com"
  issuer:   "C=US, ST=NY, L=NY, O=mydomain, OU=IT, CN=ipsecserver-ca, E=ca@mydomain.com"
  serial:    db:e9:16:e0:44:a3:57:83
  validity:  not before Mar 18 15:49:45 2015, ok
             not after  Mar 15 15:49:45 2025, ok 
  pubkey:    RSA 2048 bits
  keyid:     18:47:07:92:b8:3d:a0:bb:88:bf:27:2b:4d:0b:a7:79:8b:c1:1b:ba
  subjkey:   b7:61:d7:32:19:65:c3:10:1a:43:23:27:bc:46:29:e5:ff:df:03:1c
  authkey:   b7:61:d7:32:19:65:c3:10:1a:43:23:27:bc:46:29:e5:ff:df:03:1c

Note that if you want to enable 2 factor with this, change the openssl request for the Clients to omit the -nodes option. This will prompt you for a password during the certificate creation that must be entered every time the client wants to connect.

Using Postfix to Relay messages to an ISP Email Server

Since my ISP blocks port 25, preventing me from running my own in-house email, I relay all my in-house emails and notifications generated from various components though the ISP email servers. This allows me to have internal components send messages to my in house server on 25 and those messages are relayed out to the ISP for delivery.

This setup works on Ubuntu 12.04 and 14.04

nano /etc/postfix/main.cf

Add the following (obviously replace the domain names and IP ranges with your own.)

myhostname = server.mydomain.local
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = server.myisp.com, localhost
relayhost = [server.myisp.com]:587
mynetworks = 192.168.10.0/24 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

#fix for some isp configs being stupid.
smtp_discard_ehlo_keyword_address_maps = hash:/etc/postfix/busted-servers
smtp_connection_cache_on_demand = no
smtp_discard_ehlo_keywords = pipelining,silent-discard

### Relay client Auth
smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd

Next we need to give the server the credentials it needs to perform the relay.
Create the password file:

nano /etc/postfix/sasl_passwd

Add the following line for your ISP.

[server.myisp.com]:587 username@mydomain.com:mypassword

Create the mailname file and change what is there

Echo mydomain.com >> /etc/mailname

Change permissions and run the mapping command.

Chmod 600 sasl_passwd
Postmap hash:/etc/postfix/sasl_passwd

That’s it. Send some test messages to your internal server and it should get delivered.

OpenSwan Error – One Way Traffic with Cisco ASA

Openswan 2.6.37

Symptom: OpenSwan to Cisco ASA Site to Site Tunnel has one way traffic.
Description: The Ipsec Tunnel builds, both the openswan host and the ASA show the tunnel up but traffic only flows from the ASA into Openswan, traffic does not flow back from openswan. No errors were shown in the auth.log.

Solution: It turns out that the issue was related to the openswan ipsec conf file for this connection. The Leftid and rightid were setup as shown here in the problematic conf file:

conn tunnel-to-HQ
  left=10.1.0.50
  leftid=@openswan
  leftsubnet=10.1.0.0/24
  right=PUBLIC.IP.OF.ASA
  rightid=@asa
  rightsubnet=10.2.0.0/24
  .
  .

  auto=add

This conf file would would just fin for an Openswan to Openswan IPSEC tunnel. But for an ASA to Openswan tunnel, it failed to pass two way traffic.

The simple fix was to replace the leftid and rightid with the IP addresses of the 2 peers as shown below:

conn tunnel-to-HQ
  left=10.1.0.50
  leftid=10.1.0.50
  leftsubnet=10.1.0.0/24
  right=PUBLIC.IP.OF.ASA
  rightid=PUBLIC.IP.OF.ASA
  rightsubnet=10.2.0.0/24
  .
  .
  auto=add

The secrets file should reflect the IP addresses in the conf for this PSK setup:

10.1.0.50 PUBLIC.IP.OF.ASA: PSK "123456789"

Restart the tunnel and traffic flowed normally.

How to Log IPTABLES Dropped Packets to Syslog

Simply, I want to have IPTABLES log whenever it drops a packet.

To log all dropped incoming packets, add these entries to the bottom of your IPTABLES rules:

iptables -N LOGGING
iptables -A INPUT -j LOGGING
iptables -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-Dropped: " --log-level 4
iptables -A LOGGING -j DROP

To log all dropped outgoing packets, add these entries to the bottom of your IPTABLES rules:

iptables -N LOGGING
iptables -A OUTPUT -j LOGGING
iptables -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-Dropped: " --log-level 4
iptables -A LOGGING -j DROP

To log all dropped packets, incoming and outgoing:

iptables -N LOGGING
iptables -A INPUT -j LOGGING
iptables -A OUTPUT -j LOGGING
iptables -A LOGGING -m limit --limit 2/min -j LOG --log-prefix "IPTables-Dropped: " --log-level 4
iptables -A LOGGING -j DROP

All items will be sent to the local syslog.