Following a read of Joanna Rutkowska similar post I have finally migrated this Blog to the same type of system.
My new Blog is now hosted at blog.bowabos.com.
I have written a post on how to set-up such a system that you hopefully will find useful. If you have issues using this howto, please let me know.
Bits Of Wood And Bits Of Strings
Scala, Qubes OS, Linux, OS X...
Sunday, 7 February 2016
Sunday, 1 December 2013
How-to set up Jira and Confluence on Fedora VM in Qubes OS
This article describes how to set up Jira 6.1.4 and Confluence 5.3.4 (hosted on tomcat 7.0.42 (running on Java OpenJDK JRE 1.7.0_45) with PostgreSQL 9.2.5 as a database back-end and apache httpd 2.4.6 as a reversy proxy front-end with SSL) on Qubes OS v2 Beta 2 (Fedora 18)
Audience: Unix/Web admin who are trying to set these services up on Fedora, Fedora on Qubes or who want to get familiar with how to set-up a service on a Qubes VM.
Implementation time: If you are an old developer and know the value of copie/paste to avoid problems, it should take you between 1 and 2 hours.
Jira and Confluence provide similar functionality to trac (bugs/issues tracking and wiki). It is free for open source projects. You however have to pay for a licence otherwise. There is a 30 days trial licence if you want to check it out before you buy. I am using the 2x $10 starter licence for 10 users and find such investment very valuable. Moreover for this licence, all benefits are going to the Room to Read charity, which promotes literacy and gender equality in the developing world.
Qubes OS is at the time of writing, to my view, one of the most secure Linux based desktop operating system. It can run Linux and Windows guest virtual machines and isolate VMs, networking, usb and graphical stacks levering on Xen hypervisor. It also leverage on a templating system to minimize the disk footprint of VMs based on the same template. It does not support 3D graphical operations out of the box but advanced users can set-up a second Video card and dedicate it to one of their VM. It is aimed to be used by IT security specialist on the go but it is also a very nice fit as a home desktop that you can leverage on to host services (file share, web site, etc...) more securely while browsing the dark net or the red district at the same time on the same hardware.
This blog post is discussed in the Qubes user group.
If Apache is compromised, The attacker would be able to impersonate any active user, and at some point be able to retrieve all the data in the database.
If Jira or Confluence is compromised, the credentials to the database would therefore be compromised as they have to be readable by the application daemon. Even if encrypted with a pass-phrase provided during boot time, a valid session is present in memory.
We are therefore for now going to run Apache, Jira, Confluence and PostgreSQL in the same VM.
In order to build and be able to test the various components as we stack them up, we will start with the far end by setting up the database, then Jira and Confluence and finally the reverse proxy which will sit in front.
Let's create the Jira and Confluence user.
For these changes to be visible to the wiki VM, shutdown the template VM.
In Wiki VM's Terminal:
Start the VM by opening its Terminal.
Let's test our databases.
Try to connect to the Jira database.
We also have to provide a copy of /usr/lib/systemd/system/tomcat for our service.
Let's generate our private key. I will not get it signed as I do not have a fixed IP address and do not want to trust Certificates Authorities.
Which should therefore be followed by another 302 response for /jira/ redirecting you again.
Which should then be followed by a 200 response for /jira/secure/SetupDatabase!default.jspa.
If you are missing the 302 to /jira/ you may have an incorrect Connector set-up in /opt/jira/conf/server.xml. Verify your proxyName, proxyPort and scheme values. They should be so that whatever the server is sending to the client allows the client to connect back.
Audience: Unix/Web admin who are trying to set these services up on Fedora, Fedora on Qubes or who want to get familiar with how to set-up a service on a Qubes VM.
Implementation time: If you are an old developer and know the value of copie/paste to avoid problems, it should take you between 1 and 2 hours.
Jira and Confluence provide similar functionality to trac (bugs/issues tracking and wiki). It is free for open source projects. You however have to pay for a licence otherwise. There is a 30 days trial licence if you want to check it out before you buy. I am using the 2x $10 starter licence for 10 users and find such investment very valuable. Moreover for this licence, all benefits are going to the Room to Read charity, which promotes literacy and gender equality in the developing world.
Qubes OS is at the time of writing, to my view, one of the most secure Linux based desktop operating system. It can run Linux and Windows guest virtual machines and isolate VMs, networking, usb and graphical stacks levering on Xen hypervisor. It also leverage on a templating system to minimize the disk footprint of VMs based on the same template. It does not support 3D graphical operations out of the box but advanced users can set-up a second Video card and dedicate it to one of their VM. It is aimed to be used by IT security specialist on the go but it is also a very nice fit as a home desktop that you can leverage on to host services (file share, web site, etc...) more securely while browsing the dark net or the red district at the same time on the same hardware.
This blog post is discussed in the Qubes user group.
Target architecture
Risk analysis
The only information of value is the data in the database.If Apache is compromised, The attacker would be able to impersonate any active user, and at some point be able to retrieve all the data in the database.
If Jira or Confluence is compromised, the credentials to the database would therefore be compromised as they have to be readable by the application daemon. Even if encrypted with a pass-phrase provided during boot time, a valid session is present in memory.
We are therefore for now going to run Apache, Jira, Confluence and PostgreSQL in the same VM.
Note: Based on the information provided in this tutorial it would however be easy to split into a three tier model Apache, Jira/Confluence, PostgreSQL with two legs (Jira/PostgreSQL and Confluence/PostgreSQL).
Methodology
Qubes OS offers template VM which present its file system to other VM in read only mode and allow patching of fedora packages. We will therefore leverage as much as possible on this by using fedora packages whenever possible.In order to build and be able to test the various components as we stack them up, we will start with the far end by setting up the database, then Jira and Confluence and finally the reverse proxy which will sit in front.
Preparing the Template
Installing PostgreSQL
Launch the terminal in the template.- Click Application Menu (top left)
- Select Template fedora-18-x64 / fedora-18-x64: Terminal
- sudo yum install postgresql-server
Installing Tomcat
Atlassian states that only Oracle JRE is supported. I am going to take the risk of using OpenJDK instead as I will have automatic update via fedora patches. I will also use the Tomcat package provided by fedora for the same reasons.Note: If you want to use the Oracle JRE, do not install it in the template but in the app VM unless your are ready to patch it when required to ensure all your VMs use a patched version or if you are building a dedicated template.Let's install tomcat.
- sudo yum install tomcat
Let's create the Jira and Confluence user.
- sudo /usr/sbin/useradd -r --comment "Account to run JIRA" --shell /bin/bash jira
- sudo /usr/sbin/useradd -r --comment "Account to run Confluence" --shell /bin/bash confluence
Installing Apache httpd
Let's install httpd and make sure we can SSL enable it.- sudo yum install httpd mod_ssl openssl
For these changes to be visible to the wiki VM, shutdown the template VM.
- sudo halt
Preparing the Wiki VM
Creating a Wiki VM
In Qubes OS, this is trivial. In Qubes VM Manager:- Select in the menu VM / Create AppVM
- Give it the name wiki
- Select the color blue
- Launch the creation by pressing OK
Note: The attack surface of this VM is of medium size. However the threat is fairly small as I am planning on using this infrastructure only for personal use and will either access it within the same Qubes OS with a Disposable VM or possibly remotely after having connected to it via a reverse proxy hosted as a VM in my Qubes server with Client IP and Certificate validation and a Disposable VM Browser as a client. I have therefore used the blue color.
Preparing the PostgreSQL databases
Preparing the read-write partition
In the read-write partition, let's first prepare a set of folders on which we will deploy our data so that it survives a reboot.In Wiki VM's Terminal:
- cd /rw
- sudo mkdir -p var/pgsql/data
- sudo chown postgres:postgres var/pgsql/data
- sudo chmod 700 var/pgsql/data
- sudo mkdir -p var/pgsql/backups
- sudo chown postgres:postgres var/pgsql/backups
- sudo chmod 700 var/pgsql/backups
Initializing the database
Let's initialize the database cluster.- sudo su
- rm -rf /var/lib/pgsql
- ln -s /rw/var/pgsql /var/lib/pgsql
- postgresql-setup initdb
- exit
Configuring the database cluster
Let's make sure we can connect to the database cluster.- sudo su - postgres
- vi /var/lib/pgsql/data/pg_hba.conf
- host confluence confluence 127.0.0.1/32 md5
- host jira jira 127.0.0.1/32 md5
Add the following lines toward the end of the file, below the IPv4 local connections.
Note: If you wish to change the port, or make the database listen to another address than the local loop 127.0.0.1, you will need to edit pgsql/data/postgresql.conf and change the address above in pgsql/data/pg_hba.conf.Once done exit from the posgres sudo session.
- exit
Configuring the jira database
Let's create the jira database user and the database.- sudo systemctl start postgresql
- sudo -s -H -u postgres
- /usr/bin/createuser -S -P -E jira
- /usr/bin/createdb --owner jira --encoding utf8 jira
Configuring the confluence database
Let's create the confluence database user and the database.- /usr/bin/createuser -S -P -E confluence
- /usr/bin/createdb --owner confluence --encoding utf8 confluence
- exit
Preparing for a reboot
Let's now edit our rc.local script so that we set everything in place when the VM reboots. In Wiki VM's Terminal:- sudo vi config/rc.local
- #!/bin/bash
- rm -rf /var/lib/pgsql
- ln -s /rw/var/pgsql /var/lib/pgsql
- /usr/bin/systemctl enable postgresql &
- /usr/bin/systemctl start postgresql &
- sudo chmod u+x config/rc.local
Testing the databases
Stop the wiki VM.- sudo halt
Start the VM by opening its Terminal.
Let's test our databases.
- sudo systemctl status postgresql
Try to connect to the Jira database.
- psql -U jira -h 127.0.0.1 -p 5432 -d jira
Note: Exit the database by typing "\q" and pressing [Enter]Try to connect to the Confluence database.
- psql -U confluence -h 127.0.0.1 -p 5432 -d confluence
Preparing the Jira Tomcat instance
Preparing the read-write partition
In the read-write partition, let's first prepare a set of folders on which we will deploy our application and data so that they survive a reboot.In Wiki VM's Terminal:- cd /rw
- sudo mkdir -p opt/jira
- sudo mkdir -p var/log/jira
- sudo mkdir -p var/jira
Integrating with systemd
Looking into systemd tomcat's service config (/usr/lib/systemd/system/tomcat), it states that to run a new instance of tomcat, we need to copy /etc/sysconfig/tomcat to /etc/sysconfig/jira (jira being the name we want to give to this tomcat instance) and set the variables we need. As the file will not be persisted in our Qubes VM, let's prepare one that we will copy at boot time.- sudo mkdir config/sysconfig
- sudo vi config/sysconfig/jira
- CATALINA_BASE="/opt/jira"
- CATALINA_TMPDIR="/opt/jira/temp"
- TOMCAT_USER="jira"
- CATALINA_PID="/var/run/jira.pid"
Note: If you are new to linux a more natural text editor than vi can be used. nano is a good one... replace vi with nano in all instructions. You just have to know to use [Ctrl] + [x] to exit and save your file.
We also have to provide a copy of /usr/lib/systemd/system/tomcat for our service.
Note: This copy needs to have SERVICE_NAME set prior to launching the tomcat start-up script. As systemd mandate an absolute path for the command to be executed, we are going to need to pass it via a shell command.In Wiki VM's Terminal:
- sudo mkdir -p config/systemd/system
- sudo cp /usr/lib/systemd/system/tomcat.service config/systemd/system/jira.service
- sudo vi config/systemd/system/jira.service
- ExecStart=/bin/bash -c 'export SERVICE_NAME="jira"; /usr/sbin/tomcat-sysd start'
- ExecStop=/bin/bash -c 'export SERVICE_NAME="jira"; /usr/sbin/tomcat-sysd stop'
- User=jira
- Group=jira
Preparing CATALINA_BASE
Let's prepare our CATALINA_BASE (/opt/jira), but we first need our jira account set-up.- sudo chown jira:root var/log/jira
- sudo chmod 770 var/log/jira
- sudo chown jira:root var/jira
- sudo chmod 770 var/jira
- cd opt/jira
- sudo mkdir -p conf/Catalina/localhost logs temp webapps work
- sudo chown jira logs temp work
- sudo chmod 770 logs temp work
Preparing for a reboot
Let's now edit our rc.local script so that we set everything in place when the VM reboots.- cd /rw
- sudo vi config/rc.local
- rmdir /opt
- ln -s /rw/opt /opt
- ln -s /rw/var/jira /var/jira
- ln -s /rw/var/log/jira /var/log/jira
- cp /rw/config/sysconfig/jira /etc/sysconfig/
- cp /rw/config/systemd/system/jira.service /usr/lib/systemd/system/
- touch /var/run/jira.pid
- chown jira:jira /var/run/jira.pid
- /usr/bin/systemctl enable jira &
- /usr/bin/systemctl start jira &
Preparing the Confluence Tomcat instance
Preparing the read-write partition
In the read-write partition, let's first prepare a set of folders on which we will deploy our application and data so that they survive a reboot.In Wiki VM's Terminal:- sudo mkdir -p opt/confluence
- sudo mkdir -p var/log/confluence
- sudo mkdir -p var/confluence
Integrating with systemd
Looking into systemd tomcat's service config (/usr/lib/systemd/system/tomcat), it states that to run a new instance of tomcat, we need to copy /etc/sysconfig/tomcat to /etc/sysconfig/confluence (confluence being the name we want to give to this tomcat instance) and set the variables we need. As the file will not be persisted in our Qubes VM, let's prepare one that we will copy at boot time.- sudo vi config/sysconfig/confluence
- CATALINA_BASE="/opt/confluence"
- CATALINA_TMPDIR="/opt/confluence/temp"
- TOMCAT_USER="confluence"
- CATALINA_PID="/var/run/confluence.pid"
Note: This copy needs to have SERVICE_NAME set prior to launching the tomcat start-up script. As systemd mandate an absolute path for the command to be executed, we are going to need to pass it via a shell command.In Wiki VM's Terminal:
- sudo cp config/systemd/system/jira.service config/systemd/system/confluence.service
- sudo vi config/systemd/system/confluence.service
- ExecStart=/bin/bash -c 'export SERVICE_NAME="confluence"; /usr/sbin/tomcat-sysd start'
- ExecStop=/bin/bash -c 'export SERVICE_NAME="confluence"; /usr/sbin/tomcat-sysd stop'
- User=confluence
- Group=confluence
Preparing CATALINA_BASE
Let's prepare our CATALINA_BASE (/opt/confluence), but we first need our confluence account set-up.- sudo chown confluence:root var/log/confluence
- sudo chmod 770 var/log/confluence
- sudo chown confluence:root var/confluence
- sudo chmod 770 var/confluence
- cd opt/confluence
- sudo mkdir -p conf/Catalina/localhost logs temp webapps work
- sudo chown confluence logs temp work
- sudo chmod 770 logs temp work
Preparing for a reboot
Let's now edit our rc.local script so that we set everything in place when the VM reboots.- cd /rw
- sudo vi config/rc.local
- ln -s /rw/var/confluence /var/confluence
- ln -s /rw/var/log/confluence /var/log/confluence
- cp /rw/config/sysconfig/confluence /etc/sysconfig/
- cp /rw/config/systemd/system/confluence.service /usr/lib/systemd/system/
- touch /var/run/confluence.pid
- chown confluence:confluence /var/run/confluence.pid
- /usr/bin/systemctl enable confluence &
- /usr/bin/systemctl start confluence &
Installing Jira
We have 2 potential way to proceed. Go with the WAR distribution and configure Tomcat based on Fedora basic settings or go with the standalone distribution and modify it to use our tomcat instance. I am going to fo with the second option as the editor (Atlassian) has already done a good job at configuring it, particularly in such a simple set up as the one I want to do.Note: In the background I have looked at the differences between the two. I have used a tool to remove html comments called XMLStarlet using the command "xmlstarlet ed -d '//comment()' file.xml > file-clean.xml" and a very good graphical diff program called meld. The Atlassian files are as expected better one to use as a starting point with a few exceptions like conf/web.xml (which contains more recent mime types).
Downloading the standalone distribution
Go to Atlassian Jira download page- Toward the bottom of the page toggle the "All JIRA Download Options"
- Download the Linux tar.gz file
Copying the tar.gz file to Wiki VM
This is documented in the Qubes OS user documentation page.- In Firefox menu select Tools/Downloads.
- Right-Click on atlassian-jira-<version>.tar.gz and select Open Containing Folder.
- Right-Click on the file and select Scripts / Copy to other AppVM.
- input wiki as the destination domain name.
- Allow the transfer to happen by clicking Yes
Install Jira in the read-write partition
Let's expand our Jira distribution.- cd
- tar xzvf QubesIncoming/disp<X>/atlassian-jira-<version>.tar.gz
- cd atlassian-jira-<version>-standalone
- sudo mv conf/* /rw/opt/jira/conf
- sudo chown -R root:root /rw/opt/jira/conf
- sudo mv atlassian-jira /rw/opt/jira
- sudo chown -R root:root /rw/opt/jira/atlassian-jira
- sudo mv lib /rw/opt/jira
- sudo chown -R root:root /rw/opt/jira/lib
- sudo mv external-source /rw/opt/jira
- sudo chown -R root:root /rw/opt/jira/external-source
- sudo mv licenses /rw/opt/jira
- sudo chown -R root:root /rw/opt/jira/licenses
- sudo mv tomcat-docs /rw/opt/jira
- sudo chown -R root:root /rw/opt/jira/tomcat-docs
- sudo vi /rw/config/sysconfig/jira
- #
- # Occasionally Atlassian Support may recommend that you set some specific JVM a rguments. You can use this variable below to do that.
- #
- JVM_SUPPORT_RECOMMENDED_ARGS=""
- #
- # The following 2 settings control the minimum and maximum given to the JIRA Jav a virtual machine. In larger JIRA instances, the maximum amount will need to be increased.
- #
- JVM_MINIMUM_MEMORY="384m"
- JVM_MAXIMUM_MEMORY="768m"
- #
- # The following are the required arguments for JIRA.
- #
- JVM_REQUIRED_ARGS="-Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory"
- # Perm Gen size needs to be increased if encountering OutOfMemoryError: PermGen problems. Specifying PermGen size is not valid on IBM JDKs
- JIRA_MAX_PERM_SIZE=384m
- CATALINA_BASE="/opt/jira"
- CATALINA_TMPDIR="/opt/jira/temp"
- TOMCAT_USER="jira"
- CATALINA_PID="/var/run/jira.pid"
- #-----------------------------------------------------------------------------------
- #
- # In general don't make changes below here
- #
- #-----------------------------------------------------------------------------------
- JVM_EXTRA_ARGS="-XX:+PrintGCDateStamps -XX:-OmitStackTraceInFastThrow"
- JAVA_OPTS="-XX:MaxPermSize=${JIRA_MAX_PERM_SIZE} -Xms${JVM_MINIMUM_MEMORY} -Xmx${JVM_MAXIMUM_MEMORY} ${JAVA_OPTS} ${JVM_REQUIRED_ARGS} ${DISABLE_NOTIFICATIONS} ${JVM_SUPPORT_RECOMMENDED_ARGS} ${JVM_EXTRA_ARGS}
- cd
- rm -rf atlassian-jira-<version>-standalone
Set the Jira Home directory
Let's set our JIRA Home Directory.- sudo vi /rw/opt/jira/atlassian-jira/WEB-INF/classes/jira-application.properties
- jira.home = /var/jira
Fix the context
There is a small bug (or relaxed configuration as it is a standalone distrib) in the Atlassian package as the server.xml is referring to catalina.home instead of catalina.base.- sudo vi /rw/opt/jira/conf/server.xml
- replace the docBase value from ${catalina.home} to ${catalina.base}
Prepare Jira for reverse proxy aware responses
Set the context path
- sudo vi /rw/opt/jira/conf/server.xml
- <Context path="" docBase="${catalina.base}/atlassian-jira" reloadable="false" useHttpOnly="true">
- <Context path="/jira" docBase="${catalina.base}/atlassian-jira" reloadable="false" useHttpOnly="true">
Set the URL for redirection
- sudo vi /rw/opt/jira/conf/server.xml
- <Connector port="8080" maxThreads="150" minSpareThreads="25" connectionTimeout="20000" enableLookups="false" maxHttpHeaderSize="8192" protocol="HTTP/1.1" useBodyEncodingForURI="true" redirectPort="8443" acceptCount="100" disableUploadTimeout="true"/>
- <Connector port="8080" maxThreads="150" minSpareThreads="25" connectionTimeout="20000" enableLookups="false" maxHttpHeaderSize="8192" protocol="HTTP/1.1" useBodyEncodingForURI="true" redirectPort="8443" acceptCount="100" disableUploadTimeout="true" proxyName="www.example.com" proxyPort="443" scheme="https"/>
Installing Confluence
Downloading the standalone distribution
Go to Atlassian Confluence download page- Download the Standalone Linux tar.gz file
Copying the tar.gz file to Wiki VM
This is documented in the Qubes OS user documentation page.- In Firefox menu select Tools/Downloads.
- Right-Click on atlassian-confluence-<version>.tar.gz and select Open Containing Folder.
- Right-Click on the file and select Scripts / Copy to other AppVM.
- input wiki as the destination domain name.
- Allow the transfer to happen by clicking Yes
Install Confluence in the read-write partition
Let's expand our Confluence distribution.- cd
- tar xzvf QubesIncoming/disp<X>/atlassian-confluence-<version>.tar.gz
- cd atlassian-confluence-<version>
- sudo mv conf/* /rw/opt/confluence/conf
- sudo chown -R root:root /rw/opt/confluence/conf
- sudo mv confluence /rw/opt/confluence
- sudo chown -R root:root /rw/opt/confluence/confluence
- sudo mv lib /rw/opt/confluence
- sudo chown -R root:root /rw/opt/confluence/lib
- sudo mv licenses /rw/opt/confluence
- sudo chown -R root:root /rw/opt/confluence/licenses
- sudo vi /rw/config/sysconfig/confluence
- JAVA_OPTS="-Xms256m -Xmx512m -XX:MaxPermSize=256m $JAVA_OPTS -Djava.awt.headless=true "
- cd
- rm -rf atlassian-confluence-<version>
Set the Confluence Home directory
Let's set our Confluence Home Directory.- sudo vi /rw/opt/confluence/confluence/WEB-INF/classes/confluence-init.properties
- confluence.home = /var/confluence
Prepare Confluence for reverse proxy aware responses
Set the context path
- sudo vi /rw/opt/confluence/conf/server.xml
- <Context path="" docBase="../confluence" debug="0" reloadable="false" useHttpOnly="true">
- <Context path="/confluence" docBase="../confluence" debug="0" reloadable="false" useHttpOnly="true">
Set the URL for redirection
- sudo vi /rw/opt/jira/conf/server.xml
- <Connector className="org.apache.coyote.tomcat4.CoyoteConnector" port="8090" minProcessors="5" maxProcessors="75" enableLookups="false" redirectPort="8443" acceptCount="10" debug="0" connectionTimeout="20000" useURIValidationHack="false" URIEncoding="UTF-8"/>
- <Connector className="org.apache.coyote.tomcat4.CoyoteConnector" port="8090" minProcessors="5" maxProcessors="75" enableLookups="false" redirectPort="8443" acceptCount="10" debug="0" connectionTimeout="20000" useURIValidationHack="false" URIEncoding="UTF-8" proxyName="www.example.com" proxyPort="443" scheme="https"/>
Preparing the reverse proxy
In order to present both applications under the same web server and to map the ports to a default port, we are going to use mod_proxy and mod_proxy_http. For more information on how to set-up a more complex set-up you may want to look at the Atlassian page. We will not use mod_proxy_connect as we will decrypt the connection on our reverse proxy and forward the traffic unencrypted to tomcat.Note: If you are planning on hosting other web applications than Jira and Confluence, you may prefer to host the reverse proxy in a dedicated VM which will forward traffic to different web application back end VMs.
Note2: This set-up does not provide caching due to the small expected number of users and the increased attack surface.
Preparing the read-write partition
In the read-write partition, let's first prepare a set of folders on which we will deploy our application and data so that they survive a reboot.In Wiki VM's Terminal:- cd /rw
- sudo mkdir -p config/httpd/conf
- sudo mkdir -p config/httpd/conf.d
- sudo mkdir -p config/httpd/conf.modules.d
Configuring httpd
Let's prepare our httpd.conf file.- sudo cp /etc/httpd/conf/httpd.conf config/httpd/conf
- sudo vi config/httpd/conf/httpd.conf
Comment out the following line:
- Listen 80
- ServerAdmin root@localhost
- ServerName www.example.com
- sudo vi config/httpd/conf.d/00-proxy.conf
- ProxyRequests Off
- ProxyPreserveHost On
- <Proxy *>
- Order deny,allow
- Allow from all
- </Proxy>
- <Location /confluence>
- Order allow,deny
- Allow from all
- ProxyPass http://127.0.0.1:8090/confluence
- ProxyPassReverse http://127.0.0.1:8090/confluence
- SetEnv proxy-sendchunks 1
- SetEnv proxy-interim-response RFC
- SetEnv proxy-initial-not-pooled 1
- </Location>
- <Location /jira>
- Order allow,deny
- Allow from all
- ProxyPass http://127.0.0.1:8080/jira
- ProxyPassReverse http://127.0.0.1:8080/jira
- SetEnv proxy-sendchunks 1
- SetEnv proxy-interim-response RFC
- SetEnv proxy-initial-not-pooled 1
- </Location>
Configuring openssl
Let's start by adjusting the SSL configuration file.- sudo cp /etc/httpd/conf.d/ssl.conf config/httpd/conf.d
- sudo vi config/httpd/conf.d/ssl.conf
- SSLProtocol TLSv1.2
- SSLCipherSuite ECDHE-ECDSA-AES128-SHA256:DHE-RSA-AES128-SHA
- SSLHonorCipherOrder on
- browse to "about:config"
- accept the risk
- search for tls and set security.tls.version.max=3
/!\ Under construction /!\|!| Note: Road blocked.|!|
\!/ After this sign \!/
\!/ After this sign \!/
Let's generate our private key. I will not get it signed as I do not have a fixed IP address and do not want to trust Certificates Authorities.
- sudo yum install crypto-utils
- genkey www.example.com
Note: I have used a 4096bits key without certificate request or passphrase.
Note2: There is a bug in fedora broken by upstream package for self signed certificate (as reliant at the moment on md5).
/!\ Under construction /!\|!| Note: Road blocked.|!|
\!/ Before this sign \!/
\!/ Before this sign \!/
Preparing for a reboot
Let's now edit our rc.local script so that we set everything in place when the VM reboots.- sudo vi config/rc.local
- mv /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.origin
- cp /rw/config/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf
- cp /rw/config/httpd/conf.d/00-proxy.conf /etc/httpd/conf.d
- cp /rw/config/httpd/conf.d/ssl.conf /etc/httpd/conf.d
- /usr/bin/systemctl enable httpd &
- /usr/bin/systemctl start httpd &
- /usr/sbin/iptables -I INPUT 5 -j ACCEPT -d 10.137.2.x -s y.y.y.y/z -p tcp --dport 443 -m state --state NEW
- echo 10.137.2.x www.example.com >> /etc/hosts
Note: becareful to have two superior sign so that the line is appended to the file.
Testing the httpd service
Stop the wiki VM.- sudo halt
- https://www.example.com
- Click on I Understand the Risks
- Add Exception...
- Confirm Security Exception...
Note: If you get a "Secure Connection Failed", remember that you have to enable TLSv1.2 in Firefox as mentioned above.
Troubleshooting httpd
If you have issues. Troubleshoot meticulously. Let's first do very basic check on the server side.
Make sure you can resolve this name using a command line toolsuch as nslookup or dig.
If it can't check you /etc/hosts file then your /etc/resolv.conf file where your dns server is defined.
Make sure you can connect over TCP to the web server by issuing telnet 10.137.2.<x> 443. you should get a promt. If you don't, you can install a tool such as tcptraceroute to help identify where routing is broken or where a firewall is in the way.
If you manage to connect you can view which cipher suite you used by looking in the httpd logs:
Apache httpd service is up
Check the service is started:- sudo systemctl status httpd
Apache httpd service is listening on the right port
Check the service is listening on the right port (443)- netstat -an | grep LIST | grep -v STREAM
Browser proxy settings
Your browser will use your browser proxy settings and either go direct or via proxy.DNS resolution
If the browser is going via a proxy, the proxy will do the dns resolution. otherwise, the first thing your browser does is a dns query to the name you entered in the browser destination bar.Make sure you can resolve this name using a command line toolsuch as nslookup or dig.
If it can't check you /etc/hosts file then your /etc/resolv.conf file where your dns server is defined.
TCP Connection
The next thing your browser (or proxy) will do is establish a TCP connection on port 443.This should give you a status of active (running).Make sure you can connect over TCP to the web server by issuing telnet 10.137.2.<x> 443. you should get a promt. If you don't, you can install a tool such as tcptraceroute to help identify where routing is broken or where a firewall is in the way.
SSL Handshake
If you have gone this far you should be OK unless there is a mismatch between the Protocol and Cipher supported by the web server and your browser. You can check the protocols you browser support by going to this Hannover's university web site.If you manage to connect you can view which cipher suite you used by looking in the httpd logs:
- sudo su
- tail /var/log/httpd/ssl_request_log
- exit
Testing the Jira and Confluence tomcat instances
Browse to Jira and Confluence:- https://www.example.com/jira
- https://www.example.com/confluence
Troubleshooting Jira and Confluence tomcat instances
Jira tomcat instance is up
Check the service is started:- sudo systemctl status jira
- sudo cat /opt/jira/logs/catalina.out
Jira tomcat instance is visible through the reverse proxy
Check that your request is hitting httpd:- sudo su
- tail /var/log/httpd/ssl_request_log
- tail /var/log/httpd/ssl_access_log
- tail /var/log/httpd/ssh_error_log
- exit
- sudo su
- tail /opt/jira/logs/access_log.<date>
- exit
Which should therefore be followed by another 302 response for /jira/ redirecting you again.
Which should then be followed by a 200 response for /jira/secure/SetupDatabase!default.jspa.
If you are missing the 302 to /jira/ you may have an incorrect Connector set-up in /opt/jira/conf/server.xml. Verify your proxyName, proxyPort and scheme values. They should be so that whatever the server is sending to the client allows the client to connect back.
Configuring Jira and Confluence
Configuring Jira
Open the wiki VM's Firefox browser and browse to:- https://www.example.com/jira
- Select the server Language.
- Select My Own Database as the Database Connection
- Select PostgreSQL as the Database Type
- Input 127.0.0.1 as the Hostname
- Input 5432 as the Port
- Input jira as the Database
- Input jira as the Username
- Input the jira's database password as Password
- Input public as the Schema
- Click on Test Connection
- Click on Next
- Input an Application Title
- Select the Mode
- Verify the Base URL (https://www.example.com/jira)
- Select the I have a licence option
- Copy the licence from the disposable VM to the wiki VM browser.
Configuring Confluence
Open the wiki VM's Firefox browser and browse to:- https://www.example.com/confluence
- Copy the licence from the disposable VM to the wiki VM browser.
- Select the external database option
- select the JDBC connector option
- Verify org.postsgresql.Driver is the Driver Class Name
- Verify jdbc:postgresql://localhost:5432/confluence is the Database URL
- Input confluence as the User Name
- Input the confluence's database password as the Password
- Click Next
Tuesday, 19 November 2013
How-to set up dnscrypt-proxy and unbound on Qubes OS
This article describes how to set up dnscrypt-proxy v1.3.3 and unbound 1.4.32 on Qubes OS v2 Beta 2.
dnscrypt-proxy is a tool which will encrypt DNS queries between the client and the resolver. It relies on libsodium which is an encryption library trying to make use of only well respected algorithms by the cryptographic community.
unbound is a is a validating, recursive, and caching DNS resolver.
Qubes OS is at the time of writing, to my view, one of the most secure Linux based desktop operating system. It can run Linux and Windows guest virtual machines and isolate VMs, networking, usb and graphical stacks levering on Xen hypervisor. It also leverage on a templating system to minimize the disk footprint of VMs based on the same template. It does not support 3D graphical operations out of the box but advanced users can set-up a second Video card and dedicate it to one of their VM.
The aim of this post is to allow new Qubes OS users to set-up a DNS service VM which will serve DNS queries for their VMs while ensuring encryption of the communication between their desktop and the DNS provider of their choice which support dnscrypt. unbound will be use to cache DNS queries and serve them to your VMs.
Discussion about this post in the Qubes OS user group can be found here
In Qubes VM Manager:
Attacks could be launched by malformed DNS queries (from an untrusted VM aiming at compromising the DNS service data, i.e. Cache poisoning, or service) or responses (from a DNS provider). To mitigate the first risk, you may want to implement a trusted dnsVM and an untrusted dnsVM aimed at serving VM of different level of trust in you Qubes VMs setup.
Attacks could also be launched by a man in the middle (external network, or possibly from the netVM) by attacking the decryption stack that we will use, libsodium, to either gain visibility on the data transmitted or possibly compromising the dns service.
Overall you may want to use different client stack for the dns resolution by the dnsVM and your other AppVMs so that a vulnerability in one cannot be used to swim back to initial requester.
In dns: Terminal
Let's launch a web browser in a disposable VM and download our source...
In Application Menu:
Note: If after a reboot you try again the nslookup and is does not work, you probably forgot to update the libraries using ldconfig as described above.
dnscrypt-proxy is a tool which will encrypt DNS queries between the client and the resolver. It relies on libsodium which is an encryption library trying to make use of only well respected algorithms by the cryptographic community.
unbound is a is a validating, recursive, and caching DNS resolver.
Qubes OS is at the time of writing, to my view, one of the most secure Linux based desktop operating system. It can run Linux and Windows guest virtual machines and isolate VMs, networking, usb and graphical stacks levering on Xen hypervisor. It also leverage on a templating system to minimize the disk footprint of VMs based on the same template. It does not support 3D graphical operations out of the box but advanced users can set-up a second Video card and dedicate it to one of their VM.
The aim of this post is to allow new Qubes OS users to set-up a DNS service VM which will serve DNS queries for their VMs while ensuring encryption of the communication between their desktop and the DNS provider of their choice which support dnscrypt. unbound will be use to cache DNS queries and serve them to your VMs.
Discussion about this post in the Qubes OS user group can be found here
Preparing the Template
Installing Unbound
Launch the terminal in the Template VM.- Click Application Menu (top left)
- Select Template fedora-18-x64 / fedora-18-x64: Terminal
- sudo yum install unbound
- sudo halt
Setting up dnscrypt-proxy on a DNS VM
Creating a DNS VM
In Qubes OS, this is trivial.In Qubes VM Manager:
- Select in the menu VM / Create AppVM
- Give it the name dns
- Select the color yellow
- Launch the creation by pressing OK
Attacks could be launched by malformed DNS queries (from an untrusted VM aiming at compromising the DNS service data, i.e. Cache poisoning, or service) or responses (from a DNS provider). To mitigate the first risk, you may want to implement a trusted dnsVM and an untrusted dnsVM aimed at serving VM of different level of trust in you Qubes VMs setup.
Attacks could also be launched by a man in the middle (external network, or possibly from the netVM) by attacking the decryption stack that we will use, libsodium, to either gain visibility on the data transmitted or possibly compromising the dns service.
Overall you may want to use different client stack for the dns resolution by the dnsVM and your other AppVMs so that a vulnerability in one cannot be used to swim back to initial requester.
Starting the DNS VM
Launch the terminal in the dnsVM. In the Application Menu:- Select the Domain: dns / dns: Terminal
Installating the development tools
In Qubes OS, it is usually preferable to install software packages in the template VM from which the VM you will use the tool from is based on. In this case, these tools will only be required to compile 2 packages. It is therefore preferable to install them in the dnsVM. They will not be available any more after a dnsVM reboot, but we will have make sure everything is OK before proceeding.In dns: Terminal
- sudo yum groupinstall "Development Tools"
Downloading the source code
Note: The most detailed information on how to install dnscrypt-proxy was available on github.Let's launch a web browser in a disposable VM and download our source...
In Application Menu:
- Select DisposableVM / DispVM: Firefox web browser.
- Browse to dnscrypt.org
- Click on "Download packages" or go to dnscrypt.org/dnscrypt-proxy/downloads
- Download the lastest version and save the file.
- Browse to Libsodium https://github.com/jedisct1/libsodium
- Click on "Download the tarball" or go to https://download.libsodium.org/libsodium/releases/
- Download the lastest version and save the file.
Copying the source code to the dns VM
This is documented in the Qubes OS user documentation page.- In Firefox menu select Tools/Downloads.
- Right-Click on dnscrypt-proxy-<version>.tar.bz2 and select Open Containing Folder.
- Right-Click on the file and select Scripts / Copy to other AppVM.
- input dns as the destination domain name.
- Allow the transfer to happen by clicking Yes
- Repeat the operation for the file libsodium-<version>.tar.gz.
Verifying the source
In the dns VM Terminal:- Calculate the hash of dnscrypt-proxy: openssl dgst -sha256 ~/QubesIncoming/dispXX/dnscrypt-proxy-<version>.tar.bz2
- Launch terminal by selecting your new disposable VM in Qubes VM Manager and in the menu VM / Run command in VM, putting terminal as the command to execute.
- Retrieve the original hash using DnsSec: dig +short +dnssec TXT dnscrypt-proxy-<version>.tar.bz2.download.dnscrypt.org
- Compare the 2 hashes, they must be identical.
- Repeat the process for the second file but this time check against: dig +dnssec +short TXT <file>.download.libsodium.org
Building the source
Let's start with Libsodium. In dns VM Terminal:- cd Documents
- mkdir libsodium
- cd libsodium
- tar xzvf ~/QubesIncoming/disp<XX>/libsodium-<version>.tar.gz
- cd libsodium-<version>
- ./configure
- make -j<numberOfCoresOnYourSystem>
- sudo make install
- sudo su -
- echo /usr/local/lib > /etc/ld.so.conf.d/usr_local_lib.conf
- ldconfig
- exit
- cd Documents
- mkdir dnscrypt-proxy
- cd dnscrypt-proxy
- tar xjvf ~/QubesIncoming/disp<XX>/dnscrypt-proxy-<version>.tar.bz2
- cd dnscrypt-proxy-<version>
- ./configure
- make -j<numberOfCoresOnYourSystem>
- sudo make install
Configuring the start up script
In dns VM Terminal:- cd /rw/config
- sudo chmod u+x rc.local
- sudo vi rc.local
- #!/bin/bash
- /bin/echo /usr/local/lib > /etc/ld.so.conf.d/usr_local_lib.conf
- /sbin/ldconfig
- /usr/local/sbin/dnscrypt-proxy --provider-key=B735:1140:206F:225D:3E2B:D822:D7FD:691E:A1C3:3CC8:D666:8D0C:BE04:BFAB:CA43:FB79 --provider-name=2.dnscrypt-cert.opendns.com. --resolver-address=208.67.220.220:443 --daemonize
Testing the daemon
Before rebooting the VM and loosing our development tools, let's check everything went according to plan... In dns VM Terminal:- cd /rw/config
- sudo ./rc.local
- nslookup
- server 127.0.0.1
- www.opendns.com
Note: If after a reboot you try again the nslookup and is does not work, you probably forgot to update the libraries using ldconfig as described above.
Installing Unbound
Preparing the read-write partition
In the read-write partition, let's first prepare a set of folders on which we will deploy our configuration files so that they survive a reboot. In DNS VM's Terminal:- cd /rw
- sudo mkdir -p config/unbound/conf.d
- sudo mkdir -p config/unbound/local.d
- sudo cp /etc/unbound/unbound.conf config/unbound
Configuring Unbound
Let's configure Unbound so that:- It listen to our eth0 interface IP Address (ifconfig | grep -i ast)
- It control who can do DNS queries
- It host a local zone
- And forward to dns-crypt the rest...
- sudo vi config/unbound/unbound.conf
- interface: 10.137.2.x
- access-control: 10.137.2.0/24 allow
- access-control: 10.138.2.0/24 allow
- access-control: x.x.x.x/y allow
- harden-large-queries: yes
- private-address: 10.0.0.0/8
- private-address: 192.168.0.0/16
- val-permissive-mode: yes
- do-not-query-localhost: no
- sudo vi config/unbound/local.d/local.conf
- local-zone: "my-intranet.com" transparent
- local-data: "www.my-intranet.com A x.x.x.x"
- local-data: "my-share.my-intranet.com A x.x.x.x"
- forward-zone:
- name: "."
- forward-addr: 127.0.0.1@53
Subscribe to:
Posts (Atom)