Month: December 2014

How to configure a writable upload folder on openshift for a PHP cartridge

When creating a Openshift gear for a GNU Social deployment I found that the files could not be uploaded. I needed
to create writable folder for the webserver to place the files. Openshift allows you to update the code via a git repository, but I did not want to
upload the files directly into the git repository on the server, because that would introduce problems everytime I updated the code. Openshift
also offers a separate dir for user data files and preferably the files are stored there. After some tinkering I found a solution.
There are some specific notes for Windows users at the end of the post.

The GNU social package has two folders which it uses for storing uploaded files /avatar and /file. I will be explaining how to perform this for
both folders, but you can execute this for any type of package you’d like, not only GNU social. The principle is the same.

The solution involves two separate steps: preparation and scripting. Some of the paths in the explanation start with /var/lib/openshift/[your app id]/, in this
case you need to replace the value [your app id] with the identifier of your app. You can find the value, if you don’t know it, by executing the following
command when signed into the console on the server:

echo $OPENSHIFT_DATA_DIR

Preparation

Make sure that the folder(s) you want to make world-writable for the uploads are not in version control (if they are, you might run into problems because then you are
basically updating version controlled folders when running the scripts. There is probably a way around that, but I tried to stay away from such a solution).

First, ssh into your server and execute the commands below, this will create the upload folders in the data part where the files can be uploaded:

mkdir $OPENSHIFT_DATA_DIR/avatar
mkdir $OPENSHIFT_DATA_DIR/file

Second, create the links into the repository and make them world-writable. The ln command does not really support environment variables, this is why
we are using the absolute paths here.

ln -s /var/lib/openshift/[your app id]/app-root/data/avatar /var/lib/openshift/[your app id]/app-root/runtime/repo/avatar
ln -s /var/lib/openshift/[your app id]/app-root/data/file /var/lib/openshift/[your app id]/app-root/runtime/repo/file
chmod -R o+rw $OPENSHIFT_REPO_DIR/avatar
chmod -R o+rw $OPENSHIFT_REPO_DIR/file

This completes the manual preparation.

Scripts

When performing a code update via git push we need to temporarily remove the link and recreate it again. Openshift offers the possibility to have scripts
executed in the different steps of deploying the code. In the folder in your local cloned git repository there is a folder .openshift/action_hooks. Two
files must be created here, pre_build and post_deploy. For more background information read the manual.

On Linux, execute chmod x .openshift/action_hooks/* to make the scripts executable. For Windows, see the special note at the end of the post.

pre_build

Remove the links:

#!/bin/bash
rm -f $OPENSHIFT_REPO_DIR/avatar
rm -f $OPENSHIFT_REPO_DIR/file

post_deploy

Re-create the links and make sure that they are world-writable:

#!/bin/bash
ln -s /var/lib/openshift/[your app id]/app-root/data/avatar /var/lib/openshift/[your app id]/app-root/runtime/repo/avatar
ln -s /var/lib/openshift/[your app id]/app-root/data/file /var/lib/openshift/[your app id]/app-root/runtime/repo/file
chmod -R o+rw $OPENSHIFT_REPO_DIR/avatar
chmod -R o+rw $OPENSHIFT_REPO_DIR/file

Notes for Windows

The scripts in the repository need to have the correct line endings and have the execution bit set. The manual
explains how to do this for Windows. In short execute in the root of your local clone of the git repository after creating the build hooks:

git config core.autocrlf input # use `true` on Windows
git config core.safecrlf true
git update-index --chmod=+x .openshift/action_hooks/pre_build
git update-index --chmod=+x .openshift/action_hooks/post_deploy

Message error: .openshift/action_hooks/*: does not exist and —remove not passed

.openshift/action_hooks/*: does not exist and --remove not passed

I got this message when attempting to execute the update-index command on a wildcard. When providing the individual filenames it worked.

Leave a Comment

Ruby Gem install fails on Windows because of SSL

When installing the Openshift rhc client I need to install a Ruby gem on Windows. This resulted in an error message:

C:\Users\user>gem install rhc
ERROR:  Could not find a valid gem 'rhc' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - SSL_connect retur
          ned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (
    https://api.rubygems.org/latest_specs.4.8.gz)

This is because the Ruby network client is unable to verify the SSL Certificate Authority. Via Google a found a Gist
which explains how this can be fixed. I found the manual way the fastest way, since I
will not be using gem for other projects.

In short I did this:

Download http://curl.haxx.se/ca/cacert.pem to c:\temp

Executed in command prompt:
set SSL_CERT_FILE=c:\temp\cacert.pem

After this executing the command to install the client tools worked:

C:\temp>gem install rhc
Fetching: net-ssh-2.9.2.beta.gem (100%)
Successfully installed net-ssh-2.9.2.beta
Fetching: net-scp-1.2.1.gem (100%)
Successfully installed net-scp-1.2.1
Fetching: net-ssh-gateway-1.2.0.gem (100%)
Successfully installed net-ssh-gateway-1.2.0
Fetching: net-ssh-multi-1.2.0.gem (100%)
Successfully installed net-ssh-multi-1.2.0
Fetching: archive-tar-minitar-0.5.2.gem (100%)
Successfully installed archive-tar-minitar-0.5.2
Fetching: highline-1.6.21.gem (100%)
Successfully installed highline-1.6.21
Fetching: commander-4.2.1.gem (100%)
Successfully installed commander-4.2.1
Fetching: httpclient-2.5.3.3.gem (100%)
Successfully installed httpclient-2.5.3.3
Fetching: open4-1.3.4.gem (100%)
Successfully installed open4-1.3.4
Fetching: rhc-1.33.4.gem (100%)
===========================================================================

If this is your first time installing the RHC tools, please run 'rhc setup'

===========================================================================
Successfully installed rhc-1.33.4
Parsing documentation for archive-tar-minitar-0.5.2
Installing ri documentation for archive-tar-minitar-0.5.2
Parsing documentation for commander-4.2.1
Installing ri documentation for commander-4.2.1
Parsing documentation for highline-1.6.21
Installing ri documentation for highline-1.6.21
Parsing documentation for httpclient-2.5.3.3
Installing ri documentation for httpclient-2.5.3.3
Parsing documentation for net-scp-1.2.1
Installing ri documentation for net-scp-1.2.1
Parsing documentation for net-ssh-2.9.2.beta
Installing ri documentation for net-ssh-2.9.2.beta
Parsing documentation for net-ssh-gateway-1.2.0
Installing ri documentation for net-ssh-gateway-1.2.0
Parsing documentation for net-ssh-multi-1.2.0
Installing ri documentation for net-ssh-multi-1.2.0
Parsing documentation for open4-1.3.4
Installing ri documentation for open4-1.3.4
Parsing documentation for rhc-1.33.4
Installing ri documentation for rhc-1.33.4
Done installing documentation for archive-tar-minitar, commander, highline, http
client, net-scp, net-ssh, net-ssh-gateway, net-ssh-multi, open4, rhc after 27 se
conds
10 gems installed

Home

After fixing the SSL issue I encountered a second problem during the setup command, after the SSH key step
the client would attempt to write a file to the H:\ drive. It assumed that that was my homedrive, while it does not exist.

Generating an authorization token for this client ... C:/Ruby21-x64/lib/ruby/2.1
.0/fileutils.rb:250:in `mkdir': No such file or directory @ dir_s_mkdir - H: (Er
rno::ENOENT)
        from C:/Ruby21-x64/lib/ruby/2.1.0/fileutils.rb:250:in `fu_mkdir'
        from C:/Ruby21-x64/lib/ruby/2.1.0/fileutils.rb:224:in `block (2 levels)
in mkdir_p'
        from C:/Ruby21-x64/lib/ruby/2.1.0/fileutils.rb:222:in `reverse_each'
        from C:/Ruby21-x64/lib/ruby/2.1.0/fileutils.rb:222:in `block in mkdir_p'

        from C:/Ruby21-x64/lib/ruby/2.1.0/fileutils.rb:208:in `each'
        from C:/Ruby21-x64/lib/ruby/2.1.0/fileutils.rb:208:in `mkdir_p'
        from C:/Ruby21-x64/lib/ruby/gems/2.1.0/gems/rhc-1.33.4/lib/rhc/auth/toke
n_store.rb:34:in `[]='
        from C:/Ruby21-x64/lib/ruby/gems/2.1.0/gems/rhc-1.33.4/lib/rhc/auth/toke
n_store.rb:14:in `put'
        from C:/Ruby21-x64/lib/ruby/gems/2.1.0/gems/rhc-1.33.4/lib/rhc/auth/toke
n.rb:50:in `save'
        from C:/Ruby21-x64/lib/ruby/gems/2.1.0/gems/rhc-1.33.4/lib/rhc/wizard.rb
:243:in `block in login_stage'
        from C:/Ruby21-x64/lib/ruby/gems/2.1.0/gems/rhc-1.33.4/lib/rhc/highline_
extensions.rb:190:in `call'

It turns out that there are two environment variables that define the HOME folder on Windows: HOMEPATH and HOMEDRIVE. Set
the combination of these two to a writable folder. For example to your user dir on Windows. These environment variables can be
set in two ways: via the commandline (where they will be forgotten after closing the shell windows), or via the Advanced Properties
of the About dialog of My Computer.

The simplest way via the commandline is:

    SET HOMEPATH=\users\user
    SET HOMEDRIVE=C:

Leave a Comment

Install Unbound for local network lookups

I am running a local server for some private websites. The problem is that from within the local network I cannot
lookup the public DNS entries that are set for these websites. My router does not understand where to route
the requests to. I used to solve this by creating a separate DNS entry prefix with l. for every domain name. Recently
I found that you can run a local DNS server that is only to be used locally, which can translate the lookups to the local IP
address instead of the public one. Unbound is a DNS server which can provide this. It will proxy all
DNS requests and only alter the ones that are configured to be redirected locally. Below I’ve have described manual installation
and installation using the apt-get package manager on raspbian.

Installing

Manually from source

I am running a server with Archlinux which did not provide a package, so I had to install it manually. I used the following commands:

cd /tmp
wget https://unbound.net/downloads/unbound-latest.tar.gz
./configure --prefix=/usr --sysconfdir=/etc
make
make install

This will compile and install unbound in /usr/bin and its configuration to /etc/unbound.

Service on Archlinux

With the manual installation I needed to also define a service to start and stop unbound. I create the file /usr/lib/systemd/system/unbound.service:

[Unit]
Description=Unbound DNS Resolver
After=network.target

[Service]
PIDFile=/run/unbound.pid
ExecStart=/usr/bin/unbound -d
ExecReload=/bin/kill -HUP $MAINPID
Restart=always

[Install]
WantedBy=multi-user.target

I also need to add a user to run unbound for:

useradd unbound

Using apt-get

apt-get install unbound

Configuration

I placed two configuration files in the /etc/unbound folder. This will configure the unbound server to listen to all bound IP addresses and
to allow DNS request from the local network (in my case 192.168.1.*, and from localhost. It will also include a file that defines the
static internal IP addresses for the domain names which are hosted locally.

The first line, local-zone, defines that for the root domain example.com all requests can be forwarded to the actual DNS server, if there
is no exception defined. local-data defines an exception for a specific entry.

/etc/unbound/unbound.conf
server:
# The following line will configure unbound to perform cryptographic
# DNSSEC validation using the root trust anchor.
auto-trust-anchor-file: "/var/lib/unbound/root.key"

include: "/etc/unbound/localnetwork.conf"
interface: 0.0.0.0
access-control: 192.168.1.0/24 allow
access-control: 127.0.0.0/8 allow

/etc/unbound/localnetwork.conf
local-zone: "example.com." transparent
local-data: "foo.example.com. IN A 192.168.1.1"

In order for the server itself to also use these IP address I updated /etc/resolv.conf to also use this DNS server:

nameserver 192.168.1.1

Leave a Comment