HOWTO: Test php7 on Ubuntu 14.04 LAMP stack

So, you've heard php7 is going to be fast... you've heard it's not going to be released until late this year... and you'd really like to TEST it out with a test copy of an existing LAMP stack running on Ubuntu 14.04 now versus waiting until November or December or until the release of Ubuntu 16.04?

Yeah, that's what I was thinking. I figured there would be a ppa and packages, but I didn't find any... UPDATE: August 10th, 2015 - there is a PPA now -- you should try it first. https://launchpad.net/~ondrej/+archive/ubuntu/php-7.0

[historical content]... So... with a little help from: http://www.zimuel.it/install-php-7/ , I managed to refine a test process so that it is relatively easy and quick and works with apache.

Before you start... you may want to look at what has been or quickly will be deprecated in php7 and start getting your code in gear. https://wiki.php.net/rfc/remove_deprecated_functionality_in_php7

GIANT DISCLAIMER:

This is a gross hack. You'd be silly to do this on any production server. Do this on a throw-away VM, just for testing. My recipe below OVERWRITES the system php -- because I wanted a quick and dirty hack to keep as many settings the same as possible. If you don't have the ability to clone a vm or build a new test LAMP stack in a few minutes... do not proceed. You shouldn't be reading any further. Here be dragons!

OK... so you're still with me? You understand this is an EXPERIMENT... good.

Build a test Ubuntu 14.04 LAMP stack.
I'm *ASSUMING* you have some workload already that you want to test with php7 and that you're running Ubuntu 14.04 and your LAMP stack is already running.

All we're going to do is:

• Add missing packages required to build php7 from latest source
• Overwrite php5 with php7
• Disable php5 in apache
• Make existing Apache work with libphp7.so
• restart apache and hope it worked
• do some testing on our own

Add build dependencies for php5 -- this is probably a good idea and MAY pull in most if not all of the packages below.

sudo apt-get build-dep php5

Add missing requisite packages:

sudo apt-get install build-essential libt1-dev bison libbz2-dev libjpeg62-dev libpng12-dev libfreetype6-dev libgmp3-dev libmcrypt-dev libmysqlclient-dev libpspell-dev librecode-dev apache2-dev git

Symlink the gmp.h file into /usr/include/ so the makefile can find it. I tried explicitly specifying the path in configure and it didn't work, so... unless you know why, just make this symlink

ln -s /usr/include/x86_64-linux-gnu/gmp.h /usr/include/gmp.h

Check out the latest php source code

mkdir $HOME/php7
cd $HOME/php7
git clone https://git.php.net/repository/php-src.git

Change into the source directory and run buildconf

cd php-src
./buildconf

Configure php7 source code

./configure \
    --prefix=/usr \
    --with-config-file-path=/etc/php7/apache2 \
    --with-config-file-scan-dir=/etc/php7/apache2/conf.d \
    --enable-mbstring \
    --enable-zip \
    --enable-bcmath \
    --enable-pcntl \
    --enable-ftp \
    --enable-exif \
    --enable-calendar \
    --enable-sysvmsg \
    --enable-sysvsem \
    --enable-sysvshm \
    --enable-wddx \
    --with-curl \
    --with-mcrypt \
    --with-iconv \
    --with-gmp=/usr/include/x86_64-linux-gnu \
    --with-pspell \
    --with-gd \
    --with-jpeg-dir=/usr \
    --with-png-dir=/usr \
    --with-zlib-dir=/usr \
    --with-xpm-dir=/usr \
    --with-freetype-dir=/usr \
    --enable-gd-native-ttf \
    --enable-gd-jis-conv \
    --with-openssl \
    --with-pdo-mysql=/usr \
    --with-gettext=/usr \
    --with-zlib=/usr \
    --with-bz2=/bin \
    --with-recode=/usr \
    --with-mysqli=/usr/bin/mysql_config \
    --with-apxs2 \
    --enable-dtrace

NOTE: As of March 4th code, I had to remove the following line from configure above:
    --with-mysql=/usr \

NOTE: As of August 6th, I had to remove the following line from the configure above:
    --with-t1lib=/usr \

Build the code - now would be a good time for a cup of coffee.

make

Might as well run the tests, I got two errors that seemed safe to ignore out of over 10K tests. I chose not to email the dev list.

** Mar 5th, 2015 - UPDATE - after this code update ... there are a lot of failing tests that will need to be removed. #FIXME This update and removal of the mysql code broke my app on March 5th - looks like we're using ext/mysql - time to dig into code and use mysqli http://www.saotn.org/migrate-php-mysql-mysqli/

make test

Install php7 -AGAIN.... if you type the command below, you're overwriting your php-5.5.9 install and should ubuntu php packages be updated in the future and upgraded, you might be unhappy. We're just trying to do a quick test of php7... remember?

make install

Now... In case your app specificially calls out php5 binary as /usr/bin/php5, let's force php apps to use the new php...

cd /usr/bin
mv php5 php5.orig
mv phpize5 phpize5.orig
mv php-config5 php-config5.orig
ln -s php php5
ln -s phpize phpize5
ln -s php-config php-config5

We need a php7.conf file. Assuming your LAMP stack already worked with php5, path of least resistance is to copy existing one to new name.

cp /etc/apache2/mods-enabled/php5.conf /etc/apache2/mods-enabled/php7.conf

Disable php5 in apache - we can't have php5 and php7 enabled - then restart apache

a2dismod php5
/etc/init.d/apache2 restart

Create /etc/php7 directory by making a symlink to /etc/php5 - not clear if this is required - I just did it since I'd already hacked up so much of the php install already.

cd /etc/ && ln -s php5 php7

Check to see if this worked..

From commandline:

# php --version
PHP 7.0.0-dev (cli) (built: Mar  4 2015 20:49:17)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v3.0.0-dev, Copyright (c) 1998-2015 Zend Technologies

From Apache webserver:

tail -200 /var/log/apache2/error.log
[Wed Mar 04 21:28:21.603995 2015] [mpm_prefork:notice] [pid 20692] AH00163: Apache/2.4.7 (Ubuntu) PHP/7.0.0-dev OpenSSL/1.0.1f configured -- resuming normal operations

Success!  (or... if not, maybe you need to do some debugging)

Now what?

You can do whatever you want. I chose to do an application benchmark, some manual QA testing of my app, and I wrote a blog post.

* opcache is now working once I updated the --with-config* statements above.

Just remember - when you're done testing and playing, destroy this test environment before a php package update overwrites everything. This was just for some quick tests, remember?

Boot an unbootable Ubuntu Linux system with chroot

How to recover an unbootable Linux system. This is mostly documentation for me since I KNOW I will needs this again.

Situation:

1) You have a somewhat unique Linux system and your system does not successfully complete the upgrade process,
2) You made some critical updates to system configuration or boot files and now the compuiter won't boot.

Here's what to do: Materials: 2GB USB stick for boot device. Optional: Reasonably large USB drive for data recovery if needed

Make a bootable USB stick or DVD of the latest Ubuntu as described here

Boot your troubled system from an alternate boot device - namely the USB stick. If your computer doesn't support boot from USB and you need a DVD, you can make one and boot from it, but it is significantly slower.

Choose: 'Try Ubuntu', and you should have a mostly fully functioning Ubuntu system that has already mounted any other block devices visible to the system

Find mounted devices with: mount |grep dev - find your other boot partition - it's probably something like /dev/sda1 or /dev/sda2.

Assuming it's /dev/sda2, do this to create a chroot environment with networking out of your problematic partition that won't otherwise boot:

sudo mount /dev/sda2 /mnt/chroot/
sudo mount -o bind /proc /mnt/chroot/pro1
sudo mount -o bind /dev /mnt/chroot/dev
sudo mount -o bind /dev/pts /mnt/chroot/dev/pts
sudo mount -o bind /sys /mnt/chroot/sys
sudo cp /etc/resolv.conf /mnt/chroot/etc/resolve.conf
sudo chroot /mnt/chroot /bin/bash

 The one modification to this that I had to make in my Ubuntu 13.10 --> 14.04 mess image was that I had to move the /mnt/chroot/etc/resolv.conf manually to another file and then copy over a simple text file for resolv.conf

Open another terminal window - one for USB booted OS, one for chrooted OS.
In the USB window, enter 'xhost +' to allow displaying GUI apps from the chroot environment running as root.

Now, in chroot window, edit or fix whatever you know is broken if you just broke it. If you're in the middle of an upgrade, try to continue it.

sudo apt-get update
sudo apt-get dist-upgrade

If there are any errors, try 'sudo apt-get dist-upgrade -f'
Watch the updates, and any interactive prompts that come up.

When it completes, or fails...

sudo apt-get update    <--- again
synaptic &    <--- use GUI to resolve as many of the upgrade dependencies as possible

shut down USB booted OS. Remove USB. Boot system normally. Cross fingers.

Linux users - Upgrade skype to 4.3 now

I've recently found that skype for Linux versions prior to 4.3 are having problems with chat, and other things that made using skype between my Ubuntu desktop, Macbook, and my iPhone make me scratch my head and wonder what's wrong with linux Skype. It almost felt like something of a skype split brain problem. Anyway... if you use Linux, you probably want to/ need to upgrade to Skype 4.3 or greater and you're going to need to do it manually.

Background: Linux users have been trained over many years to just be thankful that Skype for Linux EXISTS, that Microsoft didn't kill the port, and that an occasional update for Linux still gets pushed out. Somehow... over time... someone produces proper 3rd party packages with proper dependency tracking and puts them in a repository, and life has been mostly good.


I run Ubuntu, and I've gotten used to installing Skype from the partner repository as described here in the official Ubuntu Skype guide.

To install the latest version of Skype, I agree with the recommendations here:

1. If skype is not already installed, install the packaged version of skype from the Canonical partners repository to get proper dependencies installed first
2. uninstall skype
3. download and install the new skype from skype.com

The 4.3 version has solved my Skype 'split brain' problems, random weirdness, and issues with Skype chat.

HOWTO: Enable SSL, notifications, and auto-updates in Phabricator install

Phabricator on Ubuntu 14.04 - post-install server configuration: SSL, notifications, auto-update, auto-start

I searched around for agile development and code review tools and was very pleasantly surprised with Phabricator. Used heavily at Facebook, it's got just about everything a development team could want or need.

I followed this setup guide to get up and running quickly. Nothing I've added below is particularly complex or difficult. I did deviate from this guide in two areas:

1. I used Ubuntu 14.04 -- it's solid. No reason to use 12.04 for this.
2. I did NOT install phpMyAdmin -- It's a nice tool, but it's also one of the most popular attack vectors for internet facing servers. I don't need it, so I follow the rule of occam's razor - "Everything should be as simple as possible - and no simpler"

Follow above guide and you've got a default phabricator instance up and running. Cool, Huh?

Here's what I think is missing from the above guide:

Enable SSL and force all traffic to SSL

If you have a wildcard SSL cert... you probably want to use it to protect sites containing your source code. If you don't have an SSL certificate, you might want consider getting one.

Deploy your ssl certs in the OS instance:
mkdir -p /etc/nginx/ssl
copy yoursite.com.crt and yoursite.com.key to /etc/nginx/ssl

Update nginx webserver config

edit /etc/nginx/sites-enabled/phabricator

Add a section that rewrites port 80 (http) traffic to port 443 (https) at the top:
server {
    root /opt/phabricator/phabricator/webroot;
    location / {
        rewrite ^ https://$http_host$request_uri? permanent;
    }
}

Add or Modify the second, original server secton below to be rules for our new default port 443 server to look like this:
server {
  set $fqdn phabricator.vettersoftware.com;
  set $phabWebRoot /opt/phabricator/phabricator/webroot;

  listen 443 ssl;

  server_name $fqdn;

  root      $phabWebRoot;
  ssl on;
  ssl_certificate /etc/nginx/ssl/yoursite.com.crt;
  ssl_certificate_key /etc/nginx/ssl/yoursite.com.key;
  ssl_session_timeout 30m;

...

Update the internal phabricator settings for base-uri

/opt/phabricator/phabricator/bin/config set phabricator.base-uri 'https://phabricator.yoursite.com/'

Restart nginx
service nginx restart

Enable Phabricator's aphlict notifications

See: https://secure.phabricator.com/book/phabricator/article/notifications/

Install pre-requestites for aphlict
sudo apt-get install nodejs

Set notification config option
/opt/phabricator/phabricator/bin/config set notification.enabled true

Start notification service
/opt/phabricator/phabricator/bin/aphlict start

Start all services on instance boot

This is not included in the guide I used. It's not complex. I include it here in hopes it helps others who haven't been building UNIX systems for decades.

edit /etc/rc.local and add the following lines:

#start phabricator daemons
/opt/phabricator/phabricator/bin/phd start
#start aphlict notification server
/opt/phabricator/phabricator/bin/aphlict start

Automate phabricator upgrades

Phabricator code gets updated frequently. You want those new features and bug fixes, right? You believe in continuous integration, right? Right!

cd /opt/phabricator
wget http://www.phabricator.com/rsrc/install/update_phabricator.sh
chmod a+x update_phabricator.sh

edit /opt/phabricator/update_phabricator.sh

change line 14: ROOT=`pwd` to:
ROOT='/opt/phabricator'

change all references to /etc/init.d/httpd to /etc/init.d/nginx since we're using nginx
uncomment the notification server stop and start lines

Add '--force' option and end of '$ROOT/phabricator/bin/storage upgrade' line so it looks like this"

$ROOT/phabricator/bin/storage upgrade --force

save

Test the upgrade script.

./update_phabricator.sh

Add cronjob to run upgrade script weekly.

crontab -e

add following lines, season to taste, and save:

# update phabricator every Saturday at 9:35AM
35 9 * * Sat /opt/phabricator/update_phabricator.sh


You're done!

HOWTO: Configure Gitlab 7 Omnibus install on Ubuntu to use SSL

Gitlab is a great tool for managing source code. Gitlab's omnibus install on Ubuntu is great too -- an easy install, and a clearly documented, simple upgrade path. I wanted to make gitlab use SSL, but I found the documentation for what I thought was the most common install to be sparse, outdated, and confusing. (Then again, I've found many things about SSL certificates in general to be poorly documented and confusing... maybe it's just me)

This guide assumes that you have a running gitlab instance on Linux installed via above Gitlab omnibus install. It is also assumed that you already have either a specific SSL certificate for your site (ex: gitlab-foo.yourdomain.com) or wildcard SSL certificates for your site (ex: *.yourdomain.com).

Below is a summary of what I did to convert my Gitlab instance to use SSL. I hope this is helpful to others:

My server: Ubuntu 12.04 x64 HVM - running in Amazon Web services on a C3.Large instance. At the time I installed this, there were some issues that prevented me from deploying Ubuntu 14.04, but I expect everything below should work exactly the same for Ubuntu 14.04

Put your ssl certificates on gitlab server

mkdir -p /etc/nginx/ssl

put your ssl certificates in this directory as:
   server.crt <-- public key
   server.key <-- private key in .pem format (first line contains something like BEGIN ___ PRIVATE KEY = .pem format)

Edit gitlab.rb template config file

edit /etc/gitlab/gitlab.rb and add or modify following lines:

  external_url 'https://your.domain.com'
  nginx['redirect_http_to_https'] = true
  nginx['ssl_certificate'] = "/etc/nginx/ssl/server.crt"
  nginx['ssl_certificate_key'] = "/etc/nginx/ssl/server.key"

Edit gitlab.yml config file

edit /var/opt/gitlab/gitlab-rails/etc/gitlab.yml

  port: 443
  https=true

Edit gitlab-shell/config.yml

edit /var/opt/gitlab/gitlab-shell/config.yml

  ca_file: /etc/nginx/ssl/server.crt 
  ca_path: /etc/nginx/ssl
  gitlab_url: "https://127.0.0.1:8080"

Reconfigure with chef and restart

sudo gitlab-ctl reconfigure
sudo gitlab-ctl restart

.... and that should do it!

Long time, no blog...

Hello, there blog. It's been a few years... I have 2 kids now and changed jobs. Maybe I'll start posting more here again.

Dinner with Jeff & Pearl