Tuesday, October 16, 2012

PHP vs Node.js vs Native Speed Comparison

A quick speed comparison between PHP, Node.js and native code. This is obviously an artificial benchmark. The code generates four random numbers between 0 and 100 and uses them as coordinates of two points on a plane, and then calculates the distance between them. This is repeated 10 million times. Only the loop is timed, so process startup time is not part of the timing. The results are:

PHP - 12.692 seconds
Node.js - 0.328 seconds
Native - 0.308 seconds

All tests were run on a MacBook Pro. PHP is almost 40 times slower than Node.js. Node.js is almost as fast as native code, which makes sense since it does just-in-time compilation.

PHP code:
<?php

function distance($x1, $y1, $x2, $y2) {
  $dx = $x2 - $x1;
  $dy = $y2 - $y1;
  return sqrt($dx * $dx + $dy * $dy);
}

$start = microtime(TRUE);
for($i = 0; $i < 10000000; $i++) {
  $x1 = rand(0, 100);
  $x2 = rand(0, 100);
  $y1 = rand(0, 100);
  $y2 = rand(0, 100);
  distance($x1, $y1, $x2, $y2);
}
$stop = microtime(TRUE);

echo "Elapsed time: " . ($stop - $start) . " s\n";

Node.js code:
function distance(x1, y1, x2, y2) {
  var dx = x2 - x1;
  var dy = y2 - y1;
  return Math.sqrt(dx * dx + dy * dy);
}

var start = Date.now()/1000;
for(var i = 0; i < 10000000; i++) {
  x1 = Math.random() * 100;
  x2 = Math.random() * 100;
  y1 = Math.random() * 100;
  y2 = Math.random() * 100;
  distance(x1, y1, x2, y2);
}
var stop = Date.now()/1000;

console.log("Elapsed time: " + (stop - start) + " s\n");

C code, compiled with gcc -Wall:
#include <stdlib.h>
#include <stdio.h>
#include <time.h>
#include <sys/time.h>
#include <math.h>

double distance(double x1, double y1, double x2, double y2) {
  double dx, dy;
  dx = x2 - x1;
  dy = y2 - y1;
  return sqrt(dx * dx + dy * dy);
}

int main(void) {
  int i;
  double x1, x2, y1, y2;
  struct timeval start, stop;

  srand(time(NULL));
  gettimeofday(&start, NULL);
  for(i = 0; i < 10000000; i++) {
    x1 = rand() * 100;
    x2 = rand() * 100;
    y1 = rand() * 100;
    y2 = rand() * 100;
    distance(x1, y1, x2, y2);
  }
  gettimeofday(&stop, NULL);
  printf("Elapsed time: %.3f s\n", (stop.tv_sec-start.tv_sec) + (stop.tv_usec-start.tv_usec) / 1000000.0);
  return 0;
}

Sunday, October 7, 2012

MongoDB 2.2 Aggregation And MapReduce Performance

TL;DR - 1 million documents with readings across 20 sensors (1 reading per document), find the number of readings and their average per sensor. Aggregation framework does it in 4.1 seconds, map/reduce in 67.2 seconds, map/reduce with jsMode=true in 45.5 seconds, and reading all documents into PHP and doing it there takes 3.3 seconds. Conclusion - MongoDB's map/reduce has horrible performance for some reason.

Load the initial data with
time mongo load_initial.js
which on my laptop took 1m32.271s.

// load_initial.js
//
// Create a collection of documents where each document is
// { "sensor_id": 4, "ts": 123456.78, "reading": 2.3 }

for(var i=0; i < 1000000; i++) {
  sensor_id = Math.floor(Math.random() * 20 + 1); // Random int 1-20.
  ts = Math.random() * 3000000; // Random float from 0-2999999.9999
  reading = Math.random() * 100; // Random float 0.0-99.9999
  obj = {"sensor_id": sensor_id, "ts": ts, "reading": reading};
  db.sensors.save(obj);
}
Run
mongo --eval 'db.sensors.count()'
as a sanity check, which on my laptop printed
MongoDB shell version: 2.2.0
connecting to: localhost:27017/test
1000000
For each sensor, let's count the number of readings and find the average reading. First let's do it using the aggregation framework:
// count_avg_by_sensor_aggregate.js
//
// For each sensor, find the number of readings, and their average value.

res = db.sensors.aggregate(
  {
    $group: {
      _id: "$sensor_id",
      "readings": {$sum: 1},
      "average": {$avg: "$reading"}
    }
  }
).result.sort(
  function(a, b) {
    return a._id - b._id;
  }
);

printjson(res);
Run it with
time mongo count_avg_by_sensor_aggregate.js
I omitted the output, but this took 0m4.111s. Now let's do this with a map/reduce:
// count_avg_by_sensor_mr.js
//
// For each sensor, find the number of readings, and their average value.

if(typeof jsMode === "undefined") {
  jsMode = false;
}

res = db.sensors.mapReduce(
  function() {
    emit(this.sensor_id, { "readings": 1, "total": this.reading });
  },
  function(key, values) {
    var result = { "readings": 0, "total": 0 };
    for(var i = 0; i < values.length; i++) {
      result.readings += values[i].readings;
      result.total += values[i].total;
    }
    return result;
  },
  {
    "out": { "inline": 1 },
    "finalize": function(key, value) {
      return {
        "readings": value.readings,
        "average": value.total / value.readings
      };
    },
    "jsMode": jsMode
  }
).results;

newres = [];
for(var i = 0; i < res.length; i++) {
  newres.push(
    {
      // When jsMode is true, _id becomes a string for some reason.
      "_id": parseInt(res[i]._id),
      "readings": res[i].value.readings,
      "average": res[i].value.average
    }
  );
}

newres = newres.sort(
  function(a, b) {
    return a._id - b._id;
  }
);

printjson(newres);
Run it with
time mongo count_avg_by_sensor_mr.js
This took 1m7.171s . Now do the same thing, but use jsMode=true:
time mongo --eval jsMode=true count_avg_by_sensor_mr.js
This took 0m45.459s . And finally, let's do the same calculation in PHP:
<?php

// count_avg_by_sensor.php
//
// For each sensor, find the number of readings, and their average value.

$mongo = new Mongo('mongodb://localhost:27017', array(
  'db' => 'test'
));
$db = $mongo->selectDB('test');
$sensors = $db->selectCollection('sensors');

$cursor = $sensors->find();
$result = array();
foreach($cursor as $doc) {
  $_id = (int)$doc['sensor_id'];
  if(!array_key_exists($_id, $result)) {
    $result[$_id] = array("readings" => 0, "total" => 0);
  }
  $result[$_id]["readings"] += 1;
  $result[$_id]["total"] += $doc['reading'];
}
ksort($result);

$newresult = array();
foreach($result as $_id => $value) {
  $newresult[] = array(
    "_id" => $_id,
    "readings" => $value["readings"],
    "average" => $value["total"] / $value["readings"]
  );
}

echo json_encode($newresult, JSON_PRETTY_PRINT);
Run it:
time php count_avg_by_sensor.php
This took 0m3.313s .

Wednesday, September 26, 2012

Encrypted RAID Disk on OS X Mountain Lion and Mavericks

UPDATE: This works on Mavericks as well

The Disk Utility application does not allow you to create an encrypted filesystem on a RAID volume. However, it is possible from the command line. WARNING: this will erase everything on those disks - do a backup if you want any of that data. The basic method is to create an Apple RAID volume, create a coreStorage logical volume group on it, and then create an encrypted logical volume on the logical volume group.

These are the unformatted disks before RAID.



You can see them from the command line as disk1 and disk2.

$ diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *121.3 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:          Apple_CoreStorage                         120.5 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   *2.0 TB     disk1
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   *2.0 TB     disk2
/dev/disk3
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS Macintosh HD           *120.2 GB   disk3

The coreStorage subsystem knows nothing about them yet, and only shows my Macintosh HD.

$ diskutil cs list
CoreStorage logical volume groups (1 found)
|
+-- Logical Volume Group B42959AC-207C-45CE-AC5B-A3B9E5289368
    =========================================================
    Name:         Macintosh HD
    Size:         120473067520 B (120.5 GB)
    Free Space:   0 B (0 B)
    |
    +-< Physical Volume 33B112ED-10BF-452E-BC96-1761AE2FFDC7
    |   ----------------------------------------------------
    |   Index:    0
    |   Disk:     disk0s2
    |   Status:   Online
    |   Size:     120473067520 B (120.5 GB)
    |
    +-> Logical Volume Family BE76718E-765A-4797-B7FD-9B743B6E28E9
        ----------------------------------------------------------
        Encryption Status:       Unlocked
        Encryption Type:         AES-XTS
        Conversion Status:       Complete
        Conversion Direction:    -none-
        Has Encrypted Extents:   Yes
        Fully Secure:            Yes
        Passphrase Required:     Yes
        |
        +-> Logical Volume 46D952CD-311E-476E-8C19-CE2392FBABCE
            ---------------------------------------------------
            Disk:               disk3
            Status:             Online
            Size (Total):       120154296320 B (120.2 GB)
            Size (Converted):   -none-
            Revertible:         Yes (unlock and decryption required)
            LV Name:            Macintosh HD
            Volume Name:        Macintosh HD
            Content Hint:       Apple_HFS

Create a RAID volume from the disks by dragging them into the RAID set and giving it a name (StorageRAID).



Under Options, tell it to automatically rebuild RAID sets if you want to.



Click Create, and then Create again to confirm.



It will create the RAID volume and mount it.



Now back to the command line.

$ diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *121.3 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:          Apple_CoreStorage                         120.5 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 TB     disk1
   1:                        EFI                         209.7 MB   disk1s1
   2:                 Apple_RAID                         2.0 TB     disk1s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk1s3
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 TB     disk2
   1:                        EFI                         209.7 MB   disk2s1
   2:                 Apple_RAID                         2.0 TB     disk2s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk2s3
/dev/disk3
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS Macintosh HD           *120.2 GB   disk3
/dev/disk4
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS StorageRAID           *2.0 TB     disk4

You can see that it created a disk4 as the RAID volume. Create a logical volume group named StorageLVG on disk4.

$ sudo diskutil cs createLVG StorageLVG disk4
Password:
Started CoreStorage operation
Unmounting AppleRAID set at disk4
Adding disk4 to Logical Volume Group
Creating Core Storage Logical Volume Group
Switching disk4 to Core Storage
Waiting for Logical Volume Group to appear
Discovered new Logical Volume Group "20D5D037-F88C-4F05-AD28-E569E9564FC0"
Core Storage LVG UUID: 20D5D037-F88C-4F05-AD28-E569E9564FC0
Finished CoreStorage operation

If you do a diskutil cs list now, you'll see the new LVG with the same UUID as above.

$ diskutil cs list
CoreStorage logical volume groups (2 found)
|
+-- Logical Volume Group B42959AC-207C-45CE-AC5B-A3B9E5289368
|   =========================================================
|   Name:         Macintosh HD
|   Size:         120473067520 B (120.5 GB)
|   Free Space:   0 B (0 B)
|   |
|   +-< Physical Volume 33B112ED-10BF-452E-BC96-1761AE2FFDC7
|   |   ----------------------------------------------------
|   |   Index:    0
|   |   Disk:     disk0s2
|   |   Status:   Online
|   |   Size:     120473067520 B (120.5 GB)
|   |
|   +-> Logical Volume Family BE76718E-765A-4797-B7FD-9B743B6E28E9
|       ----------------------------------------------------------
|       Encryption Status:       Unlocked
|       Encryption Type:         AES-XTS
|       Conversion Status:       Complete
|       Conversion Direction:    -none-
|       Has Encrypted Extents:   Yes
|       Fully Secure:            Yes
|       Passphrase Required:     Yes
|       |
|       +-> Logical Volume 46D952CD-311E-476E-8C19-CE2392FBABCE
|           ---------------------------------------------------
|           Disk:               disk3
|           Status:             Online
|           Size (Total):       120154296320 B (120.2 GB)
|           Size (Converted):   -none-
|           Revertible:         Yes (unlock and decryption required)
|           LV Name:            Macintosh HD
|           Volume Name:        Macintosh HD
|           Content Hint:       Apple_HFS
|
+-- Logical Volume Group 20D5D037-F88C-4F05-AD28-E569E9564FC0
    =========================================================
    Name:         StorageLVG
    Size:         2000054943744 B (2.0 TB)
    Free Space:   1999736168448 B (2.0 TB)
    |
    +-< Physical Volume AB193FA5-822F-479B-9D74-AAEC1BC22632
        ----------------------------------------------------
        Index:    0
        Disk:     disk4
        Status:   Online
        Size:     2000054943744 B (2.0 TB)

In diskutil list you can see that it changed the type of StorageRAID from Apple_HFS to Apple_CoreStorage.

$ diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *121.3 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:          Apple_CoreStorage                         120.5 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 TB     disk1
   1:                        EFI                         209.7 MB   disk1s1
   2:                 Apple_RAID                         2.0 TB     disk1s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk1s3
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 TB     disk2
   1:                        EFI                         209.7 MB   disk2s1
   2:                 Apple_RAID                         2.0 TB     disk2s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk2s3
/dev/disk3
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS Macintosh HD           *120.2 GB   disk3
/dev/disk4
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:         Apple_CoreStorage StorageRAID            *2.0 TB     disk4

Create an encrypted logical volume on this new LVG.

$ sudo diskutil cs createLV 20D5D037-F88C-4F05-AD28-E569E9564FC0 jhfs+ Storage 100% -stdinpassphrase
Passphrase for new volume:
Started CoreStorage operation
Waiting for Logical Volume to appear
Formatting file system for Logical Volume
Initialized /dev/rdisk5 as a 2 TB HFS Plus volume with a 155648k journal
Mounting disk
Core Storage LV UUID: F490C159-4CAB-463A-BAB8-3A6468CF1FE5
Core Storage disk: disk5
Finished CoreStorage operation

If you look at diskutil cs list now, you'll see the new volume.

$ diskutil cs list
CoreStorage logical volume groups (2 found)
|
+-- Logical Volume Group B42959AC-207C-45CE-AC5B-A3B9E5289368
|   =========================================================
|   Name:         Macintosh HD
|   Size:         120473067520 B (120.5 GB)
|   Free Space:   0 B (0 B)
|   |
|   +-< Physical Volume 33B112ED-10BF-452E-BC96-1761AE2FFDC7
|   |   ----------------------------------------------------
|   |   Index:    0
|   |   Disk:     disk0s2
|   |   Status:   Online
|   |   Size:     120473067520 B (120.5 GB)
|   |
|   +-> Logical Volume Family BE76718E-765A-4797-B7FD-9B743B6E28E9
|       ----------------------------------------------------------
|       Encryption Status:       Unlocked
|       Encryption Type:         AES-XTS
|       Conversion Status:       Complete
|       Conversion Direction:    -none-
|       Has Encrypted Extents:   Yes
|       Fully Secure:            Yes
|       Passphrase Required:     Yes
|       |
|       +-> Logical Volume 46D952CD-311E-476E-8C19-CE2392FBABCE
|           ---------------------------------------------------
|           Disk:               disk3
|           Status:             Online
|           Size (Total):       120154296320 B (120.2 GB)
|           Size (Converted):   -none-
|           Revertible:         Yes (unlock and decryption required)
|           LV Name:            Macintosh HD
|           Volume Name:        Macintosh HD
|           Content Hint:       Apple_HFS
|
+-- Logical Volume Group 20D5D037-F88C-4F05-AD28-E569E9564FC0
    =========================================================
    Name:         StorageLVG
    Size:         2000054943744 B (2.0 TB)
    Free Space:   0 B (0 B)
    |
    +-< Physical Volume AB193FA5-822F-479B-9D74-AAEC1BC22632
    |   ----------------------------------------------------
    |   Index:    0
    |   Disk:     disk4
    |   Status:   Online
    |   Size:     2000054943744 B (2.0 TB)
    |
    +-> Logical Volume Family AC7F549F-1D6F-4E22-B050-34791ABF53FB
        ----------------------------------------------------------
        Encryption Status:       Unlocked
        Encryption Type:         AES-XTS
        Conversion Status:       Complete
        Conversion Direction:    -none-
        Has Encrypted Extents:   Yes
        Fully Secure:            Yes
        Passphrase Required:     Yes
        |
        +-> Logical Volume F490C159-4CAB-463A-BAB8-3A6468CF1FE5
            ---------------------------------------------------
            Disk:               disk5
            Status:             Online
            Size (Total):       1999736168448 B (2.0 TB)
            Size (Converted):   -none-
            Revertible:         No
            LV Name:            Storage
            Volume Name:        Storage
            Content Hint:       Apple_HFS

And in diskutil list.

$ diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *121.3 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:          Apple_CoreStorage                         120.5 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 TB     disk1
   1:                        EFI                         209.7 MB   disk1s1
   2:                 Apple_RAID                         2.0 TB     disk1s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk1s3
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 TB     disk2
   1:                        EFI                         209.7 MB   disk2s1
   2:                 Apple_RAID                         2.0 TB     disk2s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk2s3
/dev/disk3
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS Macintosh HD           *120.2 GB   disk3
/dev/disk4
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:          Apple_CoreStorage StorageRAID            *2.0 TB     disk4
/dev/disk5
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS Storage                *2.0 TB     disk5

This is what it looks like in Disk Utility after everything is done.


Thursday, September 13, 2012

Compiling PHP 5.4.6 On OS X Mountain Lion

Mountain Lion came out, and still no PHP 5.4. It removed X11 though, which I used on Lion for the PNG and FreeType libraries when I compiled PHP 5.4 there. Instead of installing X11, I just compiled and installed the missing libraries this time. Command line tools have to be installed separately from inside XCode Preferences -> Downloads.

Here's how to install PHP 5.4.6 on Mountain Lion. I made a standalone installation, without affecting the Apache server that is already on the machine. PHP 5.4 has a built-in web server, so I don't need Apache on my laptop for development.

First, download, compile and install libjpeg. I downloaded it from http://www.ijg.org/files/, specifically the jpegsrc.v8d.tar.gz. Compile and install:

./configure --prefix=/usr/local/libjpeg8d
make
sudo make install

Download and install libfreetype from http://freetype.sourceforge.net/download.html#stable :

./configure --prefix=/usr/local/libfreetype2410
make
sudo make install

Download and install libpng from http://www.libpng.org/pub/png/libpng.html:

./configure --prefix=/usr/local/libpng1512
make
sudo make install

Finally, configure, compile and install PHP:

./configure \ 
  --prefix=/usr/local/php546 \ 
  --with-openssl \ 
  --with-pcre-regex \ 
  --with-zlib \ 
  --enable-bcmath \ 
  --with-bz2 \ 
  --enable-calendar \ 
  --with-curl \ 
  --enable-exif \ 
  --enable-ftp \ 
  --with-gd \ 
  --with-jpeg-dir=/usr/local/libjpeg8d \ 
  --with-png-dir=/usr/local/libpng1512 \ 
  --with-freetype-dir=/usr/local/libfreetype2410 \ 
  --enable-gd-native-ttf \ 
  --with-ldap \ 
  --with-ldap-sasl \ 
  --enable-mbstring \ 
  --with-mysql \ 
  --with-pdo-mysql \ 
  --with-mysqli \ 
  --with-libedit \ 
  --enable-pcntl \ 
  --enable-shmop \ 
  --with-snmp \ 
  --enable-soap \ 
  --enable-sockets \ 
  --enable-sysvmsg \ 
  --enable-sysvsem \ 
  --enable-sysvshm \ 
  --with-tidy \ 
  --enable-wddx \ 
  --with-xmlrpc \ 
  --with-xsl \ 
  --enable-zip

make
sudo make install

I also used pecl to install the MongoDB driver, but that made me realize that Mountain Lion did not have autoconf installed. So I downloaded it from http://ftp.gnu.org/gnu/autoconf/, compiled and installed it:

./configure --prefix=/usr/local/autoconf269
make
sudo make install
sudo sh -c 'echo "/usr/local/autoconf269/bin" > /etc/paths.d/autoconf'

That last line adds this autoconf to the PATH for all users on the system, but you'll need to open a new Terminal to see it.

Then install the MongoDB driver:

sudo /usr/local/php546/bin/pecl install mongo
sudo sh -c 'echo "extension=mongo.so" >> /usr/local/php546/lib/php.ini'

Monday, April 30, 2012

UTF-8 Regular Expressions in PHP

While PHP itself doesn't know about different character sets and treats all characters as being one byte long, the PCRE engine understands UTF-8. There's also mb_ereg_match(), but I prefer the PCRE functions (preg_...). Here's a piece of code to see if your PHP was compiled with PCRE UTF-8 support.

$str = 'ありがとう';
echo "strlen('$str') = " . strlen($str) . "\n";
echo "preg_match_all('/./', '$str', \$matches) = " .
  preg_match_all('/./', $str, $matches) . "\n";
echo "preg_match_all('/(*UTF8)./u', '$str', \$matches) = " .
  preg_match_all('/(*UTF8)./u', $str, $matches) . "\n";

Which outputs the correct length of 5 characters when you start your regular expresssion with (*UTF8) and use the /u modifier.

strlen('ありがとう') = 15
preg_match_all('/./', 'ありがとう', $matches) = 15
preg_match_all('/(*UTF8)./u', 'ありがとう', $matches) = 5

You can also use Unicode character properties to match only letters (in any language) for example:

// The WRONG way to do it, only works for ASCII:
preg_match_all('/[a-zA-Z]/', $str, $matches);

// This way it works with any language:
preg_match_all('/(*UTF8)\p{L}/u', $str, $matches);

You can see other Unicode character properties in the PHP Manual.

Wednesday, April 25, 2012

Logging fatal PHP errors

If you turned off the display_errors setting in your php.ini in production (as you should), then when your code dies with a fatal error, you can't see the message anywhere. It would be better to log these errors to the Apache error log (this is true even if you didn't disable display_errors, for debugging errors that other users might report.) PHP has a log_errors directive in php.ini, but it doesn't seem to log anything for me. Instead, I used register_shutdown_function() to make PHP log the errors:

register_shutdown_function(function() {
  $error = error_get_last();
  if($error !== NULL) {
  error_log('PHP Fatal: file:' . $error['file'] . ' line:' . $error['line'] .
            ' type:' . $error['type'] . ' message:' . $error['message']);
  }
});

This causes PHP to log the error on shutdown.

Saturday, March 31, 2012

Compiling PHP 5.4 on Mac OS X Lion

PHP 5.4 came out, but Apple hasn't updated Mac OS Lion with it yet:

$ php --version
PHP 5.3.8 with Suhosin-Patch (cli) (built: Nov 15 2011 15:33:15) 
Copyright (c) 1997-2011 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2011 Zend Technologies

To compile PHP 5.4, I had to download and install libjpeg first. I downloaded it from http://www.ijg.org/files/, specifically the jpegsrc.v8d.tar.gz. Comiplation and installation were easy:

./configure --prefix=/usr/local/libjpeg8d
make
sudo make install

After that I configured PHP with the following options:

./configure \
  --prefix=/usr/local/php540 \
  --with-apxs2=/usr/sbin/apxs \
  --with-openssl \
  --with-pcre-regex \
  --with-zlib \
  --enable-bcmath \
  --with-bz2 \
  --enable-calendar \
  --with-curl \
  --enable-exif \
  --enable-ftp \
  --with-gd \
  --with-jpeg-dir=/usr/local/libjpeg8d \
  --with-png-dir=/usr/X11 \
  --with-freetype-dir=/usr/X11 \
  --enable-gd-native-ttf \
  --with-ldap \
  --with-ldap-sasl \
  --enable-mbstring \
  --with-mysql \
  --with-pdo-mysql \
  --with-libedit \
  --enable-pcntl \
  --enable-shmop \
  --with-snmp \
  --enable-soap \
  --enable-sockets \
  --enable-sysvmsg \
  --enable-sysvsem \
  --enable-sysvshm \
  --with-tidy \
  --enable-wddx \
  --with-xmlrpc \
  --with-xsl \
  --enable-zip

Followed by:

make
make test

Some tests failed, a few of which already had bugs created for them. I sent the failed test report to PHP so they can debug the rest of them. The CLI binary works:

$ sapi/cli/php --version
PHP 5.4.0 (cli) (built: Mar 31 2012 14:49:14) 
Copyright (c) 1997-2012 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2012 Zend Technologies

Then finally installed PHP with:

sudo make install

Monday, February 13, 2012

Compiling Ruby MRI on Mac OS X

EDIT: include libyaml

UPDATE 2012-05-30: I have successfully installed ruby-1.9.3-p194 (latest stable) using the same procedure.

Mac OS Snow Leopard comes with a pretty old Ruby:

$ ruby --version
ruby 1.8.7 (2010-01-10 patchlevel 249) [universal-darwin11.0]

You can download and compile the latest Ruby from source. You'll need to have XCode and libyaml installed first. Download libyaml from http://pyyaml.org/wiki/LibYAML and extract it in your Source directory:

$ cd Source
$ tar xzf ~/Downloads/yaml-0.1.4.tar.gz
$ cd yaml-0.1.4/
$ CC=clang ./configure --prefix=/usr/local/yaml-0.1.4
... output omitted ...
$ make
... output omitted ...
$ sudo make install
... output omitted ...

Download the latest Ruby from http://www.ruby-lang.org/en/downloads/ and then extract it in your Source directory:

$ cd Source/
$ tar xzf ~/Downloads/ruby-1.9.3-p0.tar.gz
$ cd ruby-1.9.3-p0/

Configure for compilation with clang and installation in /usr/local/:

$ CC=clang LDFLAGS=-L/usr/local/yaml-0.1.4/lib CPPFLAGS=-I/usr/local/yaml-0.1.4/include ./configure --prefix=/usr/local/ruby-1.9.3-p0
... output omitted ...

Compile:

$ make
... output omitted ...
$ make test
... output omitted ...
PASS all 943 tests
... output omitted ...
PASS all 1 tests

(I tried that with the latest stable, ruby-1.9.2-p290, and some tests failed there, so I didn't use it.)

Install:
$ sudo make install
... output omitted ...
$ /usr/local/ruby-1.9.3-p0/bin/ruby --version
ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-darwin11.2.0]

You can either always specify the path to it or you can add it to your PATH variable.