Monday, February 14, 2011

Making an executable Ruby archive distribution

As a follow-up to my previous post, you can even make a single executable file of your Ruby archive. It relies on the fact that zip files are pretty resilient to extra garbage data in the beginning. Let's say I have two files, lib/greetings.rb and bin/hello.rb. greetings.rb defines a make_hello() function that is used by hello.rb. Here's hello.rb:

require "lib/greetings"

puts "Enter your name:"
name = gets.strip

puts "\n" + make_hello(name)

You can run it like this:

$ ruby bin/hello.rb 
Enter your name:
Steve

Hello, Steve!
$

First, we make a zip file that includes all the required source files:

$ zip -r hello.zip bin lib
  adding: bin/ (stored 0%)
  adding: bin/hello.rb (deflated 11%)
  adding: lib/ (stored 0%)
  adding: lib/greetings.rb (deflated 7%)
$ unzip -l hello.zip 
Archive:  hello.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  02-14-11 21:01   bin/
       98  02-14-11 21:01   bin/hello.rb
        0  02-14-11 20:59   lib/
       46  02-14-11 20:59   lib/greetings.rb
 --------                   -------
      144                   4 files

Now we need a ruby header for the zip file, which we'll put into header.rb:

#!/usr/bin/ruby -rubygems -x

require "zip/ziprequire"
$:.push $0
require "bin/hello.rb"
__END__

It runs the ruby interpreter, which then loads zip/ziprequire, adds itself (which will be the zip file) to the path, and then requires our main file, bin/hello.rb. We now prepend this header to the zip file using cat header.rb hello.zip > hello_tmp.zip. If you now try to run unzip -l on this file, you'll get:

$ unzip -l hello_tmp.zip 
Archive:  hello_tmp.zip
warning [hello_tmp.zip]:  97 extra bytes at beginning or within zipfile
  (attempting to process anyway)
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  02-14-11 21:01   bin/
       98  02-14-11 21:01   bin/hello.rb
        0  02-14-11 20:59   lib/
       46  02-14-11 20:59   lib/greetings.rb
 --------                   -------
      144                   4 files

We need to fix this zip file to make it valid. Thankfully zip has an option for this:


$ zip --fix hello_tmp.zip --out hellox.zip
Fix archive (-F) - assume mostly intact archive
Zip entry offsets appear off by 97 bytes - correcting...
 copying: bin/
 copying: bin/hello.rb
 copying: lib/
 copying: lib/greetings.rb

We now have a valid zip archive in hellox.zip, we just need to make it executable by running chmod +x hellox.zip. Now your whole application is a single executable file that you can run:


$ ./hellox.zip 
Enter your name:
Mike

Hello, Mike!

Tuesday, February 8, 2011

Distributing a Ruby application as an archive

Let's say you have a Ruby application you've written, and it consists of multiple files that you require inside your code. You want to run this application on some remote machines. To make it easier to deploy this application, you want to distribute it as a single file (archive.) This is possible with the rubyzip gem. Let's say your main application file (myapp.rb) looks like this:

require "lib/mylib"
require "lib/otherlib"
require "vendorlib/something"

# Do stuff.

Normally you might run your application with ruby ./myapp.rb. To run it on another machine, we can use the zip/ziprequire library in rubyzip. First, make a zip file containing all your application files:

zip -r myapp.zip myapp.rb lib vendorlib

Copy myapp.zip to the remote machine, and you can run it like this:

ruby -rubygems -Imyapp.zip -e 'require "zip/ziprequire"' -e 'require "myapp"'

See rubyzip documentation, specifically ziprequire.rb for more information.

Tuesday, January 11, 2011

Using FileMerge with Mercurial

Mac OS comes with a GUI merge tool called FileMerge. This can be used for merges in Mercurial - Mercurial will do this if its internal merge fails. The binary for FileMerge (opendiff) cannot be used as is, so we need to create a small shell script. You can put this script anywhere in your path, and call it hopendiff:

#!/bin/sh
`/usr/bin/opendiff "$@"`

Then modify your ~/.hgrc and add:

[extensions]
hgext.extdiff =

[extdiff]
cmd.opendiff = hopendiff

[merge-tools]
filemerge.executable = hopendiff
filemerge.args = $local $other -ancestor $base -merge $output 

If you already have some of those sections (like [extensions]), then just add the corresponding lines to those sections.

That's it. When you do a merge in Mercurial, it'll open FileMerge if the internal merge fails. You can also use FileMerge for normal diffs by using hg opendiff instead of hg diff.

Thursday, January 6, 2011

Binary and character set safe MySQL dump and restore

To dump some MySQL databases in a binary-safe way:

mysqldump -uroot -p -hdbhost \ 
  --skip-extended-insert \ 
  --default-character-set=binary \ 
  --databases dbone dbtwo dbthree \ 
  --add-drop-database --master-data=2 | gzip > /tmp/dump.sql.gz

Replace --master-data=2 with --lock-all-tables if dumping from a slave, and record the output of SHOW SLAVE STATUS while everything is locked.

To import this dump:

zcat /tmp/dump.sql.gz | mysql -uroot -p -hotherdbhost \ 
   --unbuffered --batch --default-character-set=binary

To set up replication:

Use the Relay_Master_Log_File:Exec_Master_Log_Pos values from SHOW SLAVE STATUS that you recorded. See this page for explanation:

http://www.mysqlperformanceblog.com/2008/07/07/how-show-slave-status-relates-to-change-master-to/

For example:

CHANGE MASTER TO MASTER_HOST = 'abc123.dklfjalddkj.com',
  MASTER_USER = 'rep', MASTER_PASSWORD = 'CHANGEME',
  MASTER_LOG_FILE = 'abc123-bin.003292',
  MASTER_LOG_POS = 283072761;