The MacRuby Blog

After several months of development we are very glad to announce the first beta release of MacRuby 0.5. As this release has very ambitious goals we will go through a few intermediate releases before shipping the final 0.5.

You can download it from here. Please note that this package is for Intel machines running Mac OS X v10.6 or higher.

Remember that this is a development version of MacRuby provided for testing and experimentation purposes only. Even if a few things are quite stable, others are still under development. Please give it a try and report us any problem you find, to make sure the final release will be great.

A lot of progress has been made in this version. Here is a summary of what’s new.

Compilation

MacRuby was based on YARV, the original Ruby 1.9 virtual machine. Two things worried us about YARV: the lack of machine code compilation and the global interpreter lock (GIL) preventing true concurrency. Since concurrency and machine code compilation are so critical to performance, we decided to write a replacement, a very ambitious target.

The new MacRuby 0.5 runtime is built upon LLVM, a compiler infrastructure also sponsored by Apple. Thanks to LLVM, MacRuby is able to transform the Ruby abstract syntax tree (AST) from the parser directly into highly optimized machine code. MacRuby supports both Just in Time (JIT) and Ahead of Time (AOT) compilation. The JIT mode will compile down the code at runtime, and the AOT mode will allow you to save on disk the compilation result. AOT compilation makes MacRuby a true Ruby compiler.

MacRuby ships with the macrubyc command line utility, which is an interface to the AOT compiler. It can produce a Mach-O compatible object file (.o) out of a given Ruby source file. You can also assemble multiple files to get a standalone executable binary ready for deployment. This can be used to compile a MacRuby Cocoa application, dramatically improving its startup time while making sure the original Ruby source code is not readable.

$ echo "p ARGV.map { |x| x.to_i }.inject(0) { |x, y| x + y }" > t.rb
$ macrubyc t.rb -o t
$ file t
t: Mach-O 64-bit executable x86_64
$ ./t 1 2 3 4 5
15

We also introduced a mode that generates a dynamic shared library out of a Ruby source file, using the new .rbo extension. MacRuby at runtime will #require these .rbo files in priority. We pre-compiled parts of the standard library using this technique.

Concurrency

The new MacRuby 0.5 runtime is designed to support concurrent execution, allowing a MacRuby program to use more than one CPU core at a time and multiple threads to call into MacRuby at the same time. This new design is critical as we enter the multicore age.

MacRuby uses native POSIX threads to represent the Thread class. Every native thread has its own VM instance, which allows multiple threads to dispatch code without interrupting others. The MacRuby garbage collector is now able to finalize objects on separate threads, preventing pauses in execution during GC.

MacRuby ships with an API for Grand Central Dispatch (GCD), a multicore computing functionality present in Mac OS 10.6, Snow Leopard. The significant advantage of GCD is not having to deal with threading directly. With our new GCD API you set up queues and dispatch Ruby blocks to them. GCD will handle executing those blocks on system threads, ensuring all your CPU cores are used.

waiting_chairs = Dispatch::Queue.new('com.apple.waiting_chairs')
semaphore = Dispatch::Semaphore.new(3)
index = -1
while true
  index += 1
  if semaphore.wait(Dispatch::TIME_NOW) != 0
    puts "Customer turned away #{index}"
    next
  end
  waiting_chairs.dispatch do
    semaphore.signal
    puts "Shave and a haircut #{index}"
  end
end

Optimizations

MacRuby 0.5 includes significant runtime-level optimizations, such as a much faster method dispatcher, cheap local variables, fast instance variable access, zero-cost DWARF exceptions, immediate floating point types, and many more.

At the core classes level, we introduced a ByteString class to properly handle raw data, rewrote the IO layer for better stability and performance, and introduced a new implementation of Array, optimized for handling Ruby immediate types.

Most of these optimizations were implemented after profiling a large MacRuby application.

Cocoa Development

We re-implemented the C and Objective-C support of 0.4 in this new release. An application written for 0.4 should continue to work on 0.5, but you should see it execute faster.

In 0.4 we used to use the libffi library to call C and Objective-C functions and to create closures. RubyCocoa is also based on libffi. While libffi is a great library to use for external dispatch, it has definite performance limitations. In 0.5 MacRuby generates stubs using LLVM at runtime, enabling much better performance (from 3 to 4 times faster).

Compatibility

We want MacRuby to be as compatible as possible with existing Ruby programs. We have been working hard on MacRuby to make sure it behaves like MRI 1.9.×.

Like most other Ruby implementations, we are now using the RubySpec project to check our compatibility. RubySpec is a set of executable specifications about the Ruby language and is maintained by the community.

At the time of this writing, MacRuby passes about 91% of the language specs, 80% of the core specs and 72% of the library specs. Our objective is to pass as many RubySpecs as possible.

Thanks to a better compatibility, MacRuby is able to run complex Ruby programs like irb, rake, rubygems, and more. Full compatibility is still a work in progress, but we expect MacRuby to run many more Ruby programs in the near future as we fix bugs and implement missing features.

As you might have noticed, MacRuby 0.4 has just been released a few days ago. It’s now time to focus on the next release, 0.5, which will be principally dedicated to performance.

New Virtual Machine

A new virtual machine based on the LLVM compiler infrastructure has been developed and integrated as a replacement for YARV.

The new VM transforms the abstract source tree (AST) directly from the parser into LLVM’s intermediate representation (IR), which is then compiled into optimized machine code. A number of other key optimizations have also been completed, such as fast fixnum arithmetic, cached method dispatch, zero-cost IA64 exceptions, fast instance variable access, constants caching, fast break/next, cheap local variables, tail call optimization and much more.

While the amount of work remaining is significant, the compiler is already able to compile most of the Ruby syntax and the new runtime is mostly complete. Early performance measurements are also very promising, with substantial room for improvement remaining.

New IO Subsystem

A new IO implementation based on CoreFoundation streams is under active development. The objective of this work is to provide a cleaner and faster implementation of the various IO operations; additionally, we will be able to perform asynchronous operations through the principal run loop.

A ByteString class which silently inherits from String has been introduced and is now used as the return value of all IO operations. Its backing store is a CFMutableDataRef, which integrates better with the new IO subsystem and offers substantially better performance.

RubySpec Integration

Performance without compatibility is, of course, meaningless. We have therefore integrated the RubySpec project and are aiming to pass all the specifications.

The current experimental branch is already able to pass most of the language specifications and will be a standard part of our regression test suite. We are also actively working on upgrading the specifications to match the Ruby 1.9 behaviors.

Want to Help?

If you want to help or simply give the new changes a try, everything is on the experimental SVN branch.

$ svn co http://svn.macosforge.org/repository/ruby/MacRuby/branches/experimental macruby-experimental

Please make sure to read the README file for information regarding requirements, dependencies and build instructions. Do not hesitate to contact us if you need more info!

After several months of development and some slight delays, MacRuby 0.4 is now available. Get it here while it’s still hot!

This is quite an important release that brings new features and fixes several problems. The changes are too numerous to be all mentioned, so here is a selection of the most interesting ones.

Threaded GC

The MacRuby garbage collector is now running in multi-threaded mode by default. That means that MacRuby will always do garbage collections on a separate thread and therefore not interrupt the program’s flow.

Full 64-bit Support

MacRuby is now fully working in both Intel 32-bit and 64-bit modes. If you are running a recent Mac chances are that it is 64-bit and MacRuby will run faster on it. This is mainly due to the fact that the underlying infrastructure has been significantly improved for 64-bit processors.

DTrace Probes

DTrace probes have been added to the core of the interpreter. You can now trace various things such as method calls or exceptions. All of that can be done on any MacRuby processes in the system.

provider macruby {
    probe insn__entry(char *insnname, char *sourcefile, int sourceline);
    probe insn__return(char *insnname, char *sourcefile, int sourceline);
    probe method__entry(char *classname, char *methodname, char *sourcefile, int sourceline);
    probe method__return(char *classname, char *methodname, char *sourcefile, int sourceline);
    probe raise(char *classname, char *sourcefile, int sourceline);
    probe rescue(char *sourcefile, int sourceline);
};

DTrace is a very powerful tool that has proven to be extremely useful when debugging live applications. MacRuby ships with several DTrace scripts in /Developer/Examples/Ruby/MacRuby/DTrace to profile various things such as method count, duration or even objects collected by the GC.

Runtime Objective-C API

MacRuby now exposes an Objective-C API that can be used to control the runtime from a pure Cocoa environment.

$ cat hello_macruby.m 
#import <Foundation/Foundation.h>
#import <MacRuby/MacRuby.h>

int main(void) {
    id proc = [[MacRuby sharedRuntime] evaluateString:@"proc { |x| puts \"hello #{x}\"}"];
    [proc performRubySelector:@selector(call:) withArguments:@"MacRuby"];
    return 0;
}
$ gcc hello_macruby.m -o hello_macruby -framework Foundation -framework MacRuby -fobjc-gc
$ ./hello_macruby 
hello MacRuby

It should be helpful if you are working on an Objective-C Cocoa application and are considering using MacRuby to either implement new functionalities or to provide a scripting interface to your native objects.

New Xcode Templates

A “MacRuby Core Data Application” template is now available, as well as an “Embed MacRuby” target. The latter can be used to embed MacRuby.framework inside your application bundle. Embedding the framework allows you to distribute your application to users and not require them to install MacRuby.

A Better HotCocoa

HotCocoa is a thin, idiomatic Ruby layer that sits above Cocoa and other frameworks. It was introduced in MacRuby 0.3 and it has been significantly improved in 0.4.

New mappings for XML parser, KVO array/set accessors, property lists and more AppKit components were added as well as lots of bug fixes and improvements.

A “deploy” task was added to the project’s Rakefile. This new task prepares an application for deployment by embedding the MacRuby runtime inside its bundle (similar to what the Xcode’s Embed MacRuby target does). Once the “macrake deploy” task is complete, you can share your .app with friends who do not have MacRuby installed on their machine.

But the most interesting change is probably the new graphics layer, called HotCocoa::Graphics. It provides a simple object-oriented interface into the power of Mac OS X’s Core Graphics and Core Image drawing libraries.

Misc Changes

Over the list of minor changes, Set has been reimplemented on top of NSSet and NSNumber implements the Numeric methods. Finally, the standard library was updated to Ruby 1.9.1’s version.

Caveat Lector

While MacRuby 0.4 turns out to be pretty stable when it comes to Cocoa development, there are still a few problems with C extensions, RubyGems and Ruby IOs. We recommend to use the corresponding Cocoa APIs (when they exist) in the meantime we work on these issues.

We hope that you will enjoy this release. Please let us know if you find any problems.

Stay tuned for more details about the next release: 0.5, which will be focusing on performance.