Blog Overview
The MacRuby Blog
After 5 months of development, we are happy to announce the immediate availability of MacRuby 0.7.
This release does not bring any significant features but consolidates the existing functionality of MacRuby by improving its Ruby compatibility, concurrency, Cocoa support, and overall stability and performance.
You can download it from here. Please note that this package only runs on Snow Leopard (10.6). Older Mac OS X versions are no longer supported.
Please give it a try and let us know of any problems you find!
You can check out the complete release notes of this release, or continue with most visible changes here.
Cocoa Support
MacRuby’s previous release, 0.6, announced that support for Cocoa development was now stable. As a matter of fact, the Cocoa layer of MacRuby hasn’t changed much in this release, which confirms us that it is indeed stable.
A long-awaited enhancement to MacRuby’s Cocoa layer, support for C blocks, has been implemented in this release. You can now pass a Ruby Proc object to an API that expects a C block, similarly if this API expected a C function pointer. However, this new change requires additional BridgeSupport metadata which will be installed from our new, upcoming BridgeSupport release.
$ cat t.rb
framework 'Foundation'
a = [1, 2, 3, 4, 5]
a.enumerateObjectsUsingBlock(Proc.new { |obj, index, stop|
p obj
stop.assign(true) if index == 2
})
$ macruby t.rb
1
2
3
The sandbox facility is now exposed in MacRuby via the Sandbox class, and can now be used to restrict access to certain facilities in the current process and therefore limit potential damage that can happen if a vulnerability is exploited. It is a good companion to the Ruby standard $SAFE functionality, also supported in MacRuby, because it can constrain access to all system services used by the process, including by C extensions or Cocoa APIs, not just from pure-Ruby APIs.
Sandbox.no_internet # TCP/IP networking is prohibited.
Sandbox.no_network # All sockets-based networking is prohibited.
Sandbox.no_writes # File system writes are prohibited.
Sandbox.temporary_writes # File system writes are restricted to temporary folders.
Sandbox.pure_computation # All operating system services are prohibited.
Sandbox#apply! # Apply a given sandbox. Once a sandbox is applied,
# it can not be removed and no other sandbox
# can be applied on top.
In this release we also reduced the memory and disk footprint of the runtime.
Concurrency and Performance
The dispatcher has been rewritten to use a per-thread cache, which is a more efficient approach compared to the previous release. All reported race conditions and deadlocks have been fixed. We have been successfully running Sinatra applications under the ControlTower HTTP server in a concurrent fashion for the past weeks.
The MacRuby kernel, which contains the primitive functions called by the code generated by the compiler, is now pre-compiled into bitcode and included in the program’s module. Further optimization passes can now be applied and result in a significant runtime performance improvement.
A basic interpreter has been implemented and is now used to evaluate cold paths, determined at compilation time using simple heuristics. As an example, simple #eval statements creating code, a common pattern in Ruby libraries, now run significantly faster.
Further minor optimizations have been implemented, such as optimized code generation, faster instance variables access, and symmetric multiple assignments simplification.
Ruby Compatibility
A lot of work has been put in this release to increase the level of compatibility with existing Ruby programs and libraries. MacRuby 0.7 is the first release of MacRuby that targets the 1.9.2 version of Ruby, and therefore a number of new features have been re-implemented or backported.
This release passes an average of 90% of total RubySpecs, from about 82% for the previous release.
As always, Ruby compatibility is a work in progress, and MacRuby is still not able to run Rails out of the box. We believe that the next release of MacRuby will eventually implement the missing pieces.
We hope you enjoy this release and thank you for helping us converging MacRuby to a first stable release!
Way back in the announcement for MacRuby 0.5, we cryptically mentioned that we had gotten a web server to run a basic Sinatra app. While that statement was true, the server we had wasn’t even really alpha worthy. At that point, it was more about proof of concept. If you have been watching the subversion repository you might have noticed that a new top-level project appeared about 4 months ago. If you’re subscribed to the mailing list, you probably have even seen a bit of discussion take place about this project, and yet we still weren’t quite ready to announce it to the world. Well, that changes today!
Today we are officially announcing version 1.0 of ControlTower, a web server specifically designed for MacRuby.
If you’re a casual follower of MacRuby, here is what you need to know in a nut-shell:
- ControlTower is a web application server based on Rack, written from the ground for MacRuby.
- It is released under the Ruby license.
- It takes advantage of MacRuby’s concurrency facilities and ability to mix-n-match Cocoa APIs with Ruby, so it only runs on MacRuby.
- It can be used on its own or with a reverse proxy to serve any Rack-based web app whose libraries are supported by MacRuby (i.e. Sinatra, but not Rails…yet).
- We are also interested in other non-traditional ways an embeddable Ruby web app server might be useful for native app development.
- It’s small, it’s light weight, and it’s reasonably fast.
ControlTower 1.0 works on MacRuby 0.7, which is to be released soon, but can still be installed from nightly builds.
How do I use it?
The easiest way to grab the latest version of ControlTower is to use macgem:
$ sudo macgem install control_tower
The source code of the project is also available on our repository and ControlTower can also be installed from there.
There are two basic ways to interact with a Rack-based web server: create a rack-up file, or use a Rack::Handler in your code. Currently, both of these mechanisms work with ControlTower. Using a rack-up file is probably the simplest and most straight forward way to get up and running, and we’ve included a number of sample rack-up files in the “sample” directory of the repository. Once you have built and installed the ControlTower gem, you can use the control_tower command to run any rack-up config:
$ cat hello.ru
class Hello
def call(env)
[200, { 'Content-Type' => 'text/plain' }, "Hello, world! Your environment is #{env}"]
end
end
run Hello.new
$ control_tower -R hello.ru
You are cleared for take-off!
Listening on 0.0.0.0:3000
Alternatively, if you would like to start up the ControlTower server from within your code, you can use the Rack::Handler::ControlTower class included with the ControlTower gem. This class has a single run method that takes a rack “app” object and an options hash as its arguments. The options hash can be used to set the same server parameters as the command line tool. So, for example you can do:
app = MyApp.new Rack::Handle::ControlTower.run(app, {:port => 80, :host => 'myserver.com', :concurrent => true })
Though, keep in mind that this method will block untill the server stops running, so if you want to do something else while the server is running you should put this code in a thread.
What’s in a Web Server?
In general, all Ruby web app servers consist of the same three components: a socket manager, a header parser, and a request handler. Incoming requests initially encounter the socket manager which begins receiving data from the incoming connection. This data is then fed to the header parser until all of the headers have been received. These headers, along with any request body data (e.g. a form being POSTed), are then passed on to the request handler. The request handler makes any necessary changes on the server, gathers any data needed for a reply, and then sends that reply back to the socket manager which will send the reply back to the client.
Ever since Mongrel, just about every Ruby web app server has used Zed Shaw’s Ragel-based HTTP parser to parse headers on incoming requests, and ControlTower does too. Since the appearance of Rack, servers can count on an “app” object provided by a web app framework to handle the requests, in accordance with the Rack spec, and being a Rack-based server ControlTower relies on this as well. That leaves the socket manager as the only one of the three components that will differ considerably between servers. In the case of Thin the socket manager is an EventMachine event loop. In the case of Unicorn sockets are managed using a prefork mechanism. In ControlTower, we do our socket management with Grand Central Dispatch.
The way this works is that each new incoming request is asynchronously dispatched to a GCD queue to be parsed and handled. In the default case, this queue is a serial queue, so requests are handled first-in first-out. By placing incoming requests onto a queue immediately, the server is quickly able to accept the next incoming request. If the -c switch is used with the control_tower command or the :concurrent key is set to true in the handler’s options hash, ControlTower will use a concurrent GCD queue in place of the serial one. This means that ControlTower can handle multiple requests concurrently inside a single process; up to 75 at once on an 8-core MacPro (compared to 20 simultaneous requests per Thin process in threaded mode). If you want to give concurrent mode a try, just make sure that your web app and database layer are truly thread-safe, and not just green-thread-safe.
How can I help?
Glad you asked! ControlTower is still a relatively young project, so there is still lots to do. Right now, it is a small, robust server capable of handling moderate loads, but it doesn’t have as many features as other Rack-based servers like Thin or Unicorn. Most notably, we do not yet support keep-alive. Other wish-list items include the addition of Rack adapters for the various common Ruby web app frameworks, support for unix domain sockets, and the ability to do Web Sockets.
Of course, you don’t have to be heavy into server development to lend a hand. We could use a lot of help just with testing and documentation (both of which are lacking currently). Perhaps the simplest and most important thing you can do to help is to try it out and send in your bug reports and feature requests. We started building ControlTower for our own purposes, but now we’re ready to share it with the community and see where you take it.
Feel also free to open new discussions about ControlTower on our mailing-list.
We are pleased to announce the immediate availability of MacRuby 0.6. In the 3 months since the previous release, a number of new features have been added and the overall stability of MacRuby has been substantially improved.
You can download it from here. Please note that this package only runs on Snow Leopard (10.6). Users on Leopard (10.5) can also use MacRuby by building the sources from our repository.
Please give it a try and let us know of any problems you find.
You can read about all the changes from the release notes e-mail on the mailing-list, or keep up with the most visible changes here.
Stable for Cocoa Development
In this release, we believe that MacRuby is now stable enough to consider using it to develop Cocoa applications.
Since 0.5’s release, we have worked closely with a number of early-adopter developers in finding and fixing a great number of bugs, as well as improving the overall process of creating Cocoa apps in MacRuby. We believe that MacRuby is now stable enough to permit the creation of complete and functional Cocoa applications that have access to the full suite of Cocoa APIs.
The “Compile” target in Xcode allows an application to be ahead-of-time compiled to machine code, allowing the developer to avoid including the Ruby source in a shipping application.
Debugging
Experimental support for debugging landed in this release.
The compiler, under debug mode, is now generating special traps inside the code. The debugger connects to these traps and allows not only basic debugging operations but also code evaluation.
The macrubyd command-line executable pilots the debugger. It provides a gdb-like experience. Please note that it is not entirely finished and it also has not been exhaustively tested.
Let’s consider the following broken program:
$ cat t.rb def foo(n) bar(n) end 5.times do |i| foo(i) end
And start it inside the debugger.
$ macrubyd t.rb Starting program. 1 def foo(n) t.rb:1> c /Users/lrz/src/t.rb:2:in `foo:': undefined method `bar' for main:TopLevel (NoMethodError) from /Users/lrz/src/t.rb:5:in `block' from /Users/lrz/src/t.rb:4:in `<main>' Program exited.
We immediately get an exception at line 2. We can restart the debugger, break at this line, then examine the argument and the backtrace.
> r Starting program. 1 def foo(n) t.rb:1> b t.rb:2 Added breakpoint 1. 1 def foo(n) t.rb:1> c 2 bar(n) t.rb:2> p n => 0 2 bar(n) t.rb:2> bt #0 /Users/lrz/src/t.rb:2:in `foo:' #1 /Users/lrz/src/t.rb:5:in `block' #2 /Users/lrz/src/t.rb:4:in `<main>' 2 bar(n)
Then, we can define the missing method on the fly and resume the execution of the program after disabling the breakpoint.
t.rb:2> p def bar(x); puts "-> #{x}"; end
=> nil
2 bar(n)
t.rb:2> disable 1
Disabled breakpoint 1.
2 bar(n)
t.rb:2> c
-> 0
-> 1
-> 2
-> 3
-> 4
Program exited.
An interesting feature of the debugger is that it has been abstracted into a simple Objective-C API, of which macrubyd is just one client. In the future we might see other clients.
Higher-Level APIs for Grand Central Dispatch
GCD is a revolutionary approach to multicore computing that is woven throughout the fabric of Mac OS X version 10.6 Snow Leopard. It is supported in MacRuby since version 0.5, through the Dispatch module.
In MacRuby 0.6, we improved the existing native bindings with additional, higher-level abstractions and convenience APIs in the “dispatch” library. We believe these features will help reduce the learning curve for GCD.
The new “dispatch” library exposes the Job class, which is likely the easiest way to perform concurrent work. Say you have a complex, long-running calculation you want to happen in the background. Create a job by passing Dispatch::Job’s initializer the block you want to execute:
$ /usr/local/bin/macirb --simple-prompt
>> require 'dispatch'
=> true
>> job = Dispatch::Job.new { Math.sqrt(10**100) }
=> #<Dispatch::Job:0x40075d1e0 @queue=com.apple.root.default-priority @group=#<Dispatch::Group:0x40073c2c0> @values=[1.0e+50]>
This atomically adds the block to GCD’s default concurrent queue, then returns immediately, so you don’t stall the main thread.
The downside is that you don’t know exactly when your job will execute. The #value method can be used to obtain the result of executing that block.
>> value = job.value => 1.0e+50
This will wait until the value has been calculated, allowing it to be used as an explicit Future. However, this may stall the main thread indefinitely, which reduces the benefits of concurrency.
There are much more convenience APIs in the “dispatch” library, such as the Proxy class or concurrent extensions to the builtin Enumerable module. Check out the tutorial for more information.
Solid Foundations
We have rewritten much of the foundation layer of MacRuby in this release, with the goal of increasing the overall robustness of the system and providing a better platform for future enhancements.
The Hash class, which used to be an alias to NSMutableDictionary, is now a new class that inherits from the latter. It can more efficiently handle immediate types (such as fixnums and floats) and honors insertion ordering.
The String class has also been changed. It is now a fresh new implementation that can handle both character and byte strings. It also uses the ICU framework to perform encoding conversions on the fly. This new class inherits from NSMutableString. Symbol was also rewritten to handle multibyte (Unicode) characters.
Finally, the Regexp class has been totally rewritten in this release. It is now using the ICU framework instead of Oniguruma for regular expression compilation and pattern matching. Since ICU is thread-safe, MacRuby 0.6 allows multiple threads to utilize regular expressions in a very efficient way, which was not possible previously.
All these changes were designed and implemented to honor compatibility with previous releases of MacRuby. They should not be directly visible.
Better Ruby Compatibility
Ruby compatibility is still progressing but is now much better than in 0.5.
MacRuby 0.6 provides support for C extensions written for the original implementation of ruby. We were able to successfully use the Nokogiri, SQLite3 and PostgreSQL extensions from MacRuby.
This release also passes about 85% of RubySpecs, is able to run a modified version of Rails 3, and implements better support for Ruby 1.9 encodings.
There are still several problems to address in order to provide a full-fidelity replacement for all Ruby features. We intend to continue working on this by focusing on RubySpecs and Rails.
We hope you enjoy the many improvements in this release!
Development on the next release, 0.7 just started. In that release, we intend to deliver a next generation compiler and virtual machine that provides very good runtime performance for multicore environments. Stay tuned for more updates!