Unpack inconsistencies (possibly related to issue #605?)

[babs.devs@…]

I am attempting to use the Grit( http://github.com/mojombo/grit) gem (after various patches to work w/ macruby), but there seems to be some inconsistency between unpacking strings w ruby 1.9 and MacRuby (nightly as of 6/3/10).

Specifically with this method:

    def init_pack
          with_idx do |idx|
            @offsets = [0]
            FanOutCount.times do |i|
              //MacRuby vs Ruby assign VERY different values to pos
              pos = idx[i * IdxOffsetSize,IdxOffsetSize].unpack('N')[0]
              if pos < @offsets[i]
                raise PackFormatError, "pack #@name has discontinuous index #{i}"
              end
              @offsets << pos
            end
            @size = @offsets[-1]
          end
        end

In MacRuby the var 'pos' evaluates to [0, 4285812579] in every other block.

In Ruby 1.9 'pos' evaluates correctly (ex. after 7 blocks: [0, 11, 24, 39, 48, 56, 63])

This bug may be invalid, and/or related to the Grit gem, but at first glance appears to be MacRuby related.

@see also:  http://www.macruby.org/trac/ticket/605

[babs.devs@…]

Grit Gem

[babs.devs@…]

Pack (indx) File

[babs.devs@…]

Reduction/clarification of bug

[babs.devs@…]

The original bug-report is unrelated to actual issue (how do you invalidate a bug in Trac?).

This issue is apparently related to a MacRuby comparison bug. When == comparing a hex String value read from a file to a String representation of the same value (defined in a Constant), MacRuby evaluates to false, while Ruby 1.9 evaluates to true. I attached a reduction of the bug, and a pack-file to test against.

[macruby@…]

This might be down to the encodings, what encoding do you get the constant and packed string? The packed string I believe will be using the BINARY or ASCII-8BIT encoding, in order to compare the two strings you might need to use :force_encoding on the constant, provided its a Ruby string and not an NSString.

[babs.devs@…]

Replying to macruby@…:

This might be down to the encodings, what encoding do you get the constant and packed string? The packed string I believe will be using the BINARY or ASCII-8BIT encoding, in order to compare the two strings you might need to use :force_encoding on the constant, provided its a Ruby string and not an NSString.

Yep. The Constant was encoded as UTF-8 and the packed string was ASCII-8BIT. This alleviated the problem:

PACK_IDX_SIGNATURE.force_encoding(Encoding::ASCII_8BIT)

From what I've read, the encoding in Ruby 1.9 was the culprit here. Is this a common issue? :force_encoding seems a bit hackish, is there a "best practice" way to ensure consistent encoding in Ruby?

[vincent.isambart@…]

In this sample code the difference is that MacRuby handles all Ruby files as encoded in UTF-8, whereas Ruby 1.9 handles them as ASCII-8BIT by default. To have 1.9 behave as MacRuby, you can add "# coding: UTF-8" to the first line of the Ruby file.

Currently the only way to have it behave as you want is indeed to replace 'PACK_IDX_SIGNATURE = "\377tOc"' by 'PACK_IDX_SIGNATURE = "\377tOc".force_encoding(Encoding::BINARY)'. This might not be very clean but it will work in any implementation of Ruby 1.9.

Alternatively you can also do PACK_IDX_SIGNATURE = [0xff, 0x74, 0x4f, 0x63].pack('C*')

[mattaimonetti@…]

duplicate of #823 closing this ticket since I feel the other ticket is a bit less busy.

[mattaimonetti@…]

#823 is closed and I re investigated this ticket. The problem is that CRuby and MacRuby have 2 different default encodings. To get the reduction example to work you need to specify the encoding of the reference string as Vincent mentioned:

PACK_IDX_SIGNATURE = "\377tOc".force_encoding(Encoding::BINARY)

If you don't do that, you will try to compare the same string but with 2 different encodings: Encoding:ASCII-8BIT and Encoding:UTF-8.