"sudo rake install" fails with ArgumentError

[dmitry@…]

sudo rake install fails on one of my machines:

./miniruby instruby.rb --make="/usr/bin/make" --dest-dir="" --extout=".ext" --mflags="" 
--make-flags="" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list 
--mantype="doc" --sym-dest-dir="/usr/local"
given string has unrecognized encoding (ArgumentError)
rake aborted!
Command failed with status (1): [./miniruby instruby.rb --make="/usr/bin/ma...]

I've tracked the problem down to val.gsub!(/\$\$|\$\(([^()]+)\)|\$\{([^{}]+)\}/) do in rbconfig.rb, where it throws the exception on the first argument ( val = "$(datarootdir)/doc/$(PACKAGE)" ).

MacRuby-trunk svn revision 2359 Mac OS X 10.5.8 rake, version 0.8.7 MacBookPro2,1

On another machine (MacBook Air) it doesn't fail and installs OK.

Am I doing something wrong? Please let me know if you need more information.

[mattaimonetti@…]

This should be fixed now.

[dmitry@…]

Unfortunately, SVN revision 2404 gives the same error.

Log with "--trace":  http://gist.github.com/176995

[lsansonetti@…]

Re-opening. This is a very strange error indeed, just curious, what's your environment variables?

$ env

[dmitry@…]

env:

TERM_PROGRAM=Apple_Terminal
TERM=xterm-color
SHELL=/bin/bash
CLICOLOR=1
TMPDIR=/var/folders/GT/GT8bzdyOGL8mePa+2N0it++++TQ/-Tmp-/
Apple_PubSub_Socket_Render=/tmp/launch-BqaJLu/Render
TERM_PROGRAM_VERSION=272
USER=dmitry
COMMAND_MODE=unix2003
SSH_AUTH_SOCK=/tmp/launch-macYTP/Listeners
__CF_USER_TEXT_ENCODING=0x1F7:0:0
PATH=/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/opt/local/bin:/usr/local/git/bin
PWD=/Users/dmitry
LANG=ru_RU.UTF-8
PS1=\[\]\w\[\]$(parse_hg_branch) \[\]\[\033]0;\w\007\]\[\]$EXITCODE\[\]\[\033]0;\w\007\]\[\]$\[\] 
SHLVL=1
HOME=/Users/dmitry
LOGNAME=dmitry
DISPLAY=/tmp/launch-Jtl0d1/:0
_GOOGLE_GILD_RUNTIME_FRAMEWORK_ABSOLUTE_PATH_=/Library/Google/Frameworks/Gild.framework
_=/usr/bin/env

Also tried with default $PATH, clean .bash_profile, and LANG=en-US.UTF8 with no luck.

[dmitry@…]

BTW, rake and ruby are from default installation:

~ $ which rake
/usr/bin/rake
~ $ which ruby
/usr/bin/ruby

[dmitry@…]

I inserted "p CONFIG" before "CONFIG.each_value ... " in rbconfig.rb. Here's the output.

{"MINOR"=>"9", "DLDLIBS"=>"", "PACKAGE_BUGREPORT"=>"", "LIBRUBYARG"=>"$(LIBRUBYARG_SHARED)",
"NROFF"=>"/usr/bin/nroff", "CP"=>"cp", "LN_S"=>"ln -s", "sysconfdir"=>"$(prefix)/etc", "LIBS"=>"",
"INSTALL_SCRIPT"=>"$(INSTALL)", "ENABLE_SHARED"=>"yes", "arch"=>"universal-darwin10.0",
"datadir"=>"$(datarootdir)", "bindir"=>"$(exec_prefix)/bin", "host_os"=>"darwin9.0.0",
"host_vendor"=>"apple", "ECHO_C"=>"\\\\c", "rubyhdrdir"=>"$(includedir)/ruby-$(MAJOR).$(MINOR).$(TEENY)",
"WINDRES"=>"", "libexecdir"=>"$(exec_prefix)/libexec", "LIBPATHENV"=>"DYLD_LIBRARY_PATH",
"INSTALL"=>"/usr/bin/install -c", "host_cpu"=>"i686", "vendordir"=>"$(prefix)/lib/ruby/vendor_ruby",
"SHELL"=>"/bin/sh", "EXTOUT"=>".ext", "target_cpu"=>"i686", "configure_args"=>"", "target_vendor"=>"apple",
"htmldir"=>"$(docdir)", "psdir"=>"$(docdir)", "build_alias"=>"", "ARCHFILE"=>"", "warnflags"=>"-Wall
-Wno-parentheses", "PACKAGE_VERSION"=>"", "RANLIB"=>"ranlib", "cflags"=>"$(optflags) $(debugflags)
$(warnflags)", "sitehdrdir"=>"$(rubyhdrdir)/site_ruby", "INSTALL_DATA"=>"$(INSTALL) -m 644",
"exec_prefix"=>"$(prefix)", "NM"=>"", "THREAD_MODEL"=>"pthread", "EXTSTATIC"=>"", "SOLIBS"=>"",
"prefix"=>"/Library/Frameworks/MacRuby.framework/Versions/0.5/usr", "sitedir"=>"$(libdir)/ruby/site_ruby",
"build_cpu"=>"i686", "MANTYPE"=>"doc", "CPPFLAGS"=>"$(cppflags)", "DLDFLAGS"=>"", "TRY_LINK"=>"",
"LIBRUBY_LDSHARED"=>"/usr/bin/gcc -dynamiclib -undefined suppress -flat_namespace", "COMMON_LIBS"=>"",
"EXPORT_PREFIX"=>"", "CCDLFLAGS"=>"-fno-common", "STATIC"=>"", "topdir"=>".", "cppflags"=>"",
"RUBY_INSTALL_NAME"=>"macruby", "GNU_LD"=>"no", "pdfdir"=>"$(docdir)", "LIBRUBY"=>"$(LIBRUBY_SO)",
"DLEXT"=>"bundle", "rubyw_install_name"=>"", "LIBEXT"=>"a", "vendorhdrdir"=>"$(rubyhdrdir)/vendor_ruby",
"rubylibdir"=>"$(libdir)/ruby/$(ruby_version)", "LINK_SO"=>"", "host"=>"i686-apple-darwin9.0.0",
"LIBRUBY_ALIASES"=>"lib$(RUBY_SO_NAME).$(MAJOR).$(MINOR).dylib lib$(RUBY_SO_NAME).dylib",
"LIBRUBYARG_SHARED"=>"-l$(RUBY_SO_NAME)", "COMMON_HEADERS"=>"", "BASERUBY"=>"ruby", "ECHO_T"=>"",
"archdir"=>"$(rubylibdir)/$(arch)", "sharedstatedir"=>"$(prefix)/com", "PREP"=>"miniruby$(EXEEXT)",
"localstatedir"=>"$(prefix)/var", "LIBRUBY_SO"=>"lib$(RUBY_SO_NAME).$(MAJOR).$(MINOR).$(TEENY).dylib",
"vendorarchdir"=>"$(vendorlibdir)/$(sitearch)", "LDSHARED"=>"$(CC) -dynamic -bundle -undefined suppress
-flat_namespace -arch i386 -arch x86_64", "ruby_version"=>"$(MAJOR).$(MINOR).$(TEENY)", "DLEXT2"=>"",
"ALLOCA"=>"", "EXEEXT"=>"", "sitelibdir"=>"$(sitedir)/$(ruby_version)", "ENABLE_STATIC"=>"no",
"optflags"=>"-O3", "DLLWRAP"=>"", "OUTFLAG"=>"-o ", "SET_MAKE"=>"", "sbindir"=>"$(exec_prefix)/sbin",
"datarootdir"=>"$(prefix)/share", "PACKAGE_TARNAME"=>"", "build_os"=>"darwin9.0.0", "PACKAGE_NAME"=>"",
"oldincludedir"=>"/usr/include", "CFLAGS"=>"-fno-common -pipe $(cflags)", "target_alias"=>"",
"MAKEFILES"=>"Makefile", "STRIP"=>"strip -A -n", "includedir"=>"$(prefix)/include", "TEENY"=>"0",
"target_os"=>"darwin9.0", "LIBRUBY_A"=>"lib$(RUBY_SO_NAME)-static.a", "MAINLIBS"=>"",
"RUBYW_INSTALL_NAME"=>"", "CXXFLAGS"=>"", "DESTDIR"=>"", "OBJEXT"=>"o",
"LIBRUBYARG_STATIC"=>"-l$(RUBY_SO_NAME)", "debugflags"=>"-g", "LDFLAGS"=>"",
"build"=>"i686-apple-darwin9.0.0", "setup"=>"Setup", "RM"=>"rm -f", "ruby_install_name"=>"macruby",
"CC"=>"/usr/bin/gcc", "LIBRUBY_DLDFLAGS"=>"-install_name $(libdir)/lib$(RUBY_SO_NAME).dylib
-current_version $(MAJOR).$(MINOR).$(TEENY) -compatibility_version $(MAJOR).$(MINOR)",
"infodir"=>"$(datarootdir)/info", "CPPOUTFILE"=>"-o conftest.i", "MAKEDIRS"=>"mkdir -p",
"CPP"=>"/usr/bin/gcc -E", "build_vendor"=>"apple", "CXX"=>"/usr/bin/g++", "ARCH_FLAG"=>"-arch i386 -arch
x86_64", "ASFLAGS"=>"", "LDSHAREDXX"=>"$(CXX) -dynamic -bundle -undefined suppress -flat_namespace",
"INSTALL_PROGRAM"=>"$(INSTALL)", "vendorlibdir"=>"$(vendordir)/$(ruby_version)", "COMMON_MACROS"=>"",
"OBJDUMP"=>"", "AR"=>"ar", "AS"=>"as", "EGREP"=>"/usr/bin/grep -E", "dvidir"=>"$(docdir)",
"target"=>"i686-apple-darwin9.0.0", "GREP"=>"/usr/bin/grep", "MAJOR"=>"1", "RDOCTARGET"=>"install-doc",
"PATH_SEPARATOR"=>":", "RUBY_SO_NAME"=>"macruby", "DEFS"=>"", "localedir"=>"$(datarootdir)/locale",
"libdir"=>"$(exec_prefix)/lib", "mandir"=>"$(datarootdir)/man", "docdir"=>"$(datarootdir)/doc/$(PACKAGE)",
"sitearch"=>"universal-darwin10.0", "sitearchdir"=>"$(sitelibdir)/$(sitearch)", "PACKAGE_STRING"=>"",
"LIBPATHFLAG"=>" -L%s", "RPATHFLAG"=>"", "host_alias"=>"", "ECHO_N"=>"", "COUTFLAG"=>"-o "}

Then "p val" in RbConfig::expand (right after the declaration and before val.gsub!), and one more log string in gsub! block, and ran "sudo rake install"

"9"
given string has unrecognized encoding (ArgumentError)

Also I tried creating regexp earlier with Regexp.new (and then inserting log string after that), and it created successfuly.

reg = Regexp.new('\$\$|\$\(([^()]+)\)|\$\{([^{}]+)\}'.force_encoding("ASCII-8BIT"))
p reg
p "--works--"
val.gsub!(reg) do
...

Output:

/\$\$|\$\(([^()]+)\)|\$\{([^{}]+)\}/
"--works--"
given string has unrecognized encoding (ArgumentError)

So the problem still seems to be with gsub!.

Let me know if I can help with debugging anything else.

[lsansonetti@…]

I just improved in r2430 the exception message a little bit so that we can know more about what's happening.

Could you give it a try again and copy/paste the message?

Also if you want to debug this directly, the problem is in the rb_reg_prepare_enc() function in the re.c file.

[dmitry@…]

Here's the message:

given string `"9"' has unrecognized encoding `x-mac-cyrillic' (7) (ArgumentError)

So, CFStringGetFastestEncoding(str) returns kCFStringEncodingMacCyrillic for "9".

I've been able to successfully install MacRuby by setting __CF_USER_TEXT_ENCODING environment variable to 0:0:0 (it was 0x1F7:0:0). Thanks for your help!

Now, CoreFoundation contains a lot of "External encodings" ( http://tinyurl.com/cfstringenc), I wonder how to handle them in rb_reg_prepare_enc()? I also wonder why there was __CF_USER_TEXT_ENCODING environment variable set to MacCyrillic, when Terminal preferences clearly state "Unicode (UTF-8)"... Though, this question is not for this bug tracker :-)

[lsansonetti@…]

I changed the logic in r2435 to fall-back to a 8-bit translation in case the returned encoding is not recognized. In the future we might want to look at all these exotic encodings.

Could you confirm that with this change MacRuby builds as expected, even with the original CF_USER_TEXT_ENCODING environment variable set on?

I also think that this environment variable is set because of your system preferences. If I set the language of my system as French I get "CF_USER_TEXT_ENCODING=0x1F5:0:0".

[dmitry@…]

Yes, r2435 installs well.

(I think __CF_USER_TEXT_ENCODING is set to the root user language, because both of my computers have English language set in System Preferences, and one have it set to 0x1F5, the other one is 0x1F7. However, the latter one has the Login window in Russian, which was chosen during installation:  http://support.apple.com/kb/TA20558?viewlocale=en_US).

[lsansonetti@…]

Closing :-)