My favorites | Sign in
Google Code will be turning read-only on August 25th. See this post for more information.
Project Home Downloads Wiki Issues Source
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 400: ruby-gettext crash with ./configure (invalid byte sequence in UTF-8_
1 person starred this issue and may be notified of changes. Back to list
Status:  Fixed
Owner:  boukewou...@gmail.com
Closed:  May 2010


Sign in to add a comment
 
Reported by goo...@JonnyJD.net, Feb 11, 2010
What steps will reproduce the problem?
1. get a git version including
http://github.com/rubyripperdev/rubyripper/commit/9eb44c04f95aeae8c74f7d8a569c6a4c19fdb8b7
2. ./configure --enable-gtk2 --enable-cli
3. make install

What is the expected output? What do you see instead?
Well, it should install fine, but during language file creation I get:


Gtk2 frontend will be installed
Cli frontend will be installed
Languages to be installed: nl, de, fr, hu, ru, es, se, bg, it

You can now run make install
Make sure you've got the writing privileges

ruby configure --update-lang #update the locale files
'gettext/utils.rb' is deprecated. Use gettext/tools.rb.
'gettext/utils.rb' is deprecated. Use gettext/tools.rb.
./locale/po/rubyripper.pot ........................................... done.
./locale/po/hu/rubyripper.po ............................................ done.
./locale/po/it/rubyripper.po ...................................... done.
./locale/po/nl/rubyripper.po ............................................ done.
./locale/po/se/rubyripper.po ......................................... done.
./locale/po/fr/rubyripper.po ...................................... done.
./locale/po/ru/rubyripper.po ...................................... done.
/usr/lib/ruby/gems/1.9.1/gems/gettext-2.1.0/lib/gettext/tools.rb:73:in
`sub!': invalid byte sequence in UTF-8 (ArgumentError)
	from
/usr/lib/ruby/gems/1.9.1/gems/gettext-2.1.0/lib/gettext/tools.rb:73:in
`msgmerge'
	from
/usr/lib/ruby/gems/1.9.1/gems/gettext-2.1.0/lib/gettext/tools.rb:139:in
`block in update_pofiles'
	from
/usr/lib/ruby/gems/1.9.1/gems/gettext-2.1.0/lib/gettext/tools.rb:138:in `glob'
	from
/usr/lib/ruby/gems/1.9.1/gems/gettext-2.1.0/lib/gettext/tools.rb:138:in
`update_pofiles'
	from
/usr/lib/ruby/gems/1.9.1/gems/gettext-2.1.0/lib/gettext/utils.rb:35:in
`update_pofiles'
	from configure:84:in `update_lang'
	from configure:169:in `block in <main>'
	from configure:147:in `each'
	from configure:147:in `<main>'
make: *** [all] Error 1


Reverting the mentioned commit (ru language update) using git revert
9eb44c04f95
works fine. It installs successfully then.


I pulled that change myself in my fork, without testing it. I didn't think
a language update could cause that trouble.
It might have something to do with my ruby/gettext, but reverting to old
versions did not help. Reverting the mentioned commit did help.
The oldest ruby I could try was 1.9.1_p243.
Feb 11, 2010
#1 goo...@JonnyJD.net
Okay, reverting
http://github.com/rubyripperdev/rubyripper/commit/9eb44c04f95aeae8c74f7d8a569c6a4c19fdb8b7
only helped with my branch.

Using latest git from today
(http://github.com/rubyripperdev/rubyripper/commit/c678c6d69d3d40c39d58ebcf0c8ab1283ca46826)
I have to REVERT the following commits in order to install:
http://github.com/rubyripperdev/rubyripper/commit/cb2e249bf58d3b3a81dbb47092ba26a718e11dae
http://github.com/rubyripperdev/rubyripper/commit/a5848a38cdd528944a679ed8aed612d261971531
http://github.com/rubyripperdev/rubyripper/commit/9eb44c04f95aeae8c74f7d8a569c6a4c19fdb8b7


This gets weird. There might be other problems in older language files that only
showed up with recent commits. Maybe some fuzzy messages?
I also checked all files with "isutf8" from "moreutils" and I could not find any
problems with UTF (execept the png file)


Another solution seems to be to delete the bulgarian translation (yes, there is no
direct connection to these 3 commits, especially the russian language one..)

And finally, as a good workaround:
disabling update of po-files in the Makefile/configure:
cat configure | sed -e '/update_pofiles/d'


However, I am quite certain this has to be another ruby 1.9.1 or ruby-gettext bug.
Having something to do with encoding/bytes/cyrillic chars.
I could fix it changing
/usr/lib/ruby/gems/1.9.1/gems/gettext-2.1.0/lib/gettext/tools.rb:73
to writing "content" to a file and reading it in right afterwards again
# update version and save merged data
      testFile = File.new("testString.txt", "w")
      testFile.write(content)
      testFile.close()
      testFile = File.open("testString.txt", "r")
      content = testFile.read
      testFile.close()
      testFile = File.new("testString.txt", "w")
      testFile.write(content)
      testFile.close()
      content.sub!(/(Project-Id-Version\:).*$/, "\\1 #{app_version}\\n\"")
      File.open(defpo, "w") {|f|f.write(content)}
Feb 12, 2010
#2 goo...@JonnyJD.net
This seems fixed in ruby-svn, currently 1.9.2.

It was some problem reading UTF data with `backticks`. At least
`cat testString.txt`.sub!(/.*/,"")
is what gives the error with certain testString.txt (one beeing the bg po file).

So this is not a problem in rubyripper.
Feb 12, 2010
Project Member #3 boukewou...@gmail.com
Thanks for analyzing this one :) I guess this means we can close this issue?
Status: Fixed
Owner: boukewoudstra
Feb 18, 2010
#4 goo...@JonnyJD.net
Just for reference:

I filed a backport request for ruby 1.9.1, requesting a backport of the fix in 1.9.2:
http://redmine.ruby-lang.org/issues/show/2762
May 24, 2010
Project Member #5 boukewou...@gmail.com
I'll keep this issue open since it's still relevant, at least for Arch Linux.

For creating the po- and mo-files it is sufficient to comment line 73 of the file 
tools.rb which is mentioned in the crash. Only drawback seems to be that the version 
number of the po-files is not updated. Since this affects no real users, I find it a 
good workaround.
Status: Accepted
May 24, 2010
Project Member #6 boukewou...@gmail.com
Rename this issue to more properly reflect the real problem.
Summary: ruby-gettext crash with ./configure (invalid byte sequence in UTF-8_
May 24, 2010
Project Member #7 boukewou...@gmail.com
I've put the critical call of ruby-gettext in a begin/rescue part, so the program can 
continue. See commit:
http://github.com/rubyripperdev/rubyripper/commit/31faa99f1747ca2de3464eb5d8bfa01455c6
3f81

Since I can update the po-files myself with athe workaround of comment 5, this is not 
a real big problem.
Status: Fixed
Sign in to add a comment

Powered by Google Project Hosting