portage conflicts with bash

I haven't updated Gentoo Linux for long time. When I did emerge portage, it was blocked with bash. However, the latest bash needed portage too. Because bash and portage depends on each other, it can't be resolved in one shot. The basic strategy of solving these dependencies is to update bash to certain version, update portage and update bash to the latest. Look at /usr/portage/app-shells/bash, I have following packages: -rw-r--r-- 1 root root 3408 Mar 23 2006 bash-2.05b-r11.ebuild -rw-r--r-- 1 root root 5065 Jun 25 2007 bash-3.0-r12.ebuild -rw-r--r-- 1 root root 4613 Jun 25 2007 bash-3.0-r13.ebuild -rw-r--r-- 1 root root 4724 Jun 25 2007 bash-3.0-r14.ebuild -rw-r--r-- 1 root root 4601 Feb 27 2007 bash-3.1_p17.ebuild -rw-r--r-- 1 root root 5190 Feb 9 2008 bash-3.2_p17-r1.ebuild -rw-r--r-- 1 root root 5653 Jun 1 2008 bash-3.2_p33.ebuild -rw-r--r-- 1 root root 6156 Aug 24 10:35 bash-3.2_p39.ebuild -rw-r--r-- 1 root root 6156 Nov 20 12:57 bash-3.2_p48.ebuild So, I picked 3.2_p17-r1 as the intermediate version. # emerge -1 =bash-3.2_p17-r1 # emerge -1 portage # emerge bash Yay!

I have a lighttpd + Ruby on Rails via FCGI installation and I started to see 500 - Internal Server Error from lighttpd when I accessed my rails application.

2008-11-29 01:15:11: (log.c.75) server started
2008-11-29 01:15:22: (mod_fastcgi.c.1743) connect failed: Connection refused on unix:/tmp/ruby-railsapp.fastcgi-0
2008-11-29 01:15:22: (mod_fastcgi.c.2912) backend died; we'll disable it for 5 seconds and send the request to another backend instead: reconnects: 0 load: 1
2008-11-29 01:15:23: (mod_fastcgi.c.2471) unexpected end-of-file (perhaps the fastcgi process died): pid: 2068 socket: unix:/tmp/ruby-railsapp.fastcgi-0
2008-11-29 01:15:23: (mod_fastcgi.c.3234) child exited, pid: 2068 status: 1
2008-11-29 01:15:23: (mod_fastcgi.c.3281) response not received, request sent: 867 on socket: unix:/tmp/ruby-railsapp.fastcgi-0 for /dispatch.fcgi , closing connection

There's no error in log/fastcgi.crash.log nor log/development.log of my Rails installation. It appeared that public/dispatch.fcgi was called and failed. I put a bunch of fprintf-equivarent and found out that the line with

require File.dirname(__FILE__) + "/../config/environment".

was the line where it failed. I needed to know how it failed, and I put code to dump the stack trace:

begin
require File.dirname(__FILE__) + "/../config/environment"
rescue Exception => exception
File.open('/tmp/a.log', 'a') do |f|
  f.puts exception.class.name + ":" + exception.message
  f.puts exception.backtrace.join("\n")
end
end

and it gave me some clue:

SystemExit:exit /Users/gaku/ror/iphone/public/../config/boot.rb:65:in `exit' /Users/gaku/ror/iphone/public/../config/boot.rb:65:in `load_rails_gem' /Users/gaku/ror/iphone/public/../config/boot.rb:53:in `load_initializer' /Users/gaku/ror/iphone/public/../config/boot.rb:38:in `run' /Users/gaku/ror/iphone/public/../config/boot.rb:11:in `boot!' /Users/gaku/ror/iphone/public/../config/boot.rb:109 /Users/gaku/ror/iphone/public/../config/environment.rb:11:in `require' /Users/gaku/ror/iphone/public/../config/environment.rb:11 /Users/gaku/ror/iphone/public/dispatch.fcgi:28:in `require' /Users/gaku/ror/iphone/public/dispatch.fcgi:28

It failed in load_rails_gem and there was an attempt to dump error message to stderr, but FCGI didn't spit the message anywhere. The message was:

$stderr.puts %(Missing the Rails #{version} gem. Please `gem install -v#{version} rails`, update your RAILS_GEM_VERSION settingin config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.)

I have rails 2.2.2 gem installed, and I updated RAILS_GEM_VERSION variable in config/environment.rb to 2.2.2. However, in fact, the code tried to run

gem 'rails', '2.2.2'

and threw Gem::LoadError. I need to dig deeper. I put another stack trace code to the Gem::LoadError handler and it revealed another trace:

Gem::LoadError:Could not find RubyGem activesupport (= 2.2.2) /opt/local/lib/ruby/site_ruby/1.8/rubygems.rb:636:in `report_activate_error' /opt/local/lib/ruby/site_ruby/1.8/rubygems.rb:141:in `activate' /opt/local/lib/ruby/site_ruby/1.8/rubygems.rb:165:in `activate' /opt/local/lib/ruby/site_ruby/1.8/rubygems.rb:164:in `each' /opt/local/lib/ruby/site_ruby/1.8/rubygems.rb:164:in `activate' /opt/local/lib/ruby/site_ruby/1.8/rubygems.rb:49:in `gem' /Users/gaku/ror/iphone/public/../config/boot.rb:62:in `load_rails_gem' /Users/gaku/ror/iphone/public/../config/boot.rb:53:in `load_initializer' /Users/gaku/ror/iphone/public/../config/boot.rb:38:in `run' /Users/gaku/ror/iphone/public/../config/boot.rb:11:in `boot!' /Users/gaku/ror/iphone/public/../config/boot.rb:123 /Users/gaku/ror/iphone/public/../config/environment.rb:11:in `require' /Users/gaku/ror/iphone/public/../config/environment.rb:11 /Users/gaku/ror/iphone/public/dispatch.fcgi:28:in `require' /Users/gaku/ror/iphone/public/dispatch.fcgi:28

It was not a failure to load rails 2.2.2 gem, but it failed to load underlying gem called activesupport. It says "Could not find RubyGem activesupport (= 2.2.2)". It looked to me that I also have activesupport 2.2.2 according to my gem list --local command. However, when I run gem uninstall activesupport, there's an strange error:

kvant-2% sudo gem uninstall activesupport ERROR: While executing gem ... (Gem::InstallError) Unknown gem activesupport >= 0

So, I reinstalled activesupport anyway and this fixed the problem. It looks like the cause of the problem was broken activesupport gem installation. It should have happened during my upgrade of Rails from 2.1 to 2.2. I don't know how it happen though. Lessons

• lighttpd's "child exited with status 1 /Users/gaku/ror/iphone/public/dispatch.fcgi" actually means Ruby fcgi script exit with status 1. Rails framework has such code.
• Stack trace is easy and informative in Ruby.
• gem installation could be broken. gem uninstall/install may help.