Loop, svn log, awk, cat, egrep echo Example

January 5th, 2010

So, I just checked in some files and I realized “doh! I didn’t check to see if there’s any debug trash in there!” My debug trash is fairly easy to detect: I only use raise or puts in Ruby and alert in JavaScript for basic “hello world” type things when debugging some particular issue.

Trouble is, sometimes I forget to remove that trash code before checking it in. To prevent big problems, I refrain from using the Ruby raise command and instead use puts which is fairly harmless. Anyway, I wrote a script to 1) ask SVN for the list of files modified by my checkin and 2) use awk to figure out the file names and, within a loop, 3) use cat to pass the contents of each file to 4) grep / egrep to look for alert, raise, and/or puts statements within the files. So now I have an example of a handful of common Linux commands that I can post here for future reference.

for l in `svn log -r 298 -v | egrep M | awk '{print $2}'`;
do
echo ${l};
cat .${l} |
egrep 'alert|puts|raise';
done

Which spat out:


/app/models/some_model.rb
/app/models/some_other_model.rb
/app/models/foo_model.rb
/app/controllers/home_controller.rb
    puts "hello world"
/app/controllers/foo_controller.rb
    #puts "hello world"
/app/controllers/other_controller.rb
    puts "hello world" unless some_condition

And I found three instances of debug trash… One commented out, one from a previous check-in, and one that I need to remove!

And on one line, without formatting:

for l in `svn log -r 298 -v | egrep M | awk ’{print $2}’`; do echo ${l};cat .${l} | egrep ’alert|puts|raise’;done

Make FireFox 3 Files Writable In Linux

December 8th, 2009

This is something that really pisses me off: FireFox 3 makes downloaded files read-only because they’re saved in /tmp, and OpenOffice doesn’t let you manipulate a read-only file. Even for something stupid and simple, like changing page margins, you have to click the pencil icon in the toolbar, click OK to an error message, and click Yes to something else. FF says “it’s supposed to be read-only, have OO fix it” and OO says “it’s read-only, tell FF to make it not read-only” and lowly-user daresay “M$ Turd handles this gracefully” and “Hey, FF, if you had made this a user-configurable option in FF3 instead of doing a global search & replace, this wouldn’t be an issue!”

Well to hell with it. This script fixes the problem by watching /tmp for files ending in doc,csv,xls,odf (and some others) and makes them writable by using the inotifywait command, which is part of the inotify-tools package. I use openSUSE so it was easy to find in the Install Software applet.

I was going to make an init.d script out of it, but I am busy… It’s just running in a Konsole on Desktop 4 along with all the other out-of-the-way stuff.

#!/bin/bash

# author John Stanfield
# makes files in /tmp/ writable upon creation
# this is a workaround for the religious war between OpenOffice and Mozilla regarding read-only files
# it's intended for users who want to
#  1) download a file
#  2) make a small edit (e.g. page margin, white space, etc)
#  3) read it, print it, email it, etc
#  4) IMPORTANT!
#     a) not be upset if the file is, accidentally or otherwise, deleted in the future
#     or
#     b) save the file elsewhere

inotifywait -m -e create --format %w%f /tmp/ | while read line
do
  if (echo found file... ${line} | egrep "\.od[t|s|p|g|f]$|\.csv$|\.doc$|\.rtf$|\.xls$|\.ppt$"); then
    chmod +w "${line}"
    echo `date` made ${line} writable
  fi
done

And unhighlighted:

#!/bin/bash

# author John Stanfield
# makes files in /tmp/ writable upon creation
# this is a workaround for the religious war between OpenOffice and Mozilla regarding read-only files
# it's intended for users who want to
#  1) download a file
#  2) make a small edit (e.g. page margin, white space, etc)
#  3) read it, print it, email it, etc
#  4) IMPORTANT!
#     a) not be upset if the file is, accidentally or otherwise, deleted in the future
#     or
#     b) save the file elsewhere

inotifywait -m -e create --format %w%f /tmp/ | while read line
do
  if (echo found file... ${line} | egrep "\.od[t|s|p|g|f]$|\.csv$|\.doc$|\.rtf$|\.xls$|\.ppt$"); then
    chmod +w "${line}"
    echo `date` made ${line} writable
  fi
done

Linux Command to Open Multiple Files

November 5th, 2009

UPDATE: I like vi more than Aptana now. Here’s how I use svn up to update a bunch of Rails sites, and then use vi to edit a file in each site:

for l in `ls -1`; do
  svn up ${l}/config/environment.rb;
  vi ${l}/config/environment.rb;
done

And when I’m done with my changes, I commit them all at once (sort of) with the same message:

for l in `ls -1`; do
  svn status -q ${l}/config/environment.rb |
    egrep '^M' |
      svn commit `awk '{print $2}'` -m "changed environment.rb";
done

One lovely thing about Ruby on Rails is the uniformity of the directory structure. All of my apps have pretty much the same files in pretty much the same folders. Today, I wanted to edit the environment.rb file in each of my company’s nine applications. Ugh, what a ton of mouse clicks digging through nine directory structures (note: I am very much into efficiency — every second counts, and manual repetitive tasks suck). I thought There’s gotta be a way to find all files named environment.rb and open them in Aptana. This command works! I can get my work done in seconds now instead of minutes! I use locate, which keeps a database of file names… I’ve found it to be faster than find.

The command below will first locate all files beneath the present folder (that’s what $(pwd) does) matching the regular expression /.*environment.rb$/ which means all files ending in environment.rb which will exclude any SVN files with similar names. Files that match that regular expression will be passed to Aptana, which will open any and all files passed in as arguments. Aptana will use an existing window if open, otherwise it will open a new one (precisely what I want!).

aptana $(locate $(pwd)/.*environment.rb$ -r)

It is the same as:

aptana /apps/app1/config/environment.rb /apps/app2/config/environment.rb /apps/app3/environment.rb

It’s much faster than:

aptana /apps/app1/config/environment.rb
aptana /apps/app2/config/environment.rb
aptana /apps/app3/config/environment.rb

And here’s a textarea to tinker, copy, & paste from:

Mount a Windows Share in Linux

November 4th, 2009

I always forget the syntax… Here it is:

su
mkdir -p /mnt/whatever/you/want
mount -t cifs //hostname/sharename -o username=user,password=pass /mnt/whatever/you/want

And without formatting:

su
mkdir -p /mnt/whatever/you/want
mount -t cifs //hostname/sharename -o username=user,password=pass /mnt/whatever/you/want

Linux TCP Window Scaling Problem

October 21st, 2009

The router at my office, a Linksys RV082, doesn’t like Linux computers. Supposedly it doesn’t like Vista or Windows 7 computers either (but what/who doesn’t, anyway?!?!). The problem is TCP window scaling, which (for performance reasons) tinkers with packets in a way that the router does not understand (possibly because the router is running firmware that is older than the concept of TCP window scaling, but firmware upgrades fail so I don’t know if the latest firmware fixes the problem). This command fixes the problem by turning off TCP window scaling.

echo 0 > /proc/sys/net/ipv4/tcp_window_scaling

openSUSE 11.1 + NVidia 9600 GT + Acer AL2216

October 16th, 2009

Whew… Two years ago I bought two of the Acer budget or economy LCD screens from NewEgg (newegg.com). They’re the 2216 screens that are always nicely priced (mine are AL2216wbd but the last few letters change [my company has purchased several varieties including AL2216WBD and V223WBD -- all are great monitors for typical non-graphic-artist business use]). They worked fine in Windows XP with an ATI FireGL card.

Fast forward to today: I just built a new Linux machine running openSUSE 11.1. The NVidia driver install was simple using “the hard way” described here. But the screens gave me a headache within 30 seconds of looking at them. Text was pixilated and fuzzy, and just about everything was so bright that it had white outlines. It was hard to look at. I thought I installed the driver wrong and it was using the wrong resolution and color combination — but it turns out the settings needed a little tweaking. I tried a bazillion different things, and I even contemplated going back to Windows (OUCH). Finally, I went through the painstaking effort of fiddling with all of NVidia’s settings. I am happy to say that this works:

Step 1: Edit /etc/X11/xorg.conf to add the settings described here in the section that begins with “For nVidia drivers you may have to disable automatic detection of DPI to set it manually.” It says to add these two settings to the Section “Device” section of xorg.conf:

  Option   "UseEdidDpi" "false"
  Option   "DPI" "96 x 96"

Step 2: Use the NVidia X Server Settings applet (just click the Gecko / Application Launcher and type nv in the search box) to adjust the brightness, contrast, and gamma. Those settings are under X Screen 0 -> X Server Color Correction. The brightness, contrast, and gamma settings pictured below worked best for me — the headache has gone away!!!

snapshot1

snapshot1

openSUSE Ruby on Rails Hack

October 13th, 2009

Note: This was my first attempt at using openSUSE. On my second attempt, I no longer had this problem. My guess is that I installed the wrong version of something-or-other which caused this problem.

If you have openSUSE 11.1, like I do, then you have Ruby 1.8.7. I encountered an error when trying to run ruby script/console and ruby script/server for the first time in openSUSE. This pretty much killed everything — most of my plug-ins and models didn’t load.

activesupport-1.4.2/lib/active_support/core_ext/string/access.rb:43:in `first’:NoMethodError: 
undefined method `[]‘ for #<Enumerable::Enumerator:0×7f1e1b77aa20>

The problem is that, in Ruby prior to 1.8.7 (perhaps much prior, I don’t know), the [] and/or first methods on String returned an array of characters. Now that has changed so that they return an Enumerable::Enumerator.

I added a hack to the top of /config/boot.rb so that it’d behave. I’ll figure out the real problem later — I’m busy! I found this example elsewhere, but I chose to but it in boot.rb:

unless '1.9'.respond_to?(:force_encoding)
  puts 'un-defining :chars method on String for ruby 1.8.7 compatibility'
  String.class_eval do
    begin
      remove_method :chars
    rescue NameError
      # OK
    end
  end
end

And unformatted:

unless ’1.9′.respond_to?(:force_encoding)
  puts ’un-defining :chars method on String for ruby 1.8.7 compatibility’
  String.class_eval do
    begin
      remove_method :chars
    rescue NameError
      # OK
    end
  end
end

How to Read a Rails Log File

October 5th, 2009

Frequently, I need to look at my Rails app’s log file. Normally, I use tail -f production.log and hope it doesn’t scroll by so fast that I can’t read it. Sometimes, I need to watch the log file but can’t do so because it’s just too fast. Here’s a nifty Linux command to find what you’re looking for. It uses tail to follow the production log file, but only spits out lines that you are looking for. In my case, I just want to show the URLs of slow requests (e.g. any request that takes longer than one second). The command below looks for the “(0 reqs/sec)” lines in the log file but can easily be changed to look for other things. Just press CTRL+C to terminate it when you’re done.

tail -f production.log | while read line ; do echo "$line"| grep '(0 reqs/sec)' ; done

And on one line without formatting:

tail -f production.log | while read line ; do echo "$line"| grep '(0 reqs/sec)' ; done

Kill and/or Stop Slow Queries in MySQL

September 23rd, 2009

Sometimes you run test queries against large databases, or you copy & paste a query but leave out the where clause, or otherwise make a mess. If you ever accidentally execute a query in MySQL that spikes the CPU, here’s how you kill or stop it:

show processlist;

That will return a list describing what’s going on in MySQL. Hopefully the result set is small enough that you can pinpoint your slow query. Look for a row with a large value for the Time column and something other than Sleep in the Command column. Look at the rest of the columns to be sure you’ve found what you’re looking for.

Id User Host db Command Time State Info
21445 john localhost products_db Query 94 Sending data select * from users inner join posts

Once you’re sure you have the right one, use the kill command (note: this is the MySQL kill command for killing MySQL internal threads, not the OS-level command for killing processes!).

kill query 21445;

Plug-in Load Order in pre-2.0 Rails

July 29th, 2009

In Ruby on Rails, you can specify the order in which plug-ins will load.

In Rails 2.0, if you want to put a few plug-ins at the top of the order, and you didn’t really care about the rest, you can do so with this syntax:

config.plugins = [:my_first_plugin, :my_second_plugin, :all]

Prior to Rails 2.0, in Rails 1.2.3 for example, you have to specify every plug-in in order to take advantage of this feature because the :all option is not supported. This is a pain for those of us who have some Rails 2.0 apps and some pre-2.0 apps. Especially when we do a lot of plug-in work. Below is a workaround. It takes your array of preferred plug-ins and then appends the rest. That may or may not be what Rails 2.0 does with the :all symbol.

order = ['my_first_plugin', 'my_second_plugin']
others = Dir.new("#{RAILS_ROOT}/vendor/plugins").select{|f|File.directory?("#{RAILS_ROOT}/vendor/plugins/#{f}") && f[0].chr!='.'}.sort
config.plugins = (order << (others-order)).flatten

And without the line breaks:

order = ['my_first_plugin', 'my_second_plugin']
others = Dir.new("#{RAILS_ROOT}/vendor/plugins").select{|f|File.directory?("#{RAILS_ROOT}/vendor/plugins/#{f}") && f.[0].chr!='.'}.sort
config.plugins = (order << (others-order)).flatten