John Stanfield dot com

Say you have the set 1,2,3,4,5,6 and you want to know all combinations, not permutations, of the set. Here is a Ruby script to do so.

elements=[1,2,3,4,5,6]
t1=Time.now
combinations=[]
elements.each_with_index do |element,i|
  # the by-itself combination
  combinations<<[element]
  # the last item's combinations are already defined
  unless i==elements.size-1
    elements[(i+1)..(elements.size-1)].each do |next_element|
      combinations<<(combinations.last.dup<<next_element)
    end
  end
end
t2=Time.now
combinations.each {|c|puts c.join(',')}
puts "it took #{(t2-t1)*1000}ms to generate this set"

and the output:

1
1,2
1,2,3
1,2,3,4
1,2,3,4,5
1,2,3,4,5,6
2
2,3
2,3,4
2,3,4,5
2,3,4,5,6
3
3,4
3,4,5
3,4,5,6
4
4,5
4,5,6
5
5,6
6
it took 8ms to generate this set

It also works with words. The real-world case for this is that I’m working on a keyword co-occurrence database. I want to know, out of a given set of survey responses, the frequency of words which appear in the same sentence. For the given sentence I love Ruby, it is such a great programming language — and powerful, too., with the common words removed…

elements=["love","Ruby","great","programming","language","powerful"]

and the output:

love
love,Ruby
love,Ruby,great
love,Ruby,great,programming
love,Ruby,great,programming,language
love,Ruby,great,programming,language,powerful
Ruby
Ruby,great
Ruby,great,programming
Ruby,great,programming,language
Ruby,great,programming,language,powerful
great
great,programming
great,programming,language
great,programming,language,powerful
programming
programming,language
programming,language,powerful
language
language,powerful
powerful
it took 13ms to generate this set

This script doesn’t handle large sets!

elements=(1..500).to_a
it took 30819ms to generate this set    

elements=(1..5000).to_a
#my computer crashed from lack of memory!

But if you can reduce the depth of combinations…

There isn’t much value (at least in the context of what I am doing!) in indexing every possible combination of words in a large text. Really, I only need maybe four-word combinations. This script will work with a large text because the result set is relatively small because it limits the combination depth to three elements.

max=3
elements=[1,2,3,4,5,6]
t1=Time.now
combinations=[]
elements.each_with_index do |element,i|
  # the by-itself combination
  combinations<<[element]
  # the last item's combinations are already defined
  unless i==elements.size-1
    elements[(i+1)..(elements.size-1)].each do |next_element|
      c=combinations.last.dup<<next_element
      c.delete_at(1) until c.size<=max
      combinations<<c
    end
  end
end
t2=Time.now
combinations.each {|c|puts c.join(',')}
puts "it took #{t2-t1}ms to generate this set"

1
1,2
1,2,3
1,3,4
1,4,5
1,5,6
2
2,3
2,3,4
2,4,5
2,5,6
3
3,4
3,4,5
3,5,6
4
4,5
4,5,6
5
5,6
6
it took 13ms to generate this set

Now I can work with large texts!

elements=(1..5000).to_a
it took 28370ms to generate this set

Now I have what I need to build a database index representing combinations of words in sentences. From there, I can find all sentences with, say, powerful and language in them (because all sentences will be indexed according to the combinations of words that appear in them). However, knowing to look for powerful and language in advance is not my goal -- I want the database to tell me the most frequent co-occurrences, so that I can examine them. My goal is to have the database tell me, in essence, the most common combination of words the combination of problem and feature so that I can pinpoint what people are talking about without having to read every single sentence. (I'm pretty sure this is impossible since people might not use the same terminology even though they're all talking about the same thing).

But more on this topic later...

Oh no! I just moved 3 gigs of pictures, afterwards realizing that the modified-time was changed from the time the picture was taken to the time the file was moved. What a mess, I thought, since I always refer to the modified-time when I wonder when a picture was taken.

exiv2 and touch to the rescue!

exiv2 will tell you, among a ton of other things, the time the picture was taken, and touch will allow you to change the modified-time of the image.

Note:Any command below, spread across multiple lines, is done so for clarity. To use the command, the line breaks must be removed such that the entire command is on one line!

This is the output of exiv2:

File name       : my_picture.jpg
File size       : 77443 Bytes
MIME type       : image/jpeg
Image size      : 425 x 543
Camera make     : Canon
Camera model    : Canon PowerShot SD1100 IS
Image timestamp : 2010:04:26 20:06:07
Image number    : 100-2025
Exposure time   : 1/60 s
Aperture        : F4.9
Exposure bias   : 0
Flash           : Yes, auto, red-eye reduction
Flash bias      : 0 EV
Focal length    : 18.6 mm
Subject distance: 266
ISO speed       : 250
Exposure mode   : Easy shooting (Auto)
Metering mode   : Multi-segment
Macro mode      : Off
Image quality   : Fine
Exif Resolution : 1848 x 2360
White balance   : Auto
Thumbnail       : JPEG, 7866 Bytes
Copyright       :
Exif comment    :

Now you just use grep to find the Image timestamp line:

exiv2 my_image.jpg | grep timestamp
Image timestamp : 2010:04:26 20:06:07

Now use awk to massage that date into almost the touch-required format of YYYYMMDDHHMM.SS

exiv2 my_image.jpg |
  grep timestamp |
    awk '{split($5,t,":");print $4,t[1],t[2],".",t[3]}'
2010:04:26 20 06 . 07

Now use sed to remove the space and colons to complete the formatting

exiv2 my_image.jpg |
  grep timestamp |
    awk '{split($5,t,":");print $4,t[1],t[2],".",t[3]}'  |
      sed 's/[:, ]//g'
201004262006.07

And the final command, to actually touch an image:

touch my_image.jpg -t
  `exiv2 my_image.jpg |
    grep timestamp |
      awk '{split($5,t,":");print $4,t[1],t[2],".",t[3]}' |
        sed 's/[:, ]//g'`

For an entire directory:

find *.jpg |
  while read f; do
    touch "${f}" -t `
      exiv2 "${f}" |
        grep timestamp |
          awk '{split($5,t,":");print $4,t[1],t[2],".",t[3]}' |
            sed 's/[:, ]//g'`;
  done

One annoying thing that has plagued my laptop is that right-clicking a JPEG in Dolphin makes a menu fly out with unusable options to rotate and flip the image. They don’t work because jpegorient doesn’t come with openSUSE 11.1, and I don’t know what it is or how to get it. I’ve been using the convert command (a part of ImageMagick) from a command line, so I wondered if I could make the Actions menu use it as well.

I edited the ServiceMenu file for the Actions menu, which is at /usr/share/kde4/services/ServiceMenus/jpegorient.desktop on my computer. You can execute locate jpegorient.desktop or find jpegorient.desktop if you don’t see it at that path. The changes are below in bold.

A handful of my Rails apps were created in a Windows development environment… Which means that the files in the script directory do not have the +x permission, which means I have to type ruby script/console instead of just script/console… SVN doesn’t see adding the execute permission to a file as being a change, so you can’t commit it by simply changing the permissions… So I wrote a script to copy the script folder, delete it, check it in, restore it, add execute permission, and check it back in. Ahh, I don’t have to type ruby anymore!

svn export script script2
svn delete script
svn commit -m "temporarily deleted script/*"
mv script2 script
svn add script
sudo chmod +x script/*
svn commit -m "restored script/* with +x"

I only give free advertising to truly exceptional companies. BookFinder.com is one of them. I have spent wasted countless hours looking for the best price on books from the usual suspects: Amazon, eBay, and half.com. No longer! BookFinder.com quickly and easily locates books from a multitude of sellers and includes the shipping cost in the price. And if you have trouble with the site, their customer service is fast and friendly. Two thumbs up for BookFinder.com!