How to put any directory under SVN revision control using a self-contained repository

Okay, I've not found this elsewhere so I'm writing a post about how I did it.
The tipical SVN scenario is when you and other people are working on shared files by checking out any number of working copies from a repository,

$ svn checkout http://svn.collab.net/repos/svn/trunk /home/me/src/project4/working_copy3

modifying the files and then committing the changes and let the revision control system manage conflicts and merge the updates.

$ svn commit -m "Corrected number of cheese slices."

In this scenario the best place for the repository is in a dedicated and secured server, it doesn't make any sense to have the repository inside its working copy: you want to feel free to make any kind of disaster and even delete the whole directory because you know you can start all over again by checking out a fresh original copy.

But SVN is so simple and powerful that you can let it track any changes that happens on a directory on your own personal computer, for backup or security purposes (think about /etc) or to study what an application does to the the files; and sometimes it may be convenient to create the repository in the same directory that repository is controlling, so you can always backup and move the directory without worrying to break the relationship between the working copy and the repository.
In my case I had an encrypted volume that I mount and unmount in different computers and I wanted to track the changes on this volume even without a network connection, so it turned out that making a self-contained SVN system was the best choice.

There are some simple initial steps to do this, and once completed you can work on the directory as you do usually with any other working copy. Before proceeding just make sure to make a backup of anything you are touching.


Suppose "/pwd" is the directory you want to put under revision control, and start creating a SVN repository where you want (since it will be moved later, I'll place it under /tmp):

$ svnadmin create /tmp/repo/

then import /pwd in this repository and remove it if successfully imported:

$ svn import /pwd/ file:///tmp/repo/ -m '[/] Start tracking all changes within pwd' && rm -fr /pwd/

Now if you check it out you'll have just put /pwd under revision control:

$ svn checkout file:///tmp/repo/ /pwd/

...but wait! IT'S NOT THAT SIMPLE FOR "LIVE" DIRECTORIES!

If you are working on a "live" directory (such as /etc on a busy system), it's not very safe to simply import and delete it because some files may need to change before you check out. So on a "live" directory we'll do it slightly differently:

$ svnadmin create /tmp/repo/
$ svn import /pwd/ file:///tmp/repo/ -m '[/] Start tracking all changes within pwd'

we'll check out under another temporary path

$ svn checkout file:///tmp/repo/ /tmp/pwd/

and move just the SVN system folders into the "live directory":

$ find /tmp/pwd/ -type d -name ".svn" -exec mv -vi {} /pwd/ \;
NOTE: In a Mac OS environment, use 'ditto' instead of 'mv':
$ cd /tmp/pwd/
$ find . -d -type d -name ".svn" -exec ditto {} /pwd/{} \;
$ cd ~-

Now /pwd is effectively a working copy and you can update the files that changed in the meanwhile, and get rid off the other working copy /tmp/pwd:

$ rm -fr /tmp/pwd/

And here comes the part to move the repository inside the working copy:
Change to /pwd/ and update (only needed if it was a "live" directory)

$ cd /pwd/ && svn update

tell SVN to ignore the folder ".repo" where we will put the repository, and commit.

$ svn propset svn:ignore .repo .
$ svn commit -m '[/] svn propset svn:ignore .repo .'

At this point, instead of just moving the repository in "/pwd/.repo", we make a precautionary step to ensure that if we are working on a "live" directory no changes would be lost when we relocate the repository: first we symlink the repository,

$ ln -s /tmp/repo/ .repo

then we tell what is the new location of the SVN repository

$ svn switch --relocate file:///tmp/repo/ file:///pwd/.repo/

and only after making the switch we effectively move it replacing the symlink all at once

$ rm /pwd/.repo && mv -vf /tmp/repo/ /pwd/.repo/

Now you have a self-contained revision control of the current directory.



If you find this useful please leave a comment ;)

Monkeypatching the Ruby IMAP class to build a client for Exchange email servers not compliant with RFC standards

When trying to use Ruby Class Net::IMAP to automate some searching/arranging/deleting tasks for my office email I came across some Exchange responses that are not strictly compliant with the IMAP standard (currently 4rev1, described in RFC3501)

After loading the standard IMAP class

ruby-1.9.2-p290 :001 > require 'net/imap'
=> true

we can connect to the IMAP server by instantiating an Net::IMAP object that describes the type of connection used (IP address, TCP port and whether we should use SSL to encrypt the communication).
My company uses IMAP over SSL (port 993/tcp) with a 'PLAIN' authentication, so let's start with:

ruby-1.9.2-p290 :002 > imap= Net::IMAP.new( 'outlook.h3g.it', :port=>'imaps', :ssl=>true )
=> #<Net::IMAP:0x0000010109e118 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x0000010109e0c8>, @host="outlook.h3g.it", @port="imaps", @tag_prefix="RUBY", @tagno=0, @parser=#<Net::IMAP::ResponseParser:0x0000010109e028 @str="* OK IMAP4 019 Ready\r\n", @pos=22, @lex_state=:EXPR_BEG, @token=nil, @flag_symbols={}>, @sock=#<OpenSSL::SSL::SSLSocket:0x0000010109d678>, @usessl=true, @responses={}, @tagged_responses={}, @response_handlers=[], @tagged_response_arrival=#<MonitorMixin::ConditionVariable:0x0000010109bb70 @monitor=#<Net::IMAP:0x0000010109e118 ...>, @cond=#<ConditionVariable:0x0000010109bb48 @waiters=[], @waiters_mutex=#<Mutex:0x0000010109baf8>>>, @continuation_request_arrival=#<MonitorMixin::ConditionVariable:0x0000010109bad0 @monitor=#<Net::IMAP:0x0000010109e118 ...>, @cond=#<ConditionVariable:0x0000010109baa8 @waiters=[], @waiters_mutex=#<Mutex:0x0000010109ba58>>>, @idle_done_cond=nil, @logout_command_tag=nil, @debug_output_bol=true, @exception=nil, @greeting=#<struct Net::IMAP::UntaggedResponse name="OK", data=#<struct Net::IMAP::ResponseText code=nil, text="IMAP4 019 Ready">, raw_data="* OK IMAP4 019 Ready\r\n">, @client_thread=#<Thread:0x00000100892c80 run>, @receiver_thread=#<Thread:0x0000010109afb8 run>>

So far so good, but when I introduce myself to the server I receive an error:

ruby-1.9.2-p290 :003 > imap.authenticate( 'PLAIN', my_name, my_password )
Net::IMAP::ResponseParseError: unexpected token CRLF (expected SPACE)
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:3235:in `parse_error'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:3087:in `match'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:2058:in `continue_req'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:2045:in `response'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:1973:in `parse'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:1124:in `get_response'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:1036:in `receive_responses'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:1023:in `block in initialize'

It seems that our client is expecting a T_SPACE while server is sending T_CRLF. Interesting to say, Microsoft states that Exchange is "compatible with RFC3501" but also that "(AUTH=PLAIN not supported)" (http://technet.microsoft.com/en-us/library/ff848256.aspx). Why?? It seems to me that once again they are not able to understand a clear (A)BNF specification on a standard RFC:

RFC3501: INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1
9. Formal Syntax
response = *(continue-req / response-data) response-done
continue-req = "+" SP (resp-text / base64) CRLF
response-data = "*" SP (resp-cond-state / resp-cond-bye / mailbox-data / message-data / capability-data) CRLF
response-done = response-tagged / response-fatal
response-fatal = "*" SP resp-cond-bye CRLF ; Server closes connection immediately
response-tagged = tag SP resp-cond-state CRLF

RFC clearly says that a SP(ace) is needed after the "+" sign, as correctly defined in the 'continue_req' method:

require 'net/imap'
module Net

class IMAP

class ResponseParser

def continue_req

match(T_PLUS)
match(T_SPACE)
return ContinuationRequest.new(resp_text, @str)

end

end

end

end

however we can try to fix (but I should say 'break') it to suit the Exchange server's needs, deleting the 'match(T_SPACE)' statement:

require 'net/imap'
module Net

class IMAP

class ResponseParser

def continue_req

match(T_PLUS)
return ContinuationRequest.new(resp_text, @str)

end #def continue_req

end #class ResponseParser

end #class IMAP

end #module Net

And then try again:

ruby-1.9.2-p290 :004 > imap.authenticate( 'PLAIN', my_name, my_password )
=> #, raw_data="RUBY0001 OK AUTHENTICATE completed.\r\n">

And now it's ok.
But testing some other IMAP command we soon face another error:

ruby-1.9.2-p290 :005 > imap.noop
=> #, raw_data="RUBY0002 OK NOOP completed.\r\n">

ruby-1.9.2-p290 :006 > imap.status( 'INBOX', 'MESSAGES' )
Net::IMAP::ResponseParseError: unexpected token SPACE (expected CRLF)
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:3235:in `parse_error'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:3087:in `match'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:2051:in `response'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:1973:in `parse'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:1124:in `get_response'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:1036:in `receive_responses'
from /usr/local/rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/imap.rb:1023:in `block in initialize'

Exchange server (sometimes?) adds an extra space before CRLF (again violating the RFC command syntax listed above).
This time we need to change the response method:

require 'net/imap'
module Net

class IMAP

class ResponseParser

def response

token = lookahead
case token.symbol
when T_PLUS

result = continue_req

when T_STAR

result = response_untagged

else

result = response_tagged

end
match(T_CRLF)
match(T_EOF)
return result

end

end

end

end

adding a 'match(T_SPACE) if lookahead.symbol == T_SPACE' just before the CRLF match:

require 'net/imap'
module Net

class IMAP

class ResponseParser

def response

token = lookahead
case token.symbol
when T_PLUS

result = continue_req

when T_STAR

result = response_untagged

else

result = response_tagged

end
match(T_SPACE) if lookahead.symbol == T_SPACE
match(T_CRLF)
match(T_EOF)
return result

end #def response

end #class ResponseParser

end #class IMAP

end #module Net

I've tested this 'monkeypatched' IMAP class long enough to say that fortunately this is all we need to make our client work... does it means that Microsoft is really shifting towards standards-based technology? Bah!