Improved support for --exec-prefix

[bukzor]

Virtualenv currently doesn't work with a python installation configured with different --prefix and --exec-prefix, due to the way it handles lib-dynload. Currently, it first copies over a couple files in lib-dynload ( zlib.so and readline.so ) and creates the lib-dynload directory in the process. It then tries to fixup exec-prefix installtions by copying over the lib-dynload from sys.exec_prefix, but failes with "File already exists". This results in many key modules becoming unavailable, such as _hashlib.so (dependency of setuptools) and _time.so (dependency of distribute).

My fix was to add recursion in the copyfile() function which copying a directory which already exists in the destination. This necessitates skipping site-packages during the exec-prefix setup.

Patch: ( Sorry for the format. I don't see an "attach file" option. )

--- virtualenv.orig 2011-04-26 20:27:02.557885000 -0700
+++ virtualenv.py   2011-04-26 21:49:58.111075000 -0700
@@ -394,6 +394,14 @@
         # Some bad symlink in the src
         logger.warn('Cannot find file %s (bad symlink)', src)
         return
+    if os.path.isdir(dest):
+        logger.debug('Directory %s already exists', dest)
+        logger.indent += 2
+        try:
+            for fn in os.listdir(src):
+                copyfile(join(src, fn), join(dest, fn), symlink)
+        finally:
+            logger.indent -= 2
     if os.path.exists(dest):
         logger.debug('File %s already exists', dest)
         return
@@ -884,7 +892,7 @@
 
 
 def change_prefix(filename, dst_prefix):
-    prefixes = [sys.prefix]
+    prefixes = [sys.prefix, sys.exec_prefix]
 
     if sys.platform == "darwin":
         prefixes.extend((
@@ -1004,8 +1012,14 @@
             exec_dir = join(sys.exec_prefix, 'Lib')
         else:
             exec_dir = join(sys.exec_prefix, 'lib', py_version)
-        for fn in os.listdir(exec_dir):
-            copyfile(join(exec_dir, fn), join(lib_dir, fn))
+        logger.debug('Exec dir %s', exec_dir)
+        logger.indent += 2
+        try:
+            for fn in os.listdir(exec_dir):
+                if fn != 'site-packages':
+                    copyfile(join(exec_dir, fn), join(lib_dir, fn))
+        finally:
+            logger.indent -= 2
 
     if is_jython:
         # Jython has either jython-dev.jar and javalib/ dir, or just

[pnasrat]

Yeah there is no attach file - can you either create fork in github, create a branch against develop, add your fix, push to github and create a pull request. Or put the diff in a gist at http://gist.github.com/ if that's easier and paste the URL here.

Creating a branch, etc preserves author information, etc.

[carljm]

Can you explain why site-packages has to be skipped during exec-prefix copying with your change? It's not obvious to me.

[bukzor]

It would hit the 'directory exists' state in my patched copyfile(), and
start symlinking all the contents of site-packages into the destination.

This has parity in the non-exec-prefix section of code:

            if fn != 'site-packages' and bn in REQUIRED_FILES:
                copyfile(join(stdlib_dir, fn), join(lib_dir, fn))

--Buck

On Wed, Apr 27, 2011 at 8:26 AM, carljm <
[email protected]>wrote:

Can you explain why site-packages has to be skipped during exec-prefix
copying with your change? It's not obvious to me.

Reply to this email directly or view it on GitHub:
#125 (comment)

[bukzor]

I'm using hg currently. I haven't even installed git yet.

Here's the gist:
https://gist.github.com/944589

https://gist.github.com/944589--buck

On Tue, Apr 26, 2011 at 11:44 PM, pnasrat <
[email protected]>wrote:

Yeah there is no attach file - can you either create fork in github, create
a branch against develop, add your fix, push to github and create a pull
request. Or put the diff in a gist at http://gist.github.com/ if that's
easier and paste the URL here.

Creating a branch, etc preserves author information, etc.

Reply to this email directly or view it on GitHub:
#125 (comment)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Just add a comment if you want to keep it open. Thank you for your contributions.