Issue installing newest version (cython problem?)

[murphytho]

Hi there,

I just wanted to join the ranks of others having issues with the current github build. My code/error is below. I have had Telescope installed in the past, but am on a new machine and am struggling to get it up and running.

I couldn't get around the numpy error on the bioconda distribution, but I'm also having trouble installing from github with pip.

Here is the error:

(Telescope) thomas@thomas-System-Product-Name:~$ pip install git+https://github.com/mlbendall/telescope.git
Collecting git+https://github.com/mlbendall/telescope.git
  Cloning https://github.com/mlbendall/telescope.git to /tmp/pip-req-build-q7psz3dx
  Running command git clone --filter=blob:none --quiet https://github.com/mlbendall/telescope.git /tmp/pip-req-build-q7psz3dx
  Resolved https://github.com/mlbendall/telescope.git to commit 4cf18595eea1702775e8a8bbc7b37ebac9233fd5
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [34 lines of output]
      /home/thomas/miniconda3/envs/Telescope/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-req-build-q7psz3dx/telescope/utils/calignment.pxd
        tree = Parsing.p_module(s, pxd, full_module_name)
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      # -*- coding: utf-8 -*-
      from calignment cimport AlignedPair
      ^
      ------------------------------------------------------------
      
      telescope/utils/calignment.pyx:2:0: 'calignment.pxd' not found
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      # -*- coding: utf-8 -*-
      from calignment cimport AlignedPair
      ^
      ------------------------------------------------------------
      
      telescope/utils/calignment.pyx:2:0: 'calignment/AlignedPair.pxd' not found
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-req-build-q7psz3dx/setup.py", line 39, in <module>
          extensions = cythonize(extensions)
        File "/home/thomas/miniconda3/envs/Telescope/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1115, in cythonize
          cythonize_one(*args)
        File "/home/thomas/miniconda3/envs/Telescope/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1238, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: telescope/utils/calignment.pyx
      Compiling telescope/utils/calignment.pyx because it changed.
      [1/1] Cythonizing telescope/utils/calignment.pyx
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

My guess is that it's a cython compatibility issue, but rolling back to a previous version of cython did not help.

I should also not that I had issues setting up the environment from the yml github link, so I had to download it locally and set it up from there. Others have noted the same issue, such as can be seen here

[MDisyak]

Any update on this? It's proven challenging to circumvent this issue. Can the devs just at least push the newest version to Bioconda? The current release still has the np.int incompatibility that was fixed on the latest github version.

[murphytho]

I actually figured out a couple ways to get it to work eventually. One was installing the bioconda version and fixing the numpy error. I didn't write down how I did it but I want to say I rolled back the version of python the environment was using? Something along those lines.

The other way I got it to work was installing from the sc_remove commit straight from GitHub. This is technically the most updated version, I think. I would definitely try this first before screwing with numpy-- I remember that being a pain.

Still would be nice to have a real fix, but in the meantime, I got this to work.

[adamspierer]

Thanks @murphytho for the pointer and agreeing with the request for an official fix. That said, I was able to get the test to work after a while of massaging and figured I would pay it forward.

My first issue is that I couldn't get past the beginning of the tutorial because conda (which I use) and mamba (used in the tutorial) don't play well together. So I created a conda virtual environment from the environment.yml file and then switched to that environment:

conda env create -n telescope --file=environment.yml
conda activate telescope

Then, I took the hint from above comment and did some work to make the sc_remove branch the main one:

## Sync repos
git fetch

## Switch to main branch
git checkout main

## Merge local main branch with sc_remove branch on github
git merge remotes/origin/sc_remove

I executed the test command in the tutorial:

eval $(telescope test)

But this gave me an error message that comes from using a depreciated file read mode:

File "/mnt/nimble/home/aspierer/miniconda3/envs/telescope/lib/python3.12/site-packages/telescope_ngs-1.0.3+49.g2832514-py3.12-linux-x86_64.egg/telescope/utils/_annotation_intervaltree.py", line 37, in __init__
    fh = open(gtf_file,'rU') if isinstance(gtf_file,str) else gtf_file
         ^^^^^^^^^^^^^^^^^^^
ValueError: invalid mode: 'rU'

I fixed this by using nano to edit the mode from 'rU' to 'r':

    fh = open(gtf_file,'r') if isinstance(gtf_file,str) else gtf_file

I ran the test command above once more and got the output I was promised (Final log-likelihood of 95252.596293 and two tsv files).

Hopefully the last few hours of troubleshooting can save others in the future.

[tmontserrat]

I managed to install telescope without modifying any code. Just using a custom environment.yml file with the correct versions of the packages solved the problem.
This is the environment.yml:

name: telescope
channels:
  - conda-forge
  - bioconda
  - defaults
dependencies:
  - python=3.7
  - future
  - pip
  - pyyaml
  - cython
  - numpy=1.21
  - scipy
  - pysam=0.19
  - htslib
  - intervaltree
  - pandas
  - samtools
  - openssl=1.1

Then, create the environment:
conda env create -n telescope --file=environment.yml

Activate the environment:
conda activate telescope

And install the last version of telescope:
pip install git+https://github.com/mlbendall/telescope.git

The test command worked perfectly.

[kajalpanchal314]

@tmontserrat hi, I ran your .yml also, but didn't work. Shows the same Cython error. Can you guide me on how to resolve the installation issue?

[marcos-da-conceicao]

I was able to get the Telescope to work using the following codes:

First, I created an environment with the following dependencies:
conda create -n telescope_env python=3.6 future pyyaml cython=0.29.7 numpy=1.16.3 pandas=1.1.3 scipy=1.2.1 pysam=0.15.2 htslib=1.9 intervaltree=3.0.2

Then I tried to install the latest version with bioconda and conda-forge already installed:
pip install git+https://github.com/mlbendall/telescope.git

And that's it.

[aydemiro]

I was able to get the Telescope to work using the following codes:

First, I created an environment with the following dependencies: conda create -n telescope_env python=3.6 future pyyaml cython=0.29.7 numpy=1.16.3 pandas=1.1.3 scipy=1.2.1 pysam=0.15.2 htslib=1.9 intervaltree=3.0.2

Then I tried to install the latest version with bioconda and conda-forge already installed: pip install git+https://github.com/mlbendall/telescope.git

And that's it.

This worked for me with one issue which was missing gcc. I was trying to create a container from a barebones base, so I had to add apt-get update && apt-get install -y build-essential python3-dev to my container. I also added the pip installation to the yml file to skip the extra environment activation step which is a pain when building a container. So I create the environment with mamba env create -f environment.yml with the following yml and no additional installation step:

name: telescope

channels:
  - conda-forge
  - bioconda
  - defaults
dependencies:
  - python = 3.6
  - future
  - pip
  - pyyaml
  - cython = 0.29.7
  - numpy = 1.16.3
  - scipy = 1.2.1
  - pysam = 0.15.2
  - htslib = 1.9
  - intervaltree = 3.0.2
  - pandas = 1.1.3
  - samtools
  - openssl = 1.1
  - pip:
    - git+https://github.com/mlbendall/telescope.git

I made the container publicly available in case anyone needs it. Pull the container with
singularity pull docker://quay.io/umassbio/telescope:1.0.3--0
or
docker pull quay.io/umassbio/telescope:1.0.3--0