Skip to content

Search for interpreters in all paths found in PATH variable #2575

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 13, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions news/1 Enhancements/2398.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Search for python interpreters in all paths found in the `PATH`/`Path` environment variable.
4 changes: 3 additions & 1 deletion src/client/interpreter/contracts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ export interface IInterpreterVersionService {
}

export const IKnownSearchPathsForInterpreters = Symbol('IKnownSearchPathsForInterpreters');

export interface IKnownSearchPathsForInterpreters {
getSearchPaths(): string[];
}
export const IVirtualEnvironmentsSearchPathProvider = Symbol('IVirtualEnvironmentsSearchPathProvider');
export interface IVirtualEnvironmentsSearchPathProvider {
getSearchPaths(resource?: Uri): string[];
Expand Down
2 changes: 1 addition & 1 deletion src/client/interpreter/locators/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ export class PythonInterpreterLocatorService implements IInterpreterLocatorServi
[PIPENV_SERVICE, ''],
[GLOBAL_VIRTUAL_ENV_SERVICE, ''],
[WORKSPACE_VIRTUAL_ENV_SERVICE, ''],
[KNOWN_PATH_SERVICE, '-win'],
[KNOWN_PATH_SERVICE, ''],
[CURRENT_PATH_SERVICE, '']
];
return getLocators(keys, this.platform, (key) => {
Expand Down
54 changes: 32 additions & 22 deletions src/client/interpreter/locators/services/KnownPathsService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,20 @@ import * as _ from 'lodash';
import * as path from 'path';
import { Uri } from 'vscode';
import { fsExistsAsync } from '../../../../utils/fs';
import { IS_WINDOWS } from '../../../common/util';
import { IPlatformService } from '../../../common/platform/types';
import { ICurrentProcess, IPathUtils } from '../../../common/types';
import { IServiceContainer } from '../../../ioc/types';
import { IInterpreterHelper, IKnownSearchPathsForInterpreters, InterpreterType, PythonInterpreter } from '../../contracts';
import { lookForInterpretersInDirectory } from '../helpers';
import { CacheableLocatorService } from './cacheableLocatorService';

// tslint:disable-next-line:no-require-imports no-var-requires
const untildify = require('untildify');

/**
* Locates "known" paths.
*/
@injectable()
export class KnownPathsService extends CacheableLocatorService {
public constructor(
@inject(IKnownSearchPathsForInterpreters) private knownSearchPaths: string[],
@inject(IKnownSearchPathsForInterpreters) private knownSearchPaths: IKnownSearchPathsForInterpreters,
@inject(IInterpreterHelper) private helper: IInterpreterHelper,
@inject(IServiceContainer) serviceContainer: IServiceContainer
) {
Expand Down Expand Up @@ -46,7 +44,7 @@ export class KnownPathsService extends CacheableLocatorService {
* Return the located interpreters.
*/
private suggestionsFromKnownPaths() {
const promises = this.knownSearchPaths.map(dir => this.getInterpretersInDirectory(dir));
const promises = this.knownSearchPaths.getSearchPaths().map(dir => this.getInterpretersInDirectory(dir));
return Promise.all<string[]>(promises)
// tslint:disable-next-line:underscore-consistent-invocation
.then(listOfInterpreters => _.flatten(listOfInterpreters))
Expand Down Expand Up @@ -79,22 +77,34 @@ export class KnownPathsService extends CacheableLocatorService {
}
}

/**
* Return the paths where Python interpreters might be found.
*/
export function getKnownSearchPathsForInterpreters(): string[] {
if (IS_WINDOWS) {
return [];
} else {
const paths = ['/usr/local/bin', '/usr/bin', '/bin', '/usr/sbin', '/sbin', '/usr/local/sbin'];
paths.forEach(p => {
paths.push(untildify(`~${p}`));
});
// Add support for paths such as /Users/xxx/anaconda/bin.
if (process.env.HOME) {
paths.push(path.join(process.env.HOME, 'anaconda', 'bin'));
paths.push(path.join(process.env.HOME, 'python', 'bin'));
@injectable()
export class KnownSearchPathsForInterpreters implements IKnownSearchPathsForInterpreters {
constructor(@inject(IServiceContainer) private readonly serviceContainer: IServiceContainer) { }
/**
* Return the paths where Python interpreters might be found.
*/
public getSearchPaths(): string[] {
const currentProcess = this.serviceContainer.get<ICurrentProcess>(ICurrentProcess);
const platformService = this.serviceContainer.get<IPlatformService>(IPlatformService);
const pathUtils = this.serviceContainer.get<IPathUtils>(IPathUtils);

const searchPaths = currentProcess.env[platformService.pathVariableName]!
.split(pathUtils.delimiter)
.map(p => p.trim())
.filter(p => p.length > 0);

if (!platformService.isWindows) {
['/usr/local/bin', '/usr/bin', '/bin', '/usr/sbin', '/sbin', '/usr/local/sbin']
.forEach(p => {
searchPaths.push(p);
searchPaths.push(path.join(pathUtils.home, p));
});
// Add support for paths such as /Users/xxx/anaconda/bin.
if (process.env.HOME) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interesting. We know that Anaconda does something 'bespoke' and so we inject it into the search paths.

This is one of the reasons we might want to have the specific IInterpreter classes that you and I discussed the other day - you could simply query each known (registered) IInterpreter for getKnownSearchPaths or something like that.

Another one that comes to mind is pyenv, who's prescribed behaviour is to shove all the installed versions under $HOME/.pyenv...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But its a chicken or the egg...
IInterpreter is the interpreter object that represents the interpreter info. Where do we get that object from. I.e. where do we get the python interpreter info from to begin with.

searchPaths.push(path.join(pathUtils.home, 'anaconda', 'bin'));
searchPaths.push(path.join(pathUtils.home, 'python', 'bin'));
}
}
return paths;
return searchPaths;
}
}
7 changes: 3 additions & 4 deletions src/client/interpreter/serviceRegistry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,15 @@ import { CondaEnvService } from './locators/services/condaEnvService';
import { CondaService } from './locators/services/condaService';
import { CurrentPathService } from './locators/services/currentPathService';
import { GlobalVirtualEnvironmentsSearchPathProvider, GlobalVirtualEnvService } from './locators/services/globalVirtualEnvService';
import { getKnownSearchPathsForInterpreters, KnownPathsService } from './locators/services/KnownPathsService';
import { KnownPathsService, KnownSearchPathsForInterpreters } from './locators/services/KnownPathsService';
import { PipEnvService } from './locators/services/pipEnvService';
import { WindowsRegistryService } from './locators/services/windowsRegistryService';
import { WorkspaceVirtualEnvironmentsSearchPathProvider, WorkspaceVirtualEnvService } from './locators/services/workspaceVirtualEnvService';
import { VirtualEnvironmentManager } from './virtualEnvs/index';
import { IVirtualEnvironmentManager } from './virtualEnvs/types';

export function registerTypes(serviceManager: IServiceManager) {
serviceManager.addSingletonInstance<string[]>(IKnownSearchPathsForInterpreters, getKnownSearchPathsForInterpreters());
serviceManager.addSingleton<IKnownSearchPathsForInterpreters>(IKnownSearchPathsForInterpreters, KnownSearchPathsForInterpreters);
serviceManager.addSingleton<IVirtualEnvironmentsSearchPathProvider>(IVirtualEnvironmentsSearchPathProvider, GlobalVirtualEnvironmentsSearchPathProvider, 'global');
serviceManager.addSingleton<IVirtualEnvironmentsSearchPathProvider>(IVirtualEnvironmentsSearchPathProvider, WorkspaceVirtualEnvironmentsSearchPathProvider, 'workspace');

Expand All @@ -70,9 +70,8 @@ export function registerTypes(serviceManager: IServiceManager) {
const isWindows = serviceManager.get<boolean>(IsWindows);
if (isWindows) {
serviceManager.addSingleton<IInterpreterLocatorService>(IInterpreterLocatorService, WindowsRegistryService, WINDOWS_REGISTRY_SERVICE);
} else {
serviceManager.addSingleton<IInterpreterLocatorService>(IInterpreterLocatorService, KnownPathsService, KNOWN_PATH_SERVICE);
}
serviceManager.addSingleton<IInterpreterLocatorService>(IInterpreterLocatorService, KnownPathsService, KNOWN_PATH_SERVICE);
serviceManager.addSingleton<IInterpreterService>(IInterpreterService, InterpreterService);
serviceManager.addSingleton<IInterpreterDisplay>(IInterpreterDisplay, InterpreterDisplay);

Expand Down
105 changes: 105 additions & 0 deletions src/test/interpreters/knownPathService.unit.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

'use strict';

// tslint:disable:max-func-body-length no-any

import { expect } from 'chai';
import * as path from 'path';
import * as TypeMoq from 'typemoq';
import { IPlatformService } from '../../client/common/platform/types';
import { ICurrentProcess, IPathUtils } from '../../client/common/types';
import { IKnownSearchPathsForInterpreters } from '../../client/interpreter/contracts';
import { KnownSearchPathsForInterpreters } from '../../client/interpreter/locators/services/KnownPathsService';
import { IServiceContainer } from '../../client/ioc/types';

suite('Interpreters Known Paths', () => {
let serviceContainer: TypeMoq.IMock<IServiceContainer>;
let currentProcess: TypeMoq.IMock<ICurrentProcess>;
let platformService: TypeMoq.IMock<IPlatformService>;
let pathUtils: TypeMoq.IMock<IPathUtils>;
let knownSearchPaths: IKnownSearchPathsForInterpreters;

setup(async () => {
serviceContainer = TypeMoq.Mock.ofType<IServiceContainer>();
currentProcess = TypeMoq.Mock.ofType<ICurrentProcess>();
platformService = TypeMoq.Mock.ofType<IPlatformService>();
pathUtils = TypeMoq.Mock.ofType<IPathUtils>();
serviceContainer.setup(c => c.get(TypeMoq.It.isValue(ICurrentProcess), TypeMoq.It.isAny())).returns(() => currentProcess.object);
serviceContainer.setup(c => c.get(TypeMoq.It.isValue(IPlatformService), TypeMoq.It.isAny())).returns(() => platformService.object);
serviceContainer.setup(c => c.get(TypeMoq.It.isValue(IPathUtils), TypeMoq.It.isAny())).returns(() => pathUtils.object);

knownSearchPaths = new KnownSearchPathsForInterpreters(serviceContainer.object);
});

test('Ensure known list of paths are returned', async () => {
const pathDelimiter = 'X';
const pathsInPATHVar = [path.join('a', 'b', 'c'), '', path.join('1', '2'), '3'];
pathUtils.setup(p => p.delimiter).returns(() => pathDelimiter);
platformService.setup(p => p.isWindows).returns(() => true);
platformService.setup(p => p.pathVariableName).returns(() => 'PATH');
currentProcess.setup(p => p.env).returns(() => {
return { PATH: pathsInPATHVar.join(pathDelimiter) };
});

const expectedPaths = [...pathsInPATHVar].filter(item => item.length > 0);

const paths = knownSearchPaths.getSearchPaths();

expect(paths).to.deep.equal(expectedPaths);
});

test('Ensure known list of paths are returned on non-windows', async () => {
const homeDir = '/users/peter Smith';
const pathDelimiter = 'X';
pathUtils.setup(p => p.delimiter).returns(() => pathDelimiter);
pathUtils.setup(p => p.home).returns(() => homeDir);
platformService.setup(p => p.isWindows).returns(() => false);
platformService.setup(p => p.pathVariableName).returns(() => 'PATH');
currentProcess.setup(p => p.env).returns(() => {
return { PATH: '' };
});

const expectedPaths: string[] = [];
['/usr/local/bin', '/usr/bin', '/bin', '/usr/sbin', '/sbin', '/usr/local/sbin']
.forEach(p => {
expectedPaths.push(p);
expectedPaths.push(path.join(homeDir, p));
});

expectedPaths.push(path.join(homeDir, 'anaconda', 'bin'));
expectedPaths.push(path.join(homeDir, 'python', 'bin'));

const paths = knownSearchPaths.getSearchPaths();

expect(paths).to.deep.equal(expectedPaths);
});

test('Ensure PATH variable and known list of paths are merged on non-windows', async () => {
const homeDir = '/users/peter Smith';
const pathDelimiter = 'X';
const pathsInPATHVar = [path.join('a', 'b', 'c'), '', path.join('1', '2'), '3'];
pathUtils.setup(p => p.delimiter).returns(() => pathDelimiter);
pathUtils.setup(p => p.home).returns(() => homeDir);
platformService.setup(p => p.isWindows).returns(() => false);
platformService.setup(p => p.pathVariableName).returns(() => 'PATH');
currentProcess.setup(p => p.env).returns(() => {
return { PATH: pathsInPATHVar.join(pathDelimiter) };
});

const expectedPaths = [...pathsInPATHVar].filter(item => item.length > 0);
['/usr/local/bin', '/usr/bin', '/bin', '/usr/sbin', '/sbin', '/usr/local/sbin']
.forEach(p => {
expectedPaths.push(p);
expectedPaths.push(path.join(homeDir, p));
});

expectedPaths.push(path.join(homeDir, 'anaconda', 'bin'));
expectedPaths.push(path.join(homeDir, 'python', 'bin'));

const paths = knownSearchPaths.getSearchPaths();

expect(paths).to.deep.equal(expectedPaths);
});
});
10 changes: 2 additions & 8 deletions src/test/interpreters/locators/index.unit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,7 @@ suite('Interpreters - Locators Index', () => {
locatorsTypes.push(PIPENV_SERVICE);
locatorsTypes.push(GLOBAL_VIRTUAL_ENV_SERVICE);
locatorsTypes.push(WORKSPACE_VIRTUAL_ENV_SERVICE);

if (osType.value !== OSType.Windows) {
locatorsTypes.push(KNOWN_PATH_SERVICE);
}
locatorsTypes.push(KNOWN_PATH_SERVICE);
locatorsTypes.push(CURRENT_PATH_SERVICE);

const locatorsWithInterpreters = locatorsTypes.map(typeName => {
Expand Down Expand Up @@ -115,10 +112,7 @@ suite('Interpreters - Locators Index', () => {
locatorsTypes.push(PIPENV_SERVICE);
locatorsTypes.push(GLOBAL_VIRTUAL_ENV_SERVICE);
locatorsTypes.push(WORKSPACE_VIRTUAL_ENV_SERVICE);

if (osType.value !== OSType.Windows) {
locatorsTypes.push(KNOWN_PATH_SERVICE);
}
locatorsTypes.push(KNOWN_PATH_SERVICE);
locatorsTypes.push(CURRENT_PATH_SERVICE);

const locatorsWithInterpreters = locatorsTypes.map(typeName => {
Expand Down