Skip to content

Commit 867e155

Browse files
cognifloydmichaelpro1
cognifloyd
authored and
michaelpro1
committed
Add pythonSrcRoot option to python servers (aiohttp/flask/blueplanet) to support src/ layout projects [and reenable/fix all python server tests] (OpenAPITools#5423)
* python server: Add pythonSrcRoot option This will allow the python project to be in a subdirectory (as specified in pythonSrcRoot). That could mean following the src layout with sources under src/ or lib/. Multi-language projects might use a sub directory like python/, or whatever makes sense for the project. By default, the pythonSrcRoot is "", meaning the existing behavior is the default. * python server: update template files to support pythonSrcRoot * python server: update docs to add pythonSrcRoot option * python server: add pythonSrcRoot sample script * python server: build sample srclayout project * [Python] copy test files preserving history * [Python] Make a conflict to preserve file copy history * [Python] customize pom.xml for src layout tests * [Python] add python-aiohttp-srclayout tests * [Python] Fix server tests by updating requirements Reverts the PR that disabled python2 server tests: OpenAPITools#4949 Reverts commits that disabled python3 server tests: 9adfedb 17ee990 Issue about the python 3 tests: OpenAPITools#5235 I couldn't find an issue about the python2 tests being disabled. I'm guessing build errors like the following were the trigger: https://travis-ci.org/github/OpenAPITools/openapi-generator/builds/634238181 Let's see what breaks! * [Python] Copy setup.py to python-aiohttp * [Python] Save history while copying setup.py to python-aiohttp * [Python] Add aiohttp server setup.py * [Python] Fix python server tests with src layout * [Python] bump Flask-Testing version * [Python] Pin pyyaml for py2.7 flask server * [Python] simplify flask server requirements * consolidate server tests * [Python] rebuild python server samples * [Python] Fix python server requirements for older pythons Documented minimum python version for our aiohttp server is 3.5. Documented minimum python version for our flask server is 3.4. Connexion 2.3 is the last version to support python 3.4 and 3.5, so fix the version pinning to correctly select <=2.3 for these EOL python versions. Newer pythons should get the latest if possible as there many relevant bug fixes. Werkzeug also needs to be pinned for these old versions in the aiohttp server just like for the flask server. 3.4 and 3.5 are EOL. We really should increase the minimum supported version, but that is for another PR to do.
1 parent a59d082 commit 867e155

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

63 files changed

+3260
-41
lines changed

‎bin/python-server-aiohttp-srclayout-petstore.sh

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
#!/bin/sh
2+
3+
SCRIPT="$0"
4+
echo "# START SCRIPT: $SCRIPT"
5+
6+
while [ -h "$SCRIPT" ] ; do
7+
ls=`ls -ld "$SCRIPT"`
8+
link=`expr "$ls" : '.*-> \(.*\)$'`
9+
if expr "$link" : '/.*' > /dev/null; then
10+
SCRIPT="$link"
11+
else
12+
SCRIPT=`dirname "$SCRIPT"`/"$link"
13+
fi
14+
done
15+
16+
if [ ! -d "${APP_DIR}" ]; then
17+
APP_DIR=`dirname "$SCRIPT"`/..
18+
APP_DIR=`cd "${APP_DIR}"; pwd`
19+
fi
20+
21+
executable="./modules/openapi-generator-cli/target/openapi-generator-cli.jar"
22+
23+
if [ ! -f "$executable" ]
24+
then
25+
mvn -B clean package
26+
fi
27+
28+
generator=python-aiohttp
29+
input=modules/openapi-generator/src/test/resources/2_0/petstore.yaml
30+
out_folder=samples/server/petstore/${generator}-srclayout
31+
resources=modules/openapi-generator/src/main/resources/${generator}
32+
33+
# if you've executed sbt assembly previously it will use that instead.
34+
export JAVA_OPTS="${JAVA_OPTS} -Xmx1024M -DloggerPath=conf/log4j.properties -Dservice"
35+
ags="generate -t $resources -i $input -g $generator -o $out_folder --additional-properties pythonSrcRoot=src $@"
36+
37+
rm -rf $out_folder/.openapi*
38+
rm -rf $out_folder/openapi_server
39+
rm -rf $out_folder/tests*
40+
rm $out_folder/README.md
41+
rm $out_folder/requirements.txt
42+
rm $out_folder/test-requirements.txt
43+
44+
java $JAVA_OPTS -jar $executable $ags

‎bin/python-server-all.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
#!/bin/sh
22

33
./bin/python-server-aiohttp-petstore.sh
4+
./bin/python-server-aiohttp-srclayout-petstore.sh
45
./bin/python-server-flask-petstore.sh
56
./bin/python-server-flask-petstore-python2.sh
67
./bin/python-server-blueplanet-petstore.sh

‎docs/generators/python-aiohttp.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ sidebar_label: python-aiohttp
1212
|packageName|python package name (convention: snake_case).| |openapi_server|
1313
|packageVersion|python package version.| |1.0.0|
1414
|prependFormOrBodyParameters|Add form or body parameters to the beginning of the parameter list.| |false|
15+
|pythonSrcRoot|put python sources in this subdirectory of output folder (defaults to &quot;&quot; for). Use this for src/ layout.| ||
1516
|serverPort|TCP port to listen to in app.run| |8080|
1617
|sortModelPropertiesByRequiredFlag|Sort model properties to place required parameters before optional parameters.| |true|
1718
|sortParamsByRequiredFlag|Sort method arguments to place required parameters before optional parameters.| |true|

‎docs/generators/python-blueplanet.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ sidebar_label: python-blueplanet
1212
|packageName|python package name (convention: snake_case).| |openapi_server|
1313
|packageVersion|python package version.| |1.0.0|
1414
|prependFormOrBodyParameters|Add form or body parameters to the beginning of the parameter list.| |false|
15+
|pythonSrcRoot|put python sources in this subdirectory of output folder (defaults to &quot;&quot; for). Use this for src/ layout.| ||
1516
|serverPort|TCP port to listen to in app.run| |8080|
1617
|sortModelPropertiesByRequiredFlag|Sort model properties to place required parameters before optional parameters.| |true|
1718
|sortParamsByRequiredFlag|Sort method arguments to place required parameters before optional parameters.| |true|

‎docs/generators/python-flask.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ sidebar_label: python-flask
1212
|packageName|python package name (convention: snake_case).| |openapi_server|
1313
|packageVersion|python package version.| |1.0.0|
1414
|prependFormOrBodyParameters|Add form or body parameters to the beginning of the parameter list.| |false|
15+
|pythonSrcRoot|put python sources in this subdirectory of output folder (defaults to &quot;&quot; for). Use this for src/ layout.| ||
1516
|serverPort|TCP port to listen to in app.run| |8080|
1617
|sortModelPropertiesByRequiredFlag|Sort model properties to place required parameters before optional parameters.| |true|
1718
|sortParamsByRequiredFlag|Sort method arguments to place required parameters before optional parameters.| |true|

‎modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonAbstractConnexionServerCodegen.java

Lines changed: 39 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ public class PythonAbstractConnexionServerCodegen extends DefaultCodegen impleme
5151
public static final String SUPPORT_PYTHON2 = "supportPython2";
5252
// nose is a python testing framework, we use pytest if USE_NOSE is unset
5353
public static final String USE_NOSE = "useNose";
54+
public static final String PYTHON_SRC_ROOT = "pythonSrcRoot";
5455
static final String MEDIA_TYPE = "mediaType";
5556

5657
protected int serverPort = 8080;
@@ -61,6 +62,7 @@ public class PythonAbstractConnexionServerCodegen extends DefaultCodegen impleme
6162
protected Map<Character, String> regexModifiers;
6263
protected boolean fixBodyName;
6364
protected boolean useNose = Boolean.FALSE;
65+
protected String pythonSrcRoot;
6466

6567
public PythonAbstractConnexionServerCodegen(String templateDirectory, boolean fixBodyNameValue) {
6668
super();
@@ -165,6 +167,8 @@ public PythonAbstractConnexionServerCodegen(String templateDirectory, boolean fi
165167
defaultValue("8080"));
166168
cliOptions.add(CliOption.newBoolean(USE_NOSE, "use the nose test framework").
167169
defaultValue(Boolean.FALSE.toString()));
170+
cliOptions.add(new CliOption(PYTHON_SRC_ROOT, "put python sources in this subdirectory of output folder (defaults to \"\" for). Use this for src/ layout.").
171+
defaultValue(""));
168172
}
169173

170174
protected void addSupportingFiles() {
@@ -212,6 +216,12 @@ public void processOpts() {
212216
if (additionalProperties.containsKey(USE_NOSE)) {
213217
setUseNose((String) additionalProperties.get(USE_NOSE));
214218
}
219+
if (additionalProperties.containsKey(PYTHON_SRC_ROOT)) {
220+
setPythonSrcRoot((String) additionalProperties.get(PYTHON_SRC_ROOT));
221+
additionalProperties.put(PYTHON_SRC_ROOT, pythonSrcRoot);
222+
} else {
223+
setPythonSrcRoot("");
224+
}
215225
supportingFiles.add(new SupportingFile("__main__.mustache", packagePath(), "__main__.py"));
216226
supportingFiles.add(new SupportingFile("util.mustache", packagePath(), "util.py"));
217227
supportingFiles.add(new SupportingFile("typing_utils.mustache", packagePath(), "typing_utils.py"));
@@ -230,6 +240,25 @@ public void setUseNose(String val) {
230240
this.useNose = Boolean.valueOf(val);
231241
}
232242

243+
public void setPythonSrcRoot(String val) {
244+
String pySrcRoot;
245+
if (val == null) {
246+
pySrcRoot = "";
247+
} else {
248+
pySrcRoot = val.replaceAll("[/\\\\]+$", "");
249+
}
250+
251+
if (pySrcRoot.isEmpty() || pySrcRoot == ".") {
252+
this.pythonSrcRoot = "";
253+
} else {
254+
this.pythonSrcRoot = pySrcRoot + File.separator;
255+
}
256+
}
257+
258+
public String pythonSrcOutputFolder() {
259+
return outputFolder + File.separator + pythonSrcRoot;
260+
}
261+
233262
private static String packageToPath(String pkg) {
234263
return pkg.replace(".", File.separator);
235264
}
@@ -304,7 +333,14 @@ public String escapeReservedWord(String name) {
304333
*/
305334
@Override
306335
public String apiFileFolder() {
307-
return outputFolder + File.separator + apiPackage().replace('.', File.separatorChar);
336+
String pkgPath = apiPackage().replace('.', File.separatorChar);
337+
return pythonSrcOutputFolder() + pkgPath;
338+
}
339+
340+
@Override
341+
public String modelFileFolder() {
342+
String pkgPath = modelPackage().replace('.', File.separatorChar);
343+
return pythonSrcOutputFolder() + pkgPath;
308344
}
309345

310346
@Override
@@ -799,7 +835,8 @@ public void setPackageVersion(String packageVersion) {
799835
}
800836

801837
public String packagePath() {
802-
return packageName.replace('.', File.separatorChar);
838+
String pkgPath = packageName.replace('.', File.separatorChar);
839+
return pythonSrcRoot + pkgPath;
803840
}
804841

805842
@Override

‎modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonAiohttpConnexionServerCodegen.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ protected void addSupportingFiles() {
7171
supportingFiles.add(new SupportingFile("conftest.mustache", testPackage, "conftest.py"));
7272
supportingFiles.add(new SupportingFile("__init__test.mustache", testPackage, "__init__.py"));
7373
supportingFiles.add(new SupportingFile("__init__main.mustache", packagePath(), "__init__.py"));
74+
supportingFiles.add(new SupportingFile("setup.mustache", "", "setup.py"));
7475
supportingFiles.add(new SupportingFile("tox.mustache", "", "tox.ini"));
7576
supportingFiles.add(new SupportingFile("gitignore.mustache", "", ".gitignore"));
7677
}

‎modules/openapi-generator/src/main/resources/python-aiohttp/conftest.mustache

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@ def client(loop, aiohttp_client):
1111
options = {
1212
"swagger_ui": True
1313
}
14-
specification_dir = os.path.join(os.path.dirname(__file__), '..',
14+
specification_dir = os.path.join(os.path.dirname(__file__), '..',{{#pythonSrcRoot}}
15+
"{{{.}}}",{{/pythonSrcRoot}}
1516
'{{packageName}}',
1617
'openapi')
1718
app = connexion.AioHttpApp(__name__, specification_dir=specification_dir,

‎modules/openapi-generator/src/main/resources/python-aiohttp/requirements.mustache

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1-
connexion[aiohttp,swagger-ui] == 2.0.2
2-
swagger-ui-bundle == 0.0.2
3-
aiohttp_jinja2 == 1.1.0
1+
connexion[aiohttp,swagger-ui] >= 2.6.0; python_version>="3.6"
2+
# 2.3 is the last version that supports python 3.5
3+
connexion[aiohttp,swagger-ui] <= 2.3.0; python_version=="3.5" or python_version=="3.4"
4+
# connexion requires werkzeug but connexion < 2.4.0 does not install werkzeug
5+
# we must peg werkzeug versions below to fix connexion
6+
# https://github.com/zalando/connexion/pull/1044
7+
werkzeug == 0.16.1; python_version=="3.5" or python_version=="3.4"
8+
swagger-ui-bundle == 0.0.6
9+
aiohttp_jinja2 == 1.2.0

‎modules/openapi-generator/src/main/resources/python-aiohttp/setup.mustache

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# coding: utf-8
2+
3+
import sys
4+
from setuptools import setup, find_packages
5+
6+
NAME = "{{packageName}}"
7+
VERSION = "{{packageVersion}}"
8+
{{#apiInfo}}{{#apis}}{{^hasMore}}
9+
# To install the library, run the following
10+
#
11+
# python setup.py install
12+
#
13+
# prerequisite: setuptools
14+
# http://pypi.python.org/pypi/setuptools
15+
16+
REQUIRES = [
17+
"connexion==2.6.0",
18+
"swagger-ui-bundle==0.0.6",
19+
"aiohttp_jinja2==1.2.0",
20+
]
21+
22+
setup(
23+
name=NAME,
24+
version=VERSION,
25+
description="{{appName}}",
26+
author_email="{{infoEmail}}",
27+
url="{{packageUrl}}",
28+
keywords=["OpenAPI", "{{appName}}"],
29+
install_requires=REQUIRES,
30+
packages=find_packages({{#pythonSrcRoot}}"{{{.}}}"{{/pythonSrcRoot}}),{{#pythonSrcRoot}}
31+
package_dir={"": "{{{.}}}"},{{/pythonSrcRoot}}
32+
package_data={'': ['{{#pythonSrcRoot}}{{{.}}}/{{/pythonSrcRoot}}openapi/openapi.yaml']},
33+
include_package_data=True,
34+
entry_points={
35+
'console_scripts': ['{{packageName}}={{packageName}}.__main__:main']},
36+
long_description="""\
37+
{{appDescription}}
38+
"""
39+
)
40+
{{/hasMore}}{{/apis}}{{/apiInfo}}

0 commit comments

Comments
 (0)