Skip to content

Simplify Travis CI configuration #2575

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
reyoung merged 4 commits into from
Jun 23, 2017
Merged

Simplify Travis CI configuration #2575

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c2fc896
Simplify Travis CI configuration
Jun 23, 2017
fba4649
Remove `BUILD_AND_TEST` section in travis.yaml
reyoung Jun 23, 2017
72c1a7f
Remove common.sh
wangkuiyi Jun 23, 2017
0cbe120
Remove paddle/script/travis/main.sh
wangkuiyi Jun 23, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 4 additions & 20 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,20 @@ group: deprecated-2017Q2
language: cpp
cache:
directories:
- $HOME/third_party
- $HOME/.ccache
- $HOME/.cache/pip
sudo: required
dist: trusty
os:
- linux
env:
- JOB=DOCS
- JOB=BUILD_AND_TEST
- JOB=PRE_COMMIT
- JOB=build_doc
- JOB=check_style
addons:
apt:
packages:
Copy link
Collaborator

@reyoung reyoung Jun 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this file, JOB=BUILD_AND_TEST should be removed.

- gcc-4.8
- g++-4.8
- gfortran-4.8
- git
- build-essential
- python
Expand All @@ -35,18 +32,7 @@ addons:
- libtool
- ccache
before_install:
- |
if [ ${JOB} == "BUILD_AND_TEST" ]; then
local change_list=`git diff --name-only $TRAVIS_COMMIT_RANGE`
if [ $? -eq 0 ]; then # if git diff return no zero, then rerun unit test.
if ! echo ${change_list} | grep -qvE '(\.md$)|(\.rst$)|(\.jpg$)|(\.png$)'
then
echo "Only markdown docs were updated, stopping build process."
exit
fi
fi
fi
- if [[ "$JOB" == "PRE_COMMIT" ]]; then sudo ln -s /usr/bin/clang-format-3.8 /usr/bin/clang-format; fi
- if [[ "$JOB" == "check_style" ]]; then sudo ln -s /usr/bin/clang-format-3.8 /usr/bin/clang-format; fi
# Paddle is using protobuf 3.1 currently. Protobuf 3.2 breaks the compatibility. So we specify the python
# protobuf version.
- pip install numpy wheel 'protobuf==3.1' sphinx==1.5.6 recommonmark sphinx-rtd-theme==0.1.9 virtualenv pre-commit requests==2.9.2 LinkChecker
Expand All @@ -55,9 +41,7 @@ before_install:
- |
function timeout() { perl -e 'alarm shift; exec @ARGV' "$@"; }
script:
- |
timeout 2580 paddle/scripts/travis/main.sh # 43min timeout
RESULT=$?; if [ $RESULT -eq 0 ] || [ $RESULT -eq 142 ]; then true; else false; fi;
- paddle/scripts/travis/$JOB.sh
notifications:
email:
on_success: change
Expand Down
12 changes: 0 additions & 12 deletions paddle/scripts/travis/build_and_test.sh
View file
Open in desktop

This file was deleted.

13 changes: 8 additions & 5 deletions paddle/scripts/travis/docs.sh → paddle/scripts/travis/build_doc.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
#!/bin/bash
set -e

# Create the build directory for CMake.
mkdir -p $TRAVIS_BUILD_DIR/build
cd $TRAVIS_BUILD_DIR/build

# Add set -e, cd to directory.
source ./common.sh
# Compile Documentation only.
cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_Fortran_COMPILER=/usr/bin/gfortran-4.8 -DWITH_GPU=OFF -DWITH_DOC=OFF -DWITH_STYLE_CHECK=OFF ${EXTRA_CMAKE_OPTS}
Copy link
Collaborator

@reyoung reyoung Jun 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gfortran is used to compile openblas, which is used by Paddle.

Maybe remove openblas will cause building error.

Copy link
Contributor

@gangliao gangliao Jun 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

current paddle build openblas does not need gfortran any more.

cmake .. -DCMAKE_BUILD_TYPE=Debug -DWITH_GPU=OFF -DWITH_DOC=OFF -DWITH_STYLE_CHECK=OFF
mkdir output
make -j `nproc`
Copy link
Contributor

@helinwang helinwang Jun 23, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why there are two cmake (line 9 and 14) and make (line 11 and 15) when building doc only?

Copy link
Collaborator Author

@wangkuiyi wangkuiyi Jun 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was told once by @QiJune this is because PaddlePaddle relies on SWIG, instead of a C-API, to expose functionalities to Python. But I don't understand the direct reason between SWIG and two runs of CMake.

Copy link
Collaborator

@reyoung reyoung Jun 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because part of Paddle's Python code is generated by C++ header. Only after we complete compiling PaddlePaddle, Paddle's Python package can be built. And Sphinx can load this python package to generate documentation.

So, generate documentation needs Python package. Python package needs to compile Paddle.

Copy link
Collaborator Author

@wangkuiyi wangkuiyi Jun 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But why we need to run CMake twice to compile Paddle?

Copy link
Collaborator

@reyoung reyoung Jun 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paddle is using Sphinx to generate documentation. Sphinx will load all Paddle's python package, py_paddle and paddle. Part of py_paddle is not pure Python. It is currently using SWIG to generate and linking all Paddle libraries.

So, we must create py_paddle and paddle Python packages first(the first cmake), and install them to current Python interpreter. And then let sphinx generate all documentations(the second cmake).

If we use C-API to expose Paddle library, then paddle Python packages could be pure Python. And then, we could just use sphinx to generate documentation.

find .. -name '*whl' | xargs pip install # install all wheels.
rm -rf *
cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_Fortran_COMPILER=/usr/bin/gfortran-4.8 -DWITH_GPU=OFF -DWITH_DOC=ON ${EXTRA_CMAKE_OPTS}
make paddle_docs paddle_docs_cn
cmake .. -DCMAKE_BUILD_TYPE=Debug -DWITH_GPU=OFF -DWITH_DOC=ON
make -j `nproc` paddle_docs paddle_docs_cn

# check websites for broken links
linkchecker doc/en/html/index.html
Expand Down
8 changes: 4 additions & 4 deletions paddle/scripts/travis/precommit.sh → paddle/scripts/travis/check_style.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
#!/bin/bash
function abort(){
echo "Your commit not fit PaddlePaddle code style" 1>&2
echo "Please use pre-commit scripts to auto-format your code" 1>&2
echo "Your change doesn't follow PaddlePaddle's code style." 1>&2
echo "Please use pre-commit to reformat your code and git push again." 1>&2
exit 1
}

trap 'abort' 0
set -e
source common.sh
cd ..

cd $TRAVIS_BUILD_DIR
export PATH=/usr/bin:$PATH
pre-commit install
clang-format --version
Expand Down
6 changes: 0 additions & 6 deletions paddle/scripts/travis/common.sh
View file
Open in desktop

This file was deleted.

13 changes: 0 additions & 13 deletions paddle/scripts/travis/main.sh
View file
Open in desktop

This file was deleted.