Skip to content

Added Manual dataset download #286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
66 changes: 42 additions & 24 deletions functions/colab.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,47 @@ exports.handler = async function (event, _) {
.map((v) => v[0].toUpperCase() + v.slice(1))
.join(' ')
// notebook cell structure

function create_nb_cell(source_array, cell_type) {
return {
cell_type: cell_type,
metadata: {},
execution_count: null,
outputs: [],
source: source_array
}
}

let specific_commands = []

if (title === 'Template Vision Segmentation') {
specific_commands.push(
'!python -c "from data import download_datasets; download_datasets(\'./\')"'
)
}

const md_cell = [
`# ${title} by PyTorch-Ignite Code-Generator\n\n`,
'Please, run the cell below to execute your code.'
]

const common_nb_commands = [
`!wget ${zipRes}\n`,
`!unzip ${template}.zip\n`,
'!pip install -r requirements.txt'
]

const execution_nb_commands = ['!python main.py config.yaml']

let nb_cells = [
create_nb_cell(md_cell, 'markdown'),
create_nb_cell(common_nb_commands, 'code')
]
if (specific_commands.length > 0) {
nb_cells.push(create_nb_cell(specific_commands, 'code'))
}
nb_cells.push(create_nb_cell(execution_nb_commands, 'code'))

const nb = {
nbformat: 4,
nbformat_minor: 0,
Expand All @@ -52,30 +93,7 @@ exports.handler = async function (event, _) {
},
accelerator: 'GPU'
},
cells: [
{
cell_type: 'markdown',
metadata: {},
execution_count: null,
outputs: [],
source: [
`# ${title} by PyTorch-Ignite Code-Generator\n\n`,
'Please, run the cell below to execute your code.'
]
},
{
cell_type: 'code',
metadata: {},
execution_count: null,
outputs: [],
source: [
`!wget ${zipRes}\n`,
`!unzip ${template}.zip\n`,
'!pip install -r requirements.txt\n',
'!python main.py config.yaml\n'
]
}
]
cells: nb_cells
}
// Create the notebook on GitHub
await pushToGitHub(
Expand Down
16 changes: 16 additions & 0 deletions src/templates/template-vision-segmentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,20 @@

This is the segmentation template by Code-Generator using `deeplabv3_resnet101` and `cifar10` dataset from TorchVision and training is powered by PyTorch and PyTorch-Ignite.

**Note:**
The dataset used in this template is quite substantial, with a size of several GBs and automatically downloading it can be seen as an unexpected behaviour. To prevent unexpected behavior or excessive bandwidth usage, the automatic downloading of the dataset has been disabled by default.

To download the dataset:

```python
python -c "from data import download_datasets; download_datasets('/path/to/data')"
```

or

```py
from data import download_datasets
download_datasets('/path/to/data')
```

#::= from_template_common ::#