Skip to content

Commit 6308620

Browse files
authored
Docs update for release (#317)
* Update the version number Update the doc to include Vue Update the doc to give more details on the sync_cycle Update the doc to include python 3 * Update the LogWriter constructor doc. * Update the package copy directory. * Update the version to 0.0.2 (implying beta now) * Update the vdl_create_scratch_log to python 3 as well.
1 parent 3d09cd1 commit 6308620

File tree

9 files changed

+59
-24
lines changed

9 files changed

+59
-24
lines changed

README.cn.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,15 +70,18 @@ visualDL --logdir=scratch_log --port=8080
7070

7171
## SDK
7272
VisualDL 同时提供了python SDK 和 C++ SDK 来实现不同方式的使用。
73+
7374
### Python SDK
75+
VisualDL 现在支持 Python 2和 Python 3。
76+
7477
以最简单的Scalar组件为例,尝试创建一个scalar组件并插入多个时间步的数据:
7578

7679
```python
7780
import random
7881
from visualdl import LogWriter
7982

8083
logdir = "./tmp"
81-
logger = LogWriter(logdir, sync_cycle=10)
84+
logger = LogWriter(logdir, sync_cycle=10000)
8285

8386
# mark the components with 'train' label.
8487
with logger.mode("train"):
@@ -102,7 +105,7 @@ namespace cp = visualdl::components;
102105

103106
int main() {
104107
const std::string dir = "./tmp";
105-
vs::LogWriter logger(dir, 10);
108+
vs::LogWriter logger(dir, 10000);
106109

107110
logger.SetMode("train");
108111
auto tablet = logger.AddTablet("scalars/scalar0");

README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ New features will be continuously added.
1717
At present, most DNN frameworks use Python as their primary language. VisualDL supports Python by nature.
1818
Users can get plentiful visualization results by simply add a few lines of Python code into their model before training.
1919

20-
2120
Besides Python SDK, VisualDL was writen in C++ on the low level. It also provides C++ SDK that
2221
can be integrated into other platforms.
2322

@@ -80,14 +79,15 @@ VisualDL provides both Python SDK and C++ SDK in order to fit more use cases.
8079

8180

8281
### Python SDK
82+
VisualDL now supports both Python 2 and Python 3.
8383
Below is an example of creating a simple Scalar component and inserting data from different timestamps:
8484

8585
```python
8686
import random
8787
from visualdl import LogWriter
8888

8989
logdir = "./tmp"
90-
logger = LogWriter(logdir, sync_cycle=10)
90+
logger = LogWriter(logdir, sync_cycle=10000)
9191

9292
# mark the components with 'train' label.
9393
with logger.mode("train"):
@@ -112,7 +112,7 @@ namespace cp = visualdl::components;
112112

113113
int main() {
114114
const std::string dir = "./tmp";
115-
vs::LogWriter logger(dir, 10);
115+
vs::LogWriter logger(dir, 10000);
116116

117117
logger.SetMode("train");
118118
auto tablet = logger.AddTablet("scalars/scalar0");
@@ -161,7 +161,8 @@ pip install --upgrade dist/visualdl-*.whl
161161
162162
### Run a demo from scratch
163163
```
164-
vdl_scratch.py
164+
# vdl_create_scratch_log is a helper commend that creates mock data.
165+
vdl_create_scratch_log
165166
visualDL --logdir=scratch_log --port=8080
166167
```
167168
that will start a server locally on port 8080, then

VERSION_NUMBER

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
0.0.1-alpha.2
1+
0.0.2

demo/vdl_create_scratch_log

Lines changed: 21 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ from visualdl.server.log import logger as log
99

1010
logdir = './scratch_log'
1111

12-
logw = LogWriter(logdir, sync_cycle=30)
12+
logw = LogWriter(logdir, sync_cycle=2000)
1313

1414
# create scalars in mode train and test.
1515
with logw.mode('train') as logger:
@@ -50,13 +50,13 @@ with logw.mode("train") as logger:
5050
image0 = logger.image("scratch/random", 4)
5151

5252
dog_jpg = Image.open(os.path.join(ROOT, 'python/dog.jpg'))
53-
dog_jpg = dog_jpg.resize(np.array(dog_jpg.size) / 2)
53+
dog_jpg = dog_jpg.resize(np.floor_divide(np.array(dog_jpg.size), 2))
5454
shape = [dog_jpg.size[1], dog_jpg.size[0], 3]
5555

5656
# add dog's image
57-
for pass_ in xrange(4):
57+
for pass_ in range(4):
5858
image.start_sampling()
59-
for sample in xrange(10):
59+
for sample in range(10):
6060
# randomly crop a dog's image.
6161
target_shape = [100, 100, 3] # width, height, channels(3 for RGB)
6262
left_x = random.randint(0, shape[1] - target_shape[1])
@@ -82,9 +82,9 @@ with logw.mode("train") as logger:
8282
image.finish_sampling()
8383

8484
# add randomly generated image
85-
for pass_ in xrange(4):
85+
for pass_ in range(4):
8686
image0.start_sampling()
87-
for sample in xrange(10):
87+
for sample in range(10):
8888
shape = [40, 30, 3]
8989
data = np.random.random(shape).flatten()
9090
image0.add_sample(shape, list(data))
@@ -98,10 +98,23 @@ def download_graph_image():
9898
9999
For real cases, just refer to README.
100100
'''
101-
import urllib
101+
102+
import sys
103+
104+
if sys.version_info[0] == 3:
105+
import urllib.request as ur
106+
107+
else:
108+
# Not Python 3 - today, it is most likely to be Python 2
109+
import urllib as ur
110+
111+
import ssl
112+
myssl = ssl.create_default_context()
113+
myssl.check_hostname = False
114+
myssl.verify_mode = ssl.CERT_NONE
102115
image_url = "https://github.com/PaddlePaddle/VisualDL/blob/develop/demo/mxnet/super_resolution_graph.png?raw=true"
103116
log.warning('download graph demo from {}'.format(image_url))
104-
graph_image = urllib.urlopen(image_url).read()
117+
graph_image = ur.urlopen(image_url, context=myssl).read()
105118
with open(os.path.join(logdir, 'graph.jpg'), 'wb') as f:
106119
f.write(graph_image)
107120
log.warning('graph ready!')

docs/how_to_dev_frontend_en.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ The VisualDL Web app uses multiple frameworks to help manage the project. They a
3131

3232
1. webpack: To manage all assets
3333
1. npm: To manage dependencies
34-
1. San: Javascript Component framework
34+
1. Vue: Javascript Component framework
3535
1. ECharts: To pilot charts
3636

3737
## Webpack
@@ -63,13 +63,13 @@ npm install
6363

6464
This command will go through `package.json` and install the dependencies in the local node_modules folder.
6565

66-
## San
66+
## Vue
6767

68-
San is a JavaScript component framework that helps the developer to implement web component in MVVM architecture pattern.
68+
Vue is a JavaScript component framework that helps the developer to implement web component in MVVM architecture pattern.
6969

70-
San allows you to define a self-contained view model in a .san file and attach view model objects to DOM objects.
70+
Vue allows you to define a self-contained view model in a .vue file and attach view model objects to DOM objects.
7171

72-
To learn more about [san](https://github.com/ecomfe/san)
72+
To learn more about [Vue](https://vuejs.org/)
7373

7474
## ECharts
7575

docs/quick_start_cn.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,16 @@ VisualDL提供原生的Python和C++ SDK,可以支持多种深度学习平台
2020
from VisualDL import LogWriter
2121
from random import random
2222

23-
logw = LogWriter("./random_log", sync_cycle=30)
23+
logw = LogWriter("./random_log", sync_cycle=10000)
2424
```
2525

2626
其中, 第一个参数指定存储数据的目录;第二个参数 `sync_cycle` 指定多少次写操作执行一次内存到磁盘的数据持久化。
2727

28+
### sync_cycle
29+
写IO是一项繁重的工作。设置` sync_cycle `太低可能会减慢你的训练。
30+
我们建议将 `sync_cycle` 设置为约要捕捉的数据点的两倍。
31+
32+
2833
模型训练会有不同的模式,比如训练、验证、测试等,这些对应到 VisualDL中就是 `mode`,可以用如下方式指定一个训练模式
2934

3035
```python

docs/quick_start_en.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,15 @@ The first step of using VisualDL is to create a `LogWriter' that can store visua
2323
from VisualDL import LogWriter
2424
from random import random
2525

26-
logw = LogWriter("./random_log", sync_cycle=30)
26+
logw = LogWriter("./random_log", sync_cycle=10000)
2727
```
2828

2929
The first parameter points to a folder; the second parameter `sync_cycle` specifies out of how memory operations should be
30-
store the data into hard drive.
30+
store the data into hard drive.
31+
32+
### sync_cycle
33+
Writing is a heavy operation. Setting `sync_cycle` might slow down your training.
34+
A good starting point is to set the `sync_cycle` to be at least twice the amount of data point your would like to capture.
3135

3236
There are different modes for model training, such as training, validating and testing. All these correspond to `mode' in VisualDL.
3337
We can use the following pattern to specify mode:

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ def run(self):
9090
install_requires=install_requires,
9191
package_data={
9292
'visualdl.server':
93-
['dist/*.js', 'dist/*.html', 'dist/fonts/*', 'dist/images/*'],
93+
['dist/*.js', 'dist/*.html', 'dist/fonts/*', 'dist/assets/*'],
9494
'visualdl': ['core.so'],
9595
'visualdl.python': ['core.so', 'dog.jpg']
9696
},

visualdl/python/storage.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,15 @@ class LogWriter(object):
126126
"""LogWriter is a Python wrapper to write data to log file with the data
127127
format defined in storage.proto. A User can get Scalar Reader/Image Reader/
128128
Histogram Reader from this module and use them to write the data to log file.
129+
130+
:param dir: The directory path to the saved log files.
131+
:type dir: basestring
132+
:param sync_cycle: Specify how often should the system store data into the file system.
133+
Typically adding a record requires 6 operations.
134+
System will save the data into the file system once operations count reaches sync_cycle.
135+
:type sync_cycle: integer
136+
:return: a new LogWriter instance
137+
:rtype: LogWriter
129138
"""
130139

131140
cur_mode = None

0 commit comments

Comments
 (0)