Skip to content

Commit e9b5c52

Browse files
authored
Merge pull request #1808 from BDoignies/Octree
SVO and SVDag
2 parents 21f68f2 + a3c1a0d commit e9b5c52

File tree

16 files changed

+1886
-5
lines changed

16 files changed

+1886
-5
lines changed

ChangeLog.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ git fetch origin
1010
git branch -u origin/main main
1111
git remote set-head origin -a
1212
```
13+
- *Kernel*
14+
- New DigitalSet model with SVO or SVDAG (Sparse Voxel Octree w/o acyclic graph features) for efficient representation of large digital objects (Bastien Doignies, [#1808](https://github.com/DGtal-team/DGtal/pull/1808))
1315

1416
- *Geometry*
1517
- Add a generic quick hull variant that can process arbitrary input lattice points and outputs their convex hull, even if it is not full dimensional (Jacques-Olivier Lachaud, [#1803](https://github.com/DGtal-team/DGtal/pull/1803))
@@ -59,8 +61,6 @@ git remote set-head origin -a
5961
- Build backend upgraded to scikit-build-core (Bastien Doignies, [#1804](https://github.com/DGtal-team/DGtal/pull/1804))
6062
- New CI for pypi deployement with cibuildwheel (Bastien Doignies, [#1804](https://github.com/DGtal-team/DGtal/pull/1804))
6163

62-
63-
6464
# DGtal 2.0
6565

6666
## New features

src/DGtal/doc/global.bib

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1631,3 +1631,28 @@ @article{Sharp:2019:VHM
16311631
address = {New York, NY, USA},
16321632
}
16331633

1634+
@inproceedings{laine2010SVO
1635+
author = {Laine, Samuli and Karras, Tero},
1636+
title = {Efficient sparse voxel octrees},
1637+
year = {2010},
1638+
isbn = {9781605589398},
1639+
publisher = {Association for Computing Machinery},
1640+
address = {New York, NY, USA},
1641+
url = {https://doi.org/10.1145/1730804.1730814},
1642+
doi = {10.1145/1730804.1730814},
1643+
booktitle = {Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games},
1644+
pages = {55–63},
1645+
numpages = {9},
1646+
location = {Washington, D.C.},
1647+
series = {I3D '10}
1648+
}
1649+
1650+
@article{Kampe2013SVDag,
1651+
title={High resolution sparse voxel DAGs},
1652+
author={Viktor K{\"a}mpe and Erik Sintorn and Ulf Assarsson},
1653+
journal={ACM Transactions on Graphics (TOG)},
1654+
year={2013},
1655+
volume={32},
1656+
pages={1 - 13},
1657+
url={https://api.semanticscholar.org/CorpusID:13022297}
1658+
}

src/DGtal/geometry/surfaces/estimation/estimationFunctors/SphereFittingEstimator.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
*
3131
* @see testLengthEstimators.cpp, testTrueLocalEstimator.cpp
3232
*/
33-
33+
3434
#if defined(SphereFittingEstimator_RECURSES)
3535
#error Recursive header files inclusion detected in SphereFittingEstimator.h
3636
#else // defined(SphereFittingEstimator_RECURSES)

src/DGtal/io/doc/moduleIO.dox

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -284,6 +284,56 @@ The mesh importation can be done automatically from the extension file name by u
284284
You can also export a Mesh object by using the operator
285285
(">>").
286286

287+
\subsection svoio Sparse Voxel Octree and Sparse Voxel Direct Acyclic Graphs
288+
289+
The static classes \c SVOReader and \c SVOWriter allow to import and export \c DigitalSetByOctree from and to files. In order to maintain an efficient data representation, DGtal uses a custom file format described below. Both SVO and SVDag are stored using the same file fomat.
290+
291+
Theses classes can be used as follows to import or export SVO:
292+
293+
@code
294+
// Writes the Octree
295+
SVOWriter<Z3i::Space>::exportSVO("file.svo", octree, compressed /* boolean */);
296+
// Reads it back
297+
auto newOctree = SVOReader<Z3i::Space>::importSVO("file.svo");
298+
@endcode
299+
300+
The reader allows an octree to be read in a different space of the same dimension. However, there may be integer truncation that may lead to unexpected results (a warning is issued in this case). Please note that BigIntegers are not supported for reading or writing.
301+
302+
\subsubsection svoformat SVO File format
303+
304+
The SVO file format is comprised of a header in ascii, and data in binary.
305+
306+
The header is written in ascii. Each line represents a different fields, whose name is space and case sensitive and for which the value is separated by a ':'. The value is written in ASCII format, and surrounding whitespace (spaces, tabs, carriage returns) is trimmed. There is no specified order, but we recommand following the order bellow. The header ends with a single '.' on a separated line. The expected fields are:
307+
- 'Format': always 'SVO'.
308+
- 'Version': unsigned integer, for now only '1' is supported.
309+
- 'Size': unsigned integer representing the number of voxels stored.
310+
- 'Dim': unsigned integer representing the dimension of voxels.
311+
- 'LowerBound': exactly 'Dim' integers separated by spaces representing the domain lower bound.
312+
- 'UpperBound': exactly 'Dim' integers separated by spaces representing the domain upper bound.
313+
- 'State': Indicates whether this is an SVO ('1') or an SVDag ('2').
314+
- 'Compression': Indicates whether data is compressed ('1') or not ('0').
315+
316+
For example, a header could be:
317+
318+
@code
319+
Format: SVO
320+
Version: 1
321+
Size: 5
322+
Dim: 3
323+
LowerBound: 5 5 5
324+
UpperBound: 12 12 12
325+
State: 2
326+
Compression: 1
327+
.
328+
@endcode
329+
330+
The data is always written in binary in little-endian order (regardless of host system). When compressed, data is compressed using zlib algorithm with the best compression level (9). The data is layed out as follows:
331+
- 8 bytes: 'SizeSize', the number of bytes array length occupies in memory
332+
- 8 bytes: 'PtrSize', the number of bytes array indices occupies in memory
333+
- [SizeSize] bytes: 'Depth', the depth of the underlying tree.
334+
- [Depth] * [SizeSize] bytes: 'ChildrenCount', the number of children at each depth of the tree.
335+
/bin/bash: line 1: :w: command not found
336+
287337
\section io_examples Examples
288338

289339
\subsection importDigitalSet Importing a 3D digital set from volume file:

src/DGtal/io/readers/SVOReader.h

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
/**
2+
* This program is free software: you can redistribute it and/or modify
3+
* it under the terms of the GNU Lesser General Public License as
4+
* published by the Free Software Foundation, either version 3 of the
5+
* License, or (at your option) any later version.
6+
*
7+
* This program is distributed in the hope that it will be useful,
8+
* but WITHOUT ANY WARRANTY; without even the implied warranty of
9+
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
10+
* GNU General Public License for more details.
11+
*
12+
* You should have received a copy of the GNU General Public License
13+
* along with this program. If not, see <http://www.gnu.org/licenses/>.
14+
*
15+
**/
16+
17+
#pragma once
18+
19+
/**
20+
* @file SVOReader.h
21+
* @author Bastien Doignies (bastien.doignies@liris.cnrs.fr)
22+
* Laboratoire d'InfoRmatique en Image et Systèmes d'information - LIRIS (CNRS, UMR 5205), CNRS, France
23+
*
24+
* @date 2025/10/20
25+
*
26+
* This file is part of the DGtal library.
27+
*/
28+
29+
#include <fstream>
30+
#include <map>
31+
#include <boost/endian/conversion.hpp>
32+
#include "DGtal/kernel/sets/DigitalSetByOctree.h"
33+
34+
namespace DGtal
35+
{
36+
/**
37+
* @brief Class to read SVO file.
38+
*
39+
* The file format is defined in SVOWriter.h
40+
*
41+
* @tparam Space The space on which the octree is defined
42+
* @see SVOWriter
43+
*/
44+
template <class Space>
45+
class SVOReader
46+
{
47+
public:
48+
using Octree = DigitalSetByOctree<Space>;
49+
/**
50+
* @brief Imports an octree from a file
51+
*
52+
* @param filename The file in which the octree is in
53+
*/
54+
static Octree importSVO(const std::string& filename);
55+
};
56+
}
57+
58+
#include "SVOReader.ih"

src/DGtal/io/readers/SVOReader.ih

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
/**
2+
* This program is free software: you can redistribute it and/or modify
3+
* it under the terms of the GNU Lesser General Public License as
4+
* published by the Free Software Foundation, either version 3 of the
5+
* License, or (at your option) any later version.
6+
*
7+
* This program is distributed in the hope that it will be useful,
8+
* but WITHOUT ANY WARRANTY; without even the implied warranty of
9+
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
10+
* GNU General Public License for more details.
11+
*
12+
* You should have received a copy of the GNU General Public License
13+
* along with this program. If not, see <http://www.gnu.org/licenses/>.
14+
*
15+
**/
16+
17+
#pragma once
18+
19+
/**
20+
* @file SVOReader.ih
21+
* @author Bastien Doignies (bastien.doignies@liris.cnrs.fr)
22+
* Laboratoire d'InfoRmatique en Image et Systèmes d'information - LIRIS (CNRS, UMR 5205), CNRS, France
23+
*
24+
* @date 2025/10/20
25+
*
26+
* This file is part of the DGtal library.
27+
*/
28+
29+
#include <boost/iostreams/filtering_stream.hpp>
30+
31+
namespace DGtal
32+
{
33+
template<typename T>
34+
inline static T getAs(const std::map<std::string, std::string>& fields, const std::string& key)
35+
{
36+
T result;
37+
38+
auto it = fields.find(key);
39+
if (it == fields.end())
40+
{
41+
trace.error() << "SVOReader: No such field \"" << key << "\"" << std::endl;
42+
throw IOException{};
43+
}
44+
45+
if constexpr (std::is_same_v<T, std::string>)
46+
{
47+
return it->second;
48+
}
49+
else
50+
{
51+
std::istringstream iss(it->second);
52+
iss >> result;
53+
54+
if (iss.fail())
55+
{
56+
trace.error() << "SVOReader: Can't convert \"" << key << "\" to desired type." << std::endl;
57+
throw IOException{};
58+
}
59+
60+
}
61+
return result;
62+
}
63+
64+
template <typename T>
65+
inline std::vector<T> getMultipleAs(const std::map<std::string, std::string>& fields, unsigned int count, const std::string& key)
66+
{
67+
std::vector<T> result;
68+
69+
auto it = fields.find(key);
70+
if (it == fields.end())
71+
{
72+
trace.error() << "SVOReader: No such field \"" << key << "\"" << std::endl;
73+
throw IOException{};
74+
}
75+
76+
if constexpr (std::is_same_v<T, std::string>)
77+
{
78+
return it->second;
79+
}
80+
else
81+
{
82+
std::istringstream iss(it->second);
83+
for (unsigned int i = 0; i < count; ++i)
84+
{
85+
T tmp; iss >> tmp;
86+
result.push_back(tmp);
87+
}
88+
89+
if (!iss.fail() && !iss.eof())
90+
{
91+
trace.error() << "SVOReader: Can't convert \"" << key << "\" to desired type." << std::endl;
92+
throw IOException{};
93+
}
94+
}
95+
return result;
96+
}
97+
98+
template<typename Int, typename Stream>
99+
Int readBinInt(Stream& stream, unsigned int count)
100+
{
101+
std::uint64_t value = 0;
102+
char byte;
103+
for (size_t i = 0; i < count; ++i)
104+
{
105+
if (!stream.get(byte)) throw IOException{};
106+
value |= static_cast<std::uint64_t>(byte) << (8 * i);
107+
}
108+
boost::endian::little_to_native_inplace(value);
109+
return static_cast<Int>(value);
110+
}
111+
112+
template <class Space>
113+
DigitalSetByOctree<Space> SVOReader<Space>::importSVO(const std::string& filename)
114+
{
115+
std::ifstream file(filename.c_str(), std::ios::binary);
116+
if (!file)
117+
{
118+
trace.warning() << "SVOReader: can't open file '" << filename << "'" << std::endl;
119+
throw IOException{};
120+
}
121+
122+
std::string line;
123+
std::map<std::string, std::string> header;
124+
while (std::getline(file, line))
125+
{
126+
if (!line.empty() && line.back() == '.') break;
127+
128+
auto split = line.find(":");
129+
if (split == std::string::npos)
130+
{
131+
trace.warning() << "SVOReader: unrecognized line \"" << line << "\"" << std::endl;
132+
continue;
133+
}
134+
135+
std::string key = line.substr(0, split);
136+
std::string val = line.substr(split + 1);
137+
138+
// Trim value
139+
auto start = val.find_first_not_of(" \t\r\n");
140+
auto end = val.find_last_not_of(" \t\r\n");
141+
if (start == std::string::npos)
142+
{
143+
trace.error() << "SVOReader: invalid value for line \"" << line << "\"" << std::endl;
144+
throw DGtal::IOException{};
145+
}
146+
147+
header[key] = val.substr(start, end - start + 1);
148+
}
149+
150+
const std::string format = getAs<std::string>(header, "Format");
151+
// const std::int32_t version = getAs<std::int32_t>(header, "Version");
152+
const std::int32_t compress = getAs<std::int32_t>(header, "Compression");
153+
const std::int32_t dim = getAs<std::int32_t>(header, "Dim");
154+
const std::int32_t state = getAs<std::int32_t>(header, "State");
155+
const std::uint64_t size = getAs<std::uint64_t>(header, "Size");
156+
157+
if (dim != Space::dimension)
158+
{
159+
trace.error() << "SVOReader: Dimension mismatch, space is " << Space::dimension << "d and Octree is " << dim << "d" << std::endl;
160+
throw IOException{};
161+
}
162+
163+
const auto lbdata = getMultipleAs<std::int64_t>(header, dim, "LowerBound");
164+
const auto ubdata = getMultipleAs<std::int64_t>(header, dim, "UpperBound");
165+
166+
typename Space::Point lb, ub;
167+
for (unsigned int i = 0; i < dim; ++i)
168+
{
169+
lb[i] = lbdata[i];
170+
ub[i] = ubdata[i];
171+
}
172+
173+
Octree octree(typename Octree::Domain(lb, ub));
174+
octree.mySize = size;
175+
octree.myState = static_cast<Octree::State>(state);
176+
177+
boost::iostreams::filtering_istream main;
178+
if (compress) main.push(boost::iostreams::zlib_decompressor());
179+
main.push(file);
180+
181+
const int64_t sizesize = readBinInt<int64_t>(main, sizeof(int64_t));
182+
const int64_t cellsize = readBinInt<int64_t>(main, sizeof(int64_t));
183+
184+
using CellI = typename Octree::CellIndex;
185+
using Size = typename Octree::Size;
186+
187+
if (sizesize > sizeof(Size))
188+
trace.warning() << "SVOReader: integer size mismatch, this may cause unwanted truncation" << std::endl;
189+
190+
if (cellsize > sizeof(CellI))
191+
trace.warning() << "SVOReader: integer size mismatch, this may cause unwanted truncation" << std::endl;
192+
193+
194+
octree.myNodes.resize(readBinInt<Size>(main, sizesize));
195+
for (size_t i = 0; i < octree.myNodes.size(); ++i)
196+
octree.myNodes[i].resize(readBinInt<Size>(main, sizesize));
197+
198+
try
199+
{
200+
for (size_t i = 0; i < octree.myNodes.size(); ++i)
201+
for (size_t j = 0; j < octree.myNodes[i].size(); ++j)
202+
for (unsigned int k = 0; k < Octree::CELL_COUNT; ++k)
203+
octree.myNodes[i][j].children[k] = readBinInt<CellI>(main, cellsize);
204+
}
205+
catch(...)
206+
{
207+
trace.error() << "SVOReader: Unexpected EOF while reading data." << std::endl;
208+
throw;
209+
}
210+
211+
return octree;
212+
}
213+
}

0 commit comments

Comments
 (0)