Commit e43edde
[SPARK-28878][SQL][FOLLOWUP] Remove extra project for DSv2 streaming scan
### What changes were proposed in this pull request?
Remove the project node if the streaming scan is columnar
### Why are the changes needed?
This is a followup of apache#25586. Batch and streaming share the same DS v2 read API so both can support columnar reads. We should apply apache#25586 to streaming scan as well.
### Does this PR introduce any user-facing change?
no
### How was this patch tested?
existing tests
Closes apache#25727 from cloud-fan/follow.
Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>1 parent 7c281b5 commit e43edde
1 file changed
Lines changed: 21 additions & 8 deletions
File tree
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2
Lines changed: 21 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
162 | 169 | | |
163 | 170 | | |
164 | 171 | | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
169 | 182 | | |
170 | 183 | | |
171 | 184 | | |
| |||
0 commit comments