Commit 78ce2cb
fix(streaming): _S3SeekableIO.__next__/__iter__ don't advance self._position, corrupting subsequent seek()/read() (#8389)
* fix(streaming): _S3SeekableIO.__next__/__iter__ don't advance self._position, corrupting subsequent seek()/read()
__next__ and __iter__ delegated straight to raw_stream.__next__()/raw_stream.__iter__(),
bypassing self._position bookkeeping entirely -- unlike read()/readline()/readlines(),
which all correctly advance self._position by the number of bytes actually consumed.
seek() uses self._position as ground truth to compute the S3 GetObject Range header on
the next raw_stream access. Consuming any data via the iterator protocol (e.g.
"for line in s3_object: ...", a fully supported documented usage) left self._position
stuck at its pre-iteration value. A subsequent seek()/read() would then reopen the S3
stream with a Range header computed from that stale position -- silently returning the
wrong slice of the object (skipped or duplicated bytes), with no exception raised.
Fix: route __next__ through the already position-tracked readline(), and have __iter__
return self, matching the standard Python file-iterator protocol.
* fix(streaming): preserve iteration chunk behavior
---------
Co-authored-by: Leandro <lcdama@amazon.pt>1 parent e0566b0 commit 78ce2cb
2 files changed
Lines changed: 60 additions & 2 deletions
File tree
- aws_lambda_powertools/utilities/streaming
- tests/functional/streaming/_boto3
Lines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
169 | 169 | | |
170 | 170 | | |
171 | 171 | | |
172 | | - | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
173 | 175 | | |
174 | 176 | | |
175 | | - | |
| 177 | + | |
176 | 178 | | |
177 | 179 | | |
178 | 180 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
163 | 163 | | |
164 | 164 | | |
165 | 165 | | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
166 | 222 | | |
167 | 223 | | |
168 | 224 | | |
| |||
0 commit comments