Iterator only returns a maximum of the PrefetchSize. Bug?

Users Badger
1

What version of Go are you using (go version)?

go1.16.5 darwin/amd64

What operating system are you using?

OSX

What version of Badger are you using?

v3.2103.0

When using an iterator, I can only get the number of items specified in PrefetchSize. Specifically, I get the first x items (PrefetchSize/2) followed by the last x items (also PrefetchSize/2), and anything in the middle is not returned.

As a workaround I started using a prefetchsize of 1<<32 - 1 so that I get everything (assuming less than that many entries of course).

I’m 99% sure this is not a bug because it would be been noticed long ago, so I’m probably doing something wrong. How should I be iterating to get all entries if not like the example below?

err := db.View(func(txn *badger.Txn) error {
				opts := badger.DefaultIteratorOptions
				opts.PrefetchSize = 1<<32 - 1
				it := txn.NewIterator(opts)
				defer it.Close()
				for it.Rewind(); it.Valid(); it.Next() {
					item := it.Item()
					err := item.Value(func(v []byte) error {
						m = append(m, DBMap{
							Key:   item.Key(),
							Value: v,
						})
						return nil
					})
					if err != nil {
						return err
					}
				}
				return nil
			})