The loop
- Send the request with your filters.
- Process
data. - If
has_moreisfalse, stop. - Otherwise send the same request again, adding
cursorset tonext_cursor, and go to step 2.
limit, include and include_total may change between pages. A cursor sent with different filters gets 400 cursor-mismatch.
Short and empty pages
Do not use the page size to decide whether you are done. A page can hold fewer records thanlimit, or none at all, while has_more is still true:
- a broad search returns what each scanned window contains, and some windows are sparse;
- records that cannot be assembled at that moment are left out of the page.
has_more is false. Continuation pages with no records are not billed.
Consistency between pages
Pages are read live, not from a snapshot. A record that changes while you page can appear on two pages, or move past your position. Treat results as a set keyed byid: upsert, don’t append. To capture changes that happen during a long walk, take a changes feed cursor before you start and follow it afterwards.
Cursors
- Treat cursors as opaque strings; their format can change at any time.
- A search cursor is only meant for continuing that search. Store it if you want to resume a long walk later, but do not depend on old search cursors.
- A feed cursor is meant to be stored: it is your position in the feed. Store it after every page.
- A cursor that is damaged, truncated, or signed with a key that has been retired gets 400
invalid-cursor. Restart the search, or for the feed, restart withsince(see Syncing).
Page sizes
Use
limit=100 for bulk work. Every page is one billed call, whatever its size.