Retrieve Large Amounts of Entities Efficiently

If you are retrieving groups of records, it is possible that someone else will delete a record in one of the groups that you have already retrieved. Because entity.find counts each group of records from the beginning of the list, your next group may miss a record that filled in the space of the deleted record.

To avoid such issues, follow these best practice rules:

When sorting large groups of records:

sort_on = [a]
max_result = n


1. filter = f, and call entity.find

2. Lookup the last value in the result, get the value of a, which we’ll call x here.

3. Repeat the call, changing:
filter = f and (a>x)

This results in a fast search, with no chance of missing any records.

See also the documentation for the entity.find call.