Stage 3 of 4, lesson 18 of 31. Improver

Collect a long history one page at a time

Broker APIs hand back long histories in pages. Loop until the last page and collect every row. It introduces while loops with a stop condition and separating the call from the logic.

Independent and unofficialCode run 12 Oct 2026

Risk warning: CFDs and spread bets are complex instruments and come with a high risk of losing money rapidly due to leverage. You can lose money on these products, and each provider publishes its own loss figure under FCA rules. You should consider whether you understand how they work and whether you can afford to take the high risk of losing your money. Automated and API trading can amplify losses through bugs, connectivity failures and runaway orders.

You need: The lesson on reading an API response. No broker account.

Ask a broker for a year of transactions and you will not get them in one reply. You get the first page and a note saying how many pages there are, or where the next one is. Miss that note and your export is silently incomplete.

The script keeps asking for the next page until the reply says there are no more. The call itself is kept in its own small function. Here it reads from a file. In real use that one function is the only part that changes to a real request.

Each broker pages differently: some give a page number and a total, some give a link to follow, some give a cursor. The loop is always the same shape. Check the history endpoint on your broker's page for which it uses.

Files for this lesson

mock-history-pages.json, sample data with invented values

The code

# Collect a long history that arrives one page at a time.
# Download mock-history-pages.json from this lesson and keep it beside this script.
# It stands in for a broker's history endpoint. The values are invented.
import json

with open("mock-history-pages.json") as f:
    PAGES = json.load(f)["pages"]


def get_history(page_number):
    """Pretend to call the API. A real call would send the page number as a parameter."""
    return PAGES[page_number - 1]


everything = []
page = 1
while True:
    answer = get_history(page)
    everything.extend(answer["transactions"])
    print("Page", page, "of", answer["paging"]["totalPages"], "gave", len(answer["transactions"]), "rows")
    if page >= answer["paging"]["totalPages"]:
        break        # stop when the API says there are no more pages
    page += 1

print("Rows collected:", len(everything))
print("First:", everything[0]["reference"], "Last:", everything[-1]["reference"])
print("Total profit and loss:", round(sum(row["profitAndLoss"] for row in everything), 2))

What you should see

Page 1 of 3 gave 4 rows
Page 2 of 3 gave 4 rows
Page 3 of 3 gave 2 rows
Rows collected: 10
First: MOCK001 Last: MOCK010
Total profit and loss: 103.45

Run with Python 3.12 on Windows against the sample files on this page. No network and no account.

Endpoints this lesson uses

Check yourself

Answer each question from the code above before you open it. The answers were written from the script on this page and what it printed.

What makes the loop stop?

After each page the script compares the page number with totalPages from that reply. When the page number has reached it, break leaves the loop.

What would happen on this mock if the two lines with the break were removed?

The loop would ask for a fourth page. The mock holds three, so get_history would fail with an IndexError. Against a real API, a loop with no stop condition keeps sending requests.

Why is get_history a function of its own when it is one line long?

It is the only part that knows where the data comes from. Swapping the file for a real request means changing that function and nothing in the loop.

Exercise

After the total, print how many of the collected rows are losses, as Losing rows.

What the changed script prints

Page 1 of 3 gave 4 rows
Page 2 of 3 gave 4 rows
Page 3 of 3 gave 2 rows
Rows collected: 10
First: MOCK001 Last: MOCK010
Total profit and loss: 103.45
Losing rows: 3

Captured from a run of the solution on 12 Oct 2026. Run with Python 3.12 on Windows against the sample files on this page. No network and no account.

One solution

# Exercise solution: the lesson script with the one change the exercise asks for.
# Collect a long history that arrives one page at a time.
# Download mock-history-pages.json from this lesson and keep it beside this script.
# It stands in for a broker's history endpoint. The values are invented.
import json

with open("mock-history-pages.json") as f:
    PAGES = json.load(f)["pages"]


def get_history(page_number):
    """Pretend to call the API. A real call would send the page number as a parameter."""
    return PAGES[page_number - 1]


everything = []
page = 1
while True:
    answer = get_history(page)
    everything.extend(answer["transactions"])
    print("Page", page, "of", answer["paging"]["totalPages"], "gave", len(answer["transactions"]), "rows")
    if page >= answer["paging"]["totalPages"]:
        break        # stop when the API says there are no more pages
    page += 1

print("Rows collected:", len(everything))
print("First:", everything[0]["reference"], "Last:", everything[-1]["reference"])
print("Total profit and loss:", round(sum(row["profitAndLoss"] for row in everything), 2))
print("Losing rows:", len([row for row in everything if row["profitAndLoss"] < 0]))

Your progress

Next: Follow the next page

Your marks are kept in this browser only, in its local storage. They are not sent anywhere, there is no account, and they will not follow you to another browser or device. To remove them, use Clear my progress on the learning track page, or clear this site's data in your browser's settings. Marking needs JavaScript. With scripts off the track reads the same, without the ticks.

Where this sits in the path

Stage 3 of 4: Talking to a broker. What an API sends back, real requests to a practice API that needs no account, how to keep a key safe, a first read only call on a demo account, and histories that arrive in pages.

The whole path, every stage

Back: First API call Next: Follow the next page