Broker API HubMenu

Stage 4 of 4, lesson 22 of 31. Improver

Stay under a rate limit on purpose

Build a small pacer that knows how many calls you have made in the last window and tells you how long to wait. It introduces classes, a sliding window and testing with a pretend clock.

Independent and unofficialCode run

Risk warning: CFDs and spread bets are complex instruments and come with a high risk of losing money rapidly due to leverage. You can lose money on these products, and each provider publishes its own loss figure under FCA rules. You should consider whether you understand how they work and whether you can afford to take the high risk of losing your money. Automated and API trading can amplify losses through bugs, connectivity failures and runaway orders.

You need: The lesson on retry and backoff. No broker account.

Backing off after a refusal is the safety net. Better is never to be refused: know the limit and pace yourself under it.

The pacer remembers when each recent call was made. Before the next call it asks one question: how many have I sent in the last window? If that is under the limit it goes ahead. If not, it waits exactly until the oldest call falls out of the window.

The example uses a pretend clock, so eight calls that would take twelve seconds run instantly and the output shows exactly when each would be sent. Passing the clock in like this is a habit worth keeping: it is what makes time-based code testable.

Limits come in several scopes, such as per application, per account and per kind of call. The table below shows how one broker publishes them. Use one pacer for each limit that applies to you, and where a broker's own pages disagree, pace against the lower figure.

The code

# Stay under a rate limit on purpose, instead of finding it by being refused.
# No broker and no internet. A pretend clock lets the example finish instantly.


class Pacer:
    """Allows at most `limit` calls in any `window` seconds, and says how long to wait otherwise."""

    def __init__(self, limit, window, clock):
        self.limit = limit
        self.window = window
        self.clock = clock
        self.sent = []          # the times of recent calls

    def wait_needed(self):
        now = self.clock()
        self.sent = [t for t in self.sent if now - t < self.window]   # forget calls outside the window
        if len(self.sent) < self.limit:
            return 0.0
        return self.window - (now - self.sent[0])    # wait until the oldest call drops out

    def record(self):
        self.sent.append(self.clock())


fake_now = 0.0


def clock():
    return fake_now


# Pace at 5 calls every 10 seconds, to keep the example short.
pacer = Pacer(limit=5, window=10, clock=clock)

for call_number in range(1, 9):
    wait = pacer.wait_needed()
    if wait > 0:
        print("Call", call_number, "must wait", round(wait, 1), "seconds")
        fake_now += wait            # a real program would call time.sleep(wait) here
    pacer.record()
    print("Call", call_number, "sent at", round(fake_now, 1), "seconds")
    fake_now += 1                   # one second of work between calls

What you should see

Call 1 sent at 0.0 seconds
Call 2 sent at 1.0 seconds
Call 3 sent at 2.0 seconds
Call 4 sent at 3.0 seconds
Call 5 sent at 4.0 seconds
Call 6 must wait 5.0 seconds
Call 6 sent at 10.0 seconds
Call 7 sent at 11.0 seconds
Call 8 sent at 12.0 seconds

Run with Python 3.12 on Windows. No network and no account.

IG limits, from the register

LimitValueScopeSource
Historical price data points a week10,000Per account, price history endpoints onlylabs.ig.comchecked
Non trading requests a minute30Per account, non trading calls onlylabs.ig.comchecked
Non trading requests a minute60Per application, that is per API key, non trading calls onlylabs.ig.comchecked
Concurrent streaming subscriptions40Per streaming connectionlabs.ig.comchecked corroborated by labs.ig.com/streaming-api-guide.html
Trading requests a minute100Sources conflict ig.com states around 40, checked . IG's own pages disagree. The conflict is labelled, not resolved. Pace against 40.Per account, trading calls onlylabs.ig.comchecked

Errors you may meet

Check yourself

Answer each question from the code above before you open it. The answers were written from the script on this page and what it printed.

Why must call 6 wait exactly 5.0 seconds?

Five calls went out at 0, 1, 2, 3 and 4 seconds, which fills the limit of five in ten seconds. The oldest leaves the window at 10 seconds. The clock reads 5, so the wait is 10 minus 5.

What does the list comprehension at the top of wait_needed do?

It keeps only the calls made less than one window ago and forgets the rest, so the count that follows is the number of calls inside the window.

Why is the clock passed into the Pacer instead of the Pacer reading the time itself?

So that a test can hand it a pretend clock. The example moves time forward by adding to fake_now, which lets twelve seconds of pacing run in an instant and print the same thing every time.

Exercise

Lower the limit from 5 calls to 3 calls in the same ten second window and run it. Work out when call 4 will be sent before you look.

What the changed script prints
Call 1 sent at 0.0 seconds
Call 2 sent at 1.0 seconds
Call 3 sent at 2.0 seconds
Call 4 must wait 7.0 seconds
Call 4 sent at 10.0 seconds
Call 5 sent at 11.0 seconds
Call 6 sent at 12.0 seconds
Call 7 must wait 7.0 seconds
Call 7 sent at 20.0 seconds
Call 8 sent at 21.0 seconds

Captured from a run of the solution on . Run with Python 3.12 on Windows. No network and no account.

One solution
# Exercise solution: the lesson script with the one change the exercise asks for.
# Stay under a rate limit on purpose, instead of finding it by being refused.
# No broker and no internet. A pretend clock lets the example finish instantly.


class Pacer:
    """Allows at most `limit` calls in any `window` seconds, and says how long to wait otherwise."""

    def __init__(self, limit, window, clock):
        self.limit = limit
        self.window = window
        self.clock = clock
        self.sent = []          # the times of recent calls

    def wait_needed(self):
        now = self.clock()
        self.sent = [t for t in self.sent if now - t < self.window]   # forget calls outside the window
        if len(self.sent) < self.limit:
            return 0.0
        return self.window - (now - self.sent[0])    # wait until the oldest call drops out

    def record(self):
        self.sent.append(self.clock())


fake_now = 0.0


def clock():
    return fake_now


# Pace at 3 calls every 10 seconds, to keep the example short.
pacer = Pacer(limit=3, window=10, clock=clock)

for call_number in range(1, 9):
    wait = pacer.wait_needed()
    if wait > 0:
        print("Call", call_number, "must wait", round(wait, 1), "seconds")
        fake_now += wait            # a real program would call time.sleep(wait) here
    pacer.record()
    print("Call", call_number, "sent at", round(fake_now, 1), "seconds")
    fake_now += 1                   # one second of work between calls

Your progress

Next: Handle an error reply

Your marks are kept in this browser only, in its local storage. They are not sent anywhere, there is no account, and they will not follow you to another browser or device. To remove them, use Clear my progress on the learning track page, or clear this site's data in your browser's settings. Marking needs JavaScript. With scripts off the track reads the same, without the ticks.

Where this sits in the path

Stage 4 of 4: Building things that survive. Failure, limits, repeated orders, logs, settings, tests and backtests that do not fool you: what separates a script from a tool.

  1. Retry and backoff
  2. Stay under a rate limitYou are here
  3. Handle an error reply
  4. Idempotent orders
  5. Log what happened
  6. Log your API calls
  7. Backtest an idea
  8. The look ahead mistake
  9. Settings from environment
  10. Test with saved replies
  11. A command line tool
  12. The capstone project

The whole path, every stage

Back: Retry and backoffNext: Handle an error reply