Stage 4 of 4, lesson 26 of 31. Improver

Log every API call, and keep secrets out of the log

Put logging inside the one function that makes requests, so that every call to the practice API leaves a line saying what was asked and what came back, and nothing secret is ever written down.

Independent and unofficialCode run 12 Oct 2026

Risk warning: CFDs and spread bets are complex instruments and come with a high risk of losing money rapidly due to leverage. You can lose money on these products, and each provider publishes its own loss figure under FCA rules. You should consider whether you understand how they work and whether you can afford to take the high risk of losing your money. Automated and API trading can amplify losses through bugs, connectivity failures and runaway orders.

You need: The lessons on logging and on your first HTTP request. An internet connection. No broker account. No key.

The logging lesson logged rows from a list. The thing most worth logging in a trading program is its conversation with the API: what it asked for, when, and what it was told. When something goes wrong at three in the morning, that file is the only witness.

Because every request goes through one function, one place is enough. get logs the request at DEBUG before it is sent and the outcome after it: INFO when the reply is what was asked for, WARNING when the body carries an error. The screen shows INFO and above. The file keeps everything, with the time on each line.

What you leave out matters as much. A log is copied, attached to messages and pasted into forums. So the script logs the names in a body, not the body, and it passes headers through safe, which replaces the value of any header that proves who you are. The practice API needs no such header and none is sent. The example line shows safe at work on a made up token, so that you can see it hidden.

The script then reads its own log back and checks two things: that every request left a line, and that the made up token is nowhere in the file. The second check is one to keep. Search your logs for your secrets before someone else does.

The practice API this lesson calls

Practice data, not any broker's API: fixed files of invented values on this site. Each opens in a browser tab, so you can read what the script will be sent.

Every address on the practice API, and what it cannot do

The code

# Keep a log of every request your program makes: what it asked for and what came back.
# The practice API is a set of fixed files of invented values. It is not any broker's API.
# It needs no account and no key. The token below is made up, and it is never sent anywhere.
import json
import logging
import os
import sys
import time
import urllib.request

BASE_URL = os.environ.get("PRACTICE_API", "https://openbrokerapi.com/mock/v1")
PAUSE = 1   # seconds to leave between calls
LOG_FILE = "api-calls.log"
PATHS = ["/account.json", "/positions.json", "/errors/rate-limited.json"]

log = logging.getLogger("api")
log.setLevel(logging.DEBUG)
to_screen = logging.StreamHandler(sys.stdout)
to_screen.setLevel(logging.INFO)
to_screen.setFormatter(logging.Formatter("%(levelname)s %(message)s"))
log.addHandler(to_screen)
to_file = logging.FileHandler(LOG_FILE, mode="w")
to_file.setLevel(logging.DEBUG)
to_file.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
log.addHandler(to_file)

HIDE = ("authorization", "cookie", "x-api-key")


def safe(headers):
    """A copy of the headers that is fit to log: anything that proves who you are is hidden."""
    return {name: "[hidden]" if name.lower() in HIDE else value for name, value in headers.items()}


def get(path):
    headers = {"Accept": "application/json", "User-Agent": "log-your-api-calls-lesson"}
    log.debug("GET %s headers %s", path, safe(headers))
    request = urllib.request.Request(BASE_URL + path, headers=headers)
    with urllib.request.urlopen(request, timeout=10) as reply:
        status, body = reply.status, json.load(reply)
    if "error" in body:
        # Log the status and the code, which say what happened. Not the whole body.
        log.warning("GET %s status %s, and the body says %s %s", path, status, body["error"]["status"], body["error"]["code"])
    else:
        log.info("GET %s status %s, body has %s", path, status, ", ".join(sorted(body)))
    return body


# What safe does, on headers of the kind a broker's API asks for. Nothing here is sent.
example = {"Accept": "application/json", "Authorization": "Bearer not-a-real-token"}
log.info("Headers as they are logged: %s", safe(example))

log.info("Run started: %d requests to make", len(PATHS))
for number, path in enumerate(PATHS):
    if number:
        time.sleep(PAUSE)
    get(path)
log.info("Run finished")

# Close the log, then read it back as you would the morning after.
logging.shutdown()
with open(LOG_FILE) as f:
    text = f.read()
lines = text.splitlines()
print(LOG_FILE, "holds", len(lines), "lines")
for line in lines:
    print("  in the file:", line.split(" ", 2)[2])      # the line without its date and time

# The check. Every request left a line of its own, and the made up token is nowhere in the file.
requests_logged = [line for line in lines if " status " in line]
assert len(requests_logged) == len(PATHS), "a request was made and not logged"
assert "not-a-real-token" not in text, "a secret reached the log"
print("Check passed:", len(requests_logged), "requests logged, and no token in the file.")

What you should see

INFO Headers as they are logged: {'Accept': 'application/json', 'Authorization': '[hidden]'}
INFO Run started: 3 requests to make
INFO GET /account.json status 200, body has account, asOf, notice, practice
INFO GET /positions.json status 200, body has asOf, notice, positions, practice
WARNING GET /errors/rate-limited.json status 200, and the body says 429 rate_limited
INFO Run finished
api-calls.log holds 9 lines
  in the file: INFO Headers as they are logged: {'Accept': 'application/json', 'Authorization': '[hidden]'}
  in the file: INFO Run started: 3 requests to make
  in the file: DEBUG GET /account.json headers {'Accept': 'application/json', 'User-Agent': 'log-your-api-calls-lesson'}
  in the file: INFO GET /account.json status 200, body has account, asOf, notice, practice
  in the file: DEBUG GET /positions.json headers {'Accept': 'application/json', 'User-Agent': 'log-your-api-calls-lesson'}
  in the file: INFO GET /positions.json status 200, body has asOf, notice, positions, practice
  in the file: DEBUG GET /errors/rate-limited.json headers {'Accept': 'application/json', 'User-Agent': 'log-your-api-calls-lesson'}
  in the file: WARNING GET /errors/rate-limited.json status 200, and the body says 429 rate_limited
  in the file: INFO Run finished
Check passed: 3 requests logged, and no token in the file.

Run with Python 3.12 on Windows against a copy of the practice API files served over HTTP on the same machine, with PRACTICE_API set to the address of the copy. They are the files this site serves. No account and no key.

Check yourself

Answer each question from the code above before you open it. The answers were written from the script on this page and what it printed.

The file holds nine lines and the screen showed six from the logger. Which three are missing from the screen?

The three DEBUG lines, one written before each request with the headers. The screen handler is set to INFO, so it drops them. The file handler is set to DEBUG and keeps them.

Why is the rate limit request a WARNING when its status is 200?

The practice API always answers 200. get looks inside the body, finds an error object and logs the status and code it carries, 429 rate_limited, at WARNING.

How does safe decide which headers to hide?

It puts each header name into lower case and looks for it in HIDE: authorization, cookie and x-api-key. A match has its value replaced with [hidden]. Every other header is logged as it is.

Exercise

Add /errors/session-expired.json to PATHS. A 401 means the program cannot carry on, which is worse than being asked to slow down, so make get log a body that says 401 at ERROR and leave other errors at WARNING.

What the changed script prints

INFO Headers as they are logged: {'Accept': 'application/json', 'Authorization': '[hidden]'}
INFO Run started: 4 requests to make
INFO GET /account.json status 200, body has account, asOf, notice, practice
INFO GET /positions.json status 200, body has asOf, notice, positions, practice
WARNING GET /errors/rate-limited.json status 200, and the body says 429 rate_limited
ERROR GET /errors/session-expired.json status 200, and the body says 401 session_expired
INFO Run finished
api-calls.log holds 11 lines
  in the file: INFO Headers as they are logged: {'Accept': 'application/json', 'Authorization': '[hidden]'}
  in the file: INFO Run started: 4 requests to make
  in the file: DEBUG GET /account.json headers {'Accept': 'application/json', 'User-Agent': 'log-your-api-calls-lesson'}
  in the file: INFO GET /account.json status 200, body has account, asOf, notice, practice
  in the file: DEBUG GET /positions.json headers {'Accept': 'application/json', 'User-Agent': 'log-your-api-calls-lesson'}
  in the file: INFO GET /positions.json status 200, body has asOf, notice, positions, practice
  in the file: DEBUG GET /errors/rate-limited.json headers {'Accept': 'application/json', 'User-Agent': 'log-your-api-calls-lesson'}
  in the file: WARNING GET /errors/rate-limited.json status 200, and the body says 429 rate_limited
  in the file: DEBUG GET /errors/session-expired.json headers {'Accept': 'application/json', 'User-Agent': 'log-your-api-calls-lesson'}
  in the file: ERROR GET /errors/session-expired.json status 200, and the body says 401 session_expired
  in the file: INFO Run finished
Check passed: 4 requests logged, and no token in the file.

Captured from a run of the solution on 12 Oct 2026. Run with Python 3.12 on Windows against a copy of the practice API files served over HTTP on the same machine, with PRACTICE_API set to the address of the copy. They are the files this site serves. No account and no key.

One solution

# Exercise solution: the lesson script with the one change the exercise asks for.
# Keep a log of every request your program makes: what it asked for and what came back.
# The practice API is a set of fixed files of invented values. It is not any broker's API.
# It needs no account and no key. The token below is made up, and it is never sent anywhere.
import json
import logging
import os
import sys
import time
import urllib.request

BASE_URL = os.environ.get("PRACTICE_API", "https://openbrokerapi.com/mock/v1")
PAUSE = 1   # seconds to leave between calls
LOG_FILE = "api-calls.log"
PATHS = ["/account.json", "/positions.json", "/errors/rate-limited.json", "/errors/session-expired.json"]

log = logging.getLogger("api")
log.setLevel(logging.DEBUG)
to_screen = logging.StreamHandler(sys.stdout)
to_screen.setLevel(logging.INFO)
to_screen.setFormatter(logging.Formatter("%(levelname)s %(message)s"))
log.addHandler(to_screen)
to_file = logging.FileHandler(LOG_FILE, mode="w")
to_file.setLevel(logging.DEBUG)
to_file.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
log.addHandler(to_file)

HIDE = ("authorization", "cookie", "x-api-key")


def safe(headers):
    """A copy of the headers that is fit to log: anything that proves who you are is hidden."""
    return {name: "[hidden]" if name.lower() in HIDE else value for name, value in headers.items()}


def get(path):
    headers = {"Accept": "application/json", "User-Agent": "log-your-api-calls-lesson"}
    log.debug("GET %s headers %s", path, safe(headers))
    request = urllib.request.Request(BASE_URL + path, headers=headers)
    with urllib.request.urlopen(request, timeout=10) as reply:
        status, body = reply.status, json.load(reply)
    if "error" in body:
        # Log the status and the code, which say what happened. Not the whole body.
        level = logging.ERROR if body["error"]["status"] == 401 else logging.WARNING
        log.log(level, "GET %s status %s, and the body says %s %s", path, status, body["error"]["status"], body["error"]["code"])
    else:
        log.info("GET %s status %s, body has %s", path, status, ", ".join(sorted(body)))
    return body


# What safe does, on headers of the kind a broker's API asks for. Nothing here is sent.
example = {"Accept": "application/json", "Authorization": "Bearer not-a-real-token"}
log.info("Headers as they are logged: %s", safe(example))

log.info("Run started: %d requests to make", len(PATHS))
for number, path in enumerate(PATHS):
    if number:
        time.sleep(PAUSE)
    get(path)
log.info("Run finished")

# Close the log, then read it back as you would the morning after.
logging.shutdown()
with open(LOG_FILE) as f:
    text = f.read()
lines = text.splitlines()
print(LOG_FILE, "holds", len(lines), "lines")
for line in lines:
    print("  in the file:", line.split(" ", 2)[2])      # the line without its date and time

# The check. Every request left a line of its own, and the made up token is nowhere in the file.
requests_logged = [line for line in lines if " status " in line]
assert len(requests_logged) == len(PATHS), "a request was made and not logged"
assert "not-a-real-token" not in text, "a secret reached the log"
print("Check passed:", len(requests_logged), "requests logged, and no token in the file.")

Your progress

Next: Backtest an idea

Your marks are kept in this browser only, in its local storage. They are not sent anywhere, there is no account, and they will not follow you to another browser or device. To remove them, use Clear my progress on the learning track page, or clear this site's data in your browser's settings. Marking needs JavaScript. With scripts off the track reads the same, without the ticks.

Where this sits in the path

Stage 4 of 4: Building things that survive. Failure, limits, repeated orders, logs, settings, tests and backtests that do not fool you: what separates a script from a tool.

The whole path, every stage

Back: Log what happened Next: Backtest an idea