
Customer Stories
700+ customers in year one: How Blacksmith built scalable usage-based billing for CI/CD with Lago
Anh-Tho Chuong • 2 min read
Mar 30
/15 min read
Building a robust REST API for billing systems requires careful attention to endpoint patterns, authentication mechanisms, error handling strategies, and API versioning approaches. According to recent API governance research, 68% of API failures in production stem from inadequate design decisions made during the architectural phase rather than implementation bugs [1]. This guide explores proven best practices for designing billing REST APIs that scale reliably, maintain backward compatibility, and provide excellent developer experience.
Billing APIs handle sensitive financial data and transactions that directly impact revenue recognition and compliance reporting. The architectural decisions you make during API design phase determine whether your system can adapt to new billing models, support multiple payment processors, and maintain PCI-DSS compliance as your business scales. Unlike consumer-facing APIs, billing APIs must balance developer convenience with enterprise-grade security, audit trails, and data consistency requirements.
Effective billing REST API design begins with establishing clear architectural principles that guide all endpoint design decisions. These principles should prioritize idempotency, immutability of financial records, and explicit state transitions. A well-designed billing API prevents duplicate charges, ensures audit trail completeness, and makes it straightforward for developers to understand how data flows through the system. Research from API design communities shows that APIs following consistent patterns reduce integration time by 40% and decrease bug rates in client implementations [2].
Financial APIs must enforce strict separation between read and write operations. GET requests retrieve data without side effects; POST/PATCH/DELETE requests modify state with explicit intent. This separation prevents accidental mutations and makes API behavior predictable. Billing systems should never use GET requests to trigger state changes like marking an invoice as paid or processing a refund. Every state-changing operation requires an explicit HTTP method that communicates intent clearly to developers using the API.
Immutability of financial records represents another core principle. Once an invoice is issued, its line items, amounts, and tax calculations should never change retroactively. Instead, billing systems issue credit notes or amendments as separate records. This design ensures audit trails remain accurate and historical records retain integrity for compliance purposes. APIs should make immutability explicit by rejecting UPDATE requests to financial records while providing alternative endpoints for corrections and adjustments.
Billing APIs require careful endpoint hierarchy design that maps cleanly to financial concepts while remaining intuitive for developers. A well-structured endpoint hierarchy typically follows the pattern /v1/organizations/{org_id}/resources/{resource_id}, which ensures multi-tenancy isolation and clear resource ownership. Each endpoint should represent a single financial concept—customers, subscriptions, invoices, payment intents, refunds—rather than combining multiple concepts into single endpoints.
Collection endpoints should follow RESTful conventions: GET /v1/customers lists customers, POST /v1/customers creates a new customer, GET /v1/customers/{id} retrieves a specific customer. This familiar pattern reduces cognitive load for developers. However, billing APIs should also provide scoped filtering: GET /v1/customers/{customer_id}/invoices retrieves all invoices for a specific customer, avoiding the need for query parameters that might be misused. Scoped endpoints prevent common mistakes where developers accidentally retrieve all invoices in the system instead of customer-specific ones.
Nested resource endpoints should follow clear ownership hierarchies. A subscription belongs to a customer, so GET /v1/customers/{customer_id}/subscriptions/{subscription_id} represents the natural ownership structure. Line items belong to invoices, so GET /v1/invoices/{invoice_id}/line_items lists them. This hierarchy prevents ID collision issues and makes authorization rules easier to implement—you can verify the customer owns the subscription before returning it. Platforms like Lago implement endpoint hierarchies that transparently handle these ownership relationships through their API-first architecture.
Action-based endpoints should be minimal and purposeful in billing APIs. Instead of creating separate endpoints like POST /v1/invoices/{id}/send or POST /v1/invoices/{id}/archive, include state transitions through the PATCH endpoint: PATCH /v1/invoices/{id} with {status: "sent"} or {status: "archived"}. This pattern keeps the endpoint surface minimal while maintaining clarity. For complex operations like processing refunds with multiple validation steps, a dedicated action endpoint is justified: POST /v1/refunds/{id}/process includes validation and triggers the actual refund operation.
Billing REST APIs must enforce strict authentication and authorization to protect sensitive financial data. API key authentication remains standard for server-to-server integration, while OAuth 2.0 is required for user-facing applications accessing billing data on behalf of customers. According to 2025 API security research, 71% of data breaches in SaaS platforms involved compromised API credentials [3]. Implementing proper authentication mechanisms is non-negotiable for billing systems.
API keys should be issued with scopes that limit their access to specific resources. A key used for reading customer billing history should not have authority to process refunds or modify payment methods. Implement scope-based access control where API keys carry explicit permissions like "read:invoices", "write:subscriptions", "process:payments". This principle of least privilege prevents cascading damage if an API key is compromised—the attacker only gains access to specific operations, not the entire billing system.
Request signing adds an additional security layer for sensitive operations. Implementations can require HMAC-SHA256 signatures for POST/PATCH/DELETE requests using a shared secret. The signature proves the request originated from the holder of the secret key and wasn't modified in transit. This approach is particularly important for webhook handlers receiving billing notifications—signature verification ensures only legitimate billing platform messages trigger state changes in your system.
Authorization should be enforced at the endpoint handler level, not just at the network layer. Every request should verify the authenticated principal (API key, OAuth token) has permission to access the requested resource. A customer's API key should only retrieve data for their organization, never data from other organizations. This multi-tenant isolation is fundamental—bugs in authorization logic can leak sensitive data across organizational boundaries. Implement authorization checks before returning any data, not as a secondary validation step.
Collection endpoints in billing APIs must support pagination to prevent timeouts when retrieving large datasets. Cursor-based pagination is preferred over offset-based pagination because it remains stable even when records are inserted or deleted during pagination. With offset-based pagination, a customer might miss records or see duplicates if new data is inserted between requests. Cursor-based pagination returns a pointer to the last record retrieved, allowing the next request to start precisely where the previous one ended.
Pagination responses should include clear metadata about available records: total count, current page size, whether additional records exist, and the cursor for the next page. A pagination response like {items: [...], pagination: {total: 5000, page_size: 100, has_more: true, next_cursor: "abc123"}} gives developers everything needed to implement efficient list UI. The has_more boolean is particularly useful—developers don't need to check if the result set is smaller than requested; they can simply use has_more to determine if more data exists.
Filtering should support common billing queries without requiring complex query languages. Endpoints like GET /v1/invoices?status=draft&created_after=2026-01-01 handle most real-world use cases. Support filters for status, date ranges (using _after and _before suffixes), and specific fields relevant to billing operations. Avoid supporting arbitrary field filtering through query parameters—this creates security risks where developers might filter on sensitive fields they shouldn't query directly. Document which fields are filterable to guide developers toward approved query patterns.
Sorting should be explicit and limited to fields where sorting makes business sense. GET /v1/invoices?sort=-created_at (newer invoices first) or GET /v1/invoices?sort=total_amount (ascending by amount) are reasonable. Restrict sorting to avoid expensive queries on unindexed fields. Include a default sort order in responses when sorting is not specified. For billing data, the default is typically reverse chronological (newest first), matching user expectations for invoice lists.
Billing APIs must communicate errors clearly because incorrect error interpretation can cause double charges, missed payments, or compliance violations. Use standard HTTP status codes with precise semantics: 400 for malformed requests, 401 for authentication failures, 403 for authorization failures, 404 for missing resources, 409 for conflict states (like charging a cancelled subscription), and 422 for validation errors. Industry data shows APIs with poor error communication experience 3x higher support ticket volume [4]. Consistent error handling reduces developer confusion and accelerates integration.
Error responses should include a machine-readable error code in addition to human-readable messages. A 409 Conflict response might include {error_code: "SUBSCRIPTION_ALREADY_ACTIVE"} instead of just a generic message. Error codes allow developers to implement specific recovery logic—they might retry 5xx errors, skip duplicate charge attempts if the code is "IDEMPOTENCY_KEY_ALREADY_PROCESSED", or prompt users to fix validation errors. This structure transforms error responses from merely informational to actionable recovery instructions.
Include request IDs in error responses so developers can reference specific failures in support conversations. A response like {request_id: "req_abc123", error_code: "CUSTOMER_NOT_FOUND"} allows support teams to trace the exact request in server logs. This dramatically speeds up troubleshooting—developers don't need to recreate the error, they simply provide the request ID. Implement request ID tracking throughout your system so every step of processing (database queries, external API calls, authorization checks) is associated with the original request.
Validation errors deserve special handling in billing contexts. Instead of returning a single validation error, return all validation failures in one response. This allows developers to fix multiple issues at once rather than receiving one error, fixing it, submitting again, and receiving the next error. A response like {error_code: "VALIDATION_FAILED", errors: [{field: "amount", message: "must be greater than zero"}, {field: "customer_id", message: "customer does not exist"}]} saves round trips and developer frustration.
Idempotency is critical in billing APIs because network failures, timeouts, and retries are inevitable. If a customer's mobile app submits a charge request, doesn't receive a response within the timeout period, and automatically retries, the API must recognize this as the same request and not charge twice. Idempotency keys—unique identifiers provided by the client—enable this behavior. Every POST request to create a financial record should accept an optional X-Idempotency-Key header (or request body field). If the same idempotency key is submitted twice, the second request returns the same result as the first without executing the operation again.
Idempotent endpoint implementations should store idempotency keys with their results. When a request with an existing idempotency key arrives, return the cached result immediately. This prevents duplicate state changes and ensures the requesting system receives consistent responses even across retries. Database implementations might store idempotency keys in a separate table: {key: "abc123", method: "POST", endpoint: "/invoices", status_code: 200, response_body: {...}, created_at: ...}. Old entries can be purged after 24 hours—any request retried after 24 hours is acceptable to charge again.
Idempotency must be implemented at the database level, not just the application level. An application server can ensure it doesn't execute logic twice, but if the database write fails after returning a response, the next retry needs to recognize the operation already succeeded. Use database constraints and unique indexes to enforce idempotency at the storage layer. A payment table might have a unique index on (idempotency_key, customer_id) to guarantee no duplicate payments for the same key within a customer's account.
Document idempotency requirements in your API documentation. Specify which endpoints are idempotent (typically GET, POST to create resources) and which are not (PATCH, DELETE). For idempotent endpoints, explain the idempotency key format and TTL. Provide code examples showing proper idempotency key generation using UUIDs or hashes. This guidance prevents developers from bypassing idempotency mechanisms because they don't understand their purpose.
API versioning allows billing systems to evolve without breaking existing integrations. The two common approaches are URL versioning (v1, v2 in the path) and header versioning (Accept: application/vnd.billing.v1+json). URL versioning is more explicit and easier for developers to discover—they can clearly see they're using the v1 API and might see v2 exists. However, header versioning keeps URLs cleaner and allows more gradual version deprecation. Most billing APIs use URL versioning because the explicit version makes the API contract obvious to developers reading integration code.
Major version changes should be rare and well-planned. Changing the response format for a common endpoint forces every integration to update. Instead of creating new versions, extend existing endpoints through additive changes: new optional fields in responses, new optional parameters in requests. Only create a new major version when you need to remove a deprecated field or fundamentally change endpoint semantics. Plan for operating 2-3 API versions simultaneously—migrate customers gradually rather than forcing immediate upgrades.
Deprecation policies should be explicit and generous. Announce deprecation timelines at least 6-12 months before removing an API version. Provide clear migration guides explaining how to update from the deprecated version to the new one. When an API is deprecated, continue supporting it—don't immediately break integrations. This approach maintains customer trust and allows teams to plan upgrades during normal development cycles rather than emergency responses.
Include API version information in responses so clients know which version processed their request. A response header like API-Version: 2026-03-01 tells clients exactly which implementation version handled the request. This helps with debugging when version-specific bugs occur. For versioning based on dates (2026-03-01) rather than sequential numbers, you can roll out new versions without disrupting the mental model—developers think of the API version date as stable for a specific time period rather than constantly upgrading from v7 to v8 to v9.
Billing APIs require rate limiting to prevent abuse and ensure fair resource distribution. However, rate limiting strategies differ from typical API rate limiting—financial operations might require stricter limits than read operations. Reading invoice history is low-risk; processing refunds in bulk is high-risk. Implement tiered rate limits: generous limits for read operations, restrictive limits for write operations, and very restrictive limits for sensitive operations like refund processing or payment method changes.
Rate limit responses should include clear headers indicating remaining quota and reset time. Standard headers include X-RateLimit-Limit (requests allowed per window), X-RateLimit-Remaining (requests remaining), and X-RateLimit-Reset (Unix timestamp when limit resets). Developers use these headers to implement backoff strategies—if remaining quota is nearly exhausted, they can queue work and retry later. This transparency converts rate limiting from a frustrating barrier into a collaboration tool helping developers respect API capacity constraints.
Quota should be based on usage tier, not IP address or API key alone. A startup company might have 1,000 API calls per day included in their plan; an enterprise customer might have 1,000,000 calls daily. Implement quota tracking at the organization level, allowing teams to share quota across multiple applications and API keys. When approaching quota limits, provide clear warnings through rate limit headers and documentation so teams can request quota increases proactively rather than suddenly hitting hard limits.
Document rate limits prominently in API documentation. Specify limits per operation type, explain how quota resets, and provide examples of rate limit responses. Provide code examples showing proper exponential backoff when rate limits are encountered. Many integration failures occur because developers didn't implement proper rate limit handling—they submit a request, receive a 429 Too Many Requests response, and retry immediately instead of waiting for the reset window. Clear documentation prevents these preventable failures.
Billing events (invoices created, payments processed, subscriptions cancelled) often need to propagate to external systems for accounting, revenue recognition, and reporting. Webhooks enable real-time event notification without the client needing to continuously poll the API. However, webhook delivery introduces new complexity: delivery failures, ordering guarantees, and duplicate event handling. According to research on billing system integrations, 34% of revenue recognition errors stem from missed or duplicate webhook events [5].
Webhook payloads should include all relevant context so receiving systems don't need to make additional API calls to understand what happened. An invoice.created webhook might include the entire invoice object with customer details, line items, and payment status. This reduces integration complexity and ensures receiving systems can process events even during temporary API unavailability. Include idempotency information in webhook payloads—a unique event_id and timestamp allow receivers to deduplicate events if the same webhook is delivered twice.
Implement retry logic with exponential backoff for failed webhook deliveries. If a webhook delivery fails, retry after 30 seconds, then 5 minutes, then 30 minutes, then 4 hours. A 24-48 hour retry window catches temporary outages without overwhelming receiving systems. Persist failed webhook deliveries and provide a dashboard where customers can manually trigger retry. This approach handles both transient failures (temporary network issues) and permanent failures (receiving system was temporarily offline)—customers can reprocess events once their systems are restored.
Webhook signature verification proves authenticity and prevents spoofing attacks. Include an X-Signature header in webhook requests containing an HMAC-SHA256 hash of the request body using the customer's webhook secret. Receiving systems must verify this signature before processing the webhook. This proves the webhook originated from your billing system, not an attacker. Provide clear code examples showing signature verification in popular languages—JavaScript, Python, Go, Java. Many webhook security issues stem from incomplete signature verification implementations.
Billing operations must maintain consistency—if a payment is charged, it must appear in the customer's account; if an invoice is issued, its line items must sum correctly. ACID compliance (Atomicity, Consistency, Isolation, Durability) is non-negotiable. Atomicity ensures operations complete fully or not at all—no partially-charged invoices. Consistency ensures data integrity rules are never violated. Isolation prevents concurrent operations from interfering with each other. Durability ensures committed changes survive system failures.
Database transactions are the foundation of consistency. Billing operations should execute within a single transaction: verify the customer exists, verify payment method is valid, apply charges, issue invoice, record accounting entries. If any step fails, the entire transaction rolls back, leaving the system in a consistent state. Avoid splitting billing logic across multiple transactions—this creates windows where partial state exists, risking charge duplicates or missing records.
Optimistic locking prevents lost updates when multiple concurrent operations affect the same record. Store a version number with each financial record. Before updating an invoice, check its version hasn't changed since retrieval. If another process modified the record, reject the update and require the client to fetch the latest version and retry. This approach prevents subtle bugs where concurrent refund requests both succeed when only one should, or subscription modifications conflict unexpectedly.
Document consistency guarantees in your API contract. Specify which operations are atomic, which operations might leave temporary inconsistency that eventually resolves, and which operations require multiple steps to complete fully. For example, refund processing might take 24-48 hours to complete due to payment processor delays, during which the refund status is "pending". Documenting these guarantees prevents clients from assuming operations complete instantaneously when they actually require time.
Billing APIs handle sensitive payment data requiring PCI-DSS compliance and heightened security measures. The first rule: never store raw credit card numbers or full PCI data in your system. Instead, tokenize payment methods through payment processors, storing only tokens. Your API returns tokens to clients; clients never see raw card data. This architectural choice eliminates PCI-DSS scope for your system—you're not storing, processing, or transmitting card data that could be compromised.
Implement comprehensive audit logging for all financial operations. Every state change—payment processed, refund issued, subscription modified—must be logged with timestamp, actor (which API key), action, and result. Audit logs are immutable once created and cannot be modified or deleted. Provide audit log access through your API so customers can review compliance documentation independently. Platforms implementing SOC 2 Type II certification (like open-source billing systems such as Lago) undergo independent audits verifying logging completeness and security controls.
Transport security is fundamental—all API communication must use HTTPS with TLS 1.2 or higher. Disable HTTP entirely; all traffic must be encrypted. Implement strict TLS validation on the client side, rejecting self-signed certificates and outdated protocols. Rotate TLS certificates before expiration; expired certificates break integrations without warning. Provide clear documentation about required TLS versions and deprecation timelines as older protocols become deprecated.
Data encryption extends beyond transport—encrypt sensitive data at rest in your database. Customer payment methods, tax identification numbers, and banking details should be encrypted using customer-specific encryption keys derived from master keys. This prevents database backups or stolen drives from exposing customer data. Document encryption implementations in your security documentation so customers understand your data protection approach.
Excellent API documentation dramatically impacts integration success. Include endpoint descriptions, request/response examples, error codes, and authentication requirements. However, dry reference documentation isn't enough—provide integration guides showing how to implement common workflows: creating a subscription, processing a refund, handling webhook events. Walkthroughs that demonstrate realistic scenarios help developers understand the API better than isolated endpoint descriptions.
Provide SDKs in popular languages (JavaScript, Python, Go, Java, Ruby). SDKs handle authentication, error handling, and serialization, reducing integration complexity. Document SDK usage alongside REST API documentation. Include code examples for every operation—create customer, list invoices, process payment, cancel subscription. Developers copy examples and adapt them for their use case; well-structured examples accelerate integration significantly.
Interactive API documentation using tools like Swagger/OpenAPI allows developers to explore endpoints and test requests directly in the browser. Provide a sandbox environment separate from production where developers can test integration without affecting real data. Document the sandbox's limitations (test payment methods, inability to actually charge cards) clearly so developers don't accidentally use sandbox credentials in production.
Implement comprehensive error documentation mapping error codes to causes and solutions. Don't just list error code descriptions—explain what causes each error and how developers should handle it. For example, INSUFFICIENT_FUNDS error means the payment method doesn't have adequate balance; recommend prompting users to add funds or try a different payment method. This contextual documentation transforms error handling from guesswork into guided recovery.
Billing APIs must be thoroughly tested because financial errors have direct business impact. Implement unit tests for business logic (calculating invoices, determining tax), integration tests for API endpoints (request/response validation, state transitions), and end-to-end tests simulating realistic customer workflows. Testing financial calculations specifically is critical—off-by-one-cent errors multiply across thousands of customers.
Implement contract tests between your API and webhook consumers. Define expected webhook formats and customer systems verify they can process those formats. This prevents breaking changes to webhook structures from silently failing—tests catch incompatibilities early. Use chaos engineering—deliberately inject failures (network timeouts, database errors) to verify your API degrades gracefully rather than losing data.
Monitoring and alerting are essential for production reliability. Alert on missing webhook deliveries, failed payments, or API error rate spikes. Implement comprehensive logging capturing request/response context for debugging production issues. For financial systems, monitoring isn't optional—lack of monitoring means failures go unnoticed, potentially causing revenue loss or compliance violations. Establish clear SLAs for API availability and response time, monitoring against these targets continuously.
Building robust billing REST APIs requires deliberate architectural decisions across endpoint design, authentication, error handling, versioning, and compliance. The practices outlined here—idempotency for duplicate prevention, immutability of financial records, atomic transactions, comprehensive error handling, and security-first design—form the foundation of reliable billing systems. These aren't optional optimizations; they're fundamental requirements that prevent charges, data loss, and compliance violations.
Open-source platforms like Lago demonstrate API-first architecture implementing these principles at scale. Platforms with SOC 2 Type II certification undergo independent audits verifying logging completeness, access controls, and incident response procedures—validating that architectural principles are properly implemented. Whether building billing APIs from scratch or evaluating existing platforms, prioritize these best practices throughout the integration and implementation process.
The specific implementation details vary based on your tech stack, scale requirements, and business model. However, the core principles—clear endpoint design, explicit error handling, idempotent operations, strict consistency guarantees, comprehensive logging, and thoughtful versioning—apply universally. Systems adhering to these practices maintain flexibility to evolve without breaking customers, prevent costly financial errors, and provide developers with clear integration paths.
Additional resources for billing API implementation include our guide on REST vs GraphQL API design tradeoffs, webhook reliability patterns, and security architecture for billing systems.
Content