Internet-Draft Parallel Resumable Uploads October 2026
Pallagatti & Maheshwari Expires 14 April 2027 [Page]
Workgroup:
HTTP
Internet-Draft:
draft-spk-httpbis-parallel-upload-00
Published:
Intended Status:
Standards Track
Expires:
Authors:
S. Pallagatti
Zscaler
A. Maheshwari
Zscaler

Parallel Appends for Resumable Uploads

Abstract

The resumable upload protocol lets a client continue an interrupted upload, but the upload has to be sent in order, one request at a time. This document extends that protocol so that a client can send different parts of the same upload in parallel. Servers and intermediaries can limit both the number of concurrent requests and the amount of data that arrives out of order. The document also describes how intermediaries that inspect uploads (for data loss prevention or malware scanning, say) can continue to do so, and how the extension behaves over HTTP/1.1, HTTP/2, and HTTP/3.

About This Document

This note is to be removed before publishing as an RFC.

The latest revision of this draft can be found at https://santoshpallagatti.github.io/draft-spk-httpbis-parallel-upload/draft-spk-httpbis-parallel-upload.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-spk-httpbis-parallel-upload/.

Discussion of this document takes place on the HTTP Working Group mailing list (mailto:ietf-http-wg@w3.org), which is archived at https://lists.w3.org/Archives/Public/ietf-http-wg/.

Source for this draft and an issue tracker can be found at https://github.com/santoshpallagatti/draft-spk-httpbis-parallel-upload.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 14 April 2027.

▲

Table of Contents

1. Introduction

The resumable upload protocol [RESUMABLE] lets a client continue an interrupted upload instead of starting over. Its state is a single offset: all bytes before the offset have been stored, and each append has to start exactly at that offset. Section 4.6 of [RESUMABLE] forbids a client from transferring data for the same upload in parallel.

HTTP has had a standard mechanism for parallel downloads since HTTP/1.1: a client requests different byte ranges of the same resource at the same time using the Range header field (Section 14.2 of [RFC9110]), and download managers and cloud storage clients rely on it. Uploads have no counterpart. Section 14.5 of [RFC9110] describes Content-Range in a PUT request but leaves its handling undefined, and [RESUMABLE] requires appends to be sequential.

Parallel uploads are nevertheless widely deployed in service-specific forms: cloud storage APIs, and the Concatenation extension of the tus protocol [TUS], upload parts as separate resources and join them at the end. What is missing is a standard way for a client to send several byte ranges of one upload at the same time. The Byte Range PATCH draft [BYTERANGE] mentions parallel uploads and leaves them for "a later document".

This document defines such an extension for [RESUMABLE]. A client asks for parallel mode when it creates the upload (Section 4) and then sends appends for different byte ranges at the same time, within a window (Section 5). The server reports which ranges it has in a new response field (Section 6), and Section 7 says when the upload is complete. Requirements for servers and for intermediaries are in Section 10 and Section 11.

Unless this document says otherwise (Section 8), all rules of [RESUMABLE] still apply. A client or server that does not implement this extension simply keeps using the sequential protocol.

1.1. Use Cases

The main motivation is large uploads over paths where a single connection cannot use the available bandwidth: video and audio, disk images and backups, and datasets or model files that often run to several gigabytes. Section 12.1 explains which parts of the gain come from parallel appends on one connection and which need several connections.

Uploads to AI services are a growing case. Users attach documents, recordings, and video for a model to analyze, and the model cannot start until the upload has finished, so upload time shows up directly as waiting time. For small files the gain from parallel appends is small, because connection setup and round trips dominate. For large inputs it can be significant.

The same trend affects organizations that inspect outbound uploads, including uploads to AI services, to keep sensitive data from leaving their networks. Their proxies have to handle more and larger uploads with limited memory, and must not let a file be committed before it has been checked. Section 11.3 describes how they can do that.

Mobile clients benefit as well. Their connections drop and change often, and with this extension only the missing ranges need to be sent again.

2. Conventions and Definitions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

This document uses the terms "upload resource", "append", "upload creation", "offset retrieval", and "upload cancellation" as defined in [RESUMABLE], and the terms "client", "server", "intermediary", "proxy", and "gateway" as defined in [RFC9110]. Structured Field types (Boolean, Integer, Token, List, Inner List, Dictionary) are defined in [RFC9651].

The following terms are also used:

Parallel mode:

The mode of an upload for which the extension defined in this document has been negotiated.

Contiguous offset (C):

The number of bytes, starting at byte 0, that have been received without a gap. This is the value that [RESUMABLE] conveys in the Upload-Offset field.

Reorder window (W):

The maximum distance, in bytes, beyond the contiguous offset at which an append may end. See Section 5.1.

Received ranges:

The set of byte ranges of the upload that the server has received and stored.

Tail append:

The append that carries Upload-Complete set to true. With explicit commit it is an empty append at the upload length. With automatic commit it is normally the data append that ends at the upload length, although an empty append at the upload length is also accepted (see Section 7). Without it the upload cannot complete.

Inspecting intermediary:

An intermediary that examines the content of an upload before allowing it to reach or be committed at the server, typically to apply data loss prevention (DLP) or malware scanning policy.

Byte ranges in this document are written as [start, end), meaning the range includes start and excludes end. "MiB" means 1,048,576 bytes.

3. Overview

A client asks for parallel mode when it creates an upload. The server, and any intermediary on the path, answer with the limits they accept: how many appends may be in progress at once, how large the reorder window is, or both. Intermediaries can only make these limits stricter.

The client then sends several appends at the same time, each covering its own byte range. Every append has to end within the reorder window, which starts at the contiguous offset. As gaps at the start of the window fill, the contiguous offset advances and the window slides forward. The server reports the ranges it holds in the Upload-Received field, so a client that loses an append re-sends only the missing range. The upload completes when the received ranges cover the whole upload.

 0                 C                              C+W        Length
 |=================|----+=====+------+=====+------|----------|
   all received     gap  recv   gap    recv  gap    outside
                   |<------ reorder window ------>|  window
Figure 1: The Reorder Window

Figure 2 shows a 12 MiB upload sent as 2 MiB appends with a window of 6 MiB and at most three appends in progress.

Client                                   Server
  |  POST /files                            |
  |  Upload-Parallel: ?1                    |
  |  Upload-Length: 12582912                |
  |---------------------------------------->|
  |  201 Created                            |
  |  Location: /uploads/abc                 |
  |  Upload-Limit: max-parallel=3,          |
  |                reorder-window=6291456   |
  |<----------------------------------------|
  |  PATCH, Upload-Offset: 0         ------>|  \
  |  PATCH, Upload-Offset: 2097152   ------>|   > concurrent,
  |  PATCH, Upload-Offset: 4194304   ------>|  /  window [0, 6 MiB)
  |<------------------ 204 (three times) ---|
  |  PATCH, Upload-Offset: 6291456   ------>|  \
  |  PATCH, Upload-Offset: 8388608   ------>|   > window slid to
  |  PATCH, Upload-Offset: 10485760  ------>|  /  [6 MiB, 12 MiB)
  |         Upload-Complete: ?1             |
  |<----------- 204, 204, 201 Created ------|  upload complete
Figure 2: Parallel Upload Overview

Inspecting intermediaries use the same window to process content in order with bounded memory, and hold the tail append until they reach a verdict (Section 11.3).

4. Negotiating Parallel Mode

4.1. The Upload-Parallel Request Field

The Upload-Parallel request header field is a Boolean [RFC9651]. A client sends it with the value true in an upload creation request to ask for parallel mode:

Upload-Parallel: ?1

A client that sends Upload-Parallel with the value true MUST also send Upload-Length in the same request. Parallel mode requires the total length to be known before appends arrive out of order. A server that receives Upload-Parallel without Upload-Length MUST treat the upload as sequential.

A client MAY create the upload with an empty body: a POST carrying Upload-Parallel, Upload-Length, Upload-Complete set to false, and no content. The response gives the client the Location of the upload resource and the Upload-Limit field, so it knows the upload URI and the limits that apply, that is how many appends may be in flight, the reorder window, any alignment, and the commit mode, before it sends any content. All content is then sent in appends.

POST /files HTTP/1.1
Host: example.com
Upload-Parallel: ?1
Upload-Length: 12582912
Upload-Complete: ?0
Content-Length: 0

HTTP/1.1 201 Created
Location: /uploads/abc
Upload-Limit: max-parallel=3, reorder-window=6291456

This is RECOMMENDED when HTTP/1.1 is used, because [RESUMABLE] otherwise carries the upload URI in a 104 (Upload Resumption Supported) interim response, which some intermediaries do not forward (Section 12.2).

4.2. Upload-Limit Parameters

This document defines the following parameters for the Upload-Limit response field of [RESUMABLE]:

max-parallel:

An Integer. The maximum number of appends to this upload that may be in progress at the same time. A value of 1 in the upload creation response means the upload is sequential (Section 4.4). If the parameter is absent, this field places no limit on the number of appends; concurrency is then bounded only by the reorder window and by what the server accepts at the time (Section 10.2).

reorder-window:

An Integer. The reorder window W in bytes. If absent, the window is unlimited, that is, equal to the upload length.

append-alignment:

An Integer. The offset of every append, and the length of every append except the tail append, MUST be a multiple of this value. If absent, no alignment applies.

commit:

A Token, either "auto" or "explicit". See Section 7. If absent, the value is "auto".

A server that accepts parallel mode MUST include at least one of max-parallel with a value greater than 1, or reorder-window, in the Upload-Limit field of the upload creation response, and MUST include its current values of these parameters in every later Upload-Limit field it sends for the upload, including in offset retrieval responses. A server that cares only about how much out-of-order data it has to hold can send reorder-window alone; one that cares only about the number of concurrent requests can send max-parallel alone.

4.3. Tightening by Intermediaries

An intermediary MAY modify the Upload-Limit field in a response it forwards, but only in ways that make the limits stricter:

  • it MAY lower max-parallel, or add it if absent;

  • it MAY lower reorder-window, or add it if absent;

  • it MAY replace append-alignment with a multiple of the current value, or add it if absent;

  • it MAY change commit from "auto" to "explicit".

An intermediary MUST NOT raise max-parallel or reorder-window, remove any of these parameters, or change commit from "explicit" to "auto". Because each hop can only tighten the limits, the values a client receives are the strictest on the path.

Only the party that has to satisfy a limit can raise it. Replacing append-alignment with a multiple of its current value is a further restriction on the client, not a new permission, which is why an intermediary may do it. Raising max-parallel or reorder-window would commit the server to work it has not agreed to, so an intermediary cannot do that even though it could reassemble the data itself. An intermediary that takes over parallel mode on the client-facing hop (Section 11.2) is a different case: there it acts as the server, and it MAY advertise its own limits, including larger ones, because it services them itself.

An intermediary that tightens a limit MUST enforce the value it advertised. The server knows only its own limits, so it would accept an append that breaks a tighter one. The intermediary therefore rejects or delays such an append itself, in the same way a server does (Section 10.2, Section 12.1.2).

Changing commit from "auto" to "explicit" works with any server that implements this document, because such a server accepts an empty append as the tail append in either mode (Section 7). An intermediary that makes this change tracks which ranges the server has received and applies the explicit-commit rules toward the client: if the client sends the commit request while ranges are still missing, the intermediary responds with 409 (Conflict) and "incomplete-ranges" instead of forwarding it (Section 7.2).

4.4. Falling Back to Sequential Uploads

An upload is in parallel mode if, and only if, the client sent Upload-Parallel with the value true and the Upload-Limit field in the response to upload creation contains either max-parallel with a value greater than 1, or reorder-window without max-parallel. A max-parallel value of 1 in that response always means sequential, even when reorder-window is also present. In every other case, client and server MUST follow [RESUMABLE] without the changes in this document. This is the only rule that decides the mode; other sections refer to it.

The mode is fixed when the upload is created, but the limits can change during the upload. Any response for the upload resource, whether to an append or to offset retrieval, MAY carry a different Upload-Limit. Each Upload-Limit field carries the complete current values: a parameter that is absent from it has the default meaning given in Section 4.2, not its previous value. A server MAY raise or lower its own limits. An intermediary recomputes its values from each new value it receives from upstream and still only tightens it (Section 4.3). A client MUST apply the most recent values to appends it starts after receiving them, and a change never invalidates an append already in progress (Section 10.2). A later max-parallel of 1 limits the client to one append at a time but does not end parallel mode; Upload-Received and the other rules of this document still apply.

5. Parallel Appends

5.1. The Reorder Window

Let C be the contiguous offset, L the upload length, and W the reorder window. In parallel mode, an append that covers bytes [o, o + n), where n is the length of its content, is inside the window if and only if:

    o + n <= min(C + W, L)

A client MUST NOT start an append that is outside the window as computed from the most recent Upload-Offset it has received. Because C only grows, an append that was inside the window when it started stays inside it.

A window equal to L allows unrestricted parallelism. A window equal to the size of one append results in sequential behavior while still permitting a new append to start as soon as the previous one completes.

5.2. Append Requests

Appends in parallel mode use the PATCH method and the application/partial-upload media type, as defined in [RESUMABLE]. The Upload-Offset request field carries the offset o at which the content starts; the content length gives n.

In parallel mode, Upload-Offset in an append request is not required to equal the server's current offset. It is the start of the byte range the append writes. A client MUST send Content-Length on every append in parallel mode, so that a recipient knows the full range before the content arrives.

If max-parallel is present, a client MUST NOT have more than that many appends to the same upload in progress at once. An append is in progress from when its request starts until its response is received or the request is abandoned.

A client SHOULD include a Content-Digest field [RFC9530] on each append so that the server and intermediaries can verify each range on its own.

PATCH /uploads/abc HTTP/1.1
Host: example.com
Content-Type: application/partial-upload
Upload-Offset: 4194304
Upload-Complete: ?0
Content-Length: 2097152
Content-Digest: sha-256=:(base64-encoded SHA-256 of the content):

[2097152 bytes]

5.3. Overlapping and Repeated Ranges

An append may overlap ranges the server has already received, for example when a client retries an append whose response it did not receive. Retrying a range with identical content is always safe.

A server MAY discard overlapping bytes without comparing them. If a server detects that overlapping bytes differ from the stored bytes (using Content-Digest, say), it MUST reject the append with status 409 (Conflict) and the problem type "range-conflict" (Section 13).

Two appends in flight for the same range with identical content are not a conflict. The server treats the newer one as a retry of the older (see below), and the range is stored once. The problem type "range-conflict" applies only when overlapping bytes differ.

If a new append overlaps a range that is being written by an append still in progress, the server SHOULD terminate the older append before processing the new one. This applies the rule of Section 4.6 of [RESUMABLE] per byte range instead of per upload: the newer request is taken to be a retry of the older one. Bytes the older append stored before termination remain received.

5.4. Interrupted Appends

When an append is interrupted, the server MAY keep the bytes it received as a received range, as in [RESUMABLE]. In parallel mode, this can leave a gap after the kept bytes. The client learns about the gap through offset retrieval (Section 6) and re-sends only the missing bytes.

6. Reporting Received Ranges

The Upload-Received response header field is a List of Inner Lists [RFC9651]. Each Inner List contains exactly two Integers, the start and end of a received range [start, end):

Upload-Offset: 1048576
Upload-Received: (0 1048576), (2097152 6291456)

The ranges MUST be listed in ascending order and MUST NOT overlap or touch; adjacent ranges are merged. If the contiguous offset is greater than 0, the first range starts at 0 and ends at the contiguous offset.

In parallel mode, a server MUST include Upload-Received in every offset retrieval response and SHOULD include it in responses to appends. The Upload-Offset field keeps the meaning defined in [RESUMABLE]: it is the contiguous offset, and it never decreases.

A server MUST NOT list a range it has not stored. A server MAY omit ranges beyond the contiguous offset, for example to limit the size of the field; the client then re-sends those bytes, which is safe (Section 5.3).

7. Upload Completion

In parallel mode an upload is complete when its received ranges cover [0, L) and the server has received the tail append. The tail append is the append that carries Upload-Complete with the value true. Every other append MUST carry Upload-Complete with the value false.

7.1. Automatic Commit

When commit is "auto", the tail append is normally the data append that ends at L. A client MUST set Upload-Complete to true on the append that ends at L and on no other append. A server that receives Upload-Complete set to true on an append that does not end at L MUST reject it with status 400 (Bad Request).

In either commit mode, a server MUST also accept an empty append at offset L carrying Upload-Complete set to true as the tail append, if it has not already received one. [RESUMABLE] already allows such an append to complete an upload whose content has all arrived; accepting it in automatic mode is what lets an intermediary change the commit mode (Section 4.3).

If gaps remain when the tail append has been received, the server responds to the tail append with 204 (No Content) and the current Upload-Offset and Upload-Received. The upload completes when the last gap is filled. The append that fills it receives the final response for the upload, as described in [RESUMABLE]. Other appends receive 204 (No Content).

7.2. Explicit Commit

When commit is "explicit", every data append, including the one that ends at L, carries Upload-Complete set to false. After the client has received successful responses for all ranges, it sends an append with empty content, Upload-Offset set to L, and Upload-Complete set to true. That empty append is the tail append and receives the final response.

If the server receives the tail append while gaps remain, it MUST reject it with status 409 (Conflict), the problem type "incomplete-ranges", and the Upload-Received field.

Explicit commit separates receiving the data from committing it. It lets a client check the whole representation, using Repr-Digest [RFC9530], before committing. It also keeps the tail append small, which helps intermediaries that hold it (Section 11.3.3).

8. Changes to Resumable Upload Rules in Parallel Mode

For uploads in parallel mode only, this document changes the following rules of [RESUMABLE]. Section numbers refer to version 13 of that draft; each rule is also described in words, so that it can be found if the numbering changes. All other rules apply unchanged.

Table 1: Rules Changed in Parallel Mode
Rule in RESUMABLE Change in parallel mode
Section 4.6: a client must not transfer data for one upload in parallel Several appends may be in progress, within the limits of Section 4.2 (Section 5.2).
Section 4.6: a new request terminates the older one Applies per overlapping byte range (Section 5.3).
Section 4.4.2: Upload-Offset must match the current offset, otherwise 409 Any append inside the reorder window is accepted (Section 5.1).
Upload-Complete marks the final append Marks the tail append; completion requires full coverage (Section 7).

[RESUMABLE] is sequential for a good reason. Without negotiation, a server cannot tell a deliberate concurrent append from a stale request that the client gave up on, such as one left behind on a half-open connection. Negotiating parallel mode removes that ambiguity, and applying the termination rule per range keeps the protection against stale requests.

9. Client Behavior

10. Server Requirements

10.1. Upload State

A server that supports parallel mode MUST track the received ranges of each upload in parallel mode, and update them atomically: a range is either fully recorded or not recorded at all. Offset retrieval responses MUST reflect every range recorded before the request was received.

When requests for one upload can reach different server instances, for example behind a load balancer, all instances MUST share a consistent view of the received ranges, or requests for an upload MUST be routed to a single instance.

10.2. Enforcing Limits

A server MUST reject an append that is outside the reorder window with status 409 (Conflict), the problem type "offset-outside-window", and the current Upload-Offset, unless it delays the append as described in Section 12.1.2.

A server MAY reject an append with status 429 (Too Many Requests), the problem type "too-many-parallel", and a Retry-After field when accepting it would exceed max-parallel or, if max-parallel is absent, the number of concurrent appends the server can currently handle.

A server MUST reject an append whose offset or length violates append-alignment with status 400 (Bad Request) and the problem type "misaligned-append".

A server MAY raise or lower its limits during an upload, for example as its load changes, and signals new values in its next responses (Section 4.4). It MUST NOT reject an append that was inside the limits it last advertised when the append started solely because the limits changed.

10.3. Storage and Visibility

A server MAY write ranges directly into a single sparse object or store them separately and join them on completion. The choice is not visible to clients.

Until an upload is complete, a server MUST NOT make its content available as the representation of any target resource, and MUST NOT fill gaps with placeholder bytes that could be mistaken for uploaded content. An upload that is cancelled or expires MUST be discarded without being committed.

If a Repr-Digest was provided, the server SHOULD verify it against the joined content before committing and MUST NOT commit on a mismatch.

10.4. Resource Management

The reorder window bounds how much out-of-order data a server has to buffer per upload. A server that buffers in memory SHOULD choose reorder-window and max-parallel according to its memory budget, and SHOULD also limit the number of concurrent uploads per client.

10.5. Expiry, Failure, and Loss of State

Uploads expire as described in [RESUMABLE], and a server SHOULD state its expiry through the max-age parameter of Upload-Limit so that clients and intermediaries can bound how long they keep state for an upload. An upload that expires or is cancelled MUST be discarded without being committed (Section 10.3).

Upload-Offset never decreases, and Upload-Received lists only ranges the server holds. A server that loses recorded ranges, for example when an instance without shared state restarts, therefore MUST NOT report a smaller contiguous offset or fewer ranges for that upload. It MUST instead fail subsequent requests for the upload resource with 404 (Not Found) or 410 (Gone), so that the client creates a new upload rather than completing one with missing bytes.

11. Intermediary Requirements

11.1. All Intermediaries

Appends in parallel mode are ordinary HTTP requests. An intermediary that does not implement this document forwards them without change, and parallel mode works through it.

An intermediary that implements this document:

  • MAY tighten Upload-Limit as described in Section 4.3, and MUST NOT loosen it;

  • MAY force sequential behavior by removing Upload-Parallel from an upload creation request or by setting max-parallel to 1 in the upload creation response (Section 4.4);

  • MUST enforce any limit it tightens (Section 4.3);

  • MUST NOT remove or alter Upload-Received, except as described in Section 11.3.4;

  • when it balances load across servers, MUST meet the consistency requirement of Section 10.1, for example by routing all requests for an upload resource to the same server.

Responses to appends and to offset retrieval are not cacheable, as specified in [RESUMABLE]; this is unchanged.

11.2. Terminating Parallel Mode at an Intermediary

An intermediary MAY accept parallel mode from clients while forwarding the upload to the server sequentially, for example because the server does not support this extension or is reached over a different HTTP version. Such an intermediary acts as the server for parallel mode on the client-facing hop: it MUST meet the requirements of Section 10 for that hop, buffer out-of-order data within the window it advertises, and forward bytes upstream in order as the contiguous offset advances.

11.3. Inspecting Intermediaries

An inspecting intermediary examines upload content before it reaches, or is committed at, the server. Such intermediaries only see content where they already terminate TLS, usually under an enterprise policy. This document does not change that.

11.3.1. In-Order Processing

Many inspection engines need content in order: file types are recognized from the first bytes, and patterns can span append boundaries. An inspecting intermediary that processes content in order:

  • SHOULD advertise a reorder-window no larger than the buffer it can dedicate to the upload;

  • buffers appends that arrive ahead of the contiguous offset and passes bytes to its engines in order as gaps fill;

  • MUST NOT evaluate appends only in isolation when its policy can match content that spans appends;

  • MAY use Content-Digest to recognize a retried range it has already inspected and skip inspecting it again.

In-order inspection needs a single view of the whole upload. When several instances of an intermediary share the load, all requests for one upload resource MUST reach the same instance, or the instances MUST share the buffered and inspection state of the upload. An instance that receives an append for an upload whose earlier content it has not inspected, and cannot obtain that state, MUST treat this as lost state (Section 11.3.7).

Some formats can only be analyzed as a whole, such as archives that keep their index at the end. For those, an intermediary keeps the content as it passes and inspects the complete content before releasing the tail. This needs storage for the whole upload but not a larger reorder window, since the content still arrives in order within the window; an intermediary MUST NOT raise reorder-window for this purpose (Section 4.3).

11.3.2. Forwarding Strategies

The RECOMMENDED approach is to forward every append as soon as it arrives, inspect the content in order alongside, and hold back only the tail append until there is a verdict (Section 11.3.3). Upload throughput is unaffected; the only added delay is the time needed to inspect the last data received.

The simplest way to do this is to set commit to "explicit" when tightening limits (Section 4.3). Every append then reaches the server unchanged, and the only request the intermediary holds is the empty commit request at the end, which it forwards once its verdict allows the content.

An intermediary that does not want the server to store uninspected bytes at all can instead forward each range only after inspecting it, at the cost of more delay. It can also forward the upload sequentially, as described in Section 11.2.

11.3.3. Holding the Tail Append

Whatever forwarding strategy it uses, an inspecting intermediary MUST NOT forward the tail append to the server before its verdict for the upload is that the content is allowed. Since a server cannot complete an upload without its tail append (Section 7), this is enough to keep uninspected content from being committed, even if the server knows nothing about inspection or about this document.

The intermediary always knows which append is the tail: it carries Upload-Complete set to true and, with automatic commit, ends at the upload length. With explicit commit the tail append is the empty commit request, so holding it delays nothing but the commit itself. That is the same rule as the strategy in Section 11.3.2, not an alternative to it.

With automatic commit the tail append can be large. The intermediary MAY forward all of it except a final part, as an append with Upload-Complete set to false, and hold only that final part as the tail it forwards later. The forwarded part is not a tail append, so if append-alignment applies it MUST end at a multiple of append-alignment: the intermediary forwards up to the largest such multiple that is less than the upload length and holds the rest. If no such multiple lies within the tail append, the intermediary holds all of it. With explicit commit the tail append is empty and nothing needs to be split.

11.3.4. Offset Retrieval While Holding

With automatic commit, the held tail carries content the server has not yet received. While it holds it, the intermediary SHOULD answer offset retrieval requests for the upload itself, or modify the server's response, so that the held range is reported as received and the Upload-Inspection field (Section 11.3.6) is included. Otherwise the client would see a gap at the end and re-send the tail, which is safe but wasteful. With explicit commit no content is held, and the intermediary only adds Upload-Inspection to these responses.

11.3.5. Verdicts

Allowed:

The intermediary forwards the tail append. It relays the server's final response to the client and adds Upload-Inspection with state "clean".

Blocked:

The intermediary MUST NOT forward the tail append. It SHOULD cancel the upload at the server using upload cancellation as defined in [RESUMABLE], and responds to the client with status 403 (Forbidden) and the problem type "upload-blocked". Later requests for the upload receive the same response.

Pending:

If inspection takes longer than the intermediary is willing to hold the request, it MAY respond to the tail append with status 202 (Accepted), a Retry-After field, and Upload-Inspection with state "scanning". The client then uses offset retrieval until the response shows the upload as complete or blocked.

11.3.6. The Upload-Inspection Response Field

The Upload-Inspection response header field is a Dictionary [RFC9651] that an inspecting intermediary uses to report progress:

Upload-Inspection: state=scanning, scanned=7340032
state:

A Token: "scanning", "clean", or "blocked".

scanned:

An Integer, optional: the offset up to which content has been inspected.

Only inspecting intermediaries set this field. A client MUST NOT treat it as a statement by the server, and MAY use it to explain delays to users. Unknown keys MUST be ignored.

11.3.7. Timeouts and Failures

An intermediary SHOULD bound how long it holds a tail append. If it cannot reach a verdict, because an inspection engine is unavailable or the upstream server is failing, it MUST NOT forward the tail. It SHOULD cancel the upload at the server and fail the client's request, so that content that was never inspected is not left stored and cannot be committed later.

An intermediary that abandons an upload, for example when the client disconnects or the intermediary itself restarts, SHOULD cancel the upload at the server rather than rely on expiry.

An intermediary that loses its buffered or inspection state for an upload, for example after a restart, cannot recover by inspecting again. The content before the contiguous offset is already at the server and will not be sent again, and in-order engines would also miss the start of the content, where file types are recognized. The intermediary therefore MUST NOT forward the tail append for that upload. It MUST cancel the upload at the server and respond to the client's requests for it with 410 (Gone), so that the client creates a new upload (Section 9). An intermediary that terminates parallel mode (Section 11.2) acts as the server on the client-facing hop and follows Section 10.5 in the same way.

If forwarding a held tail append fails because the upload has expired or the server has lost it, the intermediary reports the failure to the client and discards the held content.

11.3.8. Reusing Earlier Verdicts

If the client sends Repr-Digest at upload creation, an inspecting intermediary MAY look up an earlier verdict for that digest under the same policy. A digest known to be blocked can be refused at upload creation, before any content is transferred. For a digest known to be allowed, the intermediary MAY skip its inspection engines, but MUST still compute the digest of the content as it passes and treat a mismatch as blocked.

12. HTTP Version Considerations

12.1. HTTP/2 and HTTP/3

Over HTTP/2 [RFC9113] and HTTP/3 [RFC9114], each append is a request on its own stream, and several appends can share one connection. Parallel appends on one connection already remove the idle round trip that sequential appends spend waiting for each response, and a slow append no longer holds up the ranges after it. With HTTP/3, a lost packet also delays only the stream it belongs to; with HTTP/2 it delays every stream on the connection.

Parallel appends on one connection do not raise that connection's throughput limit, however. All streams on a connection share one congestion controller, whether the transport is TCP or QUIC, so on a path with a long round-trip time or packet loss a single connection can still use only part of the available capacity. Only additional connections, each with its own congestion controller, lift that limit. A client MAY therefore spread appends over more than one connection when a single connection does not use the available capacity, subject to Section 12.3. The total number of appends in progress still stays within max-parallel when it is present.

12.1.1. Priorities

A client SHOULD use the Priority field [RFC9218] to give the append that starts at the contiguous offset a lower urgency value (that is, higher importance) than appends further into the window, for example u=1 for the first and u=3 to u=5 for later ones. Servers and intermediaries MAY send PRIORITY_UPDATE frames to adjust this. Delivering the start of the window first lets the window slide sooner and reduces the data buffered out of order. Priorities are hints; the protocol works without them.

12.1.2. Flow Control

A request's header section, which carries Upload-Offset and Content-Length, arrives before its content. A recipient therefore knows the range of an append before any of its content arrives, and can use per-stream flow control to delay an append that is outside the window instead of rejecting it:

  • For an append inside the window, the recipient grants flow control credit as usual (WINDOW_UPDATE in HTTP/2, MAX_STREAM_DATA [RFC9000] in HTTP/3).

  • For an append outside the window, the recipient withholds credit beyond the initial stream window until the window reaches the append. The sender pauses on that stream only.

  • The recipient MUST continue to grant connection-level credit to other streams, so that a delayed stream does not stall them.

  • A recipient MAY reject an append that has been delayed longer than it is willing to wait, as described in Section 10.2.

A delayed stream can still send up to its initial stream window before pausing. A recipient that delays appends can lower its initial stream window to bound that data.

Client                           Recipient
  | HEADERS 1, Upload-Offset: 0       |  C = 0,
  | HEADERS 3, Upload-Offset: 2097152 |  window [0, 6 MiB)
  | HEADERS 5, Upload-Offset: 8388608 |
  |---------------------------------->|
  |      WINDOW_UPDATE streams 1, 3   |  stream 5 is outside
  |<----------------------------------|  the window: no
  | DATA streams 1, 3                 |  further credit
  |---------------------------------->|
  |                                   |  C reaches 4 MiB,
  |      WINDOW_UPDATE stream 5       |  window [4, 10 MiB)
  |<----------------------------------|
  | DATA stream 5                     |
  |---------------------------------->|
Figure 3: Delaying an Append with Flow Control (W = 6 MiB)

12.1.3. Cancellation and Early Data

A client cancels a redundant or abandoned append with RST_STREAM in HTTP/2 or by resetting the stream in HTTP/3. The server treats it as an interrupted append (Section 5.4).

Upload creation and appends MUST NOT be sent in TLS early data. A server that receives them in early data responds with status 425 (Too Early) [RFC8470].

12.2. HTTP/1.1

Over HTTP/1.1 [RFC9112], parallel appends use separate connections, one append per connection at a time. The following rules apply:

  • If max-parallel is present, a client MUST NOT use more connections for appends to one upload than that value. Browsers are additionally limited by their per-origin connection limits.

  • HTTP/1.1 has no per-request flow control. A client MUST NOT start an append outside the window, and a recipient SHOULD reject such an append with 409 and "offset-outside-window" rather than delay it.

  • HTTP/1.1 has no priorities. A client keeps the append at the contiguous offset in progress first and MAY replace it with a new request on another connection if it stalls (Section 9).

  • Some intermediaries do not forward informational responses. A client MUST NOT depend on receiving 104 (Upload Resumption Supported) and SHOULD create the upload with an empty body, taking the upload resource and limits from the final response.

  • A client MUST NOT pipeline requests.

  • Appends in parallel mode carry Content-Length (Section 5.2) and therefore do not use chunked transfer coding.

  • A client MAY send Expect: 100-continue, and MUST continue if no 100 (Continue) response arrives, as described in [RFC9110].

If the upload is not in parallel mode (Section 4.4), client and server use the sequential behavior of [RESUMABLE] unchanged.

12.3. Congestion Control and Fairness

An upload that uses several connections can take more than one connection's share of a bottleneck it shares with other traffic. Additional connections are meant for paths where a single connection is limited by round-trip time or loss, not for gaining a larger share of a congested link. Therefore:

  • A client SHOULD start with one connection and open additional connections only while doing so increases the upload's throughput. It SHOULD stop adding connections when it does not.

  • A client SHOULD NOT use more connections for one upload than it needs, and MUST stay within max-parallel when it is present.

  • Servers and intermediaries can limit concurrency for an upload with max-parallel, and can reject excess appends with 429 (Too Many Requests) (Section 10.2).

  • Each connection runs the congestion control specified for its transport. This document does not change congestion control.

Multipath extensions to QUIC, under development in the IETF, could offer a single connection with a separate congestion controller for each path. This document does not depend on them.

13. Problem Types

Error responses defined in this document use Problem Details [RFC9457] with the following problem types. Each type URI has the form https://iana.org/assignments/http-problem-types#NAME.

Table 2: Problem Types
Name Status Meaning
offset-outside-window 409 The append ends beyond the reorder window. The response includes Upload-Offset.
range-conflict 409 The append overlaps received bytes with different content.
incomplete-ranges 409 An explicit commit was sent while gaps remain. The response includes Upload-Received.
misaligned-append 400 The offset or length violates append-alignment.
too-many-parallel 429 More appends are in progress than max-parallel allows, or than the server can currently handle.
upload-blocked 403 An inspecting intermediary blocked the upload by policy.

A repeated append for a range that is already stored, or for one that another append is still writing, is not a conflict when the content is identical; see Section 5.3.

Problem details for "upload-blocked" MUST NOT identify the inspection engine, signature, or policy rule that matched.

14. Security Considerations

The security considerations of [RESUMABLE] apply to this extension as well.

A client cannot get more parallelism than every party on the path allows: intermediaries can only tighten the limits, and they enforce the limits they tighten (Section 4.3), so a client that ignores them cannot overrun an intermediary's buffers.

An inspecting intermediary controls whether an upload is committed by holding its tail append (Section 11.3.3). This relies on the intermediary seeing all of the upload's content in order. An attacker could try to break that view, for example by steering appends for one upload to different instances of a load-balanced intermediary, or by provoking a restart partway through an upload. An intermediary therefore keeps each upload on one instance or shares its state (Section 11.3.1), and cancels any upload it cannot inspect completely rather than releasing it (Section 11.3.7).

An attacker may try to split content across appends so that no single append matches an inspection policy. Processing appends in order, across append boundaries (Section 11.3.1), handles this. Evaluating each append on its own does not.

A digest sent by the client is only a claim. Servers and intermediaries can use it to detect errors or to look up earlier verdicts, but have to verify it against the actual content before relying on it (Section 11.3.8).

The reorder window bounds how much out-of-order data is buffered per upload, but an attacker can still open many uploads. Servers and intermediaries need to limit the number of uploads per client, the total amount of buffered data, and how long they hold delayed appends. They can respond with 429 (Too Many Requests) when overloaded.

Incomplete uploads contain gaps and possibly uninspected data. They are never exposed as the representation of a resource (Section 10.3), and gaps are never filled with bytes that could reveal other stored data.

Detailed inspection results would help an attacker adjust content until it passes. For that reason Upload-Inspection and "upload-blocked" responses carry only coarse information (Section 13).

Parallel mode sends the upload resource URI over more connections. As in [RESUMABLE], the URI needs to be hard to guess, and access to it needs to be tied to the client's authorization.

Appends are not sent in TLS early data (Section 12.1.3), and repeating an append with the same content has no effect, so replay does not cause harm.

15. IANA Considerations

15.1. HTTP Field Names

IANA is asked to register the following fields in the "Hypertext Transfer Protocol (HTTP) Field Name Registry" established by [RFC9110]:

Table 3: Field Registrations
Field Name Status Structured Type Reference
Upload-Parallel permanent Item Section 4.1 of this document
Upload-Received permanent List Section 6 of this document
Upload-Inspection permanent Dictionary Section 11.3.6 of this document

15.2. Upload-Limit Parameters

[RESUMABLE] does not establish a registry for Upload-Limit parameters, so the parameters max-parallel, reorder-window, append-alignment, and commit are defined in Section 4.2 of this document without a registration. If such a registry is created, IANA is asked to add these four parameters to it, with a reference to Section 4.2.

15.3. HTTP Problem Types

IANA is asked to register the problem types listed in Table 2 in the "HTTP Problem Types" registry established by [RFC9457], each with a reference to Section 13 of this document.

16. References

16.1. Normative References

[RESUMABLE]
Kleidl, M., Zhang, G., and L. Pardue, "Resumable Uploads for HTTP", Work in Progress, Internet-Draft, draft-ietf-httpbis-resumable-upload-13, , <https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-resumable-upload-13>.
[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
[RFC8470]
Thomson, M., Nottingham, M., and W. Tarreau, "Using Early Data in HTTP", RFC 8470, DOI 10.17487/RFC8470, , <https://www.rfc-editor.org/rfc/rfc8470>.
[RFC9000]
Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, , <https://www.rfc-editor.org/rfc/rfc9000>.
[RFC9110]
Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, , <https://www.rfc-editor.org/rfc/rfc9110>.
[RFC9112]
Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP/1.1", STD 99, RFC 9112, DOI 10.17487/RFC9112, , <https://www.rfc-editor.org/rfc/rfc9112>.
[RFC9113]
Thomson, M., Ed. and C. Benfield, Ed., "HTTP/2", RFC 9113, DOI 10.17487/RFC9113, , <https://www.rfc-editor.org/rfc/rfc9113>.
[RFC9114]
Bishop, M., Ed., "HTTP/3", RFC 9114, DOI 10.17487/RFC9114, , <https://www.rfc-editor.org/rfc/rfc9114>.
[RFC9218]
Oku, K. and L. Pardue, "Extensible Prioritization Scheme for HTTP", RFC 9218, DOI 10.17487/RFC9218, , <https://www.rfc-editor.org/rfc/rfc9218>.
[RFC9457]
Nottingham, M., Wilde, E., and S. Dalal, "Problem Details for HTTP APIs", RFC 9457, DOI 10.17487/RFC9457, , <https://www.rfc-editor.org/rfc/rfc9457>.
[RFC9530]
Polli, R. and L. Pardue, "Digest Fields", RFC 9530, DOI 10.17487/RFC9530, , <https://www.rfc-editor.org/rfc/rfc9530>.
[RFC9651]
Nottingham, M. and P. Kamp, "Structured Field Values for HTTP", RFC 9651, DOI 10.17487/RFC9651, , <https://www.rfc-editor.org/rfc/rfc9651>.

16.2. Informative References

[BYTERANGE]
Wright, A., "Byte Range PATCH", Work in Progress, Internet-Draft, draft-ietf-httpapi-patch-byterange-04, , <https://datatracker.ietf.org/doc/html/draft-ietf-httpapi-patch-byterange-04>.
[TUS]
tus project, "tus resumable upload protocol, version 1.0.0", , <https://tus.io/protocols/resumable-upload>.

Acknowledgments

This document builds on the work of the authors of [RESUMABLE] and [BYTERANGE], and on the tus protocol [TUS].

Authors' Addresses

Santosh Pallagatti
Zscaler
Akshat Maheshwari
Zscaler