Troubleshoot File Upload Issues

When file uploads are enabled, users can upload an existing audio or video file instead of recording a new one. The recording client sends the file to our media servers securely over HTTPS.

An upload can fail for two very different reasons:

  1. The server rejects the upload. Something about the upload itself is wrong, so sending it again would fail the same way. The recording client stops and shows an error.
  2. The upload doesn’t get through. The network drops, the server is temporarily unavailable or too busy. These problems are usually temporary, so the recording client retries the upload automatically.

When the ingestion server rejects an upload

The server can reject an upload. Common reasons include:

  • the account was deleted;
  • the subscription is expired, or the trial period has ended;
  • the account hash or the environment ID is missing or invalid;
  • the website isn’t on the environment’s list of allowed hosts;
  • the file is larger than the file size limit or isn’t a supported file type.

Some of these, like an expired subscription or trial, are already checked when the recording client loads, during its initial check (precheck). If that check does not pass, the recording client shows the error right away and the user never gets to upload a file. So the upload itself is rarely rejected for these reasons: it mostly happens when the page was opened before the check would have failed, or when the upload doesn’t come from a recording client that loaded normally.

Also, the recording client checks the file type and size before the upload starts, so most users with an unsupported or oversized file see an error right away, without waiting for the upload.

When a rejection happens, the recording client:

  • does not retry the upload;
  • shows the reason sent by the server (for example “This account is expired or closed”) or, if there isn’t one, “Upload failed!”;
  • if the cause is the file size or the extension, shows a link so the user can record or upload another file.

Automatic retries

The recording client retries the upload automatically when:

  • the network connection fails or drops during the upload;
  • the upload stops making progress for 60 seconds (the client stops it and tries again);
  • the server replies with 408 Request Timeout, or with a 5xx status code other than 500 (for example 502 Bad Gateway or 503 Service Unavailable, which can happen while a server restarts or is under maintenance);
  • the reply isn’t the response the media server sends, which usually means something between the user and our servers (a captive portal on public Wi-Fi, a proxy) answered instead.

Each retry sends the whole file again. While the client waits, it shows the message “Connection lost. Retrying in X seconds…”, with a countdown to the next attempt.

The delay between retries doubles after each failed attempt, up to a maximum of 30 seconds:

Failed attemptWhat happens next
1stRetries after 1.5 seconds
2ndRetries after 3 seconds
3rdSwitches to the fallback region (see below), then retries after 6 seconds
4thRetries after 12 seconds
5thRetries after 24 seconds
6th and laterRetries every 30 seconds

The delays are fixed: there is no random jitter. There is also no limit on the number of retries. The client keeps trying until the upload succeeds, the server rejects it, the user presses Cancel upload, or the user leaves the page.

Fallback between regions

If the upload fails 3 times in a row, the recording client switches the upload to the fallback region and keeps retrying there. Like the fallback for the recording connection, it only happens between regions on the same continent: US1 and US2 fall back to each other, and so do EU2 and EU3. An upload will never fall back from a US region to an EU region or vice versa.

For example, if the environment uses US1 (N. California) and the first 3 attempts fail, the 4th attempt and every later one go to US2 (New York). The client does not switch back to US1.

The fallback only affects that one upload. The next file the user uploads starts again with the original region.

When a fallback happens, the resulting files, if stored by us, are stored in the fallback region (US2 in this example).

Too many uploads (429 Too Many Requests)

To protect our media servers, we limit how many uploads a user can start, and how many can run at the same time, from the same IP address. Past those limits, the server replies with 429 Too Many Requests. This can happen when many users share the same IP address, for example in an office or a school.

A 429 reply is not counted as a failed attempt. It doesn’t bring the upload closer to a fallback and doesn’t make the next delay longer. Instead, the recording client:

  • waits for the number of seconds given in the server’s Retry-After header (30 seconds if the header is missing);
  • shows the message “Too many uploads at the moment. Retrying in X seconds…”, with a countdown;
  • then retries the upload in the same region.

Cancelling an upload

While the file is being uploaded or the client is waiting to retry, the user can press Cancel upload. The client stops the upload and any retry that was scheduled, shows the initial screen again and shows the message “Upload cancelled.” Picking a new file also cancels the upload in progress.