Docling parallelism#

Two settings decide how many documents are converted by Docling at the same time: QANSWER_CELERY_DOCLING_POOL and QANSWER_CELERY_DOCLING_CONCURRENCY. This page explains how to choose them from the number of Python backend replicas and Docling worker replicas you run.

How a document reaches Docling#

Every PDF or office document that is indexed goes through three stages:

• Each Python backend replica runs a Celery worker that consumes the docling_extraction_queue queue. These two settings configure that worker.
• The Docling API server (docling, QANSWER_DOCLING_ENDPOINT) receives each request and puts it in a Redis queue (RQ).
• The Docling worker replicas (docling-worker) take the jobs from that queue and convert them. Each replica runs one RQ worker, which takes one job at a time, so it counts as one converter.

One Celery task is one blocking HTTP request to the API server for one batch of pages. A batch is 8 pages unless you set QANSWER_INDEXING_PAGE_BATCH_SIZE (empty keeps the backend default).

What the settings control#

• QANSWER_CELERY_DOCLING_POOL is the Celery pool type. The bundle default is prefork.
• QANSWER_CELERY_DOCLING_CONCURRENCY is the number of slots in that pool. The bundle default is 2.

With prefork, N slots keep N conversions in flight per backend replica. The total number of requests sent to Docling is therefore the number of backend replicas times the concurrency.

The rule#

Aim for about two requests in flight per Docling worker: one being converted and one waiting, so a converter never sits idle between two jobs. With QANSWER_SCALING_PYTHON_REPLICAS_NUMBER backend replicas and QANSWER_SCALING_DOCLING_WORKER_REPLICAS_NUMBER Docling workers, that gives:

concurrency = max(1, round(2 × docling workers ÷ backend replicas))

Round to the nearest whole number, with a minimum of 1: with fewer Docling workers than backend replicas, keep 1 per backend.

Common layouts

The bundle defaults (2 backend replicas, 2 Docling workers) are the third row.

Worked example#

You run 3 backend replicas and 4 Docling workers. The target is 2 x 4 = 8 requests in flight, and 8 / 3 = 2.67, which rounds to a concurrency of 3. Set the following in .env.override:

QANSWER_SCALING_PYTHON_REPLICAS_NUMBER=3
QANSWER_SCALING_DOCLING_WORKER_REPLICAS_NUMBER=4
QANSWER_CELERY_DOCLING_POOL=prefork
QANSWER_CELERY_DOCLING_CONCURRENCY=3

That keeps 3 x 3 = 9 requests in flight against 4 converters: close to the target of 8.

Why not higher#

Requests beyond what the converters can take do not go faster: they wait in Docling's queue. That queue has no priorities, so every extra request a large upload has in flight waits ahead of other users' documents. QAnswer spreads each user's batches over the priority levels of its Celery broker, which only helps while the batches are still waiting on the QAnswer side.

When a conversion fails#

When a Docling request fails or times out, the backend logs a warning ("Docling serve conversion failed, falling back to vanilla parser") and converts that batch with its plain parser instead (Tesseract OCR when OCR was requested). The limits are:

• The Docling API server waits at most 200 seconds (DOCLING_SERVE_MAX_SYNC_WAIT) for a synchronous conversion, and the backend asks Docling to stop a batch after 180 seconds.
• A Docling job is stopped after 300 seconds (DOCLING_SERVE_ENG_RQ_JOB_TIMEOUT), whatever the request that created it is doing.

In practice the cause is usually a Docling worker or the API server being killed for lack of memory, which fails every request it has in flight: in one of our runs a single restart of the API server sent 121 of 255 batches to the plain parser. In the measurements above no batch fell back, even with 4 per backend.

Memory#

• Docling worker: QANSWER_DOCLING_WORKER_MEMORY_LIMIT, 8G by default.
• Docling workers grow over time (from 3 to 8 GiB over a day in our runs, docling-serve issue 366). When a worker stays above QANSWER_DOCLING_WORKER_RECYCLE_MB (6144 by default) for five checks in a row, its health check fails and autoheal restarts it gracefully, after the job in hand is finished. Keep this threshold below the memory limit by more than the peak of one job.
• Docling API server: QANSWER_DOCLING_APISERVER_MEMORY_LIMIT, 4G by default. It needs more memory as more requests are in flight: at 1 GiB it was killed with four requests in flight.
• Python backend: QANSWER_PYTHON_BACKEND_MEMORY_LIMIT, 20G by default.

With prefork, each slot is a separate process inside the backend container, so a higher concurrency counts against the backend limit. In the measurements above the backend stayed below 9.5 GiB of its 20 GiB with up to 4 slots.

Applying the change#

Set the variables in the .env.override file in the bundle directory (create it if it does not exist). It is loaded after .env and .env.specs, so it wins, and it is not committed. The replica counts live in .env.specs and can be overridden the same way.

# .env.override
QANSWER_CELERY_DOCLING_POOL=prefork
QANSWER_CELERY_DOCLING_CONCURRENCY=2

Then re-run the deployment script, which recreates the services whose configuration changed:

./deploy-scripts/up.sh

Checking the result#

If you run the monitoring bundle, two Grafana dashboards show whether the layout works:

• Celery Monitoring - Flower Metrics: the panel Currently Executing Tasks (by worker type) shows how many Docling extraction tasks run at once; it should approach backend replicas times concurrency during a large upload.
• Docling Serve - OTel Observability: Active Requests (Concurrency) and Request Duration Percentiles show the load on the API server, Error Rate Over Time (4xx / 5xx) shows failed requests, and the container health panels show worker memory and restarts.

While a large file indexes, look at how busy the Docling workers are. Well below 80% means there is room for more concurrency; around 80% or more means more concurrency only adds waiting.

In the backend logs, search for "falling back to vanilla parser". Occasional lines are normal; a steady stream usually means Docling containers are restarting.