Skip to main content

Backend Worker HTTP/1.1 Client

Use this guide when the backend fails to call a worker service even though DNS, service URL, and payload shape look correct.

Symptoms​

  • Backend calls to chatbot-svc, grading-svc, or another Python worker fail immediately.
  • Worker logs show malformed or invalid HTTP requests rather than application validation errors.
  • The same endpoint works with curl or a simple HTTP/1.1 client.
  • The failure appears after changing a RestClient builder or request factory.

Known Failure History​

CommitSurfaceFailure modeDurable fix
b8d0f278Chatbot workerJDK client negotiated HTTP/2 over plaintext while uvicorn only accepted HTTP/1.1.ChatbotClient builds HttpClient.Version.HTTP_1_1.
9dcb463dOMR/report worker callsThe same protocol risk existed for worker calls using shared RestClient wiring.RestClientConfig and OmrServiceClient pin HTTP/1.1.

Current Runtime Contract​

The backend uses Java RestClient, but worker services are Python/FastAPI processes served by uvicorn-style HTTP/1.1 endpoints. Backend clients must therefore pin HTTP/1.1 explicitly.

Current source anchors:

Source pathContract
modules/ai/src/main/java/com/lumie/ai/adapter/out/external/ChatbotClient.javaBuilds a JDK HttpClient with HttpClient.Version.HTTP_1_1.
modules/exam/src/main/java/com/lumie/exam/adapter/out/config/RestClientConfig.javaShared exam RestClient.Builder uses an HTTP/1.1 request factory.
modules/exam/src/main/java/com/lumie/exam/adapter/out/external/OmrServiceClient.javaOMR call overrides read timeout but keeps HTTP/1.1.

Diagnosis​

  1. Confirm the failing dependency is a backend-to-worker HTTP call, not RabbitMQ callback processing.
  2. Inspect worker logs for invalid HTTP request parsing, early connection close, or protocol mismatch language.
  3. Check the backend client source and verify the request factory uses a JDK HttpClient pinned to HTTP_1_1.
  4. Confirm the call still forwards X-Tenant-Slug; protocol fixes should not remove tenant propagation.
  5. Compare with a direct HTTP/1.1 probe to the same worker URL.

Source check:

cd lumie-backend
rg -n "HTTP_1_1|JdkClientHttpRequestFactory|X-Tenant-Slug" \
  modules/ai/src/main/java modules/exam/src/main/java

Fix​

Use a request factory built from an explicit HTTP/1.1 JDK client:

HttpClient http11 = HttpClient.newBuilder()
        .version(HttpClient.Version.HTTP_1_1)
        .build();
RestClient client = restClientBuilder
        .requestFactory(new JdkClientHttpRequestFactory(http11))
        .build();

For long-running image or report calls, keep the protocol pin and adjust read timeout separately. Do not replace the request factory with a default builder that may reintroduce protocol negotiation.

Verification Evidence​

  • worker logs no longer show invalid HTTP request parsing for the backend call,
  • backend logs reach application-level success or validation failure,
  • the request still includes X-Tenant-Slug,
  • OMR grading and chatbot stream paths keep their separate timeout and streaming behavior,
  • tests or source checks cover the client that was changed.

Prevention​

  • Treat backend-to-worker HTTP client configuration as part of the worker compatibility contract.
  • When adding a new worker client, copy the HTTP/1.1 request factory pattern before tuning timeouts.
  • Do not use the JDK default protocol negotiation for uvicorn-backed services.
  • Keep protocol fixes independent from payload, tenant, and timeout fixes so regressions are easy to identify.