Analysis

Purpose

analysis-svc is the worker that generates written exam commentary and per-student feedback from structured statistics. Unlike grading and report, it is not queue-driven. Callers send HTTP requests and receive a generated text response immediately.

For the repo-wide worker model, see Workers Overview.

Source Paths

Path	Role
lumie-worker/services/analysis/main.py	FastAPI app, lifespan, route handlers, metrics mount
lumie-worker/services/analysis/src/schema.py	Wire request and response models
lumie-worker/services/analysis/src/usecase.py	LLM call orchestration and metrics
lumie-worker/services/analysis/src/domain/prompts.py	Korean prompt templates and prompt builders
lumie-worker/services/analysis/src/adapters/llm.py	AsyncOpenAI client adapter
lumie-worker/services/analysis/src/joossameng/router.py	Joossameng-only AI report HTTP routes
lumie-worker/services/analysis/src/joossameng/service.py	AI report batch orchestration and ZIP assembly
lumie-worker/services/analysis/src/joossameng/renderer.py	ReportLab renderer for Joossameng AI report PDFs
lumie-worker/services/analysis/assets/fonts/	Bundled Noto Sans KR fonts used by AI report PDFs
lumie-worker/services/analysis/src/config.py	LLM_* and OTel settings
lumie-worker/services/analysis/tests/test_analysis_usecase.py	Use-case behavioral tests
lumie-worker/services/analysis/tests/test_joossameng_renderer.py	AI report font, wrapping, and pagination tests
lumie-worker/services/analysis/tests/test_joossameng_service.py	AI report job orchestration test
lumie-worker/services/analysis/tests/test_observability.py	Metrics and tracing smoke tests

Public Surface

Routes:

• GET /
• GET /health
• GET /metrics
• POST /api/analysis/exam-commentary
• POST /api/analysis/student-feedback
• POST /api/joossameng/ai-reports/exams/{exam_id}/batch
• GET /api/joossameng/ai-reports/jobs/{job_id}
• GET /api/joossameng/ai-reports/jobs/{job_id}/download

Both generation routes use GenerationResponse from lumie-worker/services/analysis/src/schema.py as the response model in lumie-worker/services/analysis/main.py:

{
  "content": "..."
}

The joossameng AI report routes are a tenant-limited extension. They require X-Tenant-Slug: joossameng; any other tenant receives 403 from the worker route before a job is created or read.

Request Models

/api/analysis/exam-commentary accepts aggregate exam data such as:

• exam name
• participant count
• average, highest, and lowest score
• grade distribution
• per-question correctness statistics

/api/analysis/student-feedback accepts student-level data such as:

• student name and exam name
• total score, grade, and exam average
• incorrect questions with selected and correct answers
• achievement grouped by question type

The request and response contracts live in services/analysis/src/schema.py and use Pydantic validation directly against the wire format.

The schemas intentionally use camelCase fields without aliases because the backend and frontend callers already speak camelCase JSON. Do not convert these models to snake_case unless every caller is migrated in the same change.

The Joossameng AI report batch route accepts only the studentIds list from AiReportBatchRequest in lumie-worker/services/analysis/src/joossameng/router.py:

{
  "studentIds": [101, 102]
}

It returns a worker-local job id. The status route reports processed, success, and failure counts; the download route returns a ZIP after the job reaches COMPLETED.

Exam Commentary Request

This example is anchored to ExamCommentaryRequest in lumie-worker/services/analysis/src/schema.py.

{
  "examName": "June Mock Exam",
  "totalParticipants": 120,
  "averageScore": 71.4,
  "highestScore": 98,
  "lowestScore": 22,
  "gradeDistribution": [
    { "grade": 1, "count": 7, "percentage": 5.8 }
  ],
  "questionStatistics": [
    {
      "questionNumber": 31,
      "correctRate": 0.42,
      "incorrectRate": 0.58,
      "questionType": "빈칸추론",
      "actualScore": 3
    }
  ]
}

Student Feedback Request

This example is anchored to StudentFeedbackRequest in lumie-worker/services/analysis/src/schema.py.

{
  "examName": "June Mock Exam",
  "studentName": "Student A",
  "totalScore": 84,
  "grade": 2,
  "averageScore": 71.4,
  "incorrectQuestions": [
    {
      "questionNumber": 31,
      "questionType": "빈칸추론",
      "selectedChoice": "2",
      "correctAnswer": "4",
      "questionCorrectRate": 42.0
    }
  ],
  "questionTypeAchievement": [
    { "type": "빈칸추론", "correctCount": 2, "totalCount": 4, "correctRate": 50.0 }
  ]
}

Generation Flow

1. FastAPI validates the incoming JSON into typed request models.
2. AnalysisUseCase builds a prompt from structured input using pure functions in src/domain/prompts.py.
3. The service calls an OpenAI-compatible chat completion API through openai.AsyncOpenAI.
4. The returned message content is wrapped as GenerationResponse.

The two generation paths differ mainly in prompt construction:

• exam commentary focuses on distribution-level trends and difficult question patterns
• student feedback focuses on one student's mistakes, question-type performance, and study guidance

The current prompt templates are written for teacher-style, plain-text output and are tuned for Korean-language responses, but the service contract itself is simple text in and text out.

Joossameng AI Report Flow

The Joossameng AI report path reuses the same AnalysisUseCase, but returns rendered PDFs instead of raw generated text:

1. The backend calls the worker POST /api/joossameng/ai-reports/exams/{exam_id}/batch route with X-Tenant-Slug: joossameng.
2. The worker creates an in-memory job and schedules JoossamengAiReportService.run_job(...) as a FastAPI background task.
3. The worker fetches report batch data from backend /internal/reports/exams/{exam_id}/batch-data with the internal HMAC signature and X-Tenant-Slug: joossameng.
4. Exam-level commentary and student feedback are read from the Joossameng AI cache when present, otherwise generated through AnalysisUseCase and saved back to the cache.
5. src/joossameng/renderer.py renders each successful student item into a PDF with ReportLab. Rendering runs through asyncio.to_thread(...) so the CPU-bound PDF work does not block the event loop.
6. Successful PDFs are written into an in-memory ZIP. The ZIP writestr(...) call also runs through asyncio.to_thread(...) so DEFLATE compression does not run on the event loop. Per-student failures increment fail_count and do not stop the whole job.
7. The download route returns the ZIP only after the job is COMPLETED.

The PDF renderer tries the bundled NotoSansKR-Regular.ttf and NotoSansKR-Bold.ttf files from services/analysis/assets/fonts/ first, then falls back to platform fonts and finally ReportLab's HYGothic-Medium CID font if no TrueType/OpenType font can be registered. It wraps Korean body text by measured ReportLab string width, preserves visible indentation for structured LLM output, and splits long body boxes across pages instead of letting text run outside the page or box boundary.

Prompt Contract

The prompt builders are pure functions. They classify exam difficulty, summarize grade distribution, choose high-incorrect-rate questions, group question types, and then render a Korean instructor prompt. The use case does not mutate the request or fetch additional data.

The generated output is expected to be:

• plain text, not Markdown;
• Korean teacher-style prose;
• bounded by max_tokens=1024 for exam commentary;
• bounded by max_tokens=2048 for student feedback.

If product requirements change to multilingual output, the prompt contract should change explicitly. The current service does not infer output language from a request field.

Configuration And Dependencies

Key settings:

• LLM_API_KEY
• LLM_BASE_URL
• LLM_MODEL
• LUMIE_BACKEND_URL
• LUMIE_INTERNAL_HMAC_SECRET
• JOOSSAMENG_DATABASE_DSN
• OTEL_ENABLED
• OTEL_ENDPOINT
• OTEL_SERVICE_NAME

Default model settings in code are:

• base URL: https://api.openai.com/v1
• model: gpt-4o-mini

The service owns the AsyncOpenAI client lifecycle in its lifespan hook and closes that client during shutdown.

LUMIE_BACKEND_URL and LUMIE_INTERNAL_HMAC_SECRET are required only by the Joossameng AI report path when it fetches backend batch data. If JOOSSAMENG_DATABASE_DSN is absent, the AI report cache falls back to in-memory process storage.

Observability And Failure Handling

The analysis worker emits:

• analysis_llm_requests_total
• analysis_llm_duration_seconds
• analysis_llm_inflight

Metrics are labeled by operation:

• exam_commentary
• student_feedback

Failures are split into:

• api_error for upstream OpenAI-compatible API failures
• handler_failure for local bugs or unexpected runtime errors

Tracing is enabled through the shared worker observability helper. FastAPI and httpx are instrumented, which means outbound LLM traffic is traced without custom client code.

Failure Semantics

Failure	Result
Invalid JSON or missing required field	FastAPI/Pydantic validation error
OpenAI-compatible API raises APIError	metric result api_error; route returns 502
Local bug or unexpected runtime error	metric result handler_failure; route returns 500
Empty LLM message content	valid GenerationResponse with empty content
Non-joossameng tenant calls AI report route	403 with joossameng tenant only
AI report job has no successful PDFs	job status FAILED
AI report ZIP requested before completion	409 with job is not ready

The service deliberately separates api_error from handler_failure so dashboards can distinguish upstream LLM problems from local worker defects.

Operational Notes

• GET /health returns {"status":"ok"}.
• GET / returns a simple liveness payload for basic checks.
• CORS is enabled because this service is currently designed to be called directly by web-facing clients in some environments.
• LLM_API_KEY has no default and should fail startup when missing.
• Unit tests mock the LLM port and validate the orchestration behavior without making network calls.
• Joossameng AI report jobs are tracked in process memory; worker restart drops job status and ZIP bytes.

Verification

cd lumie-worker
uv run pytest services/analysis/tests

cd /path/to/Lumie
rg -n "exam-commentary|student-feedback|joossameng|analysis_llm" lumie-worker/services/analysis

Expected success signals:

• pytest exits 0
• services/analysis/tests/test_analysis_usecase.py passes the commentary, student feedback, and empty-content cases
• services/analysis/tests/test_joossameng_renderer.py passes the bundled font, width wrapping, whitespace preservation, and pagination cases
• services/analysis/tests/test_joossameng_service.py confirms PDF rendering and ZIP writes are dispatched through asyncio.to_thread(...)
• services/analysis/tests/test_observability.py passes the tracing and /metrics smoke tests
• the grep shows both analysis HTTP routes in services/analysis/main.py, the Joossameng AI report router under services/analysis/src/joossameng/, and the analysis_llm_* metric names under services/analysis/src/