Analysis

Purpose

analysis-svc is the worker that generates written exam commentary and per-student feedback from structured statistics. Unlike grading and report, it is not queue-driven. Callers send HTTP requests and receive a generated text response immediately.

For the repo-wide worker model, see Workers Overview.

Source Paths

Path	Role
lumie-worker/services/analysis/main.py	FastAPI app, lifespan, route handlers, metrics mount
lumie-worker/services/analysis/src/schema.py	Wire request and response models
lumie-worker/services/analysis/src/usecase.py	LLM call orchestration and metrics
lumie-worker/services/analysis/src/domain/prompts.py	Korean prompt templates and prompt builders
lumie-worker/services/analysis/src/adapters/llm.py	AsyncOpenAI client adapter
lumie-worker/services/analysis/src/config.py	LLM_* and OTel settings
lumie-worker/services/analysis/tests/test_analysis_usecase.py	Use-case behavioral tests
lumie-worker/services/analysis/tests/test_observability.py	Metrics and tracing smoke tests

Public Surface

Routes:

• GET /
• GET /health
• GET /metrics
• POST /api/analysis/exam-commentary
• POST /api/analysis/student-feedback

Both generation routes return:

{
  "content": "..."
}

Request Models

/api/analysis/exam-commentary accepts aggregate exam data such as:

• exam name
• participant count
• average, highest, and lowest score
• grade distribution
• per-question correctness statistics

/api/analysis/student-feedback accepts student-level data such as:

• student name and exam name
• total score, grade, and exam average
• incorrect questions with selected and correct answers
• achievement grouped by question type

The request and response contracts live in services/analysis/src/schema.py and use Pydantic validation directly against the wire format.

The schemas intentionally use camelCase fields without aliases because the backend and frontend callers already speak camelCase JSON. Do not convert these models to snake_case unless every caller is migrated in the same change.

Exam Commentary Request

{
  "examName": "June Mock Exam",
  "totalParticipants": 120,
  "averageScore": 71.4,
  "highestScore": 98,
  "lowestScore": 22,
  "gradeDistribution": [
    { "grade": 1, "count": 7, "percentage": 5.8 }
  ],
  "questionStatistics": [
    {
      "questionNumber": 31,
      "correctRate": 0.42,
      "incorrectRate": 0.58,
      "questionType": "빈칸추론",
      "actualScore": 3
    }
  ]
}

Student Feedback Request

{
  "examName": "June Mock Exam",
  "studentName": "Student A",
  "totalScore": 84,
  "grade": 2,
  "averageScore": 71.4,
  "incorrectQuestions": [
    {
      "questionNumber": 31,
      "questionType": "빈칸추론",
      "selectedChoice": "2",
      "correctAnswer": "4",
      "questionCorrectRate": 42.0
    }
  ],
  "questionTypeAchievement": [
    { "type": "빈칸추론", "correctCount": 2, "totalCount": 4, "correctRate": 50.0 }
  ]
}

Generation Flow

1. FastAPI validates the incoming JSON into typed request models.
2. AnalysisUseCase builds a prompt from structured input using pure functions in src/domain/prompts.py.
3. The service calls an OpenAI-compatible chat completion API through openai.AsyncOpenAI.
4. The returned message content is wrapped as GenerationResponse.

The two generation paths differ mainly in prompt construction:

• exam commentary focuses on distribution-level trends and difficult question patterns
• student feedback focuses on one student's mistakes, question-type performance, and study guidance

The current prompt templates are written for teacher-style, plain-text output and are tuned for Korean-language responses, but the service contract itself is simple text in and text out.

Prompt Contract

The prompt builders are pure functions. They classify exam difficulty, summarize grade distribution, choose high-incorrect-rate questions, group question types, and then render a Korean instructor prompt. The use case does not mutate the request or fetch additional data.

The generated output is expected to be:

• plain text, not Markdown;
• Korean teacher-style prose;
• bounded by max_tokens=1024 for exam commentary;
• bounded by max_tokens=2048 for student feedback.

If product requirements change to multilingual output, the prompt contract should change explicitly. The current service does not infer output language from a request field.

Configuration And Dependencies

Key settings:

• LLM_API_KEY
• LLM_BASE_URL
• LLM_MODEL
• OTEL_ENABLED
• OTEL_ENDPOINT
• OTEL_SERVICE_NAME

Default model settings in code are:

• base URL: https://api.openai.com/v1
• model: gpt-4o-mini

The service owns the AsyncOpenAI client lifecycle in its lifespan hook and closes that client during shutdown.

Observability And Failure Handling

The analysis worker emits:

• analysis_llm_requests_total
• analysis_llm_duration_seconds
• analysis_llm_inflight

Metrics are labeled by operation:

• exam_commentary
• student_feedback

Failures are split into:

• api_error for upstream OpenAI-compatible API failures
• handler_failure for local bugs or unexpected runtime errors

Tracing is enabled through the shared worker observability helper. FastAPI and httpx are instrumented, which means outbound LLM traffic is traced without custom client code.

Failure Semantics

Failure	Result
Invalid JSON or missing required field	FastAPI/Pydantic validation error
OpenAI-compatible API raises APIError	metric result api_error, exception re-raised
Local bug or unexpected runtime error	metric result handler_failure, exception re-raised
Empty LLM message content	valid GenerationResponse with empty content

The service deliberately separates api_error from handler_failure so dashboards can distinguish upstream LLM problems from local worker defects.

Operational Notes

• GET /health returns {"status":"ok"}.
• GET / returns a simple liveness payload for basic checks.
• CORS is enabled because this service is currently designed to be called directly by web-facing clients in some environments.
• LLM_API_KEY has no default and should fail startup when missing.
• Unit tests mock the LLM port and validate the orchestration behavior without making network calls.

Verification

cd lumie-worker
uv run pytest services/analysis/tests

cd /path/to/Lumie
rg -n "exam-commentary|student-feedback|analysis_llm" lumie-worker/services/analysis

Expected success signals:

• pytest exits 0
• services/analysis/tests/test_analysis_usecase.py passes the commentary, student feedback, and empty-content cases
• services/analysis/tests/test_observability.py passes the tracing and /metrics smoke tests
• the grep shows both HTTP routes in services/analysis/main.py and the analysis_llm_* metric names under services/analysis/src/