4344-GRP ED-OS: Urgency Stage-Aware Scheduling of AI Tutor Agents on Low-Resource Local LLM Inference

Presenter Information

Location

https://campus.kennesaw.edu/colleges-departments/ccse/events/computing-showcase/fa26-cday-program.php

Document Type

Event

Start Date

20-11-2026 2:00 PM

Description

AI tutoring systems based on local agents repeatedly call small language models and external tools to guide students. Agents compete for limited inference resources during concurrent student request in commodity hardware. To resolve this, we build ecosystem where agents’ function as processes, tool calls as I/O operations, and inference slots as physical frames. ED-OS schedules agent operating in urgency with stage-aware priority along with aging. This helps to complete the urgent requests first while no student starves. It also releases or swaps slots during tool calls using an online cost model, shares exercise prefixes, and caches. We evaluate ED-OS by recording 204 three-agent tutor sessions with Qwen2.5-Coder-3B and replaying them on several runs with a Java discrete-event simulator. Compared with plain llama.cpp, the full layer cut the 95th-percentile time to a verified hint by 41% and raised throughput by 61%, while every student received the same hint. The urgency stage-aware scheduler matched FCFS on p95 latency (9.2 s vs. 9.4 s) while round-robin and fair share were 42-67% slower, and giving flagged students priority cut their p95 latency by 26% at a 10% cost to others. The study contributes to providing alternative solution to schedule the agent tasks based on their urgency and agent role priority on low-resource hardware.

This document is currently not available here.

Share

COinS
 
Nov 20th, 2:00 PM

4344-GRP ED-OS: Urgency Stage-Aware Scheduling of AI Tutor Agents on Low-Resource Local LLM Inference

https://campus.kennesaw.edu/colleges-departments/ccse/events/computing-showcase/fa26-cday-program.php

AI tutoring systems based on local agents repeatedly call small language models and external tools to guide students. Agents compete for limited inference resources during concurrent student request in commodity hardware. To resolve this, we build ecosystem where agents’ function as processes, tool calls as I/O operations, and inference slots as physical frames. ED-OS schedules agent operating in urgency with stage-aware priority along with aging. This helps to complete the urgent requests first while no student starves. It also releases or swaps slots during tool calls using an online cost model, shares exercise prefixes, and caches. We evaluate ED-OS by recording 204 three-agent tutor sessions with Qwen2.5-Coder-3B and replaying them on several runs with a Java discrete-event simulator. Compared with plain llama.cpp, the full layer cut the 95th-percentile time to a verified hint by 41% and raised throughput by 61%, while every student received the same hint. The urgency stage-aware scheduler matched FCFS on p95 latency (9.2 s vs. 9.4 s) while round-robin and fair share were 42-67% slower, and giving flagged students priority cut their p95 latency by 26% at a 10% cost to others. The study contributes to providing alternative solution to schedule the agent tasks based on their urgency and agent role priority on low-resource hardware.