Enterprise resource planning systems execute the core transactions of a company, but the coordination between processes still rests with people. When a new customer places an order, someone checks the customer data, looks up or creates the customer record, checks stock, informs the responsible colleague, prepares documents and follows up on missing information. Each step is trivial; the coordination across master data, sales and inventory consumes the time.

Information systems research frames this as delegation to agentic artefacts, which no longer wait to be used but take responsibility for tasks with ambiguous requirements (Baird & Maruping, 2021). LLM-based agents that plan, call tools and document their reasoning make such delegation technically feasible (Wang et al., 2024).

The open question is where autonomy should end. Automation research shows that reliance only works when trust is calibrated to actual capability, since over-trust and under-trust both degrade performance (Lee & See, 2004), and the debate on human oversight requires the supervising person to retain causal power over the system and epistemic access to it (Sterz et al., 2024). In an ERP setting this becomes concrete: which decisions may the agent execute, and which must it submit? A promising lever is the agent's own confidence estimate, yet self-assessments of large language models are frequently poorly calibrated (Geng et al., 2024).

The project asks how autonomously an agent can act in a real ERP process before human oversight becomes necessary, and how trust in such an agent can be operationalised and measured.

Project Objectives: The project develops a functional prototype of an AI employee on top of ERPNext, an open-source ERP system that can be installed and filled with data freely. The agent receives a business instruction such as a new customer order and coordinates the required steps across processes instead of executing a single task. Target components are:

• Planning: the agent decomposes the instruction into an executable plan and revises it when steps fail.
• Execution: the agent works against the ERPNext API, for example checking customer data, looking up or creating records, checking stock and preparing documents.
• Escalation: the agent identifies missing information and routes critical decisions to a human for approval instead of executing them.
• Traceability: every decision, tool call and justification is logged so that actions can be reconstructed afterwards.
• Confidence estimation: the agent reports how certain it is per decision, and this self-assessment is evaluated against actual correctness using a calibration curve.

The group defines at least two levels of autonomy and compares them empirically on a set of test scenarios, including incomplete and faulty inputs. Suggested measures are task success rate, human correction rate and calibration of the agent's self-assessment.

Project Requirements:

• Creation of a project plan and allocation of tasks among the group members using project management techniques (standard/mandatory)
• Design and implementation of the agent, including planning, tool use via the ERPNext API and a complete decision log
• Definition of at least two levels of autonomy and implementation of an escalation mechanism for human approval
• Implementation of a confidence estimate per decision and its evaluation by means of a calibration curve
• Evaluation of the prototype on a defined set of test scenarios, including incomplete and faulty inputs, using task success rate and human correction rate
• Project documentation and presentation of the results in interim and final presentations (standard/mandatory)

Prerequisites:

• Students of the Faculty of Computer Science in the fields of Information Systems or Software Engineering at bachelor's level (with at least 90 ECTS) or at master's level (standard/mandatory)
• Solid programming skills (preferably Python) and willingness to work with LLM APIs and agent frameworks
• Interest in business processes and enterprise systems; prior knowledge of ERPNext is not required

Literatur

• Baird, A., & Maruping, L. M. (2021). The next generation of research on IS use: A theoretical framework of delegation to and from agentic IS artifacts. MIS Quarterly, 45(1), 315–341. doi.org/10.25300/MISQ/2021/15882 
• Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I. (2024). A survey of confidence estimation and calibration in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (pp. 6577–6595). doi.org/10.18653/v1/2024.naacl-long.366 
• Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. doi.org/10.1518/hfes.46.1.50_30392 
• Sterz, S., Baum, K., Biewer, S., Hermanns, H., Lauber-Rönsberg, A., Meinel, P., & Langer, M. (2024). On the quest for effectiveness in human oversight: Interdisciplinary perspectives. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (pp. 2495–2507). https://doi.org/10.1145/3630106.3659051 
• Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., Zhao, W. X., Wei, Z., & Wen, J.-R. (2024). A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6), 186345. doi.org/10.1007/s11704-024-40231-1 

If you are interested in this porject, follow the steps below to submit your application.

1. Form a project group
Projects are typically carried out in groups of 3–5 students. We recommend forming a group with fellow students before applying.

2. Prepare your application
Send a short application including:

• your transcript of records
• a short motivation letter (about one page) explaining why your group is interested in the project.

3. Submit your application
Send your application via email to:jannis.nacke (at) icb.uni-due.de