Core calculator smoke — gemma4 | Version 7

Scenario

Core Calculator Tool Smoke (v7)

The core calculator tool spec is preloaded in workspace config. Reply with exactly: Spec saved.
Call self_improve_backup_live, then call self_improve_create_sandbox. Reply with exactly: Sandbox ready.
Call self_improve_generate_tool with {} to generate the core calculator tool from the saved spec. Reply with exactly: Tool generated.
Call self_improve_build_sandbox. Reply with exactly: Build done.
Call self_improve_invoke_sandbox_tool with tool_name calc_eval, arguments_json {"expression":"1+6"}, expected_result 7. If the output contains TOOL TEST PASSED, reply exactly: Tool works. Otherwise reply exactly: Tool broken.
Call self_improve_invoke_sandbox_tool with tool_name calc_eval, arguments_json {"expression":"65466641/2*6"}, expected_result 196399923. If the output contains TOOL TEST PASSED, reply exactly: Tool works. Otherwise reply exactly: Tool broken.
Call self_improve_invoke_sandbox_tool with tool_name calc_eval, arguments_json {"expression":"12*(5-2)"}, expected_result 36. If the output contains TOOL TEST PASSED, reply exactly: Tool works. Otherwise reply exactly: Tool broken.
Call self_improve_invoke_sandbox_tool with tool_name calc_eval, arguments_json {"expression":"(100-25)/5"}, expected_result 15. If the output contains TOOL TEST PASSED, reply exactly: Tool works. Otherwise reply exactly: Tool broken.
Call self_improve_verify_sandbox_tool with {}. If the output contains VERIFICATION PASSED and TOOL TEST PASSED, reply exactly: PASS. Otherwise reply exactly: FAIL.
Call self_improve_promote_sandbox. Reply with exactly: Promoted.
Use tool calling to invoke calc_eval with {"expression":"1+6"}. If the tool result is 7, reply exactly: Live OK. Otherwise reply exactly: Live FAIL.
Use tool calling to invoke calc_eval with {"expression":"65466641/2*6"}. If the tool result is 196399923, reply exactly: Live OK. Otherwise reply exactly: Live FAIL.
Use tool calling to invoke calc_eval with {"expression":"12*(5-2)"}. If the tool result is 36, reply exactly: Live OK. Otherwise reply exactly: Live FAIL.
Use tool calling to invoke calc_eval with {"expression":"(100-25)/5"}. If the tool result is 15, reply exactly: Live OK. Otherwise reply exactly: Live FAIL.

Console log

Session transcript

Loading Kokoro model (first run may download ~320MB)...
Kokoro ready (voice: af_heart).
Initializing client for model 'gemma4'...
Checking Ollama connection...
Ollama is running.
Running model 'gemma4' to ensure it is ready...
Pulling model 'gemma4'…
Model 'gemma4' ready (00:01).
Model ready.
Querying model specs...
Context window: 131,072 tokens.
=== Lilith AI Chat (gemma4 via Ollama) ===
Type your message and press Enter. Type 'exit' to quit.

[TOOL] get_memory_at_path → No memory found at path 'user/name'.

Hello! I don't seem to have your name stored. Would you mind telling me what it is? I can remember it for you.

prompt: 4419 completion: 30 total: 4449

Spec saved.

[TOOL] self_improve_backup_live → Live source backed up to E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\backups\20260527-104144

[TOOL] self_improve_create_sandbox → Sandbox created at E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox

prompt: 4477 completion: 62 total: 4539

Sandbox ready.

[TOOL] self_improve_generate_tool → Generated tool 'calc_two_plus_two' (self) at E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Lilith\Tools\Generated\CalcTwoPlusTwoTool.cs. Next: self_improve_build_sandbox, then self_improve_verify_sandbox_tool.

prompt: 4663 completion: 4 total: 4667

Tool generated.

[TOOL] self_improve_build_sandbox → Exit code: 0

Determining projects to restore...

Restored E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Lilith\Lilith.csproj (in 467 ms).

Restored E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Agent-Addons\Agent-Addons.csproj (in 467 ms).

Restored E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Interfaces\ConsoleApp\LilithConsole\LilithConsole.csproj (in 467 ms).

Restored E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Agent-Core\Agent-Core.csproj (in 467 ms).

Agent-Core -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Agent-Core\bin\Release\net8.0\Agent-Core.dll

Agent-Addons -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Agent-Addons\bin\Release\net8.0\Agent-Addons.dll

Lilith -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Lilith\bin\Release\net8.0\Lilith.dll

LilithConsole -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\LilithConsole.dll

Copied Lilith v7 source to bin\Release\net8.0\ShippedSource\

Build succeeded.

0 Warning(s)

0 Error(s)

Time Elapsed 00:00:10.85

prompt: 4817 completion: 4 total: 4821

Build done.

[TOOL] self_improve_invoke_sandbox_tool → TOOL TEST PASSED: calc_two_plus_two() => '2+2=4' (expected '2+2=4')

prompt: 5458 completion: 4 total: 5462

Tool works.

[TOOL] self_improve_verify_sandbox_tool → Exit code: 0

Determining projects to restore...

All projects are up-to-date for restore.

Agent-Core -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Agent-Core\bin\Release\net8.0\Agent-Core.dll

Agent-Addons -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Agent-Addons\bin\Release\net8.0\Agent-Addons.dll

Lilith -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Lilith\bin\Release\net8.0\Lilith.dll

Copied Lilith v7 source to bin\Release\net8.0\ShippedSource\

Build succeeded.

0 Warning(s)

0 Error(s)

Time Elapsed 00:00:03.46

Source check: tool 'calc_two_plus_two' appears in sandbox source.

Exit code: 0

Determining projects to restore...

Restored E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\DesignTests\AgentCore.DesignTests.csproj (in 430 ms).

3 of 4 projects are up-to-date for restore.

Agent-Core -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Agent-Core\bin\Release\net8.0\Agent-Core.dll

Agent-Addons -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Agent-Addons\bin\Release\net8.0\Agent-Addons.dll

Lilith -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\Lilith\bin\Release\net8.0\Lilith.dll

AgentCore.DesignTests -> E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\DesignTests\bin\Release\net8.0\AgentCore.DesignTests.dll

Test run for E:\OS-Website\src\Version7\Interfaces\ConsoleApp\LilithConsole\bin\Release\net8.0\workspace\output\self-improvement\sandbox\DesignTests\bin\Release\net8.0\AgentCore.DesignTests.dll (.NETCoreApp,Version=v8.0)

A total of 1 test files matched the specified pattern.

Passed! - Failed: 0, Passed: 5, Skipped: 0, Total: 5, Duration: 386 ms - AgentCore.DesignTests.dll (net8.0)

Tool invoke: calc_two_plus_two() => '2+2=4' (expected '2+2=4')

TOOL TEST PASSED

VERIFICATION PASSED for 'calc_two_plus_two'. You may call self_improve_promote_sandbox.

prompt: 5561 completion: 4 total: 5565

PASS

prompt: 6575 completion: 2 total: 6577

Goodbye.