AGENT SWARM v2

Visual Workspace + Agent Routing + AI-Assisted Creation
Concept for Skales | Inspired by Discussion #34
@v33-kind
kind. curious. clauding.
Summary
What this addresses, why it matters, and what it enables.
Context: Discussion #34 surfaced a real use case. A user with 20+ custom agents, 3 provider instances, and 2 billion tokens per week hit the wall: agents can be created but can't work together. The CEO agent delegates, but every subtask ignores the assignment and runs on the default provider. This proposal addresses that and more.
Problem
Ghost Field
task.agent is saved but never read during execution. All subtasks use default provider.
Impact
20+ agents idle
Custom agents with specialized prompts and providers can't be used in delegation.
Fix
~25 lines
Read task.agent, look up definition, route to that agent's provider/model/prompt. Full patch in agent-routing-patch-v2.py.
Unlock
Multi-provider orchestration
CEO delegates to agents on different providers. Each uses their own prompt and model.
Visual Workspace
Interactive agent ecosystem. Drag nodes, click to select, see connections.
+ Agent (AI)
Scrolls to the AI builder. Describe an agent in plain language, generates config, adds to canvas.
+ Skill
Adds a skill node. Auto-connects to the first available agent. Click to edit name and status.
+ Task
Adds a task node. Click it, then "Assign Agent" to connect it to one or more agents.
Discover
Auto-discovers memory and schedule nodes. Connects tasks to memory, schedules to CEO. Fits canvas.
Routing: OFF/ON
Toggles the agent routing fix. When ON, delegation routes to assigned agents. When OFF, all tasks use default.
Delegate
Runs a live delegation simulation. CEO dispatches tasks to agents. Log shows routing decisions per task.
Fit
Scales and centers all nodes within the visible canvas area.
Reset
Returns all nodes to their original positions and deselects everything.
Export
Downloads the full workspace state (nodes + edges) as a JSON file.
Click node
Select it. Detail panel shows info + actions: Edit, Assign Agent, Toggle Status, Delete.
Drag node
Reposition it. Snaps to 40px grid. Connection lines update in real time.
Ctrl + Scroll
Zoom in/out on the canvas. Normal scroll moves the page.
Routing:
Routing fix: See agent-routing-patch-v2.py for root cause, 4 code changes, and testing steps. Toggle routing ON above to see the difference.
User Stories from Discussion #34
Each issue raised, categorized and addressed. Click to expand.
US-1Agent delegation ignores assigned agentFix Ready▼
Current:
  • task.agent field is saved but executeTask() never reads it
  • All subtasks run on the default provider regardless of assignment
Root cause:
  • Missing ~25 lines in the task executor
  • No lookup of the assigned agent's definition before calling agentDecide()
Impact:
  • Multi-agent orchestration is non-functional for delegation
  • 20+ custom agents with specialized prompts are unused
Solution:
  • agent-routing-patch-v2.py Changes 1-4 (lines 75-178)
  • Full working code provided, commented out for review
US-2Custom API not working on Lio AIFixed v8.0.1▼
Current:
  • Lio AI defaulted to OpenRouter for all three roles
  • Active provider setting was ignored
Root cause:
  • Lio AI config had hardcoded OpenRouter defaults
  • Separate from main provider settings
Impact:
  • Users with custom endpoints (codex-lb, LM Studio, vLLM) could not use them in Lio AI
Solution:
  • Fixed in v8.0.1
  • Lio AI settings now show Custom (OpenAI-compatible) for all three roles
US-3Autopilot doesn't know about custom agentsFix Ready▼
Current:
  • Autopilot uses DEFAULT provider and DEFAULT persona from Settings
  • Custom agents are invisible to it
Root cause:
  • Autopilot task creation has no agent selection
  • Autonomous runner supports assignedProvider/assignedModel but they're never populated from agent definitions
Impact:
  • Users cannot assign specialized agents to scheduled or automated tasks
Solution:
  • Same pattern as Changes 1-4
  • Autonomous runner already reads assignedProvider/assignedModel
  • Linking those to agent definitions via getAgent() completes the circuit
  • Confirmed for v8.0.1
US-4Custom agents as Lio AI rolesFeature▼
Current:
  • Lio AI uses a fixed 3-role system (Architect/Reviewer/Builder)
  • Generic prompts for each role
Root cause:
  • No mechanism to override built-in roles with custom agent definitions
Impact:
  • Users who've invested in building agents with detailed prompts can't leverage them in Lio AI
Solution:
  • Add an "Agent Override" dropdown to each Lio AI role
  • When set, use that agent's system prompt and provider instead of defaults
US-5Multiple custom providers per agentFix Ready▼
Current:
  • One custom endpoint URL in settings
  • Shared by all features and agents
Root cause:
  • AgentDefinition has provider and model fields but no baseUrl field
  • All custom agents share a single endpoint
Impact:
  • Users running multiple codex-lb instances cannot route different agents to different endpoints
Solution:
  • agent-routing-patch-v2.py Enhancement 2 (line 333)
  • Adds baseUrl and apiKey fields to AgentDefinition
  • Passes them through agentDecide to the provider config
  • Includes UI field suggestion for the agent creation page
US-6Vision Provider needs Custom optionFix Ready▼
Current:
  • Vision Provider dropdown only shows Google, OpenAI, Anthropic, OpenRouter
Root cause:
  • 'custom' is not listed in the Vision Provider options array
Impact:
  • Users with vision-capable local models cannot use them for screenshots and browser control
Solution:
  • Add 'custom' to the Vision Provider dropdown
  • Custom endpoint handler already supports vision-capable models
  • Single-line change
US-7Port 3000 conflictFix Suggested▼
Current:
  • If another service runs on port 3000, Skales starts silently
  • Displays the other service's UI with no error
Root cause:
  • Electron main process spawns Next.js server on port 3000 without checking availability
Impact:
  • Users see another app's UI thinking it's Skales
  • No error, no warning, silent failure
Solution:
  • agent-routing-patch-v2.py Bug Fix 1 (line 756)
  • TCP port probe tries 3000-3009
  • Error dialog if all ports taken
  • Desktop notification if fallback port used
US-8Lio AI input box freezes after buildFix Suggested▼
Current:
  • After a Lio AI build completes, input box stops accepting text
  • Happens after a couple of follow-up attempts
Root cause:
  • Build completion handler doesn't reset isProcessing/isBuilding state on error paths
  • Input gate stays closed permanently
Impact:
  • Users must restart the app to continue working after every build
Solution:
  • agent-routing-patch-v2.py Bug Fix 2 (line 815)
  • try/finally always resets input state
  • 10-minute safety timeout force-resets if build hangs
  • Direct DOM re-enable as safety net
US-9Autopilot says file created but nothing savedFix Suggested▼
Current:
  • Autopilot tasks report "file created" in the result
  • No file exists on disk
Root cause:
  • (A) Agent hallucinates file creation without issuing the actual tool call
  • (B) Sandbox mode blocks the write silently without returning an error
Impact:
  • Users trust the output, discover nothing was saved
  • Loss of work and trust in Autopilot
Solution:
  • agent-routing-patch-v2.py Bug Fix 3 (line 878)
  • Post-execution verification parses claimed file paths and checks they exist
  • Marks task as failed if files are missing
  • Pre-write sandbox check returns clear error instead of silent block
US-10GitHub integration for Lio AIFeature▼
Current:
  • Lio AI builds projects in a local workspace folder
  • No version control integration
Root cause:
  • No Git skill exists
  • Lio AI cannot clone, branch, commit, or push
Impact:
  • Users must manually move Lio AI output into their repos
  • Cannot iterate on existing GitHub projects
Solution:
  • Git skill that can clone, branch, commit, and push
  • Combined with Lio AI code builder for autonomous development
  • Enables iteration on existing repositories
Solutions Delivered
All suggested code is in agent-routing-patch-v2.py. Commented out for review. Nothing executes without approval.
Core Fix
1.Agent Routing(Lines 75-178)
  • 4 changes across 2 files, ~25 lines total
  • Reads task.agent field and looks up agent definition
  • Routes to assigned agent's provider, model, and system prompt
  • Fully backward compatible - unassigned tasks use defaults as before
Enhancements
2.Tool Filtering(Line 256)
  • Filter available tools per agent based on capabilities whitelist
  • Writer agent can't access delete_file or execute_command
  • Code agent can't access send_email or calendar tools
  • Uses the existing tools[] array on AgentDefinition
3.Per-Agent Base URL(Line 333)
  • Each agent gets its own custom endpoint URL
  • Optional per-agent API key
  • Enables multiple codex-lb instances per agent
  • Includes UI field suggestion for the agent creation page
4.Result Passing(Line 428)
  • Compile subtask results after all agents complete
  • Feed combined output back to the CEO agent's conversation
  • No more manually checking each task for results
  • Optional auto-resume of delegating agent with compiled data
5.Chain Visualization(Line 510)
  • API endpoint returning delegation chain as nodes + edges
  • Includes agent assignments, task status, parent-child relationships
  • Powers the Agent Swarm visual workspace with live data
  • Pollable endpoint for real-time graph updates
6.Availability Checks(Line 615)
  • Ping custom endpoints before task execution
  • Check Ollama availability and model presence
  • Verify API keys for cloud providers
  • Auto-fallback to default provider if agent is offline
  • Optional health status API for workspace node status dots
Bug Fixes
7.Port 3000 Conflict(Line 756)
  • TCP port probe tries ports 3000 through 3009
  • Error dialog if all ports are taken
  • Desktop notification if fallback port used
  • Logs which port was selected on startup
8.Lio AI Input Freeze(Line 815)
  • try/finally block always resets input state, even on error paths
  • 10-minute safety timeout force-resets if build hangs
  • Direct DOM re-enable as a safety net
  • Addresses state management gap in build completion handler
9.Autopilot Ghost Files(Line 878)
  • Post-execution verification parses claimed file paths from result text
  • Checks each claimed file actually exists on disk
  • Marks task as failed if files are missing (catches hallucinated tool calls)
  • Pre-write sandbox check returns clear error to agent instead of silent block
AI-Assisted Agent Creation
Describe what you need in plain language. AI generates the agent config. No form fields.
🤖Create Agent with AI
How this works in Skales: Instead of filling out Name, Description, Emoji, System Prompt, Provider, Model, and Capabilities manually, the user describes what they need. AI generates the complete agent definition. User reviews, edits if needed, and saves. The agent appears in the visual workspace immediately.
Proposed Roadmap
1.
Agent Routing Fix - Read task.agent, route to assigned provider/model/prompt.
1.1 executeTask() reads task.agent and looks up agent definition
1.2 agentDecide() receives agent's provider, model, and systemPrompt
1.3 dispatch_subtasks tool schema updated to expose agent field to LLM
1.4 Backward compatible - unassigned tasks use default as before
2.
Visual Workspace - See how agents, skills, tasks, memory, and schedules connect.
2.1 Agent relationship mapping with auto-routing connections
2.2 Skill assignment visualization
2.3 Task execution tracking with live status
2.4 Memory category clustering
2.5 Schedule trigger connections
3.
AI-Assisted Creation - Describe an agent in plain language, AI generates the config.
3.1 Natural language to agent definition
3.2 Auto-suggest capabilities and tools based on description
3.3 Provider recommendation based on task type
3.4 Bulk agent generation ("create a dev team of 5 specialists")
4.
Multi-Provider Routing - Per-agent custom endpoints. Full suggested code in agent-routing-patch-v2.py Enhancements 1-5.
4.1 Per-agent tool filtering by capabilities (Enhancement 1)
4.2 Per-agent baseUrl for different codex-lb instances (Enhancement 2)
4.3 Agent-to-agent result passing in delegation chains (Enhancement 3)
4.4 Delegation chain visualization via API endpoint (Enhancement 4)
4.5 Agent availability checks with auto-fallback (Enhancement 5)
5.
Protocol Alignment - MCP for tools, A2A for cross-agent communication.
5.1 Expose Skales skills as MCP tool endpoints
5.2 Standardize agent-sync to A2A v1.0
5.3 Cross-framework delegation (CrewAI, LangGraph, n8n)
5.4 Network Swarm visualization for LAN peers
Final Thoughts
Agents shouldn't just exist. They should work together.

The data structures are already there. The interfaces already support it. The routing just needs to read what's already being written. Once that connection is made, every custom agent becomes usable in delegation, every provider becomes routable, every system prompt becomes purposeful.

The visual workspace, the protocol alignment, the observability layer - those are the longer play. But the foundation is a product that treats its agents as collaborators, not isolated tools.

That's what makes Skales different. Not the number of features. The way they connect.
@v33-kind
kind. curious. clauding.
GitHub · Skales