Desktop Automation: Screen Control & App Management

The desktop tools provide system-level interaction: capturing screenshots and monitoring system resources.

Tools

take_screenshot

Captures the primary monitor and saves the image.

User: Take a screenshot of my screen
Agent: [uses take_screenshot] → Screenshot saved to ~/.pocketpaw/file_jail/screenshots/screenshot_20240115_143022.png

Screenshots are saved to ~/.pocketpaw/file_jail/screenshots/ with timestamped filenames.

get_status

Returns current system metrics - CPU usage, RAM, disk space, and battery status.

User: How's my system doing?
Agent: [uses get_status] → CPU: 23%, RAM: 8.2/16 GB (51%), Disk: 234/512 GB (46%), Battery: 78% (charging)

Configuration

The desktop tools require the desktop optional dependency:

curl -fsSL https://pocketpaw.xyz/install.sh | sh

# Or add the desktop extra manually
pip install pocketpaw[desktop]

Policy Group

Desktop tools belong to group:desktop. They are included in the full tool profile.

# Enable desktop tools
export POCKETPAW_TOOLS_ALLOW="group:desktop"

Tool Schema

{
  "name": "take_screenshot",
  "description": "Take a screenshot of the user's primary monitor",
  "input_schema": {
    "type": "object",
    "properties": {},
    "required": []
  }
}

{
  "name": "get_status",
  "description": "Get current system status (CPU, RAM, Disk, Battery)",
  "input_schema": {
    "type": "object",
    "properties": {},
    "required": []
  }
}