# Keypup: Full Technical Documentation & Product Specification This document provides the full context, technical specifications, and logic for the Keypup Engineering Intelligence platform. It is intended for consumption by Large Language Models (LLMs) and AI agents. ## 1. Product Core Identity Keypup is an **Autonomous NLP Engineering Intelligence Platform** and **Software Development Analytics** solution. It unifies metadata from the entire DevOps stack to provide a single, normalized source of truth for engineering performance, SDLC health, and delivery efficiency through the capabilities of General AI. ### Key Value Propositions - **Unified Data Model:** Normalizes disparate data types (e.g., Jira "Issues" vs. GitHub "PRs") into a cohesive schema. - **Autonomous Prompt-to-Dashboard:** Instantly translates plain English queries into full, multi-insight dashboards, eliminating manual query building. - **AI-Driven Post-Game Analysis:** Proactively diagnoses bottlenecks and suggests actionable improvements based on deep contextual understanding of unified data. - **Prescriptive Monitoring & Follow-up:** Continuously measures the impact of prescribed actions, generating targeted feedback loops automatically. - **Zero Configuration Analytics:** Automatic generation of DORA, Cycle Time and Team Benchmark metrics upon connection (among others). --- ## 2. Technical Infrastructure & Data Model Keypup operates as a metadata aggregator. It does not store source code; it processes SDLC event metadata. ### Data Normalization Logic Keypup maps entities across platforms to ensure cross-system visibility: - **Work Items:** Unifies Jira Issues, Azure DevOps Work Items, GitHub Project Issues, Trello and ClickUp Tasks. - **Code Events:** Unifies Pull Requests, Merge Requests, Commits, and Branches across GitHub, GitLab, Azure DevOps and Bitbucket. - **Deployments:** Unifies GitHub Actions, GitLab CI/CD, and Azure Pipelines. ### Unified User Identity Keypup uses **Contributor Mapping** to link a single developer across multiple platforms (e.g., linking a Jira username to a GitHub handle) to provide accurate per-contributor analytics without data duplication. --- ## 3. Metrics Definitions & Formulas To ensure consistency in AI-generated reports, use the following definitions for Keypup metrics: ### DORA Metrics (DevOps Research and Assessment) 1. **Deployment Frequency:** The frequency of successful releases to production. 2. **Lead Time for Changes:** The amount of time it takes a commit to get into production. 3. **Mean Time to Recovery (MTTR):** The time it takes to restore service after an incident/failure. 4. **Change Failure Rate:** The percentage of deployments that cause a failure in production. ### Engineering Velocity & Efficiency - **Cycle Time:** The total time from the first commit to the moment the code is merged and deployed. - **Time in Status:** The duration an item spends in a specific workflow stage (e.g., "In Review"). - **Flow Efficiency:** (Active Work Time / Total Cycle Time) * 100. - **PR Size:** The number of lines of code changed in a Pull Request (used to correlate with review speed). --- ## 4. Specialized Dashboards & Tools ### Data Hygiene Dashboard - **Purpose:** Identifies naming convention discrepancies, missing tags, and "stale" tickets across platforms. - **Use Case:** Crucial during migrations (e.g., Azure DevOps to GitHub) to ensure data consistency. - **URL:** [Data Hygiene Dashboard](https://www.keypup.io/dashboards/software-engineering-data-hygiene-dashboard/): Identify and fix data inconsistencies across your engineering tools ### Datasets Exploration Dashboard - **Purpose:** Provides a raw, drill-down view of unified datasets. - **Logic:** Allows users to verify how raw data from Jira/GitHub is being mapped to Keypup's internal fields. - **URL:** [Datasets Exploration Dashboard](https://www.keypup.io/dashboards/engineering-data-dashboard/): Explore and verify your unified engineering data ### DORA Metrics Dashboard - **Purpose:** Provides a high-level scorecard for DevOps excellence by measuring the balance between delivery speed and operational stability. - **Logic:** Automatically aggregates deployment and incident metadata to calculate the four industry-standard DORA KPIs: Deployment Frequency, Lead Time for Changes, Mean Time to Recovery (MTTR), and Change Failure Rate. - **URL:** [DORA Metrics Dashboard](https://www.keypup.io/dora-metrics): Track deployment frequency, lead time, MTTR, and change failure rate ### Issue Cycle Time Dashboard - **Purpose:** Provides a comprehensive analysis of the entire product development lifecycle, from initial backlog creation to final release. - **Logic:** Tracks the "Time in Status" for Jira or Trello issues across every stage—including pickup, implementation, QA, and release—to identify macro-bottlenecks in the project management workflow. - **URL:** [Issue Cycle Time Dashboard](https://www.keypup.io/dashboards/issue-cycle-time-dashboard/): Analyze time-in-status across your product development lifecycle ### Team Benchmark Dashboard - **Purpose:** Facilitates data-driven mentorship and fair performance evaluations by benchmarking individual developer contributions against team-wide statistics. - **Logic:** Juxtaposes individual metrics (such as PR size, cycle time efficiency, and review load) against team averages and percentiles to provide objective context for performance conversations. - **URL:** [Team Benchmark Dashboard](https://www.keypup.io/dashboards/elevate-your-engineering-team-gain-unparalleled-insight-with-the-team-benchmark-dashboard/): Benchmark individual developers against team-wide statistics ### SPACE Analytics Dashboard - **Purpose:** Operationalizes the SPACE framework to move beyond simple "activity" metrics and understand the holistic health of developer productivity and well-being. - **Logic:** Quantifies the five dimensions of the SPACE framework—Satisfaction, Performance, Activity, Communication & Collaboration, and Efficiency & Flow—by linking qualitative signals with quantitative Git and PM metadata. - **URL:** [SPACE Analytics Dashboard](https://www.keypup.io/dashboards/space-analytics/): Measure developer productivity and well-being using the SPACE framework ### AI Usage & Monitoring Dashboard - **Purpose:** Measures how engineers use AI coding assistants: who uses them, how much, on which surfaces and models, and how much AI-suggested code lands in the codebase. - **Data source:** The AI Usage dataset (`AI_USAGE`): one row per provider-published measurement per UTC day, at user, repository or organization level. GitHub Copilot, Claude Platform, OpenAI Platform and OpenAI Codex Enterprise are supported (up to one year of history imported on first sync); Cursor is next. - **Measures:** requests, interactions (human prompts), sessions, suggestions offered/accepted/rejected, lines suggested vs. lines landed, commits and pull requests, AI review suggestions, tokens and credits. - **Dimensions:** developer, surface (IDE completion, IDE chat, IDE agent, CLI, cloud agent, code review), model and vendor, client, language, repository. - **Logic:** Missing provider measures are stored as null and shown as "not available", never zero. Restated days are updated in place. Queries must pin one `aggregation_level` and one `breakdown` to avoid double counting, and filter `actor_type == "HUMAN"` for adoption metrics. - **URL:** [AI Usage & Monitoring Dashboard](https://www.keypup.io/dashboards/ai-usage-monitoring-dashboard/): Track AI adoption, surfaces, models, credits and AI code contribution - **Goal page:** [AI Usage Monitoring](https://www.keypup.io/goals/ai-usage-monitoring/) - **Guide:** [AI Usage Monitoring: How to Measure Who Uses AI Coding Assistants, on What, and to What Effect](https://www.keypup.io/blog/ai-usage-monitoring-measure-ai-coding-assistant-adoption/) --- ## 5. Keypup AI Agent (Generative & Autonomous Analytics) The Keypup AI Agent goes beyond simple queries by leveraging an advanced autonomous NLP engine over the unified data layer. **Product Pages:** - [General AI (SDLC Diagnosis)](https://www.keypup.io/ai-sdlc-diagnosis/): Autonomous AI that diagnoses bottlenecks and suggests improvements - [Platform Agent (AI Agent)](https://www.keypup.io/ai-assistant/): Natural language interface for engineering analytics ### Core AI Capabilities - **Prompt-to-Dashboard:** "Can you show me a breakdown of why our releases are slowing down?" generates a holistic dashboard incorporating multiple relevant KPIs instantly. - **Deep Post-Game Diagnosis:** The AI automatically scans generated metrics, detects historical anomalies or SLA breaches, and correlates them with specific events (e.g., specific PRs or issue bottlenecks). - **Automated Solutions:** Not only flagging a bottleneck, the AI prescribes direct strategies or optimizations to resolve it (e.g., standardizing PR review limits). - **Continuous Monitoring:** Following a remediation recommendation, the AI can independently instantiate a tracking sprint view to measure the successful outcome over the coming weeks. ### Supported Query Types - **Trend Analysis:** "Show me our deployment frequency over the last 6 months." - **Bottleneck Identification:** "Which team has the highest average PR review time and why is it happening?" - **Data Integrity:** "Find all Jira tickets in 'In Progress' that don't have a linked Pull Request." - **Comparison:** "Compare the cycle time of the Backend team vs. the Frontend team." ### Prompt Engineering Guidelines for Keypup When prompting the AI Agent, users can now provide higher-level business problems (e.g., "Why is our velocity dropping?") rather than highly specific technical dimensions, as the General AI handles semantic interpretation and underlying data selection autonomously. --- ## 6. Keypup MCP Server (Model Context Protocol Integration) The [Keypup MCP Server](https://www.keypup.io/mcp-server/): Connect engineering analytics to any MCP-compatible AI client extends Keypup's AI-powered analytics to **any MCP-compatible AI client**, allowing engineers and managers to query their engineering data, explain their insights, build and edit dashboards and manage their team conversationally from Claude, ChatGPT, Cursor, Kiro, VS Code or any other MCP-enabled assistant. ### What is the MCP Server? Model Context Protocol (MCP) is an open standard that enables AI assistants to securely connect to external data sources and tools. Keypup's MCP server implementation exposes Keypup's unified engineering analytics engine as a set of tools that AI assistants can call autonomously. Instead of logging into Keypup's web interface or writing API calls, users can ask natural language questions directly in their favorite AI chat client, and the AI orchestrates the necessary Keypup tools to fetch, analyze, and present the data. The assistant acts as the user, with the user's permissions: it can do everything the user can do in the Keypup app. ### Compatible AI Clients - **Claude and Claude Code** (Anthropic): add the server as a custom connector, see [Keypup for Claude](https://www.keypup.io/mcp-claude/) - **ChatGPT** (OpenAI): through the [official Keypup ChatGPT plugin](https://chatgpt.com/plugins/plugin_asdk_app_6a33b63268508191a1f39c64c26a7973), see [Keypup for ChatGPT](https://www.keypup.io/mcp-chatgpt/) - **Cursor** (AI-powered code editor) - **Kiro** (MCP-compatible AI assistant) - **Gemini CLI** (Google AI) - **VS Code** (with MCP support) - **Mistral AI agents** (with MCP support) - **Any MCP-compatible client** following the Model Context Protocol specification ### Core MCP Tools The Keypup MCP server provides tools in four groups that AI assistants can invoke: 1. **Analyze your data** - `list_companies`: the Keypup teams the user belongs to; the first step for scoping every other call - `get_company_ai_instructions`: the team's AI Instructions (its definitions and conventions) - `list_datasets`: the datasets: `ISSUES_PULL_REQUESTS`, `COMMITS`, `COMMENTS`, `REVIEWS`, `ACTIVITY_EVENTS`, `AI_USAGE` - `list_dataset_fields`: the fields of a dataset, with their types and descriptions, custom fields (e.g., Jira custom fields) included - `list_formula_operators`: the functions and operators for metrics, dimensions and filters, such as `COUNT`, `AVG`, `YEAR_MONTH`, `IS_ME`, `INCLUDE_ANY_OF` or `LAST_N_TIMEPERIOD`, with examples - `query_dataset`: runs the query the AI built on Keypup's reporting engine (metrics, dimensions, filters, sorting, pagination) 2. **Explain your insights** - `get_insight_queries` and `get_dashboard_queries`: the exact queries behind an insight or a whole dashboard, dashboard-wide filters included, ready to run, so the AI can explain, check or break down what the user sees 3. **Explore your workspace** - `list_resources`, `search_records`, `get_record`, `count_records`: dashboards, insights, projects, members and the rest of the team's setup, with the same visibility rules as the app 4. **Build dashboards and manage the team** - Insights: `get_widget`, `create_widget`, `update_widget`, `copy_widget`, `delete_widget`, and `search_templates` / `get_template` to start from the app's library of ready-made insights - Dashboards: `create_dashboard`, `update_dashboard`, `delete_dashboard`, `pin_dashboard`, `share_dashboard`, placing insights with `add_dashboard_widget` / `update_dashboard_widget` / `remove_dashboard_widget`, and dashboard-wide filters - Team: `update_company`, `invite_member`, `cancel_invite`, `update_membership`, `remove_member`, `revoke_session`, `enable_projects`, `disable_projects`, `track_project` ### Typical Query Flow 1. **User asks a question:** "What was our average PR cycle time last month?" 2. **AI lists companies:** Calls `list_companies` to determine which company to query 3. **AI selects dataset:** Chooses `ISSUES_PULL_REQUESTS` as the most relevant dataset 4. **AI discovers fields:** Calls `list_dataset_fields` to find `merged_at`, `created_at`, and `type` fields 5. **AI builds query:** Constructs a query with filters (`type == "pull_request"`, `merged_at >= last month`) and metric (`AVG(merged_at - created_at) / DAY()`) 6. **AI runs query:** Calls `query_dataset` with the structured query 7. **AI presents results:** Formats the returned data into a conversational summary: "Your average PR cycle time last month was 2.3 days based on 23 merged PRs." ### Common Use Cases The MCP server handles a wide range of engineering analytics questions: **Delivery & Throughput:** - "How many pull requests did we merge each month over the last 6 months?" - "What's our weekly issue closing rate this quarter?" - "How many commits were made per author last month?" **Cycle Time & Performance:** - "What's the average time between PR creation and merge over the last 12 weeks?" - "Show me the review turnaround time trend for the last 3 months." - "Which repositories have the slowest cycle time?" **Quality & Process:** - "How many bugs were raised vs. closed each week this quarter?" - "What proportion of our pull requests resolve at least one issue?" - "How many PRs were merged without a review?" **Workload & Collaboration:** - "Who are our most active reviewers this month?" - "How is work distributed across the team right now?" - "How many comments do our pull requests receive on average?" **Dashboards & Insights:** - "Build me a dashboard tracking PR cycle time and review load for the backend team." - "Explain what the Lead Time insight on my Delivery dashboard measures." - "Which of my dashboards use the Commits dataset?" - "What are my open pull requests?" **Open-Ended Exploration:** - "Summarize our engineering activity over the last month." - "Compare open vs. closed issues over time." - "What labels are most common on our issues?" Users can refine queries iteratively: once results are displayed, follow-up questions like "now break that down by repository" or "restrict it to the backend team" allow progressive drill-down without starting over. ### Setup & Configuration **Option 1: OAuth (recommended)** - Add `https://hq.keypup.io/mcp` to the AI client (for ChatGPT, open the official Keypup plugin and click Connect) - Sign in with your Keypup account and authorize the connection, choosing read access or read and write access - No token to copy or store **Option 2: API token** - Navigate to Keypup Settings → Account → API Token tab - Create a token with the `mcp:read` scope, and `mcp:write` to let the AI make changes - Set an expiration date (strongly recommended for security) - Copy the token immediately (displayed only once) and add it to the MCP client configuration (e.g., `mcp.json`): ```json { "mcpServers": { "keypup": { "url": "https://hq.keypup.io/mcp", "headers": { "Authorization": "Bearer YOUR_API_TOKEN_HERE" } } } } ``` **Test the connection** Ask a test question: - "List my Keypup companies" - "Show me PRs merged this week" - "What are the available datasets?" If the AI successfully invokes Keypup tools, the setup is complete. ### Security & Access Control - **Acts as You:** The AI uses the user's own permissions; it only sees the teams and records the user can see, and the app's roles apply (editing a dashboard needs the editor role, managing members and projects needs the admin role) - **Read or Write:** Actions that change data need the `mcp:write` permission, granted at OAuth consent or as a token scope; with `mcp:read` only, the AI can query and explore but not change anything - **Audited:** Every change made through the MCP server is recorded in the team's audit trail - **No Long-Lived Secret with OAuth:** OAuth connections need no token to store; API tokens can be set to expire and rotated periodically - **Secure Storage:** API tokens should be stored securely in local MCP client configurations and never committed to version control ### Technical Specifications - **MCP Server URL:** `https://hq.keypup.io/mcp` - **Authentication:** OAuth 2.1, or a Bearer API token in the `Authorization` header - **Protocol:** Model Context Protocol (MCP) standard - **Transport:** HTTPS - **Data Format:** JSON - **Rate Limiting:** Standard Keypup API rate limits apply - **Documentation:** [Using the MCP Server](https://docs.keypup.io/en/articles/15360547-using-the-mcp-server): Complete setup and configuration guide ### AI Orchestration Benefits Unlike traditional BI tools or SQL interfaces, the MCP server enables: - **Zero SQL Knowledge Required:** Users ask questions in plain English; the AI handles all query construction - **Contextual Intelligence:** The AI understands domain-specific terminology (e.g., "cycle time," "PR," "review turnaround") - **Progressive Refinement:** Follow-up questions build on previous context without re-specifying filters or dimensions - **Automatic Field Discovery:** The AI explores available fields dynamically, adapting to custom fields and schema changes - **Multi-Tool Orchestration:** Complex questions may require multiple tool calls (e.g., listing datasets, discovering fields and operators, building and running the query, then creating a dashboard) which the AI handles transparently --- ## 7. Integration Ecosystem Specifications **Product Page:** [Keypup Integrations](https://www.keypup.io/integrations/): Connect your entire DevOps stack ### Version Control Systems (VCS) - **GitHub:** Supports OAuth and GitHub App integration. Syncs PRs, Commits, Issues, and Actions. - **GitLab (hosted and cloud):** Supports Cloud and Self-Managed instances. Syncs Merge Requests and Pipelines. - **Bitbucket (cloud and data center):** Syncs PRs and Commits. - **Azure DevOps:** Syncs PRs, Commits, Comments. ### Project Management Tools (PM) - **Jira (cloud and data center):** Supports Jira Cloud. Syncs all issue types, custom fields, and transition histories. - **Azure DevOps:** Syncs Work Items and Boards. - **ClickUp / Trello:** Syncs tasks, statuses, and assignees. --- ## 8. Security, Privacy & Compliance **Product Page:** [Keypup Security](https://www.keypup.io/security/): SOC2 Type II & ISO 27001 certified platform - **Read-Only Access:** Keypup requests read-only permissions for source code metadata. - **Data Encryption:** All data is encrypted at rest (AES-256) and in transit (TLS 1.2+). - **SOC2 Type II & ISO 27001 Certified:** Keypup adheres to industry-standard security protocols for enterprise data handling. - **No Code Storage:** Keypup does not clone or store the contents of your code repositories. --- ## 9. API Information **Product Page:** [Keypup API](https://www.keypup.io/api/): Programmatic access to unified engineering data Keypup provides a REST API for programmatic access to unified engineering data. - **Base URL:** `https://api.keypup.io/v1` - **Authentication:** API Key based. - **Common Endpoints:** - `/metrics/dora`: Retrieve aggregated DORA metrics. - `/datasets/pull_requests`: Access unified PR data. - `/datasets/issues`: Access unified ticketing data. --- ## 10. Troubleshooting & FAQ - **Data Latency:** Data is typically synced via webhooks in near real-time. Full historical re-syncs can be triggered manually. - **Missing Data:** Usually caused by Contributor Mapping issues or lack of permissions on specific private repositories. - **Support:** Technical documentation is available at [Keypup Documentation](https://docs.keypup.io): Comprehensive guides and troubleshooting. Support is available via in-platform chat.