Building a Continuous AI Workflow with PostHog and GitHub

Build an automated system that continuously monitors PostHog session recordings, analyzes UX issues using Continue CLI, and creates GitHub issues automatically.

What You'll Build

A fully automated workflow that uses Continue CLI to fetch PostHog session data, analyze user experience issues with AI, and automatically create GitHub issues with specific technical recommendations

Prerequisites

Before starting, ensure you have:

• GitHub repository where you want to create issues
• PostHog account with session recordings enabled and collecting data
• Node.js 18+ installed locally
• Continue CLI account with active credits (required for API usage)

1. Install Continue CLI

   npm i -g @continuedev/cli
   

2. Set up Continue CLI Account & API Key

   1. Visit Continue Organizations
   2. Sign up or log in to your Continue account
   3. Navigate to your organization settings
   4. Click “API Keys” and then ”+ New API Key”
   5. Copy the API key immediately (you won’t see it again!)
   6. Login to the CLI: cn login
3. Add Required Secrets to Continue CLI

   Continue CLI will securely store your API keys as secrets that can be referenced in prompts.

Continue CLI handles the complex API interactions - you just need to provide the right prompts!

Step 1: Set Up Your Credentials

First, you’ll need to gather your PostHog and GitHub API credentials and add them as secrets in Continue CLI.

PostHog API Credentials

You’ll need a Personal API Key (not a Project API key) to access session recordings:

1. Go to Personal API Keys in PostHog
2. Click + Create a personal API Key
3. Name it “Continue CLI Session Analysis”
4. Select these scopes:
   • session_recording:read - Required for accessing session data
   • insight:read
   • query:read
   • session_recording_playlist:read
5. Copy the key immediately (you won’t see it again!)
6. Note your Project ID from your PostHog project settings
7. Note your PostHog host URL (e.g., https://us.posthog.com or your custom domain)
8. You’ll also need your POSTHOG_AUTH_HEADER value, which is simply Bearer YOUR_API_KEY

Find Your Project Details: Your Project ID is in your PostHog URL: https://app.posthog.com/project/YOUR_PROJECT_ID

Add Secrets to Continue CLI

To add environment variables to your Continue Hub account:

1. Go to the Continue Hub
2. Sign in to your account
3. Navigate to your user settings
4. Look for the “Secrets” section
5. Add your variables there (like POSTHOG_API_KEY, POSTHOG_PROJECT_ID, POSTHOG_HOST,GITHUB_PAT, etc.)

These environment variables will then be securely stored and available for use in your Continue workflows.

PostHog GitHub Continuous AI Workflow Options

🚀 Fastest Path to Success

Skip the manual setup and use our pre-built PostHog GitHub agent that includes optimized prompts, rules, and the PostHog MCP for more consistent results.

After completing Step 1 below, you have two paths to get started:

⚡ Quick Start (Recommended)

Perfect for: Immediate results with optimized prompts and built-in debugging

1. Add the Pre-Built Agent

   Visit the PostHog GitHub Continuous AI Agent on Continue Hub and click “Install Agent”

   This agent includes:

   • Optimized prompts for PostHog analysis and GitHub issue creation
   • Built-in rules for consistent formatting and error handling
   • PostHog MCP for more reliable API interactions
2. Verify Your Secrets

   The agent uses these Hub resources automatically:

   • Environment Variables: bekahhw/env-vars
   • Analysis Rules: bekah-hawrot-weigel/posthog-github-continuous-ai-rules
   • Analysis Prompt: continuedev/posthog-analysis
   • Issue Creation Prompt: continuedev/github-issues
3. Run the Analysis

   Navigate to your project directory and run:

   cn "Give me my PostHog Session data and create GitHub issues based on the problems."
   

   That’s it! The agent handles everything automatically.

Why Use the Agent? Results are more consistent and debugging is easier thanks to the PostHog MCP integration and pre-tested prompts.

Agent Requirements

To use the pre-built agent, you need either:

• Continue CLI Pro Plan with the models add-on, OR
• Your own API keys added to Continue Hub secrets (same as Step 1 below) The agent will automatically detect and use your configuration.

---

Repository Labels Required: Make sure your GitHub repository has these labels:

• bug, enhancement
• high-priority, medium-priority, low-priority
• user-experience, automated

Create missing labels in your repo at: Settings → Labels → New label

What Continue CLI Does: - Parses your analysis results automatically - Makes authenticated GitHub API calls using your stored token - Creates properly formatted issues with appropriate labels - Checks for duplicate issues to avoid spam - Provides confirmation with issue URLs

Setting Up Automated GitHub Actions Workflow

Create a GitHub Actions workflow that runs your Continue CLI analysis automatically on a schedule.

Add GitHub Secrets

First, add these secrets to your GitHub repository:

1. Go to Repository Settings → Secrets and variables → Actions
2. Click “New repository secret” and add:
   • CONTINUE_API_KEY: Your Continue API key from hub.continue.dev

Create GitHub Actions Workflow

Create .github/workflows/posthog-analysis.yml in your repository:

⚡ Using Pre-Built Agent (Recommended)

name: PostHog Session Analysis with Continue CLI Agent

on:
schedule: - cron: "0 6 \* \* \*" # Run at 6 AM UTC daily
workflow_dispatch: # Allow manual triggering

jobs:
analyze:
runs-on: ubuntu-latest

    permissions:
      contents: read
      issues: write

    steps:
      - uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "18"

      - name: Install Continue CLI
        run: |
          npm install -g @continuedev/cli
          echo "✅ Continue CLI installed"
          cn --version

      - name: Authenticate with Continue CLI
        run: |
          # Set Continue API key
          export CONTINUE_API_KEY="${{ secrets.CONTINUE_API_KEY }}"
          # Login to Continue CLI
          if ! cn auth login --api-key "$CONTINUE_API_KEY"; then
            echo "❌ Failed to authenticate with Continue CLI"
            exit 1
          fi
          echo "✅ Continue CLI authenticated"

      - name: Load PostHog GitHub Agent
        run: |
          echo "🤖 Loading PostHog GitHub Continuous AI Agent..."
          cn --config continuedev/awesome-models-posthog-gh
          echo "✅ Agent loaded successfully"

      - name: Run PostHog Analysis and Create GitHub Issues
        run: |
          echo "🎬 Starting automated PostHog session analysis..."

          # Run the agent with the simple prompt
          cn -p "Tell me about my PostHog session recordings and create GitHub issues based on the problems."

          echo "✅ Analysis and issue creation completed"

      - name: Summary
        run: |
          echo "✅ PostHog analysis workflow completed!"
          echo "📊 Check the artifacts for detailed analysis results"
          echo "🐛 Check your repository issues for newly created UX issues"
          echo "🤖 Agent automatically handled both analysis and issue creation"

Test the GitHub Actions Workflow

1. Manual Trigger: Go to Actions tab in your repository and manually trigger the workflow
2. Check Artifacts: Download the analysis results from the workflow run
3. Verify Issues: Check that GitHub issues were created in your repository

Success Indicators:

• PostHog API returns session data (not empty)
• Continue CLI generates analysis with identified issues
• GitHub issues are created with proper labels and formatting
• Workflow completes without errors

What You’ve Built

After completing this guide, you have a complete Continuous AI system that:

• Monitors user experience - Automatically fetches and analyzes PostHog session data

• Identifies problems intelligently - Uses AI to spot patterns and technical issues

• Creates actionable tasks - Generates GitHub issues with specific recommendations

• Runs autonomously - Operates daily without manual intervention using GitHub Actions

• Scales with your team - Handles growing amounts of session data automatically

Continuous AI

Your system now operates at Level 2 Continuous AI - AI handles routine analysis tasks with human oversight through GitHub issue review and prioritization.

Security Best Practices

Protect Your API Keys:

• Store all credentials as GitHub Secrets, never in code
• Use Continue CLI’s secure secret storage
• Limit token scopes to minimum required permissions
• Rotate API keys regularly (every 90 days recommended)
• Monitor token usage for unusual activity

Advanced Prompts

Consider enhancing your workflow with these advanced Continue CLI prompts:

Performance Analysis

“Analyze PostHog performance metrics alongside session recordings to identify slow page loads affecting user experience”

Error Correlation

“Cross-reference JavaScript console errors with user actions to identify the root cause of UX issues”

Feature Flag Analysis

“Analyze how PostHog feature flags impact user behavior in session recordings”

Custom Slack Alerts

“Create Slack alerts when critical UX issues are detected in PostHog sessions”

Next Steps

1. Monitor your first issues - Review the GitHub issues created and implement fixes
2. Measure impact - Track how resolved issues improve PostHog metrics
3. Refine prompts - Adjust the Continue CLI prompts based on issue quality
4. Scale the system - Add more data sources or create specialized analysis workflows