Events API

---
order: 1
title: Events API
description: Trigger, acknowledge, and resolve incidents programmatically via the Events API
---

# Events API

The Events API allows external monitoring systems to create and manage incidents in OpsKnight programmatically.

---

## Endpoint

```
POST /api/events
```

---

## Authentication

### Integration Key (Recommended)

Use an integration key for service-specific event routing:

```http
Authorization: Token token=YOUR_INTEGRATION_KEY
```

Integration keys are created per-service and automatically route events to the correct service.

### API Key

Alternatively, use an API key with `events:write` scope:

```http
Authorization: Bearer YOUR_API_KEY
```

When using an API key, you must include `service_id` in the request body.

---

## Request Format

### Content Type

```http
Content-Type: application/json
```

### Request Body

```json
{
  "event_action": "trigger",
  "dedup_key": "unique-alert-identifier",
  "service_id": "svc_abc123",
  "payload": {
    "summary": "Brief description of the issue",
    "source": "monitoring-system",
    "severity": "critical",
    "custom_details": {
      "metric": "cpu_usage",
      "value": 95.5
    }
  }
}
```

---

## Fields Reference

| Field | Type | Required | Description |
| ----- | ---- | :------: | ----------- |
| `event_action` | string | Yes | `trigger`, `acknowledge`, or `resolve` |
| `dedup_key` | string | Yes | Unique identifier for deduplication (max 200 chars) |
| `service_id` | string | Conditional | Required when using API key authentication |
| `payload` | object | Yes | Event details |

### Payload Fields

| Field | Type | Required | Description |
| ----- | ---- | :------: | ----------- |
| `summary` | string | Yes | Brief description (max 500 chars) |
| `source` | string | Yes | Source system identifier (max 200 chars) |
| `severity` | string | Yes | `critical`, `error`, `warning`, or `info` |
| `custom_details` | object | No | Arbitrary key-value pairs for additional context |

### Severity to Urgency Mapping

| Severity | OpsKnight Urgency |
| -------- | ----------------- |
| `critical` | HIGH |
| `error` | MEDIUM |
| `warning` | LOW |
| `info` | LOW |

---

## Event Actions

### trigger

Creates a new incident or updates an existing one based on the dedup_key.

**Behavior**:
- If no open incident exists with the dedup_key: creates new incident
- If an open/acknowledged/snoozed/suppressed incident exists: adds alert to existing incident (deduplicated)
- Incident title is set from `payload.summary`
- Incident description is set from `payload.custom_details` (JSON formatted)

```json
{
  "event_action": "trigger",
  "dedup_key": "server-cpu-high-web01",
  "payload": {
    "summary": "CPU usage above 90% on web-01",
    "source": "prometheus",
    "severity": "critical",
    "custom_details": {
      "cpu_percent": 95.5,
      "host": "web-01"
    }
  }
}
```

### acknowledge

Marks an existing incident as acknowledged.

**Behavior**:
- Finds open incident with matching dedup_key and service
- Updates status to ACKNOWLEDGED
- Stops escalation (sets escalationStatus to COMPLETED)
- Records acknowledgedAt timestamp
- No effect if no matching incident exists

```json
{
  "event_action": "acknowledge",
  "dedup_key": "server-cpu-high-web01",
  "payload": {
    "summary": "Acknowledged via API",
    "source": "automation",
    "severity": "info"
  }
}
```

### resolve

Marks an incident as resolved.

**Behavior**:
- Finds open/acknowledged incident with matching dedup_key and service
- Updates status to RESOLVED
- Stops escalation
- Records resolvedAt timestamp
- Triggers Slack notification if configured
- No effect if no matching incident exists

```json
{
  "event_action": "resolve",
  "dedup_key": "server-cpu-high-web01",
  "payload": {
    "summary": "Issue resolved",
    "source": "prometheus",
    "severity": "info"
  }
}
```

---

## Response Format

### Success Response

```json
{
  "status": "success",
  "result": {
    "action": "triggered",
    "incident": {
      "id": "clxyz123",
      "title": "CPU usage above 90% on web-01",
      "status": "OPEN",
      "urgency": "HIGH"
    }
  }
}
```

**HTTP Status**: 202 Accepted

### Result Actions

| Action | Description |
| ------ | ----------- |
| `triggered` | New incident created |
| `deduplicated` | Alert added to existing incident |
| `acknowledged` | Incident acknowledged |
| `resolved` | Incident resolved |
| `ignored` | No matching incident found for acknowledge/resolve |

### Error Responses

| HTTP Status | Error | Description |
| ----------- | ----- | ----------- |
| 400 | Invalid JSON | Request body is not valid JSON |
| 400 | Invalid request body | Validation failed (missing/invalid fields) |
| 401 | Unauthorized | Missing or invalid API key |
| 403 | Invalid Integration Key | Integration key not found |
| 403 | API key missing scope | API key lacks `events:write` scope |
| 403 | Service access denied | User doesn't have access to the service |
| 404 | Service not found | service_id doesn't exist |
| 429 | Rate limit exceeded | Too many requests |
| 500 | Internal Server Error | Server error |

---

## Rate Limiting

| Limit | Value |
| ----- | ----- |
| Requests per minute | 120 |
| Window | 60 seconds |

Rate limits apply per integration key or API key.

### Rate Limit Response

```json
{
  "error": "Rate limit exceeded."
}
```

**HTTP Status**: 429 Too Many Requests

**Header**: `Retry-After: <seconds>`

---

## Deduplication

Deduplication prevents duplicate incidents from the same alert source.

### How It Works

Events with the same `dedup_key` and `service_id` are grouped:

```
Event 1: dedup_key="cpu-high" → New incident created
Event 2: dedup_key="cpu-high" → Alert added to same incident
Event 3: dedup_key="cpu-high" → Alert added to same incident
Event 4: dedup_key="disk-full" → New incident created
```

### Deduplication Scope

- Deduplication only applies to incidents with status: OPEN, ACKNOWLEDGED, SNOOZED, SUPPRESSED
- Resolved incidents are excluded from deduplication
- A new trigger event after resolution creates a new incident

### Dedup Key Best Practices

**Good dedup keys**:
```
server-cpu-high-web01           # host + metric + server
database-connection-error-prod  # service + issue + environment
api-latency-p99-exceeded        # service + specific metric
```

**Bad dedup keys**:
```
alert-123456                    # Random, not meaningful
cpu-high                        # Too generic
2024-01-15-alert               # Date-based, creates duplicates
```

---

## Examples

### cURL - Trigger Incident

```bash
curl -X POST https://your-opsknight.com/api/events \
  -H "Authorization: Token token=YOUR_INTEGRATION_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "event_action": "trigger",
    "dedup_key": "database-connection-timeout",
    "payload": {
      "summary": "Database connection timeout on primary",
      "source": "monitoring-agent",
      "severity": "critical",
      "custom_details": {
        "database": "postgres-primary",
        "timeout_ms": 30000
      }
    }
  }'
```

### cURL - Resolve Incident

```bash
curl -X POST https://your-opsknight.com/api/events \
  -H "Authorization: Token token=YOUR_INTEGRATION_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "event_action": "resolve",
    "dedup_key": "database-connection-timeout",
    "payload": {
      "summary": "Database connection restored",
      "source": "monitoring-agent",
      "severity": "info"
    }
  }'
```

### Python Example

```python
import requests

OPSKNIGHT_URL = "https://your-opsknight.com"
INTEGRATION_KEY = "your-integration-key"

def send_event(action, dedup_key, summary, source, severity, custom_details=None):
    url = f"{OPSKNIGHT_URL}/api/events"
    headers = {
        "Authorization": f"Token token={INTEGRATION_KEY}",
        "Content-Type": "application/json"
    }

    data = {
        "event_action": action,
        "dedup_key": dedup_key,
        "payload": {
            "summary": summary,
            "source": source,
            "severity": severity
        }
    }

    if custom_details:
        data["payload"]["custom_details"] = custom_details

    response = requests.post(url, headers=headers, json=data)
    return response.json()

# Trigger an incident
result = send_event(
    action="trigger",
    dedup_key="api-error-rate-high",
    summary="API error rate above 5%",
    source="prometheus",
    severity="error",
    custom_details={
        "error_rate": 7.5,
        "threshold": 5.0
    }
)
print(result)

# Resolve the incident
result = send_event(
    action="resolve",
    dedup_key="api-error-rate-high",
    summary="Error rate normalized",
    source="prometheus",
    severity="info"
)
print(result)
```

### Node.js Example

```javascript
const axios = require('axios');

const OPSKNIGHT_URL = 'https://your-opsknight.com';
const INTEGRATION_KEY = 'your-integration-key';

async function sendEvent(action, dedupKey, summary, source, severity, customDetails = null) {
  const response = await axios.post(
    `${OPSKNIGHT_URL}/api/events`,
    {
      event_action: action,
      dedup_key: dedupKey,
      payload: {
        summary,
        source,
        severity,
        ...(customDetails && { custom_details: customDetails })
      }
    },
    {
      headers: {
        'Authorization': `Token token=${INTEGRATION_KEY}`,
        'Content-Type': 'application/json'
      }
    }
  );
  return response.data;
}

// Trigger incident
sendEvent('trigger', 'memory-usage-critical', 'Memory usage above 95%', 'node-exporter', 'critical', {
  memory_percent: 97.5,
  host: 'app-server-01'
}).then(console.log);
```

---

## Integration Setup

### Creating an Integration Key

1. Go to **Services** → Select a service
2. Click **Integrations** tab
3. Click **Add Integration**
4. Enter a name (e.g., "Prometheus Alerts")
5. Copy the generated **Integration Key**

### Integration Key Format

Integration keys are 32-character hexadecimal strings, auto-generated and unique per integration.

---

## What Happens After Trigger

When a new incident is triggered:

1. **Alert Logged**: Raw alert data stored for reference
2. **Incident Created**: New incident with title from summary
3. **Escalation Started**: Escalation policy executed if configured
4. **Notifications Sent**:
   - Service-level notifications (Slack, webhooks) if escalation policy exists
   - User notifications if no escalation policy
5. **Status Page Updated**: Webhooks triggered for status page integrations

---

## Related Topics

- [Incidents API](./incidents) — Direct incident management
- [Integrations](../integrations/README) — Integration configuration
- [Services](../core-concepts/services) — Service setup