Skip to content

Generating Random Data

To get an idea of what Howler looks like with data, and to test your code, you can use the helper script api/howler/odm/random_data.py. This script contains methods for generating realistic test data for all models used in Howler.

Basic Usage

cd ~/repos/howler/api

# Run without arguments - all indexes are wiped and populated with test data
python howler/odm/random_data.py

# Populate all indexes without wiping existing data
python howler/odm/random_data.py all --no-wipe

# Populate specific indexes only
python howler/odm/random_data.py users hits analytics

# Populate specific indexes without wiping
python howler/odm/random_data.py users hits --no-wipe

Available Indexes

The following indexes can be populated with test data:

  • users - User accounts with various permission levels and API keys
  • templates - Hit display templates for different analytics and detections
  • overviews - Markdown-based overview templates with Handlebars support
  • views - Saved search queries and filters
  • hits - Alert/detection data including bundles
  • analytics - Analytic definitions with rules and triage settings
  • actions - Automated action configurations
  • dossiers - Investigation case files

Generated Test Data

Users

The script creates several predefined users for testing different scenarios:

Username Password Type Description
admin admin (or $DEV_ADMIN_PASS) admin, user, automation_basic, automation_advanced, actionrunner_basic, actionrunner_advanced Full admin access with multiple API keys
user user (or $DEV_USER_PASS) user, actionrunner_basic Standard user with impersonation keys
shawn-h shawn-h admin, user Admin user for testing
goose goose admin, user Admin user for testing
huey huey user, actionrunner_basic Standard user for testing
Custom Passwords

You can customize the admin and user passwords by setting environment variables:

export DEV_ADMIN_PASS="my_secure_password"
export DEV_USER_PASS="my_user_password"
python howler/odm/random_data.py users

Hits

By default, the script generates 200 random hits with realistic data including:

  • Various detection types and analytics
  • Different status levels (open, in-progress, resolved)
  • Random assignments to users
  • Assessment data (escalations and scrutiny levels)
  • Event categories and metadata

The script also creates bundles - groups of related hits linked together.

Analytics

The script generates analytics in several categories:

  • Existing analytics from hits - Analytics are automatically created from generated hits
  • Random analytics (10 by default) - Fully randomized analytic definitions
  • Rule-based analytics - Analytics with Lucene, EQL, and Sigma rules

Each analytic includes:

  • Detections
  • Comments (both analytic-level and detection-level)
  • Notebooks (if enabled in configuration)
  • Triage settings with valid assessments
  • Contributors and owners
Sigma Rules

For better test data using Sigma rules, run the Sigma rule generator first:

python howler/external/generate_sigma_rules.py

Templates

Templates are generated for different analytics and detections, including:

  • Global templates - Available to all users
  • Personal templates - User-specific templates

Each template defines which hit fields should be displayed and in what order.

Overviews

Overview templates use Handlebars syntax to create dynamic markdown-based views of hits. The generated overviews include examples of:

  • Conditional rendering based on hit status
  • Fetching external data via API calls
  • Displaying user avatars
  • Status badges and formatting

Actions

Random automated actions are generated with various operations such as:

  • Prioritization adjustments
  • Status transitions
  • Field updates
  • Bulk operations

Each action includes a query to match hits and a series of operations to perform.

Views

Saved views are created including:

  • Global views - Shared queries for common use cases
  • Personal views - User-specific saved searches
  • Readonly views - Pre-configured filters (e.g., "Assigned to me")

Environment Variables

The script respects several environment variables:

  • DEV_ADMIN_PASS - Password for the admin user (default: admin)
  • DEV_USER_PASS - Password for the user account (default: user)
  • HWL_PLUGIN_DIRECTORY - Location of Howler plugins (default: /etc/howler/plugins)
  • ELASTIC_HIT_SHARDS - Number of shards for hits index (set to 1 during setup)
  • ELASTIC_HIT_REPLICAS - Number of replicas for hits index (set to 1 during setup)
  • ELASTIC_USER_REPLICAS - Number of replicas for user index (set to 1 during setup)
  • ELASTIC_USER_AVATAR_REPLICAS - Number of replicas for user avatar index (set to 1 during setup)

Plugin Integration

The random data generator supports plugins through the run_modifications function. If you have custom plugins that extend Howler's data models, they will automatically be invoked during data generation to populate plugin-specific fields.

Use Cases

Development Environment Setup

Quickly populate a fresh Howler instance with realistic test data:

python howler/odm/random_data.py all

Testing Specific Features

Populate only the data needed for your feature:

# Testing user permissions
python howler/odm/random_data.py users --no-wipe

# Testing hit processing
python howler/odm/random_data.py hits analytics --no-wipe

# Testing actions
python howler/odm/random_data.py hits actions --no-wipe

Continuous Integration

Use the --no-wipe flag to add test data without destroying existing data during test runs.