Skip to content

Functional Documentation: ActiveMQ (AMQ) Queue Metric Collector

1. Overview

This document describes the monitoramq script, an automated component responsible for collecting real-time metrics from the ActiveMQ (AMQ) message broker. Its primary function is to query the AMQ administration interface, extract critical queue statistics (pending messages, consumers, enqueued/dequeued counts), and log this data to a designated file. This collected data serves as the foundation for further analysis, particularly for the amqstucknotifications script which identifies and alerts on stuck queues.

2. Functional Description

The monitoramq script performs the following sequence of operations when executed:

  1. Configuration Loading:

    • Retrieves connection details for the ActiveMQ broker (URL, username, password) from the application's configuration.
    • Loads the base path for log files and a customer identifier.
    • Loads an optional threshold value for message counts, defaulting to 100 if not specified.

  2. ActiveMQ Connection & Data Retrieval:

    • Establishes an HTTP GET connection to the configured ActiveMQ broker URL (assumed to be the web administration or JMX console that exposes queue statistics as XML).
    • Authenticates using Basic Authentication with the provided username and password.
    • Sets connection and read timeouts to prevent indefinite hanging.
    • Parses the XML response received from the ActiveMQ broker to extract individual queue details.

  3. Queue Metric Extraction:

    • For each queue found in the ActiveMQ XML response, the script extracts the following metrics:
      • queueName: The name of the queue.
      • messageCount: The number of pending messages currently in the queue (@size).
      • consumerCount: The number of active consumers currently connected to the queue (@consumerCount).
      • enqueueCount: The total number of messages ever enqueued to this queue (@enqueueCount).
      • dequeueCount: The total number of messages ever dequeued from this queue (@dequeueCount).

  4. Logging Queue Information:

    • For each extracted queue, it formats the collected metrics into a structured log entry.
    • The log entry includes a timestamp, a unique topic identifier (e.g., customer/prod/amq/INT/queueName), and a JSON string containing the extracted metrics.
    • This log entry is appended to a daily log file named amq-monitor-YYYYMMDD.log within the configured logBase directory.

  5. Local Threshold Warning (Console Output Only):

    • If a queue's messageCount exceeds the configured threshold (default 100), the script prints a warning message to the standard output (console).
    • Note: This is a local console warning and does not trigger an email notification; it's intended for immediate operational awareness during manual execution or for visibility in a job runner's logs. Email notifications are handled by the separate amqstucknotifications script.

3. Configuration Parameters (Inputs)

The script relies on the following parameters defined within the application's configuration (grailsApplication.config.gaia path):

Parameter Path Type Description Default Value
activemq.brokerUrl String The URL to the ActiveMQ broker's administration interface or API endpoint that provides XML output of queue metrics (e.g., JMX/ActiveMQ Console). Required
activemq.username String Username for Basic Authentication against the brokerUrl. Required
activemq.password String Password for Basic Authentication against the brokerUrl. Required
activemq.threshold Integer If a queue's pending message count exceeds this value, a warning message is printed to the console. This is a local warning, not an external notification. 100
location.logs String The base directory where log files (amq-monitor-*.log, exceptions-*.log) will be stored. Required
customer String A unique identifier for the customer/environment, used in the log topic for categorization. Required

4. Outputs

A. Main Log File

  • File Name: amq-monitor-YYYYMMDD.log
  • Location: [grailsApplication.config.gaia.location.logs]/amq-monitor-YYYYMMDD.log
  • Purpose: Contains a chronological record of ActiveMQ queue metrics. This log file is the primary input for the amqstucknotifications script.
  • Format (one line per queue per execution): YYYY-MM-DD HH:mm:ss.SSS;customer/prod/amq/INT/QUEUENAME;{"pendingMessages":X,"consumers":Y,"messagesEnqueued":A,"messagesDequeued":B}
    • YYYY-MM-DD HH:mm:ss.SSS: Timestamp of the data collection.
    • customer/prod/amq/INT/QUEUENAME: A structured identifier for the queue.
    • {"pendingMessages":X,"consumers":Y,"messagesEnqueued":A,"messagesDequeued":B}: A JSON object containing the collected metrics for that specific queue.

B. Console Output (Standard Out)

  • Content:
    • Connected to ActiveMQ admin successfully.
    • Failed to connect to ActiveMQ admin. Response code: [code]
    • Error connecting to ActiveMQ admin: [error message]
    • Unable to fetch queue details.
    • Warning: High pending message count in '[queueName]' queue! (if messageCount > threshold)
  • Purpose: Provides immediate feedback during execution, useful for debugging and job runner visibility.

C. Exception Log File

  • File Name: exceptions-YYYYMMDD.log
  • Location: [grailsApplication.config.gaia.location.logs]/exceptions-YYYYMMDD.log
  • Purpose: Records any unhandled exceptions that occur during the script's execution.
  • Format: [scriptName] [timestamp] [exception details]

5. Dependencies

  • ActiveMQ Broker: A running ActiveMQ instance accessible via HTTP/HTTPS.
  • ActiveMQ Admin Interface: The ActiveMQ broker must have its administration interface (e.g., Web Console or JMX MBeans exposed via HTTP) enabled and configured to return queue statistics in an XML format.
  • Network Connectivity: The server executing this script must have network access to the brokerUrl.
  • Authentication: The provided username and password must have sufficient permissions to access queue metrics on the ActiveMQ broker.
  • Groovy Runtime: The script requires a Groovy execution environment.
  • Disk Space: Sufficient disk space in the logBase directory for storing daily log files.

6. Execution and Scheduling

This script is designed to be executed periodically by an external scheduler (e.g., Cron, Jenkins, an internal job orchestration system). Its typical use case is to run frequently (e.g., every 1-5 minutes) to provide a granular time-series data feed for ActiveMQ queue health.

7. Error Handling

  • Connection Errors: If the script fails to connect to ActiveMQ or receives a non-200 HTTP response, it prints an error message to the console and returns null from connectToActiveMQ.
  • XML Parsing Errors: If the response from ActiveMQ cannot be parsed as XML, XmlParser exceptions may occur, which would be caught by the main try-catch block.
  • General Exceptions: Any unhandled exceptions during the main run block are caught and logged to the exceptions-YYYYMMDD.log file, including the script name, timestamp, and exception details. This ensures that failures are recorded even if console output is not monitored.

8. Limitations & Considerations

  • ActiveMQ Admin API Specificity: The script relies on the specific XML structure returned by the ActiveMQ administration interface. Changes to this interface in future ActiveMQ versions might require script updates.
  • Log File Size: Frequent execution will lead to large amq-monitor-YYYYMMDD.log files. Log rotation or archiving strategies should be implemented externally.
  • Local Warning Only: The threshold for messageCount only triggers a local console message; it is not an alerting mechanism. All alerting is handled by downstream processes (amqstucknotifications).