Most engineers reach for the CloudWatch Metrics API and end up writing extra SDK calls that add latency and cost. Embedded Metrics Format (EMF) lets you ship rich, query‑able metrics by just writing specially‑formatted JSON logs. Learn how to turn your AI inference code into a self‑monitoring powerhouse with no extra overhead.
An LLM (large language model) inference endpoint can handle hundreds of requests per second. Each request has a latency, a token count, and sometimes an error. If you log only the raw request/response, you must later run expensive log‑scans to calculate averages, percentiles, or error rates. Adding a second “metrics” call (e.g., PutMetricData
) creates extra network hops, adds milliseconds to every inference, and inflates your AWS bill.
Embedded Metrics Format (EMF) is a way to embed metric data directly inside a CloudWatch Logs event. Think of a log line as a post‑it note that not only tells you what happened but also carries a tiny scoreboard of numbers. CloudWatch reads the scoreboard, extracts the numbers, and stores them as regular CloudWatch metrics—without any separate API call.
In plain English:Write one JSON log line, get both a log entryanda metric for free.
console.log
.PutMetricData
.An EMF log entry is a JSON object that contains a top‑level @aws
key. Inside @aws
you must provide:
Timestamp
– epoch milliseconds.CloudWatchMetrics
– an array describing each metric you’re publishing.Namespace
– a logical bucket (e.g., MyAIService
) that groups related metrics.All other top‑level keys become dimensions (labels) or measurements (numeric values) for the metric.
{
"@aws": {
"Timestamp": 1725067200000,
"CloudWatchMetrics": [
{
"Namespace": "MyAIService",
"MetricName": "InferenceLatency",
"Dimensions": [["ModelName", "Endpoint"]]
}
]
},
"ModelName": "gpt‑4‑mini",
"Endpoint": "text‑completion",
"InferenceLatency": 124,
"TokenCount": 57,
"Success": true
}
InferenceLatency
, TokenCount
, and Success
are the ModelName
and Endpoint
are the @aws
fields
If you forget Timestamp
or Namespace
, CloudWatch silently drops the EMF data. The log still appears in CloudWatch Logs, but no metric shows up.
Tip:Validate your JSON with a small unit test before shipping it to production.
CloudWatch only parses EMF if the log group has a subscription filter that forwards logs to the CloudWatch Metrics processing pipeline. Without it, the EMF line stays a plain log.
aws logs put-subscription-filter \
--log-group-name /aws/ecs/my-ai-service \
--filter-name EMFProcessor \
--filter-pattern "" \
--destination-arn arn:aws:logs:us-east-1:123456789012:destination:CloudWatchMetrics
In plain English:Think of the subscription filter as a “mailroom clerk” that reads each envelope (log) and extracts the scorecard (metric). If the clerk isn’t hired, the scorecard never leaves the envelope.
diagnostics_channel
for Automatic EMF Emission
diagnostics_channel
? Node.js 22 introduced the diagnostics_channel module as a low‑overhead way for libraries to publish structured data about runtime events. It works like a radio station: the service (your inference code) broadcasts a message, and any listener (the EMF formatter) can pick it up without altering the main execution path.
// diagnosticsChannel.ts
import { createChannel } from 'node:diagnostics_channel';
// Create a channel named "ai-inference". The name is arbitrary but must be consistent.
export const inferenceChannel = createChannel('ai-inference');
Whenever an inference finishes, we publish a payload to this channel. The payload will later be turned into an EMF log line.
// emfLogger.ts
import { inferenceChannel } from './diagnosticsChannel.js';
import { CloudWatchLogsClient, PutLogEventsCommand } from '@aws-sdk/client-cloudwatch-logs';
import { format } from 'util';
// Create a CloudWatch Logs client – we only need it once.
const cwLogs = new CloudWatchLogsClient({ region: 'us-east-1' });
// Helper to get the current timestamp in milliseconds.
const now = () => Date.now();
inferenceChannel.subscribe((event) => {
// Build the EMF payload.
const emf = {
'@aws': {
Timestamp: now(),
CloudWatchMetrics: [
{
Namespace: 'MyAIService',
MetricName: 'InferenceLatency',
Dimensions: [['ModelName', 'Endpoint']],
},
],
},
ModelName: event.model,
Endpoint: event.endpoint,
InferenceLatency: event.latencyMs,
TokenCount: event.tokenCount,
Success: event.success,
};
// Serialize to JSON – this is the single log line CloudWatch will parse.
const message = JSON.stringify(emf);
// In a real container you would write to stdout; CloudWatch Agent picks it up.
// For demonstration we also push directly via SDK (optional, not required for EMF).
console.log(message);
});
Explanation of each line
createChannel
– creates a named broadcast pipe.subscribe
– registers a callback that runs @aws
block – follows the EMF spec described earlier.console.log(message)
– writes the JSON to stdout; the AWS CloudWatch Agent (or the container’s logging driver) ships it to CloudWatch Logs.
Key takeaway:By usingdiagnostics_channel
, you separate metric emission from business logic. The inference code just “fires an event”; the EMF formatter does the rest without adding latency.
// server.ts
import http from 'node:http';
import { inferenceChannel } from './diagnosticsChannel.js';
import { startXRay } from './xraySetup.js';
// Start X‑Ray tracing (see next section for details)
await startXRay();
// Simple HTTP server that pretends to run an LLM inference.
const server = http.createServer(async (req, res) => {
const start = Date.now();
const requestId = crypto.randomUUID(); // unique ID for tracing
// Simulate token counting and latency
const tokenCount = Math.floor(Math.random() * 200) + 1;
const latencyMs = Math.random() * 300 + 50; // 50‑350 ms
// Randomly inject an error 5% of the time
const success = Math.random() > 0.05;
// Emit the EMF event via diagnostics_channel
inferenceChannel.publish({
model: 'gpt‑4‑mini',
endpoint: 'text‑completion',
latencyMs: Math.round(latencyMs),
tokenCount,
success,
});
// Respond to the caller
res.writeHead(success ? 200 : 500, { 'Content-Type': 'application/json' });
res.end(
JSON.stringify({
requestId,
success,
latencyMs: Math.round(latencyMs),
tokenCount,
})
);
});
server.listen(8080, () => {
console.log('AI inference service listening on :8080');
});
diagnostics_channel
, keeping the request path lean.Success
) that can be aggregated into an error‑rate metric.Remember the emfLogger.ts
file from the previous section? Import it once so the subscription is active.
// index.ts
import './emfLogger.js'; // side‑effect import registers the listener
import './server.js';
When the container starts, the listener is registered, and every call to inferenceChannel.publish
results in a single JSON log line.
If you enable X‑Ray on a Lambda or a container without adjusting the daemon, the X‑Ray SDK can add 50‑100 ms to cold starts. In our container we start the daemon once at boot time, and we configure the SDK to use the sampling rule that records only 5 % of requests in production.
// xraySetup.ts
import { XRayClient, PutSamplingRulesCommand } from '@aws-sdk/client-xray';
import { config } from 'node:process';
export async function startXRay() {
// Turn off the default “all‑requests” sampler.
const xray = new XRayClient({ region: 'us-east-1' });
const cmd = new PutSamplingRulesCommand({
SamplingRuleRecords: [
{
SamplingRule: {
RuleName: 'AIInferenceLowSample',
Priority: 1,
FixedRate: 0.05, // 5 % of requests
ReservoirSize: 5,
ServiceName: '*',
ServiceType: '*',
Host: '*',
HTTPMethod: '*',
URLPath: '*',
Version: 1,
// Optional: limit to 1‑second intervals to avoid bursts.
},
// No specific tags needed here.
},
],
});
await xray.send(cmd);
console.log('X‑Ray sampler configured: 5 % of requests will be traced.');
}
Tip:Run a quick benchmark (ab -n 1000 -c 50 http://localhost:8080
) before and after adding X‑Ray to see the cold‑start impact.
diagnostics_channel
.MyAIService
.Application Signals is a newer CloudWatch feature that aggregates latency, error, and request‑count data automatically from multiple sources (including EMF) and runs machine‑learning‑based anomaly detection. It shows you “normal” ranges and flags outliers.
Because our EMF payload already includes InferenceLatency
(a latency metric) and Success
(an error flag), Application Signals can ingest them directly. You only need to enable the feature on the log group.
aws logs put-subscription-filter \
--log-group-name /aws/ecs/my-ai-service \
--filter-name ApplicationSignals \
--filter-pattern "" \
--destination-arn arn:aws:logs:us-east-1:123456789012:destination:CloudWatchApplicationSignals
After a few minutes, the CloudWatch console will show a new Application Signals view for the MyAIService
namespace, with automatic charts for:
InferenceLatency
.Success = false
.
In plain English:Application Signals works like a health‑monitoring smartwatch that reads your EMF “pulse” and alerts you when something feels off.
When you spin up a new container version, the first few seconds may not emit any EMF lines (e.g., if warm‑up logic runs before the first request). Application Signals treats those gaps as “missing data” and can mistakenly flag a spike. Configure the “missing data treatment” in the signal’s alarm settings to “ignore” or “missing = good”.
If your AI service runs in a dev account but you want metrics in a central monitoring account, you need to add an observability access policy to the log group:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowCrossAccountRead",
"Effect": "Allow",
"Principal": { "AWS": "arn:aws:iam::987654321098:root" },
"Action": ["logs:PutLogEvents", "logs:CreateLogStream"],
"Resource": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/ecs/my-ai-service:*"
}
]
}
Now the central account can subscribe to the same log group and see the EMF metrics without duplicating data pipelines.
Key points to remember
PutMetricData
call.@aws
block, correct diagnostics_channel
(Node.js 22) provides a lightweight, zero‑overhead way to broadcast inference results to an EMF formatter.@aws
fields + subscription filter = metric materialization.diagnostics_channel
Now you can instrument your AI inference service with zero added latency, keep your CloudWatch bill in check, and gain instant visibility into latency, token usage, and reliability—all from a single, well‑structured log line. Happy monitoring!
Transparency noticeThis article was written with the help of an AI system —
[Groq](GPT OSS 120B).
Published:2026-08-31 ·Primary focus:CloudWatchAll code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.
Find an error? Drop a comment — corrections are always welcome.