Building a Security Pipeline Around AI Anomaly Scoring

I've now built three different anomaly-scoring pipelines for three different security teams, and every single one started the same way: someone got excited about a model that scored 0.94 AUC in a notebook, and then nobody could explain how that score was supposed to trigger an actual response in production. The model isn't the hard part. The pipeline around it is.

This guide walks through how I actually structure these pipelines now, after getting burned by a few of the mistakes I'll describe later. We're not training a model here - I'm assuming you already have a scoring model (whether it's an isolation forest, an autoencoder, or a hosted API) that outputs a numeric anomaly score for an event or session. What we're building is the infrastructure that gets data to that model reliably, turns its output into decisions, and makes those decisions auditable.

Prerequisites

Before you touch any configuration, make sure you actually have these pieces in place. I've seen teams skip straight to "deploy the scorer" and then spend two weeks retrofitting the plumbing.

• A log/event pipeline that already normalizes data into a consistent schema - I use a lightweight JSON schema with fields like

  event_id

  ,

  src_ip

  ,

  user

  ,

  action

  , and

  timestamp

  . If your data isn't normalized before it reaches the scorer, your scores will drift for reasons that have nothing to do with actual anomalies.
• A message queue or streaming layer (Kafka, Redis Streams, or even a durable HTTP queue) sitting between your log source and the scoring service. Don't call the scoring model synchronously from your ingestion path - it will fall over the first time the model gets slow.
• A place to store scores with their inputs, not just the score itself. You will need this for retraining and for explaining decisions to an auditor six months from now.
• An alerting/SOAR layer capable of consuming a webhook or API call - Splunk SOAR, TheHive, or a custom service, doesn't matter which.
• Host and network access mapped out: which hosts can reach the scoring service, and which service accounts have write access to your alert queue.

On the infrastructure side, I set this up on a small dedicated host - in our environment that's

sw-infrarunbook-01

at

10.20.4.15

, isolated on its own VLAN so a compromised scoring service can't pivot straight into the SOC's core network.

Step-by-step setup

I'll walk through this in the order I actually build it, which is not always the order people expect.

Step 1: Build the ingestion adapter first, before the model is even wired in. The adapter's job is to pull events off the queue, validate them against your schema, and drop anything malformed into a dead-letter topic instead of silently failing. In my experience, the single biggest source of "the model isn't catching anything" complaints turns out to be malformed events getting swallowed upstream, not a bad model.

Step 2: Stand up the scoring service behind a stable internal API. I always put a thin wrapper API in front of the actual model runtime (whether that's a Python process, a TensorFlow Serving container, or a call out to a hosted inference endpoint). This wrapper does three things: validates input shape, applies any feature scaling that has to happen at inference time, and logs the raw score alongside a request ID before it ever reaches downstream consumers.

Step 3: Define your scoring buckets, not just a single threshold. A binary "anomalous or not" cutoff sounds clean, but it produces either alert fatigue or missed detections depending on which side of the threshold you err toward. I use three buckets: below 0.5 is logged only, 0.5 to 0.8 goes into a review queue for a human analyst, and above 0.8 triggers automated containment actions. Tune these numbers against your own false positive rate - the numbers I'm giving you are starting points, not gospel.

Step 4: Wire the scorer's output into your SOAR or alerting layer via a dedicated integration service, rather than having the scoring API call out to Slack or your ticketing system directly. Keeping this as a separate hop means you can change your response logic without redeploying the model service, and it gives you one place to add rate limiting so a scoring bug doesn't page your on-call team two hundred times in five minutes.

Step 5: Add a feedback loop. Every time an analyst marks an alert as a false positive or confirms a true positive, that label needs to land back in your score-storage table. Without this, you have no way to measure model drift, and you definitely have no way to justify retraining schedules to whoever owns the budget for this.

Full configuration example

Here's a config I'd actually deploy, trimmed down to the essentials. This is for the ingestion adapter and scoring service pairing - I'm using a YAML format because that's what most of the queue/scoring frameworks I've used expect, but adapt the syntax to whatever your stack uses.

pipeline:
  name: anomaly-scoring-pipeline
  host: sw-infrarunbook-01
  network: 10.20.4.0/24

ingestion:
  queue:
    type: kafka
    brokers:
      - 10.20.4.21:9092
      - 10.20.4.22:9092
    topic: normalized-events
    dead_letter_topic: normalized-events-dlq
    consumer_group: anomaly-scorer-group
  validation:
    schema_path: /etc/infrarunbook/schemas/event_v2.json
    strict_mode: true

scoring_service:
  endpoint: http://10.20.4.30:8501/v1/models/anomaly_scorer:predict
  timeout_ms: 1500
  retry:
    max_attempts: 2
    backoff_ms: 250
  feature_scaling:
    method: standard_scaler
    params_path: /etc/infrarunbook/models/scaler_params.json
  logging:
    store_raw_scores: true
    storage_backend: postgres
    connection: postgres://infrarunbook-admin@10.20.4.40:5432/anomaly_scores

thresholds:
  log_only_max: 0.50
  review_queue_min: 0.50
  review_queue_max: 0.80
  auto_response_min: 0.80

response_routing:
  review_queue:
    type: webhook
    url: http://10.20.4.50:9000/alerts/review
    auth_header: X-Infrarunbook-Token
  auto_response:
    type: soar_playbook
    playbook_id: contain-host-v3
    endpoint: http://10.20.4.50:9000/soar/execute
    rate_limit:
      max_per_minute: 10
      overflow_action: escalate_to_human

feedback_loop:
  enabled: true
  label_storage: postgres://infrarunbook-admin@10.20.4.40:5432/anomaly_scores
  retrain_trigger:
    min_new_labels: 500
    schedule: weekly

A couple of things worth calling out in this config that aren't obvious from just reading it. The

overflow_action: escalate_to_human

line is doing more work than it looks like - it exists specifically to stop an automated containment playbook from running wild if the model starts scoring everything above 0.8 due to a feature pipeline bug. That rate limit has saved me from a genuinely bad afternoon at least once.

The

dead_letter_topic

is not optional in my book. I don't care how solid your upstream schema validation is - something will eventually send malformed data, and you want visibility into that instead of a silent drop.

Verification steps

Once this is deployed, don't just trust that it's running - verify each hop individually before you consider it production-ready.

1. Send a known-benign test event through the ingestion queue and confirm it lands in the normalized-events topic with the correct schema. Check the consumer lag on

   anomaly-scorer-group

   - if it's climbing, your scoring service can't keep up with ingestion volume.
2. Send a synthetic event you know should score high (I usually craft one that mimics a credential-stuffing pattern, since it's easy to construct and easy to reason about) and confirm the raw score lands in the

   anomaly_scores

   table in Postgres before it hits any routing logic.
3. Manually push a score just above 0.80 through the routing layer and confirm the SOAR playbook actually fires - don't assume the webhook worked just because the HTTP call returned 200. Check the playbook execution log on the SOAR side directly.
4. Kill the scoring service intentionally and confirm the dead-letter/retry logic behaves the way you expect, rather than silently dropping the events that were in flight.
5. Check that the feedback loop actually writes analyst labels back to storage - I've seen this silently break because of a schema mismatch between the labeling UI and the storage table, and nobody notices until someone tries to run a retraining job three months later and finds an empty label set.
6. Confirm your rate limiter on auto-response actually limits. Fire fifty synthetic high-score events in a burst and count how many playbook executions actually happen versus how many get escalated to a human queue instead.

I'd also recommend running a full day of shadow mode before you let auto-response actually execute anything destructive - route everything to the review queue only, compare what the model would have auto-contained against what analysts actually flagged, and only flip on automated containment once those two lists mostly agree.

Common mistakes

A few patterns I keep seeing across different teams, worth calling out directly so you can avoid them.

Treating the anomaly score as a fixed, permanent threshold instead of something that needs periodic recalibration. Your baseline traffic shifts - new services get deployed, user behavior changes seasonally, and a threshold tuned in January can be producing garbage by June. Build the retrain and rethreshold step into your calendar, not just your config.

Skipping the raw score storage because it "takes up space." It doesn't take up nearly as much space as you think, and the first time someone asks "why did this alert fire" and you have no record of the input features that produced the score, you'll wish you'd kept it. Storage is cheap. Explaining an unexplainable alert to an auditor is not.

I once inherited a pipeline where the team had thrown away raw scores to save disk space on a system with 400GB free. We spent a week rebuilding partial context from application logs just to answer one compliance question. Never again.

Letting the scoring service call the response layer directly. This seems like it saves you a hop, but it means every time you want to adjust response logic - say, changing which playbook fires for a given score range - you're redeploying the model service itself. Keep these decoupled.

Not rate-limiting automated response actions. A model that starts misbehaving (bad feature drift, a broken upstream field, whatever) combined with an auto-response layer with no brakes is how you end up quarantining half your fleet on a Friday afternoon. Always have an overflow path that escalates to a human instead of just executing more actions faster.

Ignoring the review-queue bucket entirely because "the model is good enough." The middle bucket is where your model actually improves over time - it's the labeled data source for retraining. If you route everything either straight to auto-response or straight to the log, you're starving your own feedback loop.

Finally, don't underestimate how much of this pipeline's reliability comes down to boring infrastructure hygiene: consumer group lag monitoring, dead-letter queue alerting, and database connection pool sizing on whatever's storing your scores. The AI model gets all the attention in planning meetings, but it's the plumbing around it that determines whether the whole thing survives contact with real production traffic.