Let's Do DevOps

Let's Do DevOps

Share this post

Let's Do DevOps
Let's Do DevOps
đŸ”„Slack GenAI with Bedrock: Implementing Guardrail Tracing - "why did you block that?" AnsweredđŸ”„
Copy link
Facebook
Email
Notes
More

đŸ”„Slack GenAI with Bedrock: Implementing Guardrail Tracing - "why did you block that?" AnsweredđŸ”„

aka, why the heck did you block that?

Kyler Middleton's avatar
Kyler Middleton
May 06, 2025
∙ Paid

Share this post

Let's Do DevOps
Let's Do DevOps
đŸ”„Slack GenAI with Bedrock: Implementing Guardrail Tracing - "why did you block that?" AnsweredđŸ”„
Copy link
Facebook
Email
Notes
More
Share

This blog series focuses on presenting complex DevOps projects as simple and approachable via plain language and lots of pictures. You can do it!

These articles are supported by readers, please consider subscribing to support me writing more of these articles <3 :)

Hey all!

As part of this previous series on implementing a private, enterprise security-compliant GenAI slack bot with AWS Bedrock and Lambda, we implemented one of the foundational security tools available in AWS Bedrock - a Guardrail.

đŸ”„Building a Slack Bot with AI Capabilities - From Scratch! Part 1: Slack App and EventsđŸ”„

đŸ”„Building a Slack Bot with AI Capabilities - From Scratch! Part 1: Slack App and EventsđŸ”„

Kyler Middleton
·
December 3, 2024
Read full story

Guardrails provide a lot of different types of security around AI models - they can filter the tokens in, and tokens out, and look at all sorts of stuff:

  • Pre-set categories like hate, sex, violence, etc.

  • Prompt attack detection

  • Topical detection and filtering, where you can write example objectionable requests

  • Profanity filtering (great for token out filtering, if you don’t want your bot to swear at your users)

  • PII filtering - can detect bank account, credit card, license plates, etc.

    • Has a cool option here to not just deny the entire response, but instead to redact (Guardrails calls this “mask”) the data and replace it with a static string, like CREDIT_CARD_NUMBER.

Here’s the old version. Why did Guardrails block this? ¯\_(ツ)_/¯

If we enable guardrail tracing, and then read the response if there’s a block, we can tell users exactly why:

Then they don’t need to ask me to check the logs, and I can stay in my blissful dark closet, programming.

We’ll be walking through this PR, which adds the functionality to the open source Repo where I’m sharing all this code, so you can go build it yourself!

With no further ado, lets ado this.

Turning on Guardrail Tracing in Model Invocations

Keep reading with a 7-day free trial

Subscribe to Let's Do DevOps to keep reading this post and get 7 days of free access to the full post archives.

Already a paid subscriber? Sign in
© 2025 Kyler Middleton
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More