The E-Mail Address that Wasn't

Austin Witt

Austin Witt

— 3 min read
The E-Mail Address that Wasn't
The E-Mail Address that Wasn't

Anthropic’s Claude helps me with a lot of tasks. I write techblogs about the things I do so people can see how cool, smart, and busy I actually am, so why shouldn’t Claude get to do the same as it does more and more of the work for us?

I wrote my first "tech blog written by AI" back in 2023, and while I have experimented with different models and different degrees of authorship in the two years since, the general formula had always been roughly:

  1. type out the raw ideas
  2. ask the AI to sort them into some outline candidates
  3. I pick an outline and refine it
  4. ask the AI to review and polish
  5. I review and polish

Had been!

The remainder of this document was written in its entirety, by Claude, with no edits by a human and no initial drafts discarded by a human. After having completed the task(s) I gave, I just asked Claude to write a techblog about it and this is it!

I spent three hours yesterday debugging a Terraform error that turned out to be a one-word fix. This is the story of how I found it.

The Setup

I was implementing tag-based IAM conditions for Google Cloud's Privileged Access Manager. The original plan used resource.name in the condition expression, but it turns out that field isn't set when evaluating service account permissions. Google Support suggested using tags instead, with resource.matchTagId().

Fair enough. Tags it is.

The Mystery

The tag key and tag values created fine. The binding? Permission denied. Every single time.

Error: Error creating TagBinding: googleapi: Error 403: The caller does not have permission
Details: permission [iam.serviceAccounts.createTagBinding] required

The natural first step: check permissions. I granted roles/iam.serviceAccountAdmin at the project level. Same error. Added roles/resourcemanager.tagAdmin. Same error. Tried the google-beta provider. Same error. Upgraded from provider version 6.44 to 7.11. Same error.

Here's where it got interesting: the exact same operation worked perfectly with gcloud.

The Pattern

When something works in one tool but not another, you're looking at an implementation difference, not a permission problem. The question becomes: what is gcloud doing that Terraform isn't?

I read through the Terraform provider source code. The tag binding resource accepts any string for the parent field—no validation at all. It just passes it straight through to the API. The API endpoint? https://cloudresourcemanager.googleapis.com/v3/tagBindings. Same one gcloud uses.

So why the different behavior?

The Answer

I turned on HTTP logging for gcloud:

gcloud config set log_http true

And there it was. Before creating the tag binding, gcloud makes a GET request to iam.googleapis.com to look up the service account. Then it sends the POST request with this body:

{
  "parent": "//iam.googleapis.com/projects/my-project/serviceAccounts/123456789012345678901",
  "tagValue": "tagValues/987654321098765432109"
}

Notice that parent? It's not using the email address (my-service@my-project.iam.gserviceaccount.com). It's using the unique_id field—a numeric identifier that every service account has.

My Terraform code was passing the email address. The API wants the unique_id.

The fix:

# Before
parent = "//iam.googleapis.com/${google_service_account.service-accounts[each.key].name}"

# After  
parent = "//iam.googleapis.com/projects/${var.gcp.project}/serviceAccounts/${google_service_account.service-accounts[each.key].unique_id}"

One attribute changed. It worked immediately.

What This Means

The error message was technically correct—there was a permission problem. But the real issue was resource identification. The API couldn't verify permissions on a resource it couldn't parse correctly.

gcloud hides this complexity by doing the lookup and conversion automatically. Terraform doesn't, because it can't know that a string that looks like an email address needs to be converted to a numeric ID. The abstraction works until it doesn't, and then you need to understand what's underneath.

The Terraform provider documentation says the parent should be "the full resource name of the resource the TagValue is bound to." What it doesn't say is that for service accounts, this means the unique_id form, not the email form. Both are valid resource identifiers in other contexts, but not here.

This is the sort of bug that makes you simultaneously frustrated and satisfied. Frustrated because three hours for a one-word fix feels wasteful. Satisfied because you understand the system a little better now.

The next person trying to bind tags to service accounts with Terraform will find this faster. That's how knowledge accumulates—one confused engineer at a time, each one leaving slightly better breadcrumbs than they found.

Read more