Issue Description

On October 13 2022, Miao Zhang was notified by Jonas Weinz of GRADE RC that they were having trouble uploading their bi-weekly honu image transfer to AWS.

Error happened at the below script line below:

aws cognito-identity get-id --identity-pool-id $COGNITO_POOL_ID --region $COGNITO_REGION --logins "$COGNITO_LOGIN"="$COGNITO_TOKEN"

Specific error noted belop:

An error occurred (NotAuthorizedException) when calling the GetId operation: Identity pool - us-west-2:0de20cd9-d961-46b9-805b-bf2a9fdaa9c4 does not have identity providers configured.

Timeline

The specific error states that no identity provider has been configured for the Identity Pool.

When investigating the issue and looking at the specific Identity Pool (grade-rc_s3_id_pool), the identity provider configuration information was cleared with blank values.

There are two specific settings that needed to configured for the identity provider:

User Pool ID
App client id

Both of these were blank.

The correct values should be:

us-west-2_VglA0rUYZ
5m558okjar36jl4g6ojp8pvd05

This issue seemed to happen due to a terraform agent update causing an UpdateIdentityPool Event/Action which cleared the Identity Provider settings.

Here’s the last cloudtrail entry showing an UpdateIdentityPool Event for the specific identity pool in question -

{
    "eventVersion": "1.08",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROA2L3JXIRQWQP4FMW73:i-0d8d32c80de1c76e8",
        "arn": "arn:aws:sts::712649426017:assumed-role/tfc-agent-prod/i-0d8d32c80de1c76e8",
        "accountId": "712649426017",
        "accessKeyId": "ASIA2L3JXIRQ5TSZYE56",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "AROA2L3JXIRQWQP4FMW73",
                "arn": "arn:aws:iam::712649426017:role/tfc-agent-prod",
                "accountId": "712649426017",
                "userName": "tfc-agent-prod"
            },
            "webIdFederationData": {},
            "attributes": {
                "creationDate": "2022-09-30T19:32:32Z",
                "mfaAuthenticated": "false"
            },
            "ec2RoleDelivery": "2.0"
        }
    },
    "eventTime": "2022-09-30T19:34:27Z",
    "eventSource": "cognito-identity.amazonaws.com",
    "eventName": "UpdateIdentityPool",
    "awsRegion": "us-west-2",
    "sourceIPAddress": "52.32.25.32",
    "userAgent": "APN/1.0 HashiCorp/1.0 Terraform/1.1.9 (+https://www.terraform.io) terraform-provider-aws/dev (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.44.96 (go1.18.4; linux; amd64)",
    "requestParameters": {
        "identityPoolName": "grade-rc_s3_id_pool",
        "identityPoolId": "us-west-2:0de20cd9-d961-46b9-805b-bf2a9fdaa9c4",
        "allowUnauthenticatedIdentities": false,
        "allowClassicFlow": false
    },
    "responseElements": {
        "identityPoolName": "grade-rc_s3_id_pool",
        "identityPoolTags": {
            "Organization": "gRED",
            "WBS Code": "TBD",
            "Dataset": "TBD",
            "Expiration": "never",
            "Function": "ACE",
            "Project": "HONU Imaging Data Ingestion",
            "Department": "ECDi",
            "Terraform": "true",
            "Version": "3.0",
            "Enterprise": "Genentech",
            "Environment": "prod",
            "Team": "AI",
            "Component": "S3",
            "Requestor": "zhanm104@gene.com",
            "Application": "Reading Center Data Transfer"
        },
        "identityPoolId": "us-west-2:0de20cd9-d961-46b9-805b-bf2a9fdaa9c4",
        "allowUnauthenticatedIdentities": false,
        "allowClassicFlow": false
    },
    "requestID": "077a55d4-f3c1-432c-9f80-64624abe62ce",
    "eventID": "7878ed1e-c388-4923-92c0-496edd3a5a12",
    "readOnly": false,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "712649426017",
    "eventCategory": "Management"
}

How to Avoid Issue in the future

Please make sure to review terraform plans and note these changes of state before running a terraform apply.

GRADE Reading Center Data Transfer XML File Validation