Skip to content
This repository was archived by the owner on Feb 6, 2025. It is now read-only.

Add Resource Groups + AZ filter for AWS tf #1225

Merged
merged 3 commits into from
Jul 13, 2020
Merged

Add Resource Groups + AZ filter for AWS tf #1225

merged 3 commits into from
Jul 13, 2020

Conversation

spiarh
Copy link
Contributor

@spiarh spiarh commented Jul 3, 2020

Why is this PR needed?

  • This commits add the support for resource groups when deploying on AWS. This gives great benefits for operators and accountants. This also aligns this with our Azure Terraform which uses resource group as well.

Quoting official documentation:

In AWS, a resource is an entity that you can work with. 
Examples include an Amazon EC2 instance, an AWS CloudFormation stack, or an Amazon S3 bucket. 
If you work with multiple resources, you might find it useful to manage them as a group 
rather than move from one AWS service to another for each task.
If you manage large numbers of related resources, such as EC2 instances that make up an application layer, 
you likely need to perform bulk actions on these resources at one time. Examples of bulk actions include:

    Applying updates or security patches.
    Upgrading applications.
    Opening or closing ports to network traffic.
    Collecting specific log and monitoring data from your fleet of instances.

A resource group is a collection of AWS resources that are all in the same AWS region, 
and that match criteria provided in a query. In Resource Groups, there are two types of 
queries on which you can build a group. 

It is required for a user to have the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "resource-groups:*",
        "cloudformation:DescribeStacks",
        "cloudformation:ListStackResources",
        "tag:GetResources",
        "tag:TagResources",
        "tag:UntagResources",
        "tag:getTagKeys",
        "tag:getTagValues",
        "resource-explorer:*"
      ],
      "Resource": "*"
    }
  ]
}
  • This also adds a variable to filter the Availability Zones in order to select a specific one. This is a requirement for instance when the operator wants to select an instance type which is only available in a specific AZ.

What does this PR do?

  • This groups the resources in a resource groups, then the resources can be queried using cli, for example:
$ aws resource-groups list-group-resources --group-name lcavajani-resourcegroup
{
    "ResourceIdentifiers": [
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:instance/i-0cccb729b542bb524",
            "ResourceType": "AWS::EC2::Instance"
        },
        {
            "ResourceArn": "arn:aws:resource-groups:eu-west-3:434479817650:group/lcavajani-resourcegroup",
            "ResourceType": "AWS::ResourceGroups::Group"
        },
        {
            "ResourceArn": "arn:aws:elasticloadbalancing:eu-west-3:434479817650:loadbalancer/lcavajani-elb",
            "ResourceType": "AWS::ElasticLoadBalancing::LoadBalancer"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:dhcp-options/dopt-082d777578a6deb0d",
            "ResourceType": "AWS::EC2::DHCPOptions"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:subnet/subnet-0b8c8808302a90802",
            "ResourceType": "AWS::EC2::Subnet"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:internet-gateway/igw-06359216ee8a9bdb3",
            "ResourceType": "AWS::EC2::InternetGateway"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:security-group/sg-05564e7ba971934bc",
            "ResourceType": "AWS::EC2::SecurityGroup"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:route-table/rtb-007014fbfd78140be",
            "ResourceType": "AWS::EC2::RouteTable"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:security-group/sg-0ee9898ed02233532",
            "ResourceType": "AWS::EC2::SecurityGroup"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:instance/i-05b5e97dc6ba4bc22",
            "ResourceType": "AWS::EC2::Instance"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:natgateway/nat-0c3a684be52648a89",
            "ResourceType": "AWS::EC2::NatGateway"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:route-table/rtb-05ed4e0f00ed5ff0d",
            "ResourceType": "AWS::EC2::RouteTable"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:security-group/sg-07cc7e042dd8a60f7",
            "ResourceType": "AWS::EC2::SecurityGroup"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:security-group/sg-0e9315e76c12fa7d8",
            "ResourceType": "AWS::EC2::SecurityGroup"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:instance/i-018c5cf790cc6eb29",
            "ResourceType": "AWS::EC2::Instance"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:vpc/vpc-0de4553a6c791433c",
            "ResourceType": "AWS::EC2::VPC"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:elastic-ip/eipalloc-072b35e226c21325a",
            "ResourceType": "AWS::EC2::EIP"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:instance/i-0a2a30dd401595107",
            "ResourceType": "AWS::EC2::Instance"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:subnet/subnet-067027e19652efeb7",
            "ResourceType": "AWS::EC2::Subnet"
        },
        {
            "ResourceArn": "arn:aws:ec2:eu-west-3:434479817650:security-group/sg-08ebbcf8d61219632",
            "ResourceType": "AWS::EC2::SecurityGroup"
        }
    ]
}
  • AZ can be selected

If all the regions are working correctly and the operator uses the region eu-west-3.

With the default filter, the availability zone selected will be the first one in the list so eu-west-3-a.

If the operator defines the following variable:

availability_zones_filter= {
    name   = "zone-name"
    values = ["eu-west-3c"]
}

Then the subnet will be created in eu-west-3c

Docs

There is currently no doc about the required IAM permission for the operator in the doc (only for instance profiles).

cc @nkoranova @r0ckarong do you have something on your side regarding this topic ? otherwise I will open a bug.

Merge restrictions

(Please do not edit this)

We are in v4-maintenance phase, so we will restrict what can be merged to prevent unexpected surprises:

What can be merged (merge criteria):
    2 approvals:
        1 developer: code is fine
        1 QA: QA is fine
    there is a PR for updating documentation (or a statement that this is not needed)

Signed-off-by: lcavajani <lcavajani@suse.com>
Signed-off-by: lcavajani <lcavajani@suse.com>
@spiarh spiarh changed the title Add Resource Groups for AWS tf Add Resource Groups + AZ filter for AWS tf Jul 7, 2020
@spiarh
Copy link
Contributor Author

spiarh commented Jul 7, 2020

Since the PR was still open an not too fat, I'ved added support to filter the Availability Zones.

Signed-off-by: lcavajani <lcavajani@suse.com>
Copy link

@jenting jenting left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@innobead innobead requested a review from maximenoel8 July 8, 2020 01:48
Copy link
Contributor

@innobead innobead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@innobead
Copy link
Contributor

innobead commented Jul 8, 2020

cc @maximenoel8 @thehejik better to have a review and regression. Thanks.

Copy link
Member

@flavio flavio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@flavio
Copy link
Member

flavio commented Jul 10, 2020

There is currently no doc about the required IAM permission for the operator in the doc (only for instance profiles).
cc @nkoranova @r0ckarong do you have something on your side regarding this topic ? otherwise I will open a bug.

Please open a bug for that and submit some documentation for that (if you can, I would love that).

Thanks for yet another great submission 💚

@innobead innobead merged commit bd6ca7e into SUSE:master Jul 13, 2020
chentex pushed a commit to chentex/skuba-1 that referenced this pull request Jul 20, 2020
* Add resourcegroup in AWS tf

Signed-off-by: lcavajani <lcavajani@suse.com>

* Add AZ filter in AWS tf

Signed-off-by: lcavajani <lcavajani@suse.com>

* add az filter in tfvars example

Signed-off-by: lcavajani <lcavajani@suse.com>
jordimassaguerpla pushed a commit to jordimassaguerpla/skuba that referenced this pull request Aug 12, 2020
* Add resourcegroup in AWS tf

Signed-off-by: lcavajani <lcavajani@suse.com>

* Add AZ filter in AWS tf

Signed-off-by: lcavajani <lcavajani@suse.com>

* add az filter in tfvars example

Signed-off-by: lcavajani <lcavajani@suse.com>
@jordimassaguerpla
Copy link
Member

backport #1319

jordimassaguerpla added a commit that referenced this pull request Aug 12, 2020
…ckport_4_5_0

Add Resource Groups + AZ filter for AWS tf (#1225)
@r0ckarong
Copy link

Does this impact the terraform examples in the docs? https://github.com/SUSE/doc-caasp/blob/master/adoc/deployment-terraform-example.adoc

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants