EFS-ECS : Mounting an existing EFS to an ECS Task returns error Failed to resolve "fs-id"

[ettore-pelosato]

Describe the bug

Hi, hope to find you well. I am trying to mount an existing EFS to a group of ECS tasks. It works just fine when creating it the first time, however when I launch the stack again (therefore reusing the same EFS, I get the Failed to resove "fs-id" error.

Expected Behavior

Here is the code that, as said, works perfectly on the first creation, but fails when reusing the same EFS

const qmmTasksEfsSecurityGroup = new ec2.SecurityGroup(this, 'qmmTasksEfsSecurityGroup', {
            vpc: props.vpc,
            securityGroupName: 'qmmTasksEfsSecurityGroup'
        })

        let qmmTasksEfs: efs.IFileSystem

        if (!config.QlashMainClusterEFSID) {
            
            qmmTasksEfs = new efs.FileSystem(this, `${props.STAGE}qmmTasksEfs`, {
                fileSystemName: `${props.STAGE}qmmTasksEfs`,
                vpc: props.vpc,
                removalPolicy: cdk.RemovalPolicy.RETAIN,
                securityGroup: qmmTasksEfsSecurityGroup,
                encrypted: true,
                lifecyclePolicy: efs.LifecyclePolicy.AFTER_30_DAYS,
                enableAutomaticBackups: true
            })
    
            new cdk.CfnOutput(this, 'QlashMainClusterEFSID', {
                exportName: 'QlashMainClusterEFSID',
                value: qmmTasksEfs.fileSystemId
            })
        } else {
            qmmTasksEfs = efs.FileSystem.fromFileSystemAttributes(this, `${props.STAGE}qmmTasksEfs`, {
                securityGroup: qmmTasksEfsSecurityGroup,
                fileSystemId: config.QlashMainClusterEFSID
            })
        }

            
        const qmmRedisEfsAccessPoint = new efs.AccessPoint(this, `${props.STAGE}qmmRedisAccessPoint`, {
            fileSystem: qmmTasksEfs,
            path: '/redis',
            createAcl: {
                ownerGid: '1001',
                ownerUid: '1001',
                permissions: '750'
            },
            posixUser: {
                uid: '1001',
                gid: '1001'
            }
        })

        const qmmMongoEfsAccessPoint = new efs.AccessPoint(this, `${props.STAGE}qmmMongoAccessPoint`, {
            fileSystem: qmmTasksEfs,
            path: '/mongodb',
            createAcl: {
                ownerGid: '1002',
                ownerUid: '1002',
                permissions: '750'
            },
            posixUser: {
                uid: '1002',
                gid: '1002'
            }
        })

        // Redis

        const qmmRedisServiceSecurityGroup = new ec2.SecurityGroup(this, 'qmmRedisSecurityGroup', {
            vpc: props.vpc,
            securityGroupName: 'qmmRedisSecurityGroup'
        })

        qmmTasksEfsSecurityGroup.addIngressRule(
            ec2.Peer.securityGroupId(qmmRedisServiceSecurityGroup.securityGroupId),
            ec2.Port.tcp(2049),
            'Allow inbound traffic from qmm_redis to qmmTasksEfs'
        )

        if (props.qlashMainInstanceSecurityGroup) {
            qmmRedisServiceSecurityGroup.addIngressRule(
                ec2.Peer.securityGroupId(props.qlashMainInstanceSecurityGroup.securityGroupId),
                ec2.Port.tcp(6379),
                'Allow inbound traffic to qmm_redis from qmmMain instance'
            )
        }

        qmmRedisServiceSecurityGroup.addIngressRule(
            ec2.Peer.ipv4(props.vpc.vpcCidrBlock),
            ec2.Port.tcp(6379),
            'Allow inbound traffic to qmm_redis from resources in qlashMainClusterVpc'
        )

        const qmmRedisTaskDefinition = new ecs.FargateTaskDefinition(this, `${props.STAGE.toLowerCase()}qmmRedisTask`, {
            cpu: 256, //2048,
            memoryLimitMiB: 512, //8192,
            volumes: [
                {
                    name: `${props.STAGE.toLowerCase()}_qmm_redis_volume`,
                    efsVolumeConfiguration: {
                        fileSystemId: qmmTasksEfs.fileSystemId,
                        transitEncryption: 'ENABLED',
                        authorizationConfig: {
                            accessPointId: qmmRedisEfsAccessPoint.accessPointId,
                            iam: 'ENABLED'
                        }
                    }
                }
            ]
        })

        qmmRedisTaskDefinition.addToTaskRolePolicy(
            new iam.PolicyStatement({
                actions: [
                    'elasticfilesystem:ClientWrite',
                    'elasticfilesystem:ClientMount',
                    'elasticfilesystem:ClientRootAccess',
                    'elasticfilesystem:DescribeMountTargets',
                    'elasticfilesystem:CreateAccessPoint',
                    'elasticfilesystem:DeleteAccessPoint',
                    'elasticfilesystem:DescribeAccessPoints',
                    'elasticfilesystem:DescribeFileSystems'
                ],
                resources: [qmmTasksEfs.fileSystemArn],
            })
        )

        qmmRedisTaskDefinition.addToTaskRolePolicy(
            new iam.PolicyStatement({
                actions: ['ec2:DescribeAvailabilityZones'],
                resources: ['*']
            })
        )

        const qmmRedisContainer = qmmRedisTaskDefinition.addContainer(`${props.STAGE.toLowerCase()}_qmm_redis`, {
            image: ecs.ContainerImage.fromAsset('./resources/cluster-resources/redis'),
            containerName: `${props.STAGE.toLowerCase()}_qmm_redis`,
            portMappings: [{ containerPort: 6379, name: `${props.STAGE.toLowerCase()}_qmm_redis` }],
            healthCheck: {
                command: ["CMD", "redis-cli", "-h", "localhost", "-p", "6379", "ping"],
                interval: cdk.Duration.seconds(20),
                timeout: cdk.Duration.seconds(20),
                retries: 5
            },
            logging: ecs.LogDriver.awsLogs({streamPrefix: `${props.STAGE.toLowerCase()}_qmm_redis`}),
            command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
        })

        qmmRedisContainer.addMountPoints({
            sourceVolume: `${props.STAGE.toLowerCase()}_qmm_redis_volume`,
            containerPath: '/redis/data',
            readOnly: false
        })

        const qmmRedisService = new ecs.FargateService(this, `${props.STAGE}qmmRedisService`, {
            serviceName: `${props.STAGE}_qmmRedisService`,
            cluster: qlashMainCluster,
            desiredCount: 1,
            securityGroups: [qmmRedisServiceSecurityGroup],
            taskDefinition: qmmRedisTaskDefinition,
            enableExecuteCommand: true,
            vpcSubnets: {
                subnetGroupName: props.qmmRedisSubnetGroupName
            },
            serviceConnectConfiguration: {
                services: [{ portMappingName: `${props.STAGE.toLowerCase()}_qmm_redis` }]
            }
        })

Current Behavior

See above

Reproduction Steps

Launch the stack twice, adding the QlashMainClusterEFSID Manually or importing it from cdk.json. The first launch will work just fine, the second won't

Possible Solution

Honestly, I have no idea. Could it be that the access points are not getting updated? But then why having the FileSystem.fromFileSystemAttributes) in the first place?

Additional Information/Context

No response

CDK CLI Version

2.88

Framework Version

No response

Node.js Version

18.17

OS

Linux Ubuntu

Language

Typescript

Language Version

No response

Other information

No response

[ettore-pelosato]

Solved. In case you stumble upon this, know you have to also save the Access Point ID (you can find it in the EFS console or output it the first time you deploy it) and also reuse those same access points. Here is the correct code:

let qmmTasksEfs: efs.IFileSystem
        let qmmRedisEfsAccessPoint: efs.IAccessPoint
        let qmmMongoEfsAccessPoint: efs.IAccessPoint

        if (!config.QlashMainClusterEFSID) {
            qmmTasksEfs = new efs.FileSystem(this, `${props.STAGE}qmmTasksEfs`, {
                fileSystemName: `${props.STAGE}qmmTasksEfs`,
                vpc: props.vpc,
                removalPolicy: cdk.RemovalPolicy.RETAIN,
                securityGroup: qmmTasksEfsSecurityGroup,
                encrypted: true,
                lifecyclePolicy: efs.LifecyclePolicy.AFTER_30_DAYS,
                enableAutomaticBackups: true
            })
            
            new cdk.CfnOutput(this, 'QlashMainClusterEFSID', {
                exportName: 'QlashMainClusterEFSID',
                value: qmmTasksEfs.fileSystemId
            })
    
            qmmRedisEfsAccessPoint = new efs.AccessPoint(this, `${props.STAGE}qmmRedisAccessPoint`, {
                fileSystem: qmmTasksEfs,
                path: '/redis',
                createAcl: {
                    ownerGid: '1001',
                    ownerUid: '1001',
                    permissions: '750'
                },
                posixUser: {
                    uid: '1001',
                    gid: '1001'
                }
            })
    
            new cdk.CfnOutput(this, 'QlashMainClusterRedisAccessPointID', {
                exportName: 'QlashMainClusterRedisAccessPointID',
                value: qmmRedisEfsAccessPoint.accessPointId
            })
    
            qmmMongoEfsAccessPoint = new efs.AccessPoint(this, `${props.STAGE}qmmMongoAccessPoint`, {
                fileSystem: qmmTasksEfs,
                path: '/mongodb',
                createAcl: {
                    ownerGid: '1002',
                    ownerUid: '1002',
                    permissions: '750'
                },
                posixUser: {
                    uid: '1002',
                    gid: '1002'
                }
            })

            new cdk.CfnOutput(this, 'QlashMainClusterMongoAccessPointID', {
                exportName: 'QlashMainClusterMongoAccessPointID',
                value: qmmMongoEfsAccessPoint.accessPointId
            })

        } else {
            qmmTasksEfs = efs.FileSystem.fromFileSystemAttributes(this, `${props.STAGE}qmmTasksEfs`, {
                securityGroup: qmmTasksEfsSecurityGroup,
                fileSystemId: config.QlashMainClusterEFSID
            })

            qmmRedisEfsAccessPoint = efs.AccessPoint.fromAccessPointId(this, `${props.STAGE}qmmRedisAccessPoint`, config.QlashMainClusterRedisAccessPointID)

            qmmMongoEfsAccessPoint = efs.AccessPoint.fromAccessPointId(this, `${props.STAGE}qmmMongoAccessPoint`, config.QlashMainClusterMongoAccessPointID)
        }

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.