Question:
I’m trying to use a CloudFormation template to define CodeBuild and CodePipeline to automate the deployment of a static website hosted in an S3 bucket. To give credit where credit’s due, I’m largely following the template from https://dzone.com/articles/continuous-delivery-to-s3-via-codepipeline-and-cod.
The problem I can’t resolve is that after I add an environment variable for the Hugo version I’d like to use to create the static site files, I get an error from the AWS console that reads: “Template validation error: Template format error: Unresolved resource dependencies [HUGO_VERSION] in the Resources block of the template.”
Why isn’t it accepting the HUGO_VERSION environment variable that I define under environment_variables? This is version 0.1 of the format, so it’s a little different than the current 0.2, but I’ve read the following link: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#build-spec-ref-syntax
The thing that really confuses me is that if I remove the lines with ${HUGO_VERSION}, the template is accepted just fine – and then inspection of the CloudWatch logs after building shows (because of the printenv command) HUGO_VERSION=0.49! What gives?
Originally, the template looks like this.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 |
--- AWSTemplateFormatVersion: '2010-09-09' Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3 Parameters: SiteBucketName: Type: String Description: Name of bucket to create to host the website GitHubUser: Type: String Description: GitHub User Default: "stelligent" GitHubRepo: Type: String Description: GitHub Repo to pull from. Only the Name. not the URL Default: "devops-essentials" GitHubBranch: Type: String Description: GitHub Branch Default: "master" GitHubToken: NoEcho: true Type: String Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens BuildType: Type: String Default: "LINUX_CONTAINER" Description: The build container type to use for building the app BuildComputeType: Type: String Default: "BUILD_GENERAL1_SMALL" Description: The build compute type to use for building the app BuildImage: Type: String Default: "aws/codebuild/ubuntu-base:14.04" Description: The build image to use for building the app Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: "Site Configuration" Parameters: - SiteBucketName - Label: default: "GitHub Configuration" Parameters: - GitHubToken - GitHubUser - GitHubRepo - GitHubBranch - Label: default: "Build Configuration" Parameters: - BuildType - BuildComputeType - BuildImage ParameterLabels: SiteBucketName: default: Name of S3 Bucket to create for website hosting GitHubToken: default: GitHub OAuth2 Token GitHubUser: default: GitHub User/Org Name GitHubRepo: default: GitHub Repository Name GitHubBranch: default: GitHub Branch Name BuildType: default: CodeBuild type BuildComputeType: default: CodeBuild instance type BuildImage: default: CodeBuild image Resources: CodeBuildRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - codebuild.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: codebuild-service PolicyDocument: Statement: - Effect: Allow Action: "*" Resource: "*" Version: '2012-10-17' CodePipelineRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - codepipeline.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: codepipeline-service PolicyDocument: Statement: - Action: - codebuild:* Resource: "*" Effect: Allow - Action: - s3:GetObject - s3:GetObjectVersion - s3:GetBucketVersioning Resource: "*" Effect: Allow - Action: - s3:PutObject Resource: - arn:aws:s3:::codepipeline* Effect: Allow - Action: - s3:* - cloudformation:* - iam:PassRole Resource: "*" Effect: Allow Version: '2012-10-17' SiteBucket: Type: AWS::S3::Bucket DeletionPolicy: Delete Properties: AccessControl: PublicRead BucketName: !Ref SiteBucketName WebsiteConfiguration: IndexDocument: index.html PipelineBucket: Type: AWS::S3::Bucket DeletionPolicy: Delete CodeBuildDeploySite: Type: AWS::CodeBuild::Project DependsOn: CodeBuildRole Properties: Name: !Sub ${AWS::StackName}-DeploySite Description: Deploy site to S3 ServiceRole: !GetAtt CodeBuildRole.Arn Artifacts: Type: CODEPIPELINE Environment: Type: !Ref BuildType ComputeType: !Ref BuildComputeType Image: !Sub ${BuildImage} Source: Type: CODEPIPELINE BuildSpec: !Sub | version: 0.1 phases: post_build: commands: - aws s3 cp --recursive --acl public-read ./samples s3://${SiteBucketName}/samples - aws s3 cp --recursive --acl public-read ./html s3://${SiteBucketName}/ artifacts: type: zip files: - ./html/index.html TimeoutInMinutes: 10 Pipeline: Type: AWS::CodePipeline::Pipeline Properties: RoleArn: !GetAtt CodePipelineRole.Arn Stages: - Name: Source Actions: - InputArtifacts: [] Name: Source ActionTypeId: Category: Source Owner: ThirdParty Version: '1' Provider: GitHub OutputArtifacts: - Name: SourceArtifacts Configuration: Owner: !Ref GitHubUser Repo: !Ref GitHubRepo Branch: !Ref GitHubBranch OAuthToken: !Ref GitHubToken RunOrder: 1 - Name: Deploy Actions: - Name: Artifact ActionTypeId: Category: Build Owner: AWS Version: '1' Provider: CodeBuild InputArtifacts: - Name: SourceArtifacts OutputArtifacts: - Name: DeploymentArtifacts Configuration: ProjectName: !Ref CodeBuildDeploySite RunOrder: 1 ArtifactStore: Type: S3 Location: !Ref PipelineBucket Outputs: PipelineUrl: Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline} Description: CodePipeline URL SiteUrl: Value: !GetAtt [SiteBucket, WebsiteURL] Description: S3 Website URL |
Now after I try to add an environment variable to use Hugo in the pipeline, the template looks like this.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 |
--- AWSTemplateFormatVersion: '2010-09-09' Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3 Parameters: SiteBucketName: Type: String Description: Name of bucket to create to host the website GitHubUser: Type: String Description: GitHub User Default: "stelligent" GitHubRepo: Type: String Description: GitHub Repo to pull from. Only the Name. not the URL Default: "devops-essentials" GitHubBranch: Type: String Description: GitHub Branch Default: "master" GitHubToken: NoEcho: true Type: String Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens BuildType: Type: String Default: "LINUX_CONTAINER" Description: The build container type to use for building the app BuildComputeType: Type: String Default: "BUILD_GENERAL1_SMALL" Description: The build compute type to use for building the app BuildImage: Type: String Default: "aws/codebuild/ubuntu-base:14.04" Description: The build image to use for building the app Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: "Site Configuration" Parameters: - SiteBucketName - Label: default: "GitHub Configuration" Parameters: - GitHubToken - GitHubUser - GitHubRepo - GitHubBranch - Label: default: "Build Configuration" Parameters: - BuildType - BuildComputeType - BuildImage ParameterLabels: SiteBucketName: default: Name of S3 Bucket to create for website hosting GitHubToken: default: GitHub OAuth2 Token GitHubUser: default: GitHub User/Org Name GitHubRepo: default: GitHub Repository Name GitHubBranch: default: GitHub Branch Name BuildType: default: CodeBuild type BuildComputeType: default: CodeBuild instance type BuildImage: default: CodeBuild image Resources: CodeBuildRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - codebuild.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: codebuild-service PolicyDocument: Statement: - Effect: Allow Action: "*" Resource: "*" Version: '2012-10-17' CodePipelineRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - codepipeline.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: codepipeline-service PolicyDocument: Statement: - Action: - codebuild:* Resource: "*" Effect: Allow - Action: - s3:GetObject - s3:GetObjectVersion - s3:GetBucketVersioning Resource: "*" Effect: Allow - Action: - s3:PutObject Resource: - arn:aws:s3:::codepipeline* Effect: Allow - Action: - s3:* - cloudformation:* - iam:PassRole Resource: "*" Effect: Allow Version: '2012-10-17' SiteBucket: Type: AWS::S3::Bucket DeletionPolicy: Delete Properties: AccessControl: PublicRead BucketName: !Ref SiteBucketName WebsiteConfiguration: IndexDocument: index.html PipelineBucket: Type: AWS::S3::Bucket DeletionPolicy: Delete CodeBuildDeploySite: Type: AWS::CodeBuild::Project DependsOn: CodeBuildRole Properties: Name: !Sub ${AWS::StackName}-DeploySite Description: Deploy site to S3 ServiceRole: !GetAtt CodeBuildRole.Arn Artifacts: Type: CODEPIPELINE Environment: Type: !Ref BuildType ComputeType: !Ref BuildComputeType Image: !Sub ${BuildImage} Source: Type: CODEPIPELINE BuildSpec: !Sub | version: 0.1 environment_variables: plaintext: AWS_DEFAULT_REGION: "US-WEST-2" HUGO_VERSION: "0.49" phases: install: commands: - printenv - echo "Install step..." - curl -Ls https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz -o /tmp/hugo.tar.gz - tar xf /tmp/hugo.tar.gz -C /tmp - mv /tmp/hugo_${HUGO_VERSION}_linux_amd64/hugo_${HUGO_VERSION}_linux_amd64 /usr/bin/hugo - rm -rf /tmp/hugo* build: commands: - hugo post_build: commands: - aws s3 cp --recursive --acl public-read ./public s3://${SiteBucketName} artifacts: type: zip files: - ./html/index.html TimeoutInMinutes: 10 Pipeline: Type: AWS::CodePipeline::Pipeline Properties: RoleArn: !GetAtt CodePipelineRole.Arn Stages: - Name: Source Actions: - InputArtifacts: [] Name: Source ActionTypeId: Category: Source Owner: ThirdParty Version: '1' Provider: GitHub OutputArtifacts: - Name: SourceArtifacts Configuration: Owner: !Ref GitHubUser Repo: !Ref GitHubRepo Branch: !Ref GitHubBranch OAuthToken: !Ref GitHubToken RunOrder: 1 - Name: Deploy Actions: - Name: Artifact ActionTypeId: Category: Build Owner: AWS Version: '1' Provider: CodeBuild InputArtifacts: - Name: SourceArtifacts OutputArtifacts: - Name: DeploymentArtifacts Configuration: ProjectName: !Ref CodeBuildDeploySite RunOrder: 1 ArtifactStore: Type: S3 Location: !Ref PipelineBucket Outputs: PipelineUrl: Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline} Description: CodePipeline URL SiteUrl: Value: !GetAtt [SiteBucket, WebsiteURL] Description: S3 Website URL |
EDIT 10/20
Still haven’t solved this. I tried to follow the advice given below by matsev, but I’m still getting the same validation error. For completeness, the latest template I’m trying is
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 |
AWSTemplateFormatVersion: '2010-09-09' Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3 Parameters: SiteBucketName: Type: String Description: Name of bucket to create to host the website GitHubUser: Type: String Description: GitHub User Default: "stelligent" GitHubRepo: Type: String Description: GitHub Repo to pull from. Only the Name. not the URL Default: "devops-essentials" GitHubBranch: Type: String Description: GitHub Branch Default: "master" GitHubToken: NoEcho: true Type: String Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens BuildType: Type: String Default: "LINUX_CONTAINER" Description: The build container type to use for building the app BuildComputeType: Type: String Default: "BUILD_GENERAL1_SMALL" Description: The build compute type to use for building the app BuildImage: Type: String Default: "aws/codebuild/ubuntu-base:14.04" Description: The build image to use for building the app Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: "Site Configuration" Parameters: - SiteBucketName - Label: default: "GitHub Configuration" Parameters: - GitHubToken - GitHubUser - GitHubRepo - GitHubBranch - Label: default: "Build Configuration" Parameters: - BuildType - BuildComputeType - BuildImage ParameterLabels: SiteBucketName: default: Name of S3 Bucket to create for website hosting GitHubToken: default: GitHub OAuth2 Token GitHubUser: default: GitHub User/Org Name GitHubRepo: default: GitHub Repository Name GitHubBranch: default: GitHub Branch Name BuildType: default: CodeBuild type BuildComputeType: default: CodeBuild instance type BuildImage: default: CodeBuild image Resources: CodeBuildRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - codebuild.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: codebuild-service PolicyDocument: Statement: - Effect: Allow Action: "*" Resource: "*" Version: '2012-10-17' CodePipelineRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - codepipeline.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: codepipeline-service PolicyDocument: Statement: - Action: - codebuild:* Resource: "*" Effect: Allow - Action: - s3:GetObject - s3:GetObjectVersion - s3:GetBucketVersioning Resource: "*" Effect: Allow - Action: - s3:PutObject Resource: - arn:aws:s3:::codepipeline* Effect: Allow - Action: - s3:* - cloudformation:* - iam:PassRole Resource: "*" Effect: Allow Version: '2012-10-17' SiteBucket: Type: AWS::S3::Bucket DeletionPolicy: Delete Properties: AccessControl: PublicRead BucketName: !Ref SiteBucketName WebsiteConfiguration: IndexDocument: index.html PipelineBucket: Type: AWS::S3::Bucket DeletionPolicy: Delete CodeBuildDeploySite: Type: AWS::CodeBuild::Project DependsOn: CodeBuildRole Properties: Name: !Sub ${AWS::StackName}-DeploySite Description: Deploy site to S3 ServiceRole: !GetAtt CodeBuildRole.Arn Artifacts: Type: CODEPIPELINE Environment: Type: !Ref BuildType ComputeType: !Ref BuildComputeType Image: !Sub ${BuildImage} EnvironmentVariables: - Name: HUGO_VERSION Value: '0.49' Type: PLAINTEXT Source: Type: CODEPIPELINE BuildSpec: !Sub | version: 0.2 env: variables: AWS_DEFAULT_REGION: "US-WEST-2" phases: install: commands: - printenv - curl -Ls https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz -o /tmp/hugo.tar.gz - tar xf /tmp/hugo.tar.gz -C /tmp - mv /tmp/hugo_${HUGO_VERSION}_linux_amd64/hugo_${HUGO_VERSION}_linux_amd64 /usr/bin/hugo - rm -rf /tmp/hugo* build: commands: - hugo post_build: commands: - aws s3 cp --recursive --acl public-read ./samples s3://${SiteBucketName}/samples - aws s3 cp --recursive --acl public-read ./html s3://${SiteBucketName}/ artifacts: type: zip files: - ./html/index.html TimeoutInMinutes: 10 Pipeline: Type: AWS::CodePipeline::Pipeline Properties: RoleArn: !GetAtt CodePipelineRole.Arn Stages: - Name: Source Actions: - InputArtifacts: [] Name: Source ActionTypeId: Category: Source Owner: ThirdParty Version: '1' Provider: GitHub OutputArtifacts: - Name: SourceArtifacts Configuration: Owner: !Ref GitHubUser Repo: !Ref GitHubRepo Branch: !Ref GitHubBranch OAuthToken: !Ref GitHubToken RunOrder: 1 - Name: Deploy Actions: - Name: Artifact ActionTypeId: Category: Build Owner: AWS Version: '1' Provider: CodeBuild InputArtifacts: - Name: SourceArtifacts OutputArtifacts: - Name: DeploymentArtifacts Configuration: ProjectName: !Ref CodeBuildDeploySite RunOrder: 1 ArtifactStore: Type: S3 Location: !Ref PipelineBucket Outputs: PipelineUrl: Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline} Description: CodePipeline URL SiteUrl: Value: !GetAtt [SiteBucket, WebsiteURL] Description: S3 Website URL |
Answer:
Please check the Environment property of the AWS::CodeBuild::Project in your CloudFormation template. Specifically the EnvironmentVariables allows you to specify environment variables, e.g.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
CodeBuildDeploySite: Type: AWS::CodeBuild::Project DependsOn: CodeBuildRole Properties: Name: !Sub ${AWS::StackName}-DeploySite Description: Deploy site to S3 ServiceRole: !GetAtt CodeBuildRole.Arn Artifacts: Type: CODEPIPELINE Environment: Type: !Ref BuildType ComputeType: !Ref BuildComputeType Image: !Sub ${BuildImage} EnvironmentVariables: - Name: HUGO_VERSION Value: '0.49' Type: PLAINTEXT # More properties... |
Now in you can reference the
HUGO_VERSION as an environment variable in your buildspec.yml file, e.g.|
1 2 3 4 |
pre_build: commands: - echo HUGO_VERSION $HUGO_VERSION |