Migrate to aws-sdk-v3

Migrating to AWS SDK for JavaScript (v3) can significantly improve your project’s performance and maintainability. This guide outlines the challenges I faced, solutions I discovered, and the overall benefits of the migration.

Introduction

The AWS SDK for JavaScript (v3) offers modular architecture, allowing developers to import only the parts of the SDK they need. This results in smaller bundle sizes and potentially faster application startup times. However, migrating from v2 to v3 can be challenging, especially for complex projects. Here, I share our experience and insights to help you navigate this transition smoothly.

Resources Before the Migration

Before embarking on the migration, I consulted various resources:

Official AWS SDK JS v3 GitHub repository
Insights from Cloudonaut’s migration experiences
AWS’s own developer guide for migrating to v3
AWS’s own workshop with a sample app

The Challenge

Our migration path encountered several hurdles:

Migration Path: Understanding the best practices and steps for migration was initially daunting.
Automated Migration Tool: I attempted to use the following command for automatic migration. While helpful, it fell short with test files, requiring manual intervention.

npx aws-sdk-js-codemod -t v2-to-v3 <file>

Integration Testing: Our integ-runner failed to detect changes, forcing us to run all tests with the --force flag to ensure reliability.

# re-run the tests anyway
npm run integ-test -- --force
## does not detect the changes in der CR lambdas
npm run integ-test -- --updated-on-failed
npm run integ-test -- --dry-run

For completions the integ-test command looks as follows from the .projenrc.ts

project.package.setScript('integ-test', 'integ-runner --directory ./integ-tests --parallel-regions eu-west-2 --update-on-failed');
# ...
project.synth()

The solution

Our successful migration involved manual adjustments, especially for test files, and a thorough review process. The pull request detailing our journey can be found here: Migration PR.

The mentioned command npx aws-sdk-js-codemod -t v2-to-v3 <file> does most of the job
except the *.test.ts file with mocking, which I will explain more in detail now

jest mocks of the sdk

jest.mock('aws-sdk', () => ({
  SES: spySES,
}));

becomes obviously

// -------------------👇🏽 only mock the package
jest.mock('@aws-sdk/client-ses', () => ({
  SES: spySES,
}));

mocking jest methods

spyVerifyDomainIdentity.mockImplementation(() => ({
  promise() {
    return Promise.resolve({ VerificationToken: 'abc-token' });
  },
}));

becomes this code without the wrapping into a Promise

spyVerifyDomainIdentity.mockImplementation(() => (
   // 👇🏽 as now we return promise by default 
   { VerificationToken: 'abc-token' }
));

the stubbed methods

expect(spyWaitFor).toHaveBeenCalledTimes(1);

you also have to rename manually, as the migration commands does not automatically catch it in the test files:

expect(spyWaitUntilResourceRecordSetsChanged).toHaveBeenCalledTimes(1);

For completion a full *.test.ts looks as follows, however you can also take a look at the full repository on GitHub

const spyVerifyDomainIdentity = jest.fn();
const spyVerifyDomainDkim = jest.fn();
const spyDeleteIdentity = jest.fn();
const spySES = jest.fn(() => ({
  verifyDomainIdentity: spyVerifyDomainIdentity,
  verifyDomainDkim: spyVerifyDomainDkim,
  deleteIdentity: spyDeleteIdentity,
}));

jest.mock('@aws-sdk/client-ses', () => ({
  SES: spySES,
}));

// eslint-disable-next-line import/no-unresolved
import { OnEventRequest } from 'aws-cdk-lib/custom-resources/lib/provider-framework/types';
import { handler } from '../src/hosted-zone-dkim-verification-records.on-event-handler';

describe('hosted-zone-dkim-verification-records', () => {
  beforeEach(() => {
    jest.resetAllMocks();
  });

  it('verifies-create-domain-and-dkim-records', async () => {
    spyVerifyDomainIdentity.mockImplementation(() => (
      { VerificationToken: 'abc-token' }
    ));

    spyVerifyDomainDkim.mockImplementation(() => (
      { DkimTokens: ['dkim-token-1', 'dkim-token-2'] }
    ));

    const result = await handler(
      {
        RequestType: 'Create',
        ResourceProperties: {
          Domain: 'aws.manuel-vogel.de',
        },
      } as unknown as OnEventRequest,
    );

    expect(spyVerifyDomainIdentity).toHaveBeenCalledTimes(1);
    expect(spyVerifyDomainDkim).toHaveBeenCalledTimes(1);

    expect(result).toMatchObject(
      {
        Data: {
          VerificationToken: expect.stringMatching(/abc-token/),
          DkimTokens: expect.arrayContaining(['dkim-token-1', 'dkim-token-2']),
        },
      },
    );
  });
});

That’s it, what I can add to the other post mentioned beforehand. Now let’s look at the size improvements.

Before using aws-sdk-js-v2

The v2 SDK resulted in large bundle sizes for our Lambda functions, significantly impacting deployment times and efficiency.

Bundling asset RootmailStack/rootmail.hosted-zone-dkim-verification-records-provider/on-event-handler/Code/Stage...

  cdk.out/bundling-temp-e4226ae2453120434d34116e38484855ddb115808fc7de8a614f42dc17326b18/index.js  21.6mb ⚠️

⚡ Done in 94ms
Bundling asset RootmailStack/rootmail.autowire-dns-provider/is-complete-handler/Code/Stage...

  cdk.out/bundling-temp-abba88f5bc75fec8258ebc1863344fd858bd658e39162bcd53ca5f347c7051c9/index.js  21.6mb ⚠️

⚡ Done in 92ms
Bundling asset RootmailStack/rootmail.autowire-dns-provider/on-event-handler/Code/Stage...

  cdk.out/bundling-temp-f6c533a54820d647be88c1811cb8a91bbc029115c029727aab2121b1fdf561fd/index.js  21.6mb ⚠️

⚡ Done in 91ms
Bundling asset RootmailStack/rootmail.hosted-zone-dkim-propagation-provider/is-complete-handler/Code/Stage...

  cdk.out/bundling-temp-ee35158e898a20220b3ea9a8139d6e774153df57366a9f562baa80dd9cbb354c/index.js  21.6mb ⚠️

⚡ Done in 107ms
Bundling asset RootmailStack/rootmail.hosted-zone-dkim-propagation-provider/on-event-handler/Code/Stage...

  cdk.out/bundling-temp-76fb0d9bc0c04919e839ce3c2562261bc851c7ac40ab321fe6a23d383f6ec599/index.js  1.7kb

⚡ Done in 1ms
Bundling asset RootmailStack/Rootmail/SESReceive/ops-santa-handler/Code/Stage...

  cdk.out/bundling-temp-e21311716c09f1ca0683a530c7ce66b57f3551fe29aaf702114f7278b5dce8d2/index.js  23.3mb ⚠️

⚡ Done in 102ms
Bundling asset RootmailStack/rootmail.ses-receipt-ruleset-activation-provider/on-event-handler/Code/Stage...

  cdk.out/bundling-temp-b4b78c6f09091442c63ea65ef20d2ffdfa5edc502905e2a2ddfc24556ef3603f/index.js  21.6mb ⚠️

⚡ Done in 93ms
Bundling asset RootmailStack/rootmail.ses-receipt-ruleset-activation-provider/is-complete-handler/Code/Stage...

  cdk.out/bundling-temp-067069faf02bfe5210a5faac5593fc00bb778f78f461c1a8523ac8aff5047d67/index.js  21.6mb ⚠️

⚡ Done in 91ms
(node:47170) NOTE: I are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy
(Use `node --trace-warnings ...` to show where the warning was created)

with a total of ~153MB of code.

After using aws-sdk-js-v3

Switching to v3 dramatically reduced our Lambda bundle sizes, resulting in quicker deployment times and lower resource consumption.

Bundling asset RootmailStack/rootmail.hosted-zone-dkim-verification-records-provider/on-event-handler/Code/Stage...

  cdk.out/bundling-temp-e4226ae2453120434d34116e38484855ddb115808fc7de8a614f42dc17326b18/index.js  3.2kb

⚡ Done in 1ms
Bundling asset RootmailStack/rootmail.autowire-dns-provider/is-complete-handler/Code/Stage...

  cdk.out/bundling-temp-abba88f5bc75fec8258ebc1863344fd858bd658e39162bcd53ca5f347c7051c9/index.js  4.0kb

⚡ Done in 1ms
Bundling asset RootmailStack/rootmail.autowire-dns-provider/on-event-handler/Code/Stage...

  cdk.out/bundling-temp-f6c533a54820d647be88c1811cb8a91bbc029115c029727aab2121b1fdf561fd/index.js  8.6kb

⚡ Done in 1ms
Bundling asset RootmailStack/rootmail.hosted-zone-dkim-propagation-provider/is-complete-handler/Code/Stage...

  cdk.out/bundling-temp-ee35158e898a20220b3ea9a8139d6e774153df57366a9f562baa80dd9cbb354c/index.js  2.7kb

⚡ Done in 1ms
Bundling asset RootmailStack/rootmail.hosted-zone-dkim-propagation-provider/on-event-handler/Code/Stage...

  cdk.out/bundling-temp-76fb0d9bc0c04919e839ce3c2562261bc851c7ac40ab321fe6a23d383f6ec599/index.js  1.7kb

⚡ Done in 1ms
Bundling asset RootmailStack/Rootmail/SESReceive/ops-santa-handler/Code/Stage...

  cdk.out/bundling-temp-e21311716c09f1ca0683a530c7ce66b57f3551fe29aaf702114f7278b5dce8d2/index.js  1.7mb ⚠️

⚡ Done in 15ms
Bundling asset RootmailStack/rootmail.ses-receipt-ruleset-activation-provider/on-event-handler/Code/Stage...

  cdk.out/bundling-temp-b4b78c6f09091442c63ea65ef20d2ffdfa5edc502905e2a2ddfc24556ef3603f/index.js  4.0kb

⚡ Done in 1ms
Bundling asset RootmailStack/rootmail.ses-receipt-ruleset-activation-provider/is-complete-handler/Code/Stage...

  cdk.out/bundling-temp-067069faf02bfe5210a5faac5593fc00bb778f78f461c1a8523ac8aff5047d67/index.js  2.2kb

⚡ Done in 1ms

down to ~1.750MB 🎉

Conclusion

Migrating to AWS SDK for JavaScript (v3) proved immensely beneficial. Not only did I see a reduction in Lambda bundle sizes, but our overall project structure became more modular and maintainable. Despite the challenges in the integration testing process, the migration is undoubtedly worth the effort.

For those considering the migration, be prepared for some manual work, especially around test configurations. However, the benefits far outweigh the initial hurdles. Smaller bundle sizes, improved performance, and the use of modern JavaScript features make AWS SDK v3 a compelling choice for any AWS-powered project.

However, not to forget to mention that I would have expected the cdk integ tests to detect change in the generated lambdas, and rerun the tests. Well I learned this.

Like what you read? You can hire me, or drop me a message to see which services 💻 may help you 👇