State of the JavaScript ecosystem in 2023

Node, Deno, and Bun: competition is good for runtime engines

With multiple runtime engines emerging, fresh competition pushes runtime development further. This competition is highly beneficial!

For an extended period, Node.js has predominantly served as the primary server-side runtime for everyone, while variations have occurred in the package manager arena (notably npm, Yarn, pnpm), compilation processes (including TypeScript, Babel, Webpack, Esbuild, plugins, and linting), as well as testing tools (such as Vitest, Jest, Mocha, Jasmine, and Protractor).

Node.js is based on V8, an open-source JavaScript engine developed by Google in 2008, initially for Google Chrome and other Chromium-based browsers. It is the industry standard and the most “senior" engine of the bunch. The developer, OpenJS Foundation, is also the home of around 30 other projects, including Electron and jQuery.

In 2018, ten years after Node.js, came Deno – although the official 1.0 release did not see the light until 2020. Deno uses a secure-by-default model that blocks off potentially problematic operations until the user explicitly enables them at runtime. That way, the applications may get protection against some vulnerabilities.

Deno also offers a more dynamic way to load modules without a package.json definition. As the engine only supports ES Modules, the ecosystem transition to ESM should help gain traction. Deno also provides a built-in library std/node to support loading CommonJS modules, and it is also possible to transpile the CommonJS modules to ESM by the Content Delivery Network providing the modules to the application.

While I've yet to witness its extensive use in production applications, Deno's native support for TypeScript is very tempting – it is easy to use even without any configuration files.

Bun is the newest engine of the group with its 1.0 version released in September 2023. The release announcement covers the main points of why one would choose Bun as their engine: speed, built-in bundler, transpiler, runner, and a npm client. Bun promises astonishing speed improvements in the development cycle where a developer writes code, compiles it, and runs automated tests against it.

I’m expecting to see all three used in client projects over the next couple of years.

The rise of ECMAScript Modules (ESM)

ECMAScript modules are the new, official standard format to package JavaScript code for reuse.

Historically, JavaScript lacked native module support, leading Node.js to adopt the CommonJS format, which introduced "require" and "module.exports" methods for defining reusable modules, forming the foundation of the renowned NPM ecosystem.

Internet browsers have used a format called Asynchronous Module Definition (AMD), and bundlers like Webpack have consistently offered support for module requirements since we transitioned away from manually embedding <script> tags into web pages during what's often referred to as "The jQuery era."

For developers using TypeScript, the import/export clauses have been compiled to use these module concepts depending on TypeScript settings. It’s been quite a task to write modern JavaScript and support older browsers at the same time.

The good old Node.js "require" method operates synchronously, causing the program to halt until the module is loaded, evaluated, and ready for use. In contrast, the new standardized ESM (ECMAScript Modules) "import" method is asynchronous, enabling concurrent evaluation of other parts while the module loads, speeding up the bootup process.

The advancements and adoption of ESM

Another massive benefit lies in ESM being statically analyzable. This way, the browser executing the application can make intelligent decisions regarding parallel requests and evaluation. In the deployment phase, the bundlers can analyze which parts of the modules are used, optimize their references, and tree-shake (drop) those module parts that are not in use at all. And overall, it makes sense to use the same import/export scheme in both browser and server-side, so ESM as a standard brings us together.

With TypeScript 4.7 (released in May 2022), Node v18 (April 2022), and AWS Lambda support for Node 18 (November 2022), the ESM is fully ready for production use on the server side. We began adopting it in June 2022, initially compiling the final output code into the trusty CommonJS format. Since March 2023, we've seamlessly transitioned to ESM in multiple of our backend services for production use without a hitch.

Since the JavaScript module ecosystem is already rapidly moving from the CommonJS to the ESM world, our ability to update some dependencies to their latest versions would have been compromised already had we not made the jump.

Some prominent authors provide only ESM versions of their libraries already. So, the JavaScript ecosystem is adapting to the ESM, and the transition would be ahead of us in any case. Even TypeScript will ship its APIs in ESM sooner or later. Returning to the initial point, the ecosystem's ESM support is now mature and suitable for production use.

Transitioning to ESM

In 2022, as we were adopting ESM, some opportunities emerged with TypeScript and modules within the same timeframe, making it both fun and challenging to discover the optimal configuration combinations.

Compiling server-side software to a controlled environment is naturally simpler than various customer browsers. For our server-side applications, we now rely solely on TypeScript and esbuild (built-in support in AWS CDK) to emit the compiled code. We have removed Babel and Webpack completely.

Beginning the "esmification" process with server-side applications and then proceeding to frontend code is the most straightforward approach for reducing loading times.

These were our conversion steps and settings:

• Change all imports to have a ".js" suffix in the code. The code files are still .ts files.

• Add “type”: “module” to package.json

• Edit cdk.json to have "--esm" for ts-node

• Edit tsconfig.json to have module: node16 and moduleResolution: node16

• Replace requiring of JSON files to ESM compatible way

• Hotpatch aws-sdk with .replace('file://', ''), https://github.com/sindresorhus/callsites/issues/18#issuecomment-1206281003

With AWS CDK, we must use a compatibility banner as follows:

import { OutputFormat } from 'aws-cdk-lib/aws-lambda-nodejs' 

new lambdaNodejs.NodejsFunction(scope, id, { 
  bundling: { 
    format: OutputFormat.ESM, 
    banner: [ 
      `import { createRequire } from 'module';`, 
      `import { fileURLToPath } from 'node:url';`, 
      `import path from 'node:path';`, 
      `const __filename = fileURLToPath(import.meta.url);`, 
      `const __dirname = path.dirname(fileURLToPath(import.meta.url));`, 
      `const require = createRequire(import.meta.url);`, 
    ].join(' '), 
    mainFields: ['module', 'main'], 
  } 
}) 

And a tsconfig.json as follows: 

{ 
  "compilerOptions": {
    "typeRoots": ["./node_modules/@types", "./lib/types"], 
    "target": "es2022",
    "lib": ["es2022"], 
    "module": "node16", 
    "moduleResolution": "node16", 
    "esModuleInterop": true, 
    "isolatedModules": true, 
    "resolveJsonModule": true, 
    "skipLibCheck": true, // This might not be needed if your libs behave 
    "outDir": "build.out", 
    "strict": true, 
    "alwaysStrict": true, 
    "strictPropertyInitialization": false, 
    "sourceMap": true, 
    "incremental": true, 
    "declaration": true, 
    "inlineSources": true, 
    "strictNullChecks": true, 
    "experimentalDecorators": true, 
    "noEmitOnError": true, 
    "noImplicitAny": true,  
    "noImplicitThis": true, 
    "noImplicitReturns": true, 
    "noUnusedLocals": true, 
    "noUnusedParameters": true, 
    "noUncheckedIndexedAccess": true, 
    "noFallthroughCasesInSwitch": true
 }
}

Webassembly (Wasm)

Webassembly is also getting traction and for example Docker is betting on it so heavily they even updated their logo for a while. I’ll just let Docker to explain what WASM is:

WebAssembly, often shortened to Wasm, is a relatively new technology that allows you to compile application code written in over 40+ languages (including Rust, C, C++, JavaScript, and Golang) and run it inside sandboxed environments. The original use cases were focused on running native code in web browsers.

NPM ecosystem has matured

After all the left-pad meltdowns and developers occasionally pulling their popular modules from the NPM registry, the ecosystem has matured to a point where things just work. Sometimes, updating to new major versions of libraries can still be a pain, there's no denying that. But, in general, things do not break when you stick with the same versions, and they receive security patches.

It makes sense to update your production application regularly to use the most recent versions of its dependencies. This ensures you can take baby steps and refer to Changelogs’ migration guides. Accumulating several years of upgrade debt can make it painful and tedious to identify precisely where changes are needed to migrate the application for compatibility with the latest versions.

Dependabot and Renovate

Dependabot and Renovate are GitHub-based services that maintain the application dependencies automatically, easening the burden of keeping the application up to date.

The bots now help us with repetitious tasks like they should. We have them update modules as new ones are published. Additionally, when our automated tests succeed to ensure everything works, bots can merge and patch minor upgrades automatically. Major updates still require a developer review. We deploy regularly to be able to rollback effortlessly should there be any issues, and we know what changes can be suspected since each changeset is small.

AWS Node.js 16 and Node.js 18 runtime support

AWS publishing Node.js 16 and Node.js 18 Lambda runtime support quite quickly after these versions entered long-term support (LTS) has been a welcomed improvement. With Node.js 20 entering the LTS phase in October 2023, we can anticipate that AWS may release support for Node.js 20 on Lambda as early as 2023.

Yarn v3

Yarn’s v3 version has provided us with quick and reliable dependency graph and installation. We run it with the “nodeLinker: node-modules" setting. I can warmly recommend the version.

Internet Explorer 11 killed

With the last IE “killed” in June 2022, we’ve been able to cut down on some hacks needed earlier. While we continue to use polyfills in public online services, in most recent admin interfaces, we can already limit the support to most recent browser versions more rapidly, thus cutting the legacy compatibility layers. Meanwhile, the current browser engine share is not very healthy with Chromium ruling, but maybe that’s a subject for another article.

Security landscape

One more theme I’d like to bring up is artificial intelligence. While AI can be used to detect unexpected and malicious activity heuristically, it is also used to create new kinds of attacks to online services.

Once a security patch is released, the window of opportunity for exploitation is expected to decrease, as AI can be employed both for reverse engineering the patch and utilizing it as an attack vector. It won’t take long until AI can provide various permutations to find new exploits and vulnerabilities. Over time, this will make services more secure than before – it only means a higher degree of automation and an increased maintenance need for online services.

A "fire and forget" deployment strategy for services is becoming increasingly irresponsible, as is maintenance that is purely reactive without a proactive approach.