I've created an Advanced CDK course, you can check it out here!

BlogResumeTimelineGitHubTwitterLinkedInBlueSky

Deno vs Node showdown!

Cover Image for Deno vs Node showdown!

Update [2020-07-11]: Due to some feedback on Hackernews about cold-starts, I went ahead and updated the Deno deployment so that it didn't download dependencies at the start. Please see the Cold Starts and Deno below.

Deno is hot. Aimed to solve some of the more painful aspects of Node, Deno has taken the community by storm and a lot of people are looking to replace their Node-based workloads.

For me, I mostly use Node for Lambda functions. I wanted to see if Deno could replace it. Since Deno has a totally different packaging system that means that I have to wait until AWS clients are written or write them myself. There is a Deno library for DynamoDB, though. Combine that with the Lambda Layer that has been created for the community and I could get started easily enough. But, is it good?

Mainly, is it performant? I started by setting up two Lambda functions, one written for Node and one written for Deno. Each function did the same thing, wrote a DynamoDB record and then read it back. I created a third Lambda, written for Node, that would execute the other two Lambdas at a specified rate. I'd then monitor performance.

Disclaimer

Let's get this out of the way: Node and the AWS SDK are optimized. They've been around for quite a while and have gotten quite good, as you will soon see. Deno and the library used here have probably not undergone the same level of optimization. That being said, I still wanted to find out if it's worth adopting right now.

The Test

I ran the tester Lambda, asking for 5 minutes of calls every 500ms. I assumed that this would allow the runtimes to 'settle' after the cold-start and I could see representative duration metrics. This was my primary concern: what's the p99 duration of the Lambda function execution.

The results are largely what I expected:

Duration Comparisons

The Node lambda would execute around 40ms, which is quite solid. This was with the AWS_NODEJS_CONNECTION_REUSE_ENABLED environment variable. Init durations were around 350ms. Increasing memory to 1024 didn't improve either metric signficanly.

The Deno lambda averaged a 250ms duration. Worse, the init durations were in the 3.5s range. The main reason for this is the Deno runtime currently downloads external packages on cold-start. There are probably some mechanisms to speed this up, but that's for another post.

Deno getting packages

So, obviously, this is bad. Getting a 6x increase in moving from Node to Deno is not real appealing. But, does this mean Deno shouldn't be adopted?

Conclusion and Thoughts

Deno is hot. And for good reason. If you've ever worked with Node and the node_modules directory you've probably dealt with a packaging system that has a lot to be desired. Additionally, it's pretty insecure by default, allowing any codebase full access to the file system and network. This has been a recurring problem, usually requiring tools and diligent maintainers to catch problems after they've occurred. Deno aimed to solve both of these problems with a de-centralized package ecosystem and a safe-by-default runtime, requiring explicit parameters to enable access to the network and file system. Oh, and let's not forget 1st class Typescript support!

So, with these numbers, is Deno ready? For API Gateway integration, I'd say no. The cold-start and runtime performance just isn't there yet for backing client-facing API calls. But, what about other situations? Backing API Gateway isn't the only use-case for Deno. In a lot of cases Lambda functions are used for listening to event streams, responding to Step Function calls, or just a nightly background data processing process. In those cases, having a slower runtime may not be an issue (other than the additional billing costs). And, this is where a microservice architecture and highly decoupled systems really shines. You could easily adopt Deno for a few functions that don't need low latency; get used to the runtime and flesh out your development pipeline around it with low risk to your overall project (if it doesn't work, it's hopefully small enough you can just re-implement with Node).

Personally, I'm really excited about Deno. Node has baggage and some of that baggage is not easy to carry. It took years before Node was ready for production and adopted in those environments. Thanks to a vastly different landscape now, I think Deno will be in production much quicker. As such, jump in now where you can and get used to it, cause if you're running Node now I'm willing to bet you'll be running Deno in some capacity in 2 years.

Code

Btw, if you'd like the code so you can run these tests yourself, they're available here, as (what else?!) a CDK module.

Cold Starts and Deno

Someone posted this blog to Hackernews and many comments were around not bundling the dependencies. So, I went back and bundled them. The results are a little bit shocking.

First, here is an example of the logs of a cold-start without bundling:

2020-07-11T14:17:53.181-06:00
START RequestId: fe006316-6004-4fc3-ab41-2fe1b1c17c19 Version: $LATEST
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.412-06:00
warn: unable to import '.deno_dir/' as DENO_DIR
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.414-06:00
Download https://denopkg.com/chiefbiiko/dynamodb/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.684-06:00
Download https://deno.land/x/random/Random.js
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.817-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.817-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.904-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/util.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.905-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/aws_signature_v4.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.905-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/base_op.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.905-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/derive_config.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.905-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/create_headers.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:53.972-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/translator.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.042-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/deps.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.042-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/base_fetch.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.046-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/api/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.049-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/converter.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.050-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/client/create_cache.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.050-06:00
Download https://denopkg.com/chiefbiiko/std-encoding@v1.0.0/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.051-06:00
Download https://denopkg.com/chiefbiiko/get-aws-config@v1.0.1/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.051-06:00
Download https://denopkg.com/chiefbiiko/hmac@v1.0.2/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.051-06:00
Download https://deno.land/x/base64@v0.2.0/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.127-06:00
Download https://denopkg.com/chiefbiiko/sha256@v1.0.2/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.137-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/api/api.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.142-06:00
Download https://raw.githubusercontent.com/chiefbiiko/std-encoding/v1.0.0/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.172-06:00
Download https://raw.githubusercontent.com/chiefbiiko/sha256/v1.0.2/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.208-06:00
Download https://deno.land/x/base64@v0.2.0/base.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.208-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/api/collection.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.208-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/api/operation.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.212-06:00
Download https://raw.githubusercontent.com/chiefbiiko/dynamodb/master/api/shape.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.213-06:00
Download https://raw.githubusercontent.com/chiefbiiko/hmac/v1.0.2/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.227-06:00
Download https://raw.githubusercontent.com/chiefbiiko/get-aws-config/v1.0.1/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.230-06:00
Download https://raw.githubusercontent.com/chiefbiiko/sha256/v1.0.2/deps.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.306-06:00
Download https://deno.land/x/base64/base64url.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.334-06:00
Download https://raw.githubusercontent.com/chiefbiiko/hmac/v1.0.2/deps.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.397-06:00
Download https://deno.land/x/base64/base.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.398-06:00
Download https://denopkg.com/chiefbiiko/sha1@v1.0.3/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.498-06:00
Download https://denopkg.com/chiefbiiko/sha512@v1.0.3/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.498-06:00
Download https://raw.githubusercontent.com/chiefbiiko/sha512/v1.0.3/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.581-06:00
Download https://raw.githubusercontent.com/chiefbiiko/sha1/v1.0.3/mod.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.591-06:00
Download https://raw.githubusercontent.com/chiefbiiko/sha1/v1.0.3/deps.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.679-06:00
Download https://raw.githubusercontent.com/chiefbiiko/sha512/v1.0.3/deps.ts
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:54.679-06:00
Compile file:///tmp/runtime.js
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:57.181-06:00
INFO RequestId: fe006316-6004-4fc3-ab41-2fe1b1c17c19 Write db record in 126
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:57.303-06:00
INFO RequestId: fe006316-6004-4fc3-ab41-2fe1b1c17c19 Read db record in 121
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:57.303-06:00
INFO RequestId: fe006316-6004-4fc3-ab41-2fe1b1c17c19 { results: "8003122", stopwatch: 248 }
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:57.307-06:00
END RequestId: fe006316-6004-4fc3-ab41-2fe1b1c17c19
2020/07/11/[$LATEST]127bc33ce7464d8ba6662956f5097684

2020-07-11T14:17:57.307-06:00
REPORT RequestId: fe006316-6004-4fc3-ab41-2fe1b1c17c19	Duration: 256.27 ms	Billed Duration: 4200 ms	
Memory Size: 1024 MB	Max Memory Used: 132 MB	Init Duration: 3894.50 ms

Notice the Initial Duration at the end that's nearly 4 seconds. This is what everybody apparently thought was making it into the averages (despite me working hard to ensure the Lambda was warm during those calculations). I went ahead and bundled the dependencies using the Lambda Layer's docs and here are the logs from a cold-start with the deps bundled:

2020-07-11T14:35:40.798-06:00
START RequestId: 0e1752fc-cde2-453e-90ad-8eac61913e7d Version: $LATEST
2020/07/11/[$LATEST]4893f1ee5d1d40a08fed5eece82659a4

2020-07-11T14:35:40.798-06:00
Compile file:///tmp/runtime.js
2020/07/11/[$LATEST]4893f1ee5d1d40a08fed5eece82659a4

2020-07-11T14:35:43.288-06:00
INFO RequestId: 0e1752fc-cde2-453e-90ad-8eac61913e7d Write db record in 131
2020/07/11/[$LATEST]4893f1ee5d1d40a08fed5eece82659a4

2020-07-11T14:35:43.419-06:00
INFO RequestId: 0e1752fc-cde2-453e-90ad-8eac61913e7d Read db record in 132
2020/07/11/[$LATEST]4893f1ee5d1d40a08fed5eece82659a4

2020-07-11T14:35:43.421-06:00
INFO RequestId: 0e1752fc-cde2-453e-90ad-8eac61913e7d { results: "8289138", stopwatch: 263 }
2020/07/11/[$LATEST]4893f1ee5d1d40a08fed5eece82659a4

2020-07-11T14:35:43.423-06:00
END RequestId: 0e1752fc-cde2-453e-90ad-8eac61913e7d
2020/07/11/[$LATEST]4893f1ee5d1d40a08fed5eece82659a4

2020-07-11T14:35:43.423-06:00
REPORT RequestId: 0e1752fc-cde2-453e-90ad-8eac61913e7d Duration: 271.45 ms Billed Duration: 3000 ms 
Memory Size: 1024 MB Max Memory Used: 131 MB Init Duration: 2658.55 ms

Notice this time there is no downloading of external dependencies. Yet, there is still an almost 2.7s init duration! I would have thought that including the dependencies would have reduced that init time more than a few hundred milliseconds.

Furthermore, you can see with the stopwatches I put around the two DynamoDB calls that they're both taking 100ms+, with the total read/write cycle taking 263ms, even with deps bundled. Even if they weren't bundled, the stopwatch was 248ms. So, bundling has no affect on the runtime performance of the handler, only on the initialization of the function, the cold-start time, and even then it's still not that impressive a gain.

Science

This is my attempt at science. I welcome peer reviews and if my results can't be duplicated, I'd like to know it. If they can be improved, I'd like to know it. Open Issues if you see a problem.