How to get rid of unwanted requests to your Express server? If you don't protect your backend against spam/bots, then bots can make tons of requests and might steal the data from your database.

Requisite

To better understand this post, you should have basic knowledge of NodeJS and ExpressJS.

We will use cors to hammer the bots that might take advantages of unprotected routes in our express server.

Install the Modules

Use these commands to install the modules and make sure you have NodeJS installed in your system.

npm install express@4.17.1 cors@2.8.5 express-rate-limit@5.2.6

Create a basic express server

Let's create a simple express server and one simple GET method route.

// server.js

const express = require("express")
const app = express()

app.get("/", (req, res) => {
  res.send("Hi there!")
})

const PORT = 5000

app.listen(PORT, console.log(`Server running on port ${PORT}`))

Then start the server by running the command node server.

Let's Block the spam API requests

Now we need to allow requests to the server only for the domains we allow for, the rest of the domains that send requests will be blocked by cors. Let's see it in action. We will bind cors to express middleware and we will configure cors options. Create a file corsOptions.js and we need to configure it as per our wish to block unwanted domains/bots.

// server.js

const express = require("express")
const cors = require("cors")

const app = express()

// Enable CORS with only allowed domains
app.use(cors(corsOptions()))

app.get("/", (req, res) => {
  res.send("Hi there!")
})

const PORT = 5000

app.listen(PORT, console.log(`Server running on port ${PORT}`))

Configure CORS options

We will only allow our frontend domains' requests to the server and other domains will be blocked.

//corsOptions.js

const _DEV_ENV_ = process.env.NODE_ENV !== "production"

const allowedDomainsStr =
  "https://frontend-site-1.com,https://frontend-site-2.com"

let corsDomains = []
corsDomains = (allowedDomainsStr && allowedDomainsStr.split(",")) || []
corsDomains = corsDomains.map(domain => {
  let tempDomain = domain.trim()
  tempDomain = tempDomain.replace(/\/+$/, "")
  return tempDomain
})

// only for development node environment
if (_DEV_ENV_) {
  const devOrigins = [
    "undefined",
    "http://localhost:3000",
    "http://localhost:3001",
  ]
  devOrigins.map(item => corsDomains.push(item))
}

//our actual cors configuration
const corsOptions = () => {
  const whitelist = [...corsDomains]

  const corsOptions = {
    origin: (origin, callback) => {
      if (whitelist.indexOf(`${origin}`) !== -1) {
        callback(null, true)
      } else {
        callback(new Error("Not allowed by CORS"))
      }
    },
  }
  return corsOptions
}

module.exports = corsOptions

Code explanation

allowedDomainsStr - you can mention your frontend sites if they need to make API requests to this server. Notice that, each domain should be separated by comma (,) from each other. You should use nodejs environment variable for this in production.

Then we need to split it and sanitize each of the domain to get rid of trailing slash. Use regular expression to replace any trailing slash with an empty string- string.replace(/\/+$/, '').

devOrigins - It's only for the development environment. We should allow our frontend development origins i.e. localhost so that we can test the server in development. If we need to test it out with Postman or from the terminal, then we should add the origin undefined because they don't have origins. If you make a request directly typing the server URL into the browser address field, then its origin is undefined too.

whitelist - It is the array of domains that only can make requests to our server.

corsOptions - It's the main method that has business logic to block unnecessary requests. Here we need to check if the request is coming from our domains in whitelist or not. We check the compare incoming request's origin with whitelist, If it exists, then we invoke callback(null, true) by passing true as its second argument. If the origin is not in our whitelist i.e. it's a spam API request, then block it by throwing an error in callback. Voila, that's how we block the spam requests.

What if we want to block some API requests which are coming from our allowed domains, we can't trust that all users who use our frontend sites, won't spam our server - Users can repetitively interact with some components even if they don't need to do it repetitively. For example, Our frontend site has a Form, it submits the data to our server upon click on Submit button, If a user clicks the Submit button for like 50 times in a minute, then our server will receive 50 requests too, It means it will make our server consume more resource (say memory, etc), you would pay more to your service provider that hosts your server, for consuming more resources or our server will crash for getting tons of requests in a short period.

Well, you can say you can prevent multiple form submission on a frontend site but people who know how to use browser dev tools can see your server URL and payload, etc. on the network tab in dev tools. They can make tons of request to your server using a loop statement. So how do you block these spams? Allow only a certain number of requests for a given period - For example 30 API requests per minute per IP address/user. And, block the rest of the requests once this limit is reached.

Let's block too many API requests

We will use express-rate-limit to block excessive API requests. Create a customRateLimiter.js and bind it to express middleware. Note that, our rate limiter middleware should be present after cors middleware. Cors middleware will block the requests coming from unwanted domains and it will return the response right away. In this case, the request won't go through rate limiter middleware. if the request is coming from allowed origins then it will go through cors, then through rate limiter, then through the rest of things present below rate limiter.

// server.js

const express = require("express")
const cors = require("cors")

const corsOptions = require("./corsOptions")
const customRateLimiter = require("./customRateLimiter")

const app = express()

// Enable CORS with only allowed domains
app.use(cors(corsOptions()))

// Rate limiting
app.use(customRateLimiter())

app.get("/", (req, res) => {
  console.log("/")
  res.send("ok")
})

const PORT = 5000

app.listen(PORT, console.log(`Server running on port ${PORT}`))

Configure Rate Limiter

Let's write a custom rate limiter with the help of module express-rate-limit.

//customRateLimiter.js

const rateLimit = require("express-rate-limit")

const _DEV_ENV_ = process.env.NODE_ENV !== "production"
const ENV_DEV_RATE_LIMIT = Infinity

const ADMIN_RATE_LIMIT = 100

let adminPanelOrigins =
  "http://localhost:3001,http://localhost:3002,https://www.google.com"

adminPanelOrigins = (adminPanelOrigins && adminPanelOrigins.split(",")) || []

let allowedDomains = []

adminPanelOrigins.map(domain => {
  let tempDomain = (domain && domain.trim()) || ""
  tempDomain = tempDomain.replace(/\/+$/, "")

  if (!!tempDomain) {
    allowedDomains.push(tempDomain)
  }
})

const customRateLimiter = () => {
  const limiter = rateLimit({
    windowMs: 1 * 60 * 1000, // 1 mins
    max: _DEV_ENV_ ? ENV_DEV_RATE_LIMIT : 30, //max 30 req per 1 mins in production for non-admin origins
    // handle requests once the max limit is exceeded, allow some more requests for admin panel origins as per ADMIN_RATE_LIMIT
    handler: (req, res, next) => {
      let origin = req.headers.origin
      // current = current requests number
      // limit = rate limit given in "max"
      const { limit, current } = req.rateLimit

      const shouldAllowAdmins = current <= ADMIN_RATE_LIMIT

      // for admin origins check
      if (allowedDomains.indexOf(`${origin}`) !== -1 && shouldAllowAdmins) {
        //admin origins are allowed!
        next()
      } else if (req.originalUrl.includes("/api/v1/orders/stripe/confirm")) {
        // allow stripe webhook endpoint

        next()
      } else {
        res.status(429).send("Too many requests, please try again later.")
      }
    },
  })

  return limiter
}

module.exports = customRateLimiter

Code Explanation

ENV_DEV_RATE_LIMIT - It's for the development environment. Since you don't want to limit your request to the server in development, set it to Infinity.

Suppose you have a user-facing frontend web application and one admin dashboard, both make requests to this server. Your admin dashboard can make more request to the server- for example, around 100 per minute but you want to set a different rate limit to the frontend user-facing web app, how to accomplish this?

adminPanelOrigins - It's the domains separated by comma (,) that can have more rate limit than a user-facing frontend web application. You can have a NodeJS environment variable for this.

ADMIN_RATE_LIMIT - We want to allow the admin domains to make a maximum of 100 requests per minute.

We will split the adminPanelOrigins into an array and map over it and remove any trailing slashes if present, and add the domains to array allowedDomains.

customRateLimiter- This is the main method that has business logic for blocking excessive requests. It returns a function rateLimit which accept an object as an argument.

Let's go through the argument:

windowMs - How long the request will stay in memory. Its unit is milliseconds. For example, if we want only 30 requests per 1 minute, then windowMs will be 1 * 60 * 1000. if we want only 30 requests per 2 minute, then windowMs will be 2 * 60 * 1000.

max - It's the maximum number of requests allowed during windowMs. If any request exceeds this limit then the request will be blocked or an error response will be returned in handler method.

handler - This function is executed once the max limit is reached. Here we will have to check if the request is coming from our admin dashboard or frontend user-facing application.

if we destructure req.rateLimit, we will get limit and current. current is the current request number. We will compare it with ADMIN_RATE_LIMIT.

If the request is coming from the admin dashboard, then check whether it has reached the limit ADMIN_RATE_LIMIT. if it has not reached the limit, then we will allow it by invoking next(). If the limit has exceeded then we will block/reject the request with response status code 429 and a message Too many requests, please try again later..

NOTE

If you use some service's webhook endpoints that make requests to your server, you can allow all of their requests. For example, I was building an e-commerce application, the stripe can make webhook requests once a buyer makes a purchase. So I allowed all the requests of stripe coming through our server route /api/v1/orders/stripe/confirm. if I had a limit for the stripe webhook route what would happen, when a lot of users make purchases in a minute- say 200 or 300 or 150 purchases in one minute? The requests which have exceeded the limit would have blocked and the stripe won't able to make contact with the server, right?.

You can add your custom logic as per your need. cool, right?

When I was building the e-commerce web application a few years back and I wanted some help regarding some advanced configuration for cors and express rate limiter, there was not clear info available on google. So I had to dig the documentation of cors and express-rate-limit and played around with different options. I was finally able to choose which methods or modules were correct for me. Now I just shared what I figured out back then on this post. I would be sharing my own practical experiences more on this blog.