React ESI: Blazing-fast Server-Side Rendering for React and Next.js

Overview

React ESI: Blazing-fast Server-Side Rendering for React and Next.js

Build Status Coverage Status npm version MIT Licence

React ESI is a super powerful cache library for vanilla React and Next.js applications, that can make highly dynamic applications as fast as static sites. It provides a straightforward way to boost your application's performance by storing fragments of server-side rendered pages in edge cache servers. It means that after the first rendering, fragments of your pages will be served in a few milliseconds by servers close to your end users! It's a very efficient way to improve the performance and the SEO of your websites; and to dramatically reduce both your hosting costs and the energy consumption of these applications. Help the planet, use React ESI!

Because it is built on top of the Edge Side Includes (ESI) W3C specification, React ESI natively supports most of the well-known cloud cache providers including Cloudflare Workers, Akamai and Fastly. Of course, React ESI also supports the open source Varnish cache server that you can use in your own infrastructure for free (configuration example).

Also, React ESI allows to specify a different Time To Live (TTL) per React component and to generate the corresponding HTML asynchronously using a secure (signed) URL. The cache server fetches and stores in the cache all the needed fragments (the HTML corresponding to every React component), builds the final page and sends it to the browser. React ESI also allows components to (re-)render client-side without any specific configuration.

ESI example

Schema from The Varnish Book

Discover React ESI in depth with this presentation

Examples

Install

Using Yarn:

$ yarn add react-esi

Or using NPM:

$ npm install react-esi

Usage

React ESI provides a convenient Higher Order Component that will:

  • replace the wrapped component by an ESI tag server-side (don't worry React ESI also provides the tooling to generate the corresponding fragment);
  • render the wrapped component client-side, and feed it with the server-side computed props (if any).

React ESI automatically calls a static async method named getInitialProps() to populate the initial props of the component. Server-side, this method can access to the HTTP request and response, for instance, to set the Cache-Control header, or some cache tags.

These props returned by getInitialProps() will also be injected in the server-side generated HTML (in a <script> tag). Client-side the component will reuse the props coming from the server (the method will not be called a second time). If the method hasn't been called server-side, then it will be called client-side the first time the component is mounted.

The Higher Order Component

// pages/index.js
import React from 'react';
import withESI from 'react-esi';
import MyFragment from 'components/MyFragment';

const MyFragmentESI = withESI(MyFragment, 'MyFragment');
// The second parameter is an unique ID identifying this fragment.
// If you use different instances of the same component, use a different ID per instance.

const Index = () => (
  <div>
    <h1>React ESI demo app</h1>
    <MyFragmentESI greeting="Hello!" />
  </div>
);
// components/MyFragment.js
import React from 'react';

export default class MyFragment extends React.Component {
  render() {
    return (
      <section>
        <h1>A fragment that can have its own TTL</h1>

        <div>{this.props.greeting /* access to the props as usual */}</div>
        <div>{this.props.dataFromAnAPI}</div>
      </section>
    );
  }

  static async getInitialProps({ props, req, res }) {
    return new Promise(resolve => {
      if (res) {
        // Set a TTL for this fragment
        res.set('Cache-Control', 's-maxage=60, max-age=30');
      }

      // Simulate a delay (call to a remote service such as a web API)
      setTimeout(
        () =>
          resolve({
            ...props, // Props coming from index.js, passed through the internal URL
            dataFromAnAPI: 'Hello there'
          }),
        2000
      );
    });
  }
}

The initial props must be serializable using JSON.stringify(). Beware Map, Set and Symbol!

Note: for convenience, getInitialProps() has the same signature than the Next.js one. However, it's a totally independent and standalone implementation (you don't need Next.js to use it).

Serving the Fragments

To serve the fragments, React ESI provides a ready to use controller compatible with Express:

// server.js
import express from 'express';
import { path, serveFragment } from 'react-esi/lib/server';

const server = express();
server.use((req, res, next) => {
  // Send the Surrogate-Control header to announce ESI support to proxies (optional with Varnish, depending of your config)
  res.set('Surrogate-Control', 'content="ESI/1.0"');
  next();
});

server.get(path, (req, res) =>
  // "path" default to /_fragment, change it using the REACT_ESI_PATH env var
  serveFragment(
    req,
    res,
    // "fragmentID" is the second parameter passed to the "WithESI" HOC, the root component used for this fragment must be returned
    fragmentID => require(`./components/${fragmentID}`).default) 
);

// ...
// Other Express routes come here

server.listen(80);

Alternatively, here is a full example using a Next.js server:

// server.js
import express from 'express';
import next from 'next';
import { path, serveFragment } from 'react-esi/lib/server';

const port = parseInt(process.env.PORT, 10) || 3000
const dev = process.env.NODE_ENV !== 'production'
const app = next({ dev })
const handle = app.getRequestHandler()

app.prepare().then(() => {
  const server = express();
  server.use((req, res, next) => {
    // Send the Surrogate-Control header to announce ESI support to proxies (optional with Varnish)
    res.set('Surrogate-Control', 'content="ESI/1.0"');
    next();
  });

  server.get(path, (req, res) =>
    serveFragment(req, res, fragmentID => require(`./components/${fragmentID}`).default)
  );
  server.get('*', handle); // Next.js routes

  server.listen(port, err => {
    if (err) throw err;
    console.log(`> Ready on http://localhost:${port}`);
  });
});

Features

  • Support Varnish, Cloudflare Workers, Akamai, Fastly and any other cache systems having ESI support
  • Written in TypeScript
  • Next.js-friendly API

Environment Variables

React ESI can be configured using environment variables:

  • REACT_ESI_SECRET: a secret key used to sign the fragment URL (default to a random string, it's highly recommended to set it to prevent problems when the server restart, or when using multiple servers)
  • REACT_ESI_PATH: the internal path used to generate the fragment, should not be exposed publicly (default: /_fragment)

Passing Attributes to the <esi:include> Element

To pass attributes to the <esi:include> element generated by React ESI, pass a prop having the following structure to the HOC:

{
  esi: {
    attrs: {
      alt: "Alternative text",
      onerror: "continue"
    }
  }
}

Troubleshooting

The Cache is Never Hit

By default, most cache proxies, including Varnish, never serve a response from the cache if the request contains a cookie. If you test using localhost or a similar local domain, clear all pre-existing cookies for this origin. If the cookies are expected (e.g.: Google Analytics or ad cookies), then you must configure properly your cache proxy to ignore them. Here are some examples for Varnish.

Design Considerations

To allow the client-side app to reuse the props fetched or computed server-side, React ESI injects <script> tags containing them in the ESI fragments. After the assembling of the page by the cache server, these script tags end up mixed with the legit HTML. These tags are automatically removed from the DOM before the rendering phase.

Going Further

React ESI plays very well with advanced cache strategies including:

  • Cache invalidation (purge) with cache tags (Varnish / Cloudflare)
  • Warming the cache when data change in the persistence layer (Varnish)

Give them a try!

Vue.js / Nuxt

We love Vue and Nuxt as much as React and Next, so we're a currently porting React ESI for this platform. Contact us if you want to help!

Credits

Created by Kévin Dunglas. Sponsored by Les-Tilleuls.coop.

Issues
  • fix: stream props

    fix: stream props

    Stream the props instead of write. Use case where props are too large (100 object).

    opened by jamyouss 8
  • fix: content truncated because response end is called too early

    fix: content truncated because response end is called too early

    In some case the res.end is called too early. We should let the pipe end the writer when the reader ends.

    Tested with large ESI. The transformation is working but the last part is not sent.

    opened by jamyouss 7
  • Very large ESI are truncated

    Very large ESI are truncated

    Hello,

    First thanks for you work and in particular this implementation of ESI with React (and Next.js in my case).

    I am currently facing an issue with contents that have very large size (about 2mb), generated HTML from ESI is missing some ending tags. I have a Content component that is used to create the HOC (ContentESI). This component calls an API to get its data in getInitialProps. The return of API is complete, the props in window.__REACT_ESI__ are complete. Content component is using many others components to render parts of the API response. When I render Content without ESI, generated HTML is complete. But when I use the HOC, generated HTML is missing data at the end (so in my case, closing tags).

    My knowledge in Node.js & React are very limited, I don't know where to start. I think that it is an issue with Stream, Transform or renderToNodeStream (based on my reading of server.tsx).

    I have done tests with:

    • Node.JS 12.x, React 16.x, Next.js 9.5, React ESI 0.3
    • Node.JS 12.x, React 17.x, Next.js 9.5, React ESI 0.3
    • Node.JS 14.x, React 16.x, Next.js 9.5, React ESI 0.3
    • Node.JS 12.x, React 17.x, Next.js 9.5, React ESI 0.3
    • Node.JS 12.x, React 16.x, Next.js 9.5, React ESI 0.2
    opened by julienlary-dkt 6
  • cloudflare workers

    cloudflare workers

    This can be used to create an ssr application using cloudflare workers? I want React PWA, next.js, cloudflare workers

    opened by nikitapilgrim 5
  • Error with multiple esi-includes on one page

    Error with multiple esi-includes on one page

    Hello @dunglas

    Thanx for your great work!

    I have a php website where I want to include multiple esi-includes : <esi:include src="/esi/fragment?param=1" alt="Alternative text1" /> <esi:include src="/esi/fragment?param=2" alt="Alternative text2" />

    On the other hand I have nextjs in place with version 9.1.5 with customized server.js: server.js `const express = require('express') const next = require('next') const { path, serveFragment } = require('react-esi/lib/server')

    const dev = process.env.NODE_ENV !== 'production' const app = next({ dev }) const handle = app.getRequestHandler()

    app.prepare().then(() => { const server = express()

    server.use((req, res, next) => { // Send the Surrogate-Control header to announce ESI support to proxies (optional with Varnish) res.set('Surrogate-Control', 'content="ESI/1.0"') next() })

    server.get(path, (req, res) => serveFragment( req, res, fragmentID => require(./components/${fragmentID}).default ) ) server.get('*', handle)

    server.listen(3000, err => { if (err) throw err console.log(> Ready on http://localhost:3000) }) })`

    package.json { "name": "nextson", "version": "1.0.0", "private": true, "main": "dist/server.js", "scripts": { "dev": "node server.js", "build_old": "next build", "start_old": "next start", "build": "next build", "start": "NODE_ENV=production node server.js" }, "dependencies": { "@zeit/next-css": "^1.0.1", "@zeit/next-sass": "^1.0.1", "dotenv-webpack": "1.5.7", "express": "^4.17.1", "isomorphic-unfetch": "^3.0.0", "next": "^9.1.5", "nextjs-redirect": "^1.0.2", "node-sass": "^4.13.0", "react": "^16.9.0", "react-dom": "^16.9.0", "react-esi": "^0.2.0", "react-slick": "^0.25.2", "slick-carousel": "^1.8.1", "url-loader": "^2.2.0" }, "devDependencies": { "@babel/cli": "^7.6.0", "@babel/node": "^7.6.1" } }

    pages/hoc/fragment.js `import React from 'react'; import withESI from 'react-esi'; import Layout from '../../components/layout'; import myfragment from '../../components/myfragment';

    //const MyFragmentESI = withESI(myfragment, 'myfragment'); // The second parameter is an unique ID identifying this fragment. // If you use different instances of the same component, use a different ID per instance.

    class Fragment extends React.Component { static getInitialProps({query}) { return {query} }

    render() {
        console.log(this.props.query.param);
        const name = 'myfragment_'+this.props.query.param;
        console.log(name);
        const MyFragmentESI = withESI(myfragment, name);
        return(
          <Layout>
            <h1>React ESI demo app</h1>
            <MyFragmentESI greeting="test greeting!" />
          </Layout>
        );
    }
    

    }

    export default Fragment; `

    components/layout.js import Link from 'next/link'

    export default ({ children, title = 'This is the default title' }) => (

    {children}
    
    <footer>{'I`m here to stay'}</footer>
    
    )

    components/myfragment.js `import React from 'react';

    export default class myfragment extends React.Component {

    constructor(props) { super(props) this.click = this.click.bind(this) }

    click() {

    }

    render() { return (

    A fragment that can have its own TTL

    {this.props.greeting}
    {this.props.dataFromAnAPI}
    <button onClick={this.click}>Take the Shot!
    ); }

    static async getInitialProps({ props, req, res }) { if (res) { res.setHeader('Surrogate-Control','content="ESI/1.0"'); }

    return new Promise(resolve => {
      if (res) {
        // Set a TTL for this fragment
        res.setHeader('Cache-Control', 's-maxage=60, max-age=30');
      }
    
      // Simulate a delay (call to a remote service such as a web API)
      setTimeout(
        () =>
          resolve({
            ...props, // Props coming from index.js, passed through the internal URL
            dataFromAnAPI: 'Hello there'
          }),
        2000
      );
    });
    

    } } `

    varnish settings: /etc/varnish/default.vlc `

    backend nextjs1 { .host = "127.0.0.1"; .port = "3001"; .max_connections = 100; .first_byte_timeout = 800s; }

    sub vcl_recv { ... if ( req.url ~ "^/_next/") { set req.url = regsub(req.url, "^/nextjs/", "/"); set req.backend_hint = nextjs.backend(); return (hash); }

    if (req.url ~ "^/esi/") {
        set req.url = regsub(req.url, "^/esi/", "/hoc/");
        # Send Surrogate-Capability headers to announce ESI support to backend
        set req.http.Surrogate-Capability = "key=ESI/1.0";
        set req.backend_hint = nextjs.backend();
        return (hash);
    } 
    

    ... } sub vcl_backend_response {

    set beresp.do_esi = true;
    

    ... }

    ` Result: Screenshot 2019-12-16 at 23 00 16

    To u see by change what I am doing wrong? The ssr part is not working properly!

    getting following error: Screenshot 2019-12-16 at 23 01 37

    Thanx for your help! Would love to get your library into production

    Cheers Klausi

    opened by watzak 2
  • Fix typo

    Fix typo

    opened by CecileAmrl 1
  • Feature - add option to further pipe stream

    Feature - add option to further pipe stream

    Hi,

    This PR aims to expose the serveFragment stream to the library user. That is, an option to provide any extra pipes on the stream is provided. With this option, users can further transform the fragment output as desired. I created this PR because I needed to add rendered styles from emotion to the served fragment.

    Changes:

    • options I've added an options argument to serveFragments. I was assuming that more options could follow
    • options.pipeStream After removing the react root transform stream, it will check if option.pipeStream is provided and executes it.

    I'd be happy to implement any desired changes.

    opened by dpnolte 1
  • Feature - add typescript declarations

    Feature - add typescript declarations

    Hi,

    Thanks for this awesome library.

    With this PR, I'd like to add typings to the package.

    There were a couple of issues with typing withESI:

    • Return type - With declarations enabled, typescript generated an error: the esi property cannot be private/protected in the HOC class. That is, exported anonymous classes can't have private or protected members if declaration emit is enabled, because there's no way to represent that in a .d.ts file. I resolved it by providing an return type to withESI React.ComponentClass<P & IWithESIProps>
    • propTypes - The problem with this return type is that the static propTypes member did not account for child properties as provided by the generic variable P. This generated another typescript error. I resolved it by converting the propTypes member first to unknown and then to the proper WeakValidationMap type to account for HOC props and the children props.
    opened by dpnolte 1
  • Expose original component as public static member WrappedComponent

    Expose original component as public static member WrappedComponent

    Hi, this PR will add a public static property WrappedComponent (tests included) to access original component.

    It looks like a de facto standard in higher order components, at least that's what react-router and react-redux do, so I expected any higher order component to do the same. This allows to access original component in a standardized way and access its original static properties (in my case, a loadData property used for server-side data fetching, but that could be anything else).

    Thanks for your work!

    opened by naholyr 1
  • Feat: migrate to Github Actions

    Feat: migrate to Github Actions

    Related: #31

    opened by Fabious 5
  • Migrate to GitHub Actions

    Migrate to GitHub Actions

    Travis doesn't work anymore, we must migrate to GitHub Actions.

    hacktoberfest 
    opened by dunglas 0
  • Change README component from class to functional component

    Change README component from class to functional component

    This Pull Request only change in README.md ReactJs Example from Class Component to Functional Component.

    opened by jrmarqueshd 0
  • prop-types should be a dependency

    prop-types should be a dependency

    Regarding the official usage of prop-types: https://github.com/facebook/prop-types#how-to-depend-on-this-package, its needs to be a dependencies instead of devDependencies.

    I would propose a PR if you like.

    opened by bobaaaaa 1
  • next.js example was removed from upstream next.js project

    next.js example was removed from upstream next.js project

    The example, which is currently linked in the react-esi readme, no longer exists: https://github.com/zeit/next.js/blob/canary/examples/with-react-esi/

    Is there a reason this was removed? i.e., something about it doesn't work correctly? Or has something changed in next.js that makes react-esi incompatible with it?

    opened by milesrichardson 1
  • Is there any downsides of opting-out from default Next.js server?

    Is there any downsides of opting-out from default Next.js server?

    Is there any downsides of opting-out from default Next.js server?

    On Next.js website it's stated:

    Before deciding to use a custom server please keep in mind that it should only be used when the >integrated router of Next.js can't meet your app requirements. A custom server will remove >important performance optimizations, like serverless functions and Automatic Static Optimization.

    opened by maximignatev 0
  • Handling complex project structure

    Handling complex project structure

    Two pieces of the README first:

    withESI usage:

    const MyFragmentESI = withESI(MyFragment, 'MyFragment');
    // The second parameter is an unique ID identifying this fragment.
    // If you use different instances of the same component, use a different ID per instance.
    

    serving the fragments:

    // "fragmentID" is the second parameter passed to the "WithESI" HOC, the root component used for this fragment must be returned
    fragmentID => require(`./components/${fragmentID}`).default) 
    

    First though is that the fragmentID is more than just an ID, from this example it seems like it needs to match the filename for the dynamic require to work properly. If I end up using different ID per instance as suggested I don't know how I'm gonna map the fragmentID to the relevant import path.

    I'm working with a very complex project structure with custom resolvers using environment variables so I don't have a flat /components/* folder.

    Is there a way to build a map fragmentID => requirePath before starting the server ?

    I've seen such algorithm with gettext-extractor where we look for every call of the "getText" function in the source files. Something like this:

    JsExtractors.callExpression(['getText'], {
        arguments: {
          text: 0,
          context: 1,
        },
      })
    

    We could look for every usage of withESI maybe ?

    Maybe I'm missing something obvious to solve this, let me know

    opened by VincentCharpentier 1
  • How to get and serve HTML fragments?

    How to get and serve HTML fragments?

    I was wondering how can I render an HTML fragment which is not a component...

    For example: Let's say that example.com/esi-fragment is serving

    <div> content </div>
    

    So, as per ESI i can do something like this in an example.com page

    <esi:include src="/esi-fragment"/>
    

    And I should see the div rendered in my page.

    In all the examples I see online I only can see react components rendered... Which is working, but not really what I want to do.... Is there something I'm getting wrong?

    Thanks!

    opened by napolux 3
  • Context in getInitialProps

    Context in getInitialProps

    Context is not passed to catch store for ssr?

    hacktoberfest 
    opened by aiwbend 0
Owner
Kévin Dunglas
Founder of @coopTilleuls / Creator of @api-platform, Mercure.rocks and Vulcain.rocks / @symfony Core Team
Kévin Dunglas
HTML meta tags for React-based apps. Works for both client- and server-side rendering, and has a strict but flexible API.

React Document Meta HTML meta tags for React-based apps. Works for both client- and server-side rendering, and has a strict but flexible API. Built wi

kodyl 320 Nov 15, 2021
Adds server side rendering support to React Relay

Isomorphic React Relay Enables server-side rendering of React Relay containers. If you use react-router-relay you might also become interested in isom

Denis Nedelyaev 247 Nov 13, 2021
Build blazing fast, modern apps and websites with React

Gatsby v2 ⚛️ ?? ?? Fast in every way that matters Gatsby is a free and open source framework based on React that helps developers build blazing fast w

Gatsby 51.8k Nov 24, 2021
An interactive CLI automation tool 🛠️ for creating react.js and next.js projects most fast and efficiently. ⚛️

An interactive CLI automation tool ??️ for creating react.js and next.js projects most fast and efficiently. ⚛️

Alexis Guzman 26 Nov 9, 2021
Redux bindings for client-side search

redux-search Higher-order Redux library for searching collections of objects. Search algorithms powered by js-worker-search. Check out the live demo a

Brian Vaughn 1.4k Nov 17, 2021
A starter for React with Typescript with the fast Vite and all static code testing with Eslint and formatting with Prettier.

A starter for React with Typescript with the fast Vite and all static code testing with Eslint and formatting with Prettier.

The Sword Breaker 79 Nov 20, 2021
Complete token authentication system for react + redux that supports isomorphic rendering.

Simple, secure authentication for react + redux TL;DR - View the Live Demo You can see a complete working example here. The code for the demo is here.

Lynn Dylan Hurley 2.2k Nov 19, 2021
Our SDK for the 3D rendering platform React-Three-Fiber.

Zappar for React Three Fiber This library allows you use Zappar's best-in-class AR technology with content built using the 3D rendering platform React

Zappar 2 Nov 8, 2021
An example using universal client/server routing and data in React with AWS DynamoDB

react-server-routing-example A simple (no compile) example of how to do universal server/browser rendering, routing and data fetching with React and A

Michael Hart 298 Sep 15, 2021
A curated list of resources to learn React and related web technologies as fast as possible.

React Learning Resources A curated list of resources to learn React and related web technologies as fast as possible. The goal is to help you create p

Naresh Bhatia 25 Nov 7, 2021
:fire: An extremely fast, React-like JavaScript library for building modern user interfaces

Inferno is an insanely fast, React-like library for building high-performance user interfaces on both the client and server. Description The main obje

Inferno 15.2k Dec 3, 2021
Fast Touch Events for React

React Fastclick Instantly make your desktop / hybrid apps more responsive on touch devices. React Fastclick automatically adds fastclick touch events

Jake 'Sid' Smith 481 Sep 2, 2021
🦁 <1kb compiler-augmented virtual DOM. It's fast!

English | 中文 <1kb compiler-augmented virtual DOM. It's fast! Current Virtual DOM implementations are inadequate—Ranging from overcomplicated to abando

Aiden Bai 1.3k Nov 21, 2021
Compares Discord libraries and their support of new API features. Made with React, Next.js, and Bulma.

Compares Discord libraries and their support of new API features. Made with React, Next.js, and Bulma.

Advaith 42 Nov 29, 2021
Open source authentication and authorization service, container-native, PassportJS-native, built with React and Node.

id6 Authentication and authorization as a service Docs - Twitter Why id6 ? I wrote id6 out of frustration of re-writing authentication and authorizati

id6 12 Oct 31, 2021
React project with Google Firebase API, create partys with admin view and user view. Users can ask topics to admin, and support other topics.

?? Tecnologias Esse projeto foi desenvolvido com as seguintes tecnologias: React Firebase TypeScript ?? Como executar Clone o projeto e acesse a pasta

Felipe Souza 3 Aug 27, 2021
⚛️ Deliver UI for Web and Mobile platforms without taking care about complexity on how to style there, learn React once and code everywhere

SkynexUI Use the platform, don't care about how to style there ⚠️ Version 1.x.x is alpha, trust only in v2 A set of components writen on top of React

SkynexUI 229 Nov 22, 2021
PixelsMetaverse, Truffle and react and web3js team up to develop a metaverse game

English Pixels Metaverse Pixels Metaverse Introduction: Pixel Metaverse was born during the competition of the first Metaverse Training Camp held

null 13 Nov 24, 2021
In this assignment you will start with a basic calculator and style it using CSS, inline styles, and styled-components.

In this assignment you will start with a basic calculator and style it using CSS, inline styles, and styled-components.

Brandon Cloutier 1 Oct 4, 2021