Syntax highlighting, like what GitHub uses to highlight code, but free and open source and JavaScript

Last update: Jun 9, 2022

Close up of The Starry Night by Vincent van Gogh (1889)
with examples of starry-night over it


starry-night

Build Coverage Downloads

Syntax highlighting, like what GitHub uses to highlight code, but free and open source and JavaScript!

Contents

What is this?

This package is an open source version of GitHub’s closed-source PrettyLights project (more on that later). It supports 490 grammars and its extremely high quality. It uses TextMate grammars which are also used in popular editors (SublimeText, Atom, VS Code, &c). They’re heavy but high quality.

When should I use this?

starry-night is a high quality highlighter (when your readers or authors are programmers, you want this!) that can support tons of grammars (from new things like Astro to much more!) which approaches how GitHub renders code.

It has a WASM dependency, and rather big grammars, which means that starry-night might be too heavy particularly in browsers, in which case lowlight or refractor might be more suitable.

This project is similar to the excellent shiki, and it uses the same underlying dependencies, but starry-night is meant to match GitHub in that it produces classes and works with the CSS it ships, making it easier to add dark mode and other themes with CSS compared to inline styles.

Finally, this package produces objects (an AST), which makes it useful when you want to perform syntax highlighting in a place where serialized HTML wouldn’t work or wouldn’t work well. For example, when you want to show code in a CLI by rendering to ANSI sequences, when you’re using virtual DOM frameworks (such as React or Preact) so that diffing can be performant, or when you’re working with hast or rehype.

Bundled, minified, and gzipped, starry-night and the WASM binary are 185 kB. There are two lists of grammars you can use: common (33 languages, good for your own site) adds 160 kB and all (490 languages, useful if are making a site like GitHub) is 1.35 MB. You can also manually choose which grammars to include (or add to common): a language is typically between 3 and 5 kB. As an example, adding Astro to starry-night with the common grammars costs an additional 1.5 kB.

What is PrettyLights?

PrettyLights is the syntax highlighter that GitHub uses to turn this:

```markdown
# Hello, world!
```

…into this:

# Hello, world!">
<span class="pl-mh"><span class="pl-mh">#span><span class="pl-mh"> span>Hello, world!span>

…which is what starry-night does too (some small differences in markup, but essentially the same)!

PrettyLights is responsible for taking the flag markdown, looking it up in languages.yml from github/linguist to figure out that that means markdown, taking a corresponding grammar (in this case atom/language-gfm), doing some GPL magic in C, and turning it into spans with classes.

GitHub is using PrettyLights since December 2014, when it replaced Pygments. They wanted to open source it, but were unable due to licensing issues. Recently (Feb 2019?), GitHub has slowly started to move towards TreeLights, which is based on TreeSitter, and also closed source. If TreeLights includes a language (currently: CSS, CodeQL, EJS, Elixir, Go, HTML, JS, PHP, Python, Ruby, TS), that’ll be used, for everything else PrettyLights is used.

starry-night does what PrettyLights does, not what TreeLights does. I’m hopeful that that will be open sourced in the future and we can mimic both.


Install

This package is ESM only. In Node.js (version 12.20+, 14.14+, 16.0+, 18.0+), install with npm:

npm install @wooorm/starry-night

In Deno with esm.sh:

import {createStarryNight, common} from 'https://esm.sh/@wooorm/[email protected]'

In browsers with esm.sh:

import {createStarryNight, common} from 'https://esm.sh/@wooorm/[email protected]?bundle' ">
<script type="module">
  import {createStarryNight, common} from 'https://esm.sh/@wooorm/[email protected]?bundle'
script>

To get the CSS in browsers, do (see CSS for more info):

">

<link rel="stylesheet" href="https://esm.sh/@wooorm/[email protected]/style/both.css">

Use

import {createStarryNight, common} from '@wooorm/starry-night'

const starryNight = await createStarryNight(common)

const scope = starryNight.flagToScope('markdown')
const tree = starryNight.highlight('# hi', scope)

console.log(tree)

Yields:

{
  type: 'root',
  children: [
    {
      type: 'element',
      tagName: 'span',
      properties: {className: ['pl-mh']},
      children: [{type: 'text', value: '# hi'}]
    }
  ]
}

API

This package exports the identifiers createStarryNight, common, and all. There is no default export.

createStarryNight(grammars)

Create a StarryNight that can highlight things based on the given grammars. This is async to facilitate async loading and registering, which is currently only used for WASM.

Parameters
  • grammars (Array) — grammars to support
Returns

Promise that resolves to an instance which highlights based on the bound grammars (Promise).

starryNight.highlight(value, scope)

Highlight value (code) as scope (a TextMate scope).

Parameters
  • value (string) — code to highlight
  • scope (string) — registered grammar scope to highlight as (such as 'source.gfm')
Returns

Node representing highlighted code (Root).

Example
import {createStarryNight} from '@wooorm/starry-night'
import sourceCss from '@wooorm/starry-night/lang/source.css.js'

const starryNight = await createStarryNight([sourceCss])

console.log(starryNight.highlight('em { color: red }', 'source.css'))

Yields:

{
  type: 'root',
  children: [
    {type: 'element', tagName: 'span', properties: [Object], children: [Array]},
    {type: 'text', value: ' { '},
    // …
    {type: 'element', tagName: 'span', properties: [Object], children: [Array]},
    {type: 'text', value: ' }'}
  ]
}

starryNight.flagToScope(flag)

Get the grammar scope (such as source.gfm) associated with a grammar name (such as markdown or pandoc) or grammar extension (such as .md or .rmd). Note that grammars can use the same extensions, in which case GitHub chooses the first. Notably, .md is registered by a lisp-like language instead of markdown. 🤷‍♂️

Parameters
  • flag (string) — grammar name (such as 'markdown' or 'pandoc') or grammar extension (such as '.md' or '.rmd')
Returns

Grammar scope, such as 'source.gfm' (string?)

Example
import {createStarryNight, common} from '@wooorm/starry-night'

const starryNight = await createStarryNight(common)

console.log(starryNight.flagToScope('pandoc')) // `'source.gfm'`
console.log(starryNight.flagToScope('workbook')) // `'source.gfm'`
console.log(starryNight.flagToScope('.workbook')) // `'source.gfm'`
console.log(starryNight.flagToScope('whatever')) // `undefined`

starryNight.scopes()

List all registered scopes.

Returns

List of grammar scopes, such as 'source.gfm' (Array).

Example
import {createStarryNight, common} from '@wooorm/starry-night'

const starryNight = await createStarryNight(common)

console.log(starryNight.scopes())

Yields:

[
  'source.c',
  'source.c++',
  // …
  'text.xml',
  'text.xml.svg'
]

starryNight.register(grammars)

Add more grammars.

Parameters
  • grammars (Array) — grammars to support
Returns

A promise resolving to nothing (Promise).

Example
import {toHtml} from 'hast-util-to-html'
import {createStarryNight} from '@wooorm/starry-night'
import sourceGfm from '@wooorm/starry-night/lang/source.gfm.js'
import sourceCss from '@wooorm/starry-night/lang/source.css.js'

const markdown = '```css\nem { color: red }\n```'

const starryNight = await createStarryNight([sourceGfm])

console.log(toHtml(starryNight.highlight(markdown, 'source.gfm')))

await starryNight.register([sourceCss])

console.log(toHtml(starryNight.highlight(markdown, 'source.gfm')))

Yields:

```css em { color: red } ```">
<span class="pl-c1">```cssspan>
em { color: red }
<span class="pl-c1">```span>
```css em { color: red } ```">
<span class="pl-c1">```cssspan>
<span class="pl-ent">emspan> { <span class="pl-c1">colorspan>: <span class="pl-c1">redspan> }
<span class="pl-c1">```span>

Examples

Example: serializing hast as html

hast trees as returned by starry-night can be serialized with hast-util-to-html:

import {toHtml} from 'hast-util-to-html'
import {starryNight, common} from '@wooorm/starry-night'

const starryNight = await createStarryNight(common)

const tree = starryNight.highlight('"use strict";', 'source.js')

console.log(toHtml(tree))

Yields:

"use strict";">
<span class="pl-s"><span class="pl-pds">"span>use strict<span class="pl-pds">"span>span>;

Example: turning hast into react nodes

hast trees as returned by starry-night can be turned into React (or Preact, Vue, &c) with hast-to-hyperscript:

import {createStarryNight, common} from '@wooorm/starry-night'
import {toH} from 'hast-to-hyperscript'
import React from 'react'

const starryNight = await createStarryNight(common)

const tree = starryNight.highlight('"use strict";', 'source.js')
const reactNode = toH(React.createElement, tree)

console.log(reactNode)

Yields:

{
  '$$typeof': Symbol(react.element),
  type: 'div',
  key: 'h-1',
  ref: null,
  props: {children: [[Object], ';']},
  _owner: null,
  _store: {}
}

Example: integrate with unified, remark, and rehype

This example shows how to combine starry-night with unified: using remark to parse the markdown and transforming it to HTML with rehype. If we have a markdown file example.md:

# Hello

…world!

```js
console.log('it works!')
```

…and a plugin rehype-starry-night.js:

/**
 * @typedef {import('hast').Root} Root
 * @typedef {import('hast').ElementContent} ElementContent
 * @typedef {import('@wooorm/starry-night').Grammar} Grammar
 *
 * @typedef Options
 *   Configuration (optional)
 * @property {Array} [grammars]
 *   Grammars to support (defaults: `common`).
 */

import {createStarryNight, common} from '@wooorm/starry-night'
import {visit} from 'unist-util-visit'
import {toString} from 'hast-util-to-string'

/**
 * Plugin to highlight code with `starry-night`.
 *
 * @type {import('unified').Plugin<[Options?], Root>}
 */
export default function rehypeStarryNight(options = {}) {
  const grammars = options.grammars || common
  const starryNightPromise = createStarryNight(grammars)
  const prefix = 'language-'

  return async function (tree) {
    const starryNight = await starryNightPromise

    visit(tree, 'element', function (node, index, parent) {
      if (!parent || index === null || node.tagName !== 'pre') {
        return
      }

      const head = node.children[0]

      if (
        !head ||
        head.type !== 'element' ||
        head.tagName !== 'code' ||
        !head.properties
      ) {
        return
      }

      const classes = head.properties.className

      if (!Array.isArray(classes)) return

      const language = classes.find(
        (d) => typeof d === 'string' && d.startsWith(prefix)
      )

      if (typeof language !== 'string') return

      const scope = starryNight.flagToScope(language.slice(prefix.length))

      // Maybe warn?
      if (!scope) return

      const fragment = starryNight.highlight(toString(head), scope)
      const children = /** @type {Array} */ (fragment.children)

      parent.children.splice(index, 1, {
        type: 'element',
        tagName: 'div',
        properties: {
          className: [
            'highlight',
            'highlight-' + scope.replace(/^source\./, '').replace(/\./g, '-')
          ]
        },
        children: [{type: 'element', tagName: 'pre', properties: {}, children}]
      })
    })
  }
}

…and finally a module example.js:

import fs from 'node:fs/promises'
import {unified} from 'unified'
import remarkParse from 'remark-parse'
import remarkRehype from 'remark-rehype'
import rehypeStringify from 'rehype-stringify'
import rehypeStarryNight from './rehype-starry-night.js'

const file = await unified()
  .use(remarkParse)
  .use(remarkRehype)
  .use(rehypeStarryNight)
  .use(rehypeStringify)
  .process(await fs.readFile('example.md'))

console.log(String(file))

Now running node example.js yields:

console.log('it works!')
">
<h1>Helloh1>
<p>…world!p>
<div class="highlight highlight-js"><pre><span class="pl-en">consolespan>.<span class="pl-c1">logspan>(<span class="pl-s"><span class="pl-pds">'span>it works!<span class="pl-pds">'span>span>)
pre>div>

Example: integrating with markdown-it

This example shows how to combine starry-night with markdown-it. If we have a markdown file example.md:

# Hello

…world!

```js
console.log('it works!')
```

…and a module example.js:

import fs from 'node:fs/promises'
import {createStarryNight, common} from '@wooorm/starry-night'
import {toHtml} from 'hast-util-to-html'
import markdownIt from 'markdown-it'

const file = await fs.readFile('example.md')
const starryNight = await createStarryNight(common)

const markdownItInstance = markdownIt({
  highlight(value, lang) {
    const scope = starryNight.flagToScope(lang)

    return toHtml({
      type: 'element',
      tagName: 'pre',
      properties: {
        className: scope
          ? [
              'highlight',
              'highlight-' + scope.replace(/^source\./, '').replace(/\./g, '-')
            ]
          : undefined
      },
      children: scope
        ? starryNight.highlight(value, scope).children
        : [{type: 'text', value}]
    })
  }
})

const html = markdownItInstance.render(String(file))

console.log(html)

Now running node example.js yields:

console.log('it works!') ">
<h1>Helloh1>
<p>…world!p>
<pre class="highlight highlight-js"><span class="pl-en">consolespan>.<span class="pl-c1">logspan>(<span class="pl-s"><span class="pl-pds">'span>it works!<span class="pl-pds">'span>span>)
pre>

Syntax tree

The generated hast starts with a root node, that represents the fragment. It contains up to three levels of elements, each with a single class. All these levels can contain text nodes with the actual code. Interestingly, TextMate grammars work per line, so all line endings are in the root directly, meaning that creating a gutter to display line numbers can be generated rather naïvely by only looking through the root node.

CSS

starry-night does not inject CSS for the syntax highlighted code (because well, starry-night doesn’t have to be turned into HTML and might not run in a browser!). If you are in a browser, you can use the packaged themes, or get creative with CSS! 💅

All themes accept CSS variables (custom properties). With the theme core.css, you have to define your own properties. All other themes define the colors on :root. Themes either have a dark or light suffix, or none, in which case they automatically switch colors based on a @media (prefers-color-scheme: dark). All themes are tiny (under 1 kB). The shipped themes are as follows:

name Includes light scheme Includes dark scheme
core.css
light.css
dark.css
both.css
colorblind-light.css
colorblind-dark.css
colorblind.css
dimmed-dark.css
dimmed.css
high-contrast-light.css
high-contrast-dark.css
high-contrast.css
tritanopia-light.css
tritanopia-dark.css
tritanopia.css

Languages

Checked grammars are included in common. Everything is available through all. You can add more grammars as you please.

Each grammar has several associated names and extensions. See source files for which are known and use flagToScope to turn them into scopes.

All licenses are permissive and made available in notice. Changes should go to upstream repos and languages.yml in github/linguist.

Types

This package is fully typed with TypeScript. It exports additional Grammar and Root types that model their respective interfaces.

Compatibility

This package is at least compatible with all maintained versions of Node.js. As of now, that is Node.js 14.14+, 16.0+, and 18.0+. It also works in Deno and modern browsers.

You can pass your own TextMate grammars, provided that they work with vscode-textmate, and that they have the added fields scopeName, names, and extensions (see types for the definitions and the grammars in lang/ for examples).

Security

This package is safe.

Related

  • lowlight — similar but based on highlight.js
  • refractor — similar but based on Prism

Contribute

Yes please! See How to Contribute to Open Source.

License

The grammars included in this package are covered by their repositories’ respective licenses, which are permissive (apache-2.0, mit, etc), and made available in notice.

All other files MIT © Titus Wormer


GitHub

https://github.com/wooorm/starry-night
You might also like...

Personal blog and portfolio with a basic comment system created with react, php and mysql, hosted on github pages and backend hosted on heroku and clever cloud for free!

Personal blog and portfolio with a basic comment system created with react, php and mysql, hosted on github pages and backend hosted on heroku and clever cloud for free!

Personal blog and portfolio with a basic comment system created with react, php and mysql, hosted on github pages and backend hosted on heroku and clever cloud for free!

Mar 16, 2022

A tool to search a user from Github using the Github's API.

A tool to search a user from Github using the Github's API.

Github Finder Find github user easily A tool to search a user from Github using the Github's API. It's a mini-project created thanks to bradtraversy's

Dec 6, 2021

Github-user-finder - Github User Finder With React.js

Github-user-finder - Github User Finder With React.js

Github User Finder Aplicação que permite a busca por nome de usuários do Github

Apr 30, 2022

Any GitHub user analysis application based on React, Auth0, GitHub API, FusionCharts, Netlify

About In This Project, You can get the analysis of Any GitHub User. Here I used React, Auth0( for authentication), GitHub API ( for data fetch), Fusio

Mar 15, 2022

Github Explorer - 🔍 Repository search using the Github API.

 Github Explorer - 🔍 Repository search using the Github API.

Github Explorer 🔍 Repository search using the Github API. Link to the Project Installation Before starting, you'll need to have the following tools i

Jun 12, 2022

A free e-library for developer to read and upload ebooks they would like to share with the community and help other developers grow

A free e-library for developer to read and upload ebooks they would like to share with the community and help other developers grow

A free e-library for developer to read and upload ebooks they would like to share with the community and help other developers grow

Apr 5, 2022

MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API.

MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API.

MediaCMS is a modern, fully featured open source video and media CMS. It is developed to meet the needs of modern web platforms for viewing and sharing media. It can be used to build a small to medium video and media portal within minutes.

Jun 21, 2022

Chroma is an open source design system from the team at LifeOmic built with React and TypeScript.

Chroma is an open source design system from the team at LifeOmic built with React and TypeScript.

Chroma is an open source design system from the team at LifeOmic. It is built with React and TypeScript. The goal of Chroma is to provide design-approved components to developers to speed up their development process and build visual consistency throughout web applications.

Jun 6, 2022

Project flat is the Web, Windows and macOS client of Agora Flat open source classroom.

Project flat is the Web, Windows and macOS client of Agora Flat open source classroom.

Project flat is the Web, Windows and macOS client of Agora Flat open source classroom.

Jun 21, 2022
Comments
  • 1. Is this package still private on npm?

    Running the install instructions:

    npm install starry-night
    

    Results in a 404 error from npm.

    Have you published the package yet? Or is the package still private?

    Reviewed by macdonst at 2022-05-12 18:48
  • 2. Rehype/Remark Plugin?

    This looks great! I'd love to use it with a static site generator. Do you have any plans for a Remark/Rehype plugin wrapping this or would that be fair game for the community?

    Reviewed by eligundry at 2022-05-11 22:55
A bot Similar to the reaction roles discord bot, but free. Written in JavaScript, and uses MongoDB.

An open source reaction roles bot to anyone who needs one in their server. It's easy to use, similar to the reaction roles bot and Totally Free. The bot also used MongoDB as a database!

Jun 20, 2022
GitHub Desktop is an open source Electron-based GitHub app.
GitHub Desktop is an open source Electron-based GitHub app.

GitHub Desktop is an open source Electron-based GitHub app. It is written in TypeScript and uses React.

Jun 16, 2022
it is a React application which uses SpaceX open source graphql APIs

spaceX-mini-project it is a React application which uses SpaceX open source graphql APIs

Nov 5, 2021
Highlightjs-blade - Blade language definition for Highlight.js
Highlightjs-blade - Blade language definition for Highlight.js

Blade language definition for Highlight.js Syntax implementation of Laravel's's

Feb 25, 2022
An open source Electron-based GitHub app written in React
An open source Electron-based GitHub app written in React

GitHub Desktop GitHub Desktop is an open source Electron-based GitHub app. It is written in TypeScript and uses React. Where can I get it? Download th

Feb 10, 2022
Open source platform to manage Firestore data in a spreadsheet-like UI, deploy Cloud Functions easily, and connect to your favorite third-party platforms.⚡️✨
Open source platform to manage Firestore data in a spreadsheet-like UI, deploy Cloud Functions easily, and connect to your favorite third-party platforms.⚡️✨

Open source platform to manage Firestore data in a spreadsheet-like UI, deploy Cloud Functions easily, and connect to your favorite third-party platforms.⚡️✨

Jun 20, 2022
ToolJet is an open-source no-code framework to build and deploy internal tools quickly without much effort from the engineering teams
ToolJet is an open-source no-code framework to build and deploy internal tools quickly without much effort from the engineering teams

ToolJet is an open-source no-code framework to build and deploy internal tools quickly without much effort from the engineering teams. You can connect to your data sources such as databases ( like PostgreSQL, MongoDB, Elasticsearch, etc ), API endpoints ( ToolJet supports importing OpenAPI spec & OAuth2 authorization) and external services ( like Stripe, Slack, Google Sheets, Airtable ) and use our pre-built UI widgets to build internal tools.

Jun 27, 2022
A lightweight open source alternative to linktree, complete with Docker container that hosts your web server and code. Written in ReactJS SSR.
A lightweight open source alternative to linktree, complete with Docker container that hosts your web server and code.  Written in ReactJS SSR.

?? LittleLink-Server This project is based on the great work from littlelink It takes the same simple approach to a link page and hosts it within a No

Jun 19, 2022
This is a React Portfolio that uses a combination of React, Bootstrap, and Github Pages.
This is a React Portfolio that uses a combination of React, Bootstrap, and Github Pages.

This is a React Portfolio that uses a combination of React, Bootstrap, and Github Pages. I built the entirety of the portfolio from scratch using React's functional components, react-bootstrap for styling, and Vectr for all of my graphic design needs.

Jan 5, 2022
This AWS Ecommerce Website is a multi-tier ecommerce web application that uses React, HTML, typescript, and Javascript

This AWS Ecommerce Website is a multi-tier ecommerce web application that uses React, HTML, typescript, and Javascript Outline Overview The goal of th

Jun 4, 2022