Skip to content

Commit a5ee1d9

Browse files
authored
feat: support arbitrary colors value in theme, support colorReplacements field (#68)
1 parent 65bf56f commit a5ee1d9

15 files changed

+719
-77
lines changed

docs/.vitepress/config.ts

+2-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,8 @@ const GUIDES: DefaultTheme.NavItemWithLink[] = [
1313
{ text: 'Bundles', link: '/guide/bundles' },
1414
{ text: 'Dual Themes', link: '/guide/dual-themes' },
1515
{ text: 'Transformers', link: '/guide/transformers' },
16-
{ text: 'Compat Build', link: '/guide/compat' },
16+
{ text: 'Theme Colors Manipulation', link: '/guide/theme-colors' },
17+
{ text: 'Compatibly Build', link: '/guide/compat' },
1718
{ text: 'Custom Themes', link: '/guide/load-theme' },
1819
{ text: 'Custom Languages', link: '/guide/load-lang' },
1920
]

docs/guide/compat.md

+4-3
Original file line numberDiff line numberDiff line change
@@ -22,15 +22,15 @@ Set the alias to `shiki` in your `package.json`:
2222

2323
## Breaking Changes from Shiki
2424

25-
As of [`shiki@0.14.3`](https://github.com/shikijs/shiki/releases/tag/v0.14.3), the breaking changes between Shiiki and Shikiji are:
25+
Compare to [`shiki@0.14.3`](https://github.com/shikijs/shiki/releases/tag/v0.14.3), the breaking changes between Shiki and Shikiji are:
2626

2727
### Hard Breaking Changes
2828

2929
Breaking changes applied to both `shikiji` and `shikiji-compat`:
3030

31-
- CJS and IIFE builds are dropped. See [CJS Usage](#cjs-usage) and [CDN Usage](#cdn-usage) for more details.
31+
- CJS and IIFE builds are dropped. See [CJS Usage](/guide/install#cjs-usage) and [CDN Usage](/guide/install#cdn-usage) for more details.
3232
- `codeToHtml` uses [`hast`](https://github.com/syntax-tree/hast) internally. The generated HTML will be a bit different but should behave the same.
33-
- `css-variables` theme is not supported. Use the [dual themes](#lightdark-dual-themes) approach instead.
33+
- `css-variables` theme is not supported. Use the [dual themes](/guide/dual-themes) approach instead, or learn more at the [Theme Colors Manipulation](/guide/theme-colors).
3434

3535
### Soft Breaking Changes
3636

@@ -40,6 +40,7 @@ Breaking changes applies to `shikiji`, but are shimmed by `shikiji-compat`:
4040
- `BUNDLED_LANGUAGES`, `BUNDLED_THEMES` are moved to `shikiji/langs` and `shikiji/themes` and renamed to `bundledLanguages` and `bundledThemes` respectively.
4141
- `theme` option for `getHighlighter` is dropped, use `themes` with an array instead.
4242
- Highlighter does not maintain an internal default theme context. `theme` option is required for `codeToHtml` and `codeToThemedTokens`.
43+
- `codeToThemedTokens` set `includeExplanation` to `false` by default.
4344
- `.ansiToHtml` is merged into `.codeToHtml` as a special language `ansi`. Use `.codeToHtml(code, { lang: 'ansi' })` instead.
4445
- `lineOptions` is dropped in favor of the fully customizable `transforms` option.
4546
- `LanguageRegistration`'s `grammar` field is flattened to `LanguageRegistration` itself, refer to the types for more details.

docs/guide/load-theme.md

+14-2
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,18 @@ You can load custom themes by passing a `Theme` object into the themes array.
77
```ts
88
import { getHighlighter } from 'shikiji'
99

10-
const myTheme = JSON.parse(fs.readFileSync('my-theme.json', 'utf8'))
10+
const myTheme = {
11+
name: 'my-theme',
12+
settings: [
13+
{
14+
scope: ['comment'],
15+
settings: {
16+
foreground: '#888'
17+
}
18+
},
19+
// ...
20+
]
21+
}
1122

1223
const highlighter = await getHighlighter({
1324
themes: [myTheme]
@@ -24,6 +35,7 @@ You can also load themes after the highlighter has been created.
2435
```ts
2536
import { getHighlighter } from 'shikiji'
2637

38+
// Load the theme object from a file, a network request, or anywhere
2739
const myTheme = JSON.parse(fs.readFileSync('my-theme.json', 'utf8'))
2840

2941
const highlighter = await getHighlighter()
@@ -36,4 +48,4 @@ const html = highlighter.codeToHtml(code, {
3648
})
3749
```
3850

39-
The theme should be a TextMate theme in JSON object. For example, [it should looks like this](https://github.com/antfu/vscode-theme-vitesse/blob/main/themes/vitesse-dark.json).
51+
The theme is a TextMate theme in JavaScript object. For example, [it should looks like this](https://github.com/antfu/textmate-grammars-themes/blob/main/packages/tm-themes/themes/dark-plus.json).

docs/guide/theme-colors.md

+144
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# Theme Colors Manipulation
2+
3+
## Arbitrary Color Values
4+
5+
Usually, TextMate themes would expect the color values of each token to be a valid hex color value, the limitation is inherit from [`vscode-textmate`](https://github.com/microsoft/vscode-textmate). Since Shikiji v0.9.15, we introduced an automatic workaround by replacing non-hex color values with a placeholder and replacing them back on tokenization. This would allows you to use themes with arbitrary color values for the rendering without worrying about the technical details:
6+
7+
```ts twoslash
8+
import { getHighlighter } from 'shikiji'
9+
10+
const highlighter = await getHighlighter({
11+
langs: ['javascript'],
12+
themes: [
13+
{
14+
name: 'my-theme',
15+
settings: [
16+
{
17+
scope: ['comment'],
18+
settings: {
19+
// use `rgb`, `hsl`, `hsla`, // [!code hl:3]
20+
// or any anything supported by your renderer
21+
foreground: 'rgb(128, 128, 128)'
22+
}
23+
},
24+
{
25+
scope: ['string'],
26+
settings: {
27+
foreground: 'var(--code-string)' // CSS variable // [!code hl:1]
28+
}
29+
},
30+
// ...more
31+
],
32+
// Background and foreground colors // [!code hl:3]
33+
bg: 'var(--code-bg)',
34+
fg: 'var(--code-fg)'
35+
}
36+
]
37+
})
38+
39+
const html = highlighter.codeToHtml('const foo = "bar"', { lang: 'javascript', theme: 'my-theme' })
40+
```
41+
42+
::: info Notice
43+
Use this with caution as this will diverge from the TextMate theme compatibility.
44+
45+
This may make the theme incompatible with non-web usage like [`shikiji-cli`](/packages/cli) and [`shikiji-monaco`](/packages/monaco).
46+
:::
47+
48+
Learn more about how to [load themes](./load-theme).
49+
50+
## Color Replacements
51+
52+
You can also use the `colorReplacements` option to replace the color values of the theme. This is useful when you want to use a theme with a different color palette. It can be provided on both the theme object and the `codeToHast` `codeToHtml` options.
53+
54+
## CSS Variables Theme
55+
56+
::: warning Experimental
57+
This feature is experimental and may change without following semver.
58+
:::
59+
60+
Shikiji provides a factory function helper `createCssVariablesTheme` to for creating a theme that uses CSS variables easier. Note that this theme is a lot less granular than most of the other themes and requires to define the CSS variables in your app. This is provided for easier migration from Shiki's [`css-variables theme`](https://github.com/shikijs/shiki/blob/main/docs/themes.md#theming-with-css-variables). For better highlighting result, we recommend construct the theme manually with [Arbitrary Color Values](#arbitrary-color-values) or use [Color Replacements](#color-replacements) to override existing theme.
61+
62+
This theme is **not included by default** and requires to be registered explicitly:
63+
64+
```ts twoslash
65+
import { createCssVariablesTheme, getHighlighter } from 'shikiji'
66+
67+
// Create a custom CSS variables theme, the following are the default values
68+
const myTheme = createCssVariablesTheme({ // [!code hl:6]
69+
name: 'css-variables',
70+
variablePrefix: '--shiki-',
71+
variableDefaults: {},
72+
fontStyle: true
73+
})
74+
75+
const highlighter = await getHighlighter({
76+
langs: ['javascript'],
77+
themes: [myTheme] // register the theme // [!code hl]
78+
})
79+
80+
const html = highlighter.codeToHtml('const foo = "bar"', {
81+
lang: 'javascript',
82+
theme: 'css-variables' // use the theme // [!code hl]
83+
})
84+
```
85+
86+
CSS variables example:
87+
88+
```css
89+
:root {
90+
--shiki-foreground: #eeeeee;
91+
--shiki-background: #333333;
92+
--shiki-token-constant: #660000;
93+
--shiki-token-string: #770000;
94+
--shiki-token-comment: #880000;
95+
--shiki-token-keyword: #990000;
96+
--shiki-token-parameter: #aa0000;
97+
--shiki-token-function: #bb0000;
98+
--shiki-token-string-expression: #cc0000;
99+
--shiki-token-punctuation: #dd0000;
100+
--shiki-token-link: #ee0000;
101+
102+
/* Only required if using lang: 'ansi' */
103+
--shiki-ansi-black: #000000;
104+
--shiki-ansi-black-dim: #00000080;
105+
--shiki-ansi-red: #bb0000;
106+
--shiki-ansi-red-dim: #bb000080;
107+
--shiki-ansi-green: #00bb00;
108+
--shiki-ansi-green-dim: #00bb0080;
109+
--shiki-ansi-yellow: #bbbb00;
110+
--shiki-ansi-yellow-dim: #bbbb0080;
111+
--shiki-ansi-blue: #0000bb;
112+
--shiki-ansi-blue-dim: #0000bb80;
113+
--shiki-ansi-magenta: #ff00ff;
114+
--shiki-ansi-magenta-dim: #ff00ff80;
115+
--shiki-ansi-cyan: #00bbbb;
116+
--shiki-ansi-cyan-dim: #00bbbb80;
117+
--shiki-ansi-white: #eeeeee;
118+
--shiki-ansi-white-dim: #eeeeee80;
119+
--shiki-ansi-bright-black: #555555;
120+
--shiki-ansi-bright-black-dim: #55555580;
121+
--shiki-ansi-bright-red: #ff5555;
122+
--shiki-ansi-bright-red-dim: #ff555580;
123+
--shiki-ansi-bright-green: #00ff00;
124+
--shiki-ansi-bright-green-dim: #00ff0080;
125+
--shiki-ansi-bright-yellow: #ffff55;
126+
--shiki-ansi-bright-yellow-dim: #ffff5580;
127+
--shiki-ansi-bright-blue: #5555ff;
128+
--shiki-ansi-bright-blue-dim: #5555ff80;
129+
--shiki-ansi-bright-magenta: #ff55ff;
130+
--shiki-ansi-bright-magenta-dim: #ff55ff80;
131+
--shiki-ansi-bright-cyan: #55ffff;
132+
--shiki-ansi-bright-cyan-dim: #55ffff80;
133+
--shiki-ansi-bright-white: #ffffff;
134+
--shiki-ansi-bright-white-dim: #ffffff80;
135+
}
136+
```
137+
138+
If you are migrating form Shiki, some variables are renamed from Shiki's `css-variables`:
139+
140+
| Shiki | Shikiji |
141+
| -------------------------- | -------------------- |
142+
| `--shiki-color-text` | `--shiki-forground` |
143+
| `--shiki-color-background` | `--shiki-background` |
144+
| `--shiki-color-ansi-*` | `--shiki-ansi-*` |

packages/shikiji-compat/src/index.ts

+15-9
Original file line numberDiff line numberDiff line change
@@ -33,16 +33,22 @@ export async function getHighlighter(options: HighlighterOptions = {}) {
3333

3434
const defaultTheme = shikiji.getLoadedThemes()[0]
3535

36-
function codeToThemedTokens(code: string, options: CodeToThemedTokensOptions<BuiltinLanguage, BuiltinTheme>): ThemedToken[][]
37-
function codeToThemedTokens(code: string, lang: BuiltinLanguage, theme?: BuiltinTheme): ThemedToken[][]
38-
function codeToThemedTokens(code: string, lang: BuiltinLanguage | CodeToThemedTokensOptions<BuiltinLanguage, BuiltinTheme>, theme?: BuiltinTheme): ThemedToken[][] {
39-
if (typeof lang === 'string') {
40-
return shikiji.codeToThemedTokens(code, {
41-
lang,
42-
theme: (theme || defaultTheme) as BuiltinTheme,
36+
function codeToThemedTokens(code: string, lang: BuiltinLanguage, theme?: BuiltinTheme, options?: CodeToThemedTokensOptions<BuiltinLanguage, BuiltinTheme>): ThemedToken[][] {
37+
const tokens = shikiji.codeToThemedTokens(code, {
38+
includeExplanation: true,
39+
lang,
40+
theme: (theme || defaultTheme) as BuiltinTheme,
41+
...options,
42+
})
43+
44+
tokens.forEach((line) => {
45+
line.forEach((token) => {
46+
// Shiki always provides `explanation` array, even it's disabled
47+
token.explanation ||= []
4348
})
44-
}
45-
return shikiji.codeToThemedTokens(code, lang)
49+
})
50+
51+
return tokens
4652
}
4753

4854
function codeToHtml(code: string, options: CodeToHtmlOptions): string

packages/shikiji-compat/test/types.test.ts

+3
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ it('run', async () => {
1919
expect(sj.codeToThemedTokens('const a = 1', 'javascript'))
2020
.toEqual(s.codeToThemedTokens('const a = 1', 'javascript'))
2121

22+
expect(sj.codeToThemedTokens('const a = 1', 'javascript', 'nord', { includeExplanation: false }))
23+
.toEqual(s.codeToThemedTokens('const a = 1', 'javascript', 'nord', { includeExplanation: false }))
24+
2225
s.codeToHtml('const a = 1', 'javascript')
2326
sj.codeToHtml('const a = 1', 'javascript')
2427
s.codeToHtml('const a = 1', 'javascript', 'nord')

0 commit comments

Comments
 (0)