Gatsby: Question - How to read Markdown from frontmatter

The tutorial walks through this, particularly parts 5-7 https://www.gatsbyjs.org/tutorial/

You might also want to start with one of the starters — many of which have markdown support already setup — https://www.gatsbyjs.org/docs/gatsby-starters/

The question is more nuanced and not covered by the tutorial, so I'll reopen this.

In my opinion you have 2 options:

1. You can separate your markdown content to separate files and use path links:

Separate file - let's call it `someContent.md`
```md
### This is the actual content in **MarkDown** format.

- This is the first row
- This is second row
- This is third row
```
and reference that file in your main file (by relative path):
```md
halfBlocks:
  - title: This is first title
    content: "./someContent.md"
```
then in query you could
```
halfBlocks {
  content {
    childMarkdownRemark {
      html
    }
}
```

1. Other approach would be handling this programatically - creating markdown nodes for your frontmatter fields and adding them via createNodeField. This is more involved. You would probably need to browse the Contentful source plugin do see how to create MarkdownRemark node.

Hi @KyleAMathews thank you for your suggestion. I did read them but it was hard for me to completely understand how to go about this. I am using a starter but this was more involved. Starter used: https://github.com/v4iv/gatsby-starter-business

@pieh Thank you so much for guiding me. You are right. I figured I will have too many small files so I got this working with the code below. I am documenting this here so that if someone else has the same problem they can also see this.

Step 1: Reading content

I am reading the content normally via Graphql. This gives me the markdown as a string. I still need to convert that.

Step 2: Converting to HTML

For this I decided to let the content from as string till we reach the actual component that is going to display this. There I convert it to markdown.

Add remark for doing this programmatically. You may be able to ignore remark-preset-lint-recommended:

Install Remark

yarn add remark remark-preset-lint-recommended remark-html

Import

import remark from 'remark';
import recommended from 'remark-preset-lint-recommended';
import remarkHtml from 'remark-html';

Render
Later in the render part assuming content is the markdown that was read as string:

content = remark()
      .use(recommended)
      .use(remarkHtml)
      .processSync(content).toString();

Now I am able to reinterpret content as HTML.

Step 3: Adding content in markdown

There is one more gotcha I came across. The formatting was not proper when I was using multiline with >-:

content: >-
    ### This is the actual content in **MarkDown** format.
    - This is the first row
    - This is second row

But with pipe symbol | it works great.

content: |
    ### This is the actual content in **MarkDown** format.
    - This is the first row
    - This is second row

For now I am closing this. Please feel free to reopen if you would like.

Thanks!!

I would like to use markdown for the frontmatter (title, and excerpt to be exact) as well, and I think it should be supported by default.

Would be great to have a naming convention, so that gatsby-transformer-remark could understand that e.g. title.md is a markdown field.

@omeid @thorn0 this might be something we support directly in gatsby-transformer-remark but in the meantime, you can create a plugin which does this for you e.g. https://github.com/gatsbyjs/gatsby/issues/5729#issuecomment-395701042 and createNodeField

Apologies for commenting on an already closed issue, but just wanted to share a snippet that I used in my own gatsby-node.js that worked for me, following what you all have referenced:

// Need to `yarn add remark remark-html`, then include the following code in
// gatsby-node.js.
const remark = require('remark');
const remarkHTML = require('remark-html');

exports.onCreateNode = ({ node }) => {
  // Conditionals, etc. can be used here, but I omitted those just for example's sake.
  const markdown = node.frontmatter.my_field;
  node.frontmatter.my_field = remark()
    .use(remarkHTML)
    .processSync(markdown)
    .toString();
  return node;
};

So is it okay to do so without using createNodeField? I'm confused.

@thorn0 better use createNodeField instead of node.frontmatter.my_field = because mutating node may results in hard to debug bugs

@amitjindal @nshki It's working nicely but totally crash my production build process since I have installed the "react-flickity-component" library:

success delete html and css files from previous builds — 0.626 s
success open and validate gatsby-config — 0.018 s
success copy gatsby files — 0.075 s
success onPreBootstrap — 2.782 s
error UNHANDLED EXCEPTION


  TypeError: Cannot set property 'Compiler' of null

  - index.js:16 plugin
    [blog]/[remark-html]/index.js:16:17

  - index.js:271 Function.use
    [blog]/[unified]/index.js:271:25

  - gatsby-node.js:63 exports.onCreateNode.postscriptumsMarkdown.forEach.postscr    iptum
    /home/projects/blog/gatsby-node.js:63:12

  - Array.forEach

  - gatsby-node.js:61 Object.exports.onCreateNode
    /home/projects/blog/gatsby-node.js:61:29

  - api-runner-node.js:110 runAPI
    [blog]/[gatsby]/dist/utils/api-runner-node.js:110:36

  - api-runner-node.js:187 
    [blog]/[gatsby]/dist/utils/api-runner-node.js:187:33

  - map.js:27 
    [blog]/[async]/internal/map.js:27:9

  - eachOfLimit.js:66 replenish
    [blog]/[async]/internal/eachOfLimit.js:66:17

  - eachOfLimit.js:50 iterateeCallback
    [blog]/[async]/internal/eachOfLimit.js:50:17

  - onlyOnce.js:12 module.exports
    [blog]/[async]/internal/onlyOnce.js:12:16

  - map.js:29 
    [blog]/[async]/internal/map.js:29:13

  - util.js:16 tryCatcher
    [blog]/[bluebird]/js/release/util.js:16:23

  - nodeify.js:23 Promise.successAdapter
    [blog]/[bluebird]/js/release/nodeify.js:23:30

  - promise.js:566 Promise.module.exports.Promise._settlePromise
    [blog]/[bluebird]/js/release/promise.js:566:21

  - promise.js:606 Promise.module.exports.Promise._settlePromiseCtx
    [blog]/[bluebird]/js/release/promise.js:606:10


Waiting for the debugger to disconnect...

Process finished with exit code 130 (interrupted by signal 2: SIGINT)

Trying to exclude library from Webpack not working (@see https://github.com/gatsbyjs/gatsby/issues/7599)

Hi David (@comxd), Sorry I was traveling.
Unfortunately I have no insight into this. I tried to check in the code. The Compiler on null seems be coming from the remark library.

You seem to be using a loop in your gatsby-node.js file.
It might be related to some content coming in that is not markdown or worse empty and you are trying to process it. Try putting some console.log statements in that and see if you find a pattern where something empty is causing this.

This could be totally off but couldn't you just create a markdown component like below that you could then use in your templates anywhere markdown is needed to be converted to HTML

import React from 'react'
import PropTypes from 'prop-types'
import showdown from 'showdown'

const converter = new showdown.Converter()

const MarkdownContent = ({ content, className }) => (
  <div className={className} dangerouslySetInnerHTML={{ __html: converter.makeHtml(content) }} />
)

MarkdownContent.propTypes = {
  content: PropTypes.string,
  className: PropTypes.string,
}


export default MarkdownContent

@blakenoll definitely not totally off! It's a reasonable approach.

That said, one of the big benefits of Gatsby is that you do those operations at build time, which is nice because then we don't need to shop a Markdown parser to the end user!

@DSchau Won't the Markdown parser actually be shipped down to to the user when the app is re-hydrated?

@blakenoll Love the out-of-the-box thinking!

Do we have anything available to us to pull-in all of the remark plugins/configuration that we included within gatsby-config.js so we don't need to duplicate all the functionality done behind the scenes Gatsby's implementation of remark is providing us? That would make generating the node fields a bit easier; but incredibly cumbersome when you think about nested, repeatable fields and the variation of content per page.

@blakenoll Thanks so much for your showdown tip. Super helpful and did what I needed to use HTML in frontmatter. That said, seems a clunky approach to using Markdown to create a web page that needs to pass different pieces of content to different parts of the page.

Is there any way we can apply some sort of markdownParser function within the frontmatter section of our graphQL query (similar to the way we can manipulate images) that will parse the incoming markdown string in the frontmatter and convert it to HTML? Not a graphQL expert...just trying to think.

This is an important issue IMO b/c if one is using Gatsby with Netifly CMS, Netifly provides the option to have various frontmatter fields accept Markdown as values. But when querying those fields, they are returned as markdown in a string instead of parsed to HTML. @amitjindal and @blakenoll's solutions work, but as @DSchau mentioned, it's not preferable to ship a markdown parse to the user if we can avoid it. Any thoughts from someone who's more familiar with Gatbsy than I?

@skylarweaver I'm definitely on the same page with ya. While I do understand the nature of creating a node field that can have this information parsed, it also becomes a bit unwieldy with CMS data that may have repeatable fields and a large amount of variations of field names to sift through; on-top of not having a clear way of re-using any/all Gatsby Remark plugins at that time.

+1 what @skylarweaver said!

@amitjindal Maybe stupid question but what does the ">-" even do? Using the netlify cms starter and seems to make absolutely no difference in the generated output whether I have >, >-, |, or nothing at all.

@nol13 See block scalars, it’s about newlines.

How could I insert a table? This does not work

content: |
        |   |   |   |   |   |
        |---|---|---|---|---|
        |   |   |   |   |   |
        |   |   |   |   |   |
        |   |   |   |   |   |

@qnguyen12 I'd try using a tool like https://jmalarcon.github.io/markdowntables/ to help you with the conversion.

Yes, I mean it render as the source, not table
For example:

text: |
        test   
        ### This is the actual content in **MarkDown** format.  
        |Month|Savings|Spending|
        |--- |--- |--- |
        |January|$100|$900|
        |July|$750|$1000|
        |December|$250|$300|
        |April|$400|$700|

it generates:
test

This is the actual content in MarkDown format.

|Month|Savings|Spending| |--- |--- |--- | |January|$100|$900| |July|$750|$1000| |December|$250|$300| |April|$400|$700|

@KyleAMathews Thanks, that sort of work, but obviously you either need to reply the remark plugin config and plugins or you get different results. Any plans to support markdown frontmatter? Perhaps a frontmattermd field along with the raw frontmatter?

@omeid I don't have any plans no — this would be a great plugin for someone to create and share with the community!

I created a plugin that should do this: gatsby-transformer-remark-frontmatter. It seems to work from my testing, and I'm planning on using it in a project that I'm doing for a client, but I'd appreciate it if you guys would take a look and tell me if there's anything that looks incorrect, as it's my first time writing a gatsby plugin. It takes the route suggested by @omeid and adds a frontmattermd field to the MarkdownRemark node.

Initially, before I realized that I could just create new markdown file nodes for gatsby-transformer-remark to consume, I came up with a really hacky solution involving calling the resolvers exported by gatsby-transformer-remark's setFieldsOnGraphQLNodeType function, and passing in a new markdown node created within another resolver. This allowed querying any field on the MarkdownRemark node using a field enum like is used for the group function, which I really liked, but it felt like to much of a hack to actually use for anything. I've saved it here for posterity.

hi @WhiteAbeLincoln i tried to install and test:
npm i gatsby-transformer-remark-frontmatter npm ERR! code ENOVERSIONS npm ERR! No valid versions available for gatsby-transformer-remark-frontmatter

Sorry, I realized haven't published to npm yet. I'll publish it once I get off work and let you know.

— Abe White

On Jun 17, 2019, at 22:53, broeker notifications@github.com wrote:

hi @WhiteAbeLincoln i tried to install and test:
npm i gatsby-transformer-remark-frontmatter npm ERR! code ENOVERSIONS npm ERR! No valid versions available for gatsby-transformer-remark-frontmatter

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@WhiteAbeLincoln I tried gatsby-transformer-remark-frontmatter but it gave me an error.

ERROR #11325

Your site's "gatsby-node.js" created a page with a component that doesn't exist.

Do you get this error?

It was originally reported by @obeid on your repo's issue log.

Maybe I just ain't using it correctly. So some help appreciated.

Building on top of @nshki answer and with @pieh comment about node mutation. This totally works for me:

const remark = require("remark");
const remarkHTML = require("remark-html");

exports.onCreateNode = ({ node, actions: { createNodeField } }) => {
  const my_field = node.frontmatter.my_field;

  if (my_field) {
    const value = remark()
      .use(remarkHTML)
      .processSync(my_field)
      .toString();

    // new node at:
    // fields {
    //   my_field_html
    // }
    createNodeField({
      name: `my_field_html`,
      node,
      value
    });
  }
};

edit: my_field => my_field_html

@aziaziazi How can I do the same thing but for a field nested in array?

---
pressEventsList:
  - body: >-
      *My md content...*
    image: 'https://res.cloudinary.com/press/01.jpg'
  - body: >-
      *My md content...*
    image: 'https://res.cloudinary.com/press/02.jpg'
---

I need to convert each pressEventsList[i].body.

@alexeychikk i guess you may look for pressEventList and then map through the content to create an array of results:

const remark = require("remark");
const remarkHTML = require("remark-html");

exports.onCreateNode = ({ node, actions: { createNodeField } }) => {
const pressEventList = node.frontmatter.pressEventList;

if (pressEventList) {
  const value = pressEventList.map(event =>
     remark()
    .use(remarkHTML)
    .processSync(event.body)
    .toString()
  )

  createNodeField({
    name: `pressEventList`,
    node,
    value
  });
}
};

I'm interested in creating a plugin to parse custom YAML tags to achieve the above without using createNodeField (the same way sharp parses image URLs).
Can anyone point me to the code where image URLs are parsed to see an example of how this is currently done with sharp?

👋 For those using MDX, I created a plugin to add frontmatter support https://www.gatsbyjs.org/packages/gatsby-plugin-mdx-frontmatter/

@zslabs, it's not often you see a solution posted "9 hours ago"! I'll give it a spin! Nice work.

I've been struggling with this since I wanted to use a more complex data structure for one of my pages.
In frontmatter I had an array of sections, with a few fields like title and featured image, and then on each I had a body made with markdown.
Using createNodeField wasn't doing it for me because I had trouble linking them logically since they are created in their own field, not appended to the existing frontmatter structure.
I ended up using createFieldExtension so that when my section.body is queried, it's returned in HTML.
Please someone correct me if this isn't a good solution, it seems to work for me but I have that nagging feeling it's the wrong way of going about this.

my frontmatter structure looks like this:

templateKey: project-entry
date: 2020-06-22T13:16:57.702Z
featuredproject: true
title: Project title
description: Description for listing the project on other pages
featuredimage: Image for listing the project on other pages
featuredpost: false
sections:
  - heading: Section heading
    standout: false
    intro: >-
      Introduction to be displayed separately to body
    body: >-
       ## section title
       * bullet point
       * bullet point
       Some other text here

And the code I used in gatsby-node.js

exports.createSchemaCustomization = ({actions}) => {
  const { createTypes, createFieldExtension} = actions

  createFieldExtension({
    name: 'toHTML',
    extend:() => ({
        resolve(source) {
          return remark().use(remarkHTML).processSync(source.body).toString()
        }
      })
  })
  const typeDefs = `
  type MarkdownRemark implements Node {
    frontmatter: Frontmatter
  }
  type Frontmatter @infer {
    sections: [section]
  }
  type section @infer {
    body: String @toHTML
  }
  `
  createTypes(typeDefs)
}

For anyone else that's interested I solved it using a custom YAML type to allow parsing any arbitrary field as markdown like so:

---
title: My Page
inline: !md Some **bold** and _italic_ text
block: !md |
  ## I'm a H2 title
  [I'm an inline-style link](https://www.google.com)
---

To do so create a custom type and then override grey-matter's YAML parser:

// custom-yaml.js
const yaml = require('js-yaml')
const remark = require('remark')
const remarkHTML = require('remark-html')

const MarkdownYamlType = new yaml.Type('!md', {
  kind: 'scalar',
  construct: data => remark().use(remarkHTML).processSync(data).toString(),
})

const MARKDOWN_SCHEMA = yaml.Schema.create(MarkdownYamlType)

module.exports = doc => yaml.safeLoad(doc, { schema: MARKDOWN_SCHEMA })

// gatsby-config.js
module.exports = {
  plugins: [
    {
      resolve: `gatsby-transformer-remark`,
      options: {
        engines: { yaml: require("path/to/custom-yaml.js") },
      },
    }
  ]
}

I resolved this a little different. I created a field extension called @md and used it in the frontmatter type definition combinated with the field renaming we could achieve the desired abstraction.

exports.createSchemaCustomization = ({ actions }) => {
  actions.createFieldExtension({
    name: "md",
    args: {
      from: {
        type: "String!",
        defaultValue: true,
      },
    },

    extend() {
      return {
        args: {
          from: "String!",
        },
        resolve(source, args) {
          const fieldValue = source[args.from]
          return convertToHTML(fieldValue)
        },
      }
    },
  })
  const typeDefs = `
    type MarkdownRemark implements Node @infer {
      frontmatter: Frontmatter
    }
    type Frontmatter {
      markdownField: String! @md
    }
  `
  actions.createTypes(typeDefs)
}

here is an example usage:

...
frontmatter {
        title: markdownField(from: "title")
        subtitle: markdownField(from: "subtitle")
}

I resolved this a little different.

This doesn't quite work for me. First I get a error rejecting the defaultValue: true for the argument from - it must be a string. Changing that to defaultValue: '', I then get this error:

Encountered an error parsing the provided GraphQL type definitions:
Argument "from" of required type "String!" was not provided.

  1 |
  2 |     type MarkdownRemark implements Node @infer {
  3 |       frontmatter: Frontmatter
  4 |     }
  5 |     type Frontmatter {
> 6 |       markdownField: String! @md
    |                              ^
  7 |     }

This I don't know how to solve.

For anyone else that's interested I solved it using a custom YAML type to allow parsing any arbitrary field as markdown like so:

---
title: My Page
inline: !md Some **bold** and _italic_ text
block: !md |
  ## I'm a H2 title
  [I'm an inline-style link](https://www.google.com)
---

To do so create a custom type and then override grey-matter's YAML parser:

// custom-yaml.js
const yaml = require('js-yaml')
const remark = require('remark')
const remarkHTML = require('remark-html')

const MarkdownYamlType = new yaml.Type('!md', {
  kind: 'scalar',
  construct: data => remark().use(remarkHTML).processSync(data).toString(),
})

const MARKDOWN_SCHEMA = yaml.Schema.create(MarkdownYamlType)

module.exports = doc => yaml.safeLoad(doc, { schema: MARKDOWN_SCHEMA })

// gatsby-config.js
module.exports = {
  plugins: [
    {
      resolve: `gatsby-transformer-remark`,
      options: {
        engines: { yaml: require("path/to/custom-yaml.js") },
      },
    }
  ]
}

I get 'Invalid plugin options for "gatsby-transformer-remark":' if i try this method?