Here's the code I was just in the middle of writing when I realized I should stop and try to use webpack.
First off, what is this code trying to do? There's a babel transform, a vm context passing code strings around, and a React server render. Basically, I was trying to achieve getting HTML output for MDX content in gatsby-mdx. If you're server-rendered React before, you'll see that this seems like very complicated code to simply render a component to an html representation.
so... why is it so complicated?
The short answer is that gatsby-mdx rips the imports out of
every MDX node and stores the imports in separate files for
each node in a special directory called
MDX_SCOPES_LOCATION. So, since anyone could be using the
config for shortcodes, we have to go collect all of the
possible scopes so that MDX content can be rendered with the
React components they expect to have available.
The scope files are written using ES2015 module syntax, and are usually just a set of imports and an export.
This ES2015 syntax means that we can't require these files
into a node process without compiling them (remember that
although MDX users tend to be on the later releases, Gatsby
currently supports all the way back to node 6). Once
scooping up all of the files as strings, we need to compile
them and combine them into a single scope object to pass to
MDXScopeProvider, just like we do for
normal operation in wrapRootElement.
so... how did I know it was time for webpack?
I was making only incremental progress (first
then babel, then
vm, etc) and while incremental progress
is good, there are a few thoughts I had that told me there
was a leap I could make.
As soon as I chose to switch to using Webpack for this, I made progress quickly. There are a couple of keys to understanding what this code is doing.
cloneDeep. We take Gatsby's processed webpack config from the redux store so that the scopes and anything else we have to process are dealt with in a predictable way. By doing this, we could accidentally overwrite the webpack config and end up with the same HTML output for every
MemoryFSis an extremely simple in-memory filesystem that we can use in place of the real filesystem. Using an in-memory filesystem means we don't have to worry about accidentally overwriting any files needed for the real Gatsby build.
localsand define global variables easily with
Now, this is a heavy key in the GraphQL API because of all
the additional webpack work (a webpack process for every
html field queried). The approach I'm taking here is:
"Make it work, make it right, make it fast". We're still in
the "Make it work" phase. We will need to speed this up or
cache it heavily in the future.