Skip to content
mdlake

Importing content

How to bring existing material into the lake - a Word, PowerPoint or PDF file as one draft, or a whole folder of Markdown files with images as many drafts. Covers the dry run, re-imports, the frontmatter fields including order and _folder.yaml, limits and the report.

On this page

You do not have to retype what you already have. mdlake has two ways to bring content in. Both create drafts, so nothing is public until you publish it.

You haveUseWho can do it
One Word, PowerPoint or PDF fileImport a Word, PowerPoint or PDF fileEditors and admins
A folder of Markdown files, with imagesMarkdown folder importAdmins, and tools with a token that has the import permission

Before you start

  • You have a workspace and you are a member of it (Users and invites).
  • For Word, PowerPoint and PDF files, the document converter service runs. It is part of the standard install (Install the CMS). Without it, the import says “The document converter is not available right now. Try again in a moment.”

Import a Word, PowerPoint or PDF file

This turns one file into one draft document. The file itself is not kept. If you want to keep the original too, upload it to the media library as well (The media library).

  1. Open the workspace home page. It always shows three cards at the top.
  2. Choose Import Word, PowerPoint or PDF (“Turn a file into a draft document”). Pick a .docx, .pptx or .pdf file.
  3. Wait while the card says Importing….

Expected result: a new draft opens in the editor. If the file had pictures, or something needs your attention, a notice appears first: “Fleet services is a draft now. 3 pictures were added to the media library.” Choose Open the document, or Open the media library.

What the import does

  • Title. The title in the file is used. If there is none, the file name is.
  • Text. The converter turns the file into Markdown: headings, paragraphs, lists and tables. Anything the editor cannot show may be flattened.
  • Folder. The draft goes into the folder you are working in.
  • Pictures in Word and PowerPoint files become media files in a folder named after the document. The alt text comes from the picture description in the file, when the author wrote one. Pictures with identical content are stored once and reused.
  • Pictures in PDF files are not extracted. Only the text is imported. The notice says “Images in PDF files are not extracted yet; only the text was imported. Upload the pictures to the media library and add them to the document.”
  • Pictures without a description get no alt text. They are listed under Needs attention on the home page, and cannot be published until you add it (The media library).

Limits and failures

What you seeWhat to do
“The file is larger than 25 MB.”Use a smaller file, or split it. The limit is the media upload limit (Assets:MaxUploadBytes).
“Choose a Word (.docx), PowerPoint (.pptx) or PDF file.”Other types are refused. The check looks at the contents, not the name. Old .doc and .ppt files must be saved as .docx and .pptx first.
“The text of this file could not be read. It may be damaged or password protected.”Remove the password or save the file again.
“The document converter took too long. Try again or use a smaller file.”Try again, or a smaller file.
“The file is empty.”Check that you chose the right file.

After the import, read the draft with care. Check headings, tables and lists, then add a summary and tags (Writing documents).

Markdown folder import

This is for a larger move: a documentation folder, an export from another system, or a site’s content. You give the CMS a folder of .md files with the images and files they use. Folders become folders, images become media, and links between the files become lake:// links that survive renaming.

It is a feature for admins. The menu item Import Markdown… under New is shown only to them. Tools can do the same through the API with a workspace token that has the import permission (Workspace tokens).

Four rules protect your workspace:

  1. Everything arrives as a draft. The menu never publishes.
  2. Nothing is ever deleted. A file that disappears from your folder does not remove the document it made earlier.
  3. You see a dry run first, and nothing is written until you confirm.
  4. The import can be repeated. A second run updates the drafts it made before.

Prepare the folder

  • Use .md or .markdown files. Put images and files (.jpg, .jpeg, .png, .webp, .pdf, .docx, .pptx, .xlsx) next to the Markdown that uses them, and refer to them with relative paths, as in ![Fleet map](images/map.png).
  • Link between files with relative paths too: [Pricing](../pricing.md). A link to a heading works when the heading is lower-case words joined by hyphens, as in pricing.md#per-seat.
  • Give each file frontmatter if you want control. Every field is optional:
YAML
---
id: 01M5YPMJRG2SN2C75GJWV9DAWF
title: Fleet services
summary: What the fleet team offers and how to book it.
tags: [services, fleet]
slug: fleet-services
template: service
order: 20
---
FieldWhat the import does
titleThe document title. Without it, the first level-1 heading is used and removed from the text. Without that, the file name. The title is cut at 200 characters.
summaryThe summary. description is read as a summary when summary is missing. Over 300 characters it is shortened and a warning is given.
tagsA list, or one comma-separated string. The workspace’s own spelling is used when a tag exists. Invalid tags are dropped with a warning. At most 20 are kept.
slugKept. A value that is not lower-case hyphenated words is converted if possible (Writing documents).
templateKept. A hint for the site about which layout to use. Up to 100 characters.
idThe document’s ID, a valid ULID. If a draft with that ID exists in the workspace, the file updates it. If the ID is new, it is used for the new document. An ID that belongs to another workspace, or to a document in the trash, is ignored and the file gets a new ID.
orderThe document’s place among the items of its folder: a number, lower first. The CMS keeps it as the document’s order and not as an extra field, so you can drag the document to another place afterwards (Reorder documents and folders). A number with decimals is rounded to a whole number. Text that reads as a number, such as "20", is accepted. Without order, the document keeps the place it has.
any other fieldKept as an extra field of the document and published with it. A site can read it.
status, created, updatedManaged by the CMS. Not imported.

The first-level heading (# Title) is removed from the body because the title is a field. Other headings are moved into the levels the editor has (2 to 4). Raw HTML, MDX components and other things the editor cannot show stay as written, and are reported. Fenced code blocks stay as they are, and indented code gets a fence so that it opens as a code block.

A folder can carry its own place in the same way. Put a file called _folder.yaml inside the folder, with one line:

YAML
order: 20

The import reads it, gives that folder the number, and does not import the file as anything else. Other keys in the file are ignored. A _folder.yaml at the top of the zip does nothing when you import into the top level of the workspace, because the top level has no parent to be ordered in. When you import into a folder (Put it in), the top-level _folder.yaml of the zip orders that folder.

Folder names in the zip become document folders. Names that do not fit the folder rules are fixed, and the report says so. Hidden files and folders (names that start with a dot), node_modules and __MACOSX are left out.

Run the import

  1. Open the workspace. Choose New, then Import Markdown…. The dialog Import Markdown opens.
  2. Choose Choose a zip, or Choose a folder. A folder is zipped in your browser. The line under the buttons says what you picked, for example “Folder docs: 142 files (8.1 MB), 12 other files left out”.
  3. Under Put it in, pick the folder in the CMS where everything should go. The default is the folder you have open. The line below says “The top level of the workspace.” or “Inside guide.”.
  4. Decide on Replace drafts that were edited here since the last import. Leave it off the first time. Off means that documents someone changed in the CMS are left alone and listed in the report.
  5. Choose Check. The dialog says Checking… and then Check the import.
  6. Read the plan, described below. Nothing has been written yet.
  7. Choose Import 24 documents as drafts. The button says Nothing to import when there is nothing to do, and is disabled.
  8. The dialog Importing… shows “12 of 40 done”. You can leave it open while it works.
  9. When it says Import finished, read the report. Choose Done.

Expected result: the folders and drafts appear in the tree and the pictures in the media library. Each document shows a hollow dot, because it is a draft.

If you close the dialog after checking and do not confirm within 60 minutes, the staged upload is thrown away and you must start again.

Read the plan and the report

The plan before you confirm and the report after the import have the same shape. They show eight numbers: documents to create, update, skip or fail; media to upload or already there; links rewritten; links kept as text.

Below the numbers, a sentence says “Nothing needs your attention.” or “3 warnings and 0 errors to look at.” Tick Only show items with warnings or errors to focus on those.

The sections are:

  • Documents. Each row shows the title, the source path and a badge: Will be created, Will be updated, Will be updated and moved, Unchanged, Edited in the CMS, left alone, Being edited, left alone, or Cannot be imported. Open a row to see where the file Becomes, its Id and where the ID came from (from the file, from an earlier import or new), the tags, the messages, and Kept as written, the links that could not be resolved, with their line numbers.
  • Media. Each row shows a file and Will be uploaded, Already in the library or Cannot be imported. Open a row for the folder, the size, the alt text and the documents that use it.
  • In the workspace, not in the upload. Documents that an earlier import made, which are not in this upload. Nothing is deleted. Remove them yourself if they are gone for good.
  • Ignored files. Files left out, each with a reason: not used by any document, file type not imported, no file extension, path too long, hidden file.

After the import, Download report (JSON) and Download report (Markdown) save the report. The Markdown version is readable and good to keep.

Messages come in three levels: information, warnings and errors. Messages about order are:

MessageLevelMeaning
“The order “soon” is not a number and was dropped.” (order-invalid)WarningThe order of a document is not a number. The document is imported without it.
“The order 2.5 is not a whole number and became 3.” (order-rounded)InformationThe number was rounded.
”‘_folder.yaml’ is at the top level, which has no parent to be ordered in; it was left out.” (folder-file-root)InformationA _folder.yaml at the top level of the workspace is ignored.
“The order in ‘Guides/_folder.yaml’ is not a number; it was left out.” or “‘Guides/_folder.yaml’ could not be read (…); it was left out.” (folder-file-invalid)WarningThe file is not valid YAML, has no number, or sits in a folder whose name cannot be used. That folder gets no number.
“2 folders got their order from _folder.yaml.”InformationHow many folder numbers the import will set.
“The order of the documents and folders could not be set. Import again to retry.” (order-failed)WarningThe documents and folders were imported but the numbers were not. Run the import again.

Typical warnings are “images/map.png is not in the upload; kept as written”, “No alt text in the Markdown. Add it in the media library: an image without alt text cannot be published.” and “TOML frontmatter (+++) is not supported; it was kept as text.”

Import again

Run the import again with a changed folder to update what you imported before. The CMS recognises a document by, in this order, the id in its frontmatter and the path it had in the earlier import.

The path includes the folder under Put it in, so choose the same folder as the first time. In another folder, files without an id become new documents. When most of the new documents match what an earlier import put in another folder, the plan warns you and names that folder (other-folder).

For each file, the plan decides:

SituationResult
The file is newA new draft is created
The file changedThe draft is updated
The file did not change since the last importUnchanged, skipped
Someone changed the draft in the CMS after the last importEdited in the CMS, left alone. Tick the box Replace drafts that were edited here since the last import to overwrite it
Someone is editing the document right nowBeing edited, left alone
The file moved to another folder in the sourceThe document is updated and moved, keeping its ID and its links
The document an earlier import made is in the trashA new document is created
A file is gone from the sourceNothing happens. It is listed under In the workspace, not in the upload

Media is matched by content, wherever the file was. Identical files are stored once. A changed image is uploaded as a new file, and the old one is kept.

Published documents keep their published version until you publish again. An import never changes the text of what is public. The one exception is order: when the import gives a published document or folder a different number, the CMS writes the new number to the lake in a small commit, Reorder: <folder>, as it does when you drag (Reorder documents and folders). Documents that were left alone because they were edited in the CMS or are being edited keep their number.

Limits

These limits are checked while the zip is read. Admins can change them under MarkdownImport (CMS configuration).

LimitDefaultSetting
Size of the zip100 MBMarkdownImport:MaxZipBytes
Files and folders in the zip2,000MarkdownImport:MaxEntries
Everything together, unpacked300 MBMarkdownImport:MaxTotalBytes
Packing ratio of a file of 1 MB or more200 timesMarkdownImport:MaxCompressionRatio
Wait between Check and Import60 minutesMarkdownImport:StagingMinutes
One image or file25 MBAssets:MaxUploadBytes
One Markdown document2,000,000 charactersnot configurable
Path of a file260 charactersnot configurable

The zip is also refused when it contains an absolute path, a .. path, control characters in a name, a symbolic link or two entries with the same path. These checks protect the server.

If something goes wrong

What you seeWhat to do
“That folder has no Markdown files (.md) in it.”Choose the folder that holds the .md files.
“The file is not a readable zip archive.”Make the zip again.
“The zip has 2400 entries; at most 2000 are accepted.”Split the folder, or ask an admin to raise MarkdownImport:MaxEntries.
“The files add up to 340 MB unpacked; at most 300 MB are accepted.”Remove large files, or raise MarkdownImport:MaxTotalBytes.
“The checked upload has expired”Choose the zip or folder again.
“The import was already confirmed”Each check can be confirmed once. Start again to run it a second time.
Cannot be imported on a documentOpen the row. The reason is one of: the frontmatter cannot be read, the document is too large, the path is too long, or the file is not text.
A link is Kept as writtenThe target is not in the upload or earlier imports. Add it and import again, or fix the link in the editor.
Images appear but cannot be publishedThey have no alt text. Add it in the media library (The media library).

What’s next