Focus On: HelpNDoc (part 2)

This is the second part of a series of blog posts taking a close look at HelpNDoc. Here are links to the other posts in the series:

• part one
• part three
• part four

In the first part of this deep dive into using HelpNDoc we ended up with a complete help document for my Unit2NS program. There was more to do though, which I listed at the end of that post:

1. Scripting.
2. Conditional content.
3. Source code generation.
4. Keywords.
5. Inhibiting the Copyright message in help topics.
6. Custom variables build tags.
7. Customising document output.
8. Various topic properties not covered previously.

In this post we'll be taking a close look at the first two items in the list. The others will be returned to in subsequent posts.

Scripting

In part 1 I said I wanted to create four different document types from our single HelpNDoc project:

1. HTML Help file.
2. HTML5 web site.
3. Markdown.
4. PDF.

We managed to generate 1, 2 & 4 in part 1, but we left the Markdown version for later. Recall that, although HelpNDoc could have generated Markdown files for each topic (and for the table of contents), I wanted to change how it was presented. There were two problems with the default output.

The problems

TOC

By default, when generating a Markdown document, HelpNDoc generates a Table Of Contents page that links to each topic. This is just what I wanted but I noticed that, when one of the topics is empty and has child nodes, the TOC it generates has the wrong structure. The following image illustrates the problem:

Comparison of generated Markdown TOC with project outline

On the left is a screen grab of the Markdown TOC generated by default (previewed in VS Code). All the entries are links to the associated Markdown topic files. On the left is the document outline (from the HelpNDoc UI). It can be seen that the Using the Program topic has not been rendered. This is, I suppose, because that's the only topic that is empty, i.e. has the topic kind of "empty". All the child nodes of Using the Program have been placed (incorrectly) under the Getting Started parent topic.

I decided to fix that by including the Using the Program heading in the rendered TOC, but without linking it to anything.

Breadcrumbs

While HelpNDoc helpfully generates the TOC with links to all the topic files, there is no corresponding link back to the TOC in those topic files. Instead of adding simple links back to the TOC page, I decided implement a breadcrumb style trail from each topic back to all its parent pages, up to the TOC. Of course I would need to avoid empty topic pages within the trail having links - they must appear in plain text.

I could have hard coded the breadcrumb trails into the HelpNDoc project, but I didn't want to do that for two reasons:

1. If I later moved any topic to a different section I would have to update the breadcrumb trail manually. Can you imagine the resulting bugs?
2. I didn't need the breadcrumb trail in any other representation of the document, e.g. when generating HTML output. (Yes, I know there's a simple way to hide the breadcrumb trails from other document formats, but only when they're hard-coded. The Conditional Content section below explains.)

It should come as no surprise that both the TOC and Breadcrumb problems were resolved using scripting.

Document Generation Using Scripts

Behind the scenes, HelpNDoc uses scripts to generate its output for several of its supported document types, including Markdown. Each such document type has a default template that can be customised. One way to do this is to duplicate the default template under a new name, tell HelpNDoc to use the new template, and then customise the scripts that the new template inherited from the default template. Wisely, HelpNDoc doesn't let you modify the scripts used by the actual default template.

To duplicate the default template, click the body of the Generate Help button on the Home ribbon to display the Generate documentation dialogue box. Select the Build markdown documentation item in the Build list. The Build settings pane will change to show the Default Markdown template. Now click the Edit link to display the Edit template dialogue box, then click the button with the double folder icon to duplicate the default template. Give the new template a suitable name in the resulting Duplicate template dialogue box. I called it Unit2NSMarkdown.

Creating a duplicate Markdown template

Now we've created the new template we can return to the Generate documentation dialogue box and click the blue Default Markdown Template link to drop down a menu. From that menu select the name of the template you just created:

Selecting newly created Markdown template

HelpNDoc uses two different script files to generate the Markdown output:

1. index.pas.md - generates the "index" page, i.e. the Table Of Contents page.
2. topics.pas.md - generates the individual topic pages.

As the .pas part of the file extension implies, the scripting language is Object Pascal based. Happy days! But the template is actually Markdown. Any Markdown in this script will be output verbatim. The Pascal script is embedded in the Markdown content within one or more pairs of <?...?> delimiter tags. The Pascal script outputs text into the Markdown document using one of the built in printXXX functions. In our case we won't be writing any fixed Markdown - all the content will be rendered using scripting.

HelpNDoc has a script editor that we can use to edit the code. As we left things, we had the Generate documentation dialogue box open and the Unit2NSMarkdown template selected. To get to the script editor:

• click the Edit link to re-display the Edit template dialogue box;
• pick the Script Files page from the Edit template pane;
• choose the script you want to edit in the Script files pane;
• click the Edit Script button to display the script ready for editing in the Script authoring and execution dialogue box.

Phew!

Getting the script ready for editing

Editing The Scripts

At last we get to edit the scripts.

I'm not going to pretend that the scripts I present below were my first attempt - far from it. In fact at first I didn't notice the "empty topics" problem: My TOC page was broken, as noted above, and I had broken links to empty topics in the breadcrumb trail! Eventually, quite a bit of head scratching and many revisions later, I came up with the following scripts.

Table of contents

Because we've duplicated the default template we have a copy of the default index script (index.pas.md) to work on. Here's my revised version:

HndGeneratorInfo.CurrentFile := ExtractFileName(HndGeneratorInfo.OutputFile);

println('# ' + HndTopics.GetTopicCaption(HndTopics.GetProjectTopic()));
println('');

println('## Table of contents');
println('');

// Get a list of generated topics
var aTopicList := HndTopicsEx.GetTopicListGenerated(False, False);

// Each individual topics...
for var nCurTopic := 0 to length(aTopicList) - 1 do
begin
  // Notify about the topic being generated
  HndGeneratorInfo.CurrentTopic := aTopicList[nCurTopic].id;

  // Topic kind
  //if (aTopicList[nCurTopic].Kind = 1) then continue;  // Empty topic: do not generate anything

  // URL
  var sTopicUrl := '';
  if aTopicList[nCurTopic].Kind = 2 then sTopicUrl := HndTopics.GetTopicUrlLink(HndGeneratorInfo.CurrentTopic)  // Link to URL
  else if aTopicList[nCurTopic].Kind = 1 then sTopicUrl := ''  // Empty
  else sTopicUrl := format('%s.md', [aTopicList[nCurTopic].HelpId]);  // Normal topic

  // Indentation
  for var nLevel := 2 to HndTopics.GetTopicLevel(HndGeneratorInfo.CurrentTopic) do
    print('  ');

  // Caption
  if sTopicUrl <> '' then
    printfln('- [%s](<%s>)', [HndTopics.GetTopicCaption(HndGeneratorInfo.CurrentTopic), sTopicUrl])
      else
    println('- ' + HndTopics.GetTopicCaption(HndGeneratorInfo.CurrentTopic));
end;

You will notice that there are many identifiers of the form HndXXXX. These are built in global objects that give access to the HelpNDoc API. I'm not going into the meaning of each of these, because that is explained better than I can do it in the HelpNDoc API documentation.

This code is only slightly modified from the default script. Here's a brief explanation of how it works  (as far as I understand it anyway).

Firstly the HndGeneratorInfo object has the correct file name set - this is the file that will be used for output. This is pretty much boilerplate code you'll need in every TOC script.

Next, the correct caption for the project's root topic is output - Unit2NS in my case. The HndTopics global object is used to get the required information. We'll be using this object, and the associated HndTopicEx object, a lot in this code to get information about various topics.

Now we store a copy of a list of the project's topics in aTopicList and iterate through that list. Within the loop we use another piece of boilerplate to tell the generator which topic we are currently outputting.

In the original code (commented out) the rest of the loop was skipped if the topic kind was 1, which indicates an empty topic. It was this line that was causing the incorrectly formatted output in my project by ignoring my empty topic, despite it having child topics.

The lines following the // URL comment find any URL associated with the topic. For topic kind 2 an external URL is used, while for normal topics (kind 0), a link to the related topic file is used. The name of this file is taken from the topic's Help ID property (of which more in part four). Note that, in the original code the line for topic kind 1 would never be reached, and is actually redundant here anyway, since it replicates the action of the catch all var sTopicUrl := '' line. It could have come out, but I let it be. To be honest, I didn't notice it until just now!

Next, the level of indentation of each topic list item is calculated using topic's level in the hierarchy: two spaces are written for each level of nesting, as is required in Markdown.

Finally we get to the second bit of code I changed. Originally the printfln built-in function was called for every topic to create a list item containing a Markdown link to the topic file, but I changed that so that the printfln function is only called if the topic is non-empty. For an empty topic, a list item containing the topic "caption" in plain text is output.

This last change satisfies my requirement to include the names of empty topics in the TOC, without linking them anywhere.

The following image shows the TOC output from the revised script, once again rendered in VS Code. You can see that the TOC now matches the project outline:

Comparison of revised Markdown TOC with project outline

Breadcrumbs

Now let's look at the breadcrumb code. Because this is to appear in each topic we need to edit topics.pas.md which, as we now know, is a copy of the default script. I made quite a lot of changes to this script. Here's the final version:

function AppendBreadcrumbLink(const Breadcrumbs: string;
  const Text, Link: string; const HasContent: Boolean): string;
begin
  if HasContent then
    Result := '[' + Text + '](<./' + Link + '>)'
  else
    Result := Text;
  if Breadcrumbs <> '' then
    Result := Result + ' >> ' + Breadcrumbs;
end;

// Get a list of generated topics
var aTopicList := HndTopicsEx.GetTopicListGenerated(False, False);

// Each individual topics...
for var nCurTopic := 0 to length(aTopicList) - 1 do
begin
  // Get ID of topic being generated
  var iID := aTopicList[nCurTopic].Id;

  // Notify about the topic being generated
  HndGeneratorInfo.CurrentTopic := iId;

  // Topic kind: if it's an empty topic (Kind = 1) don't generate anything
  if (aTopicList[nCurTopic].Kind = 1) then
    Continue;  

  // Build the file name
  HndGeneratorInfo.CurrentFile := aTopicList[nCurTopic].HelpId + '.md';

  // Output title
  println('# ' + HndTopics.GetTopicCaption(HndGeneratorInfo.CurrentTopic));
  println('');

  // Calculate breadcrumbs
  var sBreadcrumbs = '';
  var sParentID := HndTopics.GetTopicParent(iID);
  var iLevel := HndTopics.GetTopicLevel(sParentID);
  while iLevel > 0 do
  begin
    sBreadcrumbs := AppendBreadcrumbLink(
      sBreadcrumbs, 
      HndTopics.GetTopicCaption(sParentID),
      HndTopics.GetTopicHelpId(sParentID) + '.md',
      HndTopics.GetTopicKind(sParentID) = 0
    );
    sParentID := HndTopics.GetTopicParent(sParentID);
    iLevel := HndTopics.GetTopicLevel(sParentID);
  end; 
  sBreadcrumbs := '::: *' 
    + AppendBreadcrumbLink(
        sBreadcrumbs,
        'Home',
        ExtractFileName(HndGeneratorInfo.OutputFile),
        True
      ) 
    + ' >> ' 
    + HndTopics.GetTopicCaption(iID)
    + '*';

  // Output breadcrumbs
  println(sBreadcrumbs);
  println('');

  // Output content
  print(HndTopics.GetTopicContentAsMarkdown(HndGeneratorInfo.CurrentTopic));
  println('');
end;

First up I've created a function - AppendBreadcrumbLink - that adds a suitable breadcrumb item to an existing breadcrumb string, Breadcrumbs. It creates a Markdown link from the given Text and Link parameters unless HasContent is false, in which case Text is simply added to the breadcrumb string and Link is ignored. (BTW I just noticed that AppendBreadcrumbLink should really be called PrependBreadcrumbLink since it prepends, rather than appends, the new breadcrumb to the existing string!)

Now we come to the body of the code. The line following // Get a list of generated topics is from the default code - it simply gets a list of all topics in the project for us to iterate over.

The next line in the original code was a redundant variable assignment that I removed.

Sticking closely to the standard code again, we begin looping through the topic list. Within the loop we first record the ID of the current topic (I added this line). Next we inform the generator object of the current topic we're operating on. Now we check for an empty topic (Kind = 1) and skip the rest of the code as there's nothing to output for such a topic. We then tell the generator what file name to use for the current topic's Markdown page before outputting the topic title as a level one Markdown heading.

My new code starts in earnest at the // Calculate breadcrumbs comment. The sBreadcrumbs variable is initialised to the empty string: it is used to build up the Markdown for the whole breadcrumb trail. Next sParentID is set to the ID of any parent topic of the current topic and iLevel is set to the level of the current topic (where the level represents how deeply the topic is nested, with 0 being the top level).

The following while loop builds the breadcrumb trail, starting from the current topic's parent (which it must have if it's level is greater than zero), and working up from the parent topic's parent, until we reach the root. AppendBreadcrumbLink is then called to add each item to the breadcrumb trail. The caption of the relevant topic is displayed in the link while the help topic ID is used to name the Markdown file being linked to. (The topic's caption is what appears in HelpNDoc's Table of contents pane and the topic ID can be seen - and edited - by right clicking the topic in the Table of contents pane.) The HasContent parameter of  AppendBreadcrumbLink is set true only if the parent topic's kind is 0 - i.e. it's a topic with content.

Before the end of the loop the sParentID and iLevel variables are updated to walk up the chain of parent topics.

Once the loop ends, the link back to the Table Of Contents page is prepended to the breadcrumb trail and the caption of the current topic is appended as plain text. The trail is made to appear in italics by use of the "*" Markdown italics delimiter. Note that when the current topic's level is zero the loop is not executed at all.

Here's a montage of a few of the help topics with the breadcrumb trail in place, rendered in VS Code. It demonstrates a long trail which includes the Using the Program empty topic (top), a top level topic (middle) and a topic that has a non-empty parent topic (bottom):

Montage of renders of various Markdown topic pages

Now there's a lot more to scripting than I have gone into here, but I hope I've presented a practical example of how scripts can be used in a real life project.

It was mentioned above that scripting is available for output formats other than Markdown. These are: CHM, HTML, EPUB, MOBI and QTHELP.

Conditional Content

Related to scripting, but much easier to use, is the ability to conditionally include or exclude content. Conditional tests depend on the value of what HelpNDoc calls "build tags". Quite frankly I've not dug into build tags in any depth yet. But, for the purposes of this example, I only needed to use one of HelpNDoc's pre-defined build tags.

The Problem

In the Lookup a unit topic I used an animated GIF. That works fine in HTML, CHM and Markdown documents, but the GIF doesn't animate in PDF output. Well, not in my reader anyway.

Consequently I don't want to display the GIF in the PDF version of the documentation. How to do that? Unsurprisingly I'm going to conditionally exclude the GIF from PDF output.

The Solution

First decide on the block(s) of text to be conditionally rendered. Place the cursor at the start of the block then click the Insert / Edit condition button on the Insert ribbon. This displays the Insert conditional operation dialogue box:

Insert conditional operation dialogue box - entering if condition

Here I've selected the If operation, the IF NOT test, and chosen the pdf tag. This tells HelpNDoc to only render the following content if the pdf tag is not defined. Given that pdf is only defined when rendering a PDF document, this is what we want.

But where is the end of this block of conditionally rendered text? In my case it's the end of the topic. So I closed the above dialogue box by clicking OK, moved the cursor to just after the animated GIF then clicked Insert / Edit condition again to re-display the dialogue box. This time I selected the End operation and clicked OK once more:

Insert conditional operation dialogue box - entering end condition

As you can see, you can also have an If..Else..End construct to render alternative content depending on the state of some build tag, but I didn't need to do that. I suppose I could have included an alternative image or images in the PDF, and might return to that.

Here's how the document appears in the HelpNDoc editor after both of the above stages:

Appearance of conditional tags in HelpNDoc editor.

If you now render a PDF document the text and image surrounded by the IF NOT PDF and END tags will not be included, but it will be included in other document types.

Next Time

There are still some outstanding items to cover. They are:

1. Source code generation.
2. Keywords.
3. Inhibiting the Copyright message in help topics.
4. Custom variables build tags.
5. Customising document output.
6. Various topic properties not covered in parts 1 & 2.

The first three of those items are covered in part three and the rest are discussed in part four.

Project Files

As in part 1, I've included the a zip file containing the project as it was after writing the scripts and adding the conditional code. It's available from my Google Drive account.

→ Download the code