Documentation references/bibliography with Sandcastle

Apr 22, 2008 at 12:04 AM
Hi,

I have a list of documents (scientific books, articles and internal research papers) that I want to cite in my classes library comments and the conceptual topics of the library user guide.
The list forms somehow a database of references which I want to maintain in xml format. Each item in the list represents a biblio entry that has an id, author, title, link, ...etc
Does MAML have something similar to this to define a bibliography?

Moreover, the biblio items must be referenced with their id such as <see href="id" /> and the reference is replaced with the item short name or number in the list for example, depending on the style used (idea inspired from LaTeX).

For the moment, I looked at "address" attribute of MAML elements to create links. I also looked at build component from DocProject project called ResolveExternalLinksComponent. But I couldn't come up with something that works as desired.

Is their anything in MAML that handles this issue that I have missed? Any hints to make this works will be of a great help.

Thanks
SY
Apr 22, 2008 at 1:49 AM
Hi SY,

Could you be more specific as to what the limitations of ResolveExternalLinksComponent are? I'll add support for your needs if it makes sense for this component :)

You may be able to accomplish what you need, however, without using ResolveExternalLinksComponent. (Note also that I haven't tested it in conceptual builds.)

If you check the conceptual configuration file you'll see a component being used to replace tokens. It works like shared content.

Create a shared content file in the Content folder and name it tokens.xml:

<content xml:space="preserve" xmlns:MSHelp="http://msdn.microsoft.com/mshelp">
  <item id="ref1">the first book.</item>
  <item id="ref2">the second book.</item>
</content>
Then you can use the content items that it defines in conceptual builds. I've only hard-coded text in this example, but you can place HTML in the items as well.

(Note that shared content items will not work recursively, although they do in reference builds.)

To use token replacement in conceptual builds, update the first instance of SharedContentComponent in the conceptual configuration file so that it looks similar to this (I'm using a DocProject path here):

<!-- resolve tokens -->
<component type="Microsoft.Ddue.Tools.SharedContentComponent" assembly="%DXROOT%\ProductionTools\BuildComponents.dll">
  <context prefix="ddue" name="http://ddue.schemas.microsoft.com/authoring/2003/5" />
  <content file="..\..\Help\Presentation\Style\Content\tokens.xml" />
  <replace elements="/*//ddue:token" item="string(.)" />
</component>
Now you can use the token element in MAML.

<para>
  <token>ref1</token>
</para>
Unfortunately, you can't include these content items in XML documentation comments the easy way (probably since the include tag is removed by C# and VB.NET compilers). The only way that I can think of is to define an XSL transformation for a custom XML documentation tag and add it to main_sandcastle.xsl; something like this:

<xsl:template match="token">
  <xsl:element name="include">
    <xsl:attribute name="item">
      <xsl:value-of select="."/>
    </xsl:attribute>
  </xsl:element>
</xsl:template>
Then you should be able to use the token element in your XML documentation as well.

(Note that I haven't tested this approach for reference documentation so the XSL will probably need some adjustment.)

- Dave
Apr 24, 2008 at 1:14 PM
Hi Dave,

Thank for the fast reply.

I don't see limitations in ResolveExternalLinksComponent itself but as you have said maybe I can accomplish what I need without ResolveExternalLinksComponent. In fact, I am going to using ResolveExternalLinksComponent in general in my documentation. Nevertheless, maybe a feature of custom replacement of the self-closing tag <see xref="id" /> would be useful. Just a thought.

As for the solution/ideas that you have proposed for adding a References or Bibliography to sandcastle documentation: I don't master xslt or sandcastle workflow but I have some questions:
- How to create the Bibliography section in the documentation with an anchor on each item?
- Will the <token>'s tag be replace by a link to the item in the bibliography or do i have to put the link the text of the <item>'s content?

In fact I wasn't aware of the difference between the code and conceptual differences, thanks for the clarification.

Just to make things more concrete, I wish to have some thing similar to the bibliography of scientific documents or any book in general (like in docbook or latex (bibtex))
  • Usage example
    • In the comments: /// This method implements the algorithm described in <cite ref="doc2" />
    • As output: This method implements the algorithm described in[2]
  • The Bibliography section:
[1]Author 1, document1 title, publisher
[2]Author 2, document2 title, link
...etc

where the content is described in xml, for example:
<item ref="doc2">
<author>Author 1</author>
<title>document1 title</title>
<link></link>
<publisher></publisher>
</item>

Thanks,
sy
Apr 24, 2008 at 2:09 PM
Hi Sy,

I like this idea a lot. I have a few things I'm going to try now and I'll get back to you soon.

- Dave
Apr 24, 2008 at 5:59 PM
Edited Apr 25, 2008 at 1:23 AM
EDIT: Correction to the bibliography_common.xsl file

Hi Sy,

I was able to successfully implement an automatic bibliography feature in the VS 2005 presentation style for both conceptual and reference builds.

If you have any trouble following these steps, please let me know and I'll try to explain them better.

Note: I'm going to post the solution for conceptual builds in my next post since I ran out of space in this post, according to CodePlex :D

Auto-Generated Reference Bibliography

First, we'll start with the required updates for reference documentation. Open up the Transforms\skeleton.xml file and add a bibliography element.

<document>
  <reference />
  <bibliography /> <!-- add this - it doesn't really matter where since it's only for meta data -->
  <syntax />
  <comments />
  <metadata />
</document>
Now open up Transforms\main_sandcastle.xsl and add an additional include element near the other include elements at the top of the document.

<xsl:include href="bibliography.xsl"/>  <!-- add this -->
 <xsl:include href="htmlBody.xsl"/>
  <xsl:include href="utilities_reference.xsl" />
Look down a bit and you'll see <xsl:template name="body">. Near the end of this template, add a call to our new bibliography template (which we haven't defined yet).

    <!--versions-->
    <xsl:if test="not($group='list' or $group='namespace' or $group='root' )">
      <xsl:apply-templates select="/document/reference/versions" />
    </xsl:if>
    <!-- bibliography -->
    <xsl:call-template name="bibliography" />  <!-- add this here  ** location matters ** -->
    <!-- see also -->
    <xsl:call-template name="seealso" />
Now create a new file in the Transforms folder and name it, bibliograpy.xsl. Here are the contents:

Important: If you create the file in Visual Studio do not let it auto-format the document.
<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 
  <xsl:include href="bibliography_common.xsl"/>
  
  <xsl:key name="citations" match="//cite" use="text()" />
 
  <xsl:variable name="hasCitations" select="boolean(count(//cite) > 0)"/>
 
  <xsl:template match="cite">
    <xsl:variable name="currentCitation" select="text()"/>
    <xsl:for-each select="//cite[generate-id(.)=generate-id(key('citations',text()))]"> <!-- Distinct citations only -->
      <xsl:if test="$currentCitation=.">
        <xsl:element name="a"><xsl:attribute name="href">#cite<xsl:value-of select="position()"/></xsl:attribute>[<xsl:value-of select="position()"/>]</xsl:element>
      </xsl:if>
    </xsl:for-each>
  </xsl:template>
 
  <xsl:template name="bibliography">
    <xsl:if test="$hasCitations">
      <xsl:call-template name="section">
        <xsl:with-param name="toggleSwitch" select="'cite'" />
        <xsl:with-param name="title">
          <!--
          <include item="bibliographyTitle" />
          A content item should be used here instead, but for the sake of simplicity I'm hard-coding a value.
          -->
          Bibliography
        </xsl:with-param>
        <xsl:with-param name="content">
          <xsl:call-template name="autogenBibliographyLinks"/>
        </xsl:with-param>
      </xsl:call-template>
    </xsl:if>
  </xsl:template>
 
  <xsl:template name="autogenBibliographyLinks">
    <xsl:for-each select="//cite[generate-id(.)=generate-id(key('citations',text()))]"> <!-- Distinct citations only -->
      <xsl:variable name="citation" select="."/>
      <xsl:variable name="entry" select="/document/bibliography/reference[@name=normalize-space($citation)]"/>
 
      <xsl:call-template name="bibliographyReference">
        <xsl:with-param name="number" select="position()"/>
        <xsl:with-param name="data" select="$entry"/>
      </xsl:call-template>
    </xsl:for-each>
  </xsl:template>
</xsl:stylesheet>
Create another file in the Transforms folder and name it, bibliography_common.xsl. Here are the contents:

Important: If you create the file in Visual Studio do not let it auto-format the document.
<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 
  <xsl:template name="bibliographyReference">
    <xsl:param name="number" />
    <xsl:param name="data" />
    
    <!-- The value of the class attribute should be changed to bibliographyStyle -->
    <div class="seeAlsoStyle">
      <xsl:element name="a">
        <xsl:attribute name="name">cite<xsl:value-of select="$number"/></xsl:attribute>
        <xsl:attribute name="id">cite<xsl:value-of select="$number"/></xsl:attribute>
      </xsl:element>
      [<xsl:value-of select="$number"/>]
      <strong><xsl:value-of select="$data/author/text()" /></strong>, 
      <em><xsl:value-of select="$data/title/text()" /></em><xsl:if test="$data/publisher">, 
        <xsl:value-of select="$data/publisher/text()" />
      </xsl:if><xsl:if test="$data/link">, 
        <xsl:element name="a">
          <xsl:attribute name="target">_blank</xsl:attribute>
          <xsl:attribute name="href">
            <xsl:value-of select="$data/link/text()" />
          </xsl:attribute>
          <xsl:value-of select="$data/link/text()" />
        </xsl:element>
      </xsl:if>
    </div>
  </xsl:template>
  
</xsl:stylesheet>
Next, open the sandcastle.config file and add the following component configuration immediately after the "Copy in reflection data" component, which is near the top of the file. (Note that you'll have to update the path of the file attribute for your own environment if you're not using DocProject.)

     <!--Copy in reflection data-->
     ....
 
        <!--Copy in bibliography data-->
        <component type="Microsoft.Ddue.Tools.CopyFromFileComponent" assembly="%DXROOT%\ProductionTools\BuildComponents.dll">
          <data file="..\..\Help\Settings\bibliography.xml" />
          <copy source="/bibliography/*" target="/document/bibliography" />
        </component>
Now we can easily add a bibliography to our XML documentation comments. Simply use <cite>reference name</cite>, as in the following example:

<remarks>
<para>This class provides the algorithm defined in <cite>Direct3D</cite>.</para>
<para>However, a related algorithm <cite>DirectX Algorithms</cite> is not provided.</para>
</remarks>
Obviously, we still need to define our references. Create a new file (in the path that corresponds to the file attribute from the component configuration above) and name it, bibliography.xml.

<?xml version="1.0" encoding="utf-8" ?>
<bibliography>
  <reference name="Direct3D">
    <title>Direct3D</title> <!-- required -->
    <author>Jack and Jill</author>  <!-- required -->
    <publisher>Example Printing, Inc.</publisher>  <!-- optional -->
    <link>http://www.example.com?book=Direct3D</link>  <!-- optional -->
  </reference>
  <reference name="DirectX Algorithms">
    <title>DirectX Algorithms</title>
    <author>Mario</author>
    <publisher>Publisher, Ltd.</publisher>
    <link>http://www.example.com?book=DirectXAlgorithms</link>
  </reference>
</bibliography>
Here is what Sandcastle produces for my example:

...
<p>
This class provides the algorithm defined in <a href="#cite1">[1]</a>.
</p><p>
However, a related algorithm <a href="#cite2">[2]</a> is not provided.
</p>
...
<h1 class="heading">
<span onclick="ExpandCollapse(citeToggle)" style="cursor:default;" onkeypress="ExpandCollapse_CheckKey(citeToggle, event)" tabindex="0">
<img id="citeToggle" class="toggle" name="toggleSwitch" src="../icons/collapse_all.gif" />
          Bibliography
        </span></h1>
<div id="citeSection" class="section" name="collapseableSection" style="">
<div class="seeAlsoStyle"><a name="cite1" id="cite1" />
      [1]
      <strong>Jack and Jill</strong>, 
      <em>Direct3D</em>, 
        Example Printing, Inc., 
        <a target="_blank" href="http://www.example.com?book=Direct3D">http://www.example.com?book=Direct3D</a></div>
<div class="seeAlsoStyle"><a name="cite2" id="cite2" />
      [2]
      <strong>Mario</strong>, 
      <em>DirectX Algorithms</em>, 
        Publisher, Ltd., 
        <a target="_blank" href="http://www.example.com?book=DirectXAlgorithms">http://www.example.com?book=DirectXAlgorithms</a>
</div></div>
The bibliography is automatically generated near the bottom of the topic if one or more <cite> elements are used.

If you have any problems let me know since it's quite possible that I simply forgot to mention a step that I took to get this working.
Apr 24, 2008 at 6:07 PM
Continued from my previous post...

Auto-Generated Conceptual Bibliography

To get the same functionality in conceptual builds we have to do a bit more work, although the steps are similar to those above (so I'm going to be lite on the comments here....)

First, update skeleton_conceptual.xml.

<document>
  <metadata />
  <bibliography />
</document>
Next, add an include to main_conceptual.xsl.

<xsl:include href="bibliography_conceptual.xsl"/>  <!-- add me -->
  <xsl:include href="htmlBody.xsl" />
<xsl:include href="utilities_dduexml.xsl" />
Create the bibliography_conceptual.xsl file.

Note: This stylesheet requires bibliography_common.xsl as well. The contents are shown in my previous post.
<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:ddue="http://ddue.schemas.microsoft.com/authoring/2003/5">
 
  <xsl:include href="bibliography_common.xsl"/>
  
  <xsl:key name="citations" match="//ddue:cite" use="text()" />
 
  <xsl:variable name="hasCitations" select="boolean(count(//ddue:cite) > 0)"/>
  
  <xsl:template match="ddue:cite">
    <xsl:variable name="currentCitation" select="text()"/>
    <xsl:for-each select="//ddue:cite[generate-id(.)=generate-id(key('citations',text()))]"> <!-- Distinct citations only -->
      <xsl:if test="$currentCitation=.">
        <xsl:element name="a"><xsl:attribute name="href">#cite<xsl:value-of select="position()"/></xsl:attribute>[<xsl:value-of select="position()"/>]</xsl:element>
      </xsl:if>
    </xsl:for-each>
  </xsl:template>
 
  <xsl:template match="ddue:bibliography">
    <xsl:if test="$hasCitations">
      <xsl:call-template name="section">
        <xsl:with-param name="toggleSwitch" select="'cite'" />
        <xsl:with-param name="title">
          <!--
          <include item="bibliographyTitle" />
          A content item should be used here instead, but for the sake of simplicity I'm hard-coding a value.
          -->
          Bibliography
        </xsl:with-param>
        <xsl:with-param name="content">
          <xsl:call-template name="autogenBibliographyLinks"/>
        </xsl:with-param>
      </xsl:call-template>
    </xsl:if>
  </xsl:template>
 
  <xsl:template name="autogenBibliographyLinks">
    <xsl:for-each select="//ddue:cite[generate-id(.)=generate-id(key('citations',text()))]"> <!-- Distinct citations only -->
      <xsl:variable name="citation" select="."/>
      <xsl:variable name="entry" select="/document/bibliography/reference[@name=normalize-space($citation)]"/>
 
      <xsl:call-template name="bibliographyReference">
        <xsl:with-param name="number" select="position()"/>
        <xsl:with-param name="data" select="$entry"/>
      </xsl:call-template>
    </xsl:for-each>
  </xsl:template>
</xsl:stylesheet>
Add a build component to conceptual.config immediately after the "Create skeleton document" component.

Note: Point to the same file as the reference documentation bibliography.
<!--Create skeleton document-->
...
        <!--Copy in bibliography data-->
        <component type="Microsoft.Ddue.Tools.CopyFromFileComponent" assembly="%DXROOT%\ProductionTools\BuildComponents.dll">
          <data file="..\..\Help\Settings\bibliography.xml" />
          <copy source="/bibliography/*" target="/document/bibliography" />
        </component>
Here's a working MAML example that has a bibliography generated automatically.

Note: You must add the bibliography element, preferably before the relatedTopics element.

<?xml version="1.0" encoding="utf-8"?>
<topic id="a2fc10e3-5724-4293-a7ce-65db64c82990" revisionNumber="0">
  <developerConceptualDocument xmlns="http://ddue.schemas.microsoft.com/authoring/2003/5" xmlns:xlink="http://www.w3.org/1999/xlink">
    <introduction>
      <para>Welcome to my test.</para>
    </introduction>
    <section>
      <title>Test Bibliography</title>
      <content>
        <para>
          This class provides the algorithm defined in <cite>Direct3D</cite>.
        </para>
        <para>
          However, a related algorithm <cite>DirectX Algorithms</cite> is not provided.
        </para>
      </content>
    </section>
    <bibliography /> <!-- you must add this element -->
    <relatedTopics>
    </relatedTopics>
  </developerConceptualDocument>
</topic>
- Dave
Apr 24, 2008 at 6:12 PM
Feature request:

Auto-generated Bibliography
Apr 27, 2008 at 9:37 AM
Hi Dave,

Great job. Thank you.

So this how we extend Sandcastle. Amazing.

I will try this. I will try to learn some xslt so that I can customize this later on.

Thanks again
Apr 27, 2008 at 12:37 PM
Hi Sy,

Learning XSLT is probably a good idea. If you want to use a different style for the bibliography, knowing how XSLT works will help you to adjust it. You certainly don't have to be an expert though (I'm far from being an expert).

I didn't realize that this post may be helpful in showing people how to extend Sandcastle themselves in various ways. I'll create a tutorial from it soon.

Here are some of the concepts that are used in the bibliography feature:

  • How to use custom tags in XML documentation comments.
  • How to use custom tags in MAML.
  • How to add new collapsible sections to both reference and conceptual topics.
  • How to use shared content documents and items. (Actually, I took a shortcut here and didn't use shared content - I placed a note in the XSL.)
  • How to add meta data to a topic so that it can be used by build components later in the stack.
  • XSLT concepts such as indexes and iteration.
- Dave