XML Reference from XSD

Dec 22, 2008 at 9:04 AM
Is there a way to generate XML Reference for a certain XML namespace having the *.XSD file for this namespace? If no, why? It would be very convenient to generate Attributes and Element Information sections and tranfser all the xs:annototations from the .xsd file to the documentation automatically.
Dec 22, 2008 at 9:43 AM
Hello,
Currently, there is no direct support made public by the Sandcastle Team.
I have done initial work on it using the Sandcastle conceptual format.
The conceptual topics include the XML Reference (developerXmlReference), which is the Microsoft
format for the XSD documentation.
I personally do not like that format, and have completed a template (based on the basic conceptual format)
for this documentation. You can see the use of this template in my HelpRegister tool documentation here...

http://www.codeplex.com/SandAssist/Release/ProjectReleases.aspx?ReleaseId=18329

NOTE: The schema used is small and the documentation was done manually.
I planned to generate and support both formats. I have still not concluded on how the schema diagram will be,
since there is no standard format for this - looking for what will provide the smallest image size
(VS 2010 may also provide some innovative diagrams for hints, since they have promised a new XSD designer).

Unfortunately, I have many other projects for the Sandcastle (in the Sandcastle Assist project), and will not
be making any release or announcement on this until March 2009, when I hope to clear the current loads :(

You can, however, post your suggestions or wishes for the XSD documentations here.

Best regards,
Paul.
Dec 22, 2008 at 1:27 PM

Hello, Paul.

Thanks for your detailed relpy.

It is a pity your ideas are not fully implemented yet. I am not very used to Sandcastle and I use it via UI provided by DocProject. It's still a black magic for me how it works internally. Because of that my suggestions wouldn't be very useful for you I think.

I still have to write a documentation for my XSD. My schema is pretty complex, it uses xs:keys and a lot of user defined types. But I don't need to document all of that. All I need is a list of possible elemenets and the information about attributes, parent/contented elements for each element in the schema. I've already spent a lot of time writing xs:annotantions for every element and I don't want to do a monkey job copying all of that into conceptual topics manually. I'm thinking about writing my own XSLT-based tool that would generate MAML files for each element. I could extend my xs:annotations with elements from some custom namespace to be able to specify usage examples in the same .xsd file. I'm concerned if writing this tool will take more time than doing the work manually.

What would you think about that?

Dec 22, 2008 at 2:32 PM

>>sergeys: It is a pity your ideas are not fully implemented yet.
I will try to raise the priority so that I could come out with something earlier than planned, until now
there was no special request for it.

>>sergeys: It's still a black magic for me how it works internally.
That was how it looks to me until I started working with it, it is really approachable.

>sergeys: Because of that my suggestions wouldn't be very useful for you I think.
I do not think so, everyone has ideas and you do not need to understand Sandcastle to
suggest what you will want to see.

>>sergeys: What would you think about that?
Schema can be very complex, and I am not sure the XSLT can easily cut it - unless you are an expert.
The .NET 2.x schema classes support has all you will need even for very complex schema parsing,
and that is what I am planning to use.

>>sergeys: I'm concerned if writing this tool will take more time than doing the work manually.
It will depend on how you look at it, investing to acquire knowledge (requires time) or getting a work done.
If you are hard press with time, getting it done manually with little automation of repeated task could be
faster, since you already understand the schema and could copy and paste the annotation texts.

Unfortunately, there is no tool in the market that can really cut this correctly. DocumentX is advertised as
supporting XSD documentation but that support is close to useless - in fact, it is part of the reason why
I decided to invest more time in the Sandcastle since it offers a lot of flexibility.

Best regards,
Paul.

Dec 22, 2008 at 3:06 PM
Hi,

Writing your own XSLT tool would certainly be more effort than simply documenting your XSDs manually using MAML, although a tool would be reusable so others would benefit from it if you donated it to the community :)

But writing a custom XSLT tool sounds a lot to me like reinventing Sandcastle.  Sandcastle, at its core, is an XSLT tool that generates documentation based on some XML inputs, which could very well be XSD files.  So instead of building your own tool, think about updating an existing Sandcastle presentation style like VS2005 to include new XSLT templates that can transform XSD files into help topics.  GUIs such as DocProject and SHFB could then add support for XSD sources later on, and in the meantime you may be able to use their extensibility features to offer support immediately.

If you are sure that you'd like to work on this then please post at the Sandcastle Styles project and we can discuss adding you to the team.

Here's the basic build process for Sandcastle with some information laced in about what you might need to do to support XSD topics, and note that we may help at the SS project if we can find the time:

  1. MRefBuilder.exe is used to generate an XML reflection file from assembly inputs (in DocProject you can find the reflection.org file in your project's buildhelp folder).  To support XSD inputs, a custom XSLT file would need to be written that added the necessary information about all of the XSD input files to the reflection file.  What's "necessary" may be up for debate, but at the very least a new <api> element would have to be added under <apis> for each XSD file, and each must have a unique value for the id attribute.  What goes inside could probably be an <apidata> element, with appropriate attributes, and a single <source> element with the full path to the XSD file.
  2. The build process then creates a build manifest for the BuildAssembler.exe tool.
  3. Build Assembler is executed.  This tool is responsible for loading the appropriate XML data into memory, structuring it as an XML document and then passing it as input to one or more XSLT files.  (In DocProject you'll find the configuration files for this tool in your project's Help\Presentation\Style\Configuration folder.  But for the sake of being tool agnostic, you should instead update the .config files in Sandcastle's presentation folders)  You will need to add build components to the configuration files to load the XSD input files into the in-memory document before it is processed by the main XSLT (e.g., main_sandcastle.xsl), which is also specified by a build component.  Loading the XSD files could be as simple as adding another CopyFromIndexComponent that expects them to be in a certain directory, in the same way that XML documentation files are loaded (see component marked with Copy in comments), or you could probably have it infer the directory based on the value of the <source> element that you previously added to the reflection file (seems to me like the way to go).
  4. The main XSLT references all shared/utility XSLT files and contains templates that output an HTML topic.  This template would have to be adjusted to handle whether it's currently being used to document an XSD file, as opposed to an API, or you could simply create a custom XSLT file for XSD documentation (preferable) and update the Build Assembler configuration file to choose the appropriate XSLT based on the current reflection element being processed (this is possible using an IfThenComponent that looks for a special apidata group attribute value such as, "xsd").  Your custom XSLT could then reference the shared/utility XSLT files to include support for things like collapsible headers.

I haven't tried nor investigated this myself, so my suggestion are up for discussion.

- Dave

Dec 24, 2008 at 8:05 AM
Hi, Dave

I don't mind donating whatever I'll have done for this XSD problem. 

I agree that it makes sense to integrate a new stuff into existing framework rather than inventing all from the beginning. I know that Sandcastle is an extensible XSLT tool and it is an ideal place to add XSD support to. It is interesting for me to dig into Sandcastle, but I'm afraid I wont have enought time to do the work completely and well enought. Let's see later if I need to join your team. For the time being I expect you provide me some help here in this topic.

I am writing an XSLT what will help me automate my task. Saying briefly I want to generate elements list looking like the lists in XML references in MSDN. For example, http://msdn.microsoft.com/en-us/library/ms256058.aspx. It requires generating of multiple files (topics): ones for each element plus the topic for the summary list. Lets see what a toptic for a certain element contains:

1. Genaral information about element. It could be taken from standard xs:annotation

2. Syntax code. It could be generated having an information about attributes.

3. List of attributes.The description for an attribute could be taken from xs:annotation for this attribute. Additinal fields like "Optional", "Default Value" are extractable from the attribute defenition in XSD.

4. Element Information with the lists of posisible parents, possible children, and an information about occurrences. First two lists are easy to retrieve from XSD. "Number of occurrences" is more compicated because there could be very non-trival combinations of <xsd:choice>, <xsd:sequence>, <xsd:all>. "Number of occurrences" could be taken from some custom element in the XSD.

5. Remarks and Examples. Defenetely should be taken from a custom element in the XSD. We could allow fragments of MAML format (or some of some other format) to be inserved into xs:annotation.

It is not clear for me yet what intermediate formats we need to have. You talked about "the necessary information about all of the XSD input files". What should it be? Might it be simply a copy of the source xs:schema?

SelormeyPaul  talked about  developerXmlReference. May we reuse it as one of the intermediate formats?

Dec 24, 2008 at 8:30 AM

Hi, Paul

>> The .NET 2.x schema classes support has all you will need even for very complex schema parsing

Yeah, I took a look at them. They appear very useful. It is not a problem to parse XSD with XSLT, I've already have written a XSLT code to extract attributes list and it took 100 lines of XML code. But I'm sure my script will hande every possible schema. Of cource it would be more reliable to use production-quatlity classes than own scripts. I already regret starting to write XSLT. I must have used the classes from the beginning.

Dec 24, 2008 at 5:56 PM
Hi,

> SelormeyPaul  talked about  developerXmlReference. May we reuse it as one of the intermediate formats?

That's the MAML template for documenting XML schemas.  I have a feeling that Microsoft has people write the XML documentation manually using that MAML template.

> It is not clear for me yet what intermediate formats we need to have. You talked about "the necessary information about all of the XSD input files". What should it be? Might it be simply a copy of the source xs:schema?

As I suggested following my first remark that you cited, "What goes inside could probably be an <apidata> element, with appropriate attributes, and a single <source> element with the full path to the XSD file."

I don't think that any new "intermediate formats" must be introduced into the build process.  It's just a matter of including the required information into the reflection.org file so that it can later be used by the XSLT.  That's why I suggested simply using the apidata element to set the node as group="xsd" and including a new <source> element with the full path to the XSD file.

> I am writing an XSLT what will help me automate my task [snip]

Thanks for the link and the explanation of your requirements.  I agree with most of your sentiments towards automatically retrieving information from schemas for inclusion into help topics.

> Let's see later if I need to join your team. For the time being I expect you provide me some help here in this topic.

That's fine, I'll try to answer any questions you may have.

> I don't mind donating whatever I'll have done for this XSD problem.

Thanks, I'm sure I'll use it :)

- Dave
Aug 4, 2009 at 11:10 PM

Did anything ever come of this thread? We're investigating different ways to document an XSD. We're pretty familiar now with Sandcastle, so would like to use that if possible.

 

Thanks,

Katy

Aug 5, 2009 at 1:02 AM

>>katydalton: Did anything ever come of this thread?
After the initial research, I have not yet started work on this. The Sandcastle official support
is not complete/usable in the recent codes checked into the source code repository.

If you use the SHFB, then there is an addin here http://xsddoc.codeplex.com/ . It is also not as
complete as I planned but very useful and may be sufficient for most uses, try it.

Best regards,
Paul.

Aug 5, 2009 at 5:22 PM

Thanks so much Paul! This has worked very well for our needs. Great job.

Katy