The launch of Chem4Word

April 2, 2010

I spoke last week at the spring meeting of the ACS in San Francisco about Chem4Word (well, at least we can still call the project that even though the product actually goes by the much catchier “Chemistry Add-in for Word”). The talk was the day after Alex Wade, Peter and I had officially announced the launch. The talk went down well and was captured on video which will be available soon. I’ll also get my slide deck online.

Since then I have been on holiday – driving up the coast (via Sonoma) to Seattle for the external research symposium at Microsoft – so have been out of contact. On reconnecting to WiFi (bless it) I am delighted to find that the add-in has been downloaded 27,000 times in the first week. Oh my!

I am going to be heading up the project when it goes open source (we are expecting this to happen in a couple of weeks and it will be available here) so I’m delighted to find so much interest out there and, of course, so many potential developers.

However, mostly the reason for this entry is to make sure that I commit myself publicly to writing up a list of TODOs for the project and where we see it going in the short to medium term. I would try to look far into the future but we all know that whatever I come back with will certainly come back to haunt me.

So there you have it. My commitment in print. It is for my own good; when I get back to Cambridge I am definitely going to have to hit the ground running. How exciting”


Extracting CML from a Chem4Word authored document (C#)

January 21, 2010

My previous post was the first in a series demonstrating how CML embedded in DOCX files could be extracted (in that case using Java). For completeness I thought I ought to post some code to accomplish the same thing in C#. This should also allow people to get used to the packaging tools before we build up functionality in later posts.

If you would like a file containing CML to test this out with, one is available here.

Now for the code:


using System.Collections.Generic;
using System.IO;
using System.IO.Packaging;
using System.Linq;
using System.Xml.Linq;
using System.Xml.XPath;

namespace Chem4Word.Tools 
{
  public class OOXMLTools 
  {
    public ICollection GetCML(string path) 
    {
      ICollection list = new List();
      using (Package package = Package.Open(path, FileMode.Open)) 
      {
        foreach (PackagePart packagePart in package.GetParts()) 
        {
          if (packagePart.ContentType == "application/xml") 
          {
            using (StreamReader streamReader = 
               new StreamReader(packagePart.GetStream())) 
            {
              try 
              {
                XDocument xDocument = 
                  XDocument.Parse(streamReader.ReadToEnd());
                if (xDocument.XPathSelectElements(
              "//*[local-name()='cml' and namespace-uri()=
              'http://www.xml-cml.org/schema']").Count() > 0) 
                {
                  list.Add(xDocument);
                }
              }
              catch 
              {
                  // not valid XML so therefore can't be CML
              }
            }
          }
        }
      }
      return list;
    }
  }
}

So there we go. Pretty similar to the Java version really. Just in case you were wondering, I know I haven’t done a load of exception checking.


Extracting CML from a Chem4Word authored document (Java)

January 20, 2010

I have been meaning to write this post for ages and thanks to a recent tweet from Egon I’ve finally got round to it. Basically what I am going to do over a series of posts is explain how CML can be extracted from a DOCX (OOXML) file authored using Chem4Word. I’ll post methods in both Java and C# but I am starting off in Java.

A very quick into to DOCX and OOXML

There are plenty of blogs, papers, videos and the like out there which explain OOXML in various levels of detail. I don’t want to replicate that here but I think it will be useful to have a quick overview for reference. Microsoft developed the Open Packaging Convention (OPC) specification as a successor to its binary Microsoft Office file formats. The file-extension DOCX indicates an OPC document which should be edited using Microsoft Office Word 2007 (as opposed to the XSLX file extension for example which are OPC documents editable using Excel). A DOCX document is effectively a zip-file (the package) which contains the original text as a marked-up XML component (document.xml), with images and other embedded objects stored as separate files.

the simplified structure of an OPC document

The package-part word\document.xml contains the main text and body of the document. Chem4Word stores CML files in the customXml folder within the package. This directory contains pairs of files with names item[\d].xml and itemProps[\d].xml – itemProps[n] contains a list of all the namespaces and schemas used in item[n].

Getting the CML out – the brute force extraction method

The first method of extracting the CML files is the simplest. This method does not allow us to know anything more about the data other than it has been included somewhere in the document by the user. For example, we don’t know where and how it is being used in the document (or how many times). So, for the algorithm: iterate through all those files where the CML may be found, attempt to build each file as a XOM document, if it builds then search within it for a cml element from the cml namespace (see code below).

import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
import java.util.zip.ZipException;
import java.util.zip.ZipFile;

import nu.xom.Builder;
import nu.xom.Document;

public class OOXMLTools {

  public static List<Document> GetCML(File file)
    throws ZipException, IOException  {

    ZipFile zipFile = new ZipFile(file);
    List<Document> list = new ArrayList<Document>();
    Builder builder = new Builder();
    Matcher m =
      Pattern.compile("customXml/item([1-9][0-9]*)\\.xml")
        .matcher("");

    for (Enumeration <? extends ZipEntry>
      entries = zipFile.entries(); entries.hasMoreElements();) {

      ZipEntry entry = entries.nextElement();
      if (m.reset(entry.getName()).matches()) {
        try {
          Document doc =
            builder.build(zipFile.getInputStream(entry));

          if (doc.query("//*[local-name()='cml' and
            namespace-uri()='http://www.xml-cml.org/schema']")
              .size() > 0) {

            list.add(doc);
          }

        } catch (Exception e) {
          // not an XML file so can't be CML
        }
      }
    }
    return list;
  }
}

If you would like to try it out here is a DOCX file with some chemistry in it. There are three chemistry zones in the document (containing testosterone [item1] and acetic acid [item3]) but only two CML files in the customXml directory because both testosterone instances point to the same backing CML file.

In following posts I will go further into how you can discover which representation is being used in the document, how many times a particular CML file is referenced and how the data is converted into the on screen representation.