Sunday 21 November 2010

Migrate SharePoint 2010 Managed Metadata to another farm or Service Application.


Managed Metadata in SharePoint 2010 is great until you want to move term groups or termsets with content types and site columns to another farm or environment. This is because the SharePoint is using GUID as reference when create a new metadata record, each GUID which used by site column will be regenerated even Import from delimiter file.

This will causing a huge problem for those develop their Managed Metadata in one environment and deploy to another for user acceptance test.

Luckily, the metadata GUID can be replicated and imported from/to Service Application by Microsoft.SharePoint.Taxonomy class. That mean implement the custom export/import process by either using .Net code or Powershell.

In this blog post I want to show you how to use the C# code with Taxonomy APIs to export and import managed metadata. At the same time I make a assumption you are experienced .Net development and with appropriate SharePoint knowledge.

First of all, ensure the account access to SharePoint has permission to read and write managed metadata store. I 'm recommend using farm account when export and import metadata from/to SharePoint.

Here the exportation of the termsets and terms to xml.
  1.  Create the SPSite object which link to managed service application which wish to export.

            SPSite site = new SPSite(siteUrl)

  1. Create a TaxonomySession and pass in SPSite created in step 1 as parameter.

 TaxonomySession session = new TaxonomySession(site);

  1. Retrieve TermStore collection from session object created in step 2.

 TermStore MetadataTermStore = session.TermStores[managedMetadataServiceName];

  1. Iterate through terms collection and extract to XML file. 
    
   foreach (Group g in MetadataTermStore.Groups)
                {
                   // some code…….
    }

And the final view of your source code.

// This is the common library for xml extraction.
           ManagedMetadata metaCommon = new ManagedMetadata();

            using (SPSite site = new SPSite(siteUrl))
            {
                TaxonomySession session = new TaxonomySession(site);
                TermStore MetadataTermStore = session.TermStores[managedMetadataServiceName];
                foreach (Group g in MetadataTermStore.Groups)
                {
                        XElement metaStore = metaCommon.GenerateNode(g);                  
                }
            }

The ManagedMetadata is the common class which I created for managed metadata to XML string, it provides several methods to extract metadata objects to xml element. For instance, below code using Linq to XML to generate Xelement from  metadata Group object.

        public XElement GenerateNode(Group g)
        {
            XElement rElement = new XElement(GroupElementName, new XAttribute(NameKey, g.Name), new          
                                                                              XAttribute(GuidKey, g.Id),
                                                                              from termSet in g.TermSets
                                                                              select GenerateNode(termSet));
            return rElement;
        }

Please note when generating xml element you MUST include metadata's GUID as it is important for import process.

I also create a Winform application to help me extract the managed metadata from SharePoint.



Your extracted xml should be something looks like this.

 <?xml version="1.0" encoding="utf-8"?>
<Group Name="Example Group" Guid="3eb956d1-2449-4eaf-be65-01579da93864">
  <TermSet Name="Accounts" Guid="d80de18e-6be1-4a4f-a157-1d1b64d17fe7">
    <Term Name="Audit" Guid="37ebd2d7-ad00-4c7f-a7b9-ad76ac3f0f57" Lcid="1033" IsDeprecated="false">
      <Term Name="External" Guid="f316c860-5727-46f7-9924-003e3c524c69" Lcid="1033" IsDeprecated="false" />
      <Term Name="Internal" Guid="559c9a12-0a6c-4546-80e3-8770b5ecf39a" Lcid="1033" IsDeprecated="false" />
    </Term>
    <Term Name="Bank" Guid="b9c52438-32f3-4597-950e-726ee9809774" Lcid="1033" IsDeprecated="false" />
    <Term Name="Budgets" Guid="4741a838-f887-464f-93bf-1622ea5ad37d" Lcid="1033" IsDeprecated="false" />
    <Term Name="B1" Guid="6876ff80-2e37-4d48-94b0-ecc4c962203e" Lcid="1033" IsDeprecated="false">
      <Term Name="B2" Guid="935bae00-b4fe-48fe-9260-660c27a62be6" Lcid="1033" IsDeprecated="false" />
      <Term Name="C2" Guid="546e29a3-a722-4b5f-80e7-5f4dc3b2355e" Lcid="1033" IsDeprecated="false" />
    </Term>
 </TermSet>
</Group>
   
Above example shown you how to extract the managed metadata to XML, and the summary at below show you how to use Taxonomy's API to import extracted xml into managed metadata store.

  1. Create the SPSite object which link to managed service application wish to export.

  1. Create a TaxonomySession and pass in SPSite created in step 1 as parameter.

  1. Retrieve TermStore collection from session object created in step 2.

  1. Iterate through extracted xml and import into term store, then commit. 

Here how it looks like when implementing in C# code.
var meta = XElement.Load(loadLocation);
using (SPSite site = new SPSite(siteUrl))
{
TaxonomySession session = new TaxonomySession(site);
managedMetadataServiceName = "Managed Metadata Service";
TermStore MetadataTermStore = session.TermStores[managedMetadataServiceName];               
// My custom common library.
ManagedMetadata manageMeta = new ManagedMetadata();
manageMeta.NewNode(MetadataTermStore, meta);
MetadataTermStore.CommitAll();
}

The custom common library will takes XElement, Group, TermSet and Term object as parameter and create appropriate entry in metadata store. As above example, the library will extract Group, TermSets and Terms accordingly when pass in whole metadata Xelement.

When creating a new termset or term, you are expecting to pass in two parameters, term key and GUID. The GUID should be re-used the value extracted from managed metadata store. Example,

// t  is Xelement which extracted from XML string.
// Group is Taxonomy's Group object.
Group.CreateTermSet(t.Attribute(NameKey).Value, new Guid(t.Attribute(GuidKey).Value));

// For create a new term under termset.
// ts is Taxonomy's TermSet object.
// LCID is local integer value.
// t is Xelement.
ts.CreateTerm(t.Attribute(NameKey).Value, LCID, new Guid(t.Attribute(GuidKey).Value));

// or create a new term under another term.
// ts is Taxonomy's Term object.
// t is Xelement.
ts.CreateTerm(t.Attribute(NameKey).Value, Convert.ToInt32(t.Attribute(LcidKey).Value), new   
                                                                                                                    Guid(t.Attribute(GuidKey).Value));

My recommendation to seamlessly  deployment for managed metadata is to using SharePoint WSP with feature, although you can write another bespoke tool for import but as best practice the SharePoint solution should always being used.

Please let me know when you need a copy of my custom managed metadata library.


    41 comments:

    1. Hi

      This is exactly what I'm looking for in a project of mine. I would be interested in getting the application you wrote for exporting/importing. Can you provide a downloadable link for the application?

      Thanks!

      Stig

      ReplyDelete
    2. Hi Stig,

      Here you go, http://www.4shared.com/file/K1xk9A5O/EKSharePointToolsWinForm.html


      Password:"0000" (four zeros)

      ReplyDelete
    3. Has anyone tried this and confirmed if it works after a Content Deployment or Site Collection Backup/Restore?

      ReplyDelete
    4. Hi Fil,

      The managed metadata is kept separately from your content database, that mean restore/backup or content deployment will not affect your website or content db if your managed metadata service application and terms remain the same ID.

      This is because each metadata term has it own unique GUID every time it created and this GUID will be referenced by metadata column, above method is to ensure the term keep same GUID when it create in different environment.

      ReplyDelete
    5. I think the part we are having trouble with is restoring the Managed Metadata Service to a different farm so it has the same service ID. We tried using backup-SPFarm and Restore-SPFarm, but having no luck. But we will give your method a shot.

      Thanks!

      ReplyDelete
    6. Thanks for putting this up here. It looks really helpful. If I use your application, can you tell me what to enter into the URL field? Is it the URL of the Central Admin web application?

      And how do I find out the "Metadata Name"?

      Sorry, this is the first time I am working with the data.

      ReplyDelete
    7. Ah, I figured it out:

      URL: [url of one of my web applications]
      Metadata Name: [name of the managed metadata service]

      Seems to work perfectly. I ran the export/import of the metadata after I had already moved my site from one farm to another, and it still worked fine - the site picked up all the imported metadata without problems.

      Very nice!

      ReplyDelete
    8. The link to the Metadata exporter no longer works - is this available elsewhere now?

      ReplyDelete
      Replies
      1. This comment has been removed by the author.

        Delete
    9. Hi Richard,

      Here the new download link, let me know if you have any question.

      http://uploading.com/files/b3ebmced/EK.SharePoint.Tools.WinForm.zip

      ReplyDelete
      Replies
      1. This comment has been removed by the author.

        Delete
      2. Your zip file is password protected, care to share?

        Delete
      3. Hi,

        More than happy to share, just to prevent it stop by anti-virus.

        try '0000' without single quote.

        Eric

        Delete
    10. My customer wants to use as much OOB solution as possible, does not want to do custom development. However, without custom solution like this or content type deployment as features, it is really hard to move metadata and content type from dev to qa to prod. What should be the best practice to accomplish a possible no code solution?

      Thanks

      ReplyDelete
    11. Hi,

      The best option for this is to package up managed metadata as a SP solution, it is custom configuration/little coding but give you a repeatable quick deployment process.

      This approach has been applied in many development environments and very effective.

      Let me know if you need any additional help.

      ReplyDelete
    12. This comment has been removed by a blog administrator.

      ReplyDelete
    13. For a no code solution I came across this http://blog.pointbeyond.com/2011/07/29/migrate-managed-metadata-service-database post for migrating managed metadata between environments.

      ReplyDelete
    14. Hi Eric,
      This looks just like what I need, but I can't get it to work. I downloaded the project from the new link you gave, but when I run it, I get an exception that points to line 138 in Form1.cs. I'm sure the names I have fed into it are correct. Can you suggest where I might be going wrong?

      Paul Rosenberg

      ReplyDelete
      Replies
      1. Hi Paul,

        Make sure you have access permission to your term store. E.g. administrator permission which configured under Central Admin -> Service Application -> Managed Metadata.

        Eric

        Delete
      2. Success! But it still does not do one vital thing: handle additional labels for a term. These are the only way to enter synonyms or acronyms for a term, and are a major reason I need the tool. It looks like I would need a " copy of my custom managed metadata library" to make this happen (if you have not done it already :-) ). Can you provide the code for the library? I will make a donation (to you or to your favorite charity) in thanks. I can be reached at paul at infodesigns dot com.

        Delete
      3. My apologies. Your program does indeed handle additional labels, as I found on doing some tests. However, it does not handle the 'IsAvailableForTagging' flag, nor does it put out the term term description (function 'GetDescription'), and it does not appear to put out the terms in correct order when a custom sort has been applied (term.CustomSortOrder). There are others that we would want to be able to manage (dates, stakeholders, etc). Any of these would require the ability to modify the EK.SharePoint.Commons code. Can you provide? I'll be happy to pass the modified code back to you.

        Delete
      4. Sure, I will send it to paul @ infodesigns email address for you. Let me know in the mean time if you have any question.

        Delete
    15. A simple code-free approach:

      1. Backup the managed metadata database in your source farm
      2. Delete the MMS service in your target farm
      3. Restore the managed metadata database to your target farm
      4. Create a new MMS service in your target farm
      5. Ensure correct service accounts have full control of MMS

      My company looked at multiple methods of migrating from our Dev environment to a Stage and multiple production farms. We also use a content type hub and migrated to test site to production. Using the method above, all GUIDs are retained and the pieces just snap into place.

      ReplyDelete
    16. I am not able to run this tool, when I run it shows window for one second and then disappear.

      ReplyDelete
    17. I'm getting an error as well:

      ************** Exception Text **************
      System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
      Parameter name: index
      at Microsoft.SharePoint.Taxonomy.Generic.IndexedCollection`1.get_Item(String index)
      at EK.SharePoint.Tools.WinForm.Form1.ExportMetadataStore(String siteUrl, String managedMetaName, String groupName, String saveLocation)
      at EK.SharePoint.Tools.WinForm.Form1.button1_Click(Object sender, EventArgs e)
      at System.Windows.Forms.Control.OnClick(EventArgs e)
      at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
      at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
      at System.Windows.Forms.Control.WndProc(Message& m)
      at System.Windows.Forms.ButtonBase.WndProc(Message& m)
      at System.Windows.Forms.Button.WndProc(Message& m)
      at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
      at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

      ReplyDelete
    18. Hi David,

      Please make sure the account run for export has permission to Metadata Term store, to run in different user just shift + right click.

      Eric

      ReplyDelete
    19. Hi Eric,
      If I wanted to migrate Content Types that have MM fields, what would be the best approach as far as a feature is concerned after the successful migration of MMS?

      Thanks

      ReplyDelete
    20. Hi Jerah,

      I would suggest to package up your content types as usual in VS and feature receiver (coding require) for MM fields.

      Regards,
      Eric

      ReplyDelete
    21. Hello, Neat post. There's an issue with your site in internet explorer, may check this? IE still is the marketplace leader and a large component to people will omit your magnificent writing because of this problem.
      My web page ... ausmalbild wildschwein

      ReplyDelete
    22. Hello Eric,

      My team needs a tool for migrating Managed Metadata content.
      Can you provide us with your tools. I ensure you feedback on how great it is :D

      Regards, Sebastien

      ReplyDelete
    23. Hi,

      Please download it from previous link or http://uploading.com/files/b3ebmced/EK.SharePoint.Tools.WinForm.zip.

      Regards,
      Eric

      ReplyDelete
      Replies
      1. Hello Eric,

        i have downloaded the zip file but when trying to unzip it is asking for password.could u please pass the password to my account.manasmoharana@gmail.com.

        Thanks in advance.

        Manas

        Delete
      2. Hi Manas,

        Have you tried '0000' without single quote?

        Regards,
        Eric

        Delete
    24. Hi Eric,
      The tool is really good. The export to xml file runs correctly. However the import of same file with one new term added results in error -"Term set update failed because of save conflict."
      Are you checking in your code to see if termset or Term already exists then do not create and create only the new once.

      Thanks,
      Poonam

      ReplyDelete
    25. Hi Eric,

      Nice work. What would need to be done to support a term store that has translations for term? I've tried the Export; some terms are exported in their english (LCID 1033) and others are exported in their french (LCID 1036). I would like to have both languages as part of the export results.

      ReplyDelete
    26. Hello,

      none of the links works, could you provide a new location for dowloading ?

      thx

      ReplyDelete
    27. Hi Eric,

      Do you have a ready made powershell script available for this operation bcoz I am not good in Dev but this is very very important task for me at the moment. The key here for us is to retain the GUID's.

      ReplyDelete
    28. Hello Eric,
      I download the file from http://uploading.com/files/b3ebmced/EK.SharePoint.Tools.WinForm.zip. It is a exe file which tried to install multiple applications (virus) to my desktop and tried to change my browser's search engine. I decline them and get no file download. If there another way I can get your source file(s)? Many thanks.

      ReplyDelete
    29. Hi Eric,

      Nice work. I've tried your utility with succes in a dev-test. However, if you need to handle multiple languages and custom properties, you might also consider ServiceAware Term Store Sync (well, it's a paid product, but there is a fully functional trial available too) which I've implemented as part of our automated deployment process.

      ReplyDelete
    30. Hi Eric,
      Would you be kind enough to re-upload the file? I owuld like to test it out for my project.
      Thanks

      ReplyDelete
      Replies
      1. Same here, if you can upload please do.

        Regards,
        Dusan

        Delete