From Research to Product

One of the things I enjoy the most about working at Microsoft is being able to work with all the really smart people in Microsoft Research. I really love taking the coolest stuff that comes out of research, and finding ways to productize that and ship it in our products and we did exactly that with the brilliant work done by Lev Nachmanson on the Microsoft Automatic Graph Layout library that we used as the graph layout engine behind DGML documents in Visual Studio 2010 Ultimate.  Lev also collaborated with folks in the Parallel Platform team and the result was a nice boost in layout performance.

Sometimes this collaboration goes both ways.  I’ve also done some work with another colleague, Kael Rowan from Microsoft Research.  Kael built a really cool Code Canvas and he ran into all the same issues we ran into while building DGML documents, namely, scale and performance of large scrolling WPF Canvas objects.  I posted some code back in 2008 on how we did the Virtual Canvas in DGML documents, and Kael has improved this in his Zoomable Canvas

If you are a WPF programmer, and you’re struggling wtih how to get lots of objects on a WPF Canvas and scroll and zoom around smoothly, you’ll be interested in Kael’s new series on the Zoomable Canvas technology.  I’m a huge fan of smooth zooming around a large canvas, zooming in and out around the mouse position as you spin the mouse wheel.  It reminds me of Supreme Commander and makes work lots of fun.

Kael and Lev are two of my favorite research software design engineers.

-Chris 

DGML with Style

In Directed Graph Documents I gave you an overview of DGML and included a Demos.zip file.  Hopefully you’ve had a chance to download those and play with them.  In this video (WMV, 40mb) I go through conditional styling in more depth to show you how to spruce up your DGML diagrams. 

I also explain the Legend, talk about its limitations, and show how to edit styles manually in the DGML and get XSD intellisense which will help you remember the various properties that are supported in conditional styles.

The thing that’s really interesting about conditional styles is when they make some hidden data in your graph really pop out.  If you combine careful use of color, font, font size, outline, icons, and grouping, not to mention link styles and labels, you can convey a lot of information in one powerful picture. 

In this video you will learn about the Legend UI, the DGML representation for Styles, and see what the GroupLabel, ValueLabel properties do to the Legend UI, and learn more about what you can do in Conditional Expressions and learn about the importance of order when creating new styles.  For more information on conditional styles the full BNF syntax is available.

Managing Large Graphs

The DGML dependency graph feature of Visual Studio 2010 include some state of the art WPF based technology for virtualizing and navigating large graphs, but I’m seeing customers still push this technology to the limits with some very large code bases.  By default the dependency graph features work best for code bases up to about 50,000 lines.  If you need to visualize dependencies in solutions larger than this then you need to use some of the tips and tricks that I take you through in this video (MP4)

This includes:

  • Garbage collection (2:00)
  • Filtering (2:23)
  • Limiting scope to selected assemblies (4:47)
  • Drilling into specific links (3:30)
  • Cut & Paste (5:20)
  • Untangling dense graphs using Quick Clusers, Neighborhood Mode and Butterfly mode (6:25)
  • Quick Grid mode on cancelled layouts (10:52)
  • Removing boring nodes using Find dialog with regular expressions (14:05)
  • Finding important nodes using “Hide Unselected” (16:08)
  • Splitting huge graphs into separate linked documents (17:02)
  • Custom reference attributes (18:23)

Architecture Explorer

The Architecture Explorer in Visual Studio 2010 allows you to navigate through all sorts of interesting relationships in your code.  It also allows you to create new Directed Graph Documents so you can visualize whatever relationships you explore and it provides a nifty ability to save and replay those queries using DGQL.  

I have posted a new Video (WMV or MP4) that shows how it works and below you will find a link to the DGQL queries that I used in this video.   I hope you enjoy the show.

DGQL.zip (1.25 kb)

Standard Dependency Graphs

Vice President of the Developer Division, Somasegar, posted a blog about architecture tools in Visual Studio 2010 including some interesting dependency graph screen shots.

So to follow up on that I have posted a new Video (WMV or MP4)on Beta 2 bits that demos how this dependency graph feature works in more detail, and the video is a lovely big HD resolution (1280 x 720) so you can really see what’s going on.

When I show this to customers the reaction I get is excitement followed by “finally, I can see the big picture” of my code.  Customers often find big things to improve in their code in the first 5 minutes of browsing with this tool.

Visualizing Runtime Object Graphs

One of the first questions people seem to ask when they see the Dependency Graph features of Visual Studio 2010 is whether it can also be used to visualized runtime object graphs.

In this Video (WMV, or MP4) I will show you how that can be done using the SOS debugger extensions and specifically the GCRoot command.  I’ve used this technique several times to help figure out some really complex memory leak bugs and so I’m confident you will find this to be a useful tool to add to your toolbox of tricks.

The code for the GCRootToDGML tool is available to download so you can try this out yourself, it should work with any recent version of Visual Studio 2010.

Adding links some excellent work done by others on this subject below:

Directed Graph Documents

Cameron Skinner first introduced DGML about 9 months ago and now that VS 2010 Beta 2 has shipped it’s time for an update.

This video (WMV or MP4) takes you through some great demos to give you an idea of what it can do.  This is just a 10 minute teaser, in subsequent posts I’ll drill into all the various software visualization features in more detail.

I think you’ll agree that this is pretty exciting stuff and I can’t wait to see all the amazing things that people will do with this new capability.

The following download contains the DGML demos used in this video: Demos.zip (542.37 kb)

Someone asked me recently how I built all those demos:

Demos.dgml – built by hand, manually adding the “Reference” attribute that point to each sample.

.NET 4.0

Assemblies.dgml: I just drag/dropped the .NET assemblies onto a new empty graph document. The green colorization was a custom category added by hand.

Grouped.dgml: was taking the above and manually grouping things then styling the group & link colors by hand.

ConditionalStyles

Baseball.dgml – by hand by copying this graph from Slate magazine

Coverage.dgml – the dependency graph is just a class to class dependency graph which I think I just used the architecture menu generate with a “Custom” settings, namely show only Types and no methods (this way the classes are leaf nodes but their links are a rollup of their inner dependencies), then I wrote a tool that mapped the Coverage report (which is in an XML format) to add coverage properties to my nodes. The hard part here is the “name mapping” because metrics analyzer uses a different “identifier” scheme.

LinesOfCode.dgml – the class dependency graph came from the Architecture menu, then wrote a tool that took a log file from the Metrics Analyzer in VS and mapped it back to the DGML graph. The hard part here is the “name mapping” because metrics analyzer uses a different “identifier” scheme.

Set.dgml – the class dependencies from VS, but the categorization by hand (using the VS Property Window you can type in your own category names), then hooking up the icons by hand.

Tagging.dgml –by hand

TeamData.dgml – was a class level dependency graph , the green was a hand categorization of interfaces, the link color styles added via the Legend.

Progression:

Most of these started as class or assembly level dependency graphs, with manual coloring and grouping added.

Overview.dgml – entirely by hand

ProgressionTypes – You can get a Type Hierarchy graph using the Architecture Explorer tool window which I explain in my view on the Architecture Explorer.

TeamArch:

CallSequence.dgml: by hand

UmlMetaModel.dgml – I built a tool that understands the MOF XML format and exported it from an internal MOF description of UML that we had, but I think the grouping was by hand.

VSBinaries:

Devdiv-grouped.dgml A friend of mine JP Duplessis wrote a tool that cracks all the binaries in a Visual Studio build (native and managed) which he then stored in a database, he then dumped that out to DGML. The grouping was then done using some regular expression matching. We knew a few central nodes that lived in each group, then based on their dependencies the tool would suck things into those groups. We call this “group by patterns”.

devdiv-just-group.dgml – collapse all, but then I have a tool that does a “shallow copy” of this graph keeping the computed group to group dependencies intact. This could be a handy power tool – sort of a “roll up and throw away the detail” kind of tool.

Wpf-by-namespace – was a straight architecture dependency graph, limited to public types only I believe.