I am having some fun learning Ruby. I'm using the Visual Studio 2010 Ruby IDE from www.sapphiresteel.com which makes things a lot more fun than using some silly interactive console window. They did a nice job.

Ruby is a strange little language. I'm quickly coming to think Ruby was deliberately designed to borrow as much stuff from as many different languages as possible. The ultimate mutt of programming languages. The following diagram illustrates what Ruby borrowed from where:

The thing that shocks me the most about Ruby is that all the strings are not Unicode. They say they are fixing that in Ruby 2.0, but that's not out till 2013. Amazing that a popular language this day and age would even dare to exist without native Unicode support.

I love DGML as you know, so I wrote the following code using the Ruby Symbol table to find all classes and generate a DGML graph of the Ruby class hierarchy:

#===============================================================================================

# This function generates a DGML graph from the superclass information in all known classes.

#===============================================================================================

class ClassHierarchy

      

       # This method finds all classes and calls the given handler passing the class.

       # (The "&" prefix here means the argument is a procedure)

       def AllClasses(&handler)

              # use the "each" method which takes a procedure (the stuff inside curly braces)

              # the |s| is the closure variable that binds to the "each" iterator.

              Symbol.all_symbols.each{

              |s|

                name = s.id2name

 

                # classes begin with uppercase letters

                # (notice the native support for regex in the language)

                if (name =~ /[A-Z].*/) then

                            begin

                             # use eval to execute the code, which is just a symbol

                             # reference, so it returns the Class instance.

                             c = eval(name)

                             # if it's really a class then it should have a superclass

                             # method

                             sc = c.superclass

                             if (sc != nil) then

                                    # now pass this to our handler.

                                    handler.call(c)

                             end                 

                           rescue Exception

                             # exceptions here mean we found something that was not

                             # a class, so ignore it.

                           end

                 end

              }

       end

      

       # Generate a DGML graph linking every class with it's superclass

       def GenerateDgml()

         puts("<DirectedGraph xmlns='http://schemas.microsoft.com/vs/2009/dgml'>")

         puts("  <Links>")

 

         # use our AllClasses method, passing a procedure that prints out a

         # DGML <Link> for each class found, including a link to it's super class.

         AllClasses{|c| puts("    <Link Source='#{c.name}' Target='#{c.superclass.name}'/>") }

 

         puts("  </Links>")

         puts("</DirectedGraph>")

       end

end

 

browser = ClassHierarchy.new

browser.GenerateDgml();

 

 

This produces the following DGML graph

See the following DGML file: RubyClasses.dgml (9.71 kb)

Listen to pod cast

Having spent the last 25 years programming everything from IBM mainframes, Apple II, OS/2, Macintosh, Unix, and Windows systems, using assembler, C, C++, Java, JScript and C# I've often thought that what we are doing as programmers is absolutely crazy.  Deep down I say to myself, “there has to be a better way”!   Ryan Dahl expresses this sentiment perfectly on his blog.

We (programmers) are all drowning in complexity and users can tell.  They get frustrated and yell at the screen every time the software does something stupid.  We programmers hate having to deliver imperfect software.  But perfection just seems unattainable on any sort of reasonable schedule which is frustrating and demoralizing. 

Some blame management for not giving us enough time to finish the job.  Some blame the tools and languages we are using.  Some blame the layer upon layer of stuff we are building on top of.  Some say we just haven’t invested enough in testing our software.

Either way, I claim it is a complexity problem.  We as an industry have hit a complexity ceiling and we can’t seem to break through.  The biggest, best, most well equipped team in the world cannot handle more than about 50,000,000 lines of code.  At this point they hit a complexity limit that can only be solved by removing stuff.

Everyone I’ve talked to about this acknowledges there is a problem.  But what amazes me is the lack of desire to do something about it.  The usual answer I hear is “let software evolve” and “it’s too expensive to re-engineer from the ground up”.  Well, we’ve let it evolve for 40 years and look at the mess we are in!  Saying it’s too expensive to make a quantum leap into something better is a ridiculous argument because we don’t know what that thing is yet, so how can we possibly know it’s too expensive?

Let me make a bold statement.  Evolution isn’t working. We are actually just going around in circles re-inventing the same old crap.  There’s a radical, scary thought.   So we chase the next new thing, jump on it, hoping it will change the world.  Some get more popular than others, and in this highly connected world, the rate of new things appearing, and the rate of adoption of those new things seems to be increasing.  But in the end we all come away with that sinking feeling that Ryan talks about so eloquently.

They say 50% of the solution is knowing what the problem is.  Well I claim the core problem is that we are using 2 dimensional tools and languages to solve an N-dimensional problem.  Languages and their accompanying tools are really great at two things, functional decomposition of a problem, and the linear execution & debugging of the resulting code.  Well it’s better than nothing, but today programmers also have to worry about threading, security, compatibility, reliability, performance, load balancing, fault tolerance, testability, installation, updating, patching, and so on.  I claim there are about 30 dimensions like this dissecting our code turning it into a big stinky pile of dung, and we are the dung beetles scurrying around busily trying to make the pile of crap less stinky.    So next time your manager asks how you are doing you can say “I’m as busy as a dung beetle” J

It is clearly crazy that we are still programming for-loops in this day and age.  Some say the solution to all this stop writing code, let the machine do it based on a Model Driven Architecture.  The idea being computers will be better at solving these N-dimensional problems and coming up with bug free code that implements the desired behavior.  Sounds good, but I haven’t really seen it achieved yet. 

Some say the answer is intentional programming; the problem is we just haven’t been formal enough about all our intentions in code, and once we add that rigor and provide tools that know how to reduce those intensions to code then we’ll be good.  The zed specification language, program proving and all that stuff fit in this camp.

Some say the answer is dynamic languages and/or aspect oriented approaches.  This camp essentially says, screw it, systems should be more dynamic, let me just add my piece of crap wherever I want to add it, and get out of my way.  Interesting approach for cowboys.

Some say we have enough software, we should just stop making more because we’ll probably only make things worse.  We should be happy with what we’ve got. This is the old fart camp and they’re really full of it. 

I say we need a quantum leap in computer science, and we need the new generation of programmers coming out of college to grab hold of this idea and run with it.  Don’t be happy with the status quo.  Continually push the boundaries on every front.  Question deeply everything you’ve been told.  Whoever discovers the quantum leap and smashes the 50m lines of code ceiling will rule the software world for the next 100 years and lead us into the Star Treckian realm of billions of lines of code working seamlessly together to create a better world. 

If you are still reading you’re probably wondering what I’m smokin’ but I claim this will happen sooner than you may think, and I also have a hunch it may well be triggered by a discovery in the rapidly advancing world of biochemistry.   Life systems are many orders of magnitude more complicated than any software system we have yet built, yet somehow all that complexity is boiled down to some seemingly simple DNA encodings (I say seemingly because I don’t think we really know yet how DNA actually works).   But what is the true DNA of software?  What are the proteins, the amino acids, the true patterns that govern entire complex software systems.  The amazingly cool thing for anyone entering the world of computer science is that we simply don’t know yet.  It is still the wild west. 

So I’m a total optimist.  While the state of the art is a big stinky pile of crap, the future will bring new and exciting breakthroughs.  The opportunity for this sort of discovery is reserved to a very small group of folks who are actually looking for it, so it is truly exciting times ahead for them.

Too Much Stupid Software.mp3 (5.14 mb)

Ever wanted to uninstall a boat load of crap in one silent uninstall batch operation?

Well I created a tool to do it. See ClickOnce installer. It requires .NET 3.5.

Here’s a couple of screen shots. It loads up, finds all products, and merges that with your existing saved baseline, in this case it found that XML notepad is new, so it puts a question mark on it, you can then decide whether to include that in your bulk uninstall, or ignore it. Put an X on all the products you want to uninstall. The blanks are ones you have explicitly marked “ignore” before, so they are still “ignored” now. If you uncheck the “Show all” checkbox the “Ignored” ones are hidden. See uninstall progress screen shot below.

Then during uninstall you get progress like this:

It was the first of the month, allowance day, and my son, Sam, had flames pouring out of his pocket, so he bought a Avatar Dragon Fighter GYRO 4CH IR Electric RTF RC Helicopter ZX-Z008 from HobbyTron.com. A week later it arrives, Sam excitedly rips open the box, we charge it up, and 40 minutes later, he starts flying. But in less than 15 minutes, I kid you not, 15 minutes, one motor dies. I can tell it is a motor because it has a sick whining sound and now Sam is devastated. Not to worry, perhaps I can do a repair job and get Sam interested in electronics. So out with the Magnification Lamp and screw drivers. We open it up and see immediately that the shaft of the motor is loose, and has dropped down so that the teeth no longer mesh properly with the gears. We can push it back up into place, and it flies again, but only for a minute until the shaft drops again.

Clearly we have a defective motor. So off we go to the local Hobby shop. We show the person there the problem, and I get the dreaded frown – we all know what it means, “you cheap Dad, how could you buy this junk for your kid, instead of buying a real helicopter from our store”. His eyes even darted over to their expensive selection of helicopters on the wall and back to our broken little toy. He shakes his head, “don’t know what the problem is, could be anything”. I ask if he sells motors that match this machine. He quickly shakes his head again. I could tell he was not going to help me fix this thing. So I have to get more precise with my questioning (which was a skill I picked up in handy class I did at Microsoft once). I ask what is the smallest motor he sells? So off he goes and comes back with this:

And it was $10, ouch, that’s 1/3 the price we paid for the entire helicopter. Oh well, my son is already checking out the $150 helicopters on the wall, so I better give it a shot, so I buy it. The guy behind the counter had that pitiful look on his face, I knew he was thinking “good luck with that sucker!!”. At least they had a motor that looks like it might work. But I knew it was a long shot. I was quietly thinking to myself we probably have a 1 in 10 chance of getting this thing working and I have no idea if I will even be able to get into the thing. You know how cheap toys are made in that “open me once and you’ll never get me back together” kind of way.

But with soldering iron ready to go, I unscrew the outer shell, and a plate covering the gears, fortunately they used tiny little metal screws that will go back together nicely. Lucky strike #1. I pull out the old motor, and find it is a different size from my new one, the new one is smaller. But fortunately, the teeth are the right size (lucky strike #2), so all I have to do is figure out how to get the motor to mount properly and re-solder the wires. But I wire up the new motor and it runs, so the voltages are good. (Lucky strike #3). Next I use electrical tape to get the new motor to fit the mount and a couple of washers to hold it at the right height so the teeth mesh, and button it all back up.

The helicopter doesn’t fly at all. I check the rotors and see they are both spinning in the same direction, so I have the wires reversed. Open it up again, reverse the wires, and we’re good. The helicopter now flies better than it did the first time, it is smooth, easier to control and Sam and I have hours of fun testing our flying skills. Given our luck we both try a “game of life” board game, and indeed it was our lucky day, we both finish the game with over 1 million dollars. And to top it off the weather outside was gorgeous. What a day!

 

 

Having written a lot of WPF code for Visual Studio 2010 I am pretty comfortable with WPF programming, but I’m ashamed to say I was really feeling rather lazy about learning yet another UI framework – Silverlight.  I played with Silverlight 3.0 a while back and was somewhat turned off by the lack of features.

But many of my friends at Microsoft dived in and shipped their own Windows Phone 7 apps, beating me to it.  Ouch, now I felt shame that I’d fallen behind in the latest technology – so with renewed competitive energy and an idea for an app that will help my kids, how could I resist any longer?

So three weeks later, in my spare time after work, with lots of help from my friends, especially Andrew Byrne, and my guinea pigs, oops, I mean kids, my app is published.  It was a fun experience, and now I’m hooked!

So, now I can’t wait to start my next app.  The code to my app is available in this Download.zip. Click here to install the app on your phone.

The app is a simple tool to help kids memorize their multiplication tables.  Here’s some screen shots.

  

It’s a brain dead simple application right?  I first prototyped it using WPF since I was familiar with that and I tried to copy the same page model that the phone has and it worked out nicely.  I had something up and running in about a day since I knew what I was doing.

Then I installed the WP7 SDK, played with a few samples, like the UnitConverterStarterKit, and got familiar with the Emulator.  I have to say right away that the Emulator was amazing.  I built the entire thing and published it before I even owned a phone and was delighted to see it running on the phone exactly as the emulator predicted, almost.  There’s one performance issue I think to do with XmlSerializer or LicenseInformation which causes the app to launch quite a bit more slowly than I’d like.  I haven’t debugged it yet.  But everything else works flawlessly on the phone exactly as it did on the emulator, so hat’s off to the emulator team J

So how does the app work?  The app has 4 pages, and a handy page base class, some user controls like the NumberPad, a BarGraph, and a simple view model that these UI components bind to in various ways.

The Data object serializes itself to and from isolated storage, and contains a list of “Sessions”, each session contains a “Cell” containing a times table problem presented to the user, what answers they tried and the time taken to answer them.  This data is graphed on the MainPage as you can see in the screen shot.

Of course, I wanted to make it polished, so I added a nice little animation to the graph, the bars grow from the bottom of the screen when you first load the app.  Much to my surprise Silverlight doesn’t even have a BeginAnimation method on UIElements, but extension methods are our friend.  We can make our favorite parts of WPF look like they are there on Silverlight, so it wasn’t a problem.

I ended up creating a bunch of other helpers which I put in my helpers namespace.  Including standard brushes, a HlsColor class for manipulating colors, the isolated storage generic which I got from a sample app, and some of my favorite things that I missed from the System namespace.

While it’s annoying to have to do this, I understand the need to keep the framework small and fast.  In fact, I noticed that the XAML designer in Visual Studio 2010 was quite snappy compared to the full WPF designer which certainly makes development fun.  So what is my app using from the .NET framework anyway?

System.Core.dll is the usual LINQ stuff.  System.Xml was the XmlSerializer used by IsolatedStorage.  System.dll and mscorlib are the usual things although I was surprised you couldn’t even get the System.Reflection.Assembly.Version property, but the forum has a post on parsing the full name to get the version that way.  Yet another helper method (YAHM).

My dependencies on the Microsoft.Phone assembly are more interesting:

The cool thing here is how few types you really need to learn in order to get a phone app up and running.  Funny that System.Diagnostics lives in here.  The LicenseInformation is needed to support my app’s “Trial version” feature.

Then from System.Windows, I use a bunch of standard Silverlight controls, media primitives, animation primitives, a couple shapes, and core UI framework base classes, plus a few extras like DispatcherTimer, NavigationEvents and some Data binding stuff.

Click here to see the full dependency graph.

NumberPad

 Now I really love the Zune HD calculator number pad because it does a nice slow fade back animation when you lift your finger off the key.  This allows you to punch in a whole slew of numbers quickly and the residual trail effect the animation leaves on your retina re-enforces in your brain that you hit the right sequence of numbers.  This is exactly what I needed for the hard core Times Tabler who is racing the clock. 

So I whipped out the XAML designer and started playing with the Visual State Manager.  But I hit a wall and couldn’t figure out how to add an animation from the “Pressed” state back to the “Normal” state.  I added it under the “Normal” state but nothing happens.  I searched around but nothing.  So I gave up and wrote it in code.  See NumberPadButton in the code.  This class simulates the Zune HD calculator button.  The NumberPad then is simply a big arrangement of these buttons in XAML, plus some simple event handling to pump these numbers into a target text block and handle clear and backspace.   Why didn’t I just use the built in Soft Input Panel (SIP)? I needed something that would allow my users to devote all their brain power to answering the problems instead of trying to be accurate with their fingers.

No Effects?

The Zune has a nice white glow effect around the button when you click on it, but shock horror, Silverlight has no Effects?  What???  I thought the rage these days was hardware accelerated graphics?!   So I simulated the glow with some different shades of rounded rectangles. Yuk, but it works, and quite honestly I doubt anyone would notice the difference. 

The Meat

The main part of the app, of course, is to test the user on the times table, so the usual state machinery exists here, for running, pausing, dealing with errors, and I also have a demo mode, so that added some more complexity.  Overall developing my app logic and debugging it was a breeze, the tools worked flawlessly.

Code Reuse

Now my Options page was more of a challenge than I was expecting.  Initially I just stuck some text boxes on there and hoped the SIP with InputScope=Digits would do the trick, but my users hated that for several reasons.  One reason was that I couldn’t figure out how to get just the digits 0 through 9 and absolutely nothing else.  The other is that the keys are smaller and harder to press than my huge number pad.

They all said, hey, you have this great number pad why can’t I use it there also?

So the work I did to make the NumberPad a reusable control paid off and it wasn’t too hard to wire it up.  The only trick was that for the bottom option “Goal” the number pad didn’t fit on screen, and the framework didn’t auto-scroll it into view.  I had to add a ScrollViewer around all this, and then when the number pad came up, figure out where the button was, add the height of the number pad (400) then set the vertical offset on the scroller to make sure it was in view.  But then “ScrollIntoView” was missing.  So I tried to just set the vertical offset on the scroller, but this didn’t work until I inserted a call to UpdateLayout.  A bit tricky, but it works. 

Then I had to test to make sure the code works on the landscape orientation.  I decided to go with a ToggleButton for the actual option instead of a Button so the user gets feedback on which option they are entering, and can click the toggle button again to hide the number pad, so you can find and edit the next option, which is something I struggled with a bit using the standard text box and the SIP. 

Tombstoning

It took me a while to figure out how to do my page state persistence properly, initially it was spread all over, then after reading around some more I found a good practice which is to put all this in a base class that knows about my View Model so all my pages work consistently.  This is TimesTablePage, and it handles the OnNavigatingFrom, OnNavigatedTo and OnBackKeyPress events, loading the data, then providing a virtual method to tell the other pages when the data has changed (OnDataChanged).

One of my guinea pigs complained that when he hit the Start button, clicked on Apps, and launched my program again, all the state was lost.  Sure enough in this case my OnNavigatedTo method is not given any of the state I persisted earlier in OnNavigatingFrom.  It’s as if the phone really doesn’t want to restore state in that case.  But my user didn’t like that, and felt it was inconsistent, so I’ll have to think about this some more.  What I found myself doing is storing less and less in the page State dictionary, and more and more in my “Data” object which is serialized back to isolated storage.  I suspect by the time I’m done everything will be in there at which point one wonders whether we really need the PhoneApplicationPage.State dictionary.

Orientation

I put off thinking about this until the end. I saw how the “cool” built in phone apps rotated smoothly as the phone was flipped, my head would spin thinking about all the UI element animations I’d have to do.  Then Andy and Rob set me straight – all I needed to do was add the following attribute to all my pages:

And boom, my pages auto-layout in each orientation, and the phone does the magic of rotating the coordinate system – in other words, my app always sees the top left corner is what the user thinks is the top left corner.  The following start page just worked with no changes.  Cool! 

 

But it didn’t do any nice smooth rotation of things and my second page wasn’t so easy:

This magic didn’t come out of the box of course.  Notice that the “problem” flips from a horizontal line layout to a vertical stacking layout, and the keypad flips for left or right handed users in the 2 landscape layouts.

So I developed each page manually in the XAML designer.  Thank heavens the designer has the ‘Switch to Landscape’ option…

Then I painstakingly copied all the Grid layout information to code so I could switch things around dynamically by setting different Grid.SetColumn, Grid.SetRow, etc.

Now, I probably could have done it all in XAML, but I didn’t want to.

Of course, I’m still mystified as to how to get things to smoothly rotate when the orientation changes, and perhaps if I did something in the Visual State Manager I’d get that for free, but so far no luck, probably have more reading to do.  If you know the answer please comment on my blog!

Publishing

So that’s really it, publishing was a breeze, had to get my icons right, I was confused by the name ‘Background.png’, it was blank so I thought it was some sort of background mask, but it’s actually the icon they show if the app is pinned to the start page.  You can actually test the icon in the emulator if you hold the mouse down on your application, the context menu pops up with the “pin to start” option.  Once I figured that out and removed the emulator chrome from my screen shots the app went through the system.   One gotcha there is don’t publish your app for a price, then try and remove the price later and make it free.  The system won’t let you.  So think carefully about pricing before you publish.

One of the things I enjoy the most about working at Microsoft is being able to work with all the really smart people in Microsoft Research. I really love taking the coolest stuff that comes out of research, and finding ways to productize that and ship it in our products and we did exactly that with the brilliant work done by Lev Nachmanson on the Microsoft Automatic Graph Layout library that we used as the graph layout engine behind DGML documents in Visual Studio 2010 Ultimate.  Lev also collaborated with folks in the Parallel Platform team and the result was a nice boost in layout performance.

Sometimes this collaboration goes both ways.  I've also done some work with another colleague, Kael Rowan from Microsoft Research.  Kael built a really cool Code Canvas and he ran into all the same issues we ran into while building DGML documents, namely, scale and performance of large scrolling WPF Canvas objects.  I posted some code back in 2008 on how we did the Virtual Canvas in DGML documents, and Kael has improved this in his Zoomable Canvas. 

If you are a WPF programmer, and you're struggling wtih how to get lots of objects on a WPF Canvas and scroll and zoom around smoothly, you'll be interested in Kael's new series on the Zoomable Canvas technology.  I'm a huge fan of smooth zooming around a large canvas, zooming in and out around the mouse position as you spin the mouse wheel.  It reminds me of Supreme Commander and makes work lots of fun.

Kael and Lev are two of my favorite research software design engineers.

-Chris 

In Directed Graph Documents I gave you an overview of DGML and included a Demos.zip file.  Hopefully you’ve had a chance to download those and play with them.  In this video (WMV, 40mb) I go through conditional styling in more depth to show you how to spruce up your DGML diagrams. 

I also explain the Legend, talk about its limitations, and show how to edit styles manually in the DGML and get XSD intellisense which will help you remember the various properties that are supported in conditional styles.

The thing that’s really interesting about conditional styles is when they make some hidden data in your graph really pop out.  If you combine careful use of color, font, font size, outline, icons, and grouping, not to mention link styles and labels, you can convey a lot of information in one powerful picture. 

In this video you will learn about the Legend UI, the DGML representation for Styles, and see what the GroupLabel, ValueLabel properties do to the Legend UI, and learn more about what you can do in Conditional Expressions and learn about the importance of order when creating new styles.  For more information on conditional styles the full BNF syntax is available.

The DGML dependency graph feature of Visual Studio 2010 include some state of the art WPF based technology for virtualizing and navigating large graphs, but I'm seeing customers still push this technology to the limits with some very large code bases.  By default the dependency graph features work best for code bases up to about 50,000 lines.  If you need to visualize dependencies in solutions larger than this then you need to use some of the tips and tricks that I take you through in this video (MP4)

This includes:

• Garbage collection (2:00)
• Filtering (2:23)
• Limiting scope to selected assemblies (4:47)
• Drilling into specific links (3:30)
• Cut & Paste (5:20)
• Untangling dense graphs using Quick Clusers, Neighborhood Mode and Butterfly mode (6:25)
• Quick Grid mode on cancelled layouts (10:52)
• Removing boring nodes using Find dialog with regular expressions (14:05)
• Finding important nodes using "Hide Unselected" (16:08)
• Splitting huge graphs into separate linked documents (17:02)
• Custom reference attributes (18:23)

The Architecture Explorer in Visual Studio 2010 allows you to navigate through all sorts of interesting relationships in your code.  It also allows you to create new Directed Graph Documents so you can visualize whatever relationships you explore and it provides a nifty ability to save and replay those queries using DGQL.  

I have posted a new Video (WMV or MP4) that shows how it works and below you will find a link to the DGQL queries that I used in this video.   I hope you enjoy the show.

DGQL.zip (1.25 kb)

Vice President of the Developer Division, Somasegar, posted a blog about architecture tools in Visual Studio 2010 including some interesting dependency graph screen shots.

So to follow up on that I have posted a new Video (WMV or MP4)on Beta 2 bits that demos how this dependency graph feature works in more detail, and the video is a lovely big HD resolution (1280 x 720) so you can really see what's going on.

When I show this to customers the reaction I get is excitement followed by "finally, I can see the big picture" of my code.  Customers often find big things to improve in their code in the first 5 minutes of browsing with this tool.