Microsoft Static Code Analysis Tools Survey


By Eugene Zakhareyev
February 2008 (originally published in February '08 issue of TFS Times newsletter)

The article discusses Static Code Analysis tools available from Microsoft, including stand-alone application and capabilities integrated into Visual Studio 2005/2008 Team System editions.

Introduction

Static Code Analysis feature in Visual Studio is different from other "edition-specific" feature because it is also available as stand-alone application for users of any Visual Studio edition. Stand-alone static code analysis engine, called FxCop, is free and currently available in two versions (version 1.35 and version 1.36).

Thus if you are not sure of what value static code analysis might bring for you, it is easy to try FxCop first to get a feeling of code analysis featureset, and then graduate to Visual Studio edition with integrated Code Analysis support (and perhaps with Team Foundation Server integration as well).

In the current article, I shall try to provide an overview of the features and the difference between different Static Code Analysis products from Microsoft - stand-alone FxCop application, Visual Studio Team Developer edition and Visual Studio Team Developer with Team Foundation Server support.

Using Code Analysis

The whole idea of the static code analysis is to find the potential issues in the software component without actually executing it (and that includes unit tests as form of execution as well). I shall assume forthwith that the concept is indisputably useful and concentrate on managed code analysis; while code analysis for C++ is also available in Visual Studio, it requires extra effort during the coding that few appear be willing to invest and therefore is not widely spread.

To give a short overview, Visual Studio Code Analysis (and FxCop, since both use same core code) runs on compiled managed code binaries, and verifies the compiled MSIL code against set of rules specified by the user. The rules available out of the box with Visual Studio Team Developer edition (or FxCop) are separated into several categories, covering such areas of software development as Design, Security, Usage, Performance etc. Since the analysis is performed on compiled code, it is impossible to enforce code syntax conventions (such as number of spaces between type and variable name in declaration); but it is possible to implement your own custom rules.

The resulting report will contain the list of rules violated, with additional metadata available for every rule, such as rule unique identifier, rule description, rule category etc. together with code specific data. Once the report is available, the user may choose to ignore certain rules violations while fixing the rest of them. Another typical use of code analysis is in an automated build, to make sure code analysis rules violations on integration are identified early on (as in nightly builds) and not propagated to released software (as in release builds). The command line code analysis execution is supported both by FxCop and Visual Studio Static Code Analysis.

Using Code Analysis

The natural start-up solution (unless you have already have VS Team Developer or Team Suite edition handy) is FxCop. The stand-alone application has lean installer and features simple yet sufficient user interface allowing user to define rule set to execute, assemblies to analyze and rules violations review screen. While both version 1.35 and the latest version (version 1.36 beta) may be used to analyze binaries built with all versions of .Net Framework available (1, 2 and 3), version 1.36 also supports .Net 3.5 specific constructs (lambda expressions), has better performance and multiple bug fixes, and rules available are similar to those in Visual Studio 2008.

FxCop screenshot

Since FxCop is a stand-alone application, active rules set and other definitions as well as analysis results are stored in separate project file, unrelated to Visual Studio solution/project.

The following are some of the advantages of FxCop over Visual Studio integrated Static Code Analysis:

  • The code analysis definitions are maintained in separate project and thus do not affect VS projects of the analyzed components.
  • FxCop provides convenient GUI for filtering of the analysis results, that shows violated rules of certain type or category (especially useful when experimenting with different code analysis rule sets)
  • FxCop provides easy visualization of analysis rules in UI - Active rule violations, rules violations excluded in project, rules violations excluded in source code (using SupressMessage attribute) and absent rules (rules available but not activated during analysis)
  • The application lightweight engine does not require loading of solutions or projects; that may be important for large solutions

On the disadvantages side it is worth to note the following:

  • Since all definitions are in separate project, it cannot be maintained in same IDE (read VS) and requires setting up external process for running code analysis and rules definitions maintenance
  • Source code navigation ("jumping" from rule violation to appropriate line in the source and back) is awkward (though it can be partially integrated into VS to overcome that)
  • The results of the code analysis cannot be merged into Team Foundation Server data store and not available for subsequent reporting
  • There are additional rules available in Visual Studio as compared to stand-alone rules set [9]

Static Code Analysis in Visual Studio 2005

In Visual Studio 2005 Team Developer (or Team Suite), Static Code Analysis is integrated into IDE. That means that the code analysis settings are part of the managed (C# or VB.NET) project definitions, and the analysis results are displayed in Error List window.

VS integrations screenshot

The advantages of integrated Static Code Analysis over FxCop are

  • The rule set definition is integral part of VS Project Properties dialog, thus providing easy access to customization with no external tools involved
  • The analysis report is in format familiar to developers, minimizing learning time
  • The navigation between Static Code Analysis report (displayed in Errors List window) and source code is simple and natural

However, there are still some areas where FxCop is better than integrated Static Code Analysis:

  • Since the rule set is defined per project, it may be not trivial to propagate same rules definitions over all projects, if only selected rules are to be activated for analysis (this issue is somewhat mitigated if using TFS – see below)
  • The filtering options of the analysis report are much poorer; the large number of violations can be easily overwhelming in simple Visual Studio interface
  • The rule violations suppressed by user or by using SuppressMessage attribute are not easily visible in Visual Studio interface

Altogether, the choice of more appropriate tool is dependent on the organization policies related to static code analysis (more on that in Best Practices section below). But that is true only if the picture does not include Team Foundation Server; with TFS Visual Studio becomes way more attractive.

Static Code Analysis in Visual Studio 2005 with Team Foundation Server

Team Foundation Server with its version control, work item tracking, reporting and build capabilities available in conjunction with Static Code Analysis integrated in Visual Studio changes quite a lot of things to the better.

First, it becomes easy to enforce running the same set of rules throughout the organization with Code Analysis check-in policy. The administrator may define a set of rules for specific Team Project, and Static Code Analysis will be run with the rule set on every check-in for that project. If the analysis displays rule violations, the user will have to fix the issues before check in or provide explanation for overriding the policy.

Code Analysis policy screenshot

Additionally, Static Code Analysis may be run as part of the code review process with the reviewer generating Work Item for certain warnings (the functionality is integrated into Errors List window). That way only selected issues identified through code analysis can be handled.

Static Code Analysis can be run as a part of Team Build project build, so that any rule violations detected would result in breaking the build, thus ensuring the proper quality. In addition to being available as part of the build report, the build results data may be also published to TFS data warehouse and thus become available for reporting using MS SQL Reporting Services.

Static Code Analysis in Visual Studio 2008

In Visual Studio 2008, the following changes occurred to Static Code Analysis:

  • Code Analysis has become more prominent in Visual Studio IDE and now has its own Analyze menu
  • Additional rules available (including custom spelling dictionaries)
  • Support for .Net Framework 3.x features (such as anonymous methods and lambda expressions)
  • VS2008 Code Analysis engine has better detection abilities as well as improved performance

And finally, new feature called Code Metrics was introduced in Visual Studio 2008. Analyzing the code with Code Metrics, the user is able to calculate such metrics as Maintainability Index, Cyclomatic Complexity, Depth of Inheritance, Class Coupling and Lines of Code. The metrics are calculated on the component, class or method level, which makes it easy to identify the problematic areas in code. Dedicated user interface is provided within Visual Studio to facilitate the detection of such areas with filtering and easy navigation to source code.

Code Metrics screenshot

The metrics are also available as rules (for example, Cycolmatic Complexity is "Avoid Excessive Complexity" rule) thus allowing for metrics to be turned on/off using uniform interface with the other rules and specified for external builds.

Static Code Analysis in Visual Studio 2008 with Team Foundation Server

Team Foundation Server 2008 provides improvements in using Code Analysis policy (specifically, handling conflicts between policy code analysis settings and project specific settings, and providing better means of synchronizing policy with projects definitions).

Additionally, since Code Metrics are available as rules, they can be specified as part of the policy definitions.

Static Code Analysis in Rosario and beyond

Rosario CTP from November 2007 included number of code analysis related features, most notably the initial implementation of rule sets. The idea behind the rule sets is to provide several out of the box sets of rules handling different development scenarios, and also to allow users generate custom rule sets answering their internal needs.

While current preview gives a glimpse of the intention behind the rule sets, the Code Analisys team is eager for the feedback on predefined rule sets contents.

Static Code Analysis best practices

While both FxCop and integrated Code Analysis in Visual Studio provide a dearth of features, the initial success of implementing code analysis practices in organization will not be solely dependent on tool features (or lack thereof).

For most organizations, the code analysis will have to be implemented over the existing code base, and without proper process in place the initial number of violations can easily overwhelm the developers and management, and potential effort of clean up overshadow any future gains.

It can be assumed that the safest way of implementing code analysis practices is to establish at least two rule sets – one for new development and one for the legacy code. The former rule set will be superset of the latter, containing all desired rules. The legacy rules set would be minimal, with the goal of gradually making it identical to the new development rules set, on rule by rule basis.

In such manner, legacy code will eventually become as compliant with organization code analysis policies as the new code, without all-at-once push for compliance.

Additionally, the internal education of users as to the purpose of the rules may be critical for the success of the effort; internal “advocates” for code analysis can ensure that relevant rules are observed and the process is dynamically adjusted (f. e., new rules are selected as needed).

Conclusion

I have tried to give a short survey of the features and differences between different flavors of Microsoft Static Code Analysis with a hope of providing a starting point for those considering implementation of static code analysis.

In my personal opinion, both FxCop and Visual Studio integrated Static Code Analysis can and should coexist peacefully in the organization, with FxCop used for development and testing of the code analysis policies and with Visual Studio being the main tool used by every single developer applying these policies.

Integration with Team Foundation Server certainly may be useful, if the automated build system is to be implemented, and the quality of the builds to be accessed based on static code analysis results as well as successful build completion; and check-in policies are to be used to enforce the usage of proper code analysis rule sets across the organization.

In the next version of Visual Studio, one may well expect some serious inroads into better management of rule definitions as well as new approaches to static code analysis. One non-Microsoft tool displaying a host of interesting features is NDepend, providing such features as query language on analysis results for better access to problematic areas on large code bases and complexity graphic visualization; and hopefully, Rosario will provide some comparable features in the near future. The Code Analysis team is open for suggestions and can be contacted through their blog.

Static Code Analysis Resources

Here is the short list of most representative resources dedicated to Static Code Analysis:

Acknowledgements

Great thanks to David M. Kean from Microsoft for technical review. SnagIt is still the best software for screen capture.

About the author

Eugene Zakhareyev is a principal consultant at Attrice Corporation in San-Jose, California. His primary focus is SCM and development methodologies using Visual Studio Team System. He may be contacted through his email.


© 2006-2012 Attrice Corporation. Last updated 02-Jan-2014 Contact us