As one of the core contributors to a compiler for a new language, I often find myself needing to explain new concepts and ideas to other developers. Over time, I've gradually found an approach that tends to work. First, where does the concept fit on the language feature or paradigm map? Second, what is the problem that it is
aimed at, and how are other perhaps more familiar solutions weak? And then -- with that critical foundation laid -- what does the new concept actually do to help?
Recently, I was asked to speak about Aspect Oriented Programming at .Net Community Day in Sweden. I've known about AOP for years and discussed it with developers before, but this was a nice opportunity for me to
spend a while thinking about how best to go about explaining it. So I set about coming up with the AOP answers for my three questions.
The paradigm jigsaw
There are a bunch of titles that we tend to put before the word "programming". "Object Oriented Programming", "Functional Programming", "Declarative Programming" and "Generic Programming" are just some examples. They're all paradigms, but the amount they try to answer differs. The first three I listed -- OOP, FP and DP -- will
happily jostle for pride of place in a language, seeking to be its "core paradigm", and posing a challenge to language designers who see value in all of them. Of course, some languages do decide to just follow a single paradigm: regexes are happily declarative, and Java has admitted little into the language that isn't squarely inside the OO box. However, most of the popular and expressive general purpose languages out there today embrace multiple of these "core paradigms", recognizing that solving every programming task with a single paradigm is like doing every garden task with a lawnmower.
At the same time, there are paradigms that simply seek to deal with a very particular set of issues. Generic programming is one of them: it deals with the situations where we want to use the same piece of code with different types, in a strongly typed way. This idea actually cross-cuts the "core paradigms"; generics are simply
a case of parametric polymorphism, which one finds in functional languages.
The purpose of Aspect-Oriented Programming
This is the kind of place that AOP sits. It's not pitching itself as a successor to -- or even a competitor of -- any of our familiar core paradigms. Instead, it takes a problem and proposes an approach for solving it. So what's the problem?
In any complex code base, there are cross-cutting concerns -- things that we need to do in many places that are incidental to the core logic. For example, logging and exception handling and reporting are generally things that a piece of code does in addition to the task that it really exists to perform. Of course, we factor as much of
the logging and reporting code out as we can, but generally we find ourselves doomed to repeat similar-looking bits of cross-cutting logic in many places. Even if it's just a call to the logger, or a catch block, it ends up duplicated.
Duplicated code is generally a bad sign. For one, it inhibits future refactoring; if we decide to switch logging library, this is now a change in many places. Since we have mentioned the logging library in many places, we have many pieces of our code that are closely coupled to it -- something that goes against our general preference for code that is loosely coupled. We also run into challenges when we want to install new cross-cutting concerns into our software -- there's so many places to change!
Aspect Oriented Programming is aimed at these kinds of challenges. It introduces the concept of join points -- places in our code that we could add some functionality -- and advice -- the functionality that we want to incorporate. It provides a mechanism to write the cross-cutting functionality in one place and have it "inserted" into the join points.
A common reaction is, "wow, magical", but if we cast our eyes back over programming history, it's really not so surprising. Once upon a time there was just assemblers. Repeating regular sequences of instructions was tiring, so macro systems were created. These "magically" got substituted for a sequence of instructions, with some parametrization thrown in to make them more powerful. A little further on, named and callable subroutines allowed for real re-use of code. We raised the abstraction bar: "you don't have to worry about what this bit of code does, just call it like this". Then came OO. Here things got really magical: invoking one of these method things leaves you with no idea what bit of code is going to be called! It could come from that familiar bit of code you wrote yesterday, or it could come from a subclass that's going to be written in 10 years time.
Over the years, we've evolved ways to manage complexity in software, many of them hanging off increasingly non-linear or loosely coupled flow of control. We've coped with the increasing complexity in our paradigms and programming style, because they more than pay themselves back by decreasing complexity in the software we produce using them. AOP's concept of join points and advice is giving us a little more conceptual overhead in exchange for a more maintainable and less coupled implementation of our cross-cutting concerns.
Looking to the future, I think it's important to remember that where we are with any paradigm today is almost certainly some way from the ideal. We've learned a lot about object oriented programming over the decades that it has existed, and continue to do so. Multiple inheritance has generally been marked down as a bad idea, and the
emergence of traits in Smalltalk (also known as roles in Perl 6) has made some -- myself included -- question whether we'd be better off dropping inheritance altogether in favor of flattening composition.
AOP has only had 15 years of lessons learned so far, and I've no doubt we've got a long road to walk from here. However, that being the case shouldn't stop us from embracing AOP today. After all, the ongoing journey of the object oriented paradigm isn't something that stops us from using it. The critical question is, "does this help me deliver higher quality, more maintainable software" -- and I think there are a bunch of use cases where today's implementations of AOP help us to do so.
About The Author
Jonathan Worthington is an architect and programming mentor at Edument. He works to help development teams improve their software architecture and work more efficiently, following best practices. He is also one of the core developers of the Rakudo Perl 6 compiler, where his work focuses on the implementation of object oriented language features and the type system. In his free time, he enjoys hiking in the mountains, good beer and curry.
In my response to Anders Hejlsberg spreading FUD about aspect-oriented programming, I mentioned the common confusion that Aspect-Oriented Programming (AOP) and Dependency Injection (DI) are competing concepts. In this post, I will further clarify the differences and similarities between AOP and DI.
The timing seems perfect, because Dino Esposito just published the article: Aspect-Oriented Programming, Interception and Unity 2.0 in the December 2010 issue of MSDN Magazine. This is a great article and I highly recommend anyone involved in .NET development to read it. Like most dependency-injection frameworks, and several general frameworks (WCF, ASP.NET MVC), Unity offers an AOP-like feature: interceptors.
What is Dependency Injection?
Dependency Injection is a design pattern that strives to reduce explicit dependencies between components.
One of the benefits of reducing dependencies (moving from a tightly coupled architecture to a loosely coupled architecture) is the reduction in the level of awareness components have of each other, thereby achieving a better separation of concerns by implementing the “program to an interface, not to an implementation” design principle.
A common misinterpretation of this design principle among C# and Java developers is that clients should consume interfaces (the thing you declare with the interface keyword) instead of classes. I prefer the following wording: “program to a contract, not to an implementation”. The contract of a class is composed of its public members, augmented by their documentation (plus their formal preconditions, post-conditions and invariants, if applicable). To be even more precise: “program to a documentation, not to an implementation”. So if your aim is just a better separation of concerns, you don’t really need interfaces, you just need the public, internal, protected and private keywords. It’s important to note that in object-oriented design, interfaces are not the silver bullet. Not all framework designers follow the same philosophy: the Java class library relies more on interfaces than that of .NET, probably because .NET designers had negative experience with the all-interface approach of COM.
Programming to an interface (instead of a class) is useful when your component has to talk to many implementations of the interface. There are innumerable cases where this is useful (after all, we all want our architecture to be extensible), but let’s face it: in as many cases, when our component needs to talk to another, we know perfectly well which implementation we have in mind, and there will never be a second implementation running side-by-side. So why use an interface here?
There are two cases when you want to program to an interface even if you know you’ll always have a single implementation:
- You depend on a component that has not been delivered yet and need to start programming – and, more importantly, testing your code – right now. You need to simulate (mock) this component until it is delivered, only for the purpose of testing.
- You depend on a component that has a persistent state (such as a database), real-world effect (such as a credit card processing system) or simply depend on a slow or unreliable resource (web service). You need to substitute these components with an implementation of your own, only for the purpose of testing.
In these situations (which all happen to be related to testing), it’s good that components are not wired together in source code – even if we know how they will be wired together in the final setup. So how do we wire these components together? This is where Dependency Injection comes into play.
With the DI design pattern, you instruct a “container” how the objects should be wired together, and you ask the container to wire the objects. Indeed, most of the time, you don’t create new objects using the class constructor, but using a factory method of the container. The container would figure out which implementation you need to be wired to, and would return it to you.
Dependency injection addresses the problem of wiring loosely-coupled components together. Aspect-oriented programming (AOP) solves a very different issue: taking a reality where some features (such as transaction handling or auditing) affect virtually all business requirements, and expressing this complex reality in a compact way, with minimal duplication of code (DRI principle). When we think of auditing, we want to look in a single source file. When we see a piece of code related to customer management, we don’t want to see auditing code. As you know, this is impossible to achieve with plain object-oriented programming, and that’s why aspect-oriented programming was invented.
Aspect-oriented programming allows you to specify an implementation pattern (using source code, not natural language) into a so-called aspect, and to apply the aspect to base code, typically to business or UI code.
Similarities and Differences between DI and AOP
There are some similarities in the objectives of DI and AOP:
- Both achieve a loosely coupled architecture.
- Both achieve a better separation of concerns.
- Both offload some concerns from the base code.
However, DI and AOP differ significantly in situations where they are found to be useful:
- DI is good when you have a dependency on a component, you just don’t know to which implementation.
- AOP is good when you need to apply a common behavior to a large number of elements of code. The target code is not necessarily dependent on this behavior to be applied.
How did dependency injection ever become associated with aspect-oriented programming? It’s simply that the DI pattern makes it easy to add behaviors to components. Here’s how: when you ask a dependency from the container, you expect to receive an implementation of an interface. You can probably guess which implementation you will receive, but since you play the game, you just program to the interface. And you’re right. If you ask the container to add a behavior (for instance tracing) to the component, you will not receive the component itself; instead, you will receive a dynamic proxy. The dynamic proxy stands between your code and the component, and applies the behaviors you added to the component.
Because DI frameworks have the ability to put behaviors between the client and the implementation of a component, DI has been identified as one of the technologies able to deliver AOP.
There are two technologies that can deliver dynamic proxies: dynamic code generation (using System.Reflection.Emit, typically), and remoting proxies (see RealProxy).
Please read Dino Esposito’s article in MSDN Magazine for details about dynamic proxies.
Proxy-Based AOP: The Good
What I love about the proxy-based approach is how AOP blends into the DI concept. DI lets you configure how the container should build the components by injecting dependencies, setting properties and calling initialization methods – you can also inject aspects at the interface of components. You can configure aspects just as other dependencies in an XML file or using C# code, just as if the issue of building and assembling components were unified.
The ability to add aspects just by changing a configuration file may prove very useful in production. Suppose there’s some issue somewhere and you want to trace all accesses to a specific component. You can do that without recompiling the application. This may be a decisive advantage in formal environments, where it can take months to go from a bug report to the deployment of a fix.
Proxy-Based AOP: The Bad
Proxy-based AOP, as implemented by DI frameworks, has serious limitations:
- It only works on the surface of components, exposed as an interface. It does not work for all methods that are internal to the component, whether public or private, static or instance. Remember that a component is an application part, and generally includes several classes.
- It only works when the client code got a reference of the component from the container. Aspects will not be applied if the component invokes itself, or if the component is invoked without the intermediary of the container. This can be seriously misleading!
- Since aspects are applied at runtime, there is no possibility for build-time verification. Build-time AOP frameworks such as PostSharp will emit compilation errors if the aspect is used incorrectly, and will even provide the aspect developer with different ways to verify, at build time, that the aspect has been used in a meaningful way (for instance, don’t add a caching aspect to a method that returns a stream, or to a method that already has a security aspect).
- It is difficult to figure out (a) which aspects are applied to the piece of code you’re working on and (b) to which pieces of code the aspect you’re working on has been applied. Full frameworks such as PostSharp or AspectJ can display this information inside the IDE, substantially improving the understandability of source code.
- Run-time AOP has higher runtime overhead than build-time frameworks. This may be of minor importance if you only apply aspects to coarsely-grained components, but just generating dynamic proxies has a significant cost if you have hundreds of classes.
- Dynamic proxies are always a side feature of dependency injection framework and do not implement the full vision, even when technically feasible. They are mostly limited to method interception, which hardly compares to what AspectJ and PostSharp have to offer.
Proxy-Based AOP: The Ugly
There’s always a risk when a practice becomes mainstream: it starts being used just for the sake of itself, because it makes you feel safe. This is when a practice becomes a dogma.
It clearly happened to test-driven development, which suddenly imposed upon the community to write tests at meaningless levels of granularity. And thus so with dependency injection, which tends to be used to isolate classes from each other. Remember that dependency injection had been initially proposed to isolate components, i.e. parts of applications of rather coarse granularity. I am concerned that we are ‘forced’ (by social pressure) to write more complex applications just to meet testing ‘requirements’ that do not result in higher quality. Remember that unit testing is an instrument, not an objective. The ultimate objective is quality of your application during its whole lifetime.
Don’t get me wrong, I am not suggesting that you should not write unit tests. I am concerned that religiously following a practice may actually degrade the quality of your code.
The same issue exists with proxy-based AOP. If DI frameworks made you excited for the power of AOP, you may be tempted to restructure your code just to use more of this proxy-based stuff. Don’t. Adding aspects to the surface of components using proxies is simple and elegant. If you are making your code DI-friendly just because you need aspects, you are probably doing something wrong. The principal mantra of AOP is that you should not change your base code. If you are, you should probably look at a real AOP framework such as PostSharp or AspectJ.
Aspect-Oriented Programming and Dependency Injection are very different concepts, but there are limited cases where they fit well together. In these situations, using the AOP facility of a DI framework makes perfect sense. For other cases you have two options: abuse from dynamic proxies (a non-solution), or use of a dedicated AOP framework.
This is not just me, not just PostSharp, and not just .NET. The same dilemma hangs over Java programmers, who have to choose between the proxy-based Spring AOP and the build-time AspectJ (read the summary of Ramnivas Laddad ‘s talk on this topic).
AOP and DI are not competing technologies. You can use both in the same project. What’s important is to understand what each technology was designed for, and to focus on the quality of your code.
I could not have received a clearer answer from Anders Hejlsberg after asking him, during PDC 2010, if there was any chance of getting AOP support in the next version of C#.
I was relieved. But then Anders started spreading myths about AOP, not only showing that he’s misinformed about the technology, but also that he’s blatantly ignoring the needs of a large portion of C# developers – including some Microsoft teams.
Anders, I am totally relieved that C# will not support AOP. I understand your reasons, as the architect of Microsoft’s flagship language, but please allow me to respectfully demystify the untruths you’re spreading.
Myth 1. “Aspect-oriented programming is interesting for debugging and instrumentation of code and is not a full-fledged programming discipline.”
Truth 1. Anders probably stopped at the “Hello, world” example.
Although code instrumentation is certainly an important use case of AOP – and the one you would see in every “getting started” documentation – the technology significantly simplifies the work of developers when it comes to implement any non-trivial, real-life application. Just to cite a few real-life scenarios where AOP really helps:
- Data Binding (INotifyPropertyChanged)
- Data Validation
- Thread switching (marshaling from/to background or GUI threads)
- Thread synchronization (locking)
- Security (authorization)
- Transaction boundaries
- Exception handling
- Sanity checks (detection of memory leaks and deadlocks)
- Transacted objects (objects supporting commit/rollback)
- Persistent objects (whether to database, XML config file, registry, app.config, …)
Myth 2. ”You can take an arbitrary body of code that you did not write and inject your code in the middle of it and start messing with invariants that programmer thought were true and probably aren’t any more.”
Truth 2. Mature aspect-oriented frameworks offer a disciplined approach to programming
To Ander’s defense, let me acknowledge that AspectJ has been used for the ugly.
You can find some terrific examples in scientific literature where people have proudly been using aspects to fix bugs in a third party library, thereby turning AOP into a patching technology.
Let me be very clear about this: patching and AOP are different things, even if the same tools can be used. These cases are pretty much limited to Java, because .NET makes it much more complex to change code you have not written yourself (because of strong names).
PostSharp only enhances the code you own. It is a build-time tool. Even if PostSharp didn’t not look at the source code, the code is typically present on the developer’s machine. If you did not write that piece of code yourself, it was probably written by your colleague. As an architect, there are actually many aspects you want to be applied to the code written by line-of-business developers. You don’t want them to care about exception handling or change notification, so you define that these features should be added to all methods or properties of all domain objects. It takes you 20 lines of code in the base class and PostSharp writes the remaining 20,000 lines of code in business objects. This is aspect-oriented programming. This is actually what you want.
PostSharp won’t let you inject code in the middle of nowhere. PostSharp uses a model named composition filters, developed by the University of Twente. This model represents a method call as an exchange of messages (request/response) between the caller and the called nodes. Aspects work as filters on these messages. They can alter them, they can choose not to pass them forward, but they cannot change the internal behavior of nodes. If you apply several filters to the same node, PostSharp will require you to specify dependencies. The aspect dependency framework is engineered to handle situations when one uses several aspects from vendors who don’t know about each other. PostSharp is robust—by design.
PostSharp makes you aware of the presence of an aspect. If your code has been modified by an aspect – even if the aspect is located in another project – Visual Studio will underline it in light blue. Move the mouse pointer to the method name, and you’ll see the list of all aspects applied there. This is a unique feature of PostSharp 2.0 Pro.
Ok, I admit it. You can add an aspect to the method Math.Sin so that it always returns 2.67. Even if you can’t break the encapsulation of a method, you can add aspects to private methods of a class, arguably breaking the encapsulation of the class. Yes, you can do some powerful stuff with a sharp tool, and you can get hurt if you don’t use it correctly. Anders knows it. C# does native pointers, doesn’t it? The solution is not to give pros toy tools, but to make the tool safe by design.
Did you ever wonder why developers move away from static languages to dynamic languages? Partly, because the strong discipline of C# prevents them from doing what they need in an effective way. Whether Anders likes it or not, developers need meta-programming. If we as language designers don’t provide them with a disciplined approach to meta-programming, developers will move away from any disciplined language.
If governments stop listening to their voters, people start voting with their feet. This is exactly what’s happening now with C# folks moving to Ruby. And this is the gap PostSharp is trying to bridge: Anders, we love the way C# makes us write well-structured, machine-verifiable, refactorable code, but SharpCrafters listens when people ask for meta-programming, and we provide a framework that lets them make code that’s still machine-verifiable, still refactorable, and even better structured and maintainable than before. And this framework is PostSharp 2.0.
Myth 3. “A lot of the same things become possible with dependency injection.”
Truth 3. Using dependency injection for the sake of AOP is like buying a car for the purpose of sleeping on the back seats.
Dependency injection exists for a single purpose: to provide a specific implementation to objects requiring an interface. Dependency injection makes it practical to follow the design principle “program to an interface, not to its implementation”.
All dependency injection frameworks now come with some AOP-like feature: when you require the implementation of an interface, you don’t receive the implementation itself, instead you receive a proxy (called DynamicProxy), and this proxy applies some aspects before invoking the final implementation. It looks like AOP, it smells like AOP, but it hardly compares to a fully-fledged AOP framework.
The first issue is that dynamic proxies only allow you to add aspects to the explicit boundaries of a service, the ones you chose to expose as an interface. You can’t add aspects to private or static methods, even if you wanted to. You can’t add aspects to fields. Even if you added aspects to public methods, the aspects methods won’t be invoked when you call them from the same object (the call would not go through the proxy).
The second issue is more dramatic. As you become addicted to some of the benefits of AOP, you start using dynamic proxies even where you don’t need dependency injection, altering the architecture of your code just because of the requirements of dynamic proxy. And this is wrong. AOP does not require you to change the architecture of your code.
Dependency injection is a great way to isolate components and services. If you are isolating your domain objects with dynamic proxies, you’re probably doing something wrong.
And have you noticed you can’t use dynamic proxies on WinForms or WPF objects? Surely you want AOP there too.
Myth 4. “[Aspect weaving happens] later through a bytecode writer, that’s sort of where I get off the bus.”
Truth 4. Anders is not on the Microsoft Code Coverage or Visual Studio Code Coverage bus.
Many people have an esthetic resentment against MSIL rewriting. When asked for a rational explanation, people usually answer that it’s “sort of dirty”.
MSIL is well documented, the CLR behaves almost perfectly according to its specification, so what’s wrong in producing verifiable MSIL instructions? What’s wrong in modifying the assembly before it’s actually being tested?
Many tools use MSIL rewriting, including Microsoft Code Contracts and Visual Studio Code Coverage. I’m happy that PostSharp drives this bus.
Myth 5. “We have better ways of addressing these scenarios.”
Truth 5. Anders has better ways because he owns C#, but we don’t. We can’t wait for C# 12 to have a solution to NotifyPropertyChanged.
I agree. C# 5.0 will be wonderful. The new keywords async and await solve the problem of asynchrony in a wonderful way. There’s no irony here, I’ve written my thoughts on it previously: the solution is better by an order of magnitude than what we can do with AOP.
PostSharp users, however, have had some solution to this issue for four years. Not a perfect solution, but some solution. And most importantly: developers have been able to develop this solution themselves. They didn’t need to wait for the C# team, or for me – they had a toolkit that allowed them to write their own solution.
I fully understand that when the C# team picks up a problem it has – and succeeds – to find a magnificent ad-hoc solution. The point is: there are lots of issues where AOP can help, and if we had to wait for the language to come with an ad-hoc solution, we would probably wait forever.
What’s more, do we really need an ad-hoc solution every time? Wouldn’t it be better to abstract this set of issues and design a solution to the abstract problem? This solution is aspect-oriented programming.
Anders Hejlsberg fails to acknowledge the need for better support for meta-programming in C#, which is what AOP addresses. It’s not just about me and several thousand PostSharp users. Many teams in Microsoft itself have identified the same need. Windows Communication Framework (WCF) has its AOP framework (generalized into custom behaviors). ASP.NET MVC has its own (filters). Unity and PIAB have their own. Folks from Code Contracts would probably have appreciated the ability to hook the compiler. This need is largely being ignored.
There’s some speculation that C# 6.0 would “open the compiler”. From what I could hear from Anders Hejsberg and Mads Togersen, there’s no indication that the compiler would allow “extensions” to hook into the pipeline, much less modify its object model. Instead, we see the sign of fear of “someone messing with invariants” and of intractable cases to be handled by Microsoft Support. With regards to supportability, designing an extensible compiler is difficult, so I would not expect the first version “managed compiler” to support any extensibility. Therefore, within a horizon of 8-10 years (beyond which it’s difficult to predict anything anyway), I would not bet on an extensible compiler.
Therefore, PostSharp is there to stay. There’s a gap between the disciplined but inflexible programming model of C#, and the flexible but undisciplined model of dynamic languages. PostSharp bridges this gap, and PostSharp may well convince some folks to stay with C#.