IoC Container Benchmark - Performance comparison
In this post I will do a performance comparison of the most popular IoC containers.
Of course performance is not the only criteria when choosing a container for a project. Perhaps you need features like interception, then not all containers are suited. But if the container is "only" used for wiring up dependencies, why not choose the fastest one?
The test setup
The contestants
- AutoFac
- Caliburn.Micro
- Catel
- Dynamo.Ioc
- Funq
- Griffin
- Hiro
- LightCore
- LightInject
- LinFu
- MEF
- MicroSliver
- Mugen
- Munq
- Ninject
- Petite
- Simple Injector
- Speedioc
- Spring.NET
- StructureMap
- TinyIoc
- Unity
- Windsor
You probably have heard of most of the frameworks, but one of them is quite new and works differently: Hiro
The author, Philip Laureano, names it an "inversion of control container compiler framework". After configuration, the container gets compiled and is immutable from now on. This concept makes this container really fast.
The test setup
To test the performance I register three interfaces and their corresponding implementation to every container.
The first one is registered as a singleton, the second one as transient (a new instance is created every time). The third also has a transient lifetime, but moreover has a dependency to the first two interfaces (which get injected by constructor injection).
Every interface is resolved 1.000.000 times and the time is measured in milliseconds.
The results
Overview
| Container | Singleton | Transient | Combined | Interception |
|---|---|---|---|---|
| No | 132 | 81 | 63 | |
| AutoFac 3.0.2 | 749 | 1806 | 4570 | 33765 |
| Caliburn.Micro 1.5.1 | 194 | 226 | 651 | |
| Catel 3.5 | 258 | 952 | 2846 | |
| Dynamo 3.0.1.0 | 72 | 84 | 139 | |
| Funq 1.0.0.0 | 81 | 92 | 277 | |
| Griffin 1.1.0 | 194 | 195 | 481 | |
| Hiro 1.0.3 | 101 | 103 | 107 | |
| LightCore 1.5.1 | 156 | 2343 | 12045 | |
| LightInject 3.0.0.5 | 171 | 178 | 298 | |
| LinFu 2.3.0.41559 | 2876 | 17564 | 47712 | |
| Mef 4.0.0.0 | 2507 | 8724 | 24937 | |
| MicroSliver 2.1.6.0 | 150 | 485 | 2019 | |
| Mugen 3.5.1 | 302 | 483 | 1404 | |
| Munq 3.1.6 | 102 | 131 | 412 | |
| Ninject 3.0.1.10 | 6791 | 19314 | 54926 | |
| Petite 0.3.2 | 86 | 86 | 260 | |
| SimpleInjector 2.2.3 | 68 | 87 | 101 | 467 |
| Speedioc 0.1.33 | 96 | 109 | 94 | |
| Spring.NET 1.3.2 | 672 | 14560 | 39065 | 666 |
| StructureMap 2.6.4.1 | 1127 | 1213 | 4379 | 7329 |
| TinyIOC 1.2 | 246 | 1500 | 6098 | |
| Unity 3.0.1304.0 | 2451 | 3807 | 11212 | 103682 |
| Windsor 3.2.0 | 456 | 2422 | 6970 | 16677 |
Singleton
Transient
Combined
Feature comparison
| Performance | Configuration | Features | Environment | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Container | Code | XML | Auto | Autowiring | Custom lifetimes | Interception | .NET | SL | WP7 | WinRT | |
| AutoFac | Average | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes |
| Caliburn.Micro | Average | No | No | No | Yes | No | No | Yes | Yes | Yes | Yes |
| Catel | Average | Yes | No | No | Yes | Yes | No | Yes | Yes | Yes | Yes |
| Dynamo | Fast | Yes | No | Yes | Yes | Yes | No | Yes | No | No | No |
| Funq | Fast | Yes | No | No | No | No | No | Yes | Yes | Yes | No |
| Griffin | Fast | Yes | No | No | Yes | No | Yes | Yes | No | No | No |
| Hiro | Fast | Yes | No | No | Yes | No | No | Yes | No | No | No |
| LightCore | Average | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No | No |
| LightInject | Fast | Yes | No | Yes | No | Yes | No | Yes | Yes | No | No |
| LinFu | Slow | Yes | No | No | Yes | No | Yes | Yes | No | No | No |
| MEF | Slow | Yes | No | No | Yes | No | No | Yes | Yes | No | Yes |
| MicroSliver | Average | Yes | No | No | Yes | No | No | Yes | Yes | No | Yes |
| Mugen | Average | Yes | No | No | Yes | Yes | No | Yes | Yes | Yes | Yes |
| Munq | Fast | Yes | No | No | Yes | Yes | No | Yes | No | Yes | No |
| Ninject | Slow | Yes | No | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes |
| Petite | Fast | Yes | No | No | No | No | No | Yes | No | No | No |
| SimpleInjector | Fast | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
| Speedioc | Fast | Yes | No | No | No | No | No | Yes | No | No | No |
| Spring.NET | Very slow | No | Yes | No | Yes | No | Yes | Yes | No | No | No |
| StructureMap | Average | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | No |
| TinyIoc | Average | Yes | No | Yes | Yes | Yes | No | Yes | Yes | Yes | No |
| Unity | Average | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | No | Yes |
| Windsor | Average | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
Conclusion
Ninject is definitely the slowest container.
MEF, LinFu and Spring.NET are faster than Ninject, but still pretty slow.
AutoFac, Catel and Windsor come next, followed by StructureMap, Unity and LightCore. A disadvantage of Spring.NET is, that can only be configured with XML.
SimpleInjector, Hiro, Funq, Munq and Dynamo offer the best performance, they are extremely fast. Give them a try!
Especially Simple Injector seems to be a good choice. It's very fast, has a good documentation and also supports advanced scenarios like interception and generic decorators.
Updates
13.09.2011: Funq and Munq have been added to the list of contestants, both frameworks are really fast. The updated charts do no more contain Spring.NET, since it was extremly slow.
04.11.2011: I added Simple Injector, the performance is the best of all contestants.
16.12.2011: I added Dynamo.Ioc, the performance is very close to Simple Injector and Hiro.
22.01.2012: Added TinyIoc.
22.02.2012: Updated IServiceLocator implementations.
12.03.2012: Added LightInject. Added feature comparison.
25.04.2012: Updated to Ninject 3.0.015 and Petite 0.3.2.
14.05.2012: Added Mugen.
14.06.2012: Added MEF.
18.06.2012: Added Griffin.
20.08.2012: Updated to Castle Windsor 3.1.0, LightInject 2.0.0.0, Simple Injector 1.5.0.12199, Structuremap 2.6.4.1, MugenInjection 2.6.0 and Unity 2.1.505.2
18.09.2012: Added Catel.
15.10.2012: Updated to Dynamo.Ioc 3.0.1.0 and MugenInjection 3.0.0
15.12.2012: Updated to Catel 3.4, Griffin.Container 1.1.0, SimpleInjector 1.6.0.12319, TinyIoC 1.2
01.01.2013: Added Caliburn.Micro 1.4
06.01.2013: Added Speedioc. Updated to Autofac 3.0.0, Caliburn.Micro.Container 1.4.1, LightCore 1.5.0
26.02.2013: Updated to Autofac 3.0.1, LightCore 1.5.1, Windsor 3.2.0
15.03.2013: Added benchmark for interception
03.04.2013: Added MicroSliver
11.04.2013: Updated several containers
09.05.2013: Updated LightInject, SimpleInjector and Unity
Source code
Latest source code is available on Github.
New comment
Comments
Andrew
09/12/2011
Would you like to include some other less prominent IoCs in the comparison?Funq http://funq.codeplex.com/
Munq http://munq.codeplex.com/
S2Container.NET http://s2container.net.seasar.org/en/index.html
PicoContainer http://docs.codehaus.org/display/PICO/Home
Daniel
09/13/2011
@Andrew:Thanks for your hint. I have added Funq and Munq to the comparison. PicoContainer is a Java container and Seasar is not very easy to use, so I did not add these frameworks.
Funq and Munq are really fast.
Ady
11/03/2011
Could you state the version of each IOC that you have compared ?Daniel
11/03/2011
@Ady:I added the version information
Pranav Shah
12/16/2011
Would you consider adding two more to the list:http://www.dynamoioc.com/
http://www.dynamoioc.com/
I came across them while reading another article:
http://blog.opennetcf.com/ctacke/2011/04/29/BenchmarkingOpenNETCFsIoCFramework.aspx
tibel
01/21/2012
Please check also:https://github.com/grumpydev/TinyIoC
http://microioc.codeplex.com/
Another interesting point would be, if the IoC container will also work on silverlight, compact framework or WP7...
Marijn
02/01/2012
Interesting read. Any thought on why you think Spring.NET is so slow? Maybe you could update your post to clarify that your test is only testing ServiceLocator style usage of the container.Daniel
02/01/2012
@Marijn:I don't know why Spring is that slow.
I'm only testing service location, since all other features like interception are not supported by all containers.
Will
02/09/2012 | http://www.storm-studios.net
Not sure why xml configuration is a detriment. I hear developers complain about that and I just scratch my head. Oh, well, Spring.NET with all of it's useful facilities works for me.Tomas Jansson
02/22/2012 | http://blog.tomasjansson.com
Interesting comparison, but...I've downloaded your code and I don't think you give the framework funq a fair comparison. The strength in funq is that it is using generics and therefor doesn't need to use any reflection and everything is already typed. If you look at munq, which I think is a fork of funq, it has really impressive numbers and funq should have approximately the same. Also, I changed your implementation and inherited directly from IServiceLocator instead of ServiceLocatorImpl and got numbers that are far from you result.
So please, when you do a benchmark do it the way that shows of the strength of all the frameworks.
Daniel
02/22/2012
@Tomas:Thanks for your feedback. I changed the code accordingly. Now I inherit directly from IServiceLocator. Funq and Petite perform much better now.
Prakash
03/03/2012
Would been interesting if you add an extra column in your to result that would show flexibility cost or feature count. Then one would be able to see how feature rich the framework is versus performance.After all its all about balance choosing the right tool for the job.
Prakash
gold price
04/18/2012 | http://goldpricetoday.ws
My site is using ninject. May be change to use Dynamo in the near feature. Thanks for your post!Martin
04/25/2012
I was wondering if you would consider upgrading the versions to the newest ones and re-running.If not, I'm sure I can try it myself, just thought it might be useful...
Specifically, Ninject 3.0 as I love the syntax, just hoping it's got some comparable performance.
Yours is the most updated blog I've found on this at the moment...
Daniel
04/25/2012
@Martin:I just updated to the latest versions. Thanks for your hint.
Martin
04/26/2012
Interesting, it doesn't seem to have affected the performance very much...I like Ninject's syntax, and it helps that it's the one suggested by Steve Sanderson in the Pro ASP.NET MVC3 book.
Do you have any thoughts on this?
Daniel
04/28/2012
@Martin:I think that the syntax doesn't matter that much.
But if you like it and the performance is good enough for your needs, keep using Ninject.
The dependencies should be configured at a single place (the composition root), so it should be easy to replace Ninject with something else as soon as required.
Dzmitry Lahoda
05/18/2012
Do not you think that using small amount of types leads to false results?Dzmitry
05/18/2012
There are features like Constructor/Property/Method Injection. Unity supports all. May be if turn some off then Unity will be faster (as containers without such features)?Dzmitry
05/18/2012
Real worlds scenarios involve complex object graphs. Some containers could work better or worse in such case. So current tests make me to doubtful about results.Daniel
05/18/2012
@Dzmitry:My performance test does not test all features of all containers and and only uses a few types.
If you have configured more types, the results may change.
I never claimed, that my test is perfect for all scenarios! But the various containers show a very different performance even with a few types.
Feel free to create you own test with more complex object graphs.
Last but not least: Performance is not the only criteria that you have to consider when choosing an IOC.
Nicholas Blumhardt
08/01/2012 | http://nblumhardt.com
Hi Daniel! Interesting to see just how many options we have now :)Autofac does have custom lifetime support- it is very flexible and is in use today supporting per-thread, per-request, per-transaction, per-message, per-view model and many other scenarios.
We did quite a bit of profiling on the MEF team when creating Microsoft.Composition (http://nuget.org/packages/microsoft.composition) and learnt some interesting things in the process.
First, concurrency makes a HUGE difference to how containers stack up; some scale linearly as CPUs are added, while others actually slow down severely. It would be interesting to see a longer-running concurrent version of these benchmarks.
Second, in real life scenarios using Composition Root, deeper graphs of transient instances appear a lot, e.g. In MVC apps. Performance on deeper graphs is also interesting.
Third, over longer (several minute) runs, the amount of garbage created by the container affects performance. This makes a big difference to throughput that may not show in short runs.
Cheers! Nick
Jonas Gauffin
08/22/2012 | http://blog.gauffin.org
My container (Griffin) now supports interceptionVan Thoai Nguyen
09/11/2012 | http://thoai-nguyen.blogspot.com
Thanks for the list.Is it really a matter of how fast the container is? Who's gonna resolve that much objects in a short time? In my opinion, just choose your favorite container. I used to be happy with Unity, StructureMap and now Autofac is my favorite choice.
Just wonder why people like to invent other libraries. If you can list out the pros and cons of these thing, that would be great.
Cheers.
Daniel
09/11/2012
@Thoai Nguyen:Performance is only one criteria among many others. In a desktop application the speed of your container doesn't matter that much. But in a web application things are different. When you handle many requests (using several servers), it makes a difference how much time you waste for setting up your dependencies.
But as I said, it's only one criteria. If your favorite container is fast enough for your needs, just keep using it.
Frantisek Jandos
10/03/2012
Spring.NET gives much better results, when object is resolved using its name, if SpringContainerAdapter.Resolve() method is rewritten to return (T)container.GetObject(typeof(T).FullName); it will give results ~2x slower for singleton and ~8x slower for transient & combined in comparison to Windsor. I vote for returning it back to the graph as Spring.NET deserves it at least for its excellent documentation in comparison to Windsor :)Martin
10/14/2012
Dynamo.IoC is in version 3 now.I just compared performance against SimpleInjector and it is almost double as fast when it comes to resolving transient instances in my test case.
I think your feature comparison should also include if the container supports multiple registrations for each type (etc. using keys). For an example SimpleInjector does not support this last time I looked at it (as far as i remember).
Daniel
10/15/2012
@Frantisek:Thanks for your hint. I updated the code accordingly.
@Martin:
Thanks for your comment. I will update the blog post soon.
I will not add a comparison that tests multiple registrations per type, since not all containers support that feature.
Martin
10/16/2012
Daniel: I just meant adding it to the feature list (not testing it).Ken
12/01/2012 | http://kennethxu.blogspot.com
>> LinFu and Ninject are both much faster than Spring.NETI didn't get this part. You date clearly indicates that Sprint.Net is much faster than that two. What was wrong? the conclusion or data?
Daniel
12/01/2012
@Ken:Thanks for your hint. The post isn't up to date in all parts.
I will update this within the next days.
John
01/01/2013
Hi,please add Caliburn.Micro to list.
Thanks.
ChrisW
01/15/2013
Someone stole your blog post !http://fukyo-it.blogspot.fi/2012/10/ioc-container-benchmark-performance.html
Daniel
01/15/2013
@ChrisW:I know. But he didn't steal the updates :-)
Steven
02/14/2013 | http://www.cuttingedge.it/blogs/steven
Martin is right, Simple Injector does not support keyed registrations. And for good reasons (http://bit.ly/Vj5epb). Keyed registrations are a design smell and if you need them, you should review your design. It's true that some containers make it impossible to implement particular features without keyed registrations (Unity needs them to register decorators for instance http://bit.ly/XQa9wy), but that's not the case with Simple Injector. The only time I ever saw keyed registrations to make sense in Simple Injector was when a user needed to implement an hybrid lifestyle, but this is fixed in Simple Injector 2.0. That version contains a feature to easily build hybrid lifestyles.I challenge Martin to come up with an example where keyed registrations are useful (with Simple Injector).
Tim
02/19/2013
Hi Daniel,Very nice post. I really like the ongoing updates!
I'm definitely going to look at the fast performance containers for my WP7 projects.
(I'm not sure Munq is right for WP7 though. Full project title is "Tools for ASP.NET MVC")
Cheers, Tim
AceHack
03/13/2013
I'm a little confused by your MEF version numbers. I'm assuming by MEF 4.0.0.0 you mean the MEF that was included with .NET framework 4 (MEF 1.0). If so can you please test MEF 4.5.0.0 or what I would call MEF 2.0, the one that is included with .NET 4.5 as they made several decisions to increase performance by dropping features. Also as a separate test could you test MEF for Windows Store Apps? I really consider this almost a different IoC than the core framework version at lest for your tests. Thanks so much this is AWESOME!!! http://nuget.org/packages/microsoft.compositionDaniel
03/13/2013
@AceHack:The version 4.0.0.0 refers to the version of 'System.ComponentModel.Composition.dll'.
I already use .NET 4.5 for executing my tests. Since .NET 4.5 replaces .NET 4.0 I'm not able to compare with the MEF version that was distributed with .NET 4.0.
Changing the "Target framework" of the solution to version 4.5 does not affect the results significantly.
AceHack
03/14/2013
@Daniel how about Microsoft.Composition package on nuget I linked to earlier? Can you incorporate those results? They are in the namespace System.Composition.AceHack
03/14/2013
@Daniel: Again thanks so much for all your work, I was curious it would be great if you also add a column for if it's supported on Windows Store Apps and WP8. Also I would be very, very grateful if you would test the DynamicProxy speeds of the ones that support it.Daniel
03/14/2013
@AceHack:I'm pretty busy at the moment. So it will take some time to add tests for WinRT.
Testing dynamic proxy performance is a good idea. I think I will add this to the benchmark.
Could you check which containers support Windows Store Apps and/or WP8? I can then add your results to the feature table.
Steve
03/20/2013
Hi Daniel,Nice writeup - there are some pretty key variances between registrations that are impacting containers differently. Lets start at the top of the list and take autofac as an example.
Autofac registration looks like this:
builder.RegisterType<Combined>().As<ICombined>();
That call relies on reflection on instantiation to decide on the ctor to use.
Funq on the other hand looks like this:
builder.RegisterType<ICombined>( c => new Combined(c.Resolve<ISingleton>(), s.Resolve<ITransient>());
That call does not rely on reflection - in fact the ability to rely reflection is not supported by Funq, so there is a key difference.
To tidy these up, in Autofac you would register as follows:
builder.Register<ICombined>(c => new Combined(c.Resolve<ISingleton>(), c.Resolve<ITransient>());
This cuts significant time off any container that falls into this distinction.
Another key is being able to imply Func<T> based upon a registration of <T>. Most containers don't allow this, but it's a key operation for anyone doing modern Dependency Inversion. Of the ones that do support it, most are faster at Func<T>() execution compared to .Resolve<T>(), but some are actually slower. Knowing this difference and capability would be helpful. When you consider this optimization, then direct ctor registration could be helped with another level of lambda lifting.
Daniel
03/20/2013
@Steve:You are absolutely right, Funq only supports explicit constructor registration. That means autowiring is not supported by Funq, which is a major drawback, since you have to change the configuration every time the constructor changes.
You should use autowiring whenever possible, because that means less maintenance work.
I did not configure explicit constructors in my benchmark, when a container supports autowiring.
It's the container's problem to resolve the constructor dependencies in a fast way. And containers like SimpleInjector show, that it is possible to do this very fast.