Invalid offsets in IL instructions after modifying assembly with Mono.Cecil

Hi

Since 2008 I've been using Mono.Cecil, an amazing piece of software, that exposes .Net assemblies contents in a relatively easy and intuitive way so one can inspect and/or change the assembly's contents. 

Unfortunately, in order to use Mono.Cecil effectively, you need a fair amount of knowledge about how MS IL works and Mono.Cecil documentation is kind of sparse (to say the least).

Some time ago I was using it to change some assembly IL and to my surprise after applying the changes peverify complained that some IL instructions had invalid offsets. 

After some head scratching I've figured out that the issue was that the target of a (short) branch instruction have crossed the threshold that would require it to be  a normal branch (i.e, one that could use 32 bits offsets instead of 8 bits of the short version).

So now the issue was that I'd be forced to scan every single IL instruction in the method body and check/fix the target offset of branches; fortunately, Mono.Cecil has  two methods: MethodBody.SimplifyMacros() and  MethodBody.OptimizeMacros() that can be used to achieve my goal. 

Basically before start doing changes to the method body's IL, you call SimplifyMacros() and when you've finished with your changes on that method body you call OptimizeMacros() and Cecil will take care of adjusting branches accordingly. Nice!


Thanks everybody that helped to develop Mono.Cecil! It's really a handy library! :)

(Leia este post em português)

Instruções com offsets IL inválidos após modificar assembly com Mono.Cecil

Desde 2008 venho usando a biblioteca (muito boa diga-se de passagem) Mono.Cecil que permite você tanto ler quanto modificar o conteúdo de assemblies .Net de uma forma relativamente simples (depois que você compreende como utilizá-la, pois infelizmente,  usar tal biblioteca de forma efetiva exige um bom conhecimento sobre como o MS IL funciona e a documentação do Mono.Cecil deixa a desejar).

Algum tempo atras, um dos testes de um dos meus aplicativos (que usa esta biblioteca) começou a falhar; para ser mais preciso peverify começou a reportar algumas instruções com offsets inválidos. 

Após investigar por algum tempo concluí que o problema se encontrava no offset (operando) usado em alguma instruções de desvio (branch), as quais estavam ultrapassando o limite de um byte (-127 / 128) (isto ocorria devido a dois motivos: i) o código IL usava a forma (short) branch, ou seja, a instrução de desvio utilizada aceitava um único byte como offset e ii) meu aplicativo adicionava instruções entre a instrução de desvio e o alvo do desvio efetivamente exigindo um offset maior que 128).

Munido desta nova informação, tudo que tinha que fazer agora era verificar instrução por instrução (é claro, apenas as de desvio) se o operando da mesma estava dentro da faixa válida (-127 a 128) e corrigir quaisquer uma que não esteja. É claro que eu queria evitar isto a qualquer custo, pois este processo adicionaria mais código (e possíveis bugs) a meu aplicativo. 

Felizmente, Mono.Cecil possui dois método (MethodBody.SimplifyMacros() e  MethodBody.OptimizeMacros()) que, quando usados, se encubem em garantir que os operandos (offset) das instruções de desvio estão dentro das faixas válidas e caso não estejam, trocam a instrução para uma instrução de desvio que suporte o offset em questão (neste  case uma que use 4 bytes).

Basicamente, antes de iniciar qualquer modificação nas instruções (IL) de um método, você executa o método SimplifyMacros() e quando finalizar suas modificações você executa OptimizeMacros() e Mono.Cecil se incumbirá em ajustar as instruções de desvio (se necessário).


Desenvolvedores do Mono.Cecil: meu muito obrigado! :)

Have fun

(Read this post in english)

Resolução de métodos sobrecarregados (overloaded) em C#

Depois de um longo e tenebroso tempo em silêncio, eu voltei :)

Desta vez com um pequeno teste.

Dado o seguinte programa em C# (não importa muito a versão da linguagem), oque você espera ver no console?

using static System.Console;

class Foo 
{ 
 public void Bar(string[] o) { WriteLine("string[]"); } 
 public void Bar(object o) { WriteLine("object"); } 

 static void Main()
 {
  var foo = new Foo();
  foo.Bar(null); 
 } 
}

Responda através dos comentários

Boa diversão!

Read this post in English!

Overload resolution in C#

After a long dark period of silence I am back :)

This time a little quiz.

Given the following C# program, what you expect to get printed?

using static System.Console;

class Foo 
{ 
 public void Bar(string[] o) { WriteLine("string[]"); } 
 public void Bar(object o) { WriteLine("object"); } 

 static void Main()
 {
  var foo = new Foo();
  foo.Bar(null); 
 } 
}

Answers (if any), in comments ;)

Have fun!

Leia este post em Português!

Co & Contra variance in C#

Hi
Every now and then I find myself thinking about 2 concepts that has “just” been “extended” in C# language version 4.0 ;): co / contravariance. The interesting fact is that it always seems like my brain is going to blow :)

PS: I used the words “just” and “extended” in quotes because version 4.0 of the language has been released in April/2010 (according to Wikipedia) and C# supported covariance (in some way or another) since version 1.0!

So, in order to try to assimilate the concept, once and for all, I decided to do what I believe to be the most effective way to learn, i.e, to try to explain it to others ;) so, if you already master the subject, go away, do something else :) (this will be an informal discussion about these topics. If you want a more formal one, please see this Wikipedia page and possible follow the links).

Basically, as the Wikipedia article (IMO) puts very well, variance refers to how type inheritance affects the relationship of more complex language constructs (such arrays of derived types instead of arrays of the base type, functions returning a derived type instead of a function return the base type, etc)

Take a look in the following piece of C# code:

using System;

namespace CoContraVariance
{
 class Base { }

 class Derived : Base { }

 class Program
 {
  static void Main(string[] args)
  {
   Derived[] derivedArray = new [] { new Derived(), new Derived() };
   Base[] baseArray = derivedArray;

   foreach (var item in baseArray)
   {
    Console.WriteLine(item);
   }
  }
 }
}

Pretty simple, huh?

In line 13 we create an array of Derived objects initialized with 2 objects; then on line 13 we assign this array to a variable declared as an ‘array of Base’ objects and it just works, after all, Derived inherits from Base, and developers expects such assignment to work.

But don’t get too excited about array covariance yet because… it is broken

The problem is that arrays in C# are covariant, pero no mucho. Suppose that in the next version of the program, you were tasked to, after printing the contents to console, replace the first element of the array with an instance of the Base class, so you come up with the following code:

namespace CoContraVariance 
{
  class Base { } 
  class Derived : Base { } 

  class Program 
  {
    static void Main(string[] args)
    { 
      Derived[] derivedArray = new [] { new Derived(), new Derived() }; 
      Base[] baseArray = derivedArray; 

      foreach (var item in baseArray) { System.Console.WriteLine(item); }

      baseArray[0] = new Base();
    }
  }
} 

Piece of cake, if you don’t mind an exception being thrown at line 13. Why? Well, because since arrays in C# are covariant, but pero no mucho, you can handle an array of a derived type as an array of a base class (like our sample) when you are reading from it, but not when you are assigning to it! It is even worse than that: every assignment need to be checked for type correctness so we are paying an extra performance price whenever we assign to any position of the array.

Another C# construct that supports co/contravariance since version 2.0 of the language is “method group to delegate conversions“. As an example, take a look in the following code:

void Foo(Func<Base> f)

{

}

Derived Bar()

{

}

Even though Foo is declared as taking a function that returns a reference to a Base object, it is perfectly valid to call it as:

Foo(Bar);

Why? Well, whatever value “f” (in this case Bar() method) returns to Foo it will be either an instance of Derived or of a sub class of Derived. In any case this object “is an” instance of Base, which Foo() is prepared to handle.

The converse is also valid for parameters (in this case it is called contravariance):

void Foo(Action<Derived> f)

{

}

void Bar(Base base)

{

}

Foo(Bar);

Note that, even though conversions from method group to delegates are co/contra variant since C# version 2.0, generic delegates are not! So, the following code is invalid on C# versions < 4.0

Action<Derived> ad = Bar; // Ok, valid, method group to delegate conversion (contravariance)

Action<Base> ac = Bar;

Action<Derived> ad_error = ac; // Error.

Ok, now, as every good C# dev is aware, arrays implements the IEnumerable<T> interface, so intuitively developers expect that the following change should be supported:

using System;
using System.Collections;
using System.Collections.Generic;

namespace CoContraVariance
{
 class Base { }

 class Derived : Base { }

 class Program
 {
  static void Main(string[] args)
  {
   Derived[] derivedArray = new [] { new Derived(), new Derived() };
   Base[] baseArray = derivedArray;

   foreach (var item in baseArray)
   {
    Console.WriteLine(item);
   }

   IEnumerable<Derived> deriveds = derivedArray;
   IEnumerable<Base> bases = deriveds;
  }
 }
}

But, if you try to compile this sample against C# version < 4.0 you’ll get an error in line 24 similar to:

Error CS0266: Cannot implicitly convert type ‘IEnumerable<Derived>’ to ‘IEnumerable<Base>’. An explicit conversion exists (are you missing a cast?)

This happens because in C# language, prior to version 4.0, generic interfaces were invariant, i.e, given an interface Itf<T> and types Base and Derived (Derived inheriting from Base) Itf<Derived> had no inheritance relationship with Itf<Base> whatsoever!

In C# version 4.0 the language designers introduced co/contra variance for generic interfaces and generic delegates! The first implication for us, is that, if we try to compile our previous sample (the one in which we play with IEnumerable<T>) with such C# compiler version it works! (that happens because MS annotated IEnumerable<T> as covariant).

The second implication for us is that now we can mark our very own interfaces as such!

namespace CoContraVariance 
{ 
  interface IFoo<out T>
  { 
    T GetValue();
  } 

  class Base { }
  class Derived : Base { } 

  class Program
  {
    static void Main(string[] args)     
    { 
      IFoo<Derived> derivedItf = null; 
      IFoo<Base> baseItf = derivedItf; 
    } 
  } 
} 

The first thing that pops out our eyes is the word “out” (no pun intended) besides the generic parameter “T” and that is the way we tell the compiler that the interface IFoo is covariant in T (if you remove this “marker” line 16 becomes invalid again).

Just in case it is not clear yet, covariant type parameters may only be used as the return type of methods / properties; if you try to define parameters of such types the compiler will kindly remind you that this is not valid ;)

The other language construct, called contravariance, allows us to handle generic interfaces / delegates of a base type as interface/delegate to a more derived one (IMO, contravariance is harder to grasp since it looks like it goes against the “normal” inheritance rules).

namespace CoContraVariance 
{ 
  interface IFoo<in T>
  { 
    void DoIt(T value);
  } 

  class Base { } 
  class Derived : Base { }   
  class MoreDerived : Derived { } 

  class Program 
  { 
    static void Main(string[] args) 
    { 
      IFoo<Base> baseItf = null; 
      IFoo<Derived> derivedItf = baseItf;

      derivedItf.DoIt( new Derived() );
      derivedItf.DoIt( new MoreDerived() );
    }
  } 
}

In this sample, when a call to derivedItf.DoIt() is made we’ll be calling baseItf.DoIt(). Since the compiler will enforce that anything we pass to the former inherits either directly or indirectly from Derived and that Derived “is a” Base, the actual method called will get an instance of Base (which is exactly what it expects)!

If you want to read more about it I recommend this excelent, in deph, series of posts and also the following links:

http://msdn.microsoft.com/en-us/library/ee207183.aspx

http://msdn.microsoft.com/en-us/library/dd799517(v=vs.110).aspx

http://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)

Happy codding.