Little handy presentation tool

Hi


Yesterday I stumbled over a really nice tool for producing screen casts (called zoomit) that allows you to zoom, draw and write directly on your desktop. If you do screen casts (or any kind of demonstration with a computer connected to a projector) I highly recommend  it (BTW I highly recommend all the Sysinternals stuff :).


Hope you find that useful.

Updating to git 1.7.11 on a Windows box

Hi


If you just updated to git version 1.7.11 and started to get the following error when you try to run git svn: 

Can't locate Git/SVN/Editor.pm in @INC (@INC contains: /lib /usr/lib/perl5/5.8.8/msys /usr/lib/perl5/5.8.8 /usr/lib/perl5/site_perl/5.8.8/msys /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl .) at C:\Program Files (x86)\Git/libexec/git-core\git-svn line 81.
BEGIN failed--compilation aborted at C:\Program Files (x86)\Git/libexec/git-core\git-svn line 81.

you can check the comments in this issue; in my case it fixed the problem.


See you!

byte code instrumentation in java...

Hi!

Are you a "low level",  developer? Have you ever disassembled a class byte code just to see what's under the hood? If not, I urge you to give it a try; IMO having at least basic knowledge about how the platform running your app works at low level may help us (developers) to write better code.
Since I joined db4o team I had the opportunity to play a lot with Mono.Cecil, a great .Net byte code manipulation lib, but I've been kind of lazy when it comes to the Java side (sure, we do employ byte code manipulation on Java code but, to be sincere, I've never touched that area of the code).
This has been slowly changing since I have joined another project at Versant (the company behind db4o) and this lead me to this post: last weekend I was debugging some Java class library code that I had no source code for¹ and I wanted to check some parameter values.
Since I was alone in my house (wife and kids went to do some shopping with my mother in law :) I decided to try 2 related technologies in the Java world: Java agents and ASM library (you can read more about Java agents here and here)².
Simply put, Java agents is a technology used to change class bytecode when it enters into the system, meaning, instead of doing the manipulation in the bytecode and saving it to the disk using java agents one can change the in memory representation of the class (without touching the actual bytes in the disk) when the JVM tries to resolve the class. If you use Sun JVM you can simply type the following command in the command line:

java -javaagent:jar-containing-agent-code.jar[=agent-args] ...

ASM is a library that allows developers to manipulate Java bytecode in a flexible, yet relatively easy way so it is a great match to use with Java agents (if you want more details I recommend to read this ASM tutorial).

Bellow you can find the code that I wrote for the agent (you can download a zipped eclipse project from here); it expects the class name and method which should be instrumented in the form className#methodName (if you omit the method name all methods will be instrumented):

package com.thinkingsoftware.diagnostics;

import java.io.*;
import java.lang.instrument.*;
import java.security.*;
import java.util.*;

import org.objectweb.asm.*;

public class DumperClassTransformer implements ClassFileTransformer {

	private final String className;
	private final String methodName;

	public DumperClassTransformer(String... args) {
		this.className = args.length > 0 ? args[0] : null;
		this.methodName = args.length > 1 ? args[1] : null;
	}
	
	private static Map> wrapperMapping = new HashMap>();
	
	{
		wrapperMapping.put(Type.INT_TYPE.getDescriptor(), Integer.class);
		wrapperMapping.put(Type.FLOAT_TYPE.getDescriptor(), Float.class);
		wrapperMapping.put(Type.DOUBLE_TYPE.getDescriptor(), Double.class);
		wrapperMapping.put(Type.BOOLEAN_TYPE.getDescriptor(), Boolean.class);
	}

	@Override
	public byte[] transform(ClassLoader loader, final String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {

		if (className.startsWith("java") || className.startsWith("sun")) {
			return classfileBuffer;
		}
		
		if (this.className == null || className.equals(this.className)) {
			
			ClassReader cr = new ClassReader(classfileBuffer);
			ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_MAXS);
			
			ClassVisitor cv = new ClassVisitor(0, cw) {
				
				@Override
				public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) {
					MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions);
					
					if (methodName != null && !name.equals(methodName)) {
						return mv;
					}
					
					try {
						String printlnStringMethodDesc = Type.getMethodDescriptor(System.err.getClass().getMethod("println", String.class));
						
						mv.visitFieldInsn(Opcodes.GETSTATIC, Type.getInternalName(System.class), "err", Type.getDescriptor(System.err.getClass()));
						mv.visitLdcInsn(name + " :");
						mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "println", printlnStringMethodDesc);
						
						Type[] args = Type.getArgumentTypes(desc);						
						for(int i = 0; i < args.length; i++) {
							
							if (args[i].getSort() == Type.ARRAY) { // not supported... :(
								continue;
							}
							
							mv.visitFieldInsn(Opcodes.GETSTATIC, Type.getInternalName(System.class), "err", Type.getDescriptor(System.err.getClass()));
							mv.visitInsn(Opcodes.DUP);
							
							mv.visitInsn(Opcodes.DUP);
							mv.visitLdcInsn("\t");
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "print", printlnStringMethodDesc);
							
							mv.visitIntInsn(Opcodes.BIPUSH, i);
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "print", "(I)V");
							
							mv.visitInsn(Opcodes.DUP);
							mv.visitLdcInsn(" = ");
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "print", printlnStringMethodDesc);							
							
							Class wrapperClass = wrapperMapping.get(args[i].getDescriptor());
							
							if (wrapperClass != null) {
								
								mv.visitVarInsn(args[i].getOpcode(Opcodes.ILOAD), i + 1);

								Type.getMethodDescriptor(Type.getType(wrapperClass), args[i]);
								mv.visitMethodInsn(
										Opcodes.INVOKESTATIC, 
										Type.getDescriptor(wrapperClass), 
										"valueOf", 
										Type.getMethodDescriptor(Type.getType(wrapperClass), args[i]));
							} else {
								mv.visitVarInsn(Opcodes.ALOAD, i + 1);
							}
							
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getType(Object.class).getDescriptor(), "toString", Type.getMethodDescriptor(Object.class.getMethod("toString")));
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "println", printlnStringMethodDesc);
						}						
					} catch (Exception ex) {
						ex.printStackTrace();
					}
					
					return mv;
				}
			};
			
			cr.accept(cv, 0);
				
			
			byte[] byteArray = cw.toByteArray();
			
			writeFile(className, byteArray);
			
			return byteArray;			
		}
		
		return classfileBuffer;
	}

	private void writeFile(String className, byte[] byteArray) {
		FileOutputStream f;
		try {
			String tempDir = System.getProperty("java.io.tmpdir");
			String outputPath = tempDir + className + "Inst.class";
			
			System.err.println();
			System.err.println("Instrumented class stored at: " + outputPath);
			System.err.println();
			System.err.println("You can look into it actuals bytecode with the following: ");
			System.err.println("javap -v -private -classpath " + tempDir + " " + className);
			System.err.println();
			
			f = new FileOutputStream(outputPath);
			f.write(byteArray, 0,  byteArray.length);
			f.close();
		} catch (IOException e) {
			e.printStackTrace();
		}
	}
}

If you want to try it you simply compile it and run some Java application; you should get  an output looking like:

running my agent against the following Java application

public class TestAgent {

	public TestAgent(String msg, int n) {
		this.msg = msg;	
	}

	public static void main(String []args) {
		TestAgent t = new TestAgent("Hallo Welt", -1);
		String msg = t.getMessage("Hello World", 42, 11.0f, true, t);
	}
	
	private String getMessage(String msg, int i, float f, boolean b, Object o) {
		return msg;
	}
	
	public String toString() {
		return "OLA MUNDO";
	}
	
	private String msg;
}

Fell free to use it anyway you like but keep in mind that this code is far from being complete or bug free or production ready (for instance it does not handle arrays) so strange errors may happen if you find one of these not supported cases.

As a last comment, if you want to play with ASM I do recommend you to take a look in the Bytecode outline plugin (but be careful to use the one in the previous link instead of the one in the Objectweb page since the later seems to have issues with newer Eclipse versions).

Hope you find this interesting!

Adriano


----

¹ Of course I could either download it from the web - after all it is an open source project- or disassembled it, but I wouldn't loose the change to play with new toys and possibly learn something new - not to mention that I severely suffer from the NIH syndrome :)


² Since I am in no way an expert in the subject, take every word with a grain of salt.

API usage rant

If you have been following me recently (Facebook, Linked-in, Tweeter, etc) you probably are aware that I have been busy taking care of the building of my new house, my job and what not (not necessarily in this order ;).

Man, building a new place proved to be a huge time sink; since we've finally finished the main construction phase I expect to blog more often.

Ok, so lets start with a rant (I am really good ranting :))

Some time ago I was playing with one of my toy projects when I got an exception while adding a new entry into a dictionary. The exception was quite self explanatory about what had happened (from the point of view of the dictionary class implementer): a duplicate key was added (which,  according to the documentation, is explicitly prohibited).

Ok, I can live with that, it was my fault after all ;) My real issue is that the message would be much more helpful had it included the actual key value (of course you override ToString() method in all of your public classes, don't you ?).
Without this information I was forced to debug the code (of course I could catch the exception and print the key but then I'd be required to change my code just to find out what was happening) to figure out what the original exception could have told me.

To my understanding, one of the reasons for this is to avoid possible sensible information from leaking and, even though IMHO this is reasonable, it makes me think whether we could do it better. 

To me looks like the dictionary class developer (product owner, or whoever) was faced with the usual coast/benefit decision: make it easier, more convenient to use or make it suitable for use when sensible information is involved and he/she decided for the later.

When writing libraries I usually tend to making it easier to be used at the coast of a higher chance of developers misuse (or using it in a way that may shut then in their foots). I confess that I had never thought about this particular scenario but now I think it could be fixed (made more flexible) by allowing the user (in this case the developer) to decide what to expose in exceptions (through an enum, an interface, you name it) and choosing a safe default value.

What do you think?

Adriano

Free (open source) RAM disk for Windows (x86 / x64)

Hi


Have you ever had the need to perform some I/O intensive operation on a relative small set of files? I do, usually when I need to compile some application.

In order to speed up this operation I've recurred to the so called "RAM Disks", applications that takes part of your computer's main memory and pretends that it is an actual disk. As you can imagine read/write/etc (I/O) operations against this virtual disk are much faster than operations against your actual hard disks (be it a traditional magnetic HD or a shine new SSD).

In the past 3 years I've used at least 3 different solutions and was happy with none. My main complain is that most of them were not capable (or it was really clumsy/hard to accomplish) of creating RAM disks dynamically, i.e, one is required to create it at OS start up time. Since my needs are dynamic (I may need more disk space at some points in time during the day) it is really hard to come up with a disk size that work for me: choose a to big disk and I am wasting precious RAM space; create a disk that's to small and I may find myself facing "out of disk space" errors (then cleaning up some junk and restarting my build tasks).

Fortunately some time ago I stumbled upon a RAM disk implementation that allows me to mount and unmount disks dynamically! Since then my work flow has been much more smooth ;) The not so good side is that it is not very easy for the "not computer savvy" user.

Bottom line is: if you need a stable and flexible (but maybe not so easy to start with) RAM disk solution you can't go wrong with this one.

What's wrong with this code - fun with assembly : the answer

In the previous post I presented the following program and asked what was the problem with it:

int fact(int n)
{
    if (n == 1)
        return 1;
    else
        n = n * fact(n - 1);
}

int main(int argc, char *argv[])
{
     int n = argc > 1 ? atoi(argv[1]) : 5;
     int i = fact(n);
   
     return printf("fact(%d): %d", n, i);
}

As I said in the previous post, the problem itself is not so hard to find out: take a close look in line 6 again! We are not missing any instruction, or are we? Actually we are missing a return  so line 6 should really look like

return n * fact(n - 1);

i.e, the developer forgot to include the return.
Note that the compiler tried to alert me with the following warning, but as you know, I just ignored it :) (as I said in the previous post, you should never ignore compiler warnings)

c:\temp\fact.c(7) : warning C4715: 'fact' : not all control paths return a value

So the interesting question is: why this program works even when it is clear that it is missing a return? In order to answer this question we are going to dive into the assembly generated code for this program. To get the assembly code just type the following in a command line (assuming you have cl.exe in your path):

cl fact.c /Fa fact.asm

Even if you have little knowledge of assembly, please, bear with me; I'll try to explain the important parts. Also I've simplified both functions (fact and main) assembly code removing not important (to this discussion) bits.

MESSAGE DB 'Fact(%d): %d', 00H
fact:
 push ebp
 mov ebp, esp
 
 cmp dword ptr [ebp+8], 1
 jne next_fact
 mov eax, 1
 jmp finish
 
next_fact:
 mov eax, [ebp+8]
 sub eax, 1
 push eax
 call fact
 add esp, 4
 imul eax, [ebp + 8]
 mov [ebp+8], eax
 
finish:
 pop ebp
 ret
 
main:
 call fact
 add esp, 4
 mov _i$[ebp], eax
 mov ecx, _i$[ebp]
 push ecx
 mov edx, _n$[ebp]
 push edx
 push OFFSET MESSAGE
 call _printf
 add esp, 12
 mov esp, ebp
 pop ebp
 ret 0

PS: If you want to generate an assembly source with more - actually lots of - information from the original C program use the following command line arguments:

cl fact.c /Fa fact.asm /FAscu

First let have some simple facts:

• fact function code starts at line 2 and extends through line 22 (ret instruction).
• Argument n is stored at address [ebp + 8]
• main function starts at line 24 and extends through line 37.
• main calls fact function on line 25 (again, please note that for brevity/simplicity reasons I removed parts of the main function, so it became easier to understand).

Now that we have a high level view of the code lets dig a little deep into the fact function code (and on main function when appropriate).

Lines 3 and 4 represents the standard C function prologue; the first interesting instruction is the one at line 6 which compares n (remember, [ebp + 8]) with 1 branching to label next_fact (line 11) if they are not equal.

Starting at line 12 the code loads n into register eax subtracts one from it and calls itself recursively. When fact returns from a previous recursive call (line 16) the code calculates n times eax i.e, the compiler used eax register to return the calculated factorial from fact. We can confirm this behavior inspecting line 27 (inside main function) which assigns eax to local variable i after calling fact.

But just using eax register to pass the return value from fact is not enough; this program works (accidentally) only because the compiler used the same register eax to perform the calculations and to pass the return value.

So we can conclude that in this particular program, omitting the return statement renders an executable equivalent to the one that would be generated had the return statement be present.

Note that different compilers (or even different versions of the same compiler) may choose other registers to perform the calculations / pass return values from functions; actually I have found at least one version of CL (from Visual Studio 6) that used register ecx to perform the multiplication and eax to pass the return values (rendering an incorrect program).

Note also that even the same version of a compiler may generate different versions of the code (one that uses eax for both, calculations and for the return values and another that uses a different set of registers) depending on options such optimization, debug, etc.


Best

What's wrong with this code - fun with assembly

Since, I'm a bit busy to create new posts I decided to cheat, translating this post from my old (Portuguese) blog :)

Before we continue let me make this clear: finding the actual issue is not terrible hard (even though it took me some time and of course I'll keep the answer to a future post :); the interesting part is to figure it out why it works (or at least, why it works when compiled with some compilers).

Ok, so lets go. Take a look in the following program: 


This program accepts a number (command line) and calculates its factorial.

I came across it while I was corrigindo a programming assignment from one of my students (yep, 14 years ago I used to work as a professor teaching C, C++, assembly and the like).

At that time I used Borland C++ 3.0 to compile and to my astonishment it worked! Last time, when I tried to compile it with VC 6 I noticed a different behavior (the application produced a wrong result) but using Visual Studio 2010 I can reproduce the same behavior as when I first compiled it with Borland C++ 3.0; of course, the compiler generated some warnings which were promptly ignored!

Warning

To ignore compiler warnings is not such a smart idea and of course I'm not recommending you should do it! Pelo contrário; check all compiler warnings in your code and understand the reasons leading the compiler to complain.Do not assume that the compiler has bugs (after all, Select is not broken). If the conclusion is that the code generating the warning is legitimate I suggest you to try to change it, or, in the worst case, disable the specific warning (via #pragma warning) just in the parts of the code that is producing the warning.

int fat(int n)
{
    if (n == 1)
        return 1;
    else
        n = n * fat(n - 1);
}

int main(int argc, char *argv[])
{
     int n = argc > 1 ? atoi(argv[1]) : 5;
     int i = fat(n);
   
     return printf("Fat(%d): %d", n, i);
 }

Have you figured it out already?


See you!