Jul 24, 2012

Arduwhat ?

Hi
Since I was 5 ~ 8 years old I was interested in mechanics, electronics and how things worked in general - my mother has some interesting histories about me disassembling all sort of devices (and usually not being able to put them together again); one in particular in which I got a saw as a "gift" (I was only 6 or 7 and someone gave me a real saw, not a toy, plastic one!) and went to put it into good use sawing some furniture :).

Even though having a high interest in electronics I ended up in the computer software field (just to make it clear: I do enjoy being involved in such fast changing, challenging industry :) and since then I've been having lots of fun (and some frustrations also).
As anyone involved with IT knows, keeping up to date with such a fast changing field requires a lot of effort and over time I put my electronics pretensions to rest (after all there are enough fun stuff in the computer world to play with to keep me busy for a long time...).
But about one year ago this started to change as I started to play with some basic (really basic) electronics stuff again (just for fun) and I stumbled on a small but very interesting piece of hardware: a small board with a microcontroller and a lot of software support with some nice characteristics (at least from my point of view ;) :
  • Open Source Software / Hardware
  • Inexpensive (will not cost you an arm and a leg :)
  • Simple to work with, at least not as complex as the other options I had seem before (no complex, expensive tools required)
  • Easily integrated with the "external" world (through sensors/actuators).
  • Extensible (shields, breakout boards and virtually anything you want)
  • Well supported
  • Large community
which perfectly describes this nice board called "Arduino" (since the name Arduino is trademarked most probably you'll  find compatible devices under some generic brand named with a "duino" suffix).
At this point you may be wondering what can be accomplished with such beautiful piece of hardware... well, I guess it is fair to say that you can find an Arduino C/C++ library for virtually anything you can come up (of course this is not completely true, but it is not too far from the truth). IMO, the important  question is not "is it possible to connect X to Arduino" but instead "how hard is to connect X to Arduino" and based on the list bellow (which I wrote based on information I found) I conclude that, excluding more exotic hardware, it does pretty well job on this topic (note that almost all items in the list bellow requires extra hardware): 
  • Sensors
    • Temperature, Pressure, Gas, Light, Distance, Color, Humidity, Accelerometers, etc.
  • Communications
    • Bluetooth, TCP/IP, UDP/IP, Serial, GPRS, Infrared, etc.
  • Storage
    • EEPROM, SD cards
  • Actuators/Output
    • Step Motors, Servos, DC Motors, LCDs, etc
As you can see it is quite flexible, and more importantly, all of the hardware above have at least one Arduino library ready to be used.

This post is not intended to be an Arduino introduction / course / tutorial. Instead it is meant to be only a teaser so if you want to learn more, you can find lots of information in the internet. Personally there are some sites I highly recommend you to give a visit:

Also, if you want to buy Arduino related hardware you can go to your favorite search engine and search for it and I am sure you'll be able to find lots of stores that sell them. But if you don't want to search, bellow you can find a list of sites in which you can buy Arduino parts (ps: I have no affiliation with any of them :).


Finally, some electronics interesting tools:


Well, that's it. Now let me get back to play with my just arrived Arduino!

In the next post I hope to shed some light about my current projects related to Arduino (both soft and hardware).

Hope you found this post interesting.

Jul 7, 2012

Did you know (Part II) File / Registry virtualization on Windows 7 / Vista

Hi

In this post I discussed some issues I faced regarding software running on Vista 64 bits. Well it ended up that all my troubles was happening because my program (64 bits) was launching IE 64 bits (instead of the 32 bits) and I had not installed Silverlight plugin on it! Solution: just switch my program to 32 bits and it worked as expected!

In this post I want to discuss some pitfalls when running software on x64 bit versions of Windows (most of the discussion applies equally well to Vista x64 ,Windows 7 x64 and probably to newer versions also - haven't tried it with Windows  8 but I bet that the behavior is the same).


Keep in mind that when talking about the reasons that lead Microsoft to add a specific feature I am just speculating, so take every word with a grain of salt :)


On previous Windows versions, to many (almost every single?) developer used to login as
admistrator into their workstations1 so there were way to many applications that relied on being able to change system resources (such as to create files/fonders under "Program Files"), after all they always succeed doing that since developers had the rights to do so (remember, administrator...).

At Vista launch time MS had a tough issue to deal with: how raise the bar for system security / stability without breaking such ("legacy") applications?


AFAIK a compromise was settled so Windows development team decided that for
some 32 bit applications such requests would not fail, but instead, Windows would lie to the calling process and would just create the file somewhere else (the same principle apply to registry access): this feature is known as file virtualization (the rules that governs whether file/registry will be enabled or not for a specific process can be found here)

Note that this feature is meant to allow
legacy applications2 to run on the new platform (at that time, Windows Vista) so Microsoft plans to remove it (as soon as your application gets compiled as 64 bits, virtualization will be disabled anyway).

If you are running the 64 bits version of Windows 7 (or Vista for that matter) y
ou can try it right now by downloading the test application source3 and running it with something like:

FileSystemVirtualizationTest.exe -e %windir%\test.txt "adriano's test"
For now, just run it with the arguments shown above and take a closer look in the produced output:

Virtualization ENABLED.

Virtualization is ON

14 bytes writen to C:\Windows\test.txt
It seams that the application succeed in creating a file under %windir% (which it should obviously not be allowed to) but how can we confirm that? Easy, just run the application again now providing only the file name (the application will try to read the file and dump its contents):
FileSystemVirtualizationTest.exe %windir%\test.txt

Virtualization is ON

adriano's test
and there it is! But wait! Let's open Windows Explorer and try to locate the file. Hey! I can't find the file anymore! Windows search can't find it either:

But the application does succeeded opening the file! how can that be? Launch Windows Explorer again on %windir% folder but this time paying attention to the little button: "Compatibility Files":

Ok, now that you noticed this button, press it and Windows Explorer will open a folder that looks like the one bellow:


Wow! Windows just created the file under %localappdata%\VirtualStore\Windows folder (which is a per user folder). 

This way Windows let applications believe they had successfully updated files / folders / registry keys when actually they didn't (preventing possible global system corruption). Clearly avoiding system corruption is a fair reason to support this functionality, but now, developers (and IT admins) must pay attention to scenarios with mixed 32 / 64 bits applications interacting in such ways that they need to observe changes from each other (since the first ones may be writing to the virtual store and the last ones will not see the virtual store).

Last, but not least :), make sure your application do not depend on registry/file virtualization; these mechanisms were introduced sonly to raise the likelihood of legacy applications to run without problems on Windows versions with stricter security settings (Vista, 7, etc). So if you have an application that does writes to protected file system areas, please do your users a favor: update it to write to a more appropriated location (%appdata%, %localappdata%, etc).

Bellow you can find some links if you want to read more about file system virtualization:


Have fun!

Adriano


[1] I am guiltily of doing the same thing :(
[2] Basically all applications developed for previous Windows OSs are handled as legacy by Vista
[3] In the first drafts of this post I used Notepad++ to write to %windir% but it looks like they added a manifest in the latest versions so it will not work for our demonstration purposes. Even in my sample application I was required to remove the manifest explicitly (VS 10 included one that disabled virtualization)

Jul 5, 2012

Little handy presentation tool

Hi


Yesterday I stumbled over a really nice tool for producing screen casts (called zoomitthat allows you to zoom, draw and write directly on your desktop. If you do screen casts (or any kind of demonstration with a computer connected to a projector) I highly recommend  it (BTW I highly recommend all the Sysinternals stuff :).


Hope you find that useful.

Jun 27, 2012

Updating to git 1.7.11 on a Windows box

Hi


If you just updated to git version 1.7.11 and started to get the following error when you try to run git svn

Can't locate Git/SVN/Editor.pm in @INC (@INC contains: /lib /usr/lib/perl5/5.8.8/msys /usr/lib/perl5/5.8.8 /usr/lib/perl5/site_perl/5.8.8/msys /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl .) at C:\Program Files (x86)\Git/libexec/git-core\git-svn line 81.
BEGIN failed--compilation aborted at C:\Program Files (x86)\Git/libexec/git-core\git-svn line 81.
you can check the comments in this issue; in my case it fixed the problem.


See you!

Jun 10, 2012

byte code instrumentation in java...

Hi!

Are you a
"low level",  developer? Have you ever disassembled a class byte code just to see what's under the hood? If not, I urge you to give it a try; IMO having at least basic knowledge about how the platform running your app works at low level may help us (developers) to write better code.
Since I joined db4o team I had the opportunity to play a lot with Mono.Cecil, a great .Net byte code manipulation lib, but I've been kind of lazy when it comes to the Java side (sure, we do employ byte code manipulation on Java code but, to be sincere, I've never touched that area of the code).
This has been slowly changing since I have joined another project at Versant (the company behind db4o) and this lead me to this post: last weekend I was debugging some Java class library code that I had no source code for1 and I wanted to check some parameter values.
Since I was alone in my house (wife and kids went to do some shopping with my mother in law :) I decided to try 2 related technologies in the Java world: Java agents and ASM library (you can read more about Java agents here and here)2.
Simply put, Java agents is a technology used to change class bytecode when it enters into the system, meaning, instead of doing the manipulation in the bytecode and saving it to the disk using java agents one can change the in memory representation of the class (without touching the actual bytes in the disk) when the JVM tries to resolve the class. If you use Sun JVM you can simply type the following command in the command line:
java -javaagent:jar-containing-agent-code.jar[=agent-args] ...
ASM is a library that allows developers to manipulate Java bytecode in a flexible, yet relatively easy way so it is a great match to use with Java agents (if you want more details I recommend to read this ASM tutorial).

Bellow you can find the code that I wrote for the agent 
(you can download a zipped eclipse project from here); it expects the class name and method which should be instrumented in the form className#methodName (if you omit the method name all methods will be instrumented):
package com.thinkingsoftware.diagnostics;

import java.io.*;
import java.lang.instrument.*;
import java.security.*;
import java.util.*;

import org.objectweb.asm.*;

public class DumperClassTransformer implements ClassFileTransformer {

	private final String className;
	private final String methodName;

	public DumperClassTransformer(String... args) {
		this.className = args.length > 0 ? args[0] : null;
		this.methodName = args.length > 1 ? args[1] : null;
	}
	
	private static Map> wrapperMapping = new HashMap>();
	
	{
		wrapperMapping.put(Type.INT_TYPE.getDescriptor(), Integer.class);
		wrapperMapping.put(Type.FLOAT_TYPE.getDescriptor(), Float.class);
		wrapperMapping.put(Type.DOUBLE_TYPE.getDescriptor(), Double.class);
		wrapperMapping.put(Type.BOOLEAN_TYPE.getDescriptor(), Boolean.class);
	}

	@Override
	public byte[] transform(ClassLoader loader, final String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {

		if (className.startsWith("java") || className.startsWith("sun")) {
			return classfileBuffer;
		}
		
		if (this.className == null || className.equals(this.className)) {
			
			ClassReader cr = new ClassReader(classfileBuffer);
			ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_MAXS);
			
			ClassVisitor cv = new ClassVisitor(0, cw) {
				
				@Override
				public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) {
					MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions);
					
					if (methodName != null && !name.equals(methodName)) {
						return mv;
					}
					
					try {
						String printlnStringMethodDesc = Type.getMethodDescriptor(System.err.getClass().getMethod("println", String.class));
						
						mv.visitFieldInsn(Opcodes.GETSTATIC, Type.getInternalName(System.class), "err", Type.getDescriptor(System.err.getClass()));
						mv.visitLdcInsn(name + " :");
						mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "println", printlnStringMethodDesc);
						
						Type[] args = Type.getArgumentTypes(desc);						
						for(int i = 0; i < args.length; i++) {
							
							if (args[i].getSort() == Type.ARRAY) { // not supported... :(
								continue;
							}
							
							mv.visitFieldInsn(Opcodes.GETSTATIC, Type.getInternalName(System.class), "err", Type.getDescriptor(System.err.getClass()));
							mv.visitInsn(Opcodes.DUP);
							
							mv.visitInsn(Opcodes.DUP);
							mv.visitLdcInsn("\t");
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "print", printlnStringMethodDesc);
							
							mv.visitIntInsn(Opcodes.BIPUSH, i);
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "print", "(I)V");
							
							mv.visitInsn(Opcodes.DUP);
							mv.visitLdcInsn(" = ");
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "print", printlnStringMethodDesc);							
							
							Class wrapperClass = wrapperMapping.get(args[i].getDescriptor());
							
							if (wrapperClass != null) {
								
								mv.visitVarInsn(args[i].getOpcode(Opcodes.ILOAD), i + 1);

								Type.getMethodDescriptor(Type.getType(wrapperClass), args[i]);
								mv.visitMethodInsn(
										Opcodes.INVOKESTATIC, 
										Type.getDescriptor(wrapperClass), 
										"valueOf", 
										Type.getMethodDescriptor(Type.getType(wrapperClass), args[i]));
							} else {
								mv.visitVarInsn(Opcodes.ALOAD, i + 1);
							}
							
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getType(Object.class).getDescriptor(), "toString", Type.getMethodDescriptor(Object.class.getMethod("toString")));
							mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, Type.getInternalName(System.err.getClass()), "println", printlnStringMethodDesc);
						}						
					} catch (Exception ex) {
						ex.printStackTrace();
					}
					
					return mv;
				}
			};
			
			cr.accept(cv, 0);
				
			
			byte[] byteArray = cw.toByteArray();
			
			writeFile(className, byteArray);
			
			return byteArray;			
		}
		
		return classfileBuffer;
	}

	private void writeFile(String className, byte[] byteArray) {
		FileOutputStream f;
		try {
			String tempDir = System.getProperty("java.io.tmpdir");
			String outputPath = tempDir + className + "Inst.class";
			
			System.err.println();
			System.err.println("Instrumented class stored at: " + outputPath);
			System.err.println();
			System.err.println("You can look into it actuals bytecode with the following: ");
			System.err.println("javap -v -private -classpath " + tempDir + " " + className);
			System.err.println();
			
			f = new FileOutputStream(outputPath);
			f.write(byteArray, 0,  byteArray.length);
			f.close();
		} catch (IOException e) {
			e.printStackTrace();
		}
	}
}


If you want to try it you simply compile it and run some Java application; you should get  an output looking like:
running my agent against the following Java application

public class TestAgent {

	public TestAgent(String msg, int n) {
		this.msg = msg;	
	}

	public static void main(String []args) {
		TestAgent t = new TestAgent("Hallo Welt", -1);
		String msg = t.getMessage("Hello World", 42, 11.0f, true, t);
	}
	
	private String getMessage(String msg, int i, float f, boolean b, Object o) {
		return msg;
	}
	
	public String toString() {
		return "OLA MUNDO";
	}
	
	private String msg;
}


Fell free to use it anyway you like but keep in mind that this code is far from being complete or bug free or production ready (for instance it does not handle arrays) so strange errors may happen if you find one of these not supported cases.

As a last comment, if you want to play with ASM I do recommend you to take a look in the
Bytecode outline plugin (but be careful to use the one in the previous link instead of the one in the Objectweb page since the later seems to have issues with newer Eclipse versions).

Hope you find this interesting!


Adriano



----

1 Of course I could either download it from the web - after all it is an open source project- or disassembled it, but I wouldn't loose the change to play with new toys and possibly learn something new - not to mention that I severely suffer from the NIH syndrome :)


2 Since I am in no way an expert in the subject, take every word with a grain of salt.

May 19, 2012

API usage rant

If you have been following me recently (Facebook, Linked-in, Tweeter, etc) you probably are aware that I have been busy taking care of the building of my new house, my job and what not (not necessarily in this order ;).

Man, building a new place proved to be a huge time sink; since we've finally finished the main construction phase I expect to blog more often.

Ok, so lets start with a rant (I am really good ranting :))

Some time ago I was playing with one of my toy projects when I got an exception while adding a new entry into a dictionaryThe exception was quite self explanatory about what had happened (from the point of view of the dictionary class implementer): a duplicate key was added (which,  according to the documentation, is explicitly prohibited).

Ok, I can live with that, it was my fault after all ;) My real issue is that the message would be much more helpful had it included the actual key value (of course you override
ToString() method in all of your public classes, don't you ?).
Without this information I was forced to debug the code (of course I could catch the exception and print the key but then I'd be required to change my code just to find out what was happening) to figure out what the original exception could have told me.

To my understanding, one of the reasons for 
this is to avoid possible sensible information from leaking and, even though IMHO this is reasonable, it makes me think whether we could do it better. 

To me looks like the dictionary class developer (product owner, or whoever) was faced with the usual coast/benefit decision: make it easier, more convenient to use or make it suitable for use when sensible information is involved and he/she decided for the later.


When writing libraries I usually tend to making it easier to be used at the coast of a higher chance of developers misuse (or using it in a way that may shut then in their foots). I confess that I had never thought about this particular scenario but now I think it could be fixed (made more flexible) by allowing the user (in this case the developer) to decide what to expose in exceptions (through an enum, an interface, you name it) and choosing a safe default value.


What do you think?


Adriano

May 9, 2012

Free (open source) RAM disk for Windows (x86 / x64)

Hi


Have you ever had the need to perform some I/O intensive operation on a relative small set of files? I do, usually when I need to compile some application.

In order to speed up this operation I've recurred to the so called "RAM Disks", applications that takes part of your computer's main memory and pretends that it is an actual disk. As you can imagine read/write/etc (I/O) operations against this virtual disk are much faster than operations against your actual hard disks (be it a traditional magnetic HD or a shine new SSD).

In the past 3 years I've used at least 3 different solutions and was happy with none. My main complain is that most of them were not capable (or it was really clumsy/hard to accomplish) of creating RAM disks dynamically, i.e, one is required to create it at OS start up time. Since my needs are dynamic (I may need more disk space at some points in time during the day) it is really hard to come up with a disk size that work for me: choose a to big disk and I am wasting precious RAM space; create a disk that's to small and I may find myself facing "out of disk space" errors (then cleaning up some junk and restarting my build tasks).

Fortunately some time ago I stumbled upon a RAM disk implementation that allows me to mount and unmount disks dynamically! Since then my work flow has been much more smooth ;) The not so good side is that it is not very easy for the "not computer savvy" user.

Bottom line is: if you need a stable and flexible (but maybe not so easy to start with) RAM disk solution you can't go wrong with this one.