MANIFEST

How to generate a MANIFEST & JAR using Maven

Posted on Updated on

Hello everyone, hope you had a pretty good time writing your own javaagent. If you did, then you may have wondered about ways you can do the following stuff.

  • Generate agent.jar
  • Avoid ClassNotFoundException and NoClassDeffFoundError

If you did, this post will give you some useful tips to achieve above targets. Assuming you have got maven installed let’s look into our solution.

In our case we need to bundle our MANIFEST file and .class files of our agent into agent.jar. There are several ways you can use to generate a JAR which bundles set of files we need.

  • Using command line : Executing following command with list of files you need to include, can be used to generate a JAR. You can find more details from here.
    • jar cf jar-file input-file(s)
  • Using the IDE : Different IDEs provide different steps to generate a JAR. You can find steps to create a JAR in Eclipse from here.

I am pretty sure no one is going to love above methods, because you have to create a jar each time you make a change to your agent. Trying to include each new file you create, to the end of the above command and trying to figure out how to maintain folder structure is going to be really annoying. Use of the IDE will also be a real pain when you get to go through all the above steps after each modification.

But, what if we can easily generate our agent.jar with just a single build command? What if it includes all the .class files you need to include, plus the MANIFEST file under correct folder structure?

Well here is the good news. Including the maven-jar-plugin as follows will do it like magic. Only thing you have to do is including following section under <build><plugins></plugins></build> tags in your pom file.

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-jar-plugin</artifactId>
   <version>2.4</version>
   <configuration>
      <archive>
         <manifest>
            <addClasspath>true</addClasspath>
            <classpathPrefix>lib/</classpathPrefix>
            <addDefaultImplementationEntries>
                  true</addDefaultImplementationEntries>
            <addDefaultSpecificationEntries>
                  true</addDefaultSpecificationEntries>
         </manifest>
         <manifestEntries>
            <Premain-Class>com.javaagent.SimpleAgent</Premain-Class>
            <Can-Redefine-Classes>false</Can-Redefine-Classes>
            <Can-Retransform-Classes>true</Can-Retransform-Classes>
         </manifestEntries>
       </archive>
   </configuration>
</plugin>

The above plugin addresses two main problems faced by those who work with a javaagent. I have already listed them at the beginning of the post. Let’s have an overall idea about what this plugin does.

Generating Agent.jar

As the name itself says, this plugin is used to create a JAR during the build time of the project. (We are using Maven to build the project) It will create a jar including all the class files in your project. The package structure would be same as it is in your src folder.

Other than creating the JAR, it will include one main thing that is needed by any JAR file. Even for the javaagent we have mentioned it as one of the main files we need. That is none other, but the MANIFEST file.

We have to provide directions on which file to look for, to access the main  method of the application. In our case it is the class with the premain method. <Premain-Class> tag  provides fully qualified class name of the class with premain method. Likewise <Main-Class> can be used to set the main class as the initiating point of the jar.

<Can-Redefine-Classes> and <Can-Retransform-Classes> allows user to specify some parameters required by the agent. Setting these values are optional, because their default values won’t conflict with our basic javaagent.

Avoid ClassNotFoundException and NoClassDeffFoundError

If you have been trying to instrument methods of JDBC or any other class, where you don’t have the paths of their class files / jars included in the classpath, agent may have aborted after throwing following Exceptions.

  • ClassNotFoundException : When an application try to load a class through its String name and no definition for the class with the given name can be found. (Eg: using forname(‘foo.com.TestClass’)
  • NoClassDefFoundError : When the JVM or classloader instance try to load definition of a class and definition can no longer be found. It may have found the class definition during compilation time, but fail to find it during run time.

Being unable to find the required class during run time of the agent can be considered as the main reason for both of these situations. This can be solved by including all the required jars to the classpath of the agent. Following tags will do it for you.

  • <addClasspath> : add all the names of jars downloaded, as dependencies of the agent.
  • <classpathPrefix> : add the given prefix (Eg: lib/) before all jar names included in the MANIFEST. This can be used as the path of each jar during the runtime of the agent.

Once you have set all correct values, you are all set to make a MANIFEST and include it in the JAR during the build time of your agent. ‘mvn clean install’ will do the magic for you. I’m pretty sure you are to love this method if you have been trying above two methods.

See you again from ‘How to copy maven dependencies’. Till then have a nice time…!!

 

How to write a Simple Agent

Posted on Updated on

After two long introduction chapters, finally we are writing our simple java agent. Main target of our agent is to add an extra print statement to one of the methods in our main class (SimpleMain). In order to do that, we need to create following three files, which are the content of our agent.

  • SimpleAgent.class
  • SimpleClassFileTransformer.class
  • MANIFEST.MF

We can generate these class files once we compile our .java files. But, before we start writing, check whether you have got maven installed with you, because we are going to need it when we generate the manifest file. Otherwise, generating the manifest file going to be a real pain when you start improving your agent. Therefore, better get use to it from the first lesson itself.

So, assuming you have already created a maven project let’s add dependency of Javassist to our pom.xml file. If you don’t include it, you won’t be able to use the methods of Javassist. (hope those experienced viewers won’t get annoyed with these details, because um trying to provide answers to all the questions i came across when i was writing my simple agent) You can obtain the dependency and jar from here based on your preference.

pom.xml 

<dependency>
 <groupId>org.javassist</groupId>
 <artifactId>javassist</artifactId>
 <version>3.20.0-GA</version>
</dependency>

SimpleClassFileTransformer.java

Next, lets write our SimpleClassFileTransformer class. This is where we include all the operations we need to do during the instrumentation process.

 
package com.testProject.javaagent;

import javassist.*;

import java.io.ByteArrayInputStream;
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.security.ProtectionDomain;

public class SimpleClassFileTransformer implements 
ClassFileTransformer {
 public byte[] transform(ClassLoader loader, 
 String className, Class classBeingRedefined,
 ProtectionDomain protectionDomain, byte[] classfileBuffer)
 throws IllegalClassFormatException {

   byte[] byteCode = classfileBuffer;
   if(className.equals("com/testMain/SimpleMain")){
      ClassPool classPool = ClassPool.getDefault(); 
      try { 
         CtClass ctClass = classPool.makeClass(
                new ByteArrayInputStream(classfileBuffer)); 
         CtMethod[] methods = ctClass.getDeclaredMethods(); 
         for (CtMethod method : methods) {  
            if (method.getName().equals("print")) {
                System.out.println("Instrumenting method : " + method.getName());
                method.insertAt(1,
                  "System.out.println(\"This is the injected line\");"); 
            } 
         } 
         byteCode = ctClass.toBytecode(); 
         ctClass.detach(); 
       } catch (Throwable e) { 
            e.printStackTrace(); 
       }
   } 
 return byteCode;
 }
}

This class implements the ClassFileTransformer interface and override the transform method which can read and write class files as byte arrays. Parameters available in this method can be used in different situations based on our requirement. ClassfileBuffer contains the content of the currently instrumenting class as a byte array. When this method is called, it will obtain the current byte array and return a byte array including all the modifications done using the byte code instrumentation library. In this example, we try to obtain the class which match the fully qualified class name of our main class (com/testMain/SimpleMain) and instrument the print(). insertAt method of javassist allows us to insert our print statement to the specified line of the method. (Let’s explore other main methods of javassist later) Once the instrumentation is complete, it will recompile and verify whether insertions are according to the standards and load the classes to JVM.

SimpleAgent.java

This is where we create a transformer object and instruct to attach instruction to class (using addTransformer()). As it is mentioned earlier, invocation method of the agent is decided based on the main method used. In our example, we will be using the premain method which pass instrumentation object and other agent parameters and invoke agent before classes are load to VM.

 
package com.testProject.javaagent;

import java.lang.instrument.Instrumentation;

public class SimpleAgent {

 public static void premain(String agentArgs, 
                   Instrumentation instrumentation){

 System.out.println("Starting Agent");
 SimpleClassFileTransformer transformer = 
                  new SimpleClassFileTransformer();
 instrumentation.addTransformer(transformer);
 }
}

MANIFEST.MF

As it is mentioned in previous post, there are few attributes that need to include in the manifest file. But, for our simple agent, we will need a simple manifest file as follows. (I’ll post the code segment needed for the generation of manifest as a separate post). Class path is provided assuming all the jars needed by the agent are included in the ‘lib’ folder

Manifest-Version: 1.0
Premain-Class: org.wso2.javaagent.JDBCAgent
Class-Path: lib/javassist-3.20-GA.jar

These are the main files of our agent. Next step would be bundling of the .class files into our Agent.jar. This can also be automated using small piece of code added to our pom file. You can generate the jar using the command line as well. But i won’t go into detail about it, because it was bit annoying. But if you dig around a bit you would be able to figure it out. But, if you can wait a little, I am sure you’ll be happy with the automated way. Anyhow, assuming we have bundled our files into our Agent.jar let’s write the main class. Because, we need a main class to instrument with our agent.

SimpleMain.java

 
package com.testMain;

public class SimpleMain {
 public static void main(String[] args){
 SimpleMain sm = new SimpleMain();
 System.out.println("Simple Main");
 sm.print();
 }
 
 public void print(){
 System.out.println("Print method of Simple Main class");
 }
}

That’s it, we have everything to test our agent. Let’s run our agent using the command line. First compile the SimpleMain class. Then run the following command after going to the location of the SimpleMain.class.

> java -javaagent:path/to/agent/Agent.jar com.testMain.SimpleMain

There is a special keyword of java called as ‘-javaagent’ which is used to pass our java agent as a parameter to JVM. Value given after jar is the fully qualified name of our main class. You can also provide the class path of the main class using ‘-cp’ attribute of java.

> java -cp path/to/main -javaagent:path/to/agent/Agent.jar com.testMain.SimpleMain

Once we run our agent using above command our output will be as follows,

Starting Agent
Instrumenting method : print
Simple Main
This is the injected line
Print method of Simple Main class

Well, now you know how to write a simple java agent. Adding more features can be done modifying the the transform method. Next post will be on ‘How to generate a MANIFEST using Maven’. Future, ill get back with another simple example which uses more features of javassist.

Hope, you guys enjoy writing your first agent. Have a nice time…!!!

Requirements of a Javaagent

Posted on Updated on

Second post is to introduce main parts of a java agent. Before we jump in an write a java agent, lets see what are the main files needed by an java agent. First of all, you should notice that all the files required by the agent need to be bundled up in a JAR file. There are many ways to prepare our agent.jar. But let’s keep the bundling part aside, till we finish writing a simple agent, and move to files needed.

agent-jar-files
Main content of a Agent.jar

Unlike normal program with just a main class and bunch of other classes, agent require three main files as described below.

SimpleAgent.class

This is the class file which contain our agent. It’ll include a premain method or agentmain method according to the requirement of the writer. There may be either one method type or both in a single class file. But the selection of method used to invoke the agent will be decided based on the time user want to invoke the agent. As it is mentioned in earlier post,

  • premain() – invoke agent before VM start
  • agentmain() – invoke agent after VM is staterd

This class will be used to pass the instrumentation object along with any parameters needed by the agent and add a transformer to classes which we need to instrument.

SimpleClassFileTransformer.class

This is where provide the instructions on all the operations we need to do on our classes. This class is written by implementing the ‘ClassFileTransformer’ interface. The transform method will carryout redefinition / re-transformation of classes as specified by the user. But the layout of the instruction given here would depend based on the bytecode instrumentation framework used. As mentioned earlier, we’ll be using Javassist through out the tutorial.

MANIFEST.MF

Other most important file is the manifest file, which provide all the parameters needed to initialize the agent. This is a special type of file, which provide information on files packaged in a JAR. (For those who has worked with jars this may not be a stranger. But for me, it was totally new thing, because it was the first time i wrote one) When we create a jar, it will add a default manifest file, with set of default values as follows including environment details as follows.

Manifest-Version: 1.0
Created-By: 1.7.0_06 (Oracle Corporation)

But, in a java agent, we need to include few more attributes as given below. Except for those mentioned as optional, we will not be able to start the agent, because it will throw some kind of an error and abort the agent.

  • Premain-class – Used when agent is invoked at JVM launch time. It need to include the fully qualified name of the agent class. In our case also, we’ll be defining this attribute, because we are working with premain method. This will provide an entry point to our agent by specifying where to look for Agent.class which need to be invoked before the main.class
  • Agent-class – Used when we are invoking the agent after VM has started. This is used when we are using agentmain method in our Agent.class.
  • Can-redefine-class – (optional) Specify as true or false, where default value is false. If it is not specified as true, it won’t allow to carryout re-transformation of classes.
  • Can-redefine-class – (optional) Specify as true or false. Won’t allow to redefine classes during the instrumentation process, if it is not specified as true.
  • Class-Path – We need to add paths of all the jars needed by the agent during run time. (Eg; javassist-xyz.jar) Importing all the relevant jars while working on the IDE will make us think everything is fine. But those files will not be available during run time. Hence it will abort the agent throwing ‘ClassNotFoundException’ and ‘NoClassDefFoundError’.

Therefore, we need to make sure that we add all these parameters and necessary files to our agent before we start our agent. At first i started to manually write this manifest files, which turned out to be a real pain when the number of libraries used in the program increased. Later, figured out the easiest way to generate the manifest file during project build time. (I’ll provide the simple modification on the pom file, needed when building with Maven in a future post)

Apart from above three main files, it may include class files of any other Java classes needed by the agent as well. (For an example, if you want to have a set of methods which you want to call from the instrumented class methods, it would be less complex if you add them in a separate class and call those methods, than adding them in the SimpleClassFileTransformer.)

Well, this is the end of Requirements of a Java Agent. Let’s start writing our simple agent in our next lesson, ‘How to write a simple Agent’. (Don’t forget to install Maven and set up your IDE to make the next step easy…) See you all in next lesson.