How to set MySql data-source on WSO2 product

Today I'm going to explain how to set your MySql datasource to WSO2 product. You can simply do this by few number of steps.

1. Create new data in your MySql database say 'userstore'
• CREATE DATABASE userstore;
2. Copy  your mysql connector jar (ex: mysql-connector-java-5.1.12-bin.jar) to PRODUCT_HOME/repository/components/lib directory.
3. Now go to master-datasource.xml file at PRODUCT_HOME/repository/conf/datasources
4. Replace the datasource tag with the following,

 <datasource> 
 <name>WSO2_CARBON_DB</name>
      <description>The datasource used for registry and user manager</description>
      <jndiConfig>
           <name>jdbc/WSO2CarbonDB</name>
      </jndiConfig>
      <definition type="RDBMS">
      <configuration>
           <url>jdbc:mysql://localhost:3306/userstore?autoReconnect=true</url>
           <username>root</username>
           <password>123</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
      </configuration>
      </definition>
 </datasource>

Set your MySql url, username and password accordingly.

Once you start your server with sh ./wso2server.sh -Dsetup all the required scripts will be run and your database base will be updated.

Now you are ready to go :)

Servicepack patches applying procedure

Recently I was involved in implementing Servicepack feature on Carbon kernel. Up to this implementation WSO2 products only supported to apply patches as needed. But when time goes this number of patches get increased(sometimes patche0001 to patch0100 so on....) and it will be difficult to maintain and sometimes some of these patches can get missed and will be difficult to check. Servicepack implementation was taken into account to address this scenario.

Let me first tell you the structure of a Servicepack. Servicepack is a collection of patches combined to one such that it will be easy to distribute to a customer. (ex: patch0001 to patch0080). It mainly contain two elemets,

• lib directory : which contain all the jar files corresponding to the patches that will be applied by Servicepack
• servicepack_patches.txt file : contain the list of patch numbers included on the servicepack

With this approach during server start up with ./wso2server.sh -DapplyPatches the code will first check on available service packs on $CARBON_HOME/repository/components/servicepack directory and apply the latest Servicepack available in the directory. Then with the help of servicepack_patches.txt file of the Servicepack patch applying process will apply the remaining patches that was not applied by Servicepack to the server.

The order the patches and Servicepacks get installed will be shown in the below diagram. It will first apply patch0000 which is the plugins backup directory if exist. Then it apply the latest Servicepack available. Finally the process apply the remaining patches which was not applied by Servicepack.

Then we will verify the components inside applied latest Servicepack and patch list with the $CARBON_HOME/repository/components/plugins directory. If plugins directory contain the latest patch list we assume the process is successfully completed :)

CarbonApp Deployment Process

In this artical I will discuss about the deployment process of CApp artifacts. CApp namly CarbonApplication Deployer is an collection of different artifacts bundled to a single deployable component. When deploying a CApp on any WSO2 product it directly deploy all the relevent artifacts for the product by calling the relevant artifact deployers programmatically. This will be done by the CAppDeployer when the corresponding CApp get deployed. According to this process the deployment of the artifacts will be synchronous and the artifact deployment will be atomic. So if the CApp successfully deployed we can guarantee that all the artifacts have successfully deployed.

Steps involved during deployment of CarbonApp,

• Artifacts inside CApp get extracted to temp location

• CAppDeployer then call the relevant deployer based on the artifact and deploy the artifacts

• When the CApp get deployed all its artifacts are up and running

The sample code to the new implementation is shown in the below segments. In this case I have given the example code for webapp deployment. The deployment of the other artifacts will be in same fashion except synapse artifacts.

Generate MD5 thumbprint form security certificate

Recently I was involved in implementing keystore validation on default wso2carbon keystore. The main idea behind this implementaion was to make customer aware about security risks by leaving default JKS in production because wso2 keystore is publically available since we are an open source company.

During system validation what I did was obtain the primary keystore of the vendor and validate its MD5 thumbprint value with default wso2carbon certificate thumbprint value. This method used to generate the thumb print form an X509Certificate is interesting and I hope this will be useful to someone someday + me :)


/**
* Generate the MD5 thumbprint of the certificate
*
* @param certificate that we need the thumbprint
* @return MD5 thumbprint value
* @throws CertificateEncodingException
* @throws NoSuchAlgorithmException
*/
private String getCertFingerprint(X509Certificate certificate) throws CertificateEncodingException, NoSuchAlgorithmException {
MessageDigest digestValue = MessageDigest.getInstance("MD5");
byte[] der = certificate.getEncoded();
digestValue.update(der);
byte[] digestInBytes = digestValue.digest();
return hexify(digestInBytes);
}

/**
* Helper method to hexify a byte array.
* @param bytes
* @return hexadecimal representation
*/
private String hexify(byte bytes[]) {

char[] hexDigits = {'0', '1', '2', '3', '4', '5', '6', '7',
'8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};
StringBuffer buf = new StringBuffer(bytes.length * 2);

// appending : marks to make fingerprint more readable
for (int i = 0; i < bytes.length; ++i) {
buf.append(hexDigits[(bytes[i] & 0xf0) >> 4]);
buf.append(hexDigits[bytes[i] & 0x0f] + ":");
}
// removing the last : value from the buffer string
buf.deleteCharAt(buf.length()-1);
return buf.toString();
}

How to upgrade SNAPSHOT version of carbon trunk

When upgrading a SNAPSHOT we need to apply this upgrade on orbit, kernal and platform trunks respectively. When upgrading we need to consider different types of SNAPSHOT entries on different files on trunk. I'm creating this tutorial because this will be useful to someone like I did :)

1) First we need to change all the x.x.0-SNAPSHOT entries of pom.xml files to next version value (ex: 4.1.0-SNAPSHOT to 4.2.0-SNAPSHOT)
this could be easily done using a single command
find . -name 'pom.xml'|xargs sed -i 's/4\.1\.0-SNAPSHOT/4\.2\.0-SNAPSHOT/g'

2) Need to change the 4.1.0.SNAPSHOT entry from carbon.product file (ex: 4.1.0.SNAPSHOT to 4.2.0.SNAPSHOT) on following directories
trunk/distribution/kernel/carbon.product
distribution/product/modules/p2-profile-gen/carbon.product

3) Need to change the x.x.0-SNAPSHOT entry on filter.properties file on following directory
kernal/trunk/distribution/product/modules/distribution/src/assembly

4) And on kernal and platform trunks extend the range of the valid snapshot versions according to our upgrade on parant/pom.xml
ex:
<carbon.platform.package.import.version.range>[4.1.0-SNAPSHOT, 4.2.0)
        </carbon.platform.package.import.version.range> 
to
<carbon.platform.package.import.version.range>[4.2.0-SNAPSHOT, 4.3.0)
        </carbon.platform.package.import.version.range>

Some Useful terminal commands for me

Building The Project

Build the project with all test cases
mvn install

Build the project by telling Maven to perform clean action in each module before running the install action. This will clear any compiled files you have and making sure that you're really compiling each module from scratch
mvn clean install

Build the project with all the test cases
mvn install -Dmaven.test.skip=true

When we use npu(no plugin update) flag maven won't check for updates on already downloaded packages and this will only download missing packages.
mvn install -npu -Dmaven.test.skip=true

This tee command can be used to export the terminal output for later referance.
mvn install -npu -Dmaven.test.skip=true | tee /home/manoj/Desktop/buildLog.txt

Running Carbon Server

Run the server
./wso2server.sh 

Run the server with OSGI console
./wso2server.sh -DosgiConsole

Run the server in remote debug mode (you need to configure remote debug in your configurations on your IDE)
./wso2server.sh --debug 5005

Creating and Applying Patch Files

To create a patch file execute from the directory location you required
svn diff > /home/manoj/Desktop/modify.txt

To apply a patch file execute from the directory location you required
patch -p0 < /home/manoj/Desktop/carbon-utils.patch

To display svn log history with specific limit (here 4 logs)
svn log -v --limit 4

To display current diff from the checkout copy
svn diff

String Find and Replace

Find a string recursively 
grep -r "text need to find" /etc/

Find a string on specific set of files
grep '4.3.0)' -R . --include pom.xml (ex: grep '4.3.0)' -R . --include pom.xml)

Find & replace a text StringOne from StringTwo
find . xargs sed -i 's/StringOne/StringTwo/g'

Find & replace a text StringOne from StringTwo from specific set of files(ex: from pom.xml files)
find . -name 'pom.xml'|xargs sed -i 's/StringOne/StringTwo/g'

Some other useful codes

Extract content of tar file
tar xvzf foo.tgz

Change user privilage of a file
chmod 777 wso2server.sh 

To edit bashrc file
gedit ~/.bashrc

To open 'Task Manager' on the terminal
top

How CarbonApp Get Deployed

CApp is a single file which is a combination of artifacts (axis2, data service, proxy service, gadjet server). When deploying a capp in to server it will first copy these artifacts to corresponding directory in CARBON_HOME/repository/deployment/server

ex : /webapp
       /jaxwebapp
       /data service
       /axis2services

On these directories a listener will periodically check for new artifacts and when found it will deploy the relevant artifact of the server. One drawback on the current system is that it's hard to check weather the artifact is actually deployed or not. So we have slightly modified this deployment approach as follows.

According to our new approach when CApp gets deployed these CApp artifacts extracted it to temp location and then CApp deployer calls the relevant deployers of artifacts to deploy. With this approch we can easily check about the artifacts.

What happens inside - When applying a patch on WSO2 kernal

To apply the patches first we need to do our changes in code base and generate jar files  corresponding to those changes by building the source. Then create directory with the name of the patch (ex: patch001) inside repository/components/patches directory and copy the generated jar files to the patch folder we previously created or copy the required patch files you need to apply

Run wso2server.sh -DapplyPatches,

First this will create a backup folder named patch0000 inside repository/components/patches directory containing the original content of repository/components/plugins folder. This step is conduct to support revert-back to the previous state if something get wrong during the operations. Next the content of the patches directory will incrementally (ex: patch0001, patch0002, etc.. ) get copied to the plugins directory.

WSO2 Carbon is implemented using OSGi bundles, so if we need to extend the platform it can be done by dropping OSGi bundles on directories provided. There are different directory locations that user can apply different patches and jars which need to be applied. The OSGi repository of WSO2 Carbon is located on  $CARBON_HOME/repository/components directory. $CARBON_HOME is the generated folder when we extracted an Carbon or Carbon based product. The following directories can be used to drop the external libraries to the Carbon server. After adding the required libraries the server need to be restarted to apply the changes.

• $CARBON_HOME/repository/components/patches

The patches inside this directory will be applied automatically when running "wso2server.sh -DapplyPatches" command and these patches will be coppied to plugins directory. This only support for plugin components.

Components other than plugins, user need to manually insert the corresponding entries to the bundles.info file at,

$CARBON_HOME/repository/components/configuration/org.eclipse.equinox.simpleconfigurator/bundles.info

• $CARBON_HOME/repository/components/dropins

This directory can be used to drop other OSGi bundles which need to be applied.

• $CARBON_HOME/repository/components/lib

This directory contain dropped jar files and during startup these jars will be converted to OSGi bundles and then will be copied to dropins directory.

• $CARBON_HOME/repository/components/extensions

This directory also contain dropped jar files and during startup these jars will be converted to OSGi framework Extension bundles and will be copied to dropins directory.

Referance,

http://stackoverflow.com/questions/11306929/how-to-apply-patch-files-in-wso2-carbon

http://wso2.org/library/knowledgebase/add-external-jar-libraries-wso2-carbon-based-products