Tuesday, July 7, 2015

Install Citrix Receiver on Ubuntu 14.04

Last week I need to install Citrix on my Ubuntu machine to access a remote server until I get the permissions for the network. I found [1] is useful during setting up Citrix on ubuntu servers.

Once installed when opening a session I got the following error ‘“Thawte Premium Server CA”, the issuer of the server’s security certificate (SSL error 61)’ to resolve this when searching I found steps to resolve on [2].

Hope this will be useful to me or anyone in future.

[1] https://help.ubuntu.com/community/CitrixICAClientHowTo
[2] https://www.geekpete.com/blog/ssl-error-61-using-citrix-ica-client-on-linux/

Monday, May 4, 2015

Carbon Pluggable Runtime Framework - Part 2

The Carbon Pluggable Runtime Framework is the core module on kernel level for handling 3rd party Runtime management on Carbon server. A high level architecture of the runtime framework is shown in below diagram.

Runtime service
For runtime to be registered on the Runtime Framework, it should extend the provided Runtime SPI and expose the corresponding runtime service.

Runtime Service Listener
This will be listening to runtime register/unregister events and once runtime service is registered, the Runtime Service Listener will notify the RuntimeManager.

Runtime Manager
This will keep the information about available runtimes.



Once all the required runtime instances get registered, the Runtime Framework will register the Runtime Service which provides the utility functionalities on the Runtime Framework.

What will be the advantage of using Pluggable Runtime Framework

With Carbon Pluggable Runtime Framework we can integrate/plug different 3rd party Runtime implementations to Carbon server. Not only that this framework facilitate the server to manage the registered Runtime's in a controlled manner. For example during server maintenance the underlying framework will handle the Runtime's by putting them on maintenance mode and start back when server is back on normal state. The developer or user did not need to know underlying process once you registered the Runtime on the framework.

Introduction to CARBON Runtimes - Part 1

What is a Runtime

When we considering a Runtime many aspects of a Runtime can be taken into consideration depending on its capabilities and features of it. For example if we consider Apache Axis2 Runtime it is a web service engine, Apache Tomcat is a implementation of the Java Servlet and JavaServer and Apache Synapse is a mediation engine likewise. Each of these Runtimes has its own features and capabilities. 


How features of a Runtime can be used

To facilitate its features a Runtime can expose different services that can be used. For example if we consider Axis2 Runtime it has Axis2Deployer which is responsible for artifact deployment and Axis2Runtime which is responsible about the Runtime aspect.


The Runtime can be utilized in Carbon by implementing the provided SPI implementations on the server. For example, by implementing the Deployer SPI, server can expose its custom deployer in such a way that it is recognised by the Deployment Engine. Likewise, Runtime SPI can be used to implement the runtime service.


Carbon Runtime Status

In the Carbon context, Runtime can be defined as an application level runtime instance that can run on top of Carbon OSGI framework. Runtime can have a different runtime status, depending on the state of the carbon server. Following diagram shows the available status options of a runtime.



Pending : A given Runtime is in idle state (before the Runtime initialization).

Inactive : The runtime has being initialized successfully. Runtimes can perform operations such as deploying artifacts.

Active : After the start() method of the runtime has been completed, the given runtime is fully functional and it can server requests to the runtime.

Maintenance : In this status, the runtime is on hold for maintenance work. That is, we are temporarily holding the serving of requests while the runtime is on intermediate/maintenance mode.

Git Branching Model

In Git version control system we can maintains multiple branches. For this example lets take Master branch and Development Branch.
  • Master Branch
This branch contains the most recently released Carbon Kernel. This is the main branch where the source code of HEAD always reflectsproduction-ready state.
  • Development Branch 
This branch is the main branch where the HEAD always reflects a state with the latest on going trunk development changes for the next release.

Start working on new feature

When you start to work on a new feature, you can create a separate feature branch for your self from the develop branch and start working. You can then merge your changes to the development branch once you complete the feature. The following instructions will guide you to start.


Task
Command
Description
Commit your changesgit commit -a -m "your commit message"Committing your changes to your local git repository
Creating a feature branchgit checkout -b myfeature developmentYou can create a new feature branch called myfeature from thedevelopment branch
Delete the worked feature branchgit branch -d myfeature
Incorporating changes to development branchgit checkout developmentThis will add your changes to your local development branch
Push the changes to development branchgit push origin developmentPush the changes to the central git repository under development branch


For more information visit http://nvie.com/posts/a-successful-git-branching-model/

Friday, September 5, 2014

Install SVN on RedHat Linux RHEL and configure DepSync on WSO2 worker manager nodes

  • After loggin in change the user into root user 
    • sudo -i 
  • You can install required packages using (This command will install Apache if its not already installed)
    • yum install mod_dav_svn subversion 
  • After this step SVN will be installed in the server and now we can configure it :) 
    • Navigate to /etc/httpd/conf.d/subversion.conf and modify it as below

LoadModule dav_svn_module modules/mod_dav_svn.so
LoadModule authz_svn_module modules/mod_authz_svn.so<Location /svn>
<Location>
   DAV svn
   SVNParentPath /var/www/svn
   AuthType Basic
   AuthName "Subversion repositories"
   AuthUserFile /etc/svn-auth-users
   Require valid-user
</Location>
  • You can create SVN users using following commands 
    • htpasswd -cm /etc/svn-auth-users testuser
  • This will request password for for the user
  • Now finally since we sucessfully installed SVN created SVN user next we can create the repository :) 
    • mkdir /var/www/svn
    • cd /var/www/svn
    • svnadmin create mySvnRepo
    • chown -R apache.apache mySvnRepo
  • Next we need to restart the Apache server 
    • service httpd restart
Goto http://localhost/svn/mySvnRepo address using your browser. Now by giving the SVN user credentials you can login to the repo and view the content.


Once you provided the credentials you will be able to check the your repository and the content if any,


    Now lets configure WSO2 servers as Manager and Workers

Enabling DepSync on the manager node

You can configure DepSync in the /repository/conf/carbon.xml file on the manager node by making the following changes,

<DeploymentSynchronizer>
    <Enabled>true</Enabled>
    <AutoCommit>true</AutoCommit>
    <AutoCheckout>true</AutoCheckout>
    <RepositoryType>svn</RepositoryType>
    <SvnUrl>http://localhost/svn/mySvnRep/</SvnUrl>
    <SvnUser>testuser</SvnUser>
    <SvnPassword>testpass</SvnPassword>
    <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

Enabling DepSync on the worker nodes

In worker node you need to set the AutoCommit property to false as below,

<DeploymentSynchronizer>
    <Enabled>true</Enabled>
    <AutoCommit>false</AutoCommit>
    <AutoCheckout>true</AutoCheckout>
    <RepositoryType>svn</RepositoryType>
    <SvnUrl>http://localhost/svn/mySvnRep/</SvnUrl>
    <SvnUser>testuser</SvnUser>
    <SvnPassword>testpass</SvnPassword>
    <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

[1] https://docs.wso2.com/display/CLUSTER420/SVN-based+Deployment+Synchronizer
[2] www.if-not-true-then-false.com/2010/install-svn-subversion-server-on-fedora-centos-red-hat-rhel

Friday, June 13, 2014

WSO2 ESB - JSON to SOAP (XML) transformation using Script sample


  • Reqired SOAP request as generated using SoapUI
 <?xml version="1.0" encoding="utf-8"?>  
 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">  
 <soapenv:Body>  
 <m:newOrder xmlns:m="http://services.com">  
 <m:customerName>WSO2</m:customerName>  
 <m:customerEmail>customer@wso2.com</m:customerEmail>  
 <m:quantity>100</m:quantity>  
 <m:recipe>check</m:recipe>  
 <m:resetFlag>true</m:resetFlag>  
 </m:newOrder>  
 </soapenv:Body>  
 </soapenv:Envelope>  

  • The object that I used to test on Advanced REST Client[2]:

POST request
Content-Type   : application/json
Payload           : {"newOrder": { "request": {"customerName":"WSO2", "customerEmail":"customer@wso2.com", "quantity":"100", "recipe":"check", "resetFlag":"true"}}}


  • Proxy configuration

 <?xml version="1.0" encoding="UTF-8"?>  
 <proxy xmlns="http://ws.apache.org/ns/synapse"  
 name="IntelProxy"  
 transports="https,http"  
 statistics="disable"  
 trace="disable"  
 startOnLoad="true">  
 <target>   
 <inSequence>  
 <script language="js"><![CDATA[  
 var customerName = mc.getPayloadXML()..*::customerName.toString();  
 var customerEmail = mc.getPayloadXML()..*::customerEmail.toString();  
 var quantity = mc.getPayloadXML()..*::quantity.toString();  
 var recipe = mc.getPayloadXML()..*::recipe.toString();  
 var resetFlag = mc.getPayloadXML()..*::resetFlag.toString();  
 mc.setPayloadXML(  
 <m:newOrder xmlns:m="http://services.com">  
 <m:request>  
 <m:customerName>{customerName}</m:customerName>  
 <m:customerEmail>{customerEmail}</m:customerEmail>  
 <m:quantity>{quantity}</m:quantity>  
 <m:recipe>{recipe}</m:recipe>  
 <m:resetFlag>{resetFlag}</m:resetFlag>  
 </m:request>  
 </m:newOrder>);  
 ]]></script>  
 <header name="Action" value="urn:newOrder"/>  
 <log level="full"/>  
 </inSequence>  
 <outSequence>  
 <log level="full"/>  
 <property name="messageType" value="application/json" scope="axis2"/>  
 <send/>  
 </outSequence>  
 <endpoint>  
 <address uri="http://localhost/services/BusinessService/" format="soap11"/>  
 </endpoint>  
 </target>  
 <description/>  
 </proxy> 


Referance

[1] https://docs.wso2.org/display/ESB481/Sample+441%3A+Converting+JSON+to+XML+Using+JavaScript

[2] https://chrome.google.com/webstore/detail/advanced-rest-client/hgmloofddffdnphfgcellkdfbfbjeloo

Did you forgot your MySQL password

I have installed MySQL server on my machine for many testing purposes and after that I forgot the password I used in many cases :D

There is a very simple way to reconfigure MySQL in Linux. 

manoj@manoj-Thinkpad:~$ sudo dpkg-reconfigure mysql-server-5.5 

This will allow us to reset the password on our MySql server.

Wednesday, April 16, 2014

How to invoke secured API using httpClient

Today I faced difficulty when try to invoke published API on WSO2 API Manager using https protocol. The reason for this was before invoking I haven't set the truststore definition in my HttpClient. So after some effort and Googling I wrote a sample client project and I believe this will be useful to other as well.

So better publish it :)



To set up the sample to test this client you can use Deploying and Testing YouTube API sample API on our API Manager docs.

And on the client code you need to update the Authorization Bearer and URL according to your published API. In this sample I have used client-trust.jks inside my client. But it is always recommended that client should generate his own keystore, and export the public key to client-trust.jks and then client should invoke the API using his own keystore.

You can download the sample client maven project here.

Carbon 5.0.0 [C5] Milestone 03 - Architecture

Carbon 5 [C5] is the next generation of WSO2 Carbon platform. The existing Carbon platform has served as a modular middleware platform for more than 5 years now. We've built many different products, solutions based on this platform. All the previous major releases of Carbon were sharing the same high level architecture, even though we've changed certain things time to time.

Base architecture of the Carbon is modeled using the Apache Axis2's kernel architecture. Apache Axis2 is Web service engine. But it also has introduced a rich extensible server framework with a configuration and runtime model, deployment engine, clustering API and a implementation, etc. We extended this architecture and built a OSGI based modular server development framework called Carbon Kernel. It is tightly coupled with Apache Axis2. But now Apache Axis2 is becoming a dead project. We don't see enough active development on the trunk. Therefore we thought of getting rid of this tight coupling to Apache Axis2.

Carbon kernel has gained weight over the time. There are many unwanted modules there. When there are more modules, the rate of patching or the rate of doing patch releases increases. This is why we had to release many patch releases of Carbon kernel in the past. This can become a maintenance nightmare for developers as well as for the users. We need to minimize Carbon kernel releases.

The other reason for C5 is to make Carbon kernel a general purpose OSGi runtime, specialized in hosting servers. We will implement the bare minimal features required for server developers in the Carbon kernel.

Our primary goal of C5 is to re-architect the Carbon platform from the ground up with the latest technologies and patterns to overcome the existing architectural limitations as well as to get rid of the dependencies to the legacy technologies like Apache Axis2. We need to build a next generation middleware platform that will last for the next 10 years.

Current WSO2 CARBON architecture consists of the following components:

  • Artifact Deployment Engine 
  • Centralized Logging 
  • Pluggable Runtime Framework 
  • Clustering Framework 
  • Configuration and Context Model (Experimental) 
  • Hierarchical Tenancy Model (Experimental) 
The diagram below depicts the architecture with its components.


  • Artifact Deployment Engine

    Carbon Artifact Deployment Framework is responsible for managing the deployment and undeployment of artifacts in a carbon server. You can find the internal design architecture of this module and on how to implement a new deployer and to plug it with the Artifact Deployment Framework on here.
  • Centralized Logging

    Carbon Logging Framwork has integrated PaxLogging, which is a well known open source project for to implement strong logging backend inside Carbon server. You can find more about supporting logging API's and configuration information on here.
  • Pluggable Runtime Framework

    Carbon Pluggable Runtime Framework is responsible for handling and managing integrated 3rd party Runtime's on Carbon server. You can find the internal design architecture of this module and on how to plug new 3rd party runtime with the Pluggable Runtime Framework on here.
  • Clustering Framework

    The Clustering Framework provides the clustering feature for carbon kernel which adds support for High Availability, Scalabilty and Failover. The overall architecture of clustering implementation in a single node and implementation details can be found here.
  • Configuration and Context

    Carbon Configuration and context, model the CarbonConfiguration and allows CarbonRuntime implementations to retrieve the configuration instance. CarbonRuntime represents the complete server space and CarbonContext is the entity which provides the Carbon runtime related contextual information of the current executing thread.
  • Hierarchical Tenancy Model

    Carbon Hierarchical Tenancy Model will create OSGi isolated region for individual tenants. Each and every tenant deployed in a single JVM will get his own OSGi region. This would ensure class space, bundles and OSGi service level isolation for tenants and also provide application-level isolation for tenants.

    You can try out our Milestone 03 release on https://github.com/wso2/carbon-kernel/releases/tag/5.0.0-M3

    How To Contribute

    You can find more instructions on how to contribute on our documentation site. If you have any suggestions or interested in C5 discussions, please do so via dev@wso2.org or architecture@wso2.org mailing lists .

Wednesday, February 19, 2014

Opensource ESB Performance Comparison

WSO2 recently released the latest version on their Enterprice Service Bus which is WSO2ESB 4.8.1
Along with this they have done a performance analysis with numer of leading opensouce ESB's and according to the results WSO2 ESB has proven to be not only the current fastest opensource ESB but also a pioneer with the set of features it has.

Their results of the latest round of performance testing can be found in the article that published by WSO2: WSO2 ESB Performance Round 7.5.





You can download the latest WSO2 ESB 4.8.1 product here - WSO2 Enterprise Service Bus

The user guide and the documentation can be found here - WSO2ESB Documentation

Wednesday, September 25, 2013

How to set MySql data-source on WSO2 product

Today I'm going to explain how to set your MySql datasource to WSO2 product. You can simply do this by few number of steps.


  1. Create new data in your MySql database say 'userstore'
      • CREATE DATABASE userstore;
  2. Copy  your mysql connector jar (ex: mysql-connector-java-5.1.12-bin.jar) to PRODUCT_HOME/repository/components/lib directory.
  3. Now go to master-datasource.xml file at PRODUCT_HOME/repository/conf/datasources
  4. Replace the datasource tag with the following,
 <datasource> 
 <name>WSO2_CARBON_DB</name>
      <description>The datasource used for registry and user manager</description>
      <jndiConfig>
           <name>jdbc/WSO2CarbonDB</name>
      </jndiConfig>
      <definition type="RDBMS">
      <configuration>
           <url>jdbc:mysql://localhost:3306/userstore?autoReconnect=true</url>
           <username>root</username>
           <password>123</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
      </configuration>
      </definition>
 </datasource>

Set your MySql url, username and password accordingly.

Once you start your server with sh ./wso2server.sh -Dsetup all the required scripts will be run and your database base will be updated.

Now you are ready to go :)


Friday, June 28, 2013

Servicepack patches applying procedure

Recently I was involved in implementing Servicepack feature on Carbon kernel. Up to this implementation WSO2 products only supported to apply patches as needed. But when time goes this number of patches get increased(sometimes patche0001 to patch0100 so on....) and it will be difficult to maintain and sometimes some of these patches can get missed and will be difficult to check. Servicepack implementation was taken into account to address this scenario.

Let me first tell you the structure of a Servicepack. Servicepack is a collection of patches combined to one such that it will be easy to distribute to a customer. (ex: patch0001 to patch0080). It mainly contain two elemets,

  • lib directory : which contain all the jar files corresponding to the patches that will be applied by Servicepack
  • servicepack_patches.txt file : contain the list of patch numbers included on the servicepack




With this approach during server start up with ./wso2server.sh -DapplyPatches the code will first check on available service packs on $CARBON_HOME/repository/components/servicepack directory and apply the latest Servicepack available in the directory. Then with the help of servicepack_patches.txt file of the Servicepack patch applying process will apply the remaining patches that was not applied by Servicepack to the server.

The order the patches and Servicepacks get installed will be shown in the below diagram. It will first apply patch0000 which is the plugins backup directory if exist. Then it apply the latest Servicepack available. Finally the process apply the remaining patches which was not applied by Servicepack.



Then we will verify the components inside applied latest Servicepack and patch list with the $CARBON_HOME/repository/components/plugins directory. If plugins directory contain the latest patch list we assume the process is successfully completed :)

CarbonApp Deployment Process


In this artical I will discuss about the deployment process of CApp artifacts. CApp namly CarbonApplication Deployer is an collection of different artifacts bundled to a single deployable component. When deploying a CApp on any WSO2 product it directly deploy all the relevent artifacts for the product by calling the relevant artifact deployers programmatically. This will be done by the CAppDeployer when the corresponding CApp get deployed. According to this process the deployment of the artifacts will be synchronous and the artifact deployment will be atomic. So if the CApp successfully deployed we can guarantee that all the artifacts have successfully deployed.



Steps involved during deployment of CarbonApp,
  • Artifacts inside CApp get extracted to temp location
  • CAppDeployer then call the relevant deployer based on the artifact and deploy the artifacts
  • When the CApp get deployed all its artifacts are up and running


The sample code to the new implementation is shown in the below segments. In this case I have given the example code for webapp deployment. The deployment of the other artifacts will be in same fashion except synapse artifacts.

Generate MD5 thumbprint form security certificate

Recently I was involved in implementing keystore validation on default wso2carbon keystore. The main idea behind this implementaion was to make customer aware about security risks by leaving default JKS in production because wso2 keystore is publically available since we are an open source company.

During system validation what I did was obtain the primary keystore of the vendor and validate its MD5 thumbprint value with default wso2carbon certificate thumbprint value. This method used to generate the thumb print form an X509Certificate is interesting and I hope this will be useful to someone someday + me :)


/**
* Generate the MD5 thumbprint of the certificate
*
* @param certificate that we need the thumbprint
* @return MD5 thumbprint value
* @throws CertificateEncodingException
* @throws NoSuchAlgorithmException
*/
private String getCertFingerprint(X509Certificate certificate) throws CertificateEncodingException, NoSuchAlgorithmException {
MessageDigest digestValue = MessageDigest.getInstance("MD5");
byte[] der = certificate.getEncoded();
digestValue.update(der);
byte[] digestInBytes = digestValue.digest();
return hexify(digestInBytes);
}

/**
* Helper method to hexify a byte array.
* @param bytes
* @return hexadecimal representation
*/
private String hexify(byte bytes[]) {

char[] hexDigits = {'0', '1', '2', '3', '4', '5', '6', '7',
'8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};
StringBuffer buf = new StringBuffer(bytes.length * 2);

// appending : marks to make fingerprint more readable
for (int i = 0; i < bytes.length; ++i) {
buf.append(hexDigits[(bytes[i] & 0xf0) >> 4]);
buf.append(hexDigits[bytes[i] & 0x0f] + ":");
}
// removing the last : value from the buffer string
buf.deleteCharAt(buf.length()-1);
return buf.toString();
}

Sunday, April 21, 2013

How to upgrade SNAPSHOT version of carbon trunk


When upgrading a SNAPSHOT we need to apply this upgrade on orbit, kernal and platform trunks respectively. When upgrading we need to consider different types of SNAPSHOT entries on different files on trunk. I'm creating this tutorial because this will be useful to someone like I did :)

1) First we need to change all the x.x.0-SNAPSHOT entries of pom.xml files to next version value (ex: 4.1.0-SNAPSHOT to 4.2.0-SNAPSHOT)
this could be easily done using a single command
find . -name 'pom.xml'|xargs sed -i 's/4\.1\.0-SNAPSHOT/4\.2\.0-SNAPSHOT/g'

2) Need to change the 4.1.0.SNAPSHOT entry from carbon.product file (ex: 4.1.0.SNAPSHOT to 4.2.0.SNAPSHOT) on following directories
trunk/distribution/kernel/carbon.product
distribution/product/modules/p2-profile-gen/carbon.product

3) Need to change the x.x.0-SNAPSHOT entry on filter.properties file on following directory
kernal/trunk/distribution/product/modules/distribution/src/assembly

4) And on kernal and platform trunks extend the range of the valid snapshot versions according to our upgrade on parant/pom.xml
ex:
<carbon.platform.package.import.version.range>[4.1.0-SNAPSHOT, 4.2.0)
        </carbon.platform.package.import.version.range> 
to
<carbon.platform.package.import.version.range>[4.2.0-SNAPSHOT, 4.3.0)
        </carbon.platform.package.import.version.range>

Some Useful terminal commands for me


Building The Project


Build the project with all test cases
mvn install

Build the project by telling Maven to perform clean action in each module before running the install action. This will clear any compiled files you have and making sure that you're really compiling each module from scratch
mvn clean install

Build the project with all the test cases
mvn install -Dmaven.test.skip=true

When we use npu(no plugin update) flag maven won't check for updates on already downloaded packages and this will only download missing packages.
mvn install -npu -Dmaven.test.skip=true

This tee command can be used to export the terminal output for later referance.
mvn install -npu -Dmaven.test.skip=true | tee /home/manoj/Desktop/buildLog.txt


Running Carbon Server

Run the server
./wso2server.sh 

Run the server with OSGI console
./wso2server.sh -DosgiConsole

Run the server in remote debug mode (you need to configure remote debug in your configurations on your IDE)
./wso2server.sh --debug 5005


Creating and Applying Patch Files

To create a patch file execute from the directory location you required
svn diff > /home/manoj/Desktop/modify.txt

To apply a patch file execute from the directory location you required
patch -p0 < /home/manoj/Desktop/carbon-utils.patch

To display svn log history with specific limit (here 4 logs)
svn log -v --limit 4

To display current diff from the checkout copy
svn diff

String Find and Replace

Find a string recursively 
grep -r "text need to find" /etc/

Find a string on specific set of files
grep '4.3.0)' -R . --include pom.xml (ex: grep '4.3.0)' -R . --include pom.xml)

Find & replace a text StringOne from StringTwo
find . xargs sed -i 's/StringOne/StringTwo/g'

Find & replace a text StringOne from StringTwo from specific set of files(ex: from pom.xml files)
find . -name 'pom.xml'|xargs sed -i 's/StringOne/StringTwo/g'

Some other useful codes

Extract content of tar file
tar xvzf foo.tgz

Change user privilage of a file
chmod 777 wso2server.sh 

To edit bashrc file
gedit ~/.bashrc

To open 'Task Manager' on the terminal
top

Saturday, April 20, 2013

How CarbonApp Get Deployed


CApp is a single file which is a combination of artifacts (axis2, data service, proxy service, gadjet server). When deploying a capp in to server it will first copy these artifacts to corresponding directory in CARBON_HOME/repository/deployment/server

ex : /webapp
       /jaxwebapp
       /data service
       /axis2services

On these directories a listener will periodically check for new artifacts and when found it will deploy the relevant artifact of the server. One drawback on the current system is that it's hard to check weather the artifact is actually deployed or not. So we have slightly modified this deployment approach as follows.

According to our new approach when CApp gets deployed these CApp artifacts extracted it to temp location and then CApp deployer calls the relevant deployers of artifacts to deploy. With this approch we can easily check about the artifacts.

Tuesday, April 16, 2013

What happens inside - When applying a patch on WSO2 kernal

To apply the patches first we need to do our changes in code base and generate jar files  corresponding to those changes by building the source. Then create directory with the name of the patch (ex: patch001) inside repository/components/patches directory and copy the generated jar files to the patch folder we previously created or copy the required patch files you need to apply

Run wso2server.sh -DapplyPatches,


First this will create a backup folder named patch0000 inside repository/components/patches directory containing the original content of repository/components/plugins folder. This step is conduct to support revert-back to the previous state if something get wrong during the operations. Next the content of the patches directory will incrementally (ex: patch0001, patch0002, etc.. ) get copied to the plugins directory.


WSO2 Carbon is implemented using OSGi bundles, so if we need to extend the platform it can be done by dropping OSGi bundles on directories provided. There are different directory locations that user can apply different patches and jars which need to be applied. The OSGi repository of WSO2 Carbon is located on  $CARBON_HOME/repository/components directory. $CARBON_HOME is the generated folder when we extracted an Carbon or Carbon based product. The following directories can be used to drop the external libraries to the Carbon server. After adding the required libraries the server need to be restarted to apply the changes.

  • $CARBON_HOME/repository/components/patches

The patches inside this directory will be applied automatically when running "wso2server.sh -DapplyPatches" command and these patches will be coppied to plugins directory. This only support for plugin components.

Components other than plugins, user need to manually insert the corresponding entries to the bundles.info file at,
$CARBON_HOME/repository/components/configuration/org.eclipse.equinox.simpleconfigurator/bundles.info

  • $CARBON_HOME/repository/components/dropins

This directory can be used to drop other OSGi bundles which need to be applied.
  • $CARBON_HOME/repository/components/lib

This directory contain dropped jar files and during startup these jars will be converted to OSGi bundles and then will be copied to dropins directory.
  • $CARBON_HOME/repository/components/extensions

This directory also contain dropped jar files and during startup these jars will be converted to OSGi framework Extension bundles and will be copied to dropins directory.

Referance,




Sunday, December 2, 2012

Different Big Data Categories

Big Data Analysis

We can simply describe the word ‘Big Data’ as the datasets that are difficult to handle and work with. We can reduce the depth of the meaning by categorizing it according to its properties. Mainly big data can be reduced to two dimensions as Bigness and Structure of the data. This can be further categorized as follows. [reference]

According to the above structure in our discussion on Big Data on RDMS we can categorize different aspects of this topic according to the properties of data set. We can simplify different uses of database implementation as follows,

Relational Databases on Big Data

When scaling relational database system to a relational big data system the system designer needs to address many aspects since traditional relational database systems are built to run on a single machine and tries to accomplish all queries that it receives.  When scaling these systems sometimes the read and write throughput of the system becomes too much to handle for the system. This is mainly because relational database systems were not designed for distributed or shared systems and the additional overheads incur during these processed would trouble the normal operation of the system. The scaling techniques used on such a system would cost significant complexity and loss of fault tolerance.

Relational Data Model
  • Handle high incoming request load in system
In a relational database system we can use batch write the incoming requests to increase the performance of the system. But if consider a system which handle 'Big Data' scale requests this much scaled system cannot be addressed using a batch writes and reading.
  • Creating database copies
When scaling up the database different copies of the database need to be maintaining to increase the availability of the data. If not when some system or network failure happens the database will be inconsistent or unavailable. Not only this is a manual and multi-step process in relational databases but also it creates more problems when new nodes are added to the system. These kind of bottle-necks are not suitable on system which handle huge amount of data since in-case if we need to add new nodes to increase the performance it must be less trouble and cost effective.
  • Complexity when changing the database scheme
In relational database systems when we need to change the database or one of its tables this schema migration need to handle manually on each and every shared node and updating a schema can be more time consuming and painful. So if we implement our system on Relational databases it will be more trouble during operations and maintaining the system.
  • Fault tolerance
In a parallel database system there is a high probability about one node might going down due to disk filling up, damaging or due to breakdown of a node. Even when the amount of system share increases the system will be less fault-tolerant.

Multi-structured Model on Big Data

One such example for this kind of database implementation would be a system which stores log files. These systems can be highly scalable, analytical and cost effective. When implementing these use cases NoSQL comes to action in most of the cases because of its properties which will be discussed in latter part of this report. As rapid evolution of Web service technologies which grow on exponential manner challenged the traditional theories and approaches of database systems. Due to this reason most of the researches done on multi-structured big data and its data management were conducted by large web companies like Google, Amazon, Facebook etc. since they were the initial party who faced these requirements on their systems before most of the industry driven applications adopt the technology.

No-SQL Data Model
  • Handle high incoming request load in system

The database systems designed for big data conceptually address this issue and it will be hidden form the user and the application.

  • Creating database copies
Maintaining different copies of databases manually become complex task for the system itself and for the designers of that system. In database systems designed for big data parallelism of the system will be hidden form the application and the system will be more user friendly to the system designers.
  • Complexity when changing the database scheme
No-SQL database systems which usually handle big data basses will usually keep and maintain some amount of metadata and row data about the system. So during a change in the system they use this metadata knowledge during modification rather than just aggregating from the data itself. So this kind of problems will not be occurring on the system.
  • Fault tolerance
No-SQL database systems which usually handle big data basses will by default make use some internal strategy on sharing and replication. These application not only handle parallelism internally they hide the code and application complexity form the developer and the user