Alexey Bokov’s weblog

By unknown reasons Sun JDK was moved to partner repository, so to use sun jdk you need to do this steps

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get update
sudo apt-get upgrade

and then enjoy :

sudo apt-cache search jdk | grep sun
sun-java6-source - Sun Java(TM) Development Kit (JDK) 6 source files
sun-java6-jre - Sun Java(TM) Runtime Environment (JRE) 6 (architecture independent files)
sun-java6-jdk - Sun Java(TM) Development Kit (JDK) 6
sun-java6-javadb - Java(TM) DB, Sun Microsystems' distribution of Apache Derby
sun-java6-demo - Sun Java(TM) Development Kit (JDK) 6 demos and examples
sun-java6-bin - Sun Java(TM) Runtime Environment (JRE) 6 (architecture dependent files)


Short how-to about configuring Ubuntu to work with EC2 API ( for Ubuntu 10.04 users – Sun JDK was moved to partner repository so please check out Where’s sun JDK on Ubuntu 10.04 ? before follow described steps ).

Continue reading ‘Configuring Ubuntu ( 9.10 ) to work with Amazon EC2’ »

If anyone interested in it – here’s new update for Sun Grid Engine 6.2 – update 4. It almost about bug fixing and man’s changes – list of changes is here. Sources’ tag for CVS is V62u4_TAG (make sense for Grid Engine, ARCo, SGE Inspect ), by the way as I know Hedeby is still 1.0u3.

If you’re looking for free light-weight Java class decompiler – take a look onto Emmanuel Dupuy’s JD – nice stuff with size about a 1 MB.

Here’s some steps which I frequently use to configure Ubuntu to work as EC2 manager
Continue reading ‘Quick steps to configure Ubuntu for with EC2 API’ »

One way to do it consists in using queues – you may create unique queue for each host in your SGE grid ( using qconf -aq ) and specify this queue name in submitting parameters -

In case if you would like do deploy jobs onto grid from application ( C or Java ) SGE supports special API – Direct esource Managment Application API – DRMAA – here’s some examples in C++ and Java which may help to figure out this stuff. There’s SGE DRMAA Javadocs, drmaa package JavaDocs and common help – C library functions listed in section 3. To specify queue name dmraa_set_attribute function should be used as shown below :

Another way to route jon onto specific host it’s to specify request attributes in qsub  : – qsub -l <request_attr_name> – for Java example please see below. Also you may add “soft” or “hard” resource requirements modifier ( for more see SGE glossary – hard/soft resource requirements).

Here’s a listing of  drmaa C++ example which runs job on specified queue – to build it you may use this simple bash script which listed below – it works on Solaris 10, for Linux I suppose it’s better to use g++ compiler :

If you got below error when you run this example

ld.so.1: sge_drmaa_test_example.out: fatal: libdrmaa.so.1.0: open failed: No such file or directory
Killed

please checkout LD_LIBRARY_PATH environment variable, it should be set in the way like ( Solaris 10 x86 )

export LD_LIBRARY_PATH=$SGE_ROOT/lib/sol-x86/

Java implementation also use DRMAA, but it looks little different from C++ : instead of  drmaaa_set_attribute it called JobTemplate::setNativeSpecification :

job_template.setNativeSpecification(“-hard -q ” + queue_name);

Another way to run job on needed host it’s to specifying hostname as request attributes – it look like

jt.setNativeSpecification(“-l hostname=dev-host1″);

Here’s an java source for sge drmaa example or Java drmaa example archive – zip  contains source file, eclipse project and compiled binaries – to create jar you may use Eclipse export  or run inside bin folder

jar cf SgeDrmaaJobRunner.jar net/bokov/sge/*.class

To run this jar ( and run /tools/job.sh which already deployed on all executors ) on Solaris 10 I use this command

java -cp $SGE_ROOT/lib/drmaa.jar:SgeDrmaaJobRunner.jar -Djava.library.path=$LD_LIBRARY_PATH net.bokov.sge.SgeDrmaaJobRunner soft host  not_wait  /tools/job.sh host2-dev-net

Also you specify not only one queue name, but use a lists of queue’s names as parameter -

qsub -q queue_1, queue_2 $SGE_ROOT/examples/jobs/simple.sh

At least qsub allows this syntax

Little hint ‘how to add $file to maven repository‘ :
mvn -X deploy:deploy-file -Dfile=$file -DrepositoryId=_rid_ -DgroupId=_gr_id_ -DartifactId=_art_id_ -Dversion=_ver_num_ -Dpackaging=jar -Durl=_url_

It may looks like a ( i add ‘str_processing.jar’ ) :
mvn -X deploy:deploy-file \
-Dfile=$PROJECT_HOME/lib/str_processing.jar \
-DrepositoryId=repo-bokov \
-DgroupId=net.bokov.str \
-DartifactId=str-processing \
-Dversion=1.0u1 \
-Dpackaging=jar \
-Durl=http://repo.bokov.net:8080/artifactory/3rdp-releases