Alexey Bokov's weblog » open source

Lucene – all about distributed and real time search

Alexey Bokov — Sat, 08 Jan 2011 15:13:31 +0000

SGE 6.2u4 – new update from Regensburg

Alexey Bokov — Mon, 26 Oct 2009 09:01:23 +0000

If anyone interested in it – here’s new update for Sun Grid Engine 6.2 – update 4. It almost about bug fixing and man’s changes – list of changes is here. Sources’ tag for CVS is V62u4_TAG (make sense for Grid Engine, ARCo, SGE Inspect ), by the way as I know Hedeby is still 1.0u3.

Microsoft contribute code to Linux under the GNU GPL v.2

Alexey Bokov — Wed, 22 Jul 2009 07:04:47 +0000

Microsoft is offering about 20,000 lines of its own device drivers to the Linux kernel that will enable Linux to run as a guest on its Hyper-V virtualization technology. Specifically, the contributed loadable kernel modules enable Linux to run in ‘enlightened mode’, giving it efficiencies equivalent to a Windows virtual machine running on Hyper-V. Also see Redmond Releases Code to Linux Kernel Community at pcworld.com

Installing Sun Grid Engine on Amazon EC2

Alexey Bokov — Tue, 07 Apr 2009 17:54:45 +0000

As base AMI i used ami-7db75014 – it’s OpenSolaris supported by Sun, common informartion about installing and using OpenSolaris in EC2 also available in Sun’s Amazon EC2 Getting started guide – in this post I will focus almost in SGE using in Amazon EC2. As SGE distributive i use all-in-one tar package – i choosed “All supported platform” in Grid Engige download page – it takes about 350 Mb, but I don’t worry about platform architecture – if sun support it – it will be in this package. This ge62u2_1.tar.gz contains bunch of other tar.gz’s ( and even hedeby’s core package ) and can be unpacked by :

root@ec2-server:~/tools/archive# gzip -dc ge62u2_1.tar.gz | tar xvpf -

So I just go inside ge6.2u2_1 and unpack them all using something like this

for myfile in *.tar.gz

gzip -dc $myfile | tar xvpf -

done

One important thing – hedeby-1.0u2-core.tar.gz contains old versions of some files from ge-6.2u2_1-common.tar.gz – there’s conflicts in files common/util/arch and common/util/arch_variables – here’s diff for them – may be sometimes it can be usefull, but for my configuration it causes very strange errors when I try to install executor host :

value == NULL for attribute “mailer” in configuration list of “ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com”

./inst_sge[261]: Translate: not found [No such file or directory]

./inst_sge[263]: Translate: not found [No such file or directory]

./inst_sge[264]: Translate: not found [No such file or directory]

When I replace this files from ge-6.2u2_1-common.tar.gz installation works as expected. Next point it’s DNS configuration – SGE is very picky to DNS and it will cause some problems in running SGE Amazon EC2 instances with SGE, this stuff can be fixed using host_aliases file in SGE, or other way it’s to use /etc/hosts file for it – some kind of this technique used in Hedeby-SGE on Amazon EC2 demo, for example if we have master this name and 2 executor hosts I put this lines into /etc/hosts :

#internal_ip external_full_name external_short_name internal_full_name internal_short_name
10.yyy.xyz.zzz ec2-RRR-TTT-ZZZ-YYY.compute-1.amazonaws.com ec2-RRR-TTT-ZZZ-YYY domU-mm-ww-PPP-WWW-FFF-GGG.compute-1.internal domU-mm-ww-PPP-WWW-FFF-GGG
10.yyy.qwe.ttt ec2-aaa-bbb-ccc-ddd.compute-1.amazonaws.com ec2-aaa-bbb-ccc-ddd domU-mm-ww-JJJ-HHH-DDD-SSS.compute-1.internal domU-mm-ww-JJJ-HHH-DDD-SSS
10.yyy.pre.ppp ec2-yyy-rrr-eee-qqq.compute-1.amazonaws.com ec2-yyy-rrr-eee-qqq domU-mm-ww-UUU-III-OOO-PPP.compute-1.internal domU-mm-ww-UUU-III-OOO-PPP

Also I use hostname ec2-RRR-TTT-ZZZ-YYY ( external_short_name ) to set instance hostname – this names I use as hostnames when I configure SGE.

Sun’s Virtual Data Center, cloud API and cloud storage services

Alexey Bokov — Fri, 03 Apr 2009 09:00:05 +0000

Shravan Kumar share with me a Lew Tucker ( Sun’s CTO ) demo where he demonstates Virtual Data Center – it’s not a usual marketing speech, it’s quite interesting. As I understand Lew Tucker talks about this one -The APIs for the Sun Cloud – a RESTful API for creating and managing cloud resources, including compute, storage, and networking components – it looks very attractive and interesting so I’m goin’ to check it out shortly in time

Here’s another Lew Tucker demos – cloud APIs and how to create and deploy My SQL Virtual Machine

Lew Tucker and Dave Douglas demonstrate web developer application for cloud storage service

All of this videos ( and other sun’s cloud-related info ) is available at Sun’s Community OneEast web event home page, also you may check out sun’s A Guide to Getting Started with Cloud Computing.

Phases of open source projects

Alexey Bokov — Tue, 16 Sep 2008 19:52:43 +0000

People at grid.com are talking about open sources- Phases of open source projects ( and SGE as example of top level open source ). This post influenced by Alex Fletcher’s The five phases of maturity for open source software ecosystems. Main idea it’s to divide lifecycle to five separate stages :
* Phase 1: Purely open source stage ( Example : open source implementations of Jingle )
* Phase 2: Entrance into the marketplace ( Example – Untangle )
* Phase 3: Momentum takeoff drives growth ( Example(s) – Alfresco, Funambol, Zimbra )
* Phase 4: Predictability ( Example(s) – Red Hat Linux, MySQL )
* Phase 5: Emergence as full-fledged platform ( Example – Sun Grid Engine )

I’m highly recommending Alex Fletcher’s blog for anyone who deal with open source project managment – there’s a lot of very interesting analytics.