Posts tagged ‘administration’

As you know we recently add DNS services into preview mode and you can subscribe to this service using Request-AzureProvideFeature like that

Request-AzureProvideFeature -ProviderNamespace Microsoft.Network -FeatureName azurednspreview

So good question here is about how get list of all possible available options for FeatureName ?

Continue reading ‘List of all available preview services in Azure’ »

I will describe how this schema works for single VM with running Linux/Jelastic. So first of all let check that do we have for that VM by using Get-AzureDeployment command and then let go deep that happened inside that file

Continue reading ‘All about Azure Service Configuration Schema .cscfg’ »


Get-AzureVM | Where-Object {$_.InstanceSize -ne 'Basic_A0'} | Set-AzureVMSize "Basic_A0" | Update-AzureVM

Q: How to get Azure VM external IP address ( if you not use PublicIP service ) ?

A:  If you’re not using ReservedIP service field PublicIP will be empty ( as soon as there’s no reserved IP ) for command Get-AzureVM. I using Swtich-AzureMode just in case to be sure that we’re in right Azure mode in powershell right now.

PS C:\> Switch-AzureMode -Name AzureServiceManagement
PS C:\> Get-AzureVM

ServiceName                             Name                                    Status
———–                             —-                                    ——
abokov-jelastic                         abokov-jelastic                         ReadyRole


PS C:\> Get-AzureVM -ServiceName "abokov-jelastic" | Select-object  Name, DNSName, *IP*

Name                    : abokov-jelastic
DNSName                 : http://abokov-jelastic.cloudapp.net/
IpAddress               : 10.0.0.4
PublicIPAddress         :
PublicIPName            :
PublicIPDomainNameLabel :
PublicIPFqdns           : {}
OperationDescription    : Get-AzureVM

 


You may see that this VM have DNS name, internal IP, but public IP is empty, nevertheless that there’s is an external IP address which is linked to that VM. To get external IP you may use command Get-AzureEndPoint which returns all information about all endpoints for that cloud service – in my case I have 7 open ports and to make this more readable I select only few fields from output.

PS C:\> Get-AzureVM -ServiceName "abokov-jelastic" | Get-AzureEndPoint | Select-Object Name, Port , *ip*

Name Port Vip VirtualIPName
—- —- — ————-
dns-TCP 53 23.99.218.74
dns-UDP 53 23.99.218.74
http 80 23.99.218.74
http4848 4848 23.99.218.74
https 443 23.99.218.74
https4949 4949 23.99.218.74
ssh 22 23.99.218.74

 

Short how-to:

root@215003:~# mysql -u root -p
Enter password:
mysql> CREATE DATABASE s_db;
mysql> GRANT ALL PRIVILEGES ON s_db.* to 's_user'@'localhost' IDENTIFIED BY '12345678';
mysql> FLUSH PRIVILEGES;
mysql>exit
mysql -u s_user -p s_db < database.sql
<
code>

By unknown reasons Sun JDK was moved to partner repository, so to use sun jdk you need to do this steps

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get update
sudo apt-get upgrade

and then enjoy :

sudo apt-cache search jdk | grep sun
sun-java6-source - Sun Java(TM) Development Kit (JDK) 6 source files
sun-java6-jre - Sun Java(TM) Runtime Environment (JRE) 6 (architecture independent files)
sun-java6-jdk - Sun Java(TM) Development Kit (JDK) 6
sun-java6-javadb - Java(TM) DB, Sun Microsystems' distribution of Apache Derby
sun-java6-demo - Sun Java(TM) Development Kit (JDK) 6 demos and examples
sun-java6-bin - Sun Java(TM) Runtime Environment (JRE) 6 (architecture dependent files)

You can use sed :

sed -e 's/\([^\d]*\)/\L\1/' in.txt > out.txt

or perl inliner :

perl -ne 'utf8::decode $_; $_ = lc $_; utf8::encode $_; print' in.txt > out.txt
.
Both guys works fine for unicode file too.

Okay, we have text file with list of urls and want to have firefox’s screenshots from this pages and also we need to have this screenshots in some normalized resolution ( like all images should be in 300×400 – thumbnails ). First of all you need to install Command line print Firefox add-on. Then create some simple script which will run firefox with needed url, print screenshot and close ( in my case via kill – may be it’s too brutal ) firefox in cycle. It may look like this ( url_list.txt – file with urls – each url on its own line :-)), after running this script you will have many *.png files which is screenshots for ulrs – 0.png – for first url in urls_list.txt, 1.png for second and so on.

#!/bin/bash
id=0
while read line
do
firefox -print $line -printmode png -printdelay 10 -printfile ${id}.png
ps ax | grep firefox | awk '{ print $1 }' | xargs kill -9 ;
id=$[$id+1]
done < urls_list.txt

And now then we have screenshots ( all this guys are in different resolution in common ) then we need to normalize them – to create thumbnails for all images in 300×400 resolution – convert helps!

for f in *.png;
do
convert -thumbnail 300x400! ${f} thumb_${f}
done

And we have many thumb_*.pn with 300×400 resolution all. A little note – using resolution without ! sign will work in another way – resize will be processed proportionally with using resize only for one dimension ( bigger one ).

We have git repo at github (  and would like to move onto some our corporate box inside our network, for example on internal.bokov.net. This repository is shared for group of developers and all who have ssh access onto internal.bokov.net shall have right to commits onto git repository. So first of all prepare place for git repo at new hosting :

ssh user@internal.bokov.net
cd var
mkdir git_repo
chmod 777 git_repo
cd git_repo
mkdir my_project
cd my_project
git init --bare --shared=all
cd objects
chmod g+ws *

Next clone existing repo and change its settings to point onto internal.bokov.net

git clone --bare git@github.com:bokov/github_project.git
git remote add -t master -m master origin ssh://internal.bokov.net/var/git_repo/my_project/
git push origin master

to start work with new repo just use

git clone ssh://internal.bokov.net/var/git_repo/my_project/

have fun.

Short how-to about configuring Ubuntu to work with EC2 API ( for Ubuntu 10.04 users – Sun JDK was moved to partner repository so please check out Where’s sun JDK on Ubuntu 10.04 ? before follow described steps ).

Continue reading ‘Configuring Ubuntu ( 9.10 ) to work with Amazon EC2’ »

Update in Amazon Web Services:

  • 2 high cpu instance types : 64 bits – Double Extra Large с 34.2 GB RAM, and 13 ECU (4 virtual cores *3.25 EC2 compute Unit=ECU), 64-bit platform
    and Quadruple Extra Large – 68.4 GB of RAM/ 26 ECU (8 virtual cores* 3.25 ECU) : New EC2 High-Memory Instances
  • Instance prices changes ( us-east is still cheaper thatn eu-west ) : Amazon EC2 – Now an Even Better Value
  • New service for relational DB ( provisioning, scaling and other nice things ) : Introducing Amazon RDS – The Amazon Relational Database Service
  • Security stuff : Vulnerability identified in Amazon’s cloud computing
  • Amazon EC2 – Ubuntu at google groups
  • 5 years ago Amazon announced Amazon Simple Queue Service – top points of AWS for last 5 years
    • Recursively add files and foldes into CVS
    find . -type d -print | grep -v CVS | xargs -n1 cvs add
    • Delete “.svn” files from current folder
    find . -type f  -exec echo \"{}\" \;  | grep ".svn" | xargs rm

    On one of my apache I use suexec to run some tasks – some weeks before I configure all it and all stuff works well, but some days later when I try to run my cgi-perl script I found I have “500 Internal error”. I change nothing in apache config or in my scripts – I just install fresh updates for ubuntu. In apache’s error log I’ve got :

    [Wed Jul 29 08:15:28 2009] [error] (13)Permission denied: exec of ‘/usr/lib/apache2/suexec’ failed
    [Wed Jul 29 08:15:28 2009] [error] [client my_ip] Premature end of script headers: script.pl

    It’s strange, but anyway it doesn’t works. I spent some time trying to fix it ( some guys recommend to rebuilt suexec – but it’s not a simplest way to fix this problem ) – in my case it was fixed by changing owner of my suexec. I check out current properties :
    ls -la /usr/lib/apache2/suexec
    -rwsr-xr– 1 root www-data 18872 2009-07-10 18:40 /usr/lib/apache2/suexec
    and change them :

    chown root:root /usr/lib/apache2/suexec
    chmod 4755 /usr/lib/apache2/suexec

    And don’t forget to restart :

    /etc/init.d/apache2 restart

    ps. by the way suexec’s id’s are :

    #id www-user
    uid=1000(www-user) gid=109(www-group) groups=109(www-group)

    Here’s some steps which I frequently use to configure Ubuntu to work as EC2 manager
    Continue reading ‘Quick steps to configure Ubuntu for with EC2 API’ »

    Sun Grid Engine’s top engineer Richard Hierlmeier wrote article ( and some bash scripts which implements it – btw why you not to put it them onto your cvs? ) about using SDM in compute cloud ( here’s EC2 as example, I suppose that GoGrid can be used also without too many changes ) – Using SDM Cloud Adapter to Manage Solaris Zones.

    Recently I found another strange thing in EC2 – if I use “shutdown now” on ec2 instance for AWS this node will be still alive and in marked as “running” in ec2-describe-instances host list. For those who want to have simple way to shutdown ec2 instances from script I recommend to use this code ( I use it in our Convergence – Data-Aware routing on EC2 project ) :

    ec2-describe-instances | grep `hostname` | awk "{ print \"ec2-terminate-instances \" \$2; }" >> /root/tools/stop_it.sh

    You may just run stop_it.sh or schedule it via at

    at -f /root/tools/stop_it.sh "now + $TIMEOUT minutes"

    just not forget to replace $TIMEOUT with something usefull 🙂

    As base AMI i used ami-7db75014 – it’s OpenSolaris supported by Sun, common informartion about installing and using OpenSolaris in EC2 also available in Sun’s Amazon EC2 Getting started guide – in this post I will focus almost in SGE using in Amazon EC2. As SGE distributive i use all-in-one tar package – i choosed “All supported platform” in Grid Engige download page – it takes about 350 Mb, but I don’t worry about platform architecture – if sun support it – it will be in this package. This ge62u2_1.tar.gz contains bunch of other tar.gz’s ( and even hedeby’s core package ) and can be unpacked by :

    root@ec2-server:~/tools/archive# gzip -dc ge62u2_1.tar.gz | tar xvpf –

    So I just go inside ge6.2u2_1 and unpack them all using something like this

    for myfile in *.tar.gz
    do
    gzip -dc $myfile | tar xvpf –
    done

    One important thing – hedeby-1.0u2-core.tar.gz contains old versions of some files from ge-6.2u2_1-common.tar.gz – there’s conflicts in files common/util/arch  and common/util/arch_variables – here’s diff for them – may be sometimes it can be usefull, but for my configuration it causes very strange errors when I try to install executor host :

    value == NULL for attribute “mailer” in configuration list of “ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com”
    ./inst_sge[261]: Translate: not found [No such file or directory]
    ./inst_sge[263]: Translate: not found [No such file or directory]
    ./inst_sge[264]: Translate: not found [No such file or directory]

    When I replace this files from ge-6.2u2_1-common.tar.gz installation works as expected. Next point it’s DNS configuration – SGE is very picky to DNS and it will cause some problems in running SGE Amazon EC2 instances with SGE, this stuff can be fixed using host_aliases file in SGE, or other way it’s to use /etc/hosts file for it – some kind of this technique used in Hedeby-SGE on Amazon EC2 demo, for example if we have master this name and 2 executor hosts I put this lines into /etc/hosts :

    #internal_ip external_full_name external_short_name internal_full_name internal_short_name
    10.yyy.xyz.zzz ec2-RRR-TTT-ZZZ-YYY.compute-1.amazonaws.com ec2-RRR-TTT-ZZZ-YYY domU-mm-ww-PPP-WWW-FFF-GGG.compute-1.internal domU-mm-ww-PPP-WWW-FFF-GGG
    10.yyy.qwe.ttt ec2-aaa-bbb-ccc-ddd.compute-1.amazonaws.com ec2-aaa-bbb-ccc-ddd domU-mm-ww-JJJ-HHH-DDD-SSS.compute-1.internal domU-mm-ww-JJJ-HHH-DDD-SSS
    10.yyy.pre.ppp ec2-yyy-rrr-eee-qqq.compute-1.amazonaws.com ec2-yyy-rrr-eee-qqq domU-mm-ww-UUU-III-OOO-PPP.compute-1.internal domU-mm-ww-UUU-III-OOO-PPP

    Also I use hostname ec2-RRR-TTT-ZZZ-YYY ( external_short_name )  to set instance hostname – this names I use as hostnames when I configure SGE.

    I got some problems with my SGE cluster – I got some amount of Solaris 10 which running under some virtualization, all servers are the same configured and have equally environment, on one machine I install SGE master, on other SGE execution hosts – and some execution hosts works well, but on another I have strange error from “install_execd” :

    Checking hostname resolving
    —————————
    Cannot contact qmaster. The command failed:
    ./bin/sol-x86/qconf -sh
    The error message was:
    error: commlib error: access denied (client IP resolved to host name “”. This is not identical to clients host name “”)
    ERROR: unable to contact qmaster using port 10500 on host “solaris-master.devnet.int.corp”

    When I run “qconf -sh” I got :

    bash-3.00# qconf -sh
    error: commlib error: access denied (client IP resolved to host name “”. This is not identical to clients host name “”)
    ERROR: unable to contact qmaster using port 10500 on host “solaris-master.devnet.int.corp

    I check out connection – ping works, hostname resolved, telnet connection on port 10500 – it works, after I check connection from master host – there’s no problems too. I compare environment on execution hosts which are worked well with hosts which have error – they got the same environment, master host configuration also have no any suspicios-looking stuff. I try to find something usefull in web – no results, some guys have same problem, but no one knows that’s happen and how to fix it. After I try to reboot execution hosts – no effect.

    But when I try run “reboot” on master host – wow, it helps! So, guys, if you’ got the same errors with SGE – try to “reboot” on your master host – it may helps.