Alexey Bokov's weblog » bash

MySQL create user and add table

Alexey Bokov — Mon, 11 Jul 2011 12:05:07 +0000

Short how-to:
root@215003:~# mysql -u root -p Enter password: mysql> CREATE DATABASE s_db; mysql> GRANT ALL PRIVILEGES ON s_db.* to 's_user'@'localhost' IDENTIFIED BY '12345678'; mysql> FLUSH PRIVILEGES; mysql>exit mysql -u s_user -p s_db < database.sql <code>

bash tips – MySQL from script

Alexey Bokov — Mon, 31 Jan 2011 12:32:51 +0000

Here it is ( with user name, password, database host and db name ) : mysql -uuser_name -ppassword -hmysql_host_server_.domain.com -Ddatabase_name < database_commands.sql

bash tips – convert to lowercase

Alexey Bokov — Wed, 11 Aug 2010 13:27:18 +0000

You can use sed :
sed -e 's/$[^\d]*$/\L\1/' in.txt > out.txt
or perl inliner :
perl -ne 'utf8::decode $_; $_ = lc $_; utf8::encode $_; print' in.txt > out.txt.
Both guys works fine for unicode file too.

telnet in bash scripts

Alexey Bokov — Thu, 29 Jul 2010 13:34:01 +0000

Using telnet in bash scripts to automate some stuff ? It’s really easy, here’s an example to tell “hello world” on telnet server :

req="hello world" server="my_server 1234" val=`( echo open ${server} sleep 3 echo "${req}" sleep 1 ) | telnet` echo $val

How to create list of screenshots for list of urls ?

Alexey Bokov — Tue, 20 Jul 2010 13:34:24 +0000

Okay, we have text file with list of urls and want to have firefox’s screenshots from this pages and also we need to have this screenshots in some normalized resolution ( like all images should be in 300×400 – thumbnails ). First of all you need to install Command line print Firefox add-on. Then create some simple script which will run firefox with needed url, print screenshot and close ( in my case via kill – may be it’s too brutal ) firefox in cycle. It may look like this ( url_list.txt – file with urls – each url on its own line ), after running this script you will have many *.png files which is screenshots for ulrs – 0.png – for first url in urls_list.txt, 1.png for second and so on.
#!/bin/bash id=0 while read line do firefox -print $line -printmode png -printdelay 10 -printfile ${id}.png ps ax | grep firefox | awk '{ print $1 }' | xargs kill -9 ; id=$[$id+1] done < urls_list.txt

And now then we have screenshots ( all this guys are in different resolution in common ) then we need to normalize them – to create thumbnails for all images in 300×400 resolution – convert helps!
for f in *.png; do convert -thumbnail 300x400! ${f} thumb_${f} done
And we have many thumb_*.pn with 300×400 resolution all. A little note – using resolution without ! sign will work in another way – resize will be processed proportionally with using resize only for one dimension ( bigger one ).

Using JSON with bash

Alexey Bokov — Tue, 27 Apr 2010 07:21:42 +0000

Using json in bash :

jsawk : work with an array of JSON objects read from stdin, filter them using JavaScript to produce a results array that is printed to stdout. For example get’s json ( from Yahoo image search by ‘madonna’ query ) and add some info ( madonna’s real name ) to this

wget http://bokov.net/json_example.json cat json_example.json | ./jsawk "this.ResultSet.RealName=\"Louise Ciccone\"" > updated_info.json

you can use some bash/awk/sed stuff to parse json. For example – list all image urls from json

wget http://bokov.net/json_example.json cat json_example.json | sed -e 's/[{}]/''/g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | grep '"ClickUrl":' | sed 's/:/ /1' | awk -F" " '{ print $2 }'

or use library from json.org for language which you like

via v.kruchkov

Bash’s find usefull stuff

Alexey Bokov — Tue, 20 Oct 2009 09:06:45 +0000

Recursively add files and foldes into CVS

find . -type d -print | grep -v CVS | xargs -n1 cvs add

Delete “.svn” files from current folder

find . -type f  -exec echo \"{}\" \;  | grep ".svn" | xargs rm

Quick steps to configure Ubuntu for with EC2 API

Alexey Bokov — Fri, 17 Jul 2009 11:28:00 +0000

Here’s some steps which I frequently use to configure Ubuntu to work as EC2 manager
1) Remove all non sun’s java – openjdk, gij

apt-get uninstall openjdk-*

apt-get uninstall gij-*

2) And install java from sun
apt-get install sun-java6-jdk

3) Install EC2 API
cd /home/abokov/

http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip

unzip ec2-api-tools.zip

4) Configure environment : I just add lines below to my ~/.bashrc
#ec2 deal with stuff
export JAVA_HOME=/usr/lib/jvm/java-6-sun
# setting up Amazon EC2
export EC2_HOME=/home/abokov/ec2-api-tools-1.3-30349
export PATH=$PATH:$EC2_HOME/bin:
export EC2_PRIVATE_KEY=/home/abokov/keys/pk-XXX.pem
export EC2_CERT=/home/abokov/keys/cert-XXX.pem

export EC2_ACCESS_KEY_ID=***
export EC2_SECRET_ACCESS_KEY=***

It’s should be ready – you may check it out by run in terminal :

ec2-describe-instances.

Using SDM to managing OpenSolaris zones

Alexey Bokov — Tue, 07 Jul 2009 08:03:44 +0000

Sun Grid Engine’s top engineer Richard Hierlmeier wrote article ( and some bash scripts which implements it – btw why you not to put it them onto your cvs? ) about using SDM in compute cloud ( here’s EC2 as example, I suppose that GoGrid can be used also without too many changes ) – Using SDM Cloud Adapter to Manage Solaris Zones.

How to dump oracle table into tab separated text file ( csv )

Alexey Bokov — Fri, 03 Jul 2009 18:35:57 +0000

My way – run sqlplus, spool in using html markup and convert this html onto text by using sed replacements. It works fast enought, and better than parse resulted text by spool without markup. Bash script which impelement it is under below

#!/bin/bash

# alexey bokov – http://bokov.net/weblog

# script converts(spool=dump) oracle table onto tab separated text file via spooling it html and convert html to text.

# strange way, but it works.

OUTPUT_TEXT_FILE=”./oracle_db.csv”

ORACLE_USER=USER
ORACLE_PASSWORD=1234
ORACLE_TABLE=MY_DATA
HTML_FILE=”./dump.html”

ORACLE_QUERY=”./req.sql”

echo “set echo off” > $ORACLE_QUERY
echo “set markup html on” >> $ORACLE_QUERY
echo “set linesize 500″ >> $ORACLE_QUERY
echo “set pagesize 0″ >> $ORACLE_QUERY
echo “set newpage 0″ >> $ORACLE_QUERY
echo “set termout off” >> $ORACLE_QUERY
echo “spool $HTML_FILE” >> $ORACLE_QUERY
echo “select * from $ORACLE_TABLE;” >> $ORACLE_QUERY
echo “spool off” >> $ORACLE_QUERY
echo “exit;” >> $ORACLE_QUERY

echo “Spool $ORACLE_TABLE onto $HTML_FILE ( take some time )…”
sqlplus $ORACLE_USER/$ORACLE_PASSWORD @$ORACLE_QUERY

echo “Convert html onto text file ( take some time also )…”
# replace all eol
tr ‘\n’ ‘ ‘ < $HTML_FILE > $OUTPUT_TEXT_FILE
sed -i ‘s/
/\n/g’ $OUTPUT_TEXT_FILE
sed -i ‘s/<\/td>/\t/g’ $OUTPUT_TEXT_FILE

# replace all most html tags
sed -i ‘s/<[^>]*>//g’ $OUTPUT_TEXT_FILE

#replace ” &
sed -i ‘s/ / /g’ $OUTPUT_TEXT_FILE
sed -i ‘s/”/”/g’ $OUTPUT_TEXT_FILE
sed -i ‘s/&/-/g’ $OUTPUT_TEXT_FILE

#cleanup
rm $HTML_FILE
rm $ORACLE_QUERY

echo ” Done, check out $OUTPUT_TEXT_FILE for result…”