Posts tagged ‘Web’
For those who like to use Azure instead of AWS :
- Shanghai releases 3-year cloud computing plan – Shanghai plans to use cloud computing technologies in urban management, industry development, e-government and small and medium-sized enterprise services
AWS related links :
- State of the cloud – 2010 August
- Amazon’s EC2 Generating 220M+ Annually – really interesting post – “at 40,000 servers evenly distributed across 6 availability zones we know, ___ 6,700 servers per zone..Most of the servers are likely in the US availability zones vs. the EU zones, maybe 75-80% of total capacity”. Also take a look on amazon ec2 instances usage rates
- Anatomy of an Amazon EC2 Resource ID and based on this anatomy EC2 usage estimates
- Rumor Mill: Google EC2 Competitor Coming in 2010?
Okay, we have text file with list of urls and want to have firefox’s screenshots from this pages and also we need to have this screenshots in some normalized resolution ( like all images should be in 300×400 – thumbnails ). First of all you need to install Command line print Firefox add-on. Then create some simple script which will run firefox with needed url, print screenshot and close ( in my case via kill – may be it’s too brutal ) firefox in cycle. It may look like this ( url_list.txt – file with urls – each url on its own line 
), after running this script you will have many *.png files which is screenshots for ulrs – 0.png – for first url in urls_list.txt, 1.png for second and so on.
#!/bin/bash
id=0
while read line
do
firefox -print $line -printmode png -printdelay 10 -printfile ${id}.png
ps ax | grep firefox | awk '{ print $1 }' | xargs kill -9 ;
id=$[$id+1]
done < urls_list.txt
And now then we have screenshots ( all this guys are in different resolution in common ) then we need to normalize them – to create thumbnails for all images in 300×400 resolution – convert helps!
for f in *.png;
do
convert -thumbnail 300x400! ${f} thumb_${f}
done
And we have many thumb_*.pn with 300×400 resolution all. A little note – using resolution without ! sign will work in another way – resize will be processed proportionally with using resize only for one dimension ( bigger one ).
- The Web as a graph by Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan,Eli Upfaly, Andrew S. Tomkins
- The Web as a graph: measurements, models, and methods
- Web Graph and PageRank algorithm by Danil Nemirovsky
- Parallel implementation of graph diameter algorithms
- Diameter of the World-Wide-Web
- Fast Computation of Empirically Tight Bounds for the Diameter of Massive Graphs Cl ?mence Magnien 1 , Matthieu Latapy 1 and Michel Habib
- Structural Analysis of the Web
- Small world phenomenon
- Fast Radius Plot and Diameter Computation for Terabyte Graphs (SDM 2010)
- Probabiblistic Counting algorithms for Database applications by Philippe Flajolet
- Philippe Flajolet home page
- HADI: Fast Diameter Estimation and Mining in Massive Graphs with Hadoop
- U Kang personal home page – one of HADI authors
- Data Mining with M AP R EDUCE: Graph and Tensor Algorithms with Applications
- PEGASUS: Mining Peta-Scale Graphs