
"Efficiency is doing things right;
effectiveness is doing the right things"
http://en.wikipedia.org/wiki/Peter_Drucker
If you improve the efficiency of something wrong, you make it MORE WRONG!
"It's better to do the right thing wrong, than the wrong thing right."
When you add that hack, are you making the wrong thing wronger?
"There are two kinds of mistakes…[errors of commission and errors of omission]
Errors of omission are much more important than errors of commission"
Lokku ImageSys works by accepting URLs from ETL and processing them by downloading, thumbnailing and then uploading the thumbnails to the frontend (n1, etc) servers.

"Opinions are like ..."
http://twiki.lon.lokku.net/twiki/bin/view/Lokku/ImageSysThumbnail
mysql> describe lis_live_images; +------------------+
| Field |
+------------------+
| image_url |
| image_hash |
| datasource_id |
| image_data |
| insert_time |
| human_classified |
| image_class |
+------------------+
7 rows in set (0.01 sec)
use Lokku::ImageSys::Interface qw(lis_get_searchapi_data_for_hash);[...]foreach my $image ( split(',', $rh_record->{image_data}) ) {## ImageSys START my $ra_image_data = lis_get_searchapi_data_for_hash( $image ); if (defined($ra_image_data)) { $rh_listing->{image_data} = $ra_image_data; last IMAGEDATA; ## we know only one image per listing atm } ## ImageSYS END }
lisinterface => Lokku::ImageSys::Interface->new({
product => $product,
}),
my $ra_images = $self->{lisinterface}->lis_images_for_si_build($raw_listing->{photo_url});
# Die if we're called from *modules* outside the image system
my $caller = caller;
if ($caller !~ m/^Lokku::ImageSys/ && $caller ne "main") {
croak "Lokku::ImageSys::Thumbnail is private code [...]";
}
loggu_init({ module => 'imagesys',
component => 'downloadd',
verbosity => defined($opt_verbose) ? $opt_verbose : 0,
screen => defined($opt_daemon) ? 0 : 1,
email_threshold => $ERROR,
});lokku@eb1:~ > ( cd common/logs/imagesys && ls downloadd* )
downloadd.error.log downloadd.log.2
downloadd.error.log.1 downloadd.log.3
downloadd.error.log.2 downloadd.log.4
downloadd.log downloadd.log.5
downloadd.log.1
# Run at 05:00 each Saturday0 5 * * sat : Error running weekly ImageSys lis_cron_frontend; $WIGDO $LOCKRUN --lockfile=$LOCKFILE_DIR/daily_lis_cron_frontend.lock -- nice -n 20 /home/lokku/code/bin/imagesys/lis_cron_frontend --all --verbose
lokku@n1:~ > lis_cron_frontend --help/home/lokku/code/bin/imagesys/lis_cron_frontend [options]
Commands:
--all Run all checks
--fs_check Run the file-system check
--db_scan Run the full live_images DB scan
Options:
--country Run only for this country (country name: uk, de, etc)
--verbose Run verbosely
--help This help
lokku@n1:~ > lis_cron_frontend --fs_check --country uk Jun 19 09:37:14 INFO Cleaner::perform_daily_fs_check[15422]:uk Starting filesystem check for 'uk' - /home/lokku/common/imagesys/limg/uk
lokku@eb1:~ > servicectl stop imagesys-downloaddservicectl stop: stopping imagesys-downloadd...
Waiting for 12899 to die...dead.
done.
lokku@eb1:~ > ~/code/bin/imagesys/lis_downloadd --country uk --verbose Jun 19 09:11:26 INFO Utils::lis_worker[6881]: Worker: eb1 Jun 19 09:11:26 INFO Download::_get_chunk_of_work[6881]:UK Doing chunk of work for UK (size 500) Jun 19 09:11:26 INFO DB::lis_gain_locks[6881]:UK UPDATE returned '0E0' Jun 19 09:11:26 INFO DB::lis_gain_locks[6881]:UK No rows gained. No work to do. Jun 19 09:11:26 INFO Daemon::END[6881]:UK Exiting(time_to_die: 0) lokku@eb1:~ >
Manager.pm
sub _check_stuck_jobs { }
sub _check_bad_retry_jobs { }
sub _check_bad_datasources { }Lokku::ImageSys::DB::lis_worker_report
my $image_hash = lis_hash_for_url( "http://a.com/b.jpg" );my $tree_path = lis_path_for_hash( $image_hash );
my $rh_process_details = lis_datasource_process_details('uk', 16);
my $rh_result = $Thumbnailer->create_thumbnails_of_image({
http_response => $Response, # HTTP::Response objectimage_hash => $image_hash, process_details => {sharpen => '0x1', # a string gamma => 1.0, # 0-1 quality => 90, # 0-100 }, });
my $Download = Lokku::ImageSys::Download->new({
worker => 'workerid', # Optional
country => 'uk', # Optional
noproxy => 0, # Optional
});