Support

Support is part of Event Gallery Extended.

Please get a subscription if you need support. Feel free to use the ticket system or the contact form for reporting defects or pre-sale questions. Make sure you're logged in in order to be able to create a new ticket.

For general information you can also jump to the manual.

Subscribe now!

#3159 Fastest way of creating 100.000 photos gallery

Posted in ‘Event Gallery Extended’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Latest post by sbluege on Thursday, 24 May 2018 06:23 UTC

videostream
 Hello, thank you for your work!
I have a questions.
Every week I have 10000+ photos. And all of them I want to sell.
I try to upload on my hosting only 100 photos, and it doesn't work fast enough.

Please tell me, what is the fastest settings to upload and show to people 10.000+ photos?
Maybe better way to upload it to google, or amazon?
And what is the fastest settings for cache?
Thank you for answers!

videostream
http://sun-paulo.studio/testphoto/event/mezhdunarodnyj-konkurs-festival-melodii-morya-g-sochi/melodii-morya-otkritie.html

I upload 1000+ photos, and it work too slow :(

sbluege
Hi, 
10k? This is a lot! This will make a lot of disk space necessary. Don't forget the necessary space for thumbnails. 
I recommend to use the backend for uploading. Make the image 1600px large. You don't need to change any cache settings. Uploads depend only one the upload speed of your internet connection. 
You can also upload via ftp and use the sync database button. There is a cli script as well. I recommend that for such a large amount of images. However, this might take a while. 
Google Photos would work as well. I don't recommend Amazon. It is very slow. 
How many images to you plan to stuff into Event Gallery? 

sbluege
I just saw, you want to sell those images. If people need to download the original file automatically, you need to upload the large files. You can't use Google since there is no protection for those files. 

sbluege
The link you send shows some images. If you upload an image, no thumbnails exist. If users access your images, the thumbnail generation is triggered. Thumbnail calculation needs some time but happens only once per image/thumbnailsize. Every subsequent request uses a cached version of the thumbnail.

But there are 13 different thumbnail sizes! Event Gallery tries to deliver the best fitting size. This requires a couple of different sizes.

To avoid random layouts and different thumbnail-sizes in the same browser you can set the jitter-value for the image list layout to 0. You can do this in the components options or the menu item settings.

videostream
This is statistic of using CPU of my hosting. Purple line is my hosting plan, green line on the top - the most expensive plan.

In this time I upload 1000 photos to event gallery. The same CPU usage was when I open event first time, and thumbnail generation is triggered.
Maybe I can make all thumbnails on my computer, and event gallery will not use hosting CPU a lot? Or maybe such CPU usage is because adding watermark is ON?

I need speed of opening pages like that - http://www.photodance.ru/ph/1779/13.php
All images\sizes\watermarks\thumbnails I can do on my computer, I need only one thing - fast speed of viewing 10000+ images for my clients :-)

Attachments

2018-05-13_10-12-32.png

sbluege
The CPU usage is caused by creating thumbnails. During upload, the first thumbnail is created because you see a preview of the uploaded image. While opening the event page, more thumbnails are created. Having a watermark on or not does not make much of a difference.

How large are your images? This has a huge impact on CPU usage. 30MP images are more expensive than 2MP images. There is a tiny trick you can try: https://www.svenbluege.de/joomla-event-gallery/event-gallery-manual/faq-original-images

Upload smaller images and the original files separately. This will allow fast thumbnail creation but you can sell the original image.

If you want to calculate the thumbnail on your local machine, you can try this too. Please have a look at the /cache/com_eventgallery_images folder.

Valid and necessary thumbnail sizes are: 48, 104, 160, 288, 320, 400, 512, 640, 720, 800, 1024, 1280, 1440, 1600
This describes always the larger edge or an image. The images are prefixed with crop or nocrop. Crop means a square sized image. If you miss to create a thumbnail or used an invalid file name, Event Gallery will still create the thumbnail itself.

videostream
I upload 800px images, 100kb :-) And I don't upload original images for clients. After client make order I take original photos, retouch it and send to client.
Ok, I will try to upload photo via FTP, and try make thumbnails by myself.

sbluege
Those files are small enough, the CPU need should not be that much. It seems like that depends on what your hosting provider measures.

If you create those thumbnails manually, don't forget that you always have to create all sizes. Even if your images are smaller than some of them. Don't enlarge thumbnails. A 1600px thumb will contain your 800px file. Event Gallery expects that and can handle it.

videostream
Hello sbluege, thank you for your help.
I upload only 1000px images, and use pageable list.

Today I try upload first 10.000 images, via ftp. When I start "sync database", CPU go overload :( Maybe there is some other way to upload so many images?

And you say that there are 14 different sizes of thumbnails. 10.000 images X 14 sizes = 140.000 files for one event. I have 2 events every week, and I can make only 300.000 files on my hosting :)

I try to make thumbnails nocrop288 on my computer - works great, no CPU loading. But sometimes event gallery make nocrop160, and CPU start overload (I think because there is 300 images per page).
Maybe there is some way to make only one thumbnail for all small previews - nocrop288? And when open page of image of lightbox - original 1000px image will be taken?

Thank you for your help!

videostream
Here is new link - http://sun-paulo.studio/zakaz-photo-video/

sbluege
If you perform the "sync database"-process, Event Gallery ready all your images to collect metadata like EXIF, image dimension and so on. There is no way around that.

All the different thumbnail sizes are needed. You already saw that nonexisting thumbnails are generated. You can try to modify the file components\com_eventgallery\helpers\sizeset.php and remove thumbnail sizes you don't need. I don't recommend that, but it might work.

The image in the lightbox is the COM_EVENTGALLERY_IMAGE_ORIGINAL_MAX_WIDTH image size. Even if you upload 1000px images, COM_EVENTGALLERY_IMAGE_ORIGINAL_MAX_WIDTH thumbnail file will contain something. So please generate this thumbnail too. COM_EVENTGALLERY_IMAGE_ORIGINAL_MAX_WIDTH is by default 1600.

videostream
I try to edit sizeset.php, after this all images turn to white.

Can you help me? Maybe I can pay you for that.
There is too much CPU usage on my site, because there is 20+ online users, and they choose 10.000+ photos.
I think there will be no CPU usage if there will be only two types of images - 288 thumbnail and 1000px original. I can make this types on my computer and upload it to server. When people see list of images, only 288 thumbnail will be used. When they open image on single page - original size image will used.

Can you change code for me? How much it will be cost?

sbluege
I can do a modification to move those sizes into something which does not get lost with every update. I'm happy to talk with you about prices then.

Before we do that, make sure the file looks like the one attached. I just tried it, and it already works. With my example only two thumbnails are generated.

Attachments

24-05-_2018_08-21-18.png