Gallery importer / deleting needs to be broken into smaller pieces to accomidate 1and1 hosting. (maybe others?)

marshra

Joined: 2005-09-11
Posts: 4
Posted: Sun, 2006-07-23 01:18

I'm not sure if this is the right place to post this, but I have done my best to ensure that it is :)

I was having problems with importing G1 data into the new G2 system. Nothing was wrong with your code at all, it's just that with my webhost (which is quite popular) 1and1.com, they stopped the script from running due to high cpu usage.

From what I got from them it's becuase of the high cpu usage on the import script. They only allow you to have 100% cpu time for 10 seconds. I asked about pasuing and going again, but he said it doesn't matter how long it takes to get 100% but it's cumlative. He suggested that if you broke down processes (max of 12 simotanious). Anyway, here is the convo. My hope for this is to get you guys to rewrite your code so that it breaks down the load on the import. I had to contact tech support and have them remove those limits for 24 hours, and import it. I also had problem deleting large almbums at a time for the saem reason.

Here is my prior post:

So after fighting with my hosting company (1and1.com), I was able to get them to raise the limits. Here is what I got from their tech support.

Thank you for contacting us.

This is definitely a problem of excessive CPU usage, this is a very resource intensive script. In both cases (importing and deleting a gallery), the process will use 10 seconds of CPU-time (equating to usually around 20 seconds using about 50% of the server's resources), after which time the process is killed. I have seen this problem before with this same gallery script. The bottom line is that it is probably too resource intensive to be used in a shared hosting environment, unless you can just do smaller operations, such as deleting or importing only a few images at a time.

After a few more questions about the restrictions on the cpu usage here is what I was told:

To expand further on how the limits work:

The limit that is relevant in this case is the CPU time. A given process can use up to 10 seconds of CPU time before it is killed. CPU time means the number of seconds that the script uses 100% of the CPU. In practice the process will almost never use 100% of the CPU since it is shared with other users. So if the CPU usage averages at around 50%, the process will run for around 20 seconds before being killed. If the process averages 25%, it will last for 40 seconds, and so on. "Pausing" the process won't make a difference, although if the script can spawn
several processes to each to part of the job it could help; however there are also limits on the number of simultaneous processes that can exists. The limit on simultaneous processes is 12.