Walk hangs

edev · Post by **edev** » Wed Sep 27, 2006 10:48 am

Hi,

recently my walk has stopped around 50,000 pages, on the "Walk Status" page it seems still going but shows "0 pages visited in the last hour" and it seems to be hung. I got the following error:

Timed out waiting for Mutex in the function semlock while processing url http://....

This walk consists of over 15,000 URLs from a file on the server (URL File), goes only 1 level deep, and the load average is set to 0.5. I have more than enough storage on this server. There is 1 thread on 3 servers at a time.

Any idea what may have gone wrong? thank you...

Post by **mark** » Wed Sep 27, 2006 1:22 pm

Sounds like database locks got stuck for a time. But that shouldn't have stopped the whole thing.

Try "pause and live" from the walk status page. Then resume the walk as mode refresh to pickup where it left off.

edev · Post by **edev** » Wed Sep 27, 2006 1:34 pm

Thank you. I've started a new walk and I'll see if the problem persists.

edev · Post by **edev** » Mon Oct 02, 2006 10:23 am

Started a new walk and it was hung around 113,000 pages. I tried to pause it and live, but it has been an hour and the walk is still going. I did it a few times through the administration interface. Is there a way to stop it from the concole? I tried dowalk/hold.txt but it didn't work. Not sure what to do at this point, I have 15,000 URLs need to be walked soon and should I split them into 10 files?

Post by **John** » Mon Oct 02, 2006 10:49 am

You may have too many servers configured. You can either run the dowalk/hold.txt every 30 seconds or so until it does stop, or run it once, wait 20-30 seconds, and then kill the child processes by pid (there should be running.pid files there).

edev · Post by **edev** » Mon Oct 02, 2006 11:38 am

Thanks John, I'll try that. I only have 3 servers each with 1 thread configured, before it wasn't a problem at all so I'm not sure what I did wrong.

At the end of the error report I'm still getting the following message:
Timed out waiting for Mutex in the function semlock while processing url http://www.....

I still have no idea what's causing that...